From jose.valim@REDACTED Mon Feb 1 09:44:16 2016 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Mon, 1 Feb 2016 09:44:16 +0100 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: Message-ID: Thank you Bj?rn! Should we expect list_to_binary to work? list_to_binary(atom_to_list(Atom)) list_to_binary(io_lib:format("~s", [Atom])) list_to_binary expects a list of bytes, instead of a list of codepoints, so I would except those cases to always raise for a UTF8 encoded atom. The same way this raises: list_to_binary(io_lib:format("~ts", [<<12494/utf8>>])) On the other hand, the line below works but the result is in the wrong encoding: list_to_binary(io_lib:format("~s", [<<12494/utf8>>])) So I would say list_to_binary is behaving as expected and that it should not change as those "limitations" are there today. Same for port_command, as it expects iodata. Or am I missing something? Regarding the raising issue, one possible option is to introduce atom_to_list(Atom, Encoding), similar to atom_to_binary/2, with the default encoding of Latin1. If you call atom_to_list(Atom, latin1) with a UTF-8 encoded atom, it will raise. This way the code semantics won't change and we won't have to worry about UTF-8 atoms creeping in unless explicitly allowed. *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D On Mon, Feb 1, 2016 at 8:14 AM, Bj?rn Gustavsson wrote: > On Sat, Jan 30, 2016 at 9:04 PM, Jos? Valim > wrote: > > > > With all that said, are there any plans of supporting UTF-8 encoded > atoms on > > Erlang R19? If the feature is desired but not planned, I would love to > > contribute the compiler and bytecode changes above although I will likely > > need some guidance. If that is an option, I would love to get in touch. > > > > It is not planned for OTP 19. IMO, the feature is desired, > but it is probably too late for OTP 19. > > Extending the BEAM format is necessary but not sufficient. > It is also necessary to make sure that other code in OTP > doesn't break. For example: > > list_to_binary(atom_to_list(Atom)) > > list_to_binary(io_lib:format("~s", [Atom])) > > erlang:port_command(Port, N, atom_to_list(Atom)) > > list_to_atom/1 could also potentially be problematic > if the code expects an exception for any non-latin1 > characters. > > Other things to be done is to update the documentation > and specs. > > I think that the community could help us there, both > in collecting a list of things that must be fixed > or modified, and also in helping fixing them. > > /Bj?rn > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn@REDACTED Mon Feb 1 08:14:47 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 1 Feb 2016 08:14:47 +0100 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: Message-ID: On Sat, Jan 30, 2016 at 9:04 PM, Jos? Valim wrote: > > With all that said, are there any plans of supporting UTF-8 encoded atoms on > Erlang R19? If the feature is desired but not planned, I would love to > contribute the compiler and bytecode changes above although I will likely > need some guidance. If that is an option, I would love to get in touch. > It is not planned for OTP 19. IMO, the feature is desired, but it is probably too late for OTP 19. Extending the BEAM format is necessary but not sufficient. It is also necessary to make sure that other code in OTP doesn't break. For example: list_to_binary(atom_to_list(Atom)) list_to_binary(io_lib:format("~s", [Atom])) erlang:port_command(Port, N, atom_to_list(Atom)) list_to_atom/1 could also potentially be problematic if the code expects an exception for any non-latin1 characters. Other things to be done is to update the documentation and specs. I think that the community could help us there, both in collecting a list of things that must be fixed or modified, and also in helping fixing them. /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bjorn@REDACTED Mon Feb 1 12:59:45 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 1 Feb 2016 12:59:45 +0100 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: Message-ID: On Mon, Feb 1, 2016 at 9:44 AM, Jos? Valim wrote: > So I would say list_to_binary is behaving as expected and that it should not > change as those "limitations" are there today. Same for port_command, as it > expects iodata. Or am I missing something? My point is that we must look for code in OTP that will break when the change to the atoms are made. As an hypothetical example, say that we find the following code in some application: Str = atom_to_list(Atom), . . . port_command(Port, Cmd, Str) We must look at the context to determine what we should do. There could be one of several solutions, for example: 1. If the atoms that can be passed to this code have been internally generated we could know that the resulting list is always safe to send to the port. In that case we don't need to update the code. 2. If the origin of the atom is unknown, and the driver cannot handle UTF-8, the solution could be to return an error to the caller if the atom contains non-latin1 characters. 3. If the driver can handle UTF-8 or can be modified to handle UTF-8, the solution could be to use atom_to_binary(Atom, utf8) instead of atom_to_list/1. Basically, we must look at every atom_to_list/1 in the OTP code base and determine whether it is safe or if it must be modified in some way. /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From jose.valim@REDACTED Mon Feb 1 13:08:51 2016 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Mon, 1 Feb 2016 13:08:51 +0100 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: Message-ID: Understood! Thank you. *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D On Mon, Feb 1, 2016 at 12:59 PM, Bj?rn Gustavsson wrote: > On Mon, Feb 1, 2016 at 9:44 AM, Jos? Valim > wrote: > > So I would say list_to_binary is behaving as expected and that it should > not > > change as those "limitations" are there today. Same for port_command, as > it > > expects iodata. Or am I missing something? > > My point is that we must look for code in > OTP that will break when the change to > the atoms are made. > > As an hypothetical example, say that we > find the following code in some application: > > Str = atom_to_list(Atom), > . > . > . > port_command(Port, Cmd, Str) > > We must look at the context to determine > what we should do. There could be one > of several solutions, for example: > > 1. If the atoms that can be passed to this > code have been internally generated we > could know that the resulting list is always > safe to send to the port. In that case we > don't need to update the code. > > 2. If the origin of the atom is unknown, > and the driver cannot handle UTF-8, > the solution could be to return an error > to the caller if the atom contains > non-latin1 characters. > > 3. If the driver can handle UTF-8 or can > be modified to handle UTF-8, the solution > could be to use atom_to_binary(Atom, utf8) > instead of atom_to_list/1. > > Basically, we must look at every atom_to_list/1 > in the OTP code base and determine whether > it is safe or if it must be modified in some way. > > /Bj?rn > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Mon Feb 1 15:13:32 2016 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Mon, 1 Feb 2016 15:13:32 +0100 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: Message-ID: I have pushed some initial work that creates a separate chunk called AtU8: https://github.com/erlang/otp/compare/master...josevalim:jv-utf8-atom First the atoms from the Atom chunk are loaded, followed by atoms from the new AtU8 chunk. I have, however, stumbled against one complication. When we generate the atom table in the compiler, the indexes are built based on first-seen case. So there is a chance we will have an atom table that looks like this: #{0 => module_name, 1 => 'some_utf8_atom_?', 2 => latin1_atom} However, because we first load the latin chunk, the loaded atom table would look like this: 0 => module_name 1 => latin1_atom 2 => 'some_utf8_atom_?' Which won't work. I have thought of a couple solutions to this problem. I would appreciate some feedback on which one is preferred: 1. Keep on using two separate chunks and change beam_asm to do two passes. One first collecting the latin1 atoms so we know how to generate indexes for the utf8 ones. 2. Keep on using two separate chunks and use negative indexes to encode utf8 atoms. This means the compiler will build a table that looks like this: #{-1 => 'some_utf8_atom_?', 0 => module_name 1 => latin1_atom} Which we will load to the same table as before: 0 => module_name 1 => latin1_atom 2 => 'some_utf8_atom_?' And we will translate negative indexes to the proper position by calculating "num_atoms + index" when loading the bytecode. 3. Introduce a new chunk called AtoE (Atoms with Encoding) to *replace* the existing Atom chunk. It will be quite similar to the current Atom chunk except we will also include the encoding alongside each atom size and contents. The Atom chunk will only be loaded if there is no AtoE chunk. If we choose this option, we can also choose to always emit the new chunk or emit the AtoE chunk only if there are Unicode atoms. 4. Similar to option 3) except that we introduce a new Atom chunk that will keep all atoms but always in Unicode. Since the runtime translates them to latin1 later anyway, this is an option if we don't want to store the encoding in the table. Thank you! PS: If you'd prefer this conversation to be moved off the list, please let me know. *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D On Mon, Feb 1, 2016 at 1:08 PM, Jos? Valim wrote: > Understood! Thank you. > > > > > *Jos? Valim* > www.plataformatec.com.br > Skype: jv.ptec > Founder and Director of R&D > > On Mon, Feb 1, 2016 at 12:59 PM, Bj?rn Gustavsson > wrote: > >> On Mon, Feb 1, 2016 at 9:44 AM, Jos? Valim >> wrote: >> > So I would say list_to_binary is behaving as expected and that it >> should not >> > change as those "limitations" are there today. Same for port_command, >> as it >> > expects iodata. Or am I missing something? >> >> My point is that we must look for code in >> OTP that will break when the change to >> the atoms are made. >> >> As an hypothetical example, say that we >> find the following code in some application: >> >> Str = atom_to_list(Atom), >> . >> . >> . >> port_command(Port, Cmd, Str) >> >> We must look at the context to determine >> what we should do. There could be one >> of several solutions, for example: >> >> 1. If the atoms that can be passed to this >> code have been internally generated we >> could know that the resulting list is always >> safe to send to the port. In that case we >> don't need to update the code. >> >> 2. If the origin of the atom is unknown, >> and the driver cannot handle UTF-8, >> the solution could be to return an error >> to the caller if the atom contains >> non-latin1 characters. >> >> 3. If the driver can handle UTF-8 or can >> be modified to handle UTF-8, the solution >> could be to use atom_to_binary(Atom, utf8) >> instead of atom_to_list/1. >> >> Basically, we must look at every atom_to_list/1 >> in the OTP code base and determine whether >> it is safe or if it must be modified in some way. >> >> /Bj?rn >> >> -- >> Bj?rn Gustavsson, Erlang/OTP, Ericsson AB >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn@REDACTED Mon Feb 1 16:34:19 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 1 Feb 2016 16:34:19 +0100 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: Message-ID: On Mon, Feb 1, 2016 at 3:13 PM, Jos? Valim wrote: > 3. Introduce a new chunk called AtoE (Atoms with Encoding) to *replace* the > existing Atom chunk. It will be quite similar to the current Atom chunk > except we will also include the encoding alongside each atom size and > contents. The Atom chunk will only be loaded if there is no AtoE chunk. If > we choose this option, we can also choose to always emit the new chunk or > emit the AtoE chunk only if there are Unicode atoms. > > 4. Similar to option 3) except that we introduce a new Atom chunk that will > keep all atoms but always in Unicode. Since the runtime translates them to > latin1 later anyway, this is an option if we don't want to store the > encoding in the table. I prefer 4 because it is the simplest. For most modules, the chunk will be smaller than for option 3. > Thank you! > > PS: If you'd prefer this conversation to be moved off the list, please let > me know. Unless the list minds, I think that it is better to keep the conversation on the list so that other can benefit from it. Personally, I have learned a lot from following conversations about design decisions on other mailing lists (for example, the Git mailing list). /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From jose.valim@REDACTED Mon Feb 1 16:48:01 2016 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Mon, 1 Feb 2016 16:48:01 +0100 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: Message-ID: > > I prefer 4 because it is the simplest. For most modules, the > chunk will be smaller than for option 3. > Should we always emit the new chunk or only if there are unicode atoms? A nice side-effect of this solution is that it would also allow Unicode module names as well. -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Mon Feb 1 17:01:30 2016 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 1 Feb 2016 17:01:30 +0100 Subject: [erlang-questions] string concatenation efficiency In-Reply-To: References: <56AABF3E.7010800@cs.otago.ac.nz> Message-ID: On Sat, Jan 30, 2016 at 5:04 PM, Khitai Pang wrote: > Thank you all for your answers. > > The concatenated string is a redis key, it will be part of an erlang message > that will be sent to a redis client erlang process. I tried sending the > iolist ["Item:{",ItemId,"}"], and it works fine. I think this is the most > efficient way. It *is* the most efficient way - but remember what you sent, some other program has to receive - the amount of work done in transporting the message and delivering it to the receiver will far outweigh the cost of building the message prior to sending it. The rule of thumb is that parsing is *always* slower than serialising messages - why? in parsing you must take a decision at every byte. In serializing a message you take a decision at every node in a parse-tree. So you should not be worrying about what you send - but you should worry about how it will be received. Building the message from variables into an I/O list or with "++" has a trivial cost compared to all the machine cycles taken up transporting the message. Even measuring the cost of different methods won't tell you much. You'll find out that A (IO-lists) is faster or slower than B (++) but all the real time is taken up in X and Y (which you didn't know about) People happily send KBytes encrypted data in XML, when single unencrypted bytes might have sufficed. The only thing you should worry about is end-to-end efficiency. So: 1) Make everything 2) Measure 3) Fast enough? Yes Happy No Can I wait ten years? No measure to find bit that takes the most time fix worse bit goto 2) Yes wait ten years - go to 2) Patience is a virtue - the programs I wrote 20 years ago now run thousands of times faster - not because I'm a better programmer but because we have GHz clocks now (they were MHz 20 years ago) You want to write clear short, elegant code that will still run in 20 years time Given the choice between clear code that is slow but fast-enough and fast code that is unclear I always choose the clear clode. The key point is fast-enough. Cheers /Joe > > > Thanks > Khitai > > > On 2016/1/29 9:24, Richard A. O'Keefe wrote: >> >> >> On 28/01/16 8:22 pm, Khitai Pang wrote: >>> >>> For string concatenation, which one of the following is the most >>> efficient? >>> >>> 1) >>> strings:join(["Item:{", ItemID, "}")], ""] >>> >>> 2) >>> lists:append(["Item:{", ItemID, "}")]) >>> >>> 3) >>> "Item:{" ++ ItemID ++ "}" >>> >>> Here ItemID is a UUID. >> >> For something this size, it hardly matters. >> >> The definition of lists:append/1 is >> append([E]) -> E; >> append([H|T]) -> H ++ append(T); >> append([]) -> []. >> so lists:append([A,B,C]) -> A ++ lists:append([B,C]) >> -> A ++ B ++ lists:append([C]) >> -> A ++ B ++ C. >> Clearly, there is no way for this to be more efficient than >> writing A ++ B ++ C directly. >> >> The definition of string:join/2 is >> join([], Sep) when is_list(Sep) -> []; >> join([H|T], Sep) -> H ++ lists:append([Sep ++ X >> || X <- T]). >> -- at least in 18.1 -- which rather startled me because I was expecting >> join([E], Sep) when is_list(Sep) -> E; >> join([H|T], Sep) -> H ++ Sep ++ join(T, Sep); >> join([], Sep) when is_list(Sep) -> []. >> Clearly, there is no way that either version of join/2 can be faster than >> append/1. In fact >> append/1 is measurably faster than >> the revised join/2, which is measurably faster than >> the original join/2, >> but you have to push the sizes ridiculously high to make these >> measurements. >> >> Really, I suggest you write whichever makes your intentions clearest >> to human readers, get the program going, then measure it. I would >> be surprised if this particular issue made any significant difference. >> >> But why are you using strings at all? >> Strings are an *interface* data type; you process them when you receive >> data from an external source and you generate them (or rather you >> generate iolists) when you are sending data to an external source, but >> for internal processing you usually want some sort of tree. >> >> Seeing "Item:{$ItemID}" suggests that maybe you *are* generating output >> for some external process, but in that case, perhaps you should just be >> sending along the iolist ["Item:{",ItemId,"}"] and *not* concatenating the >> pieces. >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bjorn@REDACTED Mon Feb 1 17:47:53 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 1 Feb 2016 17:47:53 +0100 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: Message-ID: On Mon, Feb 1, 2016 at 4:48 PM, Jos? Valim wrote: >> I prefer 4 because it is the simplest. For most modules, the >> chunk will be smaller than for option 3. > > > Should we always emit the new chunk or only if there are unicode atoms? > The default should probably be to always use the new chunk type. However, there must be an option to make the compiler generate the old chunk type if possible. There is module called test_server_node that must be possible to load in both new and old releases. We use it for compatibility testing. (Note the 'compile(r12)' attribute in the module.) See compile:expand_opt/2 and how the options r12, r13, and r14 are handled. (I suggest that you add a new r18 option that turns off the new atom chunk. You will also need to to update the r12, r13, and r14 options because io_proto_SUITE in the stdlib test suite still depends on r12.) /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From buaatianwanli@REDACTED Mon Feb 1 15:28:08 2016 From: buaatianwanli@REDACTED (=?ISO-8859-1?B?ZXJpYw==?=) Date: Mon, 1 Feb 2016 22:28:08 +0800 Subject: [erlang-questions] {"init terminating in do_boot", {'cannot load', elf_format, get_files} In-Reply-To: References: Message-ID: how to resolve this -------------- next part -------------- An HTML attachment was scrubbed... URL: From Gustav.Spellauge@REDACTED Mon Feb 1 17:54:32 2016 From: Gustav.Spellauge@REDACTED (Gustav Spellauge) Date: Mon, 1 Feb 2016 17:54:32 +0100 Subject: [erlang-questions] Erlang/OTP 17.0 on android Message-ID: <56AF8DC8.5050807@softing.com> good evening, at least the networking related functions were unable to run. i got the erlang sources, compiled everything and uploded to my android. erlang is running fine without any warnings. g. From charles@REDACTED Mon Feb 1 18:10:51 2016 From: charles@REDACTED (Charles Weitzer) Date: Mon, 1 Feb 2016 17:10:51 +0000 Subject: [erlang-questions] Software Engineering Opportunity with Voleon Capital Management Message-ID: The Voleon Group is a science-driven systematic trading firm, built on the principle that statistical machine learning provides the best solutions to the scientific problems we must solve. The firm researches and deploys systematic trading strategies designed to generate attractive returns without being dependent on the performance of the overall market. We would like to hire an exceptionally creative and capable software engineer. You will architect and implement new production trading systems, machine learning infrastructure, data integration pipelines, and large-scale storage systems. In the past, we have had great success in hiring software engineers with experience with functional programming languages such as Erlang and Haskell. Voleon's founders previously worked together at one of the most successful quantitative hedge funds in the world. Our CEO earned a PhD in Computer Science from Stanford and has been CEO and founder of a successful Internet infrastructure startup. Our Chief Investment Officer and Head of Research earned his PhD in Statistics from and is currently on the faculty at UC Berkeley. Voleon's team includes PhDs from leading departments in statistics, computer science, and mathematics from Berkeley, Stanford, CMU, Caltech, MIT, University of Chicago, and other top-tier schools. We have made several unpublished advances in the field of machine learning and in other areas as well. You will have a high impact, and you can expect frequent interaction with other researchers, officers, and our founders. We take a rigorous approach to building trading strategies and systems, and actively foster a highly collaborative and collegial intellectual environment. We are growing rapidly. Willingness to take initiative and a gritty determination to productize are essential. We seek candidates with a proven track record of writing correct, well-designed software, solving hard problems, and delivering complex projects on time. You should preferably have experience designing and implementing fault-tolerant distributed systems. Experience with building large-scale data infrastructure, stream processing systems, or latency-sensitive programs is a bonus. Required experience: - experience with functional programming languages such as Erlang, Haskell, etc. - developing with C/C++/Python/Go in a Linux environment with a focus on performance, concurrency, and correctness. - working in TCP/IP networking, multi-threading, and server development. - working with common Internet protocols (IP, TCP/UDP, SSL/TLS, HTTP, SNMP, etc.). - architecting and designing highly available systems. - architecting and designing large-scale data management infrastructure. - working in large codebases and building modular, manageable code. Preferred experience: - debugging and performance profiling, including the use of tools such as strace, valgrind, gdb, tcpdump, etc. - working with build and test automation tools. - working with well-defined change management processes. - diagnosing RDBMS performance problems, exploiting indexing, using EXPLAIN PLAN, optimizing at the code layer, etc. - working with messaging queues (RabbitMQ, Redis, etc.) as well as distributed caching systems. Interest in financial applications is essential, but experience in finance is not a primary factor in our hiring. We hire on the basis of exceptional talent. Benefits and compensation are highly competitive. We encourage you to apply on our website with your full and complete CV: http://voleon.com/apply/. The Voleon Group is an Equal Opportunity employer. Applicants are considered without regard to race, color, religion, creed, national origin, age, sex, gender, marital status, sexual orientation and identity, genetic information, veteran status, citizenship, or any other factors prohibited by local, state, or federal law. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ameretat.reith@REDACTED Mon Feb 1 20:40:49 2016 From: ameretat.reith@REDACTED (Ameretat Reith) Date: Mon, 1 Feb 2016 23:10:49 +0330 Subject: [erlang-questions] Heavy duty UDP server performance Message-ID: <20160201231049.08b28dc7@gmail.com> I'm playing with Erlang to make an experimental protocol. I'm trying to make it use full of 1Gbit link but It won't scale that much and I'm failing to found a bottleneck in my code or even anything I could call it bottleneck. My software is very like a messaging server software in behavior, with bigger packets, many clients (more than 4k) and uses more complex sub-components, like a distributed database but those components are not blocking other portions of system; It's just the client-server channel that is heavy IO and involve some encryption and decryption. I made a gen_server process for each UDP socket to clients. There is a central process registry but It being called just for new clients and Its message queue is often empty. I found there was a bottleneck in `scheduler_wait` when I had few clients (around 400) and It consumed around 50% of total CPU usage. I found an old patch by Wei Cao [1] which seemed to target same issue. But on a modern version of Erlang (18.0) blockage in `scheduler_wait` dropped well in more congested network, specifically to around 10% when my software reached Its apparent limit, around 600Mbit/s read and write to network. At this point my incoming UDP packet rate is around 24K/s. Maybe an experienced Erlang developer here can remember that problem and can tell whether Erlang is now optimized to poll for network packets more often or not.. I also concerned async pool since there was fairly high work in Erlang work with pthread but found those threads just used for file IO operations. I didn't found any assuring documentation about this, just saw the only user of this dirty IO thing is `io.c` in otp source code. I'm very grateful if anyone clear the usage and effect of this pool. I made flame graphs of function calls both inside VM (using eflame2 [2]) which is very even and cannot find any outstanding usage [3]. And made another flamegraph of perf report outside of VM which cannot find some symbols [4]. I doubt whether process_main shoud take that much work itself or not. Apparently encryption and decryption (enacl_nif calls) didn't take much time too. Do you have any suggestion for me to analyze better my software and understand VM working? Is It those limits I should expect and there is not more room for optimizations? Thanks in advance 1: http://erlang.org/pipermail/erlang-questions/2012-July/067868.html 2: https://github.com/slfritchie/eflame 3: http://file.reith.ir/in-erl-3k.gif 4: http://file.reith.ir/out-erl-perf.svg (interactive, use web browser) From n.oxyde@REDACTED Mon Feb 1 23:40:11 2016 From: n.oxyde@REDACTED (Anthony Ramine) Date: Mon, 1 Feb 2016 23:40:11 +0100 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: Message-ID: Let's be serious two minutes, this new chunk should definitely be named JOS?. > Le 1 f?vr. 2016 ? 17:47, Bj?rn Gustavsson a ?crit : > > From max.lapshin@REDACTED Tue Feb 2 07:13:01 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 2 Feb 2016 09:13:01 +0300 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: Message-ID: What are the reasons to have unicode atoms? To make code non-editable by foreign programmers? Are you ready to edit chineese or thai? -------------- next part -------------- An HTML attachment was scrubbed... URL: From mohit3081989@REDACTED Tue Feb 2 04:56:32 2016 From: mohit3081989@REDACTED (mohit Agrawal) Date: Tue, 2 Feb 2016 09:26:32 +0530 Subject: [erlang-questions] Erlang + Ejabberd Message-ID: Hi Team, I am new to Erlang, I have been trying to install Erlang and Ejabberd on EC2 ubuntu machine, everything went well till I started compiling some external modules in ejabberd. It started throwing error "undefined parse transform 'lager_transform'". I tried everything which is as below : 1. Did rebar get-deps, make clean, make deps, make install. After this I am able to see that lager_transform.beam is made and present in //lib/ folder. 2. Checked rebar.config file, it had lager deps on top, which is widely suggested, no help even after that. 3. Added -compile[{parse_tranform},{lager_transform}] on top of module, even then their is no luck. I am really blocked on this, and not able to complete the installation. I have done this before on fedora with Ejabberd 15.11 and otp 18.2, at that time it was using p1_logger instead of lager. But now when I am installing fresh with otp 18.2 and Ejabberd 16.0 or 15.11, I am getting lager_transform undefined error. Please suggest and help -- Mohit Agrawal -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Tue Feb 2 08:11:20 2016 From: zxq9@REDACTED (zxq9) Date: Tue, 02 Feb 2016 16:11:20 +0900 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: Message-ID: <6851594.7I7yDfpvbI@changa> On 2016?2?2? ??? 09:13:01 Max Lapshin wrote: > What are the reasons to have unicode atoms? > > To make code non-editable by foreign programmers? Are you ready to edit > chineese or thai? As someone who works mostly in Japanese on Japanese projects... I have to admit, while there is a fair amount of lisp in Japanese, I can't think of a practical use case where I would want to use unicode atoms in Erlang. It would be just as odd as wanting to use capital-letter atoms in Erlang, actually -- the quoting and upper/lower distinctions are actually quite convenient. If I was going to use Japanese atoms, I would also want to use Japanese variable names -- and I would then feel a very strong urge to create the same distinction: - atoms are always katakana - variables are anything else that is a letter But now we're really opening a can of worms. When I'm in Japanese-writing mode I actually *expect* my input method to change things like '->' to ?, among *many* other cases of dramatically more accurate character representations for things we use the pitiful range of "safe ASCII" characters to draw in a clumsy way. I would also expect '<<' to become ? and wonder why we have to use #{} for a map and #name{} for a record and {} for a tuple when we have ?? and ?? and ?? and... (etc.) available. As an actual daily user of UTF8 values that aren't just the tiny corner used in the West, I feel like it opens the door to a lot more confusion without being very careful about how it is implemented. It is, of course, "easy" to just say "anything in ASCII single-quotes is an atom, and the only way to no single-quote an atom is to use lower-case ASCII letters" -- but in so doing you also guarantee that almost nobody will ever take advantage of UTF8 atoms. This will make their use very rare, and so will make their presence in a codebase somewhat annoying. (Prepare to see comments on github commits like "Wtf, normal atoms aren't l33t enough for this clown?!?") I'm not saying its a bad idea or a bad thing to think about -- but a lot more thought would need to go into this than simply defining a parsing rule or cobbling together a patch that builds cleanly. Just like making snap decisions about what "whitespace" means, this will be something we're stuck with forever (all of us -- including people who program in languages that have analogs to the upper/lower distinction). -Craig From jose.valim@REDACTED Tue Feb 2 08:52:55 2016 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Tue, 2 Feb 2016 08:52:55 +0100 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: <6851594.7I7yDfpvbI@changa> References: <6851594.7I7yDfpvbI@changa> Message-ID: > Are you ready to edit chineese or thai? I can already write atoms in Afrikaans, Albanian, Basque, Breton, Corsican, Danish, English, Faroese, Galician, German, Icelandic, Indonesian, Italian, Kurdish, Leonese, Luxembourgish, Malay, Manx, Norwegian, Occitan, Portuguese, Rhaeto-Romanic, Scottish Gaelic, Spanish, Swahili, Swedish, Walloon. Source: https://en.wikipedia.org/wiki/ISO/IEC_8859-1 And I can only effectively speak 3 of those. There are already a lot of programmers who can write in their own language and I don't expect to edit their code. Not only that, there are other programming languages which have already adopted Unicode Support and the amount of times I had to read code in a language I don't understand has been exactly 0. I recommend reading Richard O'Keefe replies on the matter back in 2012: http://erlang.org/pipermail/erlang-questions/2012-October/070187.html > but a lot more thought would need to go into this than simply defining a parsing rule or cobbling together a patch that builds cleanly. I agree with this. That's why I want to focus on the underlying runtime support and not on how it will affect parsing. Specially because according to the OTP board post I mentioned above, they haven't made a decision if Unicode in variables should be allowed. Today Unicode is already supported in double quotes (and double quotes only) so I would say it is reasonable to allow them in single quotes as well. Should we move this discussion to another thread? I know this has been extensively discussed in the past and it would be nice to keep this discussion focused on the implementation details for whoever is following the trail in the future. *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue Feb 2 09:33:22 2016 From: ok@REDACTED (ok@REDACTED) Date: Tue, 2 Feb 2016 21:33:22 +1300 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: <6851594.7I7yDfpvbI@changa> References: <6851594.7I7yDfpvbI@changa> Message-ID: I should point out that Unicode isn't just useful for Chinese, Japanese, Korean, Mongolian, Thai. Hindi, Ancient Egyptian, & so on. There are plenty of languages basically using the Latin script, but with some extra letters or tweaks. Take as an example one of the official languages of my country, which uses lower- and upper-case vowels with macrons. If one is allowed to write english_name and ?venjulegt_nafn (Icelandic) it is a little difficult to explain why ingoa_māori (Māori) should be forbidden. Now Māori is definitely a minority language, but then so are Faroese, G?idhlig, Leonese, and Manx, supported by Latin 1. As a specific technical issue, Erlang allows unquoted atoms like roller_ball@REDACTED The internet has had internationalised domain names since 1998 (that would be about 17 years) and yes, they're mapped through Punycode, but if you have a domain name like pūtaiao-rorohiko.otāgo.wānanga.nz (which, to be fair, I *don't*, but it's surely only a matter of time) won't you want to be able to use it as or in an atom? By the way this means that domain names are one of the areas in Erlang/OTP that need review before Unicode atoms can be adopted. From ivan@REDACTED Tue Feb 2 09:54:31 2016 From: ivan@REDACTED (Ivan Uemlianin) Date: Tue, 02 Feb 2016 08:54:31 +0000 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: <6851594.7I7yDfpvbI@changa> Message-ID: <1454403295675-fb032df6-2dbc0a32-8edd8c89@llaisdy.com> Welsh (spoken in UK) is not supported by Latin-1. On Tue, Feb 2, 2016 at 8:33 am, ok@REDACTED wrote: I should point out that Unicode isn't just useful for Chinese, Japanese, Korean, Mongolian, Thai. Hindi, Ancient Egyptian, & so on. There are plenty of languages basically using the Latin script, but with some extra letters or tweaks. Take as an example one of the official languages of my country, which uses lower- and upper-case vowels with macrons. If one is allowed to write english_name and ?venjulegt_nafn (Icelandic) it is a little difficult to explain why ingoa_māori (Māori) should be forbidden. Now Māori is definitely a minority language, but then so are Faroese, G?idhlig, Leonese, and Manx, supported by Latin 1. As a specific technical issue, Erlang allows unquoted atoms like roller_ball@REDACTED The internet has had internationalised domain names since 1998 (that would be about 17 years) and yes, they're mapped through Punycode, but if you have a domain name like pūtaiao-rorohiko.otāgo.wānanga.nz (which, to be fair, I *don't*, but it's surely only a matter of time) won't you want to be able to use it as or in an atom? By the way this means that domain names are one of the areas in Erlang/OTP that need review before Unicode atoms can be adopted. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Tue Feb 2 09:56:45 2016 From: ivan@REDACTED (Ivan Uemlianin) Date: Tue, 02 Feb 2016 08:56:45 +0000 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: <6851594.7I7yDfpvbI@changa> Message-ID: <1454403406896-dc0b056b-d4839bba-fec36673@llaisdy.com> Japanese prolog programmers have been writing their entire programs in Kanji for years. This whole debate is silly and parochial. Erlang programmers should be able to use full Unicode for atoms, including module and function names, and for variables. Ivan On Tue, Feb 2, 2016 at 7:52 am, Jos? Valim wrote: > Are you ready to edit chineese or thai? I can already write atoms in Afrikaans, Albanian, Basque, Breton, Corsican, Danish, English, Faroese, Galician, German, Icelandic, Indonesian, Italian, Kurdish, Leonese, Luxembourgish, Malay, Manx, Norwegian, Occitan, Portuguese, Rhaeto-Romanic, Scottish Gaelic, Spanish, Swahili, Swedish, Walloon. Source: [https://en.wikipedia.org/wiki/ISO/IEC_8859-1] https://en.wikipedia.org/wiki/ISO/IEC_8859-1 [https://en.wikipedia.org/wiki/ISO/IEC_8859-1] And I can only effectively speak 3 of those. There are already a lot of programmers who can write in their own language and I don't expect to edit their code. Not only that, there are other programming languages which have already adopted Unicode Support and the amount of times I had to read code in a language I don't understand has been exactly 0. I recommend reading Richard O'Keefe replies on the matter back in 2012: [http://erlang.org/pipermail/erlang-questions/2012-October/070187.html] http://erlang.org/pipermail/erlang-questions/2012-October/070187.html [http://erlang.org/pipermail/erlang-questions/2012-October/070187.html] > but a lot more thought would need to go into this than simply defining a parsing rule or cobbling together a patch that builds cleanly. I agree with this. That's why I want to focus on the underlying runtime support and not on how it will affect parsing. Specially because according to the OTP board post I mentioned above, they haven't made a decision if Unicode in variables should be allowed. Today Unicode is already supported in double quotes (and double quotes only) so I would say it is reasonable to allow them in single quotes as well. Should we move this discussion to another thread? I know this has been extensively discussed in the past and it would be nice to keep this discussion focused on the implementation details for whoever is following the trail in the future. Jos? Valim [http://www.plataformatec.com.br/] www.plataformatec.com.br [http://www.plataformatec.com.br] Skype: jv.ptec Founder and Director of R&D -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Tue Feb 2 13:26:25 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 2 Feb 2016 07:26:25 -0500 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: <6851594.7I7yDfpvbI@changa> Message-ID: <20160202122620.GA41498@fhebert-ltm2.internal.salesforce.com> On 02/02, Jos? Valim wrote: >> Are you ready to edit chineese or thai? > >I can already write atoms in Afrikaans, Albanian, Basque, Breton, Corsican, >Danish, English, Faroese, Galician, German, Icelandic, Indonesian, Italian, >Kurdish, Leonese, Luxembourgish, Malay, Manx, Norwegian, Occitan, >Portuguese, Rhaeto-Romanic, Scottish Gaelic, Spanish, Swahili, Swedish, >Walloon. > >Source: https://en.wikipedia.org/wiki/ISO/IEC_8859-1 > >And I can only effectively speak 3 of those. There are already a lot of >programmers who can write in their own language and I don't expect to edit >their code. > >Not only that, there are other programming languages which have already >adopted Unicode Support and the amount of times I had to read code in a >language I don't understand has been exactly 0. > I want to also voice my support for Unicode support. I don't know why everytime unicode is brought up in this mailing list a bunch of people suddenly fear having to edit code in a language they don't understand. This has been possible already for a long time (as pointed out by the list of languages Jos? added there). I think last time someone was being preemptively angry because they could be buying a business where code was in a different language and then they would be screwed! The horror. I've given presentations in a local user group where I used French for my programs. Nobody here has had to suffer through a non-english language when working or communicating with me though. It turns out people tend to be reasonable when it comes to communicating with various communities and preparing for that. I can vow to try to keep it so everyone's eyes don't get to suffer the view of my native language. There's plenty of other ways to make code unreadable anyway[1]. If the objective of refusing unicode would be to avoid making code unreadable, I'd suggest we also ban one-letter variables and atoms because they are clearly not English words (aside from 'A' and 'I', and maybe 'O' if you're feeling poetic in your interjections) and therefore would be too confusing. Regards, Fred. [1]: http://ferd.ca/obfuscation.html From jesper.louis.andersen@REDACTED Tue Feb 2 13:46:56 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 2 Feb 2016 13:46:56 +0100 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: <20160202122620.GA41498@fhebert-ltm2.internal.salesforce.com> References: <6851594.7I7yDfpvbI@changa> <20160202122620.GA41498@fhebert-ltm2.internal.salesforce.com> Message-ID: On Tue, Feb 2, 2016 at 1:26 PM, Fred Hebert wrote: > I want to also voice my support for Unicode support. > > I don't know why everytime unicode is brought up in this mailing list a > bunch of people suddenly fear having to edit code in a language they don't > understand. This has been possible already for a long time (as pointed out > by the list of languages Jos? added there). I think last time someone was > being preemptively angry because they could be buying a business where code > was in a different language and then they would be screwed! The horror. > I'm in favor of Unicode support for atoms as well as a generic construction. Squint your eyes enough and an atom() is merely a string literal of a specific kind. It makes sense to allow these to vary inside the Unicode world. When used as an "identifier" it may be necessary to define certain restrictions as to what symbol is allowed. The Go language defines that an identifier must start with either '_' or a Unicode symbol from the Letter class in order to avoid certain notational headaches such as the symbol '123' which cannot be written as 123 in Erlang either. In go, you could write _123, but that will get through the erlang parser: it is a variable since it starts with the wildcard character. There are some places where notation can help a lot. When transcribing math algorithms, it is often more readable to use the symbols used in the mathematical text directly in the source code as binding values. And the above remarks also suggest it is better to give the programmer the freedom to choose here, rather than restricting what the user can do. Syntax is nothing, semantics is everything :) -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Tue Feb 2 13:59:59 2016 From: zxq9@REDACTED (zxq9) Date: Tue, 02 Feb 2016 21:59:59 +0900 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: <20160202122620.GA41498@fhebert-ltm2.internal.salesforce.com> References: <20160202122620.GA41498@fhebert-ltm2.internal.salesforce.com> Message-ID: <3830815.3tYlNSuWDZ@changa> On 2016?2?2? ??? 07:26:25 you wrote: > On 02/02, Jos? Valim wrote: > >> Are you ready to edit chineese or thai? > > > >I can already write atoms in Afrikaans, Albanian, Basque, Breton, Corsican, > >Danish, English, Faroese, Galician, German, Icelandic, Indonesian, Italian, > >Kurdish, Leonese, Luxembourgish, Malay, Manx, Norwegian, Occitan, > >Portuguese, Rhaeto-Romanic, Scottish Gaelic, Spanish, Swahili, Swedish, > >Walloon. > > > >Source: https://en.wikipedia.org/wiki/ISO/IEC_8859-1 > > > >And I can only effectively speak 3 of those. There are already a lot of > >programmers who can write in their own language and I don't expect to edit > >their code. > > > >Not only that, there are other programming languages which have already > >adopted Unicode Support and the amount of times I had to read code in a > >language I don't understand has been exactly 0. > > > > I want to also voice my support for Unicode support. > > I don't know why everytime unicode is brought up in this mailing list a > bunch of people suddenly fear having to edit code in a language they > don't understand. This has been possible already for a long time (as > pointed out by the list of languages Jos? added there). I think last > time someone was being preemptively angry because they could be buying a > business where code was in a different language and then they would be > screwed! The horror. I don't know that anyone is objecting, exactly. For my part I'm just hoping that its not (yet another) naive implementation that traps us with a "darn, that was soooo close to actually being really useful" sort of situation. For example, if there were a way to leave the door open to atoms being recognized unquoted based on script set (the same way capitalization works with Latin characters), that could indeed be super useful (I'm being selfish here, of course, and thinking of the kana situation in Japanese). A comment earlier was that "Japanese prolog programmers have been writing their entire programs in Kanji for years." <- something I have seen exactly zero cases of outside of teaching examples. I've never even seen this at NEC, and they are pretty big on taking advantage of native language facilities or building them themselves when possible. Sure, its *possible*, but its also a pita most of the time. Its a lot easier to just stay in romaji input mode than flip around between kana and kanji symbols and halfwidth ASCII symbols necessary for brackets, colons, arrows and whatnot (granted, some languages actually do accept full-width symbols for arrows, brackets, etc, but that's a LOT more rare than just accepting kana/kanji characters and thinking that's sufficient). The only place in CJK land I've ever seen much use of unicode source is in various lisp-based scripting and control languages (because they usually accept both types of parens and full-width numbers). I'm all for unicode. It is indeed looooong overdue. I hope that whatever way it happens to get implemented makes it actually useful instead of just being another "me, too!" sort of feature... which is how I will feel if it turns out like most languages: Possible -> Yes; practical -> no. Some of Richard's examples were especially tricky-looking, and they weren't even hw/fw type issues. Speaking of being overdue... when we say "atoms" do we also mean modules and functions or... ? The concern that suddenly code will become unreadable because the Tower of ASCII Babel has fallen is ridiculous. I'm more interested specifically in the opposite problem, where unicode support exists, but it isn't useful enough to make it a practical option in real code intended to do more than just introduce kids to programming. -Craig From hugo@REDACTED Tue Feb 2 14:35:36 2016 From: hugo@REDACTED (Hugo Mills) Date: Tue, 2 Feb 2016 13:35:36 +0000 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: <20160202122620.GA41498@fhebert-ltm2.internal.salesforce.com> References: <6851594.7I7yDfpvbI@changa> <20160202122620.GA41498@fhebert-ltm2.internal.salesforce.com> Message-ID: <20160202133536.GF8313@carfax.org.uk> On Tue, Feb 02, 2016 at 07:26:25AM -0500, Fred Hebert wrote: > On 02/02, Jos? Valim wrote: > >>Are you ready to edit chineese or thai? > > > >I can already write atoms in Afrikaans, Albanian, Basque, Breton, Corsican, > >Danish, English, Faroese, Galician, German, Icelandic, Indonesian, Italian, > >Kurdish, Leonese, Luxembourgish, Malay, Manx, Norwegian, Occitan, > >Portuguese, Rhaeto-Romanic, Scottish Gaelic, Spanish, Swahili, Swedish, > >Walloon. > > > >Source: https://en.wikipedia.org/wiki/ISO/IEC_8859-1 > > > >And I can only effectively speak 3 of those. There are already a lot of > >programmers who can write in their own language and I don't expect to edit > >their code. > > > >Not only that, there are other programming languages which have already > >adopted Unicode Support and the amount of times I had to read code in a > >language I don't understand has been exactly 0. > > > > I want to also voice my support for Unicode support. > > I don't know why everytime unicode is brought up in this mailing > list a bunch of people suddenly fear having to edit code in a > language they don't understand. This has been possible already for > a long time (as pointed out by the list of languages Jos? added > there). I think last time someone was being preemptively angry > because they could be buying a business where code was in a > different language and then they would be screwed! The horror. As a native English speaker, I've encountered code in a non-English language (Greek, transliterated to ASCII, in this case) precisely once, and that was merely a dismal (failed) attempt at hiding the fact that they'd copied their coursework from someone else, rather than a genuine need or desire to use Greek. As an (ex-)mathematician, I really like the idea of having support for non-ASCII symbols in code. It's not something to use as a matter of course, but sometimes, very occasionally, I want to be able to use notation in my code which is closer to the way I'd write it down on a piece of paper. So, yes, plenty of encouragement from the peanut gallery, with an observation that the cased nature of Roman script is the exception rather than the norm. Using a case distinction for the atom/variable distinction in Erlang is likely to be problematic, and is probably going to need some serious thought about how to solve it, possibly on a per-script basis. Hugo. > I've given presentations in a local user group where I used French > for my programs. Nobody here has had to suffer through a non-english > language when working or communicating with me though. It turns out > people tend to be reasonable when it comes to communicating with > various communities and preparing for that. > > I can vow to try to keep it so everyone's eyes don't get to suffer > the view of my native language. There's plenty of other ways to make > code unreadable anyway[1]. > > If the objective of refusing unicode would be to avoid making code > unreadable, I'd suggest we also ban one-letter variables and atoms > because they are clearly not English words (aside from 'A' and 'I', > and maybe 'O' if you're feeling poetic in your interjections) and > therefore would be too confusing. > > Regards, > Fred. > > [1]: http://ferd.ca/obfuscation.html > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Hugo Mills | Klytus, I'm bored. What plaything can you offer me hugo@REDACTED carfax.org.uk | today? http://carfax.org.uk/ | PGP: E2AB1DE4 | Ming the Merciless, Flash Gordon -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From pierrefenoll@REDACTED Tue Feb 2 14:52:39 2016 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Tue, 2 Feb 2016 14:52:39 +0100 Subject: [erlang-questions] Erlang + Ejabberd In-Reply-To: References: Message-ID: Mohit, The syntax of compile is wrong. Replace -compile[{$1}, {$2}] With -compile([{$1, $2}]). > On 02 Feb 2016, at 04:56, mohit Agrawal wrote: > > Hi Team, > > I am new to Erlang, I have been trying to install Erlang and Ejabberd on EC2 ubuntu machine, everything went well till I started compiling some external modules in ejabberd. It started throwing error "undefined parse transform 'lager_transform'". I tried everything which is as below : > > Did rebar get-deps, make clean, make deps, make install. After this I am able to see that lager_transform.beam is made and present in //lib/ folder. > Checked rebar.config file, it had lager deps on top, which is widely suggested, no help even after that. > Added -compile[{parse_tranform},{lager_transform}] on top of module, even then their is no luck. > I am really blocked on this, and not able to complete the installation. I have done this before on fedora with Ejabberd 15.11 and otp 18.2, at that time it was using p1_logger instead of lager. But now when I am installing fresh with otp 18.2 and Ejabberd 16.0 or 15.11, I am getting lager_transform undefined error. > > Please suggest and help > > -- > Mohit Agrawal > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmitriid@REDACTED Tue Feb 2 14:59:07 2016 From: dmitriid@REDACTED (Dmitrii Dimandt) Date: Tue, 02 Feb 2016 13:59:07 +0000 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: Message-ID: On Sat, Jan 30, 2016 at 9:04 PM, Jos? Valim > wrote: > > > > With all that said, are there any plans of supporting UTF-8 encoded > atoms on > > Erlang R19? If the feature is desired but not planned, I would love to > > contribute the compiler and bytecode changes above although I will likely > > need some guidance. If that is an option, I would love to get in touch. > > > > It is not planned for OTP 19. IMO, the feature is desired, > but it is probably too late for OTP 19. > > Extending the BEAM format is necessary but not sufficient. > It is also necessary to make sure that other code in OTP > doesn't break. > > In order to try and derail the "omg why unicode in atoms" discussion, I have a more pressing questions: are there plans for expanding Unicode support elsewehere? Hoping for at least a subset of https://github.com/erlang-unicode/i18n namely length, to_lower, to_upper etc. :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Tue Feb 2 16:30:28 2016 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 2 Feb 2016 16:30:28 +0100 Subject: [erlang-questions] [ANN] erlavro - Apache Avro implementation Message-ID: https://github.com/klarna/erlavro License: Apache 2.0 Caveat: It currently only implements the parts that we have needed for our internal use. /Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: From mohit3081989@REDACTED Tue Feb 2 15:38:36 2016 From: mohit3081989@REDACTED (mohit Agrawal) Date: Tue, 2 Feb 2016 20:08:36 +0530 Subject: [erlang-questions] Erlang + Ejabberd In-Reply-To: References: Message-ID: Thanks Pierre, Sorry for typo, I did the same thing as you mentioned. Because even after putting compile thing I was getting same error. On 2 Feb 2016 7:22 pm, "Pierre Fenoll" wrote: > Mohit, > > The syntax of compile is wrong. > > Replace > -compile[{$1}, {$2}] > With > -compile([{$1, $2}]). > > > On 02 Feb 2016, at 04:56, mohit Agrawal wrote: > > Hi Team, > > I am new to Erlang, I have been trying to install Erlang and Ejabberd on > EC2 ubuntu machine, everything went well till I started compiling some > external modules in ejabberd. It started throwing error "undefined parse > transform 'lager_transform'". I tried everything which is as below : > > > 1. Did rebar get-deps, make clean, make deps, make install. After this > I am able to see that lager_transform.beam is made and present in //lib/ > folder. > 2. Checked rebar.config file, it had lager deps on top, which is > widely suggested, no help even after that. > 3. Added -compile[{parse_tranform},{lager_transform}] on top of > module, even then their is no luck. > > I am really blocked on this, and not able to complete the installation. I > have done this before on fedora with Ejabberd 15.11 and otp 18.2, at that > time it was using p1_logger instead of lager. But now when I am installing > fresh with otp 18.2 and Ejabberd 16.0 or 15.11, I am getting > lager_transform undefined error. > > Please suggest and help > > -- > Mohit Agrawal > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mremond@REDACTED Tue Feb 2 16:39:12 2016 From: mremond@REDACTED (=?utf-8?Q?Micka=C3=ABl_R=C3=A9mond?=) Date: Tue, 2 Feb 2016 16:39:12 +0100 Subject: [erlang-questions] Erlang + Ejabberd In-Reply-To: References: Message-ID: Hello, > On 02 Feb 2016, at 04:56, mohit Agrawal wrote: > > Hi Team, > > I am new to Erlang, I have been trying to install Erlang and Ejabberd on EC2 ubuntu machine, everything went well till I started compiling some external modules in ejabberd. It started throwing error "undefined parse transform 'lager_transform'". I tried everything which is as below : Please make sure you compile ejabberd as described in documentation: http://docs.ejabberd.im/admin/guide/installation/#installing-ejabberd-from-source-code For example, you can compile it with: ./configure --enable-lager --enable-mysql make You need to run configure and make to set properly the build chain. -- Micka?l R?mond http://www.process-one.net -------------- next part -------------- An HTML attachment was scrubbed... URL: From ggaliens@REDACTED Tue Feb 2 16:41:43 2016 From: ggaliens@REDACTED (ggaliens@REDACTED) Date: Tue, 2 Feb 2016 15:41:43 +0000 Subject: [erlang-questions] Erlang Travelling Salesman or floyd-warshall example. Message-ID: <20160202154143.1FTG9.4955.root@cdptpa-web13> Looking for an Erlang Travelling Salesman or floyd-warshall example. To take a graph of path weights between nodes and prune it down to shortest path. I found one example on net ... but after cut/paste ... it just seems to be the identify function for its matrix inputs. Seems like something I'll debug/implement if I don't find a canned example soon. From felixgallo@REDACTED Tue Feb 2 17:02:12 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Tue, 2 Feb 2016 08:02:12 -0800 Subject: [erlang-questions] Erlang Travelling Salesman or floyd-warshall example. In-Reply-To: <20160202154143.1FTG9.4955.root@cdptpa-web13> References: <20160202154143.1FTG9.4955.root@cdptpa-web13> Message-ID: try the 'digraph' module on for size. Might not be an exact fit, but might be shoehornable. http://erlang.org/doc/man/digraph.html F. On Tue, Feb 2, 2016 at 7:41 AM, wrote: > Looking for an Erlang Travelling Salesman or floyd-warshall example. > > To take a graph of path weights between nodes and prune it down to > shortest path. I found one example on net ... but after cut/paste ... it > just seems to be the identify function for its matrix inputs. > > Seems like something I'll debug/implement if I don't find a canned example > soon. > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From donpedrothird@REDACTED Tue Feb 2 17:55:46 2016 From: donpedrothird@REDACTED (John Doe) Date: Tue, 2 Feb 2016 19:55:46 +0300 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: Message-ID: Yes, to_upper and to_lower would be nice to have, but I think the most important is to_nfc(). to_nfc() and to_nfd() are the basis for unicode support, and for distinct string() type, incompatible with both lists and binaries. About unicode atoms and nfc - are you ready to have atoms 'e?' and '?', which are actually both the same and different in the same time? The one of them is 2 byte long in utf-8, another is 3 bytes long. But it is still the same character, just in nfc and in nfd forms. Both variants are used in the wild by different editors. It may look harmless in most cases, but wait until you hash the value and use the hash for the password check. I myself born and live in a country with non-latin script, and use unicode every day in software, but allowing unicode atoms isn't the most important thing about supporting proper unicode in Erlang. 2016-02-02 16:59 GMT+03:00 Dmitrii Dimandt : > On Sat, Jan 30, 2016 at 9:04 PM, Jos? Valim > >> wrote: >> > >> > With all that said, are there any plans of supporting UTF-8 encoded >> atoms on >> > Erlang R19? If the feature is desired but not planned, I would love to >> > contribute the compiler and bytecode changes above although I will >> likely >> > need some guidance. If that is an option, I would love to get in touch. >> > >> >> It is not planned for OTP 19. IMO, the feature is desired, >> but it is probably too late for OTP 19. >> >> Extending the BEAM format is necessary but not sufficient. >> It is also necessary to make sure that other code in OTP >> doesn't break. >> >> > > In order to try and derail the "omg why unicode in atoms" discussion, I > have a more pressing questions: are there plans for expanding Unicode > support elsewehere? Hoping for at least a subset of > https://github.com/erlang-unicode/i18n namely length, to_lower, to_upper > etc. :) > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Tue Feb 2 21:05:41 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Tue, 2 Feb 2016 20:05:41 +0000 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: <20160201231049.08b28dc7@gmail.com> References: <20160201231049.08b28dc7@gmail.com> Message-ID: Hi, Can you share some information about how you are setting up your UDP sockets? Buffer sizes, active mode etc? Are all 400 clients sending to the same UDP socket? cheers, Chandru On 1 February 2016 at 19:40, Ameretat Reith wrote: > I'm playing with Erlang to make an experimental protocol. I'm trying > to make it use full of 1Gbit link but It won't scale that much and I'm > failing to found a bottleneck in my code or even anything I could call > it bottleneck. > > My software is very like a messaging server software in behavior, with > bigger packets, many clients (more than 4k) and uses more complex > sub-components, like a distributed database but those components are > not blocking other portions of system; It's just the client-server > channel that is heavy IO and involve some encryption and decryption. > > I made a gen_server process for each UDP socket to clients. There is a > central process registry but It being called just for new clients and > Its message queue is often empty. > > I found there was a bottleneck in `scheduler_wait` when I had few > clients (around 400) and It consumed around 50% of total CPU usage. I > found an old patch by Wei Cao [1] which seemed to target same issue. > But on a modern version of Erlang (18.0) blockage in `scheduler_wait` > dropped well in more congested network, specifically to around 10% > when my software reached Its apparent limit, around 600Mbit/s read and > write to network. At this point my incoming UDP packet rate is around > 24K/s. Maybe an experienced Erlang developer here can remember that > problem and can tell whether Erlang is now optimized to poll for > network packets more often or not.. > > I also concerned async pool since there was fairly high work in Erlang > work with pthread but found those threads just used for file IO > operations. I didn't found any assuring documentation about this, just > saw the only user of this dirty IO thing is `io.c` in otp source code. > I'm very grateful if anyone clear the usage and effect of this pool. > > > I made flame graphs of function calls both inside VM (using eflame2 > [2]) which is very even and cannot find any outstanding usage [3]. And > made another flamegraph of perf report outside of VM which cannot find > some symbols [4]. I doubt whether process_main shoud take that much > work itself or not. Apparently encryption and decryption (enacl_nif > calls) didn't take much time too. > > Do you have any suggestion for me to analyze better my software and > understand VM working? Is It those limits I should expect and there is > not more room for optimizations? > > Thanks in advance > > 1: http://erlang.org/pipermail/erlang-questions/2012-July/067868.html > 2: https://github.com/slfritchie/eflame > 3: http://file.reith.ir/in-erl-3k.gif > 4: http://file.reith.ir/out-erl-perf.svg (interactive, use web browser) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stamarit@REDACTED Tue Feb 2 23:23:09 2016 From: stamarit@REDACTED (Salvador Tamarit) Date: Tue, 2 Feb 2016 23:23:09 +0100 Subject: [erlang-questions] PROHA 2016 (@ CGO'16): Early Registration Deadline (Feb 3) Message-ID: ****************************************************************************** PROHA 2016 -- CALL FOR PARTICIPATION First Workshop on Program Transformation for Programmability in Heterogeneous Architectures Website: http://goo.gl/RzGbzY Barcelona, 12th March 2016 In conjunction with the CGO'16 Conference ==== Early Bird Registration: February 3rd, 2016 ==== For registration information, please see http://cgo.org/cgo2016/registration/ ****************************************************************************** The PROHA workshop focuses on techniques and foundations to make it possible to perform source code transformations that preserve the intended semantics of the original code and improve efficiency, portability or maintainability. The topics of interest for the workshop include, non-exclusively: program annotations to capture algorithmic properties and intended code semantics; programming paradigms able to express underlying (mathematical) properties of code; usage of dynamic and static mechanisms to infer relevant code properties; transformations which preserve intended semantics; strategies to apply transformations; heuristics to guide program transformation and techniques to synthesize / learn these heuristics; tools supporting the aforementioned topics. Venue: Gran Hotel Princesa Sofia, Barcelona, Spain. Please consult the workshop website (http://goo.gl/RzGbzY) for an up-to-date program. ****************************************************************************** -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Wed Feb 3 00:08:59 2016 From: n.oxyde@REDACTED (Anthony Ramine) Date: Wed, 3 Feb 2016 00:08:59 +0100 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: Message-ID: That feature is being worked on by someone from the Elixir community. They already solved the normalization forms and case conversions. I don't think we can blame Jos? for working on something else. :) > Le 2 f?vr. 2016 ? 17:55, John Doe a ?crit : > > I myself born and live in a country with non-latin script, and use unicode every day in software, but allowing unicode atoms isn't the most important thing about supporting proper unicode in Erlang. -------------- next part -------------- An HTML attachment was scrubbed... URL: From yueyoum@REDACTED Wed Feb 3 04:28:40 2016 From: yueyoum@REDACTED (=?UTF-8?B?5pyI5b+n6IyX?=) Date: Wed, 3 Feb 2016 11:28:40 +0800 Subject: [erlang-questions] How to write a MMO game server in Erlang? Message-ID: I try to using Erlang to implement my tiny mmo game server. I know the skeleton of c++ version is like this: int main() { GameApp app; int interval = 50; //50ms, 20 ticks per second. int sleep = 0; uint64_t tick_start, tick_end, tick_passed; while(true) { tick_start = CurrentTick(); // all logic, timer will be drive in the app.tick app.tick(tick); tick_end = CurrentTick(); tick_passed = tick_end - tick_start; sleep = interval - tick_passed; if (sleep < 0) sleep = 0; SleepMilliSeconds(sleep); } } But now, I have no idea about the Erlang version. in the erlang world, one client is one process (maybe gen_server). the process has it's own loop, and timer can use erlang:send_after or erlang:start_timer . I want to know how to implement a MMO server in erlang ? consider this situation: Player A VS Player B. A cast a skill X to some direction, then X flying. the server need to detect whether X is hit on B, When hit, add X's debuff on B, debuff will remove after 3 seconds. ...... many things , many times , many sync ... c++ has a signal tick, But how about erlang ? -- My GitHub https://github.com/yueyoum -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Wed Feb 3 04:45:06 2016 From: zxq9@REDACTED (zxq9) Date: Wed, 03 Feb 2016 12:45:06 +0900 Subject: [erlang-questions] How to write a MMO game server in Erlang? In-Reply-To: References: Message-ID: <96988179.bqEsaCeARG@changa> On 2016?2?3? ??? 11:28:40 ??? wrote: > I try to using Erlang to implement my tiny mmo game server. > > I know the skeleton of c++ version is like this: > > int main() > { > GameApp app; > int interval = 50; //50ms, 20 ticks per second. > int sleep = 0; > > uint64_t tick_start, tick_end, tick_passed; > while(true) > { > tick_start = CurrentTick(); > // all logic, timer will be drive in the app.tick > app.tick(tick); > tick_end = CurrentTick(); > tick_passed = tick_end - tick_start; > > sleep = interval - tick_passed; > if (sleep < 0) sleep = 0; > SleepMilliSeconds(sleep); > > } > } > > But now, I have no idea about the Erlang version. > > in the erlang world, one client is one process (maybe gen_server). the > process has it's own loop, and timer can use erlang:send_after or > erlang:start_timer . > > I want to know how to implement a MMO server in erlang ? > > consider this situation: > > Player A VS Player B. > > A cast a skill X to some direction, then X flying. > > the server need to detect whether X is hit on B, > > When hit, add X's debuff on B, debuff will remove after 3 seconds. > > ...... > > many things , many times , many sync ... > > c++ has a signal tick, > > But how about erlang ? It depends on the combat system, specifically, but generally speaking spawning a new process to represent the combat itself is the easiest way to handle this. In a turn-based fight this is easy: Spawn a combat process with the involved characters communicating with it. The characters send their combat inputs as messages (synch and asynch), and the combat process itself moderates interaction rules, timing and renders outcomes -- broadcasting events to all parties. Because the combat process is broadcasting interactions it can also determine if some actions are supposed to not be broadcast to some parties -- for example, if one character is a thief and has "steal" X% and steals from character A, character A may not be able to detect the attempt but character B (who is not the thief) may have noticed it. Etc. In a moderated-time fight things work similarly (like World of Warcraft): Spawn a process to manage the combat, and add characters as they become involved in that particular fight. Fights generally work as meshes in this case, where if A is targetting B, and B is targetting C, they are all three managed by a single combat process. If A and B are fighting and C and D are fighting, and then A and C wind up targetting each other then the older process takes over completely and inherits the younger fight's history. In games where there are complex area effect and relative experience effects (based on party grouping, etc.) then it is more common to have the process that controls the *location* be the combat arbitration process. In a real-time combat game... I'm not sure, because I've never written one that was FPS, but I think the only solution would be to have the location manager itself do combat arbitration. An alternative approach is to have the character processes themselves moderate their own combat interactions -- and this is totally possible -- but it can be extremely complicated because there will be a mix of synch and asynch messages necessary based on the type of interaction involved, and having symmetric protocols between identical peer processes is a recipe for creating lots of weird deadlocks. (The best way I've seen to deal with peer combat deadlocks, by the way, is to actually use them as part of the combat system itself -- having timeouts become a useful effect of their own. This also means that the connection process, the character control process and the character process itself have to each be separate, though, otherwise a combat deadlock also becomes a network, chat, and admin deadlock, which is usually not acceptable.) There are many approaches -- but first you have to let go of this idea that there is a single global tic. One of these days I will finally get back to working on the example game server... -Craig From zxq9@REDACTED Wed Feb 3 07:00:02 2016 From: zxq9@REDACTED (zxq9) Date: Wed, 03 Feb 2016 15:00:02 +0900 Subject: [erlang-questions] Request feedback on example project Message-ID: <1771611.uqjCtSRlIV@changa> Two days ago I wrote a simple UUID utility as an example of bit syntax. Yesterday I decided I wanted it to be an example of documentation and readability as well. I would like feedback from new and old Erlangers in terms of readability, documentation, what is easy to understand, what is hard to understand, etc. My goal is to develop a style plain enough that a utility application just above the threshold of clear triviality (like a UUID utility) is mostly understandable even to someone new to Erlang. https://github.com/zxq9/zuuid No ninja tricks should be in there. If there are any please bring them to my attention so they can be removed. Thanks -Craig From bjorn@REDACTED Wed Feb 3 07:17:02 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 3 Feb 2016 07:17:02 +0100 Subject: [erlang-questions] Bit syntax matching gotchas Message-ID: There are some gotchas in the bit syntax that comes up now and then on the mailing list, and also as a bug report at the end of last year: http://bugs.erlang.org/browse/ERL-44 We have decided to try do something about it in OTP 19. We have discussed various solutions internally in the OTP group, and I have spent far too much time lately thinking about it. Here follows first my summary of the issues, and then my suggestion how the compiler should be modified. BACKGROUND ABOUT BIT SYNTAX CONSTRUCTION When constructing binaries, there is an implicit masking of the values. All of the following constructions give the same result: <<255>> <<16#FF>> <<16#FFFF>> <<-1>> There have been complaints about the implicit masking behaviour, but there is a lot of code that depends on it, so it would be unwise to change it. THE PROBLEM There is no similar masking when matching values. That means that all of the following expressions will all fail to match: <<-1>> = <<-1>> <<-1/unsigned>> = <<-1> <<16#FF>> = <<16#FFFF>> <<-12345/signed>> = <<-12345/signed>> Let's look at how the compiler internally implements matching. Take this function as an example: f(<<-1:8/unsigned>>) -> ok. It will be rewritten to: f(<>) when V =:= -1 -> ok. That is, an unsigned value (in the range 0-255) will be stored in the variable V, which will then be compared to -1. POSSIBLE SOLUTION #1 The most obvious solution is probably to let the compiler warn for the above cases. The matching would still fail. The developer will need to fix their code. For example: <<-1/signed>> = <<-1>> POSSIBLE SOLUTION #2 There is one problem with the solution #1. It is not possible to produce a warning for the following example: f(Val) -> <> = <>, Val. So in addition to warning when possible, another solution is to mask values also when matching. Internally, the compiler could rewrite the function to something like: f(Val) -> <> = <>, Val = NewVar band 16#FF, Val. Similar rewriting should be done for literal integer, so the following expression would now match: <<-1>> = <<-1>> WHICH SOLUTION? Just to make to sure that I don't reject solution #2 just because it seems like a lot work to implement, I have actually implemented it. Now that I have implemented solution #2, I want to reject it. The reason I reject it is that the matching previously bound variables is uncommon. Even in the compiler test suites it is uncommon (test suites typically match bound variables more often than production code do). Therefore, solution #2 would make behaviour of matching more consistent with construction, but would not significantly increase the number of correct programs. Also, if clauses that previously didn't match start to match, then code that has not been executed before will be executed. Code that has not been tested usually doesn't work. Solution #1 would point all cases when literal integers could not possibly match and force the developer to fix them. Therefore I choose solution #1. YOUR INPUT? Are there better way to fix bit syntax matching? Anything I have forgotten or not thought about? /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bjorn@REDACTED Wed Feb 3 07:33:57 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 3 Feb 2016 07:33:57 +0100 Subject: [erlang-questions] Request feedback on example project In-Reply-To: <1771611.uqjCtSRlIV@changa> References: <1771611.uqjCtSRlIV@changa> Message-ID: On Wed, Feb 3, 2016 at 7:00 AM, zxq9 wrote: > I would like feedback from new and old Erlangers in terms of readability, > documentation, what is easy to understand, what is hard to understand, etc. > My goal is to develop a style plain enough that a utility application just > above the threshold of clear triviality (like a UUID utility) is mostly > understandable even to someone new to Erlang. After a quick look, I have only one nit to pick (and it is of course a matter of taste): You wrote: when V > 0 andalso V < 6 -> I would write it like: when 0 < V, V < 6 -> First, by putting the variable V in the middle it easier to see that the guard is a range test and not any two unrelated conditions. Second, I never use 'andalso' when a simple ',' would do. /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From zxq9@REDACTED Wed Feb 3 08:10:06 2016 From: zxq9@REDACTED (zxq9) Date: Wed, 03 Feb 2016 16:10:06 +0900 Subject: [erlang-questions] Request feedback on example project In-Reply-To: References: <1771611.uqjCtSRlIV@changa> Message-ID: <21732146.Rou2x0oCsi@changa> On 2016?2?3? ??? 07:33:57 you wrote: > On Wed, Feb 3, 2016 at 7:00 AM, zxq9 wrote: > > > I would like feedback from new and old Erlangers in terms of readability, > > documentation, what is easy to understand, what is hard to understand, etc. > > My goal is to develop a style plain enough that a utility application just > > above the threshold of clear triviality (like a UUID utility) is mostly > > understandable even to someone new to Erlang. > > After a quick look, I have only one nit to pick > (and it is of course a matter of taste): > > You wrote: > > when V > 0 andalso V < 6 -> > > I would write it like: > > when 0 < V, V < 6 -> > > First, by putting the variable V in the middle it > easier to see that the guard is a range test > and not any two unrelated conditions. Second, > I never use 'andalso' when a simple ',' would do. Hi Bj?rn! This particular spot actually made me pause for a moment. There should never be a chance where 0 > V because V is extracted from a match on bits: version({uuid, <<_:48, V:4, _:12, 2:2, _:62>>}) when V > 0 andalso V < 6 -> Therefore the following should be just as safe: version({uuid, <<_:48, V:4, _:12, 2:2, _:62>>}) when V < 6 -> But... 1- How obvious is this for someone new to Erlang? 2- If they are seeing this then they are reading the source; would a comment not be more understandable and leave the code more correct? This is certainly a nitpick -- but its the sort of tiny reader-focused nitpick I'm looking for. Thanks! -Craig From bjorn@REDACTED Wed Feb 3 08:33:28 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 3 Feb 2016 08:33:28 +0100 Subject: [erlang-questions] Request feedback on example project In-Reply-To: <21732146.Rou2x0oCsi@changa> References: <1771611.uqjCtSRlIV@changa> <21732146.Rou2x0oCsi@changa> Message-ID: On Wed, Feb 3, 2016 at 8:10 AM, zxq9 wrote: > > There should never be a chance where 0 > V because V is > extracted from a match on bits: > > version({uuid, <<_:48, V:4, _:12, 2:2, _:62>>}) > when V > 0 andalso V < 6 -> V > 0 means that V can't be 0... > Therefore the following should be just as safe: > > version({uuid, <<_:48, V:4, _:12, 2:2, _:62>>}) > when V < 6 -> ... therefore the modification is not safe. /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From zxq9@REDACTED Wed Feb 3 08:34:51 2016 From: zxq9@REDACTED (zxq9) Date: Wed, 03 Feb 2016 16:34:51 +0900 Subject: [erlang-questions] Request feedback on example project In-Reply-To: References: <1771611.uqjCtSRlIV@changa> <21732146.Rou2x0oCsi@changa> Message-ID: <2403938.jCW4QpEI81@changa> On 2016?2?3? ??? 08:33:28 you wrote: > On Wed, Feb 3, 2016 at 8:10 AM, zxq9 wrote: > > > > There should never be a chance where 0 > V because V is > > extracted from a match on bits: > > > > version({uuid, <<_:48, V:4, _:12, 2:2, _:62>>}) > > when V > 0 andalso V < 6 -> > > V > 0 means that V can't be 0... > > > Therefore the following should be just as safe: > > > > version({uuid, <<_:48, V:4, _:12, 2:2, _:62>>}) > > when V < 6 -> > > ... therefore the modification is not safe. Indeed! ...coffee... From dmitriid@REDACTED Wed Feb 3 08:38:02 2016 From: dmitriid@REDACTED (Dmitrii Dimandt) Date: Wed, 03 Feb 2016 07:38:02 +0000 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: Message-ID: I guess the question is for Erlang/OTP developers. I think Elixir has had Unicode support since early pre-pre-alphas :) On Wed, 3 Feb 2016 at 00:08, Anthony Ramine wrote: > That feature is being worked on by someone from the Elixir community. They > already solved the normalization forms and case conversions. > > I don't think we can blame Jos? for working on something else. :) > > Le 2 f?vr. 2016 ? 17:55, John Doe a ?crit : > > I myself born and live in a country with non-latin script, and use unicode > every day in software, but allowing unicode atoms isn't the most important > thing about supporting proper unicode in Erlang. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Wed Feb 3 09:52:48 2016 From: sergej.jurecko@REDACTED (=?UTF-8?Q?Sergej_Jure=C4=8Dko?=) Date: Wed, 3 Feb 2016 09:52:48 +0100 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: <20160201231049.08b28dc7@gmail.com> References: <20160201231049.08b28dc7@gmail.com> Message-ID: UDP performance in erlang is not that good. If I were you I would write a NIF. UDP is relatively simple to work with in C/C++. Sergej On Mon, Feb 1, 2016 at 8:40 PM, Ameretat Reith wrote: > I'm playing with Erlang to make an experimental protocol. I'm trying > to make it use full of 1Gbit link but It won't scale that much and I'm > failing to found a bottleneck in my code or even anything I could call > it bottleneck. > > My software is very like a messaging server software in behavior, with > bigger packets, many clients (more than 4k) and uses more complex > sub-components, like a distributed database but those components are > not blocking other portions of system; It's just the client-server > channel that is heavy IO and involve some encryption and decryption. > > I made a gen_server process for each UDP socket to clients. There is a > central process registry but It being called just for new clients and > Its message queue is often empty. > > I found there was a bottleneck in `scheduler_wait` when I had few > clients (around 400) and It consumed around 50% of total CPU usage. I > found an old patch by Wei Cao [1] which seemed to target same issue. > But on a modern version of Erlang (18.0) blockage in `scheduler_wait` > dropped well in more congested network, specifically to around 10% > when my software reached Its apparent limit, around 600Mbit/s read and > write to network. At this point my incoming UDP packet rate is around > 24K/s. Maybe an experienced Erlang developer here can remember that > problem and can tell whether Erlang is now optimized to poll for > network packets more often or not.. > > I also concerned async pool since there was fairly high work in Erlang > work with pthread but found those threads just used for file IO > operations. I didn't found any assuring documentation about this, just > saw the only user of this dirty IO thing is `io.c` in otp source code. > I'm very grateful if anyone clear the usage and effect of this pool. > > > I made flame graphs of function calls both inside VM (using eflame2 > [2]) which is very even and cannot find any outstanding usage [3]. And > made another flamegraph of perf report outside of VM which cannot find > some symbols [4]. I doubt whether process_main shoud take that much > work itself or not. Apparently encryption and decryption (enacl_nif > calls) didn't take much time too. > > Do you have any suggestion for me to analyze better my software and > understand VM working? Is It those limits I should expect and there is > not more room for optimizations? > > Thanks in advance > > 1: http://erlang.org/pipermail/erlang-questions/2012-July/067868.html > 2: https://github.com/slfritchie/eflame > 3: http://file.reith.ir/in-erl-3k.gif > 4: http://file.reith.ir/out-erl-perf.svg (interactive, use web browser) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Wed Feb 3 12:00:48 2016 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Wed, 3 Feb 2016 12:00:48 +0100 Subject: [erlang-questions] Bit syntax matching gotchas In-Reply-To: References: Message-ID: Bj?rn, in solution #1, would you warn only when matching or also when constructing? Is the warning only at compile-time or also at runtime? For example, would you warn for: X = -1. <>. We may have a third option which is to control the masking behaviour with a new flag. From Erlang 19, we could warn if you are relying on masking and you would need to pick to mask (/integer-mask) or fix your code. The default after the warning is removed is to always raise. The mask option wouldn't be supported when matching, making it clear the masking behaviour is construction only. I am not sure this is a *good* option but I thought I would mention it anyway. :) *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D On Wed, Feb 3, 2016 at 7:17 AM, Bj?rn Gustavsson wrote: > There are some gotchas in the bit syntax that comes up now and then on > the mailing list, and also as a bug report at the end of last year: > http://bugs.erlang.org/browse/ERL-44 > > We have decided to try do something about it in OTP 19. We have > discussed various solutions internally in the OTP group, and I have > spent far too much time lately thinking about it. > > Here follows first my summary of the issues, and then my suggestion > how the compiler should be modified. > > BACKGROUND ABOUT BIT SYNTAX CONSTRUCTION > > When constructing binaries, there is an implicit masking of the > values. All of the following constructions give the same result: > > <<255>> > <<16#FF>> > <<16#FFFF>> > <<-1>> > > There have been complaints about the implicit masking behaviour, but > there is a lot of code that depends on it, so it would be unwise to > change it. > > THE PROBLEM > > There is no similar masking when matching values. That means that all > of the following expressions will all fail to match: > > <<-1>> = <<-1>> > <<-1/unsigned>> = <<-1> > <<16#FF>> = <<16#FFFF>> > <<-12345/signed>> = <<-12345/signed>> > > Let's look at how the compiler internally implements matching. Take > this function as an example: > > f(<<-1:8/unsigned>>) -> ok. > > It will be rewritten to: > > f(<>) when V =:= -1 -> ok. > > That is, an unsigned value (in the range 0-255) will be stored in the > variable V, which will then be compared to -1. > > POSSIBLE SOLUTION #1 > > The most obvious solution is probably to let the compiler warn for the > above cases. The matching would still fail. The developer will need to > fix their code. For example: > > <<-1/signed>> = <<-1>> > > > POSSIBLE SOLUTION #2 > > There is one problem with the solution #1. It is not possible to > produce a warning for the following example: > > f(Val) -> > <> = <>, > Val. > > So in addition to warning when possible, another solution is to mask > values also when matching. Internally, the compiler could rewrite the > function to something like: > > f(Val) -> > <> = <>, > Val = NewVar band 16#FF, > Val. > > Similar rewriting should be done for literal integer, so the following > expression would now match: > > <<-1>> = <<-1>> > > > WHICH SOLUTION? > > Just to make to sure that I don't reject solution #2 just because it > seems like a lot work to implement, I have actually implemented it. > > Now that I have implemented solution #2, I want to reject it. > > The reason I reject it is that the matching previously bound variables > is uncommon. Even in the compiler test suites it is uncommon (test > suites typically match bound variables more often than production code > do). > > Therefore, solution #2 would make behaviour of matching more > consistent with construction, but would not significantly increase the > number of correct programs. Also, if clauses that previously didn't > match start to match, then code that has not been executed before will > be executed. Code that has not been tested usually doesn't work. > > Solution #1 would point all cases when literal integers could not > possibly match and force the developer to fix them. > > Therefore I choose solution #1. > > > YOUR INPUT? > > Are there better way to fix bit syntax matching? Anything I have > forgotten or not thought about? > > /Bj?rn > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Wed Feb 3 12:53:17 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Wed, 03 Feb 2016 11:53:17 +0000 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: Message-ID: unicode support... we lack of unicode collation though , would be cool to have something like ux shipped with erlang :) On Wed, 3 Feb 2016 at 08:38, Dmitrii Dimandt wrote: > I guess the question is for Erlang/OTP developers. I think Elixir has had > Unicode support since early pre-pre-alphas :) > On Wed, 3 Feb 2016 at 00:08, Anthony Ramine wrote: > >> That feature is being worked on by someone from the Elixir community. >> They already solved the normalization forms and case conversions. >> >> I don't think we can blame Jos? for working on something else. :) >> >> Le 2 f?vr. 2016 ? 17:55, John Doe a ?crit : >> >> I myself born and live in a country with non-latin script, and use >> unicode every day in software, but allowing unicode atoms isn't the most >> important thing about supporting proper unicode in Erlang. >> >> >> _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Feb 3 13:52:44 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 3 Feb 2016 15:52:44 +0300 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: Message-ID: Ok, ok, maybe I'm wrong and it is ok to have isolated pieces of code that is impossible to edit by foreign programmer. Problem in UTF8 is not only in nice hieroglyphs, but also in 20 different dash-like symbols and about 8 different spaces. This is one single atom: my package It is not 2 words, it is a single word that has non-breakable space inside it. Good luck for debugging =) Of course you may say me: hire programmer that makes such things. Ok, no problems. But what to do with copy-paste from skype/slack, where such symbols are translated into nice utf8 automatically? It is very good that we all have about 80-90 symbols to write code that other people understand, but I really don't understand what is the profit of adding ability to make code non-understandable by people from other cultures. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Wed Feb 3 14:20:31 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 3 Feb 2016 08:20:31 -0500 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: Message-ID: <20160203132030.GB41498@fhebert-ltm2.internal.salesforce.com> On 02/03, Max Lapshin wrote: >This is one single atom: my package >It is not 2 words, it is a single word that has non-breakable space inside >it. Good luck for debugging =) I believe the non-breakable space character is still in the space, separator [Zs] character class and should ideally be handled as such by a utf8-aware compiler. So the same way you'd need to type 'my package' (with a regular space), you'd need to type 'my?package' (with a nbsp). If this isn't respected, I'd probably expect this to be a language problem, not a unicode issue. I believe Erlang 17 and earlier would complain about invalid syntax there. Starting in 18, such characters are seen as valid spaces in a program and just go through directly the way a regular space does. > >Of course you may say me: hire programmer that makes such things. Ok, no >problems. But what to do with copy-paste from skype/slack, where such >symbols are translated into nice utf8 automatically? > Because I am a French-speaking user and non-breakable spaces have their place in regular usage. For example, : takes a leading narrow non-breakable space, and that space must be there while keeping the punctiation mark on the same line as its leading word. I have my editor set to hilight such leading spaces with a special character: set list listchars=tab:??,trail:?,nbsp:? All tabs, trailing spaces, and nonbreakable white space characters will show up in text. so 'my package' actually shows up as 'my?package' here, with some hilight color to make sure it's not just the literal '?' So assuming the code does not work properly, and that you are one of these programmers working with these characters on a day-to-day basis, there are still ways to work around it without confusion. That of course ignores specially crafted code built with the sole intention of confusing people (such as using the greek ? rather than the [whatever your locale] A in function or variable identifiers). >It is very good that we all have about 80-90 symbols to write code that >other people understand, but I really don't understand what is the profit >of adding ability to make code non-understandable by people from other >cultures. You make the assumption that without unicode, Japanese programmers would write code in English rather than transliterating it in a latin alphabet (say with ISO 3602) for example. This doesn't happen if the programmer does not know English, or if their target audience (coworkers for example) do not speak English. They just find a very annoying workaround to get their meaning across in the language they feel they should use. The reality is that if people feel like writing code in their own language, they will do so. If I'm writing code about an ATM in French, I might use the word 'guichet_automatique' or 'gab' instead of 'atm'. You would still be lost with a latin alphabet and lose all meaning. And the comments may very well be in French too, since they'd be to the attention of French speakers. So the benefit is that people can write code in their own native language unhindered, and it won't change anything to your comprehension of code because you likely wouldn't understand it anyway, or wouldn't be the target audience of said code and will in all likelihood just not see it in the first place. Regards, Fred. From pierrefenoll@REDACTED Wed Feb 3 14:52:15 2016 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Wed, 3 Feb 2016 14:52:15 +0100 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: <20160203132030.GB41498@fhebert-ltm2.internal.salesforce.com> References: <20160203132030.GB41498@fhebert-ltm2.internal.salesforce.com> Message-ID: <3724ADD3-90C9-4050-8A01-E033C9AC717E@gmail.com> What about re.erl character classes? I believe the regular expression [\s] does not match Unicode spaces, even when giving the unicode atom flag to re.erl functions. And there are other classes that Unicode defines that would be great for re.erl to support. > On 03 Feb 2016, at 14:20, Fred Hebert wrote: > >> On 02/03, Max Lapshin wrote: >> This is one single atom: my package >> It is not 2 words, it is a single word that has non-breakable space inside >> it. Good luck for debugging =) > > I believe the non-breakable space character is still in the space, separator [Zs] character class and should ideally be handled as such by a utf8-aware compiler. So the same way you'd need to type 'my package' (with a regular space), you'd need to type 'my package' (with a nbsp). > > If this isn't respected, I'd probably expect this to be a language problem, not a unicode issue. > > I believe Erlang 17 and earlier would complain about invalid syntax there. Starting in 18, such characters are seen as valid spaces in a program and just go through directly the way a regular space does. > >> >> Of course you may say me: hire programmer that makes such things. Ok, no >> problems. But what to do with copy-paste from skype/slack, where such >> symbols are translated into nice utf8 automatically? > > Because I am a French-speaking user and non-breakable spaces have their place in regular usage. For example, : takes a leading narrow non-breakable space, and that space must be there while keeping the punctiation mark on the same line as its leading word. > > I have my editor set to hilight such leading spaces with a special character: > > set list listchars=tab:??,trail:?,nbsp:? > > All tabs, trailing spaces, and nonbreakable white space characters will show up in text. so 'my package' actually shows up as 'my?package' here, with some hilight color to make sure it's not just the literal '?' > > So assuming the code does not work properly, and that you are one of these programmers working with these characters on a day-to-day basis, there are still ways to work around it without confusion. > > That of course ignores specially crafted code built with the sole intention of confusing people (such as using the greek ? rather than the [whatever your locale] A in function or variable identifiers). > >> It is very good that we all have about 80-90 symbols to write code that >> other people understand, but I really don't understand what is the profit >> of adding ability to make code non-understandable by people from other >> cultures. > > You make the assumption that without unicode, Japanese programmers would write code in English rather than transliterating it in a latin alphabet (say with ISO 3602) for example. This doesn't happen if the programmer does not know English, or if their target audience (coworkers for example) do not speak English. They just find a very annoying workaround to get their meaning across in the language they feel they should use. > > The reality is that if people feel like writing code in their own language, they will do so. If I'm writing code about an ATM in French, I might use the word 'guichet_automatique' or 'gab' instead of 'atm'. You would still be lost with a latin alphabet and lose all meaning. And the comments may very well be in French too, since they'd be to the attention of French speakers. > > So the benefit is that people can write code in their own native language unhindered, and it won't change anything to your comprehension of code because you likely wouldn't understand it anyway, or wouldn't be the target audience of said code and will in all likelihood just not see it in the first place. > > Regards, > Fred. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bchesneau@REDACTED Wed Feb 3 15:06:24 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Wed, 3 Feb 2016 15:06:24 +0100 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: <3724ADD3-90C9-4050-8A01-E033C9AC717E@gmail.com> References: <20160203132030.GB41498@fhebert-ltm2.internal.salesforce.com> <3724ADD3-90C9-4050-8A01-E033C9AC717E@gmail.com> Message-ID: > On 03 Feb 2016, at 14:52, Pierre Fenoll wrote: > > What about re.erl character classes? > > I believe the regular expression [\s] does not match Unicode spaces, even when giving the unicode atom flag to re.erl functions. > > And there are other classes that Unicode defines that would be great for re.erl to support. Yeah like the binary/string comparison that should respect the collation :) - benoit > >> On 03 Feb 2016, at 14:20, Fred Hebert wrote: >> >>> On 02/03, Max Lapshin wrote: >>> This is one single atom: my package >>> It is not 2 words, it is a single word that has non-breakable space inside >>> it. Good luck for debugging =) >> >> I believe the non-breakable space character is still in the space, separator [Zs] character class and should ideally be handled as such by a utf8-aware compiler. So the same way you'd need to type 'my package' (with a regular space), you'd need to type 'my package' (with a nbsp). >> >> If this isn't respected, I'd probably expect this to be a language problem, not a unicode issue. >> >> I believe Erlang 17 and earlier would complain about invalid syntax there. Starting in 18, such characters are seen as valid spaces in a program and just go through directly the way a regular space does. >> >>> >>> Of course you may say me: hire programmer that makes such things. Ok, no >>> problems. But what to do with copy-paste from skype/slack, where such >>> symbols are translated into nice utf8 automatically? >> >> Because I am a French-speaking user and non-breakable spaces have their place in regular usage. For example, : takes a leading narrow non-breakable space, and that space must be there while keeping the punctiation mark on the same line as its leading word. >> >> I have my editor set to hilight such leading spaces with a special character: >> >> set list listchars=tab:??,trail:?,nbsp:? >> >> All tabs, trailing spaces, and nonbreakable white space characters will show up in text. so 'my package' actually shows up as 'my?package' here, with some hilight color to make sure it's not just the literal '?' >> >> So assuming the code does not work properly, and that you are one of these programmers working with these characters on a day-to-day basis, there are still ways to work around it without confusion. >> >> That of course ignores specially crafted code built with the sole intention of confusing people (such as using the greek ? rather than the [whatever your locale] A in function or variable identifiers). >> >>> It is very good that we all have about 80-90 symbols to write code that >>> other people understand, but I really don't understand what is the profit >>> of adding ability to make code non-understandable by people from other >>> cultures. >> >> You make the assumption that without unicode, Japanese programmers would write code in English rather than transliterating it in a latin alphabet (say with ISO 3602) for example. This doesn't happen if the programmer does not know English, or if their target audience (coworkers for example) do not speak English. They just find a very annoying workaround to get their meaning across in the language they feel they should use. >> >> The reality is that if people feel like writing code in their own language, they will do so. If I'm writing code about an ATM in French, I might use the word 'guichet_automatique' or 'gab' instead of 'atm'. You would still be lost with a latin alphabet and lose all meaning. And the comments may very well be in French too, since they'd be to the attention of French speakers. >> >> So the benefit is that people can write code in their own native language unhindered, and it won't change anything to your comprehension of code because you likely wouldn't understand it anyway, or wouldn't be the target audience of said code and will in all likelihood just not see it in the first place. >> >> Regards, >> Fred. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mononcqc@REDACTED Wed Feb 3 15:11:01 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 3 Feb 2016 09:11:01 -0500 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: <3724ADD3-90C9-4050-8A01-E033C9AC717E@gmail.com> References: <20160203132030.GB41498@fhebert-ltm2.internal.salesforce.com> <3724ADD3-90C9-4050-8A01-E033C9AC717E@gmail.com> Message-ID: <20160203141100.GC41498@fhebert-ltm2.internal.salesforce.com> On 02/03, Pierre Fenoll wrote: >What about re.erl character classes? > >I believe the regular expression [\s] does not match Unicode spaces, even when giving the unicode atom flag to re.erl functions. > >And there are other classes that Unicode defines that would be great for re.erl to support. Pass in the `ucp' option: ucp Specifies that Unicode Character Properties should be used when resolving \B, \b, \D, \d, \S, \s, \W and \w. Without this flag, only ISO-Latin-1 properties are used. Using Unicode properties hurts performance, but is semantically correct when working with Unicode characters beyond the ISO-Latin-1 range. From seancribbs@REDACTED Wed Feb 3 15:51:32 2016 From: seancribbs@REDACTED (Sean Cribbs) Date: Wed, 3 Feb 2016 08:51:32 -0600 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: References: <20160201231049.08b28dc7@gmail.com> Message-ID: On what basis do you make that claim? Also, writing a NIF that actually provides better performance without blocking the scheduler is non-trivial, even if UDP is simple to work with in C. On Wed, Feb 3, 2016 at 2:52 AM, Sergej Jure?ko wrote: > UDP performance in erlang is not that good. If I were you I would write a > NIF. UDP is relatively simple to work with in C/C++. > > Sergej > > On Mon, Feb 1, 2016 at 8:40 PM, Ameretat Reith > wrote: > >> I'm playing with Erlang to make an experimental protocol. I'm trying >> to make it use full of 1Gbit link but It won't scale that much and I'm >> failing to found a bottleneck in my code or even anything I could call >> it bottleneck. >> >> My software is very like a messaging server software in behavior, with >> bigger packets, many clients (more than 4k) and uses more complex >> sub-components, like a distributed database but those components are >> not blocking other portions of system; It's just the client-server >> channel that is heavy IO and involve some encryption and decryption. >> >> I made a gen_server process for each UDP socket to clients. There is a >> central process registry but It being called just for new clients and >> Its message queue is often empty. >> >> I found there was a bottleneck in `scheduler_wait` when I had few >> clients (around 400) and It consumed around 50% of total CPU usage. I >> found an old patch by Wei Cao [1] which seemed to target same issue. >> But on a modern version of Erlang (18.0) blockage in `scheduler_wait` >> dropped well in more congested network, specifically to around 10% >> when my software reached Its apparent limit, around 600Mbit/s read and >> write to network. At this point my incoming UDP packet rate is around >> 24K/s. Maybe an experienced Erlang developer here can remember that >> problem and can tell whether Erlang is now optimized to poll for >> network packets more often or not.. >> >> I also concerned async pool since there was fairly high work in Erlang >> work with pthread but found those threads just used for file IO >> operations. I didn't found any assuring documentation about this, just >> saw the only user of this dirty IO thing is `io.c` in otp source code. >> I'm very grateful if anyone clear the usage and effect of this pool. >> >> >> I made flame graphs of function calls both inside VM (using eflame2 >> [2]) which is very even and cannot find any outstanding usage [3]. And >> made another flamegraph of perf report outside of VM which cannot find >> some symbols [4]. I doubt whether process_main shoud take that much >> work itself or not. Apparently encryption and decryption (enacl_nif >> calls) didn't take much time too. >> >> Do you have any suggestion for me to analyze better my software and >> understand VM working? Is It those limits I should expect and there is >> not more room for optimizations? >> >> Thanks in advance >> >> 1: http://erlang.org/pipermail/erlang-questions/2012-July/067868.html >> 2: https://github.com/slfritchie/eflame >> 3: http://file.reith.ir/in-erl-3k.gif >> 4: http://file.reith.ir/out-erl-perf.svg (interactive, use web browser) >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn@REDACTED Wed Feb 3 16:06:49 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Wed, 3 Feb 2016 16:06:49 +0100 Subject: [erlang-questions] Bit syntax matching gotchas In-Reply-To: References: Message-ID: On Wed, Feb 3, 2016 at 12:00 PM, Jos? Valim wrote: > Bj?rn, in solution #1, would you warn only when matching or also when > constructing? Is the warning only at compile-time or also at runtime? For > example, would you warn for: > > X = -1. > <>. Yes, I have considered warning for construction too. The warnings would only occur at compile-time. There would NOT be a warning for your example, though. More on that below. > We may have a third option which is to control the masking behaviour with a > new flag. From Erlang 19, we could warn if you are relying on masking and > you would need to pick to mask (/integer-mask) or fix your code. The default > after the warning is removed is to always raise. The mask option wouldn't be > supported when matching, making it clear the masking behaviour is > construction only. How could the compiler warn you that you depend on masking? Consider this code: f(A) -> <>. The compiler has no way of knowing whether masking is needed or not. We have no plans for changing how construction works in OTP 19. Regarding construction, note that signed/unsigned is ignored. All of the following examples construct the same binary: <<255>> <<-1>> <<-1/signed>> <<-1/unsigned>> Since signed/unsigned has never been enforced for construction, starting to warn for <<-1>> and <<-1/unsigned>> would cause many new annoying warnings. Therefore, no warnings should be produced. However, the following examples should produce warnings: <<-55555:8>> <<100000:8>> The two values above cannot be represented in 8 bits, which points to a real bug. /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From max.lapshin@REDACTED Wed Feb 3 17:23:08 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 3 Feb 2016 19:23:08 +0300 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: References: <20160201231049.08b28dc7@gmail.com> Message-ID: When we use plain gen_udp to accept 200-400 mbit of incoming MPEG-TS traffic in flussonic, we use about 50% of moderate 4 core xeon e3 server. When we switch to our driver implementation of udp that collapses several contiguous udp messages into single big message (it is allowed for mpegts) we reduce usage to 15-20% I can't tell that it is "badly written udp in erlang", just messaging is rather expensive. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Wed Feb 3 17:26:15 2016 From: sergej.jurecko@REDACTED (=?utf-8?Q?Sergej_Jure=C4=8Dko?=) Date: Wed, 3 Feb 2016 17:26:15 +0100 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: References: <20160201231049.08b28dc7@gmail.com> Message-ID: Yeah same experience and solution that we used. I presume there is a lot allocation/fragmentation going on with udp, since packets tend to be small and there are so many of them. Sergej > On 03 Feb 2016, at 17:23, Max Lapshin wrote: > > When we use plain gen_udp to accept 200-400 mbit of incoming MPEG-TS traffic in flussonic, we use about 50% of moderate 4 core xeon e3 server. > > When we switch to our driver implementation of udp that collapses several contiguous udp messages into single big message (it is allowed for mpegts) we reduce usage to 15-20% > > I can't tell that it is "badly written udp in erlang", just messaging is rather expensive. From lloyd@REDACTED Wed Feb 3 17:27:56 2016 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Wed, 3 Feb 2016 11:27:56 -0500 Subject: [erlang-questions] Request feedback on example project In-Reply-To: <2403938.jCW4QpEI81@changa> References: <1771611.uqjCtSRlIV@changa> <21732146.Rou2x0oCsi@changa> <2403938.jCW4QpEI81@changa> Message-ID: Three cheers, Craig, for focusing on clarity and quality of Erlang documentation. And apologies for the length of this post. As a know-nothing with little experience when I first set foot on the Erlang learning curve, I can attest to the frustration and rage that ensues when insufficient or opaque documentation blocks progress toward understanding and mastery. That said, in general, Erlang documentation is outstanding in its scope and detail. Let me express profound gratitude to the many who have labored to make it so. But there are holes and lapses. Here are a few that come to mind: - Libraries that offer no clue as to what problems they solve and why one would want to use them - lack of tutorials and working code that demonstrate how to use them - documentation that is overly abstract, appealing perhaps to mathematicians, but gibberish to the literal-minded newbie But I understand. Software documentation is damned difficult. Keeping code and docs in sync, providing sufficient detail for newbies without wasting the time of gurus, finding time and incentive to document all contribute to the challenge. Why bother? I'd argue that the easier it is to learn and master a language the wider the adoption. The wider the adoption, the more vibrant the community. The more vibrant the community, the more innovative, richer, and more functional the language. Thus, my suggestion toward better documentation is simply this: A simple, explicit, well-publicized, and widely adopted statement of documentation best practices with examples. I know that tidbits exist in the corpus here and there. But they need to be pulled together and prominently displayed So, Craig, thank you for moving the ball forward. Best wishes, LRP P.S. Just so happens that I was looking for UUID code the other day. Sent from my iPad > On Feb 3, 2016, at 2:34 AM, zxq9 wrote: > >> On 2016?2?3? ??? 08:33:28 you wrote: >>> On Wed, Feb 3, 2016 at 8:10 AM, zxq9 wrote: >>> >>> There should never be a chance where 0 > V because V is >>> extracted from a match on bits: >>> >>> version({uuid, <<_:48, V:4, _:12, 2:2, _:62>>}) >>> when V > 0 andalso V < 6 -> >> >> V > 0 means that V can't be 0... >> >>> Therefore the following should be just as safe: >>> >>> version({uuid, <<_:48, V:4, _:12, 2:2, _:62>>}) >>> when V < 6 -> >> >> ... therefore the modification is not safe. > > Indeed! > > > > ...coffee... > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From felixgallo@REDACTED Wed Feb 3 17:32:27 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Wed, 3 Feb 2016 08:32:27 -0800 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: <20160203141100.GC41498@fhebert-ltm2.internal.salesforce.com> References: <20160203132030.GB41498@fhebert-ltm2.internal.salesforce.com> <3724ADD3-90C9-4050-8A01-E033C9AC717E@gmail.com> <20160203141100.GC41498@fhebert-ltm2.internal.salesforce.com> Message-ID: There's also an interesting security issue around Unicode source code. Take for example the recent hack of Cryptsy, which involved a guy taking what looked like an innocent and safe pull request to fix an issue in one part of his software, but through the magic of the preprocessor, turned out to do something else entirely: http://earlz.net/view/2016/01/16/0717/analyzing-the-56-million-exploit-and-cryptsys-security and then set your gaze on: https://github.com/reinderien/mimic and then consider that, e.g., atoms are dynamically created on mention, without warning. F. On Wed, Feb 3, 2016 at 6:11 AM, Fred Hebert wrote: > On 02/03, Pierre Fenoll wrote: > >> What about re.erl character classes? >> >> I believe the regular expression [\s] does not match Unicode spaces, even >> when giving the unicode atom flag to re.erl functions. >> >> And there are other classes that Unicode defines that would be great for >> re.erl to support. >> > > Pass in the `ucp' option: > > ucp > Specifies that Unicode Character Properties should be used when > resolving \B, \b, \D, \d, \S, \s, \W and \w. Without this flag, only > ISO-Latin-1 properties are used. Using Unicode properties hurts > performance, but is semantically correct when working with Unicode > characters beyond the ISO-Latin-1 range. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Wed Feb 3 17:47:59 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 3 Feb 2016 11:47:59 -0500 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: <20160203132030.GB41498@fhebert-ltm2.internal.salesforce.com> <3724ADD3-90C9-4050-8A01-E033C9AC717E@gmail.com> <20160203141100.GC41498@fhebert-ltm2.internal.salesforce.com> Message-ID: <20160203164758.GD41498@fhebert-ltm2.internal.salesforce.com> On 02/03, Felix Gallo wrote: >There's also an interesting security issue around Unicode source code. > >Take for example the recent hack of Cryptsy, which involved a guy taking >what looked like an innocent and safe pull request to fix an issue in one >part of his software, but through the magic of the preprocessor, turned out >to do something else entirely: > >http://earlz.net/view/2016/01/16/0717/analyzing-the-56-million-exploit-and-cryptsys-security My counter-argument to that is that you don't need any of that cool UTF stuff to do that. See: - http://www.underhanded-c.org/ underhanded C contest is all about writing regular looking C code doing nasty stuff - http://arstechnica.co.uk/security/2015/12/researchers-confirm-backdoor-password-in-juniper-firewall-code/ juniper code was broken by someone adding in a password check that looked like a log line - http://arstechnica.com/security/2016/02/crypto-flaw-was-so-glaring-it-may-be-intentional-eavesdropping-backdoor/ using a non-prime in crypto communication, possibly being a backdoor. From felixgallo@REDACTED Wed Feb 3 18:00:39 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Wed, 3 Feb 2016 09:00:39 -0800 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: <20160203164758.GD41498@fhebert-ltm2.internal.salesforce.com> References: <20160203132030.GB41498@fhebert-ltm2.internal.salesforce.com> <3724ADD3-90C9-4050-8A01-E033C9AC717E@gmail.com> <20160203141100.GC41498@fhebert-ltm2.internal.salesforce.com> <20160203164758.GD41498@fhebert-ltm2.internal.salesforce.com> Message-ID: Oh for sure there's all sorts of hilarity in C. Doubtless in erlang, too. But the existence of other attack vectors doesn't suggest that you should ignore a new one. It'd probably be a good idea, if this were to be implemented, if there were some tooling or flags for the compiler to warn when unicode was used in a potentially dangerous setting, so that people taking pull requests on erlang code (or even just typing code wrong) could avoid some classes of possible exploits. On Wed, Feb 3, 2016 at 8:47 AM, Fred Hebert wrote: > On 02/03, Felix Gallo wrote: > >> There's also an interesting security issue around Unicode source code. >> >> Take for example the recent hack of Cryptsy, which involved a guy taking >> what looked like an innocent and safe pull request to fix an issue in one >> part of his software, but through the magic of the preprocessor, turned >> out >> to do something else entirely: >> >> >> http://earlz.net/view/2016/01/16/0717/analyzing-the-56-million-exploit-and-cryptsys-security >> > > My counter-argument to that is that you don't need any of that cool UTF > stuff to do that. > > See: > > - http://www.underhanded-c.org/ underhanded C contest is all about > writing regular looking C code doing nasty stuff > - > http://arstechnica.co.uk/security/2015/12/researchers-confirm-backdoor-password-in-juniper-firewall-code/ > juniper code was broken by someone adding in a password check that looked > like a log line > - > http://arstechnica.com/security/2016/02/crypto-flaw-was-so-glaring-it-may-be-intentional-eavesdropping-backdoor/ > using a non-prime in crypto communication, possibly being a backdoor. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Wed Feb 3 18:20:21 2016 From: zxq9@REDACTED (zxq9) Date: Thu, 04 Feb 2016 02:20:21 +0900 Subject: [erlang-questions] Bit syntax matching gotchas In-Reply-To: References: Message-ID: <1595893.feEBpiXfgA@changa> On 2016?2?3? ??? 16:06:49 Bj?rn Gustavsson wrote: > However, the following examples should produce > warnings: > > <<-55555:8>> > <<100000:8>> > > The two values above cannot be represented in > 8 bits, which points to a real bug. I have to admit that I've abused the bejesus out of this "bug" quite a bit without ever considering it to be peculiar. I do remember being surprised when I was first exploring Erlang and saw this happen, though: 1> A = 10000. 10000 2> B = <>. <<16>> It made sense after a moment, especially once I played some more, but I don't recall the peculiar properties of this ever being explained anywhere in practical terms. >How could the compiler warn you that you depend on masking? > >Consider this code: > >f(A) -> > <>. I suppose something like this would provide enough information: -spec f(<<_:32>>) -> <<_:16>>. f(A) -> <>. Or to be absolutely explicit: chop(In, Size) when is_binary(In) -> Mask = bit_size(In) - Size, <<_:Mask, Out:Size>> = In, <>. Is there a better way of doing that? I still expect to see things like `<> = SomeBin` all over the place. Meh. I do a fair amount of value assertion here and there in code, just not usually with binary syntax involved (that is, I do `A = B`, but never `<> = <>`). Some of the examples you presented have bizarre outcomes, though. When `A /= A`, syntax aside, something is wrong with the world. -Craig From jose.valim@REDACTED Wed Feb 3 18:25:55 2016 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Wed, 3 Feb 2016 18:25:55 +0100 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: <20160203132030.GB41498@fhebert-ltm2.internal.salesforce.com> <3724ADD3-90C9-4050-8A01-E033C9AC717E@gmail.com> <20160203141100.GC41498@fhebert-ltm2.internal.salesforce.com> <20160203164758.GD41498@fhebert-ltm2.internal.salesforce.com> Message-ID: I just wanted to clarify, to avoid any confusion, this patch is mostly about support for unicode atoms forms in the compiler. This is a required step if you ever want to support Unicode in the language but it does not imply such. There are three main milestones: 1. Support unicode atom forms (i.e. the form {atom, Line, Atom} can have a UTF-8 encoded atom) 2. Support unicode atoms between single quotes (for example, '???', as you can already write "???") 3. Support unicode in the language (for example, being able to write variables in Japanese) This discussion was originally related to 1 but I could contribute 2 if desired. Many of the concerns raised above are related to step 3 which, afaik, is not planned. *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D On Wed, Feb 3, 2016 at 6:00 PM, Felix Gallo wrote: > Oh for sure there's all sorts of hilarity in C. Doubtless in erlang, > too. But the existence of other attack vectors doesn't suggest that you > should ignore a new one. > > It'd probably be a good idea, if this were to be implemented, if there > were some tooling or flags for the compiler to warn when unicode was used > in a potentially dangerous setting, so that people taking pull requests on > erlang code (or even just typing code wrong) could avoid some classes of > possible exploits. > > On Wed, Feb 3, 2016 at 8:47 AM, Fred Hebert wrote: > >> On 02/03, Felix Gallo wrote: >> >>> There's also an interesting security issue around Unicode source code. >>> >>> Take for example the recent hack of Cryptsy, which involved a guy taking >>> what looked like an innocent and safe pull request to fix an issue in one >>> part of his software, but through the magic of the preprocessor, turned >>> out >>> to do something else entirely: >>> >>> >>> http://earlz.net/view/2016/01/16/0717/analyzing-the-56-million-exploit-and-cryptsys-security >>> >> >> My counter-argument to that is that you don't need any of that cool UTF >> stuff to do that. >> >> See: >> >> - http://www.underhanded-c.org/ underhanded C contest is all about >> writing regular looking C code doing nasty stuff >> - >> http://arstechnica.co.uk/security/2015/12/researchers-confirm-backdoor-password-in-juniper-firewall-code/ >> juniper code was broken by someone adding in a password check that looked >> like a log line >> - >> http://arstechnica.com/security/2016/02/crypto-flaw-was-so-glaring-it-may-be-intentional-eavesdropping-backdoor/ >> using a non-prime in crypto communication, possibly being a backdoor. >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Wed Feb 3 18:30:54 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Wed, 3 Feb 2016 18:30:54 +0100 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: <20160203132030.GB41498@fhebert-ltm2.internal.salesforce.com> <3724ADD3-90C9-4050-8A01-E033C9AC717E@gmail.com> <20160203141100.GC41498@fhebert-ltm2.internal.salesforce.com> <20160203164758.GD41498@fhebert-ltm2.internal.salesforce.com> Message-ID: <12AF4B75-AAB2-45A2-9C0A-DE4C433D466D@gmail.com> > On 03 Feb 2016, at 18:25, Jos? Valim wrote: > > I just wanted to clarify, to avoid any confusion, this patch is mostly about support for unicode atoms forms in the compiler. This is a required step if you ever want to support Unicode in the language but it does not imply such. There are three main milestones: > > 1. Support unicode atom forms (i.e. the form {atom, Line, Atom} can have a UTF-8 encoded atom) > 2. Support unicode atoms between single quotes (for example, '???', as you can already write "???") > 3. Support unicode in the language (for example, being able to write variables in Japanese) > > This discussion was originally related to 1 but I could contribute 2 if desired. Many of the concerns raised above are related to step 3 which, afaik, is not planned. > > > I wasn't clear enough, but supporting UTF8 in atom is OK, but do we actually support collation on binaries and string ? Either way, how "?" compare to "?" or "e" ? What is the rule chosen? - benoit -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Wed Feb 3 18:37:59 2016 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Wed, 3 Feb 2016 18:37:59 +0100 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: <12AF4B75-AAB2-45A2-9C0A-DE4C433D466D@gmail.com> References: <20160203132030.GB41498@fhebert-ltm2.internal.salesforce.com> <3724ADD3-90C9-4050-8A01-E033C9AC717E@gmail.com> <20160203141100.GC41498@fhebert-ltm2.internal.salesforce.com> <20160203164758.GD41498@fhebert-ltm2.internal.salesforce.com> <12AF4B75-AAB2-45A2-9C0A-DE4C433D466D@gmail.com> Message-ID: The comparison on binaries are done at the byte level, so there is no collation. Lists (strings) compare integer codepoints, so no collation as well. Atoms are also byte-oriented, I believe, but we will have mixed encoding because "jos?" will be encoded as latin1 if possible, so no collation either. I believe the default comparison rules won't change. Collation support needs to be provided as a library. *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Wed Feb 3 18:41:18 2016 From: kostis@REDACTED (Kostis Sagonas) Date: Wed, 3 Feb 2016 18:41:18 +0100 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: References: <20160201231049.08b28dc7@gmail.com> Message-ID: <56B23BBE.8050809@cs.ntua.gr> On 02/03/2016 05:26 PM, Sergej Jure?ko wrote: > Yeah same experience and solution that we used. I presume there is a lot allocation/fragmentation going on with udp, since packets tend to be small and there are so many of them. > > Sergej > >> >On 03 Feb 2016, at 17:23, Max Lapshin wrote: >> > >> >When we use plain gen_udp to accept 200-400 mbit of incoming MPEG-TS traffic in flussonic, we use about 50% of moderate 4 core xeon e3 server. >> > >> >When we switch to our driver implementation of udp that collapses several contiguous udp messages into single big message (it is allowed for mpegts) we reduce usage to 15-20% >> > >> >I can't tell that it is "badly written udp in erlang", just messaging is rather expensive. To an outsider, like me, none of the above answers do not reply to the original question which read: On what basis do you make that claim? Also, writing a NIF that actually provides better performance without blocking the scheduler is non-trivial, even if UDP is simple to work with in C. Some questions: 1. Is your driver implementation one that does not block a scheduler? 2. Why is % of CPU usage (rather than whether or not you can achieve the throughput which the application requires) a useful/interesting metric here? Like Sean, I am not disputing that one can write a more performant UDP server in C rather than in Erlang, but I am just curious why writing in Erlang is not sufficient for many/most applications or advantageous for other reasons. Kostis From vasdeveloper@REDACTED Wed Feb 3 18:56:08 2016 From: vasdeveloper@REDACTED (Theepan) Date: Wed, 3 Feb 2016 23:26:08 +0530 Subject: [erlang-questions] Request feedback on example project In-Reply-To: References: <1771611.uqjCtSRlIV@changa> <21732146.Rou2x0oCsi@changa> Message-ID: version({uuid, <<_:48, 0:4, _:12, 2:2, _:62>>}) -> version({uuid, <<_:48, V:4, _:12, 2:2, _:62>>}) when V < 6 -> On Wed, Feb 3, 2016 at 1:03 PM, Bj?rn Gustavsson wrote: > On Wed, Feb 3, 2016 at 8:10 AM, zxq9 wrote: > > > > There should never be a chance where 0 > V because V is > > extracted from a match on bits: > > > > version({uuid, <<_:48, V:4, _:12, 2:2, _:62>>}) > > when V > 0 andalso V < 6 -> > > V > 0 means that V can't be 0... > > > Therefore the following should be just as safe: > > > > version({uuid, <<_:48, V:4, _:12, 2:2, _:62>>}) > > when V < 6 -> > > ... therefore the modification is not safe. > > /Bj?rn > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From seancribbs@REDACTED Wed Feb 3 19:24:40 2016 From: seancribbs@REDACTED (Sean Cribbs) Date: Wed, 3 Feb 2016 12:24:40 -0600 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: References: <20160201231049.08b28dc7@gmail.com> Message-ID: This difference (batching improves performance) is interesting to me because it reminds me of a problem we had where many small writes to gen_tcp slogged performance, whereas batching them to something closer to MTU or TCP buffer size greatly improved throughput. Calling into the driver added ~30ms when sending tiny messages, even with Nagle off. I'm curious, did you try gen_udp with the {active, N} option, or were they already running in {active, true} mode? On Wed, Feb 3, 2016 at 10:23 AM, Max Lapshin wrote: > When we use plain gen_udp to accept 200-400 mbit of incoming MPEG-TS > traffic in flussonic, we use about 50% of moderate 4 core xeon e3 server. > > When we switch to our driver implementation of udp that collapses several > contiguous udp messages into single big message (it is allowed for mpegts) > we reduce usage to 15-20% > > I can't tell that it is "badly written udp in erlang", just messaging is > rather expensive. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Feb 3 20:23:36 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 3 Feb 2016 22:23:36 +0300 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: References: <20160201231049.08b28dc7@gmail.com> Message-ID: Kostis, problem is that when load of CPU is going higher than 50% you may start loosing packets. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hellkvist@REDACTED Wed Feb 3 21:03:48 2016 From: hellkvist@REDACTED (Stefan Hellkvist) Date: Wed, 3 Feb 2016 21:03:48 +0100 Subject: [erlang-questions] Request feedback on example project In-Reply-To: <1771611.uqjCtSRlIV@changa> References: <1771611.uqjCtSRlIV@changa> Message-ID: <3AB7F447-3686-4CA3-BB08-775BE125B518@gmail.com> > > https://github.com/zxq9/zuuid > > No ninja tricks should be in there. If there are any please bring them to > my attention so they can be removed. Hi, Your example in the top Readme... 13> zuuid:string(zuuid:v1()). "{86BAD2C8-C9AF-11E5-9A38-1234567890AB}" ...at least, to me, seems to indicate that the returned UUID:s are wrapped with {}. I would expect such a call to return the bare, unwrapped string representation of the UUID. Same for the example with zuuid:binary. Or have I misunderstood your intentions with the library or the definition of a UUID? /Stefan > > Thanks > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed Feb 3 22:48:43 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 4 Feb 2016 10:48:43 +1300 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: References: Message-ID: <56B275BB.2000009@cs.otago.ac.nz> On 4/02/16 1:52 am, Max Lapshin wrote: > Ok, ok, maybe I'm wrong and it is ok to have isolated pieces of code > that is impossible to edit by foreign programmer. It is *already* possible, even *easy*, to have huge amounts of code that are impossible for a foreign programmer to understand. ISO Latin 1 is said to be adequate for writing Swahili. I don't know one word of Swahili. Write an Erlang module in Swahili -- except for keywords and library references, of course -- and I guarantee that I won't be able to do anything with it. > Problem in UTF8 is not only in nice hieroglyphs, but also in 20 > different dash-like symbols and about 8 different spaces. > > This is one single atom: my package > It is not 2 words, it is a single word that has non-breakable space > inside it. Good luck for debugging =) Bad choice. No-Break SPace (NBSP) is in Latin-1, which Erlang already accepts. You are confounding two different issues: (a) should Erlang allow programmers to write quoted and/or unquoted atoms in their own language, even if that requires characters outside Latin-1? People are arguing that the answer is "yes". UAX#31 deals with this. By the way, if you check DerivedCoreProperties.txt, you'll notice that no white space characters are included in any of the four sets used to define identifiers: {,X}ID_{Start,Continue}. (b) should Erlang accept absolutely every Unicode character? Nobody is arguing for that. There is no particular problem in allowing all the "Zs" characters as separating white space. Terminating % comments at LINE SEPARATOR and PARAGRAPH SEPARATOR doesn't seem like a huge problem either. But there's a lot we're not obliged to handle. Unicode 6.1 section 15.6 describes four invisible mathematical characters: - an invisible separator for ij subscripts and superscripts, - an invisible multiplication operator - an invisible addition operator - an invisible function application operator, to distinguish fx from fx. Does Erlang need to support those? Nope. Does Erlang have to support "20 different dash-like symbols"? Nope. More precisely, there is no obligation to process *as Erlang syntax* any Unicode character just because it is allowed *as data*. It has to be admitted that Unicode is appallingly complex, and that some things have crept into it (like language tags) which are now strongly deprecated. Let me change a word: s/appallingly/terrifyingly/. *Full* support of *everything* in this dauntingly large and still growing standard may never come in my lifetime. That doesn't mean we shouldn't do *anything*. > > Of course you may say me: hire programmer that makes such things. Ok, > no problems. But what to do with copy-paste from skype/slack, where > such symbols are translated into nice utf8 automatically? Invert the translation automatically. If your editor can't do that (and to be honest, mine can't), program a command "unskype-region". And if you can't do that, write an outboard program unskype-file to do it. If the translation done by skype/slack is not invertible, then you have a major problem due to using buggy software that destroys information without warrant. The one thing that's likely to be a problem for Erlang is "-" (number subtraction) vs "--" (list difference) where if you're handed some other "dash-like symbol", it might not be clear what to do. If anyone's interested I could suggest some ideas. Let's revert to "my package", which as noted would NOT be acceptable as an unquoted identifier under Unicode rules. There *is* a problem here. 'my package' (ASCII space) 'my package' (pasted from a file where it's Latin-1 NBSP) are visually indistinguishable. But there is a well-trodden path to safety in such cases: a programming language may ban white space characters other than the plain space from string literals, and some programming languages do. If you want something else, you have to use some sort of escape mechanism. I note that we ALREADY have this problem in ASCII, where it may be impossible to tell a space from a tab. > It is very good that we all have about 80-90 symbols to write code > that other people understand, but I really don't understand what is > the profit of adding ability to make code non-understandable by people > from other cultures. That ability ALREADY EXISTS. (Swahili, for example.) The ability to make code non-understandable by other people already existed in plain ASCII. Here's a function from one of the standard library modules in OTP 18.2, where I've replaced identifiers local to the module or function by random 2- or 3-letter words from an English Scrabble dictionary. ken(SYN, _NAG) -> EFF = bra(), case catch woo(SYN) of {'EXIT',REB} -> nus(EFF), exit(REB); {error,REF,AX} -> {error, [{nus(EFF), [{REF, ?MODULE, AX}]}], []}; EME -> case arf() of [] -> nus(EFF), EME; HO -> YAM = nus(EFF), {warning, EME, [{YAM, [{ALA, ?MODULE, AX}]} || {ALA,AX} <- HO]} end end. Why isn't this already a problem? Because Erlang programmers don't *want* to write bizarre code. I think at some point you just have to trust people[%] to be sensible. [%] Standards committees excepted (:-). > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zxq9@REDACTED Wed Feb 3 23:43:14 2016 From: zxq9@REDACTED (zxq9) Date: Thu, 04 Feb 2016 07:43:14 +0900 Subject: [erlang-questions] Request feedback on example project In-Reply-To: <3AB7F447-3686-4CA3-BB08-775BE125B518@gmail.com> References: <1771611.uqjCtSRlIV@changa> <3AB7F447-3686-4CA3-BB08-775BE125B518@gmail.com> Message-ID: <11476505.jjG1v6unmC@changa> On 2016?2?3? ??? 21:03:48 you wrote: > > > > > https://github.com/zxq9/zuuid > > > > No ninja tricks should be in there. If there are any please bring them to > > my attention so they can be removed. > > Hi, > > Your example in the top Readme... > 13> zuuid:string(zuuid:v1()). "{86BAD2C8-C9AF-11E5-9A38-1234567890AB}" > ...at least, to me, seems to indicate that the returned UUID:s are wrapped with {}. I would expect such a call to return the bare, unwrapped string representation of the UUID. Same for the example with zuuid:binary. Or have I misunderstood your intentions with the library or the definition of a UUID? I'm so used to dealing with UUIDs formatted by databases that I didn't really notice. Indeed, there is a mix of representations output by various UUID tools in different environments (most of the Erlang UUID libs output standard unbracketed strings). I've added string/2 and binary/2 to make the choice explicit. http://zxq9.com/projects/zuuid/docs/zuuid.html#string-2 Thanks for catching that. -Craig From ok@REDACTED Thu Feb 4 00:38:53 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 4 Feb 2016 12:38:53 +1300 Subject: [erlang-questions] Failed tests in OTP 18.2 Message-ID: <56B28F8D.5020202@cs.otago.ac.nz> I've just been building OTP 18.2. .../otp_src_18.2/release/tests/test_server/index.html reports 3 failures in tests.emulator_test_time_SUITE (univ_to_local, local_to_univ, consistency) and 1 failure in tests.os_mon_test_cpu_sup_SUITE.cases (util_api). What's the procedure for reporting these? During the build there are a couple of warnings about rm: conftest.dSYM: is a directory (I'm building under OS X 10.9.5 with Xcode 5.1). There are also lots of warnings about missing prototypes. The Java compiler also moans about lots of deprecated methods. How worried should I be about any of this? From kostis@REDACTED Thu Feb 4 01:20:06 2016 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 4 Feb 2016 01:20:06 +0100 Subject: [erlang-questions] Request feedback on example project In-Reply-To: <1771611.uqjCtSRlIV@changa> References: <1771611.uqjCtSRlIV@changa> Message-ID: <56B29936.2080700@cs.ntua.gr> On 02/03/2016 07:00 AM, zxq9 wrote: > Two days ago I wrote a simple UUID utility as an example of bit syntax. > Yesterday I decided I wanted it to be an example of documentation and > readability as well. > > I would like feedback from new and old Erlangers in terms of readability, > documentation, what is easy to understand, what is hard to understand, etc. Some comments concerning the specs that you have written. In zuuid_man.erl, I see: -spec init(term()) -> {ok, term()}. init(_) -> {ok, #s{}}. and my immediate reaction is: well, this function does not return a ok-pair with any term in the second position; instead, it returns an #s{} record in that element. The question that comes to my mind next, is why did the author of the code write that? Does (s)he simply want to hide the return structure? This is not the proper way to do this. Further down I see: -spec v1(State) -> {UUID, NewState} when State :: #s{}, UUID :: zuuid:uuid(), NewState :: #s{}. why not introduce a state() type for #s{} and use it throughout this module? Also, if you want other modules to not be able to inspect/rely on the structure of the state record, declare it as opaque. Small type lies also exist in zuuid.erl. The function: -spec read_uuid_string(list()) -> uuid() | bad_uuid. read_uuid_string(UUID) -> Parts = string:tokens(UUID, "{-}"), case lists:map(fun length/1, Parts) of ... clearly is not meant to accept any list; it's expecting a string(). While at it, I suggest to use a list comprehension instead of lists:map. Similar changes need to happen in read_map_string/1 further below. Finally, it's debatable whether it's a good idea for functions to return error values (e.g. 'bad_mac') instead of just crashing on inputs they are not supposed to handle, but I guess this is a matter of design. Hope these help, Kostis From khitai.pang@REDACTED Thu Feb 4 02:46:26 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Thu, 4 Feb 2016 09:46:26 +0800 Subject: [erlang-questions] string concatenation efficiency In-Reply-To: References: <56AABF3E.7010800@cs.otago.ac.nz> Message-ID: Hi Joe, Thank you for your advice! You are right, I should worry more about how the message is transported and how the receiver parses the message. And maybe I shouldn't worry about trivial performance difference until I have a whole functioning system. Thanks Khitai On 2016/2/2 0:01, Joe Armstrong wrote: > On Sat, Jan 30, 2016 at 5:04 PM, Khitai Pang wrote: >> Thank you all for your answers. >> >> The concatenated string is a redis key, it will be part of an erlang message >> that will be sent to a redis client erlang process. I tried sending the >> iolist ["Item:{",ItemId,"}"], and it works fine. I think this is the most >> efficient way. > It *is* the most efficient way - but remember what you sent, some other program > has to receive - the amount of work done in transporting the message > and delivering it to the receiver will far outweigh the cost of building the > message prior to sending it. > > The rule of thumb is that parsing is *always* slower than serialising > messages - why? in parsing you must take a decision at every byte. In > serializing a message > you take a decision at every node in a parse-tree. > > So you should not be worrying about what you send - but you should worry about > how it will be received. > > Building the message from variables into an I/O list or with "++" > has a trivial cost compared to all the machine cycles taken up > transporting the message. > > Even measuring the cost of different methods won't tell you much. > You'll find out that A (IO-lists) is faster or slower than B (++) but all the > real time is taken up in X and Y (which you didn't know about) > > People happily send KBytes encrypted data in XML, when single > unencrypted bytes might have sufficed. > > The only thing you should worry about is end-to-end efficiency. > > So: > > 1) Make everything > 2) Measure > 3) Fast enough? > Yes > Happy > No > Can I wait ten years? > No > measure to find bit that takes the most time > fix worse bit goto 2) > Yes > wait ten years - go to 2) > > Patience is a virtue - the programs I wrote 20 years ago now run > thousands of times faster - not because I'm a better programmer but > because we have > GHz clocks now (they were MHz 20 years ago) > > You want to write clear short, elegant code that will still run in 20 years time > > Given the choice between clear code that is slow but fast-enough and > fast code that is unclear I always choose the clear clode. > > The key point is fast-enough. > > Cheers > > /Joe > >> >> Thanks >> Khitai >> >> >> On 2016/1/29 9:24, Richard A. O'Keefe wrote: >>> >>> On 28/01/16 8:22 pm, Khitai Pang wrote: >>>> For string concatenation, which one of the following is the most >>>> efficient? >>>> >>>> 1) >>>> strings:join(["Item:{", ItemID, "}")], ""] >>>> >>>> 2) >>>> lists:append(["Item:{", ItemID, "}")]) >>>> >>>> 3) >>>> "Item:{" ++ ItemID ++ "}" >>>> >>>> Here ItemID is a UUID. >>> For something this size, it hardly matters. >>> >>> The definition of lists:append/1 is >>> append([E]) -> E; >>> append([H|T]) -> H ++ append(T); >>> append([]) -> []. >>> so lists:append([A,B,C]) -> A ++ lists:append([B,C]) >>> -> A ++ B ++ lists:append([C]) >>> -> A ++ B ++ C. >>> Clearly, there is no way for this to be more efficient than >>> writing A ++ B ++ C directly. >>> >>> The definition of string:join/2 is >>> join([], Sep) when is_list(Sep) -> []; >>> join([H|T], Sep) -> H ++ lists:append([Sep ++ X >>> || X <- T]). >>> -- at least in 18.1 -- which rather startled me because I was expecting >>> join([E], Sep) when is_list(Sep) -> E; >>> join([H|T], Sep) -> H ++ Sep ++ join(T, Sep); >>> join([], Sep) when is_list(Sep) -> []. >>> Clearly, there is no way that either version of join/2 can be faster than >>> append/1. In fact >>> append/1 is measurably faster than >>> the revised join/2, which is measurably faster than >>> the original join/2, >>> but you have to push the sizes ridiculously high to make these >>> measurements. >>> >>> Really, I suggest you write whichever makes your intentions clearest >>> to human readers, get the program going, then measure it. I would >>> be surprised if this particular issue made any significant difference. >>> >>> But why are you using strings at all? >>> Strings are an *interface* data type; you process them when you receive >>> data from an external source and you generate them (or rather you >>> generate iolists) when you are sending data to an external source, but >>> for internal processing you usually want some sort of tree. >>> >>> Seeing "Item:{$ItemID}" suggests that maybe you *are* generating output >>> for some external process, but in that case, perhaps you should just be >>> sending along the iolist ["Item:{",ItemId,"}"] and *not* concatenating the >>> pieces. >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From zxq9@REDACTED Thu Feb 4 02:57:52 2016 From: zxq9@REDACTED (zxq9) Date: Thu, 04 Feb 2016 10:57:52 +0900 Subject: [erlang-questions] Request feedback on example project In-Reply-To: <56B29936.2080700@cs.ntua.gr> References: <1771611.uqjCtSRlIV@changa> <56B29936.2080700@cs.ntua.gr> Message-ID: <4498980.CUMZ6B54mj@changa> Hi Kostis! Thank you for catching those! On 2016?2?4? ??? 01:20:06 Kostis Sagonas wrote: > Some comments concerning the specs that you have written. > > In zuuid_man.erl, I see: > > -spec init(term()) -> {ok, term()}. > init(_) -> > {ok, #s{}}. > > and my immediate reaction is: well, this function does not return a > ok-pair with any term in the second position; instead, it returns an > #s{} record in that element. The question that comes to my mind next, > is why did the author of the code write that? Does (s)he simply want to > hide the return structure? This is not the proper way to do this. > > Further down I see: > > -spec v1(State) -> {UUID, NewState} > when State :: #s{}, > UUID :: zuuid:uuid(), > NewState :: #s{}. > > why not introduce a state() type for #s{} and use it throughout this > module? Also, if you want other modules to not be able to inspect/rely > on the structure of the state record, declare it as opaque. Indeed! I've made this change. As for the spec for init/1, I had left a template spec line in there and never went back to tighten it -- oops. I suppose that is actually *worse* than giving Dialyzer no type at all. As for opaqueness... It wasn't about exporting the type or not (I can't recall ever exporting a gen_server's state type, come to think of it...?) but simply about having been lazy and not reviewing my typespecs again. I had simply run Dialyzer over the project, everything checked out, and... Here's to the power of the critical eye over blindly trusting automation. :-) > Small type lies also exist in zuuid.erl. The function: > > -spec read_uuid_string(list()) -> uuid() | bad_uuid. > read_uuid_string(UUID) -> > Parts = string:tokens(UUID, "{-}"), > case lists:map(fun length/1, Parts) of > ... > > clearly is not meant to accept any list; it's expecting a string(). > While at it, I suggest to use a list comprehension instead of lists:map. > > Similar changes need to happen in read_map_string/1 further below. Indeed. Fixed. Thank you. As for using list comprehensions VS lists:map/2 and lists:filtermap/2... I have noticed that I'm nearly as likely to write lists:map/2 as I am to write a list comprehension -- except in the case of unpacking tuples, where I nearly always write list comprehensions. I don't have a real reason for this and like both ways just fine. I am wondering, though, if there is any general consensus on this? comprehensions > map/filtermap ? I switched both cases to list comprehensions just because, but don't really know if I care one way or the other. > Finally, it's debatable whether it's a good idea for functions to return > error values (e.g. 'bad_mac') instead of just crashing on inputs they > are not supposed to handle, but I guess this is a matter of design. We have a convention (a guideline, not a hard rule, though) for this: * Any pure function that touches "internal" (as in, already scrubbed/checked values) data only always crashes on bad data, and is written to return a naked value instead of a tuple (to enable clean function composition). (I strive to make as much of a codebase fall into this category as possible.) * Any pure function that accepts external data and is named 'read_*' will always return an {ok, Value} tuple on success where Value's type is now guaranteed, and an atom like 'bad_*' or {error, Reason} on bad data, but not crash. (This is the case read_uuid/1 and read_mac/1 and their helper functions fall in to.) * Any pure function named Module:to_[type] or Module:from_[type] will return a naked value or crash on bad data. Forests of these tend to surround ADT modules. * Directly side-effecty functions always return an {ok, Value} tuple or 'ok' on success, {error, Reason} if the external resource is manageable but merely unavailable (usually temporary), and crash in any other situation. Any side-effecty function is expected to crash at any time, though, just on principle. I don't think I've ever even written that out in English before... I should certainly put that somewhere as a reference. On that note, I am very curious to know if anyone has a different or better idea for function return/outcome guidelines -- usually this aspect of function outcomes seems to just be left up in the air. > Hope these help, Helped quite a bit. Thanks again for your time and attention. -Craig From ok@REDACTED Thu Feb 4 04:35:26 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 4 Feb 2016 16:35:26 +1300 Subject: [erlang-questions] Request feedback on example project In-Reply-To: <1771611.uqjCtSRlIV@changa> References: <1771611.uqjCtSRlIV@changa> Message-ID: <56B2C6FE.2060901@cs.otago.ac.nz> UUID support in various flavours of Unix is ridiculously spotty, so I've been looking for a readable implementation for a while. In particular, the support for all of versions 1-5 looks useful; the C libraries available to me do not do that. I will say that stuffing curly braces around the UUID by default is both surprising and unhelpful; it's probably a bad idea for there to *be* a default. The version of Erlang I have installed doesn't support maps. I was trying to install 18.2 today but ran into problems (see earlier message from me today). So I cannot compile zuuid. (The RestartStrategy is given as a map in zuuid_sup:init/1.) Speaking of documentation, while init/1 may be a private function, that doesn't mean that someone who isn't highly skilled in OTP wouldn't benefit from an explanation of *why* the flags and child spec are what they are. Suppose that getifaddrs() returns a list in which no element contains an hwaddr. - If the list is entirely empty, {error,no_iface} results. - If the list is not empty, {error,no_address} results. Why are these different? I note that the example for get_hw_addr/1 suggests eth2, br1, birbr0, wlan0 as things to look for, none of which are reported on my machine. (I have lo0, gif0, stgf0, en0, en1, en2, en3, p2p0, bridge0.) Examples that don't work are always confusing. What's particularly confusing for the new reader is "where does this list of magic alphabet soup come from? If none of this works, is my machine set up wrong, or what?" I note that this time, a non-empty list of interface addresses that happens not to contain the one you are looking for will be reported as no_iface, not no_address. "The hardware addressed desired." Should that be "address"? It's probably there somewhere, but I haven't found it, so can I ask that every module that can report {error, Reason} should either document the Reasons explicitly itself, or point to a documented catalogue from which it draws? I wanted to report something profound and insightful, but these are just a few comparatively unimportant things that bugged me when I was reading the code. Oh, a layout issue. I don't like big black blobs (except when I write them). So %% wdrwertwertwert %% wertwertwertwrt %% wretertwertwertew -spec frgsdfgdfghsdfg asdfasdf(sdfgsdfg) -> .... is just a bit too much for me to be comfortable with. I'd find it easier to read with blank line after the %% comment block and a blank line after the -spec. I find that with my OWN code it's so obvious that packing it tightly doesn't impair readability, but with someone else's code well-placed blank lines never go amiss. I can only suppose that I'm in error about my own code and that it too could do with more blank lines. From mark@REDACTED Thu Feb 4 05:38:36 2016 From: mark@REDACTED (Mark Steele) Date: Wed, 3 Feb 2016 23:38:36 -0500 Subject: [erlang-questions] ANN: Cinched 0.0.1, an encryption microservice Message-ID: Hi list, I've just released version 0.0.1 of an application I've been working on for some time now called Cinched. It's a microservice for providing encryption/decryption (partial JSON document or blob). Some of the goals I had in mind while putting this together is to simplify key management for developers as well as centralize the encryption code to make it easier to audit. Under the hood: - riak_ensemble for key storage - cowboy for the TLS/HTTP handling - shamir secret sharing - libsodium (via NIF bindings) - OCSP checks (via NIF bindings) - poolboy for limiting CPU bound workers - exometer_core for metrics - jiffy for json parsing - ej for json document traversal - Lots of SELinux policy, Linux DAC controls I'm pretty new at Erlang so I'd appreciate any feedback as well as harsh criticism, it's all good. Code is available on Github here: https://github.com/marksteele/cinched As an FYI, it's targeted to RHEL/CentOS 7. Much of the security of the system is dependent on a well behaving SELinux system with the right reference policy. PS: Has anyone managed to layer in an attestation protocol on top of Erlang distribution in order to be able to establish trust between cluster peers? I've found some academic research, but no code. Cheers, Mark Steele CISSP, GPEN, GCIA, CSM mark@REDACTED LinkedIn: https://ca.linkedin.com/in/markrsteele Github: https://github.com/marksteele Personal: http://www.control-alt-del.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Thu Feb 4 11:55:22 2016 From: zxq9@REDACTED (zxq9) Date: Thu, 04 Feb 2016 19:55:22 +0900 Subject: [erlang-questions] Request feedback on example project In-Reply-To: <56B2C6FE.2060901@cs.otago.ac.nz> References: <1771611.uqjCtSRlIV@changa> <56B2C6FE.2060901@cs.otago.ac.nz> Message-ID: <1957725.BAnHVD50dh@changa> On 2016?2?4? ??? 16:35:26 Richard A. O'Keefe wrote: > UUID support in various flavours of Unix is ridiculously spotty, > so I've been looking for a readable implementation for a while. > > In particular, the support for all of versions 1-5 looks useful; > the C libraries available to me do not do that. The use case for version 2 UUIDs seems extremely narrow and better handled in other ways, imo, but a discussion about how odd it is and the general lack of any implementations is actually what led me to write this in the first place. It happily turned out that a UUID library is an ideal size for me to solicit some feedback on style and documentation -- so I took advantage of it. I'm really glad I did. > I will say that stuffing curly braces around the UUID by default > is both surprising and unhelpful; it's probably a bad idea for > there to *be* a default. This has been mentioned *twice* now. So... I'm deprecating zuuid:string/0 and zuuid:binary/0 and adding two forms that have been requested. As someone is already using this in a project (that was fast... actually overnight) I'll just have to also demonstrate what version numbers are supposed to mean. v2.0 will lack string/0 and binary/0. A later version may reintroduce them as an equivalent for the unbracketed canonical representation. The price of having users... paid in version numbers. > The version of Erlang I have installed doesn't support maps. > I was trying to install 18.2 today but ran into problems (see > earlier message from me today). So I cannot compile zuuid. > (The RestartStrategy is given as a map in zuuid_sup:init/1.) I don't even like the idea of using maps for this, but the "old fashioned" tuple type seems to be discouraged in the R18 docs. Meh. So much for trying to be hip to the new thing. I like my old tuples. I don't think there is anything in here that doesn't work even with rather old ERTS releases, so I'm switching this (back) to tuples. With regard to tuples VS maps for trivial things like this... I almost feel like its a case of getting a newfangled toy and simply feeling compelled to use it everywhere. > Speaking of documentation, while init/1 may be a private > function, that doesn't mean that someone who isn't highly > skilled in OTP wouldn't benefit from an explanation of *why* > the flags and child spec are what they are. That's a really good point. I remember that being something I had to take on faith early on myself and really learn later; annoyingly similar to some other languages -- though in this case its more about the system than the language. I've written an explanation in the comments. > Suppose that getifaddrs() returns a list in which no element > contains an hwaddr. > - If the list is entirely empty, {error,no_iface} results. > - If the list is not empty, {error,no_address} results. > Why are these different? Indeed, I failed to explain the difference. That is now remedied. > I note that the example for get_hw_addr/1 suggests > eth2, br1, birbr0, wlan0 > as things to look for, none of which are reported on my machine. > (I have lo0, gif0, stgf0, en0, en1, en2, en3, p2p0, bridge0.) > Examples that don't work are always confusing. > What's particularly confusing for the new reader is "where does > this list of magic alphabet soup come from? If none of this > works, is my machine set up wrong, or what?" > I note that this time, a non-empty list of interface addresses > that happens not to contain the one you are looking for will be > reported as no_iface, not no_address. > > "The hardware addressed desired." > Should that be "address"? Indeed. I just assumed familiarity with these. I've added a note to the docs with some references (sort of sloppy, though; not really satisfied, but at least its touched upon). > It's probably there somewhere, but I haven't found it, so can I ask > that every module that can report {error, Reason} should either > document the Reasons explicitly itself, or point to a documented > catalogue from which it draws? Missed this entirely. Added now, on both counts. > I wanted to report something profound and insightful, but these are > just a few comparatively unimportant things that bugged me when I > was reading the code. > > Oh, a layout issue. I don't like big black blobs (except when I > write them). So > %% wdrwertwertwert > %% wertwertwertwrt > %% wretertwertwertew > -spec frgsdfgdfghsdfg > asdfasdf(sdfgsdfg) -> > .... > > is just a bit too much for me to be comfortable with. I'd find > it easier to read with blank line after the %% comment block > and a blank line after the -spec. > > I find that with my OWN code it's so obvious that packing it > tightly doesn't impair readability, but with someone else's > code well-placed blank lines never go amiss. I can only suppose > that I'm in error about my own code and that it too could do with > more blank lines. *MY* code is always perfectly readable as big, immaculate, unbroken blocks. Until a few months later. Ugh. I'm not sure where I stand on the line break issue, really, but since you mentioned it I'm trying out a line between doc and spec, a line between spec and definition, and two lines between functions (but no lines between function clauses -- I already know that confuses me, especially with un-specced/un-docced functions). Thanks a lot! If anything else comes to mind, please let me know! -Craig From dmitriid@REDACTED Thu Feb 4 12:26:19 2016 From: dmitriid@REDACTED (Dmitrii Dimandt) Date: Thu, 04 Feb 2016 11:26:19 +0000 Subject: [erlang-questions] Atom Unicode Support In-Reply-To: <56B275BB.2000009@cs.otago.ac.nz> References: <56B275BB.2000009@cs.otago.ac.nz> Message-ID: > It has to be admitted that Unicode is appallingly complex, and that some > things > have crept into it (like language tags) which are now strongly deprecated. > Let me change a word: s/appallingly/terrifyingly/. *Full* support of > *everything* > in this dauntingly large and still growing standard may never come in my > lifetime. That doesn't mean we shouldn't do *anything*. > > I'd argue for inclusion of ICU in Erlang (probably adopting or pulling in https://github.com/erlang-unicode/i18n and https://github.com/erlang-unicode/ux). There was a hint of such a possibility at one of previous EUC's Because it looks like ICU is the only library out there (anywhere) that keeps up with Unicode. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn@REDACTED Thu Feb 4 12:31:09 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Thu, 4 Feb 2016 12:31:09 +0100 Subject: [erlang-questions] Failed tests in OTP 18.2 In-Reply-To: <56B28F8D.5020202@cs.otago.ac.nz> References: <56B28F8D.5020202@cs.otago.ac.nz> Message-ID: On Thu, Feb 4, 2016 at 12:38 AM, Richard A. O'Keefe wrote: > I've just been building OTP 18.2. > .../otp_src_18.2/release/tests/test_server/index.html > reports 3 failures in tests.emulator_test_time_SUITE > (univ_to_local, local_to_univ, consistency) > and 1 failure in tests.os_mon_test_cpu_sup_SUITE.cases > (util_api). > At least 'consistency' fails because it only works in the CET time zone: %% Test the following equations: %% date() & time() == erlang:localtime() %% erlang:universaltime() + timezone == erlang:localtime() %% %% Assumptions: %% Middle-European time zone, EU rules for daylight-saving time. %% %% Limitations: %% Localtime and universaltime must be in the same month. %% Daylight-saving calculations are incorrect from the last %% Sunday of March and October to the end of the month. I guess that the other two in time_SUITE fail for the same reason. > What's the procedure for reporting these? Report them at http://bugs.erlang.org. > During the build there are a couple of warnings about > rm: conftest.dSYM: is a directory > (I'm building under OS X 10.9.5 with Xcode 5.1). > There are also lots of warnings about missing prototypes. > The Java compiler also moans about lots of deprecated methods. > How worried should I be about any of this? Not at all worried. /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From jesper.louis.andersen@REDACTED Thu Feb 4 13:08:08 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 4 Feb 2016 13:08:08 +0100 Subject: [erlang-questions] ANN: Cinched 0.0.1, an encryption microservice In-Reply-To: References: Message-ID: On Thu, Feb 4, 2016 at 5:38 AM, Mark Steele wrote: > - libsodium (via NIF bindings) I wonder if your approach is much different from https://github.com/jlouis/enacl in any way? We may win by coalescing the work there. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nathaniel@REDACTED Thu Feb 4 13:29:06 2016 From: nathaniel@REDACTED (Nathaniel Waisbrot) Date: Thu, 4 Feb 2016 07:29:06 -0500 Subject: [erlang-questions] Request feedback on example project In-Reply-To: <1957725.BAnHVD50dh@changa> References: <1771611.uqjCtSRlIV@changa> <56B2C6FE.2060901@cs.otago.ac.nz> <1957725.BAnHVD50dh@changa> Message-ID: > > I don't even like the idea of using maps for this, but the "old > fashioned" tuple type seems to be discouraged in the R18 docs. Meh. > > So much for trying to be hip to the new thing. I like my old tuples. > I don't think there is anything in here that doesn't work even with > rather old ERTS releases, so I'm switching this (back) to tuples. > With regard to tuples VS maps for trivial things like this... I almost > feel like its a case of getting a newfangled toy and simply feeling > compelled to use it everywhere. > As someone with less than 10,000 hours of OTP code under my belt, I found this change (tuples to maps) to be so useful that I immediately upgraded all my code to require version 18. With the tuples, I had to check the documentation every single time I made a supervisor to confirm whether it was `intensity` or `period` first. I also would get copy/paste errors because I'd copy the child spec for a worker that was set to `brutal_kill` and use that as template for a supervisor without changing it to `infinity`. The new maps are self-documenting and they provide sane defaults so that if I want a vanilla supervisor I can just say `#{id => cool_super, start => {M, F, A}, type => supervisor}`. Not only do I avoid dumb bugs, but it's also clear that this is a vanilla supervisor and not something more interesting. It seems like a small change, but I thought the old tuples were _awful_ and I was very happy to see them marked as deprecated. > Oh, a layout issue. I don't like big black blobs (except when I > > write them). So > > %% wdrwertwertwert > > %% wertwertwertwrt > > %% wretertwertwertew > > -spec frgsdfgdfghsdfg > > asdfasdf(sdfgsdfg) -> > > .... > > > > is just a bit too much for me to be comfortable with. I'd find > > it easier to read with blank line after the %% comment block > > and a blank line after the -spec. > > > > I find that with my OWN code it's so obvious that packing it > > tightly doesn't impair readability, but with someone else's > > code well-placed blank lines never go amiss. I can only suppose > > that I'm in error about my own code and that it too could do with > > more blank lines. > > *MY* code is always perfectly readable as big, immaculate, unbroken > blocks. Until a few months later. Ugh. > > I'm not sure where I stand on the line break issue, really, but since > you mentioned it I'm trying out a line between doc and spec, a line > between spec and definition, and two lines between functions (but no > lines between function clauses -- I already know that confuses me, > especially with un-specced/un-docced functions). > I don't feel very strongly on this one (I do about the supervisor spec), but I find the tight-packed blocks _more_ readable in other people's code. I run my eyes down the left-hand side and see % - a And that's pretty clearly (1) some comments, (2) a compiler directive, (3) a function. They all go together: I expect the comment to be function documentation and the dash to be a function spec. If I see % - a Then the comments might be module-level comments and the directive might be an export or an if or a define or a type declaration and I have to read more characters to figure out if those three lines are related. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bog495@REDACTED Thu Feb 4 13:29:19 2016 From: bog495@REDACTED (Bogdan Andu) Date: Thu, 4 Feb 2016 14:29:19 +0200 Subject: [erlang-questions] UDP active socket timeout Message-ID: Hi, How can be detected a timeout on an active UDP socket? controlling process receives the message: {udp, Socket, Host, Port, Bin} and after a process is spawned to handle that data: spawn(fun() -> handle_pckt(Socket, Host, Port, Bin) end), Q: how the spawned or controlling process is able to detect that a timeout on this UDP socket occurred ? documentations does not mention anything about timeout on active sockets Thanks, /Bogdan -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Thu Feb 4 13:52:58 2016 From: zxq9@REDACTED (zxq9) Date: Thu, 04 Feb 2016 21:52:58 +0900 Subject: [erlang-questions] Request feedback on example project In-Reply-To: References: <1771611.uqjCtSRlIV@changa> <1957725.BAnHVD50dh@changa> Message-ID: <1618384.kpENv6aRZF@changa> On 2016?2?4? ??? 07:29:06 Nathaniel Waisbrot wrote: > > > > I don't even like the idea of using maps for this, but the "old > > fashioned" tuple type seems to be discouraged in the R18 docs. Meh. > > > > So much for trying to be hip to the new thing. I like my old tuples. > > I don't think there is anything in here that doesn't work even with > > rather old ERTS releases, so I'm switching this (back) to tuples. > > With regard to tuples VS maps for trivial things like this... I almost > > feel like its a case of getting a newfangled toy and simply feeling > > compelled to use it everywhere. > > As someone with less than 10,000 hours of OTP code under my belt, I found > this change (tuples to maps) to be so useful that I immediately upgraded > all my code to require version 18. > With the tuples, I had to check the documentation every single time I made > a supervisor to confirm whether it was `intensity` or `period` first. I > also would get copy/paste errors because I'd copy the child spec for a > worker that was set to `brutal_kill` and use that as template for a > supervisor without changing it to `infinity`. > The new maps are self-documenting and they provide sane defaults so that if > I want a vanilla supervisor I can just say `#{id => cool_super, start => > {M, F, A}, type => supervisor}`. Not only do I avoid dumb bugs, but it's > also clear that this is a vanilla supervisor and not something more > interesting. > > It seems like a small change, but I thought the old tuples were _awful_ and > I was very happy to see them marked as deprecated. Having a method for defaulting to sane values (using maps for convenience) makes perfect sense. But *deprecating* tuples bothers me -- it means that eventually every piece of OTP code is going to break or have to be (arbitrarily) moved forward to maps for no gain. Ideally, if I could have my way... and actually I can, so I'm going to add something like this to my own templating tool... vanilla_worker(Name) -> vanilla_worker(Name, []). vanilla_worker(Name, Args) -> {Name, {Name, start_link, Args}, permanent, brutal_kill, worker, [Name]}. ...And maybe one or two more really common ones. I find it every bit as annoying to define a bunch of nearly identical maps as a bunch of nearly identical tuples. I generally only have to do this from scratch *once* per project, which is really not a big deal. Its annoying either way, but its typically a one-off annoyance. (Until now I've just had a template I have a script pull over and start from there, so I sort of forgot that this could be annoying.) > > Oh, a layout issue. I don't like big black blobs (except when I > > > write them). So > > > %% wdrwertwertwert > > > %% wertwertwertwrt > > > %% wretertwertwertew > > > -spec frgsdfgdfghsdfg > > > asdfasdf(sdfgsdfg) -> > > > .... > > > > > > is just a bit too much for me to be comfortable with. I'd find > > > it easier to read with blank line after the %% comment block > > > and a blank line after the -spec. > > > > > > I find that with my OWN code it's so obvious that packing it > > > tightly doesn't impair readability, but with someone else's > > > code well-placed blank lines never go amiss. I can only suppose > > > that I'm in error about my own code and that it too could do with > > > more blank lines. > > > > *MY* code is always perfectly readable as big, immaculate, unbroken > > blocks. Until a few months later. Ugh. > > > > I'm not sure where I stand on the line break issue, really, but since > > you mentioned it I'm trying out a line between doc and spec, a line > > between spec and definition, and two lines between functions (but no > > lines between function clauses -- I already know that confuses me, > > especially with un-specced/un-docced functions). > > > > > I don't feel very strongly on this one (I do about the supervisor spec), > but I find the tight-packed blocks _more_ readable in other people's code. > I run my eyes down the left-hand side and see > > % > - > a > > And that's pretty clearly (1) some comments, (2) a compiler directive, (3) > a function. They all go together: I expect the comment to be function > documentation and the dash to be a function spec. > > If I see > > % > > - > > a > > Then the comments might be module-level comments and the directive might be > an export or an if or a define or a type declaration and I have to read > more characters to figure out if those three lines are related. I sometimes think so myself. In this case, though, where every function has an explicit `%% @doc` at the head of the function docs and everything else follows a pretty obvious pattern, I do indeed find it easier with the extra lines. With a mishmash of styles in a single source file, though... ugh. I think it would get really annoying. Tomorrow I'll check again and see which I like. Part of my goal here is to get some feedback before I start doing more open-source stuff... the place I've been coding the last year has been rather niche and impossible to make generally applicable someplace like github. On second thought, I think the extra newlines actually improve *reading* but may somewhat impare *scanning*. Consider: https://github.com/zxq9/zuuid/blob/master/src/zuuid.erl One clear advantage is that my super-lazy-man's navigation in vi is far more precise now, as each function and spec had is the start of what the editor sees as its own block. Not a big deal (I usually jump directly to wherever I want to go), but it is sort of nice. -Craig From seancribbs@REDACTED Thu Feb 4 15:43:11 2016 From: seancribbs@REDACTED (Sean Cribbs) Date: Thu, 4 Feb 2016 08:43:11 -0600 Subject: [erlang-questions] UDP active socket timeout In-Reply-To: References: Message-ID: UDP is a connection-less datagram protocol, so there's no timeout inherent in the protocol. However, you may have timeouts on: 1) receiving a datagram (nothing arrived) 2) sending a datagram (things went very badly) However, those are timeouts internal to Erlang, from your process to the socket driver. If you are using the {active, true} option (which it sounds like you are), you should use an 'after' clause in your 'receive' to implement timeouts, or use the trailing timeout in the return tuple of a gen_server/gen_fsm callback. On Thu, Feb 4, 2016 at 6:29 AM, Bogdan Andu wrote: > Hi, > > How can be detected a timeout on an active UDP socket? > > controlling process receives the message: > {udp, Socket, Host, Port, Bin} > > and after a process is spawned to handle that data: > spawn(fun() -> handle_pckt(Socket, Host, Port, Bin) end), > > Q: how the spawned or controlling process is able to detect > that a timeout on this UDP socket occurred ? > > documentations does not mention anything about timeout on active sockets > > Thanks, > > /Bogdan > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Thu Feb 4 16:36:09 2016 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Thu, 4 Feb 2016 10:36:09 -0500 Subject: [erlang-questions] Request feedback on example project In-Reply-To: <1618384.kpENv6aRZF@changa> References: <1771611.uqjCtSRlIV@changa> <1957725.BAnHVD50dh@changa> <1618384.kpENv6aRZF@changa> Message-ID: <978AAB66-4294-4AA0-BFF5-1C72FECBF609@writersglen.com> > On second thought, I think the extra newlines actually improve *reading* > but may somewhat impare *scanning*. > Consider: > https://github.com/zxq9/zuuid/blob/master/src/zuuid.erl Ah, much easier on my aging eyes. Best wishes, LRP Sent from my iPad > On Feb 4, 2016, at 7:52 AM, zxq9 wrote: > > On 2016?2?4? ??? 07:29:06 Nathaniel Waisbrot wrote: >>> >>> I don't even like the idea of using maps for this, but the "old >>> fashioned" tuple type seems to be discouraged in the R18 docs. Meh. >>> >>> So much for trying to be hip to the new thing. I like my old tuples. >>> I don't think there is anything in here that doesn't work even with >>> rather old ERTS releases, so I'm switching this (back) to tuples. >>> With regard to tuples VS maps for trivial things like this... I almost >>> feel like its a case of getting a newfangled toy and simply feeling >>> compelled to use it everywhere. >> >> As someone with less than 10,000 hours of OTP code under my belt, I found >> this change (tuples to maps) to be so useful that I immediately upgraded >> all my code to require version 18. >> With the tuples, I had to check the documentation every single time I made >> a supervisor to confirm whether it was `intensity` or `period` first. I >> also would get copy/paste errors because I'd copy the child spec for a >> worker that was set to `brutal_kill` and use that as template for a >> supervisor without changing it to `infinity`. >> The new maps are self-documenting and they provide sane defaults so that if >> I want a vanilla supervisor I can just say `#{id => cool_super, start => >> {M, F, A}, type => supervisor}`. Not only do I avoid dumb bugs, but it's >> also clear that this is a vanilla supervisor and not something more >> interesting. >> >> It seems like a small change, but I thought the old tuples were _awful_ and >> I was very happy to see them marked as deprecated. > > Having a method for defaulting to sane values (using maps for convenience) > makes perfect sense. But *deprecating* tuples bothers me -- it means that > eventually every piece of OTP code is going to break or have to be > (arbitrarily) moved forward to maps for no gain. > > Ideally, if I could have my way... and actually I can, so I'm going to add > something like this to my own templating tool... > > vanilla_worker(Name) -> > vanilla_worker(Name, []). > > vanilla_worker(Name, Args) -> > {Name, > {Name, start_link, Args}, > permanent, > brutal_kill, > worker, > [Name]}. > > ...And maybe one or two more really common ones. > > I find it every bit as annoying to define a bunch of nearly identical > maps as a bunch of nearly identical tuples. I generally only have to do > this from scratch *once* per project, which is really not a big deal. > Its annoying either way, but its typically a one-off annoyance. > > (Until now I've just had a template I have a script pull over and start > from there, so I sort of forgot that this could be annoying.) > >>> Oh, a layout issue. I don't like big black blobs (except when I >>>> write them). So >>>> %% wdrwertwertwert >>>> %% wertwertwertwrt >>>> %% wretertwertwertew >>>> -spec frgsdfgdfghsdfg >>>> asdfasdf(sdfgsdfg) -> >>>> .... >>>> >>>> is just a bit too much for me to be comfortable with. I'd find >>>> it easier to read with blank line after the %% comment block >>>> and a blank line after the -spec. >>>> >>>> I find that with my OWN code it's so obvious that packing it >>>> tightly doesn't impair readability, but with someone else's >>>> code well-placed blank lines never go amiss. I can only suppose >>>> that I'm in error about my own code and that it too could do with >>>> more blank lines. >>> >>> *MY* code is always perfectly readable as big, immaculate, unbroken >>> blocks. Until a few months later. Ugh. >>> >>> I'm not sure where I stand on the line break issue, really, but since >>> you mentioned it I'm trying out a line between doc and spec, a line >>> between spec and definition, and two lines between functions (but no >>> lines between function clauses -- I already know that confuses me, >>> especially with un-specced/un-docced functions). >> >> >> I don't feel very strongly on this one (I do about the supervisor spec), >> but I find the tight-packed blocks _more_ readable in other people's code. >> I run my eyes down the left-hand side and see >> >> % >> - >> a >> >> And that's pretty clearly (1) some comments, (2) a compiler directive, (3) >> a function. They all go together: I expect the comment to be function >> documentation and the dash to be a function spec. >> >> If I see >> >> % >> >> - >> >> a >> >> Then the comments might be module-level comments and the directive might be >> an export or an if or a define or a type declaration and I have to read >> more characters to figure out if those three lines are related. > > I sometimes think so myself. In this case, though, where every function has > an explicit `%% @doc` at the head of the function docs and everything else > follows a pretty obvious pattern, I do indeed find it easier with the extra > lines. With a mishmash of styles in a single source file, though... ugh. I > think it would get really annoying. Tomorrow I'll check again and see which > I like. Part of my goal here is to get some feedback before I start doing > more open-source stuff... the place I've been coding the last year has been > rather niche and impossible to make generally applicable someplace like > github. > > On second thought, I think the extra newlines actually improve *reading* > but may somewhat impare *scanning*. > > Consider: > https://github.com/zxq9/zuuid/blob/master/src/zuuid.erl > > One clear advantage is that my super-lazy-man's navigation in vi is far > more precise now, as each function and spec had is the start of what the > editor sees as its own block. Not a big deal (I usually jump directly to > wherever I want to go), but it is sort of nice. > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ewoznicka@REDACTED Thu Feb 4 14:18:23 2016 From: ewoznicka@REDACTED (ewoznicka@REDACTED) Date: Thu, 4 Feb 2016 14:18:23 +0100 Subject: [erlang-questions] team of Erlang developers for hire In-Reply-To: <544A172E.9050307@silversoft.pl> References: <544A172E.9050307@silversoft.pl> Message-ID: <56B34F9F.9090806@silversoft.pl> Hello, I'd like to let you know that our team has now some free man-power. If you have projects that need expert Erlang developers and/or for more information about us, please contact me. Sincerly yours, Ewa Woznicka Executive Assistant SilverSoft Sp. z o.o., Warsaw -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark@REDACTED Thu Feb 4 17:51:56 2016 From: mark@REDACTED (Mark Steele) Date: Thu, 4 Feb 2016 11:51:56 -0500 Subject: [erlang-questions] ANN: Cinched 0.0.1, an encryption microservice In-Reply-To: References: Message-ID: Hi Jesper, I'll take a look at your library, looks like it might make have some good enhancements over the bindings (especially the use of dirty schedulers) I'm currently using (which is erlang-nacl from Tony Garnock-Jones). Cheers, Mark Steele CISSP, GPEN, GCIA, CSM mark@REDACTED LinkedIn: https://ca.linkedin.com/in/markrsteele Github: https://github.com/marksteele Personal: http://www.control-alt-del.org On Thu, Feb 4, 2016 at 7:08 AM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > On Thu, Feb 4, 2016 at 5:38 AM, Mark Steele > wrote: > >> - libsodium (via NIF bindings) > > > I wonder if your approach is much different from > https://github.com/jlouis/enacl in any way? We may win by coalescing the > work there. > > > -- > J. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ameretat.reith@REDACTED Thu Feb 4 17:50:58 2016 From: ameretat.reith@REDACTED (Ameretat Reith) Date: Thu, 4 Feb 2016 20:20:58 +0330 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: References: <20160201231049.08b28dc7@gmail.com> Message-ID: <20160204202058.403be449@gmail.com> On Wed, 3 Feb 2016 12:24:40 -0600 Sean Cribbs wrote: > I'm curious, did you try gen_udp with the {active, N} option, or were > they already running in {active, true} mode? I got best results by {active, once}. I also played with {read_packets, N} socket option but nothing changed that much. From ameretat.reith@REDACTED Thu Feb 4 17:55:26 2016 From: ameretat.reith@REDACTED (Ameretat Reith) Date: Thu, 4 Feb 2016 20:25:26 +0330 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: References: <20160201231049.08b28dc7@gmail.com> Message-ID: <20160204202526.0cda43f6@gmail.com> On Wed, 3 Feb 2016 22:23:36 +0300 Max Lapshin wrote: > Kostis, problem is that when load of CPU is going higher than 50% you > may start loosing packets. How do you measure packet loss? Is there some kind of perf event for example to find them? I see no increase in interface dropped packets counter (with ifconfig), looks packets being dropped in Erlang side, not in kernel buffer. From max.lapshin@REDACTED Thu Feb 4 18:35:17 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 4 Feb 2016 20:35:17 +0300 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: <20160204202526.0cda43f6@gmail.com> References: <20160201231049.08b28dc7@gmail.com> <20160204202526.0cda43f6@gmail.com> Message-ID: /proc/net/udp I promise to extract library for reading /proc/net/udp /proc/stat /proc/diskstats /proc/meminfo /proc/net/dev that we are using to monitor server on which flussonic is running -------------- next part -------------- An HTML attachment was scrubbed... URL: From s@REDACTED Thu Feb 4 19:07:52 2016 From: s@REDACTED (Stefan Schmiedl) Date: Thu, 4 Feb 2016 19:07:52 +0100 Subject: [erlang-questions] Request feedback on example project In-Reply-To: <978AAB66-4294-4AA0-BFF5-1C72FECBF609@writersglen.com> References: <1771611.uqjCtSRlIV@changa> <1957725.BAnHVD50dh@changa> <1618384.kpENv6aRZF@changa> <978AAB66-4294-4AA0-BFF5-1C72FECBF609@writersglen.com> Message-ID: <20160204180752.GB13362@g128.fritz.box> Lloyd R. Prentice (04.02. 10:36): > > On second thought, I think the extra newlines actually improve *reading* > > but may somewhat impare *scanning*. > > > Consider: > > https://github.com/zxq9/zuuid/blob/master/src/zuuid.erl > > Ah, much easier on my aging eyes. +1. I'm stuck somewhere else, but this is something that I'd *like* to read. Well structured, not just-a-wall-of-text. s. From siraaj@REDACTED Thu Feb 4 21:09:33 2016 From: siraaj@REDACTED (Siraaj Khandkar) Date: Thu, 4 Feb 2016 15:09:33 -0500 Subject: [erlang-questions] Request feedback on example project In-Reply-To: <56B29936.2080700@cs.ntua.gr> References: <1771611.uqjCtSRlIV@changa> <56B29936.2080700@cs.ntua.gr> Message-ID: <56B3AFFD.80804@khandkar.net> On 2/3/16 7:20 PM, Kostis Sagonas wrote: >>.... > While at it, I suggest to use a list comprehension instead of lists:map. > Why? I use comprehensions occasionally, but never as first pick, for 2 reasons: (1) surprising filter semantics: 1> L = [{foo, a}, {bar, b}]. [{foo,a},{bar,b}] 2> 2> [X || {foo, X} <-L]. [a] 3> While lists:map/2 crashes, as expected: 3> lists:map(fun ({foo, X}) -> X end, L). ** exception error: no function clause matching erl_eval:'-inside-an-interpreted-fun-'({bar,b}) 4> There're certainly ways around that, such as: 4> [begin {foo, Foo} = X, Foo end || X <- L]. ** exception error: no match of right hand side value {bar,b} 5> 5> [(fun ({foo, Foo}) -> Foo end)(X) || X <- L]. ** exception error: no function clause matching erl_eval:'-inside-an-interpreted-fun-'({bar,b}) 6> But all that feels like extra gymnastics to just use list comprehensions for the sake of using list comprehensions. (2) Explicitly-declared intention. When you explicitly call "filter" and "map" - I have an immediate high-level idea of where you're going with it, but list comprehensions tempt one to conflate things, so it is harder to understand the intention (there're some gnarly ones out in the wild...). Surely, YMMV, but this is just my, correct, opinion WRT to blanket "just use list comprehensions" recommendations ;-) From kostis@REDACTED Thu Feb 4 21:18:36 2016 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 4 Feb 2016 21:18:36 +0100 Subject: [erlang-questions] Request feedback on example project In-Reply-To: <56B3AFFD.80804@khandkar.net> References: <1771611.uqjCtSRlIV@changa> <56B29936.2080700@cs.ntua.gr> <56B3AFFD.80804@khandkar.net> Message-ID: <56B3B21C.5010600@cs.ntua.gr> On 02/04/2016 09:09 PM, Siraaj Khandkar wrote: > On 2/3/16 7:20 PM, Kostis Sagonas wrote: >>> .... >> While at it, I suggest to use a list comprehension instead of lists:map. >> > > Why? Did you even bother looking at the code I was referring to, or you simply wanted to take something off your chest? The relavant code (which was included in my mail) read: ========================================================= -spec read_uuid_string(list()) -> uuid() | bad_uuid. read_uuid_string(UUID) -> Parts = string:tokens(UUID, "{-}"), case lists:map(fun length/1, Parts) of ... ========================================================= There is no filter here. The corresponding list comprehension is: [length(P) || P <- Parts] and is shorter and nicer. That's why. Kostis PS. With most of the rest that you write below, I agree. > I use comprehensions occasionally, but never as first pick, for 2 reasons: > > > (1) surprising filter semantics: > > 1> L = [{foo, a}, {bar, b}]. > [{foo,a},{bar,b}] > 2> > 2> [X || {foo, X} <-L]. > [a] > 3> > > While lists:map/2 crashes, as expected: > > 3> lists:map(fun ({foo, X}) -> X end, L). > ** exception error: no function clause matching > erl_eval:'-inside-an-interpreted-fun-'({bar,b}) > 4> > > There're certainly ways around that, such as: > > 4> [begin {foo, Foo} = X, Foo end || X <- L]. > ** exception error: no match of right hand side value {bar,b} > 5> > 5> [(fun ({foo, Foo}) -> Foo end)(X) || X <- L]. > ** exception error: no function clause matching > erl_eval:'-inside-an-interpreted-fun-'({bar,b}) > 6> > > But all that feels like extra gymnastics to just use list comprehensions > for the sake of using list comprehensions. > > > (2) Explicitly-declared intention. When you explicitly call "filter" and > "map" - I have an immediate high-level idea of where you're going with > it, but list comprehensions tempt one to conflate things, so it is > harder to understand the intention (there're some gnarly ones out in the > wild...). > > > Surely, YMMV, but this is just my, correct, opinion WRT to blanket "just > use list comprehensions" recommendations ;-) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Thu Feb 4 21:32:36 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 5 Feb 2016 09:32:36 +1300 Subject: [erlang-questions] Request feedback on example project In-Reply-To: References: <1771611.uqjCtSRlIV@changa> <56B2C6FE.2060901@cs.otago.ac.nz> <1957725.BAnHVD50dh@changa> Message-ID: <56B3B564.8010300@cs.otago.ac.nz> On 5/02/16 1:29 am, Nathaniel Waisbrot wrote: > > The new maps are self-documenting and they provide sane defaults so > that if I want a vanilla supervisor I can just say `#{id => > cool_super, start => {M, F, A}, type => supervisor}`. Not only do I > avoid dumb bugs, but it's also clear that this is a vanilla supervisor > and not something more interesting. I fail to see why something that accepts that could not also accept [{id,cool_super}, {start,{M,F,A}}, {type,supervisor}] with the same meaning and the same defaults. In fact the function maps:from_list/1 should make this trivial. as_map(Map) when is_map(Map) -> Map; as_map(List) when is_list(List) -> maps:from_list(List). Maps _as such_ don't provide defaults, sane or otherwise. There is a layout compromise that would give me the separation I need to read the lines and Nathaniel Walsbrot the continuity he wants to avoid a confusion that I've never personally experienced. %% big comment %% big comment % - spec .... % function(...) -> From vasdeveloper@REDACTED Thu Feb 4 22:00:24 2016 From: vasdeveloper@REDACTED (Theepan) Date: Fri, 5 Feb 2016 02:30:24 +0530 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: References: <20160201231049.08b28dc7@gmail.com> Message-ID: Max, You don't loose packets simply because the CPU reached 50%. or even 80%. On Thu, Feb 4, 2016 at 12:53 AM, Max Lapshin wrote: > Kostis, problem is that when load of CPU is going higher than 50% you may > start loosing packets. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Feb 4 22:06:29 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 5 Feb 2016 10:06:29 +1300 Subject: [erlang-questions] Request feedback on example project In-Reply-To: <1618384.kpENv6aRZF@changa> References: <1771611.uqjCtSRlIV@changa> <1957725.BAnHVD50dh@changa> <1618384.kpENv6aRZF@changa> Message-ID: <56B3BD55.1050007@cs.otago.ac.nz> I've now written an AWK script to insert the extra blank lines I wanted, and I found it helped me a *lot*. On 5/02/16 1:52 am, zxq9 wrote: > On second thought, I think the extra newlines actually improve *reading* > but may somewhat impare *scanning*. s/impare/impair/ Fair comment. Thing is, I was trying to *read* the code. For scanning the code, a scheme that begins every blob with %%@doc or %%@private &c is unhelpful, because all I ever see as I skip forward is those uninformative lines. When my eye (or the text editor's cursor) comes to rest, I want it to be resting on the function name. This means that to skip through the code in vi I need to do /^[a-z']/ . Let's take a look at the first thing in zuuid.erl. %% @doc %% Starts the zuuid application, spawning a supervisor and a worker to manage %% state generation details for version 1 and 2 UUIDs and ensure duplicates %% will not occur even at high call frequencies (as suggested in %% RFC 4122, 4.2.1). %% WHAT starts the application? The problem here is that I'm not so much reading internal documentation as I am some sort of assembly code for internal documentation. The information I *need* before I read these words, the information that *will* be displayed in formatted documentation before these words, is in *this* representation of the documentation many lines later. Since the -spec line has the function name and may well list the argument names and give a helpful name for the result (hint: Result is not a particularly helpful name), I personally would find it more readable if the order were -spec () -> ... %%@doc .... %% name(...) -> ... because in this order I would at least know what the @doc was *about*. In this order it might make sense *not* to have a blank line between the -spec and @doc because the @doc would be a grammatical continuation of the -spec. Maybe everybody else just runs files through edoc and puts that in one window while they edit code in another. From ok@REDACTED Thu Feb 4 22:30:28 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 5 Feb 2016 10:30:28 +1300 Subject: [erlang-questions] Request feedback on example project In-Reply-To: <56B3AFFD.80804@khandkar.net> References: <1771611.uqjCtSRlIV@changa> <56B29936.2080700@cs.ntua.gr> <56B3AFFD.80804@khandkar.net> Message-ID: <56B3C2F4.4060602@cs.otago.ac.nz> On 5/02/16 9:09 am, Siraaj Khandkar wrote: > (1) surprising filter semantics: > > 1> L = [{foo, a}, {bar, b}]. > [{foo,a},{bar,b}] > 2> > 2> [X || {foo, X} <-L]. > [a] > 3> This is how list comprehensions are supposed to work. What's surprising about that? > While lists:map/2 crashes, as expected: 3> lists:map(fun ({foo, X}) -> X end, L). Yes, but that is not a translation of the list comprehension and the list comprehension is not a translation of this. They are different, and that is one important reason to prefer the list comprehension. To get the effect of the list comprehension -- which is a commonly desired effect -- you would have to write lists:flatmap(fun ({foo,X}) -> [X] ; (_) -> [] end, L). which is a bit confusing because there isn't any useful connection between flatmap and flatten. To translate your map call as a list comprehension, F = fun ({foo,X}) -> X end, [F(X) || X <- L]. > There're certainly ways around that, such as: Why even think about "ways around that"? List comprehensions and calls to map are both doing their intended, documented, and different jobs. > > 4> [begin {foo, Foo} = X, Foo end || X <- L]. > ** exception error: no match of right hand side value {bar,b} > 5> > 5> [(fun ({foo, Foo}) -> Foo end)(X) || X <- L]. > ** exception error: no function clause matching > erl_eval:'-inside-an-interpreted-fun-'({bar,b}) > 6> > > But all that feels like extra gymnastics to just use list > comprehensions for the sake of using list comprehensions. No, it's extra gymnastics in the service of a straw man argument. In real code, I *want* the filtering behaviour of list comprehensions, so I use them instead of switching over to bulky calls to flatmap/2. > (2) Explicitly-declared intention. When you explicitly call "filter" > and "map" - I have an immediate high-level idea of where you're going > with it, but list comprehensions tempt one to conflate things, so it > is harder to understand the intention (there're some gnarly ones out > in the wild...). It is possible to write unreadable tangles of code using filter and map as well. One thing I would mention is the linguistic concept of "topicalisation", the way that people can shuffle different parts of a sentence to the beginning to highlight it. In [hypot(X, Y) || {X,Y} <- zip(Xs, Ys)] the topic is hypot(X, Y): we are invited to focus on the *result*. In map(zip(Xs,Ys), fun ({X,Y}) -> hypot(X, Y) end) the topic is zip(Xs, Ys): we are invited to focus on the *inputs*, not on what happens to them. Looking at it from this viewpoint, it is immediately obvious that *SOME* expressions will be more intention-revealing when written with the inputs first so that map or flatmap + filter is a good fit *SOME* expressions will be more intention-revealing when written with the results first. It's just like the way Haskell has *both* let Pattern = Expression in Result and Result where Pattern = Expression The fact that something is a call to map or a call to filter is NOT intention-revealing as such. All that tells me is the what, not the why, and the why is the intention. It is good that Erlang has *both* list comprehensions *and* higher- order functions. Do not blame either for not being identical to the other. From zxq9@REDACTED Thu Feb 4 23:31:08 2016 From: zxq9@REDACTED (zxq9) Date: Fri, 05 Feb 2016 07:31:08 +0900 Subject: [erlang-questions] Request feedback on example project In-Reply-To: <56B3BD55.1050007@cs.otago.ac.nz> References: <1771611.uqjCtSRlIV@changa> <1618384.kpENv6aRZF@changa> <56B3BD55.1050007@cs.otago.ac.nz> Message-ID: <9039047.3BPQCiTTLA@changa> On 2016?2?5? ??? 10:06:29 Richard A. O'Keefe wrote: > On 5/02/16 1:52 am, zxq9 wrote: > > but may somewhat impare *scanning*. > s/impare/impair/ (O.o) I actually typed that. (>o<) > Fair comment. Thing is, I was trying to *read* the code. > > For scanning the code, a scheme that begins every blob with > %%@doc > or > %%@private > &c is unhelpful, because all I ever see as I skip forward is those > uninformative lines. When my eye (or the text editor's cursor) > comes to rest, I want it to be resting on the function name. > > This means that to skip through the code in vi I need to do > /^[a-z']/ . > > Let's take a look at the first thing in zuuid.erl. > > %% @doc > %% Starts the zuuid application, spawning a supervisor and a worker to > manage > %% state generation details for version 1 and 2 UUIDs and ensure duplicates > %% will not occur even at high call frequencies (as suggested in > %% RFC 4122, > 4.2.1). > %% > > WHAT starts the application? > > The problem here is that I'm not so much reading internal documentation > as I am some sort of assembly code for internal documentation. The > information I *need* before I read these words, the information that *will* > be displayed in formatted documentation before these words, is in *this* > representation of the documentation many lines later. > > Since the -spec line has the function name and may well list the > argument names and give a helpful name for the result (hint: Result is > not a particularly helpful name), I personally would find it more > readable if the order were > > -spec () -> > ... > %%@doc > .... > %% > > name(...) -> ... > > because in this order I would at least know what the @doc was *about*. > > In this order it might make sense *not* to have a blank > line between the -spec and @doc because the @doc would > be a grammatical continuation of the -spec. This is a really good point, especially with functions that have lengthy doc comments above them. Looking at some other code for context, my eye (and editor) really does pick up about the first line or two of the doc comment, then jump to the spec or function head, then jump back up to continue reading the doc. I've never considered how silly this procedure actually is. I've switched them around now and the effect (especially now that I'm expecting it) is magical. https://github.com/zxq9/zuuid/blob/master/src/zuuid.erl Much easier to visually navigate. It is actually very readable even in plain, unhighlighted text now -- this is sort of the gold standard for readability in my mind. Thanks, again! -Craig From ameretat.reith@REDACTED Fri Feb 5 01:35:33 2016 From: ameretat.reith@REDACTED (Ameretat Reith) Date: Fri, 5 Feb 2016 04:05:33 +0330 Subject: [erlang-questions] Bit syntax matching gotchas In-Reply-To: References: Message-ID: <20160205040533.25d3fe77@gmail.com> On Wed, 3 Feb 2016 07:17:02 +0100 Bj?rn Gustavsson wrote: > We have decided to try do something about it in OTP 19. We have > discussed various solutions internally in the OTP group, and I have > spent far too much time lately thinking about it. Thanks for working on this. > The reason I reject it is that the matching previously bound variables > is uncommon. Even in the compiler test suites it is uncommon (test > suites typically match bound variables more often than production code > do). Not that common but nothing prevents coder to write: update_token(NewToken, ID) -> case NewToken of <> when (Now-Time) < 60 ... That looked logical to me and it took some time to understand I should change my code to: update_token(NewToken, ID) -> case NewToken of <> when ReceivedID =:= <>, (Now-Time) < 60 > Therefore, solution #2 would make behaviour of matching more > consistent with construction, but would not significantly increase the > number of correct programs. Also, if clauses that previously didn't > match start to match, then code that has not been executed before will > be executed. I don't think this is very bad. The code that now will be executed is most probably intended by coder to be executed and it may solve a hidden bug too. My above code surely intended to match, otherwise why I bothered myself to specify the size? Alas, this change effects cannot warned to coder. > YOUR INPUT? Now that I know how matching and construction differ, I can accept solution #1 and I prefer less language changes too. But if language correctness really matters, solution #2 is better. From mkbucc@REDACTED Fri Feb 5 02:59:47 2016 From: mkbucc@REDACTED (Mark Bucciarelli) Date: Thu, 4 Feb 2016 20:59:47 -0500 Subject: [erlang-questions] count_characters example from Programming Erlang Message-ID: Hi, I'm trying to compile the count_characters example from Programming Erlang book. But I get an error, and I can't see a typo. -module(t). -export([count_characters/1]). count_characters(Str) -> count_characters(Str, #{}). count_characters([H|T], #{ H := N }=X) -> count_characters(T, X#{ H := N+1 }); count_characters([H|T], X) -> count_characters(T, X#{ H => 1 }); count_characters([], X) -> X. 1> c(t). t.erl:7: variable 'H' is unbound error 2> This behavior contradicts the point made in the text: There are two things to note about count_characters/2. In the first clause, the variable H inside the map is also defined outside the map and thus is bound (as required). In the second clause, we use map_extend to add a new key to the map. Thanks! Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Fri Feb 5 05:58:19 2016 From: zxq9@REDACTED (zxq9) Date: Fri, 05 Feb 2016 13:58:19 +0900 Subject: [erlang-questions] count_characters example from Programming Erlang In-Reply-To: References: Message-ID: <5199281.3quezL6omW@changa> On 2016?2?4? ??? 20:59:47 Mark Bucciarelli wrote: > Hi, > > I'm trying to compile the count_characters example from Programming Erlang > book. > But I get an error, and I can't see a typo. > > -module(t). > -export([count_characters/1]). > > count_characters(Str) -> > count_characters(Str, #{}). > > count_characters([H|T], #{ H := N }=X) -> > count_characters(T, X#{ H := N+1 }); > > count_characters([H|T], X) -> > count_characters(T, X#{ H => 1 }); > > count_characters([], X) -> > X. > > 1> c(t). > t.erl:7: variable 'H' is unbound > error > 2> > > This behavior contradicts the point made in the text: > > There are two things to note about count_characters/2. > In the first clause, the variable H inside the map is also > defined outside the map and thus is bound (as required). > In the second clause, we use map_extend to add a new > key to the map. Someone will be along to sharpshoot this statement, but I'm pretty sure that the book was written based on the proposed map implementation, and the actual implementation turned out to not allow match-assignment on keys. I believe variable map keys are permitted in R18, but not sure about this particular case. A working version can be built this way, though: -module(foo). -export([count_characters/1]). count_characters(Str) -> count_characters(Str, #{}). count_characters([Char | Rest], Counts) -> Count = maps:get(Char, Counts, 0), count_characters(Rest, maps:put(Char, Count + 1, Counts)); count_characters([], Counts) -> Counts. Same principle, just slightly different way of going about it. The difficulty of writing a book against an unimplemented and still shifting standard is daunting. I'm amazed he was even able to! -Craig From bjorn@REDACTED Fri Feb 5 06:02:12 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Fri, 5 Feb 2016 06:02:12 +0100 Subject: [erlang-questions] Bit syntax matching gotchas In-Reply-To: <20160205040533.25d3fe77@gmail.com> References: <20160205040533.25d3fe77@gmail.com> Message-ID: On Fri, Feb 5, 2016 at 1:35 AM, Ameretat Reith wrote: > On Wed, 3 Feb 2016 07:17:02 +0100 > Bj?rn Gustavsson wrote: [...] >> The reason I reject it is that the matching previously bound variables >> is uncommon. Even in the compiler test suites it is uncommon (test >> suites typically match bound variables more often than production code >> do). > > Not that common but nothing prevents coder to write: > > update_token(NewToken, ID) -> > case NewToken of > <> when (Now-Time) < 60 > ... > > That looked logical to me and it took some time to understand I should > change my code to: > > update_token(NewToken, ID) -> > case NewToken of > <> when > ReceivedID =:= <>, (Now-Time) < 60 Your examples don't compile because of undefined variables. Assuming that Len and PrefLen should be the same variable, I don't see why you would need to change from the first example to the second. It seems that both of your examples would work if you fix the unbound variables. I did not write in may mail that matching of bound variables will not work. I only wrote that we would not add any automatic masking of integer values. That is, the following example works fine: t() -> t(16#FF, <<16#FF>>). t(A, Bin) -> <> = Bin. In the following example there will be a badmatch exception: t() -> t(16#FFFF, <<16#FF>>). t(A, Bin) -> <> = Bin. Solution #2 that I rejected would have made the second example match too. /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From bog495@REDACTED Fri Feb 5 09:16:38 2016 From: bog495@REDACTED (Bogdan Andu) Date: Fri, 5 Feb 2016 10:16:38 +0200 Subject: [erlang-questions] UDP active socket timeout In-Reply-To: References: Message-ID: this is easy on client side but on the server side there must be an interaction with spawned processes that actually use that socket to send data. other thing I found hard to achieve in UDP is: - to detect if a client is connected from what ports; - sending detection errors from server to client : gen_udp:send/4 always returns ok no matter what if port and/or ip is wrong. otherwise it would have been easy to implement something pinging the client with an empty udp packet and (possibly, again) to receive response; - gen_udp:recv is blocking (and with Timeout parameter even worse) and thus a design based on passive sockets leads to serialized processing of client requests; - if controlling process is used to handle other messages (from child procs) other then {udp, ...} message it would be unpractical from parallelization point of view of client request processing On Thu, Feb 4, 2016 at 4:43 PM, Sean Cribbs wrote: > UDP is a connection-less datagram protocol, so there's no timeout inherent > in the protocol. However, you may have timeouts on: > > 1) receiving a datagram (nothing arrived) > 2) sending a datagram (things went very badly) > > However, those are timeouts internal to Erlang, from your process to the > socket driver. If you are using the {active, true} option (which it sounds > like you are), you should use an 'after' clause in your 'receive' to > implement timeouts, or use the trailing timeout in the return tuple of a > gen_server/gen_fsm callback. > > On Thu, Feb 4, 2016 at 6:29 AM, Bogdan Andu wrote: > >> Hi, >> >> How can be detected a timeout on an active UDP socket? >> >> controlling process receives the message: >> {udp, Socket, Host, Port, Bin} >> >> and after a process is spawned to handle that data: >> spawn(fun() -> handle_pckt(Socket, Host, Port, Bin) end), >> >> Q: how the spawned or controlling process is able to detect >> that a timeout on this UDP socket occurred ? >> >> documentations does not mention anything about timeout on active sockets >> >> Thanks, >> >> /Bogdan >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Fri Feb 5 09:24:34 2016 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 5 Feb 2016 09:24:34 +0100 Subject: [erlang-questions] count_characters example from Programming Erlang In-Reply-To: <5199281.3quezL6omW@changa> References: <5199281.3quezL6omW@changa> Message-ID: On Fri, Feb 5, 2016 at 5:58 AM, zxq9 wrote: > On 2016?2?4? ??? 20:59:47 Mark Bucciarelli wrote: >> Hi, >> >> I'm trying to compile the count_characters example from Programming Erlang >> book. >> But I get an error, and I can't see a typo. >> >> -module(t). >> -export([count_characters/1]). >> >> count_characters(Str) -> >> count_characters(Str, #{}). >> >> count_characters([H|T], #{ H := N }=X) -> >> count_characters(T, X#{ H := N+1 }); >> >> count_characters([H|T], X) -> >> count_characters(T, X#{ H => 1 }); >> >> count_characters([], X) -> >> X. >> >> 1> c(t). >> t.erl:7: variable 'H' is unbound >> error >> 2> >> >> This behavior contradicts the point made in the text: >> >> There are two things to note about count_characters/2. >> In the first clause, the variable H inside the map is also >> defined outside the map and thus is bound (as required). >> In the second clause, we use map_extend to add a new >> key to the map. > > Someone will be along to sharpshoot this statement, but I'm pretty sure > that the book was written based on the proposed map implementation, and > the actual implementation turned out to not allow match-assignment on keys. > > I believe variable map keys are permitted in R18, but not sure about this > particular case. > > A working version can be built this way, though: > > -module(foo). > -export([count_characters/1]). > > count_characters(Str) -> > count_characters(Str, #{}). > > count_characters([Char | Rest], Counts) -> > Count = maps:get(Char, Counts, 0), > count_characters(Rest, maps:put(Char, Count + 1, Counts)); > count_characters([], Counts) -> > Counts. > > Same principle, just slightly different way of going about it. > > The difficulty of writing a book against an unimplemented and still > shifting standard is daunting. I'm amazed he was even able to! The idea was that by the time the book was published the book and implementation would be in phase with each other. There's a back story here: In the early days of Erlang the implementation and the documentation were always out of phase. This means there was a heck of a lot of code and very little documentation. When the documentation was written there was short time when it was in phase with the code but then the code changed and the documentation was not updated. The point is the two were never in phase, or at least if there were in phase it was for a short time. At the time I used to say "If the code and documentation different, read the code." But reading code is difficult - especially for a beginner. Anyway even if I can understand the code I don't know a) what it's supposed to do and b) if it is buggy. When the code and documentation differ I don't know which is definitive. At a certain point in time I thought to myself - "this is stupid" so I changed strategy. I said - "the documentation should describe that the system is supposed to do, and if the code and documentation differ it's a bug and the code is wrong." Then I wrote the documentation from the standpoint of what the system should do rather than what it actually does, and I do this in my books. Most system documentation describe what the code does, and not what it should do. In an ideal world the two should be the same - but this is only true in very mature systems. Now to the point in question. If stare hard at the clause where the problem is it looks like this: count_characters([H|T], #{ H := N }=X}) Erlang semantics dictate that if a variable occurs more than once in a pattern then it must have the same value - so reading from left to right H has a value after matching the first argument in the function and so this value can be used as a key in the hash map lookup that is implies by the second argument. This should be similar to the behavior of pattern matching in binaries where you can "propagate forward" a length of a segment, which has been pattern matched earlier in the binary. For example: <> = <<2,1,2,3,4,5>> matches with bindings Len = 2, A = <<1,2>> and B = <<3,4,5>> So Len is propagated forwards in the pattern match (in this case, and in the hashmap case we can't match all the arguments "in parallel" but have to do it in a specific order. To compile hashmap patterns like the one above that failed, is certainly possible. /Joe > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Fri Feb 5 09:47:32 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 5 Feb 2016 11:47:32 +0300 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: References: <20160201231049.08b28dc7@gmail.com> Message-ID: Well, I'm not going to argue about it, but I know that it is a serious blocker for us: when flussonic is consuming 50% of all cores only on capturing (unpacking mpegts is another pain) when code in C takes only 10-15% for this task, customers are complaining. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Fri Feb 5 10:00:02 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 05 Feb 2016 09:00:02 +0000 Subject: [erlang-questions] small eventloop in a nif? Message-ID: For a project i need to bind some some sockets events in a nif and i wonder what would be the simple cross-platform event loop i could use. select is limited to 1024 sockets on some platforms. I wonder if outside libev, libevent or libuv someone found a simpler one ? Simple means easy to embed/build statically and with a simple API. I only need to manage sockets events. Any idea? beno?t -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri Feb 5 11:37:08 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 5 Feb 2016 13:37:08 +0300 Subject: [erlang-questions] small eventloop in a nif? In-Reply-To: References: Message-ID: libevent is traditionally the most convenient, but for reasons that I don't know, it is abandoned and claimed to be bad. But it's bufferedevent is one the best things around all event-based programming. Why don't use an erlang driver? It is working inside erlang loop. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Fri Feb 5 11:40:22 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 5 Feb 2016 11:40:22 +0100 Subject: [erlang-questions] small eventloop in a nif? In-Reply-To: References: Message-ID: thanks for the answer :) > On 05 Feb 2016, at 11:37, Max Lapshin wrote: > > libevent is traditionally the most convenient, but for reasons that I don't know, it is abandoned and claimed to be bad. But it's bufferedevent is one the best things around all event-based programming. OK I see, I also hesitate with just using libev which is available if i am not wrong in the openbsd core. Picoev - I just found - also seems small but less maintained. > > Why don't use an erlang driver? It is working inside erlang loop. Not sure what you mean there. Can you elaborate? - benoit From sergej.jurecko@REDACTED Fri Feb 5 12:17:15 2016 From: sergej.jurecko@REDACTED (=?UTF-8?Q?Sergej_Jure=C4=8Dko?=) Date: Fri, 5 Feb 2016 12:17:15 +0100 Subject: [erlang-questions] small eventloop in a nif? In-Reply-To: References: Message-ID: > > > > > Why don't use an erlang driver? It is working inside erlang loop. > > Not sure what you mean there. Can you elaborate? > > http://erlang.org/doc/man/erl_driver.html#driver_select -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri Feb 5 12:53:31 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 5 Feb 2016 14:53:31 +0300 Subject: [erlang-questions] small eventloop in a nif? In-Reply-To: References: Message-ID: You can refer to this code as an example: https://github.com/erlyvideo/udpts/blob/master/c_src/mpegts_udp.c -------------- next part -------------- An HTML attachment was scrubbed... URL: From vasdeveloper@REDACTED Fri Feb 5 13:40:33 2016 From: vasdeveloper@REDACTED (Theepan) Date: Fri, 5 Feb 2016 18:10:33 +0530 Subject: [erlang-questions] Illegal Guard? Message-ID: Team - Any idea? 60> 60> 60> is_integer(A = 5). true 61> 61> fun () when is_integer(A = 5) -> ok end. * 2: illegal guard expression 62> 62> 62> 62> fun () when is_integer(5) -> ok end. #Fun 63> 63> 63> fun (A = 5) when is_integer(A) -> ok end. #Fun 64> 64> -------------- next part -------------- An HTML attachment was scrubbed... URL: From radek@REDACTED Fri Feb 5 13:51:31 2016 From: radek@REDACTED (Rad Gruchalski) Date: Fri, 5 Feb 2016 13:51:31 +0100 Subject: [erlang-questions] Illegal Guard? In-Reply-To: References: Message-ID: <0734D55E371C4F35B23864E126BFCEE1@gruchalski.com> It?s explained here: http://erlang.org/doc/reference_manual/expressions.html#id83710 The set of valid guard expressions (sometimes called guard tests) is a subset of the set of valid Erlang expressions. The reason for restricting the set of valid expressions is that evaluation of a guard expression must be guaranteed to be free of side effects. Kind regards, Radek Gruchalski radek@REDACTED (mailto:radek@REDACTED) (mailto:radek@REDACTED) de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/) Confidentiality: This communication is intended for the above-named person and may be confidential and/or legally privileged. If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately. On Friday, 5 February 2016 at 13:40, Theepan wrote: > > Team - Any idea? > > > > > > > 60> > > > 60> > > > 60> is_integer(A = 5). > > > true > > > 61> > > > 61> fun () when is_integer(A = 5) -> ok end. > > > * 2: illegal guard expression > > > 62> > > > 62> > > > 62> > > > 62> fun () when is_integer(5) -> ok end. > > > #Fun > > > 63> > > > 63> > > > 63> fun (A = 5) when is_integer(A) -> ok end. > > > #Fun > > > 64> > > > 64> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Fri Feb 5 14:09:15 2016 From: zxq9@REDACTED (zxq9) Date: Fri, 05 Feb 2016 22:09:15 +0900 Subject: [erlang-questions] Illegal Guard? In-Reply-To: References: Message-ID: <1838507.94il841jNi@changa> On 2016?2?5? ??? 18:10:33 Theepan wrote: > Team - Any idea? > > 60> is_integer(A = 5). > true > 61> fun () when is_integer(A = 5) -> ok end. > * 2: illegal guard expression > 62> fun () when is_integer(5) -> ok end. > #Fun > 63> fun (A = 5) when is_integer(A) -> ok end. > #Fun I think it has to do with binding VS masking. That is to say, what part of a function head is not yet referring to an external scope, so you can't yet create a closure over an existing value: 1> is_integer(A = 5). true 2> B = fun() -> A end. #Fun 3> C = fun(A = 6) -> A end. #Fun 4> D = fun(A) -> A end. #Fun A is now actually assigned to 5 from our first statement -- which is running within the context of an ongoing fun (I think). That being as it may, anything assigned within the head of a function, apparently to include guards, is masking whatever else existed. Its not about is_integer/1 as much as it is about where it is used. 5> A. 5 6> B(). 5 7> C(6). 6 8> D(7). 7 Now let's drop the current scope's memory of A: 9> f(A). ok 10> A. * 1: variable 'A' is unbound 11> B(). 5 12> C(7). ** exception error: no function clause matching erl_eval:'-inside-an-interpreted-fun-'(7) 13> D(7). 7 Now let's see what happens if we use an assignment within a guard within a function's inner scope: 14> E = fun(A) when is_integer(A) -> 14> case is_integer(Z = A) of 14> true -> Z; 14> false -> "strangeness" 14> end 14> end. #Fun 15> E(10). 10 I don't know for sure, but this certainly leads me to believe it has something to do with where the actual scoping boundary lies. In other news... another thought crossed my mind... WHY ARE YOU ASSIGNING INSIDE A GUARD? Just to deepen the mystery because needlessly exploring corner cases that should never occur in actual code is sort of funny to me... 16> A = 5. 5 17> F = fun(A) when is_integer(A) -> 17> case is_integer(Z = A) of 17> true -> Z; 17> false -> "strangeness" 17> end 17> end. #Fun 18> F(5). 5 19> F(10). 10 20> A. 5 Someone who really knows will get bored and give a real answer on Monday, perhaps... ? -Craig From bchesneau@REDACTED Fri Feb 5 14:56:02 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 05 Feb 2016 13:56:02 +0000 Subject: [erlang-questions] small eventloop in a nif? In-Reply-To: References: Message-ID: thanks! it's gives me a good start :) On Fri, 5 Feb 2016 at 12:53, Max Lapshin wrote: > You can refer to this code as an example: > https://github.com/erlyvideo/udpts/blob/master/c_src/mpegts_udp.c > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Trond.Endrestol@REDACTED Fri Feb 5 16:25:25 2016 From: Trond.Endrestol@REDACTED (=?UTF-8?Q?Trond_Endrest=C3=B8l?=) Date: Fri, 5 Feb 2016 16:25:25 +0100 (CET) Subject: [erlang-questions] Illegal Guard? In-Reply-To: <0734D55E371C4F35B23864E126BFCEE1@gruchalski.com> References: <0734D55E371C4F35B23864E126BFCEE1@gruchalski.com> Message-ID: On Fri, 5 Feb 2016 13:51+0100, Rad Gruchalski wrote: > The reason for restricting the set of valid expressions is that > evaluation of a guard expression must be guaranteed to be free of > side effects. Is there a way of convincing the Erlang compiler that a certain function is free of any side effects? Below is what I believe to be a valid case. While watching prof. Simon Thompson's Erlang Master Classes last year, he demonstrated in Master Class 1, Video 6, a parser where parts of the Erlang code was duplicated too much for my own taste. Ref.: http://www.kent.ac.uk/elearning/themes/masterclasses.html I typed my source file as I watched the video and tried to optimise the code by using macros. Here's a couple of fragments from my own file. Notice the smaller classifier functions and the parse function both use macros to cope with (a) code duplication, and (b) the lack of user defined guard tests: % Master Class 1, Video 6: % I really wish we could convince the compiler to believe these three % functions are without any side effects, and thus suitable as guard % tests. % Using macros is one way of managing the duplicate code. -spec is_alpha(integer()) -> boolean(). -define(IS_ALPHA(Ch), $a =< Ch andalso Ch =< $z). is_alpha(Ch) -> ?IS_ALPHA(Ch). -spec is_num(integer()) -> boolean(). -define(IS_NUM(Ch), $0 =< Ch andalso Ch =< $9). is_num(Ch) -> ?IS_NUM(Ch). -spec is_white(integer()) -> boolean(). -define(HT, 9). -define(LF, 10). -define(VT, 11). -define(FF, 12). -define(CR, 13). -define(SP, 32). -define(IS_WHITE(Ch), (Ch == ?SP) orelse (Ch == ?HT) orelse (Ch == ?LF) orelse (Ch == ?CR) orelse (Ch == ?FF) orelse (Ch == ?VT)). is_white(Ch) -> ?IS_WHITE(Ch). %% Lines omitted -spec parse(string()) -> {expr(), string()} | string(). parse([$( | Rest]) -> {E1, Rest1} = parse(Rest), [Op | Rest2] = parse(Rest1), {E2, Rest3} = parse(Rest2), [$) | RestFinal] = parse(Rest3), {case Op of $+ -> {'add', E1, E2}; $- -> {'sub', E1, E2}; $* -> {'mul', E1, E2}; $/ -> {'div', E1, E2} end, RestFinal}; parse([Ch | Rest]) when (Ch == $)) orelse (Ch == $+) orelse (Ch == $-) orelse (Ch == $*) orelse (Ch == $/) -> [Ch | Rest]; parse([Ch | Rest]) when ?IS_ALPHA(Ch) -> {Succeeds, Remainder} = get_while(fun is_alpha/1, Rest), {{var, list_to_atom([Ch | Succeeds])}, Remainder}; parse([Ch | Rest]) when ?IS_NUM(Ch) -> {Succeeds, Remainder} = get_while(fun is_num/1, Rest), {{num, list_to_integer([Ch | Succeeds])}, Remainder}; parse([Ch | Rest]) when ?IS_WHITE(Ch) -> {_Succeeds, Remainder} = get_while(fun is_white/1, Rest), parse(Remainder). -- ---------------------------------------------------------------------- Trond Endrest?l | Trond.Endrestol@REDACTED ACM, NAS, NUUG, SAGE, USENIX | FreeBSD 10.3-P & Alpine 2.20 From mkbucc@REDACTED Sat Feb 6 00:26:14 2016 From: mkbucc@REDACTED (Mark Bucciarelli) Date: Fri, 5 Feb 2016 18:26:14 -0500 Subject: [erlang-questions] count_characters example from Programming Erlang In-Reply-To: References: <5199281.3quezL6omW@changa> Message-ID: On Fri, Feb 5, 2016 at 3:24 AM, Joe Armstrong wrote: > > The idea was that by the time the book was published the book and > implementation would be in phase with each other. > > There's a back story here: > > In the early days of Erlang the implementation and the documentation were > always out of phase. This means there was a heck of a lot of code and very > little documentation. > > When the documentation was written there was short time when it was in phase > with the code but then the code changed and the documentation was not > updated. The point is the two were never in phase, or at least if there were > in phase it was for a short time. > > At the time I used to say > > "If the code and documentation different, read the code." > > But reading code is difficult - especially for a beginner. > > Anyway even if I can understand the code I don't know a) what it's supposed > to do and b) if it is buggy. > > When the code and documentation differ I don't know which is definitive. > > At a certain point in time I thought to myself - "this is stupid" so I > changed strategy. > > I said - > > "the documentation should describe that the system is supposed to do, and > if the code and documentation differ it's a bug and the code is wrong." > > Then I wrote the documentation from the standpoint of what the system should > do rather than what it actually does, and I do this in my books. > > Most system documentation describe what the code does, and not what it should > do. > > In an ideal world the two should be the same - but this is only true in very > mature systems. > So your book describes the expected behavior. And thanks for that back story. I struggle with the issue of technical documentation at my day job; if you have code, comments and technical specs, those are three version of the same state. I would prefer to let the code be the technical spec, but the systems I work with are pretty small. > > Now to the point in question. If stare hard at the clause where the problem > is it looks like this: > > count_characters([H|T], #{ H := N }=X}) > > Erlang semantics dictate that if a variable occurs more than once in a > pattern then it must have the same value - so reading from left to right H > has a value after matching the first argument in the function and so this > value can be used as a key in the hash map lookup that is implies by the > second argument. > > This should be similar to the behavior of pattern matching in binaries where > you can "propagate forward" a length of a segment, which has been pattern > matched earlier in the binary. > > For example: > > <> = <<2,1,2,3,4,5>> > > matches with bindings Len = 2, A = <<1,2>> and B = <<3,4,5>> So Len is > propagated forwards in the pattern match (in this case, and in the hashmap > case we can't match all the arguments "in parallel" but have to do it in a > specific order. > > To compile hashmap patterns like the one above that failed, is certainly > possible. > Would you like me to file a bug reports for this case? And let me take this opportunity to say I am _really_ enjoying learning Erlang. I'm about one week in and I wanted to play hooky so I could keep tinkering. Thanks, Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From lheadley@REDACTED Sat Feb 6 01:57:35 2016 From: lheadley@REDACTED (Lyn Headley) Date: Fri, 5 Feb 2016 16:57:35 -0800 Subject: [erlang-questions] Request feedback on example project In-Reply-To: <1771611.uqjCtSRlIV@changa> References: <1771611.uqjCtSRlIV@changa> Message-ID: My issue is that it doesn't explain what a UUID is, and refers me to an RFC if I want to find out. But RFC's are too detailed. So I never figured it out, and wandered off... On Tue, Feb 2, 2016 at 10:00 PM, zxq9 wrote: > Two days ago I wrote a simple UUID utility as an example of bit syntax. > Yesterday I decided I wanted it to be an example of documentation and > readability as well. > > I would like feedback from new and old Erlangers in terms of readability, > documentation, what is easy to understand, what is hard to understand, etc. > My goal is to develop a style plain enough that a utility application just > above the threshold of clear triviality (like a UUID utility) is mostly > understandable even to someone new to Erlang. > > https://github.com/zxq9/zuuid > > No ninja tricks should be in there. If there are any please bring them to > my attention so they can be removed. > > Thanks > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zxq9@REDACTED Sat Feb 6 08:48:06 2016 From: zxq9@REDACTED (zxq9) Date: Sat, 06 Feb 2016 16:48:06 +0900 Subject: [erlang-questions] Request feedback on example project In-Reply-To: References: <1771611.uqjCtSRlIV@changa> Message-ID: <8364711.uhRC1a4Jpl@changa> Hi Lyn, Oops! I assumed (wrongly) that everyone just sort of knew what this was. I've remedied that and also put the project README in line with the doc overview. http://zxq9.com/projects/zuuid/docs/ If this explanation doesn't make sense, is confusing, doesn't touch on what you think is relevant, etc. please let me know. Thanks for taking the time to comment -- I'm blind to this sort of thing without feedback. -Craig On 2016?2?5? ??? 16:57:35 you wrote: > My issue is that it doesn't explain what a UUID is, and refers me to > an RFC if I want to find out. But RFC's are too detailed. So I never > figured it out, and wandered off... > > On Tue, Feb 2, 2016 at 10:00 PM, zxq9 wrote: > > Two days ago I wrote a simple UUID utility as an example of bit syntax. > > Yesterday I decided I wanted it to be an example of documentation and > > readability as well. > > > > I would like feedback from new and old Erlangers in terms of readability, > > documentation, what is easy to understand, what is hard to understand, etc. > > My goal is to develop a style plain enough that a utility application just > > above the threshold of clear triviality (like a UUID utility) is mostly > > understandable even to someone new to Erlang. > > > > https://github.com/zxq9/zuuid > > > > No ninja tricks should be in there. If there are any please bring them to > > my attention so they can be removed. > > > > Thanks > > > > -Craig > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Sat Feb 6 09:28:40 2016 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 6 Feb 2016 09:28:40 +0100 Subject: [erlang-questions] Request feedback on example project In-Reply-To: <1771611.uqjCtSRlIV@changa> References: <1771611.uqjCtSRlIV@changa> Message-ID: On Wed, Feb 3, 2016 at 7:00 AM, zxq9 wrote: > Two days ago I wrote a simple UUID utility as an example of bit syntax. > Yesterday I decided I wanted it to be an example of documentation and > readability as well. I took a quick look and liked what I saw. In my opinion one should publish documentation and compiled code. If the user needs to see the source then the documentation is inadequate source should be available if requested and for the purpose of modification but not because the documentation is inadequate. You documentation is excellent. One tiny point in http://erlang.org/pipermail/erlang-questions/2015-March/083879.html I argued that interfaces should be *minimal* - you export uuid:string and uuid:binary I'm beginning to dislike code where I can call functions with polymorphic arguments, filenames that can be strings, atoms, I/O lists whatever. I feel like shouting - "make you mind up". Do you want me to call your library through the string or binary interface? I would reason like this. "Should a user see the printed representation of UUIDs or should they be hidden from sight?" I think UUID's should not be seen by humans only by programs - therefore only the binary interface should be available which will encourage this usage. So remove the uuid:string functions. Cheers /Joe > > I would like feedback from new and old Erlangers in terms of readability, > documentation, what is easy to understand, what is hard to understand, etc. > My goal is to develop a style plain enough that a utility application just > above the threshold of clear triviality (like a UUID utility) is mostly > understandable even to someone new to Erlang. > > https://github.com/zxq9/zuuid > > No ninja tricks should be in there. If there are any please bring them to > my attention so they can be removed. > > Thanks > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Sat Feb 6 09:33:21 2016 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 6 Feb 2016 09:33:21 +0100 Subject: [erlang-questions] count_characters example from Programming Erlang In-Reply-To: References: <5199281.3quezL6omW@changa> Message-ID: On Sat, Feb 6, 2016 at 12:26 AM, Mark Bucciarelli wrote: > On Fri, Feb 5, 2016 at 3:24 AM, Joe Armstrong wrote: >> >> The idea was that by the time the book was published the book and >> implementation would be in phase with each other. >> >> There's a back story here: >> >> In the early days of Erlang the implementation and the documentation were >> always out of phase. This means there was a heck of a lot of code and very >> little documentation. >> >> When the documentation was written there was short time when it was in >> phase >> with the code but then the code changed and the documentation was not >> updated. The point is the two were never in phase, or at least if there >> were >> in phase it was for a short time. >> >> At the time I used to say >> >> "If the code and documentation different, read the code." >> >> But reading code is difficult - especially for a beginner. >> >> Anyway even if I can understand the code I don't know a) what it's >> supposed >> to do and b) if it is buggy. >> >> When the code and documentation differ I don't know which is definitive. >> >> At a certain point in time I thought to myself - "this is stupid" so I >> changed strategy. >> >> I said - >> >> "the documentation should describe that the system is supposed to do, >> and >> if the code and documentation differ it's a bug and the code is >> wrong." >> >> Then I wrote the documentation from the standpoint of what the system >> should >> do rather than what it actually does, and I do this in my books. >> >> Most system documentation describe what the code does, and not what it >> should >> do. >> >> In an ideal world the two should be the same - but this is only true in >> very >> mature systems. >> > > So your book describes the expected behavior. What I want it to be - it's vision of the future - not a description of the current state of art. > > And thanks for that back story. I struggle with the issue of technical > documentation at my day job; if you have code, comments and technical specs, > those are three version of the same state. I would prefer to let the code > be > the technical spec, but the systems I work with are pretty small. This was the same in the early days of Erlang - I think this is always the case in a programmer lead project - sooner or later the change must occure after a few months or years > >> >> Now to the point in question. If stare hard at the clause where the >> problem >> is it looks like this: >> >> count_characters([H|T], #{ H := N }=X}) >> >> Erlang semantics dictate that if a variable occurs more than once in a >> pattern then it must have the same value - so reading from left to right H >> has a value after matching the first argument in the function and so this >> value can be used as a key in the hash map lookup that is implies by the >> second argument. >> >> This should be similar to the behavior of pattern matching in binaries >> where >> you can "propagate forward" a length of a segment, which has been pattern >> matched earlier in the binary. >> >> For example: >> >> <> = <<2,1,2,3,4,5>> >> >> matches with bindings Len = 2, A = <<1,2>> and B = <<3,4,5>> So Len is >> propagated forwards in the pattern match (in this case, and in the hashmap >> case we can't match all the arguments "in parallel" but have to do it in a >> specific order. >> >> To compile hashmap patterns like the one above that failed, is certainly >> possible. >> > > Would you like me to file a bug reports for this case? > Yes :-) > And let me take this opportunity to say I am _really_ enjoying learning > Erlang. > I'm about one week in and I wanted to play hooky so I could keep tinkering. > Glad you're enjoying it /Joe > Thanks, > > Mark > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From zxq9@REDACTED Sat Feb 6 10:39:26 2016 From: zxq9@REDACTED (zxq9) Date: Sat, 06 Feb 2016 18:39:26 +0900 Subject: [erlang-questions] Request feedback on example project In-Reply-To: References: <1771611.uqjCtSRlIV@changa> Message-ID: <1563328.VYh2viIhZm@changa> On 2016?2?6? ??? 09:28:40 you wrote: > On Wed, Feb 3, 2016 at 7:00 AM, zxq9 wrote: > > Two days ago I wrote a simple UUID utility as an example of bit syntax. > > Yesterday I decided I wanted it to be an example of documentation and > > readability as well. > > I took a quick look and liked what I saw. > > In my opinion one should publish documentation and compiled code. > If the user needs to see the source then the documentation is inadequate > source should be available if requested and for the purpose of modification > but not because the documentation is inadequate. > > You documentation is excellent. > > One tiny point in > > http://erlang.org/pipermail/erlang-questions/2015-March/083879.html > > I argued that interfaces should be *minimal* - you export uuid:string > and uuid:binary > > I'm beginning to dislike code where I can call functions with polymorphic > arguments, filenames that can be strings, atoms, I/O lists whatever. > > I feel like shouting - "make you mind up". Do you want me to call your > library through the string or binary interface? > > I would reason like this. "Should a user see the printed representation of > UUIDs or should they be hidden from sight?" I think UUID's should not > be seen by humans only by programs - therefore only the binary interface > should be available which will encourage this usage. So remove the > uuid:string functions. I agree wholeheartedly. In this case uuid() [1] is its own internal type. This is the only type accepted by zuuid:string/2 [2] and zuuid:binary/2 [3]. The string and binary functions are output-only, to convert the internal uuid() type to serialized representations required by various databases and frameworks (its not compliant with RFC 4122 if it can't at *least* output a canonical string representation of a UUID). Without those output options this application would not be very useful. You can't pass an arbitrary representation to string/2 or binary/2, but you must be able to pass an arbitrary representation (especially any form corresponding to the canonical string representation in RFC 4122) to the external-to-internal function, read_uuid/1. Without this it would be impossible to interoperate with pretty much any other UUID tools. Postgres returns the equivalent of zuuid:binary(UUID, brackets), for example -- so at a minimum that must be readable. I'm not disputing your points -- the API reflects my approach to dealing with them. If there is a better way I am all ears. If the API is *confusing* or there is an obviously better way to provide this functionality, though, I would really like to know. (I already deprecated string/1 and binary/1 because they really were half-baked.) -Craig [1. http://zxq9.com/projects/zuuid/docs/zuuid.html#type-uuid] [2. http://zxq9.com/projects/zuuid/docs/zuuid.html#string-2] [3. http://zxq9.com/projects/zuuid/docs/zuuid.html#binary-2] From mkbucc@REDACTED Sat Feb 6 12:29:56 2016 From: mkbucc@REDACTED (Mark Bucciarelli) Date: Sat, 6 Feb 2016 06:29:56 -0500 Subject: [erlang-questions] count_characters example from Programming Erlang In-Reply-To: References: <5199281.3quezL6omW@changa> Message-ID: <20160206112956.GA14778@marks-mbp-3> On Sat, Feb 06, 2016 at 09:33:21AM +0100, Joe Armstrong wrote: > On Sat, Feb 6, 2016 at 12:26 AM, Mark Bucciarelli wrote: > > > > Would you like me to file a bug reports for this case? > > > > Yes :-) > Done. http://bugs.erlang.org/browse/ERL-88 Mark From mkbucc@REDACTED Sat Feb 6 13:57:47 2016 From: mkbucc@REDACTED (Mark Bucciarelli) Date: Sat, 6 Feb 2016 07:57:47 -0500 Subject: [erlang-questions] Blog entry: Erlang: The equals sign is a matching operator Message-ID: <20160206125747.GA15141@marks-mbp-3> Hi, After working with Erlang for a couple weeks, I was inspired to write a blog entry on the lowly equals sign. http://mbucc.blogspot.com/2016/02/erlang-equals-sign-is-matching-operator.html I hope you enjoy it, and any comments/feedback/corrections are welcome. Thanks, Mark From duncan@REDACTED Sat Feb 6 13:58:46 2016 From: duncan@REDACTED (duncan@REDACTED) Date: Sat, 06 Feb 2016 05:58:46 -0700 Subject: [erlang-questions] =?utf-8?q?count=5Fcharacters_example_from_Prog?= =?utf-8?q?ramming_Erlang?= Message-ID: <20160206055846.7e43b23f706d1a78218bd3e1c66e57ee.c13703cce1.wbe@email23.secureserver.net> An HTML attachment was scrubbed... URL: From ivan@REDACTED Sat Feb 6 14:58:41 2016 From: ivan@REDACTED (Ivan Uemlianin) Date: Sat, 06 Feb 2016 13:58:41 +0000 Subject: [erlang-questions] Blog entry: Erlang: The equals sign is a matching operator In-Reply-To: <20160206125747.GA15141@marks-mbp-3> References: <20160206125747.GA15141@marks-mbp-3> Message-ID: <1454767123096-2bc46fa7-b5433d10-cf98ca8c@llaisdy.com> Nice piece On Sat, Feb 6, 2016 at 12:57 pm, Mark Bucciarelli wrote: Hi, After working with Erlang for a couple weeks, I was inspired to write a blog entry on the lowly equals sign. http://mbucc.blogspot.com/2016/02/erlang-equals-sign-is-matching-operator.html I hope you enjoy it, and any comments/feedback/corrections are welcome. Thanks, Mark _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vasdeveloper@REDACTED Sat Feb 6 17:32:28 2016 From: vasdeveloper@REDACTED (Theepan) Date: Sat, 6 Feb 2016 22:02:28 +0530 Subject: [erlang-questions] Illegal Guard? In-Reply-To: <0734D55E371C4F35B23864E126BFCEE1@gruchalski.com> References: <0734D55E371C4F35B23864E126BFCEE1@gruchalski.com> Message-ID: Hi Rad, Thanks, that helps. Regards, Theepan On Fri, Feb 5, 2016 at 6:21 PM, Rad Gruchalski wrote: > It?s explained here: > http://erlang.org/doc/reference_manual/expressions.html#id83710 > The set of valid *guard expressions* (sometimes called guard tests) is a > subset of the set of valid Erlang expressions. The reason for restricting > the set of valid expressions is that evaluation of a guard expression must > be guaranteed to be free of side effects. > > Kind regards, > Radek Gruchalski > radek@REDACTED > de.linkedin.com/in/radgruchalski/ > > > *Confidentiality:*This communication is intended for the above-named > person and may be confidential and/or legally privileged. > If it has come to you in error you must take no action based on it, nor > must you copy or show it to anyone; please delete/destroy and inform the > sender immediately. > > On Friday, 5 February 2016 at 13:40, Theepan wrote: > > Team - Any idea? > > > 60> > > 60> > > 60> is_integer(A = 5). > > true > > 61> > > 61> fun () when is_integer(A = 5) -> ok end. > > * 2: illegal guard expression > > 62> > > 62> > > 62> > > 62> fun () when is_integer(5) -> ok end. > > #Fun > > 63> > > 63> > > 63> fun (A = 5) when is_integer(A) -> ok end. > > #Fun > > 64> > > 64> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vasdeveloper@REDACTED Sat Feb 6 17:37:12 2016 From: vasdeveloper@REDACTED (Theepan) Date: Sat, 6 Feb 2016 22:07:12 +0530 Subject: [erlang-questions] Illegal Guard? In-Reply-To: <1838507.94il841jNi@changa> References: <1838507.94il841jNi@changa> Message-ID: "WHY ARE YOU ASSIGNING INSIDE A GUARD?" There was another thread regarding the issues about automatic masking of binaries during construction. I was checking to see how to convert the lvalue into a binary (so that it will get masked as well.) before matching takes place.. Sometimes I try some crazy things :) On Fri, Feb 5, 2016 at 6:39 PM, zxq9 wrote: > On 2016?2?5? ??? 18:10:33 Theepan wrote: > > Team - Any idea? > > > > 60> is_integer(A = 5). > > true > > 61> fun () when is_integer(A = 5) -> ok end. > > * 2: illegal guard expression > > 62> fun () when is_integer(5) -> ok end. > > #Fun > > 63> fun (A = 5) when is_integer(A) -> ok end. > > #Fun > > I think it has to do with binding VS masking. That is to say, what part > of a function head is not yet referring to an external scope, so you can't > yet create a closure over an existing value: > > 1> is_integer(A = 5). > true > 2> B = fun() -> A end. > #Fun > 3> C = fun(A = 6) -> A end. > #Fun > 4> D = fun(A) -> A end. > #Fun > > A is now actually assigned to 5 from our first statement -- which is > running within the context of an ongoing fun (I think). That being as > it may, anything assigned within the head of a function, apparently to > include guards, is masking whatever else existed. Its not about > is_integer/1 as much as it is about where it is used. > > 5> A. > 5 > 6> B(). > 5 > 7> C(6). > 6 > 8> D(7). > 7 > > Now let's drop the current scope's memory of A: > > 9> f(A). > ok > 10> A. > * 1: variable 'A' is unbound > 11> B(). > 5 > 12> C(7). > ** exception error: no function clause matching > erl_eval:'-inside-an-interpreted-fun-'(7) > 13> D(7). > 7 > > Now let's see what happens if we use an assignment within a guard within > a function's inner scope: > > 14> E = fun(A) when is_integer(A) -> > 14> case is_integer(Z = A) of > 14> true -> Z; > 14> false -> "strangeness" > 14> end > 14> end. > #Fun > 15> E(10). > 10 > > I don't know for sure, but this certainly leads me to believe it has > something to do with where the actual scoping boundary lies. > > In other news... another thought crossed my mind... > > WHY ARE YOU ASSIGNING INSIDE A GUARD? > > Just to deepen the mystery because needlessly exploring corner cases > that should never occur in actual code is sort of funny to me... > > 16> A = 5. > 5 > 17> F = fun(A) when is_integer(A) -> > 17> case is_integer(Z = A) of > 17> true -> Z; > 17> false -> "strangeness" > 17> end > 17> end. > #Fun > 18> F(5). > 5 > 19> F(10). > 10 > 20> A. > 5 > > Someone who really knows will get bored and give a real answer on Monday, > perhaps... ? > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mohit3081989@REDACTED Sat Feb 6 17:17:49 2016 From: mohit3081989@REDACTED (mohit Agrawal) Date: Sat, 6 Feb 2016 21:47:49 +0530 Subject: [erlang-questions] Erlang + Ejabberd In-Reply-To: References: Message-ID: Thanks. On 2 February 2016 at 21:09, Micka?l R?mond wrote: > Hello, > > On 02 Feb 2016, at 04:56, mohit Agrawal wrote: > > Hi Team, > > I am new to Erlang, I have been trying to install Erlang and Ejabberd on > EC2 ubuntu machine, everything went well till I started compiling some > external modules in ejabberd. It started throwing error "undefined parse > transform 'lager_transform'". I tried everything which is as below : > > > Please make sure you compile ejabberd as described in documentation: > http://docs.ejabberd.im/admin/guide/installation/#installing-ejabberd-from-source-code > > For example, you can compile it with: > > ./configure --enable-lager --enable-mysql > make > > You need to run configure and make to set properly the build chain. > -- > Micka?l R?mond > http://www.process-one.net > > -- Mohit Agrawal -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Sun Feb 7 03:19:36 2016 From: ok@REDACTED (ok@REDACTED) Date: Sun, 7 Feb 2016 15:19:36 +1300 Subject: [erlang-questions] Illegal Guard? In-Reply-To: References: Message-ID: > Team - Any idea? What it says: you may not use '=' in a guard. Come to think of it, > 60> is_integer(A = 5). is pretty horrible no matter where it occurs; it's one of the least likeable features of Erlang. As far as I know the reason '=' isn't allowed in guards is to avoid things like f(X, Y) when U = X+Y, U > 0 ; V = Y*X, V < 0 -> ... is it OK to use U here? or V? That problem can surely be solved, just like 'case': if a variable is bound in every alternative of a guard, it's bound, otherwise it's visible but unusable. But before something like that could be adopted, 'andalso' and 'orelse' and worse still 'and' and 'or' were allowed into guards, and things would have got even more tangled. From sargun@REDACTED Sun Feb 7 07:06:31 2016 From: sargun@REDACTED (Sargun Dhillon) Date: Sat, 6 Feb 2016 22:06:31 -0800 Subject: [erlang-questions] Trouble with ct_slave node initialization In-Reply-To: References: Message-ID: So, I think the problem stems from ct_slave's use of inet:gethostname(): https://github.com/erlang/otp/blob/maint/lib/common_test/src/ct_slave.erl#L382-L391. Even when dispatched for long names, it's effectively using inet:gethostname() -- https://github.com/erlang/otp/blob/maint/lib/kernel/src/net_adm.erl#L75-L80. That in turns calls the http://linux.die.net/man/2/gethostname, which just returns what it gets from uname. So, in theory, shortname should work as long as we start up our node like chris@$(uname -n). Then, when it tries to connect to the node, it'll connect to it as chris4@$(uname -n), and it'll try to start a local node without triggering SSH. Of course that doesn't work. Because in turn when ct_slave starts the node, the command it generates to start up erlang looks closer to this: -home /home/ubuntu -- -noshell -noinput -noshell -noinput -setcookie SCPOZKCLKFHQSBVEAKEF -sname chris4. It "calculates" that the name of the slave is going to be chris4@$(uname -n) (based upon the result of gethostname). When it starts, it calculates the hostname using inet_db:gethostname -- https://github.com/erlang/otp/blob/maint/lib/kernel/src/net_kernel.erl#L1242-L1269. This gets it from inet_config: https://github.com/erlang/otp/blob/maint/lib/kernel/src/inet_config.erl#L47-L86. Which, initially sets it via inet:gethostname(), and then a inet:gethostbyname on that name. Given this divergence, we get two different node names. The only obvious way I see to alleviate this, other than configure the utsname correctly, is ensure there is an /etc/hosts file entry for 127.0.0.1, and it should have the uname first, before localhost so it's the primary hostname, and not an alias in the gethostbyname call. Unfortunately, in many systems, this file is controlled by an outside entity so that's a no-go. I wish that common test would split node(), and get the host part, and use that to drive starting the slave at -sname chris4@$HOST. Opinions? At least add an option for it? On Fri, Jan 29, 2016 at 5:01 PM, Christopher Meiklejohn wrote: > I'm seeing the following behavior where ct_slave fails to initialize > the node within the boot_timeout, however, the node is being > registered via epmd and is pingable after initialization. > > (chris2@REDACTED)9> ct_slave:start('chris3@REDACTED'). > {error,boot_timeout,chris3@REDACTED@box545} > (chris2@REDACTED)10> net_adm:ping('chris3@REDACTED'). > pong > > (chris2@REDACTED)11> net_adm:ping('chris4'). > pang > (chris2@REDACTED)12> ct_slave:start('chris4'). > {error,boot_timeout,chris4@REDACTED} > (chris2@REDACTED)13> net_adm:ping('chris4'). > pang > (chris2@REDACTED)14> > > (chris2@REDACTED)16> net_adm:ping('chris4@REDACTED'). > pong > > The boxes hostname is box545, however Erlang is running with > distribution on, but with only shortnames enabled. > > Thanks, > Christopher > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From sargun@REDACTED Sun Feb 7 07:07:04 2016 From: sargun@REDACTED (Sargun Dhillon) Date: Sat, 6 Feb 2016 22:07:04 -0800 Subject: [erlang-questions] Erlang Time API And RFC1589? Message-ID: RFC1589 (https://tools.ietf.org/html/rfc1589) discusses a time-keeping in relation with OS kernels. This include syscalls which allow userspace programs to know if the Kernel is synced to NTP, as well as error bounds (http://man7.org/linux/man-pages/man2/adjtimex.2.html). Although this time still will not be globally accurate, it is a step to ensure that your system clock isn't either wildly out of sync, or broken. With the inclusion of the new time keeping API in Erlang 18, and the explicit os_system_time_source, was there a reason the RFC1589 time API wasn't included? Would it make sense to add a os_precise_system_time_source that queries, and exposes the information in RFC1589, or even add an RFC1589-like extension to the Erlang time API? From random.outcomes@REDACTED Sun Feb 7 14:20:34 2016 From: random.outcomes@REDACTED (Luke) Date: Mon, 8 Feb 2016 00:20:34 +1100 Subject: [erlang-questions] Architecture: How to achieve concurrency when gen_server:handle_call waits? Message-ID: Hi, The way I'm getting my head around the actor model & OTP is to imagine it's like an office building full of people with specialised jobs (gen_servers), and you send emails to people who work on your request right away and reply when they're done (for a handle_call). If I have used call then it's as if I fire off the email and sit there clicking refresh until I get a reply (or like a literal call, I am on hold until they get back to me), but if I use cast then I just get on with my day and maybe check my inbox again later. I can see a lot of benefits with this kind of organisation of your programs and like it a lot. My problem is that when a gen_server is working on something they are busy and can't answer more calls/casts. Is the correct way to achieve concurrency in Erlang to have each gen_server spawn a brand new process and then go back to checking their inbox again? In the office building metaphor, essentially each worker also has access to an infinite pool of interns and they are able to forward tasks, immediately delegating all the work away (I used to work at Ericsson, I can see how this model comes naturally to them :P) If this is indeed the correct way to achieve concurrency, I still have the following questions: 1 - Why isn't this done automatically behind the scenes in gen_server? When would you ever not want to free up your gen_server to handle more requests? 2 - Is it best practice to spawn each new process under the gen_server, a supervisor somewhere, or not all? 3 - If your gen_server is being flooded with messages, would one viable solution to achieve concurrency be creating say 10 of the same gen_server under a supervisor, and having any processes passing messages to this "job" randomly pick one, effectively using probability to reduce traffic to each by 1/10 - is there a library/methodology for doing this kind of thing already? 4 - This seems like it would be a common piece of code, is this bundled into OTP somewhere? Is this situation what I'm supposed to use gen_event for? Or if I'm completely wrong, what is the actual way programs like yaws achieve high concurrency, as reading the source code has not revealed the answer to me. Thanks, Jed -------------- next part -------------- An HTML attachment was scrubbed... URL: From askjuise@REDACTED Sun Feb 7 14:31:43 2016 From: askjuise@REDACTED (Alexander Petrovsky) Date: Sun, 7 Feb 2016 16:31:43 +0300 Subject: [erlang-questions] Right way to terminate process chain Message-ID: Hi! By example I have the follow chain: x_sup -> x:start_link -> {ok, YPid} = y:start_link(self()) {ok, ZPid} = z:start_link(self()) So, X knows only YPid, Y knows both XPid and ZPid, Z knows only YPid. All processes are gen_servers. How to terminate the whole chain of the processes when any process can terminate with 'normal' reason? I have only two ideas: - trap_exit, catch EXIT message and terminate - call gen_server(XPid|YPid|ZPid) from terminate callback in every process, but it causes 'little temporary' deadlock. -- ?????????? ????????? / Alexander Petrovsky, Skype: askjuise Phone: +7 914 8 820 815 -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Sun Feb 7 14:37:50 2016 From: sergej.jurecko@REDACTED (=?utf-8?Q?Sergej_Jure=C4=8Dko?=) Date: Sun, 7 Feb 2016 14:37:50 +0100 Subject: [erlang-questions] Architecture: How to achieve concurrency when gen_server:handle_call waits? In-Reply-To: References: Message-ID: <80E1CE80-C310-4ED7-8422-9C0FA6180E5B@gmail.com> You're thinking in traditional non-Erlang ideas of concurrency. Have X processes handling Y clients where X << Y. The Erlang way is to have X processes handling X clients. Each client gets its own process. This is how cowboy/yaws achieve concurrency. Of course not every problem falls into such an architecture. Then you deal with it as the situation demands. Also using gen_server:call for everything is not a good design choice. it should only be used when it's actually needed. Very often gen_server:cast is the right choice. Sergej > On 07 Feb 2016, at 14:20, Luke wrote: > > Hi, > > The way I'm getting my head around the actor model & OTP is to imagine it's like an office building full of people with specialised jobs (gen_servers), and you send emails to people who work on your request right away and reply when they're done (for a handle_call). If I have used call then it's as if I fire off the email and sit there clicking refresh until I get a reply (or like a literal call, I am on hold until they get back to me), but if I use cast then I just get on with my day and maybe check my inbox again later. I can see a lot of benefits with this kind of organisation of your programs and like it a lot. > > My problem is that when a gen_server is working on something they are busy and can't answer more calls/casts. Is the correct way to achieve concurrency in Erlang to have each gen_server spawn a brand new process and then go back to checking their inbox again? In the office building metaphor, essentially each worker also has access to an infinite pool of interns and they are able to forward tasks, immediately delegating all the work away (I used to work at Ericsson, I can see how this model comes naturally to them :P) > > If this is indeed the correct way to achieve concurrency, I still have the following questions: > 1 - Why isn't this done automatically behind the scenes in gen_server? When would you ever not want to free up your gen_server to handle more requests? > 2 - Is it best practice to spawn each new process under the gen_server, a supervisor somewhere, or not all? > 3 - If your gen_server is being flooded with messages, would one viable solution to achieve concurrency be creating say 10 of the same gen_server under a supervisor, and having any processes passing messages to this "job" randomly pick one, effectively using probability to reduce traffic to each by 1/10 - is there a library/methodology for doing this kind of thing already? > 4 - This seems like it would be a common piece of code, is this bundled into OTP somewhere? Is this situation what I'm supposed to use gen_event for? Or if I'm completely wrong, what is the actual way programs like yaws achieve high concurrency, as reading the source code has not revealed the answer to me. > > Thanks, > Jed > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From zxq9@REDACTED Sun Feb 7 14:49:05 2016 From: zxq9@REDACTED (zxq9) Date: Sun, 07 Feb 2016 22:49:05 +0900 Subject: [erlang-questions] Architecture: How to achieve concurrency when gen_server:handle_call waits? In-Reply-To: References: Message-ID: <1883763.FQxtlYrhkX@changa> Hi Luke. On 2016?2?8? ??? 00:20:34 Luke wrote: > Hi, > > The way I'm getting my head around the actor model & OTP is to imagine it's > like an office building full of people with specialised jobs (gen_servers), > and you send emails to people who work on your request right away and reply > when they're done (for a handle_call). If I have used call then it's as if > I fire off the email and sit there clicking refresh until I get a reply > (or like a literal call, I am on hold until they get back to me), but if I > use cast then I just get on with my day and maybe check my inbox again > later. I can see a lot of benefits with this kind of organisation of your > programs and like it a lot. > > My problem is that when a gen_server is working on something they are busy > and can't answer more calls/casts. Is the correct way to achieve > concurrency in Erlang to have each gen_server spawn a brand new process and > then go back to checking their inbox again? In the office building > metaphor, essentially each worker also has access to an infinite pool of > interns and they are able to forward tasks, immediately delegating all the > work away (I used to work at Ericsson, I can see how this model comes > naturally to them :P) There is nothing inherently wrong with this way of doing things, but consider two things: - Most of the time an Erlang process is less about doing work (else why is the caller not just spawning his own process?) and more about managing state. It is the *context* of the call that is significant more than the algorithm underlying whatever was done. - How must the reply now be achieved? Either the worker must be given the full reply reference (which is a bit more complex to deal with, but absolutely workable) or the original gen_server must receive *another* message back from its worker and forward it to the caller. This turns the gen_server into more of a dispatch server than a state server. Again, there is nothing at all wrong with this model, its just that very often this approach is more awkward than it is worth. There *are* some cases you very occasionally run in to where this approach is a very big win -- but its not so common that OTP abstracts it out for you, and often when you do need this the situation is special enough that a generic solution may not fit (but I could imagine a generalization of the idea working well all the same: the existing implementation is called spawn_link). > If this is indeed the correct way to achieve concurrency, I still have the > following questions: First off -- you're *already* achieving concurrency. The bigger issue is that you're stuck thinking in terms of synchronous calls. Don't do that. Try seeing if you can write a program that deals only in casts. Sometimes you can, sometimes you can't -- but you'll develop a sense for where casts are sufficient and calls are actually necessary. > 1 - Why isn't this done automatically behind the scenes in gen_server? When > would you ever not want to free up your gen_server to handle more requests? Usually you don't run into situations where a single gen_server's response time dictates your entire system's latency. If you have this situation you have just invented a bottleneck. That's silly. Don't do that. Don't make your bottlenecks performant, just don't have bottlenecks in the first place. It is surprising how little centralization you can achieve in most systems if you think carefully about it (or get used to the idea). > 2 - Is it best practice to spawn each new process under the gen_server, a > supervisor somewhere, or not all? It is usually a good thing to spawn them always under a supervisor. But there are a variety of approaches. Generally I prefer to have a supervisor that is simple_one_for_one per one-off task other processes in the system will want to spawn for parallel computation. But this also constrains your thinking considerably (which is also usually a good thing). Sometimes you do need the flexibility of a loose process -- but whenever you do this spawn_link it, don't just spawn it -- that way if it does its entire branch of the supervision tree dies with it and is restarted (or not) in an understandable way. You get surprising and weird things happening if you start spawning detached processes. As a guideline: Always spawn under a supervisor As a hard rule: Always ALWAYS spawn linked to the supervision tree. > 3 - If your gen_server is being flooded with messages, would one viable > solution to achieve concurrency be creating say 10 of the same gen_server > under a supervisor, and having any processes passing messages to this "job" > randomly pick one, effectively using probability to reduce traffic to each > by 1/10 - is there a library/methodology for doing this kind of thing > already? This is pooling, and yes, there are tools for it. It is sort of silly to ever have this problem, though (imo). Whenever I want pooling what I really want is a bunch of stateless workers I can ride into the ground, and the only reason I ever want to have a fixed number of them is to control resource usage. It is also possible to just spawn processes as needed, though, and let them die whenever they are done doing what they are doing. gen_server are *slightly* more overhead to spawn than pure Erlang ones (for some definition of slightly...). In brief: there are dogmatic answers either way, and neither is a clean fit for many cases you'll run into in reality. The only real answer to pooling VS spawning is: "it depends". > 4 - This seems like it would be a common piece of code, is this bundled > into OTP somewhere? Is this situation what I'm supposed to use gen_event > for? Or if I'm completely wrong, what is the actual way programs like yaws > achieve high concurrency, as reading the source code has not revealed the > answer to me. For client/connection driven services the typical way is not to have a connection pool, but to spawn a connection per client. A way of dealing with this I use in messaging systems, business software and games where a single user may concurrently log in from several clients is to have a process spawned to handle each network connection, but a separate client "controller" spawned to handle each *session* (which itself may have several connections associated with it). There isn't anything in OTP that abstracts that away. I'd love to write a library for this -- but no money and no time. Poof! Anywway, writing this isn't hard, just sort of annoying to do more than once. -Craig From roe.adrian@REDACTED Sun Feb 7 15:12:51 2016 From: roe.adrian@REDACTED (Adrian Roe) Date: Sun, 7 Feb 2016 14:12:51 +0000 Subject: [erlang-questions] Architecture: How to achieve concurrency when gen_server:handle_call waits? In-Reply-To: References: Message-ID: Luke I quite like your analogy. I think the core issue is what is it that the gen_server is for. The scenario you describe is roughly along the lines of "gen_servers seem to be a bottleneck in the way of lots of parallel calls? and that is precisely correct. You would normally only use a gen_server when that bottleneck is a good thing - when there is some sort of state in the gen_server itself that is mutated by the calls to the gen_server. If all you need are worker threads that operate completely independently then the gen_server is an unhelpful bottleneck and your code could just reside in a simple library module (i.e. just a plain old erlang module - not one adopting a gen_xxx behaviour). This library code can just do the work (typically a good choice if the work is quick, or if the callee actively wants to block until the work is done) or it can spawn a process, do the work in that process and send a message back to the caller when it is done. So, to build on your question 3) - here each call to the API would spawn its own process that would live for the duration of the work and then expire. There are common techniques (which I see mentioned in other replies coming in as I type) to throttle the total number of processes spawned if you might have very large numbers of concurrent calls and need to limit them / provide back-pressure. That said, if you are new to erlang I would err on the side of creating processes freely - if a piece of code isn?t ?just? some simple maths or state manipulation there?s a good chance a new process isn?t a bad option :) That way you have small, easily testable elements to work with. If as a side-effect you have lots of protocols (message flows) between the processes that do not in some way reflect your business rules (do this thing and also that thing at the same time, or do lots of thing X all at the same time), then you?ve probably got too many and they are likely to make the code harder to understand again, so think about whether you?d be better off merging once more. For example if you need to add X & Y and also A & B, just do them one after the other - the complexity caused by splitting the work and then merging the answers is very unlikely to be worth it (ignoring the fact that creating process, while cheap computationally is not quite free!). Of course, the other reason to have code in gen_xxx modules is so that they can easily be supervised and restarted if something goes wrong - probably not the case in the sort of scenario you described. Getting the balance right between ?library code?, ?library code that spawns processes? and "gen_xxx processes? is part of the journey to being an effective erlang programmer. You?ve come up with a really good question, so I?d say you are on the journey already :) Adrian > On 7 Feb 2016, at 13:20, Luke wrote: > > Hi, > > The way I'm getting my head around the actor model & OTP is to imagine it's like an office building full of people with specialised jobs (gen_servers), and you send emails to people who work on your request right away and reply when they're done (for a handle_call). If I have used call then it's as if I fire off the email and sit there clicking refresh until I get a reply (or like a literal call, I am on hold until they get back to me), but if I use cast then I just get on with my day and maybe check my inbox again later. I can see a lot of benefits with this kind of organisation of your programs and like it a lot. > > My problem is that when a gen_server is working on something they are busy and can't answer more calls/casts. Is the correct way to achieve concurrency in Erlang to have each gen_server spawn a brand new process and then go back to checking their inbox again? In the office building metaphor, essentially each worker also has access to an infinite pool of interns and they are able to forward tasks, immediately delegating all the work away (I used to work at Ericsson, I can see how this model comes naturally to them :P) > > If this is indeed the correct way to achieve concurrency, I still have the following questions: > 1 - Why isn't this done automatically behind the scenes in gen_server? When would you ever not want to free up your gen_server to handle more requests? > 2 - Is it best practice to spawn each new process under the gen_server, a supervisor somewhere, or not all? > 3 - If your gen_server is being flooded with messages, would one viable solution to achieve concurrency be creating say 10 of the same gen_server under a supervisor, and having any processes passing messages to this "job" randomly pick one, effectively using probability to reduce traffic to each by 1/10 - is there a library/methodology for doing this kind of thing already? > 4 - This seems like it would be a common piece of code, is this bundled into OTP somewhere? Is this situation what I'm supposed to use gen_event for? Or if I'm completely wrong, what is the actual way programs like yaws achieve high concurrency, as reading the source code has not revealed the answer to me. > > Thanks, > Jed > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Sun Feb 7 15:36:48 2016 From: erlang@REDACTED (Joe Armstrong) Date: Sun, 7 Feb 2016 15:36:48 +0100 Subject: [erlang-questions] Illegal Guard? In-Reply-To: References: Message-ID: Why use a guard at all? 1> F = fun(5)->five; (_)->not_five end. #Fun 2> F(5). five 3> F(6). not_five Cheers /Joe On Fri, Feb 5, 2016 at 1:40 PM, Theepan wrote: > Team - Any idea? > > > 60> > > 60> > > 60> is_integer(A = 5). > > true > > 61> > > 61> fun () when is_integer(A = 5) -> ok end. > > * 2: illegal guard expression > > 62> > > 62> > > 62> > > 62> fun () when is_integer(5) -> ok end. > > #Fun > > 63> > > 63> > > 63> fun (A = 5) when is_integer(A) -> ok end. > > #Fun > > 64> > > 64> > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From jesper.louis.andersen@REDACTED Sun Feb 7 15:52:44 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 7 Feb 2016 15:52:44 +0100 Subject: [erlang-questions] Right way to terminate process chain In-Reply-To: References: Message-ID: On Sun, Feb 7, 2016 at 2:31 PM, Alexander Petrovsky wrote: > I have only two ideas: > - trap_exit, catch EXIT message and terminate > - call gen_server(XPid|YPid|ZPid) from terminate callback in every > process, but it causes 'little temporary' deadlock. > When you have a simple way to close down the process chain, you can often just terminate it gracefully and message the others you are done. If you terminate abnormally, the links will make sure to reap the process tree. When you cannot close down the process chain in this way easily though, there are monitors. Call erlang:monitor(process, Pid) and handle the 'DOWN' message which arrives in your inbox. Monitors are unidirectional, so you will have to monitor those processes you need to watch out for individually. But this is the common way to handle termination by means of an async exception. Also, note the semantics of monitors if the Pid you monitor isn't there. And that you can call erlang:demonitor(MonitorRef, [flush]) to linearize monitor messages in your inbox. All in all the API is nice to work with in a safe way with a bit of attention. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sun Feb 7 16:14:03 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 7 Feb 2016 16:14:03 +0100 Subject: [erlang-questions] Architecture: How to achieve concurrency when gen_server:handle_call waits? In-Reply-To: References: Message-ID: On Sun, Feb 7, 2016 at 2:20 PM, Luke wrote: > 1 - Why isn't this done automatically behind the scenes in gen_server? > When would you ever not want to free up your gen_server to handle more > requests? > >From a perspective of increasing concurrency (and thus parallelism) this model is alluring. But the strength of processing one message at a time in order is that handling messages linearize. You are sure handling one message is either going to succeed--you reach the next state invariant and is ready for the next message--or fail. Since the gen_server is the sole owner of the data it processes, there is no room for data races. Many parts of a larger system benefit a lot from this simple model and it scales up to a point. > 2 - Is it best practice to spawn each new process under the gen_server, a > supervisor somewhere, or not all? > This .. depends. Usually the question is "how do you want your newly spawned processes and the process tree to behave when something goes wrong?". Running under the gen_server usually means that one failure means failure of all. Running in a simple-one-for-one supervisor as a sibling to the gen_server lets you handle individual errors more gracefully (also set monitors on the workers in this case). > 3 - If your gen_server is being flooded with messages, would one viable > solution to achieve concurrency be creating say 10 of the same gen_server > under a supervisor, and having any processes passing messages to this "job" > randomly pick one, effectively using probability to reduce traffic to each > by 1/10 - is there a library/methodology for doing this kind of thing > already? > Yes. There are multiple solutions and they have different failure semantics and different semantics when loaded with more requests than they can handle, or with jobs of varying latency. Can the library batch requests which are equivalent? There is usually no good one-size-fits-all here. And this is the main questions you should ask of any such library. 4 - This seems like it would be a common piece of code, is this bundled > into OTP somewhere? Is this situation what I'm supposed to use gen_event > for? Or if I'm completely wrong, what is the actual way programs like yaws > achieve high concurrency, as reading the source code has not revealed the > answer to me. > If you read the above remarks, you understand why it is hard to come up with a good design solution everyone would be happy with. You get the tools to build such a solution, but no solution. gen_event is actually two kinds of modules: an event manager which is an endpoint for publishing events, and event handlers, which runs as part of the event managers context subscribing to events as they come in and doing work on the event. The model is very suitable for loosely coupling events from one application into another as you can dynamically install and remove such handlers. One application contains the event manager and the other application installs a handler module so it can get notified when such events occur. The built-in alarm_handler in Erlang is a good example of a gen_event system: alarms in the system can be set and cleared, and you can then subscribe to alarms in order to react to those. An application could subscribe to the alarm that you lost database connectivity and change its behavior accordingly. Once the alarm clears, you can resume normal operation. Another good alarm is "some process is using more than 5% of available memory". And so on. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrtndimitrov@REDACTED Sun Feb 7 18:05:47 2016 From: mrtndimitrov@REDACTED (Martin Koroudjiev) Date: Sun, 7 Feb 2016 19:05:47 +0200 Subject: [erlang-questions] Setting in lager the log level during runtime Message-ID: <56B7796B.80902@gmail.com> Hello, I have to trace a bug on a production server running lager. I connected to the node and tried to change the log level to debug but got a strange error - bad_module. I've checked and lager is running. Here is a screenshot of the commands: Thanks in advance, Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: gaijedid.png Type: image/png Size: 10366 bytes Desc: not available URL: From darach@REDACTED Sun Feb 7 18:21:14 2016 From: darach@REDACTED (Darach Ennis) Date: Sun, 7 Feb 2016 17:21:14 +0000 Subject: [erlang-questions] [ANN] jch-erl 0.2.0 Message-ID: Hi folk. jch-erl is a NIF wrapper for the Jump Consistent Hash algorithm by John Lamping and Eric Veach developed at Google, Inc. Paper: "A Fast, Minimal Memory, Consistent Hash Algorithm". Thanks to contributions by Sergey Prokhorov &co this has now been updated to include: * For interop with other implementations, now becoming common, the original LCG from the paper. * Alternately, xorshift64* based PRNG * A fix for 32-bit platforms. * Support for release 18. Benchmark on R18 here in the travis CI logs: https://travis-ci.org/darach/jch-erl/jobs/107607039 Source there: https://github.com/darach/jch-erl Cheers, Darach. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Sun Feb 7 19:38:01 2016 From: lloyd@REDACTED (lloyd@REDACTED) Date: Sun, 7 Feb 2016 13:38:01 -0500 (EST) Subject: [erlang-questions] How best to return db query Message-ID: <1454870281.371221019@apps.rackspace.com> Hello, I have this mnesia query: get_note(ID) -> Query = fun() -> mnesia:read({?TABLE, ID}) end, {atomic, Results} = mnesia:transaction(Query), Results. It returns either a list containing one or more valid records or an empty list. Would it be better to return something like: {ok, Records} | {error, data_not_found} Thanks, LRP From sergej.jurecko@REDACTED Sun Feb 7 19:45:39 2016 From: sergej.jurecko@REDACTED (=?UTF-8?Q?Sergej_Jure=C4=8Dko?=) Date: Sun, 7 Feb 2016 19:45:39 +0100 Subject: [erlang-questions] How best to return db query In-Reply-To: <1454870281.371221019@apps.rackspace.com> References: <1454870281.371221019@apps.rackspace.com> Message-ID: An empty result to a db query is not an error. Sergej On Feb 7, 2016 7:38 PM, wrote: > Hello, > > I have this mnesia query: > > > get_note(ID) -> > Query = > fun() -> > mnesia:read({?TABLE, ID}) > end, > {atomic, Results} = mnesia:transaction(Query), > Results. > > It returns either a list containing one or more valid records or an empty > list. > > Would it be better to return something like: > > {ok, Records} | {error, data_not_found} > > Thanks, > > LRP > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vimal7370@REDACTED Sun Feb 7 19:52:49 2016 From: vimal7370@REDACTED (Vimal Kumar) Date: Mon, 8 Feb 2016 00:22:49 +0530 Subject: [erlang-questions] Mnesia schema replicas Message-ID: Hello list, Ref (Section 6.6): http://erlang.org/doc/apps/mnesia/Mnesia_chap5.html -QUOTE- The functions mnesia:add_table_copy/3 and mnesia:del_table_copy/2 can be used to add and delete replicas of the schema table. Adding a node to the list of nodes where the schema is replicated affects the following: It allows other tables to be replicated to this node. It causes Mnesia to try to contact the node at startup of disc-full nodes. -END QUOTE- I can use the functions mnesia:add_table_copy/3 and mnesia:del_table_copy/2 to add and delete replicated copies of other tables successfully, but not table 'schema'. In fact I am not sure at what point in initialization process of a new mnesia node should this be run. Please assist. ------ $ erl -mnesia -sname slave Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] Eshell V7.0 (abort with ^G) (slave@REDACTED)1> mnesia:start(). ok (slave@REDACTED)3> mnesia:change_config(extra_db_nodes, ['server@REDACTED']). {ok,['server@REDACTED']} (slave@REDACTED)5> mnesia:add_table_copy(schema, node(), disc_copies). {aborted,{already_exists,schema,'slave@REDACTED'}} ------ Regards, Vimal -------------- next part -------------- An HTML attachment was scrubbed... URL: From random.outcomes@REDACTED Sun Feb 7 17:33:08 2016 From: random.outcomes@REDACTED (Luke) Date: Mon, 8 Feb 2016 03:33:08 +1100 Subject: [erlang-questions] Architecture: How to achieve concurrency when gen_server:handle_call waits? In-Reply-To: References: Message-ID: Hi All, Firstly thank you for your answers, some of my colleagues have criticized Erlang for not having a large developer community behind it, but I would counter and say that a smaller community can be even more useful when experts are contactable directly :) the more I use Erlang the more I absolutely love it! I believe I have derived the solution the problem I was having and it's all starting to 'click'. I am a very visual thinker so I have created diagrams (and please excuse my crappy paint skills) outlining my first solution, which feels incredibly poor to me now hahaha. I have also sketched out what I believe is the correct implementation but I am very appreciative of any and all feedback. This is the architecture which I am trying to implement - I included some text to describe it but you probably get the idea: http://postimg.org/image/yz9gnpzgd/ [image: Inline image 1] This was the first way I thought of implementing it, and how I thought concurrency worked in Erlang. I feel a bit stupid looking at it now hahaha. I should have drawn this neater (especially step 2, follow the light blue line to the process just down and to the right, not the step 6 process which is spawned a long way 'down') - my apologies - just follow the numbers/colours and try to relate it back to the above architecture. "Message ! Broker" is of course an Erlang message pass, I was using spawn_link to delegate work (the "interns"), the black circles were the gen_servers I was creating to offer the functionality of each block, and I was spinning off new workers to try and deal with the obvious bottleneck of one server trying to handle all the messages. http://postimg.org/image/bnpzzcx39/ [image: Inline image 2] This is the new implementation I have devised, as I said I welcome all feedback on it as well http://s21.postimg.org/byeovx07b/good_concurrency.png [image: Inline image 3] Not only does this look a lot simpler, But each request is now being contained within the main process (I will still most likely use a gen_server), and each new request is handled simply be creating a new process. On the error handling side of the design, if say the process in Block1 crashes then the process in the main big block will eventually be able to catch an exception caused by the timeout of not getting a reply, and perhaps try again, or that supervisor could restart it I guess, what do you guys think? Also different behaviours can be captured by simply coding different implementations of gen_server and calling the correct one in the first spawn_link/3, for example if we use the insurance example above I would create a 'change customer address' server after the message is interpreted in the yaws block, but 'submit claim' would be a different module that still implements the gen_server behaviour. The only part this diagram doesn't show is how the YAWS block handles requests, but I guess that the answer is just to spawn a new process in that block similar to blocks 1 and 2, and that process will eventually receive a message back from the process in the main big block. This description is probably not sounding very clear hahaha, let me know if it needs clarification. Thanks again :) On Mon, Feb 8, 2016 at 2:14 AM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > On Sun, Feb 7, 2016 at 2:20 PM, Luke wrote: > >> 1 - Why isn't this done automatically behind the scenes in gen_server? >> When would you ever not want to free up your gen_server to handle more >> requests? >> > > From a perspective of increasing concurrency (and thus parallelism) this > model is alluring. But the strength of processing one message at a time in > order is that handling messages linearize. You are sure handling one > message is either going to succeed--you reach the next state invariant and > is ready for the next message--or fail. Since the gen_server is the sole > owner of the data it processes, there is no room for data races. Many parts > of a larger system benefit a lot from this simple model and it scales up to > a point. > > >> 2 - Is it best practice to spawn each new process under the gen_server, a >> supervisor somewhere, or not all? >> > > This .. depends. Usually the question is "how do you want your newly > spawned processes and the process tree to behave when something goes > wrong?". Running under the gen_server usually means that one failure means > failure of all. Running in a simple-one-for-one supervisor as a sibling to > the gen_server lets you handle individual errors more gracefully (also set > monitors on the workers in this case). > > >> 3 - If your gen_server is being flooded with messages, would one viable >> solution to achieve concurrency be creating say 10 of the same gen_server >> under a supervisor, and having any processes passing messages to this "job" >> randomly pick one, effectively using probability to reduce traffic to each >> by 1/10 - is there a library/methodology for doing this kind of thing >> already? >> > > Yes. There are multiple solutions and they have different failure > semantics and different semantics when loaded with more requests than they > can handle, or with jobs of varying latency. Can the library batch requests > which are equivalent? There is usually no good one-size-fits-all here. And > this is the main questions you should ask of any such library. > > 4 - This seems like it would be a common piece of code, is this bundled >> into OTP somewhere? Is this situation what I'm supposed to use gen_event >> for? Or if I'm completely wrong, what is the actual way programs like yaws >> achieve high concurrency, as reading the source code has not revealed the >> answer to me. >> > > If you read the above remarks, you understand why it is hard to come up > with a good design solution everyone would be happy with. You get the tools > to build such a solution, but no solution. > > gen_event is actually two kinds of modules: an event manager which is an > endpoint for publishing events, and event handlers, which runs as part of > the event managers context subscribing to events as they come in and doing > work on the event. The model is very suitable for loosely coupling events > from one application into another as you can dynamically install and remove > such handlers. One application contains the event manager and the other > application installs a handler module so it can get notified when such > events occur. > > The built-in alarm_handler in Erlang is a good example of a gen_event > system: alarms in the system can be set and cleared, and you can then > subscribe to alarms in order to react to those. An application could > subscribe to the alarm that you lost database connectivity and change its > behavior accordingly. Once the alarm clears, you can resume normal > operation. Another good alarm is "some process is using more than 5% of > available memory". And so on. > > > > > > -- > J. > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 40981 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 57765 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 250723 bytes Desc: not available URL: From sergej.jurecko@REDACTED Mon Feb 8 03:21:57 2016 From: sergej.jurecko@REDACTED (=?UTF-8?Q?Sergej_Jure=C4=8Dko?=) Date: Mon, 8 Feb 2016 03:21:57 +0100 Subject: [erlang-questions] How best to return db query In-Reply-To: References: <1454870281.371221019@apps.rackspace.com> Message-ID: I disagree. That kind of logic does not belong in a db interface function. Sergej On Feb 8, 2016 1:25 AM, "Bernard Duggan" wrote: > >An empty result to a db query is not an error. > That rather depends on the semantics of your program. In the general case, > you're absolutely right, of course - that's why mnesia:read() doesn't > return an error in that case. However, perhaps in the case of Lloyd's > application, there should always be something in that particular table - > maybe something populated at startup that nothing should ever delete. In > that case it would be perfectly reasonable to not just produce an error, > but in fact to crash. > So I guess what I'm saying is that my answer would be "it depends on what > you want the semantics of get_note to be". Is it expected to always return > one or more values? Is it an error if it doesn't, and if so, is that error > sensibly recoverable by the caller? > > Cheers, > > B > > On Mon, Feb 8, 2016 at 5:45 AM, Sergej Jure?ko > wrote: > >> An empty result to a db query is not an error. >> >> Sergej >> On Feb 7, 2016 7:38 PM, wrote: >> >>> Hello, >>> >>> I have this mnesia query: >>> >>> >>> get_note(ID) -> >>> Query = >>> fun() -> >>> mnesia:read({?TABLE, ID}) >>> end, >>> {atomic, Results} = mnesia:transaction(Query), >>> Results. >>> >>> It returns either a list containing one or more valid records or an >>> empty list. >>> >>> Would it be better to return something like: >>> >>> {ok, Records} | {error, data_not_found} >>> >>> Thanks, >>> >>> LRP >>> >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bernard@REDACTED Mon Feb 8 01:15:28 2016 From: bernard@REDACTED (Bernard Duggan) Date: Mon, 8 Feb 2016 11:15:28 +1100 Subject: [erlang-questions] UDP active socket timeout In-Reply-To: References: Message-ID: Let's see here: > - to detect if a client is connected from what ports; Well yes. UDP doesn't have connections, as Sean noted. So how would you even define this, nevermind detect it? > gen_udp:send/4 always returns ok no matter what if port and/or ip is wrong What do you mean "wrong"? Do you mean "not the place I wanted to send it"? Remember, there's no "connection", so there's no particular place packets "have" to go to be "correct" (at least not in any sense gen_udp could be expected to determine). > gen_udp:recv is blocking (and with Timeout parameter even worse) and thus a design based on passive sockets leads to serialized processing of client requests; gen_udp provides an "active" mode which means you don't have to call recv, but rather receive each packet as a message to your process. That should do waht you want. But it looks like you know that: > if controlling process is used to handle other messages (from child procs) other then {udp, ...} message it would be unpractical from parallelization point of view of client request processing Why? Processes are quite capable of handling messages from multiple sources. If sheer volume of messages is your concern, I'd suggest benchmarking your system before getting too hung up on receiving all the messages in one process. If your other messages result in slow or blocking behaviour then you need to reconsider your design. Cheers, B On Fri, Feb 5, 2016 at 7:16 PM, Bogdan Andu wrote: > this is easy on client side but on the server side there must be an > interaction > with spawned processes that actually use that socket to send data. > > other thing I found hard to achieve in UDP is: > - to detect if a client is connected from what ports; > - sending detection errors from server to client : > gen_udp:send/4 always returns ok no matter what if port and/or ip is > wrong. > otherwise it would have been easy to implement something pinging the > client with an empty udp packet > and (possibly, again) to receive response; > - gen_udp:recv is blocking (and with Timeout parameter even worse) and > thus a design based on passive sockets leads to serialized processing > of client requests; > - if controlling process is used to handle other messages (from child > procs) other then {udp, ...} message it would be > unpractical from parallelization point of view of client request > processing > > On Thu, Feb 4, 2016 at 4:43 PM, Sean Cribbs wrote: > >> UDP is a connection-less datagram protocol, so there's no timeout >> inherent in the protocol. However, you may have timeouts on: >> >> 1) receiving a datagram (nothing arrived) >> 2) sending a datagram (things went very badly) >> >> However, those are timeouts internal to Erlang, from your process to the >> socket driver. If you are using the {active, true} option (which it sounds >> like you are), you should use an 'after' clause in your 'receive' to >> implement timeouts, or use the trailing timeout in the return tuple of a >> gen_server/gen_fsm callback. >> >> On Thu, Feb 4, 2016 at 6:29 AM, Bogdan Andu wrote: >> >>> Hi, >>> >>> How can be detected a timeout on an active UDP socket? >>> >>> controlling process receives the message: >>> {udp, Socket, Host, Port, Bin} >>> >>> and after a process is spawned to handle that data: >>> spawn(fun() -> handle_pckt(Socket, Host, Port, Bin) end), >>> >>> Q: how the spawned or controlling process is able to detect >>> that a timeout on this UDP socket occurred ? >>> >>> documentations does not mention anything about timeout on active sockets >>> >>> Thanks, >>> >>> /Bogdan >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bernard@REDACTED Mon Feb 8 01:25:55 2016 From: bernard@REDACTED (Bernard Duggan) Date: Mon, 8 Feb 2016 11:25:55 +1100 Subject: [erlang-questions] How best to return db query In-Reply-To: References: <1454870281.371221019@apps.rackspace.com> Message-ID: >An empty result to a db query is not an error. That rather depends on the semantics of your program. In the general case, you're absolutely right, of course - that's why mnesia:read() doesn't return an error in that case. However, perhaps in the case of Lloyd's application, there should always be something in that particular table - maybe something populated at startup that nothing should ever delete. In that case it would be perfectly reasonable to not just produce an error, but in fact to crash. So I guess what I'm saying is that my answer would be "it depends on what you want the semantics of get_note to be". Is it expected to always return one or more values? Is it an error if it doesn't, and if so, is that error sensibly recoverable by the caller? Cheers, B On Mon, Feb 8, 2016 at 5:45 AM, Sergej Jure?ko wrote: > An empty result to a db query is not an error. > > Sergej > On Feb 7, 2016 7:38 PM, wrote: > >> Hello, >> >> I have this mnesia query: >> >> >> get_note(ID) -> >> Query = >> fun() -> >> mnesia:read({?TABLE, ID}) >> end, >> {atomic, Results} = mnesia:transaction(Query), >> Results. >> >> It returns either a list containing one or more valid records or an empty >> list. >> >> Would it be better to return something like: >> >> {ok, Records} | {error, data_not_found} >> >> Thanks, >> >> LRP >> >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bernard@REDACTED Mon Feb 8 03:32:31 2016 From: bernard@REDACTED (Bernard Duggan) Date: Mon, 8 Feb 2016 13:32:31 +1100 Subject: [erlang-questions] How best to return db query In-Reply-To: References: <1454870281.371221019@apps.rackspace.com> Message-ID: Sure, but who's to say get_note is intended as "just" a db interface function? That was my point in saying that it depends on the intended semantics of that function.The fact that he was considering giving the empty case an 'error' tag suggests that perhaps there's more to it. But there's not enough information in the original question to make a judgement either way. B On Mon, Feb 8, 2016 at 1:21 PM, Sergej Jure?ko wrote: > I disagree. That kind of logic does not belong in a db interface function. > > Sergej > On Feb 8, 2016 1:25 AM, "Bernard Duggan" wrote: > >> >An empty result to a db query is not an error. >> That rather depends on the semantics of your program. In the general >> case, you're absolutely right, of course - that's why mnesia:read() doesn't >> return an error in that case. However, perhaps in the case of Lloyd's >> application, there should always be something in that particular table - >> maybe something populated at startup that nothing should ever delete. In >> that case it would be perfectly reasonable to not just produce an error, >> but in fact to crash. >> So I guess what I'm saying is that my answer would be "it depends on what >> you want the semantics of get_note to be". Is it expected to always return >> one or more values? Is it an error if it doesn't, and if so, is that error >> sensibly recoverable by the caller? >> >> Cheers, >> >> B >> >> On Mon, Feb 8, 2016 at 5:45 AM, Sergej Jure?ko >> wrote: >> >>> An empty result to a db query is not an error. >>> >>> Sergej >>> On Feb 7, 2016 7:38 PM, wrote: >>> >>>> Hello, >>>> >>>> I have this mnesia query: >>>> >>>> >>>> get_note(ID) -> >>>> Query = >>>> fun() -> >>>> mnesia:read({?TABLE, ID}) >>>> end, >>>> {atomic, Results} = mnesia:transaction(Query), >>>> Results. >>>> >>>> It returns either a list containing one or more valid records or an >>>> empty list. >>>> >>>> Would it be better to return something like: >>>> >>>> {ok, Records} | {error, data_not_found} >>>> >>>> Thanks, >>>> >>>> LRP >>>> >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From dgud@REDACTED Mon Feb 8 09:41:51 2016 From: dgud@REDACTED (Dan Gudmundsson) Date: Mon, 08 Feb 2016 08:41:51 +0000 Subject: [erlang-questions] Mnesia schema replicas In-Reply-To: References: Message-ID: When you start mnesia without calling create_schema, mnesia will create an empty ram_copies schema for you. change_config(extra_db_nodes, ..) will connect and merge schema with the server. What you probably want is to call mnesia:change_table_copy_type(schema, disc_copies, node)) after change_config. /Dan On Sun, Feb 7, 2016 at 7:52 PM Vimal Kumar wrote: > Hello list, > > Ref (Section 6.6): http://erlang.org/doc/apps/mnesia/Mnesia_chap5.html > > -QUOTE- > The functions mnesia:add_table_copy/3 and mnesia:del_table_copy/2 can be > used to add and delete replicas of the schema table. Adding a node to the > list of nodes where the schema is replicated affects the following: > > It allows other tables to be replicated to this node. > It causes Mnesia to try to contact the node at startup of disc-full nodes. > -END QUOTE- > > I can use the functions mnesia:add_table_copy/3 and > mnesia:del_table_copy/2 to add and delete replicated copies of other tables > successfully, but not table 'schema'. In fact I am not sure at what point > in initialization process of a new mnesia node should this be run. Please > assist. > > ------ > $ erl -mnesia -sname slave > Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:10] > [hipe] [kernel-poll:false] > > Eshell V7.0 (abort with ^G) > (slave@REDACTED)1> mnesia:start(). > ok > > (slave@REDACTED)3> mnesia:change_config(extra_db_nodes, > ['server@REDACTED']). > {ok,['server@REDACTED']} > > (slave@REDACTED)5> mnesia:add_table_copy(schema, node(), > disc_copies). > {aborted,{already_exists,schema,'slave@REDACTED'}} > ------ > > Regards, > Vimal > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vimal7370@REDACTED Mon Feb 8 09:59:19 2016 From: vimal7370@REDACTED (Vimal Kumar) Date: Mon, 8 Feb 2016 14:29:19 +0530 Subject: [erlang-questions] Mnesia schema replicas In-Reply-To: References: Message-ID: Hi Dan, On Mon, Feb 8, 2016 at 2:11 PM, Dan Gudmundsson wrote: > When you start mnesia without calling create_schema, mnesia will create an > empty ram_copies schema for you. > > change_config(extra_db_nodes, ..) will connect and merge schema with the > server. > > What you probably want is to call mnesia:change_table_copy_type(schema, > disc_copies, node)) after change_config. > > True, and that is what I normally do. However, the documentation says: "The functions mnesia:add_table_copy/3 and mnesia:del_table_copy/2 can be used to add and delete replicas of the schema table." Is this statement correct, or a bug in documentation? Has anyone tried to add replica of schema table using mnesia:add_table_copy/3 and succeeded? Regards, Vimal > /Dan > > On Sun, Feb 7, 2016 at 7:52 PM Vimal Kumar wrote: > >> Hello list, >> >> Ref (Section 6.6): http://erlang.org/doc/apps/mnesia/Mnesia_chap5.html >> >> -QUOTE- >> The functions mnesia:add_table_copy/3 and mnesia:del_table_copy/2 can be >> used to add and delete replicas of the schema table. Adding a node to the >> list of nodes where the schema is replicated affects the following: >> >> It allows other tables to be replicated to this node. >> It causes Mnesia to try to contact the node at startup of disc-full nodes. >> -END QUOTE- >> >> I can use the functions mnesia:add_table_copy/3 and >> mnesia:del_table_copy/2 to add and delete replicated copies of other tables >> successfully, but not table 'schema'. In fact I am not sure at what point >> in initialization process of a new mnesia node should this be run. Please >> assist. >> >> ------ >> $ erl -mnesia -sname slave >> Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:10] >> [hipe] [kernel-poll:false] >> >> Eshell V7.0 (abort with ^G) >> (slave@REDACTED)1> mnesia:start(). >> ok >> >> (slave@REDACTED)3> mnesia:change_config(extra_db_nodes, >> ['server@REDACTED']). >> {ok,['server@REDACTED']} >> >> (slave@REDACTED)5> mnesia:add_table_copy(schema, node(), >> disc_copies). >> {aborted,{already_exists,schema,'slave@REDACTED'}} >> ------ >> >> Regards, >> Vimal >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dgud@REDACTED Mon Feb 8 10:13:17 2016 From: dgud@REDACTED (Dan Gudmundsson) Date: Mon, 08 Feb 2016 09:13:17 +0000 Subject: [erlang-questions] Mnesia schema replicas In-Reply-To: References: Message-ID: Once upon a time, change_config didn't exist and you added new nodes with add_table_copy, I do think that is still works. The call to the mnesia:add_table_copy(schema, slaveNode, ram_copies), must be done on the server before change_config is done. /Dan On Mon, Feb 8, 2016 at 9:59 AM Vimal Kumar wrote: > Hi Dan, > > On Mon, Feb 8, 2016 at 2:11 PM, Dan Gudmundsson wrote: > >> When you start mnesia without calling create_schema, mnesia will create >> an empty ram_copies schema for you. >> >> change_config(extra_db_nodes, ..) will connect and merge schema with the >> server. >> >> What you probably want is to call mnesia:change_table_copy_type(schema, >> disc_copies, node)) after change_config. >> >> > True, and that is what I normally do. However, the documentation says: > > "The functions mnesia:add_table_copy/3 and mnesia:del_table_copy/2 can be > used to add and delete replicas of the schema table." > > Is this statement correct, or a bug in documentation? Has anyone tried to > add replica of schema table using mnesia:add_table_copy/3 and succeeded? > > Regards, > Vimal > > >> /Dan >> >> On Sun, Feb 7, 2016 at 7:52 PM Vimal Kumar wrote: >> >>> Hello list, >>> >>> Ref (Section 6.6): http://erlang.org/doc/apps/mnesia/Mnesia_chap5.html >>> >>> -QUOTE- >>> The functions mnesia:add_table_copy/3 and mnesia:del_table_copy/2 can be >>> used to add and delete replicas of the schema table. Adding a node to the >>> list of nodes where the schema is replicated affects the following: >>> >>> It allows other tables to be replicated to this node. >>> It causes Mnesia to try to contact the node at startup of disc-full >>> nodes. >>> -END QUOTE- >>> >>> I can use the functions mnesia:add_table_copy/3 and >>> mnesia:del_table_copy/2 to add and delete replicated copies of other tables >>> successfully, but not table 'schema'. In fact I am not sure at what point >>> in initialization process of a new mnesia node should this be run. Please >>> assist. >>> >>> ------ >>> $ erl -mnesia -sname slave >>> Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:10] >>> [hipe] [kernel-poll:false] >>> >>> Eshell V7.0 (abort with ^G) >>> (slave@REDACTED)1> mnesia:start(). >>> ok >>> >>> (slave@REDACTED)3> mnesia:change_config(extra_db_nodes, >>> ['server@REDACTED']). >>> {ok,['server@REDACTED']} >>> >>> (slave@REDACTED)5> mnesia:add_table_copy(schema, node(), >>> disc_copies). >>> {aborted,{already_exists,schema,'slave@REDACTED'}} >>> ------ >>> >>> Regards, >>> Vimal >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From Tobias.Schlager@REDACTED Mon Feb 8 10:23:30 2016 From: Tobias.Schlager@REDACTED (Tobias Schlager) Date: Mon, 8 Feb 2016 09:23:30 +0000 Subject: [erlang-questions] Mnesia schema replicas In-Reply-To: References: , Message-ID: <12F2115FD1CCEE4294943B2608A18FA301A2750D68@MAIL01.win.lbaum.eu> Hi Vimal, I've researched all of this a while ago for this project [1] which manages a dynamically growing/shrinking Mnesia cluster. The source code is full of comments on my findings which may help you. However, the project only uses RAM copies. Regards Tobias [1] https://github.com/lindenbaum/lbm_kv From mkbucc@REDACTED Mon Feb 8 14:13:05 2016 From: mkbucc@REDACTED (Mark Bucciarelli) Date: Mon, 8 Feb 2016 08:13:05 -0500 Subject: [erlang-questions] count_characters example from Programming Erlang In-Reply-To: <56B7CF2F.3060407@cs.ntua.gr> References: <5199281.3quezL6omW@changa> <20160206112956.GA14778@marks-mbp-3> <56B7CF2F.3060407@cs.ntua.gr> Message-ID: <20160208131305.GA18076@marks-mbp-3> On Mon, Feb 08, 2016 at 12:11:43AM +0100, Kostis Sagonas wrote: > > ============================================================== > Erlang installed with homebrew: > ==> Options > ... > --with-native-libs > Enable native library compilation > ... > --without-hipe > Disable building hipe; fails on various OS X systems > ============================================================== > > This combination of options does not make sense. You need the HiPE compiler > enabled in order to build the libraries in native code. > This is the default configuration that comes with the homebrew recipe. If you haven't used homebrew on Mac, I installed Erlang by typing: $ brew install erlang > > If you have experienced/noticed any problem building HiPE on your system then > perhaps you should report it. > I'll try to contact the homebrew recipe maintainer and see what's up with that. Thanks, Mark From ameretat.reith@REDACTED Mon Feb 8 15:05:03 2016 From: ameretat.reith@REDACTED (Ameretat Reith) Date: Mon, 8 Feb 2016 17:35:03 +0330 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: References: <20160201231049.08b28dc7@gmail.com> Message-ID: I simplified scenario and made a stress tester for this use case: Handle each UDP socket in a gen_server and send a UDP packet in every miliseconds [1]. It won't reach more that 280Mbit/s on my Core 2 duo system without sending anything to wire. At this point CPU will be a bottleneck here. I sent perf report in `out` directory of repository [2] and it shows still time spent in process_main is high. On our production servers with Xeon E3-1230 CPUs and low latency (.20ms between servers), I can fill 1Gbits link: send 1400 byte packets each 20ms from 1800 ports to 1800 ports, and measure bandwidth by received packets. I can transfer with 1Gbit/s speed but at this point CPU usage is above 50%. By overloading system, I can see no packet drop from /proc/net/udp but response time drops considerably. I think Erlang get packets from kernel very soon and buffers getting overrun in Erlang side, not sure how to measure them then. I disabled and enabled kernel poll, perf report same time spent but on different functions. 1: https://github.com/reith/udpstress 2: https://raw.githubusercontent.com/reith/udpstress/master/out/b67acb689f-perf.svg On Fri, Feb 5, 2016 at 12:17 PM, Max Lapshin wrote: > Well, I'm not going to argue about it, but I know that it is a serious > blocker for us: when flussonic is consuming 50% of all cores only on > capturing (unpacking mpegts is another pain) when code in C takes only > 10-15% for this task, customers are complaining. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrtndimitrov@REDACTED Mon Feb 8 15:28:01 2016 From: mrtndimitrov@REDACTED (Martin Koroudjiev) Date: Mon, 8 Feb 2016 16:28:01 +0200 Subject: [erlang-questions] Erlang SMTP client Message-ID: <56B8A5F1.5030100@gmail.com> Hello, what would you recommend for a client SMTP library? Thank you, Martin From chandrashekhar.mullaparthi@REDACTED Tue Feb 9 00:58:45 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Mon, 8 Feb 2016 23:58:45 +0000 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: References: <20160201231049.08b28dc7@gmail.com> Message-ID: Hi, I rewrote your client slightly and got better throughput than what you are getting. Tests were run on a 2.8 GHz Intel Core i7 running OS X. https://github.com/cmullaparthi/udpstress 23:52:18.712 [notice] listening on udp 12000 ... 23:52:18.780 [notice] listening on udp 12993 23:52:18.781 [notice] listening on udp 12994 23:52:18.781 [notice] listening on udp 12995 23:52:18.781 [notice] listening on udp 12996 23:52:18.781 [notice] listening on udp 12997 23:52:18.781 [notice] listening on udp 12998 23:52:18.781 [notice] listening on udp 12999 23:52:18.781 [notice] listening on udp 13000 23:52:28.718 [notice] 1454975548: recv_pkts: 0 recv_size: 0 sent_pkts: 0 sent_size: 0 recv: 0.000000 Mbit/s send: 0.000000 Mbit/s 23:52:38.724 [notice] 1454975558: recv_pkts: 0 recv_size: 0 sent_pkts: 0 sent_size: 0 recv: 0.000000 Mbit/s send: 0.000000 Mbit/s 23:52:48.725 [notice] 1454975568: recv_pkts: 0 recv_size: 0 sent_pkts: 0 sent_size: 0 recv: 0.000000 Mbit/s send: 0.000000 Mbit/s 23:52:58.728 [notice] 1454975578: recv_pkts: 679648 recv_size: 951507200 sent_pkts: 679648 sent_size: 3398240 recv: 761.205760 Mbit/s send: 2.718592 Mbit/s 23:53:08.729 [notice] 1454975588: recv_pkts: 652524 recv_size: 913533600 sent_pkts: 652524 sent_size: 3262620 recv: 730.826880 Mbit/s send: 2.610096 Mbit/s 23:53:18.730 [notice] 1454975598: recv_pkts: 638936 recv_size: 894510400 sent_pkts: 638936 sent_size: 3194680 recv: 715.608320 Mbit/s send: 2.555744 Mbit/s 23:53:28.733 [notice] 1454975608: recv_pkts: 618893 recv_size: 866450200 sent_pkts: 618893 sent_size: 3094465 recv: 693.160160 Mbit/s send: 2.475572 Mbit/s 23:53:38.735 [notice] 1454975618: recv_pkts: 620698 recv_size: 868977200 sent_pkts: 620698 sent_size: 3103490 recv: 695.181760 Mbit/s send: 2.482792 Mbit/s 23:53:48.736 [notice] 1454975628: recv_pkts: 610931 recv_size: 855303400 sent_pkts: 610931 sent_size: 3054655 recv: 684.242720 Mbit/s send: 2.443724 Mbit/s 23:53:58.738 [notice] 1454975638: recv_pkts: 623615 recv_size: 873061000 sent_pkts: 623615 sent_size: 3118075 recv: 698.448800 Mbit/s send: 2.494460 Mbit/s 23:54:08.739 [notice] 1454975648: recv_pkts: 629565 recv_size: 881391000 sent_pkts: 629565 sent_size: 3147825 recv: 705.112800 Mbit/s send: 2.518260 Mbit/s 23:54:18.740 [notice] 1454975658: recv_pkts: 624504 recv_size: 874305600 sent_pkts: 624504 sent_size: 3122520 recv: 699.444480 Mbit/s send: 2.498016 Mbit/s 23:54:28.741 [notice] 1454975668: recv_pkts: 625500 recv_size: 875700000 sent_pkts: 625500 sent_size: 3127500 recv: 700.560000 Mbit/s send: 2.502000 Mbit/s 23:54:38.742 [notice] 1454975678: recv_pkts: 615165 recv_size: 861231000 sent_pkts: 615165 sent_size: 3075825 recv: 688.984800 Mbit/s send: 2.460660 Mbit/s 23:54:48.743 [notice] 1454975688: recv_pkts: 620643 recv_size: 868900200 sent_pkts: 620643 sent_size: 3103215 recv: 695.120160 Mbit/s send: 2.482572 Mbit/s 23:54:58.744 [notice] 1454975698: recv_pkts: 623126 recv_size: 872376400 sent_pkts: 623126 sent_size: 3115630 recv: 697.901120 Mbit/s send: 2.492504 Mbit/s 23:55:08.746 [notice] 1454975708: recv_pkts: 630593 recv_size: 882830200 sent_pkts: 630593 sent_size: 3152965 recv: 706.264160 Mbit/s send: 2.522372 Mbit/s 23:55:18.747 [notice] 1454975718: recv_pkts: 623336 recv_size: 872670400 sent_pkts: 623336 sent_size: 3116680 recv: 698.136320 Mbit/s send: 2.493344 Mbit/s 23:55:28.749 [notice] 1454975728: recv_pkts: 611828 recv_size: 856559200 sent_pkts: 611828 sent_size: 3059140 recv: 685.247360 Mbit/s send: 2.447312 Mbit/s 23:55:38.750 [notice] 1454975738: recv_pkts: 626984 recv_size: 877777600 sent_pkts: 626984 sent_size: 3134920 recv: 702.222080 Mbit/s send: 2.507936 Mbit/s On 8 February 2016 at 14:05, Ameretat Reith wrote: > I simplified scenario and made a stress tester for this use case: Handle > each > UDP socket in a gen_server and send a UDP packet in every miliseconds [1]. > > It won't reach more that 280Mbit/s on my Core 2 duo system without sending > anything to wire. At this point CPU will be a bottleneck here. I sent > perf > report in `out` directory of repository [2] and it shows still time spent > in > process_main is high. > > On our production servers with Xeon E3-1230 CPUs and low latency (.20ms > between servers), I can fill 1Gbits link: send 1400 byte packets each 20ms > from 1800 ports to 1800 ports, and measure bandwidth by received packets. > I can transfer with 1Gbit/s speed but at this point CPU usage is above 50%. > By overloading system, I can see no packet drop from /proc/net/udp but > response time drops considerably. I think Erlang get packets from kernel > very > soon and buffers getting overrun in Erlang side, not sure how to measure > them > then. I disabled and enabled kernel poll, perf report same time spent but > on > different functions. > > 1: https://github.com/reith/udpstress > 2: > https://raw.githubusercontent.com/reith/udpstress/master/out/b67acb689f-perf.svg > > On Fri, Feb 5, 2016 at 12:17 PM, Max Lapshin > wrote: > >> Well, I'm not going to argue about it, but I know that it is a serious >> blocker for us: when flussonic is consuming 50% of all cores only on >> capturing (unpacking mpegts is another pain) when code in C takes only >> 10-15% for this task, customers are complaining. >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue Feb 9 02:18:53 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 9 Feb 2016 14:18:53 +1300 Subject: [erlang-questions] Illegal Guard? In-Reply-To: References: <0734D55E371C4F35B23864E126BFCEE1@gruchalski.com> Message-ID: <56B93E7D.20508@cs.otago.ac.nz> On 6/02/16 4:25 am, Trond Endrest?l wrote: > Is there a way of convincing the Erlang compiler that a certain > function is free of any side effects? I did some work on this a *long* time ago (back in the 90s) but then the language changed (funs were added) making my analysis useless. More recently, see http://user.it.uu.se/~kostis/Papers/purity.pdf The thing is that "purity" isn't quite strong enough. We want *boundedness* as well: bounded time and bounded stack space. It is a real pity that Erlang allows length(_) in guards because usually what you really want is something *bounded*: has exactly N elements, has at least N elements, has at most N elements. But at least length(L) is *bounded*. Note also, as the paper linked above certainly does, that hot-loading means that you can never be certain that a function from another module will be pure when you call it. > > Below is what I believe to be a valid case. > > While watching prof. Simon Thompson's Erlang Master Classes last year, > he demonstrated in Master Class 1, Video 6, a parser where parts of > the Erlang code was duplicated too much for my own taste. > > Ref.: http://www.kent.ac.uk/elearning/themes/masterclasses.html > > % Master Class 1, Video 6: > > % I really wish we could convince the compiler to believe these three > % functions are without any side effects, and thus suitable as guard > % tests. > > % Using macros is one way of managing the duplicate code. > > -spec is_alpha(integer()) -> boolean(). > > -define(IS_ALPHA(Ch), $a =< Ch andalso Ch =< $z). > > is_alpha(Ch) -> > ?IS_ALPHA(Ch). > > -spec is_num(integer()) -> boolean(). > > -define(IS_NUM(Ch), $0 =< Ch andalso Ch =< $9). > > is_num(Ch) -> > ?IS_NUM(Ch). > > -spec is_white(integer()) -> boolean(). > > -define(HT, 9). > -define(LF, 10). > -define(VT, 11). > -define(FF, 12). > -define(CR, 13). > -define(SP, 32). > > -define(IS_WHITE(Ch), > (Ch == ?SP) orelse (Ch == ?HT) orelse (Ch == ?LF) orelse > (Ch == ?CR) orelse (Ch == ?FF) orelse (Ch == ?VT)). > > is_white(Ch) -> > ?IS_WHITE(Ch). > > %% Lines omitted > > -spec parse(string()) -> {expr(), string()} | string(). > > parse([$( | Rest]) -> > {E1, Rest1} = parse(Rest), > [Op | Rest2] = parse(Rest1), > {E2, Rest3} = parse(Rest2), > [$) | RestFinal] = parse(Rest3), > {case Op of > $+ -> > {'add', E1, E2}; > $- -> > {'sub', E1, E2}; > $* -> > {'mul', E1, E2}; > $/ -> > {'div', E1, E2} > end, > RestFinal}; > parse([Ch | Rest]) when (Ch == $)) orelse (Ch == $+) orelse > (Ch == $-) orelse (Ch == $*) orelse > (Ch == $/) -> > [Ch | Rest]; > parse([Ch | Rest]) when ?IS_ALPHA(Ch) -> > {Succeeds, Remainder} = get_while(fun is_alpha/1, Rest), > {{var, list_to_atom([Ch | Succeeds])}, Remainder}; > parse([Ch | Rest]) when ?IS_NUM(Ch) -> > {Succeeds, Remainder} = get_while(fun is_num/1, Rest), > {{num, list_to_integer([Ch | Succeeds])}, Remainder}; > parse([Ch | Rest]) when ?IS_WHITE(Ch) -> > {_Succeeds, Remainder} = get_while(fun is_white/1, Rest), > parse(Remainder). Really, user-defined guards would not make this good. Here's something better. -spec classify(integer()) -> letter | digit | space | open | error. % classify(C) says what C can be at the beginning of an expression. classify(C) when C >= $a, C =< $z -> letter; classify(C) when C >= $A, C =< $A -> letter; classify(C) when C >= $0, C =< $9 -> digit; classify(C) when C == $\t; C == $\n; C == $\r; C == $\f; C == $\v; C == $\s -> space; classify($() -> open; classify(_) -> error. parse([C|Cs]) -> case classify(C) of space -> parse(Cs) ; letter -> take(letter, Cs, [C], fun (Ts) -> {var,list_to_atom(Ts)} end) ; digit -> take(digit, Cs, [C], fun (Ts) -> {num,list_to_integer(Ts)} end) ; open -> {E1,[Op|Cs1]} = parse(Cs), {E2,[$)|Cs2]} = parse(Cs1), T = case Op of $+ -> {add,E1,E2} ; $- -> {sub,E1,E2} ; $* -> {mul,E1,E2} ; $/ -> {dvd,E1,E2} end, {T,Cs2} end. take(Class, [C|Cs], Ts, Fun) -> case classify(C) of Class -> take(Class, Cs, [C|Ts], Fun) ; _ -> {Fun(lists:reverse(Ts)), [C|Cs]} end; take(_, [], Ts, Fun) -> {Fun(lists:reverse(Ts)), []}. I've written more tokenisers in Prolog and Erlang than I care to remember, and trust me on this, you do NOT want character literals all over the place. Amongst other things, the code above is ASCII dependent, but the ASCII dependence is now in *one* place, the classify/1 function. (I'm assuming a fixed set of binary operators here.) From keita@REDACTED Tue Feb 9 01:02:23 2016 From: keita@REDACTED (Keita Kobayashi) Date: Tue, 9 Feb 2016 09:02:23 +0900 Subject: [erlang-questions] Erlang SMTP client In-Reply-To: <56B8A5F1.5030100@gmail.com> References: <56B8A5F1.5030100@gmail.com> Message-ID: <46D204B1-4797-40DA-A062-B406AD3900A2@kbys.me> I've had good experience with the gen_smtp library. https://github.com/Vagabond/gen_smtp Keita Kobayashi keita@REDACTED > On Feb 8, 2016, at 11:28 PM, Martin Koroudjiev wrote: > > Hello, what would you recommend for a client SMTP library? > > Thank you, > Martin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Tue Feb 9 10:36:02 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 9 Feb 2016 12:36:02 +0300 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: References: <20160201231049.08b28dc7@gmail.com> Message-ID: Is it on localhost? -------------- next part -------------- An HTML attachment was scrubbed... URL: From icfp.publicity@REDACTED Tue Feb 9 07:09:58 2016 From: icfp.publicity@REDACTED (Lindsey Kuper) Date: Mon, 8 Feb 2016 22:09:58 -0800 Subject: [erlang-questions] ICFP 2016 Second Call for Papers Message-ID: ICFP 2016 The 21st ACM SIGPLAN International Conference on Functional Programming http://conf.researchr.org/home/icfp-2016 Second Call for Papers Important dates --------------- Submissions due: Wednesday, March 16 2016, 15:00 (UTC) https://icfp2016.hotcrp.com (now open) Author response: Monday, 2 May, 2016, 15:00 (UTC) - Thursday, 5 May, 2016, 15:00 (UTC) Notification: Friday, 20 May, 2016 Final copy due: TBA Early registration: TBA Conference: Monday, 19 September - Wednesday, 21 September, 2016 (note updated conference dates) Scope ----- ICFP 2016 seeks original papers on the art and science of functional programming. Submissions are invited on all topics from principles to practice, from foundations to features, and from abstraction to application. The scope includes all languages that encourage functional programming, including both purely applicative and imperative languages, as well as languages with objects, concurrency, or parallelism. Topics of interest include (but are not limited to): - Language Design: concurrency, parallelism, and distribution; modules; components and composition; metaprogramming; type systems; interoperability; domain-specific languages; and relations to imperative, object-oriented, or logic programming. - Implementation: abstract machines; virtual machines; interpretation; compilation; compile-time and run-time optimization; garbage collection and memory management; multi-threading; exploiting parallel hardware; interfaces to foreign functions, services, components, or low-level machine resources. - Software-Development Techniques: algorithms and data structures; design patterns; specification; verification; validation; proof assistants; debugging; testing; tracing; profiling. - Foundations: formal semantics; lambda calculus; rewriting; type theory; monads; continuations; control; state; effects; program verification; dependent types. - Analysis and Transformation: control-flow; data-flow; abstract interpretation; partial evaluation; program calculation. - Applications: symbolic computing; formal-methods tools; artificial intelligence; systems programming; distributed-systems and web programming; hardware design; databases; XML processing; scientific and numerical computing; graphical user interfaces; multimedia and 3D graphics programming; scripting; system administration; security. - Education: teaching introductory programming; parallel programming; mathematical proof; algebra. - Functional Pearls: elegant, instructive, and fun essays on functional programming. - Experience Reports: short papers that provide evidence that functional programming really works or describe obstacles that have kept it from working. If you are concerned about the appropriateness of some topic, do not hesitate to contact the program chair. Abbreviated instructions for authors ------------------------------------ - By Wednesday, March 16 2016, 15:00 (UTC), submit a full paper of at most 12 pages (6 pages for an Experience Report), in standard SIGPLAN conference format, including figures but ***excluding bibliography***. The deadlines will be strictly enforced and papers exceeding the page limits will be summarily rejected. ***ICFP 2016 will employ a lightweight double-blind reviewing process.*** To facilitate this, submitted papers must adhere to two rules: 1. ***author names and institutions must be omitted***, and 2. ***references to authors' own related work should be in the third person*** (e.g., not "We build on our previous work ..." but rather "We build on the work of ..."). The purpose of this process is to help the PC and external reviewers come to an initial judgement about the paper without bias, not to make it impossible for them to discover the authors if they were to try. Nothing should be done in the name of anonymity that weakens the submission or makes the job of reviewing the paper more difficult (e.g., important background references should not be omitted or anonymized). In addition, authors should feel free to disseminate their ideas or draft versions of their paper as they normally would. For instance, authors may post drafts of their papers on the web or give talks on their research ideas. We have put together a document answering frequently asked questions that should address many common concerns: http://conf.researchr.org/track/icfp-2016/icfp-2016-papers#Submission-and-Reviewing-FAQ (last updated February 8, 2016). - Authors have the option to attach supplementary material to a submission, on the understanding that reviewers may choose not to look at it. The material should be uploaded at submission time, as a single pdf or a tarball, not via a URL. This supplementary material may or may not be anonymized; if not anonymized, it will only be revealed to reviewers after they have submitted their review of your paper and learned your identity. - Each submission must adhere to SIGPLAN's republication policy, as explained on the web at: http://www.sigplan.org/Resources/Policies/Republication - Authors of resubmitted (but previously rejected) papers have the option to attach an annotated copy of the reviews of their previous submission(s), explaining how they have addressed these previous reviews in the present submission. If a reviewer identifies him/herself as a reviewer of this previous submission and wishes to see how his/her comments have been addressed, the program chair will communicate to this reviewer the annotated copy of his/her previous review. Otherwise, no reviewer will read the annotated copies of the previous reviews. Overall, a submission will be evaluated according to its relevance, correctness, significance, originality, and clarity. It should explain its contributions in both general and technical terms, clearly identifying what has been accomplished, explaining why it is significant, and comparing it with previous work. The technical content should be accessible to a broad audience. Functional Pearls and Experience Reports are separate categories of papers that need not report original research results and must be marked as such at the time of submission. Detailed guidelines on both categories are given below. Presentations will be videotaped and released online if the presenter consents. The proceedings will be freely available for download from the ACM Digital Library from at least one week before the start of the conference until two weeks after the conference. Formatting: Submissions must be in PDF format printable in black and white on US Letter sized paper and interpretable by Ghostscript. Papers must adhere to the standard SIGPLAN conference format: two columns, nine-point font on a ten-point baseline, with columns 20pc (3.33in) wide and 54pc (9in) tall, with a column gutter of 2pc (0.33in). A suitable document template for LaTeX is available at http://www.sigplan.org/Resources/Author/ Submission: Submissions will be accepted at https://icfp2016.hotcrp.com. Improved versions of a paper may be submitted at any point before the submission deadline using the same web interface. Author response: Authors will have a 72-hour period, starting at 15:00 UTC on Monday, 2 May, 2016, to read reviews and respond to them. ACM Author-Izer is a unique service that enables ACM authors to generate and post links on either their home page or institutional repository for visitors to download the definitive version of their articles from the ACM Digital Library at no charge. Downloads through Author-Izer links are captured in official ACM statistics, improving the accuracy of usage and impact measurements. Consistently linking the definitive version of ACM article should reduce user confusion over article versioning. After your article has been published and assigned to your ACM Author Profile page, please visit http://www.acm.org/publications/acm-author-izer-service to learn how to create your links for free downloads from the ACM DL. AUTHORS TAKE NOTE: The official publication date is the date the proceedings are made available in the ACM Digital Library. This date may be up to two weeks prior to the first day of your conference. The official publication date affects the deadline for any patent filings related to published work. Special categories of papers ---------------------------- In addition to research papers, ICFP solicits two kinds of papers that do not require original research contributions: Functional Pearls, which are full papers, and Experience Reports, which are limited to six pages. Authors submitting such papers may wish to consider the following advice. Functional Pearls ================= A Functional Pearl is an elegant essay about something related to functional programming. Examples include, but are not limited to: - a new and thought-provoking way of looking at an old idea - an instructive example of program calculation or proof - a nifty presentation of an old or new data structure - an interesting application of functional programming techniques - a novel use or exposition of functional programming in the classroom While pearls often demonstrate an idea through the development of a short program, there is no requirement or expectation that they do so. Thus, they encompass the notions of theoretical and educational pearls. Functional Pearls are valued as highly and judged as rigorously as ordinary papers, but using somewhat different criteria. In particular, a pearl is not required to report original research, but, it should be concise, instructive, and entertaining. Your pearl is likely to be rejected if your readers get bored, if the material gets too complicated, if too much specialized knowledge is needed, or if the writing is inelegant. The key to writing a good pearl is polishing. A submission you wish to have treated as a pearl must be marked as such on the submission web page, and should contain the words ``Functional Pearl'' somewhere in its title or subtitle. These steps will alert reviewers to use the appropriate evaluation criteria. Pearls will be combined with ordinary papers, however, for the purpose of computing the conference's acceptance rate. Experience Reports ================== The purpose of an Experience Report is to help create a body of published, refereed, citable evidence that functional programming really works ? or to describe what obstacles prevent it from working. Possible topics for an Experience Report include, but are not limited to: - insights gained from real-world projects using functional programming - comparison of functional programming with conventional programming in the context of an industrial project or a university curriculum - project-management, business, or legal issues encountered when using functional programming in a real-world project - curricular issues encountered when using functional programming in education - real-world constraints that created special challenges for an implementation of a functional language or for functional programming in general An Experience Report is distinguished from a normal ICFP paper by its title, by its length, and by the criteria used to evaluate it. - Both in the proceedings and in any citations, the title of each accepted Experience Report must begin with the words ``Experience Report'' followed by a colon. The acceptance rate for Experience Reports will be computed and reported separately from the rate for ordinary papers. - An Experience Report is at most six pages long. Each accepted Experience Report will be presented at the conference, but depending on the number of Experience Reports and regular papers accepted, authors of Experience reports may be asked to give shorter talks. - Because the purpose of Experience Reports is to enable our community to accumulate a body of evidence about the efficacy of functional programming, an acceptable Experience Report need not add to the body of knowledge of the functional-programming community by presenting novel results or conclusions. It is sufficient if the Report states a clear thesis and provides supporting evidence. The thesis must be relevant to ICFP, but it need not be novel. The program committee will accept or reject Experience Reports based on whether they judge the evidence to be convincing. Anecdotal evidence will be acceptable provided it is well argued and the author explains what efforts were made to gather as much evidence as possible. Typically, more convincing evidence is obtained from papers which show how functional programming was used than from papers which only say that functional programming was used. The most convincing evidence often includes comparisons of situations before and after the introduction or discontinuation of functional programming. Evidence drawn from a single person's experience may be sufficient, but more weight will be given to evidence drawn from the experience of groups of people. An Experience Report should be short and to the point: make a claim about how well functional programming worked on your project and why, and produce evidence to substantiate your claim. If functional programming worked for you in the same ways it has worked for others, you need only to summarize the results?the main part of your paper should discuss how well it worked and in what context. Most readers will not want to know all the details of your project and its implementation, but please characterize your project and its context well enough so that readers can judge to what degree your experience is relevant to their own projects. Be especially careful to highlight any unusual aspects of your project. Also keep in mind that specifics about your project are more valuable than generalities about functional programming; for example, it is more valuable to say that your team delivered its software a month ahead of schedule than it is to say that functional programming made your team more productive. If your paper not only describes experience but also presents new technical results, or if your experience refutes cherished beliefs of the functional-programming community, you may be better off submitting it as a full paper, which will be judged by the usual criteria of novelty, originality, and relevance. If you are unsure in which category to submit, the program chair will be happy to help you decide. Organizers ---------- General Co-Chairs: Jacques Garrigue (Nagoya University) Gabriele Keller (University of New South Wales) Program Chair: Eijiro Sumii (Tohoku University) Program Committee: Koen Claessen (Chalmers University of Technology) Joshua Dunfield (University of British Columbia, Canada) Matthew Fluet (Rochester Institute of Technology) Nate Foster (Cornell University) Dan Grossman (University of Washington, USA) Jurriaan Hage (Utrecht University) Roman Leshchinskiy (Standard Chartered Bank) Keisuke Nakano (The University of Electro-Communications) Aleksandar Nanevski (IMDEA Software Institute) Scott Owens (University of Kent) Sungwoo Park (Pohang University of Science and Technology) Amr Sabry (Indiana University) Tom Schrijvers (KU Leuven) Olin Shivers (Northeastern University) Walid Taha (Halmstad University) Dimitrios Vytiniotis (Microsoft Research, Cambridge) David Walker (Princeton University) Nobuko Yoshida (Imperial College London, UK) External Review Committee: See http://conf.researchr.org/committee/icfp-2016/icfp-2016-papers-external-review-committee. From pavananms@REDACTED Tue Feb 9 14:58:08 2016 From: pavananms@REDACTED (Pavanan M S) Date: Tue, 9 Feb 2016 19:28:08 +0530 Subject: [erlang-questions] Rabbitmq examples not working Message-ID: HI all, I downloaded the complete Rabitmq examples and tried to run it according to the instructions in this link https://github.com/rabbitmq/rabbitmq-tutorials/tree/master/erlang. But when trying to run ./send the following error occurs. ./send.erl:4: can't find include lib "amqp_client/include/amqp_client.hrl" escript: There were compilation errors. I have followed the instructions carefully ..What should I do now? -------------- next part -------------- An HTML attachment was scrubbed... URL: From vasdeveloper@REDACTED Tue Feb 9 15:03:10 2016 From: vasdeveloper@REDACTED (Theepan) Date: Tue, 9 Feb 2016 19:33:10 +0530 Subject: [erlang-questions] Machine Learning Message-ID: Hi Team, I could search and find out couple of Erlang libraries in the NN space, but would you share your experience with me. What would be the best direction or tools to start with? Regards, Theepan -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Tue Feb 9 15:57:21 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Tue, 9 Feb 2016 15:57:21 +0100 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: References: <20160201231049.08b28dc7@gmail.com> Message-ID: Hello, On Wed, Feb 3, 2016 at 5:23 PM, Max Lapshin wrote: > When we use plain gen_udp to accept 200-400 mbit of incoming MPEG-TS > traffic in flussonic, we use about 50% of moderate 4 core xeon e3 server. > > When we switch to our driver implementation of udp that collapses several > contiguous udp messages into single big message (it is allowed for mpegts) > we reduce usage to 15-20% > > I can't tell that it is "badly written udp in erlang", just messaging is > rather expensive. > Do you think that it is the batching that makes the performance difference? In that case do you think adding an option to gen_udp that does batching would help enough? There is already read_packets as an option, and maybe it makes sense to have an option that batches all reads done by read_packets into one erlang message. i.e. you get something like: {udp_batch, Socket, IP, InPortNo, [Packets]} or possibly {udp_batch, Socket, [{IP, InPortNo, Packet}]} Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From ameretat.reith@REDACTED Tue Feb 9 16:10:08 2016 From: ameretat.reith@REDACTED (Ameretat Reith) Date: Tue, 9 Feb 2016 18:40:08 +0330 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: References: <20160201231049.08b28dc7@gmail.com> Message-ID: On Tue, Feb 9, 2016 at 3:28 AM, Chandru < chandrashekhar.mullaparthi@REDACTED> wrote: Hi, I rewrote your client slightly and got better throughput than what you are getting. Tests were run on a 2.8 GHz Intel Core i7 running OS X. https://github.com/cmullaparthi/udpstress Thanks. I tested your method and made same thing on server (avoiding gen_server) Here is what I'm getting: VMARGS_PATH=$PWD/server.args ./bin/udpstress foreground -extra plain_server 1400 1600 06:04:31.122 [notice] 1455023071: recv_pkts: 91000 recv_size: 127400000 sent_pkts: 91000 sent_size: 455000 recv: 101.920000 Mbit/s send: 0.364000 Mbit/s 06:04:41.123 [notice] 1455023081: recv_pkts: 642473 recv_size: 899462200 sent_pkts: 642473 sent_size: 3212365 recv: 719.569760 Mbit/s send: 2.569892 Mbit/s 06:04:51.124 [notice] 1455023091: recv_pkts: 659013 recv_size: 922618200 sent_pkts: 659013 sent_size: 3295065 recv: 738.094560 Mbit/s send: 2.636052 Mbit/s 06:05:01.126 [notice] 1455023101: recv_pkts: 656831 recv_size: 919563400 sent_pkts: 656831 sent_size: 3284155 recv: 735.650720 Mbit/s send: 2.627324 Mbit/s 06:05:11.126 [notice] 1455023111: recv_pkts: 646297 recv_size: 904815800 sent_pkts: 646297 sent_size: 3231485 recv: 723.852640 Mbit/s send: 2.585188 Mbit/s 06:05:21.127 [notice] 1455023121: recv_pkts: 638607 recv_size: 894049800 sent_pkts: 638607 sent_size: 3193035 recv: 715.239840 Mbit/s send: 2.554428 Mbit/s 06:05:31.128 [notice] 1455023131: recv_pkts: 641356 recv_size: 897898400 sent_pkts: 641356 sent_size: 3206780 recv: 718.318720 Mbit/s send: 2.565424 Mbit/s $ VMARGS_PATH=$PWD/server.args ./bin/udpstress foreground -extra genserver_server 1400 1600 06:06:41.262 [notice] 1455023201: recv_pkts: 238786 recv_size: 334300400 sent_pkts: 238786 sent_size: 1193930 recv: 267.440320 Mbit/s send: 0.955144 Mbit/s 06:06:51.262 [notice] 1455023211: recv_pkts: 646220 recv_size: 904708000 sent_pkts: 646220 sent_size: 3231100 recv: 723.766400 Mbit/s send: 2.584880 Mbit/s 06:07:01.263 [notice] 1455023221: recv_pkts: 647552 recv_size: 906572800 sent_pkts: 647552 sent_size: 3237760 recv: 725.258240 Mbit/s send: 2.590208 Mbit/s 06:07:11.264 [notice] 1455023231: recv_pkts: 642863 recv_size: 900008200 sent_pkts: 642863 sent_size: 3214315 recv: 720.006560 Mbit/s send: 2.571452 Mbit/s 06:07:21.265 [notice] 1455023241: recv_pkts: 644790 recv_size: 902706000 sent_pkts: 644790 sent_size: 3223950 recv: 722.164800 Mbit/s send: 2.579160 Mbit/s Seems both servers fullfill the request rate, now I try to flood server: request every 20 milliseconds and see receive rate in server: $ VMARGS_PATH=$PWD/client.args ./bin/udpstress foreground -extra plain_client server_addr -i 20 1400 3100 $ VMARGS_PATH=$PWD/server.args ./bin/udpstress foreground -extra plain_server 1400 3100 07:49:01.804 [notice] 1455029341: recv_pkts: 850948 recv_size: 1191327200 sent_pkts: 850948 sent_size: 4254740 recv: 953.061760 Mbit/s send: 3.403792 Mbit/s 07:49:11.805 [notice] 1455029351: recv_pkts: 851744 recv_size: 1192441600 sent_pkts: 851744 sent_size: 4258720 recv: 953.953280 Mbit/s send: 3.406976 Mbit/s while using genserver_client less packets getting routed, server log: 07:53:41.832 [notice] 1455029621: recv_pkts: 810797 recv_size: 1135115800 sent_pkts: 810797 sent_size: 4053985 recv: 908.092640 Mbit/s send: 3.243188 Mbit/s 07:53:51.833 [notice] 1455029631: recv_pkts: 810403 recv_size: 1134564200 sent_pkts: 810403 sent_size: 4052015 recv: 907.651360 Mbit/s send: 3.241612 Mbit/s Above logs are made using two Xeon E3 Quad core servers having about .20ms latency and codes I collected in [1]. I think problem is not using or not using gen_server, It's mostly because just getting UDP packets in 1Gbit/s takes that much CPU in Erlang. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Feb 9 16:19:07 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 9 Feb 2016 16:19:07 +0100 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: References: <20160201231049.08b28dc7@gmail.com> Message-ID: On Tue, Feb 9, 2016 at 4:10 PM, Ameretat Reith wrote: > I think problem is not using or not using gen_server, It's mostly because > just getting UDP packets in 1Gbit/s takes that much CPU in Erlang. Where is that time spent in the Erlang VM or in the Kernel? You are potentially on a wakeup schedule of 81 wakeups per millisecond to handle packets. Which suggests you need to understand where your CPU time is spent in the system in order to tune it for lower CPU usage. Many systems can show lower CPU load, until you are forced to do things with the packets at which point they have to pay with CPU load in order to organize the data. Chances are such organization has already been done for you by the VM. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Feb 9 17:51:34 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 9 Feb 2016 17:51:34 +0100 Subject: [erlang-questions] Machine Learning In-Reply-To: References: Message-ID: On Tue, Feb 9, 2016 at 3:03 PM, Theepan wrote: > What would be the best direction or tools to start with? I would start by writing a tensorflow program as something you can talk to via a port, should I start today. Tensorflow networks are configured and trained in Python, but can be run everywhere you can call into C. The nice idea is that you split network tinkering from actually using the NN you are generating. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From anpavl@REDACTED Tue Feb 9 22:08:03 2016 From: anpavl@REDACTED (Pavel Abalihin) Date: Wed, 10 Feb 2016 00:08:03 +0300 Subject: [erlang-questions] Erlang SMTP client In-Reply-To: <56B8A5F1.5030100@gmail.com> References: <56B8A5F1.5030100@gmail.com> Message-ID: Pat: https://github.com/selectel/pat On 8 February 2016 at 17:28, Martin Koroudjiev wrote: > Hello, what would you recommend for a client SMTP library? > > Thank you, > Martin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed Feb 10 06:54:50 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 10 Feb 2016 18:54:50 +1300 Subject: [erlang-questions] Bit syntax matching gotchas In-Reply-To: References: Message-ID: <56BAD0AA.3020207@cs.otago.ac.nz> On 3/02/16 7:17 pm, Bj?rn Gustavsson wrote: > There are some gotchas in the bit syntax that comes up now and then on > the mailing list, and also as a bug report at the end of last year: > http://bugs.erlang.org/browse/ERL-44 I am sorry to be so late replying to this, but things have been (a) frantic and (b) horrible. > BACKGROUND ABOUT BIT SYNTAX CONSTRUCTION When constructing binaries, > there is an implicit masking of the values. All of the following > constructions give the same result: <<255>> <<16#FF>> <<16#FFFF>> <<-1>> That always felt so WRONG to me. If I *want* masking, I can write <>. I really do not understand why this kind of thing was ever considered acceptable: 3> <> = <>. ** exception error: no match of right hand side value <<"?">> If you stop doing the masking (except where there is an explicit 'band' licensing it) then (a) some modules will break (b) the fix will be easy (c) the fixed code will be backwards portable > There have been complaints about the implicit masking behaviour, but > there is a lot of code that depends on it, so it would be unwise to > change it. I don't buy that, sorry. Make a new option: -bit_syntax(safe|dangerous). allowed once per module. dangerous is the current behaviour, safe raises an exception for overflow. In R19, introduce it, with dangerous as the default. In R20, make the compiler issue a warning if a module has a bit syntax form that would be ambiguous and doesn't state the desired behaviour. In R21, make safe the default. In R22, make the compiler issue a warning if a module has -bit_syntax(dangerous). In R23, remove the option. This process *FIXES* the problem. Instead of safe/dangerous, consistent/inconsistent might be better. > POSSIBLE SOLUTION #1 The most obvious solution is probably to let the > compiler warn for the above cases. The matching would still fail. The > developer will need to fix their code. For example: <<-1/signed>> = > <<-1>> POSSIBLE SOLUTION #2 There is one problem with the solution #1. > It is not possible to produce a warning for the following example: > f(Val) -> <> = <>, Val. Then solution #1 is not a solution. But why isn't it possible? The problem arises whenever you have <<...X:N...>> and the compiler has no proof that X fits in N bits. This is just such a case. > The proble So in addition to warning when possible, another solution > is to mask values also when matching. Internally, the compiler could > rewrite the function to something like: f(Val) -> <> = > <>, Val = NewVar band 16#FF, Val. Similar rewriting should be > done for literal integer, so the following expression would now match: > <<-1>> = <<-1>> What exactly is the rewriting? 0> <> = <<-1>>, 10> -1 = NuVar band 255. ** exception error: no match of right hand side value 255 That's not a match. > Solution #1 would point all cases when literal integers could not > possibly match and force the developer to fix them. Therefore I choose > solution #1. The problem is not a problem with literal integers. It is a problem with ANYTHING that doesn't fit being quietly truncated in one context but not the other. Anything that leaves matching and building contexts inconsistent is not a solution. If a match cannot *return* a value in a certain place, the corresponding build should not *accept* that value in that place. The idea that this can't be fixed because there is code depending on the old behaviour reminds me of two famous bugs. (C) C's switch statement is a botch, with 'break' being used both for "exit this loop" and "exit this switch". This was fixed in C's grandparent BCPL where 'break' was used for "exit this loop" only and 'endcase' was used for "exit this switch only". C's designers were aware of that correction to BCPL, but felt they could not adopt it because they "already had 100 users". (X) Excel notoriously gets its leap year calculations wrong but Microsoft have announced that this feature will not be changed in case there are users who depend on the bug. I'm also reminded of a bug that was squashed. (B) Burroughs Algol has pointers. That made it possible for you to write EBCDIC POINTER P; BEGIN EBCDIC ARRAY A[0:79]; P := A; END; and then use P. They didn't notice the problem for a while. They then made a temporary fix for the operating system so that when you exited a block that declared an array, the OS would sweep your stack looking for pointers to it. They also made an addition to the language, so that you could write EBCDIC ARRAY B[0:131]; EBCDIC POINTER P FOR B; BEGIN EBCDIC ARRAY A[0:79]; P := A; %% This is a syntax error! END; Customers were given about a year to fix their programs, then warnings were turned into hard errors and the OS patch was removed, and presto chango, absence of dangling pointers proved by the compilers. Burroughs Algol is still available -- I have a personal copy of the MCP and software that runs in a VM on a Windows box; now all I need is a Windows box -- and for 30 years that class of bugs has been GONE. The C bug has been perpetuated into C++, Java, and JavaScript and people are *still* making 'break' mistakes, all because people chose to perpetuate old bugs rather than eliminate them. The longer any match/build inconsistency is allowed to remain in bit syntax, the more pain it will cause. From touch.sereysethy@REDACTED Wed Feb 10 08:16:31 2016 From: touch.sereysethy@REDACTED (Sereysethy TOUCH) Date: Wed, 10 Feb 2016 14:16:31 +0700 Subject: [erlang-questions] Rabbitmq examples not working In-Reply-To: References: Message-ID: Hi, You just need to download rabbit_common and rabbit_client and put it in the same directory wget http://www.rabbitmq.com/releases/rabbitmq-erlang-client/v2.7.0/rabbit_common-2.7.0.ez unzip rabbit_common-2.7.0.ez ln -s rabbit_common-2.7.0 rabbit_common wget http://www.rabbitmq.com/releases/rabbitmq-erlang-client/v2.7.0/amqp_client-2.7.0.ez unzip amqp_client-2.7.0.ez ln -s amqp_client-2.7.0 amqp_client On Tue, Feb 9, 2016 at 8:58 PM, Pavanan M S wrote: > HI all, > > I downloaded the complete Rabitmq examples and tried to run > it according to the instructions in this link > https://github.com/rabbitmq/rabbitmq-tutorials/tree/master/erlang. > But when trying to run ./send the following error occurs. > > ./send.erl:4: can't find include lib > "amqp_client/include/amqp_client.hrl" > escript: There were compilation errors. > > I have followed the instructions carefully ..What should > I do now? > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From samuelrivas@REDACTED Wed Feb 10 10:34:58 2016 From: samuelrivas@REDACTED (Samuel) Date: Wed, 10 Feb 2016 10:34:58 +0100 Subject: [erlang-questions] Machine Learning In-Reply-To: References: Message-ID: >> What would be the best direction or tools to start with? > > > I would start by writing a tensorflow program as something you can talk to > via a port, should I start today. Tensorflow networks are configured and > trained in Python, but can be run everywhere you can call into C. The nice > idea is that you split network tinkering from actually using the NN you are > generating. I am not aware of existing ML or linear algrebra libraries in erlang that can be used to quick start an ML project, but just piping into tensorflow (or any other existing library/framework) isn't really doing ML with erlang, is it? You can as well just use tensorflow directly. Unless the problem to solve is orchestrating a big, distributed system that has some ML components in it; then doing the orchestration in Erlang that way might be a good idea. It still could be an interesting project to create a ML framework in erlang itself, a quick google reveals some previous works on that, but I haven't gone through any of them, so I'll let the Theepan do the research :) From ivan@REDACTED Wed Feb 10 11:07:49 2016 From: ivan@REDACTED (Ivan Uemlianin) Date: Wed, 10 Feb 2016 10:07:49 +0000 Subject: [erlang-questions] Machine Learning In-Reply-To: References: Message-ID: <1455098870919-d945cd2f-a2396c90-50be329a@llaisdy.com> There is Gene Sher's neural net / artificial life work, e.g.: https://github.com/CorticalComputer/DXNN http://www.springer.com/us/book/9781461444626 On Wed, Feb 10, 2016 at 9:35 am, Samuel wrote: >> What would be the best direction or tools to start with? > > > I would start by writing a tensorflow program as something you can talk to > via a port, should I start today. Tensorflow networks are configured and > trained in Python, but can be run everywhere you can call into C. The nice > idea is that you split network tinkering from actually using the NN you are > generating. I am not aware of existing ML or linear algrebra libraries in erlang that can be used to quick start an ML project, but just piping into tensorflow (or any other existing library/framework) isn't really doing ML with erlang, is it? You can as well just use tensorflow directly. Unless the problem to solve is orchestrating a big, distributed system that has some ML components in it; then doing the orchestration in Erlang that way might be a good idea. It still could be an interesting project to create a ML framework in erlang itself, a quick google reveals some previous works on that, but I haven't gone through any of them, so I'll let the Theepan do the research :) _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From grahamrhay@REDACTED Wed Feb 10 12:05:45 2016 From: grahamrhay@REDACTED (Graham Hay) Date: Wed, 10 Feb 2016 11:05:45 +0000 Subject: [erlang-questions] Erlang SMTP client In-Reply-To: References: <56B8A5F1.5030100@gmail.com> Message-ID: On 9 February 2016 at 21:08, Pavel Abalihin wrote: > > Pat: https://github.com/selectel/pat > Good name. From jesper.louis.andersen@REDACTED Wed Feb 10 12:31:44 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 10 Feb 2016 12:31:44 +0100 Subject: [erlang-questions] Machine Learning In-Reply-To: References: Message-ID: On Wed, Feb 10, 2016 at 10:34 AM, Samuel wrote: > I am not aware of existing ML or linear algrebra libraries in erlang > that can be used to quick start an ML project, but just piping into > tensorflow (or any other existing library/framework) isn't really > doing ML with erlang, is it? You can as well just use tensorflow > directly. > The question is if this is a practical problem which needs solving or it is for research. If you are researching how to construct, say, SVM or NNs, then surely Erlang is a good vehicle. But in practice, there is a number of things which makes Erlang unsuitable: * ML is often CPU bound. You don't want a bytecode interpreter to be a limiting factor here. Even if the interpreter in Erlang is state-of-the-art and highly optimized, it is not far fetched that a FP-intensive program will be roughly a factor of 30 faster if compiled in a lower level language. * GPUs are popular in ML models for a reason: they speed up the FP computations by a factor of 3000 or more. This in itself should hint you that you need something else than Erlang. * Erlangs word overhead per process and terms means a lower-level model can pack many more entities in memory. This affects caching behavior. Training of the model is often off-line and using the model is online in the system. How you train your model is less important. This is why I'd just outsource this problem to the libraries built and tuned for it. It is like solving LinAlg problems but forgetting everything about existing LAPACK and ATLAS routines in Fortran. A model, in Erlang, which could be viable is to use Erlang to produce programs for lower level consumption by compilation. But these are problems for which languages such as Haskell and OCaml dominates for a reason: their type systems makes it far easier to pull off. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Wed Feb 10 13:56:48 2016 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 10 Feb 2016 13:56:48 +0100 Subject: [erlang-questions] Proposal for new state machine engine: gen_statem Message-ID: <20160210125648.GA85287@erix.ericsson.se> Hi all! I have created a pull request for a new state machine engine in the gen_* framework. There is a heated discussion right now on GitHUB: https://github.com/erlang/otp/pull/960 Please contribute with an opinion. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From rok@REDACTED Wed Feb 10 14:59:19 2016 From: rok@REDACTED (Robert Kowalski) Date: Wed, 10 Feb 2016 13:59:19 +0000 Subject: [erlang-questions] Who is interested in an Erlang Performance Improvement Book? In-Reply-To: <20150810212624.1059.82914@domU-12-31-39-0A-A0-4F> References: <20150810212624.1059.82914@domU-12-31-39-0A-A0-4F> Message-ID: hi! the publishers i talked to were not interested, as the topic is too niche. i took a break, wrote a book about writing command line tools and designing CLIs (The CLI Book) and will slowly release the content I had as a series of blogposts (the projects we will cover are: couchdb, mochiweb and lager) all three use flamegraphs to find bottlenecks, if i had written the book i had focussed on other methods, too. here is the first post, enjoy: http://kowalski.gd/blog/high-performance-erlang-finding-bottlenecks-couchdb-1 On Tue, Aug 11, 2015 at 7:00 AM, Robert Kowalski wrote: > Hi list, > > I wrote an article how to learn Erlang by example [1] which got a lot of > good feedback recently when it was posted on HN. > > The past weeks I am working on finding bottlenecks and try to improve the > performance of Erlang Open Source projects. > > Based on my work, findings and insights I was asking myself if you would be > interested in a book about way to measure and improve Erlang performance. > Like my blogpost it would use real world examples, this time from more Open > Source Erlang projects. > > What do you think? > > Best, > Robert > > > [1] > http://robert-kowalski.de/blog/lets-learn-erlang-and-fix-a-bug-on-a-couchdb-cluster/ From ameretat.reith@REDACTED Wed Feb 10 21:49:19 2016 From: ameretat.reith@REDACTED (Ameretat Reith) Date: Thu, 11 Feb 2016 00:19:19 +0330 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: References: <20160201231049.08b28dc7@gmail.com> Message-ID: <20160211001919.03d4f012@gmail.com> On Tue, 9 Feb 2016 16:19:07 +0100 Jesper Louis Andersen wrote: > Where is that time spent in the Erlang VM or in the Kernel? You are > potentially on a wakeup schedule of 81 wakeups per millisecond to > handle packets. Which suggests you need to understand where your CPU > time is spent in the system in order to tune it for lower CPU usage. In more powerful machines that can handle 1Gbit/s with 50% CPU utilization, It's VM's scheduler_wait consuming about 30% of CPU usage. (xeon-e3 attachment) suggesting system can get more input. In overloaded and less powerful machines, VM task management functions get more resources and time spent in kernel is less than 20%. (corei3 attachment) -------------- next part -------------- A non-text attachment was scrubbed... Name: xeon-e3-1gbits.svg.bz2 Type: application/x-bzip Size: 61925 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: corei3-local-600mbits.svg.bz2 Type: application/x-bzip Size: 59501 bytes Desc: not available URL: From sargun@REDACTED Thu Feb 11 04:04:00 2016 From: sargun@REDACTED (Sargun Dhillon) Date: Wed, 10 Feb 2016 19:04:00 -0800 Subject: [erlang-questions] Proposal for new state machine engine: gen_statem In-Reply-To: <20160210125648.GA85287@erix.ericsson.se> References: <20160210125648.GA85287@erix.ericsson.se> Message-ID: Do you have a standalone implementation of gen_statem? It would be great to test converting some existing code, and see how it feels. On Wed, Feb 10, 2016 at 4:56 AM, Raimo Niskanen wrote: > Hi all! > > I have created a pull request for a new state machine engine in the gen_* > framework. There is a heated discussion right now on GitHUB: > > https://github.com/erlang/otp/pull/960 > > Please contribute with an opinion. > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From unix1@REDACTED Thu Feb 11 07:39:15 2016 From: unix1@REDACTED (Unix One) Date: Thu, 11 Feb 2016 06:39:15 +0000 Subject: [erlang-questions] Illegal Guard? In-Reply-To: <56B93E7D.20508@cs.otago.ac.nz> References: <0734D55E371C4F35B23864E126BFCEE1@gruchalski.com> <56B93E7D.20508@cs.otago.ac.nz> Message-ID: On 02/08/2016 05:18 PM, Richard A. O'Keefe wrote: > Note also, as the paper linked above certainly does, > that hot-loading means that you can never be certain > that a function from another module will be pure when > you call it. Please excuse my ignorance, but wouldn't this specific point potentially not be an issue if: - guards required functions to be explicitly declared pure - compiler verified the assertion of purity - hot code loader refused to load if a guard function wasn't declared pure Maybe the declaration of purity is not necessarily required, but it does make it more explicit. Thank you. From ok@REDACTED Thu Feb 11 08:01:55 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 11 Feb 2016 20:01:55 +1300 Subject: [erlang-questions] Illegal Guard? In-Reply-To: References: <0734D55E371C4F35B23864E126BFCEE1@gruchalski.com> <56B93E7D.20508@cs.otago.ac.nz> Message-ID: <56BC31E3.7090804@cs.otago.ac.nz> On 11/02/16 7:39 pm, Unix One wrote: > On 02/08/2016 05:18 PM, Richard A. O'Keefe wrote: >> Note also, as the paper linked above certainly does, >> that hot-loading means that you can never be certain >> that a function from another module will be pure when >> you call it. > Please excuse my ignorance, but wouldn't this specific point potentially > not be an issue if: > > - guards required functions to be explicitly declared pure > - compiler verified the assertion of purity > - hot code loader refused to load if a guard function wasn't declared pure > > Maybe the declaration of purity is not necessarily required, but it does > make it more explicit. I believe I mentioned that purity is not sufficient. f(X) -> f(X). is pure. But it is not bounded, and boundedness is of practical importance. We *could* have declarations like that. Currently we don't. There are some technical problems which could probably be resolved. One of them is "what does it mean to call a function pure if it is being traced?" Actually, the simplest thing would be to adopt abstract patterns, which are *necessarily* pure and bounded, so there is nothing extra to check/enforce. There have been programming languages with 'effects' as part of their type system. To a degree, Mercury and Clean do that now. Erlang has a type system now, which it didn't before. Maybe in another decade it will have a type+effects system. From lukas@REDACTED Thu Feb 11 10:22:55 2016 From: lukas@REDACTED (Lukas Larsson) Date: Thu, 11 Feb 2016 10:22:55 +0100 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: <20160211001919.03d4f012@gmail.com> References: <20160201231049.08b28dc7@gmail.com> <20160211001919.03d4f012@gmail.com> Message-ID: Hello, Would you mind sending me the "perf script" output from your profiling runs so that I can have a closer look? For the record, CPU utilization is a terrible way to measure load on a system, especially if you are running on much less than full throttle. Because of how expensive it is to go to sleep, all the threads in the Erlang VM will spin and consume CPU when it is out of work and if you are running at 25% CPU then you are running out of work very often which will cause more spinning which will increase CPU while not decreasing the amount of work the system can do. One way to work around that is to compact the load of the schedulers better, so that instead of running at 25% on each scheduler, you run on 100% on one and 0% on the other three. The default config attempts to do this, but it is a trade off in how fast you want to be able to react to an increase in load and how much CPU you want to spend spinning, so it is not as aggressive as it could be. You can change how aggressive it is by setting the scheduler wakeup threshold so a higher value, i.e. "+swt very_high". This may reduce some of the time that you see in scheduler_wait when profiling. Another way to effect the wait time is by changing the scheduler busy wait threshold so that when the schedulers run out of work they will spin more or less, i.e. "+sbwt short" or "+sbwt long". You will have to experiment and see which is best for this specific benchmark. Fun fact, in virtualized environments it is crazy crazy crazy expensive to go to sleep. By compacting load we have seen systems get a reduction of up to 10% CPU usage without changing anything but where the work is scheduled. Lukas On Wed, Feb 10, 2016 at 9:49 PM, Ameretat Reith wrote: > On Tue, 9 Feb 2016 16:19:07 +0100 > Jesper Louis Andersen wrote: > > > Where is that time spent in the Erlang VM or in the Kernel? You are > > potentially on a wakeup schedule of 81 wakeups per millisecond to > > handle packets. Which suggests you need to understand where your CPU > > time is spent in the system in order to tune it for lower CPU usage. > > In more powerful machines that can handle 1Gbit/s with 50% CPU > utilization, It's VM's scheduler_wait consuming about 30% of CPU usage. > (xeon-e3 attachment) suggesting system can get more input. In > overloaded and less powerful machines, VM task management functions get > more resources and time spent in kernel is less than 20%. (corei3 > attachment) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Thu Feb 11 12:25:31 2016 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 11 Feb 2016 12:25:31 +0100 Subject: [erlang-questions] Proposal for new state machine engine: gen_statem In-Reply-To: References: <20160210125648.GA85287@erix.ericsson.se> Message-ID: <20160211112531.GA1373@erix.ericsson.se> On Wed, Feb 10, 2016 at 07:04:00PM -0800, Sargun Dhillon wrote: > Do you have a standalone implementation of gen_statem? It would be > great to test converting some existing code, and see how it feels. Here is a tarball containing 6 ebin/*.beam files to replace in lib/stdlib in an OTP-18.2 installation: http://erlang.org/~raimo/OTP-18.2-stdlib-ebin-gen_statem.tgz Keep the original files around. > > On Wed, Feb 10, 2016 at 4:56 AM, Raimo Niskanen > wrote: > > Hi all! > > > > I have created a pull request for a new state machine engine in the gen_* > > framework. There is a heated discussion right now on GitHUB: > > > > https://github.com/erlang/otp/pull/960 > > > > Please contribute with an opinion. > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From bchesneau@REDACTED Thu Feb 11 12:32:25 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 11 Feb 2016 11:32:25 +0000 Subject: [erlang-questions] Proposal for new state machine engine: gen_statem In-Reply-To: <20160211112531.GA1373@erix.ericsson.se> References: <20160210125648.GA85287@erix.ericsson.se> <20160211112531.GA1373@erix.ericsson.se> Message-ID: hrm how does it helps to figure the changes in the source code though? On Thu, 11 Feb 2016 at 12:25, Raimo Niskanen < raimo+erlang-questions@REDACTED> wrote: > On Wed, Feb 10, 2016 at 07:04:00PM -0800, Sargun Dhillon wrote: > > Do you have a standalone implementation of gen_statem? It would be > > great to test converting some existing code, and see how it feels. > > Here is a tarball containing 6 ebin/*.beam files to replace in lib/stdlib > in an OTP-18.2 installation: > http://erlang.org/~raimo/OTP-18.2-stdlib-ebin-gen_statem.tgz > > Keep the original files around. > > > > > On Wed, Feb 10, 2016 at 4:56 AM, Raimo Niskanen > > wrote: > > > Hi all! > > > > > > I have created a pull request for a new state machine engine in the > gen_* > > > framework. There is a heated discussion right now on GitHUB: > > > > > > https://github.com/erlang/otp/pull/960 > > > > > > Please contribute with an opinion. > > > -- > > > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangsiri@REDACTED Thu Feb 11 12:40:28 2016 From: erlangsiri@REDACTED (Siri Hansen) Date: Thu, 11 Feb 2016 12:40:28 +0100 Subject: [erlang-questions] Dependent module load before supervisor update In-Reply-To: <20160108114811.1d27e233@gmail.com> References: <20160108114811.1d27e233@gmail.com> Message-ID: Hi, in case you haven't already solved this... you could have a look at the generated relup file. It contains low level instructions, based on the high level instruction 'update' given in you appup. The low level instructions are in short: suspend load code_change resume for each supervisor. You could try to re-arrange these to do something like suspend suspend load load code_change code_change resume resume Note - this comes with absolutely no warranty... so be careful! Regards /siri 2016-01-08 9:18 GMT+01:00 Ameretat Reith : > I have appup instructions like this: > ` > [ > {load_module, child_sup}, > {update, parent_sup, supervisor}, > {update, child_sup, supervisor} > ] > > I'm intending to update two supervisor states but one of them > (parent_sup) needs upgraded code (a function changed) from child_sup > module (child_spec:spec) during its init invocation. > > So I want upgrade child_sup function interfaces before supervisor > upgrade of parent_sup and postpone child_spec init call (supervisor > upgrade) to after parent_sup fully upgraded. > > Now the problem is, in above instructions, I cannot use `load_module, > child_sup` or `load, {child_sup,...,..,}` getting error "Multiply > defined module: child_sup" and If without doing this, parent_sup won't > upgrade since It depend on upgraded interface functions from child_sup. > > So, Is there a way to control supervisor state upgrade and its module > loading during release upgrade? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Thu Feb 11 14:24:23 2016 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 11 Feb 2016 14:24:23 +0100 Subject: [erlang-questions] Proposal for new state machine engine: gen_statem In-Reply-To: References: <20160210125648.GA85287@erix.ericsson.se> <20160211112531.GA1373@erix.ericsson.se> Message-ID: <20160211132423.GB1373@erix.ericsson.se> On Thu, Feb 11, 2016 at 11:32:25AM +0000, Benoit Chesneau wrote: > hrm how does it helps to figure the changes in the source code though? It certainly does not. The OP asked for something to run. The source code can be found through the pull request: https://github.com/erlang/otp/pull/960 which points at a diff towards a branch in my GitHub repository: https://github.com/erlang/otp/compare/maint...RaimoNiskanen:raimo/new-gen-state-machine/OTP-13065 In the pull request discussion i have linked to ready-built documentation: http://erlang.org/~raimo/doc-7.2/lib/stdlib-2.7/doc/html/gen_statem.html which many in the discussion dislike, but it is a reference manual so it has to be complete and therefore looks complex. There are no simple examples. Yet. > On Thu, 11 Feb 2016 at 12:25, Raimo Niskanen < > raimo+erlang-questions@REDACTED> wrote: > > > On Wed, Feb 10, 2016 at 07:04:00PM -0800, Sargun Dhillon wrote: > > > Do you have a standalone implementation of gen_statem? It would be > > > great to test converting some existing code, and see how it feels. > > > > Here is a tarball containing 6 ebin/*.beam files to replace in lib/stdlib > > in an OTP-18.2 installation: > > http://erlang.org/~raimo/OTP-18.2-stdlib-ebin-gen_statem.tgz > > > > Keep the original files around. > > > > > > > > On Wed, Feb 10, 2016 at 4:56 AM, Raimo Niskanen > > > wrote: > > > > Hi all! > > > > > > > > I have created a pull request for a new state machine engine in the > > gen_* > > > > framework. There is a heated discussion right now on GitHUB: > > > > > > > > https://github.com/erlang/otp/pull/960 > > > > > > > > Please contribute with an opinion. > > > > -- > > > > > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > > > _______________________________________________ > > > > erlang-questions mailing list > > > > erlang-questions@REDACTED > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > > > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From jesper.louis.andersen@REDACTED Thu Feb 11 15:33:07 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 11 Feb 2016 15:33:07 +0100 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: References: <20160201231049.08b28dc7@gmail.com> <20160211001919.03d4f012@gmail.com> Message-ID: On Thu, Feb 11, 2016 at 10:22 AM, Lukas Larsson wrote: > For the record, CPU utilization is a terrible way to measure load on a > system, especially if you are running on much less than full throttle. > Because of how expensive it is to go to sleep, all the threads in the > Erlang VM will spin and consume CPU when it is out of work and if you are > running at 25% CPU then you are running out of work very often which will > cause more spinning which will increase CPU while not decreasing the amount > of work the system can do. This was exactly the same idea I had when I read that. Apart from what Lukas is mentioning, you may be able to avoid some TLB shootdowns by locking CPU cores to schedulers via +sbt db. For these kinds of situations, this can improve the CPU % usage as well since one core waiting for the L1,L2,L3 cache to populate counts as active CPU time. And if other cores are able to kill the TLB, you will have lots of traps to the OS and lots of higher CPU times. Another venture you may want to try is to tune the memory allocator a bit. If all the messages you receive have the same rough structure, you can win a lot by making it easier for the system to grab memory of that size. This is quite machine-specific however and usually don't win you much unless you are up against a wall--but that seems to be the case here. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From unix1@REDACTED Thu Feb 11 17:30:48 2016 From: unix1@REDACTED (Unix One) Date: Thu, 11 Feb 2016 16:30:48 +0000 Subject: [erlang-questions] Illegal Guard? In-Reply-To: <56BC31E3.7090804@cs.otago.ac.nz> References: <0734D55E371C4F35B23864E126BFCEE1@gruchalski.com> <56B93E7D.20508@cs.otago.ac.nz> <56BC31E3.7090804@cs.otago.ac.nz> Message-ID: On 02/10/2016 11:01 PM, Richard A. O'Keefe wrote: > I believe I mentioned that purity is not sufficient. > f(X) -> f(X). > is pure. But it is not bounded, and boundedness is of practical > importance. I understand. I was merely replying to the last point about retaining purity after hot loading. > We *could* have declarations like that. Currently we don't. > There are some technical problems which could probably be resolved. > One of them is "what does it mean to call a function pure if it is being > traced?" > > Actually, the simplest thing would be to adopt abstract patterns, which > are *necessarily* pure and bounded, so there is nothing extra to > check/enforce. > > There have been programming languages with 'effects' as part of their > type system. To a degree, Mercury and Clean do that now. Erlang has a > type system now, which it didn't before. Maybe in another decade it will > have a type+effects system. Interesting... I'll go do some more reading. Thank you. From vasdeveloper@REDACTED Thu Feb 11 18:43:32 2016 From: vasdeveloper@REDACTED (Theepan) Date: Thu, 11 Feb 2016 23:13:32 +0530 Subject: [erlang-questions] Machine Learning In-Reply-To: References: Message-ID: Thanks Jesper and other for the pointers.. My work is more on texts corpuses than on images. Will revert if there is something that I can share with the team.. :) Regards, Theepan On Wed, Feb 10, 2016 at 5:01 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > On Wed, Feb 10, 2016 at 10:34 AM, Samuel wrote: > >> I am not aware of existing ML or linear algrebra libraries in erlang >> that can be used to quick start an ML project, but just piping into >> tensorflow (or any other existing library/framework) isn't really >> doing ML with erlang, is it? You can as well just use tensorflow >> directly. >> > > The question is if this is a practical problem which needs solving or it > is for research. If you are researching how to construct, say, SVM or NNs, > then surely Erlang is a good vehicle. But in practice, there is a number of > things which makes Erlang unsuitable: > > * ML is often CPU bound. You don't want a bytecode interpreter to be a > limiting factor here. Even if the interpreter in Erlang is state-of-the-art > and highly optimized, it is not far fetched that a FP-intensive program > will be roughly a factor of 30 faster if compiled in a lower level language. > > * GPUs are popular in ML models for a reason: they speed up the FP > computations by a factor of 3000 or more. This in itself should hint you > that you need something else than Erlang. > > * Erlangs word overhead per process and terms means a lower-level model > can pack many more entities in memory. This affects caching behavior. > > Training of the model is often off-line and using the model is online in > the system. How you train your model is less important. This is why I'd > just outsource this problem to the libraries built and tuned for it. It is > like solving LinAlg problems but forgetting everything about existing > LAPACK and ATLAS routines in Fortran. > > A model, in Erlang, which could be viable is to use Erlang to produce > programs for lower level consumption by compilation. But these are problems > for which languages such as Haskell and OCaml dominates for a reason: their > type systems makes it far easier to pull off. > > > > -- > J. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vasdeveloper@REDACTED Thu Feb 11 19:02:51 2016 From: vasdeveloper@REDACTED (Theepan) Date: Thu, 11 Feb 2016 23:32:51 +0530 Subject: [erlang-questions] Illegal Guard? In-Reply-To: References: Message-ID: Richard, Why do you think "f(X, Y) when U = X+Y, U > 0 ; V = Y*X, V < 0 ->" is wrong? I do think that this is ugly, and diminishes the boundary between function body and function header, and leads to cluttered programs. But in a logical sense, in the same way input arguments are allocated private stack space (it is really heap at the OS level), even guards can create variables in the private stack; and apply logical conditions on them. *I am not advocating this as a good thing to do, but thinking out loud* Regards, Theepan On Sun, Feb 7, 2016 at 7:49 AM, wrote: > > Team - Any idea? > > What it says: you may not use '=' in a guard. > > Come to think of it, > > 60> is_integer(A = 5). > is pretty horrible no matter where it occurs; it's one of > the least likeable features of Erlang. > > As far as I know the reason '=' isn't allowed in guards > is to avoid things like > > f(X, Y) when U = X+Y, U > 0 ; V = Y*X, V < 0 -> > ... is it OK to use U here? or V? > > That problem can surely be solved, just like 'case': > if a variable is bound in every alternative of a guard, > it's bound, otherwise it's visible but unusable. > But before something like that could be adopted, > 'andalso' and 'orelse' and worse still 'and' and 'or' > were allowed into guards, and things would have got > even more tangled. > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Thu Feb 11 19:04:24 2016 From: lloyd@REDACTED (lloyd@REDACTED) Date: Thu, 11 Feb 2016 13:04:24 -0500 (EST) Subject: [erlang-questions] Machine Learning In-Reply-To: References: Message-ID: <1455213864.56144829@apps.rackspace.com> Hi Teepan, While I understand that Erlang may not deliver the performance needed for serious machine learning programs, I'd much welcome anything you can contribute toward demonstrating how machine learning ideas and algorithms can be implemented in Erlang. Tinker toy models, while not fast, can be very valuable learning tools. Given my aging brain and severe time constraints, I'd much rather play around with these ideas in a language I know than jumping across who knows how many syntactic boundaries to implement something in a black box. Bests wishes, LRP P.S. I'm also interested in processing text corpuses. So I'd also welcome anything you might contribute in this area. -----Original Message----- From: "Theepan" Sent: Thursday, February 11, 2016 12:43pm To: "Jesper Louis Andersen" Cc: "Erlang Questions Mailing List" Subject: Re: [erlang-questions] Machine Learning _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions Thanks Jesper and other for the pointers.. My work is more on texts corpuses than on images. Will revert if there is something that I can share with the team.. :) Regards, Theepan On Wed, Feb 10, 2016 at 5:01 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > On Wed, Feb 10, 2016 at 10:34 AM, Samuel wrote: > >> I am not aware of existing ML or linear algrebra libraries in erlang >> that can be used to quick start an ML project, but just piping into >> tensorflow (or any other existing library/framework) isn't really >> doing ML with erlang, is it? You can as well just use tensorflow >> directly. >> > > The question is if this is a practical problem which needs solving or it > is for research. If you are researching how to construct, say, SVM or NNs, > then surely Erlang is a good vehicle. But in practice, there is a number of > things which makes Erlang unsuitable: > > * ML is often CPU bound. You don't want a bytecode interpreter to be a > limiting factor here. Even if the interpreter in Erlang is state-of-the-art > and highly optimized, it is not far fetched that a FP-intensive program > will be roughly a factor of 30 faster if compiled in a lower level language. > > * GPUs are popular in ML models for a reason: they speed up the FP > computations by a factor of 3000 or more. This in itself should hint you > that you need something else than Erlang. > > * Erlangs word overhead per process and terms means a lower-level model > can pack many more entities in memory. This affects caching behavior. > > Training of the model is often off-line and using the model is online in > the system. How you train your model is less important. This is why I'd > just outsource this problem to the libraries built and tuned for it. It is > like solving LinAlg problems but forgetting everything about existing > LAPACK and ATLAS routines in Fortran. > > A model, in Erlang, which could be viable is to use Erlang to produce > programs for lower level consumption by compilation. But these are problems > for which languages such as Haskell and OCaml dominates for a reason: their > type systems makes it far easier to pull off. > > > > -- > J. > From ameretat.reith@REDACTED Thu Feb 11 20:17:17 2016 From: ameretat.reith@REDACTED (Ameretat Reith) Date: Thu, 11 Feb 2016 22:47:17 +0330 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: References: <20160201231049.08b28dc7@gmail.com> <20160211001919.03d4f012@gmail.com> Message-ID: <20160211224717.3a78bd4d@gmail.com> On Thu, 11 Feb 2016 10:22:55 +0100 Lukas Larsson wrote: > Would you mind sending me the "perf script" output from your > profiling runs so that I can have a closer look? [1] is for E3 case which used 50% of CPU to take 1Gbit/s traffic. > > For the record, CPU utilization is a terrible way to measure load on a > system, especially if you are running on much less than full throttle. > Because of how expensive it is to go to sleep, all the threads in the > Erlang VM will spin and consume CPU when it is out of work and if you > are running at 25% CPU then you are running out of work very often > which will cause more spinning which will increase CPU while not > decreasing the amount of work the system can do. While I was monitoring CPU on above test case, All cores had even and stable load, so I think scheduler turning off and on was not an issue here. I didn't included sasl in this testing application but for my real application I hardly can see scheduler utilization jumps when CPU core usage is stable around 50%. I agree that's not really useful for performance analysis since much time would spent for polling in that case. I made other samples in my local machine, server and client running on one machine, server opens N ports for client and client responds to each packet came from client, then client sends another packet when he get response for previous packet. I told VM to open 2 schedulers, (+S2 in vm.args). CPU is dual core with hyperthreading on. I open just open port, CPU usage is about 20%, cores utilization is not even and each one jump between 0 to 60. System load is stick to 1.4. Receive rate is about 290Mbit/s. perf script [2]. Opening two ports, CPU usage become around 50, still core usage jumps but this time more just 0 or 100. I don't know why perf saw just one scheduler. System load became stable at 1.8. Receive rate is about 690Mbit/s, much better than double of previous run. perf script [3]. By 10 ports, CPU usage is 100%, system load keep increasing and after 5 minutes seems won't exceed 3.9. Top CPU consumer is process_main by 11%. Receive rate reach 750Mbit/s. perf script [4]. To see how system react by much more load, I opened 1000 ports. Run queue sometimes reach 600 but on average is about 16 in server. Receive rate drops to 520Mbit/s. perf script [5] Thanks. 1:http://file.reith.ir/erludp/perf-script-e3-1gbit-50.out.bz2 2:http://file.reith.ir/erludp/perf-script-local-i3-4170-293mbits-1x.out.bz2 3:http://file.reith.ir/erludp/perf-script-local-i3-4170-690mbits-2x.out.bz2 4:http://file.reith.ir/erludp/perf-script-local-i3-4170-750mbits-10x.out.bz2 5:http://file.reith.ir/erludp/perf-script-local-i3-4170-520mbits-1kx.out.bz2 From ameretat.reith@REDACTED Thu Feb 11 20:36:55 2016 From: ameretat.reith@REDACTED (Ameretat Reith) Date: Thu, 11 Feb 2016 23:06:55 +0330 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: References: <20160201231049.08b28dc7@gmail.com> <20160211001919.03d4f012@gmail.com> Message-ID: <20160211230655.4def4df7@gmail.com> On Thu, 11 Feb 2016 15:33:07 +0100 Jesper Louis Andersen wrote: > > This was exactly the same idea I had when I read that. Apart from what > Lukas is mentioning, you may be able to avoid some TLB shootdowns by > locking CPU cores to schedulers via +sbt db. For these kinds of > situations, this can improve the CPU % usage as well since one core > waiting for the L1,L2,L3 cache to populate counts as active CPU time. > And if other cores are able to kill the TLB, you will have lots of > traps to the OS and lots of higher CPU times. Good idea, Thanks. I'll test on real application. > If all the messages you receive have the same rough structure, > you can win a lot by making it easier for the system to grab memory > of that size. I'm not aware of any option to hint VM to grab memory of some size, VM memory options I found, are mostly about carrier size and the way some carrier and location being chosen. From jesper.louis.andersen@REDACTED Thu Feb 11 21:16:56 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 11 Feb 2016 21:16:56 +0100 Subject: [erlang-questions] Heavy duty UDP server performance In-Reply-To: <20160211230655.4def4df7@gmail.com> References: <20160201231049.08b28dc7@gmail.com> <20160211001919.03d4f012@gmail.com> <20160211230655.4def4df7@gmail.com> Message-ID: On Thu, Feb 11, 2016 at 8:36 PM, Ameretat Reith wrote: > I'm not aware of any option to hint VM to grab memory of some size, VM > memory options I found, are mostly about carrier size and the way some > carrier and location being chosen. > Setting the carrier size so you get 2 megabyte hugepages/superpages can help since you have better TLB entries this way. And this means fewer loads once the TLB is shot. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Thu Feb 11 22:04:07 2016 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Thu, 11 Feb 2016 22:04:07 +0100 Subject: [erlang-questions] Discussion and proposal regarding rpc scalability Message-ID: Hello everyone, I was reading the publication "Investigating the Scalability Limits of Distributed Erlang " and one of the conclusions is: *> We observed that distributed Erlang scales linearly up to 150 nodes when no global command is made. Our results reveal that the latency of rpc calls rises as cluster size grows. This shows that spawn scales much better than rpc and using spawn instead of rpc in the sake of scalability is advised. * The reason why is highlighted in a previous section: *> To find out why rpc?s latency increases as the cluster size grows, we need to know more about rpc. (...) There is a generic server process (gen server) on each Erlang node which is named rex. This process is responsible for receiving and handling all rpc requests that come to an Erlang node. After handling the request, generated results will be returned to the source node. In addition to user applications, rpc is also used by many built-in OTP modules, and so it can be overloaded as a shared service.* In other words, the more applications we have relying on rpc, the more likely rpc will become a bottleneck and increase latency. I believe we have three options here: 1. Promote spawn over rpc, as the paper conclusion did (i.e. mention spawn in the rpc docs and so on) 2. Leave things as is 3. Allow "more scalable" usage of rpc by supporting application specific rpc instances In particular, my proposal for 3 is to allow developers to spawn their own rpc processes. In other words, we can expose: rpc:start_link(my_app_rpc) %% start your own rpc rpc:call({my_app_rpc, nodename}, foo, bar, [1, 2, 3]) %% invoke your own rpc at the given node This is a very simple solution that moves the bottleneck away from rpc's rex process since developers can place their own rpc processes in their application's tree. The code changes required to support this feature are also minimal and are almost all at the API level, i.e. support a tuple were today a node is expected or allow the name as argument, mimicking the same API provided by gen_server that rpc relies on. We won't change implementation details. Finally, I believe it will provide a more predictable usage of rpc. Feedback is appreciated! *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Thu Feb 11 22:17:47 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 11 Feb 2016 22:17:47 +0100 Subject: [erlang-questions] Discussion and proposal regarding rpc scalability In-Reply-To: References: Message-ID: Hi, I think #3 is a great idea. regards, Vlad On Thu, Feb 11, 2016 at 10:04 PM, Jos? Valim < jose.valim@REDACTED> wrote: > Hello everyone, > > I was reading the publication "Investigating the Scalability Limits of > Distributed Erlang > " and one of > the conclusions is: > > *> We observed that distributed Erlang scales linearly up to 150 nodes > when no global command is made. Our results reveal that the latency of rpc > calls rises as cluster size grows. This shows that spawn scales much better > than rpc and using spawn instead of rpc in the sake of scalability is > advised. * > > The reason why is highlighted in a previous section: > > *> To find out why rpc?s latency increases as the cluster size grows, we > need to know more about rpc. (...) There is a generic server process (gen > server) on each Erlang node which is named rex. This process is responsible > for receiving and handling all rpc requests that come to an Erlang node. > After handling the request, generated results will be returned to the > source node. In addition to user applications, rpc is also used by many > built-in OTP modules, and so it can be overloaded as a shared service.* > > In other words, the more applications we have relying on rpc, the more > likely rpc will become a bottleneck and increase latency. I believe we have > three options here: > > 1. Promote spawn over rpc, as the paper conclusion did (i.e. mention spawn > in the rpc docs and so on) > 2. Leave things as is > 3. Allow "more scalable" usage of rpc by supporting application specific > rpc instances > > In particular, my proposal for 3 is to allow developers to spawn their own > rpc processes. In other words, we can expose: > > rpc:start_link(my_app_rpc) %% start your own rpc > > rpc:call({my_app_rpc, nodename}, foo, bar, [1, 2, 3]) %% invoke your own > rpc at the given node > > > This is a very simple solution that moves the bottleneck away from rpc's > rex process since developers can place their own rpc processes in their > application's tree. The code changes required to support this feature are > also minimal and are almost all at the API level, i.e. support a tuple were > today a node is expected or allow the name as argument, mimicking the same > API provided by gen_server that rpc relies on. We won't change > implementation details. Finally, I believe it will provide a more > predictable usage of rpc. > > Feedback is appreciated! > > *Jos? Valim* > www.plataformatec.com.br > Skype: jv.ptec > Founder and Director of R&D > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From drohrer@REDACTED Thu Feb 11 22:23:55 2016 From: drohrer@REDACTED (Douglas Rohrer) Date: Thu, 11 Feb 2016 21:23:55 +0000 Subject: [erlang-questions] Discussion and proposal regarding rpc scalability In-Reply-To: References: Message-ID: As a Basho engineer, I'd be remiss if I didn't point out the opportunity for the use of some sort of consistent hashing of Pid -> rex instance, with a configurable number of rex instances per node. More generic and easily scalable (what happens when your app overloads its one instance of rex? now you need to go modify your code to add another N, etc.) Calling node would hash the calling PID to its REX instance, which would figure out the remote Pid's rex instance in a similar manner, send the request over, and coordinate the return results. rpc:call would remain the same in this case. Doug On Thu, Feb 11, 2016 at 4:04 PM Jos? Valim wrote: > Hello everyone, > > I was reading the publication "Investigating the Scalability Limits of > Distributed Erlang > " and one of > the conclusions is: > > *> We observed that distributed Erlang scales linearly up to 150 nodes > when no global command is made. Our results reveal that the latency of rpc > calls rises as cluster size grows. This shows that spawn scales much better > than rpc and using spawn instead of rpc in the sake of scalability is > advised. * > > The reason why is highlighted in a previous section: > > *> To find out why rpc?s latency increases as the cluster size grows, we > need to know more about rpc. (...) There is a generic server process (gen > server) on each Erlang node which is named rex. This process is responsible > for receiving and handling all rpc requests that come to an Erlang node. > After handling the request, generated results will be returned to the > source node. In addition to user applications, rpc is also used by many > built-in OTP modules, and so it can be overloaded as a shared service.* > > In other words, the more applications we have relying on rpc, the more > likely rpc will become a bottleneck and increase latency. I believe we have > three options here: > > 1. Promote spawn over rpc, as the paper conclusion did (i.e. mention spawn > in the rpc docs and so on) > 2. Leave things as is > 3. Allow "more scalable" usage of rpc by supporting application specific > rpc instances > > In particular, my proposal for 3 is to allow developers to spawn their own > rpc processes. In other words, we can expose: > > rpc:start_link(my_app_rpc) %% start your own rpc > > rpc:call({my_app_rpc, nodename}, foo, bar, [1, 2, 3]) %% invoke your own > rpc at the given node > > > This is a very simple solution that moves the bottleneck away from rpc's > rex process since developers can place their own rpc processes in their > application's tree. The code changes required to support this feature are > also minimal and are almost all at the API level, i.e. support a tuple were > today a node is expected or allow the name as argument, mimicking the same > API provided by gen_server that rpc relies on. We won't change > implementation details. Finally, I believe it will provide a more > predictable usage of rpc. > > Feedback is appreciated! > > *Jos? Valim* > www.plataformatec.com.br > Skype: jv.ptec > Founder and Director of R&D > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark.geib.44@REDACTED Thu Feb 11 22:25:08 2016 From: mark.geib.44@REDACTED (Mark Geib) Date: Thu, 11 Feb 2016 14:25:08 -0700 Subject: [erlang-questions] cowboy-swagger newbie question Message-ID: I am trying to evaluate the cowboy-swagger library which runs on top of cowboy-trails. So far pretty interesting. However I need some help in how to provide the swagger metadata for an endpoint where I have a root url like /applicaiton/rest/file/[:op]/[arg1]/[arg2] that I want to provide documentation for. I assume that I want to use the swagger ?basePath? and ?paths? attributes but can?t seem to make that work. I have tried the following in all the forms I can image: trails() -> MsgTrailsMetadata = #{basePath => ?/application/rest/file/", paths => [#{<<"/list">> => #{get => #{tags => ["file"], description => ?list available mpeg files", produces => ["application/json"] }}}, #{<<"/played">> => #{get => #{tags => ["file"], description => ?list played mpeg files", produces => ["application/json"]}}}] }, [trails:trail(?/application/rest/file/:op/[:arg1/[:arg2]]", ?MODULE, [], MsgTrailsMetadata)]. Any ideas are appreciated. Mark. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From fernando.benavides@REDACTED Thu Feb 11 22:44:14 2016 From: fernando.benavides@REDACTED (Brujo Benavides) Date: Thu, 11 Feb 2016 18:44:14 -0300 Subject: [erlang-questions] cowboy-swagger newbie question In-Reply-To: References: Message-ID: Hi Mark, This email will be very brief (read: mostly useless :P) because I am on vacation and I can't actually try your example from here :( Surely someone from inaka (Harry, Carlos, I am looking at you guys) will give you a better answer soon. In any case, the problem with your example seems to be that you are trying to provide the whole swagger metadata when just the metadata for the current endpoint is needed. In other words, you don't have to provide the basepath (that comes from trails:root_path(), or similar function) and 'paths' is automatically generated by cowboy-swagger. You just need to provide the metadata (including the parameters, specifically those in your path) for your endpoints. Hope this helps. On Feb 11, 2016 7:25 PM, "Mark Geib" wrote: > I am trying to evaluate the cowboy-swagger library which runs on top of > cowboy-trails. So far pretty interesting. However I need some help in how > to provide the swagger metadata for an endpoint where I have a root url > like /applicaiton/rest/file/[:op]/[arg1]/[arg2] that I want to provide > documentation for. I assume that I want to use the swagger ?basePath? and > ?paths? attributes but can?t seem to make that work. I have tried the > following in all the forms I can image: > > trails() -> > MsgTrailsMetadata = > #{basePath => ?/application/rest/file/", > paths => [#{<<"/list">> => > #{get => #{tags => ["file"], > description => > ?list available mpeg files", > produces => > ["application/json"] > }}}, > #{<<"/played">> => > #{get => #{tags => ["file"], > description => > ?list played mpeg files", > produces => > ["application/json"]}}}] > > }, > [trails:trail(?/application/rest/file/:op/[:arg1/[:arg2]]", ?MODULE, [], > MsgTrailsMetadata)]. > > Any ideas are appreciated. > > Mark. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark.geib.44@REDACTED Thu Feb 11 22:50:37 2016 From: mark.geib.44@REDACTED (Mark Geib) Date: Thu, 11 Feb 2016 14:50:37 -0700 Subject: [erlang-questions] cowboy-swagger newbie question In-Reply-To: References: Message-ID: <4C006017-70E0-42AC-92BD-9412E0EBF196@gmail.com> Thanks for the reply. However, I am a bit confused as to how I document the different operations then. I need to be able to provide metadata for more than one GET, one for ?list? and and one for ?played?. The parameters, for example, are different for each operation. BTW, I am new to RESTful endpoints, so it could be a poor choice for an example. Mark. > On Feb 11, 2016, at 2:44 PM, Brujo Benavides wrote: > > Hi Mark, > > This email will be very brief (read: mostly useless :P) because I am on vacation and I can't actually try your example from here :( > Surely someone from inaka (Harry, Carlos, I am looking at you guys) will give you a better answer soon. > In any case, the problem with your example seems to be that you are trying to provide the whole swagger metadata when just the metadata for the current endpoint is needed. > In other words, you don't have to provide the basepath (that comes from trails:root_path(), or similar function) and 'paths' is automatically generated by cowboy-swagger. You just need to provide the metadata (including the parameters, specifically those in your path) for your endpoints. > Hope this helps. > > On Feb 11, 2016 7:25 PM, "Mark Geib" > wrote: > I am trying to evaluate the cowboy-swagger library which runs on top of cowboy-trails. So far pretty interesting. However I need some help in how to provide the swagger metadata for an endpoint where I have a root url like /applicaiton/rest/file/[:op]/[arg1]/[arg2] that I want to provide documentation for. I assume that I want to use the swagger ?basePath? and ?paths? attributes but can?t seem to make that work. I have tried the following in all the forms I can image: > > trails() -> > MsgTrailsMetadata = > #{basePath => ?/application/rest/file/", > paths => [#{<<"/list">> => > #{get => #{tags => ["file"], > description => ?list available mpeg files", > produces => ["application/json"] > }}}, > #{<<"/played">> => > #{get => #{tags => ["file"], > description => ?list played mpeg files", > produces => ["application/json"]}}}] > > }, > [trails:trail(?/application/rest/file/:op/[:arg1/[:arg2]]", ?MODULE, [], MsgTrailsMetadata)]. > > Any ideas are appreciated. > > Mark. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 495 bytes Desc: Message signed with OpenPGP using GPGMail URL: From chandrashekhar.mullaparthi@REDACTED Thu Feb 11 22:57:34 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Thu, 11 Feb 2016 21:57:34 +0000 Subject: [erlang-questions] Discussion and proposal regarding rpc scalability In-Reply-To: References: Message-ID: Here is an alternative suggestion based on my experience of implementing our own rpc mechanism precisely because of the problems you mention. I alluded briefly to this during my CodeMesh presentation last year as well ( http://www.codemesh.io/static/upload/media/14478899074266codemesh2015.pdf) - slides 30-33 The idea is to remove the use of rex completely, and extend the RPC mechanism with these features. * Named RPC endpoints which allow connections to multiple nodes So supposed you have three erlang nodes which offer the same service, let's call them db@REDACTED, db@REDACTED, db@REDACTED, it would be nice to have a single RPC endpoint called DB, which sets up and maintains connections to all 3 nodes, and to be able to invoke it as. This would also give you your application specific RPC connections. rpc:call('DB', some_module, some_function, [args,...]) * Load balancing between these connections using different load balancing strategies * Seamless failover so that clients aren't aware that one of the nodes is down (if you really wanted to know, then you could use erlang:monitor_node/2,3) * Ability to use multiple TCP connections behind the same endpoint (and this could be easily extended to use TLS instead) I had implemented all this at my previous place of work and we had a lot of success with it. This was done as a separate application obviously, but it would be good to finally get rid of rex and its associated problems. cheers, Chandru On 11 February 2016 at 21:04, Jos? Valim wrote: > Hello everyone, > > I was reading the publication "Investigating the Scalability Limits of > Distributed Erlang > " and one of > the conclusions is: > > *> We observed that distributed Erlang scales linearly up to 150 nodes > when no global command is made. Our results reveal that the latency of rpc > calls rises as cluster size grows. This shows that spawn scales much better > than rpc and using spawn instead of rpc in the sake of scalability is > advised. * > > The reason why is highlighted in a previous section: > > *> To find out why rpc?s latency increases as the cluster size grows, we > need to know more about rpc. (...) There is a generic server process (gen > server) on each Erlang node which is named rex. This process is responsible > for receiving and handling all rpc requests that come to an Erlang node. > After handling the request, generated results will be returned to the > source node. In addition to user applications, rpc is also used by many > built-in OTP modules, and so it can be overloaded as a shared service.* > > In other words, the more applications we have relying on rpc, the more > likely rpc will become a bottleneck and increase latency. I believe we have > three options here: > > 1. Promote spawn over rpc, as the paper conclusion did (i.e. mention spawn > in the rpc docs and so on) > 2. Leave things as is > 3. Allow "more scalable" usage of rpc by supporting application specific > rpc instances > > In particular, my proposal for 3 is to allow developers to spawn their own > rpc processes. In other words, we can expose: > > rpc:start_link(my_app_rpc) %% start your own rpc > > rpc:call({my_app_rpc, nodename}, foo, bar, [1, 2, 3]) %% invoke your own > rpc at the given node > > > This is a very simple solution that moves the bottleneck away from rpc's > rex process since developers can place their own rpc processes in their > application's tree. The code changes required to support this feature are > also minimal and are almost all at the API level, i.e. support a tuple were > today a node is expected or allow the name as argument, mimicking the same > API provided by gen_server that rpc relies on. We won't change > implementation details. Finally, I believe it will provide a more > predictable usage of rpc. > > Feedback is appreciated! > > *Jos? Valim* > www.plataformatec.com.br > Skype: jv.ptec > Founder and Director of R&D > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From seancribbs@REDACTED Thu Feb 11 22:58:21 2016 From: seancribbs@REDACTED (Sean Cribbs) Date: Thu, 11 Feb 2016 15:58:21 -0600 Subject: [erlang-questions] Discussion and proposal regarding rpc scalability In-Reply-To: References: Message-ID: Jos?, It's interesting to me that your example of making registered RPC receivers isn't that much different from having registered processes with specific messages (call/cast/etc) they handle. What do you see as the use-case of allowing generic RPC in that scenario? Cheers, Sean On Thu, Feb 11, 2016 at 3:04 PM, Jos? Valim wrote: > Hello everyone, > > I was reading the publication "Investigating the Scalability Limits of > Distributed Erlang > " and one of > the conclusions is: > > *> We observed that distributed Erlang scales linearly up to 150 nodes > when no global command is made. Our results reveal that the latency of rpc > calls rises as cluster size grows. This shows that spawn scales much better > than rpc and using spawn instead of rpc in the sake of scalability is > advised. * > > The reason why is highlighted in a previous section: > > *> To find out why rpc?s latency increases as the cluster size grows, we > need to know more about rpc. (...) There is a generic server process (gen > server) on each Erlang node which is named rex. This process is responsible > for receiving and handling all rpc requests that come to an Erlang node. > After handling the request, generated results will be returned to the > source node. In addition to user applications, rpc is also used by many > built-in OTP modules, and so it can be overloaded as a shared service.* > > In other words, the more applications we have relying on rpc, the more > likely rpc will become a bottleneck and increase latency. I believe we have > three options here: > > 1. Promote spawn over rpc, as the paper conclusion did (i.e. mention spawn > in the rpc docs and so on) > 2. Leave things as is > 3. Allow "more scalable" usage of rpc by supporting application specific > rpc instances > > In particular, my proposal for 3 is to allow developers to spawn their own > rpc processes. In other words, we can expose: > > rpc:start_link(my_app_rpc) %% start your own rpc > > rpc:call({my_app_rpc, nodename}, foo, bar, [1, 2, 3]) %% invoke your own > rpc at the given node > > > This is a very simple solution that moves the bottleneck away from rpc's > rex process since developers can place their own rpc processes in their > application's tree. The code changes required to support this feature are > also minimal and are almost all at the API level, i.e. support a tuple were > today a node is expected or allow the name as argument, mimicking the same > API provided by gen_server that rpc relies on. We won't change > implementation details. Finally, I believe it will provide a more > predictable usage of rpc. > > Feedback is appreciated! > > *Jos? Valim* > www.plataformatec.com.br > Skype: jv.ptec > Founder and Director of R&D > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fernando.benavides@REDACTED Thu Feb 11 23:00:46 2016 From: fernando.benavides@REDACTED (Brujo Benavides) Date: Thu, 11 Feb 2016 19:00:46 -0300 Subject: [erlang-questions] cowboy-swagger newbie question In-Reply-To: <4C006017-70E0-42AC-92BD-9412E0EBF196@gmail.com> References: <4C006017-70E0-42AC-92BD-9412E0EBF196@gmail.com> Message-ID: Of course! I missed that part as well. The keys for the metada are the verbs. You can find examples of that on https://github.com/inaka/lsl and https://github.com/inaka/canillita If you have 'list' and 'played' as parts of your paths (i.e. Urls) then you have multiple trails (one for each) possibly served by the same handler (although I strongly recommend you to use 2 different ones). On Feb 11, 2016 7:50 PM, "Mark Geib" wrote: > Thanks for the reply. > > However, I am a bit confused as to how I document the different operations > then. I need to be able to provide metadata for more than one GET, one for > ?list? and and one for ?played?. The parameters, for example, are different > for each operation. > > BTW, I am new to RESTful endpoints, so it could be a poor choice for an > example. > > Mark. > > > On Feb 11, 2016, at 2:44 PM, Brujo Benavides < > fernando.benavides@REDACTED> wrote: > > Hi Mark, > > This email will be very brief (read: mostly useless :P) because I am on > vacation and I can't actually try your example from here :( > Surely someone from inaka (Harry, Carlos, I am looking at you guys) will > give you a better answer soon. > In any case, the problem with your example seems to be that you are trying > to provide the whole swagger metadata when just the metadata for the > current endpoint is needed. > In other words, you don't have to provide the basepath (that comes from > trails:root_path(), or similar function) and 'paths' is automatically > generated by cowboy-swagger. You just need to provide the metadata > (including the parameters, specifically those in your path) for your > endpoints. > Hope this helps. > On Feb 11, 2016 7:25 PM, "Mark Geib" wrote: > >> I am trying to evaluate the cowboy-swagger library which runs on top of >> cowboy-trails. So far pretty interesting. However I need some help in how >> to provide the swagger metadata for an endpoint where I have a root url >> like /applicaiton/rest/file/[:op]/[arg1]/[arg2] that I want to provide >> documentation for. I assume that I want to use the swagger ?basePath? and >> ?paths? attributes but can?t seem to make that work. I have tried the >> following in all the forms I can image: >> >> trails() -> >> MsgTrailsMetadata = >> #{basePath => ?/application/rest/file/", >> paths => [#{<<"/list">> => >> #{get => #{tags => ["file"], >> description => >> ?list available mpeg files", >> produces => >> ["application/json"] >> }}}, >> #{<<"/played">> => >> #{get => #{tags => ["file"], >> description => >> ?list played mpeg files", >> produces => >> ["application/json"]}}}] >> >> }, >> [trails:trail(?/application/rest/file/:op/[:arg1/[:arg2]]", ?MODULE, >> [], MsgTrailsMetadata)]. >> >> Any ideas are appreciated. >> >> Mark. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Fri Feb 12 02:14:41 2016 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Fri, 12 Feb 2016 02:14:41 +0100 Subject: [erlang-questions] Discussion and proposal regarding rpc scalability In-Reply-To: References: Message-ID: Thanks everyone for the feedback so far. Douglas, I thought about such solution. The problem is that the caller does not know how many rex instances there are per node. Therefore you would need to first message a single process that does the consistent hashing on the node, which is an additional step (extra copying) and may still be a bottleneck. How would you solve this particular problem? Chandru, thanks for validating the concerns raised. I agree your proposal has many improvements. IIRC the release project from where the paper came from explored similar ideas as well. I am afraid however such solution can't be a replacement. A lot of the usage of rex in OTP and other applications is to request information from a particular node and therefore the load balancing and node grouping wouldn't be desired. Sean, the idea is to leverage all of the rpc functionality without reimplementing it. Like the group leader handling, the call/async_call API and so forth. So while we could tell folks to use a gen_server (or to use spawn), we would be neglecting what rpc provides. In other words, I would use it for the same reason I use rpc, I'd just have it in my own app's tree. *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D On Thu, Feb 11, 2016 at 10:58 PM, Sean Cribbs wrote: > Jos?, > > It's interesting to me that your example of making registered RPC > receivers isn't that much different from having registered processes with > specific messages (call/cast/etc) they handle. What do you see as the > use-case of allowing generic RPC in that scenario? > > Cheers, > > Sean > > On Thu, Feb 11, 2016 at 3:04 PM, Jos? Valim < > jose.valim@REDACTED> wrote: > >> Hello everyone, >> >> I was reading the publication "Investigating the Scalability Limits of >> Distributed Erlang >> " and one of >> the conclusions is: >> >> *> We observed that distributed Erlang scales linearly up to 150 nodes >> when no global command is made. Our results reveal that the latency of rpc >> calls rises as cluster size grows. This shows that spawn scales much better >> than rpc and using spawn instead of rpc in the sake of scalability is >> advised. * >> >> The reason why is highlighted in a previous section: >> >> *> To find out why rpc?s latency increases as the cluster size grows, we >> need to know more about rpc. (...) There is a generic server process (gen >> server) on each Erlang node which is named rex. This process is responsible >> for receiving and handling all rpc requests that come to an Erlang node. >> After handling the request, generated results will be returned to the >> source node. In addition to user applications, rpc is also used by many >> built-in OTP modules, and so it can be overloaded as a shared service.* >> >> In other words, the more applications we have relying on rpc, the more >> likely rpc will become a bottleneck and increase latency. I believe we have >> three options here: >> >> 1. Promote spawn over rpc, as the paper conclusion did (i.e. mention >> spawn in the rpc docs and so on) >> 2. Leave things as is >> 3. Allow "more scalable" usage of rpc by supporting application specific >> rpc instances >> >> In particular, my proposal for 3 is to allow developers to spawn their >> own rpc processes. In other words, we can expose: >> >> rpc:start_link(my_app_rpc) %% start your own rpc >> >> rpc:call({my_app_rpc, nodename}, foo, bar, [1, 2, 3]) %% invoke your own >> rpc at the given node >> >> >> This is a very simple solution that moves the bottleneck away from rpc's >> rex process since developers can place their own rpc processes in their >> application's tree. The code changes required to support this feature are >> also minimal and are almost all at the API level, i.e. support a tuple were >> today a node is expected or allow the name as argument, mimicking the same >> API provided by gen_server that rpc relies on. We won't change >> implementation details. Finally, I believe it will provide a more >> predictable usage of rpc. >> >> Feedback is appreciated! >> >> *Jos? Valim* >> www.plataformatec.com.br >> Skype: jv.ptec >> Founder and Director of R&D >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Fri Feb 12 05:14:07 2016 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 11 Feb 2016 20:14:07 -0800 Subject: [erlang-questions] Discussion and proposal regarding rpc scalability In-Reply-To: References: Message-ID: <56BD5C0F.1010107@gmail.com> On 02/11/2016 05:14 PM, Jos? Valim wrote: > Thanks everyone for the feedback so far. > > Douglas, I thought about such solution. The problem is that the caller does not know how many rex instances there are per node. Therefore you would need to first message a single process that does the consistent hashing on the node, which is an additional step (extra copying) and may still be a bottleneck. How would you solve this particular problem? > > Chandru, thanks for validating the concerns raised. I agree your proposal has many improvements. IIRC the release project from where the paper came from explored similar ideas as well. I am afraid however such solution can't be a replacement. A lot of the usage of rex in OTP and other applications is to request information from a particular node and therefore the load balancing and node grouping wouldn't be desired. > > Sean, the idea is to leverage all of the rpc functionality without reimplementing it. Like the group leader handling, the call/async_call API and so forth. So while we could tell folks to use a gen_server (or to use spawn), we would be neglecting what rpc provides. In other words, I would use it for the same reason I use rpc, I'd just have it in my own app's tree. It sounds like you might be suggesting a gen_rpc behaviour which could be used instead of a gen_server behaviour, to provide functionality similar to the rpc module. > > > > > *Jos? Valim* > www.plataformatec.com.br > Skype: jv.ptec > Founder and Director of R&D > > On Thu, Feb 11, 2016 at 10:58 PM, Sean Cribbs > wrote: > > Jos?, > > It's interesting to me that your example of making registered RPC receivers isn't that much different from having registered processes with specific messages (call/cast/etc) they handle. What do you see as the use-case of allowing generic RPC in that scenario? > > Cheers, > > Sean > > On Thu, Feb 11, 2016 at 3:04 PM, Jos? Valim > wrote: > > Hello everyone, > > I was reading the publication "Investigating the Scalability Limits of Distributed Erlang " and one of the conclusions is: > > /> We observed that distributed Erlang scales linearly up to 150 nodes when no global command is made. Our results reveal that the latency of rpc calls rises as cluster size grows. This shows that spawn scales much better than rpc and using spawn instead of rpc in the sake of scalability is advised. / > > The reason why is highlighted in a previous section: > > /> To find out why rpc?s latency increases as the cluster size grows, we need to know more about rpc. (...) There is a generic server process (gen server) on each Erlang node which is named rex. This process is responsible for receiving and handling all rpc requests that come to an Erlang node. After handling the request, generated results will be returned to the source node. In addition to user applications, rpc is also used by many built-in OTP modules, and so it can be overloaded as a shared service./ > > In other words, the more applications we have relying on rpc, the more likely rpc will become a bottleneck and increase latency. I believe we have three options here: > > 1. Promote spawn over rpc, as the paper conclusion did (i.e. mention spawn in the rpc docs and so on) > 2. Leave things as is > 3. Allow "more scalable" usage of rpc by supporting application specific rpc instances > > In particular, my proposal for 3 is to allow developers to spawn their own rpc processes. In other words, we can expose: > > rpc:start_link(my_app_rpc) %% start your own rpc > > rpc:call({my_app_rpc, nodename}, foo, bar, [1, 2, 3]) %% invoke your own rpc at the given node > > > This is a very simple solution that moves the bottleneck away from rpc's rex process since developers can place their own rpc processes in their application's tree. The code changes required to support this feature are also minimal and are almost all at the API level, i.e. support a tuple were today a node is expected or allow the name as argument, mimicking the same API provided by gen_server that rpc relies on. We won't change implementation details. Finally, I believe it will provide a more predictable usage of rpc. > > Feedback is appreciated! > > *Jos? Valim* > www.plataformatec.com.br > Skype: jv.ptec > Founder and Director of R&D > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Fri Feb 12 07:26:01 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 12 Feb 2016 06:26:01 +0000 Subject: [erlang-questions] Discussion and proposal regarding rpc scalability In-Reply-To: References: Message-ID: Hrm wouldn'mtwould'nt the method in gen_rpc https://github.com/priestjim/gen_rpc partially fit the bill? - benoit On Fri, 12 Feb 2016 at 02:15, Jos? Valim wrote: > Thanks everyone for the feedback so far. > > Douglas, I thought about such solution. The problem is that the caller > does not know how many rex instances there are per node. Therefore you > would need to first message a single process that does the consistent > hashing on the node, which is an additional step (extra copying) and may > still be a bottleneck. How would you solve this particular problem? > > Chandru, thanks for validating the concerns raised. I agree your proposal > has many improvements. IIRC the release project from where the paper came > from explored similar ideas as well. I am afraid however such solution > can't be a replacement. A lot of the usage of rex in OTP and other > applications is to request information from a particular node and therefore > the load balancing and node grouping wouldn't be desired. > > Sean, the idea is to leverage all of the rpc functionality without > reimplementing it. Like the group leader handling, the call/async_call API > and so forth. So while we could tell folks to use a gen_server (or to use > spawn), we would be neglecting what rpc provides. In other words, I would > use it for the same reason I use rpc, I'd just have it in my own app's tree. > > > > > *Jos? Valim* > www.plataformatec.com.br > Skype: jv.ptec > Founder and Director of R&D > > On Thu, Feb 11, 2016 at 10:58 PM, Sean Cribbs > wrote: > >> Jos?, >> >> It's interesting to me that your example of making registered RPC >> receivers isn't that much different from having registered processes with >> specific messages (call/cast/etc) they handle. What do you see as the >> use-case of allowing generic RPC in that scenario? >> >> Cheers, >> >> Sean >> >> On Thu, Feb 11, 2016 at 3:04 PM, Jos? Valim < >> jose.valim@REDACTED> wrote: >> >>> Hello everyone, >>> >>> I was reading the publication "Investigating the Scalability Limits of >>> Distributed Erlang >>> " and one of >>> the conclusions is: >>> >>> *> We observed that distributed Erlang scales linearly up to 150 nodes >>> when no global command is made. Our results reveal that the latency of rpc >>> calls rises as cluster size grows. This shows that spawn scales much better >>> than rpc and using spawn instead of rpc in the sake of scalability is >>> advised. * >>> >>> The reason why is highlighted in a previous section: >>> >>> *> To find out why rpc?s latency increases as the cluster size grows, we >>> need to know more about rpc. (...) There is a generic server process (gen >>> server) on each Erlang node which is named rex. This process is responsible >>> for receiving and handling all rpc requests that come to an Erlang node. >>> After handling the request, generated results will be returned to the >>> source node. In addition to user applications, rpc is also used by many >>> built-in OTP modules, and so it can be overloaded as a shared service.* >>> >>> In other words, the more applications we have relying on rpc, the more >>> likely rpc will become a bottleneck and increase latency. I believe we have >>> three options here: >>> >>> 1. Promote spawn over rpc, as the paper conclusion did (i.e. mention >>> spawn in the rpc docs and so on) >>> 2. Leave things as is >>> 3. Allow "more scalable" usage of rpc by supporting application specific >>> rpc instances >>> >>> In particular, my proposal for 3 is to allow developers to spawn their >>> own rpc processes. In other words, we can expose: >>> >>> rpc:start_link(my_app_rpc) %% start your own rpc >>> >>> rpc:call({my_app_rpc, nodename}, foo, bar, [1, 2, 3]) %% invoke your own >>> rpc at the given node >>> >>> >>> This is a very simple solution that moves the bottleneck away from rpc's >>> rex process since developers can place their own rpc processes in their >>> application's tree. The code changes required to support this feature are >>> also minimal and are almost all at the API level, i.e. support a tuple were >>> today a node is expected or allow the name as argument, mimicking the same >>> API provided by gen_server that rpc relies on. We won't change >>> implementation details. Finally, I believe it will provide a more >>> predictable usage of rpc. >>> >>> Feedback is appreciated! >>> >>> *Jos? Valim* >>> www.plataformatec.com.br >>> Skype: jv.ptec >>> Founder and Director of R&D >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Fri Feb 12 09:36:06 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Fri, 12 Feb 2016 08:36:06 +0000 Subject: [erlang-questions] Discussion and proposal regarding rpc scalability In-Reply-To: References: Message-ID: Hi Jos?, On 12 February 2016 at 01:14, Jos? Valim wrote: > > Chandru, thanks for validating the concerns raised. I agree your proposal > has many improvements. IIRC the release project from where the paper came > from explored similar ideas as well. I am afraid however such solution > can't be a replacement. A lot of the usage of rex in OTP and other > applications is to request information from a particular node and therefore > the load balancing and node grouping wouldn't be desired. > Agreed that load balancing and node grouping are not of immediate use to OTP applications. But if someone is going to dig around the rpc internals, it would be a good idea to improve it a lot, rather than make it just slightly better. RPC hasn't had any radical improvements since it was first implemented - all I can remember is having a timeout feature added to it. The features I suggest can be achieved without affecting the API. > Sean, the idea is to leverage all of the rpc functionality without > reimplementing it. Like the group leader handling, the call/async_call API > and so forth. So while we could tell folks to use a gen_server (or to use > spawn), we would be neglecting what rpc provides. In other words, I would > use it for the same reason I use rpc, I'd just have it in my own app's tree. > > IMHO, the group leader handling feature is a bad idea in the context of RPC. Nice to have for some noddy use cases, but no one in their right mind would use that feature when handling large amounts of traffic. I would recommend deprecating it or making it an optional feature (off by default). Chandru > On Thu, Feb 11, 2016 at 10:58 PM, Sean Cribbs > wrote: > >> Jos?, >> >> It's interesting to me that your example of making registered RPC >> receivers isn't that much different from having registered processes with >> specific messages (call/cast/etc) they handle. What do you see as the >> use-case of allowing generic RPC in that scenario? >> >> Cheers, >> >> Sean >> >> On Thu, Feb 11, 2016 at 3:04 PM, Jos? Valim < >> jose.valim@REDACTED> wrote: >> >>> Hello everyone, >>> >>> I was reading the publication "Investigating the Scalability Limits of >>> Distributed Erlang >>> " and one of >>> the conclusions is: >>> >>> *> We observed that distributed Erlang scales linearly up to 150 nodes >>> when no global command is made. Our results reveal that the latency of rpc >>> calls rises as cluster size grows. This shows that spawn scales much better >>> than rpc and using spawn instead of rpc in the sake of scalability is >>> advised. * >>> >>> The reason why is highlighted in a previous section: >>> >>> *> To find out why rpc?s latency increases as the cluster size grows, we >>> need to know more about rpc. (...) There is a generic server process (gen >>> server) on each Erlang node which is named rex. This process is responsible >>> for receiving and handling all rpc requests that come to an Erlang node. >>> After handling the request, generated results will be returned to the >>> source node. In addition to user applications, rpc is also used by many >>> built-in OTP modules, and so it can be overloaded as a shared service.* >>> >>> In other words, the more applications we have relying on rpc, the more >>> likely rpc will become a bottleneck and increase latency. I believe we have >>> three options here: >>> >>> 1. Promote spawn over rpc, as the paper conclusion did (i.e. mention >>> spawn in the rpc docs and so on) >>> 2. Leave things as is >>> 3. Allow "more scalable" usage of rpc by supporting application specific >>> rpc instances >>> >>> In particular, my proposal for 3 is to allow developers to spawn their >>> own rpc processes. In other words, we can expose: >>> >>> rpc:start_link(my_app_rpc) %% start your own rpc >>> >>> rpc:call({my_app_rpc, nodename}, foo, bar, [1, 2, 3]) %% invoke your own >>> rpc at the given node >>> >>> >>> This is a very simple solution that moves the bottleneck away from rpc's >>> rex process since developers can place their own rpc processes in their >>> application's tree. The code changes required to support this feature are >>> also minimal and are almost all at the API level, i.e. support a tuple were >>> today a node is expected or allow the name as argument, mimicking the same >>> API provided by gen_server that rpc relies on. We won't change >>> implementation details. Finally, I believe it will provide a more >>> predictable usage of rpc. >>> >>> Feedback is appreciated! >>> >>> *Jos? Valim* >>> www.plataformatec.com.br >>> Skype: jv.ptec >>> Founder and Director of R&D >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Fri Feb 12 09:40:31 2016 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Fri, 12 Feb 2016 09:40:31 +0100 Subject: [erlang-questions] Discussion and proposal regarding rpc scalability In-Reply-To: References: Message-ID: Thanks Michael and Benoit. There are some interesting ideas in gen_rpc. In particular, they have one gen_server per node. This is quite smart. Instead of naming the process "rex", they name the process node(). This way, if I have nodes named A, B and C, each node will have processes A, B and C in them. RPC between nodes goes directly to those processes. This could potentially solve the bottlenecks mentioned above without requiring developers to start their own RPC. Btw, this leads to how we could introduce consistent hashing if desired. On the first request to a node, it won't have an entry in the local ETS table, we can request the node and ask how many local rpc processes it has and their PIDs. These are relatively simple solutions which could also be backwards compatible. At this point, we have a couple different solutions, each with different complexity. I personally prefer the consistent hashing one. I am definitely willing to work on any of the solutions above and I could draft a more detailed proposal. I just need some guidance from the OTP team about if and which solution they are more inclined to accept. *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D On Fri, Feb 12, 2016 at 7:26 AM, Benoit Chesneau wrote: > Hrm wouldn'mtwould'nt the method in gen_rpc > https://github.com/priestjim/gen_rpc > > partially fit the bill? > > - benoit > > On Fri, 12 Feb 2016 at 02:15, Jos? Valim > wrote: > >> Thanks everyone for the feedback so far. >> >> Douglas, I thought about such solution. The problem is that the caller >> does not know how many rex instances there are per node. Therefore you >> would need to first message a single process that does the consistent >> hashing on the node, which is an additional step (extra copying) and may >> still be a bottleneck. How would you solve this particular problem? >> >> Chandru, thanks for validating the concerns raised. I agree your proposal >> has many improvements. IIRC the release project from where the paper came >> from explored similar ideas as well. I am afraid however such solution >> can't be a replacement. A lot of the usage of rex in OTP and other >> applications is to request information from a particular node and therefore >> the load balancing and node grouping wouldn't be desired. >> >> Sean, the idea is to leverage all of the rpc functionality without >> reimplementing it. Like the group leader handling, the call/async_call API >> and so forth. So while we could tell folks to use a gen_server (or to use >> spawn), we would be neglecting what rpc provides. In other words, I would >> use it for the same reason I use rpc, I'd just have it in my own app's tree. >> >> >> >> >> *Jos? Valim* >> www.plataformatec.com.br >> Skype: jv.ptec >> Founder and Director of R&D >> >> On Thu, Feb 11, 2016 at 10:58 PM, Sean Cribbs >> wrote: >> >>> Jos?, >>> >>> It's interesting to me that your example of making registered RPC >>> receivers isn't that much different from having registered processes with >>> specific messages (call/cast/etc) they handle. What do you see as the >>> use-case of allowing generic RPC in that scenario? >>> >>> Cheers, >>> >>> Sean >>> >>> On Thu, Feb 11, 2016 at 3:04 PM, Jos? Valim < >>> jose.valim@REDACTED> wrote: >>> >>>> Hello everyone, >>>> >>>> I was reading the publication "Investigating the Scalability Limits of >>>> Distributed Erlang >>>> " and one >>>> of the conclusions is: >>>> >>>> *> We observed that distributed Erlang scales linearly up to 150 nodes >>>> when no global command is made. Our results reveal that the latency of rpc >>>> calls rises as cluster size grows. This shows that spawn scales much better >>>> than rpc and using spawn instead of rpc in the sake of scalability is >>>> advised. * >>>> >>>> The reason why is highlighted in a previous section: >>>> >>>> *> To find out why rpc?s latency increases as the cluster size grows, >>>> we need to know more about rpc. (...) There is a generic server process >>>> (gen server) on each Erlang node which is named rex. This process is >>>> responsible for receiving and handling all rpc requests that come to an >>>> Erlang node. After handling the request, generated results will be returned >>>> to the source node. In addition to user applications, rpc is also used by >>>> many built-in OTP modules, and so it can be overloaded as a shared service.* >>>> >>>> In other words, the more applications we have relying on rpc, the more >>>> likely rpc will become a bottleneck and increase latency. I believe we have >>>> three options here: >>>> >>>> 1. Promote spawn over rpc, as the paper conclusion did (i.e. mention >>>> spawn in the rpc docs and so on) >>>> 2. Leave things as is >>>> 3. Allow "more scalable" usage of rpc by supporting application >>>> specific rpc instances >>>> >>>> In particular, my proposal for 3 is to allow developers to spawn their >>>> own rpc processes. In other words, we can expose: >>>> >>>> rpc:start_link(my_app_rpc) %% start your own rpc >>>> >>>> rpc:call({my_app_rpc, nodename}, foo, bar, [1, 2, 3]) %% invoke your >>>> own rpc at the given node >>>> >>>> >>>> This is a very simple solution that moves the bottleneck away from >>>> rpc's rex process since developers can place their own rpc processes in >>>> their application's tree. The code changes required to support this feature >>>> are also minimal and are almost all at the API level, i.e. support a tuple >>>> were today a node is expected or allow the name as argument, mimicking the >>>> same API provided by gen_server that rpc relies on. We won't change >>>> implementation details. Finally, I believe it will provide a more >>>> predictable usage of rpc. >>>> >>>> Feedback is appreciated! >>>> >>>> *Jos? Valim* >>>> www.plataformatec.com.br >>>> Skype: jv.ptec >>>> Founder and Director of R&D >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Fri Feb 12 09:56:19 2016 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 12 Feb 2016 00:56:19 -0800 Subject: [erlang-questions] Discussion and proposal regarding rpc scalability In-Reply-To: References: Message-ID: <56BD9E33.9090104@gmail.com> On 02/12/2016 12:40 AM, Jos? Valim wrote: > Thanks Michael and Benoit. There are some interesting ideas in gen_rpc. > > In particular, they have one gen_server per node. This is quite smart. Instead of naming the process "rex", they name the process node(). This way, if I have nodes named A, B and C, each node will have processes A, B and C in them. RPC between nodes goes directly to those processes. This could potentially solve the bottlenecks mentioned above without requiring developers to start their own RPC. I think using node names to name the process, is a problem due to it being dynamic based on the erl command line arguments. If the name is dynamic, it can't be provided to the registered section of the .app file and any potential conflicts wouldn't be easy to understand. The argument for it, is that there is only one process like this, who cares? However, with a complex system, it is good to nail down dependencies to what can be static and easily understood in isolation. > > Btw, this leads to how we could introduce consistent hashing if desired. On the first request to a node, it won't have an entry in the local ETS table, we can request the node and ask how many local rpc processes it has and their PIDs. These are relatively simple solutions which could also be backwards compatible. > > At this point, we have a couple different solutions, each with different complexity. I personally prefer the consistent hashing one. I am definitely willing to work on any of the solutions above and I could draft a more detailed proposal. I just need some guidance from the OTP team about if and which solution they are more inclined to accept. If consistent hashing uses the via syntax for process naming, it can be decoupled from the rpc source code itself, so then it should be more flexible for reuse. The via syntax can be used to allocate more than 1 pid per name, and there is an example of doing so with cpg at https://github.com/okeuday/cpg/blob/master/test/cpg_test.erl#L82-L105 . > > > *Jos? Valim* > www.plataformatec.com.br > Skype: jv.ptec > Founder and Director of R&D > > On Fri, Feb 12, 2016 at 7:26 AM, Benoit Chesneau > wrote: > > Hrm wouldn'mtwould'nt the method in gen_rpc > https://github.com/priestjim/gen_rpc > > partially fit the bill? > > - benoit > > On Fri, 12 Feb 2016 at 02:15, Jos? Valim > wrote: > > Thanks everyone for the feedback so far. > > Douglas, I thought about such solution. The problem is that the caller does not know how many rex instances there are per node. Therefore you would need to first message a single process that does the consistent hashing on the node, which is an additional step (extra copying) and may still be a bottleneck. How would you solve this particular problem? > > Chandru, thanks for validating the concerns raised. I agree your proposal has many improvements. IIRC the release project from where the paper came from explored similar ideas as well. I am afraid however such solution can't be a replacement. A lot of the usage of rex in OTP and other applications is to request information from a particular node and therefore the load balancing and node grouping wouldn't be desired. > > Sean, the idea is to leverage all of the rpc functionality without reimplementing it. Like the group leader handling, the call/async_call API and so forth. So while we could tell folks to use a gen_server (or to use spawn), we would be neglecting what rpc provides. In other words, I would use it for the same reason I use rpc, I'd just have it in my own app's tree. > > > > > *Jos? Valim* > www.plataformatec.com.br > Skype: jv.ptec > Founder and Director of R&D > > On Thu, Feb 11, 2016 at 10:58 PM, Sean Cribbs > wrote: > > Jos?, > > It's interesting to me that your example of making registered RPC receivers isn't that much different from having registered processes with specific messages (call/cast/etc) they handle. What do you see as the use-case of allowing generic RPC in that scenario? > > Cheers, > > Sean > > On Thu, Feb 11, 2016 at 3:04 PM, Jos? Valim > wrote: > > Hello everyone, > > I was reading the publication "Investigating the Scalability Limits of Distributed Erlang " and one of the conclusions is: > > /> We observed that distributed Erlang scales linearly up to 150 nodes when no global command is made. Our results reveal that the latency of rpc calls rises as cluster size grows. This shows that spawn scales much better than rpc and using spawn instead of rpc in the sake of scalability is advised. / > > The reason why is highlighted in a previous section: > > /> To find out why rpc?s latency increases as the cluster size grows, we need to know more about rpc. (...) There is a generic server process (gen server) on each Erlang node which is named rex. This process is responsible for receiving and handling all rpc requests that come to an Erlang node. After handling the request, generated results will be returned to the source node. In addition to user applications, rpc is also used by many built-in OTP modules, and so it can be overloaded as a shared service./ > > In other words, the more applications we have relying on rpc, the more likely rpc will become a bottleneck and increase latency. I believe we have three options here: > > 1. Promote spawn over rpc, as the paper conclusion did (i.e. mention spawn in the rpc docs and so on) > 2. Leave things as is > 3. Allow "more scalable" usage of rpc by supporting application specific rpc instances > > In particular, my proposal for 3 is to allow developers to spawn their own rpc processes. In other words, we can expose: > > rpc:start_link(my_app_rpc) %% start your own rpc > > rpc:call({my_app_rpc, nodename}, foo, bar, [1, 2, 3]) %% invoke your own rpc at the given node > > > This is a very simple solution that moves the bottleneck away from rpc's rex process since developers can place their own rpc processes in their application's tree. The code changes required to support this feature are also minimal and are almost all at the API level, i.e. support a tuple were today a node is expected or allow the name as argument, mimicking the same API provided by gen_server that rpc relies on. We won't change implementation details. Finally, I believe it will provide a more predictable usage of rpc. > > Feedback is appreciated! > > *Jos? Valim* > www.plataformatec.com.br > Skype: jv.ptec > Founder and Director of R&D > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Fri Feb 12 15:19:20 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 12 Feb 2016 15:19:20 +0100 Subject: [erlang-questions] overriding code in another application Message-ID: Hi! I need to be able to distribute a version of the debugger in a way that works with R15+, and this is how I made it work: have a general my_debugger application that contains common code and dummy modules that are implemented in separate applications (my_debugger_15 to my_debugger_18), each compiled with the respective compiler. My idea is that I start 'my_debugger' and dynamically the right 'my_debugger_X' and the dummy modules will get reloaded from the _X application. So far it seems to work, but I wonder if there may be some details that I don't see and that would make this a no-go. I won't use releases, but the applications might get stopped. Has anyone ever tried or thought about this? best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From bdurksen@REDACTED Fri Feb 12 16:31:53 2016 From: bdurksen@REDACTED (Bill Durksen) Date: Fri, 12 Feb 2016 10:31:53 -0500 Subject: [erlang-questions] Maze alternate example for Rosetta Code Message-ID: <56BDFAE9.6050500@vaxxine.com> Hi there, To learn Erlang, I have recently switched from "learn you some Erlang" (http://learnyousomeerlang.com) to building actual programs. My first real Erlang program was a multi-process, multi-host, wxWidgets-based file-transfer application with 768 lines of Erlang code. I was surprised how simple it is to get this distributed functionality built, cleanly, in Erlang. Much appreciation goes to [erlang-questions]-subscribers, who guided me to various solutions when I needed an Erlang implementation of the Singleton design pattern. The 'ets' dictionary solution worked for me, after I figured it out. Currently, I am finishing a Maze-generation program (114 lines of packed Erlang code), and have put the final version up on the Rosetta Code web-site. (see the http://rosettacode.org/wiki/Maze_generation#Erlang ...Alternative example (implemented with 2 diagraphs)). From doing this experiment, I have a few questions: 1. The original Erlang maze-generation code (same Rosetta web-site page) used a light-weight process for each vertex in the maze. This is a very good example of how the Erlang process architecture can support many light-weight processes. I was surprised that it actually worked, until I tried generating a bigger maze. When I used the original Erlang maze example to generate a maze of 25 columns x 4000 rows, it brought my PC down to a crawl. I couldn't do anything (I am running Windows 7 64-bit on 9 GB of RAM) until I killed EPMD from Task Manager. Even after killing EPMD, it took quite a while for the computer to get back to normal again, which indicates that it may not have been EPMD causing the problem. Here is the question (part a): should I expect EPMD (or the WERL process) to slow to a crawl with 100,000 Erlang processes running on a single machine? (part b): If yes, what would be the Erlang programmer's defence tactic against this scenario? 2. 114 packed lines of Erlang code just seems to over-coded for such a powerful language. Not being an expert, yet, in all the nuances of Erlang, can someone suggest ways I can decrease maze.erl code down to 99 lines or less, without cheating (removing too many newlines)? Thanks, Bill -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Fri Feb 12 17:33:18 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Fri, 12 Feb 2016 08:33:18 -0800 Subject: [erlang-questions] Maze alternate example for Rosetta Code In-Reply-To: <56BDFAE9.6050500@vaxxine.com> References: <56BDFAE9.6050500@vaxxine.com> Message-ID: my suspicion (without having run that code yet) is that you're running out of memory somehow, and your computer's becoming slow as programs are getting paged in and out. You might try starting the program while windows' process monitor is running and watching the CPU and memory graphs to see if there are any unnatural or explosive spikes. If you've compiled with wx in your otp installation, then you could also start observer (type o() in the shell before kicking off your program) and watch the pretty lights while it runs. 10,000 erlang processes is not a heck of a lot, unless each of those processes, say, recurses over a megabyte of local state or, unbeknownst to you, starts another 100 processes each or tries to send every other process a message. epmd is, as you guess, super unlikely to be the root cause of any problems; it's just a name service. F. On Fri, Feb 12, 2016 at 7:31 AM, Bill Durksen wrote: > Hi there, > > To learn Erlang, I have recently switched from "learn you some Erlang" ( > http://learnyousomeerlang.com) to building actual programs. > > My first real Erlang program was a multi-process, multi-host, > wxWidgets-based file-transfer application with 768 lines of Erlang code. > I was surprised how simple it is to get this distributed functionality > built, cleanly, in Erlang. > Much appreciation goes to [erlang-questions]-subscribers, who guided me to > various solutions when I needed an Erlang implementation of the Singleton > design pattern. > The 'ets' dictionary solution worked for me, after I figured it out. > > Currently, I am finishing a Maze-generation program (114 lines of packed > Erlang code), and have put the final version up on the Rosetta Code > web-site. > (see the http://rosettacode.org/wiki/Maze_generation#Erlang > ...Alternative example (implemented with 2 diagraphs)). > > From doing this experiment, I have a few questions: > > 1. The original Erlang maze-generation code (same Rosetta web-site page) > used a light-weight process for each vertex in the maze. > This is a very good example of how the Erlang process architecture can > support many light-weight processes. > I was surprised that it actually worked, until I tried generating a > bigger maze. > When I used the original Erlang maze example to generate a maze of 25 > columns x 4000 rows, it brought my PC down to a crawl. > I couldn't do anything (I am running Windows 7 64-bit on 9 GB of RAM) > until I killed EPMD from Task Manager. > Even after killing EPMD, it took quite a while for the computer to get > back to normal again, which indicates that it may not have been EPMD > causing the problem. > > Here is the question (part a): should I expect EPMD (or the WERL > process) to slow to a crawl with 100,000 Erlang processes running on a > single machine? > (part b): If yes, what would be the > Erlang programmer's defence tactic against this scenario? > > 2. 114 packed lines of Erlang code just seems to over-coded for such a > powerful language. > Not being an expert, yet, in all the nuances of Erlang, can someone > suggest ways I can decrease maze.erl code down to 99 lines or less, without > cheating (removing too many newlines)? > > Thanks, > Bill > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From seancribbs@REDACTED Fri Feb 12 18:53:25 2016 From: seancribbs@REDACTED (Sean Cribbs) Date: Fri, 12 Feb 2016 11:53:25 -0600 Subject: [erlang-questions] Maze alternate example for Rosetta Code In-Reply-To: References: <56BDFAE9.6050500@vaxxine.com> Message-ID: Bill, I'm poking at a refactoring of it, but my first impression is that the author didn't know the memory/complexity implications of list concatenation (++) vs cons/reverse, elements of the standard library like sets (they used a fixed array), and various other things that make the code more confusing. As a result, there is probably some hidden waste. I'll post a refactoring to this later. I wouldn't be so concerned with LoC, but you might find that clearer, simpler implementations tend to have less code anyway. On Fri, Feb 12, 2016 at 10:33 AM, Felix Gallo wrote: > my suspicion (without having run that code yet) is that you're running out > of memory somehow, and your computer's becoming slow as programs are > getting paged in and out. You might try starting the program while > windows' process monitor is running and watching the CPU and memory graphs > to see if there are any unnatural or explosive spikes. If you've compiled > with wx in your otp installation, then you could also start observer (type > o() in the shell before kicking off your program) and watch the pretty > lights while it runs. > > 10,000 erlang processes is not a heck of a lot, unless each of those > processes, say, recurses over a megabyte of local state or, unbeknownst to > you, starts another 100 processes each or tries to send every other process > a message. > > epmd is, as you guess, super unlikely to be the root cause of any > problems; it's just a name service. > > F. > > On Fri, Feb 12, 2016 at 7:31 AM, Bill Durksen > wrote: > >> Hi there, >> >> To learn Erlang, I have recently switched from "learn you some Erlang" ( >> http://learnyousomeerlang.com) to building actual programs. >> >> My first real Erlang program was a multi-process, multi-host, >> wxWidgets-based file-transfer application with 768 lines of Erlang code. >> I was surprised how simple it is to get this distributed functionality >> built, cleanly, in Erlang. >> Much appreciation goes to [erlang-questions]-subscribers, who guided me >> to various solutions when I needed an Erlang implementation of the >> Singleton design pattern. >> The 'ets' dictionary solution worked for me, after I figured it out. >> >> Currently, I am finishing a Maze-generation program (114 lines of packed >> Erlang code), and have put the final version up on the Rosetta Code >> web-site. >> (see the http://rosettacode.org/wiki/Maze_generation#Erlang >> ...Alternative example (implemented with 2 diagraphs)). >> >> From doing this experiment, I have a few questions: >> >> 1. The original Erlang maze-generation code (same Rosetta web-site page) >> used a light-weight process for each vertex in the maze. >> This is a very good example of how the Erlang process architecture >> can support many light-weight processes. >> I was surprised that it actually worked, until I tried generating a >> bigger maze. >> When I used the original Erlang maze example to generate a maze of 25 >> columns x 4000 rows, it brought my PC down to a crawl. >> I couldn't do anything (I am running Windows 7 64-bit on 9 GB of RAM) >> until I killed EPMD from Task Manager. >> Even after killing EPMD, it took quite a while for the computer to >> get back to normal again, which indicates that it may not have been EPMD >> causing the problem. >> >> Here is the question (part a): should I expect EPMD (or the WERL >> process) to slow to a crawl with 100,000 Erlang processes running on a >> single machine? >> (part b): If yes, what would be >> the Erlang programmer's defence tactic against this scenario? >> >> 2. 114 packed lines of Erlang code just seems to over-coded for such a >> powerful language. >> Not being an expert, yet, in all the nuances of Erlang, can someone >> suggest ways I can decrease maze.erl code down to 99 lines or less, without >> cheating (removing too many newlines)? >> >> Thanks, >> Bill >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Sat Feb 13 04:32:59 2016 From: g@REDACTED (Garrett Smith) Date: Sat, 13 Feb 2016 03:32:59 +0000 Subject: [erlang-questions] Machine Learning In-Reply-To: <1455213864.56144829@apps.rackspace.com> References: <1455213864.56144829@apps.rackspace.com> Message-ID: Set Erlang aside for this problem and go use TensorFlow to learn about neural network training and applications. There is way too much to learn in this field to worry about how it might apply to our favorite language. As has already been said, Erlang is well suited for coordinating process automation using these toolsets, but the architecture of running complex models on CUDA and cuDNN is completely non trivial and the entire focus of libraries like TensorFlow, Theano, Caffe, etc. On Thu, Feb 11, 2016 at 12:04 PM wrote: > Hi Teepan, > > While I understand that Erlang may not deliver the performance needed for > serious machine learning programs, I'd much welcome anything you can > contribute toward demonstrating how machine learning ideas and algorithms > can be implemented in Erlang. > > Tinker toy models, while not fast, can be very valuable learning tools. > Given my aging brain and severe time constraints, I'd much rather play > around with these ideas in a language I know than jumping across who knows > how many syntactic boundaries to implement something in a black box. > > Bests wishes, > > LRP > > P.S. I'm also interested in processing text corpuses. So I'd also welcome > anything you might contribute in this area. > > > > -----Original Message----- > From: "Theepan" > Sent: Thursday, February 11, 2016 12:43pm > To: "Jesper Louis Andersen" > Cc: "Erlang Questions Mailing List" > Subject: Re: [erlang-questions] Machine Learning > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > Thanks Jesper and other for the pointers.. My work is more on texts > corpuses than on images. Will revert if there is something that I can share > with the team.. :) > > Regards, > Theepan > > > On Wed, Feb 10, 2016 at 5:01 PM, Jesper Louis Andersen < > jesper.louis.andersen@REDACTED> wrote: > > > > > On Wed, Feb 10, 2016 at 10:34 AM, Samuel wrote: > > > >> I am not aware of existing ML or linear algrebra libraries in erlang > >> that can be used to quick start an ML project, but just piping into > >> tensorflow (or any other existing library/framework) isn't really > >> doing ML with erlang, is it? You can as well just use tensorflow > >> directly. > >> > > > > The question is if this is a practical problem which needs solving or it > > is for research. If you are researching how to construct, say, SVM or > NNs, > > then surely Erlang is a good vehicle. But in practice, there is a number > of > > things which makes Erlang unsuitable: > > > > * ML is often CPU bound. You don't want a bytecode interpreter to be a > > limiting factor here. Even if the interpreter in Erlang is > state-of-the-art > > and highly optimized, it is not far fetched that a FP-intensive program > > will be roughly a factor of 30 faster if compiled in a lower level > language. > > > > * GPUs are popular in ML models for a reason: they speed up the FP > > computations by a factor of 3000 or more. This in itself should hint you > > that you need something else than Erlang. > > > > * Erlangs word overhead per process and terms means a lower-level model > > can pack many more entities in memory. This affects caching behavior. > > > > Training of the model is often off-line and using the model is online in > > the system. How you train your model is less important. This is why I'd > > just outsource this problem to the libraries built and tuned for it. It > is > > like solving LinAlg problems but forgetting everything about existing > > LAPACK and ATLAS routines in Fortran. > > > > A model, in Erlang, which could be viable is to use Erlang to produce > > programs for lower level consumption by compilation. But these are > problems > > for which languages such as Haskell and OCaml dominates for a reason: > their > > type systems makes it far easier to pull off. > > > > > > > > -- > > J. > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serge@REDACTED Sat Feb 13 14:31:31 2016 From: serge@REDACTED (Serge Aleynikov) Date: Sat, 13 Feb 2016 08:31:31 -0500 Subject: [erlang-questions] overriding code in another application In-Reply-To: References: Message-ID: ?How about compiling code by different compiler versions and putting the produced beams into version-specific directories. ?Then at run time either add the proper versioned directory's path into ERL_LIBS, provide it explicitely via "-pa", or add it dynamically at run-time via the code:add_patha/1 call? On Fri, Feb 12, 2016 at 9:19 AM, Vlad Dumitrescu wrote: > Hi! > > I need to be able to distribute a version of the debugger in a way that > works with R15+, and this is how I made it work: have a general my_debugger > application that contains common code and dummy modules that are > implemented in separate applications (my_debugger_15 to my_debugger_18), > each compiled with the respective compiler. > > My idea is that I start 'my_debugger' and dynamically the right > 'my_debugger_X' and the dummy modules will get reloaded from the _X > application. > > So far it seems to work, but I wonder if there may be some details that I > don't see and that would make this a no-go. I won't use releases, but the > applications might get stopped. Has anyone ever tried or thought about this? > > best regards, > Vlad > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Sat Feb 13 14:50:26 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Sat, 13 Feb 2016 14:50:26 +0100 Subject: [erlang-questions] overriding code in another application In-Reply-To: References: Message-ID: Hi Serge, This is more or less what I do, except that each version is compiled as a separate application. This makes it easy to build with rebar. I suppose that I don't really have to start those applications, just make the code available in the code path. Thanks! best regards, Vlad On Sat, Feb 13, 2016 at 2:31 PM, Serge Aleynikov wrote: > ?How about compiling code by different compiler versions and putting the > produced beams into version-specific directories. ?Then at run time either > add the proper versioned directory's path into ERL_LIBS, provide it > explicitely via "-pa", or add it dynamically at run-time via the > code:add_patha/1 call? > > On Fri, Feb 12, 2016 at 9:19 AM, Vlad Dumitrescu > wrote: > >> Hi! >> >> I need to be able to distribute a version of the debugger in a way that >> works with R15+, and this is how I made it work: have a general my_debugger >> application that contains common code and dummy modules that are >> implemented in separate applications (my_debugger_15 to my_debugger_18), >> each compiled with the respective compiler. >> >> My idea is that I start 'my_debugger' and dynamically the right >> 'my_debugger_X' and the dummy modules will get reloaded from the _X >> application. >> >> So far it seems to work, but I wonder if there may be some details that I >> don't see and that would make this a no-go. I won't use releases, but the >> applications might get stopped. Has anyone ever tried or thought about this? >> >> best regards, >> Vlad >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From setori88@REDACTED Sat Feb 13 18:20:08 2016 From: setori88@REDACTED (stewart mackenzie) Date: Sun, 14 Feb 2016 01:20:08 +0800 Subject: [erlang-questions] [Nix-dev] Erlang Package Manager In-Reply-To: <54A3D7D5.1080509@ninenines.eu> References: <54A3D7D5.1080509@ninenines.eu> Message-ID: Tristan, Gleb Peregud has implemented first class support for Erlang in Nix, pleas read the docs: http://hydra.nixos.org/build/31821726/download/1/nixpkgs/manual.html#users-guide-to-the-erlang-infrastructure /sjm From setori88@REDACTED Sat Feb 13 18:21:03 2016 From: setori88@REDACTED (stewart mackenzie) Date: Sun, 14 Feb 2016 01:21:03 +0800 Subject: [erlang-questions] [Nix-dev] Erlang Package Manager In-Reply-To: References: <1420037904.3286020.208308277.3B5341AF@webmail.messagingengine.com> Message-ID: Tristan, Gleb Peregud has implemented first class support for Erlang in Nix, pleas read the docs: http://hydra.nixos.org/build/31821726/download/1/nixpkgs/manual.html#users-guide-to-the-erlang-infrastructure /sjm From matwey.kornilov@REDACTED Sat Feb 13 22:02:40 2016 From: matwey.kornilov@REDACTED (Matwey V. Kornilov) Date: Sun, 14 Feb 2016 00:02:40 +0300 Subject: [erlang-questions] Longnames vs. shortnames (yet again) Message-ID: Hello, When I should use longnames and when I should use shortnames? net_kernel:create_hostpart/2 allows me to understand the difference in behavior, but I can't understand what is the purpose to have both longnames and shortnames. I suppose that there are different use-cases for longnames and shortnames. Unfortunately, Erlang documentation lacks clear explanation here. From ok@REDACTED Sun Feb 14 00:24:08 2016 From: ok@REDACTED (ok@REDACTED) Date: Sun, 14 Feb 2016 12:24:08 +1300 Subject: [erlang-questions] Maze alternate example for Rosetta Code In-Reply-To: References: <56BDFAE9.6050500@vaxxine.com> Message-ID: <485cb13c394e19016e3660c9cdb4753b.squirrel@chasm.otago.ac.nz> For my own purposes, I have completed about 624 of the 780 Rosetta Code problems. One issue with the Rosetta Code examples is that different authors have 'optimised' different things for different problems: brevity, clarity, speed, memory, minimised library use, maximised library use,... and there is no standard way for the authors to *tell* you what they were up to. From personal experience, sometimes a solution's author is just trying to get *something* done quickly and is actively ignoring every other consideration. You'll note that quite a lot of solutions are translations from other solutions: this is often a sign that the author was fed up and now wasn't even *trying* to produce idiomatic or efficient code. Sometimes the criteria are genuinely in conflict. Let's take a simple example: left factorial. 0 to: n-1 detectSum: [:k | k factorial] is a direct and idiomatic transcription of the requirements. n = 0 ifTrue: [0] ifFalse: [|x| x := 1. n-1 to: 1 by: -1 do: [:k | x := x*k+1]. x] is much less direct, but over the range of numbers considered in the problem it is an order of magnitude faster. What I'm saying is, don't be too quick to assume that the author of the existing Maze solution in Erlang didn't *know* about this or that feature of Erlang; maybe s/he was trying to produce something that could be understood by someone with a fairly basic knowledge of Erlang. (Rosetta Code *is* a 'chrestomathy' meaning that the solutions shouldn't be *too* esoteric.) From ok@REDACTED Sun Feb 14 00:37:08 2016 From: ok@REDACTED (ok@REDACTED) Date: Sun, 14 Feb 2016 12:37:08 +1300 Subject: [erlang-questions] Illegal Guard? In-Reply-To: References: Message-ID: <17bba60332e7778d7d5e004737da2bbd.squirrel@chasm.otago.ac.nz> > Richard, > > Why do you think "f(X, Y) when U = X+Y, U > 0 ; V = Y*X, V < 0 ->" is > wrong? Because the language specification and the compiler say it is. I said that something like that creates problems in the same way that case ?? of ?? -> U = X+Y ; ?? -> V = Y*X end creates problems. It can be addressed the same way. For what it's worth, I personally believe that allowing = in guards would be an improvement to the language. It would certainly make some guard macros easier to write. > > I do think that this is ugly, and diminishes the boundary between function > body and function header, and leads to cluttered programs. Leads? I'd say "can lead". A lot of things can lead to cluttered programs. But this is far from the most pressing issue in Erlang. From vasdeveloper@REDACTED Sun Feb 14 01:12:37 2016 From: vasdeveloper@REDACTED (Theepan) Date: Sun, 14 Feb 2016 05:42:37 +0530 Subject: [erlang-questions] Illegal Guard? In-Reply-To: <17bba60332e7778d7d5e004737da2bbd.squirrel@chasm.otago.ac.nz> References: <17bba60332e7778d7d5e004737da2bbd.squirrel@chasm.otago.ac.nz> Message-ID: > > > Why do you think "f(X, Y) when U = X+Y, U > 0 ; V = Y*X, V < 0 ->" is > > wrong? > > Because the language specification and the compiler say it is. > > Rewording my question :) - why it is wrong to allow such guard expressions at the compiler level? -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn@REDACTED Sun Feb 14 11:38:48 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Sun, 14 Feb 2016 11:38:48 +0100 Subject: [erlang-questions] Maze alternate example for Rosetta Code In-Reply-To: <56BDFAE9.6050500@vaxxine.com> References: <56BDFAE9.6050500@vaxxine.com> Message-ID: On Fri, Feb 12, 2016 at 4:31 PM, Bill Durksen wrote: > > 2. 114 packed lines of Erlang code just seems to over-coded for such a > powerful language. > Not being an expert, yet, in all the nuances of Erlang, can someone > suggest ways I can decrease maze.erl code down to 99 lines or less, without > cheating (removing too many newlines)? Here are few pointers for more idiomatic Erlang. Most of the time you want to use 'case' and not 'if'. Your code: has_neighbour(Maze, Row, Col, Direction) -> V = vertex_at(Row, Col, Maze), if V >= 0 -> Adjacent = adjacent_cell(Direction, V, Maze), if length(Adjacent) > 0 -> Neighbours = digraph:out_neighbours(Maze#maze.g, lists:nth(1, Adjacent)), lists:member(V, Neighbours); true -> false end; true -> false end. This can be rewritten to: has_neighbour(Maze, Row, Col, Direction) -> case vertex_at(Row, Col, Maze) of V when V >= 0 -> case adjacent_cell(Direction, V, Maze) of [Adjacent] -> Neighbours = digraph:out_neighbours(Maze#maze.g, Adjacent), lists:member(V, Neighbours); [] -> false end; _ -> false end. I also used pattern matching to avoid using length/1 and lists:nth/1. Another example. Here is your code: vertex_at(Row, Col, Maze) -> Cell_Exists = cell_exists(Row, Col, Maze), if Cell_Exists -> Row * Maze#maze.n + Col; true -> -1 end. You return -1 if there is no vertex. More idiomatic Erlang would be: vertex_at(Row, Col, Maze) -> case cell_exists(Row, Col, Maze) of true -> Row * Maze#maze.n + Col; false -> none end. or: vertex_at(Row, Col, Maze) -> case cell_exists(Row, Col, Maze) of true -> {ok,Row * Maze#maze.n + Col}; false -> error end. /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From eric.pailleau@REDACTED Sun Feb 14 11:57:51 2016 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sun, 14 Feb 2016 11:57:51 +0100 Subject: [erlang-questions] Longnames vs. shortnames (yet again) In-Reply-To: References: Message-ID: <56C05DAF.90701@wanadoo.fr> Le 13/02/2016 22:02, Matwey V. Kornilov a ?crit : > Hello, > > When I should use longnames and when I should use shortnames? > > net_kernel:create_hostpart/2 allows me to understand the difference in > behavior, but I can't understand what is the purpose to have both > longnames and shortnames. I suppose that there are different use-cases > for longnames and shortnames. Unfortunately, Erlang documentation lacks > clear explanation here. Hi, erl manual is clear on this point for -sname : " This is sometimes the only way to run distributed Erlang if the DNS (Domain Name System) is not running. " regards From matwey.kornilov@REDACTED Sun Feb 14 12:57:32 2016 From: matwey.kornilov@REDACTED (Matwey V. Kornilov) Date: Sun, 14 Feb 2016 14:57:32 +0300 Subject: [erlang-questions] Longnames vs. shortnames (yet again) In-Reply-To: <56C05DAF.90701@wanadoo.fr> References: <56C05DAF.90701@wanadoo.fr> Message-ID: 14.02.2016 13:57, PAILLEAU Eric ?????: > Le 13/02/2016 22:02, Matwey V. Kornilov a ?crit : >> Hello, >> >> When I should use longnames and when I should use shortnames? >> >> net_kernel:create_hostpart/2 allows me to understand the difference in >> behavior, but I can't understand what is the purpose to have both >> longnames and shortnames. I suppose that there are different use-cases >> for longnames and shortnames. Unfortunately, Erlang documentation lacks >> clear explanation here. > Hi, > erl manual is clear on this point for -sname : > " This is sometimes the only way to run distributed Erlang if the DNS > (Domain Name System) is not running. " Should I avoid using -sname until DNS is not running? From eric.pailleau@REDACTED Sun Feb 14 14:06:44 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Sun, 14 Feb 2016 14:06:44 +0100 Subject: [erlang-questions] Longnames vs. shortnames (yet again) In-Reply-To: Message-ID: Hi, dns timeout can result in a long node start-up and unexpected behaviour of nodes. Unless your Dns is high available, I would recommend to force always same IP to machines, and add entries in /etc/hosts. Regards Le?14 f?vr. 2016 12:57 PM, "Matwey V. Kornilov" a ?crit?: > > 14.02.2016 13:57, PAILLEAU Eric ?????: > > Le 13/02/2016 22:02, Matwey V. Kornilov a ?crit : > >> Hello, > >> > >> When I should use longnames and when I should use shortnames? > >> > >> net_kernel:create_hostpart/2 allows me to understand the difference in > >> behavior, but I can't understand what is the purpose to have both > >> longnames and shortnames. I suppose that there are different use-cases > >> for longnames and shortnames. Unfortunately, Erlang documentation lacks > >> clear explanation here. > > Hi, > > erl manual is clear on this point for -sname : > > " This? is? sometimes the only way to run distributed Erlang if the DNS > > (Domain Name System) is not running. " > > Should I avoid using -sname until DNS is not running? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From gleber.p@REDACTED Sun Feb 14 17:37:44 2016 From: gleber.p@REDACTED (Gleb Peregud) Date: Sun, 14 Feb 2016 17:37:44 +0100 Subject: [erlang-questions] [Nix-dev] Erlang Package Manager In-Reply-To: References: <54A3D7D5.1080509@ninenines.eu> Message-ID: Ah, missed this email before. Just wanted to point out that large chunk of this support was implemented by Eric B. Merritt, so at least half of kudos goes to him. Cheers Gleb On Feb 13, 2016 18:20, "stewart mackenzie" wrote: > Tristan, Gleb Peregud has implemented first class support for Erlang > in Nix, pleas read the docs: > > http://hydra.nixos.org/build/31821726/download/1/nixpkgs/manual.html#users-guide-to-the-erlang-infrastructure > > /sjm > _______________________________________________ > nix-dev mailing list > nix-dev@REDACTED > http://lists.science.uu.nl/mailman/listinfo/nix-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrallen1@REDACTED Sun Feb 14 21:19:55 2016 From: mrallen1@REDACTED (Mark Allen) Date: Sun, 14 Feb 2016 20:19:55 +0000 (UTC) Subject: [erlang-questions] Setting in lager the log level during runtime In-Reply-To: <56B7796B.80902@gmail.com> References: <56B7796B.80902@gmail.com> Message-ID: <463923366.4135845.1455481195079.JavaMail.yahoo@mail.yahoo.com> Hi. The OTP docs for gen_event:call state: If the specified event handler is not installed, the function returns?{error,bad_module}. I see that the application is loaded from the screen shot; did you start the lager application and a console backend before trying to set a new log level? ?"Starting" the application registers lager_console_backend as a gen_event handler. You may also want to use lager:trace_console/1 Thanks. Mark On Sunday, February 7, 2016 11:06 AM, Martin Koroudjiev wrote: Hello, I have to trace a bug on a production server running lager. I connected to the node and tried to change the log level to debug but got a strange error - bad_module. I've checked and lager is running. Here is a screenshot of the commands: Thanks in advance, Martin _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Mon Feb 15 06:25:18 2016 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 15 Feb 2016 06:25:18 +0100 Subject: [erlang-questions] Maze alternate example for Rosetta Code In-Reply-To: <485cb13c394e19016e3660c9cdb4753b.squirrel@chasm.otago.ac.nz> References: <56BDFAE9.6050500@vaxxine.com> <485cb13c394e19016e3660c9cdb4753b.squirrel@chasm.otago.ac.nz> Message-ID: <56C1613E.8040609@ericsson.com> Greetings, The short story about why the Erlang program for maze generation looks like it does: Historical reasons. The long story: I found Rosetta Code when I started to learn Python. After reading what I came there for I stayed, looking at tasks done in Erlang. They where very few. Lots of the missing Erlang tasks could be done directly with a function in the standard library. So I added those. Then there where some who could be done easily with the standard library and a little glue. So I added those. I found a table, sorted after which languages that had the most tasks done. Erlang was very low. I decided to add Erlang tasks until the table looked better. At one time I did some backtracking tasks. That is not something that I have ever done during 30 years of professional programming. It was difficult. Then came the maze solving task. To me it seemed much more natural to use parallel processes, instead of backtracking, to solve a maze. Directly after maze solving I did the maze generation. Having just used processes, I did it again. No example was very large so I did not notice the limitations. bengt On 02/14/2016 12:24 AM, ok@REDACTED wrote: > For my own purposes, I have completed about 624 of the 780 Rosetta Code > problems. One issue with the Rosetta Code examples is that different > authors have 'optimised' different things for different problems: brevity, > clarity, speed, memory, minimised library use, maximised library use,... > and there is no standard way for the authors to *tell* you what they were > up to. From personal experience, sometimes a solution's author is just > trying to get *something* done quickly and is actively ignoring every > other consideration. You'll note that quite a lot of solutions are > translations from other solutions: this is often a sign that the author > was fed up and now wasn't even *trying* to produce idiomatic or efficient > code. > > Sometimes the criteria are genuinely in conflict. Let's take a > simple example: left factorial. > 0 to: n-1 detectSum: [:k | k factorial] > is a direct and idiomatic transcription of the requirements. > n = 0 ifTrue: [0] ifFalse: [|x| > x := 1. > n-1 to: 1 by: -1 do: [:k | x := x*k+1]. > x] > is much less direct, but over the range of numbers considered in the > problem it is an order of magnitude faster. > > What I'm saying is, don't be too quick to assume that the author of > the existing Maze solution in Erlang didn't *know* about this or > that feature of Erlang; maybe s/he was trying to produce something > that could be understood by someone with a fairly basic knowledge > of Erlang. (Rosetta Code *is* a 'chrestomathy' meaning that the > solutions shouldn't be *too* esoteric.) > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mkbucc@REDACTED Mon Feb 15 14:35:26 2016 From: mkbucc@REDACTED (Mark Bucciarelli) Date: Mon, 15 Feb 2016 08:35:26 -0500 Subject: [erlang-questions] course/error_handling.html Message-ID: <20160215133526.GA26445@marks-mbp-3> Hi, I'd like to use some of the images at http://erlang.org/course/error_handling.html in a blog post I am writing. Is the copyright holder on this list and is it OK with you? If not, could someone put me in touch with him or her? Thanks, Mark From erlang@REDACTED Mon Feb 15 15:27:28 2016 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 15 Feb 2016 15:27:28 +0100 Subject: [erlang-questions] course/error_handling.html In-Reply-To: <20160215133526.GA26445@marks-mbp-3> References: <20160215133526.GA26445@marks-mbp-3> Message-ID: Goodness - Well I wrote it but do I own the copyright? - I don't know? I suspect Ericsson owns the copyright. I'm sure Ericsson would be delighted if you quote them. Open source Erlang is covered by an Apache 2.0 licence - so in absence of any counter-indicators I'd assume that anything on the web site has the same licence. Adding licenses and copyrights to all documents is something that I have not spent the last 40 years doing :-) It's nice of you to ask, BTW - this material has been quoted many times *without* any permission being asked for ... Cheers /Joe On Mon, Feb 15, 2016 at 2:35 PM, Mark Bucciarelli wrote: > Hi, > > I'd like to use some of the images at > http://erlang.org/course/error_handling.html in a blog post I am writing. > > Is the copyright holder on this list and is it OK with you? > > If not, could someone put me in touch with him or her? > > Thanks, > > Mark > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mrtndimitrov@REDACTED Mon Feb 15 15:44:52 2016 From: mrtndimitrov@REDACTED (Martin Koroudjiev) Date: Mon, 15 Feb 2016 16:44:52 +0200 Subject: [erlang-questions] Setting in lager the log level during runtime In-Reply-To: <463923366.4135845.1455481195079.JavaMail.yahoo@mail.yahoo.com> References: <56B7796B.80902@gmail.com> <463923366.4135845.1455481195079.JavaMail.yahoo@mail.yahoo.com> Message-ID: <56C1E464.9050002@gmail.com> Sorry I forgot to reply to the list. Yes, the problem was that on the production server, the console backend was not started. Regards, Martin On 2/14/2016 10:19 PM, Mark Allen wrote: > Hi. > > The OTP docs for gen_event:call state: > > If the specified event handler is not installed, the function > returns {error,bad_module}. > > I see that the application is loaded from the screen shot; did you > start the lager application and a console backend before trying to set > a new log level? "Starting" the application registers > lager_console_backend as a gen_event handler. You may also want to use > lager:trace_console/1 > > Thanks. > > Mark > > > > > On Sunday, February 7, 2016 11:06 AM, Martin Koroudjiev > wrote: > > > Hello, > > I have to trace a bug on a production server running lager. I > connected to the node and tried to change the log level to debug but > got a strange error - bad_module. > I've checked and lager is running. Here is a screenshot of the commands: > > > > Thanks in advance, > Martin > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Mon Feb 15 16:27:44 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 15 Feb 2016 16:27:44 +0100 Subject: [erlang-questions] Hot code upgrade with Erlang version change Message-ID: Dear all, Is it possible to do hot code upgrade going from 17.5 to 18.2.1? I couldn't find any clear info to this regards. Thank you, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierrefenoll@REDACTED Mon Feb 15 16:38:17 2016 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Mon, 15 Feb 2016 16:38:17 +0100 Subject: [erlang-questions] Updates, lenses, and why cross-module inlining would be nice In-Reply-To: <8216D951-C321-48B2-ADFC-FA0D2D5B7D9E@cs.otago.ac.nz> References: <1448544949.1214768.450737385.156BF7D2@webmail.messagingengine.com> <5657EA88.1010901@gmail.com> <1462F19A-19B6-4A16-A26D-4643BC0830CB@cs.otago.ac.nz> <565CA6AC.9020306@gmail.com> <32BC6909-E775-48B0-98EB-D89716098F61@cs.otago.ac.nz> <565E8D7F.5030106@gmail.com> <0AB5C64B-11D3-46A5-BB6A-AF275C5D7E08@cs.otago.ac.nz> <565FEB28.4090301@gmail.com> <8216D951-C321-48B2-ADFC-FA0D2D5B7D9E@cs.otago.ac.nz> Message-ID: This is surely a very naive idea on how to one day get cross-module inlining or at least enlarge the range of possibilities for the JIT. A lot of inlining is impossible due to the possibility to hot-swap code. But what about in the context of a release? When creating a release (or updating an old one), the compiler/relx/whatever is generating the release has a clear and finite scope of all the code that will be shipped and run. Maybe not a clear idea of extra dependencies or resources, but at least all the Erlang code should be there. Then a compiler would be able to do supercompilation, agressive inlining, ? I was thinking that we could then get a release that resembles an executable and if we can link external dependencies (OpenSSL, ?) we would then have a somewhat standalone binary that kind of works like escripts. Is my reasoning right? Where am I on the scale from extremely naive all the way up to ROK? Cheers, -- Pierre Fenoll On 7 December 2015 at 05:45, Richard A. O'Keefe wrote: > > On 3/12/2015, at 8:11 pm, Michael Truog wrote: > > If we had header files that were only allowed to contain functions and > > types that we called "template files" for lack of a better name > > (ideally, they would have their own export statements to help > > distinguish between private functions/types and the interface for modules > > to use) > > AND > > we had the include files (and template files) versioned within the beam > > output (to address the untracked dependency problem). > > > > Wouldn't that approach be preferable when compared > > to trying to manage the module dependency graph during a > > runtime code upgrades? Why would the "template files" approach > > not be sufficient? > > These are the differences I can see between 'template files' > and 'import_static modules'. > > (1) 'template files' would not be modules. > Having their own export directives would make them > modul*ar*, but they could not use 'fun M:F/A' to refer > to their own functions, having no M they could use. > > This does not seem like a good thing. > > (2) Headers can be nested. If 'template files' were to be > like headers, this would create nested scopes for top > level functions, and the possibility of multiple > distinct functions with the same name and arity and > in some sense "in" the very same module. > > I don't see that as an insuperable problem, but it's a > very big change to the language in order to avoid what > is really not likely to be a practical problem. > > (3) 'template files' are copied into the including module, > which allows different modules to have included > different versions of a 'template file'. > > Like headers, this DOES NOT SOLVE the dependency and > version skew problems, IT CREATES THOSE PROBLEMS. > > So with 'template files', either you have precisely the > same problems you have with headers plus a whole lot of > extra complexity, or you have to track dependencies anyway. > > If we can take a step away from Erlang, > we can see that we have a fundamental problem. > > Resource A is prepared from foundations x and c. > Resource B is prepared from foundations y and c. > Resources A and B have to "fit together" in some fashion. > Common foundation c has something to do with how that > fitting together works. > > c is revised to c'. > A is re-prepared to A'. > If B were re-prepared, B' and A' would be compatible, > just as B and A were compatible. > But B and A' are not compatible. > > As far as I can see, there are three general ways to > deal with this kind of problem. > > 1. Detection. When you try to use A' and B together, > detect that they were prepared from c' and c and > refuse to allow this use. > > This requires dependency tracking. > > 2. Prevention. When c is revised to c', use dependencies > forward and queue A and B for rebuilding. > > This requires dependency tracking. > > 3. Avoidance. Make the preparation step fairly trivial > so that whenever A (B) makes references to c, the > latest version of c is used. > > In programming language terms, this is not even lazy > evaluation, it's call-by-name. It's the way Erlang > currently handles remote calls. (More or less.) > > As a rule of thumb, early binding is the route to > efficiency (and early error detection), late binding > is the route to flexibility (and late error detection). > The performance cost may be anywhere between slight and > scary depending on the application. > > 3'. It is only necessary to provide the *appearance* of late > binding. I have a faint and probably unreliable memory > that the SPITBOL implementation of SNOBOL4 could generate > "optimised" code but could back it out and recompile it > when the assumptions it had made turned out to be wrong. > > Amongst other things, this requires the system to keep > track of this-depends-on-that at run time, so it's still > dependency tracking, but it need not be visible to the > programmer. > > What might be noticeable would be a performance blip as > loading c' caused everything that had been run-time compiled > against c to be de-optimised. TANSTAAFL. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierrefenoll@REDACTED Mon Feb 15 16:40:19 2016 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Mon, 15 Feb 2016 16:40:19 +0100 Subject: [erlang-questions] Hot code upgrade with Erlang version change In-Reply-To: References: Message-ID: Hello Roberto, Have a look at https://github.com/crownedgrouse/geas to help you understand what changes from one release to another. Cheers, -- Pierre Fenoll On 15 February 2016 at 16:27, Roberto Ostinelli wrote: > Dear all, > Is it possible to do hot code upgrade going from 17.5 to 18.2.1? I > couldn't find any clear info to this regards. > > Thank you, > r. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Mon Feb 15 16:47:54 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Mon, 15 Feb 2016 16:47:54 +0100 Subject: [erlang-questions] Hot code upgrade with Erlang version change In-Reply-To: References: Message-ID: <56C1F32A.3020300@ninenines.eu> From what I understand this involves the restart_emulator command. See 12.16 here: http://erlang.org/doc/design_principles/appup_cookbook.html On 02/15/2016 04:27 PM, Roberto Ostinelli wrote: > Dear all, > Is it possible to do hot code upgrade going from 17.5 to 18.2.1? I > couldn't find any clear info to this regards. > > Thank you, > r. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From roger@REDACTED Mon Feb 15 17:45:20 2016 From: roger@REDACTED (Roger Lipscombe) Date: Mon, 15 Feb 2016 16:45:20 +0000 Subject: [erlang-questions] TLS: signature algorithms extension Message-ID: Does Erlang support the signature algorithms extension in TLS 1.2 (https://tools.ietf.org/html/rfc5246#section-7.4.1.4.1)? Specifically, I've got two classes of client, one of which expects a SHA1-signed certificate, and one of which expects a SHA256-signed certificate. It appears that 'certfile' can only be specified once, and -- in testing -- it appears that the file can contain only one server certificate. Can we use Erlang SSL (via ranch, if it matters) to serve a different certificate based on the signature algorithms extension sent by the client (or, if absent, a default)? From roberto@REDACTED Mon Feb 15 18:49:31 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 15 Feb 2016 18:49:31 +0100 Subject: [erlang-questions] Hot code upgrade with Erlang version change In-Reply-To: <56C1F32A.3020300@ninenines.eu> References: <56C1F32A.3020300@ninenines.eu> Message-ID: Thank you all. On the same subject: if I upgrade some dependencies, I've noted that when I generate the .appup file with rebar it generates them for every dependency: ubuntu@REDACTED:rebar generate-appups previous_release=myapp-1.3.6 ==> rel (generate-appups) Generated appup for bear Generated appup for exometer_core Generated appup for folsom Generated appup for goldrush Generated appup for jiffy Generated appup for lager Generated appup for myapp Generated appup for parse_trans Generated appup for setup Appup generation complete If I follow the steps in https://github.com/rebar/rebar/wiki/Upgrades, will these dependencies also be hot code updated? On Mon, Feb 15, 2016 at 4:47 PM, Lo?c Hoguin wrote: > From what I understand this involves the restart_emulator command. See > 12.16 here: http://erlang.org/doc/design_principles/appup_cookbook.html > > > On 02/15/2016 04:27 PM, Roberto Ostinelli wrote: > >> Dear all, >> Is it possible to do hot code upgrade going from 17.5 to 18.2.1? I >> couldn't find any clear info to this regards. >> >> Thank you, >> r. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aschultz@REDACTED Tue Feb 16 08:55:11 2016 From: aschultz@REDACTED (Andreas Schultz) Date: Tue, 16 Feb 2016 08:55:11 +0100 (CET) Subject: [erlang-questions] TLS: signature algorithms extension In-Reply-To: References: Message-ID: <1084981133.465676.1455609311806.JavaMail.zimbra@tpip.net> Hi, ----- Original Message ----- > From: "Roger Lipscombe" > To: erlang-questions@REDACTED > Sent: Monday, February 15, 2016 5:45:20 PM > Subject: [erlang-questions] TLS: signature algorithms extension > Does Erlang support the signature algorithms extension in TLS 1.2 > (https://tools.ietf.org/html/rfc5246#section-7.4.1.4.1)? Yes, and it does announce (as client) all the hash and signature algorithms that it supports. > Specifically, I've got two classes of client, one of which expects a > SHA1-signed certificate, and one of which expects a SHA256-signed > certificate. > > It appears that 'certfile' can only be specified once, and -- in > testing -- it appears that the file can contain only one server > certificate. > > Can we use Erlang SSL (via ranch, if it matters) to serve a different > certificate based on the signature algorithms extension sent by the > client (or, if absent, a default)? The certificate to use is initialized before the handshake. So there is no support for selecting different certificates from a list of candidates. Andreas From ingela.andin@REDACTED Tue Feb 16 10:20:54 2016 From: ingela.andin@REDACTED (Ingela Andin) Date: Tue, 16 Feb 2016 10:20:54 +0100 Subject: [erlang-questions] TLS: signature algorithms extension In-Reply-To: <1084981133.465676.1455609311806.JavaMail.zimbra@tpip.net> References: <1084981133.465676.1455609311806.JavaMail.zimbra@tpip.net> Message-ID: Hi! I confirm what Andreas is saying. Of course it would be an interesting functionality to add more possibilities for the server to choose certificates, as for the sni option there is one such possibility. Regards Ingela Erlang/OTP team - Ericsson AB 2016-02-16 8:55 GMT+01:00 Andreas Schultz : > Hi, > > ----- Original Message ----- > > From: "Roger Lipscombe" > > To: erlang-questions@REDACTED > > Sent: Monday, February 15, 2016 5:45:20 PM > > Subject: [erlang-questions] TLS: signature algorithms extension > > > Does Erlang support the signature algorithms extension in TLS 1.2 > > (https://tools.ietf.org/html/rfc5246#section-7.4.1.4.1)? > > Yes, and it does announce (as client) all the hash and signature > algorithms that it supports. > > > Specifically, I've got two classes of client, one of which expects a > > SHA1-signed certificate, and one of which expects a SHA256-signed > > certificate. > > > > It appears that 'certfile' can only be specified once, and -- in > > testing -- it appears that the file can contain only one server > > certificate. > > > > Can we use Erlang SSL (via ranch, if it matters) to serve a different > > certificate based on the signature algorithms extension sent by the > > client (or, if absent, a default)? > > The certificate to use is initialized before the handshake. So there > is no support for selecting different certificates from a list of > candidates. > > Andreas > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Tue Feb 16 12:22:07 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 16 Feb 2016 12:22:07 +0100 Subject: [erlang-questions] Release upgrade of an app without a top supervisor Message-ID: Dear all, I'm building a release upgrade however I'm encountering an error during the release install: 1> release_handler:install_release("1.5.0"). <0.1497.0>: reason: {'EXIT',{timeout,{sys,get_status,[<0.1497.0>]}}} 11:58:30.647 [error] release_handler: cannot find top supervisor for application setup It looks like the application setup [1] does not have a top supervisor, hence the install fails. The app setup itself has not been upgraded, it's the same app; however, it is packaged as 1.5.0 in comparison to the 1.4.6 currently running in production. Is there any way I can go around this? Thanks, r. [1] https://github.com/uwiger/setup -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Tue Feb 16 12:27:41 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 16 Feb 2016 12:27:41 +0100 Subject: [erlang-questions] Release upgrade of an app without a top supervisor In-Reply-To: References: Message-ID: I found an old PR of Ulf here: https://github.com/uwiger/setup/pull/23 However I don't understand how to use the start_setup option. If any kind soul can help me out it would be great. Thanks, r. On Tue, Feb 16, 2016 at 12:22 PM, Roberto Ostinelli wrote: > Dear all, > I'm building a release upgrade however I'm encountering an error during > the release install: > > 1> release_handler:install_release("1.5.0"). > <0.1497.0>: reason: {'EXIT',{timeout,{sys,get_status,[<0.1497.0>]}}} > 11:58:30.647 [error] release_handler: cannot find top supervisor for > application setup > > It looks like the application setup [1] does not have a top supervisor, > hence the install fails. > > The app setup itself has not been upgraded, it's the same app; however, it > is packaged as 1.5.0 in comparison to the 1.4.6 currently running in > production. > > Is there any way I can go around this? > > Thanks, > r. > > [1] https://github.com/uwiger/setup > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Tue Feb 16 13:55:49 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 16 Feb 2016 13:55:49 +0100 Subject: [erlang-questions] Release upgrade of an app without a top supervisor In-Reply-To: <2D6F7186-12EE-439C-BA3E-D0BAACB04BF7@gmail.com> References: <2D6F7186-12EE-439C-BA3E-D0BAACB04BF7@gmail.com> Message-ID: Thank you Ulf. Anything I can do with an already running system, that obviously doesn't have that PR in? The only application using setup is, as you have guessed, exometer_core. On Tue, Feb 16, 2016 at 1:43 PM, Ulf Wiger wrote: > It?s true that setup didn?t play well with the release handler [blush]. > > That was addressed in the PR you mention. > > The start_setup option was added mainly because some applications (not > least exometer) use setup simply to expand and read environment variables, > and therefore do not require setup to be started. > > Off the top of my head, I can recall the following reasons to start setup > at all: > > - When using it for installation, setup generates a boot script that loads > all required apps and sets the code path, then starts setup. Thus, you can > fire up a node where only setup is running, but all code and environment > data are available. Setup then finds environment variables identifying code > hooks to run in different phases of the installation. A corresponding > process can be used for upgrades. > > - When making use of setup?s home(), data_dir() and log_dir(), i.e. > locations where applications can safely store data and logs. During > startup, setup will verify that those directories exist, creating them if > necessary. This behavior can be disabled with the env variable -setup > verify_directories true | false. > > - You could actually run code hooks even during normal startup. Whether or > not that?s a good idea is perhaps up for debate. It is possible, though. > > Predecessors of setup have done more during startup, including reading the > boot script and preparing to respond to various configuration queries. But > setup currently does little at startup, so most people might prefer it not > to start at all. Most functions - e.g. expanding environment variables or > upgrading applications on the fly - are usable even when setup isn?t > running. > > BR, > Ulf W > > On 16 Feb 2016, at 12:27, Roberto Ostinelli wrote: > > I found an old PR of Ulf here: > https://github.com/uwiger/setup/pull/23 > > However I don't understand how to use the start_setup option. If any kind > soul can help me out it would be great. > > Thanks, > r. > > > On Tue, Feb 16, 2016 at 12:22 PM, Roberto Ostinelli > wrote: > >> Dear all, >> I'm building a release upgrade however I'm encountering an error during >> the release install: >> >> 1> release_handler:install_release("1.5.0"). >> <0.1497.0>: reason: {'EXIT',{timeout,{sys,get_status,[<0.1497.0>]}}} >> 11:58:30.647 [error] release_handler: cannot find top supervisor for >> application setup >> >> It looks like the application setup [1] does not have a top supervisor, >> hence the install fails. >> >> The app setup itself has not been upgraded, it's the same app; however, >> it is packaged as 1.5.0 in comparison to the 1.4.6 currently running in >> production. >> >> Is there any way I can go around this? >> >> Thanks, >> r. >> >> [1] https://github.com/uwiger/setup >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Tue Feb 16 14:31:00 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 16 Feb 2016 14:31:00 +0100 Subject: [erlang-questions] Release upgrade of an app without a top supervisor In-Reply-To: <81A01FB9-22A4-42ED-A493-E2C6168A6A67@gmail.com> References: <2D6F7186-12EE-439C-BA3E-D0BAACB04BF7@gmail.com> <81A01FB9-22A4-42ED-A493-E2C6168A6A67@gmail.com> Message-ID: Done that already, to no avail unfortunately. On Tue, Feb 16, 2016 at 2:30 PM, Ulf Wiger wrote: > You could perhaps create a setup.appup file in setup-1.5.0/ebin/ with > roughly the following content (not tested): > > {?1.5.0", > [{?1.4.6", [{restart_application, setup}]}], > [{?1.4.6", [{restart_application, setup}]}] > }. > > BR, > Ulf W > > On 16 Feb 2016, at 13:55, Roberto Ostinelli wrote: > > Thank you Ulf. > Anything I can do with an already running system, that obviously doesn't > have that PR in? The only application using setup is, as you have guessed, > exometer_core. > > On Tue, Feb 16, 2016 at 1:43 PM, Ulf Wiger wrote: > >> It?s true that setup didn?t play well with the release handler [blush]. >> >> That was addressed in the PR you mention. >> >> The start_setup option was added mainly because some applications (not >> least exometer) use setup simply to expand and read environment variables, >> and therefore do not require setup to be started. >> >> Off the top of my head, I can recall the following reasons to start setup >> at all: >> >> - When using it for installation, setup generates a boot script that >> loads all required apps and sets the code path, then starts setup. Thus, >> you can fire up a node where only setup is running, but all code and >> environment data are available. Setup then finds environment variables >> identifying code hooks to run in different phases of the installation. A >> corresponding process can be used for upgrades. >> >> - When making use of setup?s home(), data_dir() and log_dir(), i.e. >> locations where applications can safely store data and logs. During >> startup, setup will verify that those directories exist, creating them if >> necessary. This behavior can be disabled with the env variable -setup >> verify_directories true | false. >> >> - You could actually run code hooks even during normal startup. Whether >> or not that?s a good idea is perhaps up for debate. It is possible, though. >> >> Predecessors of setup have done more during startup, including reading >> the boot script and preparing to respond to various configuration queries. >> But setup currently does little at startup, so most people might prefer it >> not to start at all. Most functions - e.g. expanding environment variables >> or upgrading applications on the fly - are usable even when setup isn?t >> running. >> >> BR, >> Ulf W >> >> On 16 Feb 2016, at 12:27, Roberto Ostinelli wrote: >> >> I found an old PR of Ulf here: >> https://github.com/uwiger/setup/pull/23 >> >> However I don't understand how to use the start_setup option. If any kind >> soul can help me out it would be great. >> >> Thanks, >> r. >> >> >> On Tue, Feb 16, 2016 at 12:22 PM, Roberto Ostinelli >> wrote: >> >>> Dear all, >>> I'm building a release upgrade however I'm encountering an error during >>> the release install: >>> >>> 1> release_handler:install_release("1.5.0"). >>> <0.1497.0>: reason: {'EXIT',{timeout,{sys,get_status,[<0.1497.0>]}}} >>> 11:58:30.647 [error] release_handler: cannot find top supervisor for >>> application setup >>> >>> It looks like the application setup [1] does not have a top supervisor, >>> hence the install fails. >>> >>> The app setup itself has not been upgraded, it's the same app; however, >>> it is packaged as 1.5.0 in comparison to the 1.4.6 currently running in >>> production. >>> >>> Is there any way I can go around this? >>> >>> Thanks, >>> r. >>> >>> [1] https://github.com/uwiger/setup >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From darach@REDACTED Tue Feb 16 15:00:24 2016 From: darach@REDACTED (Darach Ennis) Date: Tue, 16 Feb 2016 14:00:24 +0000 Subject: [erlang-questions] ANN: hdr-histogram-erl 0.3 Message-ID: Hi folk, A new release of HDR histogram for Erlang, Elixir and LFE with the following community contributions and fixes: * Bugfix for NIF allocated resource freeing by @timofey-barmin * Bugfix for CentOS by @mniec * Rebar3 support by @jlouis https://github.com/HdrHistogram/hdr_histogram_erl hdr-histogram-erl is a high dynamic range histogram that supports recording and analyzing sampled data points across a configurable range with configurable precision within that range. The precision is expressed as a number of significant figures in the recording. https://github.com/HdrHistogram/hdr_histogram_erl This HDR histogram implementation is designed for recording histograms of value measurements in latency sensitive environments. Although the native recording times can be as low as single digit nanoseconds there is added overhead in this wrapper/binding due to both the frontend overhead of converting from native C to the NIF interface, and the erlang overhead incurred calling into the NIFs. A distinct advantage of this histogram implementation is constant space and recording (time) overhead with an ability to recycle and reset instances whilst reclaiming already allocated space for reuse thereby reducing allocation cost and garbage collection overhead in the BEAM where repeated or continuous usage is likely. For example, a gen_server recording metrics continuously and resetting and logging histogram dumps on a periodic or other windowed basis. There's now quite a few folk using this library during development and production. If you depend on it in some way please ping and make yourself known. Very interested in collecting experiences and hearing how this can be improved for your use case. Enjoy! Cheers, Darach. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Tue Feb 16 16:28:57 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 16 Feb 2016 16:28:57 +0100 Subject: [erlang-questions] Release upgrade of an app without a top supervisor In-Reply-To: References: <2D6F7186-12EE-439C-BA3E-D0BAACB04BF7@gmail.com> <81A01FB9-22A4-42ED-A493-E2C6168A6A67@gmail.com> Message-ID: If anyone else needs a solution for this, here is mine. First, build your release upgrade package as you normally would. I use rebar, so I issue: $ rebar generate-appups previous_release=myapp-1.4.6 Modify your .appup files as needed, then generate the upgrade package: $ rebar generate-upgrade previous_release=myapp-1.4.6 Then, open up the upgrade package, and look for the relup file releases/1.5.0/relup. In your relup instructions, before the point_of_no_return instruction ensure to load the setup app: {load_object_code,{setup,"1.5.0",[setup,setup_gen,setup_lib]}}, Just after the point_of_no_return, ensure to stop it: {apply,{application,stop,[setup]}}, Finally, after all the other instructions ensure to start it: {apply,{application,start,[setup,permanent]}} Now, on a live system, check which applications are live: 1> application:which_applications(). [{myapp,"My App.","1.4.6"}, {exometer_core,"Code instrumentation and metrics collection package.", "1.4.6"}, {setup,"Generic setup application for Erlang-based systems","1.4.6"}, Then proceed to the normal upgrade: 2> release_handler:unpack_release("myapp_1.5.0"). 3> release_handler:install_release("1.5.0"). 4> release_handler:make_permanent("1.5.0"). Check applications again: 5> application:which_applications(). [{setup,"Generic setup application for Erlang-based systems","1.5.0"}, {myapp,"My app.","1.5.0"}, {exometer_core,"Code instrumentation and metrics collection package.", "1.5.0"}, You can see that it upgraded successfully. Hope this can help somebody else. Best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Tue Feb 16 17:04:33 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 16 Feb 2016 17:04:33 +0100 Subject: [erlang-questions] Release upgrade of an app without a top supervisor In-Reply-To: <47C4CBA6-4DC7-4ECB-877E-A10248817A97@gmail.com> References: <2D6F7186-12EE-439C-BA3E-D0BAACB04BF7@gmail.com> <81A01FB9-22A4-42ED-A493-E2C6168A6A67@gmail.com> <47C4CBA6-4DC7-4ECB-877E-A10248817A97@gmail.com> Message-ID: > > > I have merged the PR and tagged it 1.7. > Great news, Ulf, thank you for being so responsive :) Maybe you could consider upgrading the dependency for exometer_core too: https://github.com/Feuerlabs/exometer_core/blob/master/rebar.config Right now it's easy to do it manually anyways. > If you upgrade to 1.7 instead of 1.6, the suggested fix should work in the > more general case. > > That is, even restarting setup will not suffice, if the upgrade script > contains other applications that need upgrading by locating and suspending > processes. > > The reason is that the release_handler *will* query all running > applications for processes which will need to be suspended for a given > module, and since the top process in the setup application doesn?t respond > to system messages, it will always fail there. > True. Fortunately enough the only app using setup is exometer_core, so if my understanding is correct of what you stated here above, it should be fine. > With 1.7, it works as intended (you still need to upgrade to 1.7 using an > application restart, and this should be the first thing to do). > Hopefully my described approach should work. Best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Tue Feb 16 18:26:44 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 16 Feb 2016 18:26:44 +0100 Subject: [erlang-questions] Release upgrade of an app without a top supervisor In-Reply-To: References: <2D6F7186-12EE-439C-BA3E-D0BAACB04BF7@gmail.com> <81A01FB9-22A4-42ED-A493-E2C6168A6A67@gmail.com> <47C4CBA6-4DC7-4ECB-877E-A10248817A97@gmail.com> Message-ID: Thank you for this example Ulf. However, the modifications I've illustrated here below to the relup file instruct to stop the setup application. The release_handler will query the states only of currently running applications: https://github.com/erlang/otp/blob/maint/lib/sasl/src/release_handler_1.erl#L597 Hence, since the setup app is stopped, it does not query it. Probably that's why my example here below works? :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierrefenoll@REDACTED Wed Feb 17 08:20:15 2016 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Wed, 17 Feb 2016 08:20:15 +0100 Subject: [erlang-questions] Updates, lenses, and why cross-module inlining would be nice In-Reply-To: <56C40ADB.7000301@gmail.com> References: <1448544949.1214768.450737385.156BF7D2@webmail.messagingengine.com> <5657EA88.1010901@gmail.com> <1462F19A-19B6-4A16-A26D-4643BC0830CB@cs.otago.ac.nz> <565CA6AC.9020306@gmail.com> <32BC6909-E775-48B0-98EB-D89716098F61@cs.otago.ac.nz> <565E8D7F.5030106@gmail.com> <0AB5C64B-11D3-46A5-BB6A-AF275C5D7E08@cs.otago.ac.nz> <565FEB28.4090301@gmail.com> <8216D951-C321-48B2-ADFC-FA0D2D5B7D9E@cs.otago.ac.nz> <56C40ADB.7000301@gmail.com> Message-ID: Release inlining could be a compiler option only activated when desired. It would require no additional semantics to the language, only the additional lantency associated with swapping inlined code times the number of modules, though we may be able to go sublinear here. > On 17 Feb 2016, at 06:53, Michael Truog wrote: > >> On 02/15/2016 07:38 AM, Pierre Fenoll wrote: >> This is surely a very naive idea on how to one day get cross-module inlining >> or at least enlarge the range of possibilities for the JIT. >> >> A lot of inlining is impossible due to the possibility to hot-swap code. >> But what about in the context of a release? > Inlining isn't impossible due to the possibility to hot-swap code. However, inlining is simpler when modules are grouped together into a blob you can call a release. > > Inlining could rely on the inlined version of a function and a non-inlined version of the function where the inlined version of a function contains references to all the versions of various modules that it depends on. Any hot-code loading should then create a ripple effect, changing the rest of the system in a way not limited to the module being updated, which makes the module update process error-prone (if only due to the difference in latency with a module being indirectly changed due to a function inline) when abandoning the concept of isolation. > > Tracking the dependencies that can be cyclic should be a complexity that is at least linear, so as you add more inlined functions, the latency with a module update grows in at least a linear way. That means a module update could cause longer downtime due to a module update when many functions are inlined, where the downtime is worse as more functions get inlined, and that is only when considering the inlining in the old version of the module. When inlining is used, you have the tendency to group modules together, as you have described, to try to make sure everything is inlined as a group, so you preserve the lower latency function calls. If all the modules that are inlined together are updated together, you can minimize any problems dealing with non-inlined versions of the functions due to dependencies on old modules, but that would make the module update process longer, and would avoid the isolation a single module update currently provides. > > Forcing inlined modules to be updated in a big group might be nice with application versions, if you say functions can only be inlined within a single OTP application and not across OTP application boundaries, but as the documentation does show, changing an OTP application during runtime is ad-hoc since it depends on what you are doing to your supervisor(s) and everything else, while a single module update is a simple atomic change between two module versions. > > It is a really nice property of a module update, with the current situation in Erlang, where you know only the module will be changing when you update the source code. Keeping that isolation of change to make sure potential errors have a definite and clear scope is important. The problem of dealing with dependencies due to inlining is why I suggested using a templating concept instead, since that achieves inlining without making module updates error-prone. > >> >> When creating a release (or updating an old one), >> the compiler/relx/whatever is generating the release has a clear and finite scope >> of all the code that will be shipped and run. >> Maybe not a clear idea of extra dependencies or resources, >> but at least all the Erlang code should be there. >> Then a compiler would be able to do supercompilation, agressive inlining, ? >> >> I was thinking that we could then get a release that resembles an executable >> and if we can link external dependencies (OpenSSL, ?) >> we would then have a somewhat standalone binary that kind of works like escripts. >> >> Is my reasoning right? >> Where am I on the scale from extremely naive all the way up to ROK? >> >> >> Cheers, >> -- >> Pierre Fenoll >> >> >>> On 7 December 2015 at 05:45, Richard A. O'Keefe wrote: >>> >>> On 3/12/2015, at 8:11 pm, Michael Truog wrote: >>> > If we had header files that were only allowed to contain functions and >>> > types that we called "template files" for lack of a better name >>> > (ideally, they would have their own export statements to help >>> > distinguish between private functions/types and the interface for modules >>> > to use) >>> > AND >>> > we had the include files (and template files) versioned within the beam >>> > output (to address the untracked dependency problem). >>> > >>> > Wouldn't that approach be preferable when compared >>> > to trying to manage the module dependency graph during a >>> > runtime code upgrades? Why would the "template files" approach >>> > not be sufficient? >>> >>> These are the differences I can see between 'template files' >>> and 'import_static modules'. >>> >>> (1) 'template files' would not be modules. >>> Having their own export directives would make them >>> modul*ar*, but they could not use 'fun M:F/A' to refer >>> to their own functions, having no M they could use. >>> >>> This does not seem like a good thing. >>> >>> (2) Headers can be nested. If 'template files' were to be >>> like headers, this would create nested scopes for top >>> level functions, and the possibility of multiple >>> distinct functions with the same name and arity and >>> in some sense "in" the very same module. >>> >>> I don't see that as an insuperable problem, but it's a >>> very big change to the language in order to avoid what >>> is really not likely to be a practical problem. >>> >>> (3) 'template files' are copied into the including module, >>> which allows different modules to have included >>> different versions of a 'template file'. >>> >>> Like headers, this DOES NOT SOLVE the dependency and >>> version skew problems, IT CREATES THOSE PROBLEMS. >>> >>> So with 'template files', either you have precisely the >>> same problems you have with headers plus a whole lot of >>> extra complexity, or you have to track dependencies anyway. >>> >>> If we can take a step away from Erlang, >>> we can see that we have a fundamental problem. >>> >>> Resource A is prepared from foundations x and c. >>> Resource B is prepared from foundations y and c. >>> Resources A and B have to "fit together" in some fashion. >>> Common foundation c has something to do with how that >>> fitting together works. >>> >>> c is revised to c'. >>> A is re-prepared to A'. >>> If B were re-prepared, B' and A' would be compatible, >>> just as B and A were compatible. >>> But B and A' are not compatible. >>> >>> As far as I can see, there are three general ways to >>> deal with this kind of problem. >>> >>> 1. Detection. When you try to use A' and B together, >>> detect that they were prepared from c' and c and >>> refuse to allow this use. >>> >>> This requires dependency tracking. >>> >>> 2. Prevention. When c is revised to c', use dependencies >>> forward and queue A and B for rebuilding. >>> >>> This requires dependency tracking. >>> >>> 3. Avoidance. Make the preparation step fairly trivial >>> so that whenever A (B) makes references to c, the >>> latest version of c is used. >>> >>> In programming language terms, this is not even lazy >>> evaluation, it's call-by-name. It's the way Erlang >>> currently handles remote calls. (More or less.) >>> >>> As a rule of thumb, early binding is the route to >>> efficiency (and early error detection), late binding >>> is the route to flexibility (and late error detection). >>> The performance cost may be anywhere between slight and >>> scary depending on the application. >>> >>> 3'. It is only necessary to provide the *appearance* of late >>> binding. I have a faint and probably unreliable memory >>> that the SPITBOL implementation of SNOBOL4 could generate >>> "optimised" code but could back it out and recompile it >>> when the assumptions it had made turned out to be wrong. >>> >>> Amongst other things, this requires the system to keep >>> track of this-depends-on-that at run time, so it's still >>> dependency tracking, but it need not be visible to the >>> programmer. >>> >>> What might be noticeable would be a performance blip as >>> loading c' caused everything that had been run-time compiled >>> against c to be de-optimised. TANSTAAFL. >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Wed Feb 17 08:50:45 2016 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 16 Feb 2016 23:50:45 -0800 Subject: [erlang-questions] Updates, lenses, and why cross-module inlining would be nice In-Reply-To: References: <1448544949.1214768.450737385.156BF7D2@webmail.messagingengine.com> <5657EA88.1010901@gmail.com> <1462F19A-19B6-4A16-A26D-4643BC0830CB@cs.otago.ac.nz> <565CA6AC.9020306@gmail.com> <32BC6909-E775-48B0-98EB-D89716098F61@cs.otago.ac.nz> <565E8D7F.5030106@gmail.com> <0AB5C64B-11D3-46A5-BB6A-AF275C5D7E08@cs.otago.ac.nz> <565FEB28.4090301@gmail.com> <8216D951-C321-48B2-ADFC-FA0D2D5B7D9E@cs.otago.ac.nz> <56C40ADB.7000301@gmail.com> Message-ID: <56C42655.9070103@gmail.com> On 02/16/2016 11:20 PM, Pierre Fenoll wrote: > Release inlining could be a compiler option only activated when desired. > It would require no additional semantics to the language, only the additional lantency associated with swapping inlined code times the number of modules, though we may be able to go sublinear here. If you are unable to update a system with separate components (and instead need to update everything to make sure the system is able to function correctly), the difference between that and simply restarting the operating system process becomes negligible, unless you consider the extra complexity when not simply restarting the operating system process. There are other details like keeping ports open that could work when changing everything, or it might not. > > On 17 Feb 2016, at 06:53, Michael Truog > wrote: > >> On 02/15/2016 07:38 AM, Pierre Fenoll wrote: >>> This is surely a very naive idea on how to one day get cross-module inlining >>> or at least enlarge the range of possibilities for the JIT. >>> >>> A lot of inlining is impossible due to the possibility to hot-swap code. >>> But what about in the context of a release? >> Inlining isn't impossible due to the possibility to hot-swap code. However, inlining is simpler when modules are grouped together into a blob you can call a release. >> >> Inlining could rely on the inlined version of a function and a non-inlined version of the function where the inlined version of a function contains references to all the versions of various modules that it depends on. Any hot-code loading should then create a ripple effect, changing the rest of the system in a way not limited to the module being updated, which makes the module update process error-prone (if only due to the difference in latency with a module being indirectly changed due to a function inline) when abandoning the concept of isolation. >> >> Tracking the dependencies that can be cyclic should be a complexity that is at least linear, so as you add more inlined functions, the latency with a module update grows in at least a linear way. That means a module update could cause longer downtime due to a module update when many functions are inlined, where the downtime is worse as more functions get inlined, and that is only when considering the inlining in the old version of the module. When inlining is used, you have the tendency to group modules together, as you have described, to try to make sure everything is inlined as a group, so you preserve the lower latency function calls. If all the modules that are inlined together are updated together, you can minimize any problems dealing with non-inlined versions of the functions due to dependencies on old modules, but that would make the module update process longer, and would avoid the isolation a single module update currently provides. >> >> Forcing inlined modules to be updated in a big group might be nice with application versions, if you say functions can only be inlined within a single OTP application and not across OTP application boundaries, but as the documentation does show, changing an OTP application during runtime is ad-hoc since it depends on what you are doing to your supervisor(s) and everything else, while a single module update is a simple atomic change between two module versions. >> >> It is a really nice property of a module update, with the current situation in Erlang, where you know only the module will be changing when you update the source code. Keeping that isolation of change to make sure potential errors have a definite and clear scope is important. The problem of dealing with dependencies due to inlining is why I suggested using a templating concept instead, since that achieves inlining without making module updates error-prone. >> >>> >>> When creating a release (or updating an old one), >>> the compiler/relx/whatever is generating the release has a clear and finite scope >>> of all the code that will be shipped and run. >>> Maybe not a clear idea of extra dependencies or resources, >>> but at least all the Erlang code should be there. >>> Then a compiler would be able to do supercompilation, agressive inlining, ? >>> >>> I was thinking that we could then get a release that resembles an executable >>> and if we can link external dependencies (OpenSSL, ?) >>> we would then have a somewhat standalone binary that kind of works like escripts. >>> >>> Is my reasoning right? >>> Where am I on the scale from extremely naive all the way up to ROK? >>> >>> >>> Cheers, >>> -- >>> Pierre Fenoll >>> >>> >>> On 7 December 2015 at 05:45, Richard A. O'Keefe > wrote: >>> >>> >>> On 3/12/2015, at 8:11 pm, Michael Truog > wrote: >>> > If we had header files that were only allowed to contain functions and >>> > types that we called "template files" for lack of a better name >>> > (ideally, they would have their own export statements to help >>> > distinguish between private functions/types and the interface for modules >>> > to use) >>> > AND >>> > we had the include files (and template files) versioned within the beam >>> > output (to address the untracked dependency problem). >>> > >>> > Wouldn't that approach be preferable when compared >>> > to trying to manage the module dependency graph during a >>> > runtime code upgrades? Why would the "template files" approach >>> > not be sufficient? >>> >>> These are the differences I can see between 'template files' >>> and 'import_static modules'. >>> >>> (1) 'template files' would not be modules. >>> Having their own export directives would make them >>> modul*ar*, but they could not use 'fun M:F/A' to refer >>> to their own functions, having no M they could use. >>> >>> This does not seem like a good thing. >>> >>> (2) Headers can be nested. If 'template files' were to be >>> like headers, this would create nested scopes for top >>> level functions, and the possibility of multiple >>> distinct functions with the same name and arity and >>> in some sense "in" the very same module. >>> >>> I don't see that as an insuperable problem, but it's a >>> very big change to the language in order to avoid what >>> is really not likely to be a practical problem. >>> >>> (3) 'template files' are copied into the including module, >>> which allows different modules to have included >>> different versions of a 'template file'. >>> >>> Like headers, this DOES NOT SOLVE the dependency and >>> version skew problems, IT CREATES THOSE PROBLEMS. >>> >>> So with 'template files', either you have precisely the >>> same problems you have with headers plus a whole lot of >>> extra complexity, or you have to track dependencies anyway. >>> >>> If we can take a step away from Erlang, >>> we can see that we have a fundamental problem. >>> >>> Resource A is prepared from foundations x and c. >>> Resource B is prepared from foundations y and c. >>> Resources A and B have to "fit together" in some fashion. >>> Common foundation c has something to do with how that >>> fitting together works. >>> >>> c is revised to c'. >>> A is re-prepared to A'. >>> If B were re-prepared, B' and A' would be compatible, >>> just as B and A were compatible. >>> But B and A' are not compatible. >>> >>> As far as I can see, there are three general ways to >>> deal with this kind of problem. >>> >>> 1. Detection. When you try to use A' and B together, >>> detect that they were prepared from c' and c and >>> refuse to allow this use. >>> >>> This requires dependency tracking. >>> >>> 2. Prevention. When c is revised to c', use dependencies >>> forward and queue A and B for rebuilding. >>> >>> This requires dependency tracking. >>> >>> 3. Avoidance. Make the preparation step fairly trivial >>> so that whenever A (B) makes references to c, the >>> latest version of c is used. >>> >>> In programming language terms, this is not even lazy >>> evaluation, it's call-by-name. It's the way Erlang >>> currently handles remote calls. (More or less.) >>> >>> As a rule of thumb, early binding is the route to >>> efficiency (and early error detection), late binding >>> is the route to flexibility (and late error detection). >>> The performance cost may be anywhere between slight and >>> scary depending on the application. >>> >>> 3'. It is only necessary to provide the *appearance* of late >>> binding. I have a faint and probably unreliable memory >>> that the SPITBOL implementation of SNOBOL4 could generate >>> "optimised" code but could back it out and recompile it >>> when the assumptions it had made turned out to be wrong. >>> >>> Amongst other things, this requires the system to keep >>> track of this-depends-on-that at run time, so it's still >>> dependency tracking, but it need not be visible to the >>> programmer. >>> >>> What might be noticeable would be a performance blip as >>> loading c' caused everything that had been run-time compiled >>> against c to be de-optimised. TANSTAAFL. >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Wed Feb 17 06:53:31 2016 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 16 Feb 2016 21:53:31 -0800 Subject: [erlang-questions] Updates, lenses, and why cross-module inlining would be nice In-Reply-To: References: <1448544949.1214768.450737385.156BF7D2@webmail.messagingengine.com> <5657EA88.1010901@gmail.com> <1462F19A-19B6-4A16-A26D-4643BC0830CB@cs.otago.ac.nz> <565CA6AC.9020306@gmail.com> <32BC6909-E775-48B0-98EB-D89716098F61@cs.otago.ac.nz> <565E8D7F.5030106@gmail.com> <0AB5C64B-11D3-46A5-BB6A-AF275C5D7E08@cs.otago.ac.nz> <565FEB28.4090301@gmail.com> <8216D951-C321-48B2-ADFC-FA0D2D5B7D9E@cs.otago.ac.nz> Message-ID: <56C40ADB.7000301@gmail.com> On 02/15/2016 07:38 AM, Pierre Fenoll wrote: > This is surely a very naive idea on how to one day get cross-module inlining > or at least enlarge the range of possibilities for the JIT. > > A lot of inlining is impossible due to the possibility to hot-swap code. > But what about in the context of a release? Inlining isn't impossible due to the possibility to hot-swap code. However, inlining is simpler when modules are grouped together into a blob you can call a release. Inlining could rely on the inlined version of a function and a non-inlined version of the function where the inlined version of a function contains references to all the versions of various modules that it depends on. Any hot-code loading should then create a ripple effect, changing the rest of the system in a way not limited to the module being updated, which makes the module update process error-prone (if only due to the difference in latency with a module being indirectly changed due to a function inline) when abandoning the concept of isolation. Tracking the dependencies that can be cyclic should be a complexity that is at least linear, so as you add more inlined functions, the latency with a module update grows in at least a linear way. That means a module update could cause longer downtime due to a module update when many functions are inlined, where the downtime is worse as more functions get inlined, and that is only when considering the inlining in the old version of the module. When inlining is used, you have the tendency to group modules together, as you have described, to try to make sure everything is inlined as a group, so you preserve the lower latency function calls. If all the modules that are inlined together are updated together, you can minimize any problems dealing with non-inlined versions of the functions due to dependencies on old modules, but that would make the module update process longer, and would avoid the isolation a single module update currently provides. Forcing inlined modules to be updated in a big group might be nice with application versions, if you say functions can only be inlined within a single OTP application and not across OTP application boundaries, but as the documentation does show, changing an OTP application during runtime is ad-hoc since it depends on what you are doing to your supervisor(s) and everything else, while a single module update is a simple atomic change between two module versions. It is a really nice property of a module update, with the current situation in Erlang, where you know only the module will be changing when you update the source code. Keeping that isolation of change to make sure potential errors have a definite and clear scope is important. The problem of dealing with dependencies due to inlining is why I suggested using a templating concept instead, since that achieves inlining without making module updates error-prone. > > When creating a release (or updating an old one), > the compiler/relx/whatever is generating the release has a clear and finite scope > of all the code that will be shipped and run. > Maybe not a clear idea of extra dependencies or resources, > but at least all the Erlang code should be there. > Then a compiler would be able to do supercompilation, agressive inlining, ? > > I was thinking that we could then get a release that resembles an executable > and if we can link external dependencies (OpenSSL, ?) > we would then have a somewhat standalone binary that kind of works like escripts. > > Is my reasoning right? > Where am I on the scale from extremely naive all the way up to ROK? > > > Cheers, > -- > Pierre Fenoll > > > On 7 December 2015 at 05:45, Richard A. O'Keefe > wrote: > > > On 3/12/2015, at 8:11 pm, Michael Truog > wrote: > > If we had header files that were only allowed to contain functions and > > types that we called "template files" for lack of a better name > > (ideally, they would have their own export statements to help > > distinguish between private functions/types and the interface for modules > > to use) > > AND > > we had the include files (and template files) versioned within the beam > > output (to address the untracked dependency problem). > > > > Wouldn't that approach be preferable when compared > > to trying to manage the module dependency graph during a > > runtime code upgrades? Why would the "template files" approach > > not be sufficient? > > These are the differences I can see between 'template files' > and 'import_static modules'. > > (1) 'template files' would not be modules. > Having their own export directives would make them > modul*ar*, but they could not use 'fun M:F/A' to refer > to their own functions, having no M they could use. > > This does not seem like a good thing. > > (2) Headers can be nested. If 'template files' were to be > like headers, this would create nested scopes for top > level functions, and the possibility of multiple > distinct functions with the same name and arity and > in some sense "in" the very same module. > > I don't see that as an insuperable problem, but it's a > very big change to the language in order to avoid what > is really not likely to be a practical problem. > > (3) 'template files' are copied into the including module, > which allows different modules to have included > different versions of a 'template file'. > > Like headers, this DOES NOT SOLVE the dependency and > version skew problems, IT CREATES THOSE PROBLEMS. > > So with 'template files', either you have precisely the > same problems you have with headers plus a whole lot of > extra complexity, or you have to track dependencies anyway. > > If we can take a step away from Erlang, > we can see that we have a fundamental problem. > > Resource A is prepared from foundations x and c. > Resource B is prepared from foundations y and c. > Resources A and B have to "fit together" in some fashion. > Common foundation c has something to do with how that > fitting together works. > > c is revised to c'. > A is re-prepared to A'. > If B were re-prepared, B' and A' would be compatible, > just as B and A were compatible. > But B and A' are not compatible. > > As far as I can see, there are three general ways to > deal with this kind of problem. > > 1. Detection. When you try to use A' and B together, > detect that they were prepared from c' and c and > refuse to allow this use. > > This requires dependency tracking. > > 2. Prevention. When c is revised to c', use dependencies > forward and queue A and B for rebuilding. > > This requires dependency tracking. > > 3. Avoidance. Make the preparation step fairly trivial > so that whenever A (B) makes references to c, the > latest version of c is used. > > In programming language terms, this is not even lazy > evaluation, it's call-by-name. It's the way Erlang > currently handles remote calls. (More or less.) > > As a rule of thumb, early binding is the route to > efficiency (and early error detection), late binding > is the route to flexibility (and late error detection). > The performance cost may be anywhere between slight and > scary depending on the application. > > 3'. It is only necessary to provide the *appearance* of late > binding. I have a faint and probably unreliable memory > that the SPITBOL implementation of SNOBOL4 could generate > "optimised" code but could back it out and recompile it > when the assumptions it had made turned out to be wrong. > > Amongst other things, this requires the system to keep > track of this-depends-on-that at run time, so it's still > dependency tracking, but it need not be visible to the > programmer. > > What might be noticeable would be a performance blip as > loading c' caused everything that had been run-time compiled > against c to be de-optimised. TANSTAAFL. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Wed Feb 17 10:34:09 2016 From: rvirding@REDACTED (Robert Virding) Date: Wed, 17 Feb 2016 10:34:09 +0100 Subject: [erlang-questions] Erlounge in Barcelona Message-ID: Hi, I will be in Barcelona next week. Are there any Erlang/Elixirers who would care for a light erlounge while I am there? Robert -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Wed Feb 17 11:33:28 2016 From: vances@REDACTED (Vance Shipley) Date: Wed, 17 Feb 2016 16:03:28 +0530 Subject: [erlang-questions] Erlounge in Barcelona In-Reply-To: References: Message-ID: Count me in, I'll be in Barcelona Saturday through Wednesday for Mobile World Congress. On Wed, Feb 17, 2016 at 3:04 PM, Robert Virding wrote: > Hi, > > I will be in Barcelona next week. Are there any Erlang/Elixirers who would > care for a light erlounge while I am there? > > Robert > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- -Vance From kost@REDACTED Wed Feb 17 10:45:02 2016 From: kost@REDACTED (Konstantin Vsochanov) Date: Wed, 17 Feb 2016 12:45:02 +0300 Subject: [erlang-questions] Why we need a -module() attribute? Message-ID: <56C4411E.9060106@rol.ru> Hi! I?m working with Erlang for two years now and enjoy every day of using it. However, there are some strange features I?m wondering about. One of them -module() attribute. The module name is strictly defined by the name of the .erl file. You can?t change module name with -module() attribute. But the the -module() attribute is mandatory. Why? Why we need -module() attribute at all? Maybe we can make -module() optional in next Erlang release? So we don?t need to repeat file name in first string of every .erl file. Or maybe I miss something important? Feedback is appreciated! - Konstantin Voschanov From carlosj.gf@REDACTED Wed Feb 17 13:32:06 2016 From: carlosj.gf@REDACTED (=?UTF-8?Q?Carlos_Gonz=C3=A1lez_Florido?=) Date: Wed, 17 Feb 2016 13:32:06 +0100 Subject: [erlang-questions] Erlounge in Barcelona In-Reply-To: References: Message-ID: I would love to, if Wednesday or Thursday. Carlos Gonz?lez ErlNETS Technologies https://github.com/ErlNETS El mi?rcoles, 17 de febrero de 2016, Robert Virding escribi?: > Hi, > > I will be in Barcelona next week. Are there any Erlang/Elixirers who would > care for a light erlounge while I am there? > > Robert > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matwey.kornilov@REDACTED Wed Feb 17 13:42:34 2016 From: matwey.kornilov@REDACTED (Matwey V. Kornilov) Date: Wed, 17 Feb 2016 15:42:34 +0300 Subject: [erlang-questions] howto/INSTALL.md: wxWidgets Message-ID: Hello, HOWTO/INSTALL.md says: """ * wxWidgets -- Toolkit for GUI applications. Required for building the `wx` application. At least version 3.0 of wxWidgets is required. """ But it is obviously wrong. wx can be successfully compiled with wxWidgets 2.8 also. It seems that before 107b27ef809e4356ea258cbd820967ffb8fa0875 there were no such requirement. From eric.pailleau@REDACTED Wed Feb 17 13:46:34 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Wed, 17 Feb 2016 13:46:34 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56C4411E.9060106@rol.ru> Message-ID: <56fedf59-d724-406d-95af-12f81d6bf5f3@email.android.com> Hi, I suppose it is necessary when building a module from an abstract code not coming from a file on disk. Regards. Le?17 f?vr. 2016 10:45 AM, Konstantin Vsochanov a ?crit?: > > Hi! > > I?m working with Erlang for two years now and enjoy every day of using > it. However, there are some strange? features I?m wondering about. One > of them -module() attribute. > > The module name is strictly defined by the name of the .erl file. You > can?t change module name with -module() attribute. But the the -module() > attribute is mandatory. Why? Why we need -module() attribute at all? > Maybe we can make -module() optional in next Erlang release? So we don?t > need to repeat file name in first string of every .erl file. Or maybe I > miss something important? > > Feedback is appreciated! > > - Konstantin Voschanov > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From dangud@REDACTED Wed Feb 17 14:38:37 2016 From: dangud@REDACTED (Dan Gudmundsson) Date: Wed, 17 Feb 2016 13:38:37 +0000 Subject: [erlang-questions] howto/INSTALL.md: wxWidgets In-Reply-To: References: Message-ID: Is not that instruction for OSX? It was in the OSX chapter when I wrote it. For OSX wxWidgets-3.0 is required and it is used in our build for windows as well. On *nix you can still use 2.8 but it is recommended to use the 3.0 branch for better compability with the OSX and Windows. On Wed, Feb 17, 2016 at 1:42 PM Matwey V. Kornilov < matwey.kornilov@REDACTED> wrote: > Hello, > > HOWTO/INSTALL.md says: > > """ > * wxWidgets -- Toolkit for GUI applications. > Required for building the `wx` application. At least > version 3.0 of wxWidgets is required. > """ > > But it is obviously wrong. wx can be successfully compiled with > wxWidgets 2.8 also. > > It seems that before 107b27ef809e4356ea258cbd820967ffb8fa0875 > there were no such requirement. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matwey.kornilov@REDACTED Wed Feb 17 14:42:01 2016 From: matwey.kornilov@REDACTED (Matwey V. Kornilov) Date: Wed, 17 Feb 2016 16:42:01 +0300 Subject: [erlang-questions] howto/INSTALL.md: wxWidgets In-Reply-To: References: Message-ID: The text is on line 89 of the file (section "Optional Utilities"). It is not OSX chapter. 2016-02-17 16:38 GMT+03:00 Dan Gudmundsson : > Is not that instruction for OSX? > It was in the OSX chapter when I wrote it. > > For OSX wxWidgets-3.0 is required and it is used in our build for windows as > well. > On *nix you can still use 2.8 but it is recommended to use the 3.0 branch > for better > compability with the OSX and Windows. > > > On Wed, Feb 17, 2016 at 1:42 PM Matwey V. Kornilov > wrote: >> >> Hello, >> >> HOWTO/INSTALL.md says: >> >> """ >> * wxWidgets -- Toolkit for GUI applications. >> Required for building the `wx` application. At least >> version 3.0 of wxWidgets is required. >> """ >> >> But it is obviously wrong. wx can be successfully compiled with >> wxWidgets 2.8 also. >> >> It seems that before 107b27ef809e4356ea258cbd820967ffb8fa0875 >> there were no such requirement. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -- With best regards, Matwey V. Kornilov http://blog.matwey.name xmpp://0x2207@REDACTED From cons@REDACTED Wed Feb 17 14:58:56 2016 From: cons@REDACTED (=?windows-1252?Q?Cons_T_=C5hs?=) Date: Wed, 17 Feb 2016 14:58:56 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56C4411E.9060106@rol.ru> References: <56C4411E.9060106@rol.ru> Message-ID: <3ED05F8C-39B9-4CD6-9998-0016393C5192@tail-f.com> On 17 Feb 2016, at 10:45, Konstantin Vsochanov wrote: > Hi! > > I?m working with Erlang for two years now and enjoy every day of using > it. However, there are some strange features I?m wondering about. One > of them -module() attribute. > > The module name is strictly defined by the name of the .erl file. You > can?t change module name with -module() attribute. Actually, you can. If you give the option no_error_module_mismatch to compile:file/2 as in 42> compile:file(baz, [no_error_module_mismatch]). the file baz.erl can contain a -module() directive with another name than baz. Note, however, that you need to use code:load_binary/3 to load the resulting beam file. Cons > But the the -module() > attribute is mandatory. Why? Why we need -module() attribute at all? > Maybe we can make -module() optional in next Erlang release? So we don?t > need to repeat file name in first string of every .erl file. Or maybe I > miss something important? > > Feedback is appreciated! > > - Konstantin Voschanov > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Wed Feb 17 15:26:08 2016 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 17 Feb 2016 15:26:08 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56C4411E.9060106@rol.ru> References: <56C4411E.9060106@rol.ru> Message-ID: Well suppose you sent the *content* of the file in a message - how would the receiver know the name of the module?. Actually the module attribute and file name don't have to be the same - but if they are not you'll break the autoloading mechanism. If the file name and module name are different you should know exactly what you're doing :-) Cheers /Joe On Wed, Feb 17, 2016 at 10:45 AM, Konstantin Vsochanov wrote: > Hi! > > I?m working with Erlang for two years now and enjoy every day of using > it. However, there are some strange features I?m wondering about. One > of them -module() attribute. > > The module name is strictly defined by the name of the .erl file. You > can?t change module name with -module() attribute. But the the -module() > attribute is mandatory. Why? Why we need -module() attribute at all? > Maybe we can make -module() optional in next Erlang release? So we don?t > need to repeat file name in first string of every .erl file. Or maybe I > miss something important? > > Feedback is appreciated! > > - Konstantin Voschanov > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Wed Feb 17 15:37:50 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Wed, 17 Feb 2016 15:37:50 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56C4411E.9060106@rol.ru> References: <56C4411E.9060106@rol.ru> Message-ID: <56C485BE.3030106@ninenines.eu> From what I understand it's there to make sure it works on case insensitive file systems. But I'm not sure this is an actual issue, considering you have to give the file name to the compiler anyway, and this file name could be considered the canonical module name + extension. It's a problem that I believe tools should fix, not programmers. On 02/17/2016 10:45 AM, Konstantin Vsochanov wrote: > Hi! > > I?m working with Erlang for two years now and enjoy every day of using > it. However, there are some strange features I?m wondering about. One > of them -module() attribute. > > The module name is strictly defined by the name of the .erl file. You > can?t change module name with -module() attribute. But the the -module() > attribute is mandatory. Why? Why we need -module() attribute at all? > Maybe we can make -module() optional in next Erlang release? So we don?t > need to repeat file name in first string of every .erl file. Or maybe I > miss something important? > > Feedback is appreciated! > > - Konstantin Voschanov > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From jesper.louis.andersen@REDACTED Wed Feb 17 15:45:29 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 17 Feb 2016 15:45:29 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56C4411E.9060106@rol.ru> References: <56C4411E.9060106@rol.ru> Message-ID: I usually say that any kind of assumption binding the file system to the module system of the programming language is a mistake. * You can't use another transport such as a socket for moving data around. * You are bound by file system lookups for loading modules. * You can't do nested modules well and have to rely on odd file system directory structures * The programmer can't name things logically and are forced to obey the system * You can't split the same module into multiple files * You can't have multiple modules in one file One language which doesn't make this mistake is Standard ML. You have another layer in some SML systems, the .mlb file, which then groups files into a collection of modules, including export/import renaming, stdlib selection, extension selection and so on. Were I to create a language, this would be my solution as well. On Wed, Feb 17, 2016 at 10:45 AM, Konstantin Vsochanov wrote: > Hi! > > I?m working with Erlang for two years now and enjoy every day of using > it. However, there are some strange features I?m wondering about. One > of them -module() attribute. > > The module name is strictly defined by the name of the .erl file. You > can?t change module name with -module() attribute. But the the -module() > attribute is mandatory. Why? Why we need -module() attribute at all? > Maybe we can make -module() optional in next Erlang release? So we don?t > need to repeat file name in first string of every .erl file. Or maybe I > miss something important? > > Feedback is appreciated! > > - Konstantin Voschanov > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From alin.popa@REDACTED Wed Feb 17 13:25:19 2016 From: alin.popa@REDACTED (Alin Popa) Date: Wed, 17 Feb 2016 12:25:19 +0000 Subject: [erlang-questions] logical_processors_available unknown on mac Message-ID: Hi, I was recently trying to use `erlang:system_info(logical_processors_available).` on my mac, and getting `unknown` atom back; anyone has any idea if this is really by design? I've read the official documentation on this: ``` logical_processors_available Returns the detected number of logical processors available to the Erlang runtime system. The return value is either an integer, or the atom unknown if the emulator cannot detect the available logical processors. ``` Thing is I find hard to believe that the emulator cannon detect the available logical processors, especially when this `erlang:system_info(logical_processors).` returns 8 (having 4 hyperthreaded cores). I've tried to understand the reasoning behind that by looking at the source code (`erts/lib_src/common/erl_misc_utils.c`), but due to my lack of C knowledge, I wasn't able to get very far. Thanks, Alin -------------- next part -------------- An HTML attachment was scrubbed... URL: From lfmo@REDACTED Wed Feb 17 15:11:38 2016 From: lfmo@REDACTED (Leonardo Fernandes Mendonca de Oliveira) Date: Wed, 17 Feb 2016 11:11:38 -0300 Subject: [erlang-questions] Some library similar to Haskell Accelerate Message-ID: Hi, Does anyone knows if there is some erlang library similar to Haskell Accelerate [https://hackage.haskell.org/package/accelerate]. Basically: It defines an array language for computations for high-performance computing (Computations on multi-dimensional regular arrays..that can be executed on a range of architectures). -- Atc., MSc. Leonardo Fernandes @leofernandesmo -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed Feb 17 16:32:25 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 17 Feb 2016 16:32:25 +0100 Subject: [erlang-questions] logical_processors_available unknown on mac In-Reply-To: References: Message-ID: On Wed, Feb 17, 2016 at 1:25 PM, Alin Popa wrote: > I was recently trying to use > `erlang:system_info(logical_processors_available).` on my mac, and getting > `unknown` atom back; anyone has any idea if this is really by design? > A bit of guesswork here, and someone will correct my mistakes I'm sure: You can't take your core off-line in OSX and replace it while the system is running to the best of my knowledge. So you may not be able to query this information from the kernel at all. Illumos/Solaris has support for this for instance, and I guess this is why there is an entry for it. If your Solaris server has a failing CPU package, you can take it offline from the OS, replace it, and start the new CPU package, without stopping the system. When this happens, you might have fewer logical processors available in the system for some reason. Many of the settings regarding CPUs are not that supported on OSX. Another such call is the +sbt option, which you can't use on OSX since there is no way to bind threads to CPU cores. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed Feb 17 16:38:06 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 17 Feb 2016 16:38:06 +0100 Subject: [erlang-questions] Updates, lenses, and why cross-module inlining would be nice In-Reply-To: References: <1448544949.1214768.450737385.156BF7D2@webmail.messagingengine.com> <5657EA88.1010901@gmail.com> <1462F19A-19B6-4A16-A26D-4643BC0830CB@cs.otago.ac.nz> <565CA6AC.9020306@gmail.com> <32BC6909-E775-48B0-98EB-D89716098F61@cs.otago.ac.nz> <565E8D7F.5030106@gmail.com> <0AB5C64B-11D3-46A5-BB6A-AF275C5D7E08@cs.otago.ac.nz> <565FEB28.4090301@gmail.com> <8216D951-C321-48B2-ADFC-FA0D2D5B7D9E@cs.otago.ac.nz> Message-ID: On Mon, Feb 15, 2016 at 4:38 PM, Pierre Fenoll wrote: > This is surely a very naive idea on how to one day get cross-module > inlining > or at least enlarge the range of possibilities for the JIT. > A JIT should be free to build up optimized code with full inlining as long as it knows how to deoptimize that code again should a new version of a module be loaded. One of the alluring thing of a JIT is that it is free to make dynamic choices in the running code without any priori knowledge. A traditional compiler which works on static data needs to operate on static analysis alone. This is usually easier if the program has less wiggle room of interpretation. And this is the case if you have a static type system... -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jozsef.berces@REDACTED Wed Feb 17 16:38:35 2016 From: jozsef.berces@REDACTED (=?iso-8859-1?Q?J=F3zsef_B=E9rces?=) Date: Wed, 17 Feb 2016 15:38:35 +0000 Subject: [erlang-questions] SSH tunnels Message-ID: <7460EBDDCF52084A849D0F271CE059B8183E385E@ESESSMB101.ericsson.se> Can we establish SSH tunnels with pure Erlang code? I would be interested in setting up local and dynamic port forwarding. Thanks, Jozsef -------------- next part -------------- An HTML attachment was scrubbed... URL: From seancribbs@REDACTED Wed Feb 17 16:42:54 2016 From: seancribbs@REDACTED (Sean Cribbs) Date: Wed, 17 Feb 2016 09:42:54 -0600 Subject: [erlang-questions] Some library similar to Haskell Accelerate In-Reply-To: References: Message-ID: Are you looking for something like LINPACK/LAPACK? There's this: https://github.com/sklassen/lapack-erlang-nif Cheers, Sean On Wed, Feb 17, 2016 at 8:11 AM, Leonardo Fernandes Mendonca de Oliveira < lfmo@REDACTED> wrote: > Hi, > Does anyone knows if there is some erlang library similar to Haskell > Accelerate [https://hackage.haskell.org/package/accelerate]. > Basically: It defines an array language for computations for > high-performance computing (Computations on multi-dimensional regular > arrays..that can be executed on a range of architectures). > > > -- > Atc., > MSc. Leonardo Fernandes > @leofernandesmo > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From darach@REDACTED Wed Feb 17 16:55:38 2016 From: darach@REDACTED (Darach Ennis) Date: Wed, 17 Feb 2016 15:55:38 +0000 Subject: [erlang-questions] Some library similar to Haskell Accelerate In-Reply-To: References: Message-ID: Hi Leonardo, It's also worth looking at: * CL - https://github.com/tonyrog/cl * PteraCUDA - https://github.com/kevsmith/pteracuda If you want to offload the compute intensive code to GPU acceleration these might be worth investigating. If you wish to pre- or -post- process something on a latency sensitive data path then Solarflare's AOE enables this: http://www.solarflare.com/applicationonload-engine through a firmware development kit. Cheers, Darach. On Wed, Feb 17, 2016 at 3:42 PM, Sean Cribbs wrote: > Are you looking for something like LINPACK/LAPACK? There's this: > https://github.com/sklassen/lapack-erlang-nif > > Cheers, > > Sean > > On Wed, Feb 17, 2016 at 8:11 AM, Leonardo Fernandes Mendonca de Oliveira < > lfmo@REDACTED> wrote: > >> Hi, >> Does anyone knows if there is some erlang library similar to Haskell >> Accelerate [https://hackage.haskell.org/package/accelerate]. >> Basically: It defines an array language for computations for >> high-performance computing (Computations on multi-dimensional regular >> arrays..that can be executed on a range of architectures). >> >> >> -- >> Atc., >> MSc. Leonardo Fernandes >> @leofernandesmo >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serge@REDACTED Wed Feb 17 17:48:29 2016 From: serge@REDACTED (Serge Aleynikov) Date: Wed, 17 Feb 2016 11:48:29 -0500 Subject: [erlang-questions] SSH tunnels In-Reply-To: <7460EBDDCF52084A849D0F271CE059B8183E385E@ESESSMB101.ericsson.se> References: <7460EBDDCF52084A849D0F271CE059B8183E385E@ESESSMB101.ericsson.se> Message-ID: Are you talking about running distributed Erlang transport over SSH tunnels or creating tunnels via the Erlang's SSH client/server applications? It would certainly be nice if distributed transport supported such an option, albeit it's not a trivial change, that would involve similar effort to supporting multiple transports (https://github.com/erlang/otp/pull/121). If it's just for application-level port forwarding, then I would think leveraging the SSH server's port forwarding facility with the Erlang client-side tunneling, you can look at ssh_connection:direct_tcpip/6, which does exactly that. Serge On Wed, Feb 17, 2016 at 10:38 AM, J?zsef B?rces wrote: > Can we establish SSH tunnels with pure Erlang code? I would be interested > in setting up local and dynamic port forwarding. > > > > Thanks, > > Jozsef > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From denc716@REDACTED Wed Feb 17 18:38:35 2016 From: denc716@REDACTED (derek) Date: Wed, 17 Feb 2016 09:38:35 -0800 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> Message-ID: that -module(). line sound like redundant, if it can be mismatched, why can't it be omitted? compiler can derive the module name from the filename anyway; Java also made that filename has to match class name, that is a design mistake and unnecessary requirement, to my opinion; or on the other hand, if -module() is proved to be useful, why not just ignore the filename, and always use what's defined in -module()? in many modern language design like Go and Elixir, it's allowed to define multiple Class/Module in one file, this will be really flexible: %% file1.erl -module(module1). %% [content of module1] -module(module2). %% [content of module2] -module(module3). ... On Wed, Feb 17, 2016 at 6:26 AM, Joe Armstrong wrote: > Well suppose you sent the *content* of the file in a message - how > would the receiver > know the name of the module?. > > Actually the module attribute and file name don't have to be the same - but > if they are not you'll break the autoloading mechanism. If the file name and > module name are different you should know exactly what you're doing :-) > > Cheers > > /Joe > > On Wed, Feb 17, 2016 at 10:45 AM, Konstantin Vsochanov wrote: >> Hi! >> >> I?m working with Erlang for two years now and enjoy every day of using >> it. However, there are some strange features I?m wondering about. One >> of them -module() attribute. >> >> The module name is strictly defined by the name of the .erl file. You >> can?t change module name with -module() attribute. But the the -module() >> attribute is mandatory. Why? Why we need -module() attribute at all? >> Maybe we can make -module() optional in next Erlang release? So we don?t >> need to repeat file name in first string of every .erl file. Or maybe I >> miss something important? >> >> Feedback is appreciated! >> >> - Konstantin Voschanov >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From eric.pailleau@REDACTED Wed Feb 17 18:44:28 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Wed, 17 Feb 2016 18:44:28 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: Message-ID: Hi, Mho, module name in file should be unique and same than rootname file. This avoid any module clash, not globally but in same app. BTW, namespace is probably something Erlang should handle in futur. This would avoid any module name clash. But anither story. Le?17 f?vr. 2016 6:38 PM, derek a ?crit?: > > that -module(). line sound like redundant, if it can be mismatched, > why can't it be omitted? > compiler can derive the module name from the filename anyway; Java > also made that filename has to match class name, that is a design > mistake and unnecessary requirement, to my opinion; > > or on the other hand, if -module() is proved to be useful, why not > just ignore the filename, and always use what's defined in -module()? > in many modern language design like Go and Elixir, it's allowed to > define multiple Class/Module in one file, this will be really > flexible: > > %% file1.erl > -module(module1). > > %% [content of module1] > > -module(module2). > > %% [content of module2] > > -module(module3). > > ... > > On Wed, Feb 17, 2016 at 6:26 AM, Joe Armstrong wrote: > > Well suppose you sent the *content* of the file in a message - how > > would the receiver > > know the name of the module?. > > > > Actually the module attribute and file name don't have to be the same - but > > if they are not you'll break the autoloading mechanism. If the file name and > > module name are different you should know exactly what you're doing :-) > > > > Cheers > > > > /Joe > > > > On Wed, Feb 17, 2016 at 10:45 AM, Konstantin Vsochanov wrote: > >> Hi! > >> > >> I?m working with Erlang for two years now and enjoy every day of using > >> it. However, there are some strange? features I?m wondering about. One > >> of them -module() attribute. > >> > >>? The module name is strictly defined by the name of the .erl file. You > >> can?t change module name with -module() attribute. But the the -module() > >> attribute is mandatory. Why? Why we need -module() attribute at all? > >> Maybe we can make -module() optional in next Erlang release? So we don?t > >> need to repeat file name in first string of every .erl file. Or maybe I > >> miss something important? > >> > >> Feedback is appreciated! > >> > >> - Konstantin Voschanov > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.feng.bao@REDACTED Thu Feb 18 07:47:47 2016 From: max.feng.bao@REDACTED (max_feng) Date: Thu, 18 Feb 2016 14:47:47 +0800 Subject: [erlang-questions] An elegant implementation of multi pollset for Erlang VM Message-ID: Erlang VM has only one PollSet in the same time, which is not scalable, especially with a rapid growth of NIC bandwidth and CPU cores on one single machine. In order to make full use of multi-core in Eralng network programming, we develop gen_socket module. The gen_socket creates multi pollset for Erlang VM without changing erts code! The gen_socket already has been deployed in production env, and increases throughput of echo server by 110% on a 24-core, 1000Mb/s machine based on redhat6.2. The gen_socket is completely compatible with gen_tcp interface, and very easily deployed. Just git clone and make it. For details please refer to project link. https://github.com/max-feng/erlang_multi_pollset Max Feng -------------- next part -------------- An HTML attachment was scrubbed... URL: From jozsef.berces@REDACTED Thu Feb 18 12:45:16 2016 From: jozsef.berces@REDACTED (=?utf-8?B?SsOzenNlZiBCw6lyY2Vz?=) Date: Thu, 18 Feb 2016 11:45:16 +0000 Subject: [erlang-questions] SSH tunnels In-Reply-To: References: <7460EBDDCF52084A849D0F271CE059B8183E385E@ESESSMB101.ericsson.se> Message-ID: <7460EBDDCF52084A849D0F271CE059B8183E41C0@ESESSMB101.ericsson.se> Thanks. I meant the application level port-forwarding but tunneling the Erlang transport is also interesting. From: Serge Aleynikov [mailto:serge@REDACTED] Sent: Wednesday, February 17, 2016 17:48 To: J?zsef B?rces Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] SSH tunnels Are you talking about running distributed Erlang transport over SSH tunnels or creating tunnels via the Erlang's SSH client/server applications? It would certainly be nice if distributed transport supported such an option, albeit it's not a trivial change, that would involve similar effort to supporting multiple transports (https://github.com/erlang/otp/pull/121). If it's just for application-level port forwarding, then I would think leveraging the SSH server's port forwarding facility with the Erlang client-side tunneling, you can look at ssh_connection:direct_tcpip/6, which does exactly that. Serge On Wed, Feb 17, 2016 at 10:38 AM, J?zsef B?rces > wrote: Can we establish SSH tunnels with pure Erlang code? I would be interested in setting up local and dynamic port forwarding. Thanks, Jozsef _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Thu Feb 18 12:46:31 2016 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 18 Feb 2016 12:46:31 +0100 Subject: [erlang-questions] where does output go in detached mode Message-ID: Hello, I start erlang with $ erl -detached and have two questions: 1) If program does io:format(Format, Data) where does the output go? Does the output just vanish, or do I have to unclog some buffers eventually? 2) Can I attach to the process later? And if so does this work on the three major platforms OS-X, linux and windows? Cheers /Joe From erlang@REDACTED Thu Feb 18 12:55:18 2016 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 18 Feb 2016 12:55:18 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: Message-ID: On Wed, Feb 17, 2016 at 6:44 PM, ?ric Pailleau wrote: > Hi, > Mho, module name in file should be unique and same than rootname file. This avoid any module clash, not globally but in same app. I disagree - the "real" module name should be the SHA1 checksum of the source code - the "user friendly name" should just be an alias to the SHA1 checksum. The development system should hide or reveal this information depending upon the context. When developing you probably want to talk about a module called "foo" when you ship code or send code over the network you want know *exactly* what you did. The problem with using a module name to identify the code is that the content of the module changes with time and place. In a distributed system we can imagine upgrading code for some module 'foo' bit that foo.erl is *different*on different machine because the code has not yet arrived. Using "just" the module name will and does get you very rapidly into "version nightmare land". Handling versions is much more complicated than you might think - and *nobody* seems to have got it right - (apart from NiX - which might have got things right, but I'm not sure yet) Cheers /Joe > BTW, namespace is probably something Erlang should handle in futur. This would avoid any module name clash. But anither story. > > Le 17 f?vr. 2016 6:38 PM, derek a ?crit : >> >> that -module(). line sound like redundant, if it can be mismatched, >> why can't it be omitted? >> compiler can derive the module name from the filename anyway; Java >> also made that filename has to match class name, that is a design >> mistake and unnecessary requirement, to my opinion; >> >> or on the other hand, if -module() is proved to be useful, why not >> just ignore the filename, and always use what's defined in -module()? >> in many modern language design like Go and Elixir, it's allowed to >> define multiple Class/Module in one file, this will be really >> flexible: >> >> %% file1.erl >> -module(module1). >> >> %% [content of module1] >> >> -module(module2). >> >> %% [content of module2] >> >> -module(module3). >> >> ... >> >> On Wed, Feb 17, 2016 at 6:26 AM, Joe Armstrong wrote: >> > Well suppose you sent the *content* of the file in a message - how >> > would the receiver >> > know the name of the module?. >> > >> > Actually the module attribute and file name don't have to be the same - but >> > if they are not you'll break the autoloading mechanism. If the file name and >> > module name are different you should know exactly what you're doing :-) >> > >> > Cheers >> > >> > /Joe >> > >> > On Wed, Feb 17, 2016 at 10:45 AM, Konstantin Vsochanov wrote: >> >> Hi! >> >> >> >> I?m working with Erlang for two years now and enjoy every day of using >> >> it. However, there are some strange features I?m wondering about. One >> >> of them -module() attribute. >> >> >> >> The module name is strictly defined by the name of the .erl file. You >> >> can?t change module name with -module() attribute. But the the -module() >> >> attribute is mandatory. Why? Why we need -module() attribute at all? >> >> Maybe we can make -module() optional in next Erlang release? So we don?t >> >> need to repeat file name in first string of every .erl file. Or maybe I >> >> miss something important? >> >> >> >> Feedback is appreciated! >> >> >> >> - Konstantin Voschanov >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From askjuise@REDACTED Thu Feb 18 13:01:22 2016 From: askjuise@REDACTED (Alexander Petrovsky) Date: Thu, 18 Feb 2016 15:01:22 +0300 Subject: [erlang-questions] Packets deduplication Message-ID: Hi! I have the stream of packets with ID (int), and I need to check is the packet is uniq (by ID) or not? Incoming rate is about 20k pps and ID is monotonically grows. What's the best way and data structure fit for this problem? -- ?????????? ????????? / Alexander Petrovsky, Skype: askjuise Phone: +7 914 8 820 815 -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Thu Feb 18 13:21:28 2016 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 18 Feb 2016 13:21:28 +0100 Subject: [erlang-questions] tiny erlang Message-ID: Hello I want to make a small erlang distribution for an embedded device - the process is rather simple: 1) strip the beam executable 2) for each library in {stdlib, kernerl, compiler, sasl} do Compress all the beam code and remove debug symbols squash everthing into a single packed file 3) Hack the code loader to read from the single packed beam file 4) Tweak the shell scripts to start Erlang At a guess we're down to c. 6 MB I have done this on *several* occassions - and am getting a bit fed up doing it over and over again. Has anybody done this recently - and should not something like this be in the standard distribution? Cheers /Joe From alin.popa@REDACTED Thu Feb 18 12:46:46 2016 From: alin.popa@REDACTED (Alin Popa) Date: Thu, 18 Feb 2016 11:46:46 +0000 Subject: [erlang-questions] logical_processors_available unknown on mac In-Reply-To: References: Message-ID: Hmmmm, sure, I think it makes sense, but still, it works fine on Linux as I've tested. Also, not entirely sure what should I use instead of that on OSX, any idea? Thanks, Alin On Wed, Feb 17, 2016 at 3:32 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > On Wed, Feb 17, 2016 at 1:25 PM, Alin Popa wrote: > >> I was recently trying to use >> `erlang:system_info(logical_processors_available).` on my mac, and getting >> `unknown` atom back; anyone has any idea if this is really by design? >> > > A bit of guesswork here, and someone will correct my mistakes I'm sure: > > You can't take your core off-line in OSX and replace it while the system > is running to the best of my knowledge. So you may not be able to query > this information from the kernel at all. Illumos/Solaris has support for > this for instance, and I guess this is why there is an entry for it. > > If your Solaris server has a failing CPU package, you can take it offline > from the OS, replace it, and start the new CPU package, without stopping > the system. When this happens, you might have fewer logical processors > available in the system for some reason. > > Many of the settings regarding CPUs are not that supported on OSX. Another > such call is the +sbt option, which you can't use on OSX since there is no > way to bind threads to CPU cores. > > > -- > J. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas.larsson@REDACTED Thu Feb 18 13:44:47 2016 From: lukas.larsson@REDACTED (Lukas Larsson) Date: Thu, 18 Feb 2016 13:44:47 +0100 Subject: [erlang-questions] An elegant implementation of multi pollset for Erlang VM In-Reply-To: References: Message-ID: very nice! good work! A question though; one of the reasons why doing this within erts is difficult is because after a process sets a socket from active to passive no messages should arrive from the socket. Does you solution handle this problem? If so how? Lukas On Thu, Feb 18, 2016 at 7:47 AM, max_feng wrote: > Erlang VM has only one PollSet in the same time, which is not scalable, > especially with a rapid growth of NIC bandwidth and CPU cores on one single > machine. > > In order to make full use of multi-core in Eralng network programming, we > develop gen_socket module. > The gen_socket creates multi pollset for Erlang VM without changing erts > code! > > The gen_socket already has been deployed in production env, and increases > throughput of echo server by 110% on a 24-core, 1000Mb/s machine based on > redhat6.2. > > The gen_socket is completely compatible with gen_tcp interface, and very > easily deployed. Just git clone and make it. > > For details please refer to project link. > > https://github.com/max-feng/erlang_multi_pollset > > Max Feng > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Thu Feb 18 13:49:19 2016 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 18 Feb 2016 13:49:19 +0100 Subject: [erlang-questions] live audio output from erlang on mac Message-ID: Hello, I want to generate raw sound samples and play them almost immediately on my mac. I can do this by making wav files and call afplay so I know I can make and play sounds. That I want is to do is send sound samples to a server (probably using TCP) and get the server to play them. Can anybody suggest suitable software to do this? Note I've contemplated the following: 1) I can build port audio but I'd need to write a TCP (or whatever) interface. 2) Pulse Audio seems to be able to do this - but the project for a mac port seems to have been abandoned. 3) I wrote my own TCP server and with a core audio interface (in C) but the code sucks and tricky things like splicing wave changes when the signal goes to zero seem pretty tricky to do properly - and I have to use pthreads which is unbelievably horrible. 4) The supercollider backend (scsynth) might be able to do this - but how to do this is undocumented. 5) I think I could send RTP packets to VLC and configure VLC to play the samples, but haven't tried this. From tuncer.ayaz@REDACTED Thu Feb 18 14:05:04 2016 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Thu, 18 Feb 2016 14:05:04 +0100 Subject: [erlang-questions] logical_processors_available unknown on mac In-Reply-To: References: Message-ID: On 18 February 2016 at 12:46, Alin Popa wrote: > Hmmmm, sure, I think it makes sense, but still, it works fine on > Linux as I've tested. > Also, not entirely sure what should I use instead of that on OSX, > any idea? Doesn't 'logical_processors' (ignoring _available and _online) suffice for your use case? If not, and assuming you're on Apple hardware, I'm afraid there's no way to ask OS X about hot swappable CPUs, because there is no such concept in OS X. Also, I'm not aware of current Apple hardware that supports hot-swapping CPUs. If you run OS X on non-Apple hardware, maybe a device driver (kext) could access such info. From alin.popa@REDACTED Thu Feb 18 14:11:58 2016 From: alin.popa@REDACTED (Alin Popa) Date: Thu, 18 Feb 2016 13:11:58 +0000 Subject: [erlang-questions] logical_processors_available unknown on mac In-Reply-To: References: Message-ID: Yep, I can use `logical_processors` instead, that will do for what I need. Thanks, Alin On Thu, Feb 18, 2016 at 1:05 PM, Tuncer Ayaz wrote: > On 18 February 2016 at 12:46, Alin Popa wrote: > > Hmmmm, sure, I think it makes sense, but still, it works fine on > > Linux as I've tested. > > Also, not entirely sure what should I use instead of that on OSX, > > any idea? > > Doesn't 'logical_processors' (ignoring _available and _online) suffice > for your use case? > > If not, and assuming you're on Apple hardware, I'm afraid there's no > way to ask OS X about hot swappable CPUs, because there is no such > concept in OS X. Also, I'm not aware of current Apple hardware that > supports hot-swapping CPUs. > > If you run OS X on non-Apple hardware, maybe a device driver (kext) > could access such info. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From z@REDACTED Thu Feb 18 14:20:09 2016 From: z@REDACTED (Danil Zagoskin) Date: Thu, 18 Feb 2016 16:20:09 +0300 Subject: [erlang-questions] Packets deduplication In-Reply-To: References: Message-ID: Hi! If ID grows monotinically and you have no plans of recovering after packet reordering, then you can just keep the previous ID. - If CurID > PrevID, CurID is unique; - If CurID == PrevID, it is not unique; - If CurID < PrevID, it is a bug or reordering, let it crash. On Thu, Feb 18, 2016 at 3:01 PM, Alexander Petrovsky wrote: > Hi! > > I have the stream of packets with ID (int), and I need to check is the > packet is uniq (by ID) or not? > > Incoming rate is about 20k pps and ID is monotonically grows. What's the > best way and data structure fit for this problem? > > -- > ?????????? ????????? / Alexander Petrovsky, > > Skype: askjuise > Phone: +7 914 8 820 815 > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Danil Zagoskin | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From askjuise@REDACTED Thu Feb 18 14:24:40 2016 From: askjuise@REDACTED (Alexander Petrovsky) Date: Thu, 18 Feb 2016 16:24:40 +0300 Subject: [erlang-questions] Packets deduplication In-Reply-To: References: Message-ID: Ouch, I forgot to say, the IDs can be sparse, so they not really monotonically, they just grow, and can be reordered. 2016-02-18 16:20 GMT+03:00 Danil Zagoskin : > Hi! > > If ID grows monotinically and you have no plans of recovering after packet > reordering, then you can just keep the previous ID. > - If CurID > PrevID, CurID is unique; > - If CurID == PrevID, it is not unique; > - If CurID < PrevID, it is a bug or reordering, let it crash. > > > On Thu, Feb 18, 2016 at 3:01 PM, Alexander Petrovsky > wrote: > >> Hi! >> >> I have the stream of packets with ID (int), and I need to check is the >> packet is uniq (by ID) or not? >> >> Incoming rate is about 20k pps and ID is monotonically grows. What's the >> best way and data structure fit for this problem? >> >> -- >> ?????????? ????????? / Alexander Petrovsky, >> >> Skype: askjuise >> Phone: +7 914 8 820 815 >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > Danil Zagoskin | z@REDACTED > -- ?????????? ????????? / Alexander Petrovsky, Skype: askjuise Phone: +7 914 8 820 815 -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Feb 18 14:37:10 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 18 Feb 2016 14:37:10 +0100 Subject: [erlang-questions] Packets deduplication In-Reply-To: References: Message-ID: A simple solution that may work in the end is to keep track of a window of bits: * shift the window as the ID increases * check older packets against the window This gives you "SACK" support, is fast to check against and fast to expire old data due to the shifting. Simply using an ordered list or a map could also work if you clean it up eventually. It depends on how fast your CPU core is. On Thu, Feb 18, 2016 at 2:24 PM, Alexander Petrovsky wrote: > Ouch, I forgot to say, the IDs can be sparse, so they not really > monotonically, they just grow, and can be reordered. > > 2016-02-18 16:20 GMT+03:00 Danil Zagoskin : > >> Hi! >> >> If ID grows monotinically and you have no plans of recovering after >> packet reordering, then you can just keep the previous ID. >> - If CurID > PrevID, CurID is unique; >> - If CurID == PrevID, it is not unique; >> - If CurID < PrevID, it is a bug or reordering, let it crash. >> >> >> On Thu, Feb 18, 2016 at 3:01 PM, Alexander Petrovsky >> wrote: >> >>> Hi! >>> >>> I have the stream of packets with ID (int), and I need to check is the >>> packet is uniq (by ID) or not? >>> >>> Incoming rate is about 20k pps and ID is monotonically grows. What's the >>> best way and data structure fit for this problem? >>> >>> -- >>> ?????????? ????????? / Alexander Petrovsky, >>> >>> Skype: askjuise >>> Phone: +7 914 8 820 815 >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> >> -- >> Danil Zagoskin | z@REDACTED >> > > > > -- > ?????????? ????????? / Alexander Petrovsky, > > Skype: askjuise > Phone: +7 914 8 820 815 > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From askjuise@REDACTED Thu Feb 18 14:43:02 2016 From: askjuise@REDACTED (Alexander Petrovsky) Date: Thu, 18 Feb 2016 16:43:02 +0300 Subject: [erlang-questions] Packets deduplication In-Reply-To: References: Message-ID: Yep, right now I try to do something similar with bitmap and sliding window, that will shift. Could you explain about maps? Thanks! 2016-02-18 16:37 GMT+03:00 Jesper Louis Andersen < jesper.louis.andersen@REDACTED>: > A simple solution that may work in the end is to keep track of a window of > bits: > > * shift the window as the ID increases > * check older packets against the window > > This gives you "SACK" support, is fast to check against and fast to expire > old data due to the shifting. > > Simply using an ordered list or a map could also work if you clean it up > eventually. It depends on how fast your CPU core is. > > > On Thu, Feb 18, 2016 at 2:24 PM, Alexander Petrovsky > wrote: > >> Ouch, I forgot to say, the IDs can be sparse, so they not really >> monotonically, they just grow, and can be reordered. >> >> 2016-02-18 16:20 GMT+03:00 Danil Zagoskin : >> >>> Hi! >>> >>> If ID grows monotinically and you have no plans of recovering after >>> packet reordering, then you can just keep the previous ID. >>> - If CurID > PrevID, CurID is unique; >>> - If CurID == PrevID, it is not unique; >>> - If CurID < PrevID, it is a bug or reordering, let it crash. >>> >>> >>> On Thu, Feb 18, 2016 at 3:01 PM, Alexander Petrovsky >> > wrote: >>> >>>> Hi! >>>> >>>> I have the stream of packets with ID (int), and I need to check is the >>>> packet is uniq (by ID) or not? >>>> >>>> Incoming rate is about 20k pps and ID is monotonically grows. What's >>>> the best way and data structure fit for this problem? >>>> >>>> -- >>>> ?????????? ????????? / Alexander Petrovsky, >>>> >>>> Skype: askjuise >>>> Phone: +7 914 8 820 815 >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >>> >>> -- >>> Danil Zagoskin | z@REDACTED >>> >> >> >> >> -- >> ?????????? ????????? / Alexander Petrovsky, >> >> Skype: askjuise >> Phone: +7 914 8 820 815 >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > J. > -- ?????????? ????????? / Alexander Petrovsky, Skype: askjuise Phone: +7 914 8 820 815 -------------- next part -------------- An HTML attachment was scrubbed... URL: From hellkvist@REDACTED Thu Feb 18 15:17:46 2016 From: hellkvist@REDACTED (Stefan Hellkvist) Date: Thu, 18 Feb 2016 15:17:46 +0100 Subject: [erlang-questions] live audio output from erlang on mac In-Reply-To: References: Message-ID: > On 18 Feb 2016, at 13:49, Joe Armstrong wrote: > > 5) I think I could send RTP packets to VLC and configure VLC to play > the samples, > but haven't tried this. An option similar to the VLC option (but more command line oriented perhaps) would be to use ffmpeg (with its player ffplay). Ffmpeg can receive for instance rtp (and also raw udp) over the network (example to listen for udp on specific port: "ffplay udp://localhost:4242?listen " - see https://trac.ffmpeg.org/wiki/StreamingGuide ) and it is flexible when it comes to formats such as raw PCM samples of various sample bit size (https://trac.ffmpeg.org/wiki/audio%20types ) with possibility to control buffer size to minimise latency. You can install it on the mac with ?brew install ffmpeg? if you use brew. /Stefan -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Thu Feb 18 15:21:08 2016 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 18 Feb 2016 15:21:08 +0100 Subject: [erlang-questions] Proposal for new state machine engine: gen_statem In-Reply-To: <20160210125648.GA85287@erix.ericsson.se> References: <20160210125648.GA85287@erix.ericsson.se> Message-ID: <20160218142108.GA83787@erix.ericsson.se> On Wed, Feb 10, 2016 at 01:56:48PM +0100, Raimo Niskanen wrote: > Hi all! > > I have created a pull request for a new state machine engine in the gen_* > framework. There is a heated discussion right now on GitHUB: > > https://github.com/erlang/otp/pull/960 > > Please contribute with an opinion. I just pushed a new version with the following major changes directly caused by the pull request discussion: * Change state function return format to tagged tuples {stop,...} and {next_event,...}. * An option 'callback_mode' : 'state_functions' |' handle_event_function' that selects between atom only states and any term states. * Rename the state operation 'insert_event' to 'next_event'. * Rename the state option 'retry' to 'postpone'. Updated documentation at: http://erlang.org/~raimo/doc-7.2/lib/stdlib-2.7/doc/html/gen_statem.html Updated OTP 1.2 .beam files at: https://github.com/erlang/otp/pull/erlang.org/%7Eraimo/OTP-18.2-stdlib-ebin-gen_statem.tgz keep the original around. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From jesper.louis.andersen@REDACTED Thu Feb 18 15:58:40 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 18 Feb 2016 15:58:40 +0100 Subject: [erlang-questions] Packets deduplication In-Reply-To: References: Message-ID: Keep a map, M :: #{ integer() => bool() } and ask if there is_key/2 on the ID's. Once in a while, you GC the map through maps:with(lists:seq(ID1, ID2), M) and make it small again. You would have to measure if this is actually faster/slower and by how much it is a slower solution. It isn't a priori given it would be either. jlouis/eministat will help determine if it is the case. On Thu, Feb 18, 2016 at 2:43 PM, Alexander Petrovsky wrote: > Yep, right now I try to do something similar with bitmap and sliding > window, that will shift. Could you explain about maps? > > Thanks! > > 2016-02-18 16:37 GMT+03:00 Jesper Louis Andersen < > jesper.louis.andersen@REDACTED>: > >> A simple solution that may work in the end is to keep track of a window >> of bits: >> >> * shift the window as the ID increases >> * check older packets against the window >> >> This gives you "SACK" support, is fast to check against and fast to >> expire old data due to the shifting. >> >> Simply using an ordered list or a map could also work if you clean it up >> eventually. It depends on how fast your CPU core is. >> >> >> On Thu, Feb 18, 2016 at 2:24 PM, Alexander Petrovsky >> wrote: >> >>> Ouch, I forgot to say, the IDs can be sparse, so they not really >>> monotonically, they just grow, and can be reordered. >>> >>> 2016-02-18 16:20 GMT+03:00 Danil Zagoskin : >>> >>>> Hi! >>>> >>>> If ID grows monotinically and you have no plans of recovering after >>>> packet reordering, then you can just keep the previous ID. >>>> - If CurID > PrevID, CurID is unique; >>>> - If CurID == PrevID, it is not unique; >>>> - If CurID < PrevID, it is a bug or reordering, let it crash. >>>> >>>> >>>> On Thu, Feb 18, 2016 at 3:01 PM, Alexander Petrovsky < >>>> askjuise@REDACTED> wrote: >>>> >>>>> Hi! >>>>> >>>>> I have the stream of packets with ID (int), and I need to check is the >>>>> packet is uniq (by ID) or not? >>>>> >>>>> Incoming rate is about 20k pps and ID is monotonically grows. What's >>>>> the best way and data structure fit for this problem? >>>>> >>>>> -- >>>>> ?????????? ????????? / Alexander Petrovsky, >>>>> >>>>> Skype: askjuise >>>>> Phone: +7 914 8 820 815 >>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>>> >>>> >>>> -- >>>> Danil Zagoskin | z@REDACTED >>>> >>> >>> >>> >>> -- >>> ?????????? ????????? / Alexander Petrovsky, >>> >>> Skype: askjuise >>> Phone: +7 914 8 820 815 >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> >> -- >> J. >> > > > > -- > ?????????? ????????? / Alexander Petrovsky, > > Skype: askjuise > Phone: +7 914 8 820 815 > > -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From joaohf@REDACTED Thu Feb 18 16:20:27 2016 From: joaohf@REDACTED (=?UTF-8?Q?Jo=C3=A3o_Henrique_Freitas?=) Date: Thu, 18 Feb 2016 13:20:27 -0200 Subject: [erlang-questions] tiny erlang In-Reply-To: References: Message-ID: Hi Joe, Take a look at https://github.com/joaohf/meta-erlang If it does not fit in your need, I could implement it Of course meta-erlang should be used with Yocto/Openembedded. Thanks On Thu, Feb 18, 2016 at 10:21 AM, Joe Armstrong wrote: > Hello > > I want to make a small erlang distribution for an embedded device - the > process > is rather simple: > > 1) strip the beam executable > 2) for each library in {stdlib, kernerl, compiler, sasl} do > Compress all the beam code and remove debug symbols > squash everthing into a single packed file > 3) Hack the code loader to read from the single packed beam file > 4) Tweak the shell scripts to start Erlang > > At a guess we're down to c. 6 MB > > I have done this on *several* occassions - and am getting a bit fed up > doing it over and over again. > > Has anybody done this recently - and should not something like this be > in the standard distribution? > > Cheers > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Jo?o Henrique Ferreira de Freitas - joaohf_at_gmail.com Campinas-SP-Brasil -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbhat7@REDACTED Thu Feb 18 14:18:35 2016 From: sbhat7@REDACTED (Sukumar Yethadka) Date: Thu, 18 Feb 2016 14:18:35 +0100 Subject: [erlang-questions] Erlang port of github's scientist Message-ID: Hello! Inspired by github's scientist [1], I have created an Erlang port for it called fuge [2]. Would love to have some feedback :-) 1. http://githubengineering.com/scientist/ 2. https://github.com/sthadka/fuge Thanks! Sukumar -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Thu Feb 18 18:09:06 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Thu, 18 Feb 2016 09:09:06 -0800 Subject: [erlang-questions] Packets deduplication In-Reply-To: References: Message-ID: Alexander -- what's the application supposed to do if it receives some packets in a non-monotonic order? If the answer is "ignore", then you could just check that the packet ID is greater than the last packet ID and everything else just falls out naturally. If you still need to process out-of-order packets, then consider each packet's "has been seen" value as a single bit. There is some packet count window C beyond which duplicates are not possible. Maintain a queue of bytes totalling at least C bits, as well as the id number at which the byte queue starts. When a new packet comes in, if it would require using one or more new bytes, push those to the front and drop the equivalent number off the back, and update the low byte queue id. Create a 'test byte' which has only the new bit set in the appropriate position for that byte. use binary-and ("band") to check to see if that bit has been seen in the appropriate byte in the byte queue; if the resulting byte is greater than zero, the bit has already been seen; do whatever processing is necessary. Otherwise, the bit has not been seen; set the bit in the byte queue, process, and continue. This is probably fastest in C/C++ but Erlang should be fine as long as the sparsity factor between packet ids is not generally gigantic (i.e. it's usually 1, rarely 100). If it's gigantic, then I'd probably use a queue of small pooled bloom filters in the same way so that I wouldn't have to do as much allocation. As always, measure and compare. F. On Thu, Feb 18, 2016 at 6:58 AM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > Keep a map, M :: #{ integer() => bool() } and ask if there is_key/2 on the > ID's. Once in a while, you GC the map through maps:with(lists:seq(ID1, > ID2), M) and make it small again. > > You would have to measure if this is actually faster/slower and by how > much it is a slower solution. It isn't a priori given it would be either. > jlouis/eministat will help determine if it is the case. > > > On Thu, Feb 18, 2016 at 2:43 PM, Alexander Petrovsky > wrote: > >> Yep, right now I try to do something similar with bitmap and sliding >> window, that will shift. Could you explain about maps? >> >> Thanks! >> >> 2016-02-18 16:37 GMT+03:00 Jesper Louis Andersen < >> jesper.louis.andersen@REDACTED>: >> >>> A simple solution that may work in the end is to keep track of a window >>> of bits: >>> >>> * shift the window as the ID increases >>> * check older packets against the window >>> >>> This gives you "SACK" support, is fast to check against and fast to >>> expire old data due to the shifting. >>> >>> Simply using an ordered list or a map could also work if you clean it up >>> eventually. It depends on how fast your CPU core is. >>> >>> >>> On Thu, Feb 18, 2016 at 2:24 PM, Alexander Petrovsky >> > wrote: >>> >>>> Ouch, I forgot to say, the IDs can be sparse, so they not really >>>> monotonically, they just grow, and can be reordered. >>>> >>>> 2016-02-18 16:20 GMT+03:00 Danil Zagoskin : >>>> >>>>> Hi! >>>>> >>>>> If ID grows monotinically and you have no plans of recovering after >>>>> packet reordering, then you can just keep the previous ID. >>>>> - If CurID > PrevID, CurID is unique; >>>>> - If CurID == PrevID, it is not unique; >>>>> - If CurID < PrevID, it is a bug or reordering, let it crash. >>>>> >>>>> >>>>> On Thu, Feb 18, 2016 at 3:01 PM, Alexander Petrovsky < >>>>> askjuise@REDACTED> wrote: >>>>> >>>>>> Hi! >>>>>> >>>>>> I have the stream of packets with ID (int), and I need to check is >>>>>> the packet is uniq (by ID) or not? >>>>>> >>>>>> Incoming rate is about 20k pps and ID is monotonically grows. What's >>>>>> the best way and data structure fit for this problem? >>>>>> >>>>>> -- >>>>>> ?????????? ????????? / Alexander Petrovsky, >>>>>> >>>>>> Skype: askjuise >>>>>> Phone: +7 914 8 820 815 >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Danil Zagoskin | z@REDACTED >>>>> >>>> >>>> >>>> >>>> -- >>>> ?????????? ????????? / Alexander Petrovsky, >>>> >>>> Skype: askjuise >>>> Phone: +7 914 8 820 815 >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >>> >>> -- >>> J. >>> >> >> >> >> -- >> ?????????? ????????? / Alexander Petrovsky, >> >> Skype: askjuise >> Phone: +7 914 8 820 815 >> >> > > > -- > J. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Thu Feb 18 18:27:56 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Thu, 18 Feb 2016 18:27:56 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: Message-ID: <6fd290cf-e2a3-4b30-bbeb-85c1d3c6cb24@email.android.com> Hi Joe, I was meaning at compile time, from files on disk. Having several modules in same file is dangerous. This could end by having two modules with same name in same file or two same module in two different files. With one file per module, called the same, it is unlikely to have this clash (well I see sometimes subdirectories under src/... Hirk) It was maybe the original goal of this constraint ? At runtime, I agree with you, hard to know. Le?18 f?vr. 2016 12:55 PM, Joe Armstrong a ?crit?: > > On Wed, Feb 17, 2016 at 6:44 PM, ?ric Pailleau wrote: > > Hi, > > Mho,? module name in file should be unique and same than rootname file. This avoid any module clash, not globally but in same app. > > I disagree - the "real" module name should be the SHA1 checksum of the > source code - the "user friendly name" should just be an alias to the > SHA1 checksum. > > The development system should hide or reveal this information > depending upon the context. > > When developing you probably want to talk about a module called "foo" > when you ship code or send code over the network you want know > *exactly* what you did. > > The problem with using a module name to identify the code is that the content of > the module changes with time and place. > > In a distributed system we can imagine upgrading code for some module > 'foo' bit that foo.erl is *different*on different machine because the > code has not yet arrived. > > Using "just" the module name will and does get you very rapidly into > "version nightmare land". > > Handling versions is much more complicated than you might think - and *nobody* > seems to have got it right - (apart from NiX - which might have got > things right, > but I'm not sure yet) > > Cheers > > /Joe > > > > > > > > > > BTW, namespace is probably something Erlang should handle in futur. This would avoid any module name clash. But anither story. > > > > Le 17 f?vr. 2016 6:38 PM, derek a ?crit : > >> > >> that -module(). line sound like redundant, if it can be mismatched, > >> why can't it be omitted? > >> compiler can derive the module name from the filename anyway; Java > >> also made that filename has to match class name, that is a design > >> mistake and unnecessary requirement, to my opinion; > >> > >> or on the other hand, if -module() is proved to be useful, why not > >> just ignore the filename, and always use what's defined in -module()? > >> in many modern language design like Go and Elixir, it's allowed to > >> define multiple Class/Module in one file, this will be really > >> flexible: > >> > >> %% file1.erl > >> -module(module1). > >> > >> %% [content of module1] > >> > >> -module(module2). > >> > >> %% [content of module2] > >> > >> -module(module3). > >> > >> ... > >> > >> On Wed, Feb 17, 2016 at 6:26 AM, Joe Armstrong wrote: > >> > Well suppose you sent the *content* of the file in a message - how > >> > would the receiver > >> > know the name of the module?. > >> > > >> > Actually the module attribute and file name don't have to be the same - but > >> > if they are not you'll break the autoloading mechanism. If the file name and > >> > module name are different you should know exactly what you're doing :-) > >> > > >> > Cheers > >> > > >> > /Joe > >> > > >> > On Wed, Feb 17, 2016 at 10:45 AM, Konstantin Vsochanov wrote: > >> >> Hi! > >> >> > >> >> I?m working with Erlang for two years now and enjoy every day of using > >> >> it. However, there are some strange? features I?m wondering about. One > >> >> of them -module() attribute. > >> >> > >> >>? The module name is strictly defined by the name of the .erl file. You > >> >> can?t change module name with -module() attribute. But the the -module() > >> >> attribute is mandatory. Why? Why we need -module() attribute at all? > >> >> Maybe we can make -module() optional in next Erlang release? So we don?t > >> >> need to repeat file name in first string of every .erl file. Or maybe I > >> >> miss something important? > >> >> > >> >> Feedback is appreciated! > >> >> > >> >> - Konstantin Voschanov > >> >> _______________________________________________ > >> >> erlang-questions mailing list > >> >> erlang-questions@REDACTED > >> >> http://erlang.org/mailman/listinfo/erlang-questions > >> > _______________________________________________ > >> > erlang-questions mailing list > >> > erlang-questions@REDACTED > >> > http://erlang.org/mailman/listinfo/erlang-questions > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions From hawk.mattsson@REDACTED Thu Feb 18 18:31:10 2016 From: hawk.mattsson@REDACTED (=?UTF-8?Q?H=C3=A5kan_Mattsson?=) Date: Thu, 18 Feb 2016 18:31:10 +0100 Subject: [erlang-questions] tiny erlang In-Reply-To: References: Message-ID: How much do you gain doing it your way compared to use Reltool to build the release? Assuming you instruct Reltool to only include your four applications, build compressed .ez files, strip the beams, skip unneccessary system files etc. /H?kan On Thu, Feb 18, 2016 at 1:21 PM, Joe Armstrong wrote: > Hello > > I want to make a small erlang distribution for an embedded device - the > process > is rather simple: > > 1) strip the beam executable > 2) for each library in {stdlib, kernerl, compiler, sasl} do > Compress all the beam code and remove debug symbols > squash everthing into a single packed file > 3) Hack the code loader to read from the single packed beam file > 4) Tweak the shell scripts to start Erlang > > At a guess we're down to c. 6 MB > > I have done this on *several* occassions - and am getting a bit fed up > doing it over and over again. > > Has anybody done this recently - and should not something like this be > in the standard distribution? > > Cheers > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Thu Feb 18 18:53:59 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Thu, 18 Feb 2016 18:53:59 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <6fd290cf-e2a3-4b30-bbeb-85c1d3c6cb24@email.android.com> Message-ID: Hi, Sha1 sum of source code is probably note the best solution due to the fact that same source file on Windows and Linux may differ due to different carriage return. Some are using tab and other spaces. And source with comments or without would be different in checksum but same Beam file. A canonical version of abstract code is probably better for a checksum and could be stored in a beam chunk. Le?18 f?vr. 2016 6:27 PM, ?ric Pailleau a ?crit?: > > Hi Joe, > I was meaning at compile time, from files on disk. > Having several modules in same file is dangerous. This could end by having two modules with same name in same file or two same module in two different files. > With one file per module, called the same, it is unlikely to have this clash (well I see sometimes subdirectories under src/... Hirk) > It was maybe the original goal of this constraint ? > > At runtime,? I agree with you, hard to know. > > Le?18 f?vr. 2016 12:55 PM, Joe Armstrong a ?crit?: > > > > On Wed, Feb 17, 2016 at 6:44 PM, ?ric Pailleau wrote: > > > Hi, > > > Mho,? module name in file should be unique and same than rootname file. This avoid any module clash, not globally but in same app. > > > > I disagree - the "real" module name should be the SHA1 checksum of the > > source code - the "user friendly name" should just be an alias to the > > SHA1 checksum. > > > > The development system should hide or reveal this information > > depending upon the context. > > > > When developing you probably want to talk about a module called "foo" > > when you ship code or send code over the network you want know > > *exactly* what you did. > > > > The problem with using a module name to identify the code is that the content of > > the module changes with time and place. > > > > In a distributed system we can imagine upgrading code for some module > > 'foo' bit that foo.erl is *different*on different machine because the > > code has not yet arrived. > > > > Using "just" the module name will and does get you very rapidly into > > "version nightmare land". > > > > Handling versions is much more complicated than you might think - and *nobody* > > seems to have got it right - (apart from NiX - which might have got > > things right, > > but I'm not sure yet) > > > > Cheers > > > > /Joe > > > > > > > > > > > > > > > > > > > BTW, namespace is probably something Erlang should handle in futur. This would avoid any module name clash. But anither story. > > > > > > Le 17 f?vr. 2016 6:38 PM, derek a ?crit : > > >> > > >> that -module(). line sound like redundant, if it can be mismatched, > > >> why can't it be omitted? > > >> compiler can derive the module name from the filename anyway; Java > > >> also made that filename has to match class name, that is a design > > >> mistake and unnecessary requirement, to my opinion; > > >> > > >> or on the other hand, if -module() is proved to be useful, why not > > >> just ignore the filename, and always use what's defined in -module()? > > >> in many modern language design like Go and Elixir, it's allowed to > > >> define multiple Class/Module in one file, this will be really > > >> flexible: > > >> > > >> %% file1.erl > > >> -module(module1). > > >> > > >> %% [content of module1] > > >> > > >> -module(module2). > > >> > > >> %% [content of module2] > > >> > > >> -module(module3). > > >> > > >> ... > > >> > > >> On Wed, Feb 17, 2016 at 6:26 AM, Joe Armstrong wrote: > > >> > Well suppose you sent the *content* of the file in a message - how > > >> > would the receiver > > >> > know the name of the module?. > > >> > > > >> > Actually the module attribute and file name don't have to be the same - but > > >> > if they are not you'll break the autoloading mechanism. If the file name and > > >> > module name are different you should know exactly what you're doing :-) > > >> > > > >> > Cheers > > >> > > > >> > /Joe > > >> > > > >> > On Wed, Feb 17, 2016 at 10:45 AM, Konstantin Vsochanov wrote: > > >> >> Hi! > > >> >> > > >> >> I?m working with Erlang for two years now and enjoy every day of using > > >> >> it. However, there are some strange? features I?m wondering about. One > > >> >> of them -module() attribute. > > >> >> > > >> >>? The module name is strictly defined by the name of the .erl file. You > > >> >> can?t change module name with -module() attribute. But the the -module() > > >> >> attribute is mandatory. Why? Why we need -module() attribute at all? > > >> >> Maybe we can make -module() optional in next Erlang release? So we don?t > > >> >> need to repeat file name in first string of every .erl file. Or maybe I > > >> >> miss something important? > > >> >> > > >> >> Feedback is appreciated! > > >> >> > > >> >> - Konstantin Voschanov > > >> >> _______________________________________________ > > >> >> erlang-questions mailing list > > >> >> erlang-questions@REDACTED > > >> >> http://erlang.org/mailman/listinfo/erlang-questions > > >> > _______________________________________________ > > >> > erlang-questions mailing list > > >> > erlang-questions@REDACTED > > >> > http://erlang.org/mailman/listinfo/erlang-questions > > >> _______________________________________________ > > >> erlang-questions mailing list > > >> erlang-questions@REDACTED > > >> http://erlang.org/mailman/listinfo/erlang-questions From tty.erlang@REDACTED Thu Feb 18 19:01:35 2016 From: tty.erlang@REDACTED (T Ty) Date: Thu, 18 Feb 2016 18:01:35 +0000 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <6fd290cf-e2a3-4b30-bbeb-85c1d3c6cb24@email.android.com> Message-ID: Would -vsn be enough ? its MD5 and not Sha1 though. On Thu, Feb 18, 2016 at 5:53 PM, ?ric Pailleau wrote: > Hi, > Sha1 sum of source code is probably note the best solution due to the fact > that same source file on Windows and Linux may differ due to different > carriage return. Some are using tab and other spaces. And source with > comments or without would be different in checksum but same Beam file. > A canonical version of abstract code is probably better for a checksum and > could be stored in a beam chunk. > > Le 18 f?vr. 2016 6:27 PM, ?ric Pailleau a > ?crit : > > > > Hi Joe, > > I was meaning at compile time, from files on disk. > > Having several modules in same file is dangerous. This could end by > having two modules with same name in same file or two same module in two > different files. > > With one file per module, called the same, it is unlikely to have this > clash (well I see sometimes subdirectories under src/... Hirk) > > It was maybe the original goal of this constraint ? > > > > At runtime, I agree with you, hard to know. > > > > Le 18 f?vr. 2016 12:55 PM, Joe Armstrong a ?crit : > > > > > > On Wed, Feb 17, 2016 at 6:44 PM, ?ric Pailleau < > eric.pailleau@REDACTED> wrote: > > > > Hi, > > > > Mho, module name in file should be unique and same than rootname > file. This avoid any module clash, not globally but in same app. > > > > > > I disagree - the "real" module name should be the SHA1 checksum of the > > > source code - the "user friendly name" should just be an alias to the > > > SHA1 checksum. > > > > > > The development system should hide or reveal this information > > > depending upon the context. > > > > > > When developing you probably want to talk about a module called "foo" > > > when you ship code or send code over the network you want know > > > *exactly* what you did. > > > > > > The problem with using a module name to identify the code is that the > content of > > > the module changes with time and place. > > > > > > In a distributed system we can imagine upgrading code for some module > > > 'foo' bit that foo.erl is *different*on different machine because the > > > code has not yet arrived. > > > > > > Using "just" the module name will and does get you very rapidly into > > > "version nightmare land". > > > > > > Handling versions is much more complicated than you might think - and > *nobody* > > > seems to have got it right - (apart from NiX - which might have got > > > things right, > > > but I'm not sure yet) > > > > > > Cheers > > > > > > /Joe > > > > > > > > > > > > > > > > > > > > > > > > > > > > BTW, namespace is probably something Erlang should handle in futur. > This would avoid any module name clash. But anither story. > > > > > > > > Le 17 f?vr. 2016 6:38 PM, derek a ?crit : > > > >> > > > >> that -module(). line sound like redundant, if it can be mismatched, > > > >> why can't it be omitted? > > > >> compiler can derive the module name from the filename anyway; Java > > > >> also made that filename has to match class name, that is a design > > > >> mistake and unnecessary requirement, to my opinion; > > > >> > > > >> or on the other hand, if -module() is proved to be useful, why not > > > >> just ignore the filename, and always use what's defined in > -module()? > > > >> in many modern language design like Go and Elixir, it's allowed to > > > >> define multiple Class/Module in one file, this will be really > > > >> flexible: > > > >> > > > >> %% file1.erl > > > >> -module(module1). > > > >> > > > >> %% [content of module1] > > > >> > > > >> -module(module2). > > > >> > > > >> %% [content of module2] > > > >> > > > >> -module(module3). > > > >> > > > >> ... > > > >> > > > >> On Wed, Feb 17, 2016 at 6:26 AM, Joe Armstrong > wrote: > > > >> > Well suppose you sent the *content* of the file in a message - how > > > >> > would the receiver > > > >> > know the name of the module?. > > > >> > > > > >> > Actually the module attribute and file name don't have to be the > same - but > > > >> > if they are not you'll break the autoloading mechanism. If the > file name and > > > >> > module name are different you should know exactly what you're > doing :-) > > > >> > > > > >> > Cheers > > > >> > > > > >> > /Joe > > > >> > > > > >> > On Wed, Feb 17, 2016 at 10:45 AM, Konstantin Vsochanov < > kost@REDACTED> wrote: > > > >> >> Hi! > > > >> >> > > > >> >> I?m working with Erlang for two years now and enjoy every day of > using > > > >> >> it. However, there are some strange features I?m wondering > about. One > > > >> >> of them -module() attribute. > > > >> >> > > > >> >> The module name is strictly defined by the name of the .erl > file. You > > > >> >> can?t change module name with -module() attribute. But the the > -module() > > > >> >> attribute is mandatory. Why? Why we need -module() attribute at > all? > > > >> >> Maybe we can make -module() optional in next Erlang release? So > we don?t > > > >> >> need to repeat file name in first string of every .erl file. Or > maybe I > > > >> >> miss something important? > > > >> >> > > > >> >> Feedback is appreciated! > > > >> >> > > > >> >> - Konstantin Voschanov > > > >> >> _______________________________________________ > > > >> >> erlang-questions mailing list > > > >> >> erlang-questions@REDACTED > > > >> >> http://erlang.org/mailman/listinfo/erlang-questions > > > >> > _______________________________________________ > > > >> > erlang-questions mailing list > > > >> > erlang-questions@REDACTED > > > >> > http://erlang.org/mailman/listinfo/erlang-questions > > > >> _______________________________________________ > > > >> erlang-questions mailing list > > > >> erlang-questions@REDACTED > > > >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Thu Feb 18 19:28:24 2016 From: dmkolesnikov@REDACTED (dmkolesnikov@REDACTED) Date: Thu, 18 Feb 2016 20:28:24 +0200 Subject: [erlang-questions] Packets deduplication In-Reply-To: References: Message-ID: <7AFC1CFF-ED22-4EE5-AFB5-082B1E2FEB70@gmail.com> Hello Alexandr, The right data structure is either bloom filter or scalable bloom filter. I've played with standard bloom here https://github.com/fogfish/feta/blob/master/src/bloom.erl 20k might require you to do tuning and reflect the filter to ETS. The ETS as such or any cache based on ETS might help you as well. Cache is needed to implement TTL for your ID's Best Regards, Dmitry Sent from my iPhone > On 18 Feb 2016, at 14:01, Alexander Petrovsky wrote: > > Hi! > > I have the stream of packets with ID (int), and I need to check is the packet is uniq (by ID) or not? > > Incoming rate is about 20k pps and ID is monotonically grows. What's the best way and data structure fit for this problem? > > -- > ?????????? ????????? / Alexander Petrovsky, > > Skype: askjuise > Phone: +7 914 8 820 815 > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From eric.pailleau@REDACTED Thu Feb 18 19:32:55 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Thu, 18 Feb 2016 19:32:55 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: Message-ID: <98abf496-fccf-4bf5-ba44-4e00d7ceb187@email.android.com> An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Feb 18 19:45:16 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 18 Feb 2016 19:45:16 +0100 Subject: [erlang-questions] Packets deduplication In-Reply-To: <7AFC1CFF-ED22-4EE5-AFB5-082B1E2FEB70@gmail.com> References: <7AFC1CFF-ED22-4EE5-AFB5-082B1E2FEB70@gmail.com> Message-ID: On Thu, Feb 18, 2016 at 7:28 PM, wrote: > The right data structure is either bloom filter or scalable bloom filter. You also need to delete elements, which a cuckoo filter can do. The problem with bloom filters in this situation is the probability of error as far as I see. They either say NOPE or MAYBE??? The other problem is that it probably wont beat integer operations on 60 bit words since it requires a hash. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Thu Feb 18 19:56:22 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Thu, 18 Feb 2016 10:56:22 -0800 Subject: [erlang-questions] Packets deduplication In-Reply-To: References: <7AFC1CFF-ED22-4EE5-AFB5-082B1E2FEB70@gmail.com> Message-ID: you'd use the bloom filter first to determine if it's possible that you've seen it before, and then if you get a hit, just do a lookup into some other structure. Depending on your hash functions and the sparsity, it could be significantly faster; consider, for example, if the sequence is 1,2,3,4,2^1024,2^1024+1,2^1024+2... just about anything except a bloom filter is going to have possibly-interesting behavior when the 2^1024 shows up. The bloom filter may give you some false positives and will generally be slower, but if you're trying to maintain a packet train for, like, remote medical surgery, you might accept the extra space and the slightly higher average latency for less jitter. On Thu, Feb 18, 2016 at 10:45 AM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > On Thu, Feb 18, 2016 at 7:28 PM, wrote: > >> The right data structure is either bloom filter or scalable bloom filter. > > > You also need to delete elements, which a cuckoo filter can do. > > The problem with bloom filters in this situation is the probability of > error as far as I see. They either say NOPE or MAYBE??? The other problem > is that it probably wont beat integer operations on 60 bit words since it > requires a hash. > > > -- > J. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Thu Feb 18 21:19:27 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 18 Feb 2016 23:19:27 +0300 Subject: [erlang-questions] An elegant implementation of multi pollset for Erlang VM In-Reply-To: References: Message-ID: Very cool! But frankly speaking, not clear what problem are you solving. Our Flussonic after removing 70% of NIF and other C code (that were supposed to be optimizing) is serving right now about 10 gbit/s from single machine, taking about 12 cores from 24 core Xeon on 2.1 Ghz. It is not easy, because many of thousands of connected users are connected via wifi or rotten copper so it is not a very easy task to deliver them video. Test benchmarks on localhost show about 25-30 gbit/s, but it is on localhost and benchmark which is a lie. But we have small amount of users: about 10-15 thousands of active connections. Comet server can have much more. What is your load profile? -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Thu Feb 18 21:24:34 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 18 Feb 2016 23:24:34 +0300 Subject: [erlang-questions] Packets deduplication In-Reply-To: References: <7AFC1CFF-ED22-4EE5-AFB5-082B1E2FEB70@gmail.com> Message-ID: Perhaps structure for keeping fetched packets can look like: WindowBeginning then list of binaries of N bytes each, which are bitmaps. Some of binaries can be replaced by atom 'true' if all bits are set and WindowEnd My question here is: when to send retransmit request? I suppose that this question is very close to TCP problems, so it cannot have a simple answer, but maybe you have some good enough answer for you? -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Thu Feb 18 21:29:03 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 18 Feb 2016 23:29:03 +0300 Subject: [erlang-questions] live audio output from erlang on mac In-Reply-To: References: Message-ID: I would write a small C program around libao and use it via port. forget about nif and drivers, because Mac + erlang + hardware access is something like broken: many of OSX API require to be run strictly from main thread which is used by erlang and nif will not get there. libao is good, if you need, I can share working example of code. -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Thu Feb 18 21:30:29 2016 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 18 Feb 2016 21:30:29 +0100 Subject: [erlang-questions] where does output go in detached mode In-Reply-To: References: Message-ID: /Richard 2016-02-18 12:46 GMT+01:00 Joe Armstrong : > Hello, > > I start erlang with > > $ erl -detached > > and have two questions: > > 1) If program does io:format(Format, Data) where does the output go? > > Does the output just vanish, or do I have to unclog some buffers > eventually? > > 2) Can I attach to the process later? > > And if so does this work on the three major platforms OS-X, linux and > windows? > > Cheers > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Thu Feb 18 21:42:57 2016 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Thu, 18 Feb 2016 21:42:57 +0100 Subject: [erlang-questions] where does output go in detached mode In-Reply-To: References: Message-ID: (No idea why gmail decided to send my previous reply before I got a chance to actually type anything; please ignore that one.) 2016-02-18 12:46 GMT+01:00 Joe Armstrong : > Hello, > > I start erlang with > > $ erl -detached > > and have two questions: > > 1) If program does io:format(Format, Data) where does the output go? > /dev/null > Does the output just vanish, or do I have to unclog some buffers > eventually? > It's gone. 2) Can I attach to the process later? > Yes, but only if you launch erl via the run_erl wrapper; see http://erlang.org/doc/man/run_erl.html You can then attach to the running Beam later using to_erl (which seems to be still not documented...). And if so does this work on the three major platforms OS-X, linux and > windows? > The docs say "on Solaris" in the summary line, which is probably no longer the most popular platform for Erlang, but yes, it's for Linux too - and I would guess it works on BSD/OSX and similar, but you're probably supposed to use something completely different on Windows. /Richard -------------- next part -------------- An HTML attachment was scrubbed... URL: From duncan@REDACTED Thu Feb 18 21:43:05 2016 From: duncan@REDACTED (duncan@REDACTED) Date: Thu, 18 Feb 2016 13:43:05 -0700 Subject: [erlang-questions] Why we need a -module() attribute? Message-ID: <20160218134305.7e43b23f706d1a78218bd3e1c66e57ee.8fd0e0cc87.mailapi@mailapi23.secureserver.net> Eric, I'd argue the different source codes you mentioned are actually different source codes even if they compile to the same beam file. Ie it would be ok they have a different checksum and different version. -------- Original Message -------- Subject: Re: [erlang-questions] Why we need a -module() attribute? From: ?ric Pailleau Date: Feb 18, 2016 12:54 PM To: "Joe Armstrong" CC: "Erlang Questions" Hi, Sha1 sum of source code is probably note the best solution due to the fact that same source file on Windows and Linux may differ due to different carriage return. Some are using tab and other spaces. And source with comments or without would be different in checksum but same Beam file. A canonical version of abstract code is probably better for a checksum and could be stored in a beam chunk. Le?18 f?vr. 2016 6:27 PM, ?ric Pailleau a ?crit?: > > Hi Joe, > I was meaning at compile time, from files on disk. > Having several modules in same file is dangerous. This could end by having two modules with same name in same file or two same module in two different files. > With one file per module, called the same, it is unlikely to have this clash (well I see sometimes subdirectories under src/... Hirk) > It was maybe the original goal of this constraint ? > > At runtime,? I agree with you, hard to know. > > Le?18 f?vr. 2016 12:55 PM, Joe Armstrong a ?crit?: > > > > On Wed, Feb 17, 2016 at 6:44 PM, ?ric Pailleau wrote: > > > Hi, > > > Mho,? module name in file should be unique and same than rootname file. This avoid any module clash, not globally but in same app. > > > > I disagree - the "real" module name should be the SHA1 checksum of the > > source code - the "user friendly name" should just be an alias to the > > SHA1 checksum. > > > > The development system should hide or reveal this information > > depending upon the context. > > > > When developing you probably want to talk about a module called "foo" > > when you ship code or send code over the network you want know > > *exactly* what you did. > > > > The problem with using a module name to identify the code is that the content of > > the module changes with time and place. > > > > In a distributed system we can imagine upgrading code for some module > > 'foo' bit that foo.erl is *different*on different machine because the > > code has not yet arrived. > > > > Using "just" the module name will and does get you very rapidly into > > "version nightmare land". > > > > Handling versions is much more complicated than you might think - and *nobody* > > seems to have got it right - (apart from NiX - which might have got > > things right, > > but I'm not sure yet) > > > > Cheers > > > > /Joe > > > > > > > > > > > > > > > > > > > BTW, namespace is probably something Erlang should handle in futur. This would avoid any module name clash. But anither story. > > > > > > Le 17 f?vr. 2016 6:38 PM, derek a ?crit : > > >> > > >> that -module(). line sound like redundant, if it can be mismatched, > > >> why can't it be omitted? > > >> compiler can derive the module name from the filename anyway; Java > > >> also made that filename has to match class name, that is a design > > >> mistake and unnecessary requirement, to my opinion; > > >> > > >> or on the other hand, if -module() is proved to be useful, why not > > >> just ignore the filename, and always use what's defined in -module()? > > >> in many modern language design like Go and Elixir, it's allowed to > > >> define multiple Class/Module in one file, this will be really > > >> flexible: > > >> > > >> %% file1.erl > > >> -module(module1). > > >> > > >> %% [content of module1] > > >> > > >> -module(module2). > > >> > > >> %% [content of module2] > > >> > > >> -module(module3). > > >> > > >> ... > > >> > > >> On Wed, Feb 17, 2016 at 6:26 AM, Joe Armstrong wrote: > > >> > Well suppose you sent the *content* of the file in a message - how > > >> > would the receiver > > >> > know the name of the module?. > > >> > > > >> > Actually the module attribute and file name don't have to be the same - but > > >> > if they are not you'll break the autoloading mechanism. If the file name and > > >> > module name are different you should know exactly what you're doing :-) > > >> > > > >> > Cheers > > >> > > > >> > /Joe > > >> > > > >> > On Wed, Feb 17, 2016 at 10:45 AM, Konstantin Vsochanov wrote: > > >> >> Hi! > > >> >> > > >> >> I?m working with Erlang for two years now and enjoy every day of using > > >> >> it. However, there are some strange? features I?m wondering about. One > > >> >> of them -module() attribute. > > >> >> > > >> >>? The module name is strictly defined by the name of the .erl file. You > > >> >> can?t change module name with -module() attribute. But the the -module() > > >> >> attribute is mandatory. Why? Why we need -module() attribute at all? > > >> >> Maybe we can make -module() optional in next Erlang release? So we don?t > > >> >> need to repeat file name in first string of every .erl file. Or maybe I > > >> >> miss something important? > > >> >> > > >> >> Feedback is appreciated! > > >> >> > > >> >> - Konstantin Voschanov > > >> >> _______________________________________________ > > >> >> erlang-questions mailing list > > >> >> erlang-questions@REDACTED > > >> >> http://erlang.org/mailman/listinfo/erlang-questions > > >> > _______________________________________________ > > >> > erlang-questions mailing list > > >> > erlang-questions@REDACTED > > >> > http://erlang.org/mailman/listinfo/erlang-questions > > >> _______________________________________________ > > >> erlang-questions mailing list > > >> erlang-questions@REDACTED > > >> http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From irfah.senuoy@REDACTED Thu Feb 18 21:42:01 2016 From: irfah.senuoy@REDACTED (=?UTF-8?Q?Youn=C3=A8s_HAFRI?=) Date: Thu, 18 Feb 2016 21:42:01 +0100 Subject: [erlang-questions] An elegant implementation of multi pollset for Erlang VM In-Reply-To: References: Message-ID: Compiles only on Linux and the "accept" call never returns in my case: > {ok, S} = gen_socket:accept( L ). ArchLinux 64-bit (kernel 3.14.51), RAM 128gB, Erlang R16B03. Is it production ready??? I join Max here: 1gb/sec isn't that hard to achieve in Erlang with enough Cores. Le Thursday, February 18, 2016, Max Lapshin a ?crit : > Very cool! > > But frankly speaking, not clear what problem are you solving. > > > Our Flussonic after removing 70% of NIF and other C code (that were > supposed to be optimizing) is serving right now about 10 gbit/s from single > machine, taking about 12 cores from 24 core Xeon on 2.1 Ghz. > > It is not easy, because many of thousands of connected users are connected > via wifi or rotten copper so it is not a very easy task to deliver them > video. > > Test benchmarks on localhost show about 25-30 gbit/s, but it is on > localhost and benchmark which is a lie. > > > But we have small amount of users: about 10-15 thousands of active > connections. Comet server can have much more. What is your load profile? > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Thu Feb 18 22:05:51 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Thu, 18 Feb 2016 22:05:51 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <20160218134305.7e43b23f706d1a78218bd3e1c66e57ee.8fd0e0cc87.mailapi@mailapi23.secureserver.net> Message-ID: Hi Duncan, Yes, sure. But even it is very very unlikely for sha1, collision could happen between two differents source codes if based on non strict format, due to comment feeding. And I think it is better to know that two modules are same even if source code is not exactly the same. I think to year copyright changing in source comments and having no impact to running Beam. Git is good for source, sha1sum in Beam would be good to know what module is running. Le?18 f?vr. 2016 9:43 PM, duncan@REDACTED a ?crit?: > > Eric, > I'd argue the different source codes you mentioned are actually different source codes even if they compile to the same beam file. Ie it would be ok they have a different checksum and different version. > > -------- Original Message -------- > Subject: Re: [erlang-questions] Why we need a -module() attribute? > From: ?ric Pailleau > Date: Feb 18, 2016 12:54 PM > To: "Joe Armstrong" > CC: "Erlang Questions" > > Hi, > Sha1 sum of source code is probably note the best solution due to the fact that same source file on Windows and Linux may differ due to different carriage return. Some are using tab and other spaces. And source with comments or without would be different in checksum but same Beam file. > A canonical version of abstract code is probably better for a checksum and could be stored in a beam chunk. > > Le?18 f?vr. 2016 6:27 PM, ?ric Pailleau a ?crit?: > > > > Hi Joe, > > I was meaning at compile time, from files on disk. > > Having several modules in same file is dangerous. This could end by having two modules with same name in same file or two same module in two different files. > > With one file per module, called the same, it is unlikely to have this clash (well I see sometimes subdirectories under src/... Hirk) > > It was maybe the original goal of this constraint ? > > > > At runtime,? I agree with you, hard to know. > > > > Le?18 f?vr. 2016 12:55 PM, Joe Armstrong a ?crit?: > > > > > > On Wed, Feb 17, 2016 at 6:44 PM, ?ric Pailleau wrote: > > > > Hi, > > > > Mho,? module name in file should be unique and same than rootname file. This avoid any module clash, not globally but in same app. > > > > > > I disagree - the "real" module name should be the SHA1 checksum of the > > > source code - the "user friendly name" should just be an alias to the > > > SHA1 checksum. > > > > > > The development system should hide or reveal this information > > > depending upon the context. > > > > > > When developing you probably want to talk about a module called "foo" > > > when you ship code or send code over the network you want know > > > *exactly* what you did. > > > > > > The problem with using a module name to identify the code is that the content of > > > the module changes with time and place. > > > > > > In a distributed system we can imagine upgrading code for some module > > > 'foo' bit that foo.erl is *different*on different machine because the > > > code has not yet arrived. > > > > > > Using "just" the module name will and does get you very rapidly into > > > "version nightmare land". > > > > > > Handling versions is much more complicated than you might think - and *nobody* > > > seems to have got it right - (apart from NiX - which might have got > > > things right, > > > but I'm not sure yet) > > > > > > Cheers > > > > > > /Joe > > > > > > > > > > > > > > > > > > > > > > > > > > > > BTW, namespace is probably something Erlang should handle in futur. This would avoid any module name clash. But anither story. > > > > > > > > Le 17 f?vr. 2016 6:38 PM, derek a ?crit : > > > >> > > > >> that -module(). line sound like redundant, if it can be mismatched, > > > >> why can't it be omitted? > > > >> compiler can derive the module name from the filename anyway; Java > > > >> also made that filename has to match class name, that is a design > > > >> mistake and unnecessary requirement, to my opinion; > > > >> > > > >> or on the other hand, if -module() is proved to be useful, why not > > > >> just ignore the filename, and always use what's defined in -module()? > > > >> in many modern language design like Go and Elixir, it's allowed to > > > >> define multiple Class/Module in one file, this will be really > > > >> flexible: > > > >> > > > >> %% file1.erl > > > >> -module(module1). > > > >> > > > >> %% [content of module1] > > > >> > > > >> -module(module2). > > > >> > > > >> %% [content of module2] > > > >> > > > >> -module(module3). > > > >> > > > >> ... > > > >> > > > >> On Wed, Feb 17, 2016 at 6:26 AM, Joe Armstrong wrote: > > > >> > Well suppose you sent the *content* of the file in a message - how > > > >> > would the receiver > > > >> > know the name of the module?. > > > >> > > > > >> > Actually the module attribute and file name don't have to be the same - but > > > >> > if they are not you'll break the autoloading mechanism. If the file name and > > > >> > module name are different you should know exactly what you're doing :-) > > > >> > > > > >> > Cheers > > > >> > > > > >> > /Joe > > > >> > > > > >> > On Wed, Feb 17, 2016 at 10:45 AM, Konstantin Vsochanov wrote: > > > >> >> Hi! > > > >> >> > > > >> >> I?m working with Erlang for two years now and enjoy every day of using > > > >> >> it. However, there are some strange? features I?m wondering about. One > > > >> >> of them -module() attribute. > > > >> >> > > > >> >>? The module name is strictly defined by the name of the .erl file. You > > > >> >> can?t change module name with -module() attribute. But the the -module() > > > >> >> attribute is mandatory. Why? Why we need -module() attribute at all? > > > >> >> Maybe we can make -module() optional in next Erlang release? So we don?t > > > >> >> need to repeat file name in first string of every .erl file. Or maybe I > > > >> >> miss something important? > > > >> >> > > > >> >> Feedback is appreciated! > > > >> >> > > > >> >> - Konstantin Voschanov > > > >> >> _______________________________________________ > > > >> >> erlang-questions mailing list > > > >> >> erlang-questions@REDACTED > > > >> >> http://erlang.org/mailman/listinfo/erlang-questions > > > >> > _______________________________________________ > > > >> > erlang-questions mailing list > > > >> > erlang-questions@REDACTED > > > >> > http://erlang.org/mailman/listinfo/erlang-questions > > > >> _______________________________________________ > > > >> erlang-questions mailing list > > > >> erlang-questions@REDACTED > > > >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From bernard@REDACTED Thu Feb 18 22:49:08 2016 From: bernard@REDACTED (Bernard Duggan) Date: Fri, 19 Feb 2016 08:49:08 +1100 Subject: [erlang-questions] where does output go in detached mode In-Reply-To: References: Message-ID: Hi Joe, >From your question, you might want to have a look at erld ( https://github.com/ShoreTel-Inc/erld). It provides a wrapper around Erlang which will capture the output of a "detached" Erlang node and log it. It also provides a bunch of other daemon-like behaviour (capturing signals for log rotation, startup return values etc). It doesn't currently allow for reattachment, but you can always --remsh into the node assuming it's named. Cheers, Bernard On Fri, Feb 19, 2016 at 7:42 AM, Richard Carlsson < carlsson.richard@REDACTED> wrote: > (No idea why gmail decided to send my previous reply before I got a chance > to actually type anything; please ignore that one.) > > 2016-02-18 12:46 GMT+01:00 Joe Armstrong : > >> Hello, >> >> I start erlang with >> >> $ erl -detached >> >> and have two questions: >> >> 1) If program does io:format(Format, Data) where does the output go? >> > > /dev/null > > >> Does the output just vanish, or do I have to unclog some buffers >> eventually? >> > > It's gone. > > 2) Can I attach to the process later? >> > > Yes, but only if you launch erl via the run_erl wrapper; see > http://erlang.org/doc/man/run_erl.html > You can then attach to the running Beam later using to_erl (which seems to > be still not documented...). > > And if so does this work on the three major platforms OS-X, linux and >> windows? >> > > The docs say "on Solaris" in the summary line, which is probably no longer > the most popular platform for Erlang, but yes, it's for Linux too - and I > would guess it works on BSD/OSX and similar, but you're probably supposed > to use something completely different on Windows. > > /Richard > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Thu Feb 18 23:07:15 2016 From: tony@REDACTED (Tony Rogvall) Date: Thu, 18 Feb 2016 23:07:15 +0100 Subject: [erlang-questions] where does output go in detached mode In-Reply-To: References: Message-ID: /dev/null ? :-) > On 18 feb 2016, at 21:30, Richard Carlsson wrote: > > > > > /Richard > > 2016-02-18 12:46 GMT+01:00 Joe Armstrong >: > Hello, > > I start erlang with > > $ erl -detached > > and have two questions: > > 1) If program does io:format(Format, Data) where does the output go? > > Does the output just vanish, or do I have to unclog some buffers eventually? > > 2) Can I attach to the process later? > > And if so does this work on the three major platforms OS-X, linux and > windows? > > Cheers > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From ok@REDACTED Thu Feb 18 23:54:42 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 19 Feb 2016 11:54:42 +1300 Subject: [erlang-questions] logical_processors_available unknown on mac In-Reply-To: References: Message-ID: <56C64BB2.9060704@cs.otago.ac.nz> On 19/02/16 12:46 am, Alin Popa wrote: > On Wed, Feb 17, 2016 at 1:25 PM, Alin Popa > wrote: > > I was recently trying to use > `erlang:system_info(logical_processors_available).` on my mac, > and getting `unknown` atom back; anyone has any idea if this > is really by design? > | head % sysctl -a | grep cpu | head hw.ncpu = 4 hw.cpufrequency = 3200000000 to= 4 hw.ncpu: 4 hw.activecpu: 4 hw.physicalcpu: 4 hw.physicalcpu_max: 4 hw.logicalcpu: 4 hw.logicalcpu_max: 4 hw.cputype: 7 You can get at this with, e.g., int ans; size_t anslen = sizeof ans; sysctlbyname("hw.activecpu", &ans, &anslen, 0, 0) For that matter, sysconf(_SC_NPROCESSORS_ONLN) and sysconf(_SC_NPROCESSORS_CONF) are supported. * hw.activecpu - The number of processors currently available for executing threads. * Use this number to determine the number threads to create in SMP aware applications. * This number can change when power management modes are changed. There are so many ways to count the number of CPUs on a Mac. I suspect that hw.activecpu is the one to use; if it isn't, then hw.logicalcpu may be the one. From alin.popa@REDACTED Fri Feb 19 00:42:04 2016 From: alin.popa@REDACTED (Alin Popa) Date: Thu, 18 Feb 2016 23:42:04 +0000 Subject: [erlang-questions] logical_processors_available unknown on mac In-Reply-To: <56C64BB2.9060704@cs.otago.ac.nz> References: <56C64BB2.9060704@cs.otago.ac.nz> Message-ID: Yeah, I know there are some sysctl OSX flags that can be looked up, the thing that I don't know if those are the same as those used on the Linux (or other?) implementations. Alin On Thu, Feb 18, 2016 at 10:54 PM, Richard A. O'Keefe wrote: > > > On 19/02/16 12:46 am, Alin Popa wrote: > >> On Wed, Feb 17, 2016 at 1:25 PM, Alin Popa > alin.popa@REDACTED>> wrote: >> >> I was recently trying to use >> `erlang:system_info(logical_processors_available).` on my mac, >> and getting `unknown` atom back; anyone has any idea if this >> is really by design? >> >> > | head > % sysctl -a | grep cpu | head > hw.ncpu = 4 > hw.cpufrequency = 3200000000 > to= 4 > hw.ncpu: 4 > hw.activecpu: 4 > hw.physicalcpu: 4 > hw.physicalcpu_max: 4 > hw.logicalcpu: 4 > hw.logicalcpu_max: 4 > hw.cputype: 7 > > You can get at this with, e.g., > int ans; size_t anslen = sizeof ans; > sysctlbyname("hw.activecpu", &ans, &anslen, 0, 0) > For that matter, > sysconf(_SC_NPROCESSORS_ONLN) > and > sysconf(_SC_NPROCESSORS_CONF) > are supported. > > > * hw.activecpu - The number of processors currently > available for executing threads. > * Use this number to determine the number > threads to create in SMP aware applications. > * This number can change when power > management modes are changed. > > There are so many ways to count the number of CPUs on a Mac. > I suspect that hw.activecpu is the one to use; if it isn't, then > hw.logicalcpu may be the one. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Feb 19 01:40:00 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 19 Feb 2016 13:40:00 +1300 Subject: [erlang-questions] logical_processors_available unknown on mac In-Reply-To: References: <56C64BB2.9060704@cs.otago.ac.nz> Message-ID: <56C66460.4010000@cs.otago.ac.nz> On 19/02/16 12:42 pm, Alin Popa wrote: > Yeah, I know there are some sysctl OSX flags that can be looked up, > the thing that I don't know if those are the same as those used on the > Linux (or other?) implementations. I just checked two different boxes running two different versions of Linux. Both had the sysctl program. Neither had sysctl(3) in the library part of the manual. Both included the sysctl() function but not the sysctlbyname() function. Neither of them reported ANY 'hw' or 'cpu' properties. sysctl was added to the BSDs after Solaris engulfed System V, so Solaris lacks it. https://www.gnu.org/software/gnulib/manual/html_node/sysctl.html says of sysctl() that This function is missing on some platforms: Minix 3.1.8, AIX 5.1, HP-UX 11, IRIX 5.3, OSF/1 5.1, Solaris 11 2011-11, Cygwin, mingw, MSVC 9, Interix 3.5, BeOS which is a long-winded way of saying "BSD and Linux only". I thought an Apple-specific answer to an Apple-specific issue might be useful. By the way, it *used* to be possible to turn cores on and off in Mac OS X. There used to be a program 'CPUPalette.app' and later it was possible to do it with 'Instruments'. From ok@REDACTED Fri Feb 19 05:30:44 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 19 Feb 2016 17:30:44 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56C4411E.9060106@rol.ru> References: <56C4411E.9060106@rol.ru> Message-ID: <56C69A74.2000100@cs.otago.ac.nz> On 17/02/16 10:45 pm, Konstantin Vsochanov wrote > I?m working with Erlang for two years now and enjoy every day of using > it. However, there are some strange features I?m wondering about. One > of them -module() attribute. I've never really thought of it as an attribute, but I see from http://erlang.org/doc/apps/erts/absform.html that you are correct to call it so. > The module name is strictly defined by the name of the .erl file. The meaning of a module is defined by its *contents*. More precisely, by the abstract syntax tree of the contents. You can use http://erlang.org/doc/man/merl.html#compile-1 merl:compile(Code) to compile an Erlang module that never was in a file, so *couldn't* have its name defined by the file name. Following the example of the Juice compiler for Oberon, we see that Erlang ASTs could compress very nicely. We could imagine a version of Erlang working by pulling compressed ASTs out of a CDB file. One crucial point is that we cannot rely on file names for modules because we cannot rely on file names, period. For example, my department provides me with a desktop Mac and a laptop Mac. On the desktop machine, time_of_day.erl and Time_Of_Day.erl are *different* file names; on the laptop they are the *same* file name. (I have actually been bitten by this, though not yet with an Erlang module.) Erlang as such does not have any reason why there couldn't be a module named \ In fact, this works. m% cat \\.erl -module('\\'). -export(['div'/0]). 'div'() -> quotient. m% erlc \\.erl m% file \\* \.beam: Erlang BEAM file \.erl: ASCII text m% erl 1> l('\\'). {module,'\\'} 2> '\\':'div'(). quotient I don't advise you to try porting this file to Windows... m% cat >A.erl -module(a). -export([b/0]). b() -> c. m% wc a.erl m% wc a.erl 3 5 38 a.erl m% ls -li A.erl a.erl 72869017 -rw-r--r-- 1 ok wheel 38 19 Feb 16:42 A.erl 72869017 -rw-r--r-- 1 ok wheel 38 19 Feb 16:42 a.erl m% erl 1> c('A'). A.beam: Module name 'a' does not match file name 'A' error 2> c('a'). {ok,a} You have no idea how confusing it is for the compiler to reject a file when given the name it was created under, but to accept the same file under an intrinsic alias. *Especially* as when typing file names on a case-ignoring file system, you tend to get a bit sloppy with the shift key. Frankly, I think that relying on file names is a system-level design bug in Erlang. It would be better to use something like OASIS catalogues so that there were NO unpleasant interactions between Erlang module names and file names: - no reserved characters (/. \) - no case problems - no problems with conflicts between the encoding of the contents of an Erlang module and the encoding assumed by the file system - no problems with Erlang module name length having to conform to the UNKNOWABLE file name limit of some future use Aren't these problems with C too? Not so much, because C doesn't *have* module names (and C++ namespaces are *not* tied to file names). Even so, having been burned a few times, so I have a little AWK script: #!/bin/awk -f # Scan the current directory for file names differing only in case, # which will cause problems porting to Mac OS X and Windows. BEGIN { while (("ls" | getline original) > 0) { lower = tolower(original) tally[lower]++ list[lower] = list[lower] " " original } close("ls") for (lower in tally) if (tally[lower] > 1) { print substr(list[lower], 2) } } | You can?t change module name with -module() attribute. But the the -module() | attribute is mandatory. Why? Why we need -module() attribute at all? > Maybe we can make -module() optional in next Erlang release? NO! You want to throw away the safe thing and keep the dangerous thing. The -module declaration is the ONLY way have to tell that moving files from one operating system to another HASN'T mangled them to breaking point. > So we don?t > need to repeat file name in first string of every .erl file. Or maybe I > miss something important? Why would repeating the file name be a problem? I have things set up in my text editor so that Meta-[-% inserts a standard header (based on the file name). The external file name this is based on has templates that are easy enough to modify, I get % File : foobar.erl % Author : Richard A. O'Keefe % Updated: %G% % Purpose: -module(foobar). -export([ ]). To *not* repeat the module name, I would have to do EXTRA work. Long term, we ought to be decoupling Erlang module names (and include files etc) from file names entirely. Some day someone will do % erlc grasp and that will fetch the source code (compressed) from a repository over https and produce a grasp.beam locally, plus a catalogue entry saying that module 'Grasp' is in 'grasp.beam'. Until the happy day when field system weirdness can no longer harm us, we MUST preserve -module as a safety measure. From ok@REDACTED Fri Feb 19 05:58:07 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 19 Feb 2016 17:58:07 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> Message-ID: <56C6A0DF.9020600@cs.otago.ac.nz> On 18/02/16 6:38 am, derek wrote: > that -module(). line sound like redundant, if it can be mismatched, > why can't it be omitted? > compiler can derive the module name from the filename anyway; No it cannot. I *have* had this happen to me. I write a module 'search' and put it in a 'search.erl' file. I copy it to a memory stick. I plug the stick into another machine. Both machines have case-sensitive file systems, but the file name was changed to SEARCH.ERL Now, just given the file name SEARCH.ERL, was the module name search Search SEARCH seArch or something else? Deriving module names from file names is like holding a chain-saw by the blade. Usually OK but from time to time you'll lose a hand. > Java > also made that filename has to match class name, that is a design > mistake and unnecessary requirement, to my opinion; Why? A Java source file may contain MANY classes. Here is an AWK script that generates a file that defines 10 (as in TEN, as in twice FIVE) public classes. BEGIN { c = "Hemulen" f = c ".java" print "public class " c " {" >f s = "Hattifattener" for (i = 1; i <= 9; i++) { k = s i print " public static class " k " {" >f print " public static final int v = " i ";" >f print " }" >f print " public static final " k " v" i " = new " k "();" >f print "" >f } print "}" >f print "" >f } Run that script, then compile Hemulen.java, and you get ten *.class files. Those classes cannot ALL get their names from Hemulen.java Actually, the coupling between Java file names and class names IS a major pain in the posterior. I recently picked up a repository of source code for analysis There are *hundreds* of .java files, and I CANNOT COMPILE A SINGLE ONE OF THEM because the file names got mangled. I am going to have to read hundreds of files and manually change their names. Sigh. The one and only thing that will make this situation salvageable is that the files *DO* contain the class names. > or on the other hand, if -module() is proved to be useful, why not > just ignore the filename, and always use what's defined in -module()? > in many modern language design like Go and Elixir, it's allowed to > define multiple Class/Module in one file Or you could mention Simula67, which allowed many classes in a source file, and didn't care what the source file was called. We can't just ignore the file name yet, for the same reason that Java doesn't. Java and Erlang use the file name to *find* the compiled module. It is really amazingly useful to have some confidence that you have found the *right* file, given the horrible things file systems can do to you. File name syntax is far more diverse than people realise. From ok@REDACTED Fri Feb 19 06:10:55 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 19 Feb 2016 18:10:55 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56C6A0DF.9020600@cs.otago.ac.nz> References: <56C4411E.9060106@rol.ru> <56C6A0DF.9020600@cs.otago.ac.nz> Message-ID: <56C6A3DF.1040800@cs.otago.ac.nz> On 19/02/16 5:58 pm, Richard A. O'Keefe wrote: > Java >> also made that filename has to match class name, that is a design >> mistake and unnecessary requirement, to my opinion; > Why? A Java source file may contain MANY classes. Clarification: my example was one where a .java file contains many *public* classes. Even before nested classes were introduced, a .java file could contain any number of private classes as well as one public class. The key point either way was that class names could not ALL be derived from file names. From bengt.kleberg@REDACTED Fri Feb 19 07:03:55 2016 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Fri, 19 Feb 2016 07:03:55 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56C6A0DF.9020600@cs.otago.ac.nz> References: <56C4411E.9060106@rol.ru> <56C6A0DF.9020600@cs.otago.ac.nz> Message-ID: <56C6B04B.3090903@ericsson.com> Greetings, Given the problem with case-sensitive file systems, and identifiers in code, why not ignore case? It seems to work for Eiffel. No results from searching for Eiffel and problems with case. It is not a large language, so the number of people that have been exposed to case insensitivity is small. This would not work for Erlang, but new languages could try. bengt On 02/19/2016 05:58 AM, Richard A. O'Keefe wrote: > > > On 18/02/16 6:38 am, derek wrote: >> that -module(). line sound like redundant, if it can be mismatched, >> why can't it be omitted? >> compiler can derive the module name from the filename anyway; > No it cannot. I *have* had this happen to me. > I write a module 'search' and put it in a 'search.erl' file. > I copy it to a memory stick. > I plug the stick into another machine. > Both machines have case-sensitive file systems, but > the file name was changed to SEARCH.ERL > > Now, just given the file name SEARCH.ERL, was the module name > search > Search > SEARCH > seArch > or something else? > > Deriving module names from file names is like > holding a chain-saw by the blade. > Usually OK but from time to time you'll lose a hand. > >> Java >> also made that filename has to match class name, that is a design >> mistake and unnecessary requirement, to my opinion; > Why? A Java source file may contain MANY classes. > Here is an AWK script that generates a file that defines 10 (as in TEN, > as in twice FIVE) public classes. > > BEGIN { > c = "Hemulen" > f = c ".java" > print "public class " c " {" >f > s = "Hattifattener" > for (i = 1; i <= 9; i++) { > k = s i > print " public static class " k " {" >f > print " public static final int v = " i ";" >f > print " }" >f > print " public static final " k " v" i " = new " k "();" >f > print "" >f > } > print "}" >f > print "" >f > } > > Run that script, then compile Hemulen.java, and you get ten *.class > files. > Those classes cannot ALL get their names from Hemulen.java > > Actually, the coupling between Java file names and class names IS a > major pain in the posterior. I recently picked up a repository of source > code for analysis There are *hundreds* of .java files, and > I CANNOT COMPILE A SINGLE ONE OF THEM > because the file names got mangled. I am going to have to read > hundreds of files and manually change their names. Sigh. > The one and only thing that will make this situation salvageable > is that the files *DO* contain the class names. >> or on the other hand, if -module() is proved to be useful, why not >> just ignore the filename, and always use what's defined in -module()? >> in many modern language design like Go and Elixir, it's allowed to >> define multiple Class/Module in one file > Or you could mention Simula67, which allowed many classes in a source > file, and > didn't care what the source file was called. > > We can't just ignore the file name yet, for the same reason that Java > doesn't. > Java and Erlang use the file name to *find* the compiled module. > It is really amazingly useful to have some confidence that you have > found the > *right* file, given the horrible things file systems can do to you. > > File name syntax is far more diverse than people realise. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From askjuise@REDACTED Fri Feb 19 09:51:17 2016 From: askjuise@REDACTED (Alexander Petrovsky) Date: Fri, 19 Feb 2016 11:51:17 +0300 Subject: [erlang-questions] Packets deduplication In-Reply-To: References: <7AFC1CFF-ED22-4EE5-AFB5-082B1E2FEB70@gmail.com> Message-ID: Right now I'm using something like bitmaps as Felix suggested, drop older bits and check uniques with band, and yes, I need to process reordered packets. I think bloom filter is not good solution here due the very-very old ID's can't be deleted and memory will grow, and looks like latency will grow too. But I will look at scalable bloom filter, it may be suitable. Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Feb 19 10:20:11 2016 From: ok@REDACTED (ok@REDACTED) Date: Fri, 19 Feb 2016 22:20:11 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56C6B04B.3090903@ericsson.com> References: <56C4411E.9060106@rol.ru> <56C6A0DF.9020600@cs.otago.ac.nz> <56C6B04B.3090903@ericsson.com> Message-ID: > Greetings, > > Given the problem with case-sensitive file systems, and identifiers in > code, why not ignore case? Sadly, it's not that simple. Consider a. i b. İ c. ı d. I Which of these count as equivalent-except-for-case depends on the natural language they are written in. It's not as if case-insensitivity were all that natural for natural languages either. "THE" is now "TUE" but "the" has not changed. A UN-man is not an un-man. LAX is not particularly lax. And to get away from initials, Q: What's that band? A: The Who. Q: The who? A: That's right. And I'll never forget Mrs Which, Mrs What, and Mrs Who from "A Wrinkle in Time". > It seems to work for Eiffel. No results from searching for Eiffel and > problems with case. It is not a large language, so the number of people > that have been exposed to case insensitivity is small. I suspect that most Eiffel programs are still written in Western European languages for which Latin-1 works well, except perhaps for strings. > > This would not work for Erlang, but new languages could try. Unicode makes some complicated things possible and some "simple" things extremely difficult. Dealing with alphabetic case is one of them. One of the issues with file systems and alphabetic case is that it's just not clear what actually happens. For example, in a Unix-like system where a file name is a sequence of bytes excluding 0 and 47, whether two such sequences should count as equivalent depends on what the encoding is deemed to be. I have certainly run into trouble with file names written in one encoding being displayed unsuitably in another, and the encoding is not (except in classic MacOS) something that is stored with the file name. (I believe MacOS X uses UTF8.) But if we say "let file names be interpreted as UTF-8", we are *still* left with the problem that case equivalence is determined by natural language, not by encoding alone. One problem with case-insensitive languages is that the same identifier may appear as READSYMBOL, readsymbol, ReadSymbol, or even ReadSYmbol. Some languages have dealt with this by saying that you can use any one capitalisation pattern you like: a name that appears in a scope must be capitalised identically at every occurrence. (That amounts to saying that you must write so that your program would work equally well in a case sensitive or a case insensitive language.) Then too, there are file names that are not legal identifiers. Taking Java as an example, what should we do with a :@%%.java file? Or even 2016.java? Like I said before, tying file names and module (or class) names together is like holding a chain-saw by the blade. From khitai.pang@REDACTED Fri Feb 19 10:48:00 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Fri, 19 Feb 2016 17:48:00 +0800 Subject: [erlang-questions] Validate email address Message-ID: Hi, How to write an Erlang function to check whether a string is a valid email address? Thanks Simon From marc@REDACTED Fri Feb 19 10:50:24 2016 From: marc@REDACTED (Marc Worrell) Date: Fri, 19 Feb 2016 10:50:24 +0100 Subject: [erlang-questions] Validate email address In-Reply-To: References: Message-ID: <6451A4EE-4582-4CC6-A01D-1B6DC06C53DE@worrell.nl> You can use z_stdlib: https://github.com/zotonic/z_stdlib/blob/master/src/z_email_utils.erl#L31 - Marc > On 19 feb. 2016, at 10:48, Khitai Pang wrote: > > Hi, > > How to write an Erlang function to check whether a string is a valid email address? > > Thanks > Simon > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From aschultz@REDACTED Fri Feb 19 11:15:24 2016 From: aschultz@REDACTED (Andreas Schultz) Date: Fri, 19 Feb 2016 11:15:24 +0100 Subject: [erlang-questions] An elegant implementation of multi pollset for Erlang VM In-Reply-To: References: Message-ID: <56C6EB3C.2040109@tpip.net> Hi Max, On 02/18/2016 07:47 AM, max_feng wrote: > Erlang VM has only one PollSet in the same time, which is not scalable, especially with a rapid growth of NIC bandwidth and CPU cores on one > single machine. > > In order to make full use of multi-core in Eralng network programming, we develop gen_socket module. > The gen_socket creates multi pollset for Erlang VM without changing erts code! Your code is more or less a Erlang wrapper around libev. Why do you have to import all the libev code, couldn't you just have a NIF to wrap what you need? The scheduler and CPU binding seems to give you an performance advantage. If that it so, then I would rather like to see this as a feature in the Erlang poll code than a UNIX only bolt on. > The gen_socket already has been deployed in production env, and increases throughput of echo server by 110% on a 24-core, 1000Mb/s machine > based on redhat6.2. > > The gen_socket is completely compatible with gen_tcp interface, and very easily deployed. Just git clone and make it. It is not really nice that you are using a name that is already in use (and much longer so) by another Erlang project [1] that is roughly in the same domain. Andreas [1]: https://github.com/travelping/gen_socket > > For details please refer to project link. > > https://github.com/max-feng/erlang_multi_pollset > > Max Feng > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From pierrefenoll@REDACTED Fri Feb 19 12:22:46 2016 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Fri, 19 Feb 2016 12:22:46 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6A0DF.9020600@cs.otago.ac.nz> <56C6B04B.3090903@ericsson.com> Message-ID: What about: Optional -module attribute + ASCII-only lowercase filenames, composed of only chars non-escapable in a terminal (\\, spaces, controls, ?) ? Cheers, -- Pierre Fenoll On 19 February 2016 at 10:20, wrote: > > Greetings, > > > > Given the problem with case-sensitive file systems, and identifiers in > > code, why not ignore case? > > Sadly, it's not that simple. Consider > > a. i > b. İ > c. ı > d. I > > Which of these count as equivalent-except-for-case depends on > the natural language they are written in. > > It's not as if case-insensitivity were all that natural for > natural languages either. "THE" is now "TUE" but "the" has not changed. > A UN-man is not an un-man. LAX is not particularly lax. > And to get away from initials, > Q: What's that band? > A: The Who. > Q: The who? > A: That's right. > And I'll never forget Mrs Which, Mrs What, and Mrs Who from > "A Wrinkle in Time". > > > It seems to work for Eiffel. No results from searching for Eiffel and > > problems with case. It is not a large language, so the number of people > > that have been exposed to case insensitivity is small. > > I suspect that most Eiffel programs are still written in Western > European languages for which Latin-1 works well, except perhaps > for strings. > > > > This would not work for Erlang, but new languages could try. > > Unicode makes some complicated things possible and some "simple" > things extremely difficult. Dealing with alphabetic case is one > of them. > > One of the issues with file systems and alphabetic case is that it's > just not clear what actually happens. For example, in a Unix-like > system where a file name is a sequence of bytes excluding 0 and 47, > whether two such sequences should count as equivalent depends on > what the encoding is deemed to be. I have certainly run into trouble > with file names written in one encoding being displayed unsuitably in > another, and the encoding is not (except in classic MacOS) something > that is stored with the file name. (I believe MacOS X uses UTF8.) > But if we say "let file names be interpreted as UTF-8", we are *still* > left with the problem that case equivalence is determined by > natural language, not by encoding alone. > > One problem with case-insensitive languages is that the same > identifier may appear as READSYMBOL, readsymbol, ReadSymbol, > or even ReadSYmbol. Some languages have dealt with this by saying > that you can use any one capitalisation pattern you like: a name > that appears in a scope must be capitalised identically at every > occurrence. (That amounts to saying that you must write so that > your program would work equally well in a case sensitive or a > case insensitive language.) > > Then too, there are file names that are not legal identifiers. > Taking Java as an example, what should we do with a :@%%.java > file? Or even 2016.java? > > Like I said before, tying file names and module (or class) names > together is like holding a chain-saw by the blade. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Feb 19 12:32:11 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Fri, 19 Feb 2016 12:32:11 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6A0DF.9020600@cs.otago.ac.nz> <56C6B04B.3090903@ericsson.com> Message-ID: <56C6FD3B.4000303@ninenines.eu> I've mentioned that at the beginning of the thread. We're going circles now. :-) I really don't think getting rid of the mandatory -module attribute is such a hard problem to solve... On 02/19/2016 12:22 PM, Pierre Fenoll wrote: > What about: > Optional -module attribute > + > ASCII-only lowercase filenames, composed of only chars non-escapable in > a terminal (\\, spaces, controls, ?) > ? > > > Cheers, > -- > Pierre Fenoll > > > On 19 February 2016 at 10:20, > wrote: > > > Greetings, > > > > Given the problem with case-sensitive file systems, and identifiers in > > code, why not ignore case? > > Sadly, it's not that simple. Consider > > a. i > b. İ > c. ı > d. I > > Which of these count as equivalent-except-for-case depends on > the natural language they are written in. > > It's not as if case-insensitivity were all that natural for > natural languages either. "THE" is now "TUE" but "the" has not changed. > A UN-man is not an un-man. LAX is not particularly lax. > And to get away from initials, > Q: What's that band? > A: The Who. > Q: The who? > A: That's right. > And I'll never forget Mrs Which, Mrs What, and Mrs Who from > "A Wrinkle in Time". > > > It seems to work for Eiffel. No results from searching for Eiffel and > > problems with case. It is not a large language, so the number of people > > that have been exposed to case insensitivity is small. > > I suspect that most Eiffel programs are still written in Western > European languages for which Latin-1 works well, except perhaps > for strings. > > > > This would not work for Erlang, but new languages could try. > > Unicode makes some complicated things possible and some "simple" > things extremely difficult. Dealing with alphabetic case is one > of them. > > One of the issues with file systems and alphabetic case is that it's > just not clear what actually happens. For example, in a Unix-like > system where a file name is a sequence of bytes excluding 0 and 47, > whether two such sequences should count as equivalent depends on > what the encoding is deemed to be. I have certainly run into trouble > with file names written in one encoding being displayed unsuitably in > another, and the encoding is not (except in classic MacOS) something > that is stored with the file name. (I believe MacOS X uses UTF8.) > But if we say "let file names be interpreted as UTF-8", we are *still* > left with the problem that case equivalence is determined by > natural language, not by encoding alone. > > One problem with case-insensitive languages is that the same > identifier may appear as READSYMBOL, readsymbol, ReadSymbol, > or even ReadSYmbol. Some languages have dealt with this by saying > that you can use any one capitalisation pattern you like: a name > that appears in a scope must be capitalised identically at every > occurrence. (That amounts to saying that you must write so that > your program would work equally well in a case sensitive or a > case insensitive language.) > > Then too, there are file names that are not legal identifiers. > Taking Java as an example, what should we do with a :@%%.java > file? Or even 2016.java? > > Like I said before, tying file names and module (or class) names > together is like holding a chain-saw by the blade. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From zxq9@REDACTED Fri Feb 19 13:00:07 2016 From: zxq9@REDACTED (zxq9) Date: Fri, 19 Feb 2016 21:00:07 +0900 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56C6FD3B.4000303@ninenines.eu> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> Message-ID: <1643491.uzcKq8YCN9@changa> On 2016?2?19? ??? 12:32:11 Lo?c Hoguin wrote: > I've mentioned that at the beginning of the thread. We're going circles > now. :-) > > I really don't think getting rid of the mandatory -module attribute is > such a hard problem to solve... This whole conversation is sort of weird to me. Why is having this attribute a problem? Also, this is rather clearly moving in the wrong direction. I imagine (hope?) that someday filesystems as we know them will be considered archaic vestiges of the past and a very different concept of {Aliases, Meta, Data} will evolve. Yet right now, after a few decades of going to ALLLL the horrible, torturous trouble of writing "-module(foo)." at the top of files, we are suddenly experiencing a collective urge to tie module names to filesystem names -- which aren't anywhere close to being uniform across systems. It seems more like we should be arguing for, say, formalization of version semantics (hint: semver or something like it), the runtime to maintain a mapping of code resource locations to {ModName, Version}, let the runtime know more about what the versions *mean*, treat canonical (but signed and distributed) source locations as abstract resources, etc. If we don't care about the Mod/Sourcefile mapping any longer then multiple modules becomes a non-issue and certain things would be pretty nature to write into a single file. Oh. And namespacing (of some form). Unless we decide to go down the com.example.my.backwards.modules direction Java pushed itself or the "filesystems/are/actually/object/hierarchies" of Python (which wound up necessitating that things like virtenv become core features) I just don't see this as a meaningful way to deal with the file/code mapping. And why do we have that mapping anyway? -module(same_as_filename). Is that silly? YES OF COURSE IT IS SILLY. But its a bandaid. And it won't break later when filename == modname is no longer true. Keeping it is a tiny annoyance, one we are already well conditioned to. I think keeping it is a lot better than choosing to make it optional and guaranteeing that a lot of code will have to someday be changed once 1 file /= 1 mod. Those other things I mentioned seem both more important and dramatically more interesting. I'm not arguing that the current situation is optimal or something better shouldn't eventually be put in place. I'm just saying that this is both the least important aspect of that and the one part of the module system we will actually need to keep, not toss. -Craig From dmkolesnikov@REDACTED Fri Feb 19 13:10:00 2016 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Fri, 19 Feb 2016 14:10:00 +0200 Subject: [erlang-questions] jinterface maven Message-ID: Hello, Is any one maintaining jinterface at maven? http://mvnrepository.com/artifact/org.erlang.otp/jinterface It is very old and do not have support for maps. Best Regards, Dmitry From essen@REDACTED Fri Feb 19 13:22:44 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Fri, 19 Feb 2016 13:22:44 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <1643491.uzcKq8YCN9@changa> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> Message-ID: <56C70914.4000809@ninenines.eu> On 02/19/2016 01:00 PM, zxq9 wrote: > On 2016?2?19? ??? 12:32:11 Lo?c Hoguin wrote: >> I've mentioned that at the beginning of the thread. We're going circles >> now. :-) >> >> I really don't think getting rid of the mandatory -module attribute is >> such a hard problem to solve... > > This whole conversation is sort of weird to me. > > Why is having this attribute a problem? You never had the error that the module name doesn't match the file name? > Also, this is rather clearly moving in the wrong direction. I imagine > (hope?) that someday filesystems as we know them will be considered archaic > vestiges of the past and a very different concept of {Aliases, Meta, Data} > will evolve. Yet right now, after a few decades of going to ALLLL the > horrible, torturous trouble of writing "-module(foo)." at the top of files, > we are suddenly experiencing a collective urge to tie module names to > filesystem names -- which aren't anywhere close to being uniform across > systems. Considering the current compiler code checks that you are a good little kid and wrote the same module name as the file name, I don't think that an hypothetical file system was in OTP team's mind when they made the -module mandatory. And frankly, if I were to base all my decisions based on hypothetical changes 50 years from now I wouldn't write any code. I can't predict the future. Can you? > It seems more like we should be arguing for, say, formalization of version > semantics (hint: semver or something like it), the runtime to maintain a > mapping of code resource locations to {ModName, Version}, let the runtime > know more about what the versions *mean*, treat canonical (but signed and > distributed) source locations as abstract resources, etc. If we don't care > about the Mod/Sourcefile mapping any longer then multiple modules becomes > a non-issue and certain things would be pretty nature to write into a > single file. > > Oh. And namespacing (of some form). > > Unless we decide to go down the com.example.my.backwards.modules direction > Java pushed itself or the "filesystems/are/actually/object/hierarchies" of > Python (which wound up necessitating that things like virtenv become core > features) I just don't see this as a meaningful way to deal with the > file/code mapping. And why do we have that mapping anyway? > > -module(same_as_filename). > > Is that silly? YES OF COURSE IT IS SILLY. But its a bandaid. And it won't > break later when filename == modname is no longer true. Keeping it is a tiny > annoyance, one we are already well conditioned to. I think keeping it is a lot > better than choosing to make it optional and guaranteeing that a lot of code > will have to someday be changed once 1 file /= 1 mod. Those other things I > mentioned seem both more important and dramatically more interesting. > > I'm not arguing that the current situation is optimal or something better > shouldn't eventually be put in place. I'm just saying that this is both the > least important aspect of that and the one part of the module system we will > actually need to keep, not toss. > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From bchesneau@REDACTED Fri Feb 19 13:39:44 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 19 Feb 2016 13:39:44 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56C6FD3B.4000303@ninenines.eu> References: <56C4411E.9060106@rol.ru> <56C6A0DF.9020600@cs.otago.ac.nz> <56C6B04B.3090903@ericsson.com> <56C6FD3B.4000303@ninenines.eu> Message-ID: > On 19 Feb 2016, at 12:32, Lo?c Hoguin wrote: > > I've mentioned that at the beginning of the thread. We're going circles now. :-) > > I really don't think getting rid of the mandatory -module attribute is such a hard problem to solve... for myself I am now wondering why we dont have a `-module` block and then ignoring the filename completely. -module(some). -endmodule. - benoit -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Fri Feb 19 13:54:39 2016 From: zxq9@REDACTED (zxq9) Date: Fri, 19 Feb 2016 21:54:39 +0900 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> Message-ID: <3335531.8j3mjyJq6m@changa> On 2016?2?19? ??? 13:39:44 Benoit Chesneau wrote: > > > On 19 Feb 2016, at 12:32, Lo?c Hoguin wrote: > > > > I've mentioned that at the beginning of the thread. We're going circles now. :-) > > > > I really don't think getting rid of the mandatory -module attribute is such a hard problem to solve... > > > for myself I am now wondering why we dont have a `-module` block and then ignoring the filename completely. > > -module(some). > -endmodule. > > - benoit This is a much more interesting idea, imo. The reason I had referred to the filename == modname thing as a bandaid is because it appears to be the way the code loader was kludged into finding code -- because that's a workable and expedient solution, and writing an actual resource resolver is a lot heavier (but more correct) solution. -Craig From nathaniel@REDACTED Fri Feb 19 15:03:17 2016 From: nathaniel@REDACTED (Nathaniel Waisbrot) Date: Fri, 19 Feb 2016 09:03:17 -0500 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6A0DF.9020600@cs.otago.ac.nz> <56C6B04B.3090903@ericsson.com> <56C6FD3B.4000303@ninenines.eu> Message-ID: > for myself I am now wondering why we dont have a `-module` block and then ignoring the filename completely. > > -module(some). > -endmodule. This sounds nice. I think the first thing I?d do is start putting my application and top-level supervisor code in the same file (they?re generally small enough to both fit on one screen together). The second thing would be to wish that I could declare the same module in multiple files and have the compiler combine them for me: foo.interface.erl: -module(foo). -export([poke/0]). poke() -> gen_server:cast(?MODULE, poke). -endmodule. foo.gen_server.erl: -module(foo). -export([init/1, handle_call/3, handle_cast/2, handle_info/2, terminate/2, code_change/3]). handle_cast(poke, State) -> %% ? -endmodule. After that, I?d start wondering about monkey-patching, so there?s that to watch out for. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From bjorn@REDACTED Fri Feb 19 15:23:13 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Fri, 19 Feb 2016 15:23:13 +0100 Subject: [erlang-questions] Faster loading of many modules Message-ID: Loading 500 modules one at a time can be slow. Therefore I have implemented functions in 'code' that can take a list of modules to load all at once. Here is the pull request: https://github.com/erlang/otp/pull/971 /Bj?rn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From roger.wenham@REDACTED Fri Feb 19 15:55:45 2016 From: roger.wenham@REDACTED (Roger Wenham) Date: Fri, 19 Feb 2016 15:55:45 +0100 Subject: [erlang-questions] SSL Example does not work Message-ID: <56C72CF1.3080103@gmx.net> Hi, Having had some difficulty writing a TLS server, I decided to run the example in the sources for SSL (.../erlang/lib/ssl-7.2/examples/...) and to my surprise, it fails also. What I get is: Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] Eshell V7.2.1 (abort with ^G) 1> client_server:start(). ** exception exit: {noproc,{gen_server,call, [ssl_listen_tracker_sup, {start_child,[#Port<0.748>, [{active,false}, {packet_size,0}, {packet,0}, {header,0}, {mode,list}], {ssl_options,tls, [{3,3},{3,2},{3,1}], verify_peer,undefined,#Fun,true,false, undefined,2,<<"/opt/local/lib/erlang/li"...>>,undefined, <<"/opt/local/lib/e"...>>,undefined,[],undefined,<<...>>,...}]}, infinity]}} in function gen_server:call/3 (gen_server.erl, line 212) in call from ssl:listen/2 (ssl.erl, line 155) in call from client_server:start/0 (client_server.erl, line 35) I have tried this on several platforms, and the failure appears to be the same... I checked back trough the archive for the last few months, but no joy... Anyone have any idea about what I'm doing wrong? Thanks Roger -------------- next part -------------- An HTML attachment was scrubbed... URL: From magnus@REDACTED Fri Feb 19 16:31:25 2016 From: magnus@REDACTED (Magnus Henoch) Date: Fri, 19 Feb 2016 15:31:25 +0000 Subject: [erlang-questions] SSL Example does not work In-Reply-To: <56C72CF1.3080103@gmx.net> References: <56C72CF1.3080103@gmx.net> Message-ID: You need to run ssl:start() before starting the example. Regards, Magnus On Fri, Feb 19, 2016 at 2:55 PM, Roger Wenham wrote: > Hi, > Having had some difficulty writing a TLS server, I decided to run the > example in the sources for SSL > (...*/erlang/lib/ssl-7.2/examples/*...) and to my surprise, it fails > also. > > What I get is: > > Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:8:8] [async-threads:10] > [hipe] [kernel-poll:false] > > Eshell V7.2.1 (abort with ^G) > > 1> client_server:start(). > ** exception exit: {noproc,{gen_server,call, > [ssl_listen_tracker_sup, > {start_child,[#Port<0.748>, > [{active,false}, > {packet_size,0}, > {packet,0}, > {header,0}, > {mode,list}], > {ssl_options,tls, > > [{3,3},{3,2},{3,1}], > > verify_peer,undefined,#Fun,true,false, > > undefined,2,<<"/opt/local/lib/erlang/li"...>>,undefined, > > <<"/opt/local/lib/e"...>>,undefined,[],undefined,<<...>>,...}]}, > infinity]}} > in function gen_server:call/3 (gen_server.erl, line 212) > in call from ssl:listen/2 (ssl.erl, line 155) > in call from client_server:start/0 (client_server.erl, line 35) > > I have tried this on several platforms, and the failure appears to be the > same... > > I checked back trough the archive for the last few months, but no joy... > > Anyone have any idea about what I'm doing wrong? > > Thanks > > Roger > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vimal7370@REDACTED Fri Feb 19 16:41:32 2016 From: vimal7370@REDACTED (Vimal Kumar) Date: Fri, 19 Feb 2016 21:11:32 +0530 Subject: [erlang-questions] SSL Example does not work In-Reply-To: <56C72CF1.3080103@gmx.net> References: <56C72CF1.3080103@gmx.net> Message-ID: SSL not started? Try: application:ensure_all_started(ssl). On Fri, Feb 19, 2016 at 8:25 PM, Roger Wenham wrote: > Hi, > Having had some difficulty writing a TLS server, I decided to run the > example in the sources for SSL > (...*/erlang/lib/ssl-7.2/examples/*...) and to my surprise, it fails > also. > > What I get is: > > Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:8:8] [async-threads:10] > [hipe] [kernel-poll:false] > > Eshell V7.2.1 (abort with ^G) > > 1> client_server:start(). > ** exception exit: {noproc,{gen_server,call, > [ssl_listen_tracker_sup, > {start_child,[#Port<0.748>, > [{active,false}, > {packet_size,0}, > {packet,0}, > {header,0}, > {mode,list}], > {ssl_options,tls, > > [{3,3},{3,2},{3,1}], > > verify_peer,undefined,#Fun,true,false, > > undefined,2,<<"/opt/local/lib/erlang/li"...>>,undefined, > > <<"/opt/local/lib/e"...>>,undefined,[],undefined,<<...>>,...}]}, > infinity]}} > in function gen_server:call/3 (gen_server.erl, line 212) > in call from ssl:listen/2 (ssl.erl, line 155) > in call from client_server:start/0 (client_server.erl, line 35) > > I have tried this on several platforms, and the failure appears to be the > same... > > I checked back trough the archive for the last few months, but no joy... > > Anyone have any idea about what I'm doing wrong? > > Thanks > > Roger > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Fri Feb 19 16:43:55 2016 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 19 Feb 2016 16:43:55 +0100 Subject: [erlang-questions] Proposal for new state machine engine: gen_statem In-Reply-To: <20160210125648.GA85287@erix.ericsson.se> References: <20160210125648.GA85287@erix.ericsson.se> Message-ID: <20160219154355.GA14250@erix.ericsson.se> On Wed, Feb 10, 2016 at 01:56:48PM +0100, Raimo Niskanen wrote: > Hi all! > > I have created a pull request for a new state machine engine in the gen_* > framework. There is a heated discussion right now on GitHUB: > > https://github.com/erlang/otp/pull/960 > > Please contribute with an opinion. I pushed some updates to the branch. Major changes: * New return values from the state functions: {stop,Reason} | {keep_state,...} | {keep_state_and_data,...}. It turned out that the old [Op] was essential to an internal use case of ours and that can now be done with {keep_state_and_data,[Op]}. * Change initial PrevState to undefined. * Rewrite of terminology and types: state_data() -> data(), StateOp :: state_operation() -> Op :: transition_action() and more. * Some more documentation cleanup Two questions still remain: * Should we change the return format from init/1 from {'ok',state(),data()} to {callback_mode(),state(),data()} and remove the init/enter_loop option {'callback_mode',callback_mode()}? I think I am for making the option mandatory this way or some similar. * If there are multiple transaction_option() in a state function return Ops, which one shall win, the first or the last? Currently the last wins to harmonize with that all Ops are executed in list order so the last one overrides the previous. This way you can not override a value by consing it to the head of Ops - you will have to append it instead, which is a bit uncommon. The documentation still lack early examples, but hopefully the Description is easier to understand now... Opinions on the current version, please?! Updated documentation at: http://erlang.org/~raimo/doc-7.2/lib/stdlib-2.7/doc/html/gen_statem.html Updated OTP 18.2 .beam files at: https://github.com/erlang/otp/pull/erlang.org/%7Eraimo/OTP-18.2-stdlib-ebin-gen_statem.tgz keep the original around. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From erlang@REDACTED Fri Feb 19 17:04:03 2016 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 19 Feb 2016 17:04:03 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <3335531.8j3mjyJq6m@changa> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <3335531.8j3mjyJq6m@changa> Message-ID: A bit of history .. Erlang has always had autoloading. Initially no modules are loaded. When a module is called that is not loaded the emulator calls the function error_handler:undefined_function(Mod, Func, Args) The job of this function is to find the code for the module `Mod` load the code then call apply(Mod, Func, Args) error_handler.erl calls code.erl to actually find the code. Code has a list of paths which it searches for a file called .beam It *assumes* this contains compiled code for Mod.erl. If this is not the case weird things happen. At the time very little thought was put into code.erl - it was quickly hacked together. I made a generic mechanism so that users could replace error_handler.erl with their own custom error_handler on a per-process basis - but virtually nobody ever did this it and it was not well documented. code.erl and error_handler.erl are a bit of a mess (in fact the entire code loading is a bit of a mess) - but it works and rewriting it has never been high priority. It's not really apparent, but parts of the system have had thousands of hours of work done on them (binaries, bit syntax, pattern matching, compilation) - other parts (code loading, starting the system etc. have had dozens of hours work and are pretty near to the first ever versions). This reflects priorities - saving the programmer a few minutes typing has never been a priority - making fast pattern matching algorithms etc. was and is, a priority. Today changing code.erl and error_handler.erl is pretty tricky for reasons of backwards compatibility - it's easy to write a new code loader, but it's really difficult to write one that is *beautiful* and *backwards compatible*. I've actually changed my mind on backwards compatibility - I'm overjoyed when I find that 20 year old Erlang programs still run flawlessly. Unlike which changes so fast that even programs written a few weeks ago no longer work. Cheers /Joe On Fri, Feb 19, 2016 at 1:54 PM, zxq9 wrote: > On 2016?2?19? ??? 13:39:44 Benoit Chesneau wrote: >> >> > On 19 Feb 2016, at 12:32, Lo?c Hoguin wrote: >> > >> > I've mentioned that at the beginning of the thread. We're going circles now. :-) >> > >> > I really don't think getting rid of the mandatory -module attribute is such a hard problem to solve... >> >> >> for myself I am now wondering why we dont have a `-module` block and then ignoring the filename completely. >> >> -module(some). >> -endmodule. >> >> - benoit > > This is a much more interesting idea, imo. > > The reason I had referred to the filename == modname thing as a bandaid > is because it appears to be the way the code loader was kludged into finding > code -- because that's a workable and expedient solution, and writing an > actual resource resolver is a lot heavier (but more correct) solution. > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Fri Feb 19 17:06:04 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Fri, 19 Feb 2016 17:06:04 +0100 Subject: [erlang-questions] Proposal for new state machine engine: gen_statem In-Reply-To: <20160219154355.GA14250@erix.ericsson.se> References: <20160210125648.GA85287@erix.ericsson.se> <20160219154355.GA14250@erix.ericsson.se> Message-ID: <56C73D6C.6030304@ninenines.eu> On 02/19/2016 04:43 PM, Raimo Niskanen wrote: > * If there are multiple transaction_option() in a state function return > Ops, which one shall win, the first or the last? Currently the last wins to > harmonize with that all Ops are executed in list order so the last one > overrides the previous. This way you can not override a value by consing it > to the head of Ops - you will have to append it instead, which is a bit > uncommon. I believe that when being provided with a list of commands, the commands should be run in order and therefore the last one should always win. That a command overrides a previous command is perfectly reasonable. I don't think the list of commands will ever get very big, so Ops ++ [override] shouldn't be an issue. Sure ++ is uncommon, but overriding is also uncommon, isn't it? -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From raimo+erlang-questions@REDACTED Fri Feb 19 17:12:07 2016 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 19 Feb 2016 17:12:07 +0100 Subject: [erlang-questions] Proposal for new state machine engine: gen_statem In-Reply-To: <56C73D6C.6030304@ninenines.eu> References: <20160210125648.GA85287@erix.ericsson.se> <20160219154355.GA14250@erix.ericsson.se> <56C73D6C.6030304@ninenines.eu> Message-ID: <20160219161207.GA16464@erix.ericsson.se> On Fri, Feb 19, 2016 at 05:06:04PM +0100, Lo?c Hoguin wrote: > On 02/19/2016 04:43 PM, Raimo Niskanen wrote: > > * If there are multiple transaction_option() in a state function return > > Ops, which one shall win, the first or the last? Currently the last wins to > > harmonize with that all Ops are executed in list order so the last one > > overrides the previous. This way you can not override a value by consing it > > to the head of Ops - you will have to append it instead, which is a bit > > uncommon. > > I believe that when being provided with a list of commands, the commands > should be run in order and therefore the last one should always win. > That a command overrides a previous command is perfectly reasonable. > > I don't think the list of commands will ever get very big, so Ops ++ > [override] shouldn't be an issue. Sure ++ is uncommon, but overriding is > also uncommon, isn't it? Valid points. Unfortunately this clashes with the behaviour of e.g proplists. Fred Hebert just pointed this out in the pull request. By the way I think we should keep the discussion in the pull request since the bulk is already there... https://github.com/erlang/otp/pull/960 > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From bengt.kleberg@REDACTED Fri Feb 19 17:17:34 2016 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Fri, 19 Feb 2016 17:17:34 +0100 Subject: [erlang-questions] Proposal for new state machine engine: gen_statem In-Reply-To: <56C73D6C.6030304@ninenines.eu> References: <20160210125648.GA85287@erix.ericsson.se> <20160219154355.GA14250@erix.ericsson.se> <56C73D6C.6030304@ninenines.eu> Message-ID: <56C7401E.7090904@ericsson.com> Greetings, I do [override | Ops], since the first one is picked. bengt On 02/19/2016 05:06 PM, Lo?c Hoguin wrote: > On 02/19/2016 04:43 PM, Raimo Niskanen wrote: >> * If there are multiple transaction_option() in a state function return >> Ops, which one shall win, the first or the last? Currently the >> last wins to >> harmonize with that all Ops are executed in list order so the last >> one >> overrides the previous. This way you can not override a value by >> consing it >> to the head of Ops - you will have to append it instead, which is >> a bit >> uncommon. > > I believe that when being provided with a list of commands, the > commands should be run in order and therefore the last one should > always win. That a command overrides a previous command is perfectly > reasonable. > > I don't think the list of commands will ever get very big, so Ops ++ > [override] shouldn't be an issue. Sure ++ is uncommon, but overriding > is also uncommon, isn't it? > From mononcqc@REDACTED Fri Feb 19 18:00:42 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 19 Feb 2016 12:00:42 -0500 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56C70914.4000809@ninenines.eu> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> Message-ID: <20160219170040.GA1627@fhebert-ltm2.internal.salesforce.com> On 02/19, Lo?c Hoguin wrote: > >You never had the error that the module name doesn't match the file name? > Frankly the only times I got the error was when I was either: 1. Making copies of modules for benchmarks or comparisons and forgetting to change the names. This error was necessary since I couldn't run 3 modules all with the same name without issue. Much appreciated 2. Moving files around and I forget to rename the module. I tend to like things to be in order and I appreciate the reminder that renaming a file means I should probably rename the module. It's annoying that it's mandatory, but I can't really think of too many cases where I want to look for a module, and must find it in a differently-named file. I guess I could use grep/awk/ack to move around, but I quite like just opening a file with the right name. I always thought of the feature as a net positive given it forces me to keep things clean and consistent. One module per file is something I'm more open to argue about, but filename must match the module name is definitely a plus for me. From khitai.pang@REDACTED Fri Feb 19 18:37:07 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Sat, 20 Feb 2016 01:37:07 +0800 Subject: [erlang-questions] Validate international mobile phone number Message-ID: Is there any erlang project that has code for international mobile phone number validation? I think it should be there since Erlang was originally for telecommunication :) Thanks Khitai From thomas.elsgaard@REDACTED Fri Feb 19 21:32:05 2016 From: thomas.elsgaard@REDACTED (Thomas Elsgaard) Date: Fri, 19 Feb 2016 20:32:05 +0000 Subject: [erlang-questions] Validate international mobile phone number In-Reply-To: References: Message-ID: Hi Khitai Mobile telcos are getting the information from the GSMA IR.21 database, so if you want to validate international mobile phone numbers (MSISDN), you should look into how to obtain the IR.21 information from GSMA. Based on the IR.21 information, the switch engineers are planning and implementing numbers analysis in the mobile and fixed exchanges. Thomas On Fri, 19 Feb 2016 at 18:37 Khitai Pang wrote: > Is there any erlang project that has code for international mobile phone > number validation? I think it should be there since Erlang was > originally for telecommunication :) > > Thanks > Khitai > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Fri Feb 19 21:52:50 2016 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 19 Feb 2016 21:52:50 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <20160219170040.GA1627@fhebert-ltm2.internal.salesforce.com> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <20160219170040.GA1627@fhebert-ltm2.internal.salesforce.com> Message-ID: While on the subject of modules ... I think there should be 'anonymous modules' and registered modules (just like processes): For processes: Pid = spawn(fun() -> ... end) Creates a process, but only the process which created the process can say Pid ! Message Other process do not know the value of Pid so they can't send any messages to Pid. This is inconvenient so we can say register(foo, Pid) and thereafter *any* process can say `foo ! Message` I would like to see: Mod = load_code(Bin) after which the parent can say Mod:func(....) and register(Name, Mod) and then *any* process can call foo (just like registered processes) With such a mechanism we could declare and use private modules which are not globally visible. (The same should be true for ets tables and ports) - also Modules should be linkable and sent exit signals to their link set when deleted. Just make everything behave like processes. Cheers /Joe On Fri, Feb 19, 2016 at 6:00 PM, Fred Hebert wrote: > On 02/19, Lo?c Hoguin wrote: >> >> >> You never had the error that the module name doesn't match the file name? >> > > Frankly the only times I got the error was when I was either: > > 1. Making copies of modules for benchmarks or comparisons and forgetting to > change the names. This error was necessary since I couldn't run 3 modules > all with the same name without issue. Much appreciated > > 2. Moving files around and I forget to rename the module. I tend to like > things to be in order and I appreciate the reminder that renaming a file > means I should probably rename the module. > > It's annoying that it's mandatory, but I can't really think of too many > cases where I want to look for a module, and must find it in a > differently-named file. I guess I could use grep/awk/ack to move around, but > I quite like just opening a file with the right name. > > I always thought of the feature as a net positive given it forces me to keep > things clean and consistent. > > One module per file is something I'm more open to argue about, but filename > must match the module name is definitely a plus for me. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From tony@REDACTED Fri Feb 19 23:26:04 2016 From: tony@REDACTED (Tony Rogvall) Date: Fri, 19 Feb 2016 23:26:04 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <20160219170040.GA1627@fhebert-ltm2.internal.salesforce.com> Message-ID: > On 19 feb 2016, at 21:52, Joe Armstrong wrote: > > While on the subject of modules ... > > I think there should be 'anonymous modules' and registered modules > (just like processes): > I even had it implemented in the first SMP prototype while trying to find cheap ways of doing the code loading. Just think how nice this would blend in with a gen_server, just pass it the anonymous module code and the code is upgraded. It worked like a charm. :-) Cheers /Tony > For processes: > > Pid = spawn(fun() -> ... end) > > Creates a process, but only the process which created the process can say > > Pid ! Message > > Other process do not know the value of Pid so they can't send any > messages to Pid. > > This is inconvenient so we can say register(foo, Pid) and thereafter *any* > process can say `foo ! Message` > > I would like to see: > > Mod = load_code(Bin) > > after which the parent can say Mod:func(....) and register(Name, Mod) > and then *any* process can call foo (just like registered processes) > > With such a mechanism we could declare and use private modules which are not > globally visible. > > (The same should be true for ets tables and ports) - also Modules > should be linkable > and sent exit signals to their link set when deleted. > > Just make everything behave like processes. > > Cheers > > /Joe > > > On Fri, Feb 19, 2016 at 6:00 PM, Fred Hebert wrote: >> On 02/19, Lo?c Hoguin wrote: >>> >>> >>> You never had the error that the module name doesn't match the file name? >>> >> >> Frankly the only times I got the error was when I was either: >> >> 1. Making copies of modules for benchmarks or comparisons and forgetting to >> change the names. This error was necessary since I couldn't run 3 modules >> all with the same name without issue. Much appreciated >> >> 2. Moving files around and I forget to rename the module. I tend to like >> things to be in order and I appreciate the reminder that renaming a file >> means I should probably rename the module. >> >> It's annoying that it's mandatory, but I can't really think of too many >> cases where I want to look for a module, and must find it in a >> differently-named file. I guess I could use grep/awk/ack to move around, but >> I quite like just opening a file with the right name. >> >> I always thought of the feature as a net positive given it forces me to keep >> things clean and consistent. >> >> One module per file is something I'm more open to argue about, but filename >> must match the module name is definitely a plus for me. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From roger.wenham@REDACTED Sat Feb 20 12:13:22 2016 From: roger.wenham@REDACTED (Roger Wenham) Date: Sat, 20 Feb 2016 12:13:22 +0100 Subject: [erlang-questions] SSL Example does not work Message-ID: <56C84A52.2030206@gmx.net> Hi Magnus, Thought of that, but the test program starts: start() -> %% Start ssl application application:start(crypto), application:start(public_key), application:start(ssl), ... I obviously don't understand the difference between application:start(ssl) and ssl:start(). Shouldn't they do the same thing? I will have to check that... Regards, Roger On Fri, Feb 19, 2016 at 15:31 PM, Magnus Henoch wrote: You need to run ssl:start() before starting the example. Regards, Magnus From roger.wenham@REDACTED Sat Feb 20 12:59:25 2016 From: roger.wenham@REDACTED (Roger Wenham) Date: Sat, 20 Feb 2016 12:59:25 +0100 Subject: [erlang-questions] SSL Example does not work In-Reply-To: References: <56C72CF1.3080103@gmx.net> Message-ID: <56C8551D.70604@gmx.net> Hi Magnus, I did the ssl:start() as you suggested, and it now appears to work. So why is that necessary? The program calls: application:start(ssl), which I presume would cause ssl:start() to be called... Why is that not enough? Also, is it ok to put the ssl:start() inside the program, and why wasn't that done? Lastly, Vimal Kumar suggested I use application:ensure_all_started(ssl), would that remove the need for the ssl:start() ? Sorry for all the questions :-) Regards Roger On 19/02/16 16:31, Magnus Henoch wrote: > You need to run ssl:start() before starting the example. > > Regards, > Magnus > > On Fri, Feb 19, 2016 at 2:55 PM, Roger Wenham > wrote: > > Hi, > Having had some difficulty writing a TLS server, I decided to run > the example in the sources for SSL > (...//erlang/lib/ssl-7.2/examples//...) and to my surprise, it > fails also. > > What I get is: > > Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:8:8] > [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V7.2.1 (abort with ^G) > > 1> client_server:start(). > ** exception exit: {noproc,{gen_server,call, > [ssl_listen_tracker_sup, > {start_child,[#Port<0.748>, > [{active,false}, > {packet_size,0}, > {packet,0}, > {header,0}, > {mode,list}], > {ssl_options,tls, > [{3,3},{3,2},{3,1}], > verify_peer,undefined,#Fun,true,false, > undefined,2,<<"/opt/local/lib/erlang/li"...>>,undefined, > <<"/opt/local/lib/e"...>>,undefined,[],undefined,<<...>>,...}]}, > infinity]}} > in function gen_server:call/3 (gen_server.erl, line 212) > in call from ssl:listen/2 (ssl.erl, line 155) > in call from client_server:start/0 (client_server.erl, line 35) > > I have tried this on several platforms, and the failure appears to > be the same... > > I checked back trough the archive for the last few months, but no > joy... > > Anyone have any idea about what I'm doing wrong? > > Thanks > > Roger > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Sat Feb 20 14:14:15 2016 From: zxq9@REDACTED (zxq9) Date: Sat, 20 Feb 2016 22:14:15 +0900 Subject: [erlang-questions] SSL Example does not work In-Reply-To: <56C8551D.70604@gmx.net> References: <56C72CF1.3080103@gmx.net> <56C8551D.70604@gmx.net> Message-ID: <1698946.4CB1DWytV9@changa> On 2016?2?20? ??? 12:59:25 Roger Wenham wrote: > Hi Magnus, > > I did the ssl:start() as you suggested, and it now appears to work. So > why is that necessary? > > The program calls: > > application:start(ssl), > > which I presume would cause ssl:start() to be called... Why is that not > enough? > > Also, is it ok to put the ssl:start() inside the program, and why wasn't > that done? > > Lastly, Vimal Kumar suggested I use application:ensure_all_started(ssl), > would that remove the need for the ssl:start() ? > > Sorry for all the questions :-) Hi Roger. The reason that application:start(ssl) doesn't quite cut it is because ssl has, as an Erlang application, a few of its own application dependencies that ssl:start/0,1 takes care of. https://github.com/erlang/otp/blob/82a835d94be7ee5e98d101a29999fedaf6cd75fe/lib/ssl/src/ssl.erl#L67-L77 Usually you can get away with ensure_started/1, but not in this case, it seems. -Craig From zxq9@REDACTED Sat Feb 20 14:20:45 2016 From: zxq9@REDACTED (zxq9) Date: Sat, 20 Feb 2016 22:20:45 +0900 Subject: [erlang-questions] SSL Example does not work In-Reply-To: <1698946.4CB1DWytV9@changa> References: <56C72CF1.3080103@gmx.net> <56C8551D.70604@gmx.net> <1698946.4CB1DWytV9@changa> Message-ID: <2296856.NPeYoZrNd1@changa> On 2016?2?20? ??? 22:14:15 zxq9 wrote: > On 2016?2?20? ??? 12:59:25 Roger Wenham wrote: > > Hi Magnus, > > > > I did the ssl:start() as you suggested, and it now appears to work. So > > why is that necessary? > > > > The program calls: > > > > application:start(ssl), > > > > which I presume would cause ssl:start() to be called... Why is that not > > enough? > > > > Also, is it ok to put the ssl:start() inside the program, and why wasn't > > that done? > > > > Lastly, Vimal Kumar suggested I use application:ensure_all_started(ssl), > > would that remove the need for the ssl:start() ? > > > > Sorry for all the questions :-) > > > Hi Roger. > > The reason that application:start(ssl) doesn't quite cut it is because > ssl has, as an Erlang application, a few of its own application > dependencies that ssl:start/0,1 takes care of. > > https://github.com/erlang/otp/blob/82a835d94be7ee5e98d101a29999fedaf6cd75fe/lib/ssl/src/ssl.erl#L67-L77 > > Usually you can get away with ensure_started/1, but not in this case > it seems. Sorry, just pointing that out is maybe slightly useless. Some variation of the following may be able to stand in for application:ensure_started/1 ensure_ssl_started() -> case ssl:start() of ok -> ok; {error,{already_started,ssl}} -> ok; Error -> Error end. Normally you shouldn't need this, though. If your application has crashed that far back up the chain (to the point that this would be called on a restart) you probably want to let the node die because something is super horribly wrong. Or at least that is my assumption, not knowing anything about your system. -Craig From essen@REDACTED Sat Feb 20 14:32:35 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Sat, 20 Feb 2016 14:32:35 +0100 Subject: [erlang-questions] SSL Example does not work In-Reply-To: <56C84A52.2030206@gmx.net> References: <56C84A52.2030206@gmx.net> Message-ID: <56C86AF3.4090905@ninenines.eu> On 02/20/2016 12:13 PM, Roger Wenham wrote: > Hi Magnus, > > Thought of that, but the test program starts: > > start() -> > %% Start ssl application It's missing application:start(asn1). > application:start(crypto), > application:start(public_key), > application:start(ssl), > ... -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From roger.wenham@REDACTED Sat Feb 20 17:01:42 2016 From: roger.wenham@REDACTED (Roger Wenham) Date: Sat, 20 Feb 2016 17:01:42 +0100 Subject: [erlang-questions] SSL Example does not work In-Reply-To: <56C86AF3.4090905@ninenines.eu> References: <56C84A52.2030206@gmx.net> <56C86AF3.4090905@ninenines.eu> Message-ID: <56C88DE6.9060708@gmx.net> Thanks for all the replies!, this list is brilliant! I now completely understand. ssl:start() calls: application:start(crypto), application:start(asn1), application:start(public_key), application:start(ssl). The supplied ssl example program calls: start() -> %% Start ssl application application:start(crypto), application:start(public_key), application:start(ssl), .... So as Lo?c Hoguin correctly pointed out, asn1 is not being started, causing the error... So the test program should be modified as either: start() -> %% Start ssl application application:start(crypto), application:start(asn1), application:start(public_key), application:start(ssl), .... or better: start() -> %% Start ssl application ssl:start(), .... I have tested the first fix, and it works.... Do I need to raise a bug on this, as it is *very* confusing to have an official example program not work? Thanks Roger On 20/02/16 14:32, Lo?c Hoguin wrote: > On 02/20/2016 12:13 PM, Roger Wenham wrote: >> Hi Magnus, >> >> Thought of that, but the test program starts: >> >> start() -> >> %% Start ssl application > > It's missing > application:start(asn1). > >> application:start(crypto), >> application:start(public_key), >> application:start(ssl), >> ... > From marat@REDACTED Sat Feb 20 19:09:29 2016 From: marat@REDACTED (Marat Kalibekov) Date: Sun, 21 Feb 2016 00:09:29 +0600 Subject: [erlang-questions] REST API client library Message-ID: Hello, what is the best/most used client library on Erlang? -- Kalibekov Marat -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Sun Feb 21 10:20:03 2016 From: ok@REDACTED (ok@REDACTED) Date: Sun, 21 Feb 2016 22:20:03 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56C70914.4000809@ninenines.eu> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> Message-ID: <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> > On 02/19/2016 01:00 PM, zxq9 wrote: >> On 2016???2???19??? ????????? 12:32:11 Lo??c Hoguin wrote: >> Why is having this attribute a problem? > > You never had the error that the module name doesn't match the file name? Having the file name wrong is a problem. Having -module in the file is a SOLUTION to that problem. Has everyone forgotten the Xerox Lesson? Back in the 80s it was on everyone's lips at Xerox PARC, or at any rate everyone's lips who spoke to me. A PROGRAM IS NOT A LISTING: A PROGRAM IS A DATABASE. This mantra frees you to imagine things like versioned modules in a non-versioned file system, or treating modules (or even parts of modules) as things that can be stored in a data base under some kind of authentication scheme. > Considering the current compiler code checks that you are a good little > kid and wrote the same module name as the file name, I don't think that > an hypothetical file system was in OTP team's mind when they made the > -module mandatory. No, the compiler is not checking that YOU are a good little kid. The compiler is checking that the FILE SYSTEM hasn't broken things. To repeat an earlier example, I've copied a file named 'tests.erl' onto a memory stick (system X), read it on another system (system Y) with no trouble, and failed miserably on another system (Z) because the file is actually called 'TESTS.ERL' on the memory stick. This is the file system mucking things up and it HAPPENS. > > And frankly, if I were to base all my decisions based on hypothetical > changes 50 years from now I wouldn't write any code. I can't predict the > future. Can you? How about predicting the present? We have to deal with mutually incompatible file systems NOW. According to http://erlang.org/doc/man/code.html you can load modules from .ez files NOW, and the name of a pseudo-file inside a .ez file is *not* required to follow the rules of the host file system. From ok@REDACTED Sun Feb 21 10:39:27 2016 From: ok@REDACTED (ok@REDACTED) Date: Sun, 21 Feb 2016 22:39:27 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6A0DF.9020600@cs.otago.ac.nz> <56C6B04B.3090903@ericsson.com> Message-ID: > What about: > Optional -module attribute > + > ASCII-only lowercase filenames, composed of only chars non-escapable in a > terminal (\\, spaces, controls, ???) There are two ideas here. (1) Restrict module names to a 'portable' subset. There are NO non-escapable characters in a POSIX terminal. What's more, there are NO characters that might not NEED escaping in a POSIX terminal: % stty intr s % # now try to cancel that % s % # oh dear interrupt char % \s % # oh dear no change here % ^Vstty intr '^C' But anyway, say it's [a-z0-9_] only. Still isn't enough: what if the name is too long and the file system truncates it? The POSIX limit is 14, and there have been UNIX systems with that limit, so 'abcdefghij.beam' might be too long to work as a file name, so limit module names to 9 characters? This approach to safety is not going to work very well. (2) Make module names optional. But in the name of suffering humanity, *WHY*? As a human being, when I am looking at a module, *I* want to know what the module is called. I don't give a fewmet what the *file* is called, I want to know what the *module* is called, which means I want it in the source text where *I* can see it. Is there any difficulty whatsoever in putting the -module directive there? Nope. In my editor, ESC[% inserts % File : % Author : % Updated: 20%E% % Purpose: -module(). -export([ ]). The module name comes *free* as in *zero* effort. In short, tying module names to file names IS a problem, while having -module directives is NOT a problem. From essen@REDACTED Sun Feb 21 12:08:04 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Sun, 21 Feb 2016 12:08:04 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> Message-ID: <56C99A94.2020009@ninenines.eu> On 02/21/2016 10:20 AM, ok@REDACTED wrote: >> And frankly, if I were to base all my decisions based on hypothetical >> changes 50 years from now I wouldn't write any code. I can't predict the >> future. Can you? > > How about predicting the present? > > We have to deal with mutually incompatible file systems NOW. > According to http://erlang.org/doc/man/code.html you can > load modules from .ez files NOW, and the name of a pseudo-file > inside a .ez file is *not* required to follow the rules of the > host file system. It seems that you consider beam and source files to be equivalent. They are not, though. A beam file should contain a module name, there's no doubt about this. So loading from a .ez file would work, so upgrading would work, and so on. But there's no point in having the source file require the module name directly in the source if the compiler can figure it out (and it can *because we give it* when we ask the compiler to compile files). The compiler should error out if it can't find a module name *at all*, not if it can't find it *in the source*. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From rvirding@REDACTED Sun Feb 21 12:59:36 2016 From: rvirding@REDACTED (Robert Virding) Date: Sun, 21 Feb 2016 12:59:36 +0100 Subject: [erlang-questions] Erlounge in Barcelona In-Reply-To: References: Message-ID: I would prefer Monday - Wednesday. Vance are you here on Wednesday evening? If so that would fit me very well. Otherwise I have no problems with going out and meeting people more than once. :-) Robert On 17 February 2016 at 13:32, Carlos Gonz?lez Florido wrote: > I would love to, if Wednesday or Thursday. > > Carlos Gonz?lez > ErlNETS Technologies > https://github.com/ErlNETS > > > El mi?rcoles, 17 de febrero de 2016, Robert Virding > escribi?: > >> Hi, >> >> I will be in Barcelona next week. Are there any Erlang/Elixirers who >> would care for a light erlounge while I am there? >> >> Robert >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Sun Feb 21 13:08:55 2016 From: vances@REDACTED (Vance Shipley) Date: Sun, 21 Feb 2016 17:38:55 +0530 Subject: [erlang-questions] Erlounge in Barcelona In-Reply-To: References: Message-ID: On Sun, Feb 21, 2016 at 5:29 PM, Robert Virding wrote: > I would prefer Monday - Wednesday. Vance are you here on Wednesday evening? Yes, Wednesday evening is fine for me, I leave Thursday morning. -- -Vance From fxmywc@REDACTED Sun Feb 21 13:39:27 2016 From: fxmywc@REDACTED (fxmy wang) Date: Sun, 21 Feb 2016 20:39:27 +0800 Subject: [erlang-questions] Weird lists:dropwhile/2 behavior? Message-ID: Hello guys, Just encountered this: Erlang/OTP 18 [erts-7.0.3] [source-5991161] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] Eshell V7.0.3 (abort with ^G)1> F = fun(_) -> true end.#Fun2> K = fun(Elem) -> Elem > 3 end.#Fun3> lists:dropwhile(F, [1,2,3,4,5]). []4> lists:dropwhile(K, [1,2,3,4,5]). [1,2,3,4,5] Is this behavior expected? Am I missing something? ?? ?Cheers, Fxmy.? ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Sun Feb 21 14:34:22 2016 From: sergej.jurecko@REDACTED (=?utf-8?Q?Sergej_Jure=C4=8Dko?=) Date: Sun, 21 Feb 2016 14:34:22 +0100 Subject: [erlang-questions] Weird lists:dropwhile/2 behavior? In-Reply-To: References: Message-ID: <51C08225-739D-4ECC-B1C0-8A98A8B34F7C@gmail.com> You probably should be using lists:filter for K, or better yet a list comprehension. lists:dropwhile will stop processing as soon as K returns false. Sergej > On 21 Feb 2016, at 13:39, fxmy wang wrote: > > Hello guys, > > Just encountered this: > Erlang/OTP 18 [erts-7.0.3] [source-5991161] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V7.0.3 (abort with ^G) > 1> F = fun(_) -> true end. > #Fun > 2> K = fun(Elem) -> Elem > 3 end. > #Fun > 3> lists:dropwhile(F, [1,2,3,4,5]). > [] > 4> lists:dropwhile(K, [1,2,3,4,5]). > [1,2,3,4,5] > Is this behavior expected? > Am I missing something? > > ?? > > > > > ?Cheers, > > > Fxmy.? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Sun Feb 21 14:35:23 2016 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Sun, 21 Feb 2016 15:35:23 +0200 Subject: [erlang-questions] Weird lists:dropwhile/2 behavior? In-Reply-To: References: Message-ID: <8270E9B1-C984-4009-991C-BA37380DAD89@gmail.com> Hello, This is correct behavior. http://erldocs.com/17.5/stdlib/lists.html?i=1&search=lists:drop#dropwhile/2 "Drops elements Elem from List1 while Pred(Elem) returns true and returns the remaining list." Your first case: the predicate return true, empty list is the correct result. Your second case: the predicate return false for the first element, remaining list is returned. - Dmitry > On Feb 21, 2016, at 2:39 PM, fxmy wang wrote: > > Hello guys, > > Just encountered this: > Erlang/OTP 18 [erts-7.0.3] [source-5991161] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V7.0.3 (abort with ^G > ) > > 1> F = fun(_) -> true end > . > > #Fun > > > 2> K = fun(Elem) -> Elem > 3 end > . > > #Fun > > > 3> lists:dropwhile(F, [1,2,3,4,5 > ]). > [] > > 4> lists:dropwhile(K, [1,2,3,4,5 > ]). > [ > 1,2,3,4,5 > ] > > Is this behavior expected? > Am I missing something? > > ?? > > > > > ?Cheers, > > > Fxmy.? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jesper.louis.andersen@REDACTED Sun Feb 21 14:36:05 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 21 Feb 2016 14:36:05 +0100 Subject: [erlang-questions] Weird lists:dropwhile/2 behavior? In-Reply-To: References: Message-ID: On Sun, Feb 21, 2016 at 1:39 PM, fxmy wang wrote: > Is this behavior expected? yes. To make it explicit: In lists:dropwhile(fun(E) -> E > 3 end, [1,2,3,4,5]), we first check if 1 > 3. This is false, so we stop dropping and return the remainder of the list, which is [1,2,3,4,5]. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Sun Feb 21 15:11:12 2016 From: zxq9@REDACTED (zxq9) Date: Sun, 21 Feb 2016 23:11:12 +0900 Subject: [erlang-questions] Weird lists:dropwhile/2 behavior? In-Reply-To: References: Message-ID: <1968981.SlC76hMsqk@changa> On 2016?2?21? ??? 14:36:05 Jesper Louis Andersen wrote: > On Sun, Feb 21, 2016 at 1:39 PM, fxmy wang wrote: > > > Is this behavior expected? > > > yes. To make it explicit: > > In lists:dropwhile(fun(E) -> E > 3 end, [1,2,3,4,5]), we first check if 1 > > 3. This is false, so we stop dropping and return the remainder of the list, > which is [1,2,3,4,5]. Hi Fxmy. To follow up with a counter example: 1> J = fun(Z) -> Z < 3 end. #Fun 2> lists:dropwhile(J, [1,2,3,4,5]). [3,4,5] Compare that with yours: 2> K = fun(Elem) -> Elem > 3 end. #Fun 4> lists:dropwhile(K, [1,2,3,4,5]). [1,2,3,4,5] And now an example over a valley: 3> Q = fun(Z) -> Z > 3 end. #Fun 4> lists:dropwhile(Q, [5,4,3,2,1,2,3,4,5]). [3,2,1,2,3,4,5] Just like it says on the box. -Craig From fxmywc@REDACTED Sun Feb 21 14:40:50 2016 From: fxmywc@REDACTED (fxmy wang) Date: Sun, 21 Feb 2016 21:40:50 +0800 Subject: [erlang-questions] Weird lists:dropwhile/2 behavior? In-Reply-To: References: Message-ID: Thanks guys, somehow the doc gives me a feeling that lists:dropwhile will scan through all the list no matter Pred(Elem) returns 'true' or 'false'. =) Cheers 2016-02-21 21:36 GMT+08:00 Jesper Louis Andersen < jesper.louis.andersen@REDACTED>: > > On Sun, Feb 21, 2016 at 1:39 PM, fxmy wang wrote: > >> Is this behavior expected? > > > yes. To make it explicit: > > In lists:dropwhile(fun(E) -> E > 3 end, [1,2,3,4,5]), we first check if 1 > > 3. This is false, so we stop dropping and return the remainder of the > list, which is [1,2,3,4,5]. > > > -- > J. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Sun Feb 21 19:53:52 2016 From: erlang@REDACTED (Joe Armstrong) Date: Sun, 21 Feb 2016 19:53:52 +0100 Subject: [erlang-questions] Sending message at a specific and accurate time Message-ID: I want to send a UDP message at a specific time - For each event I spawn a process at_time_send(Host, Port, Time, Socket, Bin) -> %% io:format("at time ~p send",[Time]), spawn(fun() -> send_message_at_time(Host, Port, Time, Socket, Bin) end). The process sends a message to itself at the specified time send_message_at_time(Host, Port, WantedTime, Socket, Bin) -> Tnow = my_now(), Delta = trunc((WantedTime - Tnow)*1000), if Delta > 0 -> erlang:send_after(Delta, self(), ping), receive ping -> gen_udp:send(Socket, Host, Port, Bin) end; true -> void end. my_now() -> %% seconds past epoch erlang:system_time()/1000000000. I'm getting about a 4 - 9 ms. inaccuracy in the time the messages is sent and the time I want it to be sent - but I'd like it to be much more accurate (sub ms if possible) Is there a more accurate way to do this than in my code? Should I just have one process for all events (not one per event) - can I mess with process priorities or command line flag to achieve better timing accuracy? Cheers /Joe From jesper.louis.andersen@REDACTED Sun Feb 21 20:32:03 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 21 Feb 2016 20:32:03 +0100 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: References: Message-ID: On Sun, Feb 21, 2016 at 7:53 PM, Joe Armstrong wrote: > I'm getting about a 4 - 9 ms. inaccuracy in the time the messages is sent > and > the time I want it to be sent - but I'd like it to be much more accurate > (sub ms if possible) > I would start by making measurements without the network component. When you would send the gen_udp message, you take a timestamp, and analyze the skew from the suggested skew. This lets you estimate the overhead of the rest of the system in isolation from your own code and the Erlang VM. Intuitively, 4 to 9 milliseconds is much higher than what I would expect. But note that if you sleep for, say, 40ms, you will be awoken on the 41ms flank at the earliest. This is because you are usually "inside" a millisecond when you make the call so you start by taking the ceiling of that milli-second before you. How much other work is your Erlang VM doing when you make these measurements? You are saying between 4 to 9 ms, which is variance suggesting the VM has lots of work to do at that moment. And of course such stuff will affect the running time. You can switch priority of your processes up to high, but this comes at the expense of other calculations if you can't finish your work quickly in the process with high priority. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Sun Feb 21 21:10:12 2016 From: rvirding@REDACTED (Robert Virding) Date: Sun, 21 Feb 2016 21:10:12 +0100 Subject: [erlang-questions] Erlounge in Barcelona In-Reply-To: References: Message-ID: OK, the my suggestion is that we meet on Wednesday evening for an erlounge. Where I don't know. Anyone have a suggestion? Robert On 21 February 2016 at 13:08, Vance Shipley wrote: > On Sun, Feb 21, 2016 at 5:29 PM, Robert Virding > wrote: > > I would prefer Monday - Wednesday. Vance are you here on Wednesday > evening? > > Yes, Wednesday evening is fine for me, I leave Thursday morning. > > -- > -Vance > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Sun Feb 21 22:20:33 2016 From: erlang@REDACTED (Joe Armstrong) Date: Sun, 21 Feb 2016 22:20:33 +0100 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: References: Message-ID: I tried a simpler program: test() -> test(10, []). test(0, L) -> L; test(K, L) -> T1 = ms_time(), erlang:send_after(5000, self(), ping), receive ping -> T2 = ms_time(), test(K-1, [T2-T1|L]) end. ms_time() -> erlang:system_time() div 1000000. Running this gives the following times [5001,5001,5006,5006,5002,5003,5006,5002,5002,5006] I'd expected 5000 or 5001 This is in an unloaded OS with an unloaded erlang. 6ms seems very long - there are very few processes running and the system load is virtually zero. I tried erl -snp disable and setting process_flag(priority, max) but the results are pretty much the same. Waiting for shorter times like 100 ms makes no difference - still events with a 6 ms delay. I want to use this for scheduling music events (controlling synths) and these delays are far more than I expected. /Joe On Sun, Feb 21, 2016 at 8:32 PM, Jesper Louis Andersen wrote: > > On Sun, Feb 21, 2016 at 7:53 PM, Joe Armstrong wrote: >> >> I'm getting about a 4 - 9 ms. inaccuracy in the time the messages is sent >> and >> the time I want it to be sent - but I'd like it to be much more accurate >> (sub ms if possible) > > > I would start by making measurements without the network component. When you > would send the gen_udp message, you take a timestamp, and analyze the skew > from the suggested skew. This lets you estimate the overhead of the rest of > the system in isolation from your own code and the Erlang VM. > > Intuitively, 4 to 9 milliseconds is much higher than what I would expect. > But note that if you sleep for, say, 40ms, you will be awoken on the 41ms > flank at the earliest. This is because you are usually "inside" a > millisecond when you make the call so you start by taking the ceiling of > that milli-second before you. > > How much other work is your Erlang VM doing when you make these > measurements? You are saying between 4 to 9 ms, which is variance suggesting > the VM has lots of work to do at that moment. And of course such stuff will > affect the running time. You can switch priority of your processes up to > high, but this comes at the expense of other calculations if you can't > finish your work quickly in the process with high priority. > > > -- > J. From felixgallo@REDACTED Sun Feb 21 22:34:00 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Sun, 21 Feb 2016 13:34:00 -0800 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: References: Message-ID: What os and hardware are you seeing these results on? On Feb 21, 2016 1:20 PM, "Joe Armstrong" wrote: > I tried a simpler program: > > test() -> test(10, []). > > test(0, L) -> > L; > test(K, L) -> > T1 = ms_time(), > erlang:send_after(5000, self(), ping), > receive > ping -> > T2 = ms_time(), > test(K-1, [T2-T1|L]) > end. > > ms_time() -> > erlang:system_time() div 1000000. > > Running this gives the following times > > [5001,5001,5006,5006,5002,5003,5006,5002,5002,5006] > > I'd expected 5000 or 5001 > > This is in an unloaded OS with an unloaded erlang. 6ms seems very long - > there are very few processes running and the system load is virtually zero. > > I tried erl -snp disable and setting process_flag(priority, max) but the > results > are pretty much the same. > > Waiting for shorter times like 100 ms makes no difference - still > events with a 6 ms delay. > > I want to use this for scheduling music events (controlling synths) and > these > delays are far more than I expected. > > /Joe > > > > On Sun, Feb 21, 2016 at 8:32 PM, Jesper Louis Andersen > wrote: > > > > On Sun, Feb 21, 2016 at 7:53 PM, Joe Armstrong wrote: > >> > >> I'm getting about a 4 - 9 ms. inaccuracy in the time the messages is > sent > >> and > >> the time I want it to be sent - but I'd like it to be much more accurate > >> (sub ms if possible) > > > > > > I would start by making measurements without the network component. When > you > > would send the gen_udp message, you take a timestamp, and analyze the > skew > > from the suggested skew. This lets you estimate the overhead of the rest > of > > the system in isolation from your own code and the Erlang VM. > > > > Intuitively, 4 to 9 milliseconds is much higher than what I would expect. > > But note that if you sleep for, say, 40ms, you will be awoken on the 41ms > > flank at the earliest. This is because you are usually "inside" a > > millisecond when you make the call so you start by taking the ceiling of > > that milli-second before you. > > > > How much other work is your Erlang VM doing when you make these > > measurements? You are saying between 4 to 9 ms, which is variance > suggesting > > the VM has lots of work to do at that moment. And of course such stuff > will > > affect the running time. You can switch priority of your processes up to > > high, but this comes at the expense of other calculations if you can't > > finish your work quickly in the process with high priority. > > > > > > -- > > J. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Sun Feb 21 22:40:44 2016 From: erlang@REDACTED (Joe Armstrong) Date: Sun, 21 Feb 2016 22:40:44 +0100 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: References: Message-ID: mac os-x El Capitan 10.11.2 Macbook retina pro 3.1 GHz core i7 16 GB ram Can you run the above program locally and post back the results? /Joe On Sun, Feb 21, 2016 at 10:34 PM, Felix Gallo wrote: > What os and hardware are you seeing these results on? > > On Feb 21, 2016 1:20 PM, "Joe Armstrong" wrote: >> >> I tried a simpler program: >> >> test() -> test(10, []). >> >> test(0, L) -> >> L; >> test(K, L) -> >> T1 = ms_time(), >> erlang:send_after(5000, self(), ping), >> receive >> ping -> >> T2 = ms_time(), >> test(K-1, [T2-T1|L]) >> end. >> >> ms_time() -> >> erlang:system_time() div 1000000. >> >> Running this gives the following times >> >> [5001,5001,5006,5006,5002,5003,5006,5002,5002,5006] >> >> I'd expected 5000 or 5001 >> >> This is in an unloaded OS with an unloaded erlang. 6ms seems very long - >> there are very few processes running and the system load is virtually >> zero. >> >> I tried erl -snp disable and setting process_flag(priority, max) but the >> results >> are pretty much the same. >> >> Waiting for shorter times like 100 ms makes no difference - still >> events with a 6 ms delay. >> >> I want to use this for scheduling music events (controlling synths) and >> these >> delays are far more than I expected. >> >> /Joe >> >> >> >> On Sun, Feb 21, 2016 at 8:32 PM, Jesper Louis Andersen >> wrote: >> > >> > On Sun, Feb 21, 2016 at 7:53 PM, Joe Armstrong wrote: >> >> >> >> I'm getting about a 4 - 9 ms. inaccuracy in the time the messages is >> >> sent >> >> and >> >> the time I want it to be sent - but I'd like it to be much more >> >> accurate >> >> (sub ms if possible) >> > >> > >> > I would start by making measurements without the network component. When >> > you >> > would send the gen_udp message, you take a timestamp, and analyze the >> > skew >> > from the suggested skew. This lets you estimate the overhead of the rest >> > of >> > the system in isolation from your own code and the Erlang VM. >> > >> > Intuitively, 4 to 9 milliseconds is much higher than what I would >> > expect. >> > But note that if you sleep for, say, 40ms, you will be awoken on the >> > 41ms >> > flank at the earliest. This is because you are usually "inside" a >> > millisecond when you make the call so you start by taking the ceiling of >> > that milli-second before you. >> > >> > How much other work is your Erlang VM doing when you make these >> > measurements? You are saying between 4 to 9 ms, which is variance >> > suggesting >> > the VM has lots of work to do at that moment. And of course such stuff >> > will >> > affect the running time. You can switch priority of your processes up to >> > high, but this comes at the expense of other calculations if you can't >> > finish your work quickly in the process with high priority. >> > >> > >> > -- >> > J. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From jesper.louis.andersen@REDACTED Sun Feb 21 22:56:40 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 21 Feb 2016 22:56:40 +0100 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: References: Message-ID: On a Archlinux machine, Core i7 Haswell: 6> z:test(). [5001,5001,5001,5001,5001,5001,5001,5001,5001,5001] This is what I expect to see. I'm awoken on the right flank up to one millisecond after. Immediate thought: this has to do with CPU frequency scaling and battery saving features of OSX. Since nothing is done for 5 seconds, the system throttles down and goes to sleep. When it is awoken again, you are enjoying some latency. Enable dtrace and hunt :) On Sun, Feb 21, 2016 at 10:40 PM, Joe Armstrong wrote: > mac os-x El Capitan 10.11.2 > Macbook retina pro > 3.1 GHz core i7 > 16 GB ram > > Can you run the above program locally and post back the results? > > > > /Joe > > On Sun, Feb 21, 2016 at 10:34 PM, Felix Gallo > wrote: > > What os and hardware are you seeing these results on? > > > > On Feb 21, 2016 1:20 PM, "Joe Armstrong" wrote: > >> > >> I tried a simpler program: > >> > >> test() -> test(10, []). > >> > >> test(0, L) -> > >> L; > >> test(K, L) -> > >> T1 = ms_time(), > >> erlang:send_after(5000, self(), ping), > >> receive > >> ping -> > >> T2 = ms_time(), > >> test(K-1, [T2-T1|L]) > >> end. > >> > >> ms_time() -> > >> erlang:system_time() div 1000000. > >> > >> Running this gives the following times > >> > >> [5001,5001,5006,5006,5002,5003,5006,5002,5002,5006] > >> > >> I'd expected 5000 or 5001 > >> > >> This is in an unloaded OS with an unloaded erlang. 6ms seems very long - > >> there are very few processes running and the system load is virtually > >> zero. > >> > >> I tried erl -snp disable and setting process_flag(priority, max) but the > >> results > >> are pretty much the same. > >> > >> Waiting for shorter times like 100 ms makes no difference - still > >> events with a 6 ms delay. > >> > >> I want to use this for scheduling music events (controlling synths) and > >> these > >> delays are far more than I expected. > >> > >> /Joe > >> > >> > >> > >> On Sun, Feb 21, 2016 at 8:32 PM, Jesper Louis Andersen > >> wrote: > >> > > >> > On Sun, Feb 21, 2016 at 7:53 PM, Joe Armstrong > wrote: > >> >> > >> >> I'm getting about a 4 - 9 ms. inaccuracy in the time the messages is > >> >> sent > >> >> and > >> >> the time I want it to be sent - but I'd like it to be much more > >> >> accurate > >> >> (sub ms if possible) > >> > > >> > > >> > I would start by making measurements without the network component. > When > >> > you > >> > would send the gen_udp message, you take a timestamp, and analyze the > >> > skew > >> > from the suggested skew. This lets you estimate the overhead of the > rest > >> > of > >> > the system in isolation from your own code and the Erlang VM. > >> > > >> > Intuitively, 4 to 9 milliseconds is much higher than what I would > >> > expect. > >> > But note that if you sleep for, say, 40ms, you will be awoken on the > >> > 41ms > >> > flank at the earliest. This is because you are usually "inside" a > >> > millisecond when you make the call so you start by taking the ceiling > of > >> > that milli-second before you. > >> > > >> > How much other work is your Erlang VM doing when you make these > >> > measurements? You are saying between 4 to 9 ms, which is variance > >> > suggesting > >> > the VM has lots of work to do at that moment. And of course such stuff > >> > will > >> > affect the running time. You can switch priority of your processes up > to > >> > high, but this comes at the expense of other calculations if you can't > >> > finish your work quickly in the process with high priority. > >> > > >> > > >> > -- > >> > J. > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Sun Feb 21 23:00:20 2016 From: erlang@REDACTED (Joe Armstrong) Date: Sun, 21 Feb 2016 23:00:20 +0100 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: References: Message-ID: On Sun, Feb 21, 2016 at 10:56 PM, Jesper Louis Andersen wrote: > On a Archlinux machine, Core i7 Haswell: > > 6> z:test(). > [5001,5001,5001,5001,5001,5001,5001,5001,5001,5001] > > This is what I expect to see. I'm awoken on the right flank up to one > millisecond after. > > Immediate thought: this has to do with CPU frequency scaling and battery > saving features of OSX. Since nothing is done for 5 seconds, the system > throttles down and goes to sleep. When it is awoken again, you are enjoying > some latency. It was the same with 100 ms delays > > Enable dtrace and hunt :) > > > On Sun, Feb 21, 2016 at 10:40 PM, Joe Armstrong wrote: >> >> mac os-x El Capitan 10.11.2 >> Macbook retina pro >> 3.1 GHz core i7 >> 16 GB ram >> >> Can you run the above program locally and post back the results? >> >> >> >> /Joe >> >> On Sun, Feb 21, 2016 at 10:34 PM, Felix Gallo >> wrote: >> > What os and hardware are you seeing these results on? >> > >> > On Feb 21, 2016 1:20 PM, "Joe Armstrong" wrote: >> >> >> >> I tried a simpler program: >> >> >> >> test() -> test(10, []). >> >> >> >> test(0, L) -> >> >> L; >> >> test(K, L) -> >> >> T1 = ms_time(), >> >> erlang:send_after(5000, self(), ping), >> >> receive >> >> ping -> >> >> T2 = ms_time(), >> >> test(K-1, [T2-T1|L]) >> >> end. >> >> >> >> ms_time() -> >> >> erlang:system_time() div 1000000. >> >> >> >> Running this gives the following times >> >> >> >> [5001,5001,5006,5006,5002,5003,5006,5002,5002,5006] >> >> >> >> I'd expected 5000 or 5001 >> >> >> >> This is in an unloaded OS with an unloaded erlang. 6ms seems very long >> >> - >> >> there are very few processes running and the system load is virtually >> >> zero. >> >> >> >> I tried erl -snp disable and setting process_flag(priority, max) but >> >> the >> >> results >> >> are pretty much the same. >> >> >> >> Waiting for shorter times like 100 ms makes no difference - still >> >> events with a 6 ms delay. >> >> >> >> I want to use this for scheduling music events (controlling synths) and >> >> these >> >> delays are far more than I expected. >> >> >> >> /Joe >> >> >> >> >> >> >> >> On Sun, Feb 21, 2016 at 8:32 PM, Jesper Louis Andersen >> >> wrote: >> >> > >> >> > On Sun, Feb 21, 2016 at 7:53 PM, Joe Armstrong >> >> > wrote: >> >> >> >> >> >> I'm getting about a 4 - 9 ms. inaccuracy in the time the messages is >> >> >> sent >> >> >> and >> >> >> the time I want it to be sent - but I'd like it to be much more >> >> >> accurate >> >> >> (sub ms if possible) >> >> > >> >> > >> >> > I would start by making measurements without the network component. >> >> > When >> >> > you >> >> > would send the gen_udp message, you take a timestamp, and analyze the >> >> > skew >> >> > from the suggested skew. This lets you estimate the overhead of the >> >> > rest >> >> > of >> >> > the system in isolation from your own code and the Erlang VM. >> >> > >> >> > Intuitively, 4 to 9 milliseconds is much higher than what I would >> >> > expect. >> >> > But note that if you sleep for, say, 40ms, you will be awoken on the >> >> > 41ms >> >> > flank at the earliest. This is because you are usually "inside" a >> >> > millisecond when you make the call so you start by taking the ceiling >> >> > of >> >> > that milli-second before you. >> >> > >> >> > How much other work is your Erlang VM doing when you make these >> >> > measurements? You are saying between 4 to 9 ms, which is variance >> >> > suggesting >> >> > the VM has lots of work to do at that moment. And of course such >> >> > stuff >> >> > will >> >> > affect the running time. You can switch priority of your processes up >> >> > to >> >> > high, but this comes at the expense of other calculations if you >> >> > can't >> >> > finish your work quickly in the process with high priority. >> >> > >> >> > >> >> > -- >> >> > J. >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > J. From ok@REDACTED Mon Feb 22 00:32:22 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 22 Feb 2016 12:32:22 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56C6B04B.3090903@ericsson.com> References: <56C4411E.9060106@rol.ru> <56C6A0DF.9020600@cs.otago.ac.nz> <56C6B04B.3090903@ericsson.com> Message-ID: <56CA4906.80709@cs.otago.ac.nz> On 19/02/16 7:03 pm, Bengt Kleberg wrote: > Greetings, > > Given the problem with case-sensitive file systems, and identifiers in > code, why not ignore case? Actually, there are three approaches file systems have taken: (1) Normalise case on creation, ignore case when matching. IBM MVS, DEC TOPS-10, MS-DOS. (2) Preserve case on creation, ignore case when matching. Classic Mac OS, Mac OS X (own file system), NTFS. (3) Presrve case on creation, heed case when matching. UNIX. The central problem these days is that both case normalisation and case ignoring are locale-dependent. I might write two modules in locale X which in locale X are clearly different even after converting to a standard case (Unicode recommands lower case). But in locale Y they might be converted to the same string.It seems to work for Eiffel. No results from searching for Eiffel and problems with case. It is not a large language, so the number of people that have been exposed to case insensitivity is small. The annotated Ada 2012 Reference Manual gets a little confusing: {AI95-00395-01} {AI05-0227-1} When this International Standard mentions the conversion of some character or sequence of characters to upper case, it means the character or sequence of characters obtained by using simple upper case mappinglocale-independent full case folding, as defined by documents referenced in the note in section 1 of ISO/IEC 1046:2003. Discussion: Unless otherwise specified for sequences of characters, case folding is applied to the sequence, not to individual characters. It sometimes can make a difference. ... {AI95-00395-01} We now explicitly define what the Standard means by uper case, as there are many possibilities for ISO 10646 characters. {AI05-0227-1} Correction: Upper case is defined by "simple upper case" mapping", because "full case folding" is a mapping (mostly) to lower case. {AI95-00285-01}The description of the language definition in this International Standard uses the character properties General Category, Simple Uppercase Mapping, Uppercase Mapping, and Special Case Condition of the documents referenced by the note in section 1 of ISO/IEC 10646:2003. It took *years* for the Ada people to sort out exactly what they meant by case conversion, and the result is quite clearly WRONG for some users. Tht's not really the problem. The problem is that file systems that ignore case (one way or the other) will have their OWN rules about case conversion, and we cannot expect them to match Ada's or any other language's exactly. There are plenty of arguments for case insensitivity in programming languages. I don't think this one will work. though. From ok@REDACTED Mon Feb 22 01:45:30 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 22 Feb 2016 13:45:30 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56C99A94.2020009@ninenines.eu> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> Message-ID: <56CA5A2A.9020102@cs.otago.ac.nz> On 22/02/16 12:08 am, Lo?c Hoguin wrote: > It seems that you consider beam and source files to be equivalent. Why would you say that? What *is* true is that the name of a .beam file is derived from the name of a .erl file, that the module name must match the pre-extension part of the .erl file, and that the module name must also match the pre-extension part of the .beam file. Example: % echo "-module(foo)." >foo.erl % erlc foo.erl % mv foo.beam Foo.beam % erl 1> l('Foo'). {error,badfile} =ERROR REPORT==== 22-Feb-2016::12:56:36 === Loading of /home/cshome/o/ok/COSC400/Foo.beam failed: badfile =ERROR REPORT==== 22-Feb-2016::12:56:36 === beam/beam_load.c(1278): Error loading module 'Foo': module name in object code is foo > A beam file should contain a module name, there's no doubt about this. I think you need to argue for that. > So loading from a .ez file would work, so upgrading would work, and so on. You're skipping over the key step. As far as I know, loading a file from a .ez file is NOT done by searching the contents of the pseudo-files searching for one with a matching module name, but it's based on the (pseudo-)file name, just like a file in the host file system, but with possibly different naming rules. > But there's no point in having the source file require the module name > directly in the source if the compiler can figure it out (and it can > *because we give it* when we ask the compiler to compile files). Eh? When we ask the compiler to compile FILES we give FILE names. MODULE names look like file names (as long as you close one eye and squint with the other) but they are not the same thing. To make just one little distinction, for every programming language I know, identifiers are either ALWAYS case sensitive or ALWAYS case insensitive. But in a file system, file names in some directories may be case sensitive and file names in other directories may not be case sensitive. *IF* module names were restricted to lower case letters, the compiler could figure out that 'LISTS.ERL' contained the source for 'lists'. But they aren't, so it can't. For that matter, on my laptop (a Mac), for all the compiler can tell, 'lists.erl' might be the file for 'LISTS' or 'Lists'. > The compiler should error out if it can't find a module name *at all*, > not if it can't find it *in the source*. Right now we have a combination of two things. (1) A simple obvious human-friendly -module directive that informs both human beings and compilers of the name of a module, *regardless* of the kind of container the text is held in (it could, for example, be a CLOB in a data base), and at a cost to the programmer of somewhere between negligible and zero. (2) A crude hack for autoloading that hopes file names can be long enough to hold module names (I have a VM with a mainframe operating system that limits names to 17 characters) and that hopes nobody will have two modules whose names differ only in case and that hopes Unicode isn't a problem. This really also covers loading: when you say l(foobar) Erlang thinks you're giving it a file name but what you usually want to give it is a module name. Now (2) was fair enough at the time; it was entirely typical of the days when everything was ASCII and 100 MB was a lot of disc. Those days are gone. Until (2) is fixed somehow, (1) and its analogue of 'redundantly' storing a module name in a .beam file are our principal protection against (2) going wrong, which it does. (1) is not a problem. (2) is the problem. (1) is the (current) solution. I have this mental picture of someone out at sea in a small boat saying "what's this nasty old thing sticking in the bottom?" and wanting to pull out the drain plug. From ok@REDACTED Mon Feb 22 01:51:28 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 22 Feb 2016 13:51:28 +1300 Subject: [erlang-questions] Weird lists:dropwhile/2 behavior? In-Reply-To: References: Message-ID: <56CA5B90.6020108@cs.otago.ac.nz> On 22/02/16 1:39 am, fxmy wang wrote: K = fun (E) -> E > 3 end. lists:dropwhile(K, [1,2,3,4,5]). What should we expect that to do? dropwhile(Test, List) says "throw away the longest prefix of List such that every element of the prefix satisfies the Test." What is the longest prefix of [1,2,3,4,5] such that every element is greater than 3? The empty list. So we expect the answer [1,2,3,4,5], and indeed, that's what we get. lists:filter(K, [3,1,4,2,5]) will of course answer [4,5], because filter is defined to look at all the elements, not just a prefix. Without looking at the source code, it will be something like dropwhile(F, L = [X|Xs]) -> case F(X) of true -> dropwhile(F, Xs) ; false -> L end; dropwhile(_, L) -> L. From ok@REDACTED Mon Feb 22 02:08:26 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 22 Feb 2016 14:08:26 +1300 Subject: [erlang-questions] Weird lists:dropwhile/2 behavior? In-Reply-To: References: Message-ID: <56CA5F8A.4080603@cs.otago.ac.nz> On 22/02/16 2:40 am, fxmy wang wrote: > Thanks guys, somehow the doc gives me a feeling that lists:dropwhile > will scan through all the list no matter Pred(Elem) returns 'true' or > 'false'. =) The documentation is unfortunately sparse. dropwhile(Pred, List1) -> List2 Types: Pred = fun((Elem :: T) -> boolean()) List1 = List2 = [T] T = term() Drops elements Elem from List1 while Pred(Elem) returns true and returns the remaining list. You have to read closely to see the word WHILE there. For takewhile/2 it's clearer: takewhile(Pred, List1) -> List2 Types: Pred = fun((Elem :: T) -> boolean()) List1 = List2 = [T] T = term() Takes elements Elem from List1 while Pred(Elem) returns true, that is, the function returns the longest prefix of the list for which all elements satisfy the predicate. For all L and for all F such that [F(X) || X <- L] contains only true and false, takewhile(F, L) ++ dropwhile(F, L) == L. The takewhile and dropwhile functions (why on earth are they not take_while and drop_while? since when has Erlang runwordstogetherlikeC?) are not unique to Erlang. For example, in Haskell > dropWhile (> 3) [1..5] [1,2,3,4,5] I propose the following revision to the documentation. %% dropwhile Let List == Prefix ++ Suffix where Pred(X) is true for every X in Prefix and Prefix is as long as possible. dropwhile(Pred, List) returns Prefix. See takewhile/2. %% takewhile Let List == Prefix ++ Suffix where Pred(X) is true for every X in Prefix and Prefix is as long as possible. takewhile(Pred, List) returns Suffix. See dropwhile/2. From essen@REDACTED Mon Feb 22 02:38:46 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Mon, 22 Feb 2016 02:38:46 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56CA5A2A.9020102@cs.otago.ac.nz> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> Message-ID: <56CA66A6.9070901@ninenines.eu> On 02/22/2016 01:45 AM, Richard A. O'Keefe wrote: >> But there's no point in having the source file require the module name >> directly in the source if the compiler can figure it out (and it can >> *because we give it* when we ask the compiler to compile files). > > Eh? When we ask the compiler to compile FILES we give FILE names. > MODULE names look like file names (as long as you close one eye and > squint with the other) but they are not the same thing. I don't see how that's going to be any worse than what we currently have. Today, if we take for example OSX, we have this: - Create file m.erl containing -module(m). - erlc m.erl succeeds - erlc M.erl fails with Module name 'm' does not match file name 'M' - Create file M.erl containing -module(m). - erlc m.erl succeeds - erlc M.erl fails like above Despite the fact that my filesystem says I have a file M.erl, I can compile it just fine if I tell erlc it's "m.erl" instead. If we didn't have the -module line because it became optional, we could have the module name taken from the file name we give to erlc. - If you run "erlc my_module.erl" it should create the module my_module. - If you run "erlc MY_module.erl" it should create the module MY_module. In both cases, what you see on the disk and the module name can appear different. It's no better or worse than what we currently have. One could argue that currently if you do "erlc MY_module.erl" you get an error right away (as opposed to "my_module" being not found later on if module name was taken from file name). I believe this is largely a non-issue. I have yet to hear from anyone trying to name their files with uppercase characters and ending up with the module name not matching. And despite this being the hundredth time -module has been brought up, I don't recall seeing anyone come and say they ran into this problem. On the other hand I recall seeing this topic many times because people feel this attribute shouldn't be mandatory. Concerns about encodings/charsets are also unfounded. We have the same problem in both cases, except in the current situation we take the file name and compare it to the declared module name, and for optional -module attribute you take the file name and use it as the module name. If you can do the first you can do the second. If you compile from forms, then the module name should of course remain mandatory. It's only when compiling from files that it could be optional. And there's no technical barriers to prevent it. Regardless of the technical merits of one or the other solution, when developing software it's generally a good idea to listen to what you users want, and this Nth topic about -module should be a clear indication that the current situation is not what many users want. It doesn't matter if you think you are technically correct, because you defend a point of view that many people do not care about. All solutions have their strong and weak points, and enough people are annoyed by the current solution's weak points (the mandatory -module attribute) to create topics every couple months. People tend to want different things, so when so many people agree on something it's time to take notice. This happens a lot less than you'd think. In general it's also a good idea to remove as many steps as possible to get code up and running, if only to give the illusion of Erlang being a productive language, as this attracts newcomers much more easily. Erlang is the most productive language I've seen (in large parts due to readability and low maintenance) but it's doing a terrible job showing it. We often say Erlang makes hard things easy and easy things hard. Perhaps it's time to seriously consider making easy things easier. This is the best way to get more people in. But I get the feeling not everyone wants that, considering how little effort is spent on that end (maps happened, but it seemed more like a miracle than anything). -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From vinoski@REDACTED Mon Feb 22 03:28:24 2016 From: vinoski@REDACTED (Steve Vinoski) Date: Sun, 21 Feb 2016 21:28:24 -0500 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56CA66A6.9070901@ninenines.eu> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> Message-ID: On Sun, Feb 21, 2016 at 8:38 PM, Lo?c Hoguin wrote: > Regardless of the technical merits of one or the other solution, when > developing software it's generally a good idea to listen to what you users > want, and this Nth topic about -module should be a clear indication that > the current situation is not what many users want. > > It doesn't matter if you think you are technically correct, because you > defend a point of view that many people do not care about. All solutions > have their strong and weak points, and enough people are annoyed by the > current solution's weak points (the mandatory -module attribute) to create > topics every couple months. People tend to want different things, so when > so many people agree on something it's time to take notice. This happens a > lot less than you'd think. > I don't recall seeing this topic come up "every couple months". Can you provide pointers to the various threads where this has come up so often, other than this one, to back up this claim? Google searches of erlang.org and searches through stackoverflow.com don't turn up much at all on this topic. In general it's also a good idea to remove as many steps as possible to get > code up and running, if only to give the illusion of Erlang being a > productive language, as this attracts newcomers much more easily. Over the years I've heard reasons for people avoiding Erlang, but I personally have never even once heard someone say they didn't want to use Erlang because of the burdensome -module attribute. > Erlang is the most productive language I've seen (in large parts due to > readability and low maintenance) but it's doing a terrible job showing it. > > We often say Erlang makes hard things easy and easy things hard. Perhaps > it's time to seriously consider making easy things easier. This is the best > way to get more people in. But I get the feeling not everyone wants that, > considering how little effort is spent on that end (maps happened, but it > seemed more like a miracle than anything). IMO that statement is rather insulting to those who've put a lot of time and effort into designing and implementing maps. It's not at all a simple problem. If having an optional -module attribute is that important to you, write a pull request. --steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Mon Feb 22 04:04:08 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Sun, 21 Feb 2016 22:04:08 -0500 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> Message-ID: <20160222030407.GB53561@ferdmbp.local> On 02/21, Steve Vinoski wrote: > >If having an optional -module attribute is that important to you, write a >pull request. > I have written a program that does what is required and can be quite simply added to a tool such as erlang.mk. $ awk 'NR==1 {x=FILENAME; gsub(/^.+\//, "", x); gsub(/\.erl$/, "", x); print "-module("x").\n" $0; next } !/^-module(.*).$/ {print}' $FILENAME If someone feels they would benefit from filenames taking over the module names, this script will fully edit out an existing `-module(name).` line and instead replace it taken from one using the filename. I hope this helps, please refine as you see fit, I'm releasing this as public domain. From g@REDACTED Mon Feb 22 05:24:08 2016 From: g@REDACTED (Garrett Smith) Date: Mon, 22 Feb 2016 04:24:08 +0000 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <20160222030407.GB53561@ferdmbp.local> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <20160222030407.GB53561@ferdmbp.local> Message-ID: On Sun, Feb 21, 2016, 21:04 Fred Hebert wrote: > On 02/21, Steve Vinoski wrote: > > > >If having an optional -module attribute is that important to you, write a > >pull request. > > > > I have written a program that does what is required and can be quite > simply added to a tool such as erlang.mk. > > $ awk 'NR==1 {x=FILENAME; gsub(/^.+\//, "", x); gsub(/\.erl$/, "", x); > print "-module("x").\n" $0; next } !/^-module(.*).$/ {print}' $FILENAME > > If someone feels they would benefit from filenames taking over the > module names, this script will fully edit out an existing > `-module(name).` line and instead replace it taken from one using the > filename. > > I hope this helps, please refine as you see fit, I'm releasing this as > public domain. > OMG this so awesome!!! Seriously, if you Reddit I will upvote. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vicbaz@REDACTED Sun Feb 21 23:29:12 2016 From: vicbaz@REDACTED (vicbaz@REDACTED) Date: Mon, 22 Feb 2016 01:29:12 +0300 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: References: Message-ID: <56CA3A38.4010404@yandex.ru> Hello, Try +K flag. $ erl Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] Eshell V7.2.1 (abort with ^G) 1> c(t). {ok,t} 2> t:test(). [5003,5006,5002,5003,5006,5002,5003,5006,5001,5005] $ erl +K true Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:true] Eshell V7.2.1 (abort with ^G) 1> c(t). {ok,t} 2> t:test(). [5001,5001,5001,5001,5001,5001,5001,5001,5001,5001] $ uname -a Linux frontier 4.3.0-0.bpo.1-amd64 #1 SMP Debian 4.3.3-7~bpo8+1 (2016-01-19) x86_64 GNU/Linux $ cat /proc/cpuinfo | grep 'model name' model name : Pentium(R) Dual-Core CPU E5300 @ 2.60GHz model name : Pentium(R) Dual-Core CPU E5300 @ 2.60GHz On 22/02/16 00:20, Joe Armstrong wrote: > I tried a simpler program: > > test() -> test(10, []). > > test(0, L) -> > L; > test(K, L) -> > T1 = ms_time(), > erlang:send_after(5000, self(), ping), > receive > ping -> > T2 = ms_time(), > test(K-1, [T2-T1|L]) > end. > > ms_time() -> > erlang:system_time() div 1000000. > > Running this gives the following times > > [5001,5001,5006,5006,5002,5003,5006,5002,5002,5006] > > I'd expected 5000 or 5001 > > This is in an unloaded OS with an unloaded erlang. 6ms seems very long - > there are very few processes running and the system load is virtually zero. > > I tried erl -snp disable and setting process_flag(priority, max) but the results > are pretty much the same. > > Waiting for shorter times like 100 ms makes no difference - still > events with a 6 ms delay. > > I want to use this for scheduling music events (controlling synths) and these > delays are far more than I expected. > > /Joe > > > > On Sun, Feb 21, 2016 at 8:32 PM, Jesper Louis Andersen > wrote: >> >> On Sun, Feb 21, 2016 at 7:53 PM, Joe Armstrong wrote: >>> >>> I'm getting about a 4 - 9 ms. inaccuracy in the time the messages is sent >>> and >>> the time I want it to be sent - but I'd like it to be much more accurate >>> (sub ms if possible) >> >> >> I would start by making measurements without the network component. When you >> would send the gen_udp message, you take a timestamp, and analyze the skew >> from the suggested skew. This lets you estimate the overhead of the rest of >> the system in isolation from your own code and the Erlang VM. >> >> Intuitively, 4 to 9 milliseconds is much higher than what I would expect. >> But note that if you sleep for, say, 40ms, you will be awoken on the 41ms >> flank at the earliest. This is because you are usually "inside" a >> millisecond when you make the call so you start by taking the ceiling of >> that milli-second before you. >> >> How much other work is your Erlang VM doing when you make these >> measurements? You are saying between 4 to 9 ms, which is variance suggesting >> the VM has lots of work to do at that moment. And of course such stuff will >> affect the running time. You can switch priority of your processes up to >> high, but this comes at the expense of other calculations if you can't >> finish your work quickly in the process with high priority. >> >> >> -- >> J. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From erlang@REDACTED Mon Feb 22 08:30:11 2016 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 22 Feb 2016 08:30:11 +0100 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: <56CA3A38.4010404@yandex.ru> References: <56CA3A38.4010404@yandex.ru> Message-ID: I made a new test - here's the result with kernel poll (I've also changed the units of time to microseconds - so we can see the sub-millisecond effect) $ erl +K true Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:true] Eshell V7.2.1 (abort with ^G) 1> t:test(). [1006032,1006176,1005987,1006065,1000988,1001926,1005561, 1001584,1000335,1001442] 2> t:test(). [1001963,1005706,1006087,1006171,1005994,1005992,1005985, 1001037,1005900,1005268] To test how fast we can send a message and receive it I changed erlang:send_after(1000, self(), ping), to self() ! ping and reran - now I get delays of the order of 2 microseconds admittedly the process does not now get scheduled in and out I can only conclude that send_after(Time, Pid, Msg) is deeply broken on the mac - if inter-processing scheduling and messaging had 6 millisecond delays Erlang would be unusable. The time unit in send_after is milliseconds - and it is true that sending something 6 milliseconds after I wanted it is indeed "after" - but it would also be true if the message was sent a year after I asked for it. I guess I could do a send_after 10ms before I want and then do a busy wait but this would be crazy. Here's the slightly modified program. -module(t). -compile(export_all). test() -> process_flag(priority, max), test(10, []). test(0, L) -> L; test(K, L) -> T1 = microsecond_time(), %% erlang:send_after(1000, self(), ping), self() ! ping, receive ping -> T2 = microsecond_time(), test(K-1, [T2-T1|L]) end. microsecond_time() -> erlang:system_time() div 1000. /Joe On Sun, Feb 21, 2016 at 11:29 PM, wrote: > Hello, > > Try +K flag. > > $ erl > Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:2:2] [async-threads:10] > [hipe] [kernel-poll:false] > > Eshell V7.2.1 (abort with ^G) > 1> c(t). > {ok,t} > 2> t:test(). > [5003,5006,5002,5003,5006,5002,5003,5006,5001,5005] > > $ erl +K true > Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:2:2] [async-threads:10] > [hipe] [kernel-poll:true] > > Eshell V7.2.1 (abort with ^G) > 1> c(t). > {ok,t} > 2> t:test(). > [5001,5001,5001,5001,5001,5001,5001,5001,5001,5001] > > $ uname -a > Linux frontier 4.3.0-0.bpo.1-amd64 #1 SMP Debian 4.3.3-7~bpo8+1 (2016-01-19) > x86_64 GNU/Linux > > $ cat /proc/cpuinfo | grep 'model name' > model name : Pentium(R) Dual-Core CPU E5300 @ 2.60GHz > model name : Pentium(R) Dual-Core CPU E5300 @ 2.60GHz > > > On 22/02/16 00:20, Joe Armstrong wrote: >> >> I tried a simpler program: >> >> test() -> test(10, []). >> >> test(0, L) -> >> L; >> test(K, L) -> >> T1 = ms_time(), >> erlang:send_after(5000, self(), ping), >> receive >> ping -> >> T2 = ms_time(), >> test(K-1, [T2-T1|L]) >> end. >> >> ms_time() -> >> erlang:system_time() div 1000000. >> >> Running this gives the following times >> >> [5001,5001,5006,5006,5002,5003,5006,5002,5002,5006] >> >> I'd expected 5000 or 5001 >> >> This is in an unloaded OS with an unloaded erlang. 6ms seems very long - >> there are very few processes running and the system load is virtually >> zero. >> >> I tried erl -snp disable and setting process_flag(priority, max) but the >> results >> are pretty much the same. >> >> Waiting for shorter times like 100 ms makes no difference - still >> events with a 6 ms delay. >> >> I want to use this for scheduling music events (controlling synths) and >> these >> delays are far more than I expected. >> >> /Joe >> >> >> >> On Sun, Feb 21, 2016 at 8:32 PM, Jesper Louis Andersen >> wrote: >>> >>> >>> On Sun, Feb 21, 2016 at 7:53 PM, Joe Armstrong wrote: >>>> >>>> >>>> I'm getting about a 4 - 9 ms. inaccuracy in the time the messages is >>>> sent >>>> and >>>> the time I want it to be sent - but I'd like it to be much more accurate >>>> (sub ms if possible) >>> >>> >>> >>> I would start by making measurements without the network component. When >>> you >>> would send the gen_udp message, you take a timestamp, and analyze the >>> skew >>> from the suggested skew. This lets you estimate the overhead of the rest >>> of >>> the system in isolation from your own code and the Erlang VM. >>> >>> Intuitively, 4 to 9 milliseconds is much higher than what I would expect. >>> But note that if you sleep for, say, 40ms, you will be awoken on the 41ms >>> flank at the earliest. This is because you are usually "inside" a >>> millisecond when you make the call so you start by taking the ceiling of >>> that milli-second before you. >>> >>> How much other work is your Erlang VM doing when you make these >>> measurements? You are saying between 4 to 9 ms, which is variance >>> suggesting >>> the VM has lots of work to do at that moment. And of course such stuff >>> will >>> affect the running time. You can switch priority of your processes up to >>> high, but this comes at the expense of other calculations if you can't >>> finish your work quickly in the process with high priority. >>> >>> >>> -- >>> J. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > From jean.parpaillon@REDACTED Mon Feb 22 10:59:40 2016 From: jean.parpaillon@REDACTED (Jean Parpaillon) Date: Mon, 22 Feb 2016 10:59:40 +0100 Subject: [erlang-questions] GSoC : D-Bus proposal Message-ID: <1456135180.2427.4.camel@free.fr> Dear all, I'm proposing several topics around erlang-dbus project for the 2016 GSoC: https://github.com/beamcommunity/beamcommunity.github.com/wiki/Project: -Erlang-DBus Don't hesitate to comment / propose new ideas.. The beamcommunity initiative is proposing topics on other erlang/OTP exciting projects. Please have a look and share ! https://github.com/beamcommunity/beamcommunity.github.com/wiki Regards, -- Jean Parpaillon -- Director @ OW2 Consortium OCCIware Strategic Orientation Committee Chairman Research Engineer @ Inria -- Phone: +33 6 30 10 92 86 im: jean.parpaillon@REDACTED skype: jean.parpaillon linkedin: http://www.linkedin.com/in/jeanparpaillon/en -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Feb 22 11:06:44 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 22 Feb 2016 13:06:44 +0300 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: References: <56CA3A38.4010404@yandex.ru> Message-ID: Your problem is very painful for us. There is such a task in video streaming: send udp packets with very precise timing. It is required for old hardware and non-IP media like air/cable streaming. As far as I remember, we could reach required precision for IP clients, but cannot achive microsecond precision for non-IP clients. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lemenkov@REDACTED Mon Feb 22 11:17:49 2016 From: lemenkov@REDACTED (Peter Lemenkov) Date: Mon, 22 Feb 2016 11:17:49 +0100 Subject: [erlang-questions] GSoC : D-Bus proposal In-Reply-To: <1456135180.2427.4.camel@free.fr> References: <1456135180.2427.4.camel@free.fr> Message-ID: Hello All! 2016-02-22 10:59 GMT+01:00 Jean Parpaillon : > Dear all, > I'm proposing several topics around erlang-dbus project for the 2016 GSoC: > https://github.com/beamcommunity/beamcommunity.github.com/wiki/Project:-Erlang-DBus > > Don't hesitate to comment / propose new ideas.. > The beamcommunity initiative is proposing topics on other erlang/OTP > exciting projects. Please have a look and share ! > https://github.com/beamcommunity/beamcommunity.github.com/wiki I believe it would be beneficial to implement a generic AF_UNIX interface module. There are multiple proposals and implementations, so there is certainly a demand for that. -- With best regards, Peter Lemenkov. From roberto@REDACTED Mon Feb 22 11:21:51 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 22 Feb 2016 11:21:51 +0100 Subject: [erlang-questions] Cowboy header parsing termination? Message-ID: Dear list, I keep on seeing the following in the logs: Ranch listener nucleo_listener terminated with reason: no case clause matching 123 in cowboy_protocol:parse_hd_name_ws/8 line 276 (code is here: https://github.com/ninenines/cowboy/blob/1.0.4/src/cowboy_protocol.erl#L276) Anyone using cowboy sees the same thing? I'm wondering if this is some kind of targeted attack on cowboy parsing. Thanks, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Mon Feb 22 11:18:18 2016 From: bourinov@REDACTED (Max Bourinov) Date: Mon, 22 Feb 2016 11:18:18 +0100 Subject: [erlang-questions] Assembling HTTP packets from raw TCP socket Message-ID: Hi guys, I need to assemble HTTP requests (GET, POST, etc) from RAW socket. What are easy options/libs? Best regards, Max -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Mon Feb 22 12:04:17 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 22 Feb 2016 12:04:17 +0100 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: References: <56CA3A38.4010404@yandex.ru> Message-ID: On Mon, Feb 22, 2016 at 8:30 AM, Joe Armstrong wrote: > Here's the slightly modified program. I *have* to nitpick this since I worry people will misuse the new 18.0 timing API. Changes to the program: * Use erlang:monotonic_time() which gives you native resolution by default according to the precision of your systems clock (older devices will give you micro_seconds, newer systems nano_seconds). Also, by using monotonic_time we avoid time warping, which you may get with system_time. * Request the timestamps in native format, operate on the native format and finally convert down to the desired output format. This avoids rounding errors. I get the same bad results on OSX, but not on Linux. It would be interesting to analyze why it takes so long to wake up on OSX after having slept for 1 second. It definitely looks like a problem with being woken up and having to do work different from the scheduler loop, but I have yet to investigate why that happens. -module(t). -compile(export_all). test() -> process_flag(priority, max), test(10, []). test(0, L) -> L; test(K, L) -> T1 = erlang:monotonic_time(), erlang:send_after(1000, self(), ping), receive ping -> T2 = erlang:monotonic_time(), test(K-1, [erlang:convert_time_unit(T2-T1, native, micro_seconds) | L ]) end. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From magnus@REDACTED Mon Feb 22 12:12:32 2016 From: magnus@REDACTED (Magnus Henoch) Date: Mon, 22 Feb 2016 11:12:32 +0000 Subject: [erlang-questions] SSL Example does not work In-Reply-To: <56C88DE6.9060708@gmx.net> References: <56C84A52.2030206@gmx.net> <56C86AF3.4090905@ninenines.eu> <56C88DE6.9060708@gmx.net> Message-ID: I just sent a pull request to fix the example: https://github.com/erlang/otp/pull/976 Regards, Magnus On Sat, Feb 20, 2016 at 4:01 PM, Roger Wenham wrote: > Thanks for all the replies!, this list is brilliant! > > I now completely understand. > > ssl:start() calls: > application:start(crypto), > application:start(asn1), > application:start(public_key), > application:start(ssl). > > The supplied ssl example program calls: > start() -> > %% Start ssl application > application:start(crypto), > application:start(public_key), > application:start(ssl), > .... > > So as Lo?c Hoguin correctly pointed out, asn1 is not being started, > causing the error... > > So the test program should be modified as either: > > start() -> > %% Start ssl application > application:start(crypto), > application:start(asn1), > application:start(public_key), > application:start(ssl), > .... > > or better: > > start() -> > %% Start ssl application > ssl:start(), > .... > > I have tested the first fix, and it works.... > > Do I need to raise a bug on this, as it is *very* confusing to have an > official example program not work? > > Thanks > > Roger > > On 20/02/16 14:32, Lo?c Hoguin wrote: > >> On 02/20/2016 12:13 PM, Roger Wenham wrote: >> >>> Hi Magnus, >>> >>> Thought of that, but the test program starts: >>> >>> start() -> >>> %% Start ssl application >>> >> >> It's missing >> application:start(asn1). >> >> application:start(crypto), >>> application:start(public_key), >>> application:start(ssl), >>> ... >>> >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Mon Feb 22 12:18:30 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Mon, 22 Feb 2016 12:18:30 +0100 Subject: [erlang-questions] Assembling HTTP packets from raw TCP socket In-Reply-To: References: Message-ID: You may consider cowlib: https://github.com/ninenines/cowlib Best, r. On Mon, Feb 22, 2016 at 11:18 AM, Max Bourinov wrote: > Hi guys, > > I need to assemble HTTP requests (GET, POST, etc) from RAW socket. > > What are easy options/libs? > > Best regards, > Max > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Mon Feb 22 12:51:46 2016 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 22 Feb 2016 12:51:46 +0100 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: References: <56CA3A38.4010404@yandex.ru> Message-ID: On Mon, Feb 22, 2016 at 12:04 PM, Jesper Louis Andersen wrote: > > On Mon, Feb 22, 2016 at 8:30 AM, Joe Armstrong wrote: >> >> Here's the slightly modified program. > > > I *have* to nitpick this since I worry people will misuse the new 18.0 > timing API. Changes to the program: > > * Use erlang:monotonic_time() which gives you native resolution by default > according to the precision of your systems clock (older devices will give > you micro_seconds, newer systems nano_seconds). Also, by using > monotonic_time we avoid time warping, which you may get with system_time. > > * Request the timestamps in native format, operate on the native format and > finally convert down to the desired output format. This avoids rounding > errors. > > I get the same bad results on OSX, but not on Linux. It would be interesting > to analyze why it takes so long to wake up on OSX after having slept for 1 > second. It definitely looks like a problem with being woken up and having to > do work different from the scheduler loop, but I have yet to investigate why > that happens. I get the same effect irrespective of the sleep interval if I sleep for 100 ms then the wake-up still can be still 6ms later than the requested time ... Is the timer code in linux and OS-X different, or do they call the same libraries and the libraries behave differently? I have no idea /Joe > > -module(t). > -compile(export_all). > > test() -> > process_flag(priority, max), > test(10, []). > > test(0, L) -> > L; > test(K, L) -> > T1 = erlang:monotonic_time(), > erlang:send_after(1000, self(), ping), > receive > ping -> > T2 = erlang:monotonic_time(), > test(K-1, [erlang:convert_time_unit(T2-T1, native, micro_seconds) | > L ]) > end. > > > > > -- > J. From Ola.Backstrom@REDACTED Mon Feb 22 13:16:22 2016 From: Ola.Backstrom@REDACTED (=?iso-8859-1?Q?Ola_B=E4ckstr=F6m?=) Date: Mon, 22 Feb 2016 12:16:22 +0000 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: References: <56CA3A38.4010404@yandex.ru> Message-ID: <8eea2d3933b84c35ae45e82883e873c2@Jar-EX01.ls.local> Do you get different result with power cable plugged in compared to not plugged in? As Jesper mentioned earlier does OS X have some power saving stuff, it is called "timer coalescing". See for instance http://arstechnica.com/apple/2013/06/how-os-x-mavericks-works-its-power-saving-magic/ There seems to be ways to disable it, http://www.timoliver.com.au/2014/01/25/disabling-timer-coalescing-in-os-x-mavericks/ , but that is not something I've tried. I thought this was "os X"-only stuff, but now when I google I see that linux seems to have coalescing since many years too. If it is in general enabled I don't know. /Ola -----Original Message----- From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Joe Armstrong Sent: den 22 februari 2016 12:52 To: Jesper Louis Andersen Cc: Erlang Subject: Re: [erlang-questions] Sending message at a specific and accurate time On Mon, Feb 22, 2016 at 12:04 PM, Jesper Louis Andersen wrote: > > On Mon, Feb 22, 2016 at 8:30 AM, Joe Armstrong wrote: >> >> Here's the slightly modified program. > > > I *have* to nitpick this since I worry people will misuse the new 18.0 > timing API. Changes to the program: > > * Use erlang:monotonic_time() which gives you native resolution by > default according to the precision of your systems clock (older > devices will give you micro_seconds, newer systems nano_seconds). > Also, by using monotonic_time we avoid time warping, which you may get with system_time. > > * Request the timestamps in native format, operate on the native > format and finally convert down to the desired output format. This > avoids rounding errors. > > I get the same bad results on OSX, but not on Linux. It would be > interesting to analyze why it takes so long to wake up on OSX after > having slept for 1 second. It definitely looks like a problem with > being woken up and having to do work different from the scheduler > loop, but I have yet to investigate why that happens. I get the same effect irrespective of the sleep interval if I sleep for 100 ms then the wake-up still can be still 6ms later than the requested time ... Is the timer code in linux and OS-X different, or do they call the same libraries and the libraries behave differently? I have no idea /Joe > > -module(t). > -compile(export_all). > > test() -> > process_flag(priority, max), > test(10, []). > > test(0, L) -> > L; > test(K, L) -> > T1 = erlang:monotonic_time(), > erlang:send_after(1000, self(), ping), > receive > ping -> > T2 = erlang:monotonic_time(), > test(K-1, [erlang:convert_time_unit(T2-T1, native, > micro_seconds) | L ]) > end. > > > > > -- > J. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From jesper.louis.andersen@REDACTED Mon Feb 22 13:16:53 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 22 Feb 2016 13:16:53 +0100 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: References: <56CA3A38.4010404@yandex.ru> Message-ID: On Mon, Feb 22, 2016 at 12:51 PM, Joe Armstrong wrote: > I get the same effect irrespective of the sleep interval if I sleep for > 100 ms > then the wake-up still can be still 6ms later than the requested time ... > Sleeping for 100ms is ages in modern computers, so it would not surprise me one bit that you ended up going to sleep to save power in that time frame. The Erlang VM has a compatibility layer plugging into the underlying operating system. You would be surprised over how different they are and what they support at the lowest level. And what the "correct means of operation" is on system A might be a really bad idea on system B. The right way to figure out what is happening is dtrace(1) and plugging into the timer wheel to figure out what is happening at that level. Either you are getting the message too late from the kernel, or you end up doing something weird inside Erlang as a VM. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Mon Feb 22 13:19:04 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 22 Feb 2016 13:19:04 +0100 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: <8eea2d3933b84c35ae45e82883e873c2@Jar-EX01.ls.local> References: <56CA3A38.4010404@yandex.ru> <8eea2d3933b84c35ae45e82883e873c2@Jar-EX01.ls.local> Message-ID: On Mon, Feb 22, 2016 at 1:16 PM, Ola B?ckstr?m wrote: > Do you get different result with power cable plugged in compared to not > plugged in? I controlled for that by trying it with and without the cable plugged. It isn't affecting the latencies. Any kind of dynamic timing system shouldn't really affect this either. If anything it ought to improve the precision. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Mon Feb 22 13:33:37 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Mon, 22 Feb 2016 13:33:37 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> Message-ID: <56CB0021.9050901@ninenines.eu> On 02/22/2016 03:28 AM, Steve Vinoski wrote: > > > On Sun, Feb 21, 2016 at 8:38 PM, Lo?c Hoguin > wrote: > > Regardless of the technical merits of one or the other solution, > when developing software it's generally a good idea to listen to > what you users want, and this Nth topic about -module should be a > clear indication that the current situation is not what many users want. > > It doesn't matter if you think you are technically correct, because > you defend a point of view that many people do not care about. All > solutions have their strong and weak points, and enough people are > annoyed by the current solution's weak points (the mandatory -module > attribute) to create topics every couple months. People tend to want > different things, so when so many people agree on something it's > time to take notice. This happens a lot less than you'd think. > > > I don't recall seeing this topic come up "every couple months". Can you > provide pointers to the various threads where this has come up so often, > other than this one, to back up this claim? Google searches of > erlang.org and searches through stackoverflow.com > don't turn up much at all on this topic. I guess it bubbles up to the ML less often than I thought. I still run into people having issues with it every couple months. I never know what to answer when they ask why it's this way, it just feels unnecessary to me. > In general it's also a good idea to remove as many steps as possible > to get code up and running, if only to give the illusion of Erlang > being a productive language, as this attracts newcomers much more > easily. > > > Over the years I've heard reasons for people avoiding Erlang, but I > personally have never even once heard someone say they didn't want to > use Erlang because of the burdensome -module attribute. People don't come to Erlang in droves because Erlang does not match their identity, ie it's not C/Java, Python or Ruby and does not have this familiar feeling. That's the *only reason*. Any other reasons given is just rationalization after the facts. The people who do come to Erlang have to jump a lot of barriers to get comfortable with it. Common issues involve unnecessary steps (this is where -module comes in) and missing functions for simple tasks (when this comes up this involves strings or binary strings). Some people can jump all barriers no problem, others eventually fall and give up after falling a few times. Removing barriers help. To give an example, PHP has so few barriers *to start* that even non-programmers can easily start doing things with it. They write a few lines, load up their browser and it prints things. The equivalent in Erlang involves a PhD... Of course PHP has a lot more barriers later on so I am not saying Erlang should be more like PHP, I am however saying that Erlang should be as easy to get started with as possible. You're going to suggest that I dedicate time to improve these things but I already do and my time is not infinite. Making the .app.src optional in Erlang.mk was the largest improvement. Before, people either forgot to list their apps to the 'applications' list, or made mistakes when adding them. Now they don't need to and no mistakes can be made. I have a great deal of similar simplifications coming up in Cowboy 2, again to make things easier and avoid mistakes (removing barriers). > Erlang is the most productive language I've seen (in large parts due > to readability and low maintenance) but it's doing a terrible job > showing it. > > We often say Erlang makes hard things easy and easy things hard. > Perhaps it's time to seriously consider making easy things easier. > This is the best way to get more people in. But I get the feeling > not everyone wants that, considering how little effort is spent on > that end (maps happened, but it seemed more like a miracle than > anything). > > > IMO that statement is rather insulting to those who've put a lot of time > and effort into designing and implementing maps. It's not at all a > simple problem. I really doubt Bjorn-Egil will be offended considering the discussions I had with him. I still maintain it's a miracle, and will add that he is the savior who made the miracle possible. I doubt we would have maps now if not for him. He knows how grateful I am for all his work. > If having an optional -module attribute is that important to you, write > a pull request. There's not even the slightest hint that it would get accepted. My time not being infinite, I better spend it elsewhere for now. It's just one of many barriers. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From mononcqc@REDACTED Mon Feb 22 14:57:34 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 22 Feb 2016 08:57:34 -0500 Subject: [erlang-questions] Cowboy header parsing termination? In-Reply-To: References: Message-ID: <20160222135733.GB1627@fhebert-ltm2.internal.salesforce.com> On 02/22, Roberto Ostinelli wrote: >Dear list, >I keep on seeing the following in the logs: > >Ranch listener nucleo_listener terminated with reason: no case clause >matching 123 in cowboy_protocol:parse_hd_name_ws/8 line 276 > >(code is here: >https://github.com/ninenines/cowboy/blob/1.0.4/src/cowboy_protocol.erl#L276) > >Anyone using cowboy sees the same thing? I'm wondering if this is some kind >of targeted attack on cowboy parsing. > 123 is the ASCII code for '{'. What you see here is a header fabricated to look maybe a bit like this: My-Header-Name {whatever goes in here} This is not valid content. When cowboy sees the space after the header name, it expects to see more space or a colon, possibly because the header should look like: My-Header-Name : {whatever goes in here} But that colon is missing and all you get is that bracket right there. It would be easy to expect it to just be garbage over the line or incomplete stuff than a direct attack on cowboy itself aas far as I can tell. There's not too much to be found by just sending requests that end early like that, unless someone is doing some form of fuzzing, in which case you should find a lot more varied error logs along with this. The trick would be to look at the content that was sent over the line before and after that point. One possibility could be that some proxy or intermediary (I don't think cowboy itself is sensitive to that) could be hit by request smuggling: http://www.cgisecurity.com/lib/HTTP-Request-Smuggling.pdf But only contextual information could help reveal that. Regards, Fred. From tony@REDACTED Mon Feb 22 15:31:44 2016 From: tony@REDACTED (Tony Rogvall) Date: Mon, 22 Feb 2016 15:31:44 +0100 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: References: <56CA3A38.4010404@yandex.ru> <8eea2d3933b84c35ae45e82883e873c2@Jar-EX01.ls.local> Message-ID: > L = fun Loop() -> receive after 1 -> Loop() end end. > spawn_link(L). That is a process doing minimal sleep, but still sleeps ( I think ) Adding this little ?helper? process changes the timing on my mac to the better. From 3-4 ms latency to to around 1 or less. Power cable did not affect the latency for me either. /Tony > On 22 feb 2016, at 13:19, Jesper Louis Andersen wrote: > > > On Mon, Feb 22, 2016 at 1:16 PM, Ola B?ckstr?m wrote: > Do you get different result with power cable plugged in compared to not plugged in? > > I controlled for that by trying it with and without the cable plugged. It isn't affecting the latencies. > > Any kind of dynamic timing system shouldn't really affect this either. If anything it ought to improve the precision. > > > -- > J. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From erlang@REDACTED Mon Feb 22 15:55:39 2016 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 22 Feb 2016 15:55:39 +0100 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: References: <56CA3A38.4010404@yandex.ru> <8eea2d3933b84c35ae45e82883e873c2@Jar-EX01.ls.local> Message-ID: On Mon, Feb 22, 2016 at 3:31 PM, Tony Rogvall wrote: >> L = fun Loop() -> receive after 1 -> Loop() end end. >> spawn_link(L). Busy sleep :-) < I thought busy waiting was frowned upon> Actually after 10 works as well. So a work-around is to set the timer 10 ms *before* I want and run a busy sleep loop - So I keep having to thumping the damn thing to make it doesn't fall asleep. I've only tested on an unloaded system - perhaps if the system was loaded the timings would be better? Thanks for the tip. /Joe > > That is a process doing minimal sleep, but still sleeps ( I think ) > > Adding this little ?helper? process changes the timing on my mac to the better. > > From 3-4 ms latency to to around 1 or less. > > Power cable did not affect the latency for me either. > > /Tony > >> On 22 feb 2016, at 13:19, Jesper Louis Andersen wrote: >> >> >> On Mon, Feb 22, 2016 at 1:16 PM, Ola B?ckstr?m wrote: >> Do you get different result with power cable plugged in compared to not plugged in? >> >> I controlled for that by trying it with and without the cable plugged. It isn't affecting the latencies. >> >> Any kind of dynamic timing system shouldn't really affect this either. If anything it ought to improve the precision. >> >> >> -- >> J. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From rickard@REDACTED Mon Feb 22 16:55:57 2016 From: rickard@REDACTED (Rickard Green) Date: Mon, 22 Feb 2016 16:55:57 +0100 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: References: <56CA3A38.4010404@yandex.ru> Message-ID: > I can only conclude that send_after(Time, Pid, Msg) is deeply broken on the > mac - if inter-processing scheduling and messaging had 6 millisecond > delays Erlang would be unusable. > When schedulers run out of work they eventually need to sleep. A timeout is set up in order to service next timeout. On macosx we will sleep in select(). The timeout is set on microsecond level, but on macosx the thread is often woken milliseconds after the time we requested. When the thread wakeup is delayed like this, timers handled by the thread will also be delayed. I verified the delayed timeout in select() using a simple c-program, and the figures corresponds to what can be seen in your send_after() test. That is, it is not the timer implementation in the vm that delays the timers. > I guess I could do a send_after 10ms before I want and then do a busy wait > but this would be crazy. > If the primitive used for waiting has this kind of delay, a busy wait is more or less what is left. The vm could of course perform this busy wait for you, but my guess is that this busy wait wouldn't be received well by the users. It is however an option. Regards, Rickard -- Rickard Green, Erlang/OTP, Ericsson AB From erlang@REDACTED Mon Feb 22 17:58:02 2016 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 22 Feb 2016 17:58:02 +0100 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: References: <56CA3A38.4010404@yandex.ru> Message-ID: On Mon, Feb 22, 2016 at 4:55 PM, Rickard Green wrote: >> I can only conclude that send_after(Time, Pid, Msg) is deeply broken on the >> mac - if inter-processing scheduling and messaging had 6 millisecond >> delays Erlang would be unusable. >> > > When schedulers run out of work they eventually need to sleep. A > timeout is set up in order to service next timeout. On macosx we will > sleep in select(). > The timeout is set on microsecond level, but on > macosx the thread is often woken milliseconds after the time we > requested. UUgh -- that's horrible - seems daft setting a microsecond timer if it refuses to obey - ( There might be a crazy work around - on the mac core audio can send midi events at very precise times - so if you subscribe to these you'll get a callback up at a very precise time - so you could use the midi scheduler to schedule future events. (Actually I ran into this problem when generating audio events, so it's not surprising) Midi is essentially "play this note" at time , which is not that different from "send this message" at time - and core audio supports this at a very low OS level. The audio timers are designed to take priority over other timers and are be very accurate ... A quick google search turned up several posts that pursued this idea /Joe > When the thread wakeup is delayed like this, timers handled > by the thread will also be delayed. I verified the delayed timeout in > select() using a simple c-program, and the figures corresponds to what > can be seen in your send_after() test. That is, it is not the timer > implementation in the vm that delays the timers. > >> I guess I could do a send_after 10ms before I want and then do a busy wait >> but this would be crazy. >> > > If the primitive used for waiting has this kind of delay, a busy wait > is more or less what is left. The vm could of course perform this busy > wait for you, but my guess is that this busy wait wouldn't be received > well by the users. It is however an option. > > Regards, > Rickard > -- > Rickard Green, Erlang/OTP, Ericsson AB From tlegant@REDACTED Mon Feb 22 18:14:11 2016 From: tlegant@REDACTED (Timothy Legant) Date: Mon, 22 Feb 2016 11:14:11 -0600 Subject: [erlang-questions] Strange BEAM slowdown Message-ID: <86twl0hdlo.fsf@sintara.local.catseye.net> Hello, We have an application where we read a huge volume of small messages from ZMQ sockets and distribute them to Erlang processes. We are seeing strange behavior where, after a short while, beam.smp's load drops quite a bit and then the data begins queuing, eating memory until we either stop the program or the Linux OOM killer does it for us. DETAILS ------- CentOS release 6.6 (Final) Erlang/OTP 17 [erts-6.4] [source-2e19e2f] [64-bit] [smp:56:56] [async-threads:20] [hipe] [kernel-poll:true] beam.smp is started with the flags: +sbt db +sub true We have 60+ data sources (TCP/ZMQ sockets), each of which feeds an independent set of processes; there is no interaction between the processes handling the data from one socket and the processes handling data from other sockets. Our first implementation used the erlzmq2 library to read the socket. We then parsed the messages in Erlang and sent Erlang terms to the data handling processes. After seeing the problem behavior we suspected that the repeated calls to erlzmq:recv() and parsing in Erlang might be the cause of the backup so we rewrote that code as a NIF (background thread + several API calls). Our NIF implementation reads the ZMQ socket, parses the data and then sends it to the data handling processes. We (obviously, I suppose) create one of these background threads for each of the 60+ data source sockets. Despite the entirely different implementation of ZMQ handling, parsing and dispatch of the data, we are seeing the same issue: first the load drops off precipitously and then the data starts queuing in the ZMQ socket buffers and the program is unusable. We are curious if anyone has seen this sort of behavior with BEAM or might have suggestions on where to look for the issue. Thanks, Tim From dmkolesnikov@REDACTED Mon Feb 22 19:28:34 2016 From: dmkolesnikov@REDACTED (dmkolesnikov@REDACTED) Date: Mon, 22 Feb 2016 20:28:34 +0200 Subject: [erlang-questions] Strange BEAM slowdown In-Reply-To: <86twl0hdlo.fsf@sintara.local.catseye.net> References: <86twl0hdlo.fsf@sintara.local.catseye.net> Message-ID: <7E2FEC4F-67F1-4590-8296-93BE4318A437@gmail.com> Hello, I've seen a similar behavior when message processing rate was slower then message arrival rate. I would use etop to check if some process has significant mailbox size or huge reductions; use recon tool to check for binary leaked. There is very good book by Fried, it might shed some light on your problem https://s3.amazonaws.com/erlang-in-anger/text.v1.0.2.pdf Best Regards, Dmitry Sent from my iPhone > On 22 Feb 2016, at 19:14, Timothy Legant wrote: > > Hello, > > We have an application where we read a huge volume of small messages > from ZMQ sockets and distribute them to Erlang processes. We are > seeing strange behavior where, after a short while, beam.smp's load > drops quite a bit and then the data begins queuing, eating memory > until we either stop the program or the Linux OOM killer does it for > us. > > DETAILS > ------- > CentOS release 6.6 (Final) > Erlang/OTP 17 [erts-6.4] [source-2e19e2f] [64-bit] [smp:56:56] [async-threads:20] [hipe] [kernel-poll:true] > > beam.smp is started with the flags: +sbt db +sub true > > We have 60+ data sources (TCP/ZMQ sockets), each of which feeds an > independent set of processes; there is no interaction between the > processes handling the data from one socket and the processes handling > data from other sockets. > > Our first implementation used the erlzmq2 library to read the socket. > We then parsed the messages in Erlang and sent Erlang terms to the > data handling processes. > > After seeing the problem behavior we suspected that the repeated calls > to erlzmq:recv() and parsing in Erlang might be the cause of the > backup so we rewrote that code as a NIF (background thread + several > API calls). Our NIF implementation reads the ZMQ socket, parses the > data and then sends it to the data handling processes. We (obviously, > I suppose) create one of these background threads for each of the 60+ > data source sockets. > > Despite the entirely different implementation of ZMQ handling, parsing > and dispatch of the data, we are seeing the same issue: first the load > drops off precipitously and then the data starts queuing in the ZMQ > socket buffers and the program is unusable. > > > We are curious if anyone has seen this sort of behavior with BEAM or > might have suggestions on where to look for the issue. > > > Thanks, > > Tim > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From rvirding@REDACTED Mon Feb 22 21:27:35 2016 From: rvirding@REDACTED (Robert Virding) Date: Mon, 22 Feb 2016 21:27:35 +0100 Subject: [erlang-questions] Erlounge in Barcelona In-Reply-To: References: Message-ID: Any suggestions to where we can meet on Wednesday? Robert On 21 February 2016 at 21:10, Robert Virding wrote: > OK, the my suggestion is that we meet on Wednesday evening for an > erlounge. Where I don't know. Anyone have a suggestion? > > Robert > > > On 21 February 2016 at 13:08, Vance Shipley wrote: > >> On Sun, Feb 21, 2016 at 5:29 PM, Robert Virding >> wrote: >> > I would prefer Monday - Wednesday. Vance are you here on Wednesday >> evening? >> >> Yes, Wednesday evening is fine for me, I leave Thursday morning. >> >> -- >> -Vance >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Mon Feb 22 22:00:49 2016 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 22 Feb 2016 13:00:49 -0800 Subject: [erlang-questions] Strange BEAM slowdown In-Reply-To: <86twl0hdlo.fsf@sintara.local.catseye.net> References: <86twl0hdlo.fsf@sintara.local.catseye.net> Message-ID: <56CB7701.8020705@gmail.com> If this was related to scheduler threads locking up, due to spending too much time (more than 1-2ms roughly) in the erlzmq2 NIF, when the erlzmq:recv function is called, you can change the ZeroMQ connections you create to receive with active mode instead of passive, to receive the messages in the Erlang process without the call to erlzmq:recv. I don't quite understand the need to rewrite the NIF, since it already is using a background thread for the receive (at https://github.com/zeromq/erlzmq2). An example of using the active mode for recv is at https://github.com/CloudI/cloudi_service_zeromq/blob/master/src/cloudi_service_zeromq.erl . On 02/22/2016 09:14 AM, Timothy Legant wrote: > Hello, > > We have an application where we read a huge volume of small messages > from ZMQ sockets and distribute them to Erlang processes. We are > seeing strange behavior where, after a short while, beam.smp's load > drops quite a bit and then the data begins queuing, eating memory > until we either stop the program or the Linux OOM killer does it for > us. > > DETAILS > ------- > CentOS release 6.6 (Final) > Erlang/OTP 17 [erts-6.4] [source-2e19e2f] [64-bit] [smp:56:56] [async-threads:20] [hipe] [kernel-poll:true] > > beam.smp is started with the flags: +sbt db +sub true > > We have 60+ data sources (TCP/ZMQ sockets), each of which feeds an > independent set of processes; there is no interaction between the > processes handling the data from one socket and the processes handling > data from other sockets. > > Our first implementation used the erlzmq2 library to read the socket. > We then parsed the messages in Erlang and sent Erlang terms to the > data handling processes. > > After seeing the problem behavior we suspected that the repeated calls > to erlzmq:recv() and parsing in Erlang might be the cause of the > backup so we rewrote that code as a NIF (background thread + several > API calls). Our NIF implementation reads the ZMQ socket, parses the > data and then sends it to the data handling processes. We (obviously, > I suppose) create one of these background threads for each of the 60+ > data source sockets. > > Despite the entirely different implementation of ZMQ handling, parsing > and dispatch of the data, we are seeing the same issue: first the load > drops off precipitously and then the data starts queuing in the ZMQ > socket buffers and the program is unusable. > > > We are curious if anyone has seen this sort of behavior with BEAM or > might have suggestions on where to look for the issue. > > > Thanks, > > Tim > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Mon Feb 22 22:26:51 2016 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 22 Feb 2016 22:26:51 +0100 Subject: [erlang-questions] Announce; Erlang midi on mac Message-ID: I've got erlang working with garage band - so for you mac fans who like music you can use garage band as a back-end to play midi sequences Note: Erlang is handling all the timing - so now you can hear errors in timing and GC delays :-) https://github.com/joearms/music_experiments/tree/master/midi_mac_driver There is no documentation (yet) - blogs will follow Have fun /Joe From carlosj.gf@REDACTED Mon Feb 22 23:25:50 2016 From: carlosj.gf@REDACTED (=?UTF-8?Q?Carlos_Gonz=C3=A1lez_Florido?=) Date: Mon, 22 Feb 2016 23:25:50 +0100 Subject: [erlang-questions] Erlounge in Barcelona In-Reply-To: References: Message-ID: I'm afraid I don't know Barcelona very well... I can ask some friends tomorrow. On Mon, Feb 22, 2016 at 9:27 PM, Robert Virding wrote: > Any suggestions to where we can meet on Wednesday? > > Robert > > > On 21 February 2016 at 21:10, Robert Virding wrote: > >> OK, the my suggestion is that we meet on Wednesday evening for an >> erlounge. Where I don't know. Anyone have a suggestion? >> >> Robert >> >> >> On 21 February 2016 at 13:08, Vance Shipley wrote: >> >>> On Sun, Feb 21, 2016 at 5:29 PM, Robert Virding >>> wrote: >>> > I would prefer Monday - Wednesday. Vance are you here on Wednesday >>> evening? >>> >>> Yes, Wednesday evening is fine for me, I leave Thursday morning. >>> >>> -- >>> -Vance >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Mon Feb 22 23:30:23 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 23 Feb 2016 11:30:23 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> Message-ID: <56CB8BFF.2020802@cs.otago.ac.nz> I do not recall ever seeing -module coming up here before. I certainly don't recall a long thread like this one. I believe I have pointed out how utterly trivial it is to create the -module directive and how valuable I have found it in detecting file-system-created snafus. #!/bin/sh #Usage: edit module case "$1" in */*) echo "Module name may not contain /" >/dev/stderr; exit 1 ;; -*) echo "Module name may not start with -">/dev/stderr; exit 1 ;; *\'*) echo "Module name may not contain '" >/dev/stderr; exit 1 ;; *\\*) echo "Module name may not contain \\" >/dev/stderr; exit 1 ;; *) ;; esac file="$1.erl" # Beware: file system may mangle name! if [ ! -e "$file" ]; then date=`date +%Y-%m-%d` case `expr "$1" : '^[a-z][a-zA-Z0-9_]*$'` in 0) module="'$1'" ;; *) module="$1" ;; esac echo "%%% File : $file" >"$file" echo "%%% Author : $LOGNAME" >>"$file" echo "%%% Created: $date" >>"$file" echo "%%% Purpose: " >>"$file" echo "" >>"$file" echo "-module($module)." >>"$file" echo "" >>"$file" echo "-export([" >>"$file" echo " ])." >>"$file" echo "" >>"$file" fi exec ${EDITOR:-emacs} "$file" This isn't completely bullet-proof, but it's close enough to be useful. The program I actually use fills in the full name from the pw_gecos field in the struct passwd entry for $LOGNAME and adds a copyright notice if you want one. So here's how much burden -module adds to this programmer: instead of % emacs foobar.erl I have to type % edit foobar Hang on, that's LESS work. Hmmm. From ok@REDACTED Tue Feb 23 00:12:33 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 23 Feb 2016 12:12:33 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <20160222030407.GB53561@ferdmbp.local> Message-ID: <56CB95E1.4040309@cs.otago.ac.nz> The little AWK program is indeed neat, but it goes in the broken direction (from file name to module name) not the working direction (from module name to file name). This is the whole problem: given a file name, we cannot cross-platform reliably reconstruct a module name. From rvirding@REDACTED Tue Feb 23 11:08:57 2016 From: rvirding@REDACTED (Robert Virding) Date: Tue, 23 Feb 2016 11:08:57 +0100 Subject: [erlang-questions] Erlounge in Barcelona In-Reply-To: References: Message-ID: I have an offer and suggestion: SeQura, @*sequra_es* . 5th floor (no name on the door). Carrer d'Arag?, 383, 5? at 19 in the evening, or 7pm for any transatlantic guests. All welcome, Robert On 22 February 2016 at 23:25, Carlos Gonz?lez Florido wrote: > I'm afraid I don't know Barcelona very well... I can ask some friends > tomorrow. > > On Mon, Feb 22, 2016 at 9:27 PM, Robert Virding > wrote: > >> Any suggestions to where we can meet on Wednesday? >> >> Robert >> >> >> On 21 February 2016 at 21:10, Robert Virding wrote: >> >>> OK, the my suggestion is that we meet on Wednesday evening for an >>> erlounge. Where I don't know. Anyone have a suggestion? >>> >>> Robert >>> >>> >>> On 21 February 2016 at 13:08, Vance Shipley wrote: >>> >>>> On Sun, Feb 21, 2016 at 5:29 PM, Robert Virding >>>> wrote: >>>> > I would prefer Monday - Wednesday. Vance are you here on Wednesday >>>> evening? >>>> >>>> Yes, Wednesday evening is fine for me, I leave Thursday morning. >>>> >>>> -- >>>> -Vance >>>> >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Tue Feb 23 11:16:48 2016 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 23 Feb 2016 11:16:48 +0100 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: References: <56CA3A38.4010404@yandex.ru> Message-ID: On Mon, Feb 22, 2016 at 4:55 PM, Rickard Green wrote: >> I can only conclude that send_after(Time, Pid, Msg) is deeply broken on the >> mac - if inter-processing scheduling and messaging had 6 millisecond >> delays Erlang would be unusable. >> > > When schedulers run out of work they eventually need to sleep. A > timeout is set up in order to service next timeout. On macosx we will > sleep in select(). The timeout is set on microsecond level, but on > macosx the thread is often woken milliseconds after the time we > requested. When the thread wakeup is delayed like this, timers handled > by the thread will also be delayed. I verified the delayed timeout in > select() using a simple c-program, and the figures corresponds to what > can be seen in your send_after() test. That is, it is not the timer > implementation in the vm that delays the timers. > >> I guess I could do a send_after 10ms before I want and then do a busy wait >> but this would be crazy. >> > > If the primitive used for waiting has this kind of delay, a busy wait > is more or less what is left. The vm could of course perform this busy > wait for you, but my guess is that this busy wait wouldn't be received > well by the users. It is however an option. You'd need two primitives send_after (as today) and send_after_precise(Effort, Time, ...) where Effort is high, low, ... (or something) Actually I think this is a key primitive in Erlang - far more important than maps and module names - the "essence" of Erlang has to do with messaging - this is where Erlang should (and does) excel - the rest (maps, etc.) is just syntax. Sending a message at a precise time is an underlying mechanism upon which a large number of things can be built. Why is Erlang great? - because you can have thousands of parallel processes and send and receive message quickly. The Zen of erlang involves fast process spawning, fast message sending and receiving and control over time. Personally I would put a lot of effort into getting this right - Remember it's all about messaging, all the rest is syntax Cheers /Joe > > Regards, > Rickard > -- > Rickard Green, Erlang/OTP, Ericsson AB From lukas@REDACTED Tue Feb 23 11:18:29 2016 From: lukas@REDACTED (Lukas Larsson) Date: Tue, 23 Feb 2016 11:18:29 +0100 Subject: [erlang-questions] Strange BEAM slowdown In-Reply-To: <86twl0hdlo.fsf@sintara.local.catseye.net> References: <86twl0hdlo.fsf@sintara.local.catseye.net> Message-ID: Hello Timothy, On Mon, Feb 22, 2016 at 6:14 PM, Timothy Legant wrote: > Hello, > > We have an application where we read a huge volume of small messages > from ZMQ sockets and distribute them to Erlang processes. We are > seeing strange behavior where, after a short while, beam.smp's load > drops quite a bit and then the data begins queuing, eating memory > until we either stop the program or the Linux OOM killer does it for > us. > When you say load, what do you mean? Scheduler utilization? Average CPU utilization? Requests per second? Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Tue Feb 23 11:21:33 2016 From: vances@REDACTED (Vance Shipley) Date: Tue, 23 Feb 2016 15:51:33 +0530 Subject: [erlang-questions] Erlounge in Barcelona In-Reply-To: References: Message-ID: Works for me. On Feb 23, 2016 11:09 AM, "Robert Virding" wrote: > I have an offer and suggestion: > > SeQura, @*sequra_es* . 5th floor (no name > on the door). Carrer d'Arag?, 383, 5? > > at 19 in the evening, or 7pm for any transatlantic guests. > > All welcome, > Robert > > On 22 February 2016 at 23:25, Carlos Gonz?lez Florido < > carlosj.gf@REDACTED> wrote: > >> I'm afraid I don't know Barcelona very well... I can ask some friends >> tomorrow. >> >> On Mon, Feb 22, 2016 at 9:27 PM, Robert Virding >> wrote: >> >>> Any suggestions to where we can meet on Wednesday? >>> >>> Robert >>> >>> >>> On 21 February 2016 at 21:10, Robert Virding wrote: >>> >>>> OK, the my suggestion is that we meet on Wednesday evening for an >>>> erlounge. Where I don't know. Anyone have a suggestion? >>>> >>>> Robert >>>> >>>> >>>> On 21 February 2016 at 13:08, Vance Shipley wrote: >>>> >>>>> On Sun, Feb 21, 2016 at 5:29 PM, Robert Virding >>>>> wrote: >>>>> > I would prefer Monday - Wednesday. Vance are you here on Wednesday >>>>> evening? >>>>> >>>>> Yes, Wednesday evening is fine for me, I leave Thursday morning. >>>>> >>>>> -- >>>>> -Vance >>>>> >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From henrik.x.nord@REDACTED Tue Feb 23 12:35:29 2016 From: henrik.x.nord@REDACTED (Henrik Nord X) Date: Tue, 23 Feb 2016 12:35:29 +0100 Subject: [erlang-questions] Patch package OTP 18.2.4 released Message-ID: <56CC4401.3040802@ericsson.com> Patch Package: OTP 18.2.4 Git Tag: OTP-18.2.4 Date: 2016-02-23 Trouble Report Id: OTP-13323 Seq num: System: OTP Release: 18 Application: common_test-1.11.2 Predecessor: OTP 18.2.3 Check out the git tag OTP-18.2.4, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- common_test-1.11.2 ---------------------------------------------- --------------------------------------------------------------------- The common_test-1.11.2 application can be applied independently of other applications on a full OTP 18 installation. --- Fixed Bugs and Malfunctions --- OTP-13323 Application(s): common_test If a ssh package contained more than one netconf end tag, then the second end tag was never detected in ct_netconfc:handle_data. Instead it was included in the XML data given to the xmerl parser, which then failed. The problem was introduced by OTP-13007, and has now been corrected. Full runtime dependencies of common_test-1.11.2: compiler-6.0, crypto-3.6, debugger-4.1, erts-7.0, inets-6.0, kernel-4.0, observer-2.1, runtime_tools-1.8.16, sasl-2.4.2, snmp-5.1.2, ssh-4.0, stdlib-2.5, test_server-3.9, tools-2.8, xmerl-1.3.8 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From ml-erlang@REDACTED Tue Feb 23 13:38:23 2016 From: ml-erlang@REDACTED (Matthias Rieber) Date: Tue, 23 Feb 2016 13:38:23 +0100 Subject: [erlang-questions] =?utf-8?q?mnesia=5Fext_+leveldb?= Message-ID: <6672878f38887d1b733a769e6e456347@ssl.scheff32.de> Hello, is leveldb for mnesia_ext already open-sourced? I haven't found it on https://github.com/klarna Matthias From tony@REDACTED Tue Feb 23 14:29:35 2016 From: tony@REDACTED (Tony Rogvall) Date: Tue, 23 Feb 2016 14:29:35 +0100 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: References: <56CA3A38.4010404@yandex.ru> <8eea2d3933b84c35ae45e82883e873c2@Jar-EX01.ls.local> Message-ID: > On 22 feb 2016, at 15:55, Joe Armstrong wrote: > > On Mon, Feb 22, 2016 at 3:31 PM, Tony Rogvall wrote: >>> L = fun Loop() -> receive after 1 -> Loop() end end. >>> spawn_link(L). > > Busy sleep :-) < I thought busy waiting was frowned upon> > > Actually after 10 works as well. > When running a busy sleep loop of 1 ms I get a 990 us diff from the actual time and when running a busy loop of 10 ms I get 1550 us diff. BTW The same is true for smp disable Question for Richard really is why ? :-) Is 10 ms to small amount of time to allow the schedulers to sleep? The cpu load for a busy loop 10 ms is about 2% and busy loop 1 ms is 8% This sound a lot for doing nothing, but maybe not that much for doing something important at the right time now and then? The problem might go away if running this on a slightly loaded node on a multi core machine. /Tony > So a work-around is to set the timer 10 ms *before* I want and run a busy sleep > loop - > > So I keep having to thumping the damn thing to make it doesn't fall asleep. > > I've only tested on an unloaded system - perhaps if the system was loaded > the timings would be better? > > Thanks for the tip. > > /Joe > >> >> That is a process doing minimal sleep, but still sleeps ( I think ) >> >> Adding this little ?helper? process changes the timing on my mac to the better. >> >> From 3-4 ms latency to to around 1 or less. >> >> Power cable did not affect the latency for me either. >> >> /Tony >> >>> On 22 feb 2016, at 13:19, Jesper Louis Andersen wrote: >>> >>> >>> On Mon, Feb 22, 2016 at 1:16 PM, Ola B?ckstr?m wrote: >>> Do you get different result with power cable plugged in compared to not plugged in? >>> >>> I controlled for that by trying it with and without the cable plugged. It isn't affecting the latencies. >>> >>> Any kind of dynamic timing system shouldn't really affect this either. If anything it ought to improve the precision. >>> >>> >>> -- >>> J. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From rickard@REDACTED Tue Feb 23 15:58:16 2016 From: rickard@REDACTED (Rickard Green) Date: Tue, 23 Feb 2016 15:58:16 +0100 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: <56CC72D7.900@erlang.org> References: <56CC72D7.900@erlang.org> Message-ID: <56CC7388.2090103@erlang.org> Missed reply all... -------- Forwarded Message -------- Subject: Re: [erlang-questions] Sending message at a specific and accurate time Date: Tue, 23 Feb 2016 15:55:19 +0100 From: Rickard Green Organization: Ericsson AB To: Tony Rogvall On 02/23/2016 02:29 PM, Tony Rogvall wrote: > >> On 22 feb 2016, at 15:55, Joe Armstrong wrote: >> >> On Mon, Feb 22, 2016 at 3:31 PM, Tony Rogvall wrote: >>>> L = fun Loop() -> receive after 1 -> Loop() end end. >>>> spawn_link(L). >> >> Busy sleep :-) < I thought busy waiting was frowned upon> >> >> Actually after 10 works as well. >> > > When running a busy sleep loop of 1 ms I get a 990 us diff from the actual time > and when running a busy loop of 10 ms I get 1550 us diff. > BTW The same is true for smp disable > > Question for Richard really is why ? :-) The accuracy of the timeout delivered by the underlying primitive used (select() on macosx). It seems that the accuracy is better for short timeouts than long timeouts using select() on macosx. Using the above approach you also need a bit of luck in the smp case. The looping process needs to execute on the same scheduler thread as the scheduler thread managing the timer you want to improve. Regards, Rickard From carlsson.richard@REDACTED Tue Feb 23 16:02:16 2016 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 23 Feb 2016 16:02:16 +0100 Subject: [erlang-questions] mnesia_ext +leveldb In-Reply-To: <6672878f38887d1b733a769e6e456347@ssl.scheff32.de> References: <6672878f38887d1b733a769e6e456347@ssl.scheff32.de> Message-ID: Gah. Sorry, everybody, it's just been busy times here. Will try to start pulling that string again. Thanks for the reminder. /Richard 2016-02-23 13:38 GMT+01:00 Matthias Rieber : > Hello, > > is leveldb for mnesia_ext already open-sourced? I haven't found it on > https://github.com/klarna > > Matthias > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Tue Feb 23 16:59:24 2016 From: rvirding@REDACTED (Robert Virding) Date: Tue, 23 Feb 2016 16:59:24 +0100 Subject: [erlang-questions] Erlounge in Barcelona In-Reply-To: References: Message-ID: To be precise at 19 on Wednesday 24/2. Robert On 23 February 2016 at 11:21, Vance Shipley wrote: > Works for me. > On Feb 23, 2016 11:09 AM, "Robert Virding" wrote: > >> I have an offer and suggestion: >> >> SeQura, @*sequra_es* . 5th floor (no name >> on the door). Carrer d'Arag?, 383, 5? >> >> at 19 in the evening, or 7pm for any transatlantic guests. >> >> All welcome, >> Robert >> >> On 22 February 2016 at 23:25, Carlos Gonz?lez Florido < >> carlosj.gf@REDACTED> wrote: >> >>> I'm afraid I don't know Barcelona very well... I can ask some friends >>> tomorrow. >>> >>> On Mon, Feb 22, 2016 at 9:27 PM, Robert Virding >>> wrote: >>> >>>> Any suggestions to where we can meet on Wednesday? >>>> >>>> Robert >>>> >>>> >>>> On 21 February 2016 at 21:10, Robert Virding >>>> wrote: >>>> >>>>> OK, the my suggestion is that we meet on Wednesday evening for an >>>>> erlounge. Where I don't know. Anyone have a suggestion? >>>>> >>>>> Robert >>>>> >>>>> >>>>> On 21 February 2016 at 13:08, Vance Shipley >>>>> wrote: >>>>> >>>>>> On Sun, Feb 21, 2016 at 5:29 PM, Robert Virding >>>>>> wrote: >>>>>> > I would prefer Monday - Wednesday. Vance are you here on Wednesday >>>>>> evening? >>>>>> >>>>>> Yes, Wednesday evening is fine for me, I leave Thursday morning. >>>>>> >>>>>> -- >>>>>> -Vance >>>>>> >>>>> >>>>> >>>> >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Tue Feb 23 17:06:48 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Tue, 23 Feb 2016 08:06:48 -0800 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: <56CC7388.2090103@erlang.org> References: <56CC72D7.900@erlang.org> <56CC7388.2090103@erlang.org> Message-ID: I threw every available command line option at this to see if improvement could be seen -- erl +K true +sbwt very_long +sfwi 1 +swct very_eager +swt very_low Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:true] Eshell V7.2.1 (abort with ^G) 1> t:test(). [1005971,1005730,1003351,1005205,1004924,1003375,1005130, 1006166,1005846,1003769] ...perhaps a little, but nothing significant, unfortunately. OSX turns out to have a rich tapestry of low latency timers: https://developer.apple.com/library/mac/documentation/Darwin/Conceptual/KernelProgramming/scheduler/scheduler.html#//apple_ref/doc/uid/TP30000905-CH211-BABHGEFA as do FreeBSD and Linux, although their model and guarantee are a little less real-time: https://www.freebsd.org/cgi/man.cgi?query=usleep&sektion=3&apropos=0&manpath=freebsd http://man7.org/linux/man-pages/man2/nanosleep.2.html even Windows can be, as usual, hackily coerced: http://www.codeguru.com/cpp/w-p/system/timers/article.php/c5759/Creating-a-HighPrecision-HighResolution-and-Highly-Reliable-Timer-Utilising-Minimal-CPU-Resources.htm although usleep/nanosleep would probably be strictly better and not noticeably more intensive than select() on the unices, using Mach's THREAD_TIME_CONSTRAINT_POLICY or the windows hack would put all sleeping schedulers in a busy loop, so that would be something you might want fine grained control over. ideally, you could do something like: [...] process_flag(low_latency), [...] to signal to the schedulers that you want at least one thread to be running with significantly improved timing granularity, and while at least one low latency process existed, one or more schedulers would run under the specific regime and bind all the low latency processes to it. Although perhaps the erlang way is to create quick_schedulers dual to dirty_schedulers. F. On Tue, Feb 23, 2016 at 6:58 AM, Rickard Green wrote: > Missed reply all... > > -------- Forwarded Message -------- > Subject: Re: [erlang-questions] Sending message at a specific and accurate > time > Date: Tue, 23 Feb 2016 15:55:19 +0100 > From: Rickard Green > Organization: Ericsson AB > To: Tony Rogvall > > On 02/23/2016 02:29 PM, Tony Rogvall wrote: > >> >> On 22 feb 2016, at 15:55, Joe Armstrong wrote: >>> >>> On Mon, Feb 22, 2016 at 3:31 PM, Tony Rogvall wrote: >>> >>>> L = fun Loop() -> receive after 1 -> Loop() end end. >>>>> spawn_link(L). >>>>> >>>> >>> Busy sleep :-) < I thought busy waiting was frowned upon> >>> >>> Actually after 10 works as well. >>> >>> >> When running a busy sleep loop of 1 ms I get a 990 us diff from the >> actual time >> and when running a busy loop of 10 ms I get 1550 us diff. >> BTW The same is true for smp disable >> >> Question for Richard really is why ? :-) >> > > The accuracy of the timeout delivered by the underlying primitive used > (select() on macosx). It seems that the accuracy is better for short > timeouts than long timeouts using select() on macosx. > > Using the above approach you also need a bit of luck in the smp case. > The looping process needs to execute on the same scheduler thread as the > scheduler thread managing the timer you want to improve. > > Regards, > Rickard > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Tue Feb 23 17:58:53 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 23 Feb 2016 17:58:53 +0100 Subject: [erlang-questions] Cowboy header parsing termination? In-Reply-To: <20160222135733.GB1627@fhebert-ltm2.internal.salesforce.com> References: <20160222135733.GB1627@fhebert-ltm2.internal.salesforce.com> Message-ID: Thank you Fred, This is helpful feedback. The only reason that I was thinking it might be some kind of attack it's because this happened within an hour on three different machines (different environments - dev/staging/prod). Best, r. On Mon, Feb 22, 2016 at 2:57 PM, Fred Hebert wrote: > On 02/22, Roberto Ostinelli wrote: > >> Dear list, >> I keep on seeing the following in the logs: >> >> Ranch listener nucleo_listener terminated with reason: no case clause >> matching 123 in cowboy_protocol:parse_hd_name_ws/8 line 276 >> >> (code is here: >> >> https://github.com/ninenines/cowboy/blob/1.0.4/src/cowboy_protocol.erl#L276 >> ) >> >> Anyone using cowboy sees the same thing? I'm wondering if this is some >> kind >> of targeted attack on cowboy parsing. >> >> > 123 is the ASCII code for '{'. What you see here is a header fabricated to > look maybe a bit like this: > > My-Header-Name {whatever goes in here} > > This is not valid content. When cowboy sees the space after the header > name, it expects to see more space or a colon, possibly because the header > should look like: > > My-Header-Name : {whatever goes in here} > > But that colon is missing and all you get is that bracket right there. > > It would be easy to expect it to just be garbage over the line or > incomplete stuff than a direct attack on cowboy itself aas far as I can > tell. There's not too much to be found by just sending requests that end > early like that, unless someone is doing some form of fuzzing, in which > case you should find a lot more varied error logs along with this. > > The trick would be to look at the content that was sent over the line > before and after that point. One possibility could be that some proxy or > intermediary (I don't think cowboy itself is sensitive to that) could be > hit by request smuggling: > http://www.cgisecurity.com/lib/HTTP-Request-Smuggling.pdf > > But only contextual information could help reveal that. > > Regards, > Fred. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nyarly@REDACTED Tue Feb 23 19:12:26 2016 From: nyarly@REDACTED (Judson Lester) Date: Tue, 23 Feb 2016 18:12:26 +0000 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56CB95E1.4040309@cs.otago.ac.nz> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <20160222030407.GB53561@ferdmbp.local> <56CB95E1.4040309@cs.otago.ac.nz> Message-ID: For a while now, I've wished I could have more control over where module source files were on the filesystem. For example, instead of src/myapp_part_subpart.erl: -module(myapp_part_subpart). I'd really like to be able to put that in src/myapp/part/subpart.erl - maybe this is a misapprehension turned superstition, but I was under the impression that I'd need to have a whole separate build step to copy the latter into the former before handing it to erlc. Is that not the case? I'm pretty ambivalent about the extra step and the extra set of files vs. being able to name files as I'd like to, and if there's a way to get that to happen in one step I'd love to learn about it. Judson On Mon, Feb 22, 2016 at 3:12 PM Richard A. O'Keefe wrote: > The little AWK program is indeed neat, but it goes in the broken direction > (from file name to module name) not the working direction (from module > name to file name). This is the whole problem: given a file name, > we cannot cross-platform reliably reconstruct a module name. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethlakin@REDACTED Tue Feb 23 20:53:54 2016 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Tue, 23 Feb 2016 11:53:54 -0800 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <20160222030407.GB53561@ferdmbp.local> <56CB95E1.4040309@cs.otago.ac.nz> Message-ID: <56CCB8D2.6000405@gmail.com> On 02/23/2016 10:12 AM, Judson Lester wrote: > I'd really like to be able to put that in src/myapp/part/subpart.erl - > maybe this is a misapprehension turned superstition, but I was under the > impression that I'd need to have a whole separate build step to copy the > latter into the former before handing it to erlc. Is that not the case? It doesn't seem to be the case. This works just fine for me with Erlang 18: $ ls $ mkdir -p dir/with/parents $ echo "-module (test). > -compile (export_all). > > run() -> ok." > dir/with/parents/test.erl $ erlc dir/with/parents/test.erl $ ls dir test.beam $ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From bchesneau@REDACTED Tue Feb 23 21:12:40 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Tue, 23 Feb 2016 20:12:40 +0000 Subject: [erlang-questions] inet:getifaddrs/0 and windows Message-ID: I'm looking at the result of `inet:getifaddrs/0` and i'm not sure how to use the interface name "//DEVICE/...". Is this a device id? Any idea how i could query it using wmic? The idea behind is having a way to get the default route/gateway per interface by launching the wmic command line in erlang. But first i have to understand what kind of id is exposed. Any feedback is welcome :) - beno?t -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethlakin@REDACTED Tue Feb 23 21:34:42 2016 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Tue, 23 Feb 2016 12:34:42 -0800 Subject: [erlang-questions] inet:getifaddrs/0 and windows In-Reply-To: References: Message-ID: <56CCC262.9090706@gmail.com> On 02/23/2016 12:12 PM, Benoit Chesneau wrote: > I'm looking at the result of `inet:getifaddrs/0` and i'm not sure how to > use the interface name "//DEVICE/...". Is this a device id? Any idea how > i could query it using wmic? On my machine, the GUID part of the ifname (which is formatted as \\DEvICE\TCPIP_$GUID on my Windows 7 system) corresponds to the interface with the that GUID in the "SettingID" property of the "wmic nicconfig show" spew. I've never used wmic before, so I don't know how to get the nicconfig subcommand to only emit the information for a single interface, but maybe you do? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From essen@REDACTED Tue Feb 23 22:20:04 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Tue, 23 Feb 2016 22:20:04 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56CCB8D2.6000405@gmail.com> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <20160222030407.GB53561@ferdmbp.local> <56CB95E1.4040309@cs.otago.ac.nz> <56CCB8D2.6000405@gmail.com> Message-ID: <56CCCD04.40206@ninenines.eu> On 02/23/2016 08:53 PM, Kenneth Lakin wrote: > On 02/23/2016 10:12 AM, Judson Lester wrote: >> I'd really like to be able to put that in src/myapp/part/subpart.erl - >> maybe this is a misapprehension turned superstition, but I was under the >> impression that I'd need to have a whole separate build step to copy the >> latter into the former before handing it to erlc. Is that not the case? > > It doesn't seem to be the case. This works just fine for me with Erlang 18: > > $ ls > $ mkdir -p dir/with/parents > $ echo "-module (test). >> -compile (export_all). >> >> run() -> ok." > dir/with/parents/test.erl > $ erlc dir/with/parents/test.erl > $ ls > dir test.beam Interestingly, if -module was optional, this would open the way to tools being able to create the dir_with_parents_test.beam by concatenating the path segments. Erlang.mk has an option to do this with ErlyDTL templates already, it could be interesting to see this for Erlang source files. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From bchesneau@REDACTED Tue Feb 23 22:51:16 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Tue, 23 Feb 2016 21:51:16 +0000 Subject: [erlang-questions] inet:getifaddrs/0 and windows In-Reply-To: <56CCC262.9090706@gmail.com> References: <56CCC262.9090706@gmail.com> Message-ID: On Tue, Feb 23, 2016 at 9:34 PM Kenneth Lakin wrote: > On 02/23/2016 12:12 PM, Benoit Chesneau wrote: > > I'm looking at the result of `inet:getifaddrs/0` and i'm not sure how to > > use the interface name "//DEVICE/...". Is this a device id? Any idea how > > i could query it using wmic? > > On my machine, the GUID part of the ifname (which is formatted as > \\DEvICE\TCPIP_$GUID on my Windows 7 system) corresponds to the > interface with the that GUID in the "SettingID" property of the "wmic > nicconfig show" spew. > > I've never used wmic before, so I don't know how to get the nicconfig > subcommand to only emit the information for a single interface, but > maybe you do? > > This command will return the interface with the default gateway in a parsable way: ``` C:\Users\benoitc> wmic nicconfig get SettingId,DefaultIPGateway /format:csv Node,DefaultIPGateway,SettingID DESKTOP-HV04GN1,,{F99FBC41-A2B8-4382-96C2-515CADD46DBF} DESKTOP-HV04GN1,{192.168.1.254;fe80::224:d4ff:fea6:1b91},{6C201F80-3E4C-4207-9607-2C82ABE771DE} DESKTOP-HV04GN1,,{D0A5BB3C-75DD-4224-9554-569137405B53} DESKTOP-HV04GN1,,{58EB28CD-3B79-4A6A-A075-46577E343F83} ```` now I could use the `where INDEX=X` condition if I am sure that `inet:getifaddrs/0` always return in the same order. Not sure yet how to filter using the SettingId... - benoit -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Tue Feb 23 23:44:52 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Tue, 23 Feb 2016 22:44:52 +0000 Subject: [erlang-questions] inet:getifaddrs/0 and windows In-Reply-To: References: <56CCC262.9090706@gmail.com> Message-ID: On Tue, Feb 23, 2016 at 10:51 PM Benoit Chesneau wrote: > On Tue, Feb 23, 2016 at 9:34 PM Kenneth Lakin > wrote: > >> On 02/23/2016 12:12 PM, Benoit Chesneau wrote: >> > I'm looking at the result of `inet:getifaddrs/0` and i'm not sure how to >> > use the interface name "//DEVICE/...". Is this a device id? Any idea how >> > i could query it using wmic? >> >> On my machine, the GUID part of the ifname (which is formatted as >> \\DEvICE\TCPIP_$GUID on my Windows 7 system) corresponds to the >> interface with the that GUID in the "SettingID" property of the "wmic >> nicconfig show" spew. >> >> I've never used wmic before, so I don't know how to get the nicconfig >> subcommand to only emit the information for a single interface, but >> maybe you do? >> >> > This command will return the interface with the default gateway in a > parsable way: > > ``` > C:\Users\benoitc> wmic nicconfig get SettingId,DefaultIPGateway /format:csv > > Node,DefaultIPGateway,SettingID > DESKTOP-HV04GN1,,{F99FBC41-A2B8-4382-96C2-515CADD46DBF} > > DESKTOP-HV04GN1,{192.168.1.254;fe80::224:d4ff:fea6:1b91},{6C201F80-3E4C-4207-9607-2C82ABE771DE} > DESKTOP-HV04GN1,,{D0A5BB3C-75DD-4224-9554-569137405B53} > DESKTOP-HV04GN1,,{58EB28CD-3B79-4A6A-A075-46577E343F83} > ```` > > now I could use the `where INDEX=X` condition if I am sure that > `inet:getifaddrs/0` always return in the same order. Not sure yet how to > filter using the SettingId... > > - benoit > and then: ``` PS C:\Users\benoitc> wmic nicconfig where 'SettingId="{6C201F80-3E4C-4207-9607-2C82ABE771DE}"' get DefaultIPGateway /for mat:csv Node,DefaultIPGateway DESKTOP-HV04GN1,{192.168.1.254;fe80::224:d4ff:fea6:1b91} ```` Now time to code :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed Feb 24 00:46:12 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 24 Feb 2016 12:46:12 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <20160222030407.GB53561@ferdmbp.local> <56CB95E1.4040309@cs.otago.ac.nz> Message-ID: <94a762a4-511e-00ba-eb19-7530888a13ce@cs.otago.ac.nz> On 24/02/16 7:12 am, Judson Lester wrote: > For a while now, I've wished I could have more control over where > module source files were on the filesystem. For example, instead of > > src/myapp_part_subpart.erl: > -module(myapp_part_subpart). > > I'd really like to be able to put that in src/myapp/part/subpart.erl That's pretty much what GHC does for Haskell; see https://downloads.haskell.org/~ghc/7.0.1/docs/html/users_guide/separate-compilation.html I find it a pain in the --s- (and more so in Java) because you end up with one conceptual collection shotgunned across a fairly arbitrary set of directories. We have the notion of an OASIS catalogue for SGML and XML. I've used the acronym OASES without explaining it before: Organising Application Sources for Erlang Safely At a minimum, a myapp.oases file could contain things like {module, my_app_part_subpart, "src/myapp/part/subpart.erl"}. It could allow things like {module, my_app_otherPart, {zip,"src/myapp/lib.zip","otherPart.erl"}}. There'd be a 'finderl $module' command that you could use to find the file name corresponding to a particular module. I have a half-written web page about this; it's definitely not ready to put up yet. From ok@REDACTED Wed Feb 24 00:50:48 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 24 Feb 2016 12:50:48 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56CCB8D2.6000405@gmail.com> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <20160222030407.GB53561@ferdmbp.local> <56CB95E1.4040309@cs.otago.ac.nz> <56CCB8D2.6000405@gmail.com> Message-ID: <01806799-c45f-0e84-6895-28235000a32b@cs.otago.ac.nz> On 24/02/16 8:53 am, Kenneth Lakin wrote: > It doesn't seem to be the case. This works just fine for me with Erlang 18: > $ ls > $ mkdir -p dir/with/parents > $ echo "-module (test). >> -compile (export_all). >> >> run() -> ok." > dir/with/parents/test.erl > $ erlc dir/with/parents/test.erl > $ ls > dir test.beam > $ There is an important difference. The OP wanted the equivalent of -module(dir_with_parents_test). Try it and you get m% erlc dir/with/parents/test.erl /home/cshome/o/ok/erlang.d/test.beam: Module name 'dir_with_parents_test' does not match file name 'test' > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bchesneau@REDACTED Wed Feb 24 01:35:21 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Wed, 24 Feb 2016 00:35:21 +0000 Subject: [erlang-questions] inet:getifaddrs/0 and windows In-Reply-To: <56CCC262.9090706@gmail.com> References: <56CCC262.9090706@gmail.com> Message-ID: On Tue, Feb 23, 2016 at 9:34 PM Kenneth Lakin wrote: > On 02/23/2016 12:12 PM, Benoit Chesneau wrote: > > I'm looking at the result of `inet:getifaddrs/0` and i'm not sure how to > > use the interface name "//DEVICE/...". Is this a device id? Any idea how > > i could query it using wmic? > > On my machine, the GUID part of the ifname (which is formatted as > \\DEvICE\TCPIP_$GUID on my Windows 7 system) corresponds to the > interface with the that GUID in the "SettingID" property of the "wmic > nicconfig show" spew. > > I've never used wmic before, so I don't know how to get the nicconfig > subcommand to only emit the information for a single interface, but > maybe you do? > > Thanks for the hint about SettingId, it helped a lot :) Here is the final result getting the default route for an interface: https://github.com/benoitc/inet_ext/blob/master/src/inet_ext.erl#L49-L56 the code could probably be improved, but it works enough for now :) - benoit -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.santos@REDACTED Wed Feb 24 02:18:24 2016 From: michael.santos@REDACTED (Michael Santos) Date: Tue, 23 Feb 2016 20:18:24 -0500 Subject: [erlang-questions] inet:getifaddrs/0 and windows In-Reply-To: References: <56CCC262.9090706@gmail.com> Message-ID: <20160224011824.GA24212@brk> On Wed, Feb 24, 2016 at 12:35:21AM +0000, Benoit Chesneau wrote: > On Tue, Feb 23, 2016 at 9:34 PM Kenneth Lakin > wrote: > > > On 02/23/2016 12:12 PM, Benoit Chesneau wrote: > > > I'm looking at the result of `inet:getifaddrs/0` and i'm not sure how to > > > use the interface name "//DEVICE/...". Is this a device id? Any idea how > > > i could query it using wmic? > > > > On my machine, the GUID part of the ifname (which is formatted as > > \\DEvICE\TCPIP_$GUID on my Windows 7 system) corresponds to the > > interface with the that GUID in the "SettingID" property of the "wmic > > nicconfig show" spew. > > > > I've never used wmic before, so I don't know how to get the nicconfig > > subcommand to only emit the information for a single interface, but > > maybe you do? > > > > > Thanks for the hint about SettingId, it helped a lot :) > > Here is the final result getting the default route for an interface: > https://github.com/benoitc/inet_ext/blob/master/src/inet_ext.erl#L49-L56 > > the code could probably be improved, but it works enough for now :) inet_ext:gateway_for("foo ; touch /tmp/flag; echo"). :) On linux, the default gateway can be read from /proc/net/route. The flags are defined in /usr/include/linux/route.h. So a value of 3 for flags means RTF_UP|RTF_GATEWAY. From christopher.meiklejohn@REDACTED Wed Feb 24 06:47:38 2016 From: christopher.meiklejohn@REDACTED (Christopher Meiklejohn) Date: Tue, 23 Feb 2016 21:47:38 -0800 Subject: [erlang-questions] [ANN] PMLDC 2016, Call for Papers Message-ID: First Workshop on Programming Models and Languages for Distributed Computing (PMLDC 2016) Co-located with ECOOP 2016, Rome, Italy Date: July 17th, 2016 Whether you are programming a rich web application in JavaScript that mutates state in the client?s browser, or you are building a massively deployed mobile application that will operate with client state at the device, it?s undeniable that you are building a distributed system! Two major challenges of programming distributed systems are concurrency and partial failure. Concurrency of operations can introduce accidental nondeterminism: computations may result in different outcomes with the same inputs given scheduling differences in the underlying system unless a synchronization mechanism is used to enforce some order. Synchronization is typically expensive, and reduces the efficiency of user applications. Partial failure, or the failure of one or more components in a distributed system at one time, introduces the challenge of knowing, when an operation fails, which components of the operation completed successfully. To solve these problems in practice on an unreliable, asynchronous network, atomic commit protocols and timeouts as failure detection are typically used. Because of these challenges, early approaches to providing programming abstractions for distributed computing that ignored them were inherently misguided: the canonical example being the Remote Procedure Call, still widely deployed in industry. The goal of this workshop is to discuss new approaches to distributed programming that provide efficient execution and the elimination of accidental nondeterminism resulting from concurrency and partial failure. It will bring together both practitioners and theoreticians from many disciplines: database theory, distributed systems, systems programming, programming languages, data-centric programming, web application development, and verification, to discuss the state-of-the-art of distributed programming, advancement of the state-of-the-art and paths forward to better application of theory in practice. The main objectives of this workshop are the following: To review the state-of-the-art research in languages, models, and systems for distributed programming; To identify areas of critical need where research can advance the state of the art; To create a forum for discussion; To present open problems from practitioners with an aim towards motivating academic research on relevant problems faced by industry. In the spirit of both ECOOP and Curry On, this workshop aims at favoring a multidisciplinary perspective by bringing together researchers, developers, and practitioners from both academia and industry. Submission Guidelines We solicit proposals for contributed talks. We recommend preparing proposals of 2 pages, in ACM 2 column SIGPLAN style, written in English and in PDF format. However, we will accept longer proposals or submissions to other conferences, under the understanding that PC members are only expected to read the first two pages of such longer submissions. Authors with accepted papers will have the opportunity to have their submission published on the ACM Digital Library. Important Dates Paper submission: May 6, 2016 (any place on Earth) Authors notification: June 10, 2016 Final version: June 17, 2016 From bengt.kleberg@REDACTED Wed Feb 24 08:27:09 2016 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Wed, 24 Feb 2016 08:27:09 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <94a762a4-511e-00ba-eb19-7530888a13ce@cs.otago.ac.nz> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <20160222030407.GB53561@ferdmbp.local> <56CB95E1.4040309@cs.otago.ac.nz> <94a762a4-511e-00ba-eb19-7530888a13ce@cs.otago.ac.nz> Message-ID: <56CD5B4D.7020509@ericsson.com> Greetings, Would it be possible to get a more elaborate explanation of what OASES would do? Given the {module, my_app_part_subpart, "src/myapp/part/subpart.erl"} example below, would src/myapp/part/subpart.erl be available as the module my_app_part_subpart after being loaded in the VM? How would src/myapp/part/helper.erl refer to it: my_app_part_subpart or subpart or ??? How would src/myapp/other/subpart.erl refer to it: my_app_part_subpart or part_subpart or ??? I do not understand the "zip" thing enough to ask about it. bengt On 02/24/2016 12:46 AM, Richard A. O'Keefe wrote: > > > On 24/02/16 7:12 am, Judson Lester wrote: >> For a while now, I've wished I could have more control over where >> module source files were on the filesystem. For example, instead of >> >> src/myapp_part_subpart.erl: >> -module(myapp_part_subpart). >> >> I'd really like to be able to put that in src/myapp/part/subpart.erl > That's pretty much what GHC does for Haskell; see > https://downloads.haskell.org/~ghc/7.0.1/docs/html/users_guide/separate-compilation.html > > > I find it a pain in the --s- (and more so in Java) because you end up > with one > conceptual collection shotgunned across a fairly arbitrary set of > directories. > > We have the notion of an OASIS catalogue for SGML and XML. > I've used the acronym OASES without explaining it before: > Organising Application Sources for Erlang Safely > At a minimum, a myapp.oases file could contain things like > {module, my_app_part_subpart, "src/myapp/part/subpart.erl"}. > It could allow things like > {module, my_app_otherPart, {zip,"src/myapp/lib.zip","otherPart.erl"}}. > > There'd be a 'finderl $module' command that you could use to find > the file name corresponding to a particular module. > > I have a half-written web page about this; it's definitely not ready > to put up yet. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From matthias@REDACTED Wed Feb 24 11:21:40 2016 From: matthias@REDACTED (Matthias Lang) Date: Wed, 24 Feb 2016 11:21:40 +0100 Subject: [erlang-questions] tiny erlang In-Reply-To: References: Message-ID: <20160224102140.GA8062@corelatus.se> Hi, On 18. February 2016, Joe Armstrong wrote: > I want to make a small erlang distribution for an embedded device - > the process is rather simple: > 1) strip the beam executable > 2) for each library in {stdlib, kernerl, compiler, sasl} do > Compress all the beam code and remove debug symbols > squash everthing into a single packed file > 3) Hack the code loader to read from the single packed beam file > 4) Tweak the shell scripts to start Erlang > > At a guess we're down to c. 6 MB > > I have done this on *several* occassions - and am getting a bit fed up > doing it over and over again. > > Has anybody done this recently - and should not something like this be > in the standard distribution? I do this every few years, using a Makefile. I've handed that file out a few times. It's a get-stuff-done solution, not a masterpiece. Also, I'm not on the bleeding edge of Erlang; on some hardware I use R14B03, on other hardware 17.4. Some things I do slightly differently to your steps: 2. I don't think you need to compress the beam code; Erlang's build system does it for you by default when cross compiling, since at least R14B03. 2. I don't ship the compiler library. 2. I strip some .beams within libraries, e.g. 'stdlib/win32reg.beam' is useless on my hardware. 3. I don't think you need to hack the code loader; I don't. .ez files files work as-is since at least R14B03 For R14B03, that gets me down to 2.5M. For 17.4, it's 3M. --- Over the years, I've tried a number of different solutions to "which library .beam files do I ship?". Whitelist: the script only grabs the libraries I list. That's what you suggest above. Blacklist: the script deletes a list of libraries. That's what I do now. This 'fails' more gracefully when OTP gets a new library; I have to actively decide I don't need it. test suite driven: I make a release with all the .beams in it, run our (comprehensive!) test suite and delete all .beams which didn't get loaded, using information from code:all_loaded/0. This makes the smallest releases. I've settled on the 'blacklist' approach. It's robust and release size doesn't matter as much as it did in 2001 when I started doing this, mainly because flash memory has gotten far cheaper. Matt ---------------------------------------------------------------------- Here's the 'Makefile' I use: # Build and slim Erlang # Change the line below if you want to use a different version of Erlang # (Keep in mind that patches may not apply cleanly) erlang_version=otp_src_R14B03 erlang_source_file:=$(PACKAGE_SOURCES)/$(erlang_version)/$(erlang_version).tar.gz erlang_root=$(GTH_ERLANG_ROOT)/$(erlang_version) target_prefix=/opt/erlang ifneq ($(COR_TARGET), GTH3) ifneq ($(COR_TARGET), STH3) ifneq ($(COR_TARGET), GTH2D) $(error no COR_TARGET set, or set to an unknown target, giving up) endif endif endif xconf_file=erl-xcomp-armle-linux-gnu.conf install_prefix=$($(COR_TARGET)_INSTALL_BASE) prefix=$(install_prefix)$(target_prefix) ifeq (,$(findstring $(erlang_version),$(erlang_root))) default: @echo "Aborting because Erlang versions do not match" @echo "erlang_root=$(erlang_root)" @echo "erlang_version=$(erlang_version)" @/bin/false endif # Build the actual code # The 'unset' commands are needed because otp_build misbehaves if it realises # it's being run by a makefile. See mail to erlang-bugs 2010-06-22/23 build: $(erlang_version) @if test -d $(prefix); then \ echo "$(prefix) already contains a (partial?) Erlang dist. Aborting, because building over the top of a dist is known to break. Use 'make clean_target' to wipe $(prefix)" ; \ exit 1 ; \ fi sh -c "unset MAKELEVEL; unset MAKEFLAGS; cd $(erlang_version); ./otp_build boot -a" sh -c "cd $(erlang_version); make release RELEASE_ROOT=$(prefix)" # untar, patch, run configure $(erlang_version): $(erlang_source_file) tar -xzf $(erlang_source_file) sh -c "cd $(erlang_version); patch -p 1 < ../patches/better_to_erl_messages" sh -c "cd $(erlang_version); patch -p 1 < ../patches/extern_inline_c99" sh -c "cd $(erlang_version); patch -p 1 < ../patches/perl_5_22_defined_fix" sh -c "cd $(erlang_version); ./otp_build configure --xcomp-conf=../$(xconf_file)" install: run_otp_install slim strip make zip_libraries # run in sub-make; dependencies appeared just now .PHONY run_otp_install: sh -c "cd $(prefix); umask 022; ./Install -cross -minimal /opt/erlang" slim_binaries =$(addprefix $(prefix)/bin/, beam.smp dialyzer epmd erlc heart typer escript ct_run) slim_binaries+=$(addprefix $(prefix)/erts*/bin/, \ beam.smp dialyzer dyn_erl epmd erlc heart start typer escript ct_run *.src) slim_sources +=$(prefix)/erts*/src/ slim_sources +=$(prefix)/erts*/include/ slim_sources +=$(prefix)/erts*/doc/ slim_sources +=$(prefix)/erts*/man/ slim_sources +=$(prefix)/erts*/lib/internal/README slim_sources +=$(prefix)/lib/*/src/ slim_sources +=$(prefix)/lib/*/examples/ slim_sources +=$(prefix)/lib/*/include/ slim_sources +=$(prefix)/lib/*/man/ slim_sources +=$(prefix)/usr/ slim_sources +=$(prefix)/misc/ slim_applications +=$(prefix)/lib/appmon*/ slim_applications +=$(prefix)/lib/asn1*/ slim_applications +=$(prefix)/lib/common_test*/ slim_applications +=$(prefix)/lib/compiler*/ slim_applications +=$(prefix)/lib/cos*/ slim_applications +=$(prefix)/lib/debugger*/ slim_applications +=$(prefix)/lib/dialyzer*/ slim_applications +=$(prefix)/lib/diameter*/ slim_applications +=$(prefix)/lib/docbuilder*/ slim_applications +=$(prefix)/lib/edoc*/ slim_applications +=$(prefix)/lib/erl_docgen*/ slim_applications +=$(prefix)/lib/erl_interface*/ slim_applications +=$(prefix)/lib/et*/ slim_applications +=$(prefix)/lib/eunit*/ slim_applications +=$(prefix)/lib/gs*/ slim_applications +=$(prefix)/lib/hipe*/ slim_applications +=$(prefix)/lib/ic*/ slim_applications +=$(prefix)/lib/inets*/ slim_applications +=$(prefix)/lib/inviso*/ slim_applications +=$(prefix)/lib/jinterface*/ slim_applications +=$(prefix)/lib/megaco*/ slim_applications +=$(prefix)/lib/mnesia*/ slim_applications +=$(prefix)/lib/observer*/ slim_applications +=$(prefix)/lib/orber*/ slim_applications +=$(prefix)/lib/os_mon*/ slim_applications +=$(prefix)/lib/otp_mibs*/ slim_applications +=$(prefix)/lib/parsetools*/ slim_applications +=$(prefix)/lib/percept*/ slim_applications +=$(prefix)/lib/pman*/ slim_applications +=$(prefix)/lib/public_key*/ slim_applications +=$(prefix)/lib/reltool*/ slim_applications +=$(prefix)/lib/runtime_tools*/ slim_applications +=$(prefix)/lib/snmp*/ slim_applications +=$(prefix)/lib/syntax_tools*/ slim_applications +=$(prefix)/lib/test_server*/ slim_applications +=$(prefix)/lib/toolbar*/ slim_applications +=$(prefix)/lib/tools*/ slim_applications +=$(prefix)/lib/typer*/ slim_applications +=$(prefix)/lib/tv*/ slim_applications +=$(prefix)/lib/webtool*/ slim_applications +=$(prefix)/lib/wx*/ slim_applications +=$(prefix)/lib/xmerl*/ slim_applications +=$(prefix)/lib/stdlib*/ebin/win32reg.beam slim_applications +=$(prefix)/lib/sasl*/ebin/systools*.beam slim_applications +=$(prefix)/lib/sasl*/ebin/rb.beam slim_applications +=$(prefix)/lib/sasl*/ebin/rb_format_supp.beam slim_applications +=$(prefix)/lib/sasl*/ebin/erlsrv.beam slim_applications +=$(prefix)/lib/kernel*/ebin/inet6_sctp.beam slim_applications +=$(prefix)/lib/kernel*/ebin/array.beam slim_other_junk +=$(prefix)/erts*/lib/ slim_other_junk +=$(prefix)/Install slim_other_junk +=$(prefix)/releases/ slim_other_junk +=$(prefix)/bin/start_clean.boot slim: rm -f $(slim_binaries) rm -rf $(slim_sources) rm -rf $(slim_applications) rm -rf $(slim_other_junk) $(erlang_version)/bootstrap/bin/erl -noshell -noinput -eval "beam_lib:strip_release(\"$(prefix)\"),init:stop()" rm -rf $(prefix)/bin/run_erl rm -rf $(prefix)/bin/to_erl sh -c "cd $(prefix)/bin; ln -s ../erts*/bin/run_erl ." sh -c "cd $(prefix)/bin; ln -s ../erts*/bin/to_erl ." zip_libraries: $(addsuffix .ez, $(wildcard $(prefix)/lib/*)) %.ez: % sh -c "cd `dirname $@`; zip -n .beam -mr $@ `basename $^`" STRIP=$(CROSS_COMPILE)strip strip: $(STRIP) $(prefix)/erts*/bin/run_erl $(STRIP) $(prefix)/erts*/bin/to_erl $(STRIP) $(prefix)/erts*/bin/beam $(STRIP) $(prefix)/erts*/bin/inet_gethost $(STRIP) $(prefix)/erts*/bin/erlexec $(STRIP) $(prefix)/erts*/bin/child_setup $(STRIP) $(prefix)/bin/run_erl $(STRIP) $(prefix)/bin/to_erl clean: rm -rf $(erlang_version) # removes Erlang from the GTH/STH/GMH target directory clean_target: rm -rf $(prefix) # even deletes downloaded source code dist_clean: clean rm $(erlang_source_file) From max.lapshin@REDACTED Wed Feb 24 13:14:28 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 24 Feb 2016 15:14:28 +0300 Subject: [erlang-questions] tiny erlang In-Reply-To: <20160224102140.GA8062@corelatus.se> References: <20160224102140.GA8062@corelatus.se> Message-ID: 3 MB is very cool! We have reduced to 10 MB, but it includes developer and debugging tools. Smaller package ? faster loading. Also it is very important on slow flash to change configuration of code server so that it doesn't scan all ebin directories in ERL_LIBS for each file, but loads them in a more efficient way. We do it more dumb (we do not use releases): just scan all .beam files and load them. It makes startup faster. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kost@REDACTED Wed Feb 24 10:46:03 2016 From: kost@REDACTED (Konstantin Vsochanov) Date: Wed, 24 Feb 2016 12:46:03 +0300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56CB8BFF.2020802@cs.otago.ac.nz> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> Message-ID: <56CD7BDB.3050206@rol.ru> On 2/23/16 01:30 , Richard A. O'Keefe wrote: > I do not recall ever seeing -module coming up here before. > I certainly don't recall a long thread like this one. > Maybe because people who stumble over -module() too shy to write to erlang mail list? > #!/bin/sh > #Usage: edit module > case "$1" in > */*) echo "Module name may not contain /" >/dev/stderr; exit 1 ;; > -*) echo "Module name may not start with -">/dev/stderr; exit 1 ;; > *\'*) echo "Module name may not contain '" >/dev/stderr; exit 1 ;; > *\\*) echo "Module name may not contain \\" >/dev/stderr; exit 1 ;; > *) ;; > esac > file="$1.erl" # Beware: file system may mangle name! > if [ ! -e "$file" ]; then > date=`date +%Y-%m-%d` > case `expr "$1" : '^[a-z][a-zA-Z0-9_]*$'` in > 0) module="'$1'" ;; > *) module="$1" ;; > esac > echo "%%% File : $file" >"$file" > echo "%%% Author : $LOGNAME" >>"$file" > echo "%%% Created: $date" >>"$file" > echo "%%% Purpose: " >>"$file" > echo "" >>"$file" > echo "-module($module)." >>"$file" > echo "" >>"$file" > echo "-export([" >>"$file" > echo " ])." >>"$file" > echo "" >>"$file" > fi > exec ${EDITOR:-emacs} "$file" > > So here's how much burden -module adds to this programmer: > instead of > % emacs foobar.erl > I have to type > % edit foobar > > Hang on, that's LESS work. > Thank you for the script, I think it will work for me. I personally don't use emacs, but it doesn't matter. Reading all this long thread I must agree, the -module() is not the worst problem in Erlang. Regarding modules, I think the flat space for module names is more troubling. If you look at almost any Erlang library/application, you can see, that file names are like myapp_file1.erl myapp_file2.erl myapp_subsystem1_file1.erl ... So we manually implement hierarchic module system. It's a clear sign, that something going wrong or is missing. From mononcqc@REDACTED Wed Feb 24 16:07:07 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 24 Feb 2016 10:07:07 -0500 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56CD7BDB.3050206@rol.ru> References: <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> Message-ID: <20160224150705.GA90042@fhebert-ltm2.internal.salesforce.com> On 02/24, Konstantin Vsochanov wrote: > >Regarding modules, I think the flat space for module names is more >troubling. If you look at almost any Erlang library/application, you can >see, that file names are like > >myapp_file1.erl >myapp_file2.erl >myapp_subsystem1_file1.erl >... > >So we manually implement hierarchic module system. It's a clear sign, >that something going wrong or is missing. > It is, but it has some far-reaching implications that are easy to forget about when you don't do namespacing by hand. For example, ETS tables and processes can both be named, and if you do not manually namespace them, the same clashes can happen as would with modules. So we have to be aware of that. Designing a module namespace without having a way to expand this to process names and tables can yield fairly confusing cases where only parts of the system clash, but not others, and where things can remain nearly as tricky as they would otherwise be. I'm pretty sure `user' is the module that trips people up the most for that reason. HAving a namespace letting you run the equivalent of the current `my_project_user' and calling it as `user' in code could work, but if you try to register it as `user' you will fail since `user.erl' already does this at system boot (unless the --nouser option is passed to the VM). It is easy to think of simple solutions that ultimately don't cover enough cases, I'm not sure how things could reasonably be done with the actual ecosystem though. Regards, Fred. From rickard@REDACTED Wed Feb 24 16:24:56 2016 From: rickard@REDACTED (Rickard Green) Date: Wed, 24 Feb 2016 16:24:56 +0100 Subject: [erlang-questions] Sending message at a specific and accurate time In-Reply-To: References: <56CC72D7.900@erlang.org> <56CC7388.2090103@erlang.org> Message-ID: <56CDCB48.7090309@erlang.org> On 02/23/2016 05:06 PM, Felix Gallo wrote: > I threw every available command line option at this to see if > improvement could be seen -- > erl +K true +sbwt very_long +sfwi 1 +swct very_eager +swt very_low > Erlang/OTP 18 [erts-7.2.1] [source] [64-bit] [smp:4:4] > [async-threads:10] [hipe] [kernel-poll:true] > Eshell V7.2.1 (abort with ^G) > 1> t:test(). > [1005971,1005730,1003351,1005205,1004924,1003375,1005130, > 1006166,1005846,1003769] > > ...perhaps a little, but nothing significant, unfortunately. > > OSX turns out to have a rich tapestry of low latency timers: > > https://developer.apple.com/library/mac/documentation/Darwin/Conceptual/KernelProgramming/scheduler/scheduler.html#//apple_ref/doc/uid/TP30000905-CH211-BABHGEFA > Running on a real time priority may perhaps improve things. Setting prio of the vm is something that typically should be done by the user since the vm doesn't know much about other processes on the system and how it should be prioritized against them. It would have been nice with a command line tool for setting things like this, but I didn't find any. This can of course be implemented in the vm using the above api, but there need to be options for configuration of it by the user. > as do FreeBSD and Linux, although their model and guarantee are a little > less real-time: > > https://www.freebsd.org/cgi/man.cgi?query=usleep&sektion=3&apropos=0&manpath=freebsd > http://man7.org/linux/man-pages/man2/nanosleep.2.html > These kinds of primitives aren't optimal (from a functionality perspective), and can at least not be used out of the box. Schedulers always need to be able to be woken by various events, but with the sleep time limited by a timeout. Such events might be; data has become available to read on a file descriptor. In this case you typically want to wait using select()/poll()/... Such events might also be an incoming message making a process runnable on a scheduler that currently is sleeping. In order to use plain sleep primitives like those above, either: - another thread needs to perform the sleep and then wake up the scheduler that should be woken, or - another thread needs to wait for other events (such as fd events) and wake up the scheduler that should be woken using a signal. This at least introduce scheduling of an extra thread which might make things worse. Using them for a semi-busy wait (only checking for other events when woken by these primitives) will introduce increased latency for those events. Regards, Rickard > even Windows can be, as usual, hackily coerced: > > http://www.codeguru.com/cpp/w-p/system/timers/article.php/c5759/Creating-a-HighPrecision-HighResolution-and-Highly-Reliable-Timer-Utilising-Minimal-CPU-Resources.htm > > although usleep/nanosleep would probably be strictly better and not > noticeably more intensive than select() on the unices, using Mach's > THREAD_TIME_CONSTRAINT_POLICY or the windows hack would put all sleeping > schedulers in a busy loop, so that would be something you might want > fine grained control over. > > ideally, you could do something like: > > [...] > process_flag(low_latency), > [...] > > to signal to the schedulers that you want at least one thread to be > running with significantly improved timing granularity, and while at > least one low latency process existed, one or more schedulers would run > under the specific regime and bind all the low latency processes to it. > > Although perhaps the erlang way is to create quick_schedulers dual to > dirty_schedulers. > > F. > > On Tue, Feb 23, 2016 at 6:58 AM, Rickard Green > wrote: > > Missed reply all... > > -------- Forwarded Message -------- > Subject: Re: [erlang-questions] Sending message at a specific and > accurate time > Date: Tue, 23 Feb 2016 15:55:19 +0100 > From: Rickard Green > > Organization: Ericsson AB > To: Tony Rogvall > > > On 02/23/2016 02:29 PM, Tony Rogvall wrote: > > > On 22 feb 2016, at 15:55, Joe Armstrong > wrote: > > On Mon, Feb 22, 2016 at 3:31 PM, Tony Rogvall > > wrote: > > L = fun Loop() -> receive after 1 -> Loop() end end. > spawn_link(L). > > > Busy sleep :-) < I thought busy waiting was frowned upon> > > Actually after 10 works as well. > > > When running a busy sleep loop of 1 ms I get a 990 us diff from > the actual time > and when running a busy loop of 10 ms I get 1550 us diff. > BTW The same is true for smp disable > > Question for Richard really is why ? :-) > > > The accuracy of the timeout delivered by the underlying primitive used > (select() on macosx). It seems that the accuracy is better for short > timeouts than long timeouts using select() on macosx. > > Using the above approach you also need a bit of luck in the smp case. > The looping process needs to execute on the same scheduler thread as the > scheduler thread managing the timer you want to improve. > > Regards, > Rickard > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From g@REDACTED Wed Feb 24 16:49:24 2016 From: g@REDACTED (Garrett Smith) Date: Wed, 24 Feb 2016 15:49:24 +0000 Subject: [erlang-questions] HTTP client for OAuth Message-ID: I'm taking time off from typing -module attributes to focus on my backlog of other important problems... I'm interfacing with various OAuth providers within an Erlang application and I need a simple HTTP client. I've poked around the ecosystem and have learned that HTTP clients are the new MySQL clients in terms of which-one-to-freakin-use. As I'm using this for user auth, memory and performance are considerations, though far behind stability and actual-workingness. (In the interest of the later, I'd traditionally wrap OS curl processes because curl is known to actually work, but that _might_ not be feasible as the application grows. Come to think of it, I'm going to start with a curl wrapper and iterate by measuring, so never mind.) I'd love to learn from the field experience of others. Is anyone really, really happy with the HTTP client he or she is using for semi-heavy loads like user auth? -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethlakin@REDACTED Wed Feb 24 17:25:19 2016 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Wed, 24 Feb 2016 08:25:19 -0800 Subject: [erlang-questions] HTTP client for OAuth In-Reply-To: References: Message-ID: <56CDD96F.3060605@gmail.com> On 02/24/2016 07:49 AM, Garrett Smith wrote: > I'd love to learn from the field experience of others. Is anyone really, > really happy with the HTTP client he or she is using for semi-heavy > loads like user auth? I'm not sure what request volume you're dealing with, but I'm _fairly_ pleased with shotgun . However -unlike httpc- It doesn't do connection pooling of any sort, so you've got to handle that yourself if it's something you need. Be warned that it uses gun under the hood, and gun's purpose in life is to keep your HTTP connection open forever and always. So, if you're connecting to Other People's Servers(tm) maybe think about being nice and calling shotgun:close/1 when you expect that you're not going to use the connection for an extended period of time. Additionally, gun does what it calls "HTTP keepalive", which is to send a after five seconds of idle time on an open connection. The length of idle time before a keepalive is sent *is* configurable, but shotgun doesn't currently let you pass in the options required to change this. These keepalives cause trouble with _some_ HTTP servers (such as the server that serves erlang.org). If you get request failures after five seconds of idle time, this might be the culprit. If need be, I can send along a patch that will effectively disable HTTP keepalives by setting gun's keepalive timeout to something absurdly large. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From nathaniel@REDACTED Wed Feb 24 17:26:31 2016 From: nathaniel@REDACTED (Nathaniel Waisbrot) Date: Wed, 24 Feb 2016 11:26:31 -0500 Subject: [erlang-questions] HTTP client for OAuth In-Reply-To: References: Message-ID: <172423B8-1811-4781-9C4F-EDF840539561@waisbrot.net> > (In the interest of the later, I'd traditionally wrap OS curl processes because curl is known to actually work, but that _might_ not be feasible as the application grows. Come to think of it, I'm going to start with a curl wrapper and iterate by measuring, so never mind.) You may be interested in https://github.com/puzza007/katipo which is a libcurl-based Erlang HTTP client. I?ve never used it, but it?s being actively developed. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From tony@REDACTED Wed Feb 24 17:35:35 2016 From: tony@REDACTED (Tony Rogvall) Date: Wed, 24 Feb 2016 17:35:35 +0100 Subject: [erlang-questions] tiny erlang In-Reply-To: References: <20160224102140.GA8062@corelatus.se> Message-ID: <697FB7D9-D685-4B82-96B9-1790FE43D4D9@rogvall.se> I recently put together som instructions on how to cross compile erlang and user applications. It includes how to cross compile zlib/openssl/ncurses to get a decent crypto environment and a working terminal up and running. You can find it here: https://github.com/tonyrog/otp-cross This is not a HOWTO make a tiny erlang but can be used as a template for how to build the important parts. /Tony > On 24 feb 2016, at 13:14, Max Lapshin wrote: > > 3 MB is very cool! > > We have reduced to 10 MB, but it includes developer and debugging tools. > > Smaller package ? faster loading. > Also it is very important on slow flash to change configuration of code server so that it doesn't scan all ebin directories in ERL_LIBS for each file, but loads them in a more efficient way. > > We do it more dumb (we do not use releases): just scan all .beam files and load them. It makes startup faster. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From bchesneau@REDACTED Wed Feb 24 17:49:08 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Wed, 24 Feb 2016 16:49:08 +0000 Subject: [erlang-questions] HTTP client for OAuth In-Reply-To: References: Message-ID: On Wed, Feb 24, 2016 at 4:49 PM Garrett Smith wrote: > I'm taking time off from typing -module attributes to focus on my backlog > of other important problems... > > I'm interfacing with various OAuth providers within an Erlang application > and I need a simple HTTP client. I've poked around the ecosystem and have > learned that HTTP clients are the new MySQL clients in terms of > which-one-to-freakin-use. > > As I'm using this for user auth, memory and performance are > considerations, though far behind stability and actual-workingness. > > (In the interest of the later, I'd traditionally wrap OS curl processes > because curl is known to actually work, but that _might_ not be feasible as > the application grows. Come to think of it, I'm going to start with a curl > wrapper and iterate by measuring, so never mind.) > > I'd love to learn from the field experience of others. Is anyone really, > really happy with the HTTP client he or she is using for semi-heavy loads > like user auth? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Well I would advise hackney of course :) The pool part is completely optional and it will be gone in the week release anyway. It's used as a large scale in some companies handing millions of connections. It provides you a REST client API, streaming (transparent or not) and so on. If you want to do in a minimal way, I extracted the parser in a library: https://github.com/benoitc/nplib/blob/master/src/nplib_http.erl and then uses it with gen_tcp. It provides same kind of API than you find with erlang:decode_parser/3 but with less issue. For example decode_parser is not able to parse this: https://friendpaste.com/6w5CeBKwYZnMInCRZadlIX (not sure why yet, investigating it). Hope it's useful, any feedback is welcome. I would be interested to know what your choice at then end and why. I always enjoy to improve hackney when it's possible. - beno?t -------------- next part -------------- An HTML attachment was scrubbed... URL: From skribent_har@REDACTED Wed Feb 24 18:18:29 2016 From: skribent_har@REDACTED (Martin Hedberg) Date: Wed, 24 Feb 2016 17:18:29 +0000 Subject: [erlang-questions] Erlang and QNX Message-ID: Hello everyone Just wonder if there is some way to run Erlang on a QNX system? All help appreciated :-) Best regards Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From nyarly@REDACTED Wed Feb 24 18:44:07 2016 From: nyarly@REDACTED (Judson Lester) Date: Wed, 24 Feb 2016 17:44:07 +0000 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <01806799-c45f-0e84-6895-28235000a32b@cs.otago.ac.nz> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <20160222030407.GB53561@ferdmbp.local> <56CB95E1.4040309@cs.otago.ac.nz> <56CCB8D2.6000405@gmail.com> <01806799-c45f-0e84-6895-28235000a32b@cs.otago.ac.nz> Message-ID: On Tue, Feb 23, 2016 at 3:50 PM Richard A. O'Keefe wrote: > > > On 24/02/16 8:53 am, Kenneth Lakin wrote: > > It doesn't seem to be the case. This works just fine for me with > Erlang 18: > > $ ls > > $ mkdir -p dir/with/parents > > $ echo "-module (test). > >> -compile (export_all). > >> > >> run() -> ok." > dir/with/parents/test.erl > > $ erlc dir/with/parents/test.erl > > $ ls > > dir test.beam > > $ > There is an important difference. The OP wanted the equivalent of > -module(dir_with_parents_test). Try it and you get > m% erlc dir/with/parents/test.erl > /home/cshome/o/ok/erlang.d/test.beam: Module name > 'dir_with_parents_test' does not match file name 'test' > Indeed. Looking at the docs for erlc and compile, I can see that there's a couple of things that influence this, maybe. The key mystery, for me, is how compile:file/3 determines its product's path. Maybe it uses the File argument, but that isn't 100% clear. The error can be suppressed with an option, but it appears that would wreck code loading - maybe moving the file afterwards? There'd be problems with concurrent builds, though, and it doesn't do much for avoiding an intermediate file. There's an option to set the source name, but again, not clear if that's just used by module_info/1 or to construct the output path. Finally, I suppose you could use the binary option to produce the module in memory and then write that to a file - would that work, though? I guess the point of frustration I come down to is that the file name and the module name don't have to be the same ... except that they do. One concrete use case: I've recently split up a gigantic module into several smaller ones, but they all need to share a series of record definitions. I'd love to be able to collect them all into a directory and put the record defs in a .hrl there, and be reasonably sure that the record is only available to those modules. Judson -------------- next part -------------- An HTML attachment was scrubbed... URL: From kaiduanx@REDACTED Wed Feb 24 18:47:16 2016 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Wed, 24 Feb 2016 12:47:16 -0500 Subject: [erlang-questions] Erlang and QNX In-Reply-To: References: Message-ID: https://openbbnews.wordpress.com/2012/01/13/couchdb-playbook/ This is a pretty old story ... /Kaiduan On Wed, Feb 24, 2016 at 12:18 PM, Martin Hedberg wrote: > > Hello everyone > > > Just wonder if there is some way to run Erlang on a QNX system? > > > All help appreciated :-) > > > Best regards > > > Martin > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Wed Feb 24 19:06:28 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Wed, 24 Feb 2016 18:06:28 +0000 Subject: [erlang-questions] inet:getifaddrs/0 and windows In-Reply-To: <20160224011824.GA24212@brk> References: <56CCC262.9090706@gmail.com> <20160224011824.GA24212@brk> Message-ID: On Wed, Feb 24, 2016 at 2:16 AM Michael Santos wrote: > On Wed, Feb 24, 2016 at 12:35:21AM +0000, Benoit Chesneau wrote: > > On Tue, Feb 23, 2016 at 9:34 PM Kenneth Lakin > > wrote: > > > > > On 02/23/2016 12:12 PM, Benoit Chesneau wrote: > > > > I'm looking at the result of `inet:getifaddrs/0` and i'm not sure > how to > > > > use the interface name "//DEVICE/...". Is this a device id? Any idea > how > > > > i could query it using wmic? > > > > > > On my machine, the GUID part of the ifname (which is formatted as > > > \\DEvICE\TCPIP_$GUID on my Windows 7 system) corresponds to the > > > interface with the that GUID in the "SettingID" property of the "wmic > > > nicconfig show" spew. > > > > > > I've never used wmic before, so I don't know how to get the nicconfig > > > subcommand to only emit the information for a single interface, but > > > maybe you do? > > > > > > > > Thanks for the hint about SettingId, it helped a lot :) > > > > Here is the final result getting the default route for an interface: > > https://github.com/benoitc/inet_ext/blob/master/src/inet_ext.erl#L49-L56 > > > > the code could probably be improved, but it works enough for now :) > > inet_ext:gateway_for("foo ; touch /tmp/flag; echo"). > > :) On linux, the default gateway can be read from /proc/net/route. The > flags are defined in /usr/include/linux/route.h. So a value of 3 for > flags means RTF_UP|RTF_GATEWAY. > oups :) i tagged the 0.4 that is no exposing this API anymore. Thanks! Thanks for the tip anyway I should have a look. For now the lib does the job but the final goal would be having some portable C code to skip the need to call and parse the results from a script. Hopefully that won't be that hard :) - benoi -------------- next part -------------- An HTML attachment was scrubbed... URL: From z@REDACTED Wed Feb 24 19:19:19 2016 From: z@REDACTED (Danil Zagoskin) Date: Wed, 24 Feb 2016 21:19:19 +0300 Subject: [erlang-questions] Erlang and QNX In-Reply-To: References: Message-ID: Hi! We have been using OTP on QNX Neutrino successfully. At that time the only patch we needed to build it was https://github.com/blackberry/Erlang-OTP/commit/5110c9667e1da9f278dd9e258f923303297b73a2 (CouchDB on PlayBook article mentioned by Kaiduan Xie relies on that repo). Fresh releases (R15 and later) may require some more hacks. On Wed, Feb 24, 2016 at 8:18 PM, Martin Hedberg wrote: > > Hello everyone > > > Just wonder if there is some way to run Erlang on a QNX system? > > > All help appreciated :-) > > > Best regards > > > Martin > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- Danil Zagoskin | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Wed Feb 24 20:01:59 2016 From: g@REDACTED (Garrett Smith) Date: Wed, 24 Feb 2016 19:01:59 +0000 Subject: [erlang-questions] HTTP client for OAuth In-Reply-To: References: Message-ID: On Wed, Feb 24, 2016 at 10:49 AM Benoit Chesneau wrote: > On Wed, Feb 24, 2016 at 4:49 PM Garrett Smith wrote: > >> I'm taking time off from typing -module attributes to focus on my backlog >> of other important problems... >> >> I'm interfacing with various OAuth providers within an Erlang application >> and I need a simple HTTP client. I've poked around the ecosystem and have >> learned that HTTP clients are the new MySQL clients in terms of >> which-one-to-freakin-use. >> >> As I'm using this for user auth, memory and performance are >> considerations, though far behind stability and actual-workingness. >> >> (In the interest of the later, I'd traditionally wrap OS curl processes >> because curl is known to actually work, but that _might_ not be feasible as >> the application grows. Come to think of it, I'm going to start with a curl >> wrapper and iterate by measuring, so never mind.) >> >> I'd love to learn from the field experience of others. Is anyone really, >> really happy with the HTTP client he or she is using for semi-heavy loads >> like user auth? >> > _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > Well I would advise hackney of course :) The pool part is completely > optional and it will be gone in the week release anyway. It's used as a > large scale in some companies handing millions of connections. It provides > you a REST client API, streaming (transparent or not) and so on. > I'll probably come around to this, but I was initially surprised by the four, or maybe it was five, even six additional apps that came along with hackney. Are you sure it isn't an HTTP _server_? :p > > If you want to do in a minimal way, I extracted the parser in a library: > https://github.com/benoitc/nplib/blob/master/src/nplib_http.erl > > and then uses it with gen_tcp. > > This is interesting - I'll take a look! -------------- next part -------------- An HTML attachment was scrubbed... URL: From achowdhury918@REDACTED Wed Feb 24 23:12:56 2016 From: achowdhury918@REDACTED (Akash Chowdhury) Date: Wed, 24 Feb 2016 17:12:56 -0500 Subject: [erlang-questions] Question about dtlsex Message-ID: Hi, I need to use DTLS in Erlang. Does anyone have used dtlsex ( https://github.com/RoadRunnr/dtlsex/tree/master/src)? Is it properly tested? Is it usable/reliable? I need to use it for server side. Any one has any experience? Any input will be highly appreciated. Looking forward to it. Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bernard@REDACTED Wed Feb 24 23:14:32 2016 From: bernard@REDACTED (Bernard Duggan) Date: Thu, 25 Feb 2016 09:14:32 +1100 Subject: [erlang-questions] HTTP client for OAuth In-Reply-To: References: Message-ID: On Thu, Feb 25, 2016 at 2:49 AM, Garrett Smith wrote: > I'm interfacing with various OAuth providers within an Erlang application > and I need a simple HTTP client. I've poked around the ecosystem and have > learned that HTTP clients are the new MySQL clients in terms of > which-one-to-freakin-use. > I'd be very interested to know what you turn up, since I'm currently pondering exactly the same problem (OAuth and all). This is probably an absurdly naive question, but for quick-fire simple stuff like this, is there any good reason you're aware of not to just go with httpc? Or is this more just a matter of doing due diligence on the choice? Cheers, Bernrad -------------- next part -------------- An HTML attachment was scrubbed... URL: From radek@REDACTED Wed Feb 24 23:15:47 2016 From: radek@REDACTED (Rad Gruchalski) Date: Wed, 24 Feb 2016 23:15:47 +0100 Subject: [erlang-questions] HTTP client for OAuth In-Reply-To: References: Message-ID: <0C395A7EB0E34B8FB7D0CCF917DF5835@gruchalski.com> Hackney is really decent. I used it for one of my projects and would definitely recommend! Kind regards, Radek Gruchalski radek@REDACTED (mailto:radek@REDACTED) (mailto:radek@REDACTED) de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/) Confidentiality: This communication is intended for the above-named person and may be confidential and/or legally privileged. If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately. On Wednesday, 24 February 2016 at 20:01, Garrett Smith wrote: > On Wed, Feb 24, 2016 at 10:49 AM Benoit Chesneau wrote: > > On Wed, Feb 24, 2016 at 4:49 PM Garrett Smith wrote: > > > I'm taking time off from typing -module attributes to focus on my backlog of other important problems... > > > > > > I'm interfacing with various OAuth providers within an Erlang application and I need a simple HTTP client. I've poked around the ecosystem and have learned that HTTP clients are the new MySQL clients in terms of which-one-to-freakin-use. > > > > > > As I'm using this for user auth, memory and performance are considerations, though far behind stability and actual-workingness. > > > > > > (In the interest of the later, I'd traditionally wrap OS curl processes because curl is known to actually work, but that _might_ not be feasible as the application grows. Come to think of it, I'm going to start with a curl wrapper and iterate by measuring, so never mind.) > > > > > > I'd love to learn from the field experience of others. Is anyone really, really happy with the HTTP client he or she is using for semi-heavy loads like user auth? > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > Well I would advise hackney of course :) The pool part is completely optional and it will be gone in the week release anyway. It's used as a large scale in some companies handing millions of connections. It provides you a REST client API, streaming (transparent or not) and so on. > > I'll probably come around to this, but I was initially surprised by the four, or maybe it was five, even six additional apps that came along with hackney. Are you sure it isn't an HTTP _server_? :p > > > > > If you want to do in a minimal way, I extracted the parser in a library: > > https://github.com/benoitc/nplib/blob/master/src/nplib_http.erl > > > > and then uses it with gen_tcp. > > > This is interesting - I'll take a look! > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Wed Feb 24 23:24:23 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Wed, 24 Feb 2016 14:24:23 -0800 Subject: [erlang-questions] HTTP client for OAuth In-Reply-To: References: Message-ID: I believe I've seen commits fly by which intended to fix the behavior, but the last time I used httpc, processes would occasionally get the http reply from a completely different process when run concurrently. Definitely a WAT moment. I finally ended up with a fork of lhttpc ( https://github.com/Cloven/lhttpc) which was stable, fast, and didn't get mixed up. If I would do it all again today, I would probably start with hackney. But I would definitely avoid httpc. On Wed, Feb 24, 2016 at 2:14 PM, Bernard Duggan wrote: > On Thu, Feb 25, 2016 at 2:49 AM, Garrett Smith wrote: > >> I'm interfacing with various OAuth providers within an Erlang application >> and I need a simple HTTP client. I've poked around the ecosystem and have >> learned that HTTP clients are the new MySQL clients in terms of >> which-one-to-freakin-use. >> > > I'd be very interested to know what you turn up, since I'm currently > pondering exactly the same problem (OAuth and all). This is probably an > absurdly naive question, but for quick-fire simple stuff like this, is > there any good reason you're aware of not to just go with httpc? Or is this > more just a matter of doing due diligence on the choice? > > Cheers, > > Bernrad > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Wed Feb 24 23:30:54 2016 From: roger@REDACTED (Roger Lipscombe) Date: Wed, 24 Feb 2016 14:30:54 -0800 Subject: [erlang-questions] HTTP client for OAuth In-Reply-To: References: Message-ID: We were using the ESL lhttpc library, but changed to using hackney when we moved to Erlang 17, which lhttpc doesn't officially support. Perfectly happy with hackney. We looked at gun, shotgun and fusco (referred to by the lhttpc deprecation notice), but decided that hackney was a better fit for what we were doing (one-off calls to webhooks). On 24 February 2016 at 14:24, Felix Gallo wrote: > I believe I've seen commits fly by which intended to fix the behavior, but > the last time I used httpc, processes would occasionally get the http reply > from a completely different process when run concurrently. Definitely a WAT > moment. I finally ended up with a fork of lhttpc > (https://github.com/Cloven/lhttpc) which was stable, fast, and didn't get > mixed up. > > If I would do it all again today, I would probably start with hackney. But > I would definitely avoid httpc. > > > On Wed, Feb 24, 2016 at 2:14 PM, Bernard Duggan > wrote: >> >> On Thu, Feb 25, 2016 at 2:49 AM, Garrett Smith wrote: >>> >>> I'm interfacing with various OAuth providers within an Erlang application >>> and I need a simple HTTP client. I've poked around the ecosystem and have >>> learned that HTTP clients are the new MySQL clients in terms of >>> which-one-to-freakin-use. >> >> >> I'd be very interested to know what you turn up, since I'm currently >> pondering exactly the same problem (OAuth and all). This is probably an >> absurdly naive question, but for quick-fire simple stuff like this, is there >> any good reason you're aware of not to just go with httpc? Or is this more >> just a matter of doing due diligence on the choice? >> >> Cheers, >> >> Bernrad >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ok@REDACTED Thu Feb 25 01:14:34 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 25 Feb 2016 13:14:34 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56CD5B4D.7020509@ericsson.com> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <20160222030407.GB53561@ferdmbp.local> <56CB95E1.4040309@cs.otago.ac.nz> <94a762a4-511e-00ba-eb19-7530888a13ce@cs.otago.ac.nz> <56CD5B4D.7020509@ericsson.com> Message-ID: On 24/02/16 8:27 pm, Bengt Kleberg wrote: > Greetings, > > Would it be possible to get a more elaborate explanation of what OASES > would do? Map module names, grammar names, include names, &c to data sources, so that Erlang programmers would mostly deal with portable Erlang-internal names. > Given the {module, my_app_part_subpart, "src/myapp/part/subpart.erl"} > example below, would src/myapp/part/subpart.erl be available as the > module my_app_part_subpart after being loaded in the VM? Yes. > How would src/myapp/part/helper.erl refer to it: my_app_part_subpart > or subpart or ??? By the name that you provided in the -module directive. This is not a "hierarchical module names" proposal; the OASES idea is just "referring to resources in a portable way". > How would src/myapp/other/subpart.erl refer to it: my_app_part_subpart > or part_subpart or ??? By the name that you provided in the -module directive. > I do not understand the "zip" thing enough to ask about it. This goes back to IBM mainframes, to Unix Makefiles, and to .ez files. In the MVS world, you might keep the Fortran code for your project in a "partitioned data set" (think Unix .a file) called FORTRAN, so you would refer to FORTRAN(DGEMM) -- the "DGEMM" member of the "FORTRAN" partitioned data set rather than a Unixy dgemm.f. Unix makefiles let you refer to files in a .a file using foobar.a(membername) and would automatically extract things. The idea is that you might want to package up a bunch of stuff in a ZIP file (or some other kind of archive), to save space, or to make it easier to move stuff around, or whatever. Why should you have to extract a file from an archive in order to read it? Why should the compiler not be able to read *directly* from the archive? From ok@REDACTED Thu Feb 25 01:24:08 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 25 Feb 2016 13:24:08 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56CD7BDB.3050206@rol.ru> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> Message-ID: <1bd6dfe7-3499-5030-7ce2-e651db2431d8@cs.otago.ac.nz> On 24/02/16 10:46 pm, Konstantin Vsochanov wrote: > On 2/23/16 01:30 , Richard A. O'Keefe wrote: >> I do not recall ever seeing -module coming up here before. >> I certainly don't recall a long thread like this one. >> >> Maybe because people who stumble over -module() too shy to write to >> erlang mail list? That seems highly unlikely. People complain about all sorts of other things; why should they be shy about this one alone? > Regarding modules, I think the flat space for module names is more > troubling. Agreed. There was an experimental implementation of hierarchical module names for Erlang, but it was abandoned. > So we manually implement hierarchic module system. It's a clear sign, > that something going wrong or is missing. The thing is that it's not clear *what*. Consider Haskell for instance, which has hierarchical module names. You end up with names like Data.Text.ByteString.Lazy. These things are not relative, so Haskell code gets littered with stuff like import Data.Text.ByteString.Lazy as B That's not good. SETL had hierarchical modules (described in the book "On Programming"). They were abandoned in later versions of SETL. Ada had nested modules with simple names following conventional scope rules, but added "child modules". Pop11 had modules. Java has "packages", which lead to very long-winded names. There doesn't seem to be anything around at the moment that we can just copy. One idea was to run 'applications' in their own 'containers' inside an Erlang VM, so that module names could be container-local and another container would not even be able to mention one of your application's private modules. That actually seems promising for other reasons, like being able to have two versions of a module in use in different applications. From raould@REDACTED Thu Feb 25 01:28:17 2016 From: raould@REDACTED (Raoul Duke) Date: Wed, 24 Feb 2016 16:28:17 -0800 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <1bd6dfe7-3499-5030-7ce2-e651db2431d8@cs.otago.ac.nz> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <1bd6dfe7-3499-5030-7ce2-e651db2431d8@cs.otago.ac.nz> Message-ID: > import Data.Text.ByteString.Lazy as B > That's not good. good vs. perfect? go does that, too, fwiw. From ok@REDACTED Thu Feb 25 01:36:18 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 25 Feb 2016 13:36:18 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <1bd6dfe7-3499-5030-7ce2-e651db2431d8@cs.otago.ac.nz> Message-ID: <1dd9d665-fa46-aaf2-0a5e-5ed34fb1bb11@cs.otago.ac.nz> On 25/02/16 1:28 pm, Raoul Duke wrote: >> import Data.Text.ByteString.Lazy as B >> That's not good. >> >> good vs. perfect? >> go does that, too, fwiw. The problem is that two different files may refer to a third module by different short names and may use the same short name to refer to different modules. At least with Erlang we use exactly the same name everywhere. A good solution will have some part of the name invariant. From raould@REDACTED Thu Feb 25 01:42:29 2016 From: raould@REDACTED (Raoul Duke) Date: Wed, 24 Feb 2016 16:42:29 -0800 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <1dd9d665-fa46-aaf2-0a5e-5ed34fb1bb11@cs.otago.ac.nz> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <1bd6dfe7-3499-5030-7ce2-e651db2431d8@cs.otago.ac.nz> <1dd9d665-fa46-aaf2-0a5e-5ed34fb1bb11@cs.otago.ac.nz> Message-ID: > The problem is that two different files may refer to a third module > by different short names and may use the same short name to > refer to different modules. At least with Erlang we use exactly > the same name everywhere. That doesn't break any compilation. It does make it harder for humans to grok, which is of course baaaaaad. I don't get the example, really, I guess? Is it not the case that in Java when you import foo.bar.baz, you now do not have to say baz.Thing you only have to say Thing? Not a lot of extra typing? When there's 2 packages with Things then you have to go up one level of importing, sure. From essen@REDACTED Thu Feb 25 01:53:00 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Thu, 25 Feb 2016 01:53:00 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <1bd6dfe7-3499-5030-7ce2-e651db2431d8@cs.otago.ac.nz> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <1bd6dfe7-3499-5030-7ce2-e651db2431d8@cs.otago.ac.nz> Message-ID: <56CE506C.1030700@ninenines.eu> On 02/25/2016 01:24 AM, Richard A. O'Keefe wrote: > On 24/02/16 10:46 pm, Konstantin Vsochanov wrote: >> On 2/23/16 01:30 , Richard A. O'Keefe wrote: >>> I do not recall ever seeing -module coming up here before. >>> I certainly don't recall a long thread like this one. >>> >>> Maybe because people who stumble over -module() too shy to write to >>> erlang mail list? > > That seems highly unlikely. People complain about all sorts > of other things; why should they be shy about this one alone? They're not. What is highly unlikely is to hear about small problems on the mailing list, because few beginners register here, and it's rarely to ask beginner questions (Roelof posts a lot but is just one). Few people care about that sort of thing once they're used to it. Same goes for all small annoyances brought up by beginners again and again. You are a lot more likely to hear beginner feedback in person (often in the form of a question: if a question comes up often, that's a smell, or by making mistakes) or where beginners go to get help (IRC, StackOverflow and so on). But most of that feedback will usually not go as far as the one in this thread which points out something that is felt unnecessary. A similar but larger pain point that's been rationalized by people used to Erlang is the expression separators. It comes up regularly on the mailing list, but comes up a lot more outside, and even more when teaching people. It's probably the most frequent beginner mistake. You can explain it off perfectly, sure, but it'll remain the most frequent beginner mistake. The only way to fix that is to forget those perfect explanations, admit this is a problem and fix it. (I would write the fix, Steve, but it's even less likely that this kind of patch would get merged as things stand today.) I get the feeling from this mailing list that if something has a perfect explanation there's no reason to change it. That's partly why I think maps are a miracle (we had the dict module already after all). But that's wrong. Things that give unnecessary trouble to people should be changed, regardless of the accuracy of the explanation you can give to people when they run into trouble/get fed up with it. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From ok@REDACTED Thu Feb 25 03:50:33 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 25 Feb 2016 15:50:33 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <1bd6dfe7-3499-5030-7ce2-e651db2431d8@cs.otago.ac.nz> <1dd9d665-fa46-aaf2-0a5e-5ed34fb1bb11@cs.otago.ac.nz> Message-ID: On 25/02/16 1:42 pm, Raoul Duke wrote: >> The problem is that two different files may refer to a third module >> by different short names and may use the same short name to >> refer to different modules. At least with Erlang we use exactly >> the same name everywhere. >> >> That doesn't break any compilation. Nobody ever said it did. >> >> It does make it harder for humans to grok, which is of course baaaaaad. >> >> I don't get the example, really, I guess? How could I make it more explicit? A Haskell module can refer to another module by one (or more!) internal names which are in no way coupled to the file system and are in no way coupled to the name the module believes it has. The name a module has *is* coupled to the file system. >> Is it not the case that in >> Java when you import foo.bar.baz, you now do not have to say baz.Thing >> you only have to say Thing? Not a lot of extra typing? When there's 2 >> packages with Things then you have to go up one level of importing, >> sure. Please don't get me started on Java. One problem is that you can write a file that starts with import com.example.gorilla.fossy.fearnot.*; import nz.ac.erewhon.is.weka.beak.*; and everything is hunky-dory, and then a week later those horrible people at gorilla.example.com add a new class and you have a clash. Oh wait, you got me started on Java. Summary: there are several "OK, this is just tolerable, we can limp along with this" hierarchical module systems out there, but as yet, no outstanding "wow, gotta have this", especially for a dynamic distributed language. From ok@REDACTED Thu Feb 25 04:00:12 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 25 Feb 2016 16:00:12 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56CE506C.1030700@ninenines.eu> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <1bd6dfe7-3499-5030-7ce2-e651db2431d8@cs.otago.ac.nz> <56CE506C.1030700@ninenines.eu> Message-ID: <20b85c19-876e-3a75-38f3-394933d1edce@cs.otago.ac.nz> On 25/02/16 1:53 pm, Lo?c Hoguin wrote: > > A similar but larger pain point that's been rationalized by people > used to Erlang is the expression separators. I take it you're referring to commas, semicolons, and full stops. We have a complete solution to that called LFE. We also have Elixir, which has enough syntactic weirdness of its own that I decided not to use it. I briefly played with a Haskell-inspired syntax for Erlang, which I still think would be a good idea. One thing I certainly kept in that design was -module directives because of the good they do. I wouldn't say that I've *rationalised* the issue, just that I stopped falling off that particular bike very very quickly. I don't think there is any point in trying to "fix" Erlang syntax as it stands. I *do* think there is point in developing a whole *new* self-consistent syntax from the ground up, just like D didn't so much fix C's syntax as replace it. (By the way, am I the only person here who remembers that Algol 68 used ";" for sequence and "," for parallel? Hands up everyone on the list who has an Algol 68 implementation on their machine.) From unix1@REDACTED Thu Feb 25 04:49:23 2016 From: unix1@REDACTED (Unix One) Date: Thu, 25 Feb 2016 03:49:23 +0000 Subject: [erlang-questions] HTTP client for OAuth In-Reply-To: References: Message-ID: On 02/24/2016 07:49 AM, Garrett Smith wrote: > I'm interfacing with various OAuth providers within an Erlang > application and I need a simple HTTP client. I've poked around the > ecosystem and have learned that HTTP clients are the new MySQL clients > in terms of which-one-to-freakin-use. When I was writing a client for a proprietary backend that used HTTP and required its own authentication, I decided to split things up a bit: I created an HTTP client wrapper[1] that accomplished 3 things: - acted as a wrapper over a number of actual HTTP clients via a behavior (initially supporting gun and httpc, was planning to add hackney later) - exposed a behavior for authentication[2] - exposed a behavior for application specific functionality/requests[3] You can see the sample implementation[4]. My intent was to create an HTTP client application where you would only need to plug in your custom application functions and authentication. I haven't updated it in a while, so I wouldn't be surprised if it's broken against current gun. [1] https://github.com/unix1/httpclient (I know, very poorly named) [2] https://github.com/unix1/httpclient/blob/master/guide/Login_Handler.md [3] https://github.com/unix1/httpclient/blob/master/guide/Service_Handler.md [4] https://github.com/unix1/splunkclient From bchesneau@REDACTED Thu Feb 25 08:24:20 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 25 Feb 2016 07:24:20 +0000 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <20b85c19-876e-3a75-38f3-394933d1edce@cs.otago.ac.nz> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <1bd6dfe7-3499-5030-7ce2-e651db2431d8@cs.otago.ac.nz> <56CE506C.1030700@ninenines.eu> <20b85c19-876e-3a75-38f3-394933d1edce@cs.otago.ac.nz> Message-ID: i'm not sure if people complained but that the second time at least that such topic fall on the mailing list to my knowledge. One sure thing though is that people using elixir like the possibility to mix multiples modules in one file . They find it convenient. At the end multiples beam files will be created by the elixir compiler(builder?) . Convenience is important when you have to code like we do today. I don't really understand all the complexity in that thread. Reading Joe response i understand that the current implementation is not that flexible. But how difficult it would be to trick the compiler to find module blocks? (like in c++ with objects). Namespace collision can be detected at compilation. case sensitivity is imo out of topic since anyway no real solution exist. So coming back to my initial idea why having something like -module(b). .. -endmodule. couldd't be handeld by the compiler to recreate a module file and handle it that way? I guess the main difficulty is for the debugging. The generated module will need to be annotated to tell where to find the initial line of code and there are probably some other details of implementation. Anyway what do you think about it? - beno?t On Thu, 25 Feb 2016 at 04:00, Richard A. O'Keefe wrote: > > > On 25/02/16 1:53 pm, Lo?c Hoguin wrote: > > > > A similar but larger pain point that's been rationalized by people > > used to Erlang is the expression separators. > > I take it you're referring to commas, semicolons, and full stops. > > We have a complete solution to that called LFE. > > We also have Elixir, which has enough syntactic weirdness of its own > that I decided not to use it. > > I briefly played with a Haskell-inspired syntax for Erlang, which I still > think would be a good idea. One thing I certainly kept in that design > was -module directives because of the good they do. > > I wouldn't say that I've *rationalised* the issue, just that I stopped > falling off that particular bike very very quickly. > > I don't think there is any point in trying to "fix" Erlang syntax as it > stands. I *do* think there is point in developing a whole *new* > self-consistent > syntax from the ground up, just like D didn't so much fix C's syntax as > replace it. > > (By the way, am I the only person here who remembers that Algol 68 > used ";" for sequence and "," for parallel? Hands up everyone on the > list who has an Algol 68 implementation on their machine.) > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Thu Feb 25 09:02:21 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 25 Feb 2016 08:02:21 +0000 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <1bd6dfe7-3499-5030-7ce2-e651db2431d8@cs.otago.ac.nz> <56CE506C.1030700@ninenines.eu> <20b85c19-876e-3a75-38f3-394933d1edce@cs.otago.ac.nz> Message-ID: also about namespacing that's indeed quite annoying when you have to rename your filed just because you adding a dependancy to a vendor that contain a module with the same name... Or if you want to use your patched module over the erlang core (eg ssl). We already have way to point to includes files using relative paths. would be cool to extend it to the import instruction and use paths like in go. Then using the dot internally to distinct from the ":" . One bonus point is that it would encourage peiple to create libs without to care that much if someone is already using that name... - beno?t On Thu, 25 Feb 2016 at 08:24, Benoit Chesneau wrote: > i'm not sure if people complained but that the second time at least that > such topic fall on the mailing list to my knowledge. > > One sure thing though is that people using elixir like the possibility to > mix multiples modules in one file . They find it convenient. At the end > multiples beam files will be created by the elixir compiler(builder?) . > Convenience is important when you have to code like we do today. > > I don't really understand all the complexity in that thread. Reading Joe > response i understand that the current implementation is not that flexible. > But how difficult it would be to trick the compiler to find module blocks? > (like in c++ with objects). Namespace collision can be detected at > compilation. case sensitivity is imo out of topic since anyway no real > solution exist. > > So coming back to my initial idea why having something like > > -module(b). > .. > -endmodule. > > couldd't be handeld by the compiler to recreate a module file and handle > it that way? I guess the main difficulty is for the debugging. The > generated module will need to be annotated to tell where to find the > initial line of code and there are probably some other details of > implementation. Anyway what do you think about it? > > - beno?t > On Thu, 25 Feb 2016 at 04:00, Richard A. O'Keefe > wrote: > >> >> >> On 25/02/16 1:53 pm, Lo?c Hoguin wrote: >> > >> > A similar but larger pain point that's been rationalized by people >> > used to Erlang is the expression separators. >> >> I take it you're referring to commas, semicolons, and full stops. >> >> We have a complete solution to that called LFE. >> >> We also have Elixir, which has enough syntactic weirdness of its own >> that I decided not to use it. >> >> I briefly played with a Haskell-inspired syntax for Erlang, which I still >> think would be a good idea. One thing I certainly kept in that design >> was -module directives because of the good they do. >> >> I wouldn't say that I've *rationalised* the issue, just that I stopped >> falling off that particular bike very very quickly. >> >> I don't think there is any point in trying to "fix" Erlang syntax as it >> stands. I *do* think there is point in developing a whole *new* >> self-consistent >> syntax from the ground up, just like D didn't so much fix C's syntax as >> replace it. >> >> (By the way, am I the only person here who remembers that Algol 68 >> used ";" for sequence and "," for parallel? Hands up everyone on the >> list who has an Algol 68 implementation on their machine.) >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Thu Feb 25 09:10:01 2016 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 25 Feb 2016 09:10:01 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <1bd6dfe7-3499-5030-7ce2-e651db2431d8@cs.otago.ac.nz> <56CE506C.1030700@ninenines.eu> <20b85c19-876e-3a75-38f3-394933d1edce@cs.otago.ac.nz> Message-ID: <56CEB6D9.5030604@ericsson.com> More really annoying name space problems: 1) Two binary packages/libs that I need both include the same module name. 2) Two binary packages/libs that I need must have different versions of some third lib. bengt On 02/25/2016 09:02 AM, Benoit Chesneau wrote: > also about namespacing that's indeed quite annoying when you have to > rename your filed just because you adding a dependancy to a vendor > that contain a module with the same name... Or if you want to use your > patched module over the erlang core (eg ssl). > > We already have way to point to includes files using relative paths. > would be cool to extend it to the import instruction and use paths > like in go. Then using the dot internally to distinct from the ":" . > > One bonus point is that it would encourage peiple to create libs > without to care that much if someone is already using that name... > > - beno?t > > > On Thu, 25 Feb 2016 at 08:24, Benoit Chesneau > wrote: > > i'm not sure if people complained but that the second time at > least that such topic fall on the mailing list to my knowledge. > > One sure thing though is that people using elixir like the > possibility to mix multiples modules in one file . They find it > convenient. At the end multiples beam files will be created by the > elixir compiler(builder?) . Convenience is important when you have > to code like we do today. > > I don't really understand all the complexity in that thread. > Reading Joe response i understand that the current implementation > is not that flexible. But how difficult it would be to trick the > compiler to find module blocks? (like in c++ with objects). > Namespace collision can be detected at compilation. case > sensitivity is imo out of topic since anyway no real solution exist. > > So coming back to my initial idea why having something like > > -module(b). > .. > -endmodule. > > couldd't be handeld by the compiler to recreate a module file and > handle it that way? I guess the main difficulty is for the > debugging. The generated module will need to be annotated to tell > where to find the initial line of code and there are probably some > other details of implementation. Anyway what do you think about it? > > - beno?t > On Thu, 25 Feb 2016 at 04:00, Richard A. O'Keefe > > wrote: > > > > On 25/02/16 1:53 pm, Lo?c Hoguin wrote: > > > > A similar but larger pain point that's been rationalized by > people > > used to Erlang is the expression separators. > > I take it you're referring to commas, semicolons, and full stops. > > We have a complete solution to that called LFE. > > We also have Elixir, which has enough syntactic weirdness of > its own > that I decided not to use it. > > I briefly played with a Haskell-inspired syntax for Erlang, > which I still > think would be a good idea. One thing I certainly kept in > that design > was -module directives because of the good they do. > > I wouldn't say that I've *rationalised* the issue, just that I > stopped > falling off that particular bike very very quickly. > > I don't think there is any point in trying to "fix" Erlang > syntax as it > stands. I *do* think there is point in developing a whole *new* > self-consistent > syntax from the ground up, just like D didn't so much fix C's > syntax as > replace it. > > (By the way, am I the only person here who remembers that Algol 68 > used ";" for sequence and "," for parallel? Hands up everyone > on the > list who has an Algol 68 implementation on their machine.) > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Thu Feb 25 10:05:54 2016 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 25 Feb 2016 10:05:54 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <20160222030407.GB53561@ferdmbp.local> <56CB95E1.4040309@cs.otago.ac.nz> <94a762a4-511e-00ba-eb19-7530888a13ce@cs.otago.ac.nz> <56CD5B4D.7020509@ericsson.com> Message-ID: On Thu, Feb 25, 2016 at 1:14 AM, Richard A. O'Keefe wrote: > On 24/02/16 8:27 pm, Bengt Kleberg wrote: > >> Greetings, >> >> Would it be possible to get a more elaborate explanation of what OASES >> would do? > > Map module names, grammar names, include names, &c to data sources, > so that Erlang programmers would mostly deal with portable Erlang-internal > names. >> >> Given the {module, my_app_part_subpart, "src/myapp/part/subpart.erl"} >> example below, would src/myapp/part/subpart.erl be available as the module >> my_app_part_subpart after being loaded in the VM? > > Yes. > >> How would src/myapp/part/helper.erl refer to it: my_app_part_subpart or >> subpart or ??? > > > By the name that you provided in the -module directive. > This is not a "hierarchical module names" proposal; > the OASES idea is just "referring to resources in a portable way". >> >> How would src/myapp/other/subpart.erl refer to it: my_app_part_subpart or >> part_subpart or ??? > > > By the name that you provided in the -module directive. >> >> I do not understand the "zip" thing enough to ask about it. > > > This goes back to IBM mainframes, to Unix Makefiles, and to .ez files. > > In the MVS world, you might keep the Fortran code for your project > in a "partitioned data set" (think Unix .a file) called FORTRAN, so > you would refer to FORTRAN(DGEMM) -- the "DGEMM" member > of the "FORTRAN" partitioned data set rather than a Unixy > dgemm.f. Unix makefiles let you refer to files in a .a file using > foobar.a(membername) and would automatically extract things. > > The idea is that you might want to package up a bunch of stuff in a > ZIP file (or some other kind of archive), to save space, or to make > it easier to move stuff around, or whatever. Why should you have > to extract a file from an archive in order to read it? Why should > the compiler not be able to read *directly* from the archive? Yes - precisely - I *hate* systems that create thousands of small files all over the place, why can't they tidy up their mess and stick them all in a single zip file (or whatever). The widely used convention of sticking everything in a zip file and then changing the extension to something else seems like a good idea since the majority of users will think this is a single files whose contents are unknowable (and thus cannot accidently break the system by removing files) - whereas the odd programmer among us who realise this really is a zip file can mess around with the contents. This is the basic idea of abstraction - hide everything that you don't need to know about in a single container. it's also easier to delete things - I'm fed up with applications that once installed scatter files all over the file system and have no uninstall method. Also easier to move things - if there's only one file to move things are a lot easier. /Joe > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From vladdu55@REDACTED Thu Feb 25 10:17:25 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 25 Feb 2016 10:17:25 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56CEB6D9.5030604@ericsson.com> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <1bd6dfe7-3499-5030-7ce2-e651db2431d8@cs.otago.ac.nz> <56CE506C.1030700@ninenines.eu> <20b85c19-876e-3a75-38f3-394933d1edce@cs.otago.ac.nz> <56CEB6D9.5030604@ericsson.com> Message-ID: On Thu, Feb 25, 2016 at 9:10 AM, Bengt Kleberg wrote: > 2) Two binary packages/libs that I need must have different versions of > some third lib. This isn't a namespace problem. It is a very interesting problem, though, and a difficult one. There are solutions elsewhere that one could learn from (for example, Java's OSGi), but like it was pointed out before, we have more than module names that are global and that should be compartmentalized: process registry is the most important one. Imagine if the third lib starts and registers a process - there should be two of them, and each lib instance should only know about its own process. Going back to the original question, introducing namespaces brings up similar issues: what should ?MODULE resolve to? We will probably need ?FULL_MODULE too, but when to use each? It's possible it will become necessary to convert between the short and full name of a module - how? Are macros still going to work or an introspection service is needed? I think these are definitely worth investigating, answering and solving. best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From donpedrothird@REDACTED Thu Feb 25 10:25:21 2016 From: donpedrothird@REDACTED (John Doe) Date: Thu, 25 Feb 2016 12:25:21 +0300 Subject: [erlang-questions] HTTP client for OAuth In-Reply-To: References: Message-ID: If i remember right, most of the deps in hackney are for ssl (certs etc) and idn domains support. 2016-02-24 22:01 GMT+03:00 Garrett Smith : > On Wed, Feb 24, 2016 at 10:49 AM Benoit Chesneau > wrote: > >> On Wed, Feb 24, 2016 at 4:49 PM Garrett Smith wrote: >> >>> I'm taking time off from typing -module attributes to focus on my >>> backlog of other important problems... >>> >>> I'm interfacing with various OAuth providers within an Erlang >>> application and I need a simple HTTP client. I've poked around the >>> ecosystem and have learned that HTTP clients are the new MySQL clients in >>> terms of which-one-to-freakin-use. >>> >>> As I'm using this for user auth, memory and performance are >>> considerations, though far behind stability and actual-workingness. >>> >>> (In the interest of the later, I'd traditionally wrap OS curl processes >>> because curl is known to actually work, but that _might_ not be feasible as >>> the application grows. Come to think of it, I'm going to start with a curl >>> wrapper and iterate by measuring, so never mind.) >>> >>> I'd love to learn from the field experience of others. Is anyone really, >>> really happy with the HTTP client he or she is using for semi-heavy loads >>> like user auth? >>> >> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> Well I would advise hackney of course :) The pool part is completely >> optional and it will be gone in the week release anyway. It's used as a >> large scale in some companies handing millions of connections. It provides >> you a REST client API, streaming (transparent or not) and so on. >> > > I'll probably come around to this, but I was initially surprised by the > four, or maybe it was five, even six additional apps that came along with > hackney. Are you sure it isn't an HTTP _server_? :p > > >> >> If you want to do in a minimal way, I extracted the parser in a library: >> https://github.com/benoitc/nplib/blob/master/src/nplib_http.erl >> >> and then uses it with gen_tcp. >> >> This is interesting - I'll take a look! > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Thu Feb 25 10:29:53 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 25 Feb 2016 10:29:53 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <20160222030407.GB53561@ferdmbp.local> <56CB95E1.4040309@cs.otago.ac.nz> <94a762a4-511e-00ba-eb19-7530888a13ce@cs.otago.ac.nz> <56CD5B4D.7020509@ericsson.com> Message-ID: On Thu, Feb 25, 2016 at 1:14 AM, Richard A. O'Keefe wrote: > The idea is that you might want to package up a bunch of stuff in a > ZIP file (or some other kind of archive), to save space, or to make > it easier to move stuff around, or whatever. Why should you have > to extract a file from an archive in order to read it? Why should > the compiler not be able to read *directly* from the archive? > If the compiler is supposed to read the archive, then you mean that the source files should be packed in one? I can see the use for beam files to be distributed this way, but I'm not sure about sources - if they are supposed to get edited, then we have to unpack them anyway; if not, then beams are just as good. Or am I missing something? best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Thu Feb 25 10:31:02 2016 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 25 Feb 2016 10:31:02 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56CD7BDB.3050206@rol.ru> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> Message-ID: On Wed, Feb 24, 2016 at 10:46 AM, Konstantin Vsochanov wrote: > On 2/23/16 01:30 , Richard A. O'Keefe wrote: >> I do not recall ever seeing -module coming up here before. >> I certainly don't recall a long thread like this one. >> > > Maybe because people who stumble over -module() too shy to write to > erlang mail list? > >> #!/bin/sh >> #Usage: edit module >> case "$1" in >> */*) echo "Module name may not contain /" >/dev/stderr; exit 1 ;; >> -*) echo "Module name may not start with -">/dev/stderr; exit 1 ;; >> *\'*) echo "Module name may not contain '" >/dev/stderr; exit 1 ;; >> *\\*) echo "Module name may not contain \\" >/dev/stderr; exit 1 ;; >> *) ;; >> esac >> file="$1.erl" # Beware: file system may mangle name! >> if [ ! -e "$file" ]; then >> date=`date +%Y-%m-%d` >> case `expr "$1" : '^[a-z][a-zA-Z0-9_]*$'` in >> 0) module="'$1'" ;; >> *) module="$1" ;; >> esac >> echo "%%% File : $file" >"$file" >> echo "%%% Author : $LOGNAME" >>"$file" >> echo "%%% Created: $date" >>"$file" >> echo "%%% Purpose: " >>"$file" >> echo "" >>"$file" >> echo "-module($module)." >>"$file" >> echo "" >>"$file" >> echo "-export([" >>"$file" >> echo " ])." >>"$file" >> echo "" >>"$file" >> fi >> exec ${EDITOR:-emacs} "$file" >> >> So here's how much burden -module adds to this programmer: >> instead of >> % emacs foobar.erl >> I have to type >> % edit foobar >> >> Hang on, that's LESS work. >> > > Thank you for the script, I think it will work for me. I personally > don't use emacs, but it doesn't matter. > > Reading all this long thread I must agree, the -module() is not the > worst problem in Erlang. > > Regarding modules, I think the flat space for module names is more > troubling. If you look at almost any Erlang library/application, you can > see, that file names are like > > myapp_file1.erl > myapp_file2.erl > myapp_subsystem1_file1.erl > ... > > So we manually implement hierarchic module system. It's a clear sign, > that something going wrong or is missing. No - I fundamentally disagree - I think that all module names should be unique ie in a flat name space. Why? It's all about the meanings of words - when I use a word it's meaning should not depend upon context - it would be nice if this were the case. If two people use the same word it should have the same meaning and there should be one place to look up the word. We should agree on the place to lookup the word. So for example, in English I might say all the words I'm going to use are in the Chambers English dictionary. Dictionaries have flat structures - The Second Edition of the Oxford English Dictionary has 171,476 words, in alphabetical order so it's easy to find them. If all module names are unique then we can put all the modules in one directory so we'll know where to find them - so several very difficult problems go away. This solves all these problems: 1) I know the module name but I can't find it 2) I know the module name but a module with this name is in dozens of different directories 3) I have a new module and I don't know what directory to store it in Dictionaries are fantastic because we know where to find words - file systems are horrible because we don't know where to find things. I can imagine having different dictionaries, French, German etc - so I can imaging different large collections of modules - which is akin to having a two-level namespace. With rather few top-level names (like dictionaries) If you want to simulate stucture then by all means add underscores to the names this_sub_module is fine '_' has no semantics and is purely a linguistic convention (just like this.sub.module ) /Joe > > > > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Thu Feb 25 10:37:43 2016 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 25 Feb 2016 10:37:43 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <1bd6dfe7-3499-5030-7ce2-e651db2431d8@cs.otago.ac.nz> <56CE506C.1030700@ninenines.eu> <20b85c19-876e-3a75-38f3-394933d1edce@cs.otago.ac.nz> Message-ID: On Thu, Feb 25, 2016 at 8:24 AM, Benoit Chesneau wrote: > i'm not sure if people complained but that the second time at least that > such topic fall on the mailing list to my knowledge. > > One sure thing though is that people using elixir like the possibility to > mix multiples modules in one file . They find it convenient. At the end > multiples beam files will be created by the elixir compiler(builder?) . > Convenience is important when you have to code like we do today. > > I don't really understand all the complexity in that thread. Reading Joe > response i understand that the current implementation is not that flexible. > But how difficult it would be to trick the compiler to find module blocks? > (like in c++ with objects). Namespace collision can be detected at > compilation. case sensitivity is imo out of topic since anyway no real > solution exist. > > So coming back to my initial idea why having something like > > -module(b). > .. > -endmodule. > > couldd't be handeld by the compiler to recreate a module file and handle it > that way? I guess the main difficulty is for the debugging. The generated > module will need to be annotated to tell where to find the initial line of > code and there are probably some other details of implementation. Anyway > what do you think about it? I did this a few years ago (together with some other fun stuff) https://github.com/joearms/erl2 Actually it's far easier to make a new language that compiles to Erlang than to change the existing Erlang - that's because we want backwards compatibility and tool compatibility. If you make any change like this (or omitting module declaration) you break all the tools and all the books. Eventually change becomes so difficult that adding any new features or bug fixes introduces as many problems as are solved - so a rewrite becomes necessary. Personally I hate reading books and articles on the net that say "do this" and when you do it, it does not work. Cheers /Joe > > - beno?t > > On Thu, 25 Feb 2016 at 04:00, Richard A. O'Keefe wrote: >> >> >> >> On 25/02/16 1:53 pm, Lo?c Hoguin wrote: >> > >> > A similar but larger pain point that's been rationalized by people >> > used to Erlang is the expression separators. >> >> I take it you're referring to commas, semicolons, and full stops. >> >> We have a complete solution to that called LFE. >> >> We also have Elixir, which has enough syntactic weirdness of its own >> that I decided not to use it. >> >> I briefly played with a Haskell-inspired syntax for Erlang, which I still >> think would be a good idea. One thing I certainly kept in that design >> was -module directives because of the good they do. >> >> I wouldn't say that I've *rationalised* the issue, just that I stopped >> falling off that particular bike very very quickly. >> >> I don't think there is any point in trying to "fix" Erlang syntax as it >> stands. I *do* think there is point in developing a whole *new* >> self-consistent >> syntax from the ground up, just like D didn't so much fix C's syntax as >> replace it. >> >> (By the way, am I the only person here who remembers that Algol 68 >> used ";" for sequence and "," for parallel? Hands up everyone on the >> list who has an Algol 68 implementation on their machine.) >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From on-your-mark@REDACTED Thu Feb 25 11:06:03 2016 From: on-your-mark@REDACTED (YuanZhiqian) Date: Thu, 25 Feb 2016 18:06:03 +0800 Subject: [erlang-questions] Alternative tool to replace appmon Message-ID: Hi guys, I found that appmon is removed from Erlang release since R17, and I wonder if there is any alternative tool that I can use for the same purpose. Appmon is a very handy tool to inspect inside OTP apps' hierarchy, and I do wish that this functionality still available in the newest releases. Besides, I also would like to know if there's any such tools working under command line, I have no idea how I can use tools like appmon remotely, if only there's a command-line version, that would be convenient for me to ssh to the remote node and get all the info I want with it. Kind regardsZhiqian -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Thu Feb 25 11:14:10 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 25 Feb 2016 10:14:10 +0000 Subject: [erlang-questions] HTTP client for OAuth In-Reply-To: References: Message-ID: Yes so there are external modules to handle the following: - IDNA - SSL certificates access (certifi) - SSL Host validation - Mimes handling (simple module providing a way to retrieves mimes headers from a file extension and the other way) Over the time the codebase have grown without much planning to answer to more demands. Too much for my taste I would say. The coming version that must land this week tends to more simplicity and less dependencies. The way I see hackney is as a toolbox in which you can pick what you need to interact with the web and the web of things. It means anything to interact with HTTP(1.1,2), Websockets and others specs like COAP once I figured how to handle DTLS properly in Erlang. So in short the next version is simple but will keep the current API for the HTTP requests so going from hackney 1 to 2 shouldn't be hard. - beno?t On Thu, Feb 25, 2016 at 10:25 AM John Doe wrote: > If i remember right, most of the deps in hackney are for ssl (certs etc) > and idn domains support. > > 2016-02-24 22:01 GMT+03:00 Garrett Smith : > >> On Wed, Feb 24, 2016 at 10:49 AM Benoit Chesneau >> wrote: >> >>> On Wed, Feb 24, 2016 at 4:49 PM Garrett Smith wrote: >>> >>>> I'm taking time off from typing -module attributes to focus on my >>>> backlog of other important problems... >>>> >>>> I'm interfacing with various OAuth providers within an Erlang >>>> application and I need a simple HTTP client. I've poked around the >>>> ecosystem and have learned that HTTP clients are the new MySQL clients in >>>> terms of which-one-to-freakin-use. >>>> >>>> As I'm using this for user auth, memory and performance are >>>> considerations, though far behind stability and actual-workingness. >>>> >>>> (In the interest of the later, I'd traditionally wrap OS curl processes >>>> because curl is known to actually work, but that _might_ not be feasible as >>>> the application grows. Come to think of it, I'm going to start with a curl >>>> wrapper and iterate by measuring, so never mind.) >>>> >>>> I'd love to learn from the field experience of others. Is anyone >>>> really, really happy with the HTTP client he or she is using for semi-heavy >>>> loads like user auth? >>>> >>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> Well I would advise hackney of course :) The pool part is completely >>> optional and it will be gone in the week release anyway. It's used as a >>> large scale in some companies handing millions of connections. It provides >>> you a REST client API, streaming (transparent or not) and so on. >>> >> >> I'll probably come around to this, but I was initially surprised by the >> four, or maybe it was five, even six additional apps that came along with >> hackney. Are you sure it isn't an HTTP _server_? :p >> >> >>> >>> If you want to do in a minimal way, I extracted the parser in a library: >>> https://github.com/benoitc/nplib/blob/master/src/nplib_http.erl >>> >>> and then uses it with gen_tcp. >>> >>> This is interesting - I'll take a look! >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Thu Feb 25 11:35:39 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 25 Feb 2016 10:35:39 +0000 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <1bd6dfe7-3499-5030-7ce2-e651db2431d8@cs.otago.ac.nz> <56CE506C.1030700@ninenines.eu> <20b85c19-876e-3a75-38f3-394933d1edce@cs.otago.ac.nz> Message-ID: On Thu, Feb 25, 2016 at 10:37 AM Joe Armstrong wrote: > On Thu, Feb 25, 2016 at 8:24 AM, Benoit Chesneau > wrote: > > i'm not sure if people complained but that the second time at least that > > such topic fall on the mailing list to my knowledge. > > > > One sure thing though is that people using elixir like the possibility to > > mix multiples modules in one file . They find it convenient. At the end > > multiples beam files will be created by the elixir compiler(builder?) . > > Convenience is important when you have to code like we do today. > > > > I don't really understand all the complexity in that thread. Reading Joe > > response i understand that the current implementation is not that > flexible. > > But how difficult it would be to trick the compiler to find module > blocks? > > (like in c++ with objects). Namespace collision can be detected at > > compilation. case sensitivity is imo out of topic since anyway no real > > solution exist. > > > > So coming back to my initial idea why having something like > > > > -module(b). > > .. > > -endmodule. > > > > couldd't be handeld by the compiler to recreate a module file and handle > it > > that way? I guess the main difficulty is for the debugging. The generated > > module will need to be annotated to tell where to find the initial line > of > > code and there are probably some other details of implementation. Anyway > > what do you think about it? > > I did this a few years ago (together with some other fun stuff) > > https://github.com/joearms/erl2 > > > Actually it's far easier to make a new language that compiles to Erlang > than > to change the existing Erlang - that's because we want backwards > compatibility > and tool compatibility. > > If you make any change like this (or omitting module declaration) you break > all the tools and all the books. > > Eventually change becomes so difficult that adding any new features > or bug fixes introduces as many problems as are solved - so a rewrite > becomes necessary. > > Personally I hate reading books and articles on the net that say "do this" > and when you do it, it does not work. > > Cheers > > /Joe > > Oh I didn't know it was doing it, will reread... Anyway I wasn't thinking to break the current API but expanding it. Not sure if it can be done in a backward compatible way. Need to think more about it. But yeah I also like to not have to look on the web what is the new way to do the thing that was described in the book. Also I am not very annoyed by having to create different "files" for a purpose and keep related things at the same place. It seems at then end the right way to think about a problem. The real thing that annoys me today is having to prefix all my files in my project where i just want to have a `btree.erl`file and call `btree:...` just to make sure it won't conflict with another vendor when used in another project. Where a "simple" namespacing based on the vendor directory would work. - beno?t -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Thu Feb 25 11:36:22 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Thu, 25 Feb 2016 11:36:22 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <20b85c19-876e-3a75-38f3-394933d1edce@cs.otago.ac.nz> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <1bd6dfe7-3499-5030-7ce2-e651db2431d8@cs.otago.ac.nz> <56CE506C.1030700@ninenines.eu> <20b85c19-876e-3a75-38f3-394933d1edce@cs.otago.ac.nz> Message-ID: <56CED926.9080803@ninenines.eu> On 02/25/2016 04:00 AM, Richard A. O'Keefe wrote: > On 25/02/16 1:53 pm, Lo?c Hoguin wrote: >> >> A similar but larger pain point that's been rationalized by people >> used to Erlang is the expression separators. > > I take it you're referring to commas, semicolons, and full stops. Yes. > We have a complete solution to that called LFE. I largely prefer the Erlang syntax despite its faults, at least for most of the code I write. > We also have Elixir, which has enough syntactic weirdness of its own > that I decided not to use it. We agree there. > I briefly played with a Haskell-inspired syntax for Erlang, which I still > think would be a good idea. One thing I certainly kept in that design > was -module directives because of the good they do. My problem isn't that the -module directive exists, just that it's mandatory. It's good that you are protected from every possible error if that's what you want. But it's not good that you are forcefully restricted to a particular behavior just because someone somewhere might run into issues otherwise. This reminds me of the Unicode variable/function names debate, where people argue that they might one day run into code that they can't read. > I wouldn't say that I've *rationalised* the issue, just that I stopped > falling off that particular bike very very quickly. I still run into this, mostly when refactoring. When moving code around I usually end up with wrong separators. It's often in lists or maps when writing one item per line. If I could have an optional comma for the last item in the list I wouldn't have to compile, get an error, fix it and compile again. I did rationalize it, "it's like sentences!" and you probably did too. If you really never run into issues with them you either never refactor code, do tools assisted refactoring or are smarter than everyone else. > I don't think there is any point in trying to "fix" Erlang syntax as it > stands. I *do* think there is point in developing a whole *new* > self-consistent > syntax from the ground up, just like D didn't so much fix C's syntax as > replace it. I don't think any of the things discussed here break backward compatibility, or if they do, not as much as some of the changes OTP team already does (some latin1 source files broke in 17 for example). -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From Maxim.Minin@REDACTED Thu Feb 25 12:53:15 2016 From: Maxim.Minin@REDACTED (Minin Maxim) Date: Thu, 25 Feb 2016 12:53:15 +0100 Subject: [erlang-questions] Alternative tool to replace appmon In-Reply-To: References: Message-ID: You can use observer - see http://erlang.org/doc/man/observer.html best regards maxim Von: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] Im Auftrag von YuanZhiqian Gesendet: Donnerstag, 25. Februar 2016 11:06 An: erlang-questions@REDACTED Betreff: [erlang-questions] Alternative tool to replace appmon Hi guys, I found that appmon is removed from Erlang release since R17, and I wonder if there is any alternative tool that I can use for the same purpose. Appmon is a very handy tool to inspect inside OTP apps' hierarchy, and I do wish that this functionality still available in the newest releases. Besides, I also would like to know if there's any such tools working under command line, I have no idea how I can use tools like appmon remotely, if only there's a command-line version, that would be convenient for me to ssh to the remote node and get all the info I want with it. Kind regards Zhiqian -------------- next part -------------- An HTML attachment was scrubbed... URL: From Ola.Backstrom@REDACTED Thu Feb 25 13:47:20 2016 From: Ola.Backstrom@REDACTED (=?iso-8859-1?Q?Ola_B=E4ckstr=F6m?=) Date: Thu, 25 Feb 2016 12:47:20 +0000 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <20160224150705.GA90042@fhebert-ltm2.internal.salesforce.com> References: <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <20160224150705.GA90042@fhebert-ltm2.internal.salesforce.com> Message-ID: <24b9b71226d649598cb2e5514edf7fe0@Jar-EX01.ls.local> >From this long discussion I'd like to highlight that namespace handling is larger than just filenames, and that just "fixing" the filename collisions is not good enough. I've digged up two relevant quotes: 2016-02-24 16:07 Fred Hebert wrote: > [...snip...] > It is, but it has some far-reaching implications that are easy to forget about when you don't do namespacing by hand. For example, ETS tables and processes can both be named, > and if you do not manually namespace them, the same clashes can happen as would with modules. [...snip...] 2016-02-25 10:17 Vlad Dumitrescu wrote: > [...snip...] but like it was pointed out before, we have more than module names that are global and that should be compartmentalized: process registry is the most important one. > Imagine if the third lib starts and registers a process - there should be two of them, and each lib instance should only know about its own process. [...snip...] In Erlang all atoms are put in a global namespace so every reference to, for instance, the atom 'ok' is referring to the same atom in an Erlang node (or even cluster of Erlang nodes). This is indeed what we'd like for that atom. Now in some cases we'd like to register a process or an ETS-table and in fact we'd prefer that it is only referenced by our own source code, aka "global but secret". Here's where name clashes can occur, if foreign modules use the same atom. Common Lisp resolved this a long time ago, albeit in a - at least for me - complex way. But perhaps the underlying ideas could be used: * namespaced symbols and * global keywords If I remember correctly the keywords are equivalent to Erlang atoms. The existence of two different sorts of atoms are of fundamental importance to the lisp package system and enables lispers to code in different name spaces. Regards /Ola From marat@REDACTED Thu Feb 25 11:10:10 2016 From: marat@REDACTED (Marat Kalibekov) Date: Thu, 25 Feb 2016 16:10:10 +0600 Subject: [erlang-questions] Starting new library Message-ID: Hello, I want to create new open source library. Is there any requirements/advices how to do it? Or there any tool which generates initial library files for you? -- Kalibekov Marat -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethlakin@REDACTED Thu Feb 25 17:27:24 2016 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Thu, 25 Feb 2016 08:27:24 -0800 Subject: [erlang-questions] Starting new library In-Reply-To: References: Message-ID: <56CF2B6C.3080100@gmail.com> On 02/25/2016 02:10 AM, Marat Kalibekov wrote: > Or there any tool which generates initial library files for you? rebar3 new lib $LIBNAME and rebar create-lib libid=$LIBNAME will give you library skeletons in rebar3 and rebar, respectively. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From mkbucc@REDACTED Thu Feb 25 17:33:44 2016 From: mkbucc@REDACTED (Mark Bucciarelli) Date: Thu, 25 Feb 2016 11:33:44 -0500 Subject: [erlang-questions] Starting new library In-Reply-To: References: Message-ID: <20160225163344.GA40315@marks-mbp-3> On Thu, Feb 25, 2016 at 04:10:10PM +0600, Marat Kalibekov wrote: > > Hello, I want to create new open source library. > Is there any requirements/advices how to do it? > Stuff Goes Bad: Erlang in Anger Chapter 2. Building Open Source Erlang Software http://www.erlang-in-anger.com/ Mark From marat@REDACTED Thu Feb 25 17:35:33 2016 From: marat@REDACTED (Marat Kalibekov) Date: Thu, 25 Feb 2016 22:35:33 +0600 Subject: [erlang-questions] Starting new library In-Reply-To: <20160225163344.GA40315@marks-mbp-3> References: <20160225163344.GA40315@marks-mbp-3> Message-ID: Thank you Kenneth and Mark! -- Kalibekov Marat 8 777 4826322 On Thu, Feb 25, 2016 at 10:33 PM, Mark Bucciarelli wrote: > On Thu, Feb 25, 2016 at 04:10:10PM +0600, Marat Kalibekov wrote: > > > > Hello, I want to create new open source library. > > Is there any requirements/advices how to do it? > > > > Stuff Goes Bad: Erlang in Anger > Chapter 2. Building Open Source Erlang Software > http://www.erlang-in-anger.com/ > > Mark > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From a.shneyderman@REDACTED Thu Feb 25 20:28:12 2016 From: a.shneyderman@REDACTED (Alex Shneyderman) Date: Thu, 25 Feb 2016 14:28:12 -0500 Subject: [erlang-questions] Alternative tool to replace appmon In-Reply-To: References: Message-ID: observer is an alternative as the other poster noted. > Besides, I also would like to know if there's any such tools working under > command line, I have no idea how I can use tools like appmon remotely, if > only there's a command-line version, that would be convenient for me to ssh > to the remote node and get all the info I want with it. https://mfeckie.github.io/Remote-Profiling-Elixir-Over-SSH/ https://gist.github.com/pnc/9e957e17d4f9c6c81294 to connect observer to remote node. From ok@REDACTED Thu Feb 25 23:15:27 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 26 Feb 2016 11:15:27 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <20160222030407.GB53561@ferdmbp.local> <56CB95E1.4040309@cs.otago.ac.nz> <94a762a4-511e-00ba-eb19-7530888a13ce@cs.otago.ac.nz> <56CD5B4D.7020509@ericsson.com> Message-ID: <8ca2b006-e0fb-491a-ac3c-a132cf7e0e71@cs.otago.ac.nz> On 25/02/16 10:29 pm, Vlad Dumitrescu wrote: > > If the compiler is supposed to read the archive, then you mean that > the source files should be packed in one? "Should" is up to the programmer. What I'm advocating is "could". > I can see the use for beam files to be distributed this way, but I'm > not sure about sources - if they are supposed to get edited, then we > have to unpack them anyway; Well, if it comes to that, there's no reason why the *editor* couldn't work out of a zip file. Try this: I get the Erlang/OTP sources, and I love that, and I *could* edit them if I needed to. Indeed, on very rare occasions I have. But most of the time I *don't* edit them. Or to put it another way, I have a project in another language. Out of nearly 1800 files, nearly 45 of them have been changed in the last week. That's one file out of 40. Why should 39 files be just so I can edit 1 out of 40? Even now, if you want to edit a file, you have to unpack IT, not THEM. Then too, there are lots of things one can do with source files besides compile and edit them. > if not, then beams are just as good. I explicitly indicated that OASES files might point to things other than modules. I believe it to already be the case that people are allowed to put other things than modules in .ez files. > Or am I missing something? > > best regards, > Vlad > From raould@REDACTED Thu Feb 25 23:24:34 2016 From: raould@REDACTED (Raoul Duke) Date: Thu, 25 Feb 2016 14:24:34 -0800 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <24b9b71226d649598cb2e5514edf7fe0@Jar-EX01.ls.local> References: <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <20160224150705.GA90042@fhebert-ltm2.internal.salesforce.com> <24b9b71226d649598cb2e5514edf7fe0@Jar-EX01.ls.local> Message-ID: > * namespaced symbols and > * global keywords the globals should just use a standard namespace, "GLOBAL://" (or whatever :-) From ok@REDACTED Thu Feb 25 23:24:48 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 26 Feb 2016 11:24:48 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> Message-ID: If all module names are unique then we can put all the modules in one directory > so we'll know where to find them - so several very difficult problems go away. Let me mention one problem with Erlang/OTP as it stands, which to me *DWARFS* every syntax issue and always has. One of the charges that always used to be laid against the UNIX V7 manuals was that the distinction between volume 2 (system calls) and volume 3 (library) was utterly opaque to most programmers. In precisely the same way, I have always found it annoying that we have Erlang documentation Application Groups Basic compiler erts kernel **erlang and file are here sasl stdlib **lists and io are here I constantly find myself flipflopping between kernel and stdlib grumbling "where did they hid THIS one?" Sigh. I know what I should do. I should stop kvetching and make my *own* web page with precisely the links I need, just visiting the official page when I need to add something new. But honestly, the distinction between kernel and stdlib really seems to be uncannily like the distinction between UNIX volume 2 and UNIX volume 3, and *just as unhelpful*. One thing about the Java online API documentation, you get a scrolling list with the name of *every* class. From ok@REDACTED Thu Feb 25 23:26:22 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 26 Feb 2016 11:26:22 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <1bd6dfe7-3499-5030-7ce2-e651db2431d8@cs.otago.ac.nz> <56CE506C.1030700@ninenines.eu> <20b85c19-876e-3a75-38f3-394933d1edce@cs.otago.ac.nz> Message-ID: <9916b074-3fac-fdf1-e6df-19eb1ef6f5fd@cs.otago.ac.nz> On 25/02/16 10:37 pm, Joe Armstrong wrote: > > Personally I hate reading books and articles on the net that say "do this" > and when you do it, it does not work. Hah! I got my hands on a Java textbook that promised to be helpful a couple of years ago, and it had on-line code examples I could download. EVERY SINGLE ONE OF THEM produced a flood of deprecation warnings from the compiler. Not impressed. From raould@REDACTED Thu Feb 25 23:30:48 2016 From: raould@REDACTED (Raoul Duke) Date: Thu, 25 Feb 2016 14:30:48 -0800 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <9916b074-3fac-fdf1-e6df-19eb1ef6f5fd@cs.otago.ac.nz> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <1bd6dfe7-3499-5030-7ce2-e651db2431d8@cs.otago.ac.nz> <56CE506C.1030700@ninenines.eu> <20b85c19-876e-3a75-38f3-394933d1edce@cs.otago.ac.nz> <9916b074-3fac-fdf1-e6df-19eb1ef6f5fd@cs.otago.ac.nz> Message-ID: > Hah! I got my hands on a Java textbook that promised to be helpful a couple > of > years ago, and it had on-line code examples I could download. > > EVERY SINGLE ONE OF THEM produced a flood of deprecation warnings > from the compiler. Not impressed. Things change. People die. It's sad. And, of course, agreed, is not good for end-user happiness. Still, some of it is not too surprising, really. If our languages actually required us to stipulate in metadata/annotations what version is to be used, and then you still got the warnings, then we could be really super justifiably mad, I think. But most languages are too lame and do not require explicit version statements afaik. In other words: 5 whys, root cause, yadda^3. From raould@REDACTED Thu Feb 25 23:48:28 2016 From: raould@REDACTED (Raoul Duke) Date: Thu, 25 Feb 2016 14:48:28 -0800 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <1bd6dfe7-3499-5030-7ce2-e651db2431d8@cs.otago.ac.nz> <1dd9d665-fa46-aaf2-0a5e-5ed34fb1bb11@cs.otago.ac.nz> Message-ID: > How could I make it more explicit? > A Haskell module can refer to another module by one (or more!) internal > names > which are in no way coupled to the file system and are in no way coupled to > the > name the module believes it has. The name a module has *is* coupled to the > file system. What I was meaning was not that you aren't explicit, but that on the whole I don't think it is so bad that it isn't perhaps worth considering as better than the current status quo. Food for thought. > Summary: there are several "OK, this is just tolerable, we can limp along > with this" hierarchical module systems out there, but as yet, no outstanding > "wow, gotta have this", especially for a dynamic distributed language. What are the root causes? + People want to use "human" names for things. + We want unambiguity. (Too bad we don't include versions in the names?) + People do not want to have to type a lot of ascii: labor saving usability; somewhat about readability, too. + We shouldn't have to alter the ascii of an already existing program just because somebody added a new class in one of the 3rd party libs that's used. + etc. So far I think the "let users rename their namespace" isn't all that bad. And I don't really like good-enough preventing the-perfect, so I say that not as some fan boy of the approach, I just also haven't seen or thought of anything better. It is an interesting thing to ponder. From vladdu55@REDACTED Fri Feb 26 00:09:58 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 26 Feb 2016 00:09:58 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <8ca2b006-e0fb-491a-ac3c-a132cf7e0e71@cs.otago.ac.nz> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <20160222030407.GB53561@ferdmbp.local> <56CB95E1.4040309@cs.otago.ac.nz> <94a762a4-511e-00ba-eb19-7530888a13ce@cs.otago.ac.nz> <56CD5B4D.7020509@ericsson.com> <8ca2b006-e0fb-491a-ac3c-a132cf7e0e71@cs.otago.ac.nz> Message-ID: Hi, On Thu, Feb 25, 2016 at 11:15 PM, Richard A. O'Keefe wrote: > > > On 25/02/16 10:29 pm, Vlad Dumitrescu wrote: > >> >> If the compiler is supposed to read the archive, then you mean that the >> source files should be packed in one? >> > "Should" is up to the programmer. What I'm advocating is "could". > > I can see the use for beam files to be distributed this way, but I'm not >> sure about sources - if they are supposed to get edited, then we have to >> unpack them anyway; >> > Well, if it comes to that, there's no reason why the *editor* couldn't > work out of > a zip file. Try this: I get the Erlang/OTP sources, and I love that, and > I *could* > edit them if I needed to. Indeed, on very rare occasions I have. But most > of > the time I *don't* edit them. Or to put it another way, I have a project > in another > language. Out of nearly 1800 files, nearly 45 of them have been changed in > the last week. That's one file out of 40. Why should 39 files be > just so I can edit 1 out of 40? > > Even now, if you want to edit a file, you have to unpack IT, not THEM. > > Then too, there are lots of things one can do with source files besides > compile > and edit them. I see your point. For me a directory is easier to work with than an archive, when it comes to source code, but it's just me. regards, vlad > > if not, then beams are just as good. >> > I explicitly indicated that OASES files might point to things other than > modules. > I believe it to already be the case that people are allowed to put other > things > than modules in .ez files. > > > > > > Or am I missing something? >> >> best regards, >> Vlad >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Feb 26 00:38:02 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 26 Feb 2016 12:38:02 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <56CED926.9080803@ninenines.eu> References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <1bd6dfe7-3499-5030-7ce2-e651db2431d8@cs.otago.ac.nz> <56CE506C.1030700@ninenines.eu> <20b85c19-876e-3a75-38f3-394933d1edce@cs.otago.ac.nz> <56CED926.9080803@ninenines.eu> Message-ID: <51bafabb-0c33-504e-542e-4a81adab43a6@cs.otago.ac.nz> On 25/02/16 11:36 pm, Lo?c Hoguin wrote: > > I still run into this, mostly when refactoring. When moving code > around I usually end up with wrong separators. It's often in lists or > maps when writing one item per line. If I could have an optional comma > for the last item in the list I wouldn't have to compile, get an > error, fix it and compile again. I think I wrote an EEP about that. (Search search) Yes, EEP 21, "Optional trailing comma for lists and tuples" These days it should mention records and maps as well. > I did rationalize it, "it's like sentences!" and you probably did too. Actually, I thought "it's like Prolog." Banged into a wall a couple of times: "EXCEPT that most clauses end with a semicolon instead of a full stop." Coming from Prolog meant that my personal style was to NEVER put semicolons at the end of lines. Maybe I should lay functions out like foo(...) -> ... ; foo(...) -> ... ; foo(...) -> ... . My actual preference would be to simply say that an atom or hyphen in column 1 implies either a semicolon or full stop before it, whichever is appropriate. > If you really never run into issues with them you either never > refactor code, do tools assisted refactoring or are smarter than > everyone else. There's another possibility: I might refactor, but just be too bad at it for this issue to be significant. (:-) From ok@REDACTED Fri Feb 26 00:52:38 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 26 Feb 2016 12:52:38 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: <24b9b71226d649598cb2e5514edf7fe0@Jar-EX01.ls.local> References: <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <20160224150705.GA90042@fhebert-ltm2.internal.salesforce.com> <24b9b71226d649598cb2e5514edf7fe0@Jar-EX01.ls.local> Message-ID: On 26/02/16 1:47 am, Ola B?ckstr?m wrote: > In Erlang all atoms are put in a global namespace so every reference > to, for instance, the atom 'ok' is referring to the same atom in an > Erlang node (or even cluster of Erlang nodes). This is indeed what > we'd like for that atom. Now in some cases we'd like to register a > process or an ETS-table and in fact we'd prefer that it is only > referenced by our own source code, aka "global but secret". Here's > where name clashes can occur, if foreign modules use the same atom. By golly, I must have been urging the introduction of module-local pidnames for over 20 years by now. Almost all process registrations should be local to the module that registers them. I proposed -pidname(). for declaring a module-locale atomic variable to hold pids. This would be both safer than the global registry and more efficient. The global registry would remain for those things that genuinely need to be global. I'll have to see if I can dig this out. I thought I'd written it up as an EEP, but it's not in the catalogue. > Common Lisp resolved this a long time ago, albeit in a - at least for me - complex way. But perhaps the underlying ideas could be used: > * namespaced symbols and > * global keywords > If I remember correctly the keywords are equivalent to Erlang atoms. The existence of two different sorts of atoms are of fundamental importance to the lisp package system and enables lispers to code in different name spaces. The Common Lisp solution was in my experience complex and confusing. Thinking that there are two kinds of atoms is one way of being confused. One issue, of course, is that Common Lisp packages are not modules. Very far from it, in fact. From raould@REDACTED Fri Feb 26 01:02:48 2016 From: raould@REDACTED (Raoul Duke) Date: Thu, 25 Feb 2016 16:02:48 -0800 Subject: [erlang-questions] flat namespacing (Re: Why we need a -module() attribute?) Message-ID: > No - I fundamentally disagree - I think that all module names should be unique > ie in a flat name space. Uniqueness doesn't /have/ to mean a flat namespace. > Why? It's all about the meanings of words - when I use a word it's > meaning should > not depend upon context - it would be nice if this were the case. There are many benefits to using paths, not just words, for namespacing. From on-your-mark@REDACTED Fri Feb 26 01:06:56 2016 From: on-your-mark@REDACTED (YuanZhiqian) Date: Fri, 26 Feb 2016 08:06:56 +0800 Subject: [erlang-questions] Alternative tool to replace appmon In-Reply-To: References: Message-ID: Thank you so much! ???? iPhone > ? 2016?2?26??03:28?Alex Shneyderman ??? > > observer is an alternative as the other poster noted. > >> Besides, I also would like to know if there's any such tools working under >> command line, I have no idea how I can use tools like appmon remotely, if >> only there's a command-line version, that would be convenient for me to ssh >> to the remote node and get all the info I want with it. > > https://mfeckie.github.io/Remote-Profiling-Elixir-Over-SSH/ > https://gist.github.com/pnc/9e957e17d4f9c6c81294 > > to connect observer to remote node. From ok@REDACTED Fri Feb 26 01:32:40 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 26 Feb 2016 13:32:40 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <1bd6dfe7-3499-5030-7ce2-e651db2431d8@cs.otago.ac.nz> <56CE506C.1030700@ninenines.eu> <20b85c19-876e-3a75-38f3-394933d1edce@cs.otago.ac.nz> <9916b074-3fac-fdf1-e6df-19eb1ef6f5fd@cs.otago.ac.nz> Message-ID: <591933db-cdc9-07d1-1b2f-dfc8b6a20e5e@cs.otago.ac.nz> On 26/02/16 11:30 am, Raoul Duke wrote: >> Hah! I got my hands on a Java textbook that promised to be helpful a couple >> of >> years ago, and it had on-line code examples I could download. >> >> EVERY SINGLE ONE OF THEM produced a flood of deprecation warnings >> from the compiler. Not impressed. > Things change. In two years? With a deprecation system like the one in the Mac OS X headers, all you need is a compiler option "compile like it was ". From steven.charles.davis@REDACTED Fri Feb 26 01:38:34 2016 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 25 Feb 2016 18:38:34 -0600 Subject: [erlang-questions] Why we need a -module() attribute? Message-ID: It works. It?s worked for years. Works is good. Being lazy is good as a developer if the end is to produce clearer functions. It is not to avoid a second or two of typing at a keyboard ? for sure you should always be writing code for the *maintainer* not for yourself (though you may indeed be that actor too). Erlang has so little ?boilerplate? that this discussion seems entirely pointless (IMHO). If you wish to organize your source for convenience consider an Emakefile with a couple of lines like % {"src/*", [{i, "include"}, {outdir, "ebin"}, debug_info, strict_record_tests]}. {"src/*/*", [{i, "include"}, {outdir, "ebin"}, debug_info, strict_record_tests]}. {"src/*/*/*", [{i, "include"}, {outdir, "ebin"}, debug_info, strict_record_tests]}. Yes, still the modules need to have unique names but surely duplicated module names will always lead to confused code eventually. My 2c /s From steven.charles.davis@REDACTED Fri Feb 26 01:52:31 2016 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 25 Feb 2016 18:52:31 -0600 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: Message-ID: BTW has anyone considered that this ?issue? if ?solved? would be the same solution that would solve the irksome pains caused by relative file & URI paths too? I suspect that a true solution would resolve all of that and be a major contribution to computer science? I?m listening :) > On Feb 25, 2016, at 6:38 PM, Steve Davis wrote: > > It works. It?s worked for years. Works is good. > > Being lazy is good as a developer if the end is to produce clearer functions. It is not to avoid a second or two of typing at a keyboard ? for sure you should always be writing code for the *maintainer* not for yourself (though you may indeed be that actor too). > > Erlang has so little ?boilerplate? that this discussion seems entirely pointless (IMHO). > > If you wish to organize your source for convenience consider an Emakefile with a couple of lines like > > % > {"src/*", [{i, "include"}, {outdir, "ebin"}, debug_info, strict_record_tests]}. > {"src/*/*", [{i, "include"}, {outdir, "ebin"}, debug_info, strict_record_tests]}. > {"src/*/*/*", [{i, "include"}, {outdir, "ebin"}, debug_info, strict_record_tests]}. > > Yes, still the modules need to have unique names but surely duplicated module names will always lead to confused code eventually. > > My 2c > > /s From ok@REDACTED Fri Feb 26 02:47:29 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 26 Feb 2016 14:47:29 +1300 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <20160222030407.GB53561@ferdmbp.local> <56CB95E1.4040309@cs.otago.ac.nz> <94a762a4-511e-00ba-eb19-7530888a13ce@cs.otago.ac.nz> <56CD5B4D.7020509@ericsson.com> <8ca2b006-e0fb-491a-ac3c-a132cf7e0e71@cs.otago.ac.nz> Message-ID: <0c72785c-8a46-7a11-4fe4-f945d58e78fb@cs.otago.ac.nz> On 26/02/16 12:09 pm, Vlad Dumitrescu wrote: I see your point. For me a directory is easier to work with than an archive, when it comes to source code, but it's just me. The ideal situation would in fact be for the operating system to be able to mount a compressed archive as a directory. ("It's a floor wax AND a desert topping!") (Packing the 1600 source files -- which are in several directories -- into a compressed archive reduces the space to 21% and that means less time when looking at them too.) I believe Windows can do that out of the box. fuse-zip can do it for Linux, and apparently it works with OSXFUSE, the successor to MacFUSE. I don't know if there's anything like this for OpenBSD or Solaris. I once chose to unpack an archive with oodles of data files instead of taking the time to write a program reading directly out of the archive. One of the stupidest decisions I ever made in programming, and I've made quite a few. From valentin@REDACTED Fri Feb 26 07:02:13 2016 From: valentin@REDACTED (Valentin Micic) Date: Fri, 26 Feb 2016 08:02:13 +0200 Subject: [erlang-questions] -pidname( ) Was: Why we need a -module() attribute? In-Reply-To: References: <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <20160224150705.GA90042@fhebert-ltm2.internal.salesforce.com> <24b9b71226d649598cb2e5514edf7fe0@Jar-EX01.ls.local> Message-ID: <2804FE6E-214B-4076-B2F2-C49F9500E8BA@pixie.co.za> > Almost all process registrations should be local to the module that registers > them. I proposed > > -pidname(). > > for declaring a module-locale atomic variable to hold pids. This would be both > safer than the global registry and more efficient. The global registry would > remain for those things that genuinely need to be global. Presuming that the module name is , say, test, how is: -pidname( test ) different from: register( ?MODULE, self() )? Also, how would you run multiple instances of the same module within the same run-time? Your proposal may appear to solve one problem (that is, if one chose to call it a problem), but appears to introduce at least one more. V/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From mhssler@REDACTED Fri Feb 26 07:15:19 2016 From: mhssler@REDACTED (=?UTF-8?Q?Martin_H=C3=A4ssler?=) Date: Fri, 26 Feb 2016 07:15:19 +0100 Subject: [erlang-questions] Erlang for managers Message-ID: My manager's new manager has asked me for a high level summary of Erlang compared with other languages, especially C and C++, an "Erlang for dummies" as she put it. I would like to call it "Erlang for managers" instead, as she is not a dummy. What are the best Erlang summaries for managers nowadays? From bengt.kleberg@REDACTED Fri Feb 26 07:55:14 2016 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Fri, 26 Feb 2016 07:55:14 +0100 Subject: [erlang-questions] Why we need a -module() attribute? In-Reply-To: References: <56C4411E.9060106@rol.ru> <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> Message-ID: <56CFF6D2.20305@ericsson.com> Greetings, It would be sufficient (for me) if Erlang provided dictionaries (name spaces) with words (identifiers). The modules/atoms/pids/? in one dictionary would have to be allowed to look the same as in another, without being the same. And I must be able to access them all from my code. This would be more useful (for me) than having parts of the module name taken from the file system. bengt On 02/25/2016 10:31 AM, Joe Armstrong wrote: > On Wed, Feb 24, 2016 at 10:46 AM, Konstantin Vsochanov wrote: >> On 2/23/16 01:30 , Richard A. O'Keefe wrote: >>> I do not recall ever seeing -module coming up here before. >>> I certainly don't recall a long thread like this one. >>> >> Maybe because people who stumble over -module() too shy to write to >> erlang mail list? >> >>> #!/bin/sh >>> #Usage: edit module >>> case "$1" in >>> */*) echo "Module name may not contain /" >/dev/stderr; exit 1 ;; >>> -*) echo "Module name may not start with -">/dev/stderr; exit 1 ;; >>> *\'*) echo "Module name may not contain '" >/dev/stderr; exit 1 ;; >>> *\\*) echo "Module name may not contain \\" >/dev/stderr; exit 1 ;; >>> *) ;; >>> esac >>> file="$1.erl" # Beware: file system may mangle name! >>> if [ ! -e "$file" ]; then >>> date=`date +%Y-%m-%d` >>> case `expr "$1" : '^[a-z][a-zA-Z0-9_]*$'` in >>> 0) module="'$1'" ;; >>> *) module="$1" ;; >>> esac >>> echo "%%% File : $file" >"$file" >>> echo "%%% Author : $LOGNAME" >>"$file" >>> echo "%%% Created: $date" >>"$file" >>> echo "%%% Purpose: " >>"$file" >>> echo "" >>"$file" >>> echo "-module($module)." >>"$file" >>> echo "" >>"$file" >>> echo "-export([" >>"$file" >>> echo " ])." >>"$file" >>> echo "" >>"$file" >>> fi >>> exec ${EDITOR:-emacs} "$file" >>> >>> So here's how much burden -module adds to this programmer: >>> instead of >>> % emacs foobar.erl >>> I have to type >>> % edit foobar >>> >>> Hang on, that's LESS work. >>> >> Thank you for the script, I think it will work for me. I personally >> don't use emacs, but it doesn't matter. >> >> Reading all this long thread I must agree, the -module() is not the >> worst problem in Erlang. >> >> Regarding modules, I think the flat space for module names is more >> troubling. If you look at almost any Erlang library/application, you can >> see, that file names are like >> >> myapp_file1.erl >> myapp_file2.erl >> myapp_subsystem1_file1.erl >> ... >> >> So we manually implement hierarchic module system. It's a clear sign, >> that something going wrong or is missing. > No - I fundamentally disagree - I think that all module names should be unique > ie in a flat name space. > > Why? It's all about the meanings of words - when I use a word it's > meaning should > not depend upon context - it would be nice if this were the case. > > If two people use the same word it should have the same meaning and there should > be one place to look up the word. > > We should agree on the place to lookup the word. So for example, in English > I might say all the words I'm going to use are in the Chambers English > dictionary. > > Dictionaries have flat structures - The Second Edition of the Oxford > English Dictionary has 171,476 words, in alphabetical order so it's > easy to find them. > > If all module names are unique then we can put all the modules in one directory > so we'll know where to find them - so several very difficult problems go away. > > This solves all these problems: > > 1) I know the module name but I can't find it > > 2) I know the module name but a module with this name is in > dozens of different directories > > 3) I have a new module and I don't know what directory to store it in > > Dictionaries are fantastic because we know where to find words - file systems > are horrible because we don't know where to find things. > > I can imagine having different dictionaries, French, German etc - so I can > imaging different large collections of modules - which is akin to > having a two-level > namespace. With rather few top-level names (like dictionaries) > > If you want to simulate stucture then by all means add underscores to the names > this_sub_module is fine '_' has no semantics and is purely a > linguistic convention (just like this.sub.module ) > > /Joe > > > > > > > >> >> >> >> >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From thoffmann@REDACTED Fri Feb 26 09:39:57 2016 From: thoffmann@REDACTED (Torben Hoffmann) Date: Fri, 26 Feb 2016 09:39:57 +0100 Subject: [erlang-questions] Erlang for managers In-Reply-To: References: Message-ID: <56d00f76.0b058d0a.b00f7.ffff9478@mx.google.com> Hi Martin, You can have a look at a slide deck I created at one point: http://www.slideshare.net/torbenhoffmann90/ndc-london-2014-erlang-patterns-matching-business-needs The Golden Trinity of Erlang (s.24) is the key. Fred has given a good explanation of the Zen of Erlang: http://ferd.ca/the-zen-of-erlang.html I think the "Building Microservices" by Sam Newman (http://www.amazon.com/Building-Microservices-Sam-Newman/dp/1491950358/ref=sr_1_1?ie=UTF8&qid=1450360939&sr=8-1&keywords=microservices) has a very favourable mentioning of Erlang's ability to deliver on microservices: If you are a pure Erlang shop, the quality of Erlang's module implementation may get you a very long way, but I suspect that many of you are not in that situation. For me one of the key business values of Erlang is that if you have done your supervision tree well, then you can add new services to your system without worrying. If you have created something that fails, the only that part fails. The rest of your system continues. This significantly reduces the risk of adding to your system. You have to learn how to split things into smaller pieces, but that is a sane advice in any language. jlouis has some good remarks on this in https://medium.com/this-is-not-a-monad-tutorial/interview-with-jesper-louis-andersen-about-erlang-haskell-ocaml-go-idris-the-jvm-software-and-b0de06440fbd#.6bm7lnb21 So smaller, indenpendent components that can fail on their own leads to a higher innovation rate, which leads to money. You can code just as fast in a lot of other languages, but the freedom to make mistakes that Erlang lends you is worth so much more. Feel free to reach out if you need a discussion about this. Cheers, Torben Martin H?ssler writes: > [ text/plain ] > My manager's new manager has asked me for a high level summary of Erlang > compared with other languages, especially C and C++, an "Erlang for dummies" > as she put it. I would like to call it "Erlang for managers" instead, as she > is not a dummy. > > What are the best Erlang summaries for managers nowadays? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Torben Hoffmann Architect, basho.com M: +45 25 14 05 38 From jesper.louis.andersen@REDACTED Fri Feb 26 11:45:55 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 26 Feb 2016 11:45:55 +0100 Subject: [erlang-questions] Erlang for managers In-Reply-To: <56d00f76.0b058d0a.b00f7.ffff9478@mx.google.com> References: <56d00f76.0b058d0a.b00f7.ffff9478@mx.google.com> Message-ID: On Fri, Feb 26, 2016 at 9:39 AM, Torben Hoffmann wrote: > For me one of the key business values of Erlang is that if you have done > your supervision tree well, then you can add new services to your system > without worrying. If you have created something that fails, the only > that part fails. The rest of your system continues. > IMPORTANT: Before doing anything regarding management, you should watch lots of Mike Williams talks. Some things which "sound good" to the technical programmer are bitter pills for management. You should also hope Joe Armstrong chimes in on this thread :) Erlang systems give you something almost for free due to their construction: * Systems are isolated by design in Erlang, so failure in one subsystem is less likely to affect other subsystems. In contrast to most other solutions, the granularity of an isolated unit is much smaller, so faults usually have much less impact on the perception of the system as a whole. * Failure and risk are easier to control: if a subsystem fails, Erlang reestablishes that subsystem from a known good state. This means you can make a call of when you want to fix it, rather than having to fix it ASAP in many cases. In turn, you get operational freedom to decide when to fix problems and can postpone less severe problems till later, or outright ignore them. * Erlang systems automatically snapshots system state under failure for post-mortem analysis and debugging. This helps when failures occur as the system can continue running, but you have a good data set for figuring out what went wrong so it doesn't happen again. Not only do you have a backtrace, you also have the state which lead to the failure. Compared to C/C++ specifically: * Higher level language, so you can iterate faster and try out more ideas in Erlang. It makes it more likely you get the architecture right. Once you lock onto a solution in C/C++ that decision tend to be set in stone for a long time. * If your problem has natural distribution/concurrency, Erlang is likely to perform as well as C/C++ solutions due to the amount of time sunk into the Erlang VM. * Erlang programs tend to be easier to reason about from a correctness perspective[0] [0] Don't fall into the trap of mentioning Functional programming. Yes, that is the underlying reason, but give the consequences. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Fri Feb 26 12:39:27 2016 From: vances@REDACTED (Vance Shipley) Date: Fri, 26 Feb 2016 12:39:27 +0100 Subject: [erlang-questions] Erlang for managers In-Reply-To: References: <56d00f76.0b058d0a.b00f7.ffff9478@mx.google.com> Message-ID: On Fri, Feb 26, 2016 at 11:45 AM, Jesper Louis Andersen wrote: > Some things which "sound good" to the technical programmer are bitter pills for management. Yes indeed. I wouldn't lead off by extolling the fail fast philosophy, for example. Although not about Erlang, this story [http://www.paulgraham.com/avg.html] is a good read about the business advantages of using a non-mainstream language (Lisp). How well a story like that is received by a manager depends a lot on his or her personality. When once asked why other people always had such different opinions than our team I reminded him that since he insisted on hiring from the top percentile he shouldn't expect popularity. Winners choose a different path than losers. As a manager the compelling things about Erlang and OTP -- -Vance From vances@REDACTED Fri Feb 26 12:43:17 2016 From: vances@REDACTED (Vance Shipley) Date: Fri, 26 Feb 2016 12:43:17 +0100 Subject: [erlang-questions] Erlang for managers In-Reply-To: References: <56d00f76.0b058d0a.b00f7.ffff9478@mx.google.com> Message-ID: Oops, misfire. On Fri, Feb 26, 2016 at 12:39 PM, Vance Shipley wrote: > As a manager the compelling things about Erlang and OTP ... are that it is a rapid development environment which sits atop an industrial ready framework which means that prototypes aren't mockups but instead are version 0.1 of what you'll continue to build. -- -Vance From ok@REDACTED Fri Feb 26 12:52:28 2016 From: ok@REDACTED (ok@REDACTED) Date: Sat, 27 Feb 2016 00:52:28 +1300 Subject: [erlang-questions] -pidname( ) Was: Why we need a -module() attribute? In-Reply-To: <2804FE6E-214B-4076-B2F2-C49F9500E8BA@pixie.co.za> References: <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <20160224150705.GA90042@fhebert-ltm2.internal.salesforce.com> <24b9b71226d649598cb2e5514edf7fe0@Jar-EX01.ls.local> <2804FE6E-214B-4076-B2F2-C49F9500E8BA@pixie.co.za> Message-ID: <3c3cc4e9432dd3c8541953b60c86f026.squirrel@chasm.otago.ac.nz> >> Almost all process registrations should be local to the module that >> registers >> them. I proposed >> >> -pidname(). >> >> for declaring a module-local atomic variable to hold pids. This would >> be both >> safer than the global registry and more efficient. The global registry >> would >> remain for those things that genuinely need to be global. > > Presuming that the module name is , say, test, how is: > > -pidname( test ) > > different from: > > register( ?MODULE, self() )? First off, let me repeat that a pidname directive declares a MODULE-LOCAL (as in COMPLETELY INACCESSIBLE FROM THE OUTSIDE) atomic variable. Second, a module name names a module, a pidname atom names a pid variable. They are not the same thing and there is no compelling reason for them to have the same name. Indeed, a module may declare as many pidnames as it needs; they cannot all have the same name as the module. Third, -pidname just *declares* a module-local variable for holding pids, it doesn't register anything. You'd have to do something like pidname:set(, self()). Fourth, just in case you missed it, the whole point is that a pidname is strictly LOCAL to its module. It cannot even be seen from the outside. It's private. Using the registry, every process in the whole node can not only *see* your binding, they can *delete* it or *replace* it. With -pidname, no can do. You're safe, and there's no risk of you accidentally clobbering someone else's registry entry either. > > Also, how would you run multiple instances of the same module within the > same run-time? Why would that even come close to the shadow of the hint of a problem? Each instance has its OWN variable. That's what module-local means. So if you have a module that hides a process, you can load a new version and start it, and calls still executing in the old copy will still be routed to the old process; the registry entry won't (can't) be hijacked by the new module instance. > Your proposal may appear to solve one problem (that is, if one chose to > call it a problem), but appears to introduce at least one more. It may well do so, but you have not identified any. It's an anachronism, because nobody had ever heard of Java when I invented -pidname, but think of -pidname(fred). as an analogue of private static Thread fred = null; From mahesh@REDACTED Fri Feb 26 13:12:59 2016 From: mahesh@REDACTED (Mahesh Paolini-Subramanya) Date: Fri, 26 Feb 2016 07:12:59 -0500 Subject: [erlang-questions] Erlang for managers In-Reply-To: References: <56d00f76.0b058d0a.b00f7.ffff9478@mx.google.com> Message-ID: Slightly off-tangent, but you might want to first consider *why* your manager asked you for this, and tailor your answer appropriately :-) cheers On Fri, Feb 26, 2016 at 6:43 AM, Vance Shipley wrote: > Oops, misfire. > > On Fri, Feb 26, 2016 at 12:39 PM, Vance Shipley > wrote: > > As a manager the compelling things about Erlang and OTP > > ... are that it is a rapid development environment which sits atop an > industrial ready framework which means that prototypes aren't mockups > but instead are version 0.1 of what you'll continue to build. > > > > -- > -Vance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- *Mahesh Paolini-Subramanya That tall bald Indian guy..* *Twitter | Blog | G+ | LinkedIn * -------------- next part -------------- An HTML attachment was scrubbed... URL: From nathaniel@REDACTED Fri Feb 26 13:42:41 2016 From: nathaniel@REDACTED (Nathaniel Waisbrot) Date: Fri, 26 Feb 2016 07:42:41 -0500 Subject: [erlang-questions] Erlang for managers In-Reply-To: References: <56d00f76.0b058d0a.b00f7.ffff9478@mx.google.com> Message-ID: <4E01C448-FCE9-4941-AC57-F8ED2B35CA76@waisbrot.net> > On Feb 26, 2016, at 7:12 AM, Mahesh Paolini-Subramanya wrote: > > Slightly off-tangent, but you might want to first consider why your manager asked you for this, and tailor your answer appropriately :-) A good point. In my experience, this question is often based on the worry that if you use Erlang you won't be able to hire any developers. I haven't heard many satisfying answers to this, either. Fairly well-known is http://paulgraham.com/pypar.html where Paul Graham suggests that hiring for a more esoteric language mostly filters out candidates that you wouldn't want to hire anyway. That seems to work better as a justification after making the choice, rather than an argument in favor of picking a language. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From bchesneau@REDACTED Fri Feb 26 15:50:01 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 26 Feb 2016 14:50:01 +0000 Subject: [erlang-questions] maps or records? Message-ID: Hi all, i tends these days to use maps instead of records, so you dont' have to include a file to retrieve the "schema" but I wonder if this is a good idea or not. Is there still some usage for records vs maps? - beno?t -------------- next part -------------- An HTML attachment was scrubbed... URL: From valentin@REDACTED Fri Feb 26 15:53:41 2016 From: valentin@REDACTED (Valentin Micic) Date: Fri, 26 Feb 2016 16:53:41 +0200 Subject: [erlang-questions] -pidname( ) Was: Why we need a -module() attribute? In-Reply-To: <3c3cc4e9432dd3c8541953b60c86f026.squirrel@chasm.otago.ac.nz> References: <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <20160224150705.GA90042@fhebert-ltm2.internal.salesforce.com> <24b9b71226d649598cb2e5514edf7fe0@Jar-EX01.ls.local> <2804FE6E-214B-4076-B2F2-C49F9500E8BA@pixie.co.za> <3c3cc4e9432dd3c8541953b60c86f026.squirrel@chasm.otago.ac.nz> Message-ID: <940CE93F-68F6-46DC-9634-1DBD57994D5D@pixie.co.za> On 26 Feb 2016, at 1:52 PM, wrote: >>> Almost all process registrations should be local to the module that >>> registers >>> them. I proposed >>> >>> -pidname(). >>> >>> for declaring a module-local atomic variable to hold pids. This would >>> be both >>> safer than the global registry and more efficient. The global registry >>> would >>> remain for those things that genuinely need to be global. >> >> Presuming that the module name is , say, test, how is: >> >> -pidname( test ) >> >> different from: >> >> register( ?MODULE, self() )? > > First off, let me repeat that a pidname directive declares > a MODULE-LOCAL (as in COMPLETELY INACCESSIBLE FROM THE OUTSIDE) > atomic variable. Phrase "atomic variable" in a context of Erlang is a bit counter-intuitive, maybe even to a point of being an oxymoron. And when you say COMPLETELY INACCESSIBLE FROM THE OUTSIDE, I assume this would exclude read access as well. If the whole point of registering processes is to be able to provide for a static way of sending a message to a process, how would that work under these circumstances? > > Second, a module name names a module, a pidname atom names a > pid variable. They are not the same thing and there is no > compelling reason for them to have the same name. Indeed, a > module may declare as many pidnames as it needs; they cannot > all have the same name as the module. Are you saying that one can have more than one -pidname declaration per module? Don't you think this would mean that you would have to know in advance (that is, compile-time) about all the names the process powered by this module would ever have? If so, I am quite surprised that you do not see a problem with that. Also, seeing that module name and named pidname variable are not the same, what would happen if two different modules uses the same name for a pidname? Of course, you may solve this problem by indicating that these are COMPLETELY INACCESSIBLE FROM THE OUTSIDE, but again, assuming that the reason for registration is to be able to have a "static" way to send a message to a process, what would be the point of having them in the first place if they are COMPLETELY INACCESSIBLE FROM THE OUTSIDE. > > Third, -pidname just *declares* a module-local variable for > holding pids, it doesn't register anything. You'd have to > do something like pidname:set(, self()). Given the Erlang standard syntax, it stands to reason that pidname:set( , self() ) invokes a function set, that belongs to another module (pidname), and, as such, goes against your earlier stipulation that -pidname declarations are COMPLETELY INACCESSIBLE FROM THE OUTSIDE. And, if they are accessible from the outside via pidname:set( , self() ), how is that any safer than the registration mechanism already in place? > > Fourth, just in case you missed it, the whole point is that > a pidname is strictly LOCAL to its module. It cannot even be > seen from the outside. It's private. Using the registry, > every process in the whole node can not only *see* your binding, > they can *delete* it or *replace* it. With -pidname, no can do. > You're safe, and there's no risk of you accidentally clobbering > someone else's registry entry either. See a contradiction that I attempted to point out above... > >> >> Also, how would you run multiple instances of the same module within the >> same run-time? > > Why would that even come close to the shadow of the hint of a > problem? Each instance has its OWN variable. That's what > module-local means. I wish I can understand this as easily as you can. When I say instance, well, I actually mean a process, as this is what we register today (and we do that so we can use a symbolic reference instead of pid() in order to send messages to it). In ideal situation, you would have 1:1 correspondence between a module and a process. However, this is rarely the case; e.g. what we register as a gen_server is not the gen_server module, but the actual behavior that runs on top of gen_server. So, which -pidname variable should we use? And let me just reiterate a point that I've made commenting on your Second point above -- are we to provide gen_server with as many names as we have behaviors? Of course not. However, if we have a module gen_server declaring a -pidname( gen_server ), and if we send a message to a gen_server, where would this message go? The point is, you cannot possibly predict all deployments any given module may have. > > So if you have a module that hides a process, you can load a new > version and start it, and calls still executing in the old copy > will still be routed to the old process; the registry entry won't > (can't) be hijacked by the new module instance. > >> Your proposal may appear to solve one problem (that is, if one chose to >> call it a problem), but appears to introduce at least one more. > > It may well do so, but you have not identified any. > > It's an anachronism, because nobody had ever heard of Java > when I invented -pidname, but think of > -pidname(fred). > as an analogue of > private static Thread fred = null; There are some problems outlined above. I am sure you would be able to think of more yourself. But then again, I may have missed the point completely. It wouldn't be the first time. Kind regards V/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Fri Feb 26 15:59:48 2016 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 26 Feb 2016 15:59:48 +0100 Subject: [erlang-questions] Erlang for managers In-Reply-To: References: Message-ID: It's a one liner: 13 Erlang programmers made a company worth 19 B$ (that's billion not million) Read these two articles http://www.businessinsider.com.au/sequoia-why-facebook-bought-whatsapp-2014-2 http://www.wired.com/2015/09/whatsapp-serves-900-million-users-50-engineers/ Everybody loves success stories (WhatsApp) Go very easy on the technical stuff - managers think all PLs are the same anyway you need to produce quantitative evidence that this is not the case. Plan your talk like this: 1) 19 B$ 2) Why is this the case? 3) Shown them the business insider article 4) Explain the points in the article in more detail they will get bored and fidgety 5) Just when they are about to fall asleep, say And what are we going to do about it? Are we going to lead the pack or carry on as we've always been doing? Explain that using old stuff (Java etc) is *very* risky - project failure (the competition will use Erlang/Haskell etc) Then explain the plan This is what I think we should do ... (build a prototype with a few of our best people) About 5 minutes for 1+2 10 minutes for 3, 30 mins on 4 (this is where you show you are technically competent) - 5 mins on 5. For a manager fear of the unknown and fear of failure are major concerns - you need address this by changing perceptions. I some managers mindsets Java/C++ = safe, known. Erlang = risky, unknown So you have to work at this: In my my mindset Java/C++ = risky (project failure, complex code, high budget) But Erlang = safe (projects work, simple code, low budget). The greatest risk to your project is that a competitor doing something very similar chooses Erlang/Haskell etc. (This is all qualified by the assumption that Erlang is the best language for your application) Then you should meet all the standard objections: " but we can't hire any programmers with erlang experience " " we don't need to, the problem is hiring good programmers period. Good programmers can learn any language " " programming Erlang is risky, nobody will want to do this as a career choice" 19B$ Tread lightly on the technical stuff, give examples to reduce the fear of failure. Don't make assumptions - ask lot's of questions: ' What's worrying you about all of this? ' If they don't accept your arguments ask them why. ' You didn't like the idea of using Erlang, so I haven't convinced you, can I ask why what did I do wrong? ' Remember the most important factors in decision making are trust and credibility - credibility means you do your homework. Trust means you are not 'spinning them a line' Cheers /Joe On Fri, Feb 26, 2016 at 7:15 AM, Martin H?ssler wrote: > My manager's new manager has asked me for a high level summary of Erlang > compared with other languages, especially C and C++, an "Erlang for dummies" > as she put it. I would like to call it "Erlang for managers" instead, as she > is not a dummy. > > What are the best Erlang summaries for managers nowadays? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Fri Feb 26 16:21:23 2016 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 26 Feb 2016 16:21:23 +0100 Subject: [erlang-questions] -pidname( ) Was: Why we need a -module() attribute? In-Reply-To: <2804FE6E-214B-4076-B2F2-C49F9500E8BA@pixie.co.za> References: <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <20160224150705.GA90042@fhebert-ltm2.internal.salesforce.com> <24b9b71226d649598cb2e5514edf7fe0@Jar-EX01.ls.local> <2804FE6E-214B-4076-B2F2-C49F9500E8BA@pixie.co.za> Message-ID: Yes ! I suggested something similar years ago. I think there should be a process declaration (analogous) to -module(Name) Something like -process(foo). The main reason is linguistic. We cannot talk about 'the process foo' or 'a foo process'. Processes don't have names so we can't about them, instead we have to use a linguistic circumlocution. Instead of saying " a foo process" we have to say "a process started by evaluating spawn(foo, ....)" I suggest two alternatives: -process(foo). start(Args) -> ... end And -process(foo) start_one(Args) -> ... The primitive spawn should be removed and replaced with Pid = create_process foo(Args) which would equivalent to spawn(foo, start, Args) register should be removed and replaced with create_global foo(Args) which would be equivalent to register(foo, spawn(foo, start_one, Args))All All this can be done with simple preprocessing :-) Cheers /Joe On Fri, Feb 26, 2016 at 7:02 AM, Valentin Micic wrote: > Almost all process registrations should be local to the module that > registers > them. I proposed > > -pidname(). > > for declaring a module-locale atomic variable to hold pids. This would be > both > safer than the global registry and more efficient. The global registry > would > remain for those things that genuinely need to be global. > > > Presuming that the module name is , say, test, how is: > > -pidname( test ) > > different from: > > register( ?MODULE, self() )? > > Also, how would you run multiple instances of the same module within the > same run-time? > Your proposal may appear to solve one problem (that is, if one chose to call > it a problem), but appears to introduce at least one more. > > V/ > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From siraaj@REDACTED Fri Feb 26 16:23:16 2016 From: siraaj@REDACTED (Siraaj Khandkar) Date: Fri, 26 Feb 2016 10:23:16 -0500 Subject: [erlang-questions] maps or records? In-Reply-To: References: Message-ID: <56D06DE4.8030204@khandkar.net> On 2/26/16 9:50 AM, Benoit Chesneau wrote: > Hi all, > > i tends these days to use maps instead of records, so you dont' have to > include a file to retrieve the "schema" but I wonder if this is a good idea > or not. Is there still some usage for records vs maps? When keys are defined statically (i.e. they do not change at run time), such as process state, a tuple (such as an Erlang record) is the semantically-correct choice. When keys are generated dynamically (i.e. they're not (all) known until run time), such as a hit rate counter, any dictionary data structure (such as an Erlang map, dict, ETS, etc) is the semantically-correct choice. I realize that not all will agree with prioritizing semantics over immediate convenience, but those are my personal values, so take it at that :-) I'm of the opinion that including a file (who's author is hopefully conscientious enough to not pollute your namespace) is a small price to pay for explicit expression of intention. From jesper.louis.andersen@REDACTED Fri Feb 26 17:03:21 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 26 Feb 2016 17:03:21 +0100 Subject: [erlang-questions] maps or records? In-Reply-To: <56D06DE4.8030204@khandkar.net> References: <56D06DE4.8030204@khandkar.net> Message-ID: On Fri, Feb 26, 2016 at 4:23 PM, Siraaj Khandkar wrote: > When keys are defined statically (i.e. they do not change at run time), > such as process state, a tuple (such as an Erlang record) is the > semantically-correct choice. Another important point is that all keys needs to have valid data always. Otherwise you may have a key mapping to undefined, and that is often an example of illegal state you can represent. In general I try to limit the amount of knowledge of records outside its using module and at most put them locally in an application-scoped .hrl (in the src dir). Records bind tightly in the sense that any module using a record needs to know about its structure. For ad-hoc returns, I sometimes use a map rather than a tuple. enacl has an example of you generate a key: #{ public := PK, secret := SK } = enacl:box_keypair(), which makes it harder to accidentally swap the PK and SK of a keypair since you are forced to say what goes first and what goes second. In general, I tend to do this for things where you have multiple values of the same type in statically typed languages, but in dynamically typed languages I sometimes do this even more often to avoid swaps. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Fri Feb 26 17:21:20 2016 From: sergej.jurecko@REDACTED (=?utf-8?Q?Sergej_Jure=C4=8Dko?=) Date: Fri, 26 Feb 2016 17:21:20 +0100 Subject: [erlang-questions] maps or records? In-Reply-To: <56D06DE4.8030204@khandkar.net> References: <56D06DE4.8030204@khandkar.net> Message-ID: <7E148E81-4E1F-4177-ADA3-C5C064CF1392@gmail.com> > > When keys are defined statically (i.e. they do not change at run time), such as process state, a tuple (such as an Erlang record) is the semantically-correct choice. On the other hand when debugging maps can be much more helpful. Crash logs produced with maps are so much clearer and printing a map is much easier as well. Nothing stops you from having a schema for maps. Use creator functions that return a map with predefined keys and default values. If you're replacing records for maps, this is pretty important. Otherwise it is difficult to find out what the actual state is. Sergej From bchesneau@REDACTED Fri Feb 26 17:22:43 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 26 Feb 2016 16:22:43 +0000 Subject: [erlang-questions] maps or records? In-Reply-To: References: <56D06DE4.8030204@khandkar.net> Message-ID: On Fri, Feb 26, 2016 at 5:04 PM Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > On Fri, Feb 26, 2016 at 4:23 PM, Siraaj Khandkar > wrote: > >> When keys are defined statically (i.e. they do not change at run time), >> such as process state, a tuple (such as an Erlang record) is the >> semantically-correct choice. > > > Another important point is that all keys needs to have valid data always. > Otherwise you may have a key mapping to undefined, and that is often an > example of illegal state you can represent. > > In general I try to limit the amount of knowledge of records outside its > using module and at most put them locally in an application-scoped .hrl (in > the src dir). Records bind tightly in the sense that any module using a > record needs to know about its structure. > > For ad-hoc returns, I sometimes use a map rather than a tuple. enacl has > an example of you generate a key: > > #{ public := PK, secret := SK } = enacl:box_keypair(), > > which makes it harder to accidentally swap the PK and SK of a keypair > since you are forced to say what goes first and what goes second. In > general, I tend to do this for things where you have multiple values of the > same type in statically typed languages, but in dynamically typed languages > I sometimes do this even more often to avoid swaps. > > > -- > J. > > Thanks both for the information it's useful :) I think I would use maps in state when the state is persisted in a file. Since it's make it easier when you update the file format to also upgrade the state. So would you se a maps instead of a record when you need to share the returned result with apps that depends on your library? Instead of making that result opaque and provide functions to access it? Which sound ridiculous sometimes, since at the end what you do is a get on a record key ... Curious what others do in that case? - benoit -------------- next part -------------- An HTML attachment was scrubbed... URL: From siraaj@REDACTED Fri Feb 26 17:25:01 2016 From: siraaj@REDACTED (Siraaj Khandkar) Date: Fri, 26 Feb 2016 11:25:01 -0500 Subject: [erlang-questions] maps or records? In-Reply-To: References: <56D06DE4.8030204@khandkar.net> Message-ID: <56D07C5D.5060306@khandkar.net> On 2/26/16 11:03 AM, Jesper Louis Andersen wrote: > On Fri, Feb 26, 2016 at 4:23 PM, Siraaj Khandkar > wrote: > >> When keys are defined statically (i.e. they do not change at run time), >> such as process state, a tuple (such as an Erlang record) is the >> semantically-correct choice. > > > Another important point is that all keys needs to have valid data always. > Otherwise you may have a key mapping to undefined, and that is often an > example of illegal state you can represent. > Besides Dialyzer, there's also this neat trick to assert that required fields are not accidentally left-out: -record(foo, {field_a = error(field_a_is_required_in_foo) :: a()}). Which I learned from Adam Rutkowski and Marcin G?rnik. From g@REDACTED Fri Feb 26 17:56:21 2016 From: g@REDACTED (Garrett Smith) Date: Fri, 26 Feb 2016 16:56:21 +0000 Subject: [erlang-questions] maps or records? In-Reply-To: References: <56D06DE4.8030204@khandkar.net> Message-ID: On Fri, Feb 26, 2016 at 10:23 AM Benoit Chesneau > So would you se a maps instead of a record when you need to share the > returned result with apps that depends on your library? Instead of making > that result opaque and provide functions to access it? Which sound > ridiculous sometimes, since at the end what you do is a get on a record key > ... Curious what others do in that case? > What's the nature of the data? If it's a map of keys to values, I think map is a good data type. E.g. I think a database row is well represented as a map, as columns/fields tend to differ based on what you're retrieving. Maybe a good test is the one you mentioned - if you're module is simply providing a bunch of pass-throughs to peel off record fields, it wants to be a map. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Fri Feb 26 19:28:05 2016 From: rexxe98@REDACTED (Andrew Berman) Date: Fri, 26 Feb 2016 18:28:05 +0000 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? Message-ID: Hey Fellow Erlangers, I was curious if any of you guys have switched or are contemplating using Elixir for your next project. I've been programming Erlang for a while now, but I'm about to start a new project and wanted to see where other Erlang devs stood on it. Thanks, Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From garry@REDACTED Fri Feb 26 19:35:50 2016 From: garry@REDACTED (Garry Hodgson) Date: Fri, 26 Feb 2016 13:35:50 -0500 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: Message-ID: <56D09B06.7010006@research.att.com> I've looked into Elixir a bit, but haven't found anything compelling to make me switch. The "friendly" syntax doesn't appeal, and seems to add new warts to replace the old ones in Erlang. Perhaps if I was new to both, I might be inclined toward Elixir since it's less foreign than Erlang syntax. Kinda feels like Erlang with training wheels. On 02/26/2016 01:28 PM, Andrew Berman wrote: > Hey Fellow Erlangers, > > I was curious if any of you guys have switched or are contemplating using > Elixir for your next project. I've been programming Erlang for a while now, > but I'm about to start a new project and wanted to see where other Erlang > devs stood on it. > > Thanks, > > Andrew > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Garry Hodgson Lead Member of Technical Staff AT&T Chief Security Office (CSO) "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Fri Feb 26 19:44:17 2016 From: g@REDACTED (Garrett Smith) Date: Fri, 26 Feb 2016 18:44:17 +0000 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: Message-ID: I just started a new project and momentarily had a very short thought that I might use Elixir. My crusty old engineering persona took over and started asking questions like, "why would you do that?" - then I got back to work building software. I strongly prefer the Erlang syntax over Elixir, especially for actual functional programming. I've have had a problem getting colleagues to adopt into Erlang, and I believe I have a handle on why [1]. I think there are some solid answers to "why use Elixir" if you're considering the problem of adoption. If you are working alone or in a small team that is already proficient using Erlang, I personally would just go and start your work and be happy you aren't being forced to use Java, or Ruby, or Node. How many bridges did I burn here? [1] https://www.youtube.com/watch?v=3MvKLOecT1I On Fri, Feb 26, 2016 at 12:28 PM Andrew Berman wrote: > Hey Fellow Erlangers, > > I was curious if any of you guys have switched or are contemplating using > Elixir for your next project. I've been programming Erlang for a while now, > but I'm about to start a new project and wanted to see where other Erlang > devs stood on it. > > Thanks, > > Andrew > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Fri Feb 26 20:18:34 2016 From: rexxe98@REDACTED (Andrew Berman) Date: Fri, 26 Feb 2016 19:18:34 +0000 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: Message-ID: Hey Garrett, I've watched your talk a couple times before, great stuff, thank you for posting that. Yes, I'd say that adoption is definitely something I am concerned about. I have to sometimes put on my technical hiring hat and step back from the fact that I think Erlang is incredible and that I don't have an issue with the syntax. However, I am going to eventually try to build a team (ideally very lean like WhatsApp) and with lots and lots of devs familiar with Ruby or Javascript and their related tools and frameworks, Elixir seems like a stepping stone for those devs into the Erlang world. I totally agree with many on this list that a great programmer can learn any language, but sometimes it's difficult to find those people in a reasonable time frame, especially when you're trying to light things up quickly as a startup. I definitely have not counted out moving forward with Erlang, which is really why I just wanted to get a sense of what people on this mailing list thought of Elixir. I'm leaning towards Erlang because I know that I personally can move quicker, but it can't always be about me :-) On Fri, Feb 26, 2016 at 10:44 AM Garrett Smith wrote: > I just started a new project and momentarily had a very short thought that > I might use Elixir. My crusty old engineering persona took over and started > asking questions like, "why would you do that?" - then I got back to work > building software. > > I strongly prefer the Erlang syntax over Elixir, especially for actual > functional programming. > > I've have had a problem getting colleagues to adopt into Erlang, and I > believe I have a handle on why [1]. I think there are some solid answers to > "why use Elixir" if you're considering the problem of adoption. If you are > working alone or in a small team that is already proficient using Erlang, I > personally would just go and start your work and be happy you aren't being > forced to use Java, or Ruby, or Node. > > How many bridges did I burn here? > > [1] https://www.youtube.com/watch?v=3MvKLOecT1I > > On Fri, Feb 26, 2016 at 12:28 PM Andrew Berman wrote: > >> Hey Fellow Erlangers, >> >> I was curious if any of you guys have switched or are contemplating using >> Elixir for your next project. I've been programming Erlang for a while now, >> but I'm about to start a new project and wanted to see where other Erlang >> devs stood on it. >> >> Thanks, >> >> Andrew >> > _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Feb 26 20:26:16 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Fri, 26 Feb 2016 20:26:16 +0100 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: Message-ID: <56D0A6D8.2080103@ninenines.eu> On 02/26/2016 07:28 PM, Andrew Berman wrote: > Hey Fellow Erlangers, > > I was curious if any of you guys have switched or are contemplating > using Elixir for your next project. I've been programming Erlang for a > while now, but I'm about to start a new project and wanted to see where > other Erlang devs stood on it. People end up choosing Elixir for one of two reasons: - they identify with the Ruby-like syntax that they find familiar, comforting or perhaps cool (it will always be a different reason for everyone because behavior related to identity isn't rational). They probably are/were Ruby developers or like Ruby. - they require a feature that only Elixir provides. Survival. Considering most Elixir developers come from Ruby (because of the first point), Elixir has the most effort going into Web-development related libraries, and so people coming because of point number two will most likely use it for Web stuff (or Unicode, but it's usually Unicode for Web stuff). (Some people are probably not going to be happy about me making the above observation, but Elixir would grow a lot faster if they made use of this instead of fighting it. The best way to make Elixir get big is to advertise it as a better faster Ruby on Rails. Forget other languages. Forget about Erlang. Just target Ruby people.) Personally I don't really care about it. I am irrationally put off by the syntax the same way I am irrationally put off by the Ruby syntax. But even if I wasn't, it doesn't have any particular feature that would make me want to use it. I guess you could use it to write DSLs, but LFE is probably a better choice for that, in part because calling LFE from Erlang is much easier than calling Elixir from Erlang, due to using the same conventions for modules and types. In the end it doesn't really matter. Use what works for you. Some people are horrified by Perl, and yet there's still some large Perl codebases out there, because it works for them. On a related note, if you ask someone why they choose one or another tech, you will almost always get completely different answers. The reason is that the choice is rarely rational. When the choice *is* rational it's because it wouldn't work with any other tech. But it's rare because you can do most things in most languages. Same applies to DBs, OSes... So people make the choices they identify best with (and some people make a different choice every week because they identify with tech that makes the most noise). (Been talking about identity a lot recently, because I've been reading a lot about it. The world makes a lot more sense when you realize *and accept* that people are not rational, myself included.) -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From garry@REDACTED Fri Feb 26 20:55:01 2016 From: garry@REDACTED (Garry Hodgson) Date: Fri, 26 Feb 2016 14:55:01 -0500 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: Message-ID: <56D0AD95.6070003@research.att.com> In my experience, it isn't the syntax that's hard to learn, but the concepts. and since most learning material is (at least so far) in Erlang, having to mentally translate that to Elixir would seem to make it harder, not easier. But I may not be representative. Last time I counted I've used around 40 languages in anger over the years, yet I find Ruby bewildering. On 02/26/2016 02:18 PM, Andrew Berman wrote: > Hey Garrett, > > I've watched your talk a couple times before, great stuff, thank you for > posting that. > > Yes, I'd say that adoption is definitely something I am concerned about. I > have to sometimes put on my technical hiring hat and step back from the > fact that I think Erlang is incredible and that I don't have an issue with > the syntax. However, I am going to eventually try to build a team (ideally > very lean like WhatsApp) and with lots and lots of devs familiar with Ruby > or Javascript and their related tools and frameworks, Elixir seems like a > stepping stone for those devs into the Erlang world. I totally agree with > many on this list that a great programmer can learn any language, but > sometimes it's difficult to find those people in a reasonable time frame, > especially when you're trying to light things up quickly as a startup. > > I definitely have not counted out moving forward with Erlang, which is > really why I just wanted to get a sense of what people on this mailing list > thought of Elixir. I'm leaning towards Erlang because I know that I > personally can move quicker, but it can't always be about me :-) > > On Fri, Feb 26, 2016 at 10:44 AM Garrett Smith wrote: > >> I just started a new project and momentarily had a very short thought that >> I might use Elixir. My crusty old engineering persona took over and started >> asking questions like, "why would you do that?" - then I got back to work >> building software. >> >> I strongly prefer the Erlang syntax over Elixir, especially for actual >> functional programming. >> >> I've have had a problem getting colleagues to adopt into Erlang, and I >> believe I have a handle on why [1]. I think there are some solid answers to >> "why use Elixir" if you're considering the problem of adoption. If you are >> working alone or in a small team that is already proficient using Erlang, I >> personally would just go and start your work and be happy you aren't being >> forced to use Java, or Ruby, or Node. >> >> How many bridges did I burn here? >> >> [1] https://www.youtube.com/watch?v=3MvKLOecT1I >> >> On Fri, Feb 26, 2016 at 12:28 PM Andrew Berman wrote: >> >>> Hey Fellow Erlangers, >>> >>> I was curious if any of you guys have switched or are contemplating using >>> Elixir for your next project. I've been programming Erlang for a while now, >>> but I'm about to start a new project and wanted to see where other Erlang >>> devs stood on it. >>> >>> Thanks, >>> >>> Andrew >>> >> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Garry Hodgson Lead Member of Technical Staff AT&T Chief Security Office (CSO) "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." -------------- next part -------------- An HTML attachment was scrubbed... URL: From nathaniel@REDACTED Fri Feb 26 21:10:36 2016 From: nathaniel@REDACTED (Nathaniel Waisbrot) Date: Fri, 26 Feb 2016 15:10:36 -0500 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: Message-ID: <8DB1067E-8D31-4C17-ACC4-710D420B9E5E@waisbrot.net> > a team (ideally very lean like WhatsApp) and with lots and lots of devs That was a quick change of plans! Seriously, though, I've been wondering if there are any nice examples of Elixir/Erlang mixing out there. I'm happy with Erlang, but it'd be nice to offer other developers the choice and I'd like to get some of my favorite Erlang libraries more attention in the Elixir world. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jose.valim@REDACTED Fri Feb 26 21:21:24 2016 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Fri, 26 Feb 2016 21:21:24 +0100 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: <56D0AD95.6070003@research.att.com> References: <56D0AD95.6070003@research.att.com> Message-ID: > In my experience, it isn't the syntax that's hard to learn, > but the concepts. and since most learning material is > (at least so far) in Erlang, having to mentally translate > that to Elixir would seem to make it harder, not easier. > While I would say this *was* a concern at some point, I am glad to say it isn't one anymore. Since the beginning we have focused on very good documentation, getting started experience and tooling[1]. We have books that cover from the language up to OTP[2], screencasts[2], initiatives like Elixir School in many languages[3], conferences and many more. Surely at some point you will need to use Erlang libraries, but as you said correctly, once that happens the developer will already be familiar with the majority of concepts in a way everything Erlang has to offer is an *addition* rather than a distraction, from Erlang books to Erlang conferences. For example, I have heard a lot of Elixir developers praising Erlang in Anger once they are running systems in production. > But I may not be representative. Last time I counted I've > used around 40 languages in anger over the years, yet > I find Ruby bewildering. > I am not sure Ruby is relevant here. Elixir is not Ruby (and it could never be in the Erlang VM). Elixir also isn't about Ruby syntax (the same way Erlang isn't about Prolog syntax)[4]. [1] http://elixir-lang.org/getting-started/introduction.html [2] http://elixir-lang.org/learning.html [3] https://elixirschool.com/ [4] https://www.youtube.com/watch?v=Lqo9-pQuRKE -------------- next part -------------- An HTML attachment was scrubbed... URL: From mhssler@REDACTED Fri Feb 26 21:24:59 2016 From: mhssler@REDACTED (=?UTF-8?Q?Martin_H=C3=A4ssler?=) Date: Fri, 26 Feb 2016 21:24:59 +0100 Subject: [erlang-questions] Erlang for managers In-Reply-To: <4E01C448-FCE9-4941-AC57-F8ED2B35CA76@waisbrot.net> References: <56d00f76.0b058d0a.b00f7.ffff9478@mx.google.com> <4E01C448-FCE9-4941-AC57-F8ED2B35CA76@waisbrot.net> Message-ID: On Fri, Feb 26, 2016 at 1:42 PM, Nathaniel Waisbrot wrote: > > On Feb 26, 2016, at 7:12 AM, Mahesh Paolini-Subramanya > > wrote: > > > > Slightly off-tangent, but you might want to first consider why your > > manager asked you for this, and tailor your answer appropriately :-) > > A good point. In my experience, this question is often based on the > worry that if you use Erlang you won't be able to hire any > developers. I haven't heard many satisfying answers to this, either. Yes, that is right. She had heard that another part of the company which also uses Erlang/OTP said that they only wanted Erlang programmers. That caused some concern among the management as the company has thousands of highly experienced C++ developers but very few available Erlang developers. I got a bit shocked about this as our (Erlang) team's main recruitment focus is to find great developers who are eager to learn and not afraid to talk to the end users. It is also good if the new developers have some domain experience. Prior Erlang knowledge is not important at all in a team who already has long Erlang experience. My guess why the other group only ask for Erlang programmers is a combination of two things: Paul Graham's selection process and that they already have a lot experienced C++ programmers who need guidance. I think it is a good sign that my manager is interested to know more and this thread has been great input for me. Thank you all! > Fairly well-known is http://paulgraham.com/pypar.html where Paul > Graham suggests that hiring for a more esoteric language mostly > filters out candidates that you wouldn't want to hire anyway. That > seems to work better as a justification after making the choice, > rather than an argument in favor of picking a language. From garry@REDACTED Fri Feb 26 21:47:32 2016 From: garry@REDACTED (Garry Hodgson) Date: Fri, 26 Feb 2016 15:47:32 -0500 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: <56D0AD95.6070003@research.att.com> Message-ID: <56D0B9E4.8080901@research.att.com> On 2/26/16 3:21 PM, Jos? Valim wrote: > I am not sure Ruby is relevant here. Elixir is not Ruby (and it could > never be in the Erlang VM). Elixir also isn't about Ruby syntax (the > same way Erlang isn't about Prolog syntax)[4]. of course you are right. i'm just in a bad mood from debugging chef recipes. From garry@REDACTED Fri Feb 26 21:51:29 2016 From: garry@REDACTED (Garry Hodgson) Date: Fri, 26 Feb 2016 15:51:29 -0500 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: <56D0B9E4.8080901@research.att.com> References: <56D0AD95.6070003@research.att.com> <56D0B9E4.8080901@research.att.com> Message-ID: <56D0BAD1.5060004@research.att.com> On 2/26/16 3:47 PM, Garry Hodgson wrote: > On 2/26/16 3:21 PM, Jos? Valim wrote: >> I am not sure Ruby is relevant here. Elixir is not Ruby (and it could >> never be in the Erlang VM). Elixir also isn't about Ruby syntax (the >> same way Erlang isn't about Prolog syntax)[4]. > of course you are right. i'm just in a bad mood from debugging chef > recipes. my point being that while many people describe Elixir as ruby-like, that's not a selling point for me personally. From montuori@REDACTED Fri Feb 26 22:11:06 2016 From: montuori@REDACTED (Kevin Montuori) Date: Fri, 26 Feb 2016 15:11:06 -0600 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: (Garrett Smith's message of "Fri, 26 Feb 2016 18:44:17 +0000") References: Message-ID: >>>>> "gs" == Garrett Smith writes: gs> I strongly prefer the Erlang syntax over Elixir, especially for gs> actual functional programming. This! But also: the Elixir crowd seems overawed by the pipe operator. It's pretty handy in some circumstances but the community's desire to glom additional functionality onto the thing is unnerving. I can't wait to try to maintain code that's overloaded the meaning of |>. And this is just hideous: arg1 |> fun(arg2). I'd also posit that the convention of not using -import() in Erlang is a good thing. Lots of Elixir code I've read goes nuts with import() and use(). Faster, perhaps, to type than fully qualified functions but I honestly think it leads to maintenance issues. Especially use() which can have side effects. I worked (briefly!) on a codebase that had a "use MyStuff" at the top of every module. The MyStuff module comprised one autoexecuted __using__/1 function that was merely a list of import statements. It was a miserable experience. I don't mean to conflate the community with the language though. Elixir the language has some nice bits. String handling just works, macros behave as you'd expect, mix and hex are fabulous tools, the documentation's well written, tasks and agents are useful abstractions, and this list continues. I prefer Erlang to Elixir but would sure prefer Elixir to Javascript, Ruby, or Java. k. -- Kevin Montuori montuori@REDACTED From raould@REDACTED Fri Feb 26 22:14:48 2016 From: raould@REDACTED (Raoul Duke) Date: Fri, 26 Feb 2016 13:14:48 -0800 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: Message-ID: Everybody please go try LFE. Come on! ;-) From g@REDACTED Fri Feb 26 22:39:23 2016 From: g@REDACTED (Garrett Smith) Date: Fri, 26 Feb 2016 21:39:23 +0000 Subject: [erlang-questions] Erlang for managers In-Reply-To: References: <56d00f76.0b058d0a.b00f7.ffff9478@mx.google.com> <4E01C448-FCE9-4941-AC57-F8ED2B35CA76@waisbrot.net> Message-ID: Great answers all, esp the insight that such questions are often charged with fascinating back stories! To Jesper's point, Mike Williams has a relatively unique voice in this discussion, having been there from the very start and battled through large company culture up to his retirement a few years ago. This presentation is well worth the time and I'd recommend it as a resource within your org: https://vimeo.com/42824660 On Fri, Feb 26, 2016 at 2:25 PM Martin H?ssler wrote: > On Fri, Feb 26, 2016 at 1:42 PM, Nathaniel Waisbrot > wrote: > > > On Feb 26, 2016, at 7:12 AM, Mahesh Paolini-Subramanya > > > wrote: > > > > > > Slightly off-tangent, but you might want to first consider why your > > > manager asked you for this, and tailor your answer appropriately :-) > > > > A good point. In my experience, this question is often based on the > > worry that if you use Erlang you won't be able to hire any > > developers. I haven't heard many satisfying answers to this, either. > > Yes, that is right. She had heard that another part of the company > which also uses Erlang/OTP said that they only wanted Erlang > programmers. That caused some concern among the management as the > company has thousands of highly experienced C++ developers but very > few available Erlang developers. > > I got a bit shocked about this as our (Erlang) team's main recruitment > focus is to find great developers who are eager to learn and not > afraid to talk to the end users. It is also good if the new developers > have some domain experience. Prior Erlang knowledge is not important > at all in a team who already has long Erlang experience. > > My guess why the other group only ask for Erlang programmers is a > combination of two things: Paul Graham's selection process and that > they already have a lot experienced C++ programmers who need > guidance. > > I think it is a good sign that my manager is interested to know more and > this thread has been great input for me. Thank you all! > > > Fairly well-known is http://paulgraham.com/pypar.html where Paul > > Graham suggests that hiring for a more esoteric language mostly > > filters out candidates that you wouldn't want to hire anyway. That > > seems to work better as a justification after making the choice, > > rather than an argument in favor of picking a language. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hq@REDACTED Fri Feb 26 22:41:36 2016 From: hq@REDACTED (Adam Rutkowski) Date: Fri, 26 Feb 2016 22:41:36 +0100 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: Message-ID: <1456522896.2974063.533107258.43D50C63@webmail.messagingengine.com> Hi Andrew, As a die hard, long time Erlang fan, who recently switched to Elixir full- time I can probably share my perspective. For the past 6 or 7 months I have been writing Elixir code for a Ruby shop seeking their way out ;-) At the beginning it was extremely tough. Needless to say, I have never written a single line in Ruby before, I was unfamiliar with Rails (some of the concepts/terminology obviously was transferred to Phoenix), and everything seemed just plain?weird since I got very accustomed to the rough obviousness of Erlang/OTP. The switch wasn't easy for me -- at least I had the OTP part figured out. But as I soon found out, expecting Elixir to be "Ruby on BEAM" is an oversimplification. Today I think it's a very nicely designed language combining more than just that. Plug is brilliant, definitely brings Clojure's Ring to mind. Think easily accessible, composable cowboy middlewares. They're really fun to build and easy to test. Ecto is *the* killer app. Being able to abstract my postgres schema with ease is something I always craved in the Erlang world (not to mention the gazillion of epgsql forks). Today I can use postgres almost to its full potential. PG views transparently map to predefined structs, arrays & maps (jsonp) are supported via type casting, the query interface is again, composable and powerful (borrows from LINQ) and doesn't do any magic you would expect from popular, "classic" ORMs. Migrations works as expected, rolling back a schema change requires no effort. Protocols (again, Clojure?) and modules, that can be aliased and selectively imported allow you to nicely organise large codebases into logical chunks of responsibilities. In general, I fear not anymore. Elixir is well tested, carefully crafted and evolves very dynamically. I still love Erlang, but I don't think I made a mistake of trying Elixir out in the real world. I like its expressiveness and the fact that there's still OTP available for me to use. It's fun. Cheers & happy hacking, /A. On Fri, Feb 26, 2016, at 19:28, Andrew Berman wrote: > Hey Fellow Erlangers, > > I was curious if any of you guys have switched or are contemplating > using Elixir for your next project. I've been programming Erlang for a > while now, but I'm about to start a new project and wanted to see > where other Erlang devs stood on it. > > Thanks, > > Andrew > _________________________________________________ > erlang-questions mailing list erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Fri Feb 26 23:00:21 2016 From: rexxe98@REDACTED (Andrew Berman) Date: Fri, 26 Feb 2016 22:00:21 +0000 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: <1456522896.2974063.533107258.43D50C63@webmail.messagingengine.com> References: <1456522896.2974063.533107258.43D50C63@webmail.messagingengine.com> Message-ID: Hey Adam, Good to hear that your experience is positive. Did you settle on Elixir because you were writing it for a Ruby shop? In other words, did you choose it because you have a team of Ruby devs and thought it might be easier to migrate them? On Fri, Feb 26, 2016 at 1:41 PM Adam Rutkowski wrote: > Hi Andrew, > > As a die hard, long time Erlang fan, who recently switched to Elixir > full-time I can probably share my perspective. > For the past 6 or 7 months I have been writing Elixir code for a Ruby shop > seeking their way out ;-) > > At the beginning it was extremely tough. Needless to say, I have never > written a single line in Ruby before, > I was unfamiliar with Rails (some of the concepts/terminology obviously > was transferred to Phoenix), > and everything seemed just plain weird since I got very accustomed to the > rough obviousness of Erlang/OTP. > > The switch wasn't easy for me -- at least I had the OTP part figured out. > > But as I soon found out, expecting Elixir to be "Ruby on BEAM" is an > oversimplification. > Today I think it's a very nicely designed language combining more than > just that. > > Plug is brilliant, definitely brings Clojure's Ring to mind. Think easily > accessible, composable cowboy middlewares. > They're really fun to build and easy to test. > > Ecto is *the* killer app. Being able to abstract my postgres schema with > ease is something I always > craved in the Erlang world (not to mention the gazillion of epgsql forks). > Today I can use postgres almost to its full potential. PG views > transparently map to predefined > structs, arrays & maps (jsonp) are supported via type casting, the query > interface is again, > composable and powerful (borrows from LINQ) and doesn't do any magic you > would expect > from popular, "classic" ORMs. Migrations works as expected, rolling back a > schema change requires > no effort. > > Protocols (again, Clojure?) and modules, that can be aliased and > selectively imported > allow you to nicely organise large codebases into logical chunks of > responsibilities. > > In general, I fear not anymore. Elixir is well tested, carefully crafted > and evolves very dynamically. > I still love Erlang, but I don't think I made a mistake of trying Elixir > out in the real world. > I like its expressiveness and the fact that there's still OTP available > for me to use. > It's fun. > > Cheers & happy hacking, > > /A. > > > On Fri, Feb 26, 2016, at 19:28, Andrew Berman wrote: > > Hey Fellow Erlangers, > > I was curious if any of you guys have switched or are contemplating using > Elixir for your next project. I've been programming Erlang for a while now, > but I'm about to start a new project and wanted to see where other Erlang > devs stood on it. > > Thanks, > > Andrew > > *_______________________________________________* > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hq@REDACTED Fri Feb 26 23:40:53 2016 From: hq@REDACTED (Adam Rutkowski) Date: Fri, 26 Feb 2016 23:40:53 +0100 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: <1456522896.2974063.533107258.43D50C63@webmail.messagingengine.com> Message-ID: <1456526453.2986710.533180258.000399E2@webmail.messagingengine.com> On Fri, Feb 26, 2016, at 23:00, Andrew Berman wrote: > Hey Adam, > > Good to hear that your experience is positive.? Did you settle on > Elixir because you were writing it for a Ruby shop?? In other words, > did you choose it because you have a team of Ruby devs and thought it > might be easier to migrate them? I was looking for a job with an opportunity to learn something new, I guess I just needed a change. And I really wanted to stay in my OTP comfort zone. The company I ended up working for was looking for Elixir devs for the reason you've mentioned. > > On Fri, Feb 26, 2016 at 1:41 PM Adam Rutkowski wrote: >> __ >> Hi Andrew, >> >> As a die hard, long time Erlang fan, who recently switched to Elixir >> full-time I can probably share my perspective. For the past 6 or 7 >> months I have been writing Elixir code for a Ruby shop seeking their >> way out ;-) >> >> At the beginning it was extremely tough. Needless to say, I have >> never written a single line in Ruby before, I was unfamiliar with >> Rails (some of the concepts/terminology obviously was transferred to >> Phoenix), and everything seemed just plain?weird since I got very >> accustomed to the rough obviousness of Erlang/OTP. >> >> The switch wasn't easy for me -- at least I had the OTP part >> figured out. >> >> But as I soon found out, expecting Elixir to be "Ruby on BEAM" is an >> oversimplification. Today I think it's a very nicely designed >> language combining more than just that. >> >> Plug is brilliant, definitely brings Clojure's Ring to mind. Think >> easily accessible, composable cowboy middlewares. They're really fun >> to build and easy to test. >> >> Ecto is *the* killer app. Being able to abstract my postgres schema >> with ease is something I always craved in the Erlang world (not to >> mention the gazillion of epgsql forks). Today I can use postgres >> almost to its full potential. PG views transparently map to >> predefined structs, arrays & maps (jsonp) are supported via type >> casting, the query interface is again, composable and powerful >> (borrows from LINQ) and doesn't do any magic you would expect from >> popular, "classic" ORMs. Migrations works as expected, rolling back a >> schema change requires no effort. >> >> Protocols (again, Clojure?) and modules, that can be aliased and >> selectively imported allow you to nicely organise large codebases >> into logical chunks of responsibilities. >> >> In general, I fear not anymore. Elixir is well tested, carefully >> crafted and evolves very dynamically. I still love Erlang, but I >> don't think I made a mistake of trying Elixir out in the real world. >> I like its expressiveness and the fact that there's still OTP >> available for me to use. It's fun. >> >> Cheers & happy hacking, >> >> /A. >> >> >> On Fri, Feb 26, 2016, at 19:28, Andrew Berman wrote: >>> Hey Fellow Erlangers, >>> >>> I was curious if any of you guys have switched or are contemplating >>> using Elixir for your next project. I've been programming Erlang for >>> a while now, but I'm about to start a new project and wanted to see >>> where other Erlang devs stood on it. >>> >>> Thanks, >>> >>> Andrew >>> _________________________________________________ >>> erlang-questions mailing list erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions /A. -------------- next part -------------- An HTML attachment was scrubbed... URL: From Catenacci@REDACTED Fri Feb 26 22:27:13 2016 From: Catenacci@REDACTED (Onorio Catenacci) Date: Fri, 26 Feb 2016 16:27:13 -0500 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? Message-ID: > > > Message: 16 > Date: Fri, 26 Feb 2016 18:28:05 +0000 > From: Andrew Berman > To: Erlang Questions > Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? > Message-ID: > < > CAEVpa76ftrrjTP56FPULTkDuNNqGDiXtZ-iPTEMtEKStRWjOQw@REDACTED> > Content-Type: text/plain; charset="utf-8" > > Hey Fellow Erlangers, > > I was curious if any of you guys have switched or are contemplating using > Elixir for your next project. I've been programming Erlang for a while now, > but I'm about to start a new project and wanted to see where other Erlang > devs stood on it. > > Thanks, > > Andrew > Hi Andrew, While there may be other considerations you want to keep in mind, there's one, to my mind, major point that's been neglected in the discussion of using Elixir: macros. I have been led to understand by people far more familiar with Erlang than I am that in order to use certain behaviours from OTP one has to write boilerplate code for several callbacks whether one wants to modify the behavior from defaults or not. Elixir's macros allow the library to provide default implementations for these callback functions. Of course a developer can override the defaults if needed but not having to code what I don't need strikes me as a big advantage--especially if we're discussing a new developer who doesn't already know either Erlang or Elixir. I say this with full awareness that Elixir's macros are a good thing and a bad thing too because it seems that every noob Elixir developer on our mailing list wants to use macros everywhere. That and every noob seems to be convinced that the pipe forward operator (|>) just has to be used _everywhere_. I can certainly understand the thinking of folks who are familiar with Erlang. I feel the same way when people ask me why I never learned Java (I do C# for the day job). Why would I bother? But for new developers I do believe the fact that they can use OTP without having to worry about creating a lot of boilerplate code is a significant advantage. By the way, I've never written a line of Ruby in my life and I still need to get round to looking at Phoenix. Not everyone looking at Elixir is a Ruby developer looking for a more scalable Ruby on Rails--although in fairness there are a lot of them. FWIW. -- Onorio -------------- next part -------------- An HTML attachment was scrubbed... URL: From alisdairsullivan@REDACTED Sat Feb 27 06:41:00 2016 From: alisdairsullivan@REDACTED (alisdair sullivan) Date: Fri, 26 Feb 2016 21:41:00 -0800 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: <8DB1067E-8D31-4C17-ACC4-710D420B9E5E@waisbrot.net> References: <8DB1067E-8D31-4C17-ACC4-710D420B9E5E@waisbrot.net> Message-ID: <3FE4E609-E56F-486E-A79E-0851DDB58400@yahoo.ca> > On Feb 26, 2016, at 12:10 PM, Nathaniel Waisbrot wrote: > > Seriously, though, I've been wondering if there are any nice examples of Elixir/Erlang mixing out there. I'm happy with Erlang, but it'd be nice to offer other developers the choice and I'd like to get some of my favorite Erlang libraries more attention in the Elixir world. > you might be interested in this: http://github.com/talentdeficit/posterize it?s an elixir library that wraps eric meadows-jonsson?s excellent postgrex library to provide a more convenient interface for erlang usage i think elixir has a lot to offer erlang and it?s a shame that the two communities are not more integrated From essen@REDACTED Sat Feb 27 11:54:57 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Sat, 27 Feb 2016 11:54:57 +0100 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: Message-ID: <56D18081.7030307@ninenines.eu> On 02/26/2016 10:27 PM, Onorio Catenacci wrote: > While there may be other considerations you want to keep in mind, > there's one, to my mind, major point that's been neglected in the > discussion of using Elixir: macros. > > I have been led to understand by people far more familiar with Erlang > than I am that in order to use certain behaviours from OTP one has to > write boilerplate code for several callbacks whether one wants to modify > the behavior from defaults or not. > > Elixir's macros allow the library to provide default implementations for > these callback functions. Of course a developer can override the > defaults if needed but not having to code what I don't need strikes me > as a big advantage--especially if we're discussing a new developer who > doesn't already know either Erlang or Elixir. I'm not even sure what makes you think you need macros to provide default implementations? You don't. The gen_server for example has an optional callback with a default implementation: format_error. That it doesn't provide defaults for other callbacks is purely a design choice, and not a limitation of the language. You could trivially extend gen_server to make all callbacks optional, no need to have macros for that... -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From essen@REDACTED Sat Feb 27 12:28:52 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Sat, 27 Feb 2016 12:28:52 +0100 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: <56D0AD95.6070003@research.att.com> Message-ID: <56D18874.1080002@ninenines.eu> On 02/26/2016 09:21 PM, Jos? Valim wrote: > But I may not be representative. Last time I counted I've > used around 40 languages in anger over the years, yet > I find Ruby bewildering. > > > I am not sure Ruby is relevant here. Elixir is not Ruby (and it could > never be in the Erlang VM). Elixir also isn't about Ruby syntax (the > same way Erlang isn't about Prolog syntax)[4]. Rationally, Elixir is not Ruby, and Erlang isn't Prolog. Irrationally, it is. Elixir has the same look and feel as Ruby, and Erlang has the same look and feel as Prolog. When Ruby developers look at Elixir they feel right at home. If you call yourself a Ruby developer, then you identify with certain values from Ruby, many of which can be found in Elixir. It's familiar. Again, we are on the irrational level here. Same goes for Erlang and Prolog. In fact a few days ago a few long-time Prolog developers pointed out the exact same thing when they were talking about Erlang. There is this familiarity that smoothes them in, even though the languages are fundamentally different. The thing is, if you have to convince large groups of people, you need to appeal to their irrational mind. As Scott Adams brilliantly pointed out, identity beats analogy beats reason. If you want to convince people to come to Elixir, you need to appeal to their identity, which is why targeting Ruby on Rails developers is your best bet. If you don't then you're just wasting valuable time and resources. I've pointed out a few years ago that Elixir was for Ruby developers. I didn't know why at the time. If you look at the most recent survey (http://blog.elixirsips.com/2015/12/21/elixir-users-survey-2015/), you can see that Ruby developers dominate. Other languages are little more than a statistical anomaly. Clearly you bring in a lot more Ruby developers than any other combined, and the reason for that is identity. Stop fighting it. Use it to bring more people in. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From jose.valim@REDACTED Sat Feb 27 13:14:16 2016 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Sat, 27 Feb 2016 13:14:16 +0100 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: <56D18874.1080002@ninenines.eu> References: <56D0AD95.6070003@research.att.com> <56D18874.1080002@ninenines.eu> Message-ID: > > When Ruby developers look at Elixir they feel right at home. If you call > yourself a Ruby developer, then you identify with certain values from Ruby, > many of which can be found in Elixir. It's familiar. Again, we are on the > irrational level here. > You don't need to go to an irrational level to argue that. Elixir picks some of its values from Ruby, as well as Erlang and Clojure. I don't think anybody from the Elixir community is denying or fighting against that. In any case, the "Elixir is not Ruby" is a valid disclaimer. Otherwise you get people expecting it to be exactly like Ruby and have their expectations violated and also people that completely dismiss it because they think it will be exactly like Ruby. If you look at the most recent survey ( > http://blog.elixirsips.com/2015/12/21/elixir-users-survey-2015/), you can > see that Ruby developers dominate. Other languages are little more than a > statistical anomaly. Clearly you bring in a lot more Ruby developers than > any other combined, and the reason for that is identity. > The other languages *amount to 40% of the replies*. It is not a statistical anomaly. And I would be really careful to put all of the reasoning on the identity/irrational, although I agree it does play an important role. For example, we should also consider that my biggest sphere of influence before Elixir was the Ruby community. Those were the first developers I could reach to, which in turn brought more developers from the same community, and so on. That's exactly why I would like to focus on other communities. We have already attracted many Ruby folks which are now doing the work of "spreading the word" more effectively than I ever would. Plus they have the whole identity thing going on for them. Actively investing in other communities is how we can bring more diversity into the language and ensure it grows beyond a web-centric community. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Sat Feb 27 14:01:01 2016 From: ivan@REDACTED (Ivan Uemlianin) Date: Sat, 27 Feb 2016 13:01:01 +0000 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: <56D0AD95.6070003@research.att.com> <56D18874.1080002@ninenines.eu> Message-ID: <1456578063800-dd684c6d-4ca9c33f-7db43828@llaisdy.com> On Sat, Feb 27, 2016 at 12:14 pm, Jos? Valim wrote: ... If you look at the most recent survey ( [http://blog.elixirsips.com/2015/12/21/elixir-users-survey-2015/] http://blog.elixirsips.com/2015/12/21/elixir-users-survey-2015/ [http://blog.elixirsips.com/2015/12/21/elixir-users-survey-2015/] ), you can see that Ruby developers dominate. Other languages are little more than a statistical anomaly. Clearly you bring in a lot more Ruby developers than any other combined, and the reason for that is identity. The other languages *amount to 40% of the replies*. It is not a statistical anomaly. The blog post itself says " this is ridiculously represented by Ruby, with all other answers coming in as noise." Ivan -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Sat Feb 27 14:49:45 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Sat, 27 Feb 2016 14:49:45 +0100 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: <56D0AD95.6070003@research.att.com> <56D18874.1080002@ninenines.eu> Message-ID: <56D1A979.9050908@ninenines.eu> On 02/27/2016 01:14 PM, Jos? Valim wrote: > When Ruby developers look at Elixir they feel right at home. If you > call yourself a Ruby developer, then you identify with certain > values from Ruby, many of which can be found in Elixir. It's > familiar. Again, we are on the irrational level here. > > > You don't need to go to an irrational level to argue that. Elixir picks > some of its values from Ruby, as well as Erlang and Clojure. I don't > think anybody from the Elixir community is denying or fighting against that. > > In any case, the "Elixir is not Ruby" is a valid disclaimer. Otherwise > you get people expecting it to be exactly like Ruby and have their > expectations violated and also people that completely dismiss it because > they think it will be exactly like Ruby. It's a valid disclaimer, and nobody is saying Elixir is Ruby, nor should. But it doesn't get you far. In fact it might have the opposite effect for anyone who thinks of themselves as Ruby developers. A more effective way to say it would be "inspired by Ruby" or "sharing many of the values from Ruby". You want to make people come in because of their similarities, not keep them out because of their differences. > If you look at the most recent survey > (http://blog.elixirsips.com/2015/12/21/elixir-users-survey-2015/), > you can see that Ruby developers dominate. Other languages are > little more than a statistical anomaly. Clearly you bring in a lot > more Ruby developers than any other combined, and the reason for > that is identity. > > > The other languages *amount to 40% of the replies*. It is not a > statistical anomaly. And I would be really careful to put all of the > reasoning on the identity/irrational, although I agree it does play an > important role. For example, we should also consider that my biggest > sphere of influence before Elixir was the Ruby community. Those were the > first developers I could reach to, which in turn brought more developers > from the same community, and so on. There's not been any filtering done on this data (at all, Javascript is even listed three times). The first thing a statistician would do would be to remove all statistically insignifiant answers (either completely or by having an Other category). All the languages with a handful of answers are irrelevant. You also need to relativise these answers with the size of the respective communities. Clearly you don't have much an impact on Java or C developers, despite them being the largest numbers of developers. You have a much bigger impact on Python, and yet nothing close to Ruby, in comparison. Interpretation of raw data is rarely insightful. > That's exactly why I would like to focus on other communities. We have > already attracted many Ruby folks which are now doing the work of > "spreading the word" more effectively than I ever would. Plus they have > the whole identity thing going on for them. Actively investing in other > communities is how we can bring more diversity into the language and > ensure it grows beyond a web-centric community. You need to have a common ground, though. Right now, as an *outsider observer*, I can't think of Elixir being for anything other than Web stuff. Before you start investing in other communities I would recommend investing in diversifying the Elixir ecosystem. Python is a great example of a diversified ecosystem, I can think of Python being used for Web stuff, system tools and GUI applications. How that happened, I do not know, but today I can think of Python being used in all three. It's part of Python's identity, when I think of Python I think about these use cases, about python2to3 and about significant whitespace. I am guessing that before trying to bring people in, a large amount of work was done to provide them with what they want. This is of course an iterative process. But I believe you can't have Java people too interested in Elixir unless you have the enterprisey stuff they need, to give an example. And you need to make it clearly apparent. You need to decide what to focus on and make it a part of Elixir's identity. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From jose.valim@REDACTED Sat Feb 27 15:22:53 2016 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Sat, 27 Feb 2016 15:22:53 +0100 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: <56D1A979.9050908@ninenines.eu> References: <56D0AD95.6070003@research.att.com> <56D18874.1080002@ninenines.eu> <56D1A979.9050908@ninenines.eu> Message-ID: > You also need to relativise these answers with the size of the respective > communities. Clearly you don't have much an impact on Java or C developers, > despite them being the largest numbers of developers. You have a much > bigger impact on Python, and yet nothing close to Ruby, in comparison. > > Interpretation of raw data is rarely insightful. You are still assuming all communities have had equal exposition to Elixir, which is just not true. It doesn't matter the size of the community if only a handful of developers heard about us. Maybe we have been really successful with the Python developers that heard of us, maybe even more than Ruby. Who knows? That's exactly my point. Don't jump into conclusions that it is a statistical error or that all adoption is identity centric. For certain, we have been longer at promoting Elixir in one of those communities. Other than that, I completely agree with the "web-centric" perspective. The goal is not to "bring Python developers" but to promote initiatives inside the community, like the embedded and data-centric projects and companies that are popping up. Once those use cases gain traction, we will naturally bring more developers. We do share the same impression about the Python community. :) I also associate them to the scientific community (ipython, pydata, etc). -- *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Sat Feb 27 16:34:51 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Sat, 27 Feb 2016 16:34:51 +0100 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: <56D0AD95.6070003@research.att.com> <56D18874.1080002@ninenines.eu> <56D1A979.9050908@ninenines.eu> Message-ID: <56D1C21B.7030401@ninenines.eu> On 02/27/2016 03:22 PM, Jos? Valim wrote: > > You also need to relativise these answers with the size of the > respective communities. Clearly you don't have much an impact on > Java or C developers, despite them being the largest numbers of > developers. You have a much bigger impact on Python, and yet nothing > close to Ruby, in comparison. > > Interpretation of raw data is rarely insightful. > > > You are still assuming all communities have had equal exposition to > Elixir, which is just not true. It doesn't matter the size of the > community if only a handful of developers heard about us. Maybe we have > been really successful with the Python developers that heard of us, > maybe even more than Ruby. Who knows? > > That's exactly my point. Don't jump into conclusions that it is a > statistical error or that all adoption is identity centric. For certain, > we have been longer at promoting Elixir in one of those communities. I jump to conclusions as easily as I dismiss them. It's what I believe from looking at the survey, and not anything rational. I can change my mind in 10 minutes. I'll change my mind right now. I'll posit that you attract both Ruby developers and Web developers. But I stand still on the identity (although I do admit there's other reasons, as I did in my first post). I do believe it is easier and cheaper to target Ruby developers, though. And it is therefore quicker to grow, and quicker to diversify later on, if you get many Ruby developers in first. First, build an army, and then use that army to conquer the world. > We do share the same impression about the Python community. :) I also > associate them to the scientific community (ipython, pydata, etc). I guess that despite being well aware of this, it isn't close enough to home that it would come up to my mind immediately. The brain is a strange machine. Good talk. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From jesper.louis.andersen@REDACTED Sat Feb 27 23:24:35 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 27 Feb 2016 23:24:35 +0100 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: Message-ID: On Fri, Feb 26, 2016 at 7:28 PM, Andrew Berman wrote: > I was curious if any of you guys have switched or are contemplating using > Elixir for your next project. I've been programming Erlang for a while now, > but I'm about to start a new project and wanted to see where other Erlang > devs stood on it. First and foremost, I hope that if I do good work in Erlang, then that work is somewhat easily transplantable into Elixir. So if I end up using Elixir later, then I can use some of the same work, and I did not lose too much in the process. Currently, I'm mostly interested in how the Erlang, OCaml, Haskell, and Go communities are developing and what they are doing as languages. I note that all but one of those languages are statically typed, and if I were to pick up another language in the Erlang system, I'd prefer it to have a static type system. But alas, we are still missing this, so I'll have to write my own compiler[0] [0] Or write in Pony, but I'm not sure I like the semantics of Pony too much. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Sun Feb 28 01:17:06 2016 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 27 Feb 2016 16:17:06 -0800 Subject: [erlang-questions] maps or records? In-Reply-To: References: Message-ID: <56D23C82.7060409@gmail.com> On 02/26/2016 06:50 AM, Benoit Chesneau wrote: > Hi all, > > i tends these days to use maps instead of records, so you dont' have > to include a file to retrieve the "schema" but I wonder if this is a > good idea or not. Is there still some usage for records vs maps? > > - beno?t > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Records allow you to specify type specification information, while maps do not. Records are basic and efficient with the number of elements 100 or less, while maps are great for dynamic use that doesn't depend on type specification information. So, for process state information, records provide more information when checking dialyzer for problems. Records can also provide more checking with dialyzer when storing database or protocol data (two places where types are generally important for development). For dynamic key/value data, dict can use type specification information while maps don't, so there can also be motivation to use dicts instead of maps, even if only switching between the two for running dialyzer. maps are great for efficient key/value access though. -------------- next part -------------- An HTML attachment was scrubbed... URL: From wallentin.dahlberg@REDACTED Sun Feb 28 01:42:03 2016 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Sun, 28 Feb 2016 01:42:03 +0100 Subject: [erlang-questions] maps or records? In-Reply-To: <56D23C82.7060409@gmail.com> References: <56D23C82.7060409@gmail.com> Message-ID: Whoa there .. I just want to point out that you shouldn't have record sizes of upwards 100 elements. If the only thing you do is read them you are fine but normally we do both reads and updates. I would say 50 is the limit but even then you are pushing it. Above 30 I would consider splitting the array into a tree. I think records has there place. There is some overlap between maps and records for sure and I think I agree with most of the things that has already been said. The dialyzer support for maps could be stronger. It is pretty weak at the moment. Data modeling with records is easier. I would consider using records within a module, if it's never changed or upgraded and never leaks out to any other modules. Maybe between modules within an application. Maybe. But I think you are better off with maps in that case. // Bj?rn-Egil 2016-02-28 1:17 GMT+01:00 Michael Truog : > On 02/26/2016 06:50 AM, Benoit Chesneau wrote: > > Hi all, > > i tends these days to use maps instead of records, so you dont' have to > include a file to retrieve the "schema" but I wonder if this is a good idea > or not. Is there still some usage for records vs maps? > > - beno?t > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > Records allow you to specify type specification information, while maps do > not. Records are basic and efficient with the number of elements 100 or > less, while maps are great for dynamic use that doesn't depend on type > specification information. So, for process state information, records > provide more information when checking dialyzer for problems. Records can > also provide more checking with dialyzer when storing database or protocol > data (two places where types are generally important for development). > > For dynamic key/value data, dict can use type specification information > while maps don't, so there can also be motivation to use dicts instead of > maps, even if only switching between the two for running dialyzer. maps > are great for efficient key/value access though. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Sun Feb 28 02:17:17 2016 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 27 Feb 2016 17:17:17 -0800 Subject: [erlang-questions] maps or records? In-Reply-To: References: <56D23C82.7060409@gmail.com> Message-ID: <56D24A9D.4090109@gmail.com> On 02/27/2016 04:42 PM, Bj?rn-Egil Dahlberg wrote: > Whoa there .. > > I just want to point out that you shouldn't have record sizes of > upwards 100 elements. If the only thing you do is read them you are > fine but normally we do both reads and updates. I would say 50 is the > limit but even then you are pushing it. Above 30 I would consider > splitting the array into a tree. Records are preprocessing syntax (sweet syntactic sugar) for tuples and I am just going based on my benchmarking of tuples. I know tuples are generally ok for 100 elements and less (not trying to encourage making large records with tons of values). I also know that record use can minimize tuple changes via http://erlang.org/doc/efficiency_guide/commoncaveats.html#id57442 which would not happen with small maps that are only using tuples (i.e., a map that is not larger than 32 elements, |MAP_SMALL_MAP_LIMIT).| > > I think records has there place. There is some overlap between maps > and records for sure and I think I agree with most of the things that > has already been said. > The dialyzer support for maps could be stronger. It is pretty weak at > the moment. Data modeling with records is easier. > > I would consider using records within a module, if it's never changed > or upgraded and never leaks out to any other modules. Maybe between > modules within an application. Maybe. But I think you are better off > with maps in that case. Yes, records can cause maintenance and dependency problems. If the code needs to be tied to changes of a database or protocol, it might be worth it, but it depends on the situation. > > // Bj?rn-Egil > > > 2016-02-28 1:17 GMT+01:00 Michael Truog >: > > On 02/26/2016 06:50 AM, Benoit Chesneau wrote: >> Hi all, >> >> i tends these days to use maps instead of records, so you dont' >> have to include a file to retrieve the "schema" but I wonder if >> this is a good idea or not. Is there still some usage for records >> vs maps? >> >> - beno?t >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > Records allow you to specify type specification information, while > maps do not. Records are basic and efficient with the number of > elements 100 or less, while maps are great for dynamic use that > doesn't depend on type specification information. So, for process > state information, records provide more information when checking > dialyzer for problems. Records can also provide more checking > with dialyzer when storing database or protocol data (two places > where types are generally important for development). > > For dynamic key/value data, dict can use type specification > information while maps don't, so there can also be motivation to > use dicts instead of maps, even if only switching between the two > for running dialyzer. maps are great for efficient key/value > access though. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Sun Feb 28 12:53:23 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Sun, 28 Feb 2016 12:53:23 +0100 Subject: [erlang-questions] Detect beam compilation target Message-ID: Hi! Is there some information in the beam files that would identify the compatibility with the current runtime, or is it only when trying to execute incompatible code that it is detected? best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From constantin@REDACTED Sun Feb 28 13:32:31 2016 From: constantin@REDACTED (Constantin Rack) Date: Sun, 28 Feb 2016 13:32:31 +0100 Subject: [erlang-questions] Detect beam compilation target In-Reply-To: References: Message-ID: <24EEE339-6F27-497E-B7CB-4D8D7011D614@rack.li> The BEAM format number and the number of the highest opcode are stored in the code chunk: https://github.com/erlang/otp/blob/master/erts/emulator/beam/beam_load.c#L1714-L1757 -Constantin Am 28.02.2016 um 12:53 schrieb Vlad Dumitrescu: > Hi! > > Is there some information in the beam files that would identify the compatibility with the current runtime, or is it only when trying to execute incompatible code that it is detected? > > best regards, > Vlad > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Sun Feb 28 14:51:41 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Sun, 28 Feb 2016 14:51:41 +0100 Subject: [erlang-questions] Detect beam compilation target In-Reply-To: <24EEE339-6F27-497E-B7CB-4D8D7011D614@rack.li> References: <24EEE339-6F27-497E-B7CB-4D8D7011D614@rack.li> Message-ID: Thanks! I was a bit confused that all the files I had looked at had the same value, regardless. My error was that I didn't think that the value depends on the code actually in that module. Checking for example maps.beam gives a different value. So now I only have to find out if there is a table mapping the max opcode value to the beam runtime version. best regards, Vlad On Sun, Feb 28, 2016 at 1:32 PM, Constantin Rack wrote: > The BEAM format number and the number of the highest opcode are stored in > the code chunk: > > https://github.com/erlang/otp/blob/master/erts/emulator/beam/beam_load.c#L1714-L1757 > > -Constantin > > > Am 28.02.2016 um 12:53 schrieb Vlad Dumitrescu: > > Hi! > > Is there some information in the beam files that would identify the > compatibility with the current runtime, or is it only when trying to > execute incompatible code that it is detected? > > best regards, > Vlad > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From constantin@REDACTED Sun Feb 28 15:00:09 2016 From: constantin@REDACTED (Constantin Rack) Date: Sun, 28 Feb 2016 15:00:09 +0100 Subject: [erlang-questions] Detect beam compilation target In-Reply-To: References: <24EEE339-6F27-497E-B7CB-4D8D7011D614@rack.li> Message-ID: <9C19295B-F06E-4907-911A-A93A28D8EB04@rack.li> There is a list of all opcodes with comments about the runtime version here: https://github.com/erlang/otp/blob/master/lib/compiler/src/genop.tab -Constantin Am 28.02.2016 um 14:51 schrieb Vlad Dumitrescu: > Thanks! I was a bit confused that all the files I had looked at had the same value, regardless. My error was that I didn't think that the value depends on the code actually in that module. Checking for example maps.beam gives a different value. > > So now I only have to find out if there is a table mapping the max opcode value to the beam runtime version. > > best regards, > Vlad > > > On Sun, Feb 28, 2016 at 1:32 PM, Constantin Rack wrote: > The BEAM format number and the number of the highest opcode are stored in the code chunk: > https://github.com/erlang/otp/blob/master/erts/emulator/beam/beam_load.c#L1714-L1757 > > -Constantin > > > Am 28.02.2016 um 12:53 schrieb Vlad Dumitrescu: > >> Hi! >> >> Is there some information in the beam files that would identify the compatibility with the current runtime, or is it only when trying to execute incompatible code that it is detected? >> >> best regards, >> Vlad > From vladdu55@REDACTED Sun Feb 28 15:02:44 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Sun, 28 Feb 2016 15:02:44 +0100 Subject: [erlang-questions] Detect beam compilation target In-Reply-To: <9C19295B-F06E-4907-911A-A93A28D8EB04@rack.li> References: <24EEE339-6F27-497E-B7CB-4D8D7011D614@rack.li> <9C19295B-F06E-4907-911A-A93A28D8EB04@rack.li> Message-ID: Ah, great! I was about to scan the OTP repository history and create the map from that. Thank you! regards, Vlad On Sun, Feb 28, 2016 at 3:00 PM, Constantin Rack wrote: > There is a list of all opcodes with comments about the runtime version > here: > https://github.com/erlang/otp/blob/master/lib/compiler/src/genop.tab > > -Constantin > > > Am 28.02.2016 um 14:51 schrieb Vlad Dumitrescu: > > > Thanks! I was a bit confused that all the files I had looked at had the > same value, regardless. My error was that I didn't think that the value > depends on the code actually in that module. Checking for example maps.beam > gives a different value. > > > > So now I only have to find out if there is a table mapping the max > opcode value to the beam runtime version. > > > > best regards, > > Vlad > > > > > > On Sun, Feb 28, 2016 at 1:32 PM, Constantin Rack > wrote: > > The BEAM format number and the number of the highest opcode are stored > in the code chunk: > > > https://github.com/erlang/otp/blob/master/erts/emulator/beam/beam_load.c#L1714-L1757 > > > > -Constantin > > > > > > Am 28.02.2016 um 12:53 schrieb Vlad Dumitrescu: > > > >> Hi! > >> > >> Is there some information in the beam files that would identify the > compatibility with the current runtime, or is it only when trying to > execute incompatible code that it is detected? > >> > >> best regards, > >> Vlad > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Sun Feb 28 15:13:09 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Sun, 28 Feb 2016 15:13:09 +0100 Subject: [erlang-questions] Detect beam compilation target In-Reply-To: References: <24EEE339-6F27-497E-B7CB-4D8D7011D614@rack.li> <9C19295B-F06E-4907-911A-A93A28D8EB04@rack.li> Message-ID: For the record, it's a bit complicated, as this message shows: http://erlang.org/pipermail/erlang-questions/2014-February/076868.html regards, Vlad On Sun, Feb 28, 2016 at 3:02 PM, Vlad Dumitrescu wrote: > Ah, great! I was about to scan the OTP repository history and create the > map from that. Thank you! > > regards, > Vlad > > > On Sun, Feb 28, 2016 at 3:00 PM, Constantin Rack > wrote: > >> There is a list of all opcodes with comments about the runtime version >> here: >> https://github.com/erlang/otp/blob/master/lib/compiler/src/genop.tab >> >> -Constantin >> >> >> Am 28.02.2016 um 14:51 schrieb Vlad Dumitrescu: >> >> > Thanks! I was a bit confused that all the files I had looked at had the >> same value, regardless. My error was that I didn't think that the value >> depends on the code actually in that module. Checking for example maps.beam >> gives a different value. >> > >> > So now I only have to find out if there is a table mapping the max >> opcode value to the beam runtime version. >> > >> > best regards, >> > Vlad >> > >> > >> > On Sun, Feb 28, 2016 at 1:32 PM, Constantin Rack >> wrote: >> > The BEAM format number and the number of the highest opcode are stored >> in the code chunk: >> > >> https://github.com/erlang/otp/blob/master/erts/emulator/beam/beam_load.c#L1714-L1757 >> > >> > -Constantin >> > >> > >> > Am 28.02.2016 um 12:53 schrieb Vlad Dumitrescu: >> > >> >> Hi! >> >> >> >> Is there some information in the beam files that would identify the >> compatibility with the current runtime, or is it only when trying to >> execute incompatible code that it is detected? >> >> >> >> best regards, >> >> Vlad >> > >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Mon Feb 29 04:21:56 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 29 Feb 2016 16:21:56 +1300 Subject: [erlang-questions] -pidname( ) Was: Why we need a -module() attribute? In-Reply-To: <940CE93F-68F6-46DC-9634-1DBD57994D5D@pixie.co.za> References: <56C6FD3B.4000303@ninenines.eu> <1643491.uzcKq8YCN9@changa> <56C70914.4000809@ninenines.eu> <617a34874af555f9b8de0d2604c75790.squirrel@chasm.otago.ac.nz> <56C99A94.2020009@ninenines.eu> <56CA5A2A.9020102@cs.otago.ac.nz> <56CA66A6.9070901@ninenines.eu> <56CB8BFF.2020802@cs.otago.ac.nz> <56CD7BDB.3050206@rol.ru> <20160224150705.GA90042@fhebert-ltm2.internal.salesforce.com> <24b9b71226d649598cb2e5514edf7fe0@Jar-EX01.ls.local> <2804FE6E-214B-4076-B2F2-C49F9500E8BA@pixie.co.za> <3c3cc4e9432dd3c8541953b60c86f026.squirrel@chasm.otago.ac.nz> <940CE93F-68F6-46DC-9634-1DBD57994D5D@pixie.co.za> Message-ID: <4b29e6a7-2339-e793-d1ef-f556fb28f6aa@cs.otago.ac.nz> On 27/02/16 3:53 am, Valentin Micic wrote: > > Phrase "atomic variable" in a context of Erlang is a bit > counter-intuitive, maybe even to a point of being an oxymoron. In the context of multiple processes, "atomic variable" is most definitely NOT an oxymoron. > And when you say COMPLETELY INACCESSIBLE FROM THE OUTSIDE, I assume > this would exclude read access as well. Exactly so. > If the whole point of registering processes is to be able to provide > for a static way of sending a message to a process, how would that > work under these circumstances? There are two reasons for using the registry, and the needs of one reason make for a mechanism which is dangerous for the other, which ironically seems to be its main use case. [Private]: a module starts one or more processes. Code within the module needs a way to locate that process/those processes, but the intention is that all communication with it/them should go through functions in the module. This is the use-case that -pidname is designed for. It is precisely like each module having its own registry with a fixed set of keys. [Public]: a process needs to be visible to other modules, possibly even to other nodes via the {node,registry_key}!... technique. This is not addressed by -pidname and remains exactly as it is now. It's not about eliminating the registry, but about providing a SAFER alternative for processes that don't need to be public > Are you saying that one can have more than one -pidname declaration > per module? Yes, certainly. One for each process you wish to name. > > Don't you think this would mean that you would have to know in advance > (that is, compile-time) about all the names the process powered by > this module would ever have? > If so, I am quite surprised that you do not see a problem with that. I don't see a problem with that because if you look at actual Erlang code, module authors *DO* in point of fact know EXACTLY how many names they need, very often. Sure, modules may and some do create lots of processes. But it's unusual for a module to flood the registry with a large number of new entries. Behaviours like gen_server.erl But it would be an extraordinary module that flooded the registry with a large number of new entries. Since a process can act as a concurrent data structure (such as a dictionary, hint hint), if you should happen to have a module that needs a large number of private registry entries, you only need one -pidname: register your concurrent data structure process. Consider, though, asn1rt_driver_handler.erl, which uses asn1_driver_owner and asn1_driver_port. Only asn1rt_driver_handler.erl mentions asn1_driver_owner. Only asn1rt_driver_handler.erl and asn1rt_per_bin_rt2ct.erl mention asn1_driver_port; a one-line function added to asn1rt_driver_handler could make that local too. The problem with these two things being in a global registry is that ANY module could tamper with those entries or do asn1_driver_owner!unload. *That* is what the -pidname proposal is about. > Also, seeing that module name and named pidname variable are not the > same, what would happen if two different modules uses the same > name for a pidname? What part of "local" and "private" is hard to understand? Why would any module know or care what pidnames another module uses? > Of course, you may solve this problem by indicating that these are > COMPLETELY INACCESSIBLE FROM THE OUTSIDE, but again, assuming that the > reason for registration is to be able to have a "static" way to send a > message to a process, what would be the point of having them in the > first place if they are COMPLETELY INACCESSIBLE FROM THE OUTSIDE. Once again, do bear in mind that the -pidname proposal is an ADDITION to the existing registry, not a complete replacement for it. If you have a process that you *want* other modules to be able to talk to directly, just keep on using the existing registry. A lot of registry entries created now *would* be private if there were only some way to make them so. > Given the Erlang standard syntax, it stands to reason that > pidname:set( , self() ) invokes a function set, > that belongs to another module (pidname), and, as such, goes against > your earlier stipulation that -pidname declarations are COMPLETELY > INACCESSIBLE FROM THE OUTSIDE. Oh please. The special built-in functions necessary to make the approach work at all do not honestly count as *outside*. They're *inside*; part of the system machinery. > And, if they are accessible from the outside via pidname:set( atom you declared>, self() ), how is that any safer than the > registration mechanism already in place? Because calls to the pidname: modules would be special syntax recognised by the compiler. > See a contradiction that I attempted to point out above... Nope. Can't see it. > I wish I can understand this as easily as you can. The key thing is that - there are modules that create processes that they WANT to be public For those modules, the existing registry is fine. - but there are also modules that create a fixed set of named processes that they register only so that they can find them again, which WOULD be local for safety if only there were some way to do that. The details of the -pidname proposal are not important. The idea that the second class of modules (or even the second use; a module might want to do both things) exists in abundance in the OTP sources and deserves support, THAT is the key point. > In ideal situation, you would have 1:1 correspondence between a module > and a process. Why? I've already pointed out asn1rt_driver_handler, which has TWO processes it needs to know. It's not the only one. > However, this is rarely the case; e.g. what we register as a > gen_server is not the gen_server module, but the actual behavior that > runs on top of gen_server. So, which -pidname variable should we use? I am having a seriously hard time understanding why you think there is a question here. If -pidnames existed, gen_server would only be able to use -pidnames declared inside the gen_server module, that is, it would only be able to register *locally* processes it intended to keep private. Local, private. gen_server would not be able to use any other module's pidnames. There is a way out. You could have an extended interface to gen_server in which the client module provided a call-back providing indirect, controlled, access to selected pidnames. You'd need to ensure that *only* gen_server could call that call-back, but that's what -export_to (also proposed long ago) is for. > And let me just reiterate a point that I've made commenting on your > Second point above -- are we to provide gen_server with as many names > as we have behaviors? Is that a serious question? How often does it need saying? -pidname is not a REPLACEMENT for the registry we already have. It is supposed to offer something DIFFERENT from that, and so ADDITIONAL to it. -pidnames aren't *supposed* to do everything that the registry does; in fact not being ABLE to do that is what -pidname is all about. > Of course not. However, if we have a module gen_server declaring a > -pidname( gen_server ), and if we send a message to a gen_server, > where would this message go? You've lost me. There is no magic connection between a -pidname in some module that happens to be called gen_server and "a gen_server", unless THAT module contains code that establishes such a connection, and then a message pidname:get(gen_server) ! whatever gets sent to whatever process that module's author chose to bind that pidname to. > The point is, you cannot possibly predict all deployments any given > module may have. Sorry to swear, but what the hell does that have to do with the -pidname proposal? The -pidname proposal is all about HIDING LOCAL STATE that currently has to be made public. For the purposes for which -pidname was designed, you don't know and you don't *care* about "all deployments any given module may have" EXCEPT that you don't want them tampering with your internal name::process bindings. In fact, the fact that you cannot make this prediction you've dragged in is the very reason -pidname is needed: if I want to refer to a process as fred within one of my modules, I currently have no defence against you using the same name. It would not surprise me if someone could come up with a better way to protect names for processes that should not be subject to tampering. Arguably Lawrie Brown's Safe Erlang "sandboxes" could do this by having per-sandbox registries. I spent months trying to come up with something that could be fast and safe, and then put it aside hoping someone else would do better. From Catenacci@REDACTED Mon Feb 29 02:18:21 2016 From: Catenacci@REDACTED (Onorio Catenacci) Date: Sun, 28 Feb 2016 20:18:21 -0500 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: <56D18081.7030307@ninenines.eu> References: <56D18081.7030307@ninenines.eu> Message-ID: Sorry I've not been able to get back to you sooner, Lo?c. As I said, I got that impression from people I know who are working with production Erlang systems--and I don't believe I misunderstood them on that point. Maybe they need to learn a bit more about Erlang too. -- Onorio On Sat, Feb 27, 2016 at 5:54 AM, Lo?c Hoguin wrote: > On 02/26/2016 10:27 PM, Onorio Catenacci wrote: > >> While there may be other considerations you want to keep in mind, >> there's one, to my mind, major point that's been neglected in the >> discussion of using Elixir: macros. >> >> I have been led to understand by people far more familiar with Erlang >> than I am that in order to use certain behaviours from OTP one has to >> write boilerplate code for several callbacks whether one wants to modify >> the behavior from defaults or not. >> >> Elixir's macros allow the library to provide default implementations for >> these callback functions. Of course a developer can override the >> defaults if needed but not having to code what I don't need strikes me >> as a big advantage--especially if we're discussing a new developer who >> doesn't already know either Erlang or Elixir. >> > > I'm not even sure what makes you think you need macros to provide default > implementations? You don't. The gen_server for example has an optional > callback with a default implementation: format_error. That it doesn't > provide defaults for other callbacks is purely a design choice, and not a > limitation of the language. You could trivially extend gen_server to make > all callbacks optional, no need to have macros for that... > > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > -- Onorio Catenacci http://onor.io http://www.google.com/+OnorioCatenacci -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Mon Feb 29 12:40:25 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 29 Feb 2016 12:40:25 +0100 Subject: [erlang-questions] maps or records? In-Reply-To: <56D24A9D.4090109@gmail.com> References: <56D23C82.7060409@gmail.com> <56D24A9D.4090109@gmail.com> Message-ID: On Sun, Feb 28, 2016 at 2:17 AM, Michael Truog wrote: > Records are preprocessing syntax (sweet syntactic sugar) for tuples and I > am just going based on my benchmarking of tuples. I know tuples are > generally ok for 100 elements and less (not trying to encourage making > large records with tons of values). The tipping point will vary on how many updates you are doing, the given machine architecture, and if you are looking at the added GC pressure as well in your benchmarks. Updating a tuple will copy the tuple as a whole, so if you update a single field only, this is a lot of work to do which has little value. I often tend to use a map as a monovariant construction: all the keys have the same type and all the values have the same type. In contrast, I use records as product types: they have static finite size and every key encompasses a specific different type. But of course, the neat thing is you can have polyvariant maps, in Erlang should you need them. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Mon Feb 29 13:21:40 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Mon, 29 Feb 2016 12:21:40 +0000 Subject: [erlang-questions] maps or records? In-Reply-To: References: <56D06DE4.8030204@khandkar.net> Message-ID: Thanks all for the answers it's useful ! On Fri, Feb 26, 2016 at 5:56 PM Garrett Smith wrote: > On Fri, Feb 26, 2016 at 10:23 AM Benoit Chesneau > > >> So would you se a maps instead of a record when you need to share the >> returned result with apps that depends on your library? Instead of making >> that result opaque and provide functions to access it? Which sound >> ridiculous sometimes, since at the end what you do is a get on a record key >> ... Curious what others do in that case? >> > > What's the nature of the data? If it's a map of keys to values, I think > map is a good data type. E.g. I think a database row is well represented as > a map, as columns/fields tend to differ based on what you're retrieving. > > Maybe a good test is the one you mentioned - if you're module is simply > providing a bunch of pass-throughs to peel off record fields, it wants to > be a map. > I was thinking to this simple example: I have a function hackney_url:parse_url() returning a record #hackney_url{} . This record can be used internally but also externally by the applications that need it. Which requires for now to include "hackney_lib.hrl" . The record will likely change. On the other hands I am not sure I like to have to import the include file to get its definition (or copy this definition in another file, or just use the tuple) . So in that case I wonder if using a maps wouldn't be more appropriate. - benoit -------------- next part -------------- An HTML attachment was scrubbed... URL: From santif@REDACTED Mon Feb 29 15:51:00 2016 From: santif@REDACTED (=?UTF-8?Q?Santiago_Fern=C3=A1ndez?=) Date: Mon, 29 Feb 2016 11:51:00 -0300 Subject: [erlang-questions] maps or records? In-Reply-To: References: <56D06DE4.8030204@khandkar.net> Message-ID: On Mon, Feb 29, 2016 at 9:21 AM, Benoit Chesneau wrote: > I have a function hackney_url:parse_url() returning a record > #hackney_url{} . This record can be used internally but also externally by > the applications that need it. Which requires for now to include > "hackney_lib.hrl" . The record will likely change. On the other hands I am > not sure I like to have to import the include file to get its definition > (or copy this definition in another file, or just use the tuple) . You can provide functions to access data in #hackney_url{}, like this: hackney:get_host(Url) and it isn't required to include hackney_lib.hrl in client code. -- Santiago -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Mon Feb 29 15:54:48 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Mon, 29 Feb 2016 15:54:48 +0100 Subject: [erlang-questions] maps or records? In-Reply-To: References: <56D06DE4.8030204@khandkar.net> Message-ID: On Mon, Feb 29, 2016 at 3:51 PM, Santiago Fern?ndez wrote: > > On Mon, Feb 29, 2016 at 9:21 AM, Benoit Chesneau > wrote: > >> I have a function hackney_url:parse_url() returning a record >> #hackney_url{} . This record can be used internally but also externally by >> the applications that need it. Which requires for now to include >> "hackney_lib.hrl" . The record will likely change. On the other hands I am >> not sure I like to have to import the include file to get its definition >> (or copy this definition in another file, or just use the tuple) . > > > You can provide functions to access data in #hackney_url{}, like this: > > hackney:get_host(Url) > > and it isn't required to include hackney_lib.hrl in client code. > > > Yeah but like i said my initial mail, i think it's quite embarrassing when you could just pass a maps that already have this get function. On first thought thought :) - benoit -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Mon Feb 29 16:04:03 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Mon, 29 Feb 2016 16:04:03 +0100 Subject: [erlang-questions] maps or records? In-Reply-To: References: Message-ID: <56D45DE3.7090100@ninenines.eu> On 02/26/2016 03:50 PM, Benoit Chesneau wrote: > Hi all, > > i tends these days to use maps instead of records, so you dont' have to > include a file to retrieve the "schema" but I wonder if this is a good > idea or not. Is there still some usage for records vs maps? For what it's worth, Cowboy 2 will have a (non-opaque) map for Req, and still keep functions around for those that prefer or need those. So things like method, host, path etc. can be accessed directly from the map. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From siraaj@REDACTED Mon Feb 29 16:12:52 2016 From: siraaj@REDACTED (Siraaj Khandkar) Date: Mon, 29 Feb 2016 10:12:52 -0500 Subject: [erlang-questions] maps or records? In-Reply-To: References: <56D06DE4.8030204@khandkar.net> Message-ID: <56D45FF4.5040305@khandkar.net> On 2/29/16 7:21 AM, Benoit Chesneau wrote: > > I was thinking to this simple example: > > I have a function hackney_url:parse_url() returning a record #hackney_url{} > . This record can be used internally but also externally by the > applications that need it. Which requires for now to include > "hackney_lib.hrl" . I strongly favor dedicated include files per record, as it is the least invasive to the user's namespace. Also fully-qualified (i.e. app-name-prefixed) (public) record names, which is what you already did above. > The record will likely change. This should be addressed by (hopefully semantic) versioning of the app and users' considerations of what accessors need updating upon upgrade. > On the other hands I am > not sure I like to have to import the include file to get its definition > (or copy this definition in another file, or just use the tuple) . > > So in that case I wonder if using a maps wouldn't be more appropriate. > Ideally, a language would allow you to reference exported records from other modules, but, given what we have so far in Erlang - I don't think there's an ideal answer - it's your artistic decision, what you want to compromise :) (as I already mentioned earlier, I still favor records for such cases (a product type), despite some of their inconveniences, but I also never do runtime code upgrades...) From zxq9@REDACTED Mon Feb 29 16:14:11 2016 From: zxq9@REDACTED (zxq9) Date: Tue, 01 Mar 2016 00:14:11 +0900 Subject: [erlang-questions] maps or records? In-Reply-To: References: Message-ID: <1745041.bjj7bu8HcB@burrito> On Monday 29 February 2016 11:51:00 Santiago Fern?ndez wrote: > On Mon, Feb 29, 2016 at 9:21 AM, Benoit Chesneau > wrote: > > > I have a function hackney_url:parse_url() returning a record > > #hackney_url{} . This record can be used internally but also externally by > > the applications that need it. Which requires for now to include > > "hackney_lib.hrl" . The record will likely change. On the other hands I am > > not sure I like to have to import the include file to get its definition > > (or copy this definition in another file, or just use the tuple) . > > > You can provide functions to access data in #hackney_url{}, like this: > > hackney:get_host(Url) > > and it isn't required to include hackney_lib.hrl in client code. I generally prefer this in most cases (not all, though -- sometimes you really do want a big hash). The most important thing to me about it is that you can isolate your promises from your implementations over time. It is a little more work to set up, of course, but it prevents you and all your users from having to perform major project surgery if you wind up realizing that something else is really the way to structure the data. I'm not overly crazy about -type vs -opaque, but it really *is* nice sometimes to reap the benefits of paying for a solid data abstraction up front. But sometimes its totally overkill. Figuring out which situation you're in is the trick. And I'm terrible at it, so I prefer to err on the side of caution, but then project deadlines... (>.<) -Craig From mark.h.bruch@REDACTED Mon Feb 29 16:25:19 2016 From: mark.h.bruch@REDACTED (Mark Bruch) Date: Mon, 29 Feb 2016 16:25:19 +0100 Subject: [erlang-questions] maps or records? In-Reply-To: <1745041.bjj7bu8HcB@burrito> References: <1745041.bjj7bu8HcB@burrito> Message-ID: Wouldn't it be nice with something like Elixir structs for Erlang too? Especially if you could get it to work with type spec/dialyzer (maybe it already does in Elixir?). That would totally replace records for me. From siraaj@REDACTED Mon Feb 29 16:28:25 2016 From: siraaj@REDACTED (Siraaj Khandkar) Date: Mon, 29 Feb 2016 10:28:25 -0500 Subject: [erlang-questions] maps or records? In-Reply-To: <1745041.bjj7bu8HcB@burrito> References: <1745041.bjj7bu8HcB@burrito> Message-ID: <56D46399.60907@khandkar.net> On 2/29/16 10:14 AM, zxq9 wrote: > On Monday 29 February 2016 11:51:00 Santiago Fern?ndez wrote: >> On Mon, Feb 29, 2016 at 9:21 AM, Benoit Chesneau >> wrote: >> >>> I have a function hackney_url:parse_url() returning a record >>> #hackney_url{} . This record can be used internally but also externally by >>> the applications that need it. Which requires for now to include >>> "hackney_lib.hrl" . The record will likely change. On the other hands I am >>> not sure I like to have to import the include file to get its definition >>> (or copy this definition in another file, or just use the tuple) . >> >> >> You can provide functions to access data in #hackney_url{}, like this: >> >> hackney:get_host(Url) >> >> and it isn't required to include hackney_lib.hrl in client code. > > I generally prefer this in most cases (not all, though -- sometimes you really > do want a big hash). The most important thing to me about it is that you can > isolate your promises from your implementations over time. It is a little more > work to set up, of course, but it prevents you and all your users from having > to perform major project surgery if you wind up realizing that something else > is really the way to structure the data. > > I'm not overly crazy about -type vs -opaque, but it really *is* nice sometimes > to reap the benefits of paying for a solid data abstraction up front. > > But sometimes its totally overkill. Figuring out which situation you're in is > the trick. And I'm terrible at it, so I prefer to err on the side of caution, > but then project deadlines... (>.<) I don't think [accessor functions vs record labels] is the right dichotomy in these sort of cases. The dichotomy is only that of [API vs internal representation]. You can just as well have a #bar{}, which is a public representation of hidden #foo{}; foo module then just defines foo:to_bar/1 function and you honor the structure of #bar{} just the same as you honor the names and signatures of functions in a public API. In #foo{}, you can then do whatever you want, without telling the users. In other words, I'm saying it is very possible to have this cake (of public structure) and eat it too (in private, without anyone accessing the rapidly-changing, secret cream) :) From g@REDACTED Mon Feb 29 19:12:21 2016 From: g@REDACTED (Garrett Smith) Date: Mon, 29 Feb 2016 18:12:21 +0000 Subject: [erlang-questions] maps or records? In-Reply-To: References: <56D06DE4.8030204@khandkar.net> Message-ID: On Mon, Feb 29, 2016 at 6:21 AM Benoit Chesneau wrote: > Thanks all for the answers it's useful ! > > On Fri, Feb 26, 2016 at 5:56 PM Garrett Smith wrote: > >> On Fri, Feb 26, 2016 at 10:23 AM Benoit Chesneau >> >> >>> So would you se a maps instead of a record when you need to share the >>> returned result with apps that depends on your library? Instead of making >>> that result opaque and provide functions to access it? Which sound >>> ridiculous sometimes, since at the end what you do is a get on a record key >>> ... Curious what others do in that case? >>> >> >> What's the nature of the data? If it's a map of keys to values, I think >> map is a good data type. E.g. I think a database row is well represented as >> a map, as columns/fields tend to differ based on what you're retrieving. >> >> Maybe a good test is the one you mentioned - if you're module is simply >> providing a bunch of pass-throughs to peel off record fields, it wants to >> be a map. >> > > > I was thinking to this simple example: > > I have a function hackney_url:parse_url() returning a record > #hackney_url{} . This record can be used internally but also externally by > the applications that need it. Which requires for now to include > "hackney_lib.hrl" . The record will likely change. On the other hands I am > not sure I like to have to import the include file to get its definition > (or copy this definition in another file, or just use the tuple) . > At least historically the canonical form here is to use a record and provide the definition in a public facing include. I'm thinking of file:read_info as an example. I think it's less obvious to use maps. I'm starting to use maps for process state just to see how it feels - it's not a major break through for me. I miss the very rudimentary type safety - it feels a bit odd to structure typed values as maps. I poked around corresponding Python interfaces to see what they're using and of course typey values like parsed URLs are typically objects. Using dicts would again be odd there. I say keep it old school! -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Mon Feb 29 19:39:48 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 29 Feb 2016 19:39:48 +0100 Subject: [erlang-questions] maps or records? In-Reply-To: <56D46399.60907@khandkar.net> References: <1745041.bjj7bu8HcB@burrito> <56D46399.60907@khandkar.net> Message-ID: On Mon, Feb 29, 2016 at 4:28 PM, Siraaj Khandkar wrote: > In other words, I'm saying it is very possible to have this cake (of > public structure) and eat it too (in private, without anyone accessing the > rapidly-changing, secret cream) :) You are so close to the idea of view types, and Torben hinted me I should say something about them :) One advantage of a function is that it doesn't have to give you the field outright, but can provide ways to dynamically cast that structure into another structure you export. Two really good examples is that you can return some kind of tuple depending on the contents of the data and thus you can build a case match on the data. This ad-hoc for-the-purpose-construction of a datatype can often yield far more precise code since it splits analysis of what is *in* the datastructure from computation which says what to *do* about that data. Say we have a #uri{} record. The path component has many possible representations: a binary(), a list of binaries, an iterator, and so on. Different view-functions would give you different ways to handle that binary. Another nice thing is that functions can give you are zipper/derivative/delimiting-continuation over the data in the structure. The path-component can then be unfolded one path-entry at a time: pth(X) -> case http_uri:unfold_path(X) of none -> ...; {some, C, X2} -> ... C ... pth(X2) end. or you can imagine a gb_trees/gb_sets like iterator over the data structure. A plain map() cannot give any of these ideas, and its transparency also tightly couples your code to the map() structure. So one has to carefully weigh what kind of interface you want in the long run. I much prefer views of the data structures if possible since it allows for more freedom in the long run, but it requires you to be able to figure out what kind of functions you would need a priori on the data. On the other hand, it provides for much better hiding of what is going on inside the module, which allows you more flexibility regarding backwards compatibility. In other words: exporting records across application boundaries tend to lock them down, so be pretty sure they are set in stone. Also think hard if you want every user to implement the analysis code. The view is much like in a database: do you export the raw records, or do you expose a transactional API through stored procedures which tell you what you can do with the data? -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From thoffmann@REDACTED Mon Feb 29 21:02:24 2016 From: thoffmann@REDACTED (Torben Hoffmann) Date: Mon, 29 Feb 2016 21:02:24 +0100 Subject: [erlang-questions] maps or records? In-Reply-To: References: <1745041.bjj7bu8HcB@burrito> <56D46399.60907@khandkar.net> Message-ID: <56d4a3dc.45ce190a.394c6.609a@mx.google.com> Jesper Louis Andersen writes: > [ text/plain ] > On Mon, Feb 29, 2016 at 4:28 PM, Siraaj Khandkar > wrote: > >> In other words, I'm saying it is very possible to have this cake (of >> public structure) and eat it too (in private, without anyone accessing the >> rapidly-changing, secret cream) :) > > > You are so close to the idea of view types, and Torben hinted me I should > say something about them :) > > One advantage of a function is that it doesn't have to give you the field > outright, but can provide ways to dynamically cast that structure into > another structure you export. Two really good examples is that you can > return some kind of tuple depending on the contents of the data and thus > you can build a case match on the data. This ad-hoc > for-the-purpose-construction of a datatype can often yield far more precise > code since it splits analysis of what is *in* the datastructure from > computation which says what to *do* about that data. Say we have a #uri{} > record. The path component has many possible representations: a binary(), a > list of binaries, an iterator, and so on. Different view-functions would > give you different ways to handle that binary. > > Another nice thing is that functions can give you are > zipper/derivative/delimiting-continuation over the data in the structure. > The path-component can then be unfolded one path-entry at a time: > > pth(X) -> > case http_uri:unfold_path(X) of > none -> ...; > {some, C, X2} -> > ... C ... pth(X2) > end. > > or you can imagine a gb_trees/gb_sets like iterator over the data structure. > > A plain map() cannot give any of these ideas, and its transparency also > tightly couples your code to the map() structure. So one has to carefully > weigh what kind of interface you want in the long run. I much prefer views > of the data structures if possible since it allows for more freedom in the > long run, but it requires you to be able to figure out what kind of > functions you would need a priori on the data. On the other hand, it > provides for much better hiding of what is going on inside the module, > which allows you more flexibility regarding backwards compatibility. > > In other words: exporting records across application boundaries tend to > lock them down, so be pretty sure they are set in stone. Also think hard if > you want every user to implement the analysis code. The view is much like > in a database: do you export the raw records, or do you expose a > transactional API through stored procedures which tell you what you can do > with the data? > > Why attempt to write a half-baked answer when you can ask Jesper to explain it well?!?! [1] Look at Cowboy and Webmachine for examples of how to provide functions to work with a request. Both have a very rich API for extracting relevant things out of the request record. No need to start poking into that record on your own. Cheers, Torben [1] One Erlang beer token to you, Jesper. -- Torben Hoffmann Architect, basho.com M: +45 25 14 05 38 From erlang@REDACTED Mon Feb 29 22:23:50 2016 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 29 Feb 2016 22:23:50 +0100 Subject: [erlang-questions] maps or records? In-Reply-To: References: Message-ID: On Fri, Feb 26, 2016 at 3:50 PM, Benoit Chesneau wrote: > Hi all, > > i tends these days to use maps instead of records, so you dont' have to > include a file to retrieve the "schema" but I wonder if this is a good idea > or not. Is there still some usage for records vs maps? It depends - what goals are you trying to achieve? I can imaging different goals: - efficiency - clarity - speed of prototyping - ease of debugging Depending upon what you want I'd chose different methods. Records are the most efficient, but need a bit of tender loving care when you modify code. Maps are great if you don't really know what keys you want and are doing exploratory programming. Perhaps you should start with maps and change the representation to records when you fully understand your problem. /Joe > > - beno?t > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ok@REDACTED Mon Feb 29 23:34:32 2016 From: ok@REDACTED (ok@REDACTED) Date: Tue, 1 Mar 2016 11:34:32 +1300 Subject: [erlang-questions] maps or records? In-Reply-To: References: <56D06DE4.8030204@khandkar.net> Message-ID: <957a04f3e794b9b66ff5e94448edb718.squirrel@chasm.otago.ac.nz> > On Mon, Feb 29, 2016 at 6:21 AM Benoit Chesneau > I'm starting to use maps for process state just to see how it feels - it's > not a major break through for me. I miss the very rudimentary type safety > - > it feels a bit odd to structure typed values as maps. One of the core design issues in the frames proposal -- the road not taken -- was that frames should be as amenable to type checking as records. It's not obvious to me that maps *couldn't* be type checked in like fashion. From ok@REDACTED Mon Feb 29 23:49:58 2016 From: ok@REDACTED (ok@REDACTED) Date: Tue, 1 Mar 2016 11:49:58 +1300 Subject: [erlang-questions] maps or records? In-Reply-To: <56D45FF4.5040305@khandkar.net> References: <56D06DE4.8030204@khandkar.net> <56D45FF4.5040305@khandkar.net> Message-ID: <34840908b83d221c7266dd69c9b3ed6a.squirrel@chasm.otago.ac.nz> >> The record will likely change. And this is the important clue. This being the case, (a) the module must be so written that clients don't care what the data structure is. Treat it as an opaque type. Make sure all access to the data is via functions exported from the module. (b) Now the question is what will make it easiest to move from one version of your design to another. Maps answer the question "how can we make it so that existing code isn't broken by ADDING new fields, only by renaming or deleting old ones, and reduce the breakage from changes to fields that aren't mentioned often." Records answer the question "how can we notice breakage really quickly". Maps are less ugly than records, but I think records are still probably your best bet here. Using the access functions listed under (a) inside your module will also help to make change easier.