From ok@REDACTED Mon May 2 02:28:47 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 2 May 2016 12:28:47 +1200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> Message-ID: <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> On 29/04/16 7:47 PM, Vlad Dumitrescu wrote: > > There are already tools experimenting with such ways of structuring > code. For example, Eclipse has Mylyn which can provide a filtered view > of the code (https://en.wikipedia.org/wiki/Task-focused_interface), > hiding aspects that are currently irrelevant. CodeBubbles > (http://cs.brown.edu/~spr/codebubbles/ > ) tries a non-linear editing > experience where you can see dependencies explicitly. I had a fairly brief look at CodeBubbles. The set-up guide calls it "alpha" and "version 0.50" and has a copyright date of 2010. The codebubbles Wiki doesn't seem to have been updated for quite a few years. It may be that CodeBubbles itself is more up to date, but it's a bit worrying when the documentation isn't. I am deeply unhappy about anything that requires Eclipse. I have no trouble with Xcode or NetBeans, but find trying to get even Hello World going dauntingly complex in Eclipse. I also read through the tutorial, and saw very little that I wasn't already familiar with in Smalltalk. The tutorial says that it shows relationships "between BUBBLES" and refers to "CALLS". For what it's worth, Smalltalk is precisely where my "why can't I see the structure" journey started and what the annotations I've been working on were developed for. Let's face it, an IDE cannot display information it has not been told about. Can you explain what relationships CodeBubbles displays? From spanemangalore@REDACTED Mon May 2 03:26:31 2016 From: spanemangalore@REDACTED (Sachin Panemangalore) Date: Sun, 1 May 2016 18:26:31 -0700 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> Message-ID: >From a best-practices point of view, I have found the following efforts helpful. 1. Dialyzer Annotations : - Types in function definitions or when accessing elements from a collection help big-time to build context of 'what' to expect from a particular data-structure . In absence of this you need to go back and forth between functions which write to a collection and functions which are reading from a collection. In Erlang , dialyzer annotations help to add this at the function head level . I would say , its a Must to define custom type names in annotations "term()" or "any()" should rarely if any be used. 2. Human readable comments at the top of the file help the user to impart context of the ensuing logic to the reader. The internal logic may change based on implementation , but the purpose for the existence of the file rarely changes. -behavior only specifies the type of object being used but not the human context around this. 3. Layering of Code: - Code when written , must separate data-access API and Data-representation functions. Code like lists:flatmap( fun (abc) -> ; (xyz)-> end , [ some_list ] ) should only be seen at the lowest levels. The higher-level code should always look like NewObjectState = my_action_on_my_business_object( ObjectState ). Folks who want to understand how the module works from a business-logic point of view, may read the higher-layers of code. 4. Tools like Cscope , Code-Bubbles, Source-Graph may help the reader understand the Call-graphs. 5. Comments on Concise Code : As the developer gains more expertise in functional programming , the developer writes more and more concise code , for he has moved on from being a developer to an artist. Concise is beautiful. But this also brings about a moral duty for the artist to add comments for the 'tricky parts' , which will help other developers bridge the gap and appreciate the beauty. cheers sachin On Wed, Apr 27, 2016 at 8:52 PM, Richard A. O'Keefe wrote: > I've been thinking for some time of writing a paper with the > title "Why can't I see the structure?" based on the idea that > modules in every programming language I know look like blobs. > I'm aware of visual notations like UML, BON, SDL, and what > was it, Visual Erlang? But for me, those are just spaghetti > with meatballs; once you get beyond just a handful of boxes > in your diagram, all diagrams look much the same. In any > case, I'm interested in the medium scale. > > Why can't I see the structure in a 3000-line module, or even > a 1000-line module? (I am not asserting that Erlang is > particularly bad here. It isn't.) > > The kind of structure I'm interested in can, I think, be > described as *rhetorical* structure, like relationships > between paragraphs. > > My *belief* is that if this structure were made explicit, > perhaps by textual structure, perhaps by annotations, perhaps > by some graphical form (but probably derived from annotations), > it would be easier to understand medium-sized wodges of code. > > I'm aware of annotation support in languages like Java and C# > and for that matter, Smalltalk, but with the exception of > Smalltalk, nobody seems to be using annotations in this way > (and that exception is me). > > I'd be very interested in hearing from anyone else who has been > thinking in this area. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nayibor@REDACTED Mon May 2 07:42:24 2016 From: nayibor@REDACTED (Nuku Ameyibor) Date: Mon, 2 May 2016 05:42:24 +0000 Subject: [erlang-questions] Erlang for Payment Systems In-Reply-To: <3E7E1782-0203-4F03-BD1E-6F42949E7446@redops.org> References: <3E7E1782-0203-4F03-BD1E-6F42949E7446@redops.org> Message-ID: something else you might also want to look into is pcidss and padss security requirements. you will be required to comply with these requirements if you intend to communicate with visa,mastercard and other international payment schemes . On Sat, Apr 30, 2016 at 6:32 PM, Drew Varner wrote: > Joe's #2 goal of correctness and Garrett's use of success typing (Dialyzer > and type specs) naturally point to property-based testing. Erlang > QuickCheck (EQC) and other property-based testing frameworks provide > additional confidence in the code. > > Specifically, generators and shrinking strategies work hand-in-hand with > Erlang type specs. Basic property-based testing provides a more thoughtful > way to test code with solid coverage and the ability to try many edge cases > that a developer may have missed in traditional unit tests. Advanced > property-based testing with stateful models offers the opportunity to > uncover elusive concurrency errors that typically show up in obscure bug > reports. If I were playing with money in an Erlang program, I'd leverage > these tools. > > > On Apr 30, 2016, at 11:52 AM, Garrett Smith wrote: > > > > Decoding data received on the network does not sound like string > > manipulation - but data decoding. This is one of Erlang's strengths. > > > > Once decoded, transforming data and generating string like output > > (e.g. encoding JSON or XML) can be both very fast and memory efficient > > - it depends on what you're doing as programmer. > > > > There's nothing about Erlang that's particularly weak in these areas - > > certainly not to ward you off without specifics. > > > > I'd say that Erlang's glaring weakness wrt other language options is > > lack of static typing. However, this comes with the benefit of hugely > > simplifying distribution. That's a deliberate trade off that's been > > made in this language. There are ways to mitigate the language's lack > > of static typing (e.g. see TypEr and Dialyzer). > > > > Also consider that there are classes of bugs that cannot be detected > > by static type checks. And even if you code is bug free (good luck) > > your code has nothing to say about hardware failure. You need a system > > that reaches outside your program, which means distribution of some > > sort. > > > > What you're describing is a critical, long running, unattended system. > > Those are really hard to build and you should put a premium on tools > > and languages that help you build them successfully (i.e. they > > actually do what you want at acceptable service levels) at low cost. > > You'll be hard pressed to find a better language environment along > > these lines than Erlang. > > > >> On Fri, Apr 29, 2016 at 3:09 PM, Hassan Sowwan > wrote: > >> Hello, > >> > >> I am trying to implement payment messaging middleware and would like to > >> explore the option of using Erlang/OTP. > >> > >> The application will be used in banking industry to interface with EFT > >> payment switch/networks and core banking system to process card > >> transactions. > >> > >> It will be responsible to perform following tasks: > >> > >> > >> Communicate with external interfaces using ISO 8583 messaging format ( > thru > >> TCP/IP) > >> Receive huge amount of data over the socket ( HEX, BINARY, EBCIDIC), > which > >> represents financial transactions. > >> Parse/decode the received data. > >> Perform some checking in database for validation > >> Interface with host security module to validate customer PIN and other > >> security checks. > >> Sends the request to core banking system via XML or web services call > >> Respond back to external interfaces by formulating the response message > in > >> ISO 8583 format > >> > >> > >> Obviously, such applications have to be concurrent and fast enough to > >> process transactions within few seconds. > >> > >> Now my question here, is Erlang a good choice for implementing this > type of > >> applications ? > >> Can Erlang handle string processing efficiently without impacting the > system > >> performance? > >> As stated before, there will be a lot of string manipulation to decode > data > >> received over the network, so I am not sure whether erlang fits > perfectly or > >> not. > >> > >> > >> > >> > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Mon May 2 09:51:58 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 2 May 2016 09:51:58 +0200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> Message-ID: Hi, On Mon, May 2, 2016 at 2:28 AM, Richard A. O'Keefe wrote: > On 29/04/16 7:47 PM, Vlad Dumitrescu wrote: > >> >> There are already tools experimenting with such ways of structuring code. >> For example, Eclipse has Mylyn which can provide a filtered view of the >> code (https://en.wikipedia.org/wiki/Task-focused_interface), hiding >> aspects that are currently irrelevant. CodeBubbles ( >> http://cs.brown.edu/~spr/codebubbles/ < >> http://cs.brown.edu/%7Espr/codebubbles/>) tries a non-linear editing >> experience where you can see dependencies explicitly. >> > > I had a fairly brief look at CodeBubbles. The set-up guide calls it > "alpha" and "version 0.50" > and has a copyright date of 2010. The codebubbles Wiki doesn't seem to > have been updated > for quite a few years. It may be that CodeBubbles itself is more up to > date, but > it's a bit worrying when the documentation isn't. I am deeply unhappy > about anything that > requires Eclipse. I have no trouble with Xcode or NetBeans, but find > trying to get even Hello > World going dauntingly complex in Eclipse. I also read through the > tutorial, and saw very little > that I wasn't already familiar with in Smalltalk. The tutorial says that > it shows relationships > "between BUBBLES" and refers to "CALLS". > > For what it's worth, Smalltalk is precisely where my "why can't I see the > structure" journey > started and what the annotations I've been working on were developed for. > Let's face it, > an IDE cannot display information it has not been told about. > > Can you explain what relationships CodeBubbles displays? > I haven't been using CodeBubbles, only looked at it when it was "young". I took it as an example of what could be done (a different way to look at the code than simple text), not necessarily an example of how it should be done. If I was more familiar with Smalltalk, I would have used it as an example too. Regarding Eclipse, it is a huge beast and IMHO most of the complexity and issues are because it has fallen victim to its own success - with a huge third-party ecosystem, one can't discard old APIs and improve them in non-backward compatible ways. This is a reason why I am moving (slowly, but leisurely, as a friend of mine would say) the implementation core parts of erlide from Java to Erlang, with the goal of being able to provide interesting information about a code base that any IDE or editor could use. The challenging part is that this information has to be available even for incomplete or invalid code, as well as for code containing preprocessor directives. Tools can help in many ways, I think a big problem is that every other coder would like a different kind of support :-) If there is not enough user pressure behind a tool/feature, then it will wither and eventually die. > Let's face it, an IDE cannot display information it has not been told about. Yes, it can (depending on how one understands "told about"). Consider xref -- one doesn't explicitly annotate the code, the information is there and is gathered by the tool. Finding unused functions is otherwise quite tedious. Similar gathering of relevant information from non-local places in the code are helpful for example for showing documentation or type specs while one is typing. It's possible to even scan for all spawn calls and draw a diagram of the processes in an application. Going the other way around, an IDE can make usage of tools easier - for example when starting to type a function spec, it can run typer in the background and suggest a reasonable starting point. And so on. All these small things can add up to a lot. best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Mon May 2 10:37:57 2016 From: vances@REDACTED (Vance Shipley) Date: Mon, 2 May 2016 14:07:57 +0530 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> Message-ID: I have long had an interest in visualizing code and a dream of a visual development environment. I've pretty much concluded that a "visual programming language" is a pipe dream for all but trivial use cases however graphical modeling is obvious. On May 2, 2016 1:22 PM, "Vlad Dumitrescu" wrote: > It's possible to even scan for all spawn calls and draw a diagram of the processes in an application. Many years ago I wrote a tool which parsed the abstract syntax tree of gen_fsm behaviour callback modules and output .dot files for input to Graphviz which then output a picture of the directed graph of States and Events driving state transitions. This proved very helpful for finding missing event handlers. > Going the other way around, an IDE can make usage of tools easier - I had always wanted to finish that project by going the other way, starting with a graphical editor (e.g. dotty(*)) to create the model of a communicating finite state machine in a graphviz file (foo_fsm.dot) with annotations. Feed that through a utility which created a template gen_fsm behaviour callback module (foo_fsm.erl) with annotations so that it could be read back into .dot for further graphical editing. Another, more generally useful, IDE utility I have always planned is a graphical editor for supervision hierarchies. I would love to start new Erlang applications on a canvas with a palette of application master, supervisor and worker. Drag and drop the supervisors, connect them with lines to their children, annotate with the child specifications and init arguments and save it as a template application ready to call application:start(foo). These aren't terribly difficult projects, trivial really compared to what ROK is talking about, but yet (AFAIK) they don't exist. If one was to contemplate doing such a thing I understand that Eclipse has a Graphical Editing Framework (GEF) which may form a good foundation for the graphical editor. Maybe I'll get around to it someday however it's been on my todo list for 15 years now and yet I still use vim as my "IDE". -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Mon May 2 10:57:58 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 2 May 2016 10:57:58 +0200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> Message-ID: On Mon, May 2, 2016 at 10:37 AM, Vance Shipley wrote: > I have long had an interest in visualizing code and a dream of a visual > development environment. I've pretty much concluded that a "visual > programming language" is a pipe dream for all but trivial use cases however > graphical modeling is obvious. > > The thing that I found to be most difficult to handle with graphical tools is that if automatic layout is not good enough, then storing the layout is making it very difficult to handle merges and rebases. At my current job, we just switched from a graphical modeling tool (for UML-like state machines) to a text-based representation and rendering with GraphViz. > On May 2, 2016 1:22 PM, "Vlad Dumitrescu" wrote: > > > Going the other way around, an IDE can make usage of tools easier - > > I had always wanted to finish that project by going the other way, > starting with a graphical editor (e.g. dotty(*)) to create the model of a > communicating finite state machine in a graphviz file (foo_fsm.dot) with > annotations. Feed that through a utility which created a template gen_fsm > behaviour callback module (foo_fsm.erl) with annotations so that it could > be read back into .dot for further graphical editing. > > Another, more generally useful, IDE utility I have always planned is a > graphical editor for supervision hierarchies. I would love to start new > Erlang applications on a canvas with a palette of application master, > supervisor and worker. Drag and drop the supervisors, connect them with > lines to their children, annotate with the child specifications and init > arguments and save it as a template application ready to call > application:start(foo). > A graphical interface is not the only way to improve this kind of editing. It can be done with a tree view and a form-like interface. Eclipse has for example editors for its configuration files that look like the picture here https://dl.dropboxusercontent.com/u/2517505/editor1.png, which is arguably nicer than having to plow through 3-4 files with different formats among which XML is the worst. > These aren't terribly difficult projects, trivial really compared to what > ROK is talking about, but yet (AFAIK) they don't exist. If one was to > contemplate doing such a thing I understand that Eclipse has a Graphical > Editing Framework (GEF) which may form a good foundation for the graphical > editor. Maybe I'll get around to it someday however it's been on my todo > list for 15 years now and yet I still use vim as my "IDE". > Yes, GEF is cool, but it has a quite steep learning curve (which might be added to the steep curve of learning Eclipse, and the relatively steep curve of learning Java, depending on one's current knowledge). best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Mon May 2 11:47:47 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Mon, 02 May 2016 09:47:47 +0000 Subject: [erlang-questions] erlang nif resource & thread safety Message-ID: I had a look in the doc but I can't figure exactly how much a nif resource could live outside the process that created it, ie. how others processes can write/read to it and how to make sure the resource will be accessible to them. So I created a nif resource that have the following properties: - it is ref counted, each time a call is made to it, the counter is incremented and decremented when it return. - the process that create it will live until the vm is closed - write and reads to the resource are thread safe, at least the functions that use resource For now I was thinking to share this resources between others processes via an ETS table. But I am wondering if instead it shouldn't be safer to makes all the call via a nif. With the result of creating some kind of bottleneck. Is there any other pattern I should follow? How does ETS which in my understanding is a nif? - beno?t -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Mon May 2 13:56:35 2016 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 2 May 2016 13:56:35 +0200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> Message-ID: I was thinking. There is another problem, which is not addressed. We just publish final code - but not the process involved to get to the code. Reading completed code does not answer the question "how did the author get to the point where they wrote the code" - the mistakes we made along the path and the lessons learned are not recorded. When I teach programming I always make loads of mistakes. Sometimes I see a mistake coming, which if I were programming at home I would correct "in my head" and type in the correct version - so even if you recorded my keystrokes you would not see what was going on "in my head". Now since *most* programming does go on "in my head" before I even get to type the program in, you won't learn much by watching me. When I teach - I tell people what this internal dialogue is and make the mistakes - so they can learn. I think programming is a craft - and like all crafts has to be learned from a good teacher. I "know" a good program when I see one - but I have great difficulty putting into words *why* it's good. I still find pencil and paper, whiteboards, and writing down in clear English what I'm trying to solve superior to any fancy software. Google is great for searching for solutions to problems "once you know which problem you want to solve" Programming takes a long time to get good at - I've been doing it for 47 years and am still learning - this is because learning to program is actually learning to think and different programming languages and programming styles are really different ways of thinking. After about 35 years of programming I found that most programming moved "into my head" so it was better to think about what would happen than to type in the program and see. Virtually all the code I write is in the form of "micro experiments" - since I'm not sure what will happen if I do something. The "big" or complete program can be written once all the micro-experiments have been done. When I can't write the code - it is always because I haven't understood the problem - this is very frustrating I *think* I've understood the problem - yet the fact I can't write the code is a clear sign that I have not understood the problem - but I do not yet know why. Loading the problem into my brain, sleeping, more study, playing the piano, a glass of wine, stoking the cat usually solves the problem (or not). IDE's in all their glory help me type the solution to my problem into the computer - but they do not help me formulate the correct problem in the first place. Robert: What problem are you solving? Joe: Bla bla bla Robert: That's not the problem, that's the solution. What was the problem? Joe: Ummm, Dunno Clearly writing down what you problem is before solving it would be great. Unfortunately I only ever have a clear description of my problem at the very end when the problem is solved. Chicken or egg first? /Joe On Mon, May 2, 2016 at 10:57 AM, Vlad Dumitrescu wrote: > > On Mon, May 2, 2016 at 10:37 AM, Vance Shipley wrote: >> >> I have long had an interest in visualizing code and a dream of a visual >> development environment. I've pretty much concluded that a "visual >> programming language" is a pipe dream for all but trivial use cases however >> graphical modeling is obvious. > > The thing that I found to be most difficult to handle with graphical tools > is that if automatic layout is not good enough, then storing the layout is > making it very difficult to handle merges and rebases. At my current job, we > just switched from a graphical modeling tool (for UML-like state machines) > to a text-based representation and rendering with GraphViz. > > >> >> On May 2, 2016 1:22 PM, "Vlad Dumitrescu" wrote: >> >> > Going the other way around, an IDE can make usage of tools easier - >> >> I had always wanted to finish that project by going the other way, >> starting with a graphical editor (e.g. dotty(*)) to create the model of a >> communicating finite state machine in a graphviz file (foo_fsm.dot) with >> annotations. Feed that through a utility which created a template gen_fsm >> behaviour callback module (foo_fsm.erl) with annotations so that it could be >> read back into .dot for further graphical editing. >> >> Another, more generally useful, IDE utility I have always planned is a >> graphical editor for supervision hierarchies. I would love to start new >> Erlang applications on a canvas with a palette of application master, >> supervisor and worker. Drag and drop the supervisors, connect them with >> lines to their children, annotate with the child specifications and init >> arguments and save it as a template application ready to call >> application:start(foo). > > A graphical interface is not the only way to improve this kind of editing. > It can be done with a tree view and a form-like interface. Eclipse has for > example editors for its configuration files that look like the picture here > https://dl.dropboxusercontent.com/u/2517505/editor1.png, which is arguably > nicer than having to plow through 3-4 files with different formats among > which XML is the worst. > >> >> These aren't terribly difficult projects, trivial really compared to what >> ROK is talking about, but yet (AFAIK) they don't exist. If one was to >> contemplate doing such a thing I understand that Eclipse has a Graphical >> Editing Framework (GEF) which may form a good foundation for the graphical >> editor. Maybe I'll get around to it someday however it's been on my todo >> list for 15 years now and yet I still use vim as my "IDE". > > Yes, GEF is cool, but it has a quite steep learning curve (which might be > added to the steep curve of learning Eclipse, and the relatively steep curve > of learning Java, depending on one's current knowledge). > > best regards, > Vlad > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From mahesh@REDACTED Mon May 2 14:29:00 2016 From: mahesh@REDACTED (Mahesh Paolini-Subramanya) Date: Mon, 2 May 2016 08:29:00 -0400 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> Message-ID: For my part, I find that most explorations into *structure* tend to rapidly devolve into discussions about *accuracy* and/or fidelity. This probably has a lot to do with the existing tooling, and I'm not quite sure that there is any way around it... FWIW, what I try to do when I am in FredHebertMode (circles and arrows on the back of each sheet) is to push myself to recognize that 1) What is this supposed to be doing? 2) Is this doing what it is supposed to be doing? are different beasts indeed... Cheers p.s. For extra credit, there is actually a level -0- consisting of "What is the author trying to accomplish" On Mon, May 2, 2016 at 4:57 AM, Vlad Dumitrescu wrote: > > On Mon, May 2, 2016 at 10:37 AM, Vance Shipley wrote: > >> I have long had an interest in visualizing code and a dream of a visual >> development environment. I've pretty much concluded that a "visual >> programming language" is a pipe dream for all but trivial use cases however >> graphical modeling is obvious. >> >> The thing that I found to be most difficult to handle with graphical > tools is that if automatic layout is not good enough, then storing the > layout is making it very difficult to handle merges and rebases. At my > current job, we just switched from a graphical modeling tool (for UML-like > state machines) to a text-based representation and rendering with GraphViz. > > > >> On May 2, 2016 1:22 PM, "Vlad Dumitrescu" wrote: >> >> > Going the other way around, an IDE can make usage of tools easier - >> >> I had always wanted to finish that project by going the other way, >> starting with a graphical editor (e.g. dotty(*)) to create the model of a >> communicating finite state machine in a graphviz file (foo_fsm.dot) with >> annotations. Feed that through a utility which created a template gen_fsm >> behaviour callback module (foo_fsm.erl) with annotations so that it could >> be read back into .dot for further graphical editing. >> >> Another, more generally useful, IDE utility I have always planned is a >> graphical editor for supervision hierarchies. I would love to start new >> Erlang applications on a canvas with a palette of application master, >> supervisor and worker. Drag and drop the supervisors, connect them with >> lines to their children, annotate with the child specifications and init >> arguments and save it as a template application ready to call >> application:start(foo). >> > A graphical interface is not the only way to improve this kind of editing. > It can be done with a tree view and a form-like interface. Eclipse has for > example editors for its configuration files that look like the picture here > https://dl.dropboxusercontent.com/u/2517505/editor1.png, which is > arguably nicer than having to plow through 3-4 files with different formats > among which XML is the worst. > > >> These aren't terribly difficult projects, trivial really compared to what >> ROK is talking about, but yet (AFAIK) they don't exist. If one was to >> contemplate doing such a thing I understand that Eclipse has a Graphical >> Editing Framework (GEF) which may form a good foundation for the graphical >> editor. Maybe I'll get around to it someday however it's been on my todo >> list for 15 years now and yet I still use vim as my "IDE". >> > Yes, GEF is cool, but it has a quite steep learning curve (which might be > added to the steep curve of learning Eclipse, and the relatively steep > curve of learning Java, depending on one's current knowledge). > > best regards, > Vlad > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- *Mahesh Paolini-Subramanya That tall bald Indian guy..* *Twitter | Blog | G+ | LinkedIn * -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Mon May 2 14:50:19 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 2 May 2016 14:50:19 +0200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> Message-ID: On Mon, May 2, 2016 at 1:56 PM, Joe Armstrong wrote: > Virtually all the code I write is in the form of "micro experiments" - > since I'm not sure what will happen if I do something. The "big" or > complete program > can be written once all the micro-experiments have been done. > > When I can't write the code - it is always because I haven't > understood the problem - this is very frustrating I *think* I've > understood the problem - yet the fact I can't write the code is a > clear sign that I have not understood the problem - but I do not yet > know why. > > Loading the problem into my brain, sleeping, more study, playing the piano, > a glass of wine, stoking the cat usually solves the problem (or not). > > IDE's in all their glory help me type the solution to my problem into > the computer - but they do not help me formulate the correct problem > in the first place. > All true, Joe, we can't know what we don't know and no amount of the technology we are talking about (i.e. not things that stimulate the brain's creativity and whatnot) will help with that. However, helper tools can help us be faster and more flexible when writing the micro-experiments, which means we can try more of them in the same amount of time and if we have a slightly higher-level view over them maybe we can easier see new ways to combine them, that would otherwise get hidden in the boilerplate code. The tools can also help write a correct final version, too (even if it is just by eliminating the need to always have to stop typing and ask "was it keysearch(Key, List) or keysearch(List, Key)?") Also, most of the developer time is spent not thinking about new stuff, but reading/understanding/debugging/improving existing code, and here tools can help a lot by filtering parts irrelevant for the current task, or by providing a higher-level view of the code, or by making it easy to follow the execution of the code. There was a time when I could run code in my head and I got most often correct results, but that's not working any more... Anyway, even if they are far from perfect, I think everybody can agree that tools help. I consider pen and paper as tools too, by the way. I think that the issue is that everybody has their own thought processes and need tools that fit those, but we can't create custom tools for every person. Going back to a more practical discussion and considering I am a tool developer, I welcome all inputs regarding what kind of tools would help you (as in, all of you) best. I have already started looking at a prototype for an editor for an application's supervision structure, as Vance talked about. I probably won't be able to do everything myself, but it's possible that many people have tools that they wrote for themselves and use with success - maybe those can be shared? best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From marc@REDACTED Mon May 2 15:36:04 2016 From: marc@REDACTED (Marc Worrell) Date: Mon, 2 May 2016 15:36:04 +0200 Subject: [erlang-questions] [ANN] Zotonic release 0.16.0 Message-ID: <2F94A898-0DAC-4545-AF30-11A5CBD8DC68@worrell.nl> Zotonic version 0.16.0 has been released. Zotonic is the Erlang content management system and framework. It includes everything needed to build dynamic and content-rich websites. This release adds, besides bug fixes, collaboration groups. These are groups of users creating and sharing content in a closed group. Download Zotonic 0.16.0 here: https://github.com/zotonic/zotonic/releases/tag/0.16.0 Zotonic version 0.16 and earlier is in production at various systems. Two examples are: ? Jewish Monument http://www.joodsmonument.nl ? LearningStone https://www.learningstone.com Short change log: ? Fixed #1099: added collaboration groups. ? #1227: Added ip_whitelist configuration option to block unwanted access when admin password is ?admin?. ? Serve status site with 503. ? Fixed #1229: users cannot update their person resource. ? Fixed #1236 by removing is_authoritative access check. ? Fixed #1245 by improving the 403 page. ? Fixed #1147 by passing qargs to search. ? Fixed #1230: Firefox file upload error. ? Fixed #1248: Dutch month abbreviation. ? Fixed #1250: view button in case of no update permissions. Next release will be on Monday, June 6. Kind regards, The Zotonic Developers -------------- next part -------------- An HTML attachment was scrubbed... URL: From list1@REDACTED Mon May 2 15:33:37 2016 From: list1@REDACTED (Grzegorz Junka) Date: Mon, 2 May 2016 13:33:37 +0000 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> Message-ID: <5ae800f8-53a7-92a2-47d1-d8a0e76c19e0@gjunka.com> I don't agree that tools help in general, but only in specific tasks. Dialyzer, Xref, common test, all help in their specific tasks. In particular, the tools mentioned in this thread would help other developers to understand some existing code written by someone else. They probably wouldn't help me to understand my own code better, although they could probably help me to find more bugs or to help me restructure the code if I ever needed to. Before I can start writing a new program I always start with a pen and paper, away from the computer. Only once I have everything "sorted out" in my mind, I start coding using Emacs. It's the closest you can get to pen and paper. I am comfortable using IDEs but I don't tend to use them unless required by the task at hand (e.g. when doing GUI's or when they are better supported than command line, e.g. XCode, or Visual Studio). So, I wouldn't exaggerate the tools, although different people approach this problem differently. Some people study better reading a book and some watching video. Some people are better programmers with an IDE and for some (like me) it's an unnecessary burden (and overload of my brain). Having said that, the one area when tools are indispensable and should always be worked on is the automation of daunting and obvious tasks that I, as a developer, shouldn't have to do. Running tests is one example. Installing software, keeping backups, renaming function names in multiple files, intelligently searching for specific code fragments, etc. The more of them and the more things they can do automatically for me the more time I have to write code in Emacs. However, many of those tools seem to be intelligent because they have too many options. A tool works great when it has few options working 99% of the time with default values, but can be extended to a great deal when the extra mile is needed. Many tools, often IDEs, burden users with these unnecessary options. When I need to rename a function name in all my files and a tool that could do it for me shows me 100 options, I would rather prefer to do it manually. Greg On 02/05/2016 12:50, Vlad Dumitrescu wrote: > > On Mon, May 2, 2016 at 1:56 PM, Joe Armstrong > wrote: > > Virtually all the code I write is in the form of "micro experiments" - > since I'm not sure what will happen if I do something. The "big" or > complete program > can be written once all the micro-experiments have been done. > > When I can't write the code - it is always because I haven't > understood the problem - this is very frustrating I *think* I've > understood the problem - yet the fact I can't write the code is a > clear sign that I have not understood the problem - but I do not yet > know why. > > Loading the problem into my brain, sleeping, more study, playing > the piano, > a glass of wine, stoking the cat usually solves the problem (or not). > > IDE's in all their glory help me type the solution to my problem into > the computer - but they do not help me formulate the correct problem > in the first place. > > > All true, Joe, we can't know what we don't know and no amount of the > technology we are talking about (i.e. not things that stimulate the > brain's creativity and whatnot) will help with that. > > However, helper tools can help us be faster and more flexible when > writing the micro-experiments, which means we can try more of them in > the same amount of time and if we have a slightly higher-level view > over them maybe we can easier see new ways to combine them, that would > otherwise get hidden in the boilerplate code. The tools can also help > write a correct final version, too (even if it is just by eliminating > the need to always have to stop typing and ask "was it keysearch(Key, > List) or keysearch(List, Key)?") > > Also, most of the developer time is spent not thinking about new > stuff, but reading/understanding/debugging/improving existing code, > and here tools can help a lot by filtering parts irrelevant for the > current task, or by providing a higher-level view of the code, or by > making it easy to follow the execution of the code. There was a time > when I could run code in my head and I got most often correct results, > but that's not working any more... > > Anyway, even if they are far from perfect, I think everybody can agree > that tools help. I consider pen and paper as tools too, by the way. I > think that the issue is that everybody has their own thought processes > and need tools that fit those, but we can't create custom tools for > every person. > > Going back to a more practical discussion and considering I am a tool > developer, I welcome all inputs regarding what kind of tools would > help you (as in, all of you) best. I have already started looking at a > prototype for an editor for an application's supervision structure, as > Vance talked about. I probably won't be able to do everything myself, > but it's possible that many people have tools that they wrote for > themselves and use with success - maybe those can be shared? > > best regards, > Vlad > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Mon May 2 16:24:04 2016 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Mon, 2 May 2016 16:24:04 +0200 Subject: [erlang-questions] erlang nif resource & thread safety In-Reply-To: References: Message-ID: <57276304.9070206@ericsson.com> A resource will be deallocated when * all terms created with enif_make_resource() have been garbage collected * AND the call to enif_alloc_resource() has been balanced by a call to enif_release_resource() * AND all calls to enif_keep_resource() have been balanced by an equal number of calls to enif_release_resource() On 05/02/2016 11:47 AM, Benoit Chesneau wrote: > I had a look in the doc but I can't figure exactly how much a nif > resource could live outside the process that created it, ie. how > others processes can write/read to it and how to make sure the > resource will be accessible to them. > > So I created a nif resource that have the following properties: > - it is ref counted, each time a call is made to it, the counter is > incremented and decremented when it return. A resource already has an internal ref counter, which is decremented by the GC and by enif_release_resource(). Adding your own ref counter is probably either redundant or just plain wrong. Every call to enif_release_resource() must be balanced by a prior call to either enif_alloc_resource() or enif_keep_resource(). You cannot release a resource that is still referred to by one of more terms. > - the process that create it will live until the vm is closed A resource it not tied to the process that created it, it is tied to the terms that refer it. > - write and reads to the resource are thread safe, at least the > functions that use resource > Maintaining the thread safety of the *content* of a resource is no different than the thread safety of any shared data in any multi threaded program. > For now I was thinking to share this resources between others > processes via an ETS table. But I am wondering if instead it shouldn't > be safer to makes all the call via a nif. With the result of creating > some kind of bottleneck. It's totally fine to share resources via local ETS tables. The resource does not care where the terms that are keeping it alive are residing. The only thing that does not work is sending a resource term to another node or do term_to_binary/binary_to_term. The bottom line is: If enif_get_resource() succeeds, then you will obtain a pointer to a resource that is guaranteed to be alive until the ErlNifEnv of the term is invalidated. /Sverker, Erlang/OTP -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Mon May 2 17:07:34 2016 From: comptekki@REDACTED (Wes James) Date: Mon, 2 May 2016 09:07:34 -0600 Subject: [erlang-questions] erlsrv on windows 10 problem In-Reply-To: References: Message-ID: I have it working on windows 10, as mentioned, I just need to change the start up option in the windows service from Automatic to Automatic (delayed start). What can I help with? -wes On Fri, Apr 29, 2016 at 11:59 PM, Dan Gudmundsson wrote: > Well you got it run at all, that is something, I have disabled our testing > of > services on Windows 10 for now, master branch, since I could not get it > run at all. > > From what I googled and understood you will need start the services from > an elevated shell, an administer user is not enough. > But you might already do that or I got it wrong. > > Anyway, our Windows knowledge, and priority, is not at the highest level. > So any help would be appreciated. > > /Dan > > On Fri, Apr 29, 2016 at 5:55 PM Wes James wrote: > >> I use erlsrv to install an erlang application on windows. Windows 7 >> works fine, but on windows 10, I have to go in and change the service from >> automatic to Automatic (delayed start), otherwise the app never runs on >> system start up. Any idea how to fix this? >> >> I just found this windows command: >> >> sc config SVCNAME start= delayed-auto >> >> I can use that in the install script. >> >> I'd still like it to start with just automatic. With the delayed option, >> it takes several extra minutes for the service to start running. >> >> Thanks, >> >> -wes >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jacob01@REDACTED Mon May 2 17:59:55 2016 From: jacob01@REDACTED (Jacob) Date: Mon, 2 May 2016 17:59:55 +0200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: Message-ID: <5727797B.5030208@gmx.net> Hi, I just came across the point where I wished there was a merge/3 function like in dict/ordict: merge(Fun, Map1, Map2) -> Map3 Cheers Jacob On 04/08/2016 08:31 PM, Bj?rn-Egil Dahlberg wrote: > Hi there! > > I would like to know if you have any desires that we extend the current > maps module with additional functionality? > > Have you repeated some code or built your own private lib to handle > certain maps specific tasks and thought "why isn't this in the maps module?" > > I would like to know! > > There is still time before 19.0 to extend the current API. > > We have for example discussed adding {get, put, update, > remove}_path type of functions where we handle nested maps. Would this > be useful? Would it be used? > > What have we missed? > > // Bj?rn-Egil > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From comptekki@REDACTED Mon May 2 19:27:44 2016 From: comptekki@REDACTED (Wes James) Date: Mon, 2 May 2016 11:27:44 -0600 Subject: [erlang-questions] erlsrv on windows 10 problem In-Reply-To: References: Message-ID: This is with the 18.3 windows binary from erlang.org. -wes On Fri, Apr 29, 2016 at 11:59 PM, Dan Gudmundsson wrote: > Well you got it run at all, that is something, I have disabled our testing > of > services on Windows 10 for now, master branch, since I could not get it > run at all. > > From what I googled and understood you will need start the services from > an elevated shell, an administer user is not enough. > But you might already do that or I got it wrong. > > Anyway, our Windows knowledge, and priority, is not at the highest level. > So any help would be appreciated. > > /Dan > > On Fri, Apr 29, 2016 at 5:55 PM Wes James wrote: > >> I use erlsrv to install an erlang application on windows. Windows 7 >> works fine, but on windows 10, I have to go in and change the service from >> automatic to Automatic (delayed start), otherwise the app never runs on >> system start up. Any idea how to fix this? >> >> I just found this windows command: >> >> sc config SVCNAME start= delayed-auto >> >> I can use that in the install script. >> >> I'd still like it to start with just automatic. With the delayed option, >> it takes several extra minutes for the service to start running. >> >> Thanks, >> >> -wes >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From josh.rubyist@REDACTED Mon May 2 21:31:03 2016 From: josh.rubyist@REDACTED (Josh Adams) Date: Mon, 2 May 2016 14:31:03 -0500 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <5ae800f8-53a7-92a2-47d1-d8a0e76c19e0@gjunka.com> Message-ID: I suppose I will contribute a tiny bit too this conversation. Have any of you looked at unison? http://unisonweb.org/2015-05-07/about.html#post-start I've built a visual programming language, and while it works and is in production use for some healthcare purposes, I found a million issues that made me hate it. Unison strikes me as unbelievably interesting and its editor looks like what I would want. It also covers distribution and has the concept of a dht for program functions which strikes me as awesome. Would love to hear what you good folks think of it. Josh On May 2, 2016 9:20 AM, "Grzegorz Junka" wrote: I don't agree that tools help in general, but only in specific tasks. Dialyzer, Xref, common test, all help in their specific tasks. In particular, the tools mentioned in this thread would help other developers to understand some existing code written by someone else. They probably wouldn't help me to understand my own code better, although they could probably help me to find more bugs or to help me restructure the code if I ever needed to. Before I can start writing a new program I always start with a pen and paper, away from the computer. Only once I have everything "sorted out" in my mind, I start coding using Emacs. It's the closest you can get to pen and paper. I am comfortable using IDEs but I don't tend to use them unless required by the task at hand (e.g. when doing GUI's or when they are better supported than command line, e.g. XCode, or Visual Studio). So, I wouldn't exaggerate the tools, although different people approach this problem differently. Some people study better reading a book and some watching video. Some people are better programmers with an IDE and for some (like me) it's an unnecessary burden (and overload of my brain). Having said that, the one area when tools are indispensable and should always be worked on is the automation of daunting and obvious tasks that I, as a developer, shouldn't have to do. Running tests is one example. Installing software, keeping backups, renaming function names in multiple files, intelligently searching for specific code fragments, etc. The more of them and the more things they can do automatically for me the more time I have to write code in Emacs. However, many of those tools seem to be intelligent because they have too many options. A tool works great when it has few options working 99% of the time with default values, but can be extended to a great deal when the extra mile is needed. Many tools, often IDEs, burden users with these unnecessary options. When I need to rename a function name in all my files and a tool that could do it for me shows me 100 options, I would rather prefer to do it manually. Greg On 02/05/2016 12:50, Vlad Dumitrescu wrote: On Mon, May 2, 2016 at 1:56 PM, Joe Armstrong wrote: > Virtually all the code I write is in the form of "micro experiments" - > since I'm not sure what will happen if I do something. The "big" or > complete program > can be written once all the micro-experiments have been done. > > When I can't write the code - it is always because I haven't > understood the problem - this is very frustrating I *think* I've > understood the problem - yet the fact I can't write the code is a > clear sign that I have not understood the problem - but I do not yet > know why. > > Loading the problem into my brain, sleeping, more study, playing the piano, > a glass of wine, stoking the cat usually solves the problem (or not). > > IDE's in all their glory help me type the solution to my problem into > the computer - but they do not help me formulate the correct problem > in the first place. > All true, Joe, we can't know what we don't know and no amount of the technology we are talking about (i.e. not things that stimulate the brain's creativity and whatnot) will help with that. However, helper tools can help us be faster and more flexible when writing the micro-experiments, which means we can try more of them in the same amount of time and if we have a slightly higher-level view over them maybe we can easier see new ways to combine them, that would otherwise get hidden in the boilerplate code. The tools can also help write a correct final version, too (even if it is just by eliminating the need to always have to stop typing and ask "was it keysearch(Key, List) or keysearch(List, Key)?") Also, most of the developer time is spent not thinking about new stuff, but reading/understanding/debugging/improving existing code, and here tools can help a lot by filtering parts irrelevant for the current task, or by providing a higher-level view of the code, or by making it easy to follow the execution of the code. There was a time when I could run code in my head and I got most often correct results, but that's not working any more... Anyway, even if they are far from perfect, I think everybody can agree that tools help. I consider pen and paper as tools too, by the way. I think that the issue is that everybody has their own thought processes and need tools that fit those, but we can't create custom tools for every person. Going back to a more practical discussion and considering I am a tool developer, I welcome all inputs regarding what kind of tools would help you (as in, all of you) best. I have already started looking at a prototype for an editor for an application's supervision structure, as Vance talked about. I probably won't be able to do everything myself, but it's possible that many people have tools that they wrote for themselves and use with success - maybe those can be shared? best regards, Vlad _______________________________________________ erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Mon May 2 21:51:04 2016 From: comptekki@REDACTED (Wes James) Date: Mon, 2 May 2016 13:51:04 -0600 Subject: [erlang-questions] erlsrv on windows 10 problem In-Reply-To: References: Message-ID: I have a script that I right-click on and select run as administrator, so yes, it has to be done as admin. This is what I use for win 10 (setup-erlsrv.cmd): @echo off @rem @rem sample erlang install service script @rem - add then start ecom erlang service @rem - if it already exists, stop then remove first before adding and starting @rem @set erl_service=0ecom @erlsrv list %erl_service% | @findstr /i "%erl_service%" @if errorlevel 1 goto next @echo. @rem echo Removing erl service... @erlsrv remove "%erl_service%" rem @goto:EOF :next @SETLOCAL ENABLEDELAYEDEXPANSION @SET comp=computer-name @set module=-s ecom @set onfail=-onfail restart @set node=-name ecom@%comp%.nr.usu.edu @rem echo %node% @set root=-w "C:/windows/erl" @set srvc_name=-i 0ecom @set boot=-boot c:/windows/erl/start_ssl_ecom -proto_dist inet_tls -ssl_dist_opt server_certfile c:/windows/erl/ssl/cert.pem -ssl_dist_opt server_keyfile c:/windows/erl/ssl/key.pem -ssl_dist_opt server_secure_renegotiate true client_secure_renegotiate true @rem set args=-args "%boot% %module%" @set args=-args "%boot% %module%" @echo. @rem echo Adding erl service... @erlsrv add "%erl_service%" %node% %root% %onfail% %args% @echo. @rem echo Starting erl service... @erlsrv start %erl_service% @echo. :end @pause @goto:EOF ------------- Then I paste this in to an elevated cmd shell (for some reason it won't run from a file) to set the service to automatic delayed: @FOR /F "delims=" %i IN ('sc getkeyname 0ecom') DO set z=%i @echo %z% @for /f "tokens=3" %i IN ("%z%") DO set erl_servicedispname=%i @echo %erl_servicedispname% @sc config %erl_servicedispname% start= delayed-auto ------- I use it to install ecom.beam as a service from this project: https://github.com/comptekki/esysman -wes On Mon, May 2, 2016 at 1:18 PM, Dan Gudmundsson wrote: > I do not remember the exact error message now, but what I gathered you had > to start erlsrv (and erlang) from an "elevated" shell, otherwise you don't > have the permissions to install the services. And we run our automtic tests > from a "admin" user but not from an "elevated" shell. > > But I don't really know what I'm talking about here, as I said, our > windows competence is really low. > > So how you install the services, do you run start erlang from inside > and admin cmd prompt. > > Regards > /Dan > > On Mon, May 2, 2016 at 7:27 PM Wes James wrote: > >> This is with the 18.3 windows binary from erlang.org. >> >> -wes >> On Fri, Apr 29, 2016 at 11:59 PM, Dan Gudmundsson >> wrote: >> >>> Well you got it run at all, that is something, I have disabled our >>> testing of >>> services on Windows 10 for now, master branch, since I could not get it >>> run at all. >>> >>> From what I googled and understood you will need start the services from >>> an elevated shell, an administer user is not enough. >>> But you might already do that or I got it wrong. >>> >>> Anyway, our Windows knowledge, and priority, is not at the highest level. >>> So any help would be appreciated. >>> >>> /Dan >>> >>> On Fri, Apr 29, 2016 at 5:55 PM Wes James wrote: >>> >>>> I use erlsrv to install an erlang application on windows. Windows 7 >>>> works fine, but on windows 10, I have to go in and change the service from >>>> automatic to Automatic (delayed start), otherwise the app never runs on >>>> system start up. Any idea how to fix this? >>>> >>>> I just found this windows command: >>>> >>>> sc config SVCNAME start= delayed-auto >>>> >>>> I can use that in the install script. >>>> >>>> I'd still like it to start with just automatic. With the delayed >>>> option, it takes several extra minutes for the service to start running. >>>> >>>> Thanks, >>>> >>>> -wes >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Mon May 2 22:15:32 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Mon, 02 May 2016 20:15:32 +0000 Subject: [erlang-questions] erlang nif resource & thread safety In-Reply-To: <57276304.9070206@ericsson.com> References: <57276304.9070206@ericsson.com> Message-ID: On Mon, May 2, 2016 at 4:24 PM Sverker Eriksson < sverker.eriksson@REDACTED> wrote: > A resource will be deallocated when > > * all terms created with enif_make_resource() have been garbage collected > > * AND the call to enif_alloc_resource() has been balanced by a call to > enif_release_resource() > > * AND all calls to enif_keep_resource() have been balanced by an equal > number of calls to enif_release_resource() > > > > On 05/02/2016 11:47 AM, Benoit Chesneau wrote: > > I had a look in the doc but I can't figure exactly how much a nif resource > could live outside the process that created it, ie. how others processes > can write/read to it and how to make sure the resource will be accessible > to them. > > So I created a nif resource that have the following properties: > - it is ref counted, each time a call is made to it, the counter is > incremented and decremented when it return. > > A resource already has an internal ref counter, which is decremented by > the GC and by enif_release_resource(). > > Adding your own ref counter is probably either redundant or just plain > wrong. > Every call to enif_release_resource() must be balanced by a prior call > to either enif_alloc_resource() or enif_keep_resource(). You cannot > release a resource that is still referred to by one of more terms. > > The ref counter is here mainly to makes sure that parent resources knows everything has been cleaned since it's it's needed in the other API. > > - the process that create it will live until the vm is closed > > A resource it not tied to the process that created it, > it is tied to the terms that refer it. > Oh perfect then. > > - write and reads to the resource are thread safe, at least the functions > that use resource > > Maintaining the thread safety of the *content* of a resource > is no different than the thread safety of any shared data in any multi > threaded program. > Well of course. But Isn't there some cases where the VM fork a process instead of using threads on some systems? I thought it did that on openbsd for example. > > > For now I was thinking to share this resources between others processes > via an ETS table. But I am wondering if instead it shouldn't be safer to > makes all the call via a nif. With the result of creating some kind of > bottleneck. > > It's totally fine to share resources via local ETS tables. > The resource does not care where the terms that are keeping it alive are > residing. > > The only thing that does not work is sending a resource term > to another node or do term_to_binary/binary_to_term. > > > The bottom line is: > If enif_get_resource() succeeds, then you will obtain > a pointer to a resource that is guaranteed to be alive > until the ErlNifEnv of the term is invalidated. > > > > /Sverker, Erlang/OTP > > > Thanks for the detailed answer. It's useful :) - beno?t -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue May 3 02:21:42 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 3 May 2016 12:21:42 +1200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> Message-ID: On 2/05/16 7:51 PM, Vlad Dumitrescu wrote: >> Let's face it, an IDE cannot display information it has not been told about. > Yes, it can (depending on how one understands "told about"). Cross reference is to me a prime example of information that the tool *has* been told about. I agree with everything you said about tools and what they can do, although I note that the call graph tools I've used that rely on *static* information have consistently been extremely bad at dealing with higher-order functions. Tracking (statically determinable) 'spawn' calls is a wonderful thing, but it too is an example of a tool being able to use information it has been explicitly given. These are all important and useful things. But there is other information that tools do *not* display because it *isn't* derivable from who-calls-whom and who-uses-what. From ok@REDACTED Tue May 3 02:36:52 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 3 May 2016 12:36:52 +1200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> Message-ID: On 2/05/16 11:56 PM, Joe Armstrong wrote: > I was thinking. There is another problem, which is not addressed. > > We just publish final code - but not the process involved to get to the code. How that takes me back. There was a PhD student at Edinburgh when I arrived there working on transformation-based programming. I've always remembered his name as Martin Feather, but that may be wrong. The idea was that you started with a high level executable specification and then provided a script to transform it to efficient low-level code. The idea of working via transformations from short clear code to longer code was not new even in 1979, and people have continued to work on it. What Martin Feather (if that was his name) was one of the first to do was to make the transformation process semi-automatic, with a script saying things like "unfold that function" or "use this law". (It tied in with the Burstall/Darlington work on automatic transformation.) The idea was that if you changed the high level specification a little, you might be able to reuse most of your transformation script In principle, a script could even have alternatives, and I *think* his system did this. (Again, this ties in with the work on Edinburgh LCF and the idea of controlling a theorem prover by writing tactics (do a specific thing) and tacticals (combinators for tactics.) I am going to forward your message to a class because I think you're completely right. > Unfortunately I only ever have a clear description of my problem at the > very end when the problem is solved. > > Chicken or egg first? The amniote egg is many many millions of years older than birds. From raould@REDACTED Tue May 3 02:40:17 2016 From: raould@REDACTED (Raoul Duke) Date: Mon, 2 May 2016 17:40:17 -0700 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> Message-ID: >> We just publish final code - but not the process involved to get to the >> code. > I am going to forward your message to a class because I think you're > completely right. The way I read the original note about recording the process, I took it to mean all the experiments, all the wrong turns, as well as the final "right" turn that resulted in the code as it is so far. While I can think of uses for such information, on the whole I think it is something mostly to be avoided. Mostly. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue May 3 02:42:03 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 3 May 2016 12:42:03 +1200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <5727797B.5030208@gmx.net> References: <5727797B.5030208@gmx.net> Message-ID: <110e2526-e40c-0ced-fae8-e56d2a3f6fde@cs.otago.ac.nz> On 3/05/16 3:59 AM, Jacob wrote: > Hi, > > I just came across the point where I wished there was a merge/3 function > like in dict/ordict: > > merge(Fun, Map1, Map2) -> Map3 +1 . From mahesh@REDACTED Tue May 3 02:58:27 2016 From: mahesh@REDACTED (Mahesh Paolini-Subramanya) Date: Mon, 2 May 2016 20:58:27 -0400 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> Message-ID: > > The way I read the original note about recording the process, I took it to > mean all the experiments, all the wrong turns, as well as the final "right" > turn that resulted in the code as it is so far There is *so* much of note to unpack here! It is unfortunate that most of the world tends to think of code as something that is _Correct_, with any bugs being, oh, flaws in this perfection. Sadly, with the exception of the - very few - provably correct and optimal things out there, most of everything that we do consists of an exercise in multi-dimensional optimization across Requirements (conflicting, and differing, based on source), Deadlines, People (that you work with, that are involved, etc.), Access to resources, and so on and so forth. So yes, for the *serious* practitioner, there are many many lessons to be learned in recording the process. That said, I strongly suspect that most of these lessons are of the kind that are best imparted over copious quantities of wine... Cheers p.s. Or beer. Or Pepsi. Whatever... On Mon, May 2, 2016 at 8:40 PM, Raoul Duke wrote: > >> We just publish final code - but not the process involved to get to the > >> code. > > I am going to forward your message to a class because I think you're > > completely right. > > The way I read the original note about recording the process, I took it to > mean all the experiments, all the wrong turns, as well as the final "right" > turn that resulted in the code as it is so far. While I can think of uses > for such information, on the whole I think it is something mostly to be > avoided. Mostly. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- *Mahesh Paolini-Subramanya That tall bald Indian guy..* *Twitter | Blog | G+ | LinkedIn * -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Tue May 3 03:02:01 2016 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 02 May 2016 18:02:01 -0700 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <5727797B.5030208@gmx.net> References: <5727797B.5030208@gmx.net> Message-ID: <5727F889.2010602@gmail.com> On 05/02/2016 08:59 AM, Jacob wrote: > Hi, > > I just came across the point where I wished there was a merge/3 function > like in dict/ordict: > > merge(Fun, Map1, Map2) -> Map3 I agree that it is very helpful to have the same interface as the dict module (it is unfortunate that the maps module is unable to provide the same dict interface, since that makes Erlang data structure usage more complex than it needs to be, with no justification (I understand the intent was likely to discourage doing things that are inefficient)). The maps merge function was discussed in the past at http://erlang.2086793.n4.nabble.com/Merging-maps-with-a-fun-like-dict-merge-3-td4713590.html with the thread preferring: merge1(Fun, Map1, Map2) -> maps:merge(Map1, maps:map(fun(K, V2) -> case maps:find(K, Map1) of {ok, V1} -> Fun(K, V1, V2); error -> V2 end end, Map2)). The maps:merge/3 function can also be implemented using the new maps:update_with/4 function (in the git repo, from https://github.com/erlang/otp/pull/1025 ) which is based on the dict:merge/3 documentation (http://erlang.org/doc/man/dict.html#merge-3): merge2(Fun, Map1, Map2) -> maps:fold(fun (K, V1, Map) -> maps:update_with(K, fun (V2) -> Fun(K, V1, V2) end, V1, Map) end, Map2, Map1). > > Cheers > > Jacob > > On 04/08/2016 08:31 PM, Bj?rn-Egil Dahlberg wrote: >> Hi there! >> >> I would like to know if you have any desires that we extend the current >> maps module with additional functionality? >> >> Have you repeated some code or built your own private lib to handle >> certain maps specific tasks and thought "why isn't this in the maps module?" >> >> I would like to know! >> >> There is still time before 19.0 to extend the current API. >> >> We have for example discussed adding {get, put, update, >> remove}_path type of functions where we handle nested maps. Would this >> be useful? Would it be used? >> >> What have we missed? >> >> // Bj?rn-Egil >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From raould@REDACTED Tue May 3 03:09:47 2016 From: raould@REDACTED (Raoul Duke) Date: Mon, 2 May 2016 18:09:47 -0700 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> Message-ID: Personally I think this particular hypothetical "possibly every button anybody has ever pressed in the history of this project, ever" tooling would not be a priority in terms of ROI. I would rather be talking about tools that help us study and grok the code as it stands at 'head' of the tree. But if we're going to get narrowed down on this "history" tooling: I would not want the tooling to save everything unless it were able to clearly show the path finally chosen & hide all the others, and did that by default. If I want to then add in some stuff about "you might think doing X would be 'better' but I tried that and it failed, see " based on the tooling, sure, but even then I would want that to perhaps be a compressed and cleaned up story, not the full log of every little thing I did that was down the wrong path. Since there is so much that goes into making code, I think we should strive to make a clean and coherent story as much as possible as the first way things are "rendered" to help future people just get on with life. :-) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue May 3 03:29:40 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 3 May 2016 13:29:40 +1200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> Message-ID: There's a grey area. Today I have to explain documentation to a class, and one thing I'm going to say is "you need to explain the code that isn't there." Example: a function doesn't check an argument. Why is that code missing? Because it's checked somewhere else. If the function is called from a different place, or if the code that currently calls it is altered, we may need that code after all. Example: a function which seems like it should have an inverse doesn't have one in the code. Why not? There's a difference between "we haven't needed it yet" and "it's not doable". Example: a function for merging two sets doesn't include any code to sort them. Why? Because it's the caller's job to do the sorting. But there's a reason why twists and turns might need recording. I don't know about you, but more than once I've had the experience that the 'final "right" turn' was actually wrong, and to get working code I've had to go back to something I'd rejected on (what I know see to be) mistaken grounds. Now that doesn't mean everything has to be in the version we give to the compiler. Keeping (possibly wrongly) discarded stuff is the job of the version control system. And of course an *idea* that is tried in one derivation line and then abandoned may turn out to be very useful for solving a different problem. For what it's worth, just yesterday I refactored some code. At the price of writing one new method I was able to eliminate four others, *and* halve the amount of storage allocated (and reduce the time). The old code wasn't *wrong* exactly. And the new code is definitely less obvious. The new method has a comment An earlier implementation of these methods allocated a string that was immediately discarded; this creates no garbage. which explains why the obvious code isn't there. That's not exactly keeping an abandoned version, but it does mention that there was one, and why. From erlang@REDACTED Tue May 3 08:54:27 2016 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 3 May 2016 08:54:27 +0200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> Message-ID: On Tue, May 3, 2016 at 3:29 AM, Richard A. O'Keefe wrote: > There's a grey area. > Today I have to explain documentation to a class, > and one thing I'm going to say is "you need to explain > the code that isn't there." YES ^ 100 A slight modification: You need to explain the code that isn't there that somebody "versed in the art might reasonably expect to be there but was not" > > Example: a function doesn't check an argument. > Why is that code missing? Because it's checked somewhere else. > If the function is called from a different place, or if the > code that currently calls it is altered, we may need that code > after all. > > Example: a function which seems like it should have an inverse > doesn't have one in the code. Why not? There's a difference > between "we haven't needed it yet" and "it's not doable". > > Example: a function for merging two sets doesn't include any > code to sort them. Why? Because it's the caller's job to do > the sorting. > > But there's a reason why twists and turns might need recording. > I don't know about you, but more than once I've had the experience that the > 'final "right" turn' was actually wrong, and to get working code I've had to > go > back to something I'd rejected on (what I know see to be) mistaken grounds. I think the blind alleys need documenting. I add comments like: " The obvious approach of doing XXX fails because YYY so don't go there" Most of my projects have a directories called 'research' and 'dustbin' they are huge compared to the final program - I don't commit or publish them. In particular research can be huge with PDF specs, etc one line of which has ended up in the code. The BIG question the documentation should answer is not what does this code do, or how does it work but "how did the person write the code" ie what were the thought processes necessary to write the code. Cheers > > Now that doesn't mean everything has to be in the version we give to the > compiler. Keeping (possibly wrongly) discarded stuff is the job of the > version > control system. > > And of course an *idea* that is tried in one derivation line and then > abandoned > may turn out to be very useful for solving a different problem. > > For what it's worth, just yesterday I refactored some code. At the price of > writing one new method I was able to eliminate four others, *and* halve the > amount of storage allocated (and reduce the time). The old code wasn't > *wrong* exactly. And the new code is definitely less obvious. The new > method has a comment > > An earlier implementation of these methods allocated a string > that was immediately discarded; this creates no garbage. > > which explains why the obvious code isn't there. That's not exactly keeping > an abandoned version, but it does mention that there was one, and why. > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From dangud@REDACTED Tue May 3 09:17:02 2016 From: dangud@REDACTED (Dan Gudmundsson) Date: Tue, 03 May 2016 07:17:02 +0000 Subject: [erlang-questions] erlsrv on windows 10 problem In-Reply-To: References: Message-ID: Thanks, I figured it should work, but I have disabled the testing since it requires manual starting of elevated cmd shell. On Mon, May 2, 2016 at 9:51 PM Wes James wrote: > I have a script that I right-click on and select run as administrator, so > yes, it has to be done as admin. > > This is what I use for win 10 (setup-erlsrv.cmd): > > @echo off > > @rem > @rem sample erlang install service script > @rem - add then start ecom erlang service > @rem - if it already exists, stop then remove first before adding and > starting > @rem > > @set erl_service=0ecom > @erlsrv list %erl_service% | @findstr /i "%erl_service%" > @if errorlevel 1 goto next > @echo. > @rem echo Removing erl service... > @erlsrv remove "%erl_service%" > > rem @goto:EOF > > :next > > @SETLOCAL ENABLEDELAYEDEXPANSION > @SET comp=computer-name > > @set module=-s ecom > @set onfail=-onfail restart > @set node=-name ecom@%comp%.nr.usu.edu > @rem echo %node% > @set root=-w "C:/windows/erl" > @set srvc_name=-i 0ecom > @set boot=-boot c:/windows/erl/start_ssl_ecom -proto_dist inet_tls > -ssl_dist_opt server_certfile c:/windows/erl/ssl/cert.pem -ssl_dist_opt > server_keyfile c:/windows/erl/ssl/key.pem -ssl_dist_opt > server_secure_renegotiate true client_secure_renegotiate true > @rem set args=-args "%boot% %module%" > @set args=-args "%boot% %module%" > > @echo. > @rem echo Adding erl service... > @erlsrv add "%erl_service%" %node% %root% %onfail% %args% > @echo. > @rem echo Starting erl service... > @erlsrv start %erl_service% > @echo. > > :end > @pause > @goto:EOF > > ------------- > > Then I paste this in to an elevated cmd shell (for some reason it won't > run from a file) to set the service to automatic delayed: > > @FOR /F "delims=" %i IN ('sc getkeyname 0ecom') DO set z=%i > @echo %z% > @for /f "tokens=3" %i IN ("%z%") DO set erl_servicedispname=%i > @echo %erl_servicedispname% > > @sc config %erl_servicedispname% start= delayed-auto > > ------- > > I use it to install ecom.beam as a service from this project: > > https://github.com/comptekki/esysman > > -wes > > On Mon, May 2, 2016 at 1:18 PM, Dan Gudmundsson wrote: > >> I do not remember the exact error message now, but what I gathered you >> had to start erlsrv (and erlang) from an "elevated" shell, otherwise you >> don't have the permissions to install the services. And we run our automtic >> tests from a "admin" user but not from an "elevated" shell. >> >> But I don't really know what I'm talking about here, as I said, our >> windows competence is really low. >> >> So how you install the services, do you run start erlang from inside >> and admin cmd prompt. >> >> Regards >> /Dan >> >> On Mon, May 2, 2016 at 7:27 PM Wes James wrote: >> >>> This is with the 18.3 windows binary from erlang.org. >>> >>> -wes >>> On Fri, Apr 29, 2016 at 11:59 PM, Dan Gudmundsson >>> wrote: >>> >>>> Well you got it run at all, that is something, I have disabled our >>>> testing of >>>> services on Windows 10 for now, master branch, since I could not get it >>>> run at all. >>>> >>>> From what I googled and understood you will need start the services >>>> from an elevated shell, an administer user is not enough. >>>> But you might already do that or I got it wrong. >>>> >>>> Anyway, our Windows knowledge, and priority, is not at the highest >>>> level. >>>> So any help would be appreciated. >>>> >>>> /Dan >>>> >>>> On Fri, Apr 29, 2016 at 5:55 PM Wes James wrote: >>>> >>>>> I use erlsrv to install an erlang application on windows. Windows 7 >>>>> works fine, but on windows 10, I have to go in and change the service from >>>>> automatic to Automatic (delayed start), otherwise the app never runs on >>>>> system start up. Any idea how to fix this? >>>>> >>>>> I just found this windows command: >>>>> >>>>> sc config SVCNAME start= delayed-auto >>>>> >>>>> I can use that in the install script. >>>>> >>>>> I'd still like it to start with just automatic. With the delayed >>>>> option, it takes several extra minutes for the service to start running. >>>>> >>>>> Thanks, >>>>> >>>>> -wes >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Tue May 3 09:19:36 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 3 May 2016 09:19:36 +0200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> Message-ID: On Tue, May 3, 2016 at 3:29 AM, Richard A. O'Keefe wrote: > But there's a reason why twists and turns might need recording. > I don't know about you, but more than once I've had the experience that the > 'final "right" turn' was actually wrong, and to get working code I've had > to go > back to something I'd rejected on (what I know see to be) mistaken grounds. > > Now that doesn't mean everything has to be in the version we give to the > compiler. Keeping (possibly wrongly) discarded stuff is the job of the > version > control system. > > And of course an *idea* that is tried in one derivation line and then > abandoned > may turn out to be very useful for solving a different problem. > This is true and would be useful, but I think I can see a problem (which might very well be the reason why it isn't something people already do): after a while, there will be literally thousands of discarded stuff. How am I supposed to search and find the alternative that works, especially if I am now working on a different problem than 6 months ago when the stuff was originally written? Is this search faster than starting from scratch? IMHO the answer is that what needs to be saved is a higher-level description, not a complete brain dump of the process. There is also a difference between what is interesting for teaching people (beginners) and what is interesting for future reference for the team that develops the code. These categories need different levels of detail, different kind of information. best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From list1@REDACTED Tue May 3 09:41:16 2016 From: list1@REDACTED (Grzegorz Junka) Date: Tue, 3 May 2016 07:41:16 +0000 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <5ae800f8-53a7-92a2-47d1-d8a0e76c19e0@gjunka.com> Message-ID: The idea is very good, I was often imagining a visual editor, something between this Unison and https://scratch.mit.edu/ to write Erlang code. But I must say I dislike tools that show popups obstructing the view of the code that I am editing :) This is similar to the other idea mentioned in this conversation - recording all bad and wrong turns when teaching others how to resolve a problem. Both impose or assume a specific way of thinking when working with or writing the code. It's not right or wrong in its merit, but it's very personal. No two people think the same way because of different experiences in their life. So, whilst it's often constructive to learn how others reached a specific conclusion, or helpful to use semi-automatic code completion and validation, a true virtue comes only through practice and experience build on top of good understanding of computer algorithms and programming paradigms. Greg On 02/05/2016 19:31, Josh Adams wrote: > > I suppose I will contribute a tiny bit too this conversation. Have any > of you looked at unison? > http://unisonweb.org/2015-05-07/about.html#post-start > > I've built a visual programming language, and while it works and is in > production use for some healthcare purposes, I found a million issues > that made me hate it. Unison strikes me as unbelievably interesting > and its editor looks like what I would want. > > It also covers distribution and has the concept of a dht for program > functions which strikes me as awesome. Would love to hear what you > good folks think of it. > > Josh > > From Oliver.Korpilla@REDACTED Tue May 3 09:54:12 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Tue, 3 May 2016 09:54:12 +0200 Subject: [erlang-questions] Questions about supervision Message-ID: Hello. I ran into some unexpected results with supervision and transient workers and this gives me a few questions about process shutdown and resource cleanup. I have a structure like this: * main supervisor with several permanent children, one of them the (permanent) supervisor for transient children * the supervisor for transient children creates processes grouped as follows: - a "group" supervisor - a worker maintaining a TCP connection (transient) linked with the group supervisor - a worker running all the required procedures that determine one client linked with the group supervisor - all these processes are registered under global under the same ID like this: {sup, }, {conn, }, {proc, } I read that transient children are only restarted on abnormal exit which is exactly what I want. When trying to figure out how to best clean up the group of listed last, I ran into some snags, though... I sometimes rely on the supervisor:which_children/1 call to find the IDs of all active groups. When inspecting all three layers of supervision I realized that processes are not cleaned up from the list of children like I expected. Q1) So - lets say I have a transient child that exits with :normal, will its supervisor clean it out of its child lists eventually? Furthermore, when reading through the documentation of gen_server and supervisor I came to this understanding: - If shutdown is brutal_kill, no cleanup in the children can take place. - If shutdown is infinity or , terminate in OTP worker children is called only if the child is trapping exits and the exit is not normal (like shutdown). Q2) Is this understanding correct? Q3) What is the pattern for properly and safely shutting down transient children? supervisor:terminate_child followed by supervisor:delete_child? Q4) What is the pattern for keeping supervisors inner state clean so that they don't just accumulate transient children forever? Thanks and best regards, Oliver From list1@REDACTED Tue May 3 10:13:19 2016 From: list1@REDACTED (Grzegorz Junka) Date: Tue, 3 May 2016 08:13:19 +0000 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <5727797B.5030208@gmx.net> References: <5727797B.5030208@gmx.net> Message-ID: <55443f17-c01f-3014-1fe9-9ded027a2a2c@gjunka.com> +1 for that too Bj?rn-Egil, what about matching a Key with another Variable in function head? e.g. some_fun(#{Key}, Key) -> exists. Cheers Greg On 02/05/2016 15:59, Jacob wrote: > Hi, > > I just came across the point where I wished there was a merge/3 function > like in dict/ordict: > > merge(Fun, Map1, Map2) -> Map3 > > Cheers > > Jacob > > On 04/08/2016 08:31 PM, Bj?rn-Egil Dahlberg wrote: >> Hi there! >> >> I would like to know if you have any desires that we extend the current >> maps module with additional functionality? >> >> Have you repeated some code or built your own private lib to handle >> certain maps specific tasks and thought "why isn't this in the maps module?" >> >> I would like to know! >> >> There is still time before 19.0 to extend the current API. >> >> We have for example discussed adding {get, put, update, >> remove}_path type of functions where we handle nested maps. Would this >> be useful? Would it be used? >> >> What have we missed? >> >> // Bj?rn-Egil >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From vladdu55@REDACTED Tue May 3 10:13:22 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 3 May 2016 10:13:22 +0200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> Message-ID: I remembered an interesting method that might be relevant in this discussion too. It is called the Mikado method and earlier was available as a free download, but now I see that the autors have published a book https://mikadomethod.wordpress.com/. The first chapter is available for reading at https://manning-content.s3.amazonaws.com/download/3/558b9be-92a7-4ebf-90ba-c7fdd830aea7/MikadoMethod_CH01.pdf The method is meant mainly for refactoring existing code, but I think it could work for new code too. The idea is to set a goal and try a naive implementation. See what errors there are and set fixing them as prerequisites to the goal. Draw a tree of these prerequisites and how they depend on each other. The important and unintuitive next step is to revert all changes to a working state. Only then, choose an unsolved prerequisite and iterate the process until the initial goal is solved. There are no discarded branches, but the graph of the things that were tried and their dependencies is illustrative of the development process and can be annotated with information about "why"s. Maybe this is a pragmatic middle ground between keeping everything and keeping only the final branch? best regards, Vlad On Tue, May 3, 2016 at 9:19 AM, Vlad Dumitrescu wrote: > > On Tue, May 3, 2016 at 3:29 AM, Richard A. O'Keefe > wrote: > >> But there's a reason why twists and turns might need recording. >> I don't know about you, but more than once I've had the experience that >> the >> 'final "right" turn' was actually wrong, and to get working code I've had >> to go >> back to something I'd rejected on (what I know see to be) mistaken >> grounds. >> >> Now that doesn't mean everything has to be in the version we give to the >> compiler. Keeping (possibly wrongly) discarded stuff is the job of the >> version >> control system. >> >> And of course an *idea* that is tried in one derivation line and then >> abandoned >> may turn out to be very useful for solving a different problem. >> > > This is true and would be useful, but I think I can see a problem (which > might very well be the reason why it isn't something people already do): > after a while, there will be literally thousands of discarded stuff. How am > I supposed to search and find the alternative that works, especially if I > am now working on a different problem than 6 months ago when the stuff was > originally written? Is this search faster than starting from scratch? IMHO > the answer is that what needs to be saved is a higher-level description, > not a complete brain dump of the process. > > There is also a difference between what is interesting for teaching people > (beginners) and what is interesting for future reference for the team that > develops the code. These categories need different levels of detail, > different kind of information. > > best regards, > Vlad > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Tue May 3 10:59:12 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 3 May 2016 10:59:12 +0200 Subject: [erlang-questions] [ANN] Syn 1.3.1 - Now with multi_call Message-ID: Dear All, Syn 1.3.1 has been released. For those of you who don't know it, Syn is a global Process Registry and Process Group manager for Erlang, which supports PubSub. The main additions are: - A multi_call method, which allows to call all members of a Group and get their replies. - Improve net split resolution by suspending processes while a resolution is happening. - The order of Pids in a Group is now guaranteed across nodes. Thank you to all the developers that have contributed with ideas and support. Best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Tue May 3 10:59:57 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 3 May 2016 10:59:57 +0200 Subject: [erlang-questions] [ANN] Syn 1.3.1 - Now with multi_call In-Reply-To: References: Message-ID: ...and the link would be: https://github.com/ostinelli/syn Best, r. On Tue, May 3, 2016 at 10:59 AM, Roberto Ostinelli wrote: > Dear All, > Syn 1.3.1 has been released. > > For those of you who don't know it, Syn is a global Process Registry and > Process Group manager for Erlang, which supports PubSub. > > The main additions are: > > - A multi_call method, which allows to call all members of a Group and > get their replies. > - Improve net split resolution by suspending processes while a > resolution is happening. > - The order of Pids in a Group is now guaranteed across nodes. > > Thank you to all the developers that have contributed with ideas and > support. > > Best, > r. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Tue May 3 12:51:47 2016 From: g@REDACTED (Garrett Smith) Date: Tue, 3 May 2016 03:51:47 -0700 Subject: [erlang-questions] Questions about supervision In-Reply-To: References: Message-ID: Hi Oliver, On Tue, May 3, 2016 at 12:54 AM, Oliver Korpilla wrote: > Hello. > > I ran into some unexpected results with supervision and transient workers and this gives me a few questions about process shutdown and resource cleanup. > > I have a structure like this: > > * main supervisor with several permanent children, one of them the (permanent) supervisor for transient children > * the supervisor for transient children creates processes grouped as follows: > - a "group" supervisor > - a worker maintaining a TCP connection (transient) linked with the group supervisor > - a worker running all the required procedures that determine one client linked with the group supervisor > - all these processes are registered under global under the same ID like this: {sup, }, {conn, }, {proc, } > > I read that transient children are only restarted on abnormal exit which is exactly what I want. > > When trying to figure out how to best clean up the group of listed last, I ran into some snags, though... > > I sometimes rely on the supervisor:which_children/1 call to find the IDs of all active groups. When inspecting all three layers of supervision I realized that processes are not cleaned up from the list of children like I expected. Inspecting supervisor state is not well supported - and it's not a great idea. You're better off using something like gproc [1] to register processes by types-of-interest and use it to enumerate them. > Q1) So - lets say I have a transient child that exits with :normal, will its supervisor clean it out of its child lists eventually? You're looking for the behavior of the 'simple_one_for_one' supervisor, which removes children. > Furthermore, when reading through the documentation of gen_server and supervisor I came to this understanding: > > - If shutdown is brutal_kill, no cleanup in the children can take place. > - If shutdown is infinity or , terminate in OTP worker children is called only if the child is trapping exits and the exit is not normal (like shutdown). > > Q2) Is this understanding correct? Yes > Q3) What is the pattern for properly and safely shutting down transient children? supervisor:terminate_child followed by supervisor:delete_child? You can do this, but it's not a general practice. Ideally you start your processes (either indirectly at init in your supervisor tree, or by explicitly starting via a supervisor at various points at runtime) and then completely forget about them. They terminate in response to some message or by crashing. If you want to explicitly stop a process, tell it to stop (i.e. implement the stop behavior in the process module). Don't try to control it via the supervisor. > Q4) What is the pattern for keeping supervisors inner state clean so that they don't just accumulate transient children forever? simple_one_for_one_ftw! [1] https://github.com/uwiger/gproc From g@REDACTED Tue May 3 12:57:13 2016 From: g@REDACTED (Garrett Smith) Date: Tue, 3 May 2016 03:57:13 -0700 Subject: [erlang-questions] Questions about supervision In-Reply-To: References: Message-ID: On Tue, May 3, 2016 at 3:51 AM, Garrett Smith wrote: > Hi Oliver, > > On Tue, May 3, 2016 at 12:54 AM, Oliver Korpilla wrote: > >> I sometimes rely on the supervisor:which_children/1 call to find the IDs of all active groups. When inspecting all three layers of supervision I realized that processes are not cleaned up from the list of children like I expected. > > Inspecting supervisor state is not well supported - and it's not a > great idea. You're better off using something like gproc [1] to > register processes by types-of-interest and use it to enumerate them. > > [1] https://github.com/uwiger/gproc I've long been in the habit of using gproc for this problem, but syn bears mentioning as well: https://github.com/ostinelli/syn From garret.smith@REDACTED Tue May 3 17:52:30 2016 From: garret.smith@REDACTED (Garret Smith) Date: Tue, 3 May 2016 08:52:30 -0700 Subject: [erlang-questions] Mnesia: inconsistent views without netsplit? In-Reply-To: References: Message-ID: On Wed, Apr 27, 2016 at 12:32 PM, Daniel Dormont wrote: > On Wed, Apr 27, 2016 at 3:03 AM, Dan Gudmundsson wrote: >> 1) No clue. But would be interested if you have an idea what have gone >> wrong. >> >> 2) mnesia:del_table_copy(...) followed by mnesia:add_table_copy(..) should >> re-copy the table from the other nodes. > > Thanks. I'll give that a try. > > Along the same lines I was wondering: is there a setting I can use to > adjust the sensitivity of the system's detection of node disconnects, > either generically or specifically within Mnesia? My production > environment appears to have occasional momentary network hiccups (it's > Amazon EC2 instances spanning zones within a region, for anyone > curious). I'd like to make it less likely for those hiccups to cause > Mnesia to enter an inconsistent state, even if it means real failures > take a little longer to detect. If the "hiccups" are high latency, you can look at adjusting net_ticktime, documented here. http://erlang.org/doc/man/kernel_app.html > > thanks, > Dan > > >> >> On Tue, Apr 26, 2016 at 9:30 PM Daniel Dormont >> wrote: >>> >>> Hi all, >>> >>> I have a three node Mnesia cluster (hosting a somewhat outdated >>> version of ejabberd, but I'm not sure that matters). I have a table >>> that is stored as ram_copies on all three nodes. Yet, this table has >>> differing numbers of records among the three. >>> >>> The table info from one of them is pasted below. Running the same >>> query on one of my other nodes, I get more or less the same result, >>> but the "size" is very different: 553 vs 867. And indeed, there are >>> individual records that turn up in a mnesia:read/2 or >>> mnesia:dirty_read/2 on one node and not the other. >>> >>> Yet, nothing in my log indicates that there was ever a netsplit or >>> disconnection. So I have two questions: >>> >>> 1) What might cause this? and >>> 2) Is there any way, especially given I know which records are >>> affected, to force some kind of replication on this table without >>> completely restarting one of the nodes? >>> >>> thanks, >>> Dan Dormont >>> >>> >>> [{access_mode,read_write}, >>> {active_replicas,['ejabberd@REDACTED', >>> 'ejabberd@REDACTED', >>> 'ejabberd@REDACTED']}, >>> {all_nodes,['ejabberd@REDACTED', >>> 'ejabberd@REDACTED', >>> 'ejabberd@REDACTED']}, >>> {arity,3}, >>> {attributes,[name_host,pid]}, >>> {checkpoints,[]}, >>> {commit_work,[]}, >>> {cookie,{{1341,344810,207763},'ejabberd@REDACTED'}}, >>> {cstruct,{cstruct,muc_online_room,set, >>> ['ejabberd@REDACTED', >>> 'ejabberd@REDACTED', >>> 'ejabberd@REDACTED'], >>> [],[],0,read_write,false,[],[],false,muc_online_room, >>> [name_host,pid], >>> [],[],[],{...},...}}, >>> {disc_copies,[]}, >>> {disc_only_copies,[]}, >>> {frag_properties,[]}, >>> {index,[]}, >>> {load_by_force,false}, >>> {load_node,'ejabberd@REDACTED'}, >>> {load_order,0}, >>> {load_reason,{active_remote,'ejabberd@REDACTED'}}, >>> {local_content,false}, >>> {majority,false}, >>> {master_nodes,[]}, >>> {memory,73643}, >>> {ram_copies,['ejabberd@REDACTED', >>> 'ejabberd@REDACTED', >>> 'ejabberd@REDACTED']}, >>> {record_name,muc_online_room}, >>> {record_validation,{muc_online_room,3,set}}, >>> {type,set}, >>> {size,867}, >>> {snmp,[]}, >>> {storage_properties,...}, >>> {...}|...] >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Wed May 4 02:42:24 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 4 May 2016 12:42:24 +1200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> Message-ID: <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> On 3/05/16 7:19 PM, Vlad Dumitrescu wrote: > > This is true and would be useful, but I think I can see a problem > (which might very well be the reason why it isn't something people > already do): after a while, there will be literally thousands of > discarded stuff. How am I supposed to search and find the alternative > that works, especially if I am now working on a different problem than > 6 months ago when the stuff was originally written? Is this search > faster than starting from scratch? IMHO the answer is that what needs > to be saved is a higher-level description, not a complete brain dump > of the process. I think Joe Armstrong has already answered this, and with my mention of version control I thought I had addressed it. The answer is that we should keep a lot more about the history of a program than we (specifically including me in that) do, but that it does not have to go in the source file. I think we agree that when you are trying to put out fires in the current code base, you want to read as little as you safely can, and you don't want to be distracted by stuff that doesn't seem to be relevant. But when you have decided on a fix, that's when you need to review the history to see if someone already thought of it and rejected it for good reason. I'm reminded of an anecdote I read about the Knuth-Morris-Pratt string searching algorithm. (At least, that's what I think it was.) It got built into a particular text editor. A few years later one of the authors was called in and told "your string search algorithm isn't working. Fix it." It turned out that a succession of programmers had looked at the code, thought it was broken, and "fixed" it, and all that had to be done to *really* fix it was to put the old code back. I'm not as good as Joe Armstrong (though I'd like to be). In some of my projects I keep a directory dead.d. Code that seemed like a good idea at the time, and may still have value as a source of ideas, or at least as an (explained) dreadful warning, goes there. For a file foobar.src I often keep a file foobar.txt with musings about what might or might not go in foobar.src, criticisms of what other people have done to explain why I am not going to do that, thoughts about refactoring, &c. I'm not sure that a brain dump of the whole process would be a bad thing, as long as you don't *have* to read through it to see the present source code. > > There is also a difference between what is interesting for teaching > people (beginners) and what is interesting for future reference for > the team that develops the code. I used to have large chunks of the Discworld stories memorised. The story I have in mind at the moment is the contest between Granny Weatherwax (who would be the leader of the witches in the Ramtops if Discworld witches had leaders, but they don't, so everyone just _knows_) and a new witch called, if memory serves me, Diamanda Tockley. GW's friend Nanny Ogg is acting as GW's second. Nanny Ogg [thinks] one day you're going to lose. Diamanda: What level are you on? Granny Weatherwax: Level one. Diamanda [nastily]: Just beginning? Granny Weatherwax. Yes. Every day. Just beginning. Nanny Ogg [thinks] but it's not going to be today. Tell it not in Gath, but I am now 60, and have been programming since about 16. Every day, just beginning. I'm just gone through the Swift book and need to do a lot of exercises. People are telling me I need to learn Pony because it's the latest and greatest. (I've had the Pony compiler on my laptop for *months* and still haven't written a program with it yet.) Other people are telling me I need to understand the Ambient calculus, and I'm really annoyed with myself because I can see that it's *very* closely related to some consulting I did about 10 years ago and it would have been really useful to know back then. Oh, and I've been re-learning a language I last used about 1984, and am starting to feel fluent in it again. Yet if you sit me down in front of a body of code, a project, then I am a *total* newbie all over again. With respect to the *programming language*, I'm not a beginner. With respect to the *project*, I really am. I've been converting some F# code to Smalltalk (and greatly improving it while doing so, if I may say so, boast boast). And some Python code, also to Smalltalk, which Python code turns out to have a *massive* errata list on the web, despite having been published by a previously reputable publisher) so I'm having to rely much more on the textual description than the actual code. I feel very much like a beginner struggling with new stuff, except that this is bigger. As a specific example, sit me down in front of the Cowboy source code and I'm clueless. NOT because it's bad -- I carefully chose an example that isn't -- but because it's a big chunk of code that I am not familiar with. I'm not a beginner with *Erlang* but I'm a beginner with *Cowboy*. I need all the help I can get. > These categories need different levels of detail, different kind of > information. (a) I am not sure that the categories are all that distinct, and (b) absolutely. (I'm reminded of the tracing facilities in Arthur Norman's symbolic integration program, my first introduction to serious logging.) Adequate documentation tools let you annotate a chunk of text with the level it's appropriate for (like the black or white page corners in Wheeler Thorne and Misner's "Gravity", marking out basic and advanced sections, where it looks like I'm *never* going to cope with the advanced sections). Good documentation tools make it easy for you to view a document with only the material appropriate to you visible. One more anecdote. When I joined Quintus, I was handed a ring binder and told "this is the internal documentation. It will help you find your way around the code." I promptly got lost and couldn't find a certain file. "Oh, we forgot to tell you. It's out of date. Half of it is wrong. We didn't bother keeping it up to date because we knew that stuff anyway." From vladdu55@REDACTED Wed May 4 08:49:42 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Wed, 4 May 2016 08:49:42 +0200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> Message-ID: On Wed, May 4, 2016 at 2:42 AM, Richard A. O'Keefe wrote: > On 3/05/16 7:19 PM, Vlad Dumitrescu wrote: > >> >> This is true and would be useful, but I think I can see a problem (which >> might very well be the reason why it isn't something people already do): >> after a while, there will be literally thousands of discarded stuff. How am >> I supposed to search and find the alternative that works, especially if I >> am now working on a different problem than 6 months ago when the stuff was >> originally written? Is this search faster than starting from scratch? IMHO >> the answer is that what needs to be saved is a higher-level description, >> not a complete brain dump of the process. >> > > I think Joe Armstrong has already answered this, and with my mention of > version control > I thought I had addressed it. The answer is that we should keep a lot > more about the > history of a program than we (specifically including me in that) do, but > that it does not > have to go in the source file. > > I don't disagree with you, it's just that for projects larger than toys, I don't know how to browse the history for something that i don't know what it looks like and that might or might not be there. Taking erlide as an example, there are 6000 files in 7000 commits in the main branch, going back 13-14 years and if i would have saved all experiments I'd probably have a tree of at least 5 times that much. I am certain that I wouldn't be able to find anything faster than I would write it again from scratch. We would need an index of the important experiments, with a reason why they didn't were chosen for implementation and maybe a brief description of the design, and a reference to the commits. This requires a lot of discipline to maintain (especially when a team is working on the project, with each person doing its own experiments). regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From henrik.x.nord@REDACTED Wed May 4 13:03:53 2016 From: henrik.x.nord@REDACTED (Henrik Nord X) Date: Wed, 4 May 2016 13:03:53 +0200 Subject: [erlang-questions] Patch package OTP 18.3.3 released Message-ID: <5729D719.2000008@ericsson.com> Patch Package: OTP 18.3.3 Git Tag: OTP-18.3.3 Date: 2016-05-04 Trouble Report Id: OTP-13462, OTP-13513, OTP-13525, OTP-13533, OTP-13535, OTP-13536, OTP-13537 Seq num: seq13077 System: OTP Release: 18 Application: common_test-1.12.1, inets-6.2.3, ssl-7.3.2 Predecessor: OTP 18.3.2 Check out the git tag OTP-18.3.3, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- common_test-1.12.1 ---------------------------------------------- --------------------------------------------------------------------- The common_test-1.12.1 application can be applied independently of other applications on a full OTP 18 installation. --- Fixed Bugs and Malfunctions --- OTP-13462 Application(s): common_test Related Id(s): seq13077 The nodelay option used to be enabled (true) by default for sockets opened by the Common Test telnet client. This appeared to cause communication problems with telnet servers on some systems, and therefore the option is no longer used. Its value may instead be specified in the telnet connection settings. See the man page for ct_telnet for details. Please note that the interface function connect in unix_telnet has been updated with an extra argument and is now unix_telnet:connect/7. OTP-13513 Application(s): common_test Fix bug in cth_surefire: When a pre_init_per_suite hook fails before reaching the cth_surefire:pre_init_per_suite, cth_surefire produced incorrect XML. OTP-13535 Application(s): common_test The ct:get_timetrap_info/0 function has been updated to return more information about timetrap scaling. OTP-13536 Application(s): common_test A problem with stylesheet HTML tags getting incorrectly escaped by Common Test has been corrected. OTP-13537 Application(s): common_test The ct_run start flag -no_esc_chars and ct:run_test/1 start option {esc_chars,Bool} have been introduced to make it possible to disable automatic escaping of characters. Automatic escaping of special HTML characters printed with io:format/1,2 and ct:pal/1,2,3,4 was introduced in Common Test 1.12. The new flag/option may be used to disable this feature for backwards compatibility reasons. (The option is also supported in test specifications). Full runtime dependencies of common_test-1.12.1: compiler-6.0, crypto-3.6, debugger-4.1, erts-7.0, inets-6.0, kernel-4.0, observer-2.1, runtime_tools-1.8.16, sasl-2.4.2, snmp-5.1.2, ssh-4.0, stdlib-2.5, test_server-3.9, tools-2.8, xmerl-1.3.8 --------------------------------------------------------------------- --- inets-6.2.3 ----------------------------------------------------- --------------------------------------------------------------------- The inets-6.2.3 application can be applied independently of other applications on a full OTP 18 installation. --- Improvements and New Features --- OTP-13533 Application(s): inets Put back unused module inets_regexp and remove it in OTP 19 instead as it is an incompatibility, although it is an undocumented module and should not affect other applications. Full runtime dependencies of inets-6.2.3: erts-6.0, kernel-3.0, mnesia-4.12, runtime_tools-1.8.14, ssl-5.3.4, stdlib-2.0 --------------------------------------------------------------------- --- ssl-7.3.2 ------------------------------------------------------- --------------------------------------------------------------------- The ssl-7.3.2 application can be applied independently of other applications on a full OTP 18 installation. --- Fixed Bugs and Malfunctions --- OTP-13525 Application(s): ssl Correct cipher suites conversion and gaurd expression. Caused problems with GCM cipher suites and client side option to set signature_algorithms extention values. Full runtime dependencies of ssl-7.3.2: crypto-3.3, erts-7.0, inets-5.10.7, kernel-3.0, public_key-1.0, stdlib-2.0 From erlang@REDACTED Wed May 4 16:52:21 2016 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 4 May 2016 16:52:21 +0200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> Message-ID: On Wed, May 4, 2016 at 2:42 AM, Richard A. O'Keefe wrote: > On 3/05/16 7:19 PM, Vlad Dumitrescu wrote: >> >> >> This is true and would be useful, but I think I can see a problem (which >> might very well be the reason why it isn't something people already do): >> after a while, there will be literally thousands of discarded stuff. How am >> I supposed to search and find the alternative that works, especially if I am >> now working on a different problem than 6 months ago when the stuff was >> originally written? Is this search faster than starting from scratch? IMHO >> the answer is that what needs to be saved is a higher-level description, not >> a complete brain dump of the process. > > > I think Joe Armstrong has already answered this, and with my mention of > version control > I thought I had addressed it. The answer is that we should keep a lot more > about the > history of a program than we (specifically including me in that) do, but > that it does not > have to go in the source file. > > I think we agree that when you are trying to put out fires in the current > code base, > you want to read as little as you safely can, and you don't want to be > distracted > by stuff that doesn't seem to be relevant. But when you have decided on a > fix, > that's when you need to review the history to see if someone already thought > of > it and rejected it for good reason. > > I'm reminded of an anecdote I read about the Knuth-Morris-Pratt string > searching > algorithm. (At least, that's what I think it was.) It got built into a > particular > text editor. A few years later one of the authors was called in and told > "your > string search algorithm isn't working. Fix it." It turned out that a > succession > of programmers had looked at the code, thought it was broken, and "fixed" > it, > and all that had to be done to *really* fix it was to put the old code back. > > I'm not as good as Joe Armstrong (though I'd like to be). In some of my > projects > I keep a directory dead.d. Code that seemed like a good idea at the time, > and may > still have value as a source of ideas, or at least as an (explained) > dreadful warning, > goes there. For a file foobar.src I often keep a file foobar.txt with > musings about > what might or might not go in foobar.src, criticisms of what other people > have done > to explain why I am not going to do that, thoughts about refactoring, &c. > > I'm not sure that a brain dump of the whole process would be a bad thing, > as long as you don't *have* to read through it to see the present source > code. >> >> >> There is also a difference between what is interesting for teaching people >> (beginners) and what is interesting for future reference for the team that >> develops the code. > > I used to have large chunks of the Discworld stories memorised. > The story I have in mind at the moment is the contest between Granny > Weatherwax (who would be the leader of the witches in the Ramtops > if Discworld witches had leaders, but they don't, so everyone just _knows_) > and a new witch called, if memory serves me, Diamanda Tockley. > GW's friend Nanny Ogg is acting as GW's second. > > Nanny Ogg [thinks] one day you're going to lose. > Diamanda: What level are you on? > Granny Weatherwax: Level one. > Diamanda [nastily]: Just beginning? > Granny Weatherwax. Yes. Every day. Just beginning. > Nanny Ogg [thinks] but it's not going to be today. > > Tell it not in Gath, but I am now 60, and have been programming since > about 16. Every day, just beginning. I'm just gone through the Swift > book and need to do a lot of exercises. People are telling me I need > to learn Pony because it's the latest and greatest. (I've had the Pony > compiler on my laptop for *months* and still haven't written a program > with it yet.) Other people are telling me I need to understand the > Ambient calculus, and I'm really annoyed with myself because I can see > that it's *very* closely related to some consulting I did about 10 years > ago and it would have been really useful to know back then. Oh, and > I've been re-learning a language I last used about 1984, and am starting > to feel fluent in it again. > > Yet if you sit me down in front of a body of code, a project, then I am > a *total* newbie all over again. With respect to the *programming > language*, I'm not a beginner. With respect to the *project*, I really am. > I've been converting some F# code to Smalltalk (and greatly improving it > while doing so, if I may say so, boast boast). And some Python code, > also to Smalltalk, which Python code turns out to have a *massive* errata > list on the web, despite having been published by a previously reputable > publisher) so I'm having to rely much more on the textual description > than the actual code. I feel very much like a beginner struggling with new > stuff, except that this is bigger. > > As a specific example, sit me down in front of the Cowboy source code > and I'm clueless. NOT because it's bad -- I carefully chose an example > that isn't -- but because it's a big chunk of code that I am not familiar > with. > I'm not a beginner with *Erlang* but I'm a beginner with *Cowboy*. > I need all the help I can get. Thank you Richard. I'm always appalled by people who write no documentation at all and then say "read the code" as if this were easy. Imagine the PAIN when I read a comment like this - AAAAAAAAGGGGGGGHHHHH Reading unfamiliar code is appallingly difficult - you're continually trying to guess what problem the author was trying to solve. At best you know what the code does, not what it was supposed to do - for this you must guess. Pure functional requirements help bit are not the full story - it's the "subject to the requirements that ..." which is the killer. That what? - "that it's maintainable, efficient, small, fault-tolerant, understandable, ... I've done code reviews - virtually every time I've red penned some code and said why is it done this way, there turns out to be a non-obvious and undocumented reason why. I'm reminded of a visit to Aachen (I think) - we were show a room where a hermit had been walled in for 20 odd years, getting his food through a hole in the wall. The Guide said " nobody knows why he did this, but there is a reason for everything you just have to ask why" This is like reading code - it's looks weird and there must be a reason but you don't know why, and you can't ask a text file a question. I'd be happy if could understand my own code a few years after I've written it - and *overjoyed* if I could understand other peoples code. Often the effort of "reimplementation from scratch" is less than that of understanding and changing legacy code. Case in point: Once I had to modify some open LDAP code written in C - the code was appalling difficult to understand (I know, it's because I'm the world's worse C programmer) - but nevertheless.. We didn't need all of LDAP, we were using one or two hard coded lookups - I gave up and wrote custom code in Erlang over the lunch break. - I have said several times that we write code because it is quicker to write it from scratch, than to discover code that does what we want, or modify code that does almost what we want but not quite. This is crazy - but as yet I know of no better method and I've been searching for a better method for as long as I've been programming. Using libraries is a double edged sword - great if they work - horrible if they need a slight modification because they do almost what you want but not exactly. I have wasted many hours *assuming* a library was correct, and my code was bad - only to find the opposite - Grrrrrrrrrrrrr This is particularly bad when the library/framework is highly rated and supposedly well-tested - so when I get an error I assume I've made a mistake I start looking at my code, not the library. Then when the evidence become overwhelming, you start reading the library code, and Ouchhhhhh - you have to understand it - which may or may not be easy. Programming is a craft - not a science or art. It should be a science but we're not there yet. /Joe >> >> These categories need different levels of detail, different kind of >> information. > > (a) I am not sure that the categories are all that distinct, and > (b) absolutely. (I'm reminded of the tracing facilities in Arthur Norman's > symbolic integration program, my first introduction to serious logging.) > > Adequate documentation tools let you annotate a chunk of text with > the level it's appropriate for (like the black or white page corners in > Wheeler Thorne and Misner's "Gravity", marking out basic and advanced > sections, where it looks like I'm *never* going to cope with the advanced > sections). Good documentation tools make it easy for you to view a > document with only the material appropriate to you visible. > > One more anecdote. When I joined Quintus, I was handed a ring binder > and told "this is the internal documentation. It will help you find your > way around the code." I promptly got lost and couldn't find a > certain file. "Oh, we forgot to tell you. It's out of date. Half of it is > wrong. We didn't bother keeping it up to date because we knew that > stuff anyway." > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From list1@REDACTED Wed May 4 18:45:01 2016 From: list1@REDACTED (Grzegorz Junka) Date: Wed, 4 May 2016 16:45:01 +0000 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> Message-ID: > - I have said several times that we write code because it is quicker to > write it from scratch, than to discover code that does what we want, or > modify code that does almost what we want but not quite. > That's very true, but it applies all the way through. The more time has been invested into some code the more time it would be needed to write it from scratch. For example, I would prefer Elixir's convention of listing the object on which the operation is being performed as the first argument in library function calls over the OTP convention of listing it as the last argument. But I am not going to rewrite OTP libraries for that small inconvenience. Very often understanding an existing application and fixing issues is easier and quicker than writing a new one from scratch, especially if one only need to understand a specific or small part of it, and even more if the existing application supports plugins allowing to skip large parts of its code in custom modules. This should be a lesson for those who design applications and its interfaces, and in my opinion putting effort into designing the application properly is more important than documenting everything, because it's always a trade-off. Very often the documentation isn't there not because it's not needed, but because there was not enough resources to put it there in the first place. If the architecture is good then the documentation can always be added later. But if the architecture is not good, then no matter how much documentation is written, it will be quickly outdated by the constant and often fixes and patches to that architecture to make it working. Grzegorz From raould@REDACTED Wed May 4 18:50:06 2016 From: raould@REDACTED (Raoul Duke) Date: Wed, 4 May 2016 09:50:06 -0700 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> Message-ID: > This should be a lesson for those who design applications and its interfaces +1 From eric.pailleau@REDACTED Wed May 4 20:42:21 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Wed, 04 May 2016 20:42:21 +0200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> Message-ID: Hi, It is certainly unusual, but I almost always start by writing user documentation, before coding. There is some interests to do so. First, documentation exists at end of project because it exists at beginning. Second, documentation describe simple things like you would need as a user, and not complicated things as consequences of bad coding. Coding is then driven by this goal, and generally simplify the code itself. Third, coders know the goal to achieve at start and do not loose time in, finally, unneeded things. I really recommend to do this experience. "Envoy? depuis mon mobile " Eric ---- Grzegorz Junka a ?crit ---- > >> - I have said several times that we write code because it is quicker to >> write it from scratch, than to discover code that does what we want, or >> modify code that does almost what we want but not quite. >> > >That's very true, but it applies all the way through. The more time has >been invested into some code the more time it would be needed to write >it from scratch. For example, I would prefer Elixir's convention of >listing the object on which the operation is being performed as the >first argument in library function calls over the OTP convention of >listing it as the last argument. But I am not going to rewrite OTP >libraries for that small inconvenience. > >Very often understanding an existing application and fixing issues is >easier and quicker than writing a new one from scratch, especially if >one only need to understand a specific or small part of it, and even >more if the existing application supports plugins allowing to skip large >parts of its code in custom modules. > >This should be a lesson for those who design applications and its >interfaces, and in my opinion putting effort into designing the >application properly is more important than documenting everything, >because it's always a trade-off. Very often the documentation isn't >there not because it's not needed, but because there was not enough >resources to put it there in the first place. > >If the architecture is good then the documentation can always be added >later. But if the architecture is not good, then no matter how much >documentation is written, it will be quickly outdated by the constant >and often fixes and patches to that architecture to make it working. > >Grzegorz >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfidelman@REDACTED Wed May 4 21:02:06 2016 From: mfidelman@REDACTED (Miles Fidelman) Date: Wed, 4 May 2016 15:02:06 -0400 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> Message-ID: <5d752a54-b024-e13d-7b7f-b832ae069710@meetinghouse.net> Along the lines of "write the executive summary first." On 5/4/16 2:42 PM, ?ric Pailleau wrote: > > Hi, > It is certainly unusual, but I almost always start by writing user > documentation, before coding. > There is some interests to do so. > First, documentation exists at end of project because it exists at > beginning. > Second, documentation describe simple things like you would need as a > user, and not complicated things as consequences of bad coding. Coding > is then driven by this goal, and generally simplify the code itself. > Third, coders know the goal to achieve at start and do not loose time > in, finally, unneeded things. > I really recommend to do this experience. > > "Envoy? depuis mon mobile " Eric > > > > ---- Grzegorz Junka a ?crit ---- > > > > - I have said several times that we write code because it is quicker to > > write it from scratch, than to discover code that does what we want, or > > modify code that does almost what we want but not quite. > > > > That's very true, but it applies all the way through. The more time has > been invested into some code the more time it would be needed to write > it from scratch. For example, I would prefer Elixir's convention of > listing the object on which the operation is being performed as the > first argument in library function calls over the OTP convention of > listing it as the last argument. But I am not going to rewrite OTP > libraries for that small inconvenience. > > Very often understanding an existing application and fixing issues is > easier and quicker than writing a new one from scratch, especially if > one only need to understand a specific or small part of it, and even > more if the existing application supports plugins allowing to skip large > parts of its code in custom modules. > > This should be a lesson for those who design applications and its > interfaces, and in my opinion putting effort into designing the > application properly is more important than documenting everything, > because it's always a trade-off. Very often the documentation isn't > there not because it's not needed, but because there was not enough > resources to put it there in the first place. > > If the architecture is good then the documentation can always be added > later. But if the architecture is not good, then no matter how much > documentation is written, it will be quickly outdated by the constant > and often fixes and patches to that architecture to make it working. > > Grzegorz > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra -------------- next part -------------- An HTML attachment was scrubbed... URL: From list1@REDACTED Wed May 4 21:08:17 2016 From: list1@REDACTED (Grzegorz Junka) Date: Wed, 4 May 2016 19:08:17 +0000 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> Message-ID: This is called Documentation Driven Development: http://thinkingphp.org/spliceit/docs/0.1_alpha/pages/ddd_info.html http://stackoverflow.com/questions/588760/documentation-driven-design-your-experiences https://gist.github.com/zsup/9434452 This works as long as the documentation isn't over-specific. Otherwise it turns into a waterfall model of development. Grzegorz On 04/05/2016 18:42, ?ric Pailleau wrote: > > Hi, > It is certainly unusual, but I almost always start by writing user > documentation, before coding. > There is some interests to do so. > First, documentation exists at end of project because it exists at > beginning. > Second, documentation describe simple things like you would need as a > user, and not complicated things as consequences of bad coding. Coding > is then driven by this goal, and generally simplify the code itself. > Third, coders know the goal to achieve at start and do not loose time > in, finally, unneeded things. > I really recommend to do this experience. > > "Envoy? depuis mon mobile " Eric > > > > ---- Grzegorz Junka a ?crit ---- > > > > - I have said several times that we write code because it is quicker to > > write it from scratch, than to discover code that does what we want, or > > modify code that does almost what we want but not quite. > > > > That's very true, but it applies all the way through. The more time has > been invested into some code the more time it would be needed to write > it from scratch. For example, I would prefer Elixir's convention of > listing the object on which the operation is being performed as the > first argument in library function calls over the OTP convention of > listing it as the last argument. But I am not going to rewrite OTP > libraries for that small inconvenience. > > Very often understanding an existing application and fixing issues is > easier and quicker than writing a new one from scratch, especially if > one only need to understand a specific or small part of it, and even > more if the existing application supports plugins allowing to skip large > parts of its code in custom modules. > > This should be a lesson for those who design applications and its > interfaces, and in my opinion putting effort into designing the > application properly is more important than documenting everything, > because it's always a trade-off. Very often the documentation isn't > there not because it's not needed, but because there was not enough > resources to put it there in the first place. > > If the architecture is good then the documentation can always be added > later. But if the architecture is not good, then no matter how much > documentation is written, it will be quickly outdated by the constant > and often fixes and patches to that architecture to make it working. > > Grzegorz > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfidelman@REDACTED Wed May 4 22:36:35 2016 From: mfidelman@REDACTED (Miles Fidelman) Date: Wed, 4 May 2016 16:36:35 -0400 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> Message-ID: <81338ffc-667e-bf87-d988-4f0afd95059c@meetinghouse.net> And what, exactly is wrong with waterfall? (Says this somewhat older, crusty, systems architect?) I've seen way too many agile projects that ignore things like architecture, operating environment, concept of operations, interfaces, systems management, etc., etc., etc. - all those things that are kind of important for mission critical systems. Miles Fidelman On 5/4/16 3:08 PM, Grzegorz Junka wrote: > > This is called Documentation Driven Development: > > http://thinkingphp.org/spliceit/docs/0.1_alpha/pages/ddd_info.html > http://stackoverflow.com/questions/588760/documentation-driven-design-your-experiences > https://gist.github.com/zsup/9434452 > > This works as long as the documentation isn't over-specific. Otherwise > it turns into a waterfall model of development. > > Grzegorz > > > On 04/05/2016 18:42, ?ric Pailleau wrote: >> >> Hi, >> It is certainly unusual, but I almost always start by writing user >> documentation, before coding. >> There is some interests to do so. >> First, documentation exists at end of project because it exists at >> beginning. >> Second, documentation describe simple things like you would need as a >> user, and not complicated things as consequences of bad coding. >> Coding is then driven by this goal, and generally simplify the code >> itself. >> Third, coders know the goal to achieve at start and do not loose time >> in, finally, unneeded things. >> I really recommend to do this experience. >> >> "Envoy? depuis mon mobile " Eric >> >> >> >> ---- Grzegorz Junka a ?crit ---- >> >> >> > - I have said several times that we write code because it is quicker to >> > write it from scratch, than to discover code that does what we want, or >> > modify code that does almost what we want but not quite. >> > >> >> That's very true, but it applies all the way through. The more time has >> been invested into some code the more time it would be needed to write >> it from scratch. For example, I would prefer Elixir's convention of >> listing the object on which the operation is being performed as the >> first argument in library function calls over the OTP convention of >> listing it as the last argument. But I am not going to rewrite OTP >> libraries for that small inconvenience. >> >> Very often understanding an existing application and fixing issues is >> easier and quicker than writing a new one from scratch, especially if >> one only need to understand a specific or small part of it, and even >> more if the existing application supports plugins allowing to skip large >> parts of its code in custom modules. >> >> This should be a lesson for those who design applications and its >> interfaces, and in my opinion putting effort into designing the >> application properly is more important than documenting everything, >> because it's always a trade-off. Very often the documentation isn't >> there not because it's not needed, but because there was not enough >> resources to put it there in the first place. >> >> If the architecture is good then the documentation can always be added >> later. But if the architecture is not good, then no matter how much >> documentation is written, it will be quickly outdated by the constant >> and often fixes and patches to that architecture to make it working. >> >> Grzegorz >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra -------------- next part -------------- An HTML attachment was scrubbed... URL: From raould@REDACTED Wed May 4 22:43:58 2016 From: raould@REDACTED (Raoul Duke) Date: Wed, 4 May 2016 13:43:58 -0700 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: <81338ffc-667e-bf87-d988-4f0afd95059c@meetinghouse.net> References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> <81338ffc-667e-bf87-d988-4f0afd95059c@meetinghouse.net> Message-ID: > And what, exactly is wrong with waterfall? as long as the team agrees on it and can be successful, then in theory nothing is wrong with it :-} http://www.nytimes.com/2016/02/28/magazine/what-google-learned-from-its-quest-to-build-the-perfect-team.html?_r=0 From eric.pailleau@REDACTED Wed May 4 23:42:02 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Wed, 04 May 2016 23:42:02 +0200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> Message-ID: Yes. I do not pretend invent something there. However I do not do test driven dev after documentation. looks better for the team to start coding with a fresh idea of what to do. Tests can limit code inventiveness sometimes. Postone coding start is also stressing due to short delay. I prefere to reduce time to do tests, but deliver in time the project, as far those tests are added during life of programs, while bug fixing. This justify also maintenance costs ;-)... "Envoy? depuis mon mobile " Eric ---- Grzegorz Junka a ?crit ---- >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From list1@REDACTED Thu May 5 00:29:53 2016 From: list1@REDACTED (Grzegorz Junka) Date: Wed, 4 May 2016 22:29:53 +0000 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: <81338ffc-667e-bf87-d988-4f0afd95059c@meetinghouse.net> References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> <81338ffc-667e-bf87-d988-4f0afd95059c@meetinghouse.net> Message-ID: I didn't say there is anything wrong with waterfall, did I? I only said that it would no longer be DDD development but a waterfall project instead. Grzegorz On 04/05/2016 20:36, Miles Fidelman wrote: > > And what, exactly is wrong with waterfall? (Says this somewhat older, > crusty, systems architect?) > > I've seen way too many agile projects that ignore things like > architecture, operating environment, concept of operations, > interfaces, systems management, etc., etc., etc. - all those things > that are kind of important for mission critical systems. > > Miles Fidelman > > > On 5/4/16 3:08 PM, Grzegorz Junka wrote: >> >> This is called Documentation Driven Development: >> >> http://thinkingphp.org/spliceit/docs/0.1_alpha/pages/ddd_info.html >> http://stackoverflow.com/questions/588760/documentation-driven-design-your-experiences >> https://gist.github.com/zsup/9434452 >> >> This works as long as the documentation isn't over-specific. >> Otherwise it turns into a waterfall model of development. >> >> Grzegorz >> >> >> On 04/05/2016 18:42, ?ric Pailleau wrote: >>> >>> Hi, >>> It is certainly unusual, but I almost always start by writing user >>> documentation, before coding. >>> There is some interests to do so. >>> First, documentation exists at end of project because it exists at >>> beginning. >>> Second, documentation describe simple things like you would need as >>> a user, and not complicated things as consequences of bad coding. >>> Coding is then driven by this goal, and generally simplify the code >>> itself. >>> Third, coders know the goal to achieve at start and do not loose >>> time in, finally, unneeded things. >>> I really recommend to do this experience. >>> >>> "Envoy? depuis mon mobile " Eric >>> >>> >>> >>> ---- Grzegorz Junka a ?crit ---- >>> >>> >>> > - I have said several times that we write code because it is >>> quicker to >>> > write it from scratch, than to discover code that does what we >>> want, or >>> > modify code that does almost what we want but not quite. >>> > >>> >>> That's very true, but it applies all the way through. The more time has >>> been invested into some code the more time it would be needed to write >>> it from scratch. For example, I would prefer Elixir's convention of >>> listing the object on which the operation is being performed as the >>> first argument in library function calls over the OTP convention of >>> listing it as the last argument. But I am not going to rewrite OTP >>> libraries for that small inconvenience. >>> >>> Very often understanding an existing application and fixing issues is >>> easier and quicker than writing a new one from scratch, especially if >>> one only need to understand a specific or small part of it, and even >>> more if the existing application supports plugins allowing to skip >>> large >>> parts of its code in custom modules. >>> >>> This should be a lesson for those who design applications and its >>> interfaces, and in my opinion putting effort into designing the >>> application properly is more important than documenting everything, >>> because it's always a trade-off. Very often the documentation isn't >>> there not because it's not needed, but because there was not enough >>> resources to put it there in the first place. >>> >>> If the architecture is good then the documentation can always be added >>> later. But if the architecture is not good, then no matter how much >>> documentation is written, it will be quickly outdated by the constant >>> and often fixes and patches to that architecture to make it working. >>> >>> Grzegorz >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From list1@REDACTED Thu May 5 00:37:22 2016 From: list1@REDACTED (Grzegorz Junka) Date: Wed, 4 May 2016 22:37:22 +0000 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> Message-ID: <77b05878-8d27-dad7-e8b6-a0d2b16511d3@gjunka.com> Excuses, excuses, ... :-) On 04/05/2016 21:42, ?ric Pailleau wrote: > > Yes. I do not pretend invent something there. > However I do not do test driven dev after documentation. looks better > for the team to start coding with a fresh idea of what to do. > Tests can limit code inventiveness sometimes. > Postone coding start is also stressing due to short delay. I prefere > to reduce time to do tests, but deliver in time the project, as far > those tests are added during life of programs, while bug fixing. This > justify also maintenance costs ;-)... > > "Envoy? depuis mon mobile " Eric > > > > ---- Grzegorz Junka a ?crit ---- > > This is called Documentation Driven Development: > > http://thinkingphp.org/spliceit/docs/0.1_alpha/pages/ddd_info.html > http://stackoverflow.com/questions/588760/documentation-driven-design-your-experiences > https://gist.github.com/zsup/9434452 > > This works as long as the documentation isn't over-specific. Otherwise > it turns into a waterfall model of development. > > Grzegorz > > > On 04/05/2016 18:42, ?ric Pailleau wrote: >> >> Hi, >> It is certainly unusual, but I almost always start by writing user >> documentation, before coding. >> There is some interests to do so. >> First, documentation exists at end of project because it exists at >> beginning. >> Second, documentation describe simple things like you would need as a >> user, and not complicated things as consequences of bad coding. >> Coding is then driven by this goal, and generally simplify the code >> itself. >> Third, coders know the goal to achieve at start and do not loose time >> in, finally, unneeded things. >> I really recommend to do this experience. >> >> "Envoy? depuis mon mobile " Eric >> >> >> >> ---- Grzegorz Junka a ?crit ---- >> >> >> > - I have said several times that we write code because it is quicker to >> > write it from scratch, than to discover code that does what we want, or >> > modify code that does almost what we want but not quite. >> > >> >> That's very true, but it applies all the way through. The more time has >> been invested into some code the more time it would be needed to write >> it from scratch. For example, I would prefer Elixir's convention of >> listing the object on which the operation is being performed as the >> first argument in library function calls over the OTP convention of >> listing it as the last argument. But I am not going to rewrite OTP >> libraries for that small inconvenience. >> >> Very often understanding an existing application and fixing issues is >> easier and quicker than writing a new one from scratch, especially if >> one only need to understand a specific or small part of it, and even >> more if the existing application supports plugins allowing to skip large >> parts of its code in custom modules. >> >> This should be a lesson for those who design applications and its >> interfaces, and in my opinion putting effort into designing the >> application properly is more important than documenting everything, >> because it's always a trade-off. Very often the documentation isn't >> there not because it's not needed, but because there was not enough >> resources to put it there in the first place. >> >> If the architecture is good then the documentation can always be added >> later. But if the architecture is not good, then no matter how much >> documentation is written, it will be quickly outdated by the constant >> and often fixes and patches to that architecture to make it working. >> >> Grzegorz >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu May 5 01:19:36 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 5 May 2016 11:19:36 +1200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> Message-ID: <8cc80b18-a0c1-9d4f-aa05-00e3c6f037c9@cs.otago.ac.nz> On 4/05/16 6:49 PM, Vlad Dumitrescu wrote: > > I don't disagree with you, it's just that for projects larger than > toys, I don't know how to browse the history for something that i > don't know what it looks like and that might or might not be there. > Taking erlide as an example, there are 6000 files in 7000 commits in > the main branch, going back 13-14 years and if i would have saved all > experiments I'd probably have a tree of at least 5 times that much. I > am certain that I wouldn't be able to find anything faster than I > would write it again from scratch. Well, I did ask if anyone was interested in collaborating on some research. "I don't know how" is what you say just before "Can I have some grant money to investigate how". (Joke. I don't know anyone who'd give me that money.) With 6000 files of totally unfamiliar code, there's no way I could find anything without a map and ground approach radar. (find . -type -f -print | wc actually counts 2774 files; it did report 6186 before I got rid of all the '._*' junk files you get on a Mac.) OK, so 1344 Java files, 38 Erlang files, 2 Ruby files, 1 XSLT file, and 50-odd Xtend files (which I can't read yet), even hamcrest (oh don't get me started on hamcrest)... With the ._* junk removed, I measure 33.6 MB. This one Eclipse plugin is bigger than the whole Quintus Prolog system, including manuals. Not only that, it's more than half the size of Pharo, which is a complete Smalltalk system including the refactoring browser. There seems to be something about Java that forces systems to grow exceeding large. > > We would need an index of the important experiments, with a reason why > they didn't were chosen for implementation and maybe a brief > description of the design, and a reference to the commits. This > requires a lot of discipline to maintain (especially when a team is > working on the project, with each person doing its own experiments). Such a thing would, however, be extraordinarily useful for someone in my position, with NO idea of where to look for ANYTHING, and a dead link to documentation. The README.md file contains this line: Documentation may be found at [the project site](http://erlide.org/erlide.html). That site isn't supposed to expire until next year, but right now it's not accessible. So yeah, I'd find lots of history very helpful. And lots of advice for the traveller. From lloyd@REDACTED Thu May 5 01:51:16 2016 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Wed, 4 May 2016 19:51:16 -0400 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: <8cc80b18-a0c1-9d4f-aa05-00e3c6f037c9@cs.otago.ac.nz> References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> <8cc80b18-a0c1-9d4f-aa05-00e3c6f037c9@cs.otago.ac.nz> Message-ID: <3A9895F1-A90A-40EE-9F01-AC19A547E066@writersglen.com> Hi all, What a wonderful conversation with many tantalizing insights for a non-professional programmer. I do wonder, though, about this massive erlide source corpus. Couldn't it possibly be factored down into a reasonable set of manageable components with explicit interfaces? I can't help but think of the chip-design CAD system Chuck Moore wrote in forth. Don't exactly recall it's size, but much much smaller than commercial system. Plus, Chuck has designed some amazingly performant chips on this system. His secret: "Factor. Factor. Factor." All the best, LRP Sent from my iPad > On May 4, 2016, at 7:19 PM, Richard A. O'Keefe wrote: > > > >> On 4/05/16 6:49 PM, Vlad Dumitrescu wrote: >> >> I don't disagree with you, it's just that for projects larger than toys, I don't know how to browse the history for something that i don't know what it looks like and that might or might not be there. Taking erlide as an example, there are 6000 files in 7000 commits in the main branch, going back 13-14 years and if i would have saved all experiments I'd probably have a tree of at least 5 times that much. I am certain that I wouldn't be able to find anything faster than I would write it again from scratch. > Well, I did ask if anyone was interested in collaborating on some research. > "I don't know how" is what you say just before "Can I have some grant money to > investigate how". (Joke. I don't know anyone who'd give me that money.) > > With 6000 files of totally unfamiliar code, there's no way I could find > anything without a map and ground approach radar. (find . -type -f -print | > wc actually counts 2774 files; it did report 6186 before I got rid of all > the '._*' junk files you get on a Mac.) OK, so 1344 Java files, 38 Erlang > files, 2 Ruby files, 1 XSLT file, and 50-odd Xtend files (which I can't read > yet), even hamcrest (oh don't get me started on hamcrest)... > > With the ._* junk removed, I measure 33.6 MB. This one Eclipse plugin > is bigger than the whole Quintus Prolog system, including manuals. > > Not only that, it's more than half the size of Pharo, which is a complete > Smalltalk system including the refactoring browser. There seems to be > something about Java that forces systems to grow exceeding large. >> >> We would need an index of the important experiments, with a reason why they didn't were chosen for implementation and maybe a brief description of the design, and a reference to the commits. This requires a lot of discipline to maintain (especially when a team is working on the project, with each person doing its own experiments). > > Such a thing would, however, be extraordinarily useful for someone in my > position, with NO idea of where to look for ANYTHING, and a dead link to > documentation. The README.md file contains this line: > > Documentation may be found at > [the project site](http://erlide.org/erlide.html). > > That site isn't supposed to expire until next year, but right now it's not > accessible. So yeah, I'd find lots of history very helpful. And lots of > advice for the traveller. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From vladdu55@REDACTED Thu May 5 12:07:25 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 5 May 2016 12:07:25 +0200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: <8cc80b18-a0c1-9d4f-aa05-00e3c6f037c9@cs.otago.ac.nz> References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> <8cc80b18-a0c1-9d4f-aa05-00e3c6f037c9@cs.otago.ac.nz> Message-ID: On Thu, May 5, 2016 at 1:19 AM, Richard A. O'Keefe wrote: > > On 4/05/16 6:49 PM, Vlad Dumitrescu wrote: > >> >> I don't disagree with you, it's just that for projects larger than toys, >> I don't know how to browse the history for something that i don't know what >> it looks like and that might or might not be there. Taking erlide as an >> example, there are 6000 files in 7000 commits in the main branch, going >> back 13-14 years and if i would have saved all experiments I'd probably >> have a tree of at least 5 times that much. I am certain that I wouldn't be >> able to find anything faster than I would write it again from scratch. >> > > With 6000 files of totally unfamiliar code, there's no way I could find > anything without a map and ground approach radar. (find . -type -f -print > | > wc actually counts 2774 files; it did report 6186 before I got rid of all > the '._*' junk files you get on a Mac.) OK, so 1344 Java files, 38 Erlang > files, 2 Ruby files, 1 XSLT file, and 50-odd Xtend files (which I can't > read > yet), even hamcrest (oh don't get me started on hamcrest)... > Yeah, I think I forgot to filter out the binary files. Anyway, the point was that at that size, having a multitude of alternative histories, many of which might not be relevant at all any more, it gets exponentially harder to be able to find anything in there. > With the ._* junk removed, I measure 33.6 MB. This one Eclipse plugin > is bigger than the whole Quintus Prolog system, including manuals. > > Not only that, it's more than half the size of Pharo, which is a complete > Smalltalk system including the refactoring browser. There seems to be > something about Java that forces systems to grow exceeding large. Yes, and most of the important stuff (the Erlang implementation of the kernel functionality) is located in another repository. I also had to include some third party libraries as sources, in order to not depend on external stuff whose availability was unreliable. > We would need an index of the important experiments, with a reason why >> they didn't were chosen for implementation and maybe a brief description of >> the design, and a reference to the commits. This requires a lot of >> discipline to maintain (especially when a team is working on the project, >> with each person doing its own experiments). >> > > Such a thing would, however, be extraordinarily useful for someone in my > position, with NO idea of where to look for ANYTHING, and a dead link to > documentation. The README.md file contains this line: > > Documentation may be found at > [the project site](http://erlide.org/erlide.html). > > That site isn't supposed to expire until next year, but right now it's not > accessible. So yeah, I'd find lots of history very helpful. And lots of advice for the traveller. > Thanks for pointing that out, I fixed the link. I will try to keep such a high-level history from now on, I'm sure there will be a lot to learn for myself too. best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Thu May 5 16:51:32 2016 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Thu, 5 May 2016 10:51:32 -0400 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> <8cc80b18-a0c1-9d4f-aa05-00e3c6f037c9@cs.otago.ac.nz> Message-ID: <9F954143-F33E-4E21-913F-552420E36CEC@writersglen.com> Pertinent to the discussion: PROGRAMMING BY POKING: WHY MIT STOPPED TEACHING SICP http://www.posteriorscience.net/?p=206 Best wishes, LRP Sent from my iPad > On May 5, 2016, at 6:07 AM, Vlad Dumitrescu wrote: > > > >> On Thu, May 5, 2016 at 1:19 AM, Richard A. O'Keefe wrote: >> >>> On 4/05/16 6:49 PM, Vlad Dumitrescu wrote: >>> >>> I don't disagree with you, it's just that for projects larger than toys, I don't know how to browse the history for something that i don't know what it looks like and that might or might not be there. Taking erlide as an example, there are 6000 files in 7000 commits in the main branch, going back 13-14 years and if i would have saved all experiments I'd probably have a tree of at least 5 times that much. I am certain that I wouldn't be able to find anything faster than I would write it again from scratch. >> >> With 6000 files of totally unfamiliar code, there's no way I could find >> anything without a map and ground approach radar. (find . -type -f -print | >> wc actually counts 2774 files; it did report 6186 before I got rid of all >> the '._*' junk files you get on a Mac.) OK, so 1344 Java files, 38 Erlang >> files, 2 Ruby files, 1 XSLT file, and 50-odd Xtend files (which I can't read >> yet), even hamcrest (oh don't get me started on hamcrest)... > > Yeah, I think I forgot to filter out the binary files. Anyway, the point was that at that size, having a multitude of alternative histories, many of which might not be relevant at all any more, it gets exponentially harder to be able to find anything in there. > >> With the ._* junk removed, I measure 33.6 MB. This one Eclipse plugin >> is bigger than the whole Quintus Prolog system, including manuals. >> >> Not only that, it's more than half the size of Pharo, which is a complete >> Smalltalk system including the refactoring browser. There seems to be >> something about Java that forces systems to grow exceeding large. > > Yes, and most of the important stuff (the Erlang implementation of the kernel functionality) is located in another repository. I also had to include some third party libraries as sources, in order to not depend on external stuff whose availability was unreliable. > >>> We would need an index of the important experiments, with a reason why they didn't were chosen for implementation and maybe a brief description of the design, and a reference to the commits. This requires a lot of discipline to maintain (especially when a team is working on the project, with each person doing its own experiments). >> >> Such a thing would, however, be extraordinarily useful for someone in my >> position, with NO idea of where to look for ANYTHING, and a dead link to >> documentation. The README.md file contains this line: >> >> Documentation may be found at >> [the project site](http://erlide.org/erlide.html). >> >> That site isn't supposed to expire until next year, but right now it's not >> accessible. So yeah, I'd find lots of history very helpful. And lots of >> advice for the traveller. > > Thanks for pointing that out, I fixed the link. I will try to keep such a high-level history from now on, I'm sure there will be a lot to learn for myself too. > > best regards, > Vlad > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Thu May 5 17:04:21 2016 From: g@REDACTED (Garrett Smith) Date: Thu, 5 May 2016 10:04:21 -0500 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: <9F954143-F33E-4E21-913F-552420E36CEC@writersglen.com> References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> <8cc80b18-a0c1-9d4f-aa05-00e3c6f037c9@cs.otago.ac.nz> <9F954143-F33E-4E21-913F-552420E36CEC@writersglen.com> Message-ID: I was initially excited to read what great breakthroughs in teaching/learning methods this piece would reveal. But it's just terribly sad. If programming was poking at things I didn't understand - in Python moreover - boy I'd be in another profession. I feel bad for those students. On Thu, May 5, 2016 at 9:51 AM, Lloyd R. Prentice wrote: > Pertinent to the discussion: > > PROGRAMMING BY POKING: WHY MIT STOPPED TEACHING SICP > > http://www.posteriorscience.net/?p=206 > > Best wishes, > > LRP > > Sent from my iPad > > On May 5, 2016, at 6:07 AM, Vlad Dumitrescu wrote: > > > > On Thu, May 5, 2016 at 1:19 AM, Richard A. O'Keefe > wrote: >> >> >> On 4/05/16 6:49 PM, Vlad Dumitrescu wrote: >>> >>> >>> I don't disagree with you, it's just that for projects larger than toys, >>> I don't know how to browse the history for something that i don't know what >>> it looks like and that might or might not be there. Taking erlide as an >>> example, there are 6000 files in 7000 commits in the main branch, going back >>> 13-14 years and if i would have saved all experiments I'd probably have a >>> tree of at least 5 times that much. I am certain that I wouldn't be able to >>> find anything faster than I would write it again from scratch. >> >> >> With 6000 files of totally unfamiliar code, there's no way I could find >> anything without a map and ground approach radar. (find . -type -f -print >> | >> wc actually counts 2774 files; it did report 6186 before I got rid of all >> the '._*' junk files you get on a Mac.) OK, so 1344 Java files, 38 Erlang >> files, 2 Ruby files, 1 XSLT file, and 50-odd Xtend files (which I can't >> read >> yet), even hamcrest (oh don't get me started on hamcrest)... > > > Yeah, I think I forgot to filter out the binary files. Anyway, the point was > that at that size, having a multitude of alternative histories, many of > which might not be relevant at all any more, it gets exponentially harder to > be able to find anything in there. > >> >> With the ._* junk removed, I measure 33.6 MB. This one Eclipse plugin >> is bigger than the whole Quintus Prolog system, including manuals. >> >> Not only that, it's more than half the size of Pharo, which is a complete >> Smalltalk system including the refactoring browser. There seems to be >> something about Java that forces systems to grow exceeding large. > > > Yes, and most of the important stuff (the Erlang implementation of the > kernel functionality) is located in another repository. I also had to > include some third party libraries as sources, in order to not depend on > external stuff whose availability was unreliable. > >>> >>> We would need an index of the important experiments, with a reason why >>> they didn't were chosen for implementation and maybe a brief description of >>> the design, and a reference to the commits. This requires a lot of >>> discipline to maintain (especially when a team is working on the project, >>> with each person doing its own experiments). >> >> >> Such a thing would, however, be extraordinarily useful for someone in my >> position, with NO idea of where to look for ANYTHING, and a dead link to >> documentation. The README.md file contains this line: >> >> Documentation may be found at >> [the project site](http://erlide.org/erlide.html). >> >> That site isn't supposed to expire until next year, but right now it's not >> accessible. So yeah, I'd find lots of history very helpful. And lots of >> >> advice for the traveller. > > > Thanks for pointing that out, I fixed the link. I will try to keep such a > high-level history from now on, I'm sure there will be a lot to learn for > myself too. > > best regards, > Vlad > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From g@REDACTED Thu May 5 17:15:26 2016 From: g@REDACTED (Garrett Smith) Date: Thu, 5 May 2016 10:15:26 -0500 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> <8cc80b18-a0c1-9d4f-aa05-00e3c6f037c9@cs.otago.ac.nz> <9F954143-F33E-4E21-913F-552420E36CEC@writersglen.com> Message-ID: I can't even believe I read that - is that article a troll? It's very convincing, apart from the content. On Thu, May 5, 2016 at 10:04 AM, Garrett Smith wrote: > I was initially excited to read what great breakthroughs in > teaching/learning methods this piece would reveal. > > But it's just terribly sad. If programming was poking at things I > didn't understand - in Python moreover - boy I'd be in another > profession. I feel bad for those students. > > On Thu, May 5, 2016 at 9:51 AM, Lloyd R. Prentice wrote: >> Pertinent to the discussion: >> >> PROGRAMMING BY POKING: WHY MIT STOPPED TEACHING SICP >> >> http://www.posteriorscience.net/?p=206 >> >> Best wishes, >> >> LRP >> >> Sent from my iPad >> >> On May 5, 2016, at 6:07 AM, Vlad Dumitrescu wrote: >> >> >> >> On Thu, May 5, 2016 at 1:19 AM, Richard A. O'Keefe >> wrote: >>> >>> >>> On 4/05/16 6:49 PM, Vlad Dumitrescu wrote: >>>> >>>> >>>> I don't disagree with you, it's just that for projects larger than toys, >>>> I don't know how to browse the history for something that i don't know what >>>> it looks like and that might or might not be there. Taking erlide as an >>>> example, there are 6000 files in 7000 commits in the main branch, going back >>>> 13-14 years and if i would have saved all experiments I'd probably have a >>>> tree of at least 5 times that much. I am certain that I wouldn't be able to >>>> find anything faster than I would write it again from scratch. >>> >>> >>> With 6000 files of totally unfamiliar code, there's no way I could find >>> anything without a map and ground approach radar. (find . -type -f -print >>> | >>> wc actually counts 2774 files; it did report 6186 before I got rid of all >>> the '._*' junk files you get on a Mac.) OK, so 1344 Java files, 38 Erlang >>> files, 2 Ruby files, 1 XSLT file, and 50-odd Xtend files (which I can't >>> read >>> yet), even hamcrest (oh don't get me started on hamcrest)... >> >> >> Yeah, I think I forgot to filter out the binary files. Anyway, the point was >> that at that size, having a multitude of alternative histories, many of >> which might not be relevant at all any more, it gets exponentially harder to >> be able to find anything in there. >> >>> >>> With the ._* junk removed, I measure 33.6 MB. This one Eclipse plugin >>> is bigger than the whole Quintus Prolog system, including manuals. >>> >>> Not only that, it's more than half the size of Pharo, which is a complete >>> Smalltalk system including the refactoring browser. There seems to be >>> something about Java that forces systems to grow exceeding large. >> >> >> Yes, and most of the important stuff (the Erlang implementation of the >> kernel functionality) is located in another repository. I also had to >> include some third party libraries as sources, in order to not depend on >> external stuff whose availability was unreliable. >> >>>> >>>> We would need an index of the important experiments, with a reason why >>>> they didn't were chosen for implementation and maybe a brief description of >>>> the design, and a reference to the commits. This requires a lot of >>>> discipline to maintain (especially when a team is working on the project, >>>> with each person doing its own experiments). >>> >>> >>> Such a thing would, however, be extraordinarily useful for someone in my >>> position, with NO idea of where to look for ANYTHING, and a dead link to >>> documentation. The README.md file contains this line: >>> >>> Documentation may be found at >>> [the project site](http://erlide.org/erlide.html). >>> >>> That site isn't supposed to expire until next year, but right now it's not >>> accessible. So yeah, I'd find lots of history very helpful. And lots of >>> >>> advice for the traveller. >> >> >> Thanks for pointing that out, I fixed the link. I will try to keep such a >> high-level history from now on, I'm sure there will be a lot to learn for >> myself too. >> >> best regards, >> Vlad >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> From Oliver.Korpilla@REDACTED Thu May 5 17:19:38 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Thu, 5 May 2016 17:19:38 +0200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> <8cc80b18-a0c1-9d4f-aa05-00e3c6f037c9@cs.otago.ac.nz> <9F954143-F33E-4E21-913F-552420E36CEC@writersglen.com>, Message-ID: It's kinda sad that someone would come up with such a lame conclusion and such woefully incomplete analysis, especially at MIT. Yes, I agree, I do poke libraries a lot to see what they can do. But that is only part 1 of my job. In part 2 to n I take all I know to create my part on top of it, or even to replace parts that I don't like. I mean, I had several pitfalls during learning about Erlang, OTP, and Elixir. That certainly was poking. But without the lessons from SICP and other materials I absorbed throughout the years, where would I go from there? Nowhere. Or somewhere with lots of badly hacked code that will others wish I hadn't. It seems to me this is more a tale of burnout... and maybe even spending too much time in a certain corner of academia. If you get your hands dirty for a prolonged time on a project I don't think you will poke your libraries forever, but rather write whole new ones, some of them just to compensate for the shortcomings of existing ones. And you will create. And reason. And design. And analyse. And yearn for better, more succinct, more concise ways to build software and get ideas from your head into those damn stupid computer things. I don't think poking is enough in any domain. I love Python, but I am glad I actually worked through SICP on my own. Oliver ? Gesendet:?Donnerstag, 05. Mai 2016 um 17:04 Uhr Von:?"Garrett Smith" An:?"Lloyd R. Prentice" Cc:?erlang-questions Betreff:?Re: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? I was initially excited to read what great breakthroughs in teaching/learning methods this piece would reveal. But it's just terribly sad. If programming was poking at things I didn't understand - in Python moreover - boy I'd be in another profession. I feel bad for those students. On Thu, May 5, 2016 at 9:51 AM, Lloyd R. Prentice wrote: > Pertinent to the discussion: > > PROGRAMMING BY POKING: WHY MIT STOPPED TEACHING SICP > > http://www.posteriorscience.net/?p=206 > > Best wishes, > > LRP > > Sent from my iPad > > On May 5, 2016, at 6:07 AM, Vlad Dumitrescu wrote: > > > > On Thu, May 5, 2016 at 1:19 AM, Richard A. O'Keefe > wrote: >> >> >> On 4/05/16 6:49 PM, Vlad Dumitrescu wrote: >>> >>> >>> I don't disagree with you, it's just that for projects larger than toys, >>> I don't know how to browse the history for something that i don't know what >>> it looks like and that might or might not be there. Taking erlide as an >>> example, there are 6000 files in 7000 commits in the main branch, going back >>> 13-14 years and if i would have saved all experiments I'd probably have a >>> tree of at least 5 times that much. I am certain that I wouldn't be able to >>> find anything faster than I would write it again from scratch. >> >> >> With 6000 files of totally unfamiliar code, there's no way I could find >> anything without a map and ground approach radar. (find . -type -f -print >> | >> wc actually counts 2774 files; it did report 6186 before I got rid of all >> the '._*' junk files you get on a Mac.) OK, so 1344 Java files, 38 Erlang >> files, 2 Ruby files, 1 XSLT file, and 50-odd Xtend files (which I can't >> read >> yet), even hamcrest (oh don't get me started on hamcrest)... > > > Yeah, I think I forgot to filter out the binary files. Anyway, the point was > that at that size, having a multitude of alternative histories, many of > which might not be relevant at all any more, it gets exponentially harder to > be able to find anything in there. > >> >> With the ._* junk removed, I measure 33.6 MB. This one Eclipse plugin >> is bigger than the whole Quintus Prolog system, including manuals. >> >> Not only that, it's more than half the size of Pharo, which is a complete >> Smalltalk system including the refactoring browser. There seems to be >> something about Java that forces systems to grow exceeding large. > > > Yes, and most of the important stuff (the Erlang implementation of the > kernel functionality) is located in another repository. I also had to > include some third party libraries as sources, in order to not depend on > external stuff whose availability was unreliable. > >>> >>> We would need an index of the important experiments, with a reason why >>> they didn't were chosen for implementation and maybe a brief description of >>> the design, and a reference to the commits. This requires a lot of >>> discipline to maintain (especially when a team is working on the project, >>> with each person doing its own experiments). >> >> >> Such a thing would, however, be extraordinarily useful for someone in my >> position, with NO idea of where to look for ANYTHING, and a dead link to >> documentation. The README.md file contains this line: >> >> Documentation may be found at >> [the project site](http://erlide.org/erlide.html[http://erlide.org/erlide.html]). >> >> That site isn't supposed to expire until next year, but right now it's not >> accessible. So yeah, I'd find lots of history very helpful. And lots of >> >> advice for the traveller. > > > Thanks for pointing that out, I fixed the link. I will try to keep such a > high-level history from now on, I'm sure there will be a lot to learn for > myself too. > > best regards, > Vlad > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] From felixgallo@REDACTED Thu May 5 17:31:40 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Thu, 5 May 2016 08:31:40 -0700 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> <8cc80b18-a0c1-9d4f-aa05-00e3c6f037c9@cs.otago.ac.nz> <9F954143-F33E-4E21-913F-552420E36CEC@writersglen.com> Message-ID: The current nigh-untenable mess that is computing was created by a bunch of people with rigorous, maths-based training in the formal methods of the time. Imagine what it's going to be like in 20 years when a bunch of people trained primarily to ask plaintive questions on stack overflow are the dominant development culture. Leftpad is going to seem quaint. F. On Thu, May 5, 2016 at 8:19 AM, Oliver Korpilla wrote: > It's kinda sad that someone would come up with such a lame conclusion and > such woefully incomplete analysis, especially at MIT. > > Yes, I agree, I do poke libraries a lot to see what they can do. But that > is only part 1 of my job. > > In part 2 to n I take all I know to create my part on top of it, or even > to replace parts that I don't like. > > I mean, I had several pitfalls during learning about Erlang, OTP, and > Elixir. That certainly was poking. But without the lessons from SICP and > other materials I absorbed throughout the years, where would I go from > there? Nowhere. Or somewhere with lots of badly hacked code that will > others wish I hadn't. > > It seems to me this is more a tale of burnout... and maybe even spending > too much time in a certain corner of academia. If you get your hands dirty > for a prolonged time on a project I don't think you will poke your > libraries forever, but rather write whole new ones, some of them just to > compensate for the shortcomings of existing ones. And you will create. And > reason. And design. And analyse. And yearn for better, more succinct, more > concise ways to build software and get ideas from your head into those damn > stupid computer things. > > I don't think poking is enough in any domain. I love Python, but I am glad > I actually worked through SICP on my own. > > Oliver > > > Gesendet: Donnerstag, 05. Mai 2016 um 17:04 Uhr > Von: "Garrett Smith" > An: "Lloyd R. Prentice" > Cc: erlang-questions > Betreff: Re: [erlang-questions] Rhetorical structure of code: Anyone > interested in collaborating? > I was initially excited to read what great breakthroughs in > teaching/learning methods this piece would reveal. > > But it's just terribly sad. If programming was poking at things I > didn't understand - in Python moreover - boy I'd be in another > profession. I feel bad for those students. > > On Thu, May 5, 2016 at 9:51 AM, Lloyd R. Prentice > wrote: > > Pertinent to the discussion: > > > > PROGRAMMING BY POKING: WHY MIT STOPPED TEACHING SICP > > > > http://www.posteriorscience.net/?p=206 > > > > Best wishes, > > > > LRP > > > > Sent from my iPad > > > > On May 5, 2016, at 6:07 AM, Vlad Dumitrescu wrote: > > > > > > > > On Thu, May 5, 2016 at 1:19 AM, Richard A. O'Keefe > > wrote: > >> > >> > >> On 4/05/16 6:49 PM, Vlad Dumitrescu wrote: > >>> > >>> > >>> I don't disagree with you, it's just that for projects larger than > toys, > >>> I don't know how to browse the history for something that i don't know > what > >>> it looks like and that might or might not be there. Taking erlide as an > >>> example, there are 6000 files in 7000 commits in the main branch, > going back > >>> 13-14 years and if i would have saved all experiments I'd probably > have a > >>> tree of at least 5 times that much. I am certain that I wouldn't be > able to > >>> find anything faster than I would write it again from scratch. > >> > >> > >> With 6000 files of totally unfamiliar code, there's no way I could find > >> anything without a map and ground approach radar. (find . -type -f > -print > >> | > >> wc actually counts 2774 files; it did report 6186 before I got rid of > all > >> the '._*' junk files you get on a Mac.) OK, so 1344 Java files, 38 > Erlang > >> files, 2 Ruby files, 1 XSLT file, and 50-odd Xtend files (which I can't > >> read > >> yet), even hamcrest (oh don't get me started on hamcrest)... > > > > > > Yeah, I think I forgot to filter out the binary files. Anyway, the point > was > > that at that size, having a multitude of alternative histories, many of > > which might not be relevant at all any more, it gets exponentially > harder to > > be able to find anything in there. > > > >> > >> With the ._* junk removed, I measure 33.6 MB. This one Eclipse plugin > >> is bigger than the whole Quintus Prolog system, including manuals. > >> > >> Not only that, it's more than half the size of Pharo, which is a > complete > >> Smalltalk system including the refactoring browser. There seems to be > >> something about Java that forces systems to grow exceeding large. > > > > > > Yes, and most of the important stuff (the Erlang implementation of the > > kernel functionality) is located in another repository. I also had to > > include some third party libraries as sources, in order to not depend on > > external stuff whose availability was unreliable. > > > >>> > >>> We would need an index of the important experiments, with a reason why > >>> they didn't were chosen for implementation and maybe a brief > description of > >>> the design, and a reference to the commits. This requires a lot of > >>> discipline to maintain (especially when a team is working on the > project, > >>> with each person doing its own experiments). > >> > >> > >> Such a thing would, however, be extraordinarily useful for someone in my > >> position, with NO idea of where to look for ANYTHING, and a dead link to > >> documentation. The README.md file contains this line: > >> > >> Documentation may be found at > >> [the project site]( > http://erlide.org/erlide.html[http://erlide.org/erlide.html]). > >> > >> That site isn't supposed to expire until next year, but right now it's > not > >> accessible. So yeah, I'd find lots of history very helpful. And lots of > >> > >> advice for the traveller. > > > > > > Thanks for pointing that out, I fixed the link. I will try to keep such a > > high-level history from now on, I'm sure there will be a lot to learn for > > myself too. > > > > best regards, > > Vlad > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Thu May 5 17:39:11 2016 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 5 May 2016 17:39:11 +0200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> <8cc80b18-a0c1-9d4f-aa05-00e3c6f037c9@cs.otago.ac.nz> Message-ID: Another thought. I think we worry too much about the *inputs* to a program and not the *output* To me something like Markdown is a terrible tool since it cannot produce beautiful output. Markdown is popular since the *input* is easy and the output is tolerated by most people. I find the output horrible - getting quotes right (ie using proper "curly" quotes) is a mess (or impossible) - Even Gutenberg got this right, but there is not a single JavaScript in-line browser editor that gets this right (please tell me if I'm wrong). My Erlang book was authored in XML and transformed to PDF - and while not perfect the typography is acceptable for the amount of effort involved. Once the book project is over the result is a single PDF file containing the book - all the inputs can in principle be thrown away. The directory structure and files used to create the book was a mess by the time the boom was ready. I keep all the files and so on in case they might be useful in the future. They are meaningful to me but probably not understandable by anybody else. I am personally fascinated to learn how artists/muscians/authors create their works - If I see a great painting, I think "I wonder how they did this?" and I like reading descriptions of how they worked. Why? I guess it's because I think "If I understood *how* they had done XYZ" then I myself might be able to do XYZ. I don't really understand how I write programs - I gather information, do experiments, then "something happens" and I can write the program (or not). To an outside, all they can see is (in the case of my book) - the output (one PDF file) and 6365 files used to create the book (many were generated - this is just the *total* number of files in the book directory). As I said I keep the inputs mainly because they might be useful for something else. As well as keeping files in "dustbin" directories (which are project specific) I have taken to storing these *outside* the project directory structure - but in a global directory. I have one directory with a very-large number of sub-directories - I think of this as being in "dictionary" order - ie it's like a dictionary. When I start new projects I botanise among my previous projects collecting stuff and editing into shape - I virtually never reuse code (apart from some very stable libraries) but make small changes to existing code - usually my libraries do almost, but not quite exactly what I want so need small changes. Logs of what I did, and seeing all the inputs and the output would not help at all. Since the relationship between the inputs and outputs is "what happened inside my head" and I haven't a clue how this works. All I know is that if you study a problem, and think a lot, and do loads of experiments then often you'll go "Eureka" and suddenly see how to solve a particular problem. The "Eureka moment" when it comes is highly pleasurably - which is why I guess, I still enjoy programming. Tools in all their glory provide us with information but it's a mystery to me how the brain processes this information. I've been wondering what goes on in my brain when I program for 47 years and still don't know. Cheers /Joe On Thu, May 5, 2016 at 12:07 PM, Vlad Dumitrescu wrote: > > > On Thu, May 5, 2016 at 1:19 AM, Richard A. O'Keefe > wrote: >> >> >> On 4/05/16 6:49 PM, Vlad Dumitrescu wrote: >>> >>> >>> I don't disagree with you, it's just that for projects larger than toys, >>> I don't know how to browse the history for something that i don't know what >>> it looks like and that might or might not be there. Taking erlide as an >>> example, there are 6000 files in 7000 commits in the main branch, going back >>> 13-14 years and if i would have saved all experiments I'd probably have a >>> tree of at least 5 times that much. I am certain that I wouldn't be able to >>> find anything faster than I would write it again from scratch. >> >> >> With 6000 files of totally unfamiliar code, there's no way I could find >> anything without a map and ground approach radar. (find . -type -f -print >> | >> wc actually counts 2774 files; it did report 6186 before I got rid of all >> the '._*' junk files you get on a Mac.) OK, so 1344 Java files, 38 Erlang >> files, 2 Ruby files, 1 XSLT file, and 50-odd Xtend files (which I can't >> read >> yet), even hamcrest (oh don't get me started on hamcrest)... > > > Yeah, I think I forgot to filter out the binary files. Anyway, the point was > that at that size, having a multitude of alternative histories, many of > which might not be relevant at all any more, it gets exponentially harder to > be able to find anything in there. > >> >> With the ._* junk removed, I measure 33.6 MB. This one Eclipse plugin >> is bigger than the whole Quintus Prolog system, including manuals. >> >> Not only that, it's more than half the size of Pharo, which is a complete >> Smalltalk system including the refactoring browser. There seems to be >> something about Java that forces systems to grow exceeding large. > > > Yes, and most of the important stuff (the Erlang implementation of the > kernel functionality) is located in another repository. I also had to > include some third party libraries as sources, in order to not depend on > external stuff whose availability was unreliable. > >>> >>> We would need an index of the important experiments, with a reason why >>> they didn't were chosen for implementation and maybe a brief description of >>> the design, and a reference to the commits. This requires a lot of >>> discipline to maintain (especially when a team is working on the project, >>> with each person doing its own experiments). >> >> >> Such a thing would, however, be extraordinarily useful for someone in my >> position, with NO idea of where to look for ANYTHING, and a dead link to >> documentation. The README.md file contains this line: >> >> Documentation may be found at >> [the project site](http://erlide.org/erlide.html). >> >> That site isn't supposed to expire until next year, but right now it's not >> accessible. So yeah, I'd find lots of history very helpful. And lots of >> >> advice for the traveller. > > > Thanks for pointing that out, I fixed the link. I will try to keep such a > high-level history from now on, I'm sure there will be a lot to learn for > myself too. > > best regards, > Vlad > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From zxq9@REDACTED Thu May 5 17:42:17 2016 From: zxq9@REDACTED (zxq9) Date: Fri, 06 May 2016 00:42:17 +0900 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> Message-ID: <1947673.7cKNDfIJs2@changa> On 2016?5?5? ??? 10:15:26 Garrett Smith wrote: > I can't even believe I read that - is that article a troll? It's very > convincing, apart from the content. Don't worry, you're not the only one who doesn't agree. But after years of seeing entire operating system stacks reimplemented wrongly in javascript, only to be swept away by even crappier rewrites that achieve less with more, I'm not in disbelief or even shock. You have this same understanding of things deep down but can't accept it, and shouldn't, because this approach is indeed wrong. As a counter example... I'm currently working on a team that has just arrived at its magical moment (luckily for me, just as I have arrived), pushing past a gigantic ball of mud (which one of the guys called, in a wonderfully astute use of political office language "a single-component architecture" -- most hilarious joke I've witnessed played straight IRL since leaving the Army), endowed with management that actually trusts us, and that has actually been given an appropriate amount of time for us to experiment a bit, write a lot, present some competing ideas, and collectively settle on component responsibilities and work together to define concrete interfaces among components. And... holy crap, nobody feels depressed when its time to go to work anymore. The code makes sense. We discuss amongst ourselves the best way to convey the purpose of the project, its components, and the code to different types of readers. When a new requirement comes up there is an *obvious* place it should belong, etc. I'm no longer stuck totally SOL if the guy who happens to know how to unjigger the floop in called back callback module some_unmemorable_name_that_makes_no_sense.erl is on vacation for a few days. Writing documentation is not only possible (we forget how hard it is to even document stuff beyond a certain point of project Hell), it is actually enjoyable, and presenting our new direction to the company in internal talks is actually fun again. Etc. And... despite the impulse to say "all that documentation and deliberation is a waste of time" we are *obviously* moving at a pace that dramatically outstrips that of the previous ball-of-shit approach. None of that positive action would have been possible without 1) the team being in a good state, and 2) all of us actually caring about detailing all this "stuffy old engineer stuff". Also, I'm probably the oldest guy on the team (and I'm not very old) -- the pack of young geniuses is *relieved* to do things like establish which direction we're headed before rushing off into an editor. Blah blah. There is a whole world to be learned about learning within projects. I doubt anyone cares much about my anecdote outside of hopefully being able to relate to it (and its unspoken anti-examples), but my point is that your instincts are right and the whole concept of "poking" as the sole method of coding is insane. The stuff we make is always a bit messy. Pretending that things *should* be messy is like denying that the mess exists. That's a more destructive attitude than embracing the reality that iteration toward clean implementation is the natural order of things (and that sometimes new requirements set the realization of your unending trek to perfection back a million miles or so). Meh. This whole thread has caused several cascades of thoughts, emotions and recollections that I don't have time to sort through. Your reaction to that article brought me out of lurk mode, though. -Craig From list1@REDACTED Thu May 5 17:48:31 2016 From: list1@REDACTED (Grzegorz Junka) Date: Thu, 5 May 2016 15:48:31 +0000 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: <9F954143-F33E-4E21-913F-552420E36CEC@writersglen.com> References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> <8cc80b18-a0c1-9d4f-aa05-00e3c6f037c9@cs.otago.ac.nz> <9F954143-F33E-4E21-913F-552420E36CEC@writersglen.com> Message-ID: <81243813-624b-655a-f30b-6f5b8c0d88f2@gjunka.com> Thanks for sharing. From that short article to me it seems that the SICP curriculum was taking a similar approach to programming as the waterfall model, and programming by poking is an approach similar to the Agile methodology. Agile is all about prototyping, implementing a minimum amount of code to get the desired result, and iterating (it's much more, of course, according to the Agile manifesto, but in this comparison I am only concerned with similarities). Greg On 05/05/2016 14:51, Lloyd R. Prentice wrote: > Pertinent to the discussion: > > > PROGRAMMING BY POKING: WHY MIT STOPPED TEACHING SICP > > http://www.posteriorscience.net/?p=206 > > Best wishes, > > LRP > > Sent from my iPad > > On May 5, 2016, at 6:07 AM, Vlad Dumitrescu > wrote: > >> >> >> On Thu, May 5, 2016 at 1:19 AM, Richard A. O'Keefe > > wrote: >> >> >> On 4/05/16 6:49 PM, Vlad Dumitrescu wrote: >> >> >> I don't disagree with you, it's just that for projects larger >> than toys, I don't know how to browse the history for >> something that i don't know what it looks like and that might >> or might not be there. Taking erlide as an example, there are >> 6000 files in 7000 commits in the main branch, going back >> 13-14 years and if i would have saved all experiments I'd >> probably have a tree of at least 5 times that much. I am >> certain that I wouldn't be able to find anything faster than >> I would write it again from scratch. >> >> >> With 6000 files of totally unfamiliar code, there's no way I >> could find >> anything without a map and ground approach radar. (find . -type >> -f -print | >> wc actually counts 2774 files; it did report 6186 before I got >> rid of all >> the '._*' junk files you get on a Mac.) OK, so 1344 Java files, >> 38 Erlang >> files, 2 Ruby files, 1 XSLT file, and 50-odd Xtend files (which I >> can't read >> yet), even hamcrest (oh don't get me started on hamcrest)... >> >> >> Yeah, I think I forgot to filter out the binary files. Anyway, the >> point was that at that size, having a multitude of alternative >> histories, many of which might not be relevant at all any more, it >> gets exponentially harder to be able to find anything in there. >> >> With the ._* junk removed, I measure 33.6 MB. This one Eclipse >> plugin >> is bigger than the whole Quintus Prolog system, including manuals. >> >> Not only that, it's more than half the size of Pharo, which is a >> complete >> Smalltalk system including the refactoring browser. There seems to be >> something about Java that forces systems to grow exceeding large. >> >> >> Yes, and most of the important stuff (the Erlang implementation of >> the kernel functionality) is located in another repository. I also >> had to include some third party libraries as sources, in order to not >> depend on external stuff whose availability was unreliable. >> >> We would need an index of the important experiments, with a >> reason why they didn't were chosen for implementation and >> maybe a brief description of the design, and a reference to >> the commits. This requires a lot of discipline to maintain >> (especially when a team is working on the project, with each >> person doing its own experiments). >> >> >> Such a thing would, however, be extraordinarily useful for >> someone in my >> position, with NO idea of where to look for ANYTHING, and a dead >> link to >> documentation. The README.md file contains this line: >> >> Documentation may be found at >> [the project site](http://erlide.org/erlide.html). >> >> That site isn't supposed to expire until next year, but right now >> it's not >> accessible. So yeah, I'd find lots of history very helpful. And >> lots of >> >> advice for the traveller. >> >> >> Thanks for pointing that out, I fixed the link. I will try to keep >> such a high-level history from now on, I'm sure there will be a lot >> to learn for myself too. >> >> best regards, >> Vlad >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Thu May 5 17:49:59 2016 From: g@REDACTED (Garrett Smith) Date: Thu, 5 May 2016 10:49:59 -0500 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> <8cc80b18-a0c1-9d4f-aa05-00e3c6f037c9@cs.otago.ac.nz> <9F954143-F33E-4E21-913F-552420E36CEC@writersglen.com> Message-ID: Shoot - I'm getting sucked into this increasingly off topic thread. It's great - I love it, but I know there are folks here who would prefer to read discussions about the maps module. To those, forgive me - I need to vent here. I've absolutely _loved_ computers from the age of 9 when my Dad got me a Commodore 64. When I went to college I knew I'd be programming one way or another my whole life so I focused on anything _but_ computers - Economics, music, math, religious history, etc. But I did take some computer science courses - and was privileged to study at Edinburgh in both their AI and CS departments. These programs kicked the snot out of me. We used ML, C, and Assembler. I drowned there. Everyone else seemed to get it and I was lost. I wanted to drop out but stubbornly continued to get smacked around. My grades were finally just okay and I never felt like I conquered anything. I honestly can't imagine calling myself a programmer without having gone through that. That was not Googling Stack Exchange. There were no solutions to steal. It completely sucked. I was a complete idiot. But I learned the fundamentals of this work _and use them every day I sit down with a computer_. This is not a "back in my day" thing. This is teaching reality. Fing hell, poking around at something you don't understand?? Preparing people to excel at horrible jobs?? Someone just asked me on Twitter, what's the alternative to that in education these days? Wow, that's a terrifying question, that it can even be asked seriously. On Thu, May 5, 2016 at 10:15 AM, Garrett Smith wrote: > I can't even believe I read that - is that article a troll? It's very > convincing, apart from the content. > > On Thu, May 5, 2016 at 10:04 AM, Garrett Smith wrote: >> I was initially excited to read what great breakthroughs in >> teaching/learning methods this piece would reveal. >> >> But it's just terribly sad. If programming was poking at things I >> didn't understand - in Python moreover - boy I'd be in another >> profession. I feel bad for those students. >> >> On Thu, May 5, 2016 at 9:51 AM, Lloyd R. Prentice wrote: >>> Pertinent to the discussion: >>> >>> PROGRAMMING BY POKING: WHY MIT STOPPED TEACHING SICP >>> >>> http://www.posteriorscience.net/?p=206 >>> >>> Best wishes, >>> >>> LRP >>> >>> Sent from my iPad >>> >>> On May 5, 2016, at 6:07 AM, Vlad Dumitrescu wrote: >>> >>> >>> >>> On Thu, May 5, 2016 at 1:19 AM, Richard A. O'Keefe >>> wrote: >>>> >>>> >>>> On 4/05/16 6:49 PM, Vlad Dumitrescu wrote: >>>>> >>>>> >>>>> I don't disagree with you, it's just that for projects larger than toys, >>>>> I don't know how to browse the history for something that i don't know what >>>>> it looks like and that might or might not be there. Taking erlide as an >>>>> example, there are 6000 files in 7000 commits in the main branch, going back >>>>> 13-14 years and if i would have saved all experiments I'd probably have a >>>>> tree of at least 5 times that much. I am certain that I wouldn't be able to >>>>> find anything faster than I would write it again from scratch. >>>> >>>> >>>> With 6000 files of totally unfamiliar code, there's no way I could find >>>> anything without a map and ground approach radar. (find . -type -f -print >>>> | >>>> wc actually counts 2774 files; it did report 6186 before I got rid of all >>>> the '._*' junk files you get on a Mac.) OK, so 1344 Java files, 38 Erlang >>>> files, 2 Ruby files, 1 XSLT file, and 50-odd Xtend files (which I can't >>>> read >>>> yet), even hamcrest (oh don't get me started on hamcrest)... >>> >>> >>> Yeah, I think I forgot to filter out the binary files. Anyway, the point was >>> that at that size, having a multitude of alternative histories, many of >>> which might not be relevant at all any more, it gets exponentially harder to >>> be able to find anything in there. >>> >>>> >>>> With the ._* junk removed, I measure 33.6 MB. This one Eclipse plugin >>>> is bigger than the whole Quintus Prolog system, including manuals. >>>> >>>> Not only that, it's more than half the size of Pharo, which is a complete >>>> Smalltalk system including the refactoring browser. There seems to be >>>> something about Java that forces systems to grow exceeding large. >>> >>> >>> Yes, and most of the important stuff (the Erlang implementation of the >>> kernel functionality) is located in another repository. I also had to >>> include some third party libraries as sources, in order to not depend on >>> external stuff whose availability was unreliable. >>> >>>>> >>>>> We would need an index of the important experiments, with a reason why >>>>> they didn't were chosen for implementation and maybe a brief description of >>>>> the design, and a reference to the commits. This requires a lot of >>>>> discipline to maintain (especially when a team is working on the project, >>>>> with each person doing its own experiments). >>>> >>>> >>>> Such a thing would, however, be extraordinarily useful for someone in my >>>> position, with NO idea of where to look for ANYTHING, and a dead link to >>>> documentation. The README.md file contains this line: >>>> >>>> Documentation may be found at >>>> [the project site](http://erlide.org/erlide.html). >>>> >>>> That site isn't supposed to expire until next year, but right now it's not >>>> accessible. So yeah, I'd find lots of history very helpful. And lots of >>>> >>>> advice for the traveller. >>> >>> >>> Thanks for pointing that out, I fixed the link. I will try to keep such a >>> high-level history from now on, I'm sure there will be a lot to learn for >>> myself too. >>> >>> best regards, >>> Vlad >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> From erlang@REDACTED Thu May 5 17:58:28 2016 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 5 May 2016 17:58:28 +0200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> <8cc80b18-a0c1-9d4f-aa05-00e3c6f037c9@cs.otago.ac.nz> <9F954143-F33E-4E21-913F-552420E36CEC@writersglen.com> Message-ID: On Thu, May 5, 2016 at 5:31 PM, Felix Gallo wrote: > The current nigh-untenable mess that is computing was created by a bunch of > people with rigorous, maths-based training in the formal methods of the > time. I don't think this was true - I've been reading the history of the Small Scale Experimental machine https://en.wikipedia.org/wiki/Manchester_Small-Scale_Experimental_Machine The Key players were - Turing (theory) - Williams and Kilburn (Hardware) - Tooting and Kilburn (Software) I've read transcripts of interviews with Kilburn and Tooting. It seems there were three groups of people a) Theory people (Turning) b) Hardware people (Williams and Kilburn) c) Software people (Tooting and Kilburn) I think Kilburn was mainly a software guy - but he's in both camps The transcripts indicated that - nobody understood Turing - Turing didn't understand the hardware. The hardware people didn't understand the theory or the programs. The programmers didn't understand the hardware or the theory. Pretty like today. There are CS theoreticians, programmers and hardware people and they don't understand each other. Knuth wrote paper (this is from memory, so might be wrong) where he observed that programmers were bad mathematicians and vice versa - he thought that since both involved being able to manipulate multiple levels of abstraction that programmers should be good mathematicians and vice versa - but this is not the case. So I think your original statement was a bit off. > Imagine what it's going to be like in 20 years when a bunch of people > trained primarily to ask plaintive questions on stack overflow are the > dominant development culture. Leftpad is going to seem quaint. This I agree with - /Joe > F. > > On Thu, May 5, 2016 at 8:19 AM, Oliver Korpilla > wrote: >> >> It's kinda sad that someone would come up with such a lame conclusion and >> such woefully incomplete analysis, especially at MIT. >> >> Yes, I agree, I do poke libraries a lot to see what they can do. But that >> is only part 1 of my job. >> >> In part 2 to n I take all I know to create my part on top of it, or even >> to replace parts that I don't like. >> >> I mean, I had several pitfalls during learning about Erlang, OTP, and >> Elixir. That certainly was poking. But without the lessons from SICP and >> other materials I absorbed throughout the years, where would I go from >> there? Nowhere. Or somewhere with lots of badly hacked code that will others >> wish I hadn't. >> >> It seems to me this is more a tale of burnout... and maybe even spending >> too much time in a certain corner of academia. If you get your hands dirty >> for a prolonged time on a project I don't think you will poke your libraries >> forever, but rather write whole new ones, some of them just to compensate >> for the shortcomings of existing ones. And you will create. And reason. And >> design. And analyse. And yearn for better, more succinct, more concise ways >> to build software and get ideas from your head into those damn stupid >> computer things. >> >> I don't think poking is enough in any domain. I love Python, but I am glad >> I actually worked through SICP on my own. >> >> Oliver >> >> >> Gesendet: Donnerstag, 05. Mai 2016 um 17:04 Uhr >> Von: "Garrett Smith" >> An: "Lloyd R. Prentice" >> Cc: erlang-questions >> Betreff: Re: [erlang-questions] Rhetorical structure of code: Anyone >> interested in collaborating? >> I was initially excited to read what great breakthroughs in >> teaching/learning methods this piece would reveal. >> >> But it's just terribly sad. If programming was poking at things I >> didn't understand - in Python moreover - boy I'd be in another >> profession. I feel bad for those students. >> >> On Thu, May 5, 2016 at 9:51 AM, Lloyd R. Prentice >> wrote: >> > Pertinent to the discussion: >> > >> > PROGRAMMING BY POKING: WHY MIT STOPPED TEACHING SICP >> > >> > http://www.posteriorscience.net/?p=206 >> > >> > Best wishes, >> > >> > LRP >> > >> > Sent from my iPad >> > >> > On May 5, 2016, at 6:07 AM, Vlad Dumitrescu wrote: >> > >> > >> > >> > On Thu, May 5, 2016 at 1:19 AM, Richard A. O'Keefe >> > wrote: >> >> >> >> >> >> On 4/05/16 6:49 PM, Vlad Dumitrescu wrote: >> >>> >> >>> >> >>> I don't disagree with you, it's just that for projects larger than >> >>> toys, >> >>> I don't know how to browse the history for something that i don't know >> >>> what >> >>> it looks like and that might or might not be there. Taking erlide as >> >>> an >> >>> example, there are 6000 files in 7000 commits in the main branch, >> >>> going back >> >>> 13-14 years and if i would have saved all experiments I'd probably >> >>> have a >> >>> tree of at least 5 times that much. I am certain that I wouldn't be >> >>> able to >> >>> find anything faster than I would write it again from scratch. >> >> >> >> >> >> With 6000 files of totally unfamiliar code, there's no way I could find >> >> anything without a map and ground approach radar. (find . -type -f >> >> -print >> >> | >> >> wc actually counts 2774 files; it did report 6186 before I got rid of >> >> all >> >> the '._*' junk files you get on a Mac.) OK, so 1344 Java files, 38 >> >> Erlang >> >> files, 2 Ruby files, 1 XSLT file, and 50-odd Xtend files (which I can't >> >> read >> >> yet), even hamcrest (oh don't get me started on hamcrest)... >> > >> > >> > Yeah, I think I forgot to filter out the binary files. Anyway, the point >> > was >> > that at that size, having a multitude of alternative histories, many of >> > which might not be relevant at all any more, it gets exponentially >> > harder to >> > be able to find anything in there. >> > >> >> >> >> With the ._* junk removed, I measure 33.6 MB. This one Eclipse plugin >> >> is bigger than the whole Quintus Prolog system, including manuals. >> >> >> >> Not only that, it's more than half the size of Pharo, which is a >> >> complete >> >> Smalltalk system including the refactoring browser. There seems to be >> >> something about Java that forces systems to grow exceeding large. >> > >> > >> > Yes, and most of the important stuff (the Erlang implementation of the >> > kernel functionality) is located in another repository. I also had to >> > include some third party libraries as sources, in order to not depend on >> > external stuff whose availability was unreliable. >> > >> >>> >> >>> We would need an index of the important experiments, with a reason why >> >>> they didn't were chosen for implementation and maybe a brief >> >>> description of >> >>> the design, and a reference to the commits. This requires a lot of >> >>> discipline to maintain (especially when a team is working on the >> >>> project, >> >>> with each person doing its own experiments). >> >> >> >> >> >> Such a thing would, however, be extraordinarily useful for someone in >> >> my >> >> position, with NO idea of where to look for ANYTHING, and a dead link >> >> to >> >> documentation. The README.md file contains this line: >> >> >> >> Documentation may be found at >> >> [the project >> >> site](http://erlide.org/erlide.html[http://erlide.org/erlide.html]). >> >> >> >> That site isn't supposed to expire until next year, but right now it's >> >> not >> >> accessible. So yeah, I'd find lots of history very helpful. And lots of >> >> >> >> advice for the traveller. >> > >> > >> > Thanks for pointing that out, I fixed the link. I will try to keep such >> > a >> > high-level history from now on, I'm sure there will be a lot to learn >> > for >> > myself too. >> > >> > best regards, >> > Vlad >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > >> > http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > >> > http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From raould@REDACTED Thu May 5 20:29:23 2016 From: raould@REDACTED (Raoul Duke) Date: Thu, 5 May 2016 11:29:23 -0700 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: <1947673.7cKNDfIJs2@changa> References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <1947673.7cKNDfIJs2@changa> Message-ID: > Don't worry, you're not the only one who doesn't agree. my god. there are other people out there who are sane? hallelujiah! thanks for giving me some hope. (seriously!) :-| From chassoul@REDACTED Fri May 6 18:12:45 2016 From: chassoul@REDACTED (Jean Chassoul) Date: Fri, 6 May 2016 09:12:45 -0700 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> Message-ID: On Thu, Apr 21, 2016 at 5:09 PM, Robert Virding wrote: > My maps:first/1 and maps:next/2will be coming when? :-) > > +1 (= -------------- next part -------------- An HTML attachment was scrubbed... URL: From christopher.meiklejohn@REDACTED Fri May 6 18:47:45 2016 From: christopher.meiklejohn@REDACTED (Christopher Meiklejohn) Date: Fri, 06 May 2016 16:47:45 +0000 Subject: [erlang-questions] [ANN] Second Call for Papers, May 13th, 2016, PMLDC 2016 Message-ID: First Workshop on Programming Models and Languages for Distributed Computing (PMLDC 2016) Co-located with ECOOP 2016, Rome, Italy Date: July 17th, 2016 http://2016.ecoop.org/track/PMLDC-2016 Whether you are programming a rich web application in JavaScript that mutates state in the client?s browser, or you are building a massively deployed mobile application that will operate with client state at the device, it?s undeniable that you are building a distributed system! Two major challenges of programming distributed systems are concurrency and partial failure. Concurrency of operations can introduce accidental nondeterminism: computations may result in different outcomes with the same inputs given scheduling differences in the underlying system unless a synchronization mechanism is used to enforce some order. Synchronization is typically expensive, and reduces the efficiency of user applications. Partial failure, or the failure of one or more components in a distributed system at one time, introduces the challenge of knowing, when an operation fails, which components of the operation completed successfully. To solve these problems in practice on an unreliable, asynchronous network, atomic commit protocols and timeouts as failure detection are typically used. Because of these challenges, early approaches to providing programming abstractions for distributed computing that ignored them were inherently misguided: the canonical example being the Remote Procedure Call, still widely deployed in industry. The goal of this workshop is to discuss new approaches to distributed programming that provide efficient execution and the elimination of accidental nondeterminism resulting from concurrency and partial failure. It will bring together both practitioners and theoreticians from many disciplines: database theory, distributed systems, systems programming, programming languages, data-centric programming, web application development, and verification, to discuss the state-of-the-art of distributed programming, advancement of the state-of-the-art and paths forward to better application of theory in practice. The main objectives of this workshop are the following: * To review the state-of-the-art research in languages, models, and systems for distributed programming; * To identify areas of critical need where research can advance the state of the art; * To create a forum for discussion; * To present open problems from practitioners with an aim towards motivating academic research on relevant problems faced by industry. In the spirit of both ECOOP and Curry On, this workshop aims at favoring a multidisciplinary perspective by bringing together researchers, developers, and practitioners from both academia and industry. Submission Guidelines We solicit proposals for contributed talks. We recommend preparing proposals of 2 pages, in ACM 2 column SIGPLAN style, written in English and in PDF format. However, we will accept longer proposals or submissions to other conferences, under the understanding that PC members are only expected to read the first two pages of such longer submissions. Authors with accepted papers will have the opportunity to have their submission published on the ACM Digital Library. Important Dates * Paper submission: May 6, 2016 (any place on Earth) * Extended deadline: May 13th, 2016 (any place on Earth) * Authors notification: June 10, 2016 * Final version: June 17, 2016 Program Chairs Heather Miller (Ecole Polytechnique Federale de Lausanne) Christopher Meiklejohn (Universit? catholique de Louvain) Program Committee Peter Alvaro (University of California, Santa Cruz) Annette Bieniusa (Technischen Universit?t Kaiserslautern) Sebastian Burckhardt (Microsoft Research) Natalia Chechina (University of Glasgow) Neil Conway (Mesosphere) Carla Ferreira (Universidade Nova Lisboa) Alexey Gotsman (IMDEA Software Institute) Seyed Hossein Haeri (Universit? catholique de Louvain) Philipp Haller (KTH Royal Institute of Technology) Carl Lerche (Independent Consultant) Rita Loogen (University of Marburg) Rodrigo Rodrigues (Instituto Superior T?cnico, University of Lisboa & INESC-ID) Ali Shoker (HASLab/INESC TEC & University of Minho) Phil Trinder (University of Glasgow) Jos? Valim (Plataformatec) Peter Van Roy (Universit? catholique de Louvain) Hongseok Yang (University of Oxford) -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Fri May 6 19:25:37 2016 From: rvirding@REDACTED (Robert Virding) Date: Fri, 6 May 2016 19:25:37 +0200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> Message-ID: Yes, I am not alone in wanting these! Robert On 6 May 2016 at 18:12, Jean Chassoul wrote: > > On Thu, Apr 21, 2016 at 5:09 PM, Robert Virding > wrote: > >> My maps:first/1 and maps:next/2will be coming when? :-) >> >> > +1 > > (= > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lheadley@REDACTED Fri May 6 20:06:06 2016 From: lheadley@REDACTED (Lyn Headley) Date: Fri, 6 May 2016 11:06:06 -0700 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> Message-ID: I recently conceived a sociotechnical process that would help us understand each other's code. It is simple to implement. In bare bones it is this: I am new to your project. As I read your code, certain events are recorded by my editor and sent to a log that you are monitoring. You can easily browse the log, see where I am bumping into issues, and easily provide feedback. That's it. To put some flesh on it, you might see: Lyn jumped from file:line to function f/2 in file2. Lyn expressed puzzlement. etc. Your feedback could be in the context of this series of events as well. So when I receive it, I am reminded of exactly where I was. As feedback I see the annotated log: Lyn jumped from file:line to function f/2 in file2. Lyn expressed puzzlement. [Joe said: That's an optimization for X case]. That's really it. This is easy to write and I think it's a huge win. What do you think? On Wed, May 4, 2016 at 7:52 AM, Joe Armstrong wrote: > On Wed, May 4, 2016 at 2:42 AM, Richard A. O'Keefe wrote: >> On 3/05/16 7:19 PM, Vlad Dumitrescu wrote: >>> >>> >>> This is true and would be useful, but I think I can see a problem (which >>> might very well be the reason why it isn't something people already do): >>> after a while, there will be literally thousands of discarded stuff. How am >>> I supposed to search and find the alternative that works, especially if I am >>> now working on a different problem than 6 months ago when the stuff was >>> originally written? Is this search faster than starting from scratch? IMHO >>> the answer is that what needs to be saved is a higher-level description, not >>> a complete brain dump of the process. >> >> >> I think Joe Armstrong has already answered this, and with my mention of >> version control >> I thought I had addressed it. The answer is that we should keep a lot more >> about the >> history of a program than we (specifically including me in that) do, but >> that it does not >> have to go in the source file. >> >> I think we agree that when you are trying to put out fires in the current >> code base, >> you want to read as little as you safely can, and you don't want to be >> distracted >> by stuff that doesn't seem to be relevant. But when you have decided on a >> fix, >> that's when you need to review the history to see if someone already thought >> of >> it and rejected it for good reason. >> >> I'm reminded of an anecdote I read about the Knuth-Morris-Pratt string >> searching >> algorithm. (At least, that's what I think it was.) It got built into a >> particular >> text editor. A few years later one of the authors was called in and told >> "your >> string search algorithm isn't working. Fix it." It turned out that a >> succession >> of programmers had looked at the code, thought it was broken, and "fixed" >> it, >> and all that had to be done to *really* fix it was to put the old code back. >> >> I'm not as good as Joe Armstrong (though I'd like to be). In some of my >> projects >> I keep a directory dead.d. Code that seemed like a good idea at the time, >> and may >> still have value as a source of ideas, or at least as an (explained) >> dreadful warning, >> goes there. For a file foobar.src I often keep a file foobar.txt with >> musings about >> what might or might not go in foobar.src, criticisms of what other people >> have done >> to explain why I am not going to do that, thoughts about refactoring, &c. >> >> I'm not sure that a brain dump of the whole process would be a bad thing, >> as long as you don't *have* to read through it to see the present source >> code. >>> >>> >>> There is also a difference between what is interesting for teaching people >>> (beginners) and what is interesting for future reference for the team that >>> develops the code. >> >> I used to have large chunks of the Discworld stories memorised. >> The story I have in mind at the moment is the contest between Granny >> Weatherwax (who would be the leader of the witches in the Ramtops >> if Discworld witches had leaders, but they don't, so everyone just _knows_) >> and a new witch called, if memory serves me, Diamanda Tockley. >> GW's friend Nanny Ogg is acting as GW's second. >> >> Nanny Ogg [thinks] one day you're going to lose. >> Diamanda: What level are you on? >> Granny Weatherwax: Level one. >> Diamanda [nastily]: Just beginning? >> Granny Weatherwax. Yes. Every day. Just beginning. >> Nanny Ogg [thinks] but it's not going to be today. >> >> Tell it not in Gath, but I am now 60, and have been programming since >> about 16. Every day, just beginning. I'm just gone through the Swift >> book and need to do a lot of exercises. People are telling me I need >> to learn Pony because it's the latest and greatest. (I've had the Pony >> compiler on my laptop for *months* and still haven't written a program >> with it yet.) Other people are telling me I need to understand the >> Ambient calculus, and I'm really annoyed with myself because I can see >> that it's *very* closely related to some consulting I did about 10 years >> ago and it would have been really useful to know back then. Oh, and >> I've been re-learning a language I last used about 1984, and am starting >> to feel fluent in it again. >> >> Yet if you sit me down in front of a body of code, a project, then I am >> a *total* newbie all over again. With respect to the *programming >> language*, I'm not a beginner. With respect to the *project*, I really am. >> I've been converting some F# code to Smalltalk (and greatly improving it >> while doing so, if I may say so, boast boast). And some Python code, >> also to Smalltalk, which Python code turns out to have a *massive* errata >> list on the web, despite having been published by a previously reputable >> publisher) so I'm having to rely much more on the textual description >> than the actual code. I feel very much like a beginner struggling with new >> stuff, except that this is bigger. >> >> As a specific example, sit me down in front of the Cowboy source code >> and I'm clueless. NOT because it's bad -- I carefully chose an example >> that isn't -- but because it's a big chunk of code that I am not familiar >> with. >> I'm not a beginner with *Erlang* but I'm a beginner with *Cowboy*. >> I need all the help I can get. > > Thank you Richard. > > I'm always appalled by people who write no documentation at all and then > say "read the code" as if this were easy. Imagine the PAIN when I read > a comment like this - AAAAAAAAGGGGGGGHHHHH > > Reading unfamiliar code is appallingly difficult - you're continually > trying to guess what problem the author was trying to solve. > > At best you know what the code does, not what it was supposed to do - > for this you must guess. > > Pure functional requirements help bit are not the full story - > it's the "subject to the requirements that ..." which is the killer. > > That what? - "that it's maintainable, efficient, small, fault-tolerant, > understandable, ... > > I've done code reviews - virtually every time I've red penned some code > and said why is it done this way, there turns out to be a non-obvious > and undocumented reason why. > > I'm reminded of a visit to Aachen (I think) - we were show a room where > a hermit had been walled in for 20 odd years, getting his food through > a hole in the wall. > > The Guide said " nobody knows why he did this, but there is > a reason for everything you just have to ask why" > > This is like reading code - it's looks weird and there must be a reason > but you don't know why, and you can't ask a text file a question. > > I'd be happy if could understand my own code a few years after I've written > it - and *overjoyed* if I could understand other peoples code. > > Often the effort of "reimplementation from scratch" is less than that > of understanding and changing legacy code. > > Case in point: Once I had to modify some open LDAP code written in > C - the code was appalling difficult to understand (I know, it's because > I'm the world's worse C programmer) - but nevertheless.. > > We didn't need all of LDAP, we were using one or two hard coded > lookups - I gave up and wrote custom code in Erlang over the lunch break. > > - I have said several times that we write code because it is quicker to > write it from scratch, than to discover code that does what we want, or > modify code that does almost what we want but not quite. > > > > This is crazy - but as yet I know of no better method and I've been > searching for a better method for as long as I've been programming. > > > Using libraries is a double edged sword - great if they work - > horrible if they need a slight modification because they do almost > what you want but not exactly. > > I have wasted many hours *assuming* a library was correct, and my > code was bad - only to find the opposite - Grrrrrrrrrrrrr > > This is particularly bad when the library/framework is highly rated and > supposedly well-tested - so when I get an error I assume I've made a mistake > I start looking at my code, not the library. > > Then when the evidence become overwhelming, you start reading the > library code, and Ouchhhhhh - you have to understand it - which may or > may not be easy. > > Programming is a craft - not a science or art. > > It should be a science but we're not there yet. > > /Joe > > > >>> >>> These categories need different levels of detail, different kind of >>> information. >> >> (a) I am not sure that the categories are all that distinct, and >> (b) absolutely. (I'm reminded of the tracing facilities in Arthur Norman's >> symbolic integration program, my first introduction to serious logging.) >> >> Adequate documentation tools let you annotate a chunk of text with >> the level it's appropriate for (like the black or white page corners in >> Wheeler Thorne and Misner's "Gravity", marking out basic and advanced >> sections, where it looks like I'm *never* going to cope with the advanced >> sections). Good documentation tools make it easy for you to view a >> document with only the material appropriate to you visible. >> >> One more anecdote. When I joined Quintus, I was handed a ring binder >> and told "this is the internal documentation. It will help you find your >> way around the code." I promptly got lost and couldn't find a >> certain file. "Oh, we forgot to tell you. It's out of date. Half of it is >> wrong. We didn't bother keeping it up to date because we knew that >> stuff anyway." >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From raould@REDACTED Fri May 6 20:10:14 2016 From: raould@REDACTED (Raoul Duke) Date: Fri, 6 May 2016 11:10:14 -0700 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> Message-ID: (Frankly doesn't sound exciting to me personally, BUT I think if you can get it done & find people to try it then it would be GREAT to do so. Anything and everything done along these lines should be welcomed since we are still in the brainstorming phase of this particular branch of software engineering project fu. :-) -------------- next part -------------- An HTML attachment was scrubbed... URL: From raould@REDACTED Fri May 6 20:49:52 2016 From: raould@REDACTED (Raoul Duke) Date: Fri, 6 May 2016 11:49:52 -0700 Subject: [erlang-questions] [ppig-discuss] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> Message-ID: Possibly of interest https://groups.google.com/d/msg/eve-talk/di-a-d0NJpE/fz40MrzhCAAJ http://leoeditor.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From list1@REDACTED Fri May 6 21:40:48 2016 From: list1@REDACTED (Grzegorz Junka) Date: Fri, 6 May 2016 19:40:48 +0000 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> Message-ID: How would that be maintained once the code starts changing? It's quite similar to code review that's done when merging changes from a developer's branch to the integration branch. So this could be part of a tool to do code reviews. In both cases the history of reviews or comments need to be maintained in subsequent commits. A single solution would be more coherent and easier to use. Anyway, I like the idea. Greg On 06/05/2016 18:06, Lyn Headley wrote: > I recently conceived a sociotechnical process that would help us > understand each other's code. It is simple to implement. In bare bones > it is this: > > I am new to your project. As I read your code, certain events are > recorded by my editor and sent to a log that you are monitoring. You > can easily browse the log, see where I am bumping into issues, and > easily provide feedback. > > That's it. > > To put some flesh on it, you might see: > > Lyn jumped from file:line to function f/2 in file2. > Lyn expressed puzzlement. > > etc. > > Your feedback could be in the context of this series of events as > well. So when I receive it, I am reminded of exactly where I was. As > feedback I see the annotated log: > > Lyn jumped from file:line to function f/2 in file2. > Lyn expressed puzzlement. > [Joe said: That's an optimization for X case]. > > That's really it. This is easy to write and I think it's a huge win. > What do you think? > > > > > > On Wed, May 4, 2016 at 7:52 AM, Joe Armstrong wrote: >> On Wed, May 4, 2016 at 2:42 AM, Richard A. O'Keefe wrote: >>> On 3/05/16 7:19 PM, Vlad Dumitrescu wrote: >>>> >>>> This is true and would be useful, but I think I can see a problem (which >>>> might very well be the reason why it isn't something people already do): >>>> after a while, there will be literally thousands of discarded stuff. How am >>>> I supposed to search and find the alternative that works, especially if I am >>>> now working on a different problem than 6 months ago when the stuff was >>>> originally written? Is this search faster than starting from scratch? IMHO >>>> the answer is that what needs to be saved is a higher-level description, not >>>> a complete brain dump of the process. >>> >>> I think Joe Armstrong has already answered this, and with my mention of >>> version control >>> I thought I had addressed it. The answer is that we should keep a lot more >>> about the >>> history of a program than we (specifically including me in that) do, but >>> that it does not >>> have to go in the source file. >>> >>> I think we agree that when you are trying to put out fires in the current >>> code base, >>> you want to read as little as you safely can, and you don't want to be >>> distracted >>> by stuff that doesn't seem to be relevant. But when you have decided on a >>> fix, >>> that's when you need to review the history to see if someone already thought >>> of >>> it and rejected it for good reason. >>> >>> I'm reminded of an anecdote I read about the Knuth-Morris-Pratt string >>> searching >>> algorithm. (At least, that's what I think it was.) It got built into a >>> particular >>> text editor. A few years later one of the authors was called in and told >>> "your >>> string search algorithm isn't working. Fix it." It turned out that a >>> succession >>> of programmers had looked at the code, thought it was broken, and "fixed" >>> it, >>> and all that had to be done to *really* fix it was to put the old code back. >>> >>> I'm not as good as Joe Armstrong (though I'd like to be). In some of my >>> projects >>> I keep a directory dead.d. Code that seemed like a good idea at the time, >>> and may >>> still have value as a source of ideas, or at least as an (explained) >>> dreadful warning, >>> goes there. For a file foobar.src I often keep a file foobar.txt with >>> musings about >>> what might or might not go in foobar.src, criticisms of what other people >>> have done >>> to explain why I am not going to do that, thoughts about refactoring, &c. >>> >>> I'm not sure that a brain dump of the whole process would be a bad thing, >>> as long as you don't *have* to read through it to see the present source >>> code. >>>> >>>> There is also a difference between what is interesting for teaching people >>>> (beginners) and what is interesting for future reference for the team that >>>> develops the code. >>> I used to have large chunks of the Discworld stories memorised. >>> The story I have in mind at the moment is the contest between Granny >>> Weatherwax (who would be the leader of the witches in the Ramtops >>> if Discworld witches had leaders, but they don't, so everyone just _knows_) >>> and a new witch called, if memory serves me, Diamanda Tockley. >>> GW's friend Nanny Ogg is acting as GW's second. >>> >>> Nanny Ogg [thinks] one day you're going to lose. >>> Diamanda: What level are you on? >>> Granny Weatherwax: Level one. >>> Diamanda [nastily]: Just beginning? >>> Granny Weatherwax. Yes. Every day. Just beginning. >>> Nanny Ogg [thinks] but it's not going to be today. >>> >>> Tell it not in Gath, but I am now 60, and have been programming since >>> about 16. Every day, just beginning. I'm just gone through the Swift >>> book and need to do a lot of exercises. People are telling me I need >>> to learn Pony because it's the latest and greatest. (I've had the Pony >>> compiler on my laptop for *months* and still haven't written a program >>> with it yet.) Other people are telling me I need to understand the >>> Ambient calculus, and I'm really annoyed with myself because I can see >>> that it's *very* closely related to some consulting I did about 10 years >>> ago and it would have been really useful to know back then. Oh, and >>> I've been re-learning a language I last used about 1984, and am starting >>> to feel fluent in it again. >>> >>> Yet if you sit me down in front of a body of code, a project, then I am >>> a *total* newbie all over again. With respect to the *programming >>> language*, I'm not a beginner. With respect to the *project*, I really am. >>> I've been converting some F# code to Smalltalk (and greatly improving it >>> while doing so, if I may say so, boast boast). And some Python code, >>> also to Smalltalk, which Python code turns out to have a *massive* errata >>> list on the web, despite having been published by a previously reputable >>> publisher) so I'm having to rely much more on the textual description >>> than the actual code. I feel very much like a beginner struggling with new >>> stuff, except that this is bigger. >>> >>> As a specific example, sit me down in front of the Cowboy source code >>> and I'm clueless. NOT because it's bad -- I carefully chose an example >>> that isn't -- but because it's a big chunk of code that I am not familiar >>> with. >>> I'm not a beginner with *Erlang* but I'm a beginner with *Cowboy*. >>> I need all the help I can get. >> Thank you Richard. >> >> I'm always appalled by people who write no documentation at all and then >> say "read the code" as if this were easy. Imagine the PAIN when I read >> a comment like this - AAAAAAAAGGGGGGGHHHHH >> >> Reading unfamiliar code is appallingly difficult - you're continually >> trying to guess what problem the author was trying to solve. >> >> At best you know what the code does, not what it was supposed to do - >> for this you must guess. >> >> Pure functional requirements help bit are not the full story - >> it's the "subject to the requirements that ..." which is the killer. >> >> That what? - "that it's maintainable, efficient, small, fault-tolerant, >> understandable, ... >> >> I've done code reviews - virtually every time I've red penned some code >> and said why is it done this way, there turns out to be a non-obvious >> and undocumented reason why. >> >> I'm reminded of a visit to Aachen (I think) - we were show a room where >> a hermit had been walled in for 20 odd years, getting his food through >> a hole in the wall. >> >> The Guide said " nobody knows why he did this, but there is >> a reason for everything you just have to ask why" >> >> This is like reading code - it's looks weird and there must be a reason >> but you don't know why, and you can't ask a text file a question. >> >> I'd be happy if could understand my own code a few years after I've written >> it - and *overjoyed* if I could understand other peoples code. >> >> Often the effort of "reimplementation from scratch" is less than that >> of understanding and changing legacy code. >> >> Case in point: Once I had to modify some open LDAP code written in >> C - the code was appalling difficult to understand (I know, it's because >> I'm the world's worse C programmer) - but nevertheless.. >> >> We didn't need all of LDAP, we were using one or two hard coded >> lookups - I gave up and wrote custom code in Erlang over the lunch break. >> >> - I have said several times that we write code because it is quicker to >> write it from scratch, than to discover code that does what we want, or >> modify code that does almost what we want but not quite. >> >> >> >> This is crazy - but as yet I know of no better method and I've been >> searching for a better method for as long as I've been programming. >> >> >> Using libraries is a double edged sword - great if they work - >> horrible if they need a slight modification because they do almost >> what you want but not exactly. >> >> I have wasted many hours *assuming* a library was correct, and my >> code was bad - only to find the opposite - Grrrrrrrrrrrrr >> >> This is particularly bad when the library/framework is highly rated and >> supposedly well-tested - so when I get an error I assume I've made a mistake >> I start looking at my code, not the library. >> >> Then when the evidence become overwhelming, you start reading the >> library code, and Ouchhhhhh - you have to understand it - which may or >> may not be easy. >> >> Programming is a craft - not a science or art. >> >> It should be a science but we're not there yet. >> >> /Joe >> >> >> >>>> These categories need different levels of detail, different kind of >>>> information. >>> (a) I am not sure that the categories are all that distinct, and >>> (b) absolutely. (I'm reminded of the tracing facilities in Arthur Norman's >>> symbolic integration program, my first introduction to serious logging.) >>> >>> Adequate documentation tools let you annotate a chunk of text with >>> the level it's appropriate for (like the black or white page corners in >>> Wheeler Thorne and Misner's "Gravity", marking out basic and advanced >>> sections, where it looks like I'm *never* going to cope with the advanced >>> sections). Good documentation tools make it easy for you to view a >>> document with only the material appropriate to you visible. >>> >>> One more anecdote. When I joined Quintus, I was handed a ring binder >>> and told "this is the internal documentation. It will help you find your >>> way around the code." I promptly got lost and couldn't find a >>> certain file. "Oh, we forgot to tell you. It's out of date. Half of it is >>> wrong. We didn't bother keeping it up to date because we knew that >>> stuff anyway." >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From rexxe98@REDACTED Fri May 6 22:19:47 2016 From: rexxe98@REDACTED (Andrew Berman) Date: Fri, 06 May 2016 20:19:47 +0000 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> Message-ID: Not sure if this was mentioned yet, but a maps:zip addition like what is in lists only building a map would be really handy. On Fri, May 6, 2016 at 10:26 AM Robert Virding wrote: > Yes, I am not alone in wanting these! > > Robert > > > On 6 May 2016 at 18:12, Jean Chassoul wrote: > >> >> On Thu, Apr 21, 2016 at 5:09 PM, Robert Virding >> wrote: >> >>> My maps:first/1 and maps:next/2will be coming when? :-) >>> >>> >> +1 >> >> (= >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From list1@REDACTED Fri May 6 22:27:29 2016 From: list1@REDACTED (Grzegorz Junka) Date: Fri, 6 May 2016 20:27:29 +0000 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> Message-ID: Andrew, how would the zip function know which keys to join with which values? Unless you mean only zipping values for the same keys? On 06/05/2016 20:19, Andrew Berman wrote: > Not sure if this was mentioned yet, but a maps:zip addition like what > is in lists only building a map would be really handy. > > On Fri, May 6, 2016 at 10:26 AM Robert Virding > wrote: > > Yes, I am not alone in wanting these! > > Robert > > > On 6 May 2016 at 18:12, Jean Chassoul > wrote: > > > On Thu, Apr 21, 2016 at 5:09 PM, Robert Virding > > wrote: > > My maps:first/1 and maps:next/2will be coming when? :-) > > > +1 > > (= > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Sat May 7 00:06:09 2016 From: rexxe98@REDACTED (Andrew Berman) Date: Fri, 06 May 2016 22:06:09 +0000 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> Message-ID: The same way lists:zip/2 does it with the position in each list. So, maps:zip([key1, key2, key3],[val1, val2, val3]) = #{key1 => val1, key2 => val2, key3 => val3}. http://erlang.org/doc/man/lists.html#zip-2 On Fri, May 6, 2016 at 1:27 PM Grzegorz Junka wrote: > Andrew, how would the zip function know which keys to join with which > values? Unless you mean only zipping values for the same keys? > > On 06/05/2016 20:19, Andrew Berman wrote: > > Not sure if this was mentioned yet, but a maps:zip addition like what is > in lists only building a map would be really handy. > > On Fri, May 6, 2016 at 10:26 AM Robert Virding wrote: > >> Yes, I am not alone in wanting these! >> >> Robert >> >> >> On 6 May 2016 at 18:12, Jean Chassoul wrote: >> >>> >>> On Thu, Apr 21, 2016 at 5:09 PM, Robert Virding >>> wrote: >>> >>>> My maps:first/1 and maps:next/2will be coming when? :-) >>>> >>>> >>> +1 >>> >>> (= >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Sat May 7 04:43:03 2016 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 06 May 2016 19:43:03 -0700 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> Message-ID: <572D5637.8040405@gmail.com> On 05/06/2016 10:25 AM, Robert Virding wrote: > Yes, I am not alone in wanting these! > > Robert I agree iterators on maps are helpful. However, I think it is important to make sure the iterator returned from maps:first/1 is its own separate type. In the past when you have suggested this, you appeared to indicate that the iterator would be a side-effect on the Map data. However, if that was the approach, it would be limiting the map to using a single iterator at a time. Other reasons exist why this would be a bad approach: potentially hiding side-effects of iterator usage in ways not represented by types, in ways that dialyzer wouldn't handle (the maps type is already complex), being the source of errors due to the lack of transparency (i.e., it should be breaking referential transparency). So, the interface of maps:first/1 and maps:next/2 sounds good as long as it is dealing with a maps iterator type and not the map itself. > > > On 6 May 2016 at 18:12, Jean Chassoul > wrote: > > > On Thu, Apr 21, 2016 at 5:09 PM, Robert Virding > wrote: > > My maps:first/1 and maps:next/2will be coming when? :-) > > > +1 > > (= > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From list1@REDACTED Sat May 7 08:20:40 2016 From: list1@REDACTED (Grzegorz Junka) Date: Sat, 7 May 2016 06:20:40 +0000 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> Message-ID: <2fe1d93b-3f1e-9cf3-663e-4ccc7ec1a656@gjunka.com> All right, you wanted to zip two lists into a map, rather than two maps into a new map. On 06/05/2016 22:06, Andrew Berman wrote: > The same way lists:zip/2 does it with the position in each list. So, > maps:zip([key1, key2, key3],[val1, val2, val3]) = #{key1 => val1, key2 > => val2, key3 => val3}. > > http://erlang.org/doc/man/lists.html#zip-2 > > On Fri, May 6, 2016 at 1:27 PM Grzegorz Junka > wrote: > > Andrew, how would the zip function know which keys to join with > which values? Unless you mean only zipping values for the same keys? > > > On 06/05/2016 20:19, Andrew Berman wrote: >> Not sure if this was mentioned yet, but a maps:zip addition like >> what is in lists only building a map would be really handy. >> >> On Fri, May 6, 2016 at 10:26 AM Robert Virding >> > wrote: >> >> Yes, I am not alone in wanting these! >> >> Robert >> >> >> On 6 May 2016 at 18:12, Jean Chassoul > > wrote: >> >> >> On Thu, Apr 21, 2016 at 5:09 PM, Robert Virding >> > wrote: >> >> My maps:first/1 and maps:next/2will be coming when? :-) >> >> >> +1 >> >> (= >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Sat May 7 13:21:50 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Sat, 7 May 2016 07:21:50 -0400 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> Message-ID: <20160507112149.GB11705@ferdmbp.local> On 05/06, Andrew Berman wrote: >Not sure if this was mentioned yet, but a maps:zip addition like what is in >lists only building a map would be really handy. > I kind of feel it should *not* be there unless we'd be ready to also have a maps:unzip in there; right now lists:zip and lists:unzip can be combined with maps:to_list and maps:from_list to do what you need, but if you add in maps:zip and forget maps:unzip, that feels incomplete to me! From rexxe98@REDACTED Sat May 7 23:47:38 2016 From: rexxe98@REDACTED (Andrew Berman) Date: Sat, 07 May 2016 21:47:38 +0000 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <20160507112149.GB11705@ferdmbp.local> References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> <20160507112149.GB11705@ferdmbp.local> Message-ID: That's fair, Fred. And yeah, I can definitely do the two-step process for now, but lists:zip/unzip are also just helper functions for what can be done with a list comprehension. I totally understand not bloating the maps module, but if it mimicked the lists module with certain functions, I think it would be ok, plus there's nothing really new to learn. In other words, if you know what lists:zip does, you know what maps:zip does. We're not redefining what the zip function actually does. On Sat, May 7, 2016 at 4:21 AM Fred Hebert wrote: > On 05/06, Andrew Berman wrote: > >Not sure if this was mentioned yet, but a maps:zip addition like what is > in > >lists only building a map would be really handy. > > > I kind of feel it should *not* be there unless we'd be ready to also > have a maps:unzip in there; right now lists:zip and lists:unzip can be > combined with maps:to_list and maps:from_list to do what you need, but > if you add in maps:zip and forget maps:unzip, that feels incomplete to > me! > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wallentin.dahlberg@REDACTED Sun May 8 00:58:39 2016 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Sun, 8 May 2016 00:58:39 +0200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> <20160507112149.GB11705@ferdmbp.local> Message-ID: I think it's great that you all have many suggestions. It took a month before the thread got going. =) maps:take/2 and maps:update_with/3 has already been implemented. iterators or any variants thereof are a no-go for 19.0. maps:merge with fun should be the the dict equivalent. However some thought needs to go into this - a safe way to apply a fun while in midway through a BIF - doable but needs som redesign. Some composability. maps:zip/unzip needs to be fleshed before I can consider it. After 19 I will try to shift focus to iterators and then the compiler, specifically the match compiler. This is however dependent on other prio work i have in my backlog. 2016-05-07 23:47 GMT+02:00 Andrew Berman : > That's fair, Fred. And yeah, I can definitely do the two-step process for > now, but lists:zip/unzip are also just helper functions for what can be > done with a list comprehension. I totally understand not bloating the maps > module, but if it mimicked the lists module with certain functions, I think > it would be ok, plus there's nothing really new to learn. In other words, > if you know what lists:zip does, you know what maps:zip does. We're not > redefining what the zip function actually does. > > On Sat, May 7, 2016 at 4:21 AM Fred Hebert wrote: > >> On 05/06, Andrew Berman wrote: >> >Not sure if this was mentioned yet, but a maps:zip addition like what is >> in >> >lists only building a map would be really handy. >> > >> I kind of feel it should *not* be there unless we'd be ready to also >> have a maps:unzip in there; right now lists:zip and lists:unzip can be >> combined with maps:to_list and maps:from_list to do what you need, but >> if you add in maps:zip and forget maps:unzip, that feels incomplete to >> me! >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mattevans123@REDACTED Sun May 8 02:11:45 2016 From: mattevans123@REDACTED (Matthew Evans) Date: Sat, 7 May 2016 20:11:45 -0400 Subject: [erlang-questions] CNODE performance question Message-ID: Hi, We have a bit of code that was implemented in a NIF: Erlang->NIF->Socket->Destination and the reverse went Source->Socket->NIF->Erlang.The NIF code was getting a bit flakey so we made the call to implement it as a CNODE instead to avoid the NIF taking down the VM if it crashed. Now I understand that a CNODE call will be slower due to the extra hop, but we are seeing a massive (14x) decrease for this particular call. The code is based on the example on the Erlang website (ei_send on the C side and {any,?SERVER} ! Message on the Erlang side).Before we do a deep dive into any other changes I was wondering if you were aware of any quirks in how CNODE is implemented that could cause this? I?m specifically thinking if Erlang uses any name server to identify the physical port.ThanksMatt -------------- next part -------------- An HTML attachment was scrubbed... URL: From khitai.pang@REDACTED Sun May 8 06:33:03 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Sun, 8 May 2016 12:33:03 +0800 Subject: [erlang-questions] exception error: a system limit has been reached Message-ID: When I restart my app I get "escript: exception error: a system limit has been reached". I can't figure out the reason. $ myapp/_rel/myapp/bin/myapp restart Crash dump is being written to: erl_crash.dump...done Kernel pid terminated (application_controller) ({application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{k escript: exception error: a system limit has been reached in function list_to_atom/1 called as list_to_atom("myapp_maint_5500@{error_logger,{{2016,5,8},{12,24,32}},\"Protocol: ~tp: register/listen error: ~tp~n\",[\"inet_tcp\",econnrefused]}\r\n{error_logger,{{2016,5,8},{12,24,32}},crash_report,[[{initial_call,{net_kernel,init,['Argument__1']}},{pid,<0.22.0>},{registered_name,[]},{error_info,{exit,{error,badarg},[{gen_server,init_it,6,[{file,\"gen_server.erl\"},{line,344}]},{proc_lib,init_p_do_apply,3,[{file,\"proc_lib.erl\"},{line,240}]}]}},{ancestors,[net_sup,kernel_sup,<0.10.0>]},{messages,[]},{links,[#Port<0.195>,<0.19.0>]},{dictionary,[{longnames,true}]},{trap_exit,true},{status,running},{heap_size,610},{stack_size,27},{reductions,868}],[]]}\r\n{error_logger,{{2016,5,8},{12,24,32}},supervisor_report,[{supervisor,{local,net_sup}},{errorContext,start_error},{reason,{'EXIT',nodistribution}},{offender,[{pid,undefined},{id,net_kernel},{mfargs,{net_kernel,start_link,[['longname910b8272-myapp',longnames]]}},{restart_type,permanent},{shutdown,2000},{child_type,worker}]}]}\r\n{error_logger,{{2016,5,8},{12,24,32}},supervisor_report,[{supervisor,{local,kernel_sup}},{errorContext,start_error},{reason,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}},{offender,[{pid,undefined},{id,net_sup},{mfargs,{erl_distribution,start_link,[]}},{restart_type,permanent},{shutdown,infinity},{child_type,supervisor}]}]}\r\n{error_logger,{{2016,5,8},{12,24,32}},crash_report,[[{initial_call,{application_master,init,['Argument__1','Argument__2','Argument__3','Argument__4']}},{pid,<0.9.0>},{registered_name,[]},{error_info,{exit,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}},[{application_master,init,4,[{file,\"application_master.erl\"},{line,134}]},{proc_lib,init_p_do_apply,3,[{file,\"proc_lib.erl\"},{line,240}]}]}},{ancestors,[<0.8.0>]},{messages,[{'EXIT',<0.10.0>,normal}]},{links,[<0.8.0>,<0.7.0>]},{dictionary,[]},{trap_exit,true},{status,running},{heap_size,376},{stack_size,27},{reductions,117}],[]]}\r\n{error_logger,{{2016,5,8},{12,24,32}},std_info,[{application,kernel},{exited,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}}},{type,permanent}]}\r\n{\"Kernel pid terminated\",application_controller,\"{application_start_failure,kernel,{{shutdown,{failed_to_start_child,net_sup,{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}}}},{kernel,start,[normal,[]]}}}\"}\r") It's Ubuntu 14.04.4 x86_64 running in a cloud vm with 2G memory. I have the following in /etc/sysctl.conf: ## File Handle Limits # Increate the maximum number of open files fs.file-max = 500000 ## Socket Tuning # Use the full range of ports net.ipv4.ip_local_port_range = 1024 65535 # Enables fast recycling of TIME_WAIT sockets net.ipv4.tcp_tw_recycle = 1 # Allow reuse of sockets in TIME_WAIT state for new connections net.ipv4.tcp_tw_reuse = 1 # Increase the maximum socket receive/send buffer size to 16MB net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 # Enable SYN cookies net.ipv4.tcp_syncookies = 1 # Increase the number of outstanding syn requests allowed net.ipv4.tcp_max_syn_backlog = 4096 # Increase the maximum number of requests queued to a listen socket net.core.somaxconn = 1024 # Increase the maximum number of packets queued on the INPUT side net.core.netdev_max_backlog = 4096 # TCP receive/send buffer size [min, default, max] net.ipv4.tcp_rmem = 4096 12582912 16777216 net.ipv4.tcp_wmem = 4096 12582912 16777216 net.core.somaxconn = 50000 And the following in /etc/security/limits.conf: root soft nofile 65535 root hard nofile 65535 * soft nofile 65535 * hard nofile 65535 free output: > free total used free shared buffers cached Mem: 2047276 979708 1067568 480 56272 264548 -/+ buffers/cache: 658888 1388388 Swap: 0 0 0 Any idea what system limit it reaches and why? Thanks Khitai From zxq9@REDACTED Sun May 8 06:57:03 2016 From: zxq9@REDACTED (zxq9) Date: Sun, 08 May 2016 13:57:03 +0900 Subject: [erlang-questions] exception error: a system limit has been reached In-Reply-To: References: Message-ID: <1612781.W0kHVtz2n8@changa> On 2016?5?8? ??? 12:33:03 Khitai Pang wrote: > When I restart my app I get "escript: exception error: a system limit > has been reached". I can't figure out the reason. > > > $ myapp/_rel/myapp/bin/myapp restart Whatever is happening there is trying to call list_to_atom/1 with a HUUUUUGE string (why is that even happening?). That entire blob of text starting with "myapp_maint_5500@REDACTED" is ONE huge string you are passing to list_to_atom/1. That is beyond the limit for atom sizes, so you get this error: I couldn't find it in the docs anywhere, but the max size of an atom is 256 characters: ceverett@REDACTED:~$ erl Erlang/OTP 18 [erts-7.2] [source] [64-bit] [smp:2:2] [async-threads:10] [kernel-poll:false] Eshell V7.2 (abort with ^G) 1> list_to_atom("zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz"). zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz 2> list_to_atom("zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz"). ** exception error: a system limit has been reached in function list_to_atom/1 called as list_to_atom("zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz") My suspicion is that your "restart" function is accepting as input some output of the last run of the script and expecting to receive its node name as a string (probably "myapp_maint_5500@REDACTED") but instead you are actually getting a huge string with a node name plus a crash report, and trying to restart it with a node name that is the entire previous crash report. From mails@REDACTED Sun May 8 02:19:55 2016 From: mails@REDACTED (Rodrigo Dominguez) Date: Sat, 7 May 2016 21:19:55 -0300 Subject: [erlang-questions] Web Service Samples Message-ID: I'm looking into building Web Services in Erlang as a backend for web applications. I'm wondering if there are nice open source projects I can look at, any open source app with cowboy or webmachine would be nice. I tried to look on code search engines for projects using cowboy or webmachine but I couldn't fine end user applications, I meant, most of the projects I found were tools or modules to build applications, but not complete applications that coudl be deployed and used by end users. To give you an example of applications I've been building: * A backend for game applications where a client can register, login, search for other users, get into a team, record metrics of games, reportings of it. * A backend for an auction system where users can register, publish an auction, list for auctions, bid, etc. * A backend for a CMS, where users can create and collaborate in a document at the same time. Thank you Rod -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Sun May 8 10:24:11 2016 From: roger@REDACTED (Roger Price) Date: Sun, 8 May 2016 10:24:11 +0200 (CEST) Subject: [erlang-questions] Test errors with 18.3 on opensuse 13.1 Message-ID: (Apologies if this is a duplicate posting) Hi, I'm installing 18.3 on GNU/Linux opensuse 13.1 Evergreen (long term support). ./configure reported: ********************** APPLICATIONS DISABLED ********************** jinterface : No Java compiler found odbc : ODBC library - link check failed After the make, and following the installation guide, I ran command $ERL_TOP/bin/erl -s ts install -s ts smoke_test batch -s init stop Three of the tests had non-zero "failed" counters : ... tests.emulator_test.smoke_test_SUITE 2 1 boot_combo tests.emulator_test.time_SUITE 15 2 local_to_univ_utc, time_warp_modes ... tests.kernel_test.heart_SUITE.reboot 0 1 reboot ... Are these failures due to the missing jinterface and odbc applications, or something else? Roger From jesper.louis.andersen@REDACTED Sun May 8 11:04:37 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 8 May 2016 11:04:37 +0200 Subject: [erlang-questions] Test errors with 18.3 on opensuse 13.1 In-Reply-To: References: Message-ID: On Sun, May 8, 2016 at 10:24 AM, Roger Price wrote: > Are these failures due to the missing jinterface and odbc applications, or > something else? > As a first pass, they seem to be related to timing. If the machine is a virtual machine, it could perhaps be explained by that. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Sun May 8 11:27:00 2016 From: roger@REDACTED (Roger Price) Date: Sun, 8 May 2016 11:27:00 +0200 (CEST) Subject: [erlang-questions] Test errors with 18.3 on opensuse 13.1 In-Reply-To: References: Message-ID: On Sun, 8 May 2016, Jesper Louis Andersen wrote: > On Sun, May 8, 2016 at 10:24 AM, Roger Price wrote: > Are these failures due to the missing jinterface and odbc applications, or something else? > > As a first pass, they seem to be related to timing. If the machine is a virtual machine, it could > perhaps be explained by that. The machine is physical, a Dell Precision 690 with 4 cpus and 32 Gbyte memory. Linux kernel version 3.12, gcc version 4.8.1. Maybe it's Dell 690 related. I'll look for another machine to check. Roger From roger@REDACTED Sun May 8 13:02:36 2016 From: roger@REDACTED (Roger Price) Date: Sun, 8 May 2016 13:02:36 +0200 (CEST) Subject: [erlang-questions] Test errors with 18.3 on opensuse 13.1 In-Reply-To: References: Message-ID: On Sun, 8 May 2016, Roger Price wrote: > On Sun, 8 May 2016, Jesper Louis Andersen wrote: > >> On Sun, May 8, 2016 at 10:24 AM, Roger Price wrote: >> Are these failures due to the missing jinterface and odbc >> applications, or something else? >> >> As a first pass, they seem to be related to timing. If the machine is a >> virtual machine, it could perhaps be explained by that. > > The machine is physical, a Dell Precision 690 with 4 cpus and 32 Gbyte > memory. Linux kernel version 3.12, gcc version 4.8.1. > > Maybe it's Dell 690 related. I'll look for another machine to check. The same test errors occur on a Dell Precision T7500 with 12 cpus and 48 Gbyte memory running opensuse 13.1 Evergreen. smoke_test_SUITE boot_combo < > 35.838s FAILED {smoke_test_SUITE,chk_boot,153} {badmatch,{error,timeout}} time_SUITE local_to_univ_utc < > 60.009s FAILED {time_SUITE,local_to_univ_utc,115} suite_failed time_SUITE time_warp_modes < > 38.008s FAILED {time_SUITE,check_time_warp_mode,514} {badmatch,{error,timeout}} heart_SUITE reboot < > 35.728s FAILED {heart_SUITE,start_check,106} {badmatch,{error,timeout}} Roger From marco.molteni@REDACTED Sun May 8 13:08:20 2016 From: marco.molteni@REDACTED (Marco Molteni) Date: Sun, 8 May 2016 13:08:20 +0200 Subject: [erlang-questions] Web Service Samples In-Reply-To: References: Message-ID: <5046BBFA-7CD9-483F-8B91-FF0B368E487D@laposte.net> I find this sample project very useful as an example of integration of various Erlang and web technologies. It has also th e advantage of being relatively small, so easier to understand http://inaka.net/blog/2015/11/13/erlang-serpents/ > On 8 May 2016, at 02:19, Rodrigo Dominguez wrote: > > I'm looking into building Web Services in Erlang as a backend for web applications. > > I'm wondering if there are nice open source projects I can look at, any open source app with cowboy or webmachine would be nice. > > I tried to look on code search engines for projects using cowboy or webmachine but I couldn't fine end user applications, I meant, most of the projects I found were tools or modules to build applications, but not complete applications that coudl be deployed and used by end users. > > To give you an example of applications I've been building: > > * A backend for game applications where a client can register, login, search for other users, get into a team, record metrics of games, reportings of it. > * A backend for an auction system where users can register, publish an auction, list for auctions, bid, etc. > * A backend for a CMS, where users can create and collaborate in a document at the same time. > > Thank you > > Rod > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From list1@REDACTED Sun May 8 13:11:55 2016 From: list1@REDACTED (Grzegorz Junka) Date: Sun, 8 May 2016 11:11:55 +0000 Subject: [erlang-questions] Web Service Samples In-Reply-To: References: Message-ID: <5ed0b7bd-957a-6117-5e1a-f3219cbaef44@gjunka.com> There is zotonic.com which is a complete CMS, so contains code on both, front end and back end. Do you mean a complete web application, say a discussion forum, or something like the three examples you gave? Would you be able to give examples of such open source projects in other languages? On 08/05/2016 00:19, Rodrigo Dominguez wrote: > I'm looking into building Web Services in Erlang as a backend for web > applications. > > I'm wondering if there are nice open source projects I can look at, > any open source app with cowboy or webmachine would be nice. > > I tried to look on code search engines for projects using cowboy or > webmachine but I couldn't fine end user applications, I meant, most of > the projects I found were tools or modules to build applications, but > not complete applications that coudl be deployed and used by end users. > > To give you an example of applications I've been building: > > * A backend for game applications where a client can register, login, > search for other users, get into a team, record metrics of games, > reportings of it. > * A backend for an auction system where users can register, publish an > auction, list for auctions, bid, etc. > * A backend for a CMS, where users can create and collaborate in a > document at the same time. > > Thank you > > Rod > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From fernando.benavides@REDACTED Sun May 8 13:19:12 2016 From: fernando.benavides@REDACTED (Brujo Benavides) Date: Sun, 8 May 2016 08:19:12 -0300 Subject: [erlang-questions] Web Service Samples In-Reply-To: References: Message-ID: Hi Rodrigo, We (http://inaka.github.io) have a couple of apps that fit your description. You should check: - https://github.com/inaka/canillita - https://github.com/inaka/lsl - https://github.com/inaka/serpents - https://github.com/inaka/fiar Cheers! On May 8, 2016 3:58 AM, "Rodrigo Dominguez" wrote: > I'm looking into building Web Services in Erlang as a backend for web > applications. > > I'm wondering if there are nice open source projects I can look at, any > open source app with cowboy or webmachine would be nice. > > I tried to look on code search engines for projects using cowboy or > webmachine but I couldn't fine end user applications, I meant, most of the > projects I found were tools or modules to build applications, but not > complete applications that coudl be deployed and used by end users. > > To give you an example of applications I've been building: > > * A backend for game applications where a client can register, login, > search for other users, get into a team, record metrics of games, > reportings of it. > * A backend for an auction system where users can register, publish an > auction, list for auctions, bid, etc. > * A backend for a CMS, where users can create and collaborate in a > document at the same time. > > Thank you > > Rod > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From santif@REDACTED Sun May 8 17:35:03 2016 From: santif@REDACTED (Santiago Fernandez) Date: Sun, 8 May 2016 12:35:03 -0300 Subject: [erlang-questions] exception error: a system limit has been reached In-Reply-To: <1612781.W0kHVtz2n8@changa> References: <1612781.W0kHVtz2n8@changa> Message-ID: As Craig said, there is a limit of 255 characters in an atom http://erlang.org/doc/efficiency_guide/advanced.html -- Santiago On Sun, May 8, 2016 at 1:57 AM, zxq9 wrote: > On 2016?5?8? ??? 12:33:03 Khitai Pang wrote: > > When I restart my app I get "escript: exception error: a system limit > > has been reached". I can't figure out the reason. > > > > > > $ myapp/_rel/myapp/bin/myapp restart > > Whatever is happening there is trying to call list_to_atom/1 with a > HUUUUUGE string (why is that even happening?). That entire blob of text > starting with "myapp_maint_5500@REDACTED" is ONE huge string you are passing > to list_to_atom/1. That is beyond the limit for atom sizes, so you get this > error: > > I couldn't find it in the docs anywhere, but the max size of an atom is > 256 characters: > > > ceverett@REDACTED:~$ erl > Erlang/OTP 18 [erts-7.2] [source] [64-bit] [smp:2:2] [async-threads:10] > [kernel-poll:false] > > Eshell V7.2 (abort with ^G) > 1> > list_to_atom("zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz"). > > zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz > 2> > list_to_atom("zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz"). > ** exception error: a system limit has been reached > in function list_to_atom/1 > called as > list_to_atom("zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz") > > > My suspicion is that your "restart" function is accepting as input some > output of the last run of the script and expecting to receive its node name > as a string (probably "myapp_maint_5500@REDACTED") but instead you are > actually getting a huge string with a node name plus a crash report, and > trying to restart it with a node name that is the entire previous crash > report. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Sun May 8 20:14:22 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 8 May 2016 20:14:22 +0200 Subject: [erlang-questions] [ANN] PGPool 1.1.0 - now with batch and cached queries Message-ID: Dear All, PGPool 1.1.0 has been released. For those of you who don't know what PGPool is: PGPool is a PosgreSQL client that automatically uses connection pools and handles reconnections in case of errors. Main additions: - A batch/2 function that allows to run queries in a batch. - PGPool now automatically prepares your queries and caches them, This has a considerable performance impact. You can find PGPool here: https://github.com/ostinelli/pgpool Best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Sun May 8 20:25:46 2016 From: rexxe98@REDACTED (Andrew Berman) Date: Sun, 08 May 2016 18:25:46 +0000 Subject: [erlang-questions] [ANN] PGPool 1.1.0 - now with batch and cached queries In-Reply-To: References: Message-ID: Awesome stuff, Roberto! I'm going to start using this. One thing, though, you may want to consider renaming the library to avoid confusion with pgpool ( http://www.pgpool.net/mediawiki/index.php/Documentation), which has been around for a very long time. Thanks for your hard work on this! A On Sun, May 8, 2016 at 11:14 AM Roberto Ostinelli wrote: > Dear All, > PGPool 1.1.0 has been released. > > For those of you who don't know what PGPool is: PGPool is a PosgreSQL > client that automatically uses connection pools and handles reconnections > in case of errors. > > Main additions: > > - A batch/2 function that allows to run queries in a batch. > - PGPool now automatically prepares your queries and caches them, This > has a considerable performance impact. > > You can find PGPool here: > https://github.com/ostinelli/pgpool > > Best, > r. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Sun May 8 20:31:47 2016 From: rexxe98@REDACTED (Andrew Berman) Date: Sun, 08 May 2016 18:31:47 +0000 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> <20160507112149.GB11705@ferdmbp.local> Message-ID: Sorry, Bj?rn-Egil, I just had the need come up while developing a couple days ago, so I thought I'd throw maps:zip on the radar. On Sat, May 7, 2016 at 3:58 PM Bj?rn-Egil Dahlberg < wallentin.dahlberg@REDACTED> wrote: > I think it's great that you all have many suggestions. It took a month > before the thread got going. =) > > maps:take/2 and maps:update_with/3 has already been implemented. > > iterators or any variants thereof are a no-go for 19.0. > > maps:merge with fun should be the the dict equivalent. However some > thought needs to go into this - a safe way to apply a fun while in midway > through a BIF - doable but needs som redesign. Some composability. > > maps:zip/unzip needs to be fleshed before I can consider it. > > After 19 I will try to shift focus to iterators and then the compiler, > specifically the match compiler. This is however dependent on other prio > work i have in my backlog. > > > > 2016-05-07 23:47 GMT+02:00 Andrew Berman : > >> That's fair, Fred. And yeah, I can definitely do the two-step process >> for now, but lists:zip/unzip are also just helper functions for what can be >> done with a list comprehension. I totally understand not bloating the maps >> module, but if it mimicked the lists module with certain functions, I think >> it would be ok, plus there's nothing really new to learn. In other words, >> if you know what lists:zip does, you know what maps:zip does. We're not >> redefining what the zip function actually does. >> >> On Sat, May 7, 2016 at 4:21 AM Fred Hebert wrote: >> >>> On 05/06, Andrew Berman wrote: >>> >Not sure if this was mentioned yet, but a maps:zip addition like what >>> is in >>> >lists only building a map would be really handy. >>> > >>> I kind of feel it should *not* be there unless we'd be ready to also >>> have a maps:unzip in there; right now lists:zip and lists:unzip can be >>> combined with maps:to_list and maps:from_list to do what you need, but >>> if you add in maps:zip and forget maps:unzip, that feels incomplete to >>> me! >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto.ostinelli@REDACTED Sun May 8 21:13:14 2016 From: roberto.ostinelli@REDACTED (Roberto Ostinelli) Date: Sun, 8 May 2016 21:13:14 +0200 Subject: [erlang-questions] [ANN] PGPool 1.1.0 - now with batch and cached queries In-Reply-To: References: Message-ID: Thank you Andrew. > you may want to consider renaming the library to avoid confusion with pgpool (http://www.pgpool.net/mediawiki/index.php/Documentation) You might be right. Sounds like quite a hassle though, especially for anyone already using it. Does this bother anyone else? -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Sun May 8 21:22:25 2016 From: t@REDACTED (Tristan Sloughter) Date: Sun, 08 May 2016 14:22:25 -0500 Subject: [erlang-questions] [ANN] PGPool 1.1.0 - now with batch and cached queries In-Reply-To: References: Message-ID: <1462735345.935719.601608441.11B759BA@webmail.messagingengine.com> Looks great, but you'll need to update the hex package because it is missing dependencies: https://hex.pm/packages/pgpool A hex package only allows for hex dependencies. Otherwise rebar3/mix/etc can't properly resolve the dependencies to install. Looks like this is the case for a few projects, https://hex.pm/packages/epgpool. I'll start opening issues for them as I find them. -- Tristan Sloughter t@REDACTED On Sun, May 8, 2016, at 01:14 PM, Roberto Ostinelli wrote: > Dear All, > PGPool 1.1.0 has been released. > > For those of you who don't know what PGPool is: PGPool is a PosgreSQL > client that automatically uses connection pools and handles > reconnections in case of errors. > > Main additions: > * A batch/2 function that allows to run queries in a batch. > * PGPool now automatically prepares your queries and caches them, > This has a considerable performance impact. > You can find PGPool here: > https://github.com/ostinelli/pgpool > > Best, > r. > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Sun May 8 21:30:15 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Sun, 8 May 2016 21:30:15 +0200 Subject: [erlang-questions] [ANN] PGPool 1.1.0 - now with batch and cached queries In-Reply-To: <1462735345.935719.601608441.11B759BA@webmail.messagingengine.com> References: <1462735345.935719.601608441.11B759BA@webmail.messagingengine.com> Message-ID: On Sun, May 8, 2016 at 9:22 PM, Tristan Sloughter wrote: > Looks great, but you'll need to update the hex package because it is > missing dependencies: https://hex.pm/packages/pgpool > > A hex package only allows for hex dependencies. Otherwise rebar3/mix/etc > can't properly resolve the dependencies to install. > Hey Tristan, Still relatively unfamiliar with hex packages, as you can see. Does this mean that the rebar.config file should be something like this instead? {deps, [ {epgsql, "3.2.0"}, {poolboy, "1.5.1"} ]}. If so, where can I find the docs to ensure compatibility with rebar too? -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Sun May 8 21:38:52 2016 From: t@REDACTED (Tristan Sloughter) Date: Sun, 08 May 2016 14:38:52 -0500 Subject: [erlang-questions] [ANN] PGPool 1.1.0 - now with batch and cached queries In-Reply-To: References: <1462735345.935719.601608441.11B759BA@webmail.messagingengine.com> Message-ID: <1462736332.938057.601618521.426C7755@webmail.messagingengine.com> Tell anyone on rebar2 to upgrade, hehe. Na, you can do it with rebar.config.script, http://www.rebar3.org/docs/from-rebar-2x-to-rebar3#section-maintaining-backwards-compatibility-while-using-hex-packages -- Tristan Sloughter t@REDACTED On Sun, May 8, 2016, at 02:30 PM, Roberto Ostinelli wrote: > On Sun, May 8, 2016 at 9:22 PM, Tristan Sloughter > wrote: >> __ >> Looks great, but you'll need to update the hex package because it is >> missing dependencies: https://hex.pm/packages/pgpool >> >> A hex package only allows for hex dependencies. Otherwise >> rebar3/mix/etc can't properly resolve the dependencies to install. > > Hey Tristan, > Still relatively unfamiliar with hex packages, as you can see. > > Does this mean that the rebar.config[1] file should be something like > this instead? > > {deps, [ > {epgsql, "3.2.0"}, > {poolboy, "1.5.1"} > ]}. > If so, where can I find the docs to ensure compatibility with > rebar too? Links: 1. https://github.com/ostinelli/pgpool/blob/master/rebar.config -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Sun May 8 21:46:59 2016 From: mjtruog@REDACTED (Michael Truog) Date: Sun, 08 May 2016 12:46:59 -0700 Subject: [erlang-questions] Web Service Samples In-Reply-To: References: Message-ID: <572F97B3.3010301@gmail.com> CloudI (http://cloudi.org) provides a service abstraction for Erlang and other programming languages. The services don't need to be web services, but can be by using services with cowboy and/or elli that is provided (as cloudi_service_http_cowboy and cloudi_service_http_elli, or adding a different service for a HTTP server). For Erlang-only usage the https://github.com/CloudI/cloudi_core repo is provided and https://github.com/CloudI/CloudI/tree/develop/examples/hello_world5#hello-world-5-example provides information about embedding cloudi_core usage in an Erlang application. Some examples that use non-Erlang programming languages are: https://github.com/okeuday/sillymud https://github.com/okeuday/odroid_fish On 05/07/2016 05:19 PM, Rodrigo Dominguez wrote: > I'm looking into building Web Services in Erlang as a backend for web applications. > > I'm wondering if there are nice open source projects I can look at, any open source app with cowboy or webmachine would be nice. > > I tried to look on code search engines for projects using cowboy or webmachine but I couldn't fine end user applications, I meant, most of the projects I found were tools or modules to build applications, but not complete applications that coudl be deployed and used by end users. > > To give you an example of applications I've been building: > > * A backend for game applications where a client can register, login, search for other users, get into a team, record metrics of games, reportings of it. > * A backend for an auction system where users can register, publish an auction, list for auctions, bid, etc. > * A backend for a CMS, where users can create and collaborate in a document at the same time. > > Thank you > > Rod > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Mon May 9 02:01:31 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 9 May 2016 12:01:31 +1200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> Message-ID: <3fe10814-0754-7d37-eeb9-03f6fab40795@cs.otago.ac.nz> In the frames proposal, frames were intrinsically ordered, so first and next functions made sense. I thought maps were *not* intrinsically ordered. What is the first maplet in a map and what is the next? From ok@REDACTED Mon May 9 03:01:02 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 9 May 2016 13:01:02 +1200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> Message-ID: On 7/05/16 6:06 AM, Lyn Headley wrote: > I recently conceived a sociotechnical process that would help us > understand each other's code. It is simple to implement. In bare bones > it is this: > > I am new to your project. As I read your code, certain events are > recorded by my editor and sent to a log that you are monitoring. You > can easily browse the log, see where I am bumping into issues, and > easily provide feedback. One problem with this is that it is *syntactic*, not *semantic*. The editor tells me *what* you are looking at, but not *why* you are looking at it, not even whether it is what you *intended* to look at or something you arrived at by accident. And since we are (probably) on opposite sides of the planet, you are most unlikely to be able to give me feedback at the time I need it. > That's it. > > To put some flesh on it, you might see: > > Lyn jumped from file:line to function f/2 in file2. > Lyn expressed puzzlement. How does the editor know that Lyn expressed puzzlement? Is it because the comment is in M?ori, or because it looks as though there might be a division by zero, or because this is Python3 and Lyn is used to Python2? > Your feedback could be in the context of this series of events as > well. So when I receive it, I am reminded of exactly where I was. As > feedback I see the annotated log: > > Lyn jumped from file:line to function f/2 in file2. > Lyn expressed puzzlement. > [Joe said: That's an optimization for X case]. > > That's really it. This is easy to write and I think it's a huge win. > What do you think? *What* is easy to write? I think we need a lot more detail before it is possible to make that judgement. Possibly the easiest way to do this might be through some sort of video screen capture with audio annotation, where Lyn *says* she is puzzled, in which case it may be possible to try this out using existing tools. I think something like this is DEFINITELY worth trying. It does have a downside. I started this thread talking about 1 person leaving signposts (rhetorical structure annotations) for N > 1 readers. The editing log idea is 1:1; if Helen wrote the code and Lyn recorded herself viewing it and Helen replies to Lyn, that helps Lyn, but it doesn't help Sue, unless she knows that Helen's reply to Lyn's session exists. From raould@REDACTED Mon May 9 04:24:52 2016 From: raould@REDACTED (Raoul Duke) Date: Sun, 8 May 2016 19:24:52 -0700 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> Message-ID: Not video. Audio. http://www.gridshore.nl/wp-content/uploads/code-quality-measurement.jpg -------------- next part -------------- An HTML attachment was scrubbed... URL: From oliver.bollmann@REDACTED Mon May 9 06:26:47 2016 From: oliver.bollmann@REDACTED (Oliver Bollmann(T-Online)) Date: Mon, 9 May 2016 06:26:47 +0200 Subject: [erlang-questions] Windows 10 Erlsrv Message-ID: <0867CD06DC174611B8103F2770806984@cordawaredrei.local> Seems there is a problem that erlsrv want to read HKEY_LOCAL_MACHINE\SOFTWARE\Ericsson\Erlang\ErlSrv\1.1\InternalServiceName instead of HKEY_LOCAL_MACHINE\SOFTWARE\Ericsson\Erlang\ErlSrv\1.1\DisplayName this happened only on boot of windows, starting manual the service works, Set InternalServiceName = DisplayName solves the problem! BTW: RabbitMQ does it too. you may verify this behaviour using procmon from sysinternal, https://technet.microsoft.com/en-us/sysinternals/processmonitor Oliver From cvi@REDACTED Mon May 9 08:08:48 2016 From: cvi@REDACTED (=?UTF-8?Q?Christoffer_Vikstr=C3=B6m?=) Date: Mon, 09 May 2016 06:08:48 +0000 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <3fe10814-0754-7d37-eeb9-03f6fab40795@cs.otago.ac.nz> References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> <3fe10814-0754-7d37-eeb9-03f6fab40795@cs.otago.ac.nz> Message-ID: I find myself reimplementing the dict:append/3 function but for maps a lot. Maybe it's just me, but that one would be nice to have in maps. -cvi Den m?n 9 maj 2016 02:01Richard A. O'Keefe skrev: > In the frames proposal, frames were intrinsically > ordered, so first and next functions made sense. > > I thought maps were *not* intrinsically ordered. > What is the first maplet in a map and what is the next? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Mon May 9 10:59:27 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Mon, 9 May 2016 10:59:27 +0200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <3fe10814-0754-7d37-eeb9-03f6fab40795@cs.otago.ac.nz> References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> <3fe10814-0754-7d37-eeb9-03f6fab40795@cs.otago.ac.nz> Message-ID: On Mon, May 9, 2016 at 2:01 AM, Richard A. O'Keefe wrote: > I thought maps were *not* intrinsically ordered. > They aren't. You don't need a defined order in order to iterate using first and next, see for example how ets:first/ets:next work on a set. -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.james.morgan@REDACTED Mon May 9 12:48:14 2016 From: peter.james.morgan@REDACTED (Peter Morgan) Date: Mon, 9 May 2016 11:48:14 +0100 Subject: [erlang-questions] [ANN] Tansu - An Erlang Raft based KV and lock store Message-ID: Hello - Tansu is a distributed key/value store designed to maintain configuration and other data that must be highly available. It uses the Raft consensus algorithm for leadership and distribution of state amongst its members. Node discovery is via mDNS and will automatically form a mesh of nodes sharing the same environment. It provides a simple REST based KV and lock API. Tansu is available as a Docker image - starting a 5 node cluster is as simple as: for i in {1..5}; do docker run \ --name tansu-$(printf %03d $i) \ -d shortishly/tansu; done Setting a value with a TTL of 10 seconds on a random node: curl \ -H "Content-Type: application/json" \ -H "ttl: 10" \ -i \ http://$(docker inspect \ --format={{.NetworkSettings.IPAddress}} \ tansu-$(printf %03d $[1 + $[RANDOM % 5]]))/api/keys/hello \ --data-binary '{"ephemeral": true}' Getting a value from a random node: curl \ -i \ -s \ http://$(docker inspect \ --format={{.NetworkSettings.IPAddress}} \ tansu-$(printf %03d $[1 + $[RANDOM % 5]]))/api/keys/hello Locks, event streams and more details are available at https://github.com/shortishly/tansu. Thanks, Peter. From ok@REDACTED Mon May 9 14:02:31 2016 From: ok@REDACTED (ok@REDACTED) Date: Tue, 10 May 2016 00:02:31 +1200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> <3fe10814-0754-7d37-eeb9-03f6fab40795@cs.otago.ac.nz> Message-ID: > On Mon, May 9, 2016 at 2:01 AM, Richard A. O'Keefe > wrote: > >> I thought maps were *not* intrinsically ordered. >> > > They aren't. > > You don't need a defined order in order to iterate using first and next, > see for example how ets:first/ets:next work on a set. You *do* need a defined order for iteration to *make sense*. If the value of maps:first_key(Map) and maps:next_key(Map, Key) is not *determined* by the *value* of Map (without reference to the history of how the Map was built or any other inscrutable properties such as where it happens to be stored), then these things just aren't functions. Since we *do* have an ordering on terms, it *would* be possible to define 'first' as 'least in standard order' and 'next' as 'next smallest in standard order'. Without some clearly defined history-independent order being set up for these functions, you can end up with two *equal* maps being traversed in different orders. I've been there in the Prolog world and still have the scars. From ok@REDACTED Mon May 9 14:02:38 2016 From: ok@REDACTED (ok@REDACTED) Date: Tue, 10 May 2016 00:02:38 +1200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> <3fe10814-0754-7d37-eeb9-03f6fab40795@cs.otago.ac.nz> Message-ID: <035fbeffbe4c990329a51b94f1dab88d.squirrel@chasm.otago.ac.nz> > On Mon, May 9, 2016 at 2:01 AM, Richard A. O'Keefe > wrote: > >> I thought maps were *not* intrinsically ordered. >> > > They aren't. > > You don't need a defined order in order to iterate using first and next, > see for example how ets:first/ets:next work on a set. You *do* need a defined order for iteration to *make sense*. If the value of maps:first_key(Map) and maps:next_key(Map, Key) is not *determined* by the *value* of Map (without reference to the history of how the Map was built or any other inscrutable properties such as where it happens to be stored), then these things just aren't functions. Since we *do* have an ordering on terms, it *would* be possible to define 'first' as 'least in standard order' and 'next' as 'next smallest in standard order'. Without some clearly defined history-independent order being set up for these functions, you can end up with two *equal* maps being traversed in different orders. I've been there in the Prolog world and still have the scars. From zxq9@REDACTED Mon May 9 14:14:08 2016 From: zxq9@REDACTED (zxq9) Date: Mon, 09 May 2016 21:14:08 +0900 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <035fbeffbe4c990329a51b94f1dab88d.squirrel@chasm.otago.ac.nz> References: <035fbeffbe4c990329a51b94f1dab88d.squirrel@chasm.otago.ac.nz> Message-ID: <15240420.75O2hBXNvX@burrito> On Tuesday 10 May 2016 00:02:38 ok@REDACTED wrote: > > On Mon, May 9, 2016 at 2:01 AM, Richard A. O'Keefe > > wrote: > > > >> I thought maps were *not* intrinsically ordered. > >> > > > > They aren't. > > > > You don't need a defined order in order to iterate using first and next, > > see for example how ets:first/ets:next work on a set. > > You *do* need a defined order for iteration to *make sense*. > If the value of maps:first_key(Map) and maps:next_key(Map, Key) > is not *determined* by the *value* of Map (without reference to > the history of how the Map was built or any other inscrutable > properties such as where it happens to be stored), then these > things just aren't functions. > > Since we *do* have an ordering on terms, it *would* be possible > to define 'first' as 'least in standard order' and 'next' as > 'next smallest in standard order'. > > Without some clearly defined history-independent order being > set up for these functions, you can end up with two *equal* > maps being traversed in different orders. I thought I was the only one who thought this was odd, but I was shouted down once a while back mentioning it because "I just didn't understand". "next" implies a "previous" and that implies an ordering -- and that ordering seems like it should mean something (or, in other words, be consistent). Or... what am I missing? I'm all for having a maps:next/1 & maps:previous/1, but that really seems to imply an ordering, and it also seems to imply some super explicit explanation of the behavior between calls in different scopes and between changes to the contents of the map (but since each change to the map supposedly returns a *new* map, and each time a map is read in a new scope it should also be considered to be a new map... ?). Dunno... seems more like a desire to have a maps:map/2, maps:foldl/2, maps:kfoldl/2, maps:vfoldl/2, maps:kforeach/2, maps:vforeach/2, etc. should be handled by an explicit function -- instead of succumbling to (what to me) feels like a temptation to make maps have a Python-like iterator. Can someone explain the use case for next/1 and what is different about it than having a selection of more specific and explicit functions that cover the cases where we want to iterate over the map (to include shortcutting/backtracking/reporting, as in a "pipe" type list function)? -Craig From candres.bolanos@REDACTED Mon May 9 14:34:45 2016 From: candres.bolanos@REDACTED (=?UTF-8?Q?CARLOS_ANDRES_BOLA=C3=91OS_REALPE?=) Date: Mon, 9 May 2016 07:34:45 -0500 Subject: [erlang-questions] =?utf-8?b?IFtBTk5dIHNoYXJkcyDigJMgdHJhbnNwYXJl?= =?utf-8?q?nt_sharding_support_for_ETS_tables?= Message-ID: Shards is a simple Erlang/Elixir library compatible with the ETS API that implements Sharding/Partitioning support on top of ETS, totally transparent and out-of-box. To read and know more about it, check these links out: - *Blog post*: http://cabol.github.io/posts/2016/04/14/sharding-support-for-ets.html - *GiHub*: https://github.com/cabol/shards Best, Carlos. -------------- next part -------------- An HTML attachment was scrubbed... URL: From grahamrhay@REDACTED Mon May 9 14:36:29 2016 From: grahamrhay@REDACTED (Graham Hay) Date: Mon, 9 May 2016 13:36:29 +0100 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <15240420.75O2hBXNvX@burrito> References: <035fbeffbe4c990329a51b94f1dab88d.squirrel@chasm.otago.ac.nz> <15240420.75O2hBXNvX@burrito> Message-ID: > > "next" implies a "previous" and that implies an ordering -- and that > ordering seems like it should mean something (or, in other words, be > consistent). Or... what am I missing? I think the point is that you shouldn't rely on the order, as it as an implementation detail and subject to change. JS object properties are similar: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/for...in -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Mon May 9 14:54:12 2016 From: zxq9@REDACTED (zxq9) Date: Mon, 09 May 2016 21:54:12 +0900 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <15240420.75O2hBXNvX@burrito> Message-ID: <2267355.i1Po6rEGgK@burrito> On Monday 09 May 2016 13:36:29 Graham Hay wrote: > > > > "next" implies a "previous" and that implies an ordering -- and that > > ordering seems like it should mean something (or, in other words, be > > consistent). Or... what am I missing? > > > I think the point is that you shouldn't rely on the order, as it as an > implementation detail and subject to change. That's my whole point. Why the desire for a next/1 and previous/1 instead of a list-style operation over the map as a whole, since outside of an abstract sense of "doing something with each element" there is nothing interesting that can possibly be gained from introducing an implicit concept of order? > JS object properties are similar: > > https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/for...in In other news, it is hard to take JS seriously as a source of inspiration for useful language constructs. It is a great place to take inspiration from people beating the odds to get actual work done in a totally insane world, though. From garazdawi@REDACTED Mon May 9 15:05:04 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Mon, 9 May 2016 15:05:04 +0200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> <3fe10814-0754-7d37-eeb9-03f6fab40795@cs.otago.ac.nz> Message-ID: On Mon, May 9, 2016 at 2:02 PM, wrote: > > On Mon, May 9, 2016 at 2:01 AM, Richard A. O'Keefe > > wrote: > > > >> I thought maps were *not* intrinsically ordered. > >> > > > > They aren't. > > > > You don't need a defined order in order to iterate using first and next, > > see for example how ets:first/ets:next work on a set. > > You *do* need a defined order for iteration to *make sense*. > I don't really like dealing in absolutes. I can see scenarios where it would be useful to have that and scenarios where I'd rather get the lookup speed that not having it gives. One idea could be to have both so that the user could pick and choose, i.e. ofirst/onext and ufirst/unext. However the iterator without the ordering guarantee will always be as fast or faster then the iterator with the ordering guarantee. If I've done my thinking correctly, the complexity of the iterator would be something like O(n/2) for ordered and O(log(n)) for unordered iterators. > If the value of maps:first_key(Map) and maps:next_key(Map, Key) > is not *determined* by the *value* of Map (without reference to > the history of how the Map was built or any other inscrutable > properties such as where it happens to be stored), then these > things just aren't functions. > Seems kind of harsh to disqualify them from the entire set of functions, just because they aren't pure and/or the construction of the term doesn't influence it's final state. What we are leaning towards right now is to define that two maps that have the same keys also have the same iteration order, no matter how the maps have been created. So map "A = #{ a => a }, B = A#{ b => b }." and "C = #{ b => a }, D = A#{ a => b }." would be guaranteed to iterate in the same order. So "{K,V} = maps:first(B) = maps:first(D).", but we don't guarantee which key K was bound to. So I guess in part we agree with you, it is just a matter of deciding how much we have to/should guarantee about the order of the iterators. Btw, feels like we have had exactly this discussion when looking at maps a couple of years ago, but I can't find any references to it. To order or not to order, that is the question. -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Mon May 9 15:22:17 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Mon, 9 May 2016 15:22:17 +0200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <2267355.i1Po6rEGgK@burrito> References: <15240420.75O2hBXNvX@burrito> <2267355.i1Po6rEGgK@burrito> Message-ID: On Mon, May 9, 2016 at 2:54 PM, zxq9 wrote: > > That's my whole point. Why the desire for a next/1 and previous/1 instead > of > a list-style operation over the map as a whole, since outside of an > abstract > sense of "doing something with each element" there is nothing interesting > that can possibly be gained from introducing an implicit concept of order? > For one thing Mr. Virding needs them to implement his lua erlang language thing :) We also could make great use of them in the standard libraries to build efficient variants of maps:fold and friends. I'm sure there are usecases that we have not seen yet that will come up. I've learned over the years that people never use the API:s you create the way to expect them to be used and always come up with the strangest and cleverest ways of doing things. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Mon May 9 15:28:38 2016 From: zxq9@REDACTED (zxq9) Date: Mon, 09 May 2016 22:28:38 +0900 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <2267355.i1Po6rEGgK@burrito> Message-ID: <1861666.43u3BY5VlF@burrito> On Monday 09 May 2016 15:22:17 Lukas Larsson wrote: > On Mon, May 9, 2016 at 2:54 PM, zxq9 wrote: > > > > > That's my whole point. Why the desire for a next/1 and previous/1 > > instead of a list-style operation over the map as a whole, since > > outside of an abstract sense of "doing something with each element" > > there is nothing interesting that can possibly be gained from > > introducing an implicit concept of order? > > > > For one thing Mr. Virding needs them to implement his lua erlang language > thing :) We also could make great use of them in the standard libraries to > build efficient variants of maps:fold and friends. > > I'm sure there are usecases that we have not seen yet that will come up. > I've learned over the years that people never use the API:s you create the > way to expect them to be used and always come up with the strangest and > cleverest ways of doing things. Sure, internally I imagine there are a million super slick ways to use this, and I lack the imagination to see what they may be. But is that the level of abstraction that maps are meant to be as exposed to me as a user of a hash map implementation? I'm not trying to naysay, just grasp the reasoning -- from the point of view of API and language design. This seems extremely inconsistent with what I expect of a "map" implementation. Or perhaps my expectations are flawed and "map" means something different now (just like other terms like "object", "REST" and "API"...). -Craig From the.artiegold@REDACTED Mon May 9 15:43:23 2016 From: the.artiegold@REDACTED (Artie Gold) Date: Mon, 9 May 2016 08:43:23 -0500 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> <3fe10814-0754-7d37-eeb9-03f6fab40795@cs.otago.ac.nz> Message-ID: <75F2EEE8-B8F5-4093-BFE1-FE07E40904E6@gmail.com> Old na?vist here... Why should a defined order be necessary in an immutable context? Thanks (and thanks), --ag Sent from my iPhone On May 9, 2016, at 7:02 AM, wrote: >> On Mon, May 9, 2016 at 2:01 AM, Richard A. O'Keefe >> wrote: >> >>> I thought maps were *not* intrinsically ordered. >> >> They aren't. >> >> You don't need a defined order in order to iterate using first and next, >> see for example how ets:first/ets:next work on a set. > > You *do* need a defined order for iteration to *make sense*. > If the value of maps:first_key(Map) and maps:next_key(Map, Key) > is not *determined* by the *value* of Map (without reference to > the history of how the Map was built or any other inscrutable > properties such as where it happens to be stored), then these > things just aren't functions. > > Since we *do* have an ordering on terms, it *would* be possible > to define 'first' as 'least in standard order' and 'next' as > 'next smallest in standard order'. > > Without some clearly defined history-independent order being > set up for these functions, you can end up with two *equal* > maps being traversed in different orders. > > I've been there in the Prolog world and still have the scars. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jesper.louis.andersen@REDACTED Mon May 9 15:59:05 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 9 May 2016 15:59:05 +0200 Subject: [erlang-questions] CNODE performance question In-Reply-To: References: Message-ID: On Sun, May 8, 2016 at 2:11 AM, Matthew Evans wrote: > The NIF code was getting a bit flakey so we made the call to implement it > as a CNODE instead to avoid the NIF taking down the VM if it crashed. I recently wrote a wrapper for OCaml so I could start a port program and ferry operations to that. If you pipelined requests (not waiting for an answer before sending the next request), the overhead amortized to roughly 1us. I could carry out about one million reqs/s utilizing a scheduler core in Erlang and around 0.68 core in OCaml. At this point, I decided it was fast enough for most of the work I'd normally send to a foreign process for external work. If a request completed over there in, say, 3ms, then the 1us overhead is small enough that I wouldn't worry. Requests waiting synchronously for an answer were about 30.000 reqs/s. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Mon May 9 16:54:32 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 9 May 2016 10:54:32 -0400 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <1861666.43u3BY5VlF@burrito> References: <2267355.i1Po6rEGgK@burrito> <1861666.43u3BY5VlF@burrito> Message-ID: <20160509145429.GA4362@fhebert-ltm2.internal.salesforce.com> On 05/09, zxq9 wrote: >On Monday 09 May 2016 15:22:17 Lukas Larsson wrote: >> On Mon, May 9, 2016 at 2:54 PM, zxq9 wrote: >> >> > >> > That's my whole point. Why the desire for a next/1 and previous/1 >> > instead of a list-style operation over the map as a whole, since >> > outside of an abstract sense of "doing something with each element" >> > there is nothing interesting that can possibly be gained from >> > introducing an implicit concept of order? >> > > >Sure, internally I imagine there are a million super slick ways to use this, >and I lack the imagination to see what they may be. > - partial iteration to look for an element - partial iteration to only modify a subrange of the whole map; for example, re-hashing a window of N components. If your map has 10 million items and you want to re-hash 25 of them, then going over the whole map every time is going to eat you up on the computation (this one is more useful with a total stable order defined) - implementing your own map/fold/filter combination as a single pass without needing to iterate and convert the whole map at once - ability to do lookahead/look-behind in an iteration to arbitrary levels without implementing your own ad-hoc zipper or buffering all of the content you have seen Those are a few I have in mind can be doable that way -- I've mostly seen them at work or wanted them for ETS tables, but I'm sure someone could twist a map into having the same requirements. From rvirding@REDACTED Mon May 9 18:29:28 2016 From: rvirding@REDACTED (Robert Virding) Date: Mon, 9 May 2016 18:29:28 +0200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <20160509145429.GA4362@fhebert-ltm2.internal.salesforce.com> References: <2267355.i1Po6rEGgK@burrito> <1861666.43u3BY5VlF@burrito> <20160509145429.GA4362@fhebert-ltm2.internal.salesforce.com> Message-ID: OK, I am the one who asked for first/next so this is my use case. As Lukas mentioned I need this to implement Lua tables in Erlang using maps. I need to be able to iterate over a map one key/value pair at a time. For me the order is completely irrelevant as along as if I do a sequence of first-next-next-... using the previous key in the next next I am guaranteed to *uniquely* get *all* the keys in the map. If I modify the map and try to continue from where I was then all bets are of and there are no guarantees any more that I will get all keys or that they will be unique. Lua has exactly this interface to its tables so I need to be able to do it as well. Not having it is not an option. So while having maps or folds over a map is great there is no way these can be used to implement what I need efficiently. That's it. Now I use a private 2-3 trees implementation of dict with the added functions and it works well. But using maps would seem logical and more efficient. That's about it. Robert On 9 May 2016 at 16:54, Fred Hebert wrote: > On 05/09, zxq9 wrote: > >> On Monday 09 May 2016 15:22:17 Lukas Larsson wrote: >> >>> On Mon, May 9, 2016 at 2:54 PM, zxq9 wrote: >>> >>> > >>> > That's my whole point. Why the desire for a next/1 and previous/1 >>> > instead of a list-style operation over the map as a whole, since >>> > outside of an abstract sense of "doing something with each element" >>> > there is nothing interesting that can possibly be gained from >>> > introducing an implicit concept of order? >>> > >>> >> >> Sure, internally I imagine there are a million super slick ways to use >> this, >> and I lack the imagination to see what they may be. >> >> > - partial iteration to look for an element > - partial iteration to only modify a subrange of the whole map; for > example, re-hashing a window of N components. If your map has 10 million > items and you want to re-hash 25 of them, then going over the whole map > every time is going to eat you up on the computation (this one is more > useful with a total stable order defined) > - implementing your own map/fold/filter combination as a single pass > without needing to iterate and convert the whole map at once > - ability to do lookahead/look-behind in an iteration to arbitrary levels > without implementing your own ad-hoc zipper or buffering all of the > content you have seen > > Those are a few I have in mind can be doable that way -- I've mostly seen > them at work or wanted them for ETS tables, but I'm sure someone could > twist a map into having the same requirements. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raould@REDACTED Mon May 9 18:31:42 2016 From: raould@REDACTED (Raoul Duke) Date: Mon, 9 May 2016 09:31:42 -0700 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <2267355.i1Po6rEGgK@burrito> <1861666.43u3BY5VlF@burrito> <20160509145429.GA4362@fhebert-ltm2.internal.salesforce.com> Message-ID: But, but... Lua tables are eeeeeeevil!!!!! -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn-egil.xb.dahlberg@REDACTED Mon May 9 18:57:29 2016 From: bjorn-egil.xb.dahlberg@REDACTED (=?UTF-8?Q?Bj=c3=b6rn-Egil_Dahlberg_XB?=) Date: Mon, 9 May 2016 18:57:29 +0200 Subject: [erlang-questions] Test errors with 18.3 on opensuse 13.1 In-Reply-To: References: Message-ID: <5730C179.8060205@ericsson.com> On 2016-05-08 13:02, Roger Price wrote: > On Sun, 8 May 2016, Roger Price wrote: > >> On Sun, 8 May 2016, Jesper Louis Andersen wrote: >> >>> On Sun, May 8, 2016 at 10:24 AM, Roger Price >>> wrote: >>> Are these failures due to the missing jinterface and odbc >>> applications, or something else? Nope, it's not related. >>> >>> As a first pass, they seem to be related to timing. If the machine >>> is a virtual machine, it could perhaps be explained by that. >> >> The machine is physical, a Dell Precision 690 with 4 cpus and 32 >> Gbyte memory. Linux kernel version 3.12, gcc version 4.8.1. >> >> Maybe it's Dell 690 related. I'll look for another machine to check. > > The same test errors occur on a Dell Precision T7500 with 12 cpus and > 48 Gbyte memory running opensuse 13.1 Evergreen. > > smoke_test_SUITE boot_combo < > 35.838s FAILED > {smoke_test_SUITE,chk_boot,153} > {badmatch,{error,timeout}} - failed to start slave node in test. > time_SUITE local_to_univ_utc < > 60.009s FAILED > {time_SUITE,local_to_univ_utc,115} > suite_failed - failed to start slave node in test. > time_SUITE time_warp_modes < > 38.008s FAILED > {time_SUITE,check_time_warp_mode,514} > {badmatch,{error,timeout}} - failed to start slave node in test. > heart_SUITE reboot < > 35.728s FAILED {heart_SUITE,start_check,106} > {badmatch,{error,timeout}} - failed to start slave node in test. All of the failures has the same cause, namely it cannot start a slave node. This is probably due to some faulty network configuration. Try starting two erlang nodes using -sname a, -sname b and then pinging them, i.e: (a@)1> net_adm:ping(b@). pong // Bj?rn-Egil From lheadley@REDACTED Mon May 9 19:23:01 2016 From: lheadley@REDACTED (Lyn Headley) Date: Mon, 9 May 2016 10:23:01 -0700 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> Message-ID: Thanks for the feedback all. There are many places this could go, but I'll reply with one in particular to try and keep focus. > > One problem with this is that it is *syntactic*, not *semantic*. > The editor tells me *what* you are looking at, but not *why* > you are looking at it, not even whether it is what you *intended* > to look at or something you arrived at by accident. > Two roles: elder and youth. As part of using the editor, youth needs to occasionally make some semantic gestures. She might also need to edit/annotate her own history. But this could get cumbersome so we should limit it. At all times, recording is either on or off, and youth can switch it on or off. Whenever it is on, there is an active question (which youth may give a description but which doesn't absolutely need one). Recorded events append an event record to the active question. Youth can delete, but not rearrange or insert events before posing the active question (uploading it to the server where elders are listening <-- note not "elder is"). I think these points address your comment above. > And since we are (probably) on opposite sides of the planet, you are > most unlikely to be able to give me feedback at the time I need it. Youth is happy to receive feedback tomorrow because there is plenty to think about and do in the meantime. > > How does the editor know that Lyn expressed puzzlement? > Is it because the comment is in M?ori, or because it looks as > though there might be a division by zero, or because this is > Python3 and Lyn is used to Python2? The editor has a keybinding for 'express puzzlement here.' Elder may discern the source of the puzzlement simply by observing its line of code in context (which elder's editor provides), or elder may need more information.Note that context includes the entire active question history, which helps elder reply with "not relevant to your question." Express puzzlement is a bit lazy on youth's part, so before submitting her question youth may annotate this event with language. > *What* is easy to write? > I think we need a lot more detail before it is possible to make > that judgement. > Agreed. > It does have a downside. I started this thread talking about > 1 person leaving signposts (rhetorical structure annotations) > for N > 1 readers. The editing log idea is 1:1; if Helen wrote > the code and Lyn recorded herself viewing it and Helen > replies to Lyn, that helps Lyn, but it doesn't help Sue, unless > she knows that Helen's reply to Lyn's session exists. Youth 1 posts her question to a public location where Youth 2 and Google can trawl for existing questions and answers. Of course this raises Grzegorz Junka's question about how to maintain this archive, but I don't think maintenance is critical. The resource can age, even obsolesce, and still provide great value while it is relevant. I'll close with a grander claim. Programmers are less likely to add sufficient annotations and documentation to their code than they are to answer specific questions about the code. And this Q&A process can become a kind of re-usable documentation over time. From rvirding@REDACTED Mon May 9 20:29:17 2016 From: rvirding@REDACTED (Robert Virding) Date: Mon, 9 May 2016 20:29:17 +0200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <2267355.i1Po6rEGgK@burrito> <1861666.43u3BY5VlF@burrito> <20160509145429.GA4362@fhebert-ltm2.internal.salesforce.com> Message-ID: Undoubtedly, but that's the way that Lua does so I don't have much of a choice. :-) And that is why I need it and why I only need a rather simple version. If I don't get it I will just keep on nagging about it. "And they couldn't give me this really simple feature I needed." Robert On 9 May 2016 18:31, "Raoul Duke" wrote: > But, but... Lua tables are eeeeeeevil!!!!! > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From h_sowwan@REDACTED Mon May 9 18:42:11 2016 From: h_sowwan@REDACTED (Hassan Sowwan) Date: Mon, 9 May 2016 16:42:11 +0000 Subject: [erlang-questions] Erlang for Payment Systems In-Reply-To: References: , Message-ID: Thanks for your comments, the point is clear now. Just one thing to add, for online transaction processing (OLTP), what is the best approach to deal with RDBMS? For each transaction received, the system has to access the database and perform some sort of validations and log the request, and finally the request will be updated when receiving the response back from external systems. How this can be done efficiently especially that erlang uses ODBC drivers. > Date: Sat, 30 Apr 2016 10:43:49 +0200 > Subject: Re: [erlang-questions] Erlang for Payment Systems > From: erlang@REDACTED > To: h_sowwan@REDACTED > CC: erlang-questions@REDACTED > > On Sat, Apr 30, 2016 at 12:09 AM, Hassan Sowwan wrote: > > Hello, > > > > I am trying to implement payment messaging middleware and would like to > > explore the option of using Erlang/OTP. > > > > The application will be used in banking industry to interface with EFT > > payment switch/networks and core banking system to process card > > transactions. > > > > It will be responsible to perform following tasks: > > > > > > Communicate with external interfaces using ISO 8583 messaging format ( thru > > TCP/IP) > > Receive huge amount of data over the socket ( HEX, BINARY, EBCIDIC), which > > represents financial transactions. > > Parse/decode the received data. > > Perform some checking in database for validation > > Interface with host security module to validate customer PIN and other > > security checks. > > Sends the request to core banking system via XML or web services call > > Respond back to external interfaces by formulating the response message in > > ISO 8583 format > > > > > > Obviously, such applications have to be concurrent and fast enough to > > process transactions within few seconds. > > > > Now my question here, is Erlang a good choice for implementing this type of > > applications ? > > Can Erlang handle string processing efficiently without impacting the system > > performance? > > As stated before, there will be a lot of string manipulation to decode data > > received over the network, so I am not sure whether erlang fits perfectly or > > not. > > > > I think you've missed the most important criteria for choosing a technique. > > Fault tolerance - If I was building a financial system #1 of my requirements > would be fault tolerance - people will be very cross if money just > mysteriously vanishes from the system. > > #2 would be correctness > > So my questions would be how can we make a system fault tolerant and > how can we ensure correctness. > > Making things fast is *easy* - making things fault-tolerant is > extremely difficult. > > String manipulation in Erlang is blindingly fast (it's admittedly a > lot slower than C) but your program should not be manipulation strings > - but > trees representing financial objects. > > /Joe > > > > > > > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue May 10 00:31:42 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 10 May 2016 10:31:42 +1200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <035fbeffbe4c990329a51b94f1dab88d.squirrel@chasm.otago.ac.nz> <15240420.75O2hBXNvX@burrito> Message-ID: On 10/05/16 12:36 AM, Graham Hay wrote: > > "next" implies a "previous" and that implies an ordering -- and that > ordering seems like it should mean something (or, in other words, be > consistent). Or... what am I missing? > > > I think the point is that you shouldn't rely on the order, as it as an > implementation detail and subject to change No, it's much worse than that. It's not that the order may change from one release to the next. THAT would not be such a problem. The problem is that - two EQUAL tables may have DIFFERENT orders - my possibly incorrect understanding is that a map may change its representation, so that a SINGLE table may have different orders at different times - for efficiency, a map representation should be allowed to change in response to pure queries, but that would mean that the order could change DURING A TRAVERSAL. Background: for years I've been working on a Smalltalk->C compiler and its supporting library. The library naturally includes hash tables. And yes, - two equal Dictionaries generally DO have different orders - adding associations to a Dictionary and then removing them WILL in general result in the order changing even though it's the same object with the same contents - I wanted to use the move-to-front heuristic to improve lookup times, but it DID break traversal (associations could be lost or visited twice) so I had to give up on that, because while Smalltalk forbids *changing* a collection while iterating over it, it doesn't forbid *examining* it. When we speak of "THE order", our mouths are deceiving us. There is no "THE order", and that is the heart of the problem. Smalltalk can get away with it, because Smalltalk is an Object-Oriented language, not a functional one, and the compiler is entitled to make very few assumptions, not even that an object won't change class. But Erlang is (at least within a process) a functional language, and something like maps:first(Map) is supposed to be a *function* of the *value* of its argument. If Map1 == Map2, then maps:first(Map1) must == maps:first(Map2) otherwise we are adrift without a compass. One of the goals underlying the frames proposal was that none of these problems should exist. By the way, it's easy enough to impose an order. What you do is to exploit the fact that a representation may change without the value appearing to change. If first/next (or last/previous) are used, at that point you switch from some sort of hashed structure to some sort of tree, such as a splay tree or a red-black tree. Iterating over a splay tree (once formed) takes linear time, so that would probably be the way to go. I couldn't do that in Smalltalk, because there is no total order on Smalltalk objects that could be used, but there is one on Erlang terms. > > JS object properties are similar: Yes, but JavaScript is frankly insane and while it uses functions a lot, it's basically an imperative language, much closer to Smalltalk than to Erlang, although not in a terribly useful way. JavaScript is the ultimate test of Agile programming: it's what you get from doing a coding sprint instead of design. From chandrashekhar.mullaparthi@REDACTED Tue May 10 00:33:07 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Mon, 9 May 2016 23:33:07 +0100 Subject: [erlang-questions] [ANN] Tansu - An Erlang Raft based KV and lock store In-Reply-To: References: Message-ID: Hi Peter, That's great, I've always wanted an Erlang implementation of etcd :-) That said, what are your reasons for developing this when we already have etcd? cheers, Chandru On 9 May 2016 at 11:48, Peter Morgan wrote: > Hello - > > Tansu is a distributed key/value store designed to maintain configuration > and other data that must be highly available. It uses the Raft consensus > algorithm for leadership and distribution of state amongst its members. > Node discovery is via mDNS and will automatically form a mesh of nodes > sharing the same environment. It provides a simple REST based KV and lock > API. > > Tansu is available as a Docker image - starting a 5 node cluster is as > simple as: > > for i in {1..5}; do > docker run \ > --name tansu-$(printf %03d $i) \ > -d shortishly/tansu; > done > > Setting a value with a TTL of 10 seconds on a random node: > > curl \ > -H "Content-Type: application/json" \ > -H "ttl: 10" \ > -i \ > http://$(docker inspect \ > --format={{.NetworkSettings.IPAddress}} \ > tansu-$(printf %03d $[1 + $[RANDOM % 5]]))/api/keys/hello \ > --data-binary '{"ephemeral": true}' > > Getting a value from a random node: > > curl \ > -i \ > -s \ > http://$(docker inspect \ > --format={{.NetworkSettings.IPAddress}} \ > tansu-$(printf %03d $[1 + $[RANDOM % 5]]))/api/keys/hello > > Locks, event streams and more details are available at > https://github.com/shortishly/tansu. > > Thanks, > Peter. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From list1@REDACTED Tue May 10 01:59:21 2016 From: list1@REDACTED (Grzegorz Junka) Date: Mon, 9 May 2016 23:59:21 +0000 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <2267355.i1Po6rEGgK@burrito> <1861666.43u3BY5VlF@burrito> <20160509145429.GA4362@fhebert-ltm2.internal.salesforce.com> Message-ID: <648b60ae-9084-4c09-2514-86962496f6f6@gjunka.com> I only wanted to mention that maps is not the only key-value data structure available in Erlang and may not the best one to implement such an iterator. I once did a benchmark of available key-value data stores measuring not only the speed, but also the amount of consumed memory: https://gist.github.com/amiramix/d43c9a73a6fe6d651d7f Maps are quite performant but process dictionary is still quicker and maps are the worst when it comes to consumed memory, taking twice as much as dict or process dictionary and over 5 times as much memory as ets. BTW if you have a look and find any issues please let me know and I will be happy to correct and rerun. The test simply initializes the data structure with InitSize amount of Key-Value elements, and then starts measuring the time and memory needed to write/read Accesses amount of Key-Value/Key elements with randomly generated Keys. Greg On 09/05/2016 16:29, Robert Virding wrote: > OK, I am the one who asked for first/next so this is my use case. > > As Lukas mentioned I need this to implement Lua tables in Erlang using > maps. > > I need to be able to iterate over a map one key/value pair at a time. > For me the order is completely irrelevant as along as if I do a > sequence of first-next-next-... using the previous key in the next > next I am guaranteed to *uniquely* get *all* the keys in the map. If I > modify the map and try to continue from where I was then all bets are > of and there are no guarantees any more that I will get all keys or > that they will be unique. > > Lua has exactly this interface to its tables so I need to be able to > do it as well. Not having it is not an option. So while having maps or > folds over a map is great there is no way these can be used to > implement what I need efficiently. That's it. > > Now I use a private 2-3 trees implementation of dict with the added > functions and it works well. But using maps would seem logical and > more efficient. > > That's about it. > > Robert > > > On 9 May 2016 at 16:54, Fred Hebert > wrote: > > On 05/09, zxq9 wrote: > > On Monday 09 May 2016 15:22:17 Lukas Larsson wrote: > > On Mon, May 9, 2016 at 2:54 PM, zxq9 > wrote: > > > > > That's my whole point. Why the desire for a next/1 and > previous/1 > > instead of a list-style operation over the map as a > whole, since > > outside of an abstract sense of "doing something with > each element" > > there is nothing interesting that can possibly be gained > from > > introducing an implicit concept of order? > > > > > Sure, internally I imagine there are a million super slick > ways to use this, > and I lack the imagination to see what they may be. > > > - partial iteration to look for an element > - partial iteration to only modify a subrange of the whole map; > for example, re-hashing a window of N components. If your map has > 10 million items and you want to re-hash 25 of them, then going > over the whole map every time is going to eat you up on the > computation (this one is more useful with a total stable order > defined) > - implementing your own map/fold/filter combination as a single pass > without needing to iterate and convert the whole map at once > - ability to do lookahead/look-behind in an iteration to > arbitrary levels without implementing your own ad-hoc zipper or > buffering all of the content you have seen > > Those are a few I have in mind can be doable that way -- I've > mostly seen them at work or wanted them for ETS tables, but I'm > sure someone could twist a map into having the same requirements. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue May 10 02:03:59 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 10 May 2016 12:03:59 +1200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> <3fe10814-0754-7d37-eeb9-03f6fab40795@cs.otago.ac.nz> Message-ID: On 10/05/16 1:05 AM, Lukas Larsson wrote: > > I don't really like dealing in absolutes. That is an interesting fact about you. Here's one about me: I don't like dealing with fundamentally broken libraries. > I can see scenarios where it would be useful to have that and > scenarios where I'd rather get the lookup speed that not having it gives. False dichotomy, because - a data structure can support DUAL access methods - a data structure can CHANGE its representation to optimise current usage. Both of these were known in the 1960s. > One idea could be to have both so that the user could pick and choose, > i.e. ofirst/onext and ufirst/unext. However the iterator without the > ordering guarantee will always be as fast or faster then the iterator > with the ordering guarantee. If I've done my thinking correctly, the > complexity of the iterator would be something like O(n/2) for ordered > and O(log(n)) for unordered iterators. I do not understand your reasoning here. By using a threaded tree, iteration over a key-ordered dictionary can be done in O(1) worst case time per element. Again, known since the 1960s, and explained in TAOP vol 1. If you're not willing to pay the space price of threading, iteration over an adaptive key-ordered dictionary can be done in O(1) amortised time per element, and that's been known since 1985. > > If the value of maps:first_key(Map) and maps:next_key(Map, Key) > is not *determined* by the *value* of Map (without reference to > the history of how the Map was built or any other inscrutable > properties such as where it happens to be stored), then these > things just aren't functions. > > > Seems kind of harsh to disqualify them from the entire set of > functions, just because they aren't pure and/or the construction of > the term doesn't influence it's final state. I'm perfectly happy to call them procedures, methods, operations, accessors, or what have you. But having an output that is determined by the value of the input is what a function IS. And the important thing is semantics, not names. If the behaviour is affected by things the programmer cannot even detect, let alone control, then bad things happen. OK, so Erlang is a little bit pregnant. There are ETS and DETS, and there are RPC calls and other things. But you don't want loss of control over basic data structures. You don't want to traverse the same data structure twice and get different answers even though the value is the same. Here's the kind of thing I'm talking about. Here's some Smalltalk code. "make keys a flexible array holding the keys of a dictionary" keys := OrderedCollection new. aDictionary keysAndValuesDo: [:key :value | keys addLast: key]. "assert that the atom #boojum is not a key in that dictionary" [aDictionary includesKey: #boojum] deny. "add an association for #boojum and remove it again" aDictionary at: #boojum put: #snark. aDictionary removeKey: #boojum. "the *logical* state of aDictionary is back to what it was." "make values a flexible array holding the values of the dictionary" values := OrderedCollection new. aDictionary keysAndValuesDo: [:key :value | values addLast: value]. "make a new dictionary by mapping keys to corresponding values" newDictionary := Dictionary new: aDictionary size. keys with: values do: [:key :value | newDictionary at: key put: value]. "check that the new dictionary is equal to the old" [newDictionary = aDictionary] assert. This can FAIL. It's worse than that. Suppose we DON'T add and remove a #boojum -> #snark association. It's not clear that the second call to #keysAndValuesDo: has to use the same order as the first. The only requirements I can glean from the ANSI Smalltalk standard are that - if issued at the same time, any iteration method WOULD have used the same order as any other - every key is visited exactly once. An adaptive data structure might well reorganise itself between the two iterations. Now in Smalltalk, since it's an Object Oriented Procedural language, we *expect* that kind of nonsense and take care. If it's important that two "unordered" loops use the same order, we know we have to fuse the loops. Did I mention that Smalltalk *isn't* crazy enough to have first/next operations for unordered collections? Smalltalk isn't crazy enough to have first/next operations for unordered collections. Come to think of it, I have been programming in Smalltalk for nearly 20 years, and I have never *wanted* first/next for unordered collections. (And I've thought of *oodles* of operations I did want: the Dictionary class has 760 public methods, not all of them standard.) By the way, let's take one little gem. Suppose we have a dictionary mapping some key or other to floating point numbers, and we want the sum. Since floating point addition is not associative, "aDictionary sum" is not well defined. > What we are leaning towards right now is to define that two maps that > have the same keys also have the same iteration order, no matter how > the maps have been created. And that's precisely what I was arguing for. > So map "A = #{ a => a }, B = A#{ b => b }." and "C = #{ b => a }, D = > A#{ a => b }." would be guaranteed to iterate in the same order. So > "{K,V} = maps:first(B) = maps:first(D).", but we don't guarantee which > key K was bound to. So I guess in part we agree with you, it is just a > matter of deciding how much we have to/should guarantee about the > order of the iterators. The problem is that hashed data structures do not give you this guarantee. It is not clear to me that a *stable* order is any easier to provide than a *published* order. For what it's worth, something analogous to this issue came up in the ISO Prolog standard. One of the major tools in Edinburgh Prolog was a total order on terms, including variables until they were bound. (This required a garbage collector that preserved genetic order, which is not hard to ensure if you design one from scratch to do that.) People who implemented Prolog on top of something else, like Lisp or Pop-11, wanted to live with the garbage collector they had, so we ended up with the absurd situation that the standard has a sort/2 predicate that uses an ordering that is stable *for the duration of the sort only*, so that sorting a list of variables twice can give you two different answers. Not Good! (The Poplog people handled this by defining a stable order for variables as and when they were compared, taking extra space and time.) From ok@REDACTED Tue May 10 02:26:49 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 10 May 2016 12:26:49 +1200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <15240420.75O2hBXNvX@burrito> <2267355.i1Po6rEGgK@burrito> Message-ID: On 10/05/16 1:22 AM, Lukas Larsson wrote: > On Mon, May 9, 2016 at 2:54 PM, zxq9 > wrote:der? > > For one thing Mr. Virding needs them to implement his lua erlang > language thing :)A If we're going to let Lua dictate Erlang semantics, we might we well go all the way and be fully Perl compatible as well. In fact he doesn't *need* them for Lua. They would be very *convenient*, and they would improve *efficiency* of the imitation, but at the price of a little indirection, they aren't *necessary*. > We also could make great use of them in the standard libraries to > build efficient variants of maps:fold and friends. Like I said, my Smalltalk system has 760 public methods in the interface of Dictionary, and first/next are not only not amongst the public methods, they are not amongst the private methods either. Basically, all of the "bulk" methods (maps:fold and friends) are or could be layered on top of aDictionary keysAndValuesDo: [:key :value | ... do something with a key and a value ...]. If I did have analogues of first/next, they would be O(n) worst case time and even on the average would be markedly slower than using the method above. Of course that requires being able to update a mutable state, so the analogue for Erlang would be maps:fold itself. > > I'm sure there are usecases that we have not seen yet that will come > up. I've learned over the years that people never use the API:s you > create the way to expect them to be used and always come up with the > strangest and cleverest ways of doing things. To use a phrase of my father's, "You slobbered a bibful there, bub." (Meaning heartfelt agreement.) Of course one way to deal with this is to provide lots of examples, so that the *easiest* way for people to use your procedures is to follow the pattern you've given them. The basic problem at the moment is that almost any use case for operations with undefined results has to be a use case that yields undefined results itself, and I am not sure why it is important to support use cases with undefined results. Suppose there were a first/next interface for maps relying on an adaptive data structure to provide O(lg n) per element time for a first traversal and O(1) per element time for subsequent traversal. Which is doable. Could this break any Lua program? No, an unspecified order is an unspecified order, and this particular order will do as well as any other. Could the time matter? No, there is nothing in the Lua reference manual making any guarantees whatsoever about next(table[, index]). (I just checked the Lua 5.3 manual to make sure.) Next could be O(n) time per call, and a Lua programmer might be surprised, but no promises would have been broken. next(table[,index]) isn't even the recommended way to iterate over a table, that's for k, v in pairs(t) do end From ok@REDACTED Tue May 10 02:41:40 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 10 May 2016 12:41:40 +1200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <75F2EEE8-B8F5-4093-BFE1-FE07E40904E6@gmail.com> References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> <3fe10814-0754-7d37-eeb9-03f6fab40795@cs.otago.ac.nz> <75F2EEE8-B8F5-4093-BFE1-FE07E40904E6@gmail.com> Message-ID: <1a639927-c250-7bf3-4fc8-fd584c12a617@cs.otago.ac.nz> On 10/05/16 1:43 AM, Artie Gold wrote: > Old na?vist here... > > Why should a defined order be necessary in an immutable context? To produce consistent results. You may be confusing (abstract) value immutability with (concrete) representation immutability. They are not the same thing. Example 1: a simple hash table implemented as an array of linked lists, using move-to-front to search the buckets. Looking things up doesn't change the *value*, but the *representation* adjusts itself in the expectation that something you've looked up will be looked up again soon. Example 2: a splay tree. When you look a key up, the tree adjusts itself to put that key near the root. One consequence of the way this is done is that you can do k := min-key(t) while k is not missing do use k k := next-larger-key(t, k) in O(n) time instead of O(n.log n). Here again, the *value* does not change, but the *representation* adjusts. Example 3: a hybrid data structure which uses hashing for point queries but some sort of search tree for range queries (like min-key and next-larger-key) and switches from one representation to another based on what you've been doing lately. (The Burroughs B6700 MCP used this sort of thing in its file system interface. There were access procedures optimised for sequential reading and access procedures optimised for random access and instead of getting the programmer to guess, it switched at run time.) Alternatively, you may be assuming that two equal abstract values will have the same concrete representation. Sadly, this is far from true. On the gripping hand, you may be assuming that the order in which a container is traversed, while not revealed to people, would of course be defined by the contents of the container, and not by accidents of its history. In that case, we agree that it *should* be, but sadly, in practice it is not. This was always Prolog's worst feature, that it didn't really support abstract data types. Given, for example, an implementation of AVL trees, two AVL trees with identically the same keys and corresponding values might not (no, usually DID not) have the same representation, and so were not equal. That was pretty bad for a programming language where "=" was so fundamental. I can sort of cope with library modules that make my life hard, but I expect better of basic data structures. From mjtruog@REDACTED Tue May 10 02:56:46 2016 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 09 May 2016 17:56:46 -0700 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <1a639927-c250-7bf3-4fc8-fd584c12a617@cs.otago.ac.nz> References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> <3fe10814-0754-7d37-eeb9-03f6fab40795@cs.otago.ac.nz> <75F2EEE8-B8F5-4093-BFE1-FE07E40904E6@gmail.com> <1a639927-c250-7bf3-4fc8-fd584c12a617@cs.otago.ac.nz> Message-ID: <573131CE.7000006@gmail.com> On 05/09/2016 05:41 PM, Richard A. O'Keefe wrote: > > > On 10/05/16 1:43 AM, Artie Gold wrote: >> Old na?vist here... >> >> Why should a defined order be necessary in an immutable context? > To produce consistent results. > If we have maps:first(Map) return an Iterator type with maps:next(Iterator, Map) we can provide in-order results if all the keys are stored in-order in a list (within the Iterator data type). This approach would likely consume way more memory than is necessary with other data structure usage, but paying for the computational time to get all the keys in-order before iterating may be worth it. If the map already is guilty of consuming large amounts of memory, extra memory for an iterator doesn't seem like a large sin. So, it seems like there can still be hope that a map provide a defined order. > > I can sort of cope with library modules that make my life > hard, but I expect better of basic data structures. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From wallentin.dahlberg@REDACTED Tue May 10 03:10:34 2016 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Tue, 10 May 2016 03:10:34 +0200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <648b60ae-9084-4c09-2514-86962496f6f6@gjunka.com> References: <2267355.i1Po6rEGgK@burrito> <1861666.43u3BY5VlF@burrito> <20160509145429.GA4362@fhebert-ltm2.internal.salesforce.com> <648b60ae-9084-4c09-2514-86962496f6f6@gjunka.com> Message-ID: 2016-05-10 1:59 GMT+02:00 Grzegorz Junka : > I only wanted to mention that maps is not the only key-value data > structure available in Erlang and may not the best one to implement such an > iterator. I once did a benchmark of available key-value data stores > measuring not only the speed, but also the amount of consumed memory: > > https://gist.github.com/amiramix/d43c9a73a6fe6d651d7f > > Maps are quite performant but process dictionary is still quicker and maps > are the worst when it comes to consumed memory, taking twice as much as > dict or process dictionary and over 5 times as much memory as ets. > Well, you are comparing apples and oranges. Process dictionary and ETS are something completely different from gb_trees, dict, maps or orddict. BTW if you have a look and find any issues please let me know and I will be > happy to correct and rerun. The test simply initializes the data structure > with InitSize amount of Key-Value elements, and then starts measuring the > time and memory needed to write/read Accesses amount of Key-Value/Key > elements with randomly generated Keys. > Yep, I take issue with it. I think you should look over how you measure memory. It seems very arbitrary. What are you measuring? The size of the process after it all? The gc can shrink or grow the heap during execution, trying to find a suitable size. Also look at the number of gc:s during the execution, this will indicate how garbage you are generating. Normally benchmarks ends with an explicit garbage_collect before looking at the size. This gives a more fair indication of memory consumption. It all depends on what you want to measure though, so look it over. Another approach would be to use erts_debug:flat_size/1. With the functional data structures in this test you'll get: - dict: 67568 - gb_trees: 78668 - map_match: 58543 - list: 78665 - orddict: 78665 // Bj?rn-Egil -------------- next part -------------- An HTML attachment was scrubbed... URL: From ahf@REDACTED Tue May 10 00:52:39 2016 From: ahf@REDACTED (=?UTF-8?B?QWxleGFuZGVyIEbDpnLDuHk=?=) Date: Tue, 10 May 2016 00:52:39 +0200 Subject: [erlang-questions] [ANN] Luke - Post-quantum Key Exchange: A New Hope Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hello, Luke 1.2.0 has been released. Luke is a set of Erlang NIF's for the post-quantum key exchange scheme named "A New Hope"[1]. The key exchange scheme provides 128 bits of post-quantum security with "a comfortable margin". The result of the handshake is a 256 bit (32 byte binary) shared secret which can be used together with Erlang's various symmetric ciphers found in the crypto app or together with the more modern ciphers found in Jesper Louis Andersen's enacl library[2]. I hope this library will be useful for other people who are interested in playing around with post-quantum cryptography in Erlang. Note that Luke is NOT a drop-in replacement for ordinary Diffie-Hellman or x25519 since it requires one extra message between the initiator and the responder to finalize the handshake. Luke is using the reference implementation of A New Hope[3]. The reference implementation is written in portable C and thus does not contain any fancy CPU specific optimizations. The avx2 implementation of A New Hope[4] will be added to the library at a later point in time if it becomes necessary, but the API will remain the same. A New Hope is placed into the Public Domain by the authors[1]. Luke is available from: Primary: https://lab.baconsvin.org/ahf/luke Mirror: https://github.com/ahf/luke Happy hacking. Cheers, Alex. [1]: https://github.com/tpoeppelmann/newhope/raw/master/paper/newhope.pdf [2]: https://github.com/jlouis/enacl [3]: https://github.com/tpoeppelmann/newhope/tree/master/ref [4]: https://github.com/tpoeppelmann/newhope/tree/master/avx2 -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJXMRJaAAoJEPm8L+IrCM6PgRoQAMGD5/sEsOJvxwejGPK9zQOX DZF3aAVvshxGwImkWGkmubfQqTtdnEqeK7wZYp32Ln2YnJptoym89jG+HGJ5OZ9B q+K4POGiEg7dIprdIlaYEq5H+UbG7RKsiGsAXlpkkfMoutnPDGRaDm9cuTMHlXvS 9p6WUgZOukMW+qi3Tz3HF+zxV/1axxr6d0hDlwUflszGp0roBe7buPVzVI5n9Bj+ ZM+uUhyxXE+gh95L8+tn9Ia8CB1YD5H47UCrSwXpvxt6y1Hx+Hq+I7ccOtgPIdgK tHEDA5cOXgZg+DmR+2ssrut+scYItEk1BHrQJ2A7pSq0TOF3qJ39NJz2lgEjokmt yW7o7eEByUsHeDaXun6R7HqH2d7DwfACl7QTTqxf4qpZrrJAUk8R59UB3jfePxLm hr6Yi39WpgcjJEoAjjzuNwrqGf5/jxO0e0Dy15KCMmf8b2WeSNvJEUzZfCzMVxvI b8o+2wlIZdTPUW8BCQjVu0zVAMqGR4hGOpTxELvNSLvVDgssbMQhIjGGrbn+92nL 1TkWU/yXYsw9In+dz96AfuPE+8w7vTSmDUmyx5mK3l0uD9r4tizyQPI/xyxc2PuW M2dcpvRKLwqAEy1ry0VQhP5tQ2/8u83cGzxswNciUvfey9qWNvq/Eu1Fwhc2gBc4 O1gIt+HmfgCOG7TKUQee =UgyQ -----END PGP SIGNATURE----- -- Alexander F?r?y From ok@REDACTED Tue May 10 05:38:56 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 10 May 2016 15:38:56 +1200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> Message-ID: On 10/05/16 5:23 AM, Lyn Headley wrote: > The editor has a keybinding for 'express puzzlement here.' Elder may > discern the source of the puzzlement simply by observing its line of > code in context (which elder's editor provides), or elder may need > more information. I don't think there is any doubt about it. My own experience has been that "I don't understand " invariably merits the response "*what* don't you understand about ". It doesn't always *get* that response. Let me give you a personal example, which still rankles. I happened to mention to someone (while I was still a Masters student) that I didn't understand the difference between the cardinal numbers and the ordinal numbers. What I *meant* was "I don't understand why \aleph_0 + \aleph_0 = \aleph_0 but \omega + \omega > \omega." That is, what I didn't understand was why infinite cardinals and infinite ordinals don't follow the same rules. (Please, don't anyone explain it to me now. Thanks to the book "The Joy of Sets", I do understand it.) The lecturer in question told me a little story: A little boy was helping his mother in the kitchen. He was counting beans as he put them into a bowl. His mother said "give me five beans, Johnny." He answered, "I don't remember which was five." This of course demonstrated the difference between the *linguistic* cardinal numbers and ordinal numbers, but was utterly useless for helping me with my actual problem. Worse still, it left me feeling humiliated that anyone had thought I might not understand the linguistic distinction. I never expressed any kind of ignorance or uncertainty in front of that lecturer again. So actually, there is something worse than "elder" having to ask "youth" what the puzzlement is about, and that's "elder" NOT asking, because "elder" has unwarranted confidence that s/he knows already. > Note that context includes the entire active question > history, which helps elder reply with "not relevant to your question." > Express puzzlement is a bit lazy on youth's part, so before submitting > her question youth may annotate this event with language. And why should we believe that "elder" is right? I mean, in this scenario, "elder" doesn't know for sure what "youth's" question IS. And even if the answer is not relevant to "youth's" immediate *question*, it might be extremely useful for youth's *development*. It may be that other "elders" are extremely good at reading "youths'" minds. Perhaps it is a form of clairvoyance. It would be very interesting to explore this experimentally. > > Youth 1 posts her question to a public location where Youth 2 and > Google can trawl for existing questions and answers. Of course this > raises Grzegorz Junka's question about how to maintain this archive, > but I don't think maintenance is critical. The resource can age, even > obsolesce, and still provide great value while it is relevant. As always, the problem is knowing which bits are relevant and which are not. It's not a matter of simple age. Myself, I would find trawling through Q&A sessions a frustrating squandering of my time, when the same amount of effort could have gone into straightforward stuff in the source files. I would view it as other people viewing *their* time as far more valuable than *my* time, because actually, it would be. > I'll close with a grander claim. Programmers are less likely to add > sufficient annotations and documentation to their code than they are > to answer specific questions about the code. Well, people are different. The best programmers I've met (as in the ones who could solve complex problems quickly and effectively) produced lots of internal documentation. And I really don't see why answers to specific questions cannot be recorded *in the code*. > And this Q&A process can > become a kind of re-usable documentation over time. If it is not being maintained, so that it's not even trustworthy applied to the file it pertains to, how is it going to become reusable? If I *want* documentation in the form of questions and answers, I know where to find StackOverflow. I've even answered some questions there myself. The screen-capture-as-video-with-audio-questions experiment could also be used to study whether people *are* good mind-readers about what the question is. (Turn off the audio, have the "elders" guess what the question is, turn it back on and resume.) The experiment might be worth doing. Has anyone already done it? From ok@REDACTED Tue May 10 05:40:47 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 10 May 2016 15:40:47 +1200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <2267355.i1Po6rEGgK@burrito> <1861666.43u3BY5VlF@burrito> <20160509145429.GA4362@fhebert-ltm2.internal.salesforce.com> Message-ID: <14150a2e-4a00-5e43-296f-3f8682811296@cs.otago.ac.nz> On 10/05/16 6:29 AM, Robert Virding wrote: > > Undoubtedly, but that's the way that Lua does so I don't have much of > a choice. :-) And that is why I need it and why I only need a rather > simple version. > You have explained wll how you have already solved your problem in what seems to me an excellent way. 2-3 trees seem like a perfectly fine way to implement Lua tables, which come with no performance guarantees. > > If I don't get it I will just keep on nagging about it. "And they > couldn't give me this really simple feature I needed." > It's not simple, not if you want something *coherent*. From garazdawi@REDACTED Tue May 10 08:46:51 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Tue, 10 May 2016 08:46:51 +0200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> <3fe10814-0754-7d37-eeb9-03f6fab40795@cs.otago.ac.nz> Message-ID: > On 10 maj 2016, at 02:03, Richard A. O'Keefe wrote: > The problem is that hashed data structures do not give you this guarantee. Not all hashed data structures will, but some do. The ones we use for maps is an example of one that does give this guarantee. > > It is not clear to me that a *stable* order is any easier to provide than a > *published* order. > With a non-published order the hashing algorithm can be changed without having to guarantee the iterator order, or tweak/change the algorithm as long as the property of stable order is maintained. So while it may not be easier to provide, not publishing the order makes it easier to do future optimizations. From bjorn-egil.xb.dahlberg@REDACTED Tue May 10 09:36:29 2016 From: bjorn-egil.xb.dahlberg@REDACTED (=?iso-8859-1?Q?Bj=F6rn-Egil_Dahlberg_XB?=) Date: Tue, 10 May 2016 07:36:29 +0000 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <2267355.i1Po6rEGgK@burrito> <1861666.43u3BY5VlF@burrito> <20160509145429.GA4362@fhebert-ltm2.internal.salesforce.com> <648b60ae-9084-4c09-2514-86962496f6f6@gjunka.com>, Message-ID: Another key issue I forgot to mention. It matters what payload you use and the size and structure of the keys. For the ETS measurement it matters a lot. You have to copy the entire payload for each get and insert. Performance wise Maps usually beats ETS. Apples and Oranges however. ETS has other features so it's not a fair comparison. // egil Sent from my iPhone 10 maj 2016 kl. 03:10 skrev Bj?rn-Egil Dahlberg >: 2016-05-10 1:59 GMT+02:00 Grzegorz Junka >: I only wanted to mention that maps is not the only key-value data structure available in Erlang and may not the best one to implement such an iterator. I once did a benchmark of available key-value data stores measuring not only the speed, but also the amount of consumed memory: https://gist.github.com/amiramix/d43c9a73a6fe6d651d7f Maps are quite performant but process dictionary is still quicker and maps are the worst when it comes to consumed memory, taking twice as much as dict or process dictionary and over 5 times as much memory as ets. Well, you are comparing apples and oranges. Process dictionary and ETS are something completely different from gb_trees, dict, maps or orddict. BTW if you have a look and find any issues please let me know and I will be happy to correct and rerun. The test simply initializes the data structure with InitSize amount of Key-Value elements, and then starts measuring the time and memory needed to write/read Accesses amount of Key-Value/Key elements with randomly generated Keys. Yep, I take issue with it. I think you should look over how you measure memory. It seems very arbitrary. What are you measuring? The size of the process after it all? The gc can shrink or grow the heap during execution, trying to find a suitable size. Also look at the number of gc:s during the execution, this will indicate how garbage you are generating. Normally benchmarks ends with an explicit garbage_collect before looking at the size. This gives a more fair indication of memory consumption. It all depends on what you want to measure though, so look it over. Another approach would be to use erts_debug:flat_size/1. With the functional data structures in this test you'll get: - dict: 67568 - gb_trees: 78668 - map_match: 58543 - list: 78665 - orddict: 78665 // Bj?rn-Egil _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From list1@REDACTED Tue May 10 15:39:57 2016 From: list1@REDACTED (Grzegorz Junka) Date: Tue, 10 May 2016 13:39:57 +0000 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <2267355.i1Po6rEGgK@burrito> <1861666.43u3BY5VlF@burrito> <20160509145429.GA4362@fhebert-ltm2.internal.salesforce.com> <648b60ae-9084-4c09-2514-86962496f6f6@gjunka.com> Message-ID: <48b2bc5b-113e-e67a-fdf0-8be641dc248f@gjunka.com> Thanks for the comments Bj?rn-Egil. I did consider the concerns that you mentioned, but I wanted to take a pragmatic approach. I was more interested in practical implications of using a particular data structure rather than its idealistic capabilities: 1. All test data used in all tests is generated only _once_. This is to eliminate variations where specific structures work better with specific inputs. 2. Tests are run in turns the specified amount of times and the output is not averaged. This is to verify the reproducibility of the test for the same input data. 3. Key and Value are both small integers. This is to minimize the copying in ETS and to expose the overhead of calculating a hash in hash-based data structures. 4. The data structure is pre-populated with some data before starting to measure the time. This is to simulate longer-running processes that most of the time aren't empty. 5. Readings of existing and non-existing keys is interleaved with storing new keys. This is to reflect practical usage in typical applications. 6. Each test is run in isolation, in a new process. This is to measure the consumed memory most accurately - including all generated and not collected garbage. > Performance wise Maps usually beats ETS. Apples and Oranges however. > ETS has other features so it's not a fair comparison. Yeah, that's what my test concluded. Initially for small key/values maps are slightly faster. The difference would only grow in favour of maps for more complex payload because of the copying between ETS and the process. But maps consume much more memory than ETS. I believe the memory released when reallocating structures in ETS doesn't need to be garbage collected since it's implemented in C outside of any process memory? >> https://gist.github.com/amiramix/d43c9a73a6fe6d651d7f >> >> Maps are quite performant but process dictionary is still quicker >> and maps are the worst when it comes to consumed memory, taking >> twice as much as dict or process dictionary and over 5 times as >> much memory as ets. >> >> Well, you are comparing apples and oranges. Process dictionary and >> ETS are something completely different from gb_trees, dict, maps or >> orddict. What's the point of comparing gb_tress to gb_trees or maps to maps? >> >> I think you should look over how you measure memory. It seems very >> arbitrary. What are you measuring? The size of the process after it >> all? The gc can shrink or grow the heap during execution, trying to >> find a suitable size. Also look at the number of gc:s during the >> execution, this will indicate how garbage you are generating. >> Normally benchmarks ends with an explicit garbage_collect before >> looking at the size. This gives a more fair indication of memory >> consumption. >> It all depends on what you want to measure though, so look it over. >> >> Another approach would be to use erts_debug:flat_size/1. With the >> functional data structures in this test you'll get: >> >> - dict: 67568 >> - gb_trees: 78668 >> - map_match: 58543 >> - list: 78665 >> - orddict: 78665 >> >> I am not measuring the "size of the process after it all". I am measuring how much the size of an Erlang process grows after storing and reading 30000 key/value pairs. This is to give me a better understanding of how much memory I would potentially need to store (for example) a billion of key-value data elements in one million of Erlang processes (one thousand of such elements in each Erlang process). Measuring the flat size maybe useful for developers implementing the data structure. But does it tell me how much resources (CPU & memory) the VM can consume, and thus how much hardware I would need to buy to run such VM reliably? I appreciate your comments so please let me know if there is anything you would like to add. Coming back to the iterator question, some data structures may be better suitable for implementing such iterator than others. If for now the LUA table could be implemented either in maps or dict, then maybe if say, process dictionary supported iterators efficiently, they could well be implemented using PD instead? /Grzegorz -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Tue May 10 16:15:53 2016 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 10 May 2016 16:15:53 +0200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <48b2bc5b-113e-e67a-fdf0-8be641dc248f@gjunka.com> References: <2267355.i1Po6rEGgK@burrito> <1861666.43u3BY5VlF@burrito> <20160509145429.GA4362@fhebert-ltm2.internal.salesforce.com> <648b60ae-9084-4c09-2514-86962496f6f6@gjunka.com> <48b2bc5b-113e-e67a-fdf0-8be641dc248f@gjunka.com> Message-ID: <20160510141553.GA65189@erix.ericsson.se> On Tue, May 10, 2016 at 01:39:57PM +0000, Grzegorz Junka wrote: : > > > Performance wise Maps usually beats ETS. Apples and Oranges however. > > ETS has other features so it's not a fair comparison. > > Yeah, that's what my test concluded. Initially for small key/values maps > are slightly faster. The difference would only grow in favour of maps > for more complex payload because of the copying between ETS and the > process. But maps consume much more memory than ETS. I believe the > memory released when reallocating structures in ETS doesn't need to be > garbage collected since it's implemented in C outside of any process memory? ETS does not even store the payload infact the whole stored tuple in process heap memory, so the only thing affecting the process heap are the accumulating fetched tuples that you probably throw away immediately. The memory used is taken from the memory allocators that probably will get some fragmentation but as I said it does not show up on any process heap. > > > >> https://gist.github.com/amiramix/d43c9a73a6fe6d651d7f > >> > >> Maps are quite performant but process dictionary is still quicker > >> and maps are the worst when it comes to consumed memory, taking > >> twice as much as dict or process dictionary and over 5 times as > >> much memory as ets. > >> > >> Well, you are comparing apples and oranges. Process dictionary and > >> ETS are something completely different from gb_trees, dict, maps or > >> orddict. > > What's the point of comparing gb_tress to gb_trees or maps to maps? Please misunderstand correctly. Nobody is talking about comparing gb_trees to gb_trees. What Bj?rn-Egil said was that it is fair to compare any within the group (gb_trees, dict, maps, orddict) with each other since all are functional heap based data structures. But it is not fair to compare any one in the group with neither the process dictionary nor with ETS. It is also hard to compare the process dictionary with ETS since the process dictionary stores on the process heap while ETS stores in other allocated memory. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From list1@REDACTED Tue May 10 16:30:19 2016 From: list1@REDACTED (Grzegorz Junka) Date: Tue, 10 May 2016 14:30:19 +0000 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <20160510141553.GA65189@erix.ericsson.se> References: <2267355.i1Po6rEGgK@burrito> <1861666.43u3BY5VlF@burrito> <20160509145429.GA4362@fhebert-ltm2.internal.salesforce.com> <648b60ae-9084-4c09-2514-86962496f6f6@gjunka.com> <48b2bc5b-113e-e67a-fdf0-8be641dc248f@gjunka.com> <20160510141553.GA65189@erix.ericsson.se> Message-ID: <47644702-02b6-ac83-4587-263e734f54a0@gjunka.com> >>>> https://gist.github.com/amiramix/d43c9a73a6fe6d651d7f >>>> >>>> Maps are quite performant but process dictionary is still quicker >>>> and maps are the worst when it comes to consumed memory, taking >>>> twice as much as dict or process dictionary and over 5 times as >>>> much memory as ets. >>>> >>>> Well, you are comparing apples and oranges. Process dictionary and >>>> ETS are something completely different from gb_trees, dict, maps or >>>> orddict. >> What's the point of comparing gb_tress to gb_trees or maps to maps? > Please misunderstand correctly. Nobody is talking about comparing gb_trees > to gb_trees. What Bj?rn-Egil said was that it is fair to compare any > within the group (gb_trees, dict, maps, orddict) with each other since all > are functional heap based data structures. But it is not fair to compare > any one in the group with neither the process dictionary nor with ETS. > > It is also hard to compare the process dictionary with ETS since the > process dictionary stores on the process heap while ETS stores in other > allocated memory. > If I need to store many key-value pairs I don't care which group the structure belongs to. I measure and it pick one that is most suitable for the job at hand considering all the limitations. Why should I limit the test to comparing only some data structures with some others? In my test I am adding the memory consumed by the process and in ETS: process_size(E) + ets:info(Tid, memory) * erlang:system_info(wordsize). It's hard but one need to start somewhere. It's not enough to say, it's hard, don't do it. I am not saying that maps are implemented badly or that they are bad data structures. Only that they are not the only data structure that should be considered because everything comes with some limitations and trade-offs. Grzegorz From lloyd@REDACTED Tue May 10 17:26:18 2016 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Tue, 10 May 2016 11:26:18 -0400 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> Message-ID: <1FC916F3-27E8-44F1-B4AE-829734D916BA@writersglen.com> Hello, A similar less-than-helpful response is "have you tried flurbmacburger," where flurbmacburger is as opaque as the original question. Best to all, LRP Sent from my iPad > On May 9, 2016, at 11:38 PM, Richard A. O'Keefe wrote: > > > >> On 10/05/16 5:23 AM, Lyn Headley wrote: >> The editor has a keybinding for 'express puzzlement here.' Elder may >> discern the source of the puzzlement simply by observing its line of >> code in context (which elder's editor provides), or elder may need >> more information. > I don't think there is any doubt about it. My own experience has been > that "I don't understand " invariably merits the response "*what* > don't you understand about ". It doesn't always *get* that > response. > > Let me give you a personal example, which still rankles. > > I happened to mention to someone (while I was still a Masters student) > that I didn't understand the difference between the cardinal numbers > and the ordinal numbers. What I *meant* was > "I don't understand why \aleph_0 + \aleph_0 = \aleph_0 > but \omega + \omega > \omega." > That is, what I didn't understand was why infinite cardinals and infinite > ordinals don't follow the same rules. (Please, don't anyone explain it > to me now. Thanks to the book "The Joy of Sets", I do understand it.) > The lecturer in question told me a little story: > A little boy was helping his mother in the kitchen. > He was counting beans as he put them into a bowl. > His mother said "give me five beans, Johnny." > He answered, "I don't remember which was five." > This of course demonstrated the difference between the *linguistic* > cardinal numbers and ordinal numbers, but was utterly useless for > helping me with my actual problem. Worse still, it left me feeling > humiliated that anyone had thought I might not understand the > linguistic distinction. I never expressed any kind of ignorance or > uncertainty in front of that lecturer again. > > So actually, there is something worse than "elder" having to ask "youth" > what the puzzlement is about, and that's "elder" NOT asking, because > "elder" has unwarranted confidence that s/he knows already. >> Note that context includes the entire active question >> history, which helps elder reply with "not relevant to your question." >> Express puzzlement is a bit lazy on youth's part, so before submitting >> her question youth may annotate this event with language. > > And why should we believe that "elder" is right? I mean, in this scenario, > "elder" doesn't know for sure what "youth's" question IS. And even if the > answer is not relevant to "youth's" immediate *question*, it might be > extremely useful for youth's *development*. > > It may be that other "elders" are extremely good at reading "youths'" minds. > Perhaps it is a form of clairvoyance. It would be very interesting to explore > this experimentally. >> >> Youth 1 posts her question to a public location where Youth 2 and >> Google can trawl for existing questions and answers. Of course this >> raises Grzegorz Junka's question about how to maintain this archive, >> but I don't think maintenance is critical. The resource can age, even >> obsolesce, and still provide great value while it is relevant. > > As always, the problem is knowing which bits are relevant and which are not. > It's not a matter of simple age. Myself, I would find trawling through Q&A > sessions a frustrating squandering of my time, when the same amount of > effort could have gone into straightforward stuff in the source files. I > would view it as other people viewing *their* time as far more valuable than > *my* time, because actually, it would be. >> I'll close with a grander claim. Programmers are less likely to add >> sufficient annotations and documentation to their code than they are >> to answer specific questions about the code. > Well, people are different. The best programmers I've met (as in the ones > who could solve complex problems quickly and effectively) produced lots of > internal documentation. And I really don't see why answers to specific > questions cannot be recorded *in the code*. > >> And this Q&A process can >> become a kind of re-usable documentation over time. > > If it is not being maintained, so that it's not even trustworthy applied > to the file it pertains to, how is it going to become reusable? > > If I *want* documentation in the form of questions and answers, > I know where to find StackOverflow. I've even answered some > questions there myself. > > The screen-capture-as-video-with-audio-questions experiment could > also be used to study whether people *are* good mind-readers about > what the question is. (Turn off the audio, have the "elders" guess > what the question is, turn it back on and resume.) > > The experiment might be worth doing. Has anyone already done it? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From lheadley@REDACTED Tue May 10 19:28:57 2016 From: lheadley@REDACTED (Lyn Headley) Date: Tue, 10 May 2016 10:28:57 -0700 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <545cdc1d-4cf6-75ba-5386-47b610ceb85a@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> Message-ID: > Richard A. O'Keefe wrote: > It may be that other "elders" are extremely good at reading "youths'" minds. > Perhaps it is a form of clairvoyance. It would be very interesting to > explore > this experimentally. I'm not sure what led you to characterize my proposal as involving "mind reading." As your lecturer demonstrated, even face-to-face contexts require active listening and inference about what is being asked. At least my proposal enriches the semantic environment beyond that of the lone developer staring at a text file. Human communication is imperfect. Alas, we are all cast in the role of the mind reader. But some contexts provide more context than others. From ok@REDACTED Wed May 11 02:48:45 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 11 May 2016 12:48:45 +1200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> <3fe10814-0754-7d37-eeb9-03f6fab40795@cs.otago.ac.nz> Message-ID: On 10/05/16 6:46 PM, Lukas Larsson wrote: >> On 10 maj 2016, at 02:03, Richard A. O'Keefe wrote: >> The problem is that hashed data structures do not give you this guarantee. > Not all hashed data structures will, but some do. The ones we use for > maps is an example of one that does give this guarantee. Yes, but we don't get a guarantee that the same implementation will be used forever. From ok@REDACTED Wed May 11 03:17:19 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 11 May 2016 13:17:19 +1200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> Message-ID: <75de70c6-8321-c2ce-0516-0daf0132780c@cs.otago.ac.nz> On 11/05/16 5:28 AM, Lyn Headley wrote: (a message that was sent privately, to which I sent a private reply, which I now have to reconstruct) >> Richard A. O'Keefe wrote: >> It may be that other "elders" are extremely good at reading "youths'" minds. >> Perhaps it is a form of clairvoyance. It would be very interesting to >> explore >> this experimentally. > I'm not sure what led you to characterize my proposal as involving > "mind reading." The fact that the code reader communicates a problem to the code explainer just by pressing an "I am puzzled " button. I am pretty sure that if I were the code explainer, my response would be "I am puzzled about what puzzles you ", every time. As it happens, there was a fair bit of work in the AI/Cognitive Science field back in the 70s and 80s on diagnosing novice errors (in things like arithmetic, simple programming, circuits). It now occurs to me that perhaps we can think of puzzlement as "failure to reconstruct a plan under which the line in question serves a known purpose", and perhaps we could diagnose that. (Of course, this assumes that it is the reader at fault, not the writer. You probably remember Feynman's delightful anecdote of being asked for advice about a nuclear reactor, and for lack of any better idea, stabbing his finger at random on the plan, and saying "What does that do?" The others looked, and said "You've found the problem!" I suspect his choice wasn't _quite_ random...) The point was basically that while making a transcription of a code reader's odyssey through the code might be useful, I don't think it is helpful to make the communication channel so narrow. I think that if someone is puzzled and wants help, it is not unreasonable to give them the chance to write or speak at least a sentence about what they find puzzling. "I don't understand this syntax" and "I thought that function was deprecated" can both cause puzzlement, and at the same place. > As your lecturer demonstrated, even face-to-face > contexts require active listening and inference about what is being > asked. At least my proposal enriches the semantic environment beyond > that of the lone developer staring at a text file. I think it is a stretch to call it a "semantic" environment. If someone leaves the cursor on a line of code for five minutes, what does that _mean_? Are they reading it, are they looking something up in a manual, did they just go to the toilet, or what? If they are doing a search and stop at a line, is it because this was a serious candidate for their interest or because the editor has a stupid search interface that won't let you search for a two word phrase (a problem I frequently have in a certain PDF reader)? The channel is very narrow and carries very little meaning. Let someone introspect into a microphone while they are doing this and you *might* get some serious semantic context. > Human communication is imperfect. Alas, we are all cast in the role of > the mind reader. But some contexts provide more context than others. Indeed they do, which was precisely my point. I don't see anything to be gained by deliberately choosing a narrow interface providing very little context when video screen capture with audio annotation is such an established technology that all our lecture theatres now have it, so the basic idea can be tried out with no upfront development cost at all. My research agenda is sufficiently full for this year that I was advised not to post the original message in this thread, but that's something I have put a bit of time into, and I find that in my code it's context I *want* to provide and in others' code it's context I *wish* they'd given, but the vocabulary hasn't quite gelled yet. From raimo+erlang-questions@REDACTED Wed May 11 10:55:10 2016 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 11 May 2016 10:55:10 +0200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <47644702-02b6-ac83-4587-263e734f54a0@gjunka.com> References: <1861666.43u3BY5VlF@burrito> <20160509145429.GA4362@fhebert-ltm2.internal.salesforce.com> <648b60ae-9084-4c09-2514-86962496f6f6@gjunka.com> <48b2bc5b-113e-e67a-fdf0-8be641dc248f@gjunka.com> <20160510141553.GA65189@erix.ericsson.se> <47644702-02b6-ac83-4587-263e734f54a0@gjunka.com> Message-ID: <20160511085510.GA82001@erix.ericsson.se> On Tue, May 10, 2016 at 02:30:19PM +0000, Grzegorz Junka wrote: > > >>>> https://gist.github.com/amiramix/d43c9a73a6fe6d651d7f > >>>> > >>>> Maps are quite performant but process dictionary is still quicker > >>>> and maps are the worst when it comes to consumed memory, taking > >>>> twice as much as dict or process dictionary and over 5 times as > >>>> much memory as ets. > >>>> > >>>> Well, you are comparing apples and oranges. Process dictionary and > >>>> ETS are something completely different from gb_trees, dict, maps or > >>>> orddict. > >> What's the point of comparing gb_tress to gb_trees or maps to maps? > > Please misunderstand correctly. Nobody is talking about comparing gb_trees > > to gb_trees. What Bj?rn-Egil said was that it is fair to compare any > > within the group (gb_trees, dict, maps, orddict) with each other since all > > are functional heap based data structures. But it is not fair to compare > > any one in the group with neither the process dictionary nor with ETS. > > > > It is also hard to compare the process dictionary with ETS since the > > process dictionary stores on the process heap while ETS stores in other > > allocated memory. > > > > If I need to store many key-value pairs I don't care which group the > structure belongs to. I measure and it pick one that is most suitable > for the job at hand considering all the limitations. Why should I limit > the test to comparing only some data structures with some others? I did not say you should limit your tests. I just said it is hard to do a fair comparision. In fact it would be very interesting to get a usable and understandable comparision of the different available dictionary types, if possible. > > In my test I am adding the memory consumed by the process and in ETS: > process_size(E) + ets:info(Tid, memory) * erlang:system_info(wordsize). > It's hard but one need to start somewhere. It's not enough to say, it's > hard, don't do it. I did not say "don't do it". I said it is not a fair comparision, with the underlying assumption that you only compared heap usage. But since you apparently have tried to compensate for that unfairness you just might have a fairly fair comparision. As Bj?rn-Egil pointed out, though, process_info(self(), memory) might not be a good metric without a garbage_collect() preceeding it, depending on the application you benchmark for. You do not know for certain that the final process size is the maximum during the test, depending on when garbage collections are triggered. Have a look at erlang:system_monitor(Pid, [{large_heap,Size}]). It might be relevant in this benchmark. > > I am not saying that maps are implemented badly or that they are bad > data structures. Only that they are not the only data structure that > should be considered because everything comes with some limitations and > trade-offs. Precisely. The limitations and trade-offs makes the different alternatives hard to compare. (Note that I just now did not say impossible nor that it should not be done) I just said "hard" and it it is also hard (not impossible and no that it should not be done) to present the results in an understandable, usable and trustable way for others. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From list1@REDACTED Wed May 11 11:48:41 2016 From: list1@REDACTED (Grzegorz Junka) Date: Wed, 11 May 2016 09:48:41 +0000 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <20160511085510.GA82001@erix.ericsson.se> References: <1861666.43u3BY5VlF@burrito> <20160509145429.GA4362@fhebert-ltm2.internal.salesforce.com> <648b60ae-9084-4c09-2514-86962496f6f6@gjunka.com> <48b2bc5b-113e-e67a-fdf0-8be641dc248f@gjunka.com> <20160510141553.GA65189@erix.ericsson.se> <47644702-02b6-ac83-4587-263e734f54a0@gjunka.com> <20160511085510.GA82001@erix.ericsson.se> Message-ID: <9fdca54d-9595-c442-e03d-46a19a25889e@gjunka.com> On 11/05/2016 08:55, Raimo Niskanen wrote: > On Tue, May 10, 2016 at 02:30:19PM +0000, Grzegorz Junka wrote: >> In my test I am adding the memory consumed by the process and in ETS: >> process_size(E) + ets:info(Tid, memory) * erlang:system_info(wordsize). >> It's hard but one need to start somewhere. It's not enough to say, it's >> hard, don't do it. > I did not say "don't do it". I said it is not a fair comparision, > with the underlying assumption that you only compared heap usage. > > But since you apparently have tried to compensate for that unfairness > you just might have a fairly fair comparision. > > As Bj?rn-Egil pointed out, though, process_info(self(), memory) might not > be a good metric without a garbage_collect() preceeding it, depending on > the application you benchmark for. > > You do not know for certain that the final process size is the maximum > during the test, depending on when garbage collections are triggered. Have > a look at erlang:system_monitor(Pid, [{large_heap,Size}]). It might be > relevant in this benchmark. I am not that interested in the size of the process containing the data structure after it has been garbage collected. I want to know how much garbage the data structure is generating because this directly affects the size of the VM when many of such processes are running in parallel. I don't know of a fairer way of measuring that than by getting the size of the process straight after adding and reading the specified amount of keys. Sure, the size of the process may shrink at some later time, but in a real application when I don't call gc purposefully the size of the process may also stay larger for a longer period of time before gc shrinks it. Please note that there is no deleting of key-value pairs, only adding and reading. So any garbage generated is caused by changes to the internal representation of the table holding the growing amount of elements. The allocator can't reuse memory chunks released by elements deleted previously. This may not be applicable to all real-life scenarios but should more accurately reflect the behaviour where the data structure holds many key-value pairs and the application is rarely changing the existing but is mostly adding new key-value pairs. Grzegorz From roberto@REDACTED Wed May 11 15:41:19 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 11 May 2016 15:41:19 +0200 Subject: [erlang-questions] [ANN] Cowbell Message-ID: Dear list, Cowbell has been released. Cowbell is a simple node connection manager, that will monitor nodes and reconnect them for you. You can find Cowbell on Hex or at: https://github.com/ostinelli/cowbell Best, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Wed May 11 17:40:38 2016 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 11 May 2016 17:40:38 +0200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <9fdca54d-9595-c442-e03d-46a19a25889e@gjunka.com> References: <20160509145429.GA4362@fhebert-ltm2.internal.salesforce.com> <648b60ae-9084-4c09-2514-86962496f6f6@gjunka.com> <48b2bc5b-113e-e67a-fdf0-8be641dc248f@gjunka.com> <20160510141553.GA65189@erix.ericsson.se> <47644702-02b6-ac83-4587-263e734f54a0@gjunka.com> <20160511085510.GA82001@erix.ericsson.se> <9fdca54d-9595-c442-e03d-46a19a25889e@gjunka.com> Message-ID: <20160511154038.GA97693@erix.ericsson.se> On Wed, May 11, 2016 at 09:48:41AM +0000, Grzegorz Junka wrote: > > On 11/05/2016 08:55, Raimo Niskanen wrote: > > On Tue, May 10, 2016 at 02:30:19PM +0000, Grzegorz Junka wrote: > >> In my test I am adding the memory consumed by the process and in ETS: > >> process_size(E) + ets:info(Tid, memory) * erlang:system_info(wordsize). > >> It's hard but one need to start somewhere. It's not enough to say, it's > >> hard, don't do it. > > I did not say "don't do it". I said it is not a fair comparision, > > with the underlying assumption that you only compared heap usage. > > > > But since you apparently have tried to compensate for that unfairness > > you just might have a fairly fair comparision. > > > > As Bj?rn-Egil pointed out, though, process_info(self(), memory) might not > > be a good metric without a garbage_collect() preceeding it, depending on > > the application you benchmark for. > > > > You do not know for certain that the final process size is the maximum > > during the test, depending on when garbage collections are triggered. Have > > a look at erlang:system_monitor(Pid, [{large_heap,Size}]). It might be > > relevant in this benchmark. > > I am not that interested in the size of the process containing the data > structure after it has been garbage collected. I want to know how much > garbage the data structure is generating because this directly affects > the size of the VM when many of such processes are running in parallel. > I don't know of a fairer way of measuring that than by getting the size > of the process straight after adding and reading the specified amount of > keys. Then I say that a fairer way to know how much garbage a process creates is to use erlang:trace(WorkerProcess, true, [garbage_collection,{tracer,TracerProcess}]). The messages from that feature can tell you how much garbage is being reclaimed. > > Sure, the size of the process may shrink at some later time, but in a > real application when I don't call gc purposefully the size of the > process may also stay larger for a longer period of time before gc > shrinks it. A possible problem is that the process may have just recently garbage collected for some benchmark cases but not for others, quite randomly, before your size measurement. > > Please note that there is no deleting of key-value pairs, only adding > and reading. So any garbage generated is caused by changes to the > internal representation of the table holding the growing amount of > elements. The allocator can't reuse memory chunks released by elements > deleted previously. This may not be applicable to all real-life It can on the contrary reuse such memory chunks. Tuple example: * We start with a tuple of 1000 elements, and a list of 30000 elements to insert in the tuple. Each element uses 3 words plus 2 in the list cons cell that is 5 * 30000 words plus the containing tuple of 1001 words that is 151001 words. The nearest above in the Fibonacci series that the heap algorithm uses is 189886 words. * We then add one element to the tuple. We only need to allocate 1002 words for the new containing tuple, and we have 189886 - 151001 = 38885 so that is no problem. * After creating the new containing tuple the old one is garbage not yet noticed. We continue to add elements and therefore allocate new containing tuples of 1003, 1004, ... words, so at about thirty added elements we run out of heap and allocate a new heap of 307242 (Fibonacci, again) words. * After creating the containing tuple that caused the new heap allocation we garbage collect and then find out that all but the last tuple was needed so we have allocated 307242 but only use 5 * 30000 + 1030 = 151030 words. So now we can go on with around (307242 - 151030) / 1030 ~= 150 element insertions before the next heap allocation where we again will allocate the next size 497128 and only use about 5 * 30000 + 1180 = 151180 words. That is 497128 / 151180 ~= 3.3 times more than needed. * If the garbage collect causes the new heap to be less than one fourth of the old heap size the new heap may be _shrunk_. So depending on if you happen to stop your iteration just before or just after a garbage collection, maybe a shrinking one, you will get totally different heap sizes. To furthermore complicate the situation much of your data for example the list you start with will survive two garbage collections and therefore be stored on the old-heap. This also applies to the parts of every tree data structure you benchmark that does not change often. This throws all numbers in my tuple example above out the window. Try increasing Accesses to 300000, and to make the test complete in a reasonable amount of time remove list and orddict from Types, and you will get completely different results of your benchmark. We get for example _negative_ results for dict and map looks like a winner. This kind of wobbling result depending on the scale of the test is a reliable sign of an unreliable benchmark... > scenarios but should more accurately reflect the behaviour where the > data structure holds many key-value pairs and the application is rarely > changing the existing but is mostly adding new key-value pairs. > > Grzegorz -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From chassoul@REDACTED Wed May 11 18:03:15 2016 From: chassoul@REDACTED (Jean Chassoul) Date: Wed, 11 May 2016 12:03:15 -0400 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> <20160507112149.GB11705@ferdmbp.local> Message-ID: On Sat, May 7, 2016 at 6:58 PM, Bj?rn-Egil Dahlberg < wallentin.dahlberg@REDACTED> wrote: > > After 19 I will try to shift focus to iterators and then the compiler, > specifically the match compiler. This is however dependent on other prio > work i have in my backlog. > > > That's an awesome list of good work for 19 and future releases, can't wait for more luerl news and improvements too! -------------- next part -------------- An HTML attachment was scrubbed... URL: From lheadley@REDACTED Wed May 11 18:18:08 2016 From: lheadley@REDACTED (Lyn Headley) Date: Wed, 11 May 2016 09:18:08 -0700 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: <75de70c6-8321-c2ce-0516-0daf0132780c@cs.otago.ac.nz> References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> <75de70c6-8321-c2ce-0516-0daf0132780c@cs.otago.ac.nz> Message-ID: On Tue, May 10, 2016 at 6:17 PM, Richard A. O'Keefe wrote: > The point was basically that while making a transcription of a code > reader's odyssey through the code might be useful, I don't think it > is helpful to make the communication channel so narrow. I think > that if someone is puzzled and wants help, it is not unreasonable > to give them the chance to write or speak at least a sentence about > what they find puzzling. > > "I don't understand this syntax" and > "I thought that function was deprecated" > can both cause puzzlement, and at the same place. > > I guess you missed this, so I'll repeat myself: "Express puzzlement is a bit lazy on youth's part, so before submitting her question youth may annotate this event with language." You are imagining the channel I proposed as more narrow than I am. But I also don't want to overwhelm the elder. Following every move on a video channel is probably too much work for the elder. So that's something to balance: poverty of channel semantics would be bad, but so would information overload. -Lyn From raould@REDACTED Wed May 11 18:22:15 2016 From: raould@REDACTED (Raoul Duke) Date: Wed, 11 May 2016 09:22:15 -0700 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> <75de70c6-8321-c2ce-0516-0daf0132780c@cs.otago.ac.nz> Message-ID: Hmmm... Can we just get back to guild style pairing perhaps? -------------- next part -------------- An HTML attachment was scrubbed... URL: From list1@REDACTED Wed May 11 18:50:43 2016 From: list1@REDACTED (Grzegorz Junka) Date: Wed, 11 May 2016 16:50:43 +0000 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <20160511154038.GA97693@erix.ericsson.se> References: <20160509145429.GA4362@fhebert-ltm2.internal.salesforce.com> <648b60ae-9084-4c09-2514-86962496f6f6@gjunka.com> <48b2bc5b-113e-e67a-fdf0-8be641dc248f@gjunka.com> <20160510141553.GA65189@erix.ericsson.se> <47644702-02b6-ac83-4587-263e734f54a0@gjunka.com> <20160511085510.GA82001@erix.ericsson.se> <9fdca54d-9595-c442-e03d-46a19a25889e@gjunka.com> <20160511154038.GA97693@erix.ericsson.se> Message-ID: <263673de-2fab-ada3-68b7-87488681e3e0@gjunka.com> On 11/05/2016 15:40, Raimo Niskanen wrote: > On Wed, May 11, 2016 at 09:48:41AM +0000, Grzegorz Junka wrote: >> On 11/05/2016 08:55, Raimo Niskanen wrote: >>> On Tue, May 10, 2016 at 02:30:19PM +0000, Grzegorz Junka wrote: >>>> In my test I am adding the memory consumed by the process and in ETS: >>>> process_size(E) + ets:info(Tid, memory) * erlang:system_info(wordsize). >>>> It's hard but one need to start somewhere. It's not enough to say, it's >>>> hard, don't do it. >>> I did not say "don't do it". I said it is not a fair comparision, >>> with the underlying assumption that you only compared heap usage. >>> >>> But since you apparently have tried to compensate for that unfairness >>> you just might have a fairly fair comparision. >>> >>> As Bj?rn-Egil pointed out, though, process_info(self(), memory) might not >>> be a good metric without a garbage_collect() preceeding it, depending on >>> the application you benchmark for. >>> >>> You do not know for certain that the final process size is the maximum >>> during the test, depending on when garbage collections are triggered. Have >>> a look at erlang:system_monitor(Pid, [{large_heap,Size}]). It might be >>> relevant in this benchmark. >> I am not that interested in the size of the process containing the data >> structure after it has been garbage collected. I want to know how much >> garbage the data structure is generating because this directly affects >> the size of the VM when many of such processes are running in parallel. >> I don't know of a fairer way of measuring that than by getting the size >> of the process straight after adding and reading the specified amount of >> keys. > Then I say that a fairer way to know how much garbage a process creates is > to use erlang:trace(WorkerProcess, true, > [garbage_collection,{tracer,TracerProcess}]). > > The messages from that feature can tell you how much garbage is being > reclaimed. I probably don't explain myself clearly. I am not interested in how much garbage is reclaimed either. I am interested in the effect of using one data structure or the other on the size of the VM. In other words, how much in average will my VM grow when I create a process that stores 30,000 key value pairs. This is so that I can estimate which data structure will let me store the biggest amount of key-value pairs given a specific amount of physical memory and a reasonable performance of looking up existing and storing new pairs. Imagine a huge data store all kept in memory, similar to mnesia ram disk table. >> Please note that there is no deleting of key-value pairs, only adding >> and reading. So any garbage generated is caused by changes to the >> internal representation of the table holding the growing amount of >> elements. The allocator can't reuse memory chunks released by elements >> deleted previously. This may not be applicable to all real-life > It can on the contrary reuse such memory chunks. > > Tuple example: > * We start with a tuple of 1000 elements, and a list of 30000 elements to > insert in the tuple. Each element uses 3 words plus 2 in the list cons > cell that is 5 * 30000 words plus the containing tuple of 1001 words > that is 151001 words. The nearest above in the Fibonacci series that > the heap algorithm uses is 189886 words. > * We then add one element to the tuple. We only need to allocate 1002 words > for the new containing tuple, and we have 189886 - 151001 = 38885 so that > is no problem. > * After creating the new containing tuple the old one is garbage not yet > noticed. We continue to add elements and therefore allocate new > containing tuples of 1003, 1004, ... words, so at about thirty added > elements we run out of heap and allocate a new heap of 307242 (Fibonacci, > again) words. > * After creating the containing tuple that caused the new heap allocation > we garbage collect and then find out that all but the last tuple > was needed so we have allocated 307242 but only use 5 * 30000 > + 1030 = 151030 words. So now we can go on with around (307242 - 151030) > / 1030 ~= 150 element insertions before the next heap allocation where we > again will allocate the next size 497128 and only use about 5 * 30000 + > 1180 = 151180 words. That is 497128 / 151180 ~= 3.3 times more than > needed. > * If the garbage collect causes the new heap to be less than one fourth > of the old heap size the new heap may be _shrunk_. > > So depending on if you happen to stop your iteration just before or just > after a garbage collection, maybe a shrinking one, you will get totally > different heap sizes. > > To furthermore complicate the situation much of your data for example the > list you start with will survive two garbage collections and therefore be > stored on the old-heap. This also applies to the parts of every tree data > structure you benchmark that does not change often. This throws all > numbers in my tuple example above out the window. > > Try increasing Accesses to 300000, and to make the test complete in a > reasonable amount of time remove list and orddict from Types, and you will > get completely different results of your benchmark. We get for example > _negative_ results for dict and map looks like a winner. This kind of > wobbling result depending on the scale of the test is a reliable sign of an > unreliable benchmark... I use the same input data in the same order in all tests in all cycles during a single run. I believe there is no randomness in storing the same keys in the same data structures, so even if the reallocation occurs as you mentioned, it should be deterministically reproducible in every cycle. This indeed can be seen in the results, where in three cycles the reported sizes are the same every time. On the next run some results will be different, but again the same in all three cycles in that single run. It's interesting that no matter how many times I run the test some of the processes end up with just the same amount of memory allocated to it. Even if the input data changes, most of the data structures still allocate the same amount of memory, e.g.: 15335 Hits: ets Time:22068 Size:903056 15335 Hits: pd Time:12296 Size:2709360 15335 Hits: dict Time:67726 Size:2174112 15335 Hits: gb_trees Time:77140 Size:2174112 15335 Hits: map_get Time:22862 Size:5091000 15335 Hits: map_find Time:23983 Size:5091000 15335 Hits: map_match Time:20434 Size:5091000 15335 Hits: list Time:19606771 Size:371376 15335 Hits: orddict Time:8952194 Size:371376 15181 Hits: ets Time:21946 Size:900584 15181 Hits: pd Time:12655 Size:2709360 15181 Hits: dict Time:70246 Size:5091000 15181 Hits: gb_trees Time:83523 Size:2174112 15181 Hits: map_get Time:26856 Size:5091000 15181 Hits: map_find Time:21480 Size:5091000 15181 Hits: map_match Time:20160 Size:5091000 15181 Hits: list Time:19477574 Size:600904 15181 Hits: orddict Time:9250092 Size:600904 15254 Hits: ets Time:22745 Size:903552 15254 Hits: pd Time:12688 Size:2709360 15254 Hits: dict Time:70869 Size:2174112 15254 Hits: gb_trees Time:83563 Size:2174112 15254 Hits: map_get Time:23372 Size:5091000 15254 Hits: map_find Time:24381 Size:5091000 15254 Hits: map_match Time:20008 Size:5091000 15254 Hits: list Time:19669187 Size:600904 15254 Hits: orddict Time:9139247 Size:600904 Those are different runs, which can be seen by the Hits count. And yet maps and gb_trees take the same amount of memory every time. Running with 300,000 elements doesn't change much: 281040 Hits: ets Time:217597 Size:9245448 281040 Hits: pd Time:119464 Size:8426240 281040 Hits: dict Time:700924 Size:21150800 281040 Hits: gb_trees Time:759213 Size:21150800 281040 Hits: map_get Time:232013 Size:12874336 281040 Hits: map_find Time:217398 Size:12874336 281040 Hits: map_match Time:204495 Size:12874336 280912 Hits: ets Time:221122 Size:9245448 280912 Hits: pd Time:116640 Size:8426240 280912 Hits: dict Time:705112 Size:21150800 280912 Hits: gb_trees Time:732280 Size:21150800 280912 Hits: map_get Time:232050 Size:12874336 280912 Hits: map_find Time:219269 Size:12874336 280912 Hits: map_match Time:207320 Size:12874336 Still process dictionary is the quickest. Only now dict and gb_tress eat up more memory than maps. It might be interesting to draw graphs of performance and allocated memory depending on amount of keys, but it likely won't change the fact that pd is the best :) Grzegorz From ok@REDACTED Thu May 12 03:33:56 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 12 May 2016 13:33:56 +1200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <7933567e-f5a9-ed1d-414b-ff6c09126355@cs.otago.ac.nz> <75de70c6-8321-c2ce-0516-0daf0132780c@cs.otago.ac.nz> Message-ID: <543d8c1d-65ba-f2f5-06cb-1fc30d9a7269@cs.otago.ac.nz> On 12/05/16 4:18 AM, Lyn Headley wrote:I guess you missed this, so I'll repeat myself: > "Express puzzlement is a bit lazy on youth's part, so before submitting > her question youth may annotate this event with language." My point is that "may" should be "must". "Express puzzlement" is not "a bit" lazy, it is disruptively lazy > But I also don't want to overwhelm the elder. Following every move on > a video channel is probably too much work for the elder. So fast forward to tags? I think we're starting to make a case for pair programming, and I don't want to go there. From raimo+erlang-questions@REDACTED Thu May 12 12:12:42 2016 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 12 May 2016 12:12:42 +0200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <263673de-2fab-ada3-68b7-87488681e3e0@gjunka.com> References: <648b60ae-9084-4c09-2514-86962496f6f6@gjunka.com> <48b2bc5b-113e-e67a-fdf0-8be641dc248f@gjunka.com> <20160510141553.GA65189@erix.ericsson.se> <47644702-02b6-ac83-4587-263e734f54a0@gjunka.com> <20160511085510.GA82001@erix.ericsson.se> <9fdca54d-9595-c442-e03d-46a19a25889e@gjunka.com> <20160511154038.GA97693@erix.ericsson.se> <263673de-2fab-ada3-68b7-87488681e3e0@gjunka.com> Message-ID: <20160512101242.GA12686@erix.ericsson.se> On Wed, May 11, 2016 at 04:50:43PM +0000, Grzegorz Junka wrote: > > On 11/05/2016 15:40, Raimo Niskanen wrote: > > On Wed, May 11, 2016 at 09:48:41AM +0000, Grzegorz Junka wrote: > >> On 11/05/2016 08:55, Raimo Niskanen wrote: > >>> On Tue, May 10, 2016 at 02:30:19PM +0000, Grzegorz Junka wrote: > >>>> In my test I am adding the memory consumed by the process and in ETS: > >>>> process_size(E) + ets:info(Tid, memory) * erlang:system_info(wordsize). > >>>> It's hard but one need to start somewhere. It's not enough to say, it's > >>>> hard, don't do it. > >>> I did not say "don't do it". I said it is not a fair comparision, > >>> with the underlying assumption that you only compared heap usage. > >>> > >>> But since you apparently have tried to compensate for that unfairness > >>> you just might have a fairly fair comparision. > >>> > >>> As Bj?rn-Egil pointed out, though, process_info(self(), memory) might not > >>> be a good metric without a garbage_collect() preceeding it, depending on > >>> the application you benchmark for. > >>> > >>> You do not know for certain that the final process size is the maximum > >>> during the test, depending on when garbage collections are triggered. Have > >>> a look at erlang:system_monitor(Pid, [{large_heap,Size}]). It might be > >>> relevant in this benchmark. > >> I am not that interested in the size of the process containing the data > >> structure after it has been garbage collected. I want to know how much > >> garbage the data structure is generating because this directly affects > >> the size of the VM when many of such processes are running in parallel. > >> I don't know of a fairer way of measuring that than by getting the size > >> of the process straight after adding and reading the specified amount of > >> keys. > > Then I say that a fairer way to know how much garbage a process creates is > > to use erlang:trace(WorkerProcess, true, > > [garbage_collection,{tracer,TracerProcess}]). > > > > The messages from that feature can tell you how much garbage is being > > reclaimed. > > I probably don't explain myself clearly. I am not interested in how much > garbage is reclaimed either. I am interested in the effect of using one > data structure or the other on the size of the VM. In other words, how > much in average will my VM grow when I create a process that stores > 30,000 key value pairs. This is so that I can estimate which data > structure will let me store the biggest amount of key-value pairs given > a specific amount of physical memory and a reasonable performance of > looking up existing and storing new pairs. Imagine a huge data store all > kept in memory, similar to mnesia ram disk table. I got that! I try to explain that the size of the process after the test is not a valid measurement point! Since heaps can shrink the process may have been larger during the test. > > > >> Please note that there is no deleting of key-value pairs, only adding > >> and reading. So any garbage generated is caused by changes to the > >> internal representation of the table holding the growing amount of > >> elements. The allocator can't reuse memory chunks released by elements > >> deleted previously. This may not be applicable to all real-life > > It can on the contrary reuse such memory chunks. > > > > Tuple example: > > * We start with a tuple of 1000 elements, and a list of 30000 elements to > > insert in the tuple. Each element uses 3 words plus 2 in the list cons > > cell that is 5 * 30000 words plus the containing tuple of 1001 words > > that is 151001 words. The nearest above in the Fibonacci series that > > the heap algorithm uses is 189886 words. > > * We then add one element to the tuple. We only need to allocate 1002 words > > for the new containing tuple, and we have 189886 - 151001 = 38885 so that > > is no problem. > > * After creating the new containing tuple the old one is garbage not yet > > noticed. We continue to add elements and therefore allocate new > > containing tuples of 1003, 1004, ... words, so at about thirty added > > elements we run out of heap and allocate a new heap of 307242 (Fibonacci, > > again) words. > > * After creating the containing tuple that caused the new heap allocation > > we garbage collect and then find out that all but the last tuple > > was needed so we have allocated 307242 but only use 5 * 30000 > > + 1030 = 151030 words. So now we can go on with around (307242 - 151030) > > / 1030 ~= 150 element insertions before the next heap allocation where we > > again will allocate the next size 497128 and only use about 5 * 30000 + > > 1180 = 151180 words. That is 497128 / 151180 ~= 3.3 times more than > > needed. > > * If the garbage collect causes the new heap to be less than one fourth > > of the old heap size the new heap may be _shrunk_. > > > > So depending on if you happen to stop your iteration just before or just > > after a garbage collection, maybe a shrinking one, you will get totally > > different heap sizes. > > > > To furthermore complicate the situation much of your data for example the > > list you start with will survive two garbage collections and therefore be > > stored on the old-heap. This also applies to the parts of every tree data > > structure you benchmark that does not change often. This throws all > > numbers in my tuple example above out the window. > > > > Try increasing Accesses to 300000, and to make the test complete in a > > reasonable amount of time remove list and orddict from Types, and you will > > get completely different results of your benchmark. We get for example > > _negative_ results for dict and map looks like a winner. This kind of > > wobbling result depending on the scale of the test is a reliable sign of an > > unreliable benchmark... > > I use the same input data in the same order in all tests in all cycles > during a single run. I believe there is no randomness in storing the > same keys in the same data structures, so even if the reallocation > occurs as you mentioned, it should be deterministically reproducible in > every cycle. This indeed can be seen in the results, where in three The randomness is not between different runs of the same data structure; it is between different data structures so you can not compare your result from dict with the result from gb_trees. This is because a heap may shrink at some garbage collection and exactly when that happens depends on how much garbage has been produced, and how much garbage a certain data structure produces is very hard to predict (pseudorandom). > cycles the reported sizes are the same every time. On the next run some > results will be different, but again the same in all three cycles in > that single run. > > It's interesting that no matter how many times I run the test some of > the processes end up with just the same amount of memory allocated to > it. Even if the input data changes, most of the data structures still > allocate the same amount of memory, e.g.: The heap sizes are selected from the a Fibonacci-like series. That creates a quantification that is seen here. > > 15335 Hits: ets Time:22068 Size:903056 > 15335 Hits: pd Time:12296 Size:2709360 > 15335 Hits: dict Time:67726 Size:2174112 > 15335 Hits: gb_trees Time:77140 Size:2174112 > 15335 Hits: map_get Time:22862 Size:5091000 > 15335 Hits: map_find Time:23983 Size:5091000 > 15335 Hits: map_match Time:20434 Size:5091000 > 15335 Hits: list Time:19606771 Size:371376 > 15335 Hits: orddict Time:8952194 Size:371376 > > 15181 Hits: ets Time:21946 Size:900584 > 15181 Hits: pd Time:12655 Size:2709360 > 15181 Hits: dict Time:70246 Size:5091000 > 15181 Hits: gb_trees Time:83523 Size:2174112 > 15181 Hits: map_get Time:26856 Size:5091000 > 15181 Hits: map_find Time:21480 Size:5091000 > 15181 Hits: map_match Time:20160 Size:5091000 > 15181 Hits: list Time:19477574 Size:600904 > 15181 Hits: orddict Time:9250092 Size:600904 > > 15254 Hits: ets Time:22745 Size:903552 > 15254 Hits: pd Time:12688 Size:2709360 > 15254 Hits: dict Time:70869 Size:2174112 > 15254 Hits: gb_trees Time:83563 Size:2174112 > 15254 Hits: map_get Time:23372 Size:5091000 > 15254 Hits: map_find Time:24381 Size:5091000 > 15254 Hits: map_match Time:20008 Size:5091000 > 15254 Hits: list Time:19669187 Size:600904 > 15254 Hits: orddict Time:9139247 Size:600904 > > Those are different runs, which can be seen by the Hits count. And yet > maps and gb_trees take the same amount of memory every time. > > Running with 300,000 elements doesn't change much: > > 281040 Hits: ets Time:217597 Size:9245448 > 281040 Hits: pd Time:119464 Size:8426240 > 281040 Hits: dict Time:700924 Size:21150800 > 281040 Hits: gb_trees Time:759213 Size:21150800 > 281040 Hits: map_get Time:232013 Size:12874336 > 281040 Hits: map_find Time:217398 Size:12874336 > 281040 Hits: map_match Time:204495 Size:12874336 > > 280912 Hits: ets Time:221122 Size:9245448 > 280912 Hits: pd Time:116640 Size:8426240 > 280912 Hits: dict Time:705112 Size:21150800 > 280912 Hits: gb_trees Time:732280 Size:21150800 > 280912 Hits: map_get Time:232050 Size:12874336 > 280912 Hits: map_find Time:219269 Size:12874336 > 280912 Hits: map_match Time:207320 Size:12874336 > > Still process dictionary is the quickest. Only now dict and gb_tress eat > up more memory than maps. It might be interesting to draw graphs of > performance and allocated memory depending on amount of keys, but it > likely won't change the fact that pd is the best :) All right. I'll show you mine. I made a small change to your code that you is not hard to guess: Erlang/OTP 18 [erts-7.2.1] [source-ef34b57] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] Eshell V7.2.1 (abort with ^G) 1> erl_data_bench:start(30000). 15264 Hits: ets Time:22856 Size:906312 15264 Hits: pd Time:9504 Size:2709360 15264 Hits: dict Time:49149 Size:2174112 15264 Hits: gb_trees Time:57649 Size:2174112 15264 Hits: map_get Time:19387 Size:5091000 15264 Hits: map_find Time:18641 Size:5091000 15264 Hits: map_match Time:16031 Size:5091000 done 2> erl_data_bench:start(40000). 23598 Hits: ets Time:23053 Size:-2149576 23598 Hits: pd Time:11009 Size:163864 23598 Hits: dict Time:63180 Size:5451544 23598 Hits: gb_trees Time:80919 Size:3517792 23598 Hits: map_get Time:23618 Size:5451544 23598 Hits: map_find Time:22407 Size:5451544 23598 Hits: map_match Time:21243 Size:5451544 done 3> erl_data_bench:start(50000). 32398 Hits: ets Time:28390 Size:3602912 32398 Hits: pd Time:15106 Size:4446408 32398 Hits: dict Time:81661 Size:600904 32398 Hits: gb_trees Time:96279 Size:-1573208 32398 Hits: map_get Time:29142 Size:5451544 32398 Hits: map_find Time:28677 Size:5451544 32398 Hits: map_match Time:28207 Size:5451544 done 4> erl_data_bench:start(60000). 42011 Hits: ets Time:35965 Size:4610872 42011 Hits: pd Time:20392 Size:3845496 42011 Hits: dict Time:90501 Size:5451544 42011 Hits: gb_trees Time:112758 Size:3517792 42011 Hits: map_get Time:33981 Size:5451544 42011 Hits: map_find Time:33467 Size:5451544 42011 Hits: map_match Time:32018 Size:5451544 done 5> erl_data_bench:start(70000). 51491 Hits: ets Time:40139 Size:-1433664 51491 Hits: pd Time:17873 Size:327704 51491 Hits: dict Time:106166 Size:972288 51491 Hits: gb_trees Time:132816 Size:2906040 51491 Hits: map_get Time:42655 Size:4505448 51491 Hits: map_find Time:40136 Size:4505448 51491 Hits: map_match Time:38782 Size:4505448 done 6> erl_data_bench:start(80000). 61551 Hits: ets Time:46722 Size:2459576 61551 Hits: pd Time:22964 Size:3233744 61551 Hits: dict Time:120808 Size:972288 61551 Hits: gb_trees Time:150940 Size:2906040 61551 Hits: map_get Time:47957 Size:4505448 61551 Hits: map_find Time:46025 Size:4505448 61551 Hits: map_match Time:44137 Size:4505448 done 7> erl_data_bench:start(90000). 71110 Hits: ets Time:53009 Size:4042048 71110 Hits: pd Time:27188 Size:2632832 71110 Hits: dict Time:133803 Size:2906040 71110 Hits: gb_trees Time:176076 Size:6424736 71110 Hits: map_get Time:52987 Size:4505448 71110 Hits: map_find Time:51744 Size:4505448 71110 Hits: map_match Time:49109 Size:4505448 done 8> erl_data_bench:start(100000). 81274 Hits: ets Time:61566 Size:4045352 81274 Hits: pd Time:29819 Size:3233744 81274 Hits: dict Time:143565 Size:6424736 81274 Hits: gb_trees Time:184181 Size:6424736 81274 Hits: map_get Time:55400 Size:4505448 81274 Hits: map_find Time:53725 Size:4505448 81274 Hits: map_match Time:52562 Size:4505448 done 9> erl_data_bench:start(300000). 280898 Hits: ets Time:173624 Size:9245520 280898 Hits: pd Time:83201 Size:8426240 280898 Hits: dict Time:443979 Size:-771032 280898 Hits: gb_trees Time:552626 Size:21150800 280898 Hits: map_get Time:163563 Size:12874336 280898 Hits: map_find Time:160153 Size:12874336 280898 Hits: map_match Time:154237 Size:12874336 done Notice that ets, gb_trees and dict managed to get negative numbers, that the numbers flucutate widely for small increases in the the value of Accesses, as does which is the "best" data type. For 60000 and 100000 map_* looks good, but not as much for 80000. I'd say that your benchmark does not produce reliable numbers... That the process dictionary performs well is not a surprise since it is a destructive data structure. The only garbage it produces is the data items that are replaced (or deleted), there is no container that can become garbage. > > Grzegorz > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From pablo.platt@REDACTED Thu May 12 12:47:02 2016 From: pablo.platt@REDACTED (pablo platt) Date: Thu, 12 May 2016 13:47:02 +0300 Subject: [erlang-questions] Ubuntu Snappy package from erlang release? Message-ID: Hi, Ubuntu snappy packages seems to fit well with erlang releases. They are self contained with a simple file structure. https://developer.ubuntu.com/en/snappy/build-apps/ Did anyone tried to package an erlang release with ubuntu snappy? Does it support hot release upgrades and start as a daemon with systemd? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenneth@REDACTED Thu May 12 17:18:14 2016 From: kenneth@REDACTED (Kenneth Lundin) Date: Thu, 12 May 2016 17:18:14 +0200 Subject: [erlang-questions] Erlang/OTP 19.0-rc1 is available for testing Message-ID: Erlang/OTP 19.0-rc1 is available for testing. This is the release candidate before the final OTP 19.0 product release in June 2016. Between the 19.0 rc1 and the final release there will be new updates of the master branch with corrections and minor new features. Occasionally there might be new tags which we in that case will communicate and ask you to test. Erlang/OTP 19.0 is a new major release with new features, quite a few (characteristics) improvements, as well as a few incompatibilities. See the Release Notes and the documentation for more details. We would like to ask you to build and test this release candidate and send us your feedback as soon as possible, so that we can make the necessary corrections before OTP 19.0. The release contains many changes; thus, some unexpected incompatibilities or issues may have slipped through our tests. Please try to build and run your current products/applications and let us know about any problems. Some highlights of the release are: - compiler, stdlib: - New preprocessor macros *?FUNCTION_NAME, ?FUNCTION_ARITY* - New preprocessor directives -error(Term) and -warning(Term) to cause a compilation error or warning, respectively. - *gen_statem* a new state machine behavior - *mnesia_ext*: plugin of external storage solutions to mnesia - *crypto*: uses EVP interface in OpenSSL resulting in generally better performance and support for HW acceleration - *ssh*: performance improvements, uses the new gen_statem behavior - *dialyzer*: the support for maps is very much extended both the type specification syntax and the type analysis. - *erts*: - erlang:open_port(spawn, ...) 3-5 times faster - tracing, extensive improvements (scalability, speed, match specifications on send/receive, support for lttng, ...) - dirty schedulers improved - per process configurable handling of message_queue - fast loading of multiple modules - max_heap_size process flag - and much more .... You can find the Release Notes with more detailed info at http://www.erlang.org/download/OTP-19.0-rc1.README You find the source code at github.com in the official Erlang repository. Git tag OTP-19.0-rc1 https://github.com/erlang/otp/tree/OTP-19.0-rc1 You can also read the documentation on-line here: (see the Release Notes mentioned above for release notes which are not updated in the doc, but the new functionality is) http://www.erlang.org/documentation/doc-8.0-rc1/doc/ We also want to thank all of you that have contributed with new features, bug-fixes, suggestions and bug reports. The Erlang/OTP Team at Ericsson -------------- next part -------------- An HTML attachment was scrubbed... URL: From list1@REDACTED Thu May 12 21:55:11 2016 From: list1@REDACTED (Grzegorz Junka) Date: Thu, 12 May 2016 19:55:11 +0000 Subject: [erlang-questions] How to disable parentheses matching in Erlang shell? Message-ID: <6013b959-58b0-a963-9478-d6845993f024@gjunka.com> This has been bothering me for quit a while. I am using FreeBSD for Erlang development and it happens that in the Erlang shell, when I am writing a long command, and say I am closing a bracket, the shell automatically jumps the cursor to the opening bracket to show me which bracket has been closed. That would be a great feature if it worked fine. On longer lines the cursor jumps back to some random place and then jumps back to some other random place and then the whole shell is messed up. I have to copy the command to a text editor, delete all characters from the current command line in the shell, edit the command in the editor, then copy it back to the shell. When copying the command to the shell it doesn't try to jump the cursor and doesn't mess my shell. I also saw this behaviour on other systems, e.g. Red Hat. Is there any way of disabling this "jumping" feature? Grzegorz From sdl.web@REDACTED Fri May 13 06:48:46 2016 From: sdl.web@REDACTED (Leo Liu) Date: Fri, 13 May 2016 12:48:46 +0800 Subject: [erlang-questions] Erlang/OTP 19.0-rc1 is available for testing References: Message-ID: On 2016-05-12 17:18 +0200, Kenneth Lundin wrote: > - tracing, extensive improvements (scalability, speed, match > specifications on send/receive, support for lttng, ...) Thanks for another great release (to be). I tried building it with lttng on centos 7.2 but failed with stock lttng-tools-devel and lttng-ust-devel. ,----[ erts/config.log ] | configure:21911: checking for tracepoint_enabled in lttng/tracepoint.h | configure:21932: gcc -c -g -O2 -I/home/build/otp/erts/x86_64-unknown-linux-gnu -fno-tree-copyrename -D_GNU_SOURCE conftest.c >&5 | conftest.c: In function 'main': | conftest.c:239:23: error: 'com_ericsson_otp' undeclared (first use in this function) | if(tracepoint_enabled(com_ericsson_otp,dummy)) do {} while(0) | ^ | conftest.c:239:23: note: each undeclared identifier is reported only once for each function it appears in | conftest.c:239:40: error: 'dummy' undeclared (first use in this function) | if(tracepoint_enabled(com_ericsson_otp,dummy)) do {} while(0) | ^ | configure:21932: $? = 1 `---- Leo From wallentin.dahlberg@REDACTED Fri May 13 09:35:04 2016 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Fri, 13 May 2016 09:35:04 +0200 Subject: [erlang-questions] Erlang/OTP 19.0-rc1 is available for testing In-Reply-To: References: Message-ID: 2016-05-13 6:48 GMT+02:00 Leo Liu : > On 2016-05-12 17:18 +0200, Kenneth Lundin wrote: > > - tracing, extensive improvements (scalability, speed, match > > specifications on send/receive, support for lttng, ...) > > Thanks for another great release (to be). I tried building it with lttng > on centos 7.2 but failed with stock lttng-tools-devel and > lttng-ust-devel. > You need lttng-ust version 2.7 or greater. // Bj?rn-Egil -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Fri May 13 09:36:35 2016 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Fri, 13 May 2016 19:36:35 +1200 Subject: [erlang-questions] Erlang/OTP 19.0-rc1 is available for testing In-Reply-To: References: Message-ID: DTLS was on the TODO list for Erlang 19. Did this make it in? On Fri, May 13, 2016 at 7:35 PM, Bj?rn-Egil Dahlberg < wallentin.dahlberg@REDACTED> wrote: > 2016-05-13 6:48 GMT+02:00 Leo Liu : > >> On 2016-05-12 17:18 +0200, Kenneth Lundin wrote: >> > - tracing, extensive improvements (scalability, speed, match >> > specifications on send/receive, support for lttng, ...) >> >> Thanks for another great release (to be). I tried building it with lttng >> on centos 7.2 but failed with stock lttng-tools-devel and >> lttng-ust-devel. >> > > You need lttng-ust version 2.7 or greater. > > // Bj?rn-Egil > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri May 13 10:05:58 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 13 May 2016 11:05:58 +0300 Subject: [erlang-questions] Story of one performance bug (active, once vs active, N) Message-ID: Hi. Will try to share our experience with hunting one strange performance bug. One our (Flussonic) customer asked us to help because his web UI panel started working very slowly after update. Short look in network inspector of Chrome showed that blocker was a HTTP API call to method that parses config. Our config parser (generated by sub-optimal neotoma grammar) isn't blazing fast, but small config was parsed during 3 minutes (!) 3 minutes of running erlang with some ets calls and lot of code. Usually such amount of text is parsed during 1 second. We've logged on server and in console have seen that such API call makes something with erlang and during several minutes there are 400-700 active processes: > erlang:statistics(total_active_tasks). 740 When such schedule storm comes CPU used by flussonic is lowered from 1000% to 400% I will tell in other post how we have succesfully used lcnt, but here is a story about pure in-erlang performance degradation. After several hours of bisecting flussonic versions and minor commits we have found commit that broke everything. Flussonic on that server was working with moderate amount of cameras (350 IP cameras via protocol RTSP) and 500 mbit/s of input traffic. This data flow is recorded on disks and restreamed to several clients (maybe 50 mbit/s of output). Some versions ago there was code that activated sockets with {active, N} but this code was removed (don't remember why have I done it, but git blame ruthlessly tells that I did it) and replaced by {active, once} We have immediately replaced this code by another tricky version: handle_info({tcp, _Socket, Bin}, #rtsp{} = State) -> State1 = handle_data(Bin, State), erlang:send_after(100, self(), activate_tcp), {noreply, State1}; handle_info(activate_tcp, #rtsp{socket = Socket} = State) -> inet:setopts(Socket, [{active,once}]), {noreply, State} This simple lines of code immediately made everything works ok: ~900% of CPU and immediate response to API. Here is our interpretation of this situation. When I changed activation to once, rtsp socket was sending one message per 1460 bytes (just as IP camera sends). The trick here is that all IP cameras are sending into TCP socket data in the same was as they send in UDP socket. All video frames are fragmented into pieces so that they can fit into MTU. So when IP camera makes write(fd, data, len), len is always not more than 1460. IP camera tries to send not large bursts, but rather regular packet flow. So, flussonic started receiving 500 mbit / 1460 bytes = 40 K of messages per second. About 120 messages per second per each process on 24 core Xeon server. After adding this simple trick with later activation we give gen_tcp socket some time to fill buffer and average data size raised from 1460 to 50000 bytes. So amount of messages reduced to 2000 per second. Perhaps when I changed to active,once we've got situation that each rtsp handling process was rescheduled immediately after calling inet:setopts(Socket, [{active,once}]) because data has already arrived during handling previous packet. This rescheduling storm blocks time for all other processes. When we increased average size of packet situation became better because each rtsp handling process started working more time in actual reductions and less amount of time in scheduling. So looks that batch handling of data is cheaper than often scheduling. What is unclear for me: why was active,N better than active,once? I haven't checked but I think that still average size of tcp message and amount of messages was the same as in active,once case. Next post will be about locks and ets tables. It is really interesting! -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri May 13 10:08:28 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 13 May 2016 11:08:28 +0300 Subject: [erlang-questions] Erlang/OTP 19.0-rc1 is available for testing In-Reply-To: References: Message-ID: Seems that there will be no DTLS out of the box in 19.0, but perhaps there is a chance for it =) -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Fri May 13 10:12:05 2016 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Fri, 13 May 2016 20:12:05 +1200 Subject: [erlang-questions] Erlang/OTP 19.0-rc1 is available for testing In-Reply-To: References: Message-ID: Don't care how I get it, so long as it's reliable and meets the RFC :-) Lee On Fri, May 13, 2016 at 8:08 PM, Max Lapshin wrote: > Seems that there will be no DTLS out of the box in 19.0, but perhaps there > is a chance for it =) > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Fri May 13 10:12:43 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Fri, 13 May 2016 09:12:43 +0100 Subject: [erlang-questions] Story of one performance bug (active, once vs active, N) In-Reply-To: References: Message-ID: On 13 May 2016 at 09:05, Max Lapshin wrote: > > Hi. Will try to share our experience with hunting one strange performance > bug. > > > > > What is unclear for me: why was active,N better than active,once? I > haven't checked but I think that still average size of tcp message and > amount of messages was the same as in active,once case. > > > We had a similar problem with one of our systems at bet365. And the same workaround also seemed to work. Ultimately though it was because of a kernel setting which when adjusted made the problem go away with {active, once}. Strangely enough, pretty much the same day we resolved the issue Cloudflare wrote a blog post about the exact same problem they faced and the solution! I hope this is of some use to you. https://blog.cloudflare.com/the-story-of-one-latency-spike/ cheers, Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Fri May 13 10:22:42 2016 From: ingela.andin@REDACTED (Ingela Andin) Date: Fri, 13 May 2016 10:22:42 +0200 Subject: [erlang-questions] Erlang/OTP 19.0-rc1 is available for testing In-Reply-To: References: Message-ID: Hi! gen_statem rewrite is an enabler for DTLS. I do not think there will be a production-ready DTLS for 19. But hopefully there will be runnable prototype. I am working with Andreas Shultz commits from PR 936 to see what can be done. Regards Ingela Erlang/OTP team - Ericsson AB 2016-05-13 9:36 GMT+02:00 Lee Sylvester : > DTLS was on the TODO list for Erlang 19. Did this make it in? > > On Fri, May 13, 2016 at 7:35 PM, Bj?rn-Egil Dahlberg < > wallentin.dahlberg@REDACTED> wrote: > >> 2016-05-13 6:48 GMT+02:00 Leo Liu : >> >>> On 2016-05-12 17:18 +0200, Kenneth Lundin wrote: >>> > - tracing, extensive improvements (scalability, speed, match >>> > specifications on send/receive, support for lttng, ...) >>> >>> Thanks for another great release (to be). I tried building it with lttng >>> on centos 7.2 but failed with stock lttng-tools-devel and >>> lttng-ust-devel. >>> >> >> You need lttng-ust version 2.7 or greater. >> >> // Bj?rn-Egil >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri May 13 11:01:54 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 13 May 2016 12:01:54 +0300 Subject: [erlang-questions] Story of one performance bug (active, once vs active, N) In-Reply-To: References: Message-ID: Thanks for the link, it is really amazing! -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Fri May 13 11:19:55 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Fri, 13 May 2016 11:19:55 +0200 Subject: [erlang-questions] How to disable parentheses matching in Erlang shell? In-Reply-To: <6013b959-58b0-a963-9478-d6845993f024@gjunka.com> References: <6013b959-58b0-a963-9478-d6845993f024@gjunka.com> Message-ID: I don't think there is any way to turn of only the brace matching feature. You can always use the oldshell, i.e. "erl -oldshell", but you loose a lot of other feature that you may want in the shell. The problem you are experiencing normally happens when the erlang shell has a different idea of how wide your terminal is then what it actually is. I've found that this can normally be fixed by making sure that you have a 80 characters wide terminal window that erlang is running in, as 80 is the default if erlang can't find out from ncurses what the terminal size is. Lukas On Thu, May 12, 2016 at 9:55 PM, Grzegorz Junka wrote: > This has been bothering me for quit a while. I am using FreeBSD for Erlang > development and it happens that in the Erlang shell, when I am writing a > long command, and say I am closing a bracket, the shell automatically jumps > the cursor to the opening bracket to show me which bracket has been closed. > That would be a great feature if it worked fine. On longer lines the cursor > jumps back to some random place and then jumps back to some other random > place and then the whole shell is messed up. I have to copy the command to > a text editor, delete all characters from the current command line in the > shell, edit the command in the editor, then copy it back to the shell. When > copying the command to the shell it doesn't try to jump the cursor and > doesn't mess my shell. I also saw this behaviour on other systems, e.g. Red > Hat. Is there any way of disabling this "jumping" feature? > > Grzegorz > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sdl.web@REDACTED Fri May 13 12:23:41 2016 From: sdl.web@REDACTED (Leo Liu) Date: Fri, 13 May 2016 18:23:41 +0800 Subject: [erlang-questions] Erlang/OTP 19.0-rc1 is available for testing References: Message-ID: On 2016-05-13 09:35 +0200, Bj?rn-Egil Dahlberg wrote: > You need lttng-ust version 2.7 or greater. > > // Bj?rn-Egil Thanks for the note. Leo From Oliver.Korpilla@REDACTED Fri May 13 15:25:14 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Fri, 13 May 2016 15:25:14 +0200 Subject: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) Message-ID: Hello. I have transient workers under a simple_one_for_one supervisor. I start them dynamically through start_child/2. How exactly do these get restarted? With the same parameter list I supplied when first starting them? Concrete example: I have a central process listening to a TCP port, then accepting a connection, creating a worker for it through a supervisor, and then using gen_tcp:controlling_process/2 to transfer control of that new socket to the worker. The worker starts with one parameter: the socket. Now lets say the worker crashes on unexpected input and gets restarted. Is the socket now closed? Will the same socket again be provided to the process on restart? Thanks, Oliver From chandrashekhar.mullaparthi@REDACTED Fri May 13 15:47:17 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Fri, 13 May 2016 14:47:17 +0100 Subject: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) In-Reply-To: References: Message-ID: Hi Oliver, There is no point supervising a process which is handling a TCP connection as even if the supervisor restarts this process, the socket will not be reinstated. You just have to spawn a new process which will stay alive as long as the connection is active. Once the TCP connection is lost, the process dies, and a new process is spawned when the client reconnects. See https://gist.github.com/cmullaparthi/18ba2219befbf7a3c44c28cab004456f for a barebones example. cheers, Chandru On 13 May 2016 at 14:25, Oliver Korpilla wrote: > Hello. > > I have transient workers under a simple_one_for_one supervisor. I start > them dynamically through start_child/2. > > How exactly do these get restarted? With the same parameter list I > supplied when first starting them? > > Concrete example: > > I have a central process listening to a TCP port, then accepting a > connection, creating a worker for it through a supervisor, and then using > gen_tcp:controlling_process/2 to transfer control of that new socket to the > worker. The worker starts with one parameter: the socket. > > Now lets say the worker crashes on unexpected input and gets restarted. Is > the socket now closed? Will the same socket again be provided to the > process on restart? > > Thanks, > Oliver > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zzantozz@REDACTED Sat May 14 01:59:41 2016 From: zzantozz@REDACTED (Ryan Stewart) Date: Fri, 13 May 2016 23:59:41 +0000 Subject: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) In-Reply-To: References: Message-ID: Oliver: there is no restart for children under a simple 1-1 supervisor as documented under Supervision Principles at http://erlang.org/doc/man/supervisor.html#id243029 Otoh, I have to strongly disagree with Chandru about there being "no point" in supervising a connection process. Depending on the nature of your application, there can be a *huge* benefit in supervision, which is orderly shutdown. With a proper supervisory tree, you can ensure that all connection processes have a chance to finish their work and shut down cleanly when you stop your application. On Fri, May 13, 2016 at 8:47 AM Chandru < chandrashekhar.mullaparthi@REDACTED> wrote: > Hi Oliver, > > There is no point supervising a process which is handling a TCP connection > as even if the supervisor restarts this process, the socket will not be > reinstated. You just have to spawn a new process which will stay alive as > long as the connection is active. Once the TCP connection is lost, the > process dies, and a new process is spawned when the client reconnects. > > See https://gist.github.com/cmullaparthi/18ba2219befbf7a3c44c28cab004456f > for a barebones example. > > cheers, > Chandru > > On 13 May 2016 at 14:25, Oliver Korpilla wrote: > >> Hello. >> >> I have transient workers under a simple_one_for_one supervisor. I start >> them dynamically through start_child/2. >> >> How exactly do these get restarted? With the same parameter list I >> supplied when first starting them? >> >> Concrete example: >> >> I have a central process listening to a TCP port, then accepting a >> connection, creating a worker for it through a supervisor, and then using >> gen_tcp:controlling_process/2 to transfer control of that new socket to the >> worker. The worker starts with one parameter: the socket. >> >> Now lets say the worker crashes on unexpected input and gets restarted. >> Is the socket now closed? Will the same socket again be provided to the >> process on restart? >> >> Thanks, >> Oliver >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Sat May 14 09:46:00 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Sat, 14 May 2016 08:46:00 +0100 Subject: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) In-Reply-To: References: Message-ID: On 14 May 2016 at 00:59, Ryan Stewart wrote: > Oliver: there is no restart for children under a simple 1-1 supervisor as > documented under Supervision Principles at > http://erlang.org/doc/man/supervisor.html#id243029 > > Otoh, I have to strongly disagree with Chandru about there being "no > point" in supervising a connection process. Depending on the nature of your > application, there can be a *huge* benefit in supervision, which is orderly > shutdown. With a proper supervisory tree, you can ensure that all > connection processes have a chance to finish their work and shut down > cleanly when you stop your application. > Cleanup can be done in the terminate callback if you use process_flag(trap_exit, true). You need to supervise a process primarily if you want it *restarted*, not for cleanup. Yes, you do have orderly shutdown mechanisms if the number of restarts exceed a "normal" value, but that is not the situation here. Oliver has a server process to which clients connect. He does not need a supervisor for this because it is up to the client to reconnect if a connection is lost. cheers, Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Sat May 14 10:28:36 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Sat, 14 May 2016 10:28:36 +0200 Subject: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) In-Reply-To: References: Message-ID: <5736E1B4.80106@ninenines.eu> On 05/14/2016 09:46 AM, Chandru wrote: > On 14 May 2016 at 00:59, Ryan Stewart > wrote: > > Oliver: there is no restart for children under a simple 1-1 > supervisor as documented under Supervision Principles at > http://erlang.org/doc/man/supervisor.html#id243029 > > Otoh, I have to strongly disagree with Chandru about there being "no > point" in supervising a connection process. Depending on the nature > of your application, there can be a *huge* benefit in supervision, > which is orderly shutdown. With a proper supervisory tree, you can > ensure that all connection processes have a chance to finish their > work and shut down cleanly when you stop your application. > > > Cleanup can be done in the terminate callback if you use > process_flag(trap_exit, true). You need to supervise a process primarily > if you want it *restarted*, not for cleanup. Yes, you do have orderly > shutdown mechanisms if the number of restarts exceed a "normal" value, > but that is not the situation here. This is bad advice. Cleanup should be done in a separate process, otherwise sending a kill signal sent to the wrong process will leave things dirty. > Oliver has a server process to which clients connect. He does not need a > supervisor for this because it is up to the client to reconnect if a > connection is lost. This is incorrect. The primary function of supervisors is to provide a hierarchy of processes belonging to an application. With such a hierarchy it becomes possible to find and query information about any single process in your system in a standard manner. Secondary functions of supervisors include restarting, reporting (with SASL) and upgrades. But they're entirely optional. Having processes run without being attached to a supervisor is a big mistake, because they either become invisible, or force you to implement a non-standard way to find them. This makes introspecting a system more complex than it should be. The only acceptable use for non-supervised processes is to perform asynchronous calls to functions. Those processes however should be relatively short-lived, and you should really make sure that they can't get stuck indefinitely. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From list1@REDACTED Sat May 14 12:32:57 2016 From: list1@REDACTED (Grzegorz Junka) Date: Sat, 14 May 2016 10:32:57 +0000 Subject: [erlang-questions] How to disable parentheses matching in Erlang shell? In-Reply-To: References: <6013b959-58b0-a963-9478-d6845993f024@gjunka.com> Message-ID: Fixing my terminal to 80 characters only seems like taken straight from the Windows world. I hope Erlang can do better ;) Is there any way of telling the Erlang terminal how wide is my terminal, e.g. an environment variable? On 13/05/2016 09:19, Lukas Larsson wrote: > I don't think there is any way to turn of only the brace matching > feature. You can always use the oldshell, i.e. "erl -oldshell", but > you loose a lot of other feature that you may want in the shell. > > The problem you are experiencing normally happens when the erlang > shell has a different idea of how wide your terminal is then what it > actually is. I've found that this can normally be fixed by making sure > that you have a 80 characters wide terminal window that erlang is > running in, as 80 is the default if erlang can't find out from ncurses > what the terminal size is. > > Lukas > > On Thu, May 12, 2016 at 9:55 PM, Grzegorz Junka > wrote: > > This has been bothering me for quit a while. I am using FreeBSD > for Erlang development and it happens that in the Erlang shell, > when I am writing a long command, and say I am closing a bracket, > the shell automatically jumps the cursor to the opening bracket to > show me which bracket has been closed. That would be a great > feature if it worked fine. On longer lines the cursor jumps back > to some random place and then jumps back to some other random > place and then the whole shell is messed up. I have to copy the > command to a text editor, delete all characters from the current > command line in the shell, edit the command in the editor, then > copy it back to the shell. When copying the command to the shell > it doesn't try to jump the cursor and doesn't mess my shell. I > also saw this behaviour on other systems, e.g. Red Hat. Is there > any way of disabling this "jumping" feature? > > Grzegorz > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sat May 14 12:48:24 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 14 May 2016 12:48:24 +0200 Subject: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) In-Reply-To: References: Message-ID: On Sat, May 14, 2016 at 9:46 AM, Chandru < chandrashekhar.mullaparthi@REDACTED> wrote: > Cleanup can be done in the terminate callback if you use > process_flag(trap_exit, true). There is a subtlety here which, in some circumstances, creates trouble and one has to be aware of. This is in addition to what Lo?c mentions. Observation: In many systems, processes have dependencies and needs each other to operate. Nice cleanup cannot be done in the situation where your immediate dependencies are gone, but under a controlled shutdown, you can arrange the shutdown order such that the shutdown is graceful. Example: You are processing some kind of request in a web server. You decide to close down the application. Now, what you want to happen is a graceful shutdown: you stop accepting new requests, but you run the current requests to an end (up to some timeout). In settings where the server count is very dynamic and servers can get "elastically" added or removed, this is important because otherwise you would be losing requests under shutdown. Observation: processes under supervision have an ordering imposed on them. Their termination happens in the opposite order of spawning. This can be used to enforce the gracefulness constraint. Another trick, which works cross-application, is to use the Module:prep_stop/1 phase of termination to tell your other applications (ranch, cowboy, yaws) that they should start graceful termination of their workers. Once completed, you can stop your own supervision tree. "Naked" processes which are not linked into a supervision tree can terminate in any order they see fit. In particular, the Erlang VM regards itself as being "done" once all its applications are shut down. In particular a line process with process_flag(trap_exit, true) waiting on disk is not going to be allowed to hold up termination of the VM. A naked process has its use, but one has to be extremely careful around them. If they start failing, you have no easy way to know what their relation is. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From per@REDACTED Sat May 14 13:48:20 2016 From: per@REDACTED (Per Hedeland) Date: Sat, 14 May 2016 13:48:20 +0200 (CEST) Subject: [erlang-questions] erlang:raise/3 considered harmful? Message-ID: <201605141148.u4EBmKGQ067045@pluto.hedeland.org> Hi, I happened to want something like erlang:raise/3 for a library function - I didn't think it existed:-), but found that it does, although it comes with a semi-dire warning. The loop/3 function below does exactly what I want, specifically: if the fun provided by the caller of interate/3 crashes, the caller gets the same exception as if there had been no try ... catch, can handle it with hir own try/catch or not, but in any case receives full information about *what* went wrong in hir fun - all while the "protocol" spoken over the socket remains in sync. So, to take heed of the warning - is there a way to write an equivalent function *without* using raise/3 - or should it just be considered a case of "you really know what you are doing"? I believe I am of course:-), but I also think that the code below is a perfectly reasonable implementation of a side-effecty library function, and has no more to do with debugging or knowing what you're doing than any other code... --Per Hedeland iterate(Sock, Fun, State) -> gen_tcp:send(Sock, "start"), loop(Sock, Fun, State). loop(Sock, Fun, State) -> case gen_tcp:recv(Sock, 0) of {ok, "done"} -> {ok, State}; {ok, Data} -> Res = try Fun(Data, State) catch Class:Reason -> gen_tcp:send(Sock, "done"), erlang:raise(Class, Reason, erlang:get_stacktrace()) end, case Res of {continue, NewState} -> gen_tcp:send(Sock, "continue"), loop(Sock, Fun, NewState); {done, NewState} -> gen_tcp:send(Sock, "done"), {ok, NewState}; Error -> gen_tcp:send(Sock, "done"), Error end; Error -> Error end. From Oliver.Korpilla@REDACTED Sat May 14 14:21:25 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Sat, 14 May 2016 14:21:25 +0200 Subject: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) In-Reply-To: References: , Message-ID: Hello, and thank you all for your responses. I originally adopted simple_one_for_one supervisor because I had a problem with how other supervisors clean up processes. For the TCP connectors simple_one_for_one will be fine. As noted by others, they cannot really come back unless they reconnect, so that is fine. So, a simple_one_for_one supervisor acts like every child, regardless of child spec, as if it was temporary? I have another big batch of processes independent of the connectors. These serve individual requests emanating from the TCP layer, where an ID establishes which handler belongs to which batch of messages (i.e. each TCP payload contains an ID in its own proprietary header). Now, I originally saw these as transient workers I would like to have restarted, but since they are stateless and can be created on demand, I either can supervise them simple_one_for_one (and create them on demand when the one for a given ID is missing) or I can create them as transient children under a one_for_one and let that restart it on a crash. I originally went for simple_one_for_one because of the better performance and because it cleans up children after they terminate. I guess in case of one_for_one I have to clean up all children which shut down normally by calling terminate_child and delete_child on them. (I originally hoped one_for_one would do this if a child exited normally, but either I bungled my tests or it simply doesn't, even for transient children). Any recommendations? Thanks, Oliver ? Gesendet:?Samstag, 14. Mai 2016 um 12:48 Uhr Von:?"Jesper Louis Andersen" An:?Chandru Cc:?"Erlang-Questions Questions" Betreff:?Re: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) ? On Sat, May 14, 2016 at 9:46 AM, Chandru wrote:Cleanup can be done in the terminate callback if you use process_flag(trap_exit, true). There is a subtlety here which, in some circumstances, creates trouble and one has to be aware of. This is in addition to what Lo?c mentions. Observation: In many systems, processes have dependencies and needs each other to operate. Nice cleanup cannot be done in the situation where your immediate dependencies are gone, but under a controlled shutdown, you can arrange the shutdown order such that the shutdown is graceful. ? Example: You are processing some kind of request in a web server. You decide to close down the application. Now, what you want to happen is a graceful shutdown: you stop accepting new requests, but you run the current requests to an end (up to some timeout). In settings where the server count is very dynamic and servers can get "elastically" added or removed, this is important because otherwise you would be losing requests under shutdown. ? Observation: processes under supervision have an ordering imposed on them. Their termination happens in the opposite order of spawning. This can be used to enforce the gracefulness constraint. ? Another trick, which works cross-application, is to use the Module:prep_stop/1 phase of termination to tell your other applications (ranch, cowboy, yaws) that they should start graceful termination of their workers. Once completed, you can stop your own supervision tree. ? "Naked" processes which are not linked into a supervision tree can terminate in any order they see fit. In particular, the Erlang VM regards itself as being "done" once all its applications are shut down. In particular a line process with process_flag(trap_exit, true) waiting on disk is not going to be allowed to hold up termination of the VM. A naked process has its use, but one has to be extremely careful around them. If they start failing, you have no easy way to know what their relation is. ?-- J._______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] From zxq9@REDACTED Sat May 14 14:27:36 2016 From: zxq9@REDACTED (zxq9) Date: Sat, 14 May 2016 21:27:36 +0900 Subject: [erlang-questions] erlang:raise/3 considered harmful? In-Reply-To: <201605141148.u4EBmKGQ067045@pluto.hedeland.org> References: <201605141148.u4EBmKGQ067045@pluto.hedeland.org> Message-ID: <3083043.rFbQSKMz6Z@changa> On 2016?5?14? ??? 13:48:20 Per Hedeland wrote: > Hi, > > I happened to want something like erlang:raise/3 for a library function > - I didn't think it existed:-), but found that it does, although it > comes with a semi-dire warning. The loop/3 function below does exactly > what I want, specifically: if the fun provided by the caller of > interate/3 crashes, the caller gets the same exception as if there had > been no try ... catch, can handle it with hir own try/catch or not, but > in any case receives full information about *what* went wrong in hir fun > - all while the "protocol" spoken over the socket remains in sync. > > So, to take heed of the warning - is there a way to write an equivalent > function *without* using raise/3 - or should it just be considered a > case of "you really know what you are doing"? I believe I am of > course:-), but I also think that the code below is a perfectly > reasonable implementation of a side-effecty library function, and has no > more to do with debugging or knowing what you're doing than any other > code... Two things: 1- Exceptions do indeed suck -- there is usually a better way to deal with errors in almost every case. Once you start using them there is often a strong urge to accidentally start writing imperative Erlang, and that gets so ridiculous so quickly that you'll start hating your project. 2- In Erlang I tend to not keep lambdas around for very long as labeled objects (like keeping them in main loop state as in this example). The solution? *Usually* this purpose can be better served by changing this loop into a primary/callback module pair, where you have passed the callback module name in instead of a specific function. This can be as small or as large as you want, and Erlang has a special syntax to help give you warnings at compile-time by creating behaviors -- if you haven't done this before, it can be very rewarding. (The other way is to have an initiator call a function defined in some module and have the module define its own loop -- the example provided does not include enough boilerplate/"system"-handling type code to discourage me from writing a custom module to do whatever Fun is doing there.) This principle is the whole basis for much of OTP, actually, and is a structure you will almost certainly wind up re-inventing yourself given enough time (I've re-invented OTP without realizing it at least 5 times in different languages and environments... before I realized what was happening.) If you have some specific reason a primary/callbacks structure can't work please detail it and let's see if it can be either worked around cleanly, or if some other approach is really required. -Craig From per@REDACTED Sat May 14 16:13:41 2016 From: per@REDACTED (Per Hedeland) Date: Sat, 14 May 2016 16:13:41 +0200 (CEST) Subject: [erlang-questions] erlang:raise/3 considered harmful? In-Reply-To: <3083043.rFbQSKMz6Z@changa> Message-ID: <201605141413.u4EEDfKl099963@pluto.hedeland.org> Thanks for your reply. It's always difficult to put enough info in a post while still avoiding the TL;DR syndrome, and while it is indeed often the case that the right answer is to a different question than the one that was originally asked, I'm afraid you're completely missing my point. The interface/3 is a library function, part of a larger library (and the interface of the library isn't up for discussion), and loop/3 is just the traditional we-don't-have-loops-so-I-need-a-recursive-function helper, not a "main loop" in any way. Think lists(3), some of the functions there take a fun, some don't, and I don't think I've ever heard the suggestion that it would be a better design if you had to implement a behavior with callbacks in order to use the currently fun-taking functions... And the question also isn't about *using* exceptions - just the fact that they happen due to bugs, and that it should be possible for the library user to handle the exceptions that happen due to bugs in hir fun, even if only by getting a useful crash report from them. The lists module doesn't have the problem I want to solve, since there are no side effects and thus no need for "cleanup" in the case of an exception - it can (and does) just "let it crash", but that isn't acceptable in my case - I need the "cleanup" to happen. Bottom line, the only question here really *is* the one I asked below. --Per zxq9 wrote: > >On 2016???5???14??? ????????? 13:48:20 Per Hedeland wrote: >> Hi, >> >> I happened to want something like erlang:raise/3 for a library function >> - I didn't think it existed:-), but found that it does, although it >> comes with a semi-dire warning. The loop/3 function below does exactly >> what I want, specifically: if the fun provided by the caller of >> interate/3 crashes, the caller gets the same exception as if there had >> been no try ... catch, can handle it with hir own try/catch or not, but >> in any case receives full information about *what* went wrong in hir fun >> - all while the "protocol" spoken over the socket remains in sync. >> >> So, to take heed of the warning - is there a way to write an equivalent >> function *without* using raise/3 - or should it just be considered a >> case of "you really know what you are doing"? I believe I am of >> course:-), but I also think that the code below is a perfectly >> reasonable implementation of a side-effecty library function, and has no >> more to do with debugging or knowing what you're doing than any other >> code... > >Two things: > >1- Exceptions do indeed suck -- there is usually a better way to deal > with errors in almost every case. Once you start using them there is > often a strong urge to accidentally start writing imperative Erlang, > and that gets so ridiculous so quickly that you'll start hating your > project. >2- In Erlang I tend to not keep lambdas around for very long as labeled > objects (like keeping them in main loop state as in this example). > >The solution? > >*Usually* this purpose can be better served by changing this loop into >a primary/callback module pair, where you have passed the callback module >name in instead of a specific function. This can be as small or as large >as you want, and Erlang has a special syntax to help give you warnings >at compile-time by creating behaviors -- if you haven't done this before, >it can be very rewarding. (The other way is to have an initiator call a >function defined in some module and have the module define its own loop -- >the example provided does not include enough boilerplate/"system"-handling >type code to discourage me from writing a custom module to do whatever Fun >is doing there.) > >This principle is the whole basis for much of OTP, actually, and is a >structure you will almost certainly wind up re-inventing yourself given >enough time (I've re-invented OTP without realizing it at least 5 times >in different languages and environments... before I realized what was >happening.) > >If you have some specific reason a primary/callbacks structure can't work >please detail it and let's see if it can be either worked around cleanly, >or if some other approach is really required. > >-Craig >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions > From g@REDACTED Sat May 14 17:11:14 2016 From: g@REDACTED (Garrett Smith) Date: Sat, 14 May 2016 08:11:14 -0700 Subject: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) In-Reply-To: References: Message-ID: On Sat, May 14, 2016 at 5:21 AM, Oliver Korpilla wrote: > Hello, > > and thank you all for your responses. > > I originally adopted simple_one_for_one supervisor because I had a problem with how other supervisors clean up processes. > > For the TCP connectors simple_one_for_one will be fine. As noted by others, they cannot really come back unless they reconnect, so that is fine. So, a simple_one_for_one supervisor acts like every child, regardless of child spec, as if it was temporary? > > I have another big batch of processes independent of the connectors. These serve individual requests emanating from the TCP layer, where an ID establishes which handler belongs to which batch of messages (i.e. each TCP payload contains an ID in its own proprietary header). Now, I originally saw these as transient workers I would like to have restarted, but since they are stateless and can be created on demand, I either can supervise them simple_one_for_one (and create them on demand when the one for a given ID is missing) or I can create them as transient children under a one_for_one and let that restart it on a crash. If these processes only ever act on behalf of the TCP connection, consider not using them at all. Just let the TCP connections do the work. Processes should correspond to _real world_ independent threads of execution, not mental abstractions. If you do have separate threads of execution (e.g. TCP connection is providing updates to the client while it waits on these spawned workers) use a separate simple_one_for_one (sofo) supervisor for the workers and link your connection/worker processes. > I originally went for simple_one_for_one because of the better performance and because it cleans up children after they terminate. I guess in case of one_for_one I have to clean up all children which shut down normally by calling terminate_child and delete_child on them. (I originally hoped one_for_one would do this if a child exited normally, but either I bungled my tests or it simply doesn't, even for transient children). If you're ever routinely "cleaning up" after a supervisor, it's a bad sign. Configure (one-time init payload) your supervisors and let them do their thing. If you're accumulating a lot of terminated child processes, you want a sofo supervisor. > Any recommendations? It sounds like you're motivated to get a "restart" scenario here. What is your goal from the end-user (client of your app) point of view here? Without a specific goal that you understand and can defend, your default approach I think is always crash - and let the client reestablish a connection. Some worthy goals: - Don't abruptly close the connection but return a well formed error (e.g. HTTP 500, etc.) - Handle specific well understood error conditions with limited retries (e.g. reconnect to a database with the hope the outage is short term) - Tell the client to retry a different end-point (e.g. HTTP 302) Each of these needs goals needs to be implemented - you're not going to get any of them with a supervisor process restart. Short of a worthy goal, just crash, maintaining your system integrity for processing new connections, and rely on the client (outside your system) to perform the "restart". From Oliver.Korpilla@REDACTED Sat May 14 18:15:47 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Sat, 14 May 2016 18:15:47 +0200 Subject: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) In-Reply-To: References: , Message-ID: An HTML attachment was scrubbed... URL: From g@REDACTED Sat May 14 18:19:51 2016 From: g@REDACTED (Garrett Smith) Date: Sat, 14 May 2016 09:19:51 -0700 Subject: [erlang-questions] erlang:raise/3 considered harmful? In-Reply-To: <201605141148.u4EBmKGQ067045@pluto.hedeland.org> References: <201605141148.u4EBmKGQ067045@pluto.hedeland.org> Message-ID: On Sat, May 14, 2016 at 4:48 AM, Per Hedeland wrote: > Hi, > > I happened to want something like erlang:raise/3 for a library function > - I didn't think it existed:-), but found that it does, although it > comes with a semi-dire warning. The loop/3 function below does exactly > what I want, specifically: if the fun provided by the caller of > interate/3 crashes, the caller gets the same exception as if there had > been no try ... catch, can handle it with hir own try/catch or not, but > in any case receives full information about *what* went wrong in hir fun > - all while the "protocol" spoken over the socket remains in sync. > > So, to take heed of the warning - is there a way to write an equivalent > function *without* using raise/3 - or should it just be considered a > case of "you really know what you are doing"? I believe I am of > course:-), but I also think that the code below is a perfectly > reasonable implementation of a side-effecty library function, and has no > more to do with debugging or knowing what you're doing than any other > code... I think your case justifies the use of this scary function :) That said, I don't think you actually want to use it. Your API need improving I think. The fact you're returning a tagged tuple {ok, State} puts this into the "tagged tuple" function category. This in contrast to the "value vs exception" category. See dict:find vs dict:fetch for a contrast of the two patterns. Stick with one or the other - don't mingle them. So I'd expect something like this: try Fun(Data, State) catch Class:Reason -> Err = {Class, Reason, erlang:get_stacktrace()} {error, {callback_error, Err}} end But onto other issues... It's not clear what "Error" is from looking at the code - and I'd rather not (as a reader) have to poke around type specs. If you expect the form {error, Error} there, spell that out. E.g. case Res of ... {error, Error} -> gen_tcp:send(Sock, "done"), {error, Error} If "Error" can be anything, you're on shaky ground API wise. You're asking the caller of your function to handle {ok, Good}, arbitrary exceptions, and arbitrary return values. I would be very cranky with this interface. Finally, the imperative style coding here really wants some functions, in particular the calling and handling of the callback (but also applies to the calling and handling of the socket read). I'd do something like this: handle_cb(apply_cb(Fun, Data, State), Socket, Fun) apply_cb can then tuck away your logic for how to call this function. I'd typically use "catch" (I don't care for try...catch), but this would allow "throw" errors to pass through as results. You may or may not want that - I would not want that certainly without a strong defense. So this: apply_cb(Fun, Data, State) -> try Fun(Data, State) catch Class:Reason -> {error, {cb_error, {Class, Reason, erlang:get_stacktract()}}} end. Your handler then replaces your case expression: handle_cb({continue, State}, Sock, Fun) -> gen_tcp:send(Sock, "continue"), loop(Sock, Fun, State); handle_cb({done, State}, Sock, Fun) -> gen_tcp:send(Sock, "done"), {ok, State}; handle_cb({error, Err}, Sock, _Fun) -> gen_tcp:send(Sock, "done"), {error, Error}; handle_db(Other, Sock, _Fun) -> gen_tcp:send(Sock, "done"), error({unexpected_cb_result, Other}). You get a lot of goodness here: - Clearly separated concerns, each with its own name+args (i.e. function) - Consistent use of tagged tuple pattern ({ok, Good}, {error, Bad}) - Explicit error information with enough detail to diagnose errors - Intolerance and refusal to handle a misuse of this API (the error for unexpected cb result) And then just to see it in print: loop(Sock, Fun, State) -> handle_data(recv_data(Sock), Sock, Fun, State). You get the idea. And I fully understand, this was not your original question :) From g@REDACTED Sat May 14 18:32:39 2016 From: g@REDACTED (Garrett Smith) Date: Sat, 14 May 2016 09:32:39 -0700 Subject: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) In-Reply-To: References: Message-ID: I trust your judgement on the need for separate processes :) But complexity doesn't justify a _process_ - just a relatively more complex function. If the real world entities being served map to independent _threads of execution_ in time and space, process. If not, consider not a process. I only underscore this because it's not uncommon to see folks use processes (a runtime construct) used to model abstract programming logic (a design time construct at best, often simply an emotional/mental fancy). Not suggesting you fall into that category, just a highlight ;) On Sat, May 14, 2016 at 9:15 AM, Oliver Korpilla wrote: > Hello, Garrett. > > The TCP layer is a stand-in for a real, more complex protocol stack with > very different characteristics. I assure you the abstractions are absolutely > necessary and map real-world entities being served in a system quite more > complex than a TCP server. While the project is currently at the scale of a > technology demonstration it is supposed to grow into a full-fledged > application. Sorry I have to be so vague. > > Relying on the client for retries might work. My client handlers are > "stateless" in that they can come back at any time from the DB and serve a > request, even if they have to tell the client to abort its current > operation. I have to investigate the behavior of the given clients more. > That's likely the real solution. > > Thanks! > Oliver > > Gesendet: Samstag, 14. Mai 2016 um 17:11 Uhr > Von: "Garrett Smith" > An: "Oliver Korpilla" > Cc: "Jesper Louis Andersen" , > "Erlang-Questions Questions" > Betreff: Re: [erlang-questions] simple_one_for_one supervisor - what happens > at restart? (also: gen_tcp) > On Sat, May 14, 2016 at 5:21 AM, Oliver Korpilla > wrote: >> Hello, >> >> and thank you all for your responses. >> >> I originally adopted simple_one_for_one supervisor because I had a problem >> with how other supervisors clean up processes. >> >> For the TCP connectors simple_one_for_one will be fine. As noted by >> others, they cannot really come back unless they reconnect, so that is fine. >> So, a simple_one_for_one supervisor acts like every child, regardless of >> child spec, as if it was temporary? >> >> I have another big batch of processes independent of the connectors. These >> serve individual requests emanating from the TCP layer, where an ID >> establishes which handler belongs to which batch of messages (i.e. each TCP >> payload contains an ID in its own proprietary header). Now, I originally saw >> these as transient workers I would like to have restarted, but since they >> are stateless and can be created on demand, I either can supervise them >> simple_one_for_one (and create them on demand when the one for a given ID is >> missing) or I can create them as transient children under a one_for_one and >> let that restart it on a crash. > > If these processes only ever act on behalf of the TCP connection, > consider not using them at all. Just let the TCP connections do the > work. > > Processes should correspond to _real world_ independent threads of > execution, not mental abstractions. > > If you do have separate threads of execution (e.g. TCP connection is > providing updates to the client while it waits on these spawned > workers) use a separate simple_one_for_one (sofo) supervisor for the > workers and link your connection/worker processes. > >> I originally went for simple_one_for_one because of the better performance >> and because it cleans up children after they terminate. I guess in case of >> one_for_one I have to clean up all children which shut down normally by >> calling terminate_child and delete_child on them. (I originally hoped >> one_for_one would do this if a child exited normally, but either I bungled >> my tests or it simply doesn't, even for transient children). > > If you're ever routinely "cleaning up" after a supervisor, it's a bad > sign. Configure (one-time init payload) your supervisors and let them > do their thing. If you're accumulating a lot of terminated child > processes, you want a sofo supervisor. > >> Any recommendations? > > It sounds like you're motivated to get a "restart" scenario here. What > is your goal from the end-user (client of your app) point of view > here? Without a specific goal that you understand and can defend, your > default approach I think is always crash - and let the client > reestablish a connection. > > Some worthy goals: > > - Don't abruptly close the connection but return a well formed error > (e.g. HTTP 500, etc.) > - Handle specific well understood error conditions with limited > retries (e.g. reconnect to a database with the hope the outage is > short term) > - Tell the client to retry a different end-point (e.g. HTTP 302) > > Each of these needs goals needs to be implemented - you're not going > to get any of them with a supervisor process restart. Short of a > worthy goal, just crash, maintaining your system integrity for > processing new connections, and rely on the client (outside your > system) to perform the "restart". From zzantozz@REDACTED Sat May 14 18:57:23 2016 From: zzantozz@REDACTED (Ryan Stewart) Date: Sat, 14 May 2016 16:57:23 +0000 Subject: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) In-Reply-To: <5736E1B4.80106@ninenines.eu> References: <5736E1B4.80106@ninenines.eu> Message-ID: On Sat, May 14, 2016 at 3:28 AM Lo?c Hoguin wrote: > The primary function of supervisors is to provide a hierarchy of > processes belonging to an application. With such a hierarchy it becomes > possible to find and query information about any single process in your > system in a standard manner. > > I'm rather interested in this aspect of supervised processes. Especially in the case of a simple_one_for_one supervisor, it's unlikely that the supervised processes will be registered, and it's possible that there could be a rather large number of them--maybe in the tens or hundreds of thousands, depending on the use case. I'm curious how others deal with finding a specific temporary process if, for instance, you want to check on the progress of the work it's doing. My current solution is to have a locally registered "manager" process as a supervised sibling to the SOFO worker supervisor, and the manager just has a dict that maps UUIDs to worker PIDs. I.e. creating a "worker" process entails both a supervisor:start_child() call and storing the worker id -> pid mapping in the manager. Is this a typical way to handle temporary workers? -------------- next part -------------- An HTML attachment was scrubbed... URL: From zzantozz@REDACTED Sat May 14 19:03:52 2016 From: zzantozz@REDACTED (Ryan Stewart) Date: Sat, 14 May 2016 17:03:52 +0000 Subject: [erlang-questions] erlang:raise/3 considered harmful? In-Reply-To: References: <201605141148.u4EBmKGQ067045@pluto.hedeland.org> Message-ID: +1 for "exceptions suck and lead to imperative programming patterns". That said, I'm still trying to work out the Right Way to do error handling in Erlang. On a quick scan, I like Garrett's suggestions, although I would clarify it by saying that like dict:find() vs dict:fetch(), it might be reasonable for your API to support both a "crash on failure" and a "return tagged tuple on failure" option. That gives callers more freedom in how they use your code. To me, this feels like the age-old "checked vs unchecked" exception problem, and that's all about how you force callers to interact with your code. On Sat, May 14, 2016 at 11:20 AM Garrett Smith wrote: > On Sat, May 14, 2016 at 4:48 AM, Per Hedeland wrote: > > Hi, > > > > I happened to want something like erlang:raise/3 for a library function > > - I didn't think it existed:-), but found that it does, although it > > comes with a semi-dire warning. The loop/3 function below does exactly > > what I want, specifically: if the fun provided by the caller of > > interate/3 crashes, the caller gets the same exception as if there had > > been no try ... catch, can handle it with hir own try/catch or not, but > > in any case receives full information about *what* went wrong in hir fun > > - all while the "protocol" spoken over the socket remains in sync. > > > > So, to take heed of the warning - is there a way to write an equivalent > > function *without* using raise/3 - or should it just be considered a > > case of "you really know what you are doing"? I believe I am of > > course:-), but I also think that the code below is a perfectly > > reasonable implementation of a side-effecty library function, and has no > > more to do with debugging or knowing what you're doing than any other > > code... > > I think your case justifies the use of this scary function :) > > That said, I don't think you actually want to use it. Your API need > improving I think. > > The fact you're returning a tagged tuple {ok, State} puts this into > the "tagged tuple" function category. This in contrast to the "value > vs exception" category. See dict:find vs dict:fetch for a contrast of > the two patterns. Stick with one or the other - don't mingle them. > > So I'd expect something like this: > > try > Fun(Data, State) > catch > Class:Reason -> > Err = {Class, Reason, erlang:get_stacktrace()} > {error, {callback_error, Err}} > end > > But onto other issues... > > It's not clear what "Error" is from looking at the code - and I'd > rather not (as a reader) have to poke around type specs. If you expect > the form {error, Error} there, spell that out. E.g. > > case Res of > ... > {error, Error} -> > gen_tcp:send(Sock, "done"), > {error, Error} > > If "Error" can be anything, you're on shaky ground API wise. You're > asking the caller of your function to handle {ok, Good}, arbitrary > exceptions, and arbitrary return values. I would be very cranky with > this interface. > > Finally, the imperative style coding here really wants some functions, > in particular the calling and handling of the callback (but also > applies to the calling and handling of the socket read). I'd do > something like this: > > handle_cb(apply_cb(Fun, Data, State), Socket, Fun) > > apply_cb can then tuck away your logic for how to call this function. > I'd typically use "catch" (I don't care for try...catch), but this > would allow "throw" errors to pass through as results. You may or may > not want that - I would not want that certainly without a strong > defense. So this: > > apply_cb(Fun, Data, State) -> > try > Fun(Data, State) > catch > Class:Reason -> > {error, {cb_error, {Class, Reason, erlang:get_stacktract()}}} > end. > > Your handler then replaces your case expression: > > handle_cb({continue, State}, Sock, Fun) -> > gen_tcp:send(Sock, "continue"), > loop(Sock, Fun, State); > handle_cb({done, State}, Sock, Fun) -> > gen_tcp:send(Sock, "done"), > {ok, State}; > handle_cb({error, Err}, Sock, _Fun) -> > gen_tcp:send(Sock, "done"), > {error, Error}; > handle_db(Other, Sock, _Fun) -> > gen_tcp:send(Sock, "done"), > error({unexpected_cb_result, Other}). > > You get a lot of goodness here: > > - Clearly separated concerns, each with its own name+args (i.e. function) > - Consistent use of tagged tuple pattern ({ok, Good}, {error, Bad}) > - Explicit error information with enough detail to diagnose errors > - Intolerance and refusal to handle a misuse of this API (the error > for unexpected cb result) > > And then just to see it in print: > > loop(Sock, Fun, State) -> > handle_data(recv_data(Sock), Sock, Fun, State). > > You get the idea. > > And I fully understand, this was not your original question :) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Oliver.Korpilla@REDACTED Sat May 14 19:16:35 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Sat, 14 May 2016 19:16:35 +0200 Subject: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) In-Reply-To: References: <5736E1B4.80106@ninenines.eu>, Message-ID: Hello, Ryan. Isn't this normally the job of a registry like gproc or am I misunderstanding your requirements? I usually start the worker dynamically through the supervisor and let it do its own registration. Cheers, Oliver ? Gesendet:?Samstag, 14. Mai 2016 um 18:57 Uhr Von:?"Ryan Stewart" An:?"Lo?c Hoguin" , Chandru Cc:?"Erlang-Questions Questions" Betreff:?Re: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) On Sat, May 14, 2016 at 3:28 AM Lo?c Hoguin wrote:The primary function of supervisors is to provide a hierarchy of processes belonging to an application. With such a hierarchy it becomes possible to find and query information about any single process in your system in a standard manner. ? I'm rather interested in this aspect of supervised processes. Especially in the case of a simple_one_for_one supervisor, it's unlikely that the supervised processes will be registered, and it's possible that there could be a rather large number of them--maybe in the tens or hundreds of thousands, depending on the use case.?I'm curious how others deal with finding a specific temporary process if, for instance, you want to check on the progress of the work it's doing. ? My current solution is to have a locally registered "manager" process as a supervised sibling to the SOFO worker supervisor, and the manager just has a dict that maps UUIDs to worker PIDs. I.e. creating a "worker" process entails both a supervisor:start_child() call and storing the worker id -> pid mapping in the manager. Is this a typical way to handle temporary workers?_______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] From zzantozz@REDACTED Sat May 14 19:24:03 2016 From: zzantozz@REDACTED (Ryan Stewart) Date: Sat, 14 May 2016 17:24:03 +0000 Subject: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) In-Reply-To: References: <5736E1B4.80106@ninenines.eu> Message-ID: Oliver, gproc looks exactly right; thanks for the pointer! I just didn't know about it, so essentially I was building my own with very limited features. Pardon my ignorance. I've been working in Erlang for a little over 2 years, and I only just feel like I have a handle on how everything fits together. I still don't know what all libraries are out there to do common jobs. Hmm, that seems like a good subject for a fresh thread... On Sat, May 14, 2016 at 12:16 PM Oliver Korpilla wrote: > Hello, Ryan. > > Isn't this normally the job of a registry like gproc or am I > misunderstanding your requirements? > > I usually start the worker dynamically through the supervisor and let it > do its own registration. > > Cheers, > Oliver > > > Gesendet: Samstag, 14. Mai 2016 um 18:57 Uhr > Von: "Ryan Stewart" > An: "Lo?c Hoguin" , Chandru < > chandrashekhar.mullaparthi@REDACTED> > Cc: "Erlang-Questions Questions" > Betreff: Re: [erlang-questions] simple_one_for_one supervisor - what > happens at restart? (also: gen_tcp) > > On Sat, May 14, 2016 at 3:28 AM Lo?c Hoguin > wrote:The primary function of supervisors is to provide a hierarchy of > processes belonging to an application. With such a hierarchy it becomes > possible to find and query information about any single process in your > system in a standard manner. > > I'm rather interested in this aspect of supervised processes. Especially > in the case of a simple_one_for_one supervisor, it's unlikely that the > supervised processes will be registered, and it's possible that there could > be a rather large number of them--maybe in the tens or hundreds of > thousands, depending on the use case. I'm curious how others deal with > finding a specific temporary process if, for instance, you want to check on > the progress of the work it's doing. > > My current solution is to have a locally registered "manager" process as a > supervised sibling to the SOFO worker supervisor, and the manager just has > a dict that maps UUIDs to worker PIDs. I.e. creating a "worker" process > entails both a supervisor:start_child() call and storing the worker id -> > pid mapping in the manager. Is this a typical way to handle temporary > workers?_______________________________________________ erlang-questions > mailing list erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zzantozz@REDACTED Sat May 14 19:29:33 2016 From: zzantozz@REDACTED (Ryan Stewart) Date: Sat, 14 May 2016 17:29:33 +0000 Subject: [erlang-questions] Third party libs you always use Message-ID: I've got about 2 years of Erlang work under my belt, and I come from a strong JVM background before that. On the JVM, there are a certain set of third-party libraries that I would almost always include when starting a new project, just because they add so many helpful things that aren't in the core language. Then, depending on the nature of the project, others might get added for HTTP stuff, messaging, or other concerns that need to be dealt with. I'm still learning a lot about the Erlang ecosystem and what sort of library support is out there, so my question to the Erlang experts out there is this: Q: Are there third-party libraries that you almost automatically include when you start or become involved with an Erlang project? Or to put it another way: Q: What are your go-to libraries for solving common problems in an Erlang project? -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Sat May 14 19:35:47 2016 From: sergej.jurecko@REDACTED (=?UTF-8?Q?Sergej_Jure=C4=8Dko?=) Date: Sat, 14 May 2016 19:35:47 +0200 Subject: [erlang-questions] Third party libs you always use In-Reply-To: References: Message-ID: lager Regards, Sergej On May 14, 2016 7:29 PM, "Ryan Stewart" wrote: > I've got about 2 years of Erlang work under my belt, and I come from a > strong JVM background before that. On the JVM, there are a certain set of > third-party libraries that I would almost always include when starting a > new project, just because they add so many helpful things that aren't in > the core language. Then, depending on the nature of the project, others > might get added for HTTP stuff, messaging, or other concerns that need to > be dealt with. I'm still learning a lot about the Erlang ecosystem and what > sort of library support is out there, so my question to the Erlang experts > out there is this: > > Q: Are there third-party libraries that you almost automatically include > when you start or become involved with an Erlang project? > > Or to put it another way: > > Q: What are your go-to libraries for solving common problems in an Erlang > project? > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cian@REDACTED Sat May 14 19:36:34 2016 From: cian@REDACTED (Cian Synnott) Date: Sat, 14 May 2016 18:36:34 +0100 Subject: [erlang-questions] Third party libs you always use In-Reply-To: References: Message-ID: On Sat, May 14, 2016 at 6:29 PM, Ryan Stewart wrote: > Q: Are there third-party libraries that you almost automatically include > when you start or become involved with an Erlang project? > http://ferd.github.io/recon/ (Cf. https://www.erlang-in-anger.com/) https://github.com/Feuerlabs/exometer https://github.com/basho/lager https://github.com/basho/cuttlefish https://github.com/basho/node_package Very often Cowboy or Ranch depending on what I'm doing. Cian From jargon@REDACTED Sat May 14 20:07:42 2016 From: jargon@REDACTED (Johannes =?utf-8?B?V2Vpw59s?=) Date: Sat, 14 May 2016 20:07:42 +0200 Subject: [erlang-questions] SSL problems with OTP 19 Message-ID: <20160514180742.GA21998@molb.org> Hello, We let our test suite run daily against the current OTP master branch. Since May 5th some tests using the ssl application and inets httpc fail due to an (infinite) timeout. Through "git bisect" I found out that one of the following two commits is responsible: https://github.com/erlang/otp/commit/0f782eb6377ddd1e5673c186b8b1a37edd610e4b (ssl: Adapt DTLS to gen_statem) https://github.com/erlang/otp/commit/67de1ba82d1af910dc6ab6d41a1f22dbfb39f658 (ssl: Use gen_statem instead of gen_fsm) Maybe it has something to do with our usage of ssl:controlling_process/2. I will continue to debug the problem, but wanted to check if the problem is known or if anybody else encountered similar problems? Regards, Johannes From mjtruog@REDACTED Sat May 14 20:50:41 2016 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 14 May 2016 11:50:41 -0700 Subject: [erlang-questions] SSL problems with OTP 19 In-Reply-To: <20160514180742.GA21998@molb.org> References: <20160514180742.GA21998@molb.org> Message-ID: <57377381.20208@gmail.com> On 05/14/2016 11:07 AM, Johannes Wei?l wrote: > Hello, > > We let our test suite run daily against the current OTP master branch. > Since May 5th some tests using the ssl application and inets httpc > fail due to an (infinite) timeout. > > Through "git bisect" I found out that one of the following two commits > is responsible: > > https://github.com/erlang/otp/commit/0f782eb6377ddd1e5673c186b8b1a37edd610e4b (ssl: Adapt DTLS to gen_statem) > https://github.com/erlang/otp/commit/67de1ba82d1af910dc6ab6d41a1f22dbfb39f658 (ssl: Use gen_statem instead of gen_fsm) > > Maybe it has something to do with our usage of ssl:controlling_process/2. > > I will continue to debug the problem, but wanted to check if the problem > is known or if anybody else encountered similar problems? I filed a bug for this at http://bugs.erlang.org/browse/ERL-144 due to it preventing the rebar3 bootstrap script from running. > > Regards, > Johannes > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From fernando.benavides@REDACTED Sat May 14 23:07:29 2016 From: fernando.benavides@REDACTED (Brujo Benavides) Date: Sat, 14 May 2016 18:07:29 -0300 Subject: [erlang-questions] Third party libs you always use In-Reply-To: References: Message-ID: Eper (mostly just because of redbug) Sync Katana-test For Web services: Cowboy-swagger Mixer Cheers On May 14, 2016 2:36 PM, "Cian Synnott" wrote: > On Sat, May 14, 2016 at 6:29 PM, Ryan Stewart wrote: > > Q: Are there third-party libraries that you almost automatically include > > when you start or become involved with an Erlang project? > > > http://ferd.github.io/recon/ (Cf. https://www.erlang-in-anger.com/) > https://github.com/Feuerlabs/exometer > https://github.com/basho/lager > https://github.com/basho/cuttlefish > https://github.com/basho/node_package > Very often Cowboy or Ranch depending on what I'm doing. > > Cian > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethlakin@REDACTED Sat May 14 23:19:18 2016 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Sat, 14 May 2016 14:19:18 -0700 Subject: [erlang-questions] How to disable parentheses matching in Erlang shell? In-Reply-To: References: <6013b959-58b0-a963-9478-d6845993f024@gjunka.com> Message-ID: <57379656.6010306@gmail.com> On 05/14/2016 03:32 AM, Grzegorz Junka wrote: > Fixing my terminal to 80 characters only seems like taken straight from > the Windows world. I hope Erlang can do better ;) If ncurses can't determine your terminal size, then falling back to 80x24 makes a lot of sense. Do other ncurses-based programs behave incorrectly? > Is there any way of > telling the Erlang terminal how wide is my terminal, e.g. an environment > variable? On my system, it looks like the environment variables LINES and COLUMNS correspond to the current size of my terminal window. When I set COLUMNS to a value that does not match the window's current size, then the Erlang shell's paren matching bounces the cursor backwards to the wrong place (but back forwards to the right place). Maybe this will be the band-aid you need. However, it might be more useful to fix whatever is preventing ncurses from determining the terminal size. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From list1@REDACTED Sun May 15 00:39:23 2016 From: list1@REDACTED (Grzegorz Junka) Date: Sat, 14 May 2016 22:39:23 +0000 Subject: [erlang-questions] How to disable parentheses matching in Erlang shell? In-Reply-To: <57379656.6010306@gmail.com> References: <6013b959-58b0-a963-9478-d6845993f024@gjunka.com> <57379656.6010306@gmail.com> Message-ID: <5481da92-8dbb-750e-8ced-c5ffafd2e4ce@gjunka.com> On 14/05/2016 21:19, Kenneth Lakin wrote: > On 05/14/2016 03:32 AM, Grzegorz Junka wrote: >> Fixing my terminal to 80 characters only seems like taken straight from >> the Windows world. I hope Erlang can do better ;) > If ncurses can't determine your terminal size, then falling back to > 80x24 makes a lot of sense. Do other ncurses-based programs behave > incorrectly? No. Everything else is fine. This problem is not related to my system, as I have seen the same behaviour in Erlang running on FreeBSD, Ubuntu, Red Hat and CentOS in different terminals, like putty, cygwin and Konsole. >> Is there any way of >> telling the Erlang terminal how wide is my terminal, e.g. an environment >> variable? > On my system, it looks like the environment variables LINES and COLUMNS > correspond to the current size of my terminal window. When I set COLUMNS > to a value that does not match the window's current size, then the > Erlang shell's paren matching bounces the cursor backwards to the wrong > place (but back forwards to the right place). Maybe this will be the > band-aid you need. I don't have such environment variables in my system and I don't remember having them on Ubuntu or Red Hat (however I can't double-check that now) > However, it might be more useful to fix whatever is preventing ncurses > from determining the terminal size. Yeah, and even more useful if the Erlang shell allowed me to disable a feature that isn't really necessary especially if it isn't working correctly. From joshmbarney@REDACTED Sun May 15 00:49:38 2016 From: joshmbarney@REDACTED (Joshua Barney) Date: Sat, 14 May 2016 18:49:38 -0400 Subject: [erlang-questions] How to disable parentheses matching in Erlang shell? In-Reply-To: <5481da92-8dbb-750e-8ced-c5ffafd2e4ce@gjunka.com> References: <6013b959-58b0-a963-9478-d6845993f024@gjunka.com> <57379656.6010306@gmail.com> <5481da92-8dbb-750e-8ced-c5ffafd2e4ce@gjunka.com> Message-ID: It seems unlikely that erlang shell is broken only for you on all these systems. Have you checked that your shell scripts are not setting COLUMNS? Perhaps you are ssh-ing from a client that is doing something not quite right? Sent from my iPhone > On May 14, 2016, at 6:39 PM, Grzegorz Junka wrote: > > >> On 14/05/2016 21:19, Kenneth Lakin wrote: >>> On 05/14/2016 03:32 AM, Grzegorz Junka wrote: >>> Fixing my terminal to 80 characters only seems like taken straight from >>> the Windows world. I hope Erlang can do better ;) >> If ncurses can't determine your terminal size, then falling back to >> 80x24 makes a lot of sense. Do other ncurses-based programs behave >> incorrectly? > > No. Everything else is fine. This problem is not related to my system, as I have seen the same behaviour in Erlang running on FreeBSD, Ubuntu, Red Hat and CentOS in different terminals, like putty, cygwin and Konsole. > >>> Is there any way of >>> telling the Erlang terminal how wide is my terminal, e.g. an environment >>> variable? >> On my system, it looks like the environment variables LINES and COLUMNS >> correspond to the current size of my terminal window. When I set COLUMNS >> to a value that does not match the window's current size, then the >> Erlang shell's paren matching bounces the cursor backwards to the wrong >> place (but back forwards to the right place). Maybe this will be the >> band-aid you need. > > I don't have such environment variables in my system and I don't remember having them on Ubuntu or Red Hat (however I can't double-check that now) > >> However, it might be more useful to fix whatever is preventing ncurses >> from determining the terminal size. > > Yeah, and even more useful if the Erlang shell allowed me to disable a feature that isn't really necessary especially if it isn't working correctly. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From g@REDACTED Sun May 15 02:41:56 2016 From: g@REDACTED (Garrett Smith) Date: Sat, 14 May 2016 19:41:56 -0500 Subject: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) In-Reply-To: References: <5736E1B4.80106@ninenines.eu> Message-ID: I'm glossing over these responses, but the points about gproc raises an issue I had with some earlier responses - that plugging a process into a supervisory hierarchy lets you somehow keep track of it. I don't think that's good advice - querying supervisors is a bad idea. If you want to track processes, use gproc. gproc should be in core IMO - at least the publish/discovery facility for processes. Though maybe I've missed a recent update. On Sat, May 14, 2016 at 12:24 PM, Ryan Stewart wrote: > Oliver, gproc looks exactly right; thanks for the pointer! I just didn't > know about it, so essentially I was building my own with very limited > features. Pardon my ignorance. I've been working in Erlang for a little over > 2 years, and I only just feel like I have a handle on how everything fits > together. I still don't know what all libraries are out there to do common > jobs. > > Hmm, that seems like a good subject for a fresh thread... > > On Sat, May 14, 2016 at 12:16 PM Oliver Korpilla > wrote: >> >> Hello, Ryan. >> >> Isn't this normally the job of a registry like gproc or am I >> misunderstanding your requirements? >> >> I usually start the worker dynamically through the supervisor and let it >> do its own registration. >> >> Cheers, >> Oliver >> >> >> Gesendet: Samstag, 14. Mai 2016 um 18:57 Uhr >> Von: "Ryan Stewart" >> An: "Lo?c Hoguin" , Chandru >> >> Cc: "Erlang-Questions Questions" >> Betreff: Re: [erlang-questions] simple_one_for_one supervisor - what >> happens at restart? (also: gen_tcp) >> >> On Sat, May 14, 2016 at 3:28 AM Lo?c Hoguin wrote:The >> primary function of supervisors is to provide a hierarchy of >> processes belonging to an application. With such a hierarchy it becomes >> possible to find and query information about any single process in your >> system in a standard manner. >> >> I'm rather interested in this aspect of supervised processes. Especially >> in the case of a simple_one_for_one supervisor, it's unlikely that the >> supervised processes will be registered, and it's possible that there could >> be a rather large number of them--maybe in the tens or hundreds of >> thousands, depending on the use case. I'm curious how others deal with >> finding a specific temporary process if, for instance, you want to check on >> the progress of the work it's doing. >> >> My current solution is to have a locally registered "manager" process as a >> supervised sibling to the SOFO worker supervisor, and the manager just has a >> dict that maps UUIDs to worker PIDs. I.e. creating a "worker" process >> entails both a supervisor:start_child() call and storing the worker id -> >> pid mapping in the manager. Is this a typical way to handle temporary >> workers?_______________________________________________ erlang-questions >> mailing list erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From akat.metin@REDACTED Sun May 15 08:37:07 2016 From: akat.metin@REDACTED (Metin Akat) Date: Sun, 15 May 2016 09:37:07 +0300 Subject: [erlang-questions] Third party libs you always use In-Reply-To: References: Message-ID: 1. sync - for automatic recompilation while developing 2. gproc - you almost always need process registration/locking 3. lager - logging 4. cowboy - as it almost always has some web interface 5. jsx - JSON encoding/decoding On Sun, May 15, 2016 at 12:07 AM, Brujo Benavides < fernando.benavides@REDACTED> wrote: > Eper (mostly just because of redbug) > Sync > Katana-test > > For Web services: > Cowboy-swagger > Mixer > > Cheers > On May 14, 2016 2:36 PM, "Cian Synnott" wrote: > >> On Sat, May 14, 2016 at 6:29 PM, Ryan Stewart wrote: >> > Q: Are there third-party libraries that you almost automatically include >> > when you start or become involved with an Erlang project? >> > >> http://ferd.github.io/recon/ (Cf. https://www.erlang-in-anger.com/) >> https://github.com/Feuerlabs/exometer >> https://github.com/basho/lager >> https://github.com/basho/cuttlefish >> https://github.com/basho/node_package >> Very often Cowboy or Ranch depending on what I'm doing. >> >> Cian >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From list1@REDACTED Sun May 15 14:00:09 2016 From: list1@REDACTED (Grzegorz Junka) Date: Sun, 15 May 2016 12:00:09 +0000 Subject: [erlang-questions] How to disable parentheses matching in Erlang shell? In-Reply-To: References: <6013b959-58b0-a963-9478-d6845993f024@gjunka.com> <57379656.6010306@gmail.com> <5481da92-8dbb-750e-8ced-c5ffafd2e4ce@gjunka.com> Message-ID: <03e2c191-5810-2b29-6dea-38cafaa7d2cb@gjunka.com> Joshua, This is what I said, it isn't broken only for me. I saw it being broken on other people's computers as well. It's unlikely for any scripts to set COLUMNS since I don't see that environment variable being set when running shell. I am running Konsole on the same system on which I am running Erlang, both on FreeBSD. I don't need to SSH anywhere. But the behaviour was the same no matter from where or to where I was SSH-ing or not (or someone I saw doing it), e.g.: Windows putty SSH to FreeBSD Windows putty SSH to Ubuntu Windows putty SSH to Red Hat Windows putty SSH to CentOS Windows xterm cygwin SSH to FreeBSD Windows xterm cygwin SSH To Ubuntu Windows xterm cygwin SSH to Red Hat Windows xterm cygwin SSH to CentOS FreeBSD xterm SSH to Ubuntu FreeBSD Konsole SSH to Ubuntu FreeBSD xterm SSH to FreeBSD FreeBSD Konsole SSH to Ubuntu FreeBSD xterm SSH to CentOS FreeBSD Konsole SSH to CentOS FreeBSD Konsole and Erlang shell on the same system (no SSH) FreeBSD xterm end Erlang shell on the same system (no SSH) Ubuntu SSH to CentOS It's fair to say that I haven't see a system on which this feature worked fine, I just learned to not type commands that are too long or copy-paste longer commands, and I bet most people who experience it do the same and don't bother complaining. I wonder if maybe this has something to do with escape codes and for example support for 256 colors in the terminal. Grzegorz On 14/05/2016 22:49, Joshua Barney wrote: > It seems unlikely that erlang shell is broken only for you on all these systems. > Have you checked that your shell scripts are not setting COLUMNS? Perhaps you are ssh-ing from a client that is doing something not quite right? > > Sent from my iPhone > >> On May 14, 2016, at 6:39 PM, Grzegorz Junka wrote: >> >> >>> On 14/05/2016 21:19, Kenneth Lakin wrote: >>>> On 05/14/2016 03:32 AM, Grzegorz Junka wrote: >>>> Fixing my terminal to 80 characters only seems like taken straight from >>>> the Windows world. I hope Erlang can do better ;) >>> If ncurses can't determine your terminal size, then falling back to >>> 80x24 makes a lot of sense. Do other ncurses-based programs behave >>> incorrectly? >> No. Everything else is fine. This problem is not related to my system, as I have seen the same behaviour in Erlang running on FreeBSD, Ubuntu, Red Hat and CentOS in different terminals, like putty, cygwin and Konsole. >> >>>> Is there any way of >>>> telling the Erlang terminal how wide is my terminal, e.g. an environment >>>> variable? >>> On my system, it looks like the environment variables LINES and COLUMNS >>> correspond to the current size of my terminal window. When I set COLUMNS >>> to a value that does not match the window's current size, then the >>> Erlang shell's paren matching bounces the cursor backwards to the wrong >>> place (but back forwards to the right place). Maybe this will be the >>> band-aid you need. >> I don't have such environment variables in my system and I don't remember having them on Ubuntu or Red Hat (however I can't double-check that now) >> >>> However, it might be more useful to fix whatever is preventing ncurses >>> from determining the terminal size. >> Yeah, and even more useful if the Erlang shell allowed me to disable a feature that isn't really necessary especially if it isn't working correctly. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From jesper.louis.andersen@REDACTED Sun May 15 15:07:40 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 15 May 2016 15:07:40 +0200 Subject: [erlang-questions] Third party libs you always use In-Reply-To: References: Message-ID: On Sat, May 14, 2016 at 7:29 PM, Ryan Stewart wrote: > Q: Are there third-party libraries that you almost automatically include > when you start or become involved with an Erlang project? Just mentioning the three most important: Feuerlabs Exometer_core - You need a way to add metrics into your system so you know what is going on. Usually, I'm willing to sacrifice a lot of raw processing power on a node if the upshot is I know what is going on inside the node. In distributed systems, much slowdown is not your own code, but the interaction with foreign code. Knowing what time properties the foreign systems have will make it much easier to pinpoint where an eventual problem is in your own system. Fred Hebert's Recon - when things start going wrong, it is nice to be able to trace the application. Nowadays, I add relatively little debug log statements. I just trace the system instead. Ulf Wiger's gproc (alternative: Ostinelli's syn) - gproc works around the limit of local process registration: you can only register an atom, not an arbitrary term. It also lets you subscribe to changes, which is really nice when a system tries to recover from failure. You can demand certain processes exist before you continue. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Sun May 15 16:13:58 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 16 May 2016 02:13:58 +1200 Subject: [erlang-questions] How to disable parentheses matching in Erlang shell? In-Reply-To: <03e2c191-5810-2b29-6dea-38cafaa7d2cb@gjunka.com> References: <6013b959-58b0-a963-9478-d6845993f024@gjunka.com> <57379656.6010306@gmail.com> <5481da92-8dbb-750e-8ced-c5ffafd2e4ce@gjunka.com> <03e2c191-5810-2b29-6dea-38cafaa7d2cb@gjunka.com> Message-ID: <39fd127a-f6ec-fb5d-9145-7c6107f22a2c@cs.otago.ac.nz> I've never noticed any problems with Erlang bracket matching in MacOS X with Terminal, or with Solaris accessed through MacOS X with Terminal. From vinoski@REDACTED Sun May 15 16:46:31 2016 From: vinoski@REDACTED (Steve Vinoski) Date: Sun, 15 May 2016 10:46:31 -0400 Subject: [erlang-questions] How to disable parentheses matching in Erlang shell? In-Reply-To: <39fd127a-f6ec-fb5d-9145-7c6107f22a2c@cs.otago.ac.nz> References: <6013b959-58b0-a963-9478-d6845993f024@gjunka.com> <57379656.6010306@gmail.com> <5481da92-8dbb-750e-8ced-c5ffafd2e4ce@gjunka.com> <03e2c191-5810-2b29-6dea-38cafaa7d2cb@gjunka.com> <39fd127a-f6ec-fb5d-9145-7c6107f22a2c@cs.otago.ac.nz> Message-ID: On Sun, May 15, 2016 at 10:13 AM, Richard A. O'Keefe wrote: > I've never noticed any problems with Erlang bracket matching > in MacOS X with Terminal, or with Solaris accessed through > MacOS X with Terminal. Ditto on OS X with both Terminal and iTerm2, and I've never seen problems on Linux either. --steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Sun May 15 17:24:06 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Sun, 15 May 2016 17:24:06 +0200 Subject: [erlang-questions] [poll] usage of R16 and erlide Message-ID: Hello everyone, This is only for people using or planning to use erlide (the Eclipse IDE, in case the name is not familiar) Since OTP 19 is soon to be here, I'd like to check the level of support for older releases that I need to keep for erlide. So: * Are you still targeting R16 or older? * If yes: * can you run v18+ on your development machine? * do you use the erlide debugger for the R16 code? * can you estimate for how much longer you will need to use R16? * Actually, it might be interesting to know the same information about v17 - in case everybody targets v18+, I could aim to do the same. Consider answering only to me, I don't know if the mailing list is very interested in the answers. If anyone wants to know the results, I can publish them later. Thank you very much! best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From per@REDACTED Sun May 15 17:35:36 2016 From: per@REDACTED (Per Hedeland) Date: Sun, 15 May 2016 17:35:36 +0200 (CEST) Subject: [erlang-questions] erlang:raise/3 considered harmful? In-Reply-To: Message-ID: <201605151535.u4FFZaUv039895@pluto.hedeland.org> Garrett Smith wrote: > >I think your case justifies the use of this scary function :) Thanks.:-) Though given the other things that you wrote, I have to wonder *why* you think so... >That said, I don't think you actually want to use it. Your API need >improving I think. > >The fact you're returning a tagged tuple {ok, State} puts this into >the "tagged tuple" function category. This in contrast to the "value >vs exception" category. See dict:find vs dict:fetch for a contrast of >the two patterns. Stick with one or the other - don't mingle them. (Hm, I guess I need to post more, to avoid being treated like a rookie...:-) I won't go into that other than to say that yes, this function is firmly in the "tagged tuple" category, for what I am convinced is good reasons. But even such a function may have bugs, and *I* would certainly not expect every such function to wrap each of its clauses in try/catch in order to make sure it returns {error, {i_have_a_bug, ClassAndReasonAndStacktrace}} instead of just letting the exception propagate to the caller. The question of what do with bad arguments passed to such a function is perhaps not as crystal clear, but I think you will agree that the "standard Erlang way" is that an exception is raised in that case too - the function *may* do argument validation via guards, but not in order to return a tagged tuple when there is a bad argument - the result will in such cases typically be a function_clause exception. Just as a data point related to my example, try gen_tcp:recv(foo, 0). If one of the arguments happens to be a fun, this doesn't really change anything - if the fun crashes when called, you can expect that the exception is just propagated to the caller that provided the bad fun in the first place. And there are good reasons for this of course - the {error, Foo} tuples that this type of function may return are typically things that the caller actually needs to be prepared for and handle in "normal" operation - the peer closed its socket, you don't have permission to open the file, etc etc - while the exceptions are not. >So I'd expect something like this: > >try > Fun(Data, State) >catch > Class:Reason -> > Err = {Class, Reason, erlang:get_stacktrace()} > {error, {callback_error, Err}} >end ... and thus I *really* don't want to write code like that, coercing an exception into a "normal" error, that the caller *has* to handle - or possibly, by not handling it, generate a "secondary" case_clause exception or the like, which just obscures the real problem. The whole point of my example was to show a function that had a need to interfere with this "standard" pattern of letting exceptions propagate, in order to do some "cleanup" of side effects. And this should be done in a transparent manner, i.e. the end result for the caller should be the same as if there was no interference. If it wasn't for this cleanup, there would be no reason to do the try/catch in the first place. And I don't think the function without try, catch, and erlang:raise/3 would have elicited your comment about "mingle interfaces", even though its "interface" had been exactly the same.:-) >But onto other issues... Well, this brings us way over the TL;DR threshold, and is also completely unrelated to what I'm asking about, but OK... >It's not clear what "Error" is from looking at the code - and I'd >rather not (as a reader) have to poke around type specs. If you expect >the form {error, Error} there, spell that out. E.g. > > case Res of > ... > {error, Error} -> > gen_tcp:send(Sock, "done"), > {error, Error} > >If "Error" can be anything, you're on shaky ground API wise. You're >asking the caller of your function to handle {ok, Good}, arbitrary >exceptions, and arbitrary return values. I would be very cranky with >this interface. Semi-good point:-), though the Error you are talking about here is what is returned from the fun passed in by the caller. It's spec is that it must return {continue, State} | {done, State} | {error, Reason} - if it returns something else, it will also be returned to the caller - serves him right for providing a fun that didn't comply with the spec.:-) Or maybe it's actually a feature?:-) Seriously, I have a hard time losing sleep over the absence of checks for things that aren't actual problems, but in the *real* code, that I tried to strip down in order to focus on my point (obviously failing miserably:-), the caller would actually be rewarded with a function_clause exception in that case. >Finally, the imperative style coding here really wants some functions, >in particular the calling and handling of the callback (but also >applies to the calling and handling of the socket read). I'd do >something like this: Sure, and that's actually pretty much what the *real* code already looks like. Main differences is that it doesn't do this: >apply_cb(Fun, Data, State) -> > try > Fun(Data, State) > catch > Class:Reason -> > {error, {cb_error, {Class, Reason, erlang:get_stacktract()}}} > end. - which I consider Bad(tm) for the reasons given above, nor this >handle_db(Other, Sock, _Fun) -> > gen_tcp:send(Sock, "done"), > error({unexpected_cb_result, Other}). - which I consider at least "atypical Erlang" and "unnecessary" and even a little "bad". --Per From tobias.ammann@REDACTED Sun May 15 16:26:25 2016 From: tobias.ammann@REDACTED (TA) Date: Sun, 15 May 2016 10:26:25 -0400 Subject: [erlang-questions] Newbie problems with gen_fsm Message-ID: Hello, I wrote some code for a parking meter to understand how gen_fsm works. There are two states (awaiting_payment, valid_ticket), two input messages (payment, cancel_ticket), and three output messages (unpaid, change, refund). I'm getting the following error and don't understand what's wrong. Any pointers? Error: ** exception exit: {{function_clause,[{gen,do_for_proc, [{ok,<0.40.0>},#Fun], [{file,"gen.erl"},{line,252}]}, {gen_fsm,sync_send_event,2, [{file,"gen_fsm.erl"},{line,219}]}, {ticketing_fsm,demo1,0, [{file,"ticketing_fsm.erl"},{line,25}]}, {erl_eval,do_apply,6,[{file,"erl_eval.erl"},{line,674}]}, {shell,exprs,7,[{file,"shell.erl"},{line,686}]}, {shell,eval_exprs,7,[{file,"shell.erl"},{line,641}]}, {shell,eval_loop,3,[{file,"shell.erl"},{line,626}]}]}, {gen_fsm,sync_send_event,[{ok,<0.40.0>},{payment,100}]}} in function gen_fsm:sync_send_event/2 (gen_fsm.erl, line 223) in call from ticketing_fsm:demo1/0 (ticketing_fsm.erl, line 25) Source: https://transfer.sh/M7n3b/ticketing-fsm.erl Tobias -------------- next part -------------- An HTML attachment was scrubbed... URL: From Oliver.Korpilla@REDACTED Sun May 15 19:33:37 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Sun, 15 May 2016 19:33:37 +0200 Subject: [erlang-questions] Newbie problems with gen_fsm In-Reply-To: References: Message-ID: Hello, When I see this: awaiting_payment({payment, Amount}, From, {Price, Paid}) when Amount >= (Price - Paid) -> io:format("~p paid ~p", [From, Amount]), Change = Amount + Paid - Price, {next_state, valid_ticket, {change, Change}, {Price, Price}}; There are too many elements in the tuple returned. | {next_state,NextStateName,NewStateData} | {next_state,NextStateName,NewStateData,Timeout} | {next_state,NextStateName,NewStateData,hibernate} You are allowed to return these when selecting next_state as your return value. But you return a 4-tuple instead of a 3-tuple. If one of these is a reply back to the original requestor, then you need: {reply,Reply,NextStateName,NewStateData} Also, not 100% sure, but: init(Price) -> {ok, awaiting_payment, {Price, 0}}. I think it has to be: init([Price]) -> because init/1 takes the start_link params as a list. Sorry, no Erlang system under my fingertips to test. Hope this helps. Cheers, Oliver? ? Gesendet:?Sonntag, 15. Mai 2016 um 16:26 Uhr Von:?TA An:?"Erlang Users' List" Betreff:?[erlang-questions] Newbie problems with gen_fsm Hello, ? I wrote some code for a parking meter to understand how gen_fsm works. There are two states (awaiting_payment, valid_ticket), two input messages (payment, cancel_ticket), and three output messages (unpaid, change, refund). ? I'm getting the following error and don't understand what's wrong. Any pointers? ? Error: ** exception exit: {{function_clause,[{gen,do_for_proc, ?????????????????????????????????????????? [{ok,<0.40.0>},#Fun], ?????????????????????????????????????????? [{file,"gen.erl"},{line,252}]}, ????????????????????????????????????? {gen_fsm,sync_send_event,2, ?????????????????????????????????????????????? [{file,"gen_fsm.erl"},{line,219}]}, ????????????????????????????????????? {ticketing_fsm,demo1,0, ???????????????????????????????????????????????????? [{file,"ticketing_fsm.erl"},{line,25}]}, ????????????????????????????????????? {erl_eval,do_apply,6,[{file,"erl_eval.erl"},{line,674}]}, ????????????????????????????????????? {shell,exprs,7,[{file,"shell.erl"},{line,686}]}, ????????????????????????????????????? {shell,eval_exprs,7,[{file,"shell.erl"},{line,641}]}, ????????????????????????????????????? {shell,eval_loop,3,[{file,"shell.erl"},{line,626}]}]}, ??????????????????? {gen_fsm,sync_send_event,[{ok,<0.40.0>},{payment,100}]}} ???? in function? gen_fsm:sync_send_event/2 (gen_fsm.erl, line 223) ???? in call from ticketing_fsm:demo1/0 (ticketing_fsm.erl, line 25) ? Source: https://transfer.sh/M7n3b/ticketing-fsm.erl ? Tobias_______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] From Oliver.Korpilla@REDACTED Sun May 15 19:50:00 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Sun, 15 May 2016 19:50:00 +0200 Subject: [erlang-questions] Newbie problems with gen_fsm In-Reply-To: References: , Message-ID: Further observations: request_ticket(Price) -> gen_fsm:start(?MODULE, Price, []). http://erlang.org/doc/man/gen_fsm.html#start-3 This functions returns {ok, Pid} or {error, Reason} so to fit with the rest of your program, it probably should look like this: request_ticket(Price) -> {ok, Pid} gen_fsm:start(?MODULE, Price, []), Pid. Cheers, Oliver? ? Gesendet:?Sonntag, 15. Mai 2016 um 19:33 Uhr Von:?"Oliver Korpilla" An:?tobias.ammann@REDACTED Cc:?"Erlang Users' List" Betreff:?Re: [erlang-questions] Newbie problems with gen_fsm Hello, When I see this: awaiting_payment({payment, Amount}, From, {Price, Paid}) when Amount >= (Price - Paid) -> io:format("~p paid ~p", [From, Amount]), Change = Amount + Paid - Price, {next_state, valid_ticket, {change, Change}, {Price, Price}}; There are too many elements in the tuple returned. | {next_state,NextStateName,NewStateData} | {next_state,NextStateName,NewStateData,Timeout} | {next_state,NextStateName,NewStateData,hibernate} You are allowed to return these when selecting next_state as your return value. But you return a 4-tuple instead of a 3-tuple. If one of these is a reply back to the original requestor, then you need: {reply,Reply,NextStateName,NewStateData} Also, not 100% sure, but: init(Price) -> {ok, awaiting_payment, {Price, 0}}. I think it has to be: init([Price]) -> because init/1 takes the start_link params as a list. Sorry, no Erlang system under my fingertips to test. Hope this helps. Cheers, Oliver? ? Gesendet:?Sonntag, 15. Mai 2016 um 16:26 Uhr Von:?TA An:?"Erlang Users' List" Betreff:?[erlang-questions] Newbie problems with gen_fsm Hello, ? I wrote some code for a parking meter to understand how gen_fsm works. There are two states (awaiting_payment, valid_ticket), two input messages (payment, cancel_ticket), and three output messages (unpaid, change, refund). ? I'm getting the following error and don't understand what's wrong. Any pointers? ? Error: ** exception exit: {{function_clause,[{gen,do_for_proc, ?????????????????????????????????????????? [{ok,<0.40.0>},#Fun], ?????????????????????????????????????????? [{file,"gen.erl"},{line,252}]}, ????????????????????????????????????? {gen_fsm,sync_send_event,2, ?????????????????????????????????????????????? [{file,"gen_fsm.erl"},{line,219}]}, ????????????????????????????????????? {ticketing_fsm,demo1,0, ???????????????????????????????????????????????????? [{file,"ticketing_fsm.erl"},{line,25}]}, ????????????????????????????????????? {erl_eval,do_apply,6,[{file,"erl_eval.erl"},{line,674}]}, ????????????????????????????????????? {shell,exprs,7,[{file,"shell.erl"},{line,686}]}, ????????????????????????????????????? {shell,eval_exprs,7,[{file,"shell.erl"},{line,641}]}, ????????????????????????????????????? {shell,eval_loop,3,[{file,"shell.erl"},{line,626}]}]}, ??????????????????? {gen_fsm,sync_send_event,[{ok,<0.40.0>},{payment,100}]}} ???? in function? gen_fsm:sync_send_event/2 (gen_fsm.erl, line 223) ???? in call from ticketing_fsm:demo1/0 (ticketing_fsm.erl, line 25) ? Source: https://transfer.sh/M7n3b/ticketing-fsm.erl ? Tobias_______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions][http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions]] _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] From t6sn7gt@REDACTED Sun May 15 19:52:54 2016 From: t6sn7gt@REDACTED (Donald Steven) Date: Sun, 15 May 2016 13:52:54 -0400 Subject: [erlang-questions] Updating a nested record Message-ID: <5738B776.8020400@aim.com> Hi everyone, When I update the 'note' record below it updates correctly but the value returned is only the sub-record 'pressure', not the whole 'note' record. I'd be most grateful for the right syntax. Thanks. Don -module(metatest). -export([main/0]). -record(pressure, {initial, final, offset, scope}). -record(wheel, {initial, final, contour, curve, offset, scope}). -record(breath, {initial, final, contour, curve, offset, scope}). -record(pan, {mode, initial, final, contour, curve, offset, scope}). -record(bend, {initial, final, contour, curve, offset, scope}). -record(note, {start, delta, duration, pitch, velocity, p1 = #pressure{}, p2 = #wheel{}, p3 = #breath{}, p4 = #pan{}, p5 = #bend{}}). -record(controller, {start, status, change, data}). -record(pitchbend, {start, status, data}). -record(aftertouch, {start, status, data}). main() -> N = makemetaN(10, 20, 30, 40), io:format("~nOriginal note: ~p~n", [N]), N1 = N#note{delta = 99}, io:format("~nNote after updated delta: ~p~n", [N1]), N2 = setNpressure(N, 50, 60, 70, 80), io:format("~nNote after updated pressure: ~p~n~n", [N2]). makemetaN(Delta, Duration, Pitch, Velocity) -> #note{start = undefined, delta = Delta, duration = Duration, pitch = Pitch, velocity = Velocity}. setNpressure(N, Initial, Final, Offset, Scope) -> N1 = N#note.p1#pressure{initial = Initial, final = Final, offset = Offset, scope = Scope}. From g@REDACTED Sun May 15 20:00:11 2016 From: g@REDACTED (Garrett Smith) Date: Sun, 15 May 2016 11:00:11 -0700 Subject: [erlang-questions] erlang:raise/3 considered harmful? In-Reply-To: <201605151535.u4FFZaUv039895@pluto.hedeland.org> References: <201605151535.u4FFZaUv039895@pluto.hedeland.org> Message-ID: On Sun, May 15, 2016 at 8:35 AM, Per Hedeland wrote: > Garrett Smith wrote: >> >>I think your case justifies the use of this scary function :) > > Thanks.:-) Though given the other things that you wrote, I have to > wonder *why* you think so... > >>That said, I don't think you actually want to use it. Your API need >>improving I think. >> >>The fact you're returning a tagged tuple {ok, State} puts this into >>the "tagged tuple" function category. This in contrast to the "value >>vs exception" category. See dict:find vs dict:fetch for a contrast of >>the two patterns. Stick with one or the other - don't mingle them. > > (Hm, I guess I need to post more, to avoid being treated like a > rookie...:-) I won't go into that other than to say that yes, this > function is firmly in the "tagged tuple" category, for what I am > convinced is good reasons. But even such a function may have bugs, and > *I* would certainly not expect every such function to wrap each of its > clauses in try/catch in order to make sure it returns {error, > {i_have_a_bug, ClassAndReasonAndStacktrace}} instead of just letting the > exception propagate to the caller. But that's not at all what's going on here. You're injecting behavior (side effects) here in an otherwise unexpected crash. You're taking action to send something to a socket. Then you're re-raising the function. If you look at the higher order functions in core, of course they let callback exception propagate - but they don't sneak in behavior and pretend they didn't. > The question of what do with bad arguments passed to such a function is > perhaps not as crystal clear, but I think you will agree that the > "standard Erlang way" is that an exception is raised in that case too - > the function *may* do argument validation via guards, but not in order > to return a tagged tuple when there is a bad argument - the result will > in such cases typically be a function_clause exception. I'm not suggesting anywhere to return an error tuple for a badarg. I'm suggesting that you use a consistent interface, which returns a tagged tuple with ok/error for handled scenarios. If you're altering the normal course of a true exception (i.e. unanticipated and unhandled) by taking some "cleanup" action, you ought to not pretend that it's unexpected and unhandled. Communicate it as an explicit error, providing all the details needed to detect and troubleshoot the problem. > Just as a data > point related to my example, try gen_tcp:recv(foo, 0). If one of the > arguments happens to be a fun, this doesn't really change anything - if > the fun crashes when called, you can expect that the exception is just > propagated to the caller that provided the bad fun in the first place. Yes, but again - when you perform an operation in the middle of this propagation, you ought to let the caller know. You're sending bytes to a socket _that your caller gave you_. That's not important news to share? > And there are good reasons for this of course - the {error, Foo} tuples > that this type of function may return are typically things that the > caller actually needs to be prepared for and handle in "normal" > operation - the peer closed its socket, you don't have permission to > open the file, etc etc - while the exceptions are not. > >>So I'd expect something like this: >> >>try >> Fun(Data, State) >>catch >> Class:Reason -> >> Err = {Class, Reason, erlang:get_stacktrace()} >> {error, {callback_error, Err}} >>end > > ... and thus I *really* don't want to write code like that, coercing an > exception into a "normal" error, that the caller *has* to handle - or > possibly, by not handling it, generate a "secondary" case_clause > exception or the like, which just obscures the real problem. You're not obscuring anything here. You're adding _more_ information. You're also giving the caller a _consistent_ interface for detecting errors. Other exceptions - the unhandled variety - they pass through. I would not for example ever want to see this: my_fun() -> try something() catch C:Err -> {error, {no_idea_really, {C, Err, erlang:get_stracktrace()}}} end. > The whole point of my example was to show a function that had a need to > interfere with this "standard" pattern of letting exceptions propagate, > in order to do some "cleanup" of side effects. And this should be done > in a transparent manner, i.e. the end result for the caller should be > the same as if there was no interference. Stunningly bad idea. You are actively discouraging your caller from taking further action on an error _that you know about and acted on_. What benefit do you think you're providing here? You have two choices: return as an error or generate an exception. In either case you ought to provide the original error info. In exception form: error({cb_error_but_chillax_i_sent_done, OriginalErrorInfo}) But I would not do this - you're already communicating errors with tagged tuples. Now you're asking your user to handle _known errors_ in two different ways. Here's their code: try iterate(Sock, Fun, State) of {ok, State} -> handle_good(State, Sock); {error, Err} -> handle_error(Err, Sock) catch C:Err -> handle_error({C, Err}, Sock) end handle_error can of course sort out what's to be done based on the error, but the mix-and-match of tuple and exception handling is simply a bad interface. BUT, the real fun is sorting out the case where "done" was sent due to an unhandled callback exception! > If it wasn't for this cleanup, > there would be no reason to do the try/catch in the first place. And I > don't think the function without try, catch, and erlang:raise/3 would > have elicited your comment about "mingle interfaces", even though its > "interface" had been exactly the same.:-) Correct - if you let the exception propagate without explicit handling and action, no issue whatsoever. >>But onto other issues... > > Well, this brings us way over the TL;DR threshold, and is also > completely unrelated to what I'm asking about, but OK... > >>It's not clear what "Error" is from looking at the code - and I'd >>rather not (as a reader) have to poke around type specs. If you expect >>the form {error, Error} there, spell that out. E.g. >> >> case Res of >> ... >> {error, Error} -> >> gen_tcp:send(Sock, "done"), >> {error, Error} >> >>If "Error" can be anything, you're on shaky ground API wise. You're >>asking the caller of your function to handle {ok, Good}, arbitrary >>exceptions, and arbitrary return values. I would be very cranky with >>this interface. > > Semi-good point:-), though the Error you are talking about here is what > is returned from the fun passed in by the caller. It's spec is that it > must return {continue, State} | {done, State} | {error, Reason} - if it > returns something else, it will also be returned to the caller - serves > him right for providing a fun that didn't comply with the spec.:-) Experienced programmers (language aside) don't let bad state linger for long. I wonder when an error in that case would be detected - and how much useful context information would still be available. > Or maybe it's actually a feature?:-) Yeah, no, it's not. > Seriously, I have a hard time losing > sleep over the absence of checks for things that aren't actual problems Yeah, no - a violated protocol that goes undetected is an actual problem. > but in the *real* code, that I tried to strip down in order to focus on > my point (obviously failing miserably:-) Your point I believe was a question, which seemed to seek affirmation for the use of a documented function in core. You've gotten some pretty good content in response - or, the community has. That's often the point of responses to questions here - to go on record with what one thinks is generally helpful information, for the sake of not just the OP but for everyone. It often spurs fruitful discussion about any variety of topics. I had to look up this 'raise' function because I never heard of it. Why had I never heard of this function? Maybe I'm a rookie, or maybe it's just rarely used. Why would you ever use this function? You've highlighted one reason - to trick callers. > the caller would actually be rewarded with a function_clause exception in that case. > >>Finally, the imperative style coding here really wants some functions, >>in particular the calling and handling of the callback (but also >>applies to the calling and handling of the socket read). I'd do >>something like this: > > Sure, and that's actually pretty much what the *real* code already looks > like. If the original code uses cogent functions, rather assigning case expressions to variables in hard-to-read imperative style, in the interest of avoiding TLDR in code, post that next time. Main differences is that it doesn't do this: > >>apply_cb(Fun, Data, State) -> >> try >> Fun(Data, State) >> catch >> Class:Reason -> >> {error, {cb_error, {Class, Reason, erlang:get_stacktract()}}} >> end. > > - which I consider Bad(tm) for the reasons given above, nor this > >>handle_db(Other, Sock, _Fun) -> >> gen_tcp:send(Sock, "done"), >> error({unexpected_cb_result, Other}). > > - which I consider at least "atypical Erlang" and "unnecessary" and even > a little "bad". This is certainly not atypical Erlang - invalid callback results are often highlighted with an explicit error (e.g. see gen_server). It's hardly _bad_. It's arguably unnecessary - some may enjoy translating a function/case clause reported in obscure stacks to a mistake they made. In any case, to your original point, which I think everyone is following clearly enough, go for it! From garazdawi@REDACTED Sun May 15 20:24:23 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Sun, 15 May 2016 20:24:23 +0200 Subject: [erlang-questions] erlang:raise/3 considered harmful? In-Reply-To: <201605141148.u4EBmKGQ067045@pluto.hedeland.org> References: <201605141148.u4EBmKGQ067045@pluto.hedeland.org> Message-ID: Hello Per, Could try ... after be what you are looking for? It's not a perfect fit, but I thought I'd mention it as it is a construct that few people seem to know about. > Res = try > Fun(Data, State) > after -> > gen_tcp:send(Sock, "done") > end, > Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Sun May 15 20:48:01 2016 From: g@REDACTED (Garrett Smith) Date: Sun, 15 May 2016 11:48:01 -0700 Subject: [erlang-questions] erlang:raise/3 considered harmful? In-Reply-To: References: <201605151535.u4FFZaUv039895@pluto.hedeland.org> Message-ID: I've made a mistake here and I need to apologize for it. Per, I'm simply in no position to take such a tone with you. You indeed are no rookie, and I over stepped my bounds here in a terribly embarrassing way. I'll turn my attention to cleaning up my own code and save the lectures for myself! On Sun, May 15, 2016 at 11:00 AM, Garrett Smith wrote: > On Sun, May 15, 2016 at 8:35 AM, Per Hedeland wrote: >> Garrett Smith wrote: >>> >>>I think your case justifies the use of this scary function :) >> >> Thanks.:-) Though given the other things that you wrote, I have to >> wonder *why* you think so... >> >>>That said, I don't think you actually want to use it. Your API need >>>improving I think. >>> >>>The fact you're returning a tagged tuple {ok, State} puts this into >>>the "tagged tuple" function category. This in contrast to the "value >>>vs exception" category. See dict:find vs dict:fetch for a contrast of >>>the two patterns. Stick with one or the other - don't mingle them. >> >> (Hm, I guess I need to post more, to avoid being treated like a >> rookie...:-) I won't go into that other than to say that yes, this >> function is firmly in the "tagged tuple" category, for what I am >> convinced is good reasons. But even such a function may have bugs, and >> *I* would certainly not expect every such function to wrap each of its >> clauses in try/catch in order to make sure it returns {error, >> {i_have_a_bug, ClassAndReasonAndStacktrace}} instead of just letting the >> exception propagate to the caller. > > But that's not at all what's going on here. You're injecting behavior > (side effects) here in an otherwise unexpected crash. You're taking > action to send something to a socket. Then you're re-raising the > function. > > If you look at the higher order functions in core, of course they let > callback exception propagate - but they don't sneak in behavior and > pretend they didn't. > >> The question of what do with bad arguments passed to such a function is >> perhaps not as crystal clear, but I think you will agree that the >> "standard Erlang way" is that an exception is raised in that case too - >> the function *may* do argument validation via guards, but not in order >> to return a tagged tuple when there is a bad argument - the result will >> in such cases typically be a function_clause exception. > > I'm not suggesting anywhere to return an error tuple for a badarg. > > I'm suggesting that you use a consistent interface, which returns a > tagged tuple with ok/error for handled scenarios. If you're altering > the normal course of a true exception (i.e. unanticipated and > unhandled) by taking some "cleanup" action, you ought to not pretend > that it's unexpected and unhandled. Communicate it as an explicit > error, providing all the details needed to detect and troubleshoot the > problem. > >> Just as a data >> point related to my example, try gen_tcp:recv(foo, 0). If one of the >> arguments happens to be a fun, this doesn't really change anything - if >> the fun crashes when called, you can expect that the exception is just >> propagated to the caller that provided the bad fun in the first place. > > Yes, but again - when you perform an operation in the middle of this > propagation, you ought to let the caller know. You're sending bytes to > a socket _that your caller gave you_. That's not important news to > share? > >> And there are good reasons for this of course - the {error, Foo} tuples >> that this type of function may return are typically things that the >> caller actually needs to be prepared for and handle in "normal" >> operation - the peer closed its socket, you don't have permission to >> open the file, etc etc - while the exceptions are not. >> >>>So I'd expect something like this: >>> >>>try >>> Fun(Data, State) >>>catch >>> Class:Reason -> >>> Err = {Class, Reason, erlang:get_stacktrace()} >>> {error, {callback_error, Err}} >>>end >> >> ... and thus I *really* don't want to write code like that, coercing an >> exception into a "normal" error, that the caller *has* to handle - or >> possibly, by not handling it, generate a "secondary" case_clause >> exception or the like, which just obscures the real problem. > > You're not obscuring anything here. You're adding _more_ information. > > You're also giving the caller a _consistent_ interface for detecting errors. > > Other exceptions - the unhandled variety - they pass through. I would > not for example ever want to see this: > > my_fun() -> > try > something() > catch > C:Err -> > {error, {no_idea_really, {C, Err, erlang:get_stracktrace()}}} > end. > >> The whole point of my example was to show a function that had a need to >> interfere with this "standard" pattern of letting exceptions propagate, >> in order to do some "cleanup" of side effects. And this should be done >> in a transparent manner, i.e. the end result for the caller should be >> the same as if there was no interference. > > Stunningly bad idea. You are actively discouraging your caller from > taking further action on an error _that you know about and acted on_. > > What benefit do you think you're providing here? > > You have two choices: return as an error or generate an exception. In > either case you ought to provide the original error info. In exception > form: > > error({cb_error_but_chillax_i_sent_done, OriginalErrorInfo}) > > But I would not do this - you're already communicating errors with > tagged tuples. Now you're asking your user to handle _known errors_ in > two different ways. > > Here's their code: > > try iterate(Sock, Fun, State) of > {ok, State} -> handle_good(State, Sock); > {error, Err} -> handle_error(Err, Sock) > catch > C:Err -> handle_error({C, Err}, Sock) > end > > handle_error can of course sort out what's to be done based on the > error, but the mix-and-match of tuple and exception handling is simply > a bad interface. > > BUT, the real fun is sorting out the case where "done" was sent due to > an unhandled callback exception! > >> If it wasn't for this cleanup, >> there would be no reason to do the try/catch in the first place. And I >> don't think the function without try, catch, and erlang:raise/3 would >> have elicited your comment about "mingle interfaces", even though its >> "interface" had been exactly the same.:-) > > Correct - if you let the exception propagate without explicit handling > and action, no issue whatsoever. > >>>But onto other issues... >> >> Well, this brings us way over the TL;DR threshold, and is also >> completely unrelated to what I'm asking about, but OK... >> >>>It's not clear what "Error" is from looking at the code - and I'd >>>rather not (as a reader) have to poke around type specs. If you expect >>>the form {error, Error} there, spell that out. E.g. >>> >>> case Res of >>> ... >>> {error, Error} -> >>> gen_tcp:send(Sock, "done"), >>> {error, Error} >>> >>>If "Error" can be anything, you're on shaky ground API wise. You're >>>asking the caller of your function to handle {ok, Good}, arbitrary >>>exceptions, and arbitrary return values. I would be very cranky with >>>this interface. >> >> Semi-good point:-), though the Error you are talking about here is what >> is returned from the fun passed in by the caller. It's spec is that it >> must return {continue, State} | {done, State} | {error, Reason} - if it >> returns something else, it will also be returned to the caller - serves >> him right for providing a fun that didn't comply with the spec.:-) > > Experienced programmers (language aside) don't let bad state linger > for long. I wonder when an error in that case would be detected - and > how much useful context information would still be available. > >> Or maybe it's actually a feature?:-) > > Yeah, no, it's not. > >> Seriously, I have a hard time losing >> sleep over the absence of checks for things that aren't actual problems > > Yeah, no - a violated protocol that goes undetected is an actual problem. > >> but in the *real* code, that I tried to strip down in order to focus on >> my point (obviously failing miserably:-) > > Your point I believe was a question, which seemed to seek affirmation > for the use of a documented function in core. > > You've gotten some pretty good content in response - or, the community > has. That's often the point of responses to questions here - to go on > record with what one thinks is generally helpful information, for the > sake of not just the OP but for everyone. It often spurs fruitful > discussion about any variety of topics. > > I had to look up this 'raise' function because I never heard of it. > Why had I never heard of this function? Maybe I'm a rookie, or maybe > it's just rarely used. Why would you ever use this function? You've > highlighted one reason - to trick callers. > >> the caller would actually be rewarded with a function_clause exception in that case. >> >>>Finally, the imperative style coding here really wants some functions, >>>in particular the calling and handling of the callback (but also >>>applies to the calling and handling of the socket read). I'd do >>>something like this: >> >> Sure, and that's actually pretty much what the *real* code already looks >> like. > > If the original code uses cogent functions, rather assigning case > expressions to variables in hard-to-read imperative style, in the > interest of avoiding TLDR in code, post that next time. > > Main differences is that it doesn't do this: >> >>>apply_cb(Fun, Data, State) -> >>> try >>> Fun(Data, State) >>> catch >>> Class:Reason -> >>> {error, {cb_error, {Class, Reason, erlang:get_stacktract()}}} >>> end. >> >> - which I consider Bad(tm) for the reasons given above, nor this >> >>>handle_db(Other, Sock, _Fun) -> >>> gen_tcp:send(Sock, "done"), >>> error({unexpected_cb_result, Other}). >> >> - which I consider at least "atypical Erlang" and "unnecessary" and even >> a little "bad". > > This is certainly not atypical Erlang - invalid callback results are > often highlighted with an explicit error (e.g. see gen_server). It's > hardly _bad_. It's arguably unnecessary - some may enjoy translating a > function/case clause reported in obscure stacks to a mistake they > made. > > In any case, to your original point, which I think everyone is > following clearly enough, go for it! From per@REDACTED Sun May 15 21:17:04 2016 From: per@REDACTED (Per Hedeland) Date: Sun, 15 May 2016 21:17:04 +0200 (CEST) Subject: [erlang-questions] erlang:raise/3 considered harmful? In-Reply-To: Message-ID: <201605151917.u4FJH4Id041412@pluto.hedeland.org> Lukas Larsson wrote: > >Could try ... after be what you are looking for? It's not a perfect fit, >but I thought I'd mention it as it is a construct that few people seem to >know about. Now we're talking:-) - indeed, try ... after is one of the solutions I've been thinking of, the problem is that I really can't make it fit. >> Res = try >> Fun(Data, State) >> after -> >> gen_tcp:send(Sock, "done") >> end, So this ends up *always* sending the "done" after invoking the fun, but that doesn't match the protocol - if you look at the original function, there is one case where "continue" should be sent *instead*. I've also considered wrapping the whole loop/3 function in try ... after, but that instead runs into two other cases, where *nothing* should be sent: case gen_tcp:recv(Sock, 0) of {ok, "done"} -> {ok, State}; ... Error -> Error If you can come up with a way to have try ... after handle all the cases, I'm all ears! --Per From garazdawi@REDACTED Sun May 15 21:39:49 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Sun, 15 May 2016 21:39:49 +0200 Subject: [erlang-questions] erlang:raise/3 considered harmful? In-Reply-To: <201605151917.u4FJH4Id041412@pluto.hedeland.org> References: <201605151917.u4FJH4Id041412@pluto.hedeland.org> Message-ID: On Sun, May 15, 2016 at 9:17 PM, Per Hedeland wrote: > So this ends up *always* sending the "done" after invoking the fun, but > that doesn't match the protocol - if you look at the original function, > there is one case where "continue" should be sent *instead*. yes, I noticed that, which is why it's not a perfect fit :) What you want to have is someway to know in the after clause if an exception was thrown or not.... I'm not sure that is possible. I had an idea of doing something fancy with inspecting the stacktrace in the after clause, but it seems to be indistinguishable from an error happening and being caught below the try. Sometimes erlang:raise is the right tool for the job and this may be one of them. Just make sure not to crash in the catch before the re-throw :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From per@REDACTED Sun May 15 21:52:28 2016 From: per@REDACTED (Per Hedeland) Date: Sun, 15 May 2016 21:52:28 +0200 (CEST) Subject: [erlang-questions] erlang:raise/3 considered harmful? In-Reply-To: Message-ID: <201605151952.u4FJqS13041536@pluto.hedeland.org> Garrett Smith wrote: > >I've made a mistake here and I need to apologize for it. Per, I'm >simply in no position to take such a tone with you. You indeed are no >rookie, and I over stepped my bounds here in a terribly embarrassing >way. > >I'll turn my attention to cleaning up my own code and save the >lectures for myself! Don't be silly - I can take any tone, and I think there was lots of good stuff in your reply, just that it didn't really help *me*. If you have any comments om my actual question, I would be very interested. I should say though that I've pretty much come to the conclusion that the documentation for erlang:raise/3 should be amended to say something along the lines of "this function is useful if you want to re-raise a caught exception after doing some cleanup, but for any *other* usage, take heed of the following warning: ..." The problem with the function is basically just that it *can* be used in "other" ways, such as synthesizing an exception that didn't actually occur. And it's hard (at least for me) to see how a function that *only* allowed for re-raising a caught exception could be defined, given the existing try ... catch infrastructure. --Per PS On grepping the OTP code, among the massive amount of erlang:raise/3 usage, I found at least one that perfectly matches my case: file_sorter. From joshmbarney@REDACTED Sun May 15 22:28:53 2016 From: joshmbarney@REDACTED (Joshua Barney) Date: Sun, 15 May 2016 16:28:53 -0400 Subject: [erlang-questions] How to disable parentheses matching in Erlang shell? In-Reply-To: <03e2c191-5810-2b29-6dea-38cafaa7d2cb@gjunka.com> References: <6013b959-58b0-a963-9478-d6845993f024@gjunka.com> <57379656.6010306@gmail.com> <5481da92-8dbb-750e-8ced-c5ffafd2e4ce@gjunka.com> <03e2c191-5810-2b29-6dea-38cafaa7d2cb@gjunka.com> Message-ID: Grzegorz, that?s a long list of environments! I took some time to reproduce the ?Ubuntu ssh to CentOS? (and vice versa) using vmware. on two fresh installs of ubuntu and centos, the paren matching worked correctly. On ubuntu, If I resized the window halfway through it would jump backwards to the wrong point and then forwards to the correct point?centos xterm re-flowed the text so this did not happen. This worked with $TERM set to 'xterm' or ?konsole? or ?putty?. you said earlier: >>> I don't have such environment variables (LINES and COLUMNS) in my system and I don't remember having them on Ubuntu or Red Hat (however I can't double-check that now) These variables are set by your terminal in response to resize events and are communicated over ssh as you can read about here: http://unix.stackexchange.com/a/207802 If your system does not have those variables, I would strongly suspect that the nurses library would not function in the way you desire. At this point I can only wildly speculate about ways your erlang install (or terminfo database, or ncurses library) is broken, and I don?t think you?re really looking for that. Josh > On May 15, 2016, at 8:00 AM, Grzegorz Junka wrote: > > Joshua, > > This is what I said, it isn't broken only for me. I saw it being broken on other people's computers as well. > > It's unlikely for any scripts to set COLUMNS since I don't see that environment variable being set when running shell. > > I am running Konsole on the same system on which I am running Erlang, both on FreeBSD. I don't need to SSH anywhere. But the behaviour was the same no matter from where or to where I was SSH-ing or not (or someone I saw doing it), e.g.: > > Windows putty SSH to FreeBSD > > Windows putty SSH to Ubuntu > > Windows putty SSH to Red Hat > > Windows putty SSH to CentOS > > Windows xterm cygwin SSH to FreeBSD > > Windows xterm cygwin SSH To Ubuntu > > Windows xterm cygwin SSH to Red Hat > > Windows xterm cygwin SSH to CentOS > > FreeBSD xterm SSH to Ubuntu > > FreeBSD Konsole SSH to Ubuntu > > FreeBSD xterm SSH to FreeBSD > > FreeBSD Konsole SSH to Ubuntu > > FreeBSD xterm SSH to CentOS > > FreeBSD Konsole SSH to CentOS > > FreeBSD Konsole and Erlang shell on the same system (no SSH) > > FreeBSD xterm end Erlang shell on the same system (no SSH) > > Ubuntu SSH to CentOS > > It's fair to say that I haven't see a system on which this feature worked fine, I just learned to not type commands that are too long or copy-paste longer commands, and I bet most people who experience it do the same and don't bother complaining. I wonder if maybe this has something to do with escape codes and for example support for 256 colors in the terminal. > > Grzegorz > > > On 14/05/2016 22:49, Joshua Barney wrote: >> It seems unlikely that erlang shell is broken only for you on all these systems. >> Have you checked that your shell scripts are not setting COLUMNS? Perhaps you are ssh-ing from a client that is doing something not quite right? >> >> Sent from my iPhone >> >>> On May 14, 2016, at 6:39 PM, Grzegorz Junka wrote: >>> >>> >>>> On 14/05/2016 21:19, Kenneth Lakin wrote: >>>>> On 05/14/2016 03:32 AM, Grzegorz Junka wrote: >>>>> Fixing my terminal to 80 characters only seems like taken straight from >>>>> the Windows world. I hope Erlang can do better ;) >>>> If ncurses can't determine your terminal size, then falling back to >>>> 80x24 makes a lot of sense. Do other ncurses-based programs behave >>>> incorrectly? >>> No. Everything else is fine. This problem is not related to my system, as I have seen the same behaviour in Erlang running on FreeBSD, Ubuntu, Red Hat and CentOS in different terminals, like putty, cygwin and Konsole. >>> >>>>> Is there any way of >>>>> telling the Erlang terminal how wide is my terminal, e.g. an environment >>>>> variable? >>>> On my system, it looks like the environment variables LINES and COLUMNS >>>> correspond to the current size of my terminal window. When I set COLUMNS >>>> to a value that does not match the window's current size, then the >>>> Erlang shell's paren matching bounces the cursor backwards to the wrong >>>> place (but back forwards to the right place). Maybe this will be the >>>> band-aid you need. >>> I don't have such environment variables in my system and I don't remember having them on Ubuntu or Red Hat (however I can't double-check that now) >>> >>>> However, it might be more useful to fix whatever is preventing ncurses >>>> from determining the terminal size. >>> Yeah, and even more useful if the Erlang shell allowed me to disable a feature that isn't really necessary especially if it isn't working correctly. >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethlakin@REDACTED Sun May 15 23:21:35 2016 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Sun, 15 May 2016 14:21:35 -0700 Subject: [erlang-questions] Updating a nested record In-Reply-To: <5738B776.8020400@aim.com> References: <5738B776.8020400@aim.com> Message-ID: <5738E85F.1090408@gmail.com> On 05/15/2016 10:52 AM, Donald Steven wrote: > Hi everyone, > > When I update the 'note' record below it updates correctly but the > value returned is only the sub-record 'pressure', not the whole 'note' > record. I'd be most grateful for the right syntax. Thanks. I think the syntax is: setNpressure(N, Initial, Final, Offset, Scope) -> N1 = N#note{p1 = #pressure{initial = Initial, final = Final, offset = Offset, scope = Scope}}. Otherwise, you're just returning the .p1 field. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From list1@REDACTED Sun May 15 23:45:18 2016 From: list1@REDACTED (Grzegorz Junka) Date: Sun, 15 May 2016 21:45:18 +0000 Subject: [erlang-questions] How to disable parentheses matching in Erlang shell? In-Reply-To: References: <6013b959-58b0-a963-9478-d6845993f024@gjunka.com> <57379656.6010306@gmail.com> <5481da92-8dbb-750e-8ced-c5ffafd2e4ce@gjunka.com> <03e2c191-5810-2b29-6dea-38cafaa7d2cb@gjunka.com> Message-ID: I just realized what is likely causing the problem. When I start the shell with erl and type in this testing string (with spaces so it's longer) it all works fine: lists:sort([{a, 1}, {b, 2}, {c, 3}, {d, 4}, {e, 5}, {f, 6}, {g, 7}, {h, 8}, {i, 9}, {j, 10}]). But I very rarely use Erlang shell in this way. Most of the time I am connecting to running Erlang nodes using to_erl specifying the shell's PID. Then it breaks, probably because my terminal size doesn't match the default terminal size with which the shell has been originally started. When I then try to type in the same example the shell will match each {} correctly but as soon as I type ']' to close the list the cursor jumps to the line above, then some other place when I type ')'. So it's not related to the environment but to the way I am using Erlang shell. Is there any solution to that? Grzegorz On 15/05/2016 20:28, Joshua Barney wrote: > Grzegorz, > > that?s a long list of environments! > > I took some time to reproduce the ?Ubuntu ssh to CentOS? (and vice > versa) using vmware. on two fresh installs of ubuntu and centos, the > paren matching worked correctly. On ubuntu, If I resized the window > halfway through it would jump backwards to the wrong point and then > forwards to the correct point?centos xterm re-flowed the text so this > did not happen. This worked with $TERM set to 'xterm' or ?konsole? or > ?putty?. > > you said earlier: >>>> I don't have such environment variables (LINES and COLUMNS) in my >>>> system and I don't remember having them on Ubuntu or Red Hat >>>> (however I can't double-check that now) > > These variables are set by your terminal in response to resize events > and are communicated over ssh as you can read about here: > http://unix.stackexchange.com/a/207802 > If your system does not have those variables, I would strongly suspect > that the nurses library would not function in the way you desire. > > At this point I can only wildly speculate about ways your erlang > install (or terminfo database, or ncurses library) is broken, and I > don?t think you?re really looking for that. > > Josh > >> On May 15, 2016, at 8:00 AM, Grzegorz Junka > > wrote: >> >> Joshua, >> >> This is what I said, it isn't broken only for me. I saw it being >> broken on other people's computers as well. >> >> It's unlikely for any scripts to set COLUMNS since I don't see that >> environment variable being set when running shell. >> >> I am running Konsole on the same system on which I am running Erlang, >> both on FreeBSD. I don't need to SSH anywhere. But the behaviour was >> the same no matter from where or to where I was SSH-ing or not (or >> someone I saw doing it), e.g.: >> >> Windows putty SSH to FreeBSD >> >> Windows putty SSH to Ubuntu >> >> Windows putty SSH to Red Hat >> >> Windows putty SSH to CentOS >> >> Windows xterm cygwin SSH to FreeBSD >> >> Windows xterm cygwin SSH To Ubuntu >> >> Windows xterm cygwin SSH to Red Hat >> >> Windows xterm cygwin SSH to CentOS >> >> FreeBSD xterm SSH to Ubuntu >> >> FreeBSD Konsole SSH to Ubuntu >> >> FreeBSD xterm SSH to FreeBSD >> >> FreeBSD Konsole SSH to Ubuntu >> >> FreeBSD xterm SSH to CentOS >> >> FreeBSD Konsole SSH to CentOS >> >> FreeBSD Konsole and Erlang shell on the same system (no SSH) >> >> FreeBSD xterm end Erlang shell on the same system (no SSH) >> >> Ubuntu SSH to CentOS >> >> It's fair to say that I haven't see a system on which this feature >> worked fine, I just learned to not type commands that are too long or >> copy-paste longer commands, and I bet most people who experience it >> do the same and don't bother complaining. I wonder if maybe this has >> something to do with escape codes and for example support for 256 >> colors in the terminal. >> >> Grzegorz >> >> >> On 14/05/2016 22:49, Joshua Barney wrote: >>> It seems unlikely that erlang shell is broken only for you on all >>> these systems. >>> Have you checked that your shell scripts are not setting COLUMNS? >>> Perhaps you are ssh-ing from a client that is doing something not >>> quite right? >>> >>> Sent from my iPhone >>> >>>> On May 14, 2016, at 6:39 PM, Grzegorz Junka >>> > wrote: >>>> >>>> >>>>> On 14/05/2016 21:19, Kenneth Lakin wrote: >>>>>> On 05/14/2016 03:32 AM, Grzegorz Junka wrote: >>>>>> Fixing my terminal to 80 characters only seems like taken >>>>>> straight from >>>>>> the Windows world. I hope Erlang can do better ;) >>>>> If ncurses can't determine your terminal size, then falling back to >>>>> 80x24 makes a lot of sense. Do other ncurses-based programs behave >>>>> incorrectly? >>>> No. Everything else is fine. This problem is not related to my >>>> system, as I have seen the same behaviour in Erlang running on >>>> FreeBSD, Ubuntu, Red Hat and CentOS in different terminals, like >>>> putty, cygwin and Konsole. >>>> >>>>>> Is there any way of >>>>>> telling the Erlang terminal how wide is my terminal, e.g. an >>>>>> environment >>>>>> variable? >>>>> On my system, it looks like the environment variables LINES and >>>>> COLUMNS >>>>> correspond to the current size of my terminal window. When I set >>>>> COLUMNS >>>>> to a value that does not match the window's current size, then the >>>>> Erlang shell's paren matching bounces the cursor backwards to the >>>>> wrong >>>>> place (but back forwards to the right place). Maybe this will be the >>>>> band-aid you need. >>>> I don't have such environment variables in my system and I don't >>>> remember having them on Ubuntu or Red Hat (however I can't >>>> double-check that now) >>>> >>>>> However, it might be more useful to fix whatever is preventing ncurses >>>>> from determining the terminal size. >>>> Yeah, and even more useful if the Erlang shell allowed me to >>>> disable a feature that isn't really necessary especially if it >>>> isn't working correctly. >>>> >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From t6sn7gt@REDACTED Sun May 15 23:54:13 2016 From: t6sn7gt@REDACTED (Donald Steven) Date: Sun, 15 May 2016 17:54:13 -0400 Subject: [erlang-questions] Updating a nested record In-Reply-To: <5738E85F.1090408@gmail.com> References: <5738B776.8020400@aim.com> <5738E85F.1090408@gmail.com> Message-ID: <5738F005.20106@aim.com> An HTML attachment was scrubbed... URL: From kennethlakin@REDACTED Sun May 15 23:55:09 2016 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Sun, 15 May 2016 14:55:09 -0700 Subject: [erlang-questions] erlang:raise/3 considered harmful? In-Reply-To: References: <201605141148.u4EBmKGQ067045@pluto.hedeland.org> Message-ID: <5738F03D.4000004@gmail.com> > I'd typically use "catch" (I don't care for try...catch) According to the last couple paragraphs in this message: http://erlang.org/pipermail/erlang-questions/2013-November/075928.html a bare catch will create a stacktrace when an exception of type 'error' is caught, whereas a try ... catch will only create a stacktrace if you call erlang:get_stacktrace(). I know that a while back parts of mnesia were rewritten to move to try ... catch, presumably prevent the cost of unneeded stacktrace creation. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From bernard@REDACTED Mon May 16 01:32:16 2016 From: bernard@REDACTED (Bernard Duggan) Date: Mon, 16 May 2016 09:32:16 +1000 Subject: [erlang-questions] Third party libs you always use In-Reply-To: References: Message-ID: In addition to several already mentioned, https://github.com/rabbitmq/erlando I cannot tell you how much easier the 'do' parse transform has made writing input handlers. B -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Mon May 16 05:12:43 2016 From: rexxe98@REDACTED (Andrew Berman) Date: Mon, 16 May 2016 03:12:43 +0000 Subject: [erlang-questions] Third party libs you always use In-Reply-To: References: Message-ID: More of a tool than a library, but rebar3 has been indispensable for me. You won't need sync anymore either as you can use the auto plugin. On Sun, May 15, 2016 at 4:32 PM Bernard Duggan wrote: > In addition to several already mentioned, > https://github.com/rabbitmq/erlando > I cannot tell you how much easier the 'do' parse transform has made > writing input handlers. > B > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethlakin@REDACTED Mon May 16 06:00:35 2016 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Sun, 15 May 2016 21:00:35 -0700 Subject: [erlang-questions] Third party libs you always use In-Reply-To: References: Message-ID: <573945E3.3070803@gmail.com> On 05/15/2016 08:12 PM, Andrew Berman wrote: > More of a tool than a library, but rebar3 has been indispensable for > me. You won't need sync anymore either as you can use the auto plugin. When last I checked, auto doesn't accept any args at all. Fixing this is in the issue tracker, but there are only so many hours in the day. If auto accepted the same args as shell, and either switched to a pure-erlang file change notification system or just dumped the output that we get from inotifywait on startup and shutdown to /dev/null then using it it *would* be a no-brainer. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From zxq9@REDACTED Mon May 16 12:15:37 2016 From: zxq9@REDACTED (zxq9) Date: Mon, 16 May 2016 19:15:37 +0900 Subject: [erlang-questions] Erlang/OTP 19.0-rc1 is available for testing In-Reply-To: References: Message-ID: <48793105.3OT0HLkpRl@burrito> On Thursday 12 May 2016 17:18:14 Kenneth Lundin wrote: > Erlang/OTP 19.0-rc1 is available for testing. Awesome! > We would like to ask you to build and test this release candidate and send > us your feedback as soon as possible, so that we can make the necessary > corrections before OTP 19.0. My backend and networky-thingy applications -- work like a charm. Will of course require a bit more time actually driving things around in staging to really know what gets weird where, but so far everything works for me on Linux and BSD. My GUI and clienty-thingy applications -- all screwed up if they use wx. Wx has several issues in R19rc1. A few glaringly obvious problems (only tested on Linux so far, may try Windows 7 and 10 eventually): - wxStaticBoxSizer is crazy now (at least on KDE -- still need to check against Gnome and Windows). Label text is cut in half, sizer borders are thinner than expected, and the sizer visual framing line is either not drawn or has been pushed out of the paint area by the thinness of the frame (this is probably also the same reason the label text is cut in half). - Tab navigation seems to not work at all. - Input handling within text elements is broken. Initial text input works fine, but trying to press backspace makes the input field unresponsive (you have to use a mouse to refocus, and can only delete by highlight). There may be a few other rough edges, but so far the only negatives have been in wxErlang -- everything else is great. :-) -Craig From bjorn@REDACTED Mon May 16 12:19:12 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 16 May 2016 12:19:12 +0200 Subject: [erlang-questions] erlang:raise/3 considered harmful? In-Reply-To: <201605141148.u4EBmKGQ067045@pluto.hedeland.org> References: <201605141148.u4EBmKGQ067045@pluto.hedeland.org> Message-ID: On Sat, May 14, 2016 at 1:48 PM, Per Hedeland wrote: > Hi, > > I happened to want something like erlang:raise/3 for a library function > - I didn't think it existed:-), but found that it does, although it > comes with a semi-dire warning. The loop/3 function below does exactly > what I want, specifically: if the fun provided by the caller of > interate/3 crashes, the caller gets the same exception as if there had > been no try ... catch, can handle it with hir own try/catch or not, but > in any case receives full information about *what* went wrong in hir fun > - all while the "protocol" spoken over the socket remains in sync. > > So, to take heed of the warning - is there a way to write an equivalent > function *without* using raise/3 - or should it just be considered a > case of "you really know what you are doing"? I believe I am of > course:-), but I also think that the code below is a perfectly > reasonable implementation of a side-effecty library function, and has no > more to do with debugging or knowing what you're doing than any other > code... > I agree. I will remove the warning in OTP 19. erlang:raise/3 is particularly useful when you don't want to handle the error, but let you log important information about the context. For example, most compiler passes in the BEAM compiler use code similar to: function({function,Name,Arity,Entry,Is}) -> try %% Optimize the code. catch Class:Error -> Stack = erlang:get_stacktrace(), io:format("Function: ~w/~w\n", [Name,Arity]), erlang:raise(Class, Error, Stack) end. /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From jesper.louis.andersen@REDACTED Mon May 16 12:40:23 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 16 May 2016 12:40:23 +0200 Subject: [erlang-questions] erlang:raise/3 considered harmful? In-Reply-To: References: <201605141148.u4EBmKGQ067045@pluto.hedeland.org> Message-ID: On Mon, May 16, 2016 at 12:19 PM, Bj?rn Gustavsson wrote: > erlang:raise/3 is particularly useful when you > don't want to handle the error, but let you log > important information about the context. > I think this is the key insight. The problem is that in Erlang, an exception is not a first class value. When an exception occurs, it either crashes the process, or it is caught by a "catch-style" expression. When caught, the exception is turned into two atoms, Class and Reason, and as a side-effect the VM tracks the last stacktrace. But an exception is not a first class value, as is the case in e.g., Standard ML. When the exception is caught, the VM "reflects" the exception into {Class, Reason, erlang:get_stacktrace()} which is now our representation of the exception in our code (since we have no access to first-class exceptions). The erlang:raise/3 function acts as a "reifier" and turns our "proxy-representation" back into a VM exception which the VM can work with. Any solution not involving exceptions will have to simulate them. The two obvious ways are the process dictionary to track the state of "done" on the socket, or a separate process for either the Fun or the Socket. Neither of these sound desirable to me. In Standard ML, we can write: datatype ('a, 'b) result = Ok of 'a | Error of 'b exception Argh of string fun f () = raise (Argh "Some error here") fun g () = Ok (f ()) handle Ex => Error Ex fun h () = case g () of Ok v => v | Error ex => raise ex Note how the function 'g' handles any exception and turns it into an error (wrapped in the a result datatype to make the type system happy). Finally the function 'h' can re-raise the exception in another context: > use "z.sml"; exception Argh of string val f = fn: unit -> 'a val g = fn: unit -> ('a, exn) result val h = fn: unit -> 'a datatype ('a, 'b) result = Error of 'b | Ok of 'a val it = (): unit > h (); poly: : warning: The type of (it) contains a free type variable. Setting it to a unique monotype. Exception- Argh "Some error here" raised -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Mon May 16 12:43:18 2016 From: dangud@REDACTED (Dan Gudmundsson) Date: Mon, 16 May 2016 10:43:18 +0000 Subject: [erlang-questions] Erlang/OTP 19.0-rc1 is available for testing In-Reply-To: <48793105.3OT0HLkpRl@burrito> References: <48793105.3OT0HLkpRl@burrito> Message-ID: On Mon, May 16, 2016 at 12:15 PM zxq9 wrote: > > My GUI and clienty-thingy applications -- all screwed up if they use wx. > Wx has several issues in R19rc1. A few glaringly obvious problems (only > tested on Linux so far, may try Windows 7 and 10 eventually): > > - wxStaticBoxSizer is crazy now (at least on KDE -- still need to check > against Gnome and Windows). Label text is cut in half, sizer borders > are thinner than expected, and the sizer visual framing line is either > not drawn or has been pushed out of the paint area by the thinness of > the frame (this is probably also the same reason the label text is cut > in half). > > - Tab navigation seems to not work at all. > > - Input handling within text elements is broken. Initial text input works > fine, but trying to press backspace makes the input field unresponsive > (you have to use a mouse to refocus, and can only delete by highlight). > > There may be a few other rough edges, but so far the only negatives have > been in wxErlang -- everything else is great. :-) > > -Craig > I don't feel I have done anything to get those results in wx, can you check your build environment. I.e. what wxWidgets version do you have now and what did you have before, and what gtk version was wxWidgets compiled with gtk-2 or gtk-3? The changes in wx are real bugfixes, such as out arguments moved to a return tuple, and some bugs in the command queue handling which could cause calls to be executed twice or not at all, but rendering of static boxes and tab navigation are handled by wxWidgets not the wx wrapper, so it sounds like build changes on your side. /Dan _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Mon May 16 14:42:25 2016 From: ingela.andin@REDACTED (Ingela Andin) Date: Mon, 16 May 2016 14:42:25 +0200 Subject: [erlang-questions] SSL problems with OTP 19 In-Reply-To: <57377381.20208@gmail.com> References: <20160514180742.GA21998@molb.org> <57377381.20208@gmail.com> Message-ID: 2016-05-14 20:50 GMT+02:00 Michael Truog : > On 05/14/2016 11:07 AM, Johannes Wei?l wrote: > >> Hello, >> >> We let our test suite run daily against the current OTP master branch. >> Since May 5th some tests using the ssl application and inets httpc >> fail due to an (infinite) timeout. >> >> Through "git bisect" I found out that one of the following two commits >> is responsible: >> >> >> https://github.com/erlang/otp/commit/0f782eb6377ddd1e5673c186b8b1a37edd610e4b >> (ssl: Adapt DTLS to gen_statem) >> >> https://github.com/erlang/otp/commit/67de1ba82d1af910dc6ab6d41a1f22dbfb39f658 >> (ssl: Use gen_statem instead of gen_fsm) >> >> Maybe it has something to do with our usage of ssl:controlling_process/2. >> >> I will continue to debug the problem, but wanted to check if the problem >> is known or if anybody else encountered similar problems? >> > > I filed a bug for this at http://bugs.erlang.org/browse/ERL-144 due to it > preventing the rebar3 bootstrap script from running. > > I could not reproduce this. Regards Ingela Erlang/OTP Team - Ericsson AB > > >> Regards, >> Johannes >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Mon May 16 14:58:45 2016 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 16 May 2016 14:58:45 +0200 Subject: [erlang-questions] how: Simple distributed Erlang in docker containers Message-ID: <5739C405.20205@ericsson.com> Greetings, What is needed to get two Erlang nodes talking if they are in Docker containers (on the same machine)? net_kernel:connect/1 fails. I have started 2 Docker containers (kalle and gustav) and it is possible to do ping kalle PING kalle.asd.se (172.17.0.4) 56(84) bytes of data. 64 bytes from kalle.asd.se (172.17.0.4): icmp_seq=1 ttl=64 time=0.137 ms and ping gustav PING gustav.asd.se (172.17.0.5) 56(84) bytes of data. 64 bytes from gustav.asd.se (172.17.0.5): icmp_seq=1 ttl=64 time=0.147 ms Starting Erlang like this: erl -sname n -setcookie asd Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:4:4] [async-threads:10] [kernel-poll:false] Eshell V6.3 (abort with ^G) (n@REDACTED)1> net_kernel:connect('n@REDACTED'). false and, from the other side: erl -sname n -setcookie asd Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:4:4] [async-threads:10] [kernel-poll:false] Eshell V6.3 (abort with ^G) (n@REDACTED)1> net_kernel:connect('n@REDACTED'). false bengt (*) I manually added them to their respective /etc/hosts. If there is a way to do it automatically I would like to know. From mononcqc@REDACTED Mon May 16 17:09:34 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 16 May 2016 11:09:34 -0400 Subject: [erlang-questions] Third party libs you always use In-Reply-To: <573945E3.3070803@gmail.com> References: <573945E3.3070803@gmail.com> Message-ID: <20160516150933.GA43164@fhebert-ltm2.internal.salesforce.com> On 05/15, Kenneth Lakin wrote: >If auto accepted the same args as shell, and either switched to a >pure-erlang file change notification system or just dumped the output >that we get from inotifywait on startup and shutdown to /dev/null then >using it it *would* be a no-brainer. This doesn't sound particularly difficult to do, since rebar3_auto just calls the shell provider directly: https://github.com/tsloughter/rebar3_auto/blob/master/src/rebar3_auto.erl#L52 You probably just need to patch it with the argument list accepted by shell (see https://github.com/erlang/rebar3/blob/master/src/rebar_prv_shell.erl#L59-L79) for it to be able to pass them on to the shell provider transparently. Then you could call them with the same arguments. I haven't tried this -- I myself use triggers with shellrpc from my editor (https://github.com/ferd/rebar3_shellrpc) -- but I don't think it should be much harder than that. Regards, Fred. From peter.james.morgan@REDACTED Mon May 16 17:30:57 2016 From: peter.james.morgan@REDACTED (Peter Morgan) Date: Mon, 16 May 2016 16:30:57 +0100 Subject: [erlang-questions] how: Simple distributed Erlang in docker containers In-Reply-To: <5739C405.20205@ericsson.com> References: <5739C405.20205@ericsson.com> Message-ID: Hello - Shell A: docker run -it --rm erlang bash root@REDACTED:/# ip addr show dev eth0 794: eth0@REDACTED: mtu 1500 qdisc noqueue state UP group default link/ether 02:42:ac:11:00:07 brd ff:ff:ff:ff:ff:ff inet 172.17.0.7/16 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::42:acff:fe11:7/64 scope link valid_lft forever preferred_lft forever erl -name a@REDACTED -setcookie abc Erlang/OTP 18 [erts-7.3.1] [source] [64-bit] [smp:6:6] [async-threads:10] [hipe] [kernel-poll:false] Eshell V7.3.1 (abort with ^G) (a@REDACTED)1> Shell B: docker run -it --rm erlang bash 796: eth0@REDACTED: mtu 1500 qdisc noqueue state UP group default link/ether 02:42:ac:11:00:08 brd ff:ff:ff:ff:ff:ff inet 172.17.0.8/16 scope global eth0 valid_lft forever preferred_lft forever inet6 fe80::42:acff:fe11:8/64 scope link valid_lft forever preferred_lft forever erl -name b@REDACTED -setcookie abc Erlang/OTP 18 [erts-7.3.1] [source] [64-bit] [smp:6:6] [async-threads:10] [hipe] [kernel-poll:false] Eshell V7.3.1 (abort with ^G) (b@REDACTED)1> net_adm:ping('a@REDACTED'). pong (b@REDACTED)2> Take a look at the "--add-host=? parameter to docker run, rather than modifying /etc/hosts. Thanks Peter. > On 16 May 2016, at 13:58, Bengt Kleberg wrote: > > Greetings, > > What is needed to get two Erlang nodes talking if they are in Docker containers (on the same machine)? > net_kernel:connect/1 fails. > > > I have started 2 Docker containers (kalle and gustav) and it is possible to do > ping kalle > PING kalle.asd.se (172.17.0.4) 56(84) bytes of data. > 64 bytes from kalle.asd.se (172.17.0.4): icmp_seq=1 ttl=64 time=0.137 ms > > and > ping gustav > PING gustav.asd.se (172.17.0.5) 56(84) bytes of data. > 64 bytes from gustav.asd.se (172.17.0.5): icmp_seq=1 ttl=64 time=0.147 ms > > > Starting Erlang like this: > erl -sname n -setcookie asd > Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:4:4] [async-threads:10] [kernel-poll:false] > > Eshell V6.3 (abort with ^G) > (n@REDACTED)1> net_kernel:connect('n@REDACTED'). > false > > and, from the other side: > erl -sname n -setcookie asd > Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:4:4] [async-threads:10] [kernel-poll:false] > > Eshell V6.3 (abort with ^G) > (n@REDACTED)1> net_kernel:connect('n@REDACTED'). > false > > > bengt > (*) I manually added them to their respective /etc/hosts. If there is a way to do it automatically I would like to know. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Mon May 16 23:19:29 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 17 May 2016 00:19:29 +0300 Subject: [erlang-questions] leex question about context dependent parsing. Message-ID: Hi. I'm trying to build a leex + yacc parser for our Flussonic config syntax and have met problem. I want to split following text: stream s { url rtsp://10.1.2.3/h264?user=admin&password= rtp=udp; } into following stream of lexems: {stream, 1}, {string, 1, "s"}, {'{', 1}, {url, 2}, {string, 2, "rtsp://10.1.2.3/h264?user=admin&password="}, {string, 2, "rtp"}, {eq, 2}, {string, 2, "udp"}, {';', 2}, {'}', 3} At least I think that I want it. Problem with my approach is that it requires syntax dependent lexical analysis and leex doesn't allow to do it. If I consider "=" a split symbol, then first string after keyword url will be splitted. If I consider it a part of string, then it will not split key=value part of url specification. Maybe I'm doing it wrong? -------------- next part -------------- An HTML attachment was scrubbed... URL: From mandinkamohawk@REDACTED Mon May 16 22:13:44 2016 From: mandinkamohawk@REDACTED (Mr. T) Date: Mon, 16 May 2016 23:13:44 +0300 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? Message-ID: On Mon May 9 03:01:02 2016, Richard A. O'Keefe wrote: > > On 7/05/16 6:06 AM, Lyn Headley wrote: > > I recently conceived a sociotechnical process that would help us > > understand each other's code. It is simple to implement. In bare bones > > it is this: > > > > I am new to your project. As I read your code, certain events are > > recorded by my editor and sent to a log that you are monitoring. You > > can easily browse the log, see where I am bumping into issues, and > > easily provide feedback. > > One problem with this is that it is *syntactic*, not *semantic*. > The editor tells me *what* you are looking at, but not *why* > you are looking at it, not even whether it is what you *intended* > to look at or something you arrived at by accident. > > And since we are (probably) on opposite sides of the planet, you are > most unlikely to be able to give me feedback at the time I need it. > > That's it. > > > > To put some flesh on it, you might see: > > > > Lyn jumped from file:line to function f/2 in file2. > > Lyn expressed puzzlement. > > How does the editor know that Lyn expressed puzzlement? > Is it because the comment is in M?ori, or because it looks as > though there might be a division by zero, or because this is > Python3 and Lyn is used to Python2? > > Your feedback could be in the context of this series of events as > > well. So when I receive it, I am reminded of exactly where I was. As > > feedback I see the annotated log: > > > > Lyn jumped from file:line to function f/2 in file2. > > Lyn expressed puzzlement. > > [Joe said: That's an optimization for X case]. > > > > That's really it. This is easy to write and I think it's a huge win. > > What do you think? > *What* is easy to write? > I think we need a lot more detail before it is possible to make > that judgement. > > Possibly the easiest way to do this might be through some sort > of video screen capture with audio annotation, HANG ON... > > where Lyn *says* > she is puzzled, in which case it may be possible to try this out > using existing tools. > > I think something like this is DEFINITELY worth trying. ... I'd go further and say that this is a BRILLIANT, BRILLIANT, BRILLIANT idea! I'm reminded of those audio-logs in sci-fi movies where a mad scientist records running commentary of his experiments using a hand-held reporter recorder. The hope being; should he keel over from inhaling toxic fumes, other scientists who get hold of the logs should be able to not only understand *what* he was doing but also *why*, and possibly *where* he went wrong. I would ABSOLUTELY use a tool allowed me to attach audio-logs to coding session screen-scrapes as you describe especially if it were somehow integrated into both an editor/IDE (so I could annotate as I code without switching to another tool) and version-control (so I could play back annotations for commits). Even if v1.0 of such tool were for my own personal use only and not necessarily designed for collaboration (issues you raise below), the ability to *easily* archive the evolution of my own code -- mad-scientist-style -- would be worth a million bucks because it would solve two problems I have: 1. Whilst commit messages in version control can somewhat be used towards the same goal: When I'm in the zone I don't commit frequently enough, so my commit messages frequently aren?t descriptive enough. So I find myself having to look at all the commited code anyway. 2. I use something similar to Joe's "research" directories, but I find myself not keeping the notes therein up-to-date because of the time it takes. However, if all I have to do is speak, I?d be talking to my machine all the time! - Mr. T. > > It does have a downside. I started this thread talking about > 1 person leaving signposts (rhetorical structure annotations) > for N > 1 readers. The editing log idea is 1:1; if Helen wrote > the code and Lyn recorded herself viewing it and Helen > replies to Lyn, that helps Lyn, but it doesn't help Sue, unless > she knows that Helen's reply to Lyn's session exists. From serge@REDACTED Mon May 16 23:43:53 2016 From: serge@REDACTED (Serge Aleynikov) Date: Mon, 16 May 2016 17:43:53 -0400 Subject: [erlang-questions] leex question about context dependent parsing. In-Reply-To: References: Message-ID: Max, Take a look at the following project https://github.com/saleyn/erlcfg. Its syntax is very similar to yours and it's got the corresponding lexer/parser grammars there that you could use as the starting point. Regards, Serge On Mon, May 16, 2016 at 5:19 PM, Max Lapshin wrote: > Hi. I'm trying to build a leex + yacc parser for our Flussonic config > syntax and have met problem. > > I want to split following text: > > > stream s { > url rtsp://10.1.2.3/h264?user=admin&password= rtp=udp; > } > > into following stream of lexems: > > {stream, 1}, > {string, 1, "s"}, > {'{', 1}, > {url, 2}, > {string, 2, "rtsp://10.1.2.3/h264?user=admin&password="}, > {string, 2, "rtp"}, > {eq, 2}, > {string, 2, "udp"}, > {';', 2}, > {'}', 3} > > At least I think that I want it. > > Problem with my approach is that it requires syntax dependent lexical > analysis and leex doesn't allow to do it. If I consider "=" a split > symbol, then first string after keyword url will be splitted. If I consider > it a part of string, then it will not split key=value part of url > specification. > > Maybe I'm doing it wrong? > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon May 16 23:45:30 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 17 May 2016 00:45:30 +0300 Subject: [erlang-questions] leex question about context dependent parsing. In-Reply-To: References: Message-ID: Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Mon May 16 23:51:50 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Mon, 16 May 2016 22:51:50 +0100 Subject: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) In-Reply-To: <5736E1B4.80106@ninenines.eu> References: <5736E1B4.80106@ninenines.eu> Message-ID: On 14 May 2016 at 09:28, Lo?c Hoguin wrote: > On 05/14/2016 09:46 AM, Chandru wrote: > >> On 14 May 2016 at 00:59, Ryan Stewart > > wrote: >> >> Oliver: there is no restart for children under a simple 1-1 >> supervisor as documented under Supervision Principles at >> http://erlang.org/doc/man/supervisor.html#id243029 >> >> Otoh, I have to strongly disagree with Chandru about there being "no >> point" in supervising a connection process. Depending on the nature >> of your application, there can be a *huge* benefit in supervision, >> which is orderly shutdown. With a proper supervisory tree, you can >> ensure that all connection processes have a chance to finish their >> work and shut down cleanly when you stop your application. >> >> >> Cleanup can be done in the terminate callback if you use >> process_flag(trap_exit, true). You need to supervise a process primarily >> if you want it *restarted*, not for cleanup. Yes, you do have orderly >> shutdown mechanisms if the number of restarts exceed a "normal" value, >> but that is not the situation here. >> > > This is bad advice. Cleanup should be done in a separate process, > otherwise sending a kill signal sent to the wrong process will leave things > dirty. No, it's not. The reason a terminate callback is provided in a gen_server is so that a process can clean up when it terminates, not to delegate it to other processes. > Oliver has a server process to which clients connect. He does not need a >> supervisor for this because it is up to the client to reconnect if a >> connection is lost. >> > > This is incorrect. > > The primary function of supervisors is to provide a hierarchy of processes > belonging to an application. With such a hierarchy it becomes possible to > find and query information about any single process in your system in a > standard manner. > No, it's not. From the manual: The supervisor is responsible for starting, stopping and monitoring its child processes. The basic idea of a supervisor is that it shall keep its child processes alive by restarting them when necessary. > > Secondary functions of supervisors include restarting, reporting (with > SASL) and upgrades. But they're entirely optional. > See above - what you state is secondary is its primary process. In fact, making the process hierarchy discoverable is a secondary benefit of using a supervisor. > Having processes run without being attached to a supervisor is a big > mistake, because they either become invisible, or force you to implement a > non-standard way to find them. This makes introspecting a system more > complex than it should be. Look carefully at the example I provided in the gist and Oliver's use case. It is perfectly sound advice. If you are ever walking your supervisor hierarchy do something with your application, you are doing it wrong. Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Tue May 17 01:49:18 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 16 May 2016 19:49:18 -0400 Subject: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) In-Reply-To: References: <5736E1B4.80106@ninenines.eu> Message-ID: <20160516234917.GA3618@ferdmbp.local> On 05/16, Chandru wrote: > >No, it's not. The reason a terminate callback is provided in a gen_server >is so that a process can clean up when it terminates, not to delegate it to >other processes. > I'm gonna side with Lo?c here. The terminate callback is good for any process-local cleanup or optimistic work, but is by no means a safe way to terminate anything. For example, if you have many children to terminate and through some interleaving brutall_kill is triggered (or anyone calls exit(Pid, kill)), whatever work you wanted to do in terminate will be skipped by a non-trappable exit signal. Using terminate as your sole termination clean up is risky. It is better to assume that it will not be called every time, only in controlled terminations and some accidental ones. This is especially true of non-collected resources -- not ports nor ETS tables -- specifically live dependencies such as other processes mid-discussion. The other side has to be able to cope with the termination of its peer; this can be done through monitors, sometimes through link+trap_exit. If recovery is not possible, just dying is appropriate. > >No, it's not. From the manual: > >The supervisor is responsible for starting, stopping and monitoring its >child processes. The basic idea of a supervisor is that it shall keep its >child processes alive by restarting them when necessary. > In practice, the release handling mechanisms will make use of that supervision structure to walk the tree: that's why you declare whether a supervisor's child are workers or supervisors (leaf or inner node!) The tree is being walked the entire way through. That being said, I personally try to avoid calling the supervisor to know who its children are and prefer named nodes. For me the supervisor is first and foremost a definition of a unit of failure, of dependencies between workers or subtrees. > >Look carefully at the example I provided in the gist and Oliver's use case. >It is perfectly sound advice. If you are ever walking your supervisor >hierarchy do something with your application, you are doing it wrong. > See release upgrades; if you need to walk your entire system at once, doing it through supervisors is not a bad idea. Funnily enough, the supervision structure isn't all that is being trusted though. When an app is shut down, the application controller (or is it the master?) also runs through all of the processes on the node and looks for those for whose it is the group leader and then force kills them -- preventing the terminate function from being called. Regards, Fred. From mononcqc@REDACTED Tue May 17 01:51:06 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 16 May 2016 19:51:06 -0400 Subject: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) In-Reply-To: <20160516234917.GA3618@ferdmbp.local> References: <5736E1B4.80106@ninenines.eu> <20160516234917.GA3618@ferdmbp.local> Message-ID: <20160516235105.GB3618@ferdmbp.local> On 05/16, Fred Hebert wrote: >Funnily enough, the supervision structure isn't all that is being >trusted though. When an app is shut down, the application controller >(or is it the master?) also runs through all of the processes on the >node and looks for those for whose it is the group leader and then >force kills them -- preventing the terminate function from being >called. > I wanted to add that this is only done once all of the supervision trees are shut down recursively for that app. What is then cleaned up brutally is unsupervised processes. From nistrigunya@REDACTED Tue May 17 08:56:15 2016 From: nistrigunya@REDACTED (Avinash Dhumane) Date: Tue, 17 May 2016 12:26:15 +0530 Subject: [erlang-questions] Pattern match error in Maps Message-ID: avinash@REDACTED:~/tws$ erl Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] Eshell V7.3 (abort with ^G) 1> M = #{{a, 1} => 100, {a, 2} => 200, {a, 3} => 300}. #{{a,1} => 100,{a,2} => 200,{a,3} => 300} 2> lists:map(fun(N) -> #{{a, N} := V}= M, V div 10 end, lists:seq(1,3)). * 1: illegal map key in pattern 3> I can get it working with maps:get/2 But, I seem not to understand when and where may variables be used in the pattern. What am I missing? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Tue May 17 09:01:51 2016 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 17 May 2016 09:01:51 +0200 Subject: [erlang-questions] how: Simple distributed Erlang in docker containers In-Reply-To: References: <5739C405.20205@ericsson.com> Message-ID: <573AC1DF.9080807@ericsson.com> Thank you for showing a way forward. Since "-name n@REDACTED" is undocumented, I wonder if there is another possibility to use distributed Erlang with Docker? bengt On 05/16/2016 05:30 PM, Peter Morgan wrote: > Hello - > > Shell A: > > docker run -it --rm erlang bash > root@REDACTED:/# ip addr show dev eth0 > 794: eth0@REDACTED: mtu 1500 qdisc noqueue state UP group default > link/ether 02:42:ac:11:00:07 brd ff:ff:ff:ff:ff:ff > inet 172.17.0.7/16 scope global eth0 > valid_lft forever preferred_lft forever > inet6 fe80::42:acff:fe11:7/64 scope link > valid_lft forever preferred_lft forever > > erl -name a@REDACTED -setcookie abc > Erlang/OTP 18 [erts-7.3.1] [source] [64-bit] [smp:6:6] [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V7.3.1 (abort with ^G) > (a@REDACTED)1> > > > Shell B: > > docker run -it --rm erlang bash > 796: eth0@REDACTED: mtu 1500 qdisc noqueue state UP group default > link/ether 02:42:ac:11:00:08 brd ff:ff:ff:ff:ff:ff > inet 172.17.0.8/16 scope global eth0 > valid_lft forever preferred_lft forever > inet6 fe80::42:acff:fe11:8/64 scope link > valid_lft forever preferred_lft forever > > erl -name b@REDACTED -setcookie abc > Erlang/OTP 18 [erts-7.3.1] [source] [64-bit] [smp:6:6] [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V7.3.1 (abort with ^G) > (b@REDACTED)1> net_adm:ping('a@REDACTED'). > pong > (b@REDACTED)2> > > > Take a look at the "--add-host=? parameter to docker run, rather than modifying /etc/hosts. > > Thanks > Peter. > > >> On 16 May 2016, at 13:58, Bengt Kleberg wrote: >> >> Greetings, >> >> What is needed to get two Erlang nodes talking if they are in Docker containers (on the same machine)? >> net_kernel:connect/1 fails. >> >> >> I have started 2 Docker containers (kalle and gustav) and it is possible to do >> ping kalle >> PING kalle.asd.se (172.17.0.4) 56(84) bytes of data. >> 64 bytes from kalle.asd.se (172.17.0.4): icmp_seq=1 ttl=64 time=0.137 ms >> >> and >> ping gustav >> PING gustav.asd.se (172.17.0.5) 56(84) bytes of data. >> 64 bytes from gustav.asd.se (172.17.0.5): icmp_seq=1 ttl=64 time=0.147 ms >> >> >> Starting Erlang like this: >> erl -sname n -setcookie asd >> Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:4:4] [async-threads:10] [kernel-poll:false] >> >> Eshell V6.3 (abort with ^G) >> (n@REDACTED)1> net_kernel:connect('n@REDACTED'). >> false >> >> and, from the other side: >> erl -sname n -setcookie asd >> Erlang/OTP 17 [erts-6.3] [source] [64-bit] [smp:4:4] [async-threads:10] [kernel-poll:false] >> >> Eshell V6.3 (abort with ^G) >> (n@REDACTED)1> net_kernel:connect('n@REDACTED'). >> false >> >> >> bengt >> (*) I manually added them to their respective /etc/hosts. If there is a way to do it automatically I would like to know. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From wallentin.dahlberg@REDACTED Tue May 17 09:28:29 2016 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Tue, 17 May 2016 09:28:29 +0200 Subject: [erlang-questions] Pattern match error in Maps In-Reply-To: References: Message-ID: The problem is the key {a,N} in #{{a, N} := V}. The compiler only accepts literals or variables, not arbitrary expressions. Here you need to eval {a,N} first and bind it to a variable. // Bj?rn-Egil 2016-05-17 8:56 GMT+02:00 Avinash Dhumane : > avinash@REDACTED:~/tws$ erl > Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:2:2] [async-threads:10] > [hipe] [kernel-poll:false] > > Eshell V7.3 (abort with ^G) > 1> M = #{{a, 1} => 100, {a, 2} => 200, {a, 3} => 300}. > #{{a,1} => 100,{a,2} => 200,{a,3} => 300} > 2> lists:map(fun(N) -> #{{a, N} := V}= M, V div 10 end, lists:seq(1,3)). > * 1: illegal map key in pattern > 3> > > I can get it working with maps:get/2 > > But, I seem not to understand when and where may variables be used in the > pattern. > > What am I missing? > > Thanks > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Oliver.Korpilla@REDACTED Tue May 17 10:04:21 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Tue, 17 May 2016 10:04:21 +0200 Subject: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) In-Reply-To: <20160516234917.GA3618@ferdmbp.local> References: <5736E1B4.80106@ninenines.eu> , <20160516234917.GA3618@ferdmbp.local> Message-ID: Hello, Fred. ? > Funnily enough, the supervision structure isn't all that is being > trusted though. When an app is shut down, the application controller (or > is it the master?) also runs through all of the processes on the node > and looks for those for whose it is the group leader and then force > kills them -- preventing the terminate function from being called. Thanks for this information, I did not know that! Soooo... Do supervisors with strategies _other than_ simple_one_for_one restart dynamically started children? Like: I add a transient child dynamically with its full child spec. Will one_for_one restart it with its original parameters if it fails? Beyond that, my scenario is the following: I have thousands of clients at any time. These may run semi-complex procedures, setting stuff up, changing their centrally managed communication, quitting. I personally thought this was exactly the scenario transient was made for - cleaning up behind workers who exit normally (I mean: releasing the child info from the supervisor data structures), allowing restart of those who crashed. But in my system even those exiting normally seem to persist in the supervisor DS, which (I thought) according to the very definition of transient should not happen and will leak memory (and performance when walking lists) eventually. Now, since I call terminate_child in the supervisor itself on the children that may cause the problem. This may be a bug in my design, which is like this: Level A) Supervisor (one_for_one strategy) -- 1 to n relationship --> Level B) Semi-permanent Worker that runs individual procedures in parallel and acts as monitor -- 1 to n relationship --> Level C) Individual short-lived procedure spawned to run one short message sequence that ends with an update of system or client state Now, I want my Semi-Permanent Worker (B) as stateless as possible. It maintains a set of flags that can be reloaded from DB at any time so that it can be determined which procedures are allowed to start. It starts C procedures (or routes messages to running ones) and monitors them. It acts like a supervisor that needs to know more about the children because it is designed to start the right ones, so I implemented it as a worker. The individual C procedures are short, either honoring the OTP principles or simple gen_fsms, walking through one or several steps of message exchange in the predefined protocol with the client. This would all be fine and dandy, but it is an individual procedure that is handling client shutdown (and hence the need to terminate its "boss" B). Currently I finish it by making a call to the supervisor to terminate the boss child and then cleanup occurs. This works but requires me still to call delete_child in order to make sure the supervisor data structures are not full of zombie children. I could also signal in the C child procedure exit to its B boss monitor that this shutdown means a takedown of B altogether ("no more interaction with this client"). I don't want to have an error message pop up because of this (as some EXIT signals produce automatically, it seems) as this is supposed to be a regular case. So, maybe exit with shutdown? The worst solution to me would be if the boss worker needs to track if one of the procedures he runs is a shutdown of a client and finish when it finishes. That would imply giving it more state and logic which I hoped to avoid. The end result, however, is that I want the supervisor to keep as clean an internal state as possible because not only there will be 1000s of clients in the system at any time, hopefully this will sum up to millions of clients during its uptime due to the transient nature of these clients. Now, can this requirement be fulfilled with existing one_for_one supervisor? Is there any scenario where one_for_one guaranteedly cleans up the child from its data structures when it exits normally when it is configured transient? Or is there no such scenario? Previous emails left me confused about this. Thank you and cheers, Oliver From per@REDACTED Tue May 17 11:05:14 2016 From: per@REDACTED (Per Hedeland) Date: Tue, 17 May 2016 11:05:14 +0200 (CEST) Subject: [erlang-questions] erlang:raise/3 considered harmful? In-Reply-To: Message-ID: <201605170905.u4H95E8F050625@pluto.hedeland.org> =?UTF-8?Q?Bj=C3=B6rn_Gustavsson?= wrote: > >I agree. I will remove the warning in OTP 19. Well, I think a "milder" warning, excluding the specific case of re-raising a caught exception, could be motivated - e.g. using it to synthesize an exception that didn't actually occur is probably on par with using list_to_pid/1 (or worse). Or, perhaps it would be possible to have a raise/0, that was restricted to raising the "currently caught" exception? That one wouln't need a warning IMHO... --Per From grimmer.matthias@REDACTED Tue May 17 09:09:23 2016 From: grimmer.matthias@REDACTED (Matthias Grimmer) Date: Tue, 17 May 2016 09:09:23 +0200 Subject: [erlang-questions] Call for Papers: 1st Workshop on ReUsable and Modular Programming Language Ecosystems (RUMPLE'16) Message-ID: ============================================================================ Call for Papers: RUMPLE?16 1st Workshop on ReUsable and Modular Programming Language Ecosystems Co-located with SPLASH Oct/Nov, 2016, Amsterdam, Netherlands http://2016.splashcon.org/track/rumple2016 ============================================================================ The RUMPLE workshop is a venue for discussing modular approaches to programming language implementations, extensible virtual machine architectures, as well as reusable runtime components such as dynamic compilers, interpreters, or garbage collectors. The main goal of the workshop is to bring together researchers and practitioners, and facilitate the sharing of experiences and ideas. Relevant topics include, but are definitely not limited to, the following: - Extensible VM design (compiler- or interpreter-based VMs) - Reusable implementation of runtime components (e.g. interpreters, garbage collectors, intermediate representations) - Static and dynamic compiler techniques for different languages - Multi-language runtimes and mechanisms for cross-language interoperability between different languages - Tooling support for different languages (e.g. debugging, profiling, etc.) - Modular language implementations that use existing frameworks and systems - Case studies of existing language implementations, virtual machines, and runtime components (e.g. design choices, tradeoffs, etc.) - New research ideas on how we want to build languages in the future. Workshop Format and Submissions This workshop welcomes the presentation and discussion of new ideas and emerging problems that give a chance for interaction and exchange. We accept presentation proposals in the form of extended abstracts (1-4 pages). Accepted abstracts will be published on the workshop's website before the workshop date. Submissions should use the ACM SIGPLAN Conference Format, 10 point font, using the font family Times New Roman and numeric citation style. All submissions should be in PDF format. Please submit abstracts through http://ssw.jku.at/rumple/ Important Dates - Exended abstract submission: 1 Aug 2016 - Author notification: 5 Sep 2016 All deadlines are Anywhere on Earth (AoE), i.e. GMT/UTC?12:00 hour - Workshop: 31 Oct 2016 Program Committee Walter Binder, University of Lugano Carl Friedrich Bolz, King's College London Richard Jones, University of Kent Stephen Kell, University of Cambridge Jan Vitek, Northeastern University Christian Wimmer, Oracle Labs Workshop Organizers Matthias Grimmer, Johannes Kepler University Linz, Austria Laurence Tratt, King's College London, United Kingdom Adam Welc, Oracle Labs, United States For questions or concerns, please mail to matthias.grimmer at jku.at From liuzhongzheng2012@REDACTED Tue May 17 12:08:31 2016 From: liuzhongzheng2012@REDACTED (Zhongzheng Liu) Date: Tue, 17 May 2016 18:08:31 +0800 Subject: [erlang-questions] How to compile faster ? Message-ID: Hi maillist: I have a erlang project with ~500 modules, all modules including a hrl file which contains over 1000 record definition (generated by protobuf). It takes a long time to compile this project. Is there any compile option which can make compiling faster? I don't care the performance of the code running (because I use it in develop environment only), and I don't want to change the source code (cannot spit the big hrl to smaller files). Thank you. From sverker.eriksson@REDACTED Tue May 17 15:10:48 2016 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Tue, 17 May 2016 15:10:48 +0200 Subject: [erlang-questions] How to disable parentheses matching in Erlang shell? In-Reply-To: References: <6013b959-58b0-a963-9478-d6845993f024@gjunka.com> <57379656.6010306@gmail.com> <5481da92-8dbb-750e-8ced-c5ffafd2e4ce@gjunka.com> <03e2c191-5810-2b29-6dea-38cafaa7d2cb@gjunka.com> Message-ID: <573B1858.1070203@ericsson.com> Could you test this fix and see if it makes the cursor behave when connected with to_erl/run_erl. git fetch https://github.com/sverker/otp sverker/ttsl_drv-move-down /Sverker, Erlang/OTP On 05/15/2016 11:45 PM, Grzegorz Junka wrote: > > I just realized what is likely causing the problem. When I start the > shell with erl and type in this testing string (with spaces so it's > longer) it all works fine: > > lists:sort([{a, 1}, {b, 2}, {c, 3}, {d, 4}, {e, 5}, {f, 6}, {g, 7}, > {h, 8}, {i, 9}, {j, 10}]). > > But I very rarely use Erlang shell in this way. Most of the time I am > connecting to running Erlang nodes using to_erl specifying the shell's > PID. Then it breaks, probably because my terminal size doesn't match > the default terminal size with which the shell has been originally > started. > > When I then try to type in the same example the shell will match each > {} correctly but as soon as I type ']' to close the list the cursor > jumps to the line above, then some other place when I type ')'. > > So it's not related to the environment but to the way I am using > Erlang shell. Is there any solution to that? > > Grzegorz > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From list1@REDACTED Tue May 17 15:20:17 2016 From: list1@REDACTED (Grzegorz Junka) Date: Tue, 17 May 2016 13:20:17 +0000 Subject: [erlang-questions] How to compile faster ? In-Reply-To: References: Message-ID: Ho do you compile? Do you use a makefile or rebar or something else? Also, are all those modules in the same application/src file or in different applications? On 17/05/2016 10:08, Zhongzheng Liu wrote: > Hi maillist: > > I have a erlang project with ~500 modules, all modules including a hrl > file which contains over 1000 record definition (generated by > protobuf). > > It takes a long time to compile this project. Is there any compile > option which can make compiling faster? > > > I don't care the performance of the code running (because I use it in > develop environment only), > and I don't want to change the source code (cannot spit the big hrl > to smaller files). > > > Thank you. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From list1@REDACTED Tue May 17 19:01:28 2016 From: list1@REDACTED (Grzegorz Junka) Date: Tue, 17 May 2016 17:01:28 +0000 Subject: [erlang-questions] How to disable parentheses matching in Erlang shell? In-Reply-To: <573B1858.1070203@ericsson.com> References: <6013b959-58b0-a963-9478-d6845993f024@gjunka.com> <57379656.6010306@gmail.com> <5481da92-8dbb-750e-8ced-c5ffafd2e4ce@gjunka.com> <03e2c191-5810-2b29-6dea-38cafaa7d2cb@gjunka.com> <573B1858.1070203@ericsson.com> Message-ID: :( As soon as I closed the bracket after typing {e, 5} from my example the cursor jumped some 10 columns to the right and then to the beginning of the line. That's how far it got (without the fix the cursor would jump incorrectly only after closing ']'). (sc-1@REDACTED)1> lists:sort([{a, 1}, {b, 2}, {c, 3}, {d, 4}, {e, 5} Grzegorz On 17/05/2016 13:10, Sverker Eriksson wrote: > Could you test this fix and see if it makes the cursor behave > when connected with to_erl/run_erl. > > git fetch https://github.com/sverker/otp sverker/ttsl_drv-move-down > > /Sverker, Erlang/OTP > > > On 05/15/2016 11:45 PM, Grzegorz Junka wrote: >> >> I just realized what is likely causing the problem. When I start the >> shell with erl and type in this testing string (with spaces so it's >> longer) it all works fine: >> >> lists:sort([{a, 1}, {b, 2}, {c, 3}, {d, 4}, {e, 5}, {f, 6}, {g, 7}, >> {h, 8}, {i, 9}, {j, 10}]). >> >> But I very rarely use Erlang shell in this way. Most of the time I am >> connecting to running Erlang nodes using to_erl specifying the >> shell's PID. Then it breaks, probably because my terminal size >> doesn't match the default terminal size with which the shell has been >> originally started. >> >> When I then try to type in the same example the shell will match each >> {} correctly but as soon as I type ']' to close the list the cursor >> jumps to the line above, then some other place when I type ')'. >> >> So it's not related to the environment but to the way I am using >> Erlang shell. Is there any solution to that? >> >> Grzegorz >> >> > From list1@REDACTED Tue May 17 19:55:22 2016 From: list1@REDACTED (Grzegorz Junka) Date: Tue, 17 May 2016 17:55:22 +0000 Subject: [erlang-questions] How to disable parentheses matching in Erlang shell? In-Reply-To: References: <6013b959-58b0-a963-9478-d6845993f024@gjunka.com> <57379656.6010306@gmail.com> <5481da92-8dbb-750e-8ced-c5ffafd2e4ce@gjunka.com> <03e2c191-5810-2b29-6dea-38cafaa7d2cb@gjunka.com> <573B1858.1070203@ericsson.com> Message-ID: Sverker, I have a simple application which you could quickly compile to be able to easily reproduce this issue. Just execute those steps (one by one): mkdir test_sc cd test_sc/ git clone https://github.com/builderl/ex1_simple_cache.git cd ex1_simple_cache/ gmake get-deps gmake dev ./bin/init.esh ./bin/start.esh to_erl ../sc-1/shell/ This will pull an application, get all its dependencies, compile, install, configure, start and connect to one of the running nodes (there will be two running nodes from the same release). I hope that will be helpful but of course I am happy to recompile the OTP from your branch and retry if needed. Grzegorz On 17/05/2016 17:01, Grzegorz Junka wrote: > :( > > As soon as I closed the bracket after typing {e, 5} from my example > the cursor jumped some 10 columns to the right and then to the > beginning of the line. That's how far it got (without the fix the > cursor would jump incorrectly only after closing ']'). > > (sc-1@REDACTED)1> lists:sort([{a, 1}, {b, 2}, {c, 3}, {d, > 4}, {e, 5} > > Grzegorz > > > On 17/05/2016 13:10, Sverker Eriksson wrote: >> Could you test this fix and see if it makes the cursor behave >> when connected with to_erl/run_erl. >> >> git fetch https://github.com/sverker/otp sverker/ttsl_drv-move-down >> >> /Sverker, Erlang/OTP >> >> >> On 05/15/2016 11:45 PM, Grzegorz Junka wrote: >>> >>> I just realized what is likely causing the problem. When I start the >>> shell with erl and type in this testing string (with spaces so it's >>> longer) it all works fine: >>> >>> lists:sort([{a, 1}, {b, 2}, {c, 3}, {d, 4}, {e, 5}, {f, 6}, {g, 7}, >>> {h, 8}, {i, 9}, {j, 10}]). >>> >>> But I very rarely use Erlang shell in this way. Most of the time I >>> am connecting to running Erlang nodes using to_erl specifying the >>> shell's PID. Then it breaks, probably because my terminal size >>> doesn't match the default terminal size with which the shell has >>> been originally started. >>> >>> When I then try to type in the same example the shell will match >>> each {} correctly but as soon as I type ']' to close the list the >>> cursor jumps to the line above, then some other place when I type ')'. >>> >>> So it's not related to the environment but to the way I am using >>> Erlang shell. Is there any solution to that? >>> >>> Grzegorz >>> >>> >> > From silviu.cpp@REDACTED Tue May 17 21:38:36 2016 From: silviu.cpp@REDACTED (Caragea Silviu) Date: Tue, 17 May 2016 22:38:36 +0300 Subject: [erlang-questions] best way to update a nested map Message-ID: Hello, What's the best way to update a nested map ? I tried something like : M = #{ a => 1, c =>#{x => 1, y => 7}}. M1 = M#{c}#{y => 4711}. As described here: http://www.erlang-factory.com/upload/presentations/779/WhereareweontheMap.pdf But for some reason I get an error : syntax error before: '}' Silviu -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethlakin@REDACTED Tue May 17 21:53:50 2016 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Tue, 17 May 2016 12:53:50 -0700 Subject: [erlang-questions] best way to update a nested map In-Reply-To: References: Message-ID: <573B76CE.2020203@gmail.com> On 05/17/2016 12:38 PM, Caragea Silviu wrote: > M = #{ a => 1, c =>#{x => 1, y => 7}}. > M1 = M#{c}#{y => 4711}. AFAIK, you have to capture the nested map to update (rather than replace) it: M = #{ a => 1, c => #{x => 1, y => 7}}. #{c := C} = M. M1 = M#{c := C#{y := 4771}}. The slide that lists the syntax you want to use (and the two proceeding it) *appear* to be slides that describe possible future syntax. You must currently use pattern matching or maps:get/2,3 to pluck out values from a map. This might be better to learn maps syntax from: http://erlang.org/doc/reference_manual/expressions.html#id81479 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From chandrashekhar.mullaparthi@REDACTED Tue May 17 22:09:14 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Tue, 17 May 2016 21:09:14 +0100 Subject: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) In-Reply-To: <20160516234917.GA3618@ferdmbp.local> References: <5736E1B4.80106@ninenines.eu> <20160516234917.GA3618@ferdmbp.local> Message-ID: On 17 May 2016 at 00:49, Fred Hebert wrote: > On 05/16, Chandru wrote: > >> >> No, it's not. The reason a terminate callback is provided in a gen_server >> is so that a process can clean up when it terminates, not to delegate it >> to >> other processes. >> >> > I'm gonna side with Lo?c here. The terminate callback is good for any > process-local cleanup or optimistic work, but is by no means a safe way to > terminate anything. > > For example, if you have many children to terminate and through some > interleaving brutall_kill is triggered (or anyone calls exit(Pid, kill)), > whatever work you wanted to do in terminate will be skipped by a > non-trappable exit signal. > Agreed, but how does adding the connection handling process (in Oliver's use case) to a simple_one_for_one supervisor help? It doesn't give him anything other than the illusion of being "supervised". Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Tue May 17 22:14:27 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Tue, 17 May 2016 21:14:27 +0100 Subject: [erlang-questions] [ANN] erpc Message-ID: I'm pleased to announce the first release of erpc. From the README This application is intended to be a replacement for native RPC. It tries to mimic the native rpc API as far as possible. It allows unidirectional connections between nodes. Where bi-directional connections are required, they need to be configured explicitly. There is host level and node level ACLs configurable. It is possible to setup application specific connections between nodes. It supports multiple TCP/TLS connections per host and load balancing of traffic across nodes. We haven't yet deployed this in production but it is giving us good results in load tests for some of our systems. We are releasing it early so that we can develop this in the open with community feedback and contributions. https://github.com/bet365/erpc Many thanks to bet365 for open sourcing this. cheers, Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Tue May 17 22:31:03 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Tue, 17 May 2016 21:31:03 +0100 Subject: [erlang-questions] how: Simple distributed Erlang in docker containers In-Reply-To: <573AC1DF.9080807@ericsson.com> References: <5739C405.20205@ericsson.com> <573AC1DF.9080807@ericsson.com> Message-ID: On 17 May 2016 at 08:01, Bengt Kleberg wrote: > Thank you for showing a way forward. > > > Since "-name n@REDACTED" is undocumented, I wonder if there is another > possibility to use distributed Erlang with Docker? > > > Why do you say it is undocumented? From http://erlang.org/doc/man/erl.html *-name Name* Makes the Erlang runtime system into a distributed node. This flag invokes all network servers necessary for a node to become distributed. See net_kernel(3) . It is also ensured that epmd runs on the current host before Erlang is started. See epmd(1) . *The name of the node will be Name@REDACTED, where Host is the fully qualified host name of the current host*. For short names, use the -sname flag instead. Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Tue May 17 23:31:25 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Tue, 17 May 2016 21:31:25 +0000 Subject: [erlang-questions] [ANN] erpc In-Reply-To: References: Message-ID: nice! How does it compare to gen_rpc? When you say "It supports multiple TCP/TLS connections per host " does it means that we can multiple clients connected to the same node? - beno?t On Tue, 17 May 2016 at 22:14, Chandru wrote: > I'm pleased to announce the first release of erpc. From the README > > This application is intended to be a replacement for native > RPC. It tries to mimic the native rpc API as far as possible. It > allows unidirectional connections between nodes. Where > bi-directional connections are required, they need to be > configured explicitly. There is host level and node level ACLs > configurable. It is possible to setup application specific > connections between nodes. It supports multiple TCP/TLS > connections per host and load balancing of traffic across nodes. > > We haven't yet deployed this in production but it is giving us good > results in load tests for some of our systems. We are releasing it early so > that we can develop this in the open with community feedback and > contributions. > > https://github.com/bet365/erpc > > Many thanks to bet365 for open sourcing this. > > cheers, > Chandru > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Tue May 17 23:41:16 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Tue, 17 May 2016 22:41:16 +0100 Subject: [erlang-questions] [ANN] erpc In-Reply-To: References: Message-ID: On 17 May 2016 at 22:31, Benoit Chesneau wrote: > nice! How does it compare to gen_rpc? > Thank you. Similar aims (i.e. remove the problems caused by native RPC implementation). - Tries to reduce the use of message passing. - Supports pluggable transport protocols so that you can run erpc over TLS. - SCTP support is on the roadmap. - Supports load balancing across multiple nodes so the client doesn't need to care about trying multiple nodes in case one of the server nodes is down. Currently planning to add a consistent-hashing based load balancing but haven't quite decided on an API yet. > > When you say "It supports multiple TCP/TLS connections per host " does it > means that we can multiple clients connected to the same node? > Yes, with different names. That way each application can have its own dedicated connection(s) so that it doesn't suffer from sharing connections. For a single named connection, you can also have multiple transport connections. Also, connections are uni-directional so you have complete control over traffic flows. cheers, Chandru > On Tue, 17 May 2016 at 22:14, Chandru < > chandrashekhar.mullaparthi@REDACTED> wrote: > >> I'm pleased to announce the first release of erpc. From the README >> >> This application is intended to be a replacement for native >> RPC. It tries to mimic the native rpc API as far as possible. It >> allows unidirectional connections between nodes. Where >> bi-directional connections are required, they need to be >> configured explicitly. There is host level and node level ACLs >> configurable. It is possible to setup application specific >> connections between nodes. It supports multiple TCP/TLS >> connections per host and load balancing of traffic across nodes. >> >> We haven't yet deployed this in production but it is giving us good >> results in load tests for some of our systems. We are releasing it early so >> that we can develop this in the open with community feedback and >> contributions. >> >> https://github.com/bet365/erpc >> >> Many thanks to bet365 for open sourcing this. >> >> cheers, >> Chandru >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.ashmun@REDACTED Wed May 18 01:47:35 2016 From: john.ashmun@REDACTED (John R. Ashmun) Date: Tue, 17 May 2016 16:47:35 -0700 Subject: [erlang-questions] Can werl be detached? Message-ID: Perhaps my subject is not quite the correct question. Here's my context: I'm beginning to recreate in Erlang on Microsoft Windows 10 a public domain program I used to enjoy on my Commodore Amiga: mATC was a game in which the user acted as a military Air Traffic Controller. I am using a wx window to draw the player's map with aircraft data blocks overlaid. I don't have a use for the initial window that opens when werl is started. http://erlang.org/doc/man/werl.html says, in part, "All flags except -oldshell work as they do for the erl command." I don't have a UNIX-y system available to me to learn how they work for the erl command. (The erl command as an interactive program is iffy in a Windows Command Prompt window.) In particular, the description of -detached in http://erlang.org/doc/man/erl.html would seem to be what I want: "Starts the Erlang runtime system detached from the system console. Useful for running daemons and backgrounds processes. Implies -noinput." I want the Erlang VM to run my game without an Erlang shell's having opened a window, and I want my game to be in control of its wx window. Further, following what I think I learned from "Erlang and OTP in Action", I have an OTP application being started by a boot file created from a .rel file. All of this is in the setting of an ancient Erlang release: R15B. If I use this Command Prompt batch file, mATC.bat: werl -pa ebin -pa mATC_app\ebin -detached -boot mATC -config sys a Windows error message window appears that says: "Failed to execute C:\Program Files\erl5.9\erts5.9\bin\beam.smp.dll The system cannot find the file specified." The DLL is actually present at that location, and of course everything runs well when I don't use -detached. Please advise. Regards, John Ashmun -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Wed May 18 02:42:14 2016 From: rexxe98@REDACTED (Andrew Berman) Date: Wed, 18 May 2016 00:42:14 +0000 Subject: [erlang-questions] Releases and Different OSs Message-ID: Hi List, So today I tried to create a release using rebar3 from my Mac and deploy it to a CentOS box. Everything works when I do not include ERTS (and have Erlang installed on the target), but if I do include ERTS, things fail (can't run run_erl), I'm assuming because the Mac version of Erlang can't be run on CentOS, which makes sense. So, I wanted to know how people do deployments of their Erlang releases. Do you set up boxes with each OS that matches your target OS and compile on there, including ERTS in the release? Do you not include ERTS and make sure that all your Erlang installs are the exact same version? Just trying to figure out a good workflow. Thanks for any advice! Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Wed May 18 02:58:38 2016 From: rexxe98@REDACTED (Andrew Berman) Date: Wed, 18 May 2016 00:58:38 +0000 Subject: [erlang-questions] Releases and Different OSs In-Reply-To: References: Message-ID: So you're saying to download a binary of Erlang compiled on Linux and put it on my Mac and then have it include that when building the release tar? On Tue, May 17, 2016 at 5:55 PM Vincent A Okonkwo wrote: > Hi Andrew, > > Been awhile since I tried this but if you are using relx there is > a --include-erts=$PATH_TO_EXTRACTED_LINUX_ERTS ( {include_erts, > $PATH_TO_EXTRACTED_LINUX_ERTS} for rebar) that you can use > > On Wed, May 18, 2016 at 12:42 AM Andrew Berman wrote: > >> Hi List, >> >> So today I tried to create a release using rebar3 from my Mac and deploy >> it to a CentOS box. Everything works when I do not include ERTS (and have >> Erlang installed on the target), but if I do include ERTS, things fail >> (can't run run_erl), I'm assuming because the Mac version of Erlang can't >> be run on CentOS, which makes sense. So, I wanted to know how people do >> deployments of their Erlang releases. Do you set up boxes with each OS >> that matches your target OS and compile on there, including ERTS in the >> release? Do you not include ERTS and make sure that all your Erlang >> installs are the exact same version? Just trying to figure out a good >> workflow. >> >> Thanks for any advice! >> >> Andrew >> > _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Wed May 18 03:32:03 2016 From: zxq9@REDACTED (zxq9) Date: Wed, 18 May 2016 10:32:03 +0900 Subject: [erlang-questions] Can werl be detached? In-Reply-To: References: Message-ID: <6284265.hgQ7dGlg6e@burrito> On Tuesday 17 May 2016 16:47:35 John R. Ashmun wrote: > Perhaps my subject is not quite the correct question. Here's my context: > > I'm beginning to recreate in Erlang on Microsoft Windows 10 a public domain > program I used to enjoy on my Commodore Amiga: mATC was a game in which > the user acted as a military Air Traffic Controller. I am using a wx > window to draw the player's map with aircraft data blocks overlaid. I > don't have a use for the initial window that opens when werl is started. Awesome! > If I use this Command Prompt batch file, mATC.bat: > > werl -pa ebin -pa mATC_app\ebin -detached -boot mATC -config sys > > a Windows error message window appears that says: > > "Failed to execute C:\Program Files\erl5.9\erts5.9\bin\beam.smp.dll > The system cannot find the file specified." > > The DLL is actually present at that location, and of course everything runs > well when I don't use -detached. Hrm... that seems odd, but I have to admit I have no experience running actual releases on Windows. What I usually do instead is have the runtime installed on the Windows machine, unpack project code and kick things off with an escript that builds and then launches the code I want to run. Changing from releases to from-source build->run may require a bit of shuffling -- and may or may not be worth it depending on the project. But this has been very effective and lightweight for my purposes. Like everything else on Windows there are a few quirks to making escripts a click-to-run experience (specifying full path to "escript.exe" instead of "escript" in your launcher, space for unheeded shebang, etc.). I've wanted for a few years now to formalize and simplify a solution to this... but real life gets in the way of the things I wish I could do as community project work. :-( The escript-to-bootstrap approach has made my Erlang GUI experience on Windows fairly painless -- I can develop on Linux or BSD and almost always get away with relying on anything that works there (to include wx) working pretty much the same way on Windows. I have not tried this on Windows 10 yet, but I assume it will continue to work the same way it does on Windows 7 and 8.1. -Craig From john.ashmun@REDACTED Wed May 18 03:48:35 2016 From: john.ashmun@REDACTED (John R. Ashmun) Date: Tue, 17 May 2016 18:48:35 -0700 Subject: [erlang-questions] Can werl be detached? In-Reply-To: <6284265.hgQ7dGlg6e@burrito> References: <6284265.hgQ7dGlg6e@burrito> Message-ID: On Tue, May 17, 2016 at 6:32 PM, zxq9 wrote: > On Tuesday 17 May 2016 16:47:35 John R. Ashmun wrote: > > Perhaps my subject is not quite the correct question. Here's my context: > > > > I'm beginning to recreate in Erlang on Microsoft Windows 10 a public > domain > > program I used to enjoy on my Commodore Amiga: mATC was a game in which > > the user acted as a military Air Traffic Controller. I am using a wx > > window to draw the player's map with aircraft data blocks overlaid. I > > don't have a use for the initial window that opens when werl is started. > > Awesome! > > > If I use this Command Prompt batch file, mATC.bat: > > > > werl -pa ebin -pa mATC_app\ebin -detached -boot mATC -config sys > > > > a Windows error message window appears that says: > > > > "Failed to execute C:\Program Files\erl5.9\erts5.9\bin\beam.smp.dll > > The system cannot find the file specified." > > > > The DLL is actually present at that location, and of course everything > runs > > well when I don't use -detached. > > Hrm... that seems odd, but I have to admit I have no experience running > actual releases on Windows. What I usually do instead is have the runtime > installed on the Windows machine, unpack project code and kick things off > with an escript that builds and then launches the code I want to run. > > Changing from releases to from-source build->run may require a bit of > shuffling -- and may or may not be worth it depending on the project. > But this has been very effective and lightweight for my purposes. > > Like everything else on Windows there are a few quirks to making escripts > a click-to-run experience (specifying full path to "escript.exe" instead > of "escript" in your launcher, space for unheeded shebang, etc.). I've > wanted for a few years now to formalize and simplify a solution to this... > but real life gets in the way of the things I wish I could do as community > project work. :-( > > The escript-to-bootstrap approach has made my Erlang GUI experience on > Windows fairly painless -- I can develop on Linux or BSD and almost always > get away with relying on anything that works there (to include wx) working > pretty much the same way on Windows. > > I have not tried this on Windows 10 yet, but I assume it will continue to > work the same way it does on Windows 7 and 8.1. > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > I will look into escript, thanks. John -------------- next part -------------- An HTML attachment was scrubbed... URL: From codmajik@REDACTED Wed May 18 02:55:06 2016 From: codmajik@REDACTED (Vincent A Okonkwo) Date: Wed, 18 May 2016 00:55:06 +0000 Subject: [erlang-questions] Releases and Different OSs In-Reply-To: References: Message-ID: Hi Andrew, Been awhile since I tried this but if you are using relx there is a --include-erts=$PATH_TO_EXTRACTED_LINUX_ERTS ( {include_erts, $PATH_TO_EXTRACTED_LINUX_ERTS} for rebar) that you can use On Wed, May 18, 2016 at 12:42 AM Andrew Berman wrote: > Hi List, > > So today I tried to create a release using rebar3 from my Mac and deploy > it to a CentOS box. Everything works when I do not include ERTS (and have > Erlang installed on the target), but if I do include ERTS, things fail > (can't run run_erl), I'm assuming because the Mac version of Erlang can't > be run on CentOS, which makes sense. So, I wanted to know how people do > deployments of their Erlang releases. Do you set up boxes with each OS > that matches your target OS and compile on there, including ERTS in the > release? Do you not include ERTS and make sure that all your Erlang > installs are the exact same version? Just trying to figure out a good > workflow. > > Thanks for any advice! > > Andrew > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From codmajik@REDACTED Wed May 18 03:05:54 2016 From: codmajik@REDACTED (Vincent A Okonkwo) Date: Wed, 18 May 2016 01:05:54 +0000 Subject: [erlang-questions] Releases and Different OSs In-Reply-To: References: Message-ID: Yes, that's what I meant. On Wed, May 18, 2016 at 12:58 AM Andrew Berman wrote: > So you're saying to download a binary of Erlang compiled on Linux and put > it on my Mac and then have it include that when building the release tar? > > On Tue, May 17, 2016 at 5:55 PM Vincent A Okonkwo > wrote: > >> Hi Andrew, >> >> Been awhile since I tried this but if you are using relx there is >> a --include-erts=$PATH_TO_EXTRACTED_LINUX_ERTS ( {include_erts, >> $PATH_TO_EXTRACTED_LINUX_ERTS} for rebar) that you can use >> >> On Wed, May 18, 2016 at 12:42 AM Andrew Berman wrote: >> >>> Hi List, >>> >>> So today I tried to create a release using rebar3 from my Mac and deploy >>> it to a CentOS box. Everything works when I do not include ERTS (and have >>> Erlang installed on the target), but if I do include ERTS, things fail >>> (can't run run_erl), I'm assuming because the Mac version of Erlang can't >>> be run on CentOS, which makes sense. So, I wanted to know how people >>> do deployments of their Erlang releases. Do you set up boxes with each OS >>> that matches your target OS and compile on there, including ERTS in the >>> release? Do you not include ERTS and make sure that all your Erlang >>> installs are the exact same version? Just trying to figure out a good >>> workflow. >>> >>> Thanks for any advice! >>> >>> Andrew >>> >> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From liuzhongzheng2012@REDACTED Wed May 18 04:42:16 2016 From: liuzhongzheng2012@REDACTED (Zhongzheng Liu) Date: Wed, 18 May 2016 10:42:16 +0800 Subject: [erlang-questions] How to compile faster ? Message-ID: Hi Junka: My develop environment is in Windows. All the modules are in the same application. I am using compile:file/2 to compile them. I wrote a Erlang script spawned 10+ process to compile the modules in parallel, which make CPU usage 100% during compling in my PC. Using my script is faster than rebar 2. Rebar 2 make CPU usage on ~70%. Current compile options are [verbose,report,debug_info, warnings_as_errors,{inline_size,0}]. Can it be faster if I add or remove some options? Or any other idea to make compiling faster? > Ho do you compile? Do you use a makefile or rebar or something else? > Also, are all those modules in the same application/src file or in > different applications? > > On 17/05/2016 10:08, Zhongzheng Liu wrote: > > Hi maillist: > > > > I have a erlang project with ~500 modules, all modules including a hrl > > file which contains over 1000 record definition (generated by > > protobuf). > > > > It takes a long time to compile this project. Is there any compile > > option which can make compiling faster? > > > > > > I don't care the performance of the code running (because I use it in > > develop environment only), > > and I don't want to change the source code (cannot spit the big hrl > > to smaller files). > > > > > > Thank you. > > _______________________________________________ > > erlang-questions mailing list > > > > http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Wed May 18 05:32:52 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 18 May 2016 15:32:52 +1200 Subject: [erlang-questions] How to compile faster ? In-Reply-To: References: Message-ID: On 18/05/16 2:42 PM, Zhongzheng Liu wrote: > My develop environment is in Windows. Ah, there's your problem! (Joke.) > I am using compile:file/2 to compile them. Why are you recompiling all the files? What's wrong with Erlang's 'make' module? http://erlang.org/doc/man/make.html From bengt.kleberg@REDACTED Wed May 18 07:15:48 2016 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Wed, 18 May 2016 07:15:48 +0200 Subject: [erlang-questions] how: Simple distributed Erlang in docker containers In-Reply-To: References: <5739C405.20205@ericsson.com> <573AC1DF.9080807@ericsson.com> Message-ID: <573BFA84.40702@ericsson.com> Since the original message has been removed it is difficult to explain what I mean with undocumented. But interested parties can look it up. Especially the "Name" argument, that included an IP address. The resulting behaviour is very useful if the computer has been configured in a way where Erlang/OTP can not find out the fully qualified host name. But I have not seen this documented. bengt On 05/17/2016 10:31 PM, Chandru wrote: > On 17 May 2016 at 08:01, Bengt Kleberg > wrote: > > Thank you for showing a way forward. > > > Since "-name n@REDACTED" is undocumented, I wonder if there is > another possibility to use distributed Erlang with Docker? > > > > Why do you say it is undocumented? From http://erlang.org/doc/man/erl.html > > *-name Name* > > Makes the Erlang runtime system into a distributed node. This flag > invokes all network servers necessary for a node to become > distributed. See net_kernel(3) > . It is also ensured > that epmd runs on the current host before Erlang is started. See > epmd(1) . > > *The name of the node will be Name@REDACTED, where Host is the fully > qualified host name of the current host*. For short names, use the > -sname flag instead. > > > Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed May 18 08:37:50 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 18 May 2016 09:37:50 +0300 Subject: [erlang-questions] locking caveats in file module Message-ID: We have been hunting a performance bug on customer installation during some time and have found that group of processes (about 300) each hour (when new archive file is opened) got locked with following stacktrace: [{gen,do_call,4,[{file,"gen.erl"},{line,168}]}, {gen_server,call,3,[{file,"gen_server.erl"},{line,208}]}, {file,call,2,[{file,"file.erl"},{line,1487}]}, {filelib,do_is_dir,2,[{file,"filelib.erl"},{line,130}]}, {filelib,ensure_dir,1,[{file,"filelib.erl"},{line,231}]}, {dvr_hour_storage,append1,2,... Just a friendly reminder that most of file access functions are called via singleton process. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Wed May 18 08:47:31 2016 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Wed, 18 May 2016 09:47:31 +0300 Subject: [erlang-questions] locking caveats in file module In-Reply-To: References: Message-ID: <72C97681-55A7-4C2B-976A-6C4F784BB7FD@gmail.com> Hello, Have you tried a raw file access? - Dmitry > On May 18, 2016, at 9:37 AM, Max Lapshin wrote: > > We have been hunting a performance bug on customer installation during some time and have found that group of processes (about 300) each hour (when new archive file is opened) got locked with following stacktrace: > > [{gen,do_call,4,[{file,"gen.erl"},{line,168}]}, > > {gen_server,call,3,[{file,"gen_server.erl"},{line,208}]}, > > {file,call,2,[{file,"file.erl"},{line,1487}]}, > > {filelib,do_is_dir,2,[{file,"filelib.erl"},{line,130}]}, > > {filelib,ensure_dir,1,[{file,"filelib.erl"},{line,231}]}, > > {dvr_hour_storage,append1,2,... > > > > > > Just a friendly reminder that most of file access functions are called via singleton process. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mjtruog@REDACTED Wed May 18 09:01:28 2016 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 18 May 2016 00:01:28 -0700 Subject: [erlang-questions] locking caveats in file module In-Reply-To: References: Message-ID: <573C1348.30107@gmail.com> If this is related to the async thread pool, you could increase the number of async threads. Since the async threads have no shared job queue, long filesystem blocking can cause async threads to accumulate jobs, and the only option is to increase the number of threads (despite the CPU/core count) if you have variable latency with filesystem use (which causes async threads to get clogged up unpredictably). On 05/17/2016 11:37 PM, Max Lapshin wrote: > We have been hunting a performance bug on customer installation during some time and have found that group of processes (about 300) each hour (when new archive file is opened) got locked with following stacktrace: > > [{gen,do_call,4,[{file,"gen.erl"},{line,168}]}, > > {gen_server,call,3,[{file,"gen_server.erl"},{line,208}]}, > > {file,call,2,[{file,"file.erl"},{line,1487}]}, > > {filelib,do_is_dir,2,[{file,"filelib.erl"},{line,130}]}, > > {filelib,ensure_dir,1,[{file,"filelib.erl"},{line,231}]}, > > {dvr_hour_storage,append1,2,... > > > > Just a friendly reminder that most of file access functions are called via singleton process. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Wed May 18 09:02:40 2016 From: dangud@REDACTED (Dan Gudmundsson) Date: Wed, 18 May 2016 07:02:40 +0000 Subject: [erlang-questions] Can werl be detached? In-Reply-To: References: <6284265.hgQ7dGlg6e@burrito> Message-ID: At least this works as it should: werl -detached -run wx demo On Wed, May 18, 2016 at 3:48 AM John R. Ashmun wrote: > > > On Tue, May 17, 2016 at 6:32 PM, zxq9 wrote: > >> On Tuesday 17 May 2016 16:47:35 John R. Ashmun wrote: >> > Perhaps my subject is not quite the correct question. Here's my >> context: >> > >> > I'm beginning to recreate in Erlang on Microsoft Windows 10 a public >> domain >> > program I used to enjoy on my Commodore Amiga: mATC was a game in which >> > the user acted as a military Air Traffic Controller. I am using a wx >> > window to draw the player's map with aircraft data blocks overlaid. I >> > don't have a use for the initial window that opens when werl is started. >> >> Awesome! >> >> > If I use this Command Prompt batch file, mATC.bat: >> > >> > werl -pa ebin -pa mATC_app\ebin -detached -boot mATC -config sys >> > >> > a Windows error message window appears that says: >> > >> > "Failed to execute C:\Program Files\erl5.9\erts5.9\bin\beam.smp.dll >> > The system cannot find the file specified." >> > >> > The DLL is actually present at that location, and of course everything >> runs >> > well when I don't use -detached. >> >> Hrm... that seems odd, but I have to admit I have no experience running >> actual releases on Windows. What I usually do instead is have the runtime >> installed on the Windows machine, unpack project code and kick things off >> with an escript that builds and then launches the code I want to run. >> >> Changing from releases to from-source build->run may require a bit of >> shuffling -- and may or may not be worth it depending on the project. >> But this has been very effective and lightweight for my purposes. >> >> Like everything else on Windows there are a few quirks to making escripts >> a click-to-run experience (specifying full path to "escript.exe" instead >> of "escript" in your launcher, space for unheeded shebang, etc.). I've >> wanted for a few years now to formalize and simplify a solution to this... >> but real life gets in the way of the things I wish I could do as community >> project work. :-( >> >> The escript-to-bootstrap approach has made my Erlang GUI experience on >> Windows fairly painless -- I can develop on Linux or BSD and almost always >> get away with relying on anything that works there (to include wx) working >> pretty much the same way on Windows. >> >> I have not tried this on Windows 10 yet, but I assume it will continue to >> work the same way it does on Windows 7 and 8.1. >> >> -Craig >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > I will look into escript, thanks. > > John > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From liuzhongzheng2012@REDACTED Wed May 18 09:10:22 2016 From: liuzhongzheng2012@REDACTED (Zhongzheng Liu) Date: Wed, 18 May 2016 15:10:22 +0800 Subject: [erlang-questions] How to compile faster ? Message-ID: Thank you O'Keefe. I use make:file/2 in my script, it is faster if hrl file is not change. But the checking process is also not very fast. But all module share a same hrl, a slight change in hrl will cause all modules recompile, even most modules did not cite the changed part. Can it be more faster? > On 18/05/16 2:42 PM, Zhongzheng Liu wrote: > > My develop environment is in Windows. > Ah, there's your problem! (Joke.) > > I am using compile:file/2 to compile them. > Why are you recompiling all the files? > What's wrong with Erlang's 'make' module? > http://erlang.org/doc/man/make.html From zxq9@REDACTED Wed May 18 09:45:01 2016 From: zxq9@REDACTED (zxq9) Date: Wed, 18 May 2016 16:45:01 +0900 Subject: [erlang-questions] How to compile faster ? In-Reply-To: References: Message-ID: <6939290.PatoCe46tH@burrito> On Wednesday 18 May 2016 15:10:22 Zhongzheng Liu wrote: > Thank you O'Keefe. > > I use make:file/2 in my script, it is faster if hrl file is not change. > But the checking process is also not very fast. > > But all module share a same hrl, a slight change in hrl will cause all > modules recompile, even most modules did not cite the changed part. > > Can it be more faster? Probably only possible if you can re-structure the project to avoid having a hrl that is both huge *and* universal across the entire project -- which you already said is not possible (which I interpret to mean is something out of your control or would be more of a pain than just suffering through long build times). On that note, a project I'm working on right now has slowly worked its way out of a similar case of multiply-nested records living in a huge, universal hrl import. Now we have small ADTs and modules that generally correspond either strictly to to process definitions or *_lib.erl files. ...and life is much better. If you can possibly move in that direction you will probably enjoy your time more (and find that both development and building gets faster). Good luck! -Craig From max.lapshin@REDACTED Wed May 18 11:57:19 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 18 May 2016 12:57:19 +0300 Subject: [erlang-questions] locking caveats in file module In-Reply-To: <573C1348.30107@gmail.com> References: <573C1348.30107@gmail.com> Message-ID: async thread pool cannot help filelib:ensure_dir, because this library method is using only global singleton file server. Customer has mounted NFS and thus file_server_1 is blocked waiting for lost UDP packet. We have moved almost all file:read_file_info to explicit opening port and calling not via singleton, but via separate port, but filelib:ensure_dir is using singleton process. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nistrigunya@REDACTED Wed May 18 05:35:49 2016 From: nistrigunya@REDACTED (Avinash Dhumane) Date: Wed, 18 May 2016 09:05:49 +0530 Subject: [erlang-questions] Performance of maps:get() Message-ID: For my algorithm trading application, gains even in 5 to 10 microseconds matter. I used maps extensively to improve expressiveness of the design. But, here is a small observation about maps. I am not sure if the performance variation seen is indeed true, or if my test case is wrong. In either case, please point me to the right way of efficiently using of maps. avinash@REDACTED:~/tws$ erl Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] Eshell V7.3 (abort with ^G) %%% compute average of timer:tc() return values (timings) 1> Avg = fun(PerfList) -> Sum = lists:sum(lists:map(fun({Time, _}) -> Time end, PerfList)), Len = erlang:length(PerfList), Sum div Len end. #Fun %%% a test map of size 3 2> Map = #{{a, 1} => 100, {a, 2} => 200, {a, 3} => 300}. #{{a,1} => 100,{a,2} => 200,{a,3} => 300} %%% performance: without maps:get() 3> PerfList1 = lists:map(fun(_) -> timer:tc(fun() -> K1 = {a, 1}, K2 = {a, 2}, K3 = {a, 3}, #{K1 := V1, K2 := V2, K3 := V3} = Map, {V1, V2, V3} end) end, lists:seq(1,100)). [{111,{100,200,300}}, {81,{100,200,300}}, {88,{100,200,300}}, .....details deleted..... {91,{100,200,...}}, {83,{100,...}}, {85,{...}}, {107,...}, {...}|...] %%% performance: with maps:get() 4> PerfList2 = lists:map(fun(_) -> timer:tc(fun() -> {maps:get({a,1}, Map), maps:get({a,2}, Map), maps:get({a,3}, Map)} end) end, lists:seq(1, 100)). [{13,{100,200,300}}, {12,{100,200,300}}, {33,{100,200,300}}, .....details deleted..... {13,{100,200,...}}, {12,{100,...}}, {12,{...}}, {12,...}, {...}|...] %%% compare performance: maps:get() is 7-times faster 5> {Avg(PerfList1), Avg(PerfList2)}. {89,13} 6> -------------- next part -------------- An HTML attachment was scrubbed... URL: From donpedrothird@REDACTED Wed May 18 13:06:35 2016 From: donpedrothird@REDACTED (John Doe) Date: Wed, 18 May 2016 14:06:35 +0300 Subject: [erlang-questions] Releases and Different OSs In-Reply-To: References: Message-ID: I have four prebuilt ERTS for centos 32/64 and debian 32/64. ERTS were configured with --disable-dynamic-ssl-lib --without-termcap flags and built on centos 5.8 and debian 6 They are still compatible with the latest versions of Centos/Ubuntu/Debian, which are used on 99% of commercial hosts. Some library apps which I don't need are stripped as well. Then ERTS together with my app is packed into deb and rpm packages. All paths' and environment variables are hardcoded in startup scripts. This saves me from a lot of headche, as I have a lot of small customers without admins at all. If you have inhouse app this would be very different of course. 2016-05-18 3:42 GMT+03:00 Andrew Berman : > Hi List, > > So today I tried to create a release using rebar3 from my Mac and deploy > it to a CentOS box. Everything works when I do not include ERTS (and have > Erlang installed on the target), but if I do include ERTS, things fail > (can't run run_erl), I'm assuming because the Mac version of Erlang can't > be run on CentOS, which makes sense. So, I wanted to know how people do > deployments of their Erlang releases. Do you set up boxes with each OS > that matches your target OS and compile on there, including ERTS in the > release? Do you not include ERTS and make sure that all your Erlang > installs are the exact same version? Just trying to figure out a good > workflow. > > Thanks for any advice! > > Andrew > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From donpedrothird@REDACTED Wed May 18 13:15:03 2016 From: donpedrothird@REDACTED (John Doe) Date: Wed, 18 May 2016 14:15:03 +0300 Subject: [erlang-questions] Releases and Different OSs In-Reply-To: References: Message-ID: Forgot to add. Some openssl packages shipped with Centos are incompatible with Erlang crypto app and the openssl needs to built as well from the sources prior to linking it to ERTS. So the process of building fully compatible ERTS in my case looks like this: wget https://www.openssl.org/source/openssl-1.0.1j.tar.gz tar -xvzf openssl-1.0.0j.tar.gz cd openssl-1.0.0j/ ./config --prefix=/usr shared -fPIC sudo make sudo checkinstall then Erlang (R16B03-1) is configured with ./configure --disable-dynamic-ssl-lib --with-ssl=/usr/ --enable-smp-support --without-termcap 2016-05-18 14:06 GMT+03:00 John Doe : > I have four prebuilt ERTS for centos 32/64 and debian 32/64. ERTS were > configured with --disable-dynamic-ssl-lib --without-termcap > flags and built on centos 5.8 and debian 6 They are still compatible with > the latest versions of Centos/Ubuntu/Debian, which are used on 99% of > commercial hosts. Some library apps which I don't need are stripped as > well. Then ERTS together with my app is packed into deb and rpm packages. > All paths' and environment variables are hardcoded in startup scripts. This > saves me from a lot of headche, as I have a lot of small customers without > admins at all. > > If you have inhouse app this would be very different of course. > > > > 2016-05-18 3:42 GMT+03:00 Andrew Berman : > >> Hi List, >> >> So today I tried to create a release using rebar3 from my Mac and deploy >> it to a CentOS box. Everything works when I do not include ERTS (and have >> Erlang installed on the target), but if I do include ERTS, things fail >> (can't run run_erl), I'm assuming because the Mac version of Erlang can't >> be run on CentOS, which makes sense. So, I wanted to know how people do >> deployments of their Erlang releases. Do you set up boxes with each OS >> that matches your target OS and compile on there, including ERTS in the >> release? Do you not include ERTS and make sure that all your Erlang >> installs are the exact same version? Just trying to figure out a good >> workflow. >> >> Thanks for any advice! >> >> Andrew >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Wed May 18 13:39:39 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Wed, 18 May 2016 13:39:39 +0200 Subject: [erlang-questions] Performance of maps:get() In-Reply-To: References: Message-ID: Hello, You do not ever want to run benchmarking in the shell. The code in the shell and .beam files are very different and the performance differs wildly. Compile your tests into a module and try again to see if you get better results. When I do it I get {0,0} as the result. Lukas On Wed, May 18, 2016 at 5:35 AM, Avinash Dhumane wrote: > For my algorithm trading application, gains even in 5 to 10 microseconds > matter. > > I used maps extensively to improve expressiveness of the design. > > But, here is a small observation about maps. I am not sure if the > performance variation seen is indeed true, or if my test case is wrong. In > either case, please point me to the right way of efficiently using of maps. > > avinash@REDACTED:~/tws$ erl > Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:2:2] [async-threads:10] > [hipe] [kernel-poll:false] > > Eshell V7.3 (abort with ^G) > > %%% compute average of timer:tc() return values (timings) > > 1> Avg = fun(PerfList) -> Sum = lists:sum(lists:map(fun({Time, _}) -> Time > end, PerfList)), Len = erlang:length(PerfList), Sum div Len end. > #Fun > > %%% a test map of size 3 > > 2> Map = #{{a, 1} => 100, {a, 2} => 200, {a, 3} => 300}. > #{{a,1} => 100,{a,2} => 200,{a,3} => 300} > > %%% performance: without maps:get() > > 3> PerfList1 = lists:map(fun(_) -> timer:tc(fun() -> K1 = {a, 1}, K2 = {a, > 2}, K3 = {a, 3}, #{K1 := V1, K2 := V2, K3 := V3} = Map, {V1, V2, V3} end) > end, lists:seq(1,100)). > [{111,{100,200,300}}, > {81,{100,200,300}}, > {88,{100,200,300}}, > .....details deleted..... > {91,{100,200,...}}, > {83,{100,...}}, > {85,{...}}, > {107,...}, > {...}|...] > > %%% performance: with maps:get() > > 4> PerfList2 = lists:map(fun(_) -> timer:tc(fun() -> {maps:get({a,1}, > Map), maps:get({a,2}, Map), maps:get({a,3}, Map)} end) end, lists:seq(1, > 100)). > [{13,{100,200,300}}, > {12,{100,200,300}}, > {33,{100,200,300}}, > .....details deleted..... > {13,{100,200,...}}, > {12,{100,...}}, > {12,{...}}, > {12,...}, > {...}|...] > > %%% compare performance: maps:get() is 7-times faster > > 5> {Avg(PerfList1), Avg(PerfList2)}. > {89,13} > 6> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From be.dmitry@REDACTED Wed May 18 13:55:29 2016 From: be.dmitry@REDACTED (Dmitry Belyaev) Date: Wed, 18 May 2016 21:55:29 +1000 Subject: [erlang-questions] Performance of maps:get() In-Reply-To: References: Message-ID: <25018386-7536-4259-AE3F-A6BE84964A59@gmail.com> As a first step try a compiled version, not a test in a shell. On 18 May 2016 1:35:49 PM AEST, Avinash Dhumane wrote: >For my algorithm trading application, gains even in 5 to 10 >microseconds >matter. > >I used maps extensively to improve expressiveness of the design. > >But, here is a small observation about maps. I am not sure if the >performance variation seen is indeed true, or if my test case is wrong. >In >either case, please point me to the right way of efficiently using of >maps. > >avinash@REDACTED:~/tws$ erl >Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:2:2] [async-threads:10] >[hipe] [kernel-poll:false] > >Eshell V7.3 (abort with ^G) > >%%% compute average of timer:tc() return values (timings) > >1> Avg = fun(PerfList) -> Sum = lists:sum(lists:map(fun({Time, _}) -> >Time >end, PerfList)), Len = erlang:length(PerfList), Sum div Len end. >#Fun > >%%% a test map of size 3 > >2> Map = #{{a, 1} => 100, {a, 2} => 200, {a, 3} => 300}. >#{{a,1} => 100,{a,2} => 200,{a,3} => 300} > >%%% performance: without maps:get() > >3> PerfList1 = lists:map(fun(_) -> timer:tc(fun() -> K1 = {a, 1}, K2 = >{a, >2}, K3 = {a, 3}, #{K1 := V1, K2 := V2, K3 := V3} = Map, {V1, V2, V3} >end) >end, lists:seq(1,100)). >[{111,{100,200,300}}, > {81,{100,200,300}}, > {88,{100,200,300}}, >.....details deleted..... > {91,{100,200,...}}, > {83,{100,...}}, > {85,{...}}, > {107,...}, > {...}|...] > >%%% performance: with maps:get() > >4> PerfList2 = lists:map(fun(_) -> timer:tc(fun() -> {maps:get({a,1}, >Map), >maps:get({a,2}, Map), maps:get({a,3}, Map)} end) end, lists:seq(1, >100)). >[{13,{100,200,300}}, > {12,{100,200,300}}, > {33,{100,200,300}}, >.....details deleted..... > {13,{100,200,...}}, > {12,{100,...}}, > {12,{...}}, > {12,...}, > {...}|...] > >%%% compare performance: maps:get() is 7-times faster > >5> {Avg(PerfList1), Avg(PerfList2)}. >{89,13} >6> > > >------------------------------------------------------------------------ > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -- Best wishes, Dmitry Belyaev -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Wed May 18 14:13:21 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Wed, 18 May 2016 12:13:21 +0000 Subject: [erlang-questions] erlang nif resource & thread safety In-Reply-To: <57276304.9070206@ericsson.com> References: <57276304.9070206@ericsson.com> Message-ID: On Mon, May 2, 2016 at 4:24 PM Sverker Eriksson < sverker.eriksson@REDACTED> wrote: > A resource will be deallocated when > > * all terms created with enif_make_resource() have been garbage collected > > * AND the call to enif_alloc_resource() has been balanced by a call to > enif_release_resource() > > * AND all calls to enif_keep_resource() have been balanced by an equal > number of calls to enif_release_resource() > > > > On 05/02/2016 11:47 AM, Benoit Chesneau wrote: > > I had a look in the doc but I can't figure exactly how much a nif resource > could live outside the process that created it, ie. how others processes > can write/read to it and how to make sure the resource will be accessible > to them. > > So I created a nif resource that have the following properties: > - it is ref counted, each time a call is made to it, the counter is > incremented and decremented when it return. > > A resource already has an internal ref counter, which is decremented by > the GC and by enif_release_resource(). > > Adding your own ref counter is probably either redundant or just plain > wrong. > Every call to enif_release_resource() must be balanced by a prior call > to either enif_alloc_resource() or enif_keep_resource(). You cannot > release a resource that is still referred to by one of more terms. > > > - the process that create it will live until the vm is closed > > A resource it not tied to the process that created it, > it is tied to the terms that refer it. > Related to that I recently came with the following question: How to tie a nif resource to a owner process to make sure it's cleaned if the process crash? Looking at the source of ETS it seems to use unexposed functions from https://github.com/erlang/otp/blob/maint/erts/emulator/beam/erl_process.h I think I miss a C function like "nif_tie_resource(pid, somecalleback)" . Or is there any code that already does that? - beno?t -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Wed May 18 14:49:58 2016 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Wed, 18 May 2016 14:49:58 +0200 Subject: [erlang-questions] erlang nif resource & thread safety In-Reply-To: References: <57276304.9070206@ericsson.com> Message-ID: <573C64F6.9000904@ericsson.com> On 05/18/2016 02:13 PM, Benoit Chesneau wrote: > > > On Mon, May 2, 2016 at 4:24 PM Sverker Eriksson > > > wrote: > > A resource will be deallocated when > > * all terms created with enif_make_resource() have been garbage > collected > > * AND the call to enif_alloc_resource() has been balanced by a > call to enif_release_resource() > > * AND all calls to enif_keep_resource() have been balanced by an > equal number of calls to enif_release_resource() > > > > On 05/02/2016 11:47 AM, Benoit Chesneau wrote: >> I had a look in the doc but I can't figure exactly how much a nif >> resource could live outside the process that created it, ie. how >> others processes can write/read to it and how to make sure the >> resource will be accessible to them. >> >> So I created a nif resource that have the following properties: >> - it is ref counted, each time a call is made to it, the counter >> is incremented and decremented when it return. > A resource already has an internal ref counter, which is > decremented by the GC and by enif_release_resource(). > > Adding your own ref counter is probably either redundant or just > plain wrong. > Every call to enif_release_resource() must be balanced by a prior call > to either enif_alloc_resource() or enif_keep_resource(). You cannot > release a resource that is still referred to by one of more terms. > > >> - the process that create it will live until the vm is closed > A resource it not tied to the process that created it, > it is tied to the terms that refer it. > > > Related to that I recently came with the following question: > How to tie a nif resource to a owner process to make sure it's cleaned > if the process crash? > > Looking at the source of ETS it seems to use unexposed functions from > https://github.com/erlang/otp/blob/maint/erts/emulator/beam/erl_process.h > > I think I miss a C function like "nif_tie_resource(pid, > somecalleback)" . Or is there any code that already does that? > > - beno?t If the crashing process contains the last reference to your resource then it will be destructed as part of the process termination cleanup and you don't have to do anything. If you want your resource to be "cleaned" at process crash even though it is still referred by other processes or ETS tables, then there is no such callback today. You can accomplish that however by creating a second "owner" resource with a pointer to the "payload" resource. Keep the owner resource term in your owning process and let it clean the payload resource in its destructor. The destructor of the payload resource (if any) and all code doing enif_get_resource must recognize and handle an already cleaned resource. /Sverker, Erlang/OTP -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Wed May 18 15:22:59 2016 From: t@REDACTED (Tristan Sloughter) Date: Wed, 18 May 2016 08:22:59 -0500 Subject: [erlang-questions] Releases and Different OSs In-Reply-To: References: Message-ID: <1463577779.3602082.611473737.4788CABC@webmail.messagingengine.com> Right, we have a small section in the docs about doing this: http://www.rebar3.org/docs/releases#cross-compiling -- Tristan Sloughter t@REDACTED On Tue, May 17, 2016, at 08:05 PM, Vincent A Okonkwo wrote: > Yes, that's what I meant. > > On Wed, May 18, 2016 at 12:58 AM Andrew Berman > wrote: >> So you're saying to download a binary of Erlang compiled on Linux and >> put it on my Mac and then have it include that when building the >> release tar? >> >> On Tue, May 17, 2016 at 5:55 PM Vincent A Okonkwo >> wrote: >>> Hi Andrew, >>> >>> Been awhile since I tried this but if you are using relx there is a >>> --include-erts=$PATH_TO_EXTRACTED_LINUX_ERTS ( {include_erts, >>> $PATH_TO_EXTRACTED_LINUX_ERTS} for rebar) that you can use >>> >>> >>> On Wed, May 18, 2016 at 12:42 AM Andrew Berman >>> wrote: >>>> Hi List, >>>> >>>> So today I tried to create a release using rebar3 from my Mac and >>>> deploy it to a CentOS box.? Everything works when I do not include >>>> ERTS (and have Erlang installed on the target), but if I do include >>>> ERTS, things fail (can't run run_erl), I'm assuming because the Mac >>>> version of Erlang can't be run on CentOS, which makes sense. ?So, I >>>> wanted to know how people do deployments of their Erlang releases. >>>> Do you set up boxes with each OS that matches your target OS and >>>> compile on there, including ERTS in the release?? Do you not >>>> include ERTS and make sure that all your Erlang installs are the >>>> exact same version? ? Just trying to figure out a good workflow. >>>> >>>> Thanks for any advice! >>>> >>>> Andrew >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed May 18 15:52:41 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 18 May 2016 15:52:41 +0200 Subject: [erlang-questions] Performance of maps:get() In-Reply-To: <25018386-7536-4259-AE3F-A6BE84964A59@gmail.com> References: <25018386-7536-4259-AE3F-A6BE84964A59@gmail.com> Message-ID: Rewriting into a module and passing through eministat: -module(maps_get). -export([datasets/0, t/0]). without(Map) -> K1 = {a, 1}, K2 = {a, 2}, K3 = {a, 3}, #{K1 := V1, K2 := V2, K3 := V3} = Map, {V1, V2, V3}. with(Map) -> {maps:get({a,1}, Map), maps:get({a,2}, Map), maps:get({a,3}, Map)}. datasets() -> Map = #{{a, 1} => 100, {a, 2} => 200, {a, 3} => 300}, [eministat:s("without", fun() -> without(Map) end, 100, us), eministat:s("with", fun() -> with(Map) end, 100, us)]. t() -> [Without | With] = datasets(), eministat:x(95.0, Without, With). yields the following result: 9> maps_get:t(). x without + with +--------------------------------------------------------------------------+ | * * + x x| | * * | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | * x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + x | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | | + | ||________A_M_______| | | |_A_| | +--------------------------------------------------------------------------+ Dataset: x N=100 CI=95.0000 Statistic Value [ Bias] (Bootstrapped LB?UB) Min: 0.00000e+0 1st Qu. 0.00000e+0 Median: 1.00000 3rd Qu. 1.00000 Max: 10.0000 Average: 0.820000 [ -8.45000e-4] ( 0.630000 ? 1.23000) Std. Dev: 1.33621 [ -7.87598e-2] ( 0.482418 ? 2.33919) Outliers: 0/2 = 2 (?=0.819155, ?=1.25745) Outlier variance: 0.989942 (severe, the data set is probably unusable) ------ Dataset: + N=100 CI=95.0000 Statistic Value [ Bias] (Bootstrapped LB?UB) Min: 0.00000e+0 1st Qu. 0.00000e+0 Median: 0.00000e+0 3rd Qu. 0.00000e+0 Max: 2.00000 Average: 4.00000e-2 [ -4.33000e-4] ( 0.00000e+0 ? 0.110000) Std. Dev: 0.242878 [ -2.09268e-2] ( 0.100000 ? 0.455716) Outliers: 0/3 = 3 (?=3.95670e-2, ?=0.221952) Outlier variance: 0.989996 (severe, the data set is probably unusable) Difference at 95.0% confidence -0.780000 ? 0.266188 -95.1220% ? 32.4620% (Student's t, pooled s = 0.960324) ------ The result is largely unusable. We can't measure at the microsecond level precisely enough to conclude anything. You will probably have to run several rounds of the selection in order to get a useful value which isn't severely pertubed by e.g. garbage collection or taking an interrupt on the OSX machine I ran this on. You will have to do something about the outlier variance. Also note that the confidence intervals overlaps a lot. Even though we have significant difference at 95.0% confidence, the difference is probably affected too much by measurement errors to be conclusive. A test could easily run 1000 rounds of the operation: Altering the code to 10000 rounds and 25 trials yields the following code: -module(maps_get). -export([datasets/0, t/0]). without(0, _) -> ok; without(K, Map) -> K1 = {a, 1}, K2 = {a, 2}, K3 = {a, 3}, #{K1 := V1, K2 := V2, K3 := V3} = Map, {V1, V2, V3}, without(K-1, Map). with(0, _) -> ok; with(K, Map) -> {maps:get({a,1}, Map), maps:get({a,2}, Map), maps:get({a,3}, Map)}, with(K-1, Map). datasets() -> Map = #{{a, 1} => 100, {a, 2} => 200, {a, 3} => 300}, [eministat:s("without", fun() -> without(10000, Map) end, 25, us), eministat:s("with", fun() -> with(10000, Map) end, 25, us)]. t() -> [Without | With] = datasets(), eministat:x(95.0, Without, With). And the following output from eministat: 11> maps_get:t(). x without + with +--------------------------------------------------------------------------+ | x xx x x + +| | x x + | | x x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | x + | | + | | + | | + | | + | | + | ||MA__| | | |A| | +--------------------------------------------------------------------------+ Dataset: x N=25 CI=95.0000 Statistic Value [ Bias] (Bootstrapped LB?UB) Min: 408.000 1st Qu. 410.000 Median: 411.000 3rd Qu. 412.000 Max: 460.000 Average: 418.880 [ -2.47040e-2] ( 413.760 ? 426.680) Std. Dev: 16.3078 [ -0.661582] ( 10.6226 ? 21.5513) Outliers: 0/6 = 6 (?=418.855, ?=15.6462) Outlier variance: 0.149444 (moderate) ------ Dataset: + N=25 CI=95.0000 Statistic Value [ Bias] (Bootstrapped LB?UB) Min: 860.000 1st Qu. 860.000 Median: 861.000 3rd Qu. 861.000 Max: 902.000 Average: 862.440 [ -9.31200e-3] ( 860.680 ? 869.040) Std. Dev: 8.26680 [ -1.72872] ( 0.583095 ? 16.8891) Outliers: 0/1 = 1 (?=862.431, ?=6.53808) Outlier variance: 3.84000e-2 (slight) Difference at 95.0% confidence 443.560 ? 7.35359 105.892% ? 1.75554% (Student's t, pooled s = 12.9283) ------ ok 12> Conclusion: maps:get/2 is likely around 443us (+/- 7.4us) slower in this test than ignoring it. But also note that in both cases the compiler is warning us about term construction which is unused. So the compiler might optimize stuff away. Though running a loop with no effect as a control ticks out at 170us, so it is unlikely. On Wed, May 18, 2016 at 1:55 PM, Dmitry Belyaev wrote: > As a first step try a compiled version, not a test in a shell. > > On 18 May 2016 1:35:49 PM AEST, Avinash Dhumane > wrote: >> >> For my algorithm trading application, gains even in 5 to 10 microseconds >> matter. >> >> I used maps extensively to improve expressiveness of the design. >> >> But, here is a small observation about maps. I am not sure if the >> performance variation seen is indeed true, or if my test case is wrong. In >> either case, please point me to the right way of efficiently using of maps. >> >> avinash@REDACTED:~/tws$ erl >> Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:2:2] [async-threads:10] >> [hipe] [kernel-poll:false] >> >> Eshell V7.3 (abort with ^G) >> >> %%% compute average of timer:tc() return values (timings) >> >> 1> Avg = fun(PerfList) -> Sum = lists:sum(lists:map(fun({Time, _}) -> >> Time end, PerfList)), Len = erlang:length(PerfList), Sum div Len end. >> #Fun >> >> %%% a test map of size 3 >> >> 2> Map = #{{a, 1} => 100, {a, 2} => ; 200, {a, 3} => 300}. >> >> #{{a,1} => 100,{a,2} => 200,{a,3} => 300} >> >> %%% performance: without maps:get() >> >> 3> PerfList1 = lists:map(fun(_) -> timer:tc(fun() -> K1 = {a, 1}, K2 = >> {a, 2}, K3 = {a, 3}, #{K1 := V1, K2 := V2, K3 := V3} = Map, {V1, V2, V3} >> end) end, lists:seq(1,100)). >> [{111,{100,200,300}}, >> {81,{100,200,300}}, >> {88,{100,200,300}}, >> .....details deleted..... >> {91,{100,200,...}}, >> {83,{100,...}}, >> {85,{...}}, >> {107,...}, >> {...}|...] >> >> %%% performance: with maps:get() >> >> 4> PerfList2 = lists:map(fun(_) -> timer:tc(fun() -> {maps:get({a,1}, >> Map), maps:get({a,2}, Map), maps:get({a,3}, Map)} end) end, lists:seq(1, >> 100)). >> [{13,{100,200,300}}, >> {12,{100,200,300}}, >> {33,{100,200,300}}, >> .....details deleted..... >> {13,{100,200,...}}, >> {12,{100,...}}, >> {12,{...}}, >> {12,...}, >> {...}|...] >> >> %%% compare performance: maps:get() is 7-times faster >> >> 5> {Avg(PerfList1), Avg(PerfList2)}. >> {89,13} >> 6> >> >> ------------------------------ >> >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -- > Best wishes, > Dmitry Belyaev > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.ashmun@REDACTED Wed May 18 18:41:34 2016 From: john.ashmun@REDACTED (John R. Ashmun) Date: Wed, 18 May 2016 09:41:34 -0700 Subject: [erlang-questions] Can werl be detached? In-Reply-To: References: <6284265.hgQ7dGlg6e@burrito> Message-ID: On Wed, May 18, 2016 at 12:02 AM, Dan Gudmundsson wrote: > At least this works as it should: > > werl -detached -run wx demo > > > On Wed, May 18, 2016 at 3:48 AM John R. Ashmun > wrote: > >> >> >> On Tue, May 17, 2016 at 6:32 PM, zxq9 wrote: >> >>> On Tuesday 17 May 2016 16:47:35 John R. Ashmun wrote: >>> > Perhaps my subject is not quite the correct question. Here's my >>> context: >>> > >>> > I'm beginning to recreate in Erlang on Microsoft Windows 10 a public >>> domain >>> > program I used to enjoy on my Commodore Amiga: mATC was a game in >>> which >>> > the user acted as a military Air Traffic Controller. I am using a wx >>> > window to draw the player's map with aircraft data blocks overlaid. I >>> > don't have a use for the initial window that opens when werl is >>> started. >>> >>> Awesome! >>> >>> > If I use this Command Prompt batch file, mATC.bat: >>> > >>> > werl -pa ebin -pa mATC_app\ebin -detached -boot mATC -config sys >>> > >>> > a Windows error message window appears that says: >>> > >>> > "Failed to execute C:\Program Files\erl5.9\erts5.9\bin\beam.smp.dll >>> > The system cannot find the file specified." >>> > >>> > The DLL is actually present at that location, and of course everything >>> runs >>> > well when I don't use -detached. >>> >>> Hrm... that seems odd, but I have to admit I have no experience running >>> actual releases on Windows. What I usually do instead is have the runtime >>> installed on the Windows machine, unpack project code and kick things off >>> with an escript that builds and then launches the code I want to run. >>> >>> Changing from releases to from-source build->run may require a bit of >>> shuffling -- and may or may not be worth it depending on the project. >>> But this has been very effective and lightweight for my purposes. >>> >>> Like everything else on Windows there are a few quirks to making escripts >>> a click-to-run experience (specifying full path to "escript.exe" instead >>> of "escript" in your launcher, space for unheeded shebang, etc.). I've >>> wanted for a few years now to formalize and simplify a solution to >>> this... >>> but real life gets in the way of the things I wish I could do as >>> community >>> project work. :-( >>> >>> The escript-to-bootstrap approach has made my Erlang GUI experience on >>> Windows fairly painless -- I can develop on Linux or BSD and almost >>> always >>> get away with relying on anything that works there (to include wx) >>> working >>> pretty much the same way on Windows. >>> >>> I have not tried this on Windows 10 yet, but I assume it will continue to >>> work the same way it does on Windows 7 and 8.1. >>> >>> -Craig >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> I will look into escript, thanks. >> >> John >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > Thank you. I will try to find it and run it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From evnu@REDACTED Wed May 18 20:25:34 2016 From: evnu@REDACTED (Magnus Ottenklinger) Date: Wed, 18 May 2016 20:25:34 +0200 Subject: [erlang-questions] How to compile faster ? In-Reply-To: References: Message-ID: <146359593428.5875.10549984088359428205@central-dogma.fritz.box> Quoting Richard A. O'Keefe (2016-05-18 05:32:52) > Why are you recompiling all the files? > What's wrong with Erlang's 'make' module? > http://erlang.org/doc/man/make.html Just a small note on 'make.erl': 'make:check_includes/3' slows down compilation a lot if no header file actually changed. The function has to peek into modules to check if they include a header file which is newer than the object file. We circumvented that by building a simple dependency graph on the first compilation run on a clean project. In essence, this comes down to what erlang.mk is apparently doing as well in erlc.mk. Magnus From t6sn7gt@REDACTED Wed May 18 22:40:36 2016 From: t6sn7gt@REDACTED (Donald Steven) Date: Wed, 18 May 2016 16:40:36 -0400 Subject: [erlang-questions] Pinging the great unknown In-Reply-To: Message-ID: <154c59944fa-6fda-26d43@webprd-m47.mail.aol.com> Hi all, Simple newbie question for someone just learning. I'm getting the hang of sending and receiving messages work and pids, but how does a process ping a message into the great unknown that says basically: "Hello out there, I'm here, I exist..." if you don't know what if any processes are 'out there' to actually receive a message? Thanks. Don -------------- next part -------------- An HTML attachment was scrubbed... URL: From hugo@REDACTED Wed May 18 23:12:54 2016 From: hugo@REDACTED (Hugo Mills) Date: Wed, 18 May 2016 21:12:54 +0000 Subject: [erlang-questions] Pinging the great unknown In-Reply-To: <154c59944fa-6fda-26d43@webprd-m47.mail.aol.com> References: <154c59944fa-6fda-26d43@webprd-m47.mail.aol.com> Message-ID: <20160518211254.GA16712@carfax.org.uk> On Wed, May 18, 2016 at 04:40:36PM -0400, Donald Steven wrote: > Hi all, > > Simple newbie question for someone just learning. I'm getting the hang of sending and receiving messages work and pids, but how does a process ping a message into the great unknown that says basically: "Hello out there, I'm here, I exist..." if you don't know what if any processes are 'out there' to actually receive a message? One approach: Process A (the receipient) registers itself with register/2, using an atom as a name. Process B (the sender) uses the same atom to look up the pid of A, with whereis/1. So, the initial "connection" between the two comes from the programmer knowing that process A registers under a particular name, and then telling process B to use that name. There's a simple built-in registry (register/2, whereis/1), or there are more complex third-party registries (like gproc). Another approach: Process A starts. It starts process B, passing in its own pid as a parameter. Alternatively, process C could have started both A and B, getting the pid of A after it's running, and passing that to B as a parameter on startup. The first approach is more typical, particularly in the case where process A is a server of some kind. Hugo. -- Hugo Mills | Be pure. hugo@REDACTED carfax.org.uk | Be vigilant. http://carfax.org.uk/ | Behave. PGP: E2AB1DE4 | Torquemada, Nemesis -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From ted.ml.erlang@REDACTED Wed May 18 23:08:51 2016 From: ted.ml.erlang@REDACTED (Ted Burghart) Date: Wed, 18 May 2016 17:08:51 -0400 Subject: [erlang-questions] Thoughts on enif_whereis? Message-ID: I have a NIF library from which I want to send messages to a registered local service. The service should be quite long-lived, so it's reasonable for me to hold on to its pid in the NIF's private data, but if it crashes and is re-started/re-registered I need to be able to pick up its new pid. I can initialize the NIF library with the pid at the time it's loaded, and my current approach is to have an Erlang process monitoring the service so it can call into the NIF library with the new pid if it changes, but it would be much less convoluted if I could just respond to a (presumably very rare) failure result from enif_send() by looking up the new pid. Before anyone warns of the perils of long-running NIFs, I do intend to sprinkle enif_consume_timeslice() and enif_schedule_nif() calls liberally throughout this code. erts_whereis_name_to_id() is pretty straightforward, and looks like it would be no more onerous to call from a NIF than, say, enif_send(). Has an enif_whereis() function been considered and, if so, have any conclusions been reached? I can see where a stern warning about not holding any lock while calling it would be in order due to the global mutex in register.c, but beyond that it seems relatively simple and really handy. Am I missing something? ? Ted -- Ted Burghart Senior Engineer Basho Technologies http://www.basho.com From fritchie@REDACTED Wed May 18 23:42:35 2016 From: fritchie@REDACTED (Scott Fritchie) Date: Wed, 18 May 2016 16:42:35 -0500 Subject: [erlang-questions] [ANN] Call for papers: ACM Erlang Workshop 2016 Message-ID: Erlang and Elixir folks, good afternoon. Melinda T?th and I are co-chairs of this year's ACM workshop on Erlang, Elixir, LFE, and (new this year) concurrency-oriented programming in environments such as Akka, CloudHaskell, Occam, and Pony. The workshop will be held in Nara, Japan on September 23, 2016. If you or someone you know are working on a project that could teach the ErlangOTP//BeamVM community something new, please consider submitting a proposal for a paper (in either category) or poster. If there's a local user group in your area that wouldn't mind receiving a copy of this call for papers, we would be grateful if you would forward this announcement and/or a link to the CFP at http://erlang.org/workshop/2016/. Thank you! -Scott CALL FOR PAPERS =============== Fifteenth ACM SIGPLAN Erlang Workshop ------------------------------------- Nara, Japan, September 23, 2016 Satellite event of the 21st ACM SIGPLAN International Conference on Functional Programming (ICFP 2016) September 18-24, 2016 The Erlang Workshop aims to bring together the open source, academic, and industrial communities of Erlang, to discuss technologies and languages related to Erlang. The Erlang model of concurrent programming has been widely emulated, for example by Akka in Scala, and even new programming languages were designed atop of the Erlang VM, such as Elixir. Therefore we would like to broaden the scope of the workshop to include systems like those mentioned above. The workshop will enable participants to familiarize themselves with recent developments on new techniques and tools, novel applications, draw lessons from users' experiences and identify research problems and common areas relevant to the practice of Erlang, Erlang-like languages, functional programming, distribution, concurrency etc. We invite three types of submissions. 1. Technical papers describing interesting contributions either in theoretical work or real world applications. Submission related to Erlang, Elixir, Akka, CloudHaskell, Occam, and functional programming are welcome and encouraged. Topics of interest include (but are not limited to): - virtual machine extensions and compilation techniques - implementations and interfaces of Erlang in/with other languages - new tools (profilers, tracers, debuggers, testing frameworks etc.) - language extensions - formal semantics, correctness and verification - testing Erlang programs - program analysis and transformation - Erlang-like languages and technologies - functional languages and multi-processing - concurrency in functional languages - functional languages and distributed computing - parallel programming - pattern based programming - Erlang in education The maximum length for technical papers is restricted to 12 pages. 2. Experience reports describing uses of Erlang in the "real-world", Erlang libraries for specific tasks, experiences from using Erlang in specific application domains, reusable programming idioms and elegant new ways of using Erlang to approach or solve a particular problem. The maximum length for the experience report is restricted to 2 pages. 3. Poster presentations describing topics related to the workshop goals. Each includes a maximum of 2 pages of the abstract and summary. Presentations in this category will be given an hour of shared simultaneous demonstration time. Workshop Co-Chairs ------------------ Melinda T?th, E?tv?s Lor?nd University, Hungary Scott Lystig Fritchie, Basho Japan KK Program Committee ----------------------------- (Note: the Workshop Co-Chairs are also committee members) Jamie Allen, Typesafe Laura M. Castro, University of A Coru?a, Spain Natalia Chechina, University of Glasgow Viktoria F?rd?s, Erlang Solutions Yosuke Hara, Rakuten, Inc. Kenji Rikitake, KRPEO Bruce Tate, iCanMakeItBetter Simon Thompson, University of Kent, UK Important Dates ----------------------- Submissions due: Friday, 3 June, 2016 Author notification: Friday, 8 July, 2016 Final copy due: Sunday, 31 July, 2016 Workshop date: September 23, 2016 Instructions to authors -------------------------------- Papers must be submitted online via EasyChair (via the "Erlang2016" event). The submission page is https://www.easychair.org/conferences/?conf=erlang2016 Submitted papers should be in portable document format (PDF), formatted using the ACM SIGPLAN style guidelines. Each submission must adhere to SIGPLAN's republication policy. Violation risks summary rejection of the offending submission. Accepted papers will be published by the ACM and will appear in the ACM Digital Library. Paper submissions will be considered for poster submission in the case they are not accepted as full papers. Venue & Registration Details ------------------------------------------ For registration, please see the ICFP 2016 web site at: http://conf.researchr.org/home/icfp-2016 Related Links -------------------- CFP: http://conf.researchr.org/track/icfp-2016/erlang-2016-papers ICFP 2016 web site: http://conf.researchr.org/home/icfp-2016 Past ACM SIGPLAN Erlang workshops: http://www.erlang.org/workshop/ Open Source Erlang: http://www.erlang.org/ EasyChair submission site: https://www.easychair.org/conferences/?conf=erlang2016 Author Information for SIGPLAN Conferences: http://www.sigplan.org/authorInformation.htm Attendee Information for SIGPLAN Events: http://www.sigplan.org/Resources/Policies/Anti-harassment From jargon@REDACTED Thu May 19 00:07:59 2016 From: jargon@REDACTED (Johannes =?utf-8?B?V2Vpw59s?=) Date: Thu, 19 May 2016 00:07:59 +0200 Subject: [erlang-questions] SSL problems with OTP 19 In-Reply-To: References: <20160514180742.GA21998@molb.org> <57377381.20208@gmail.com> Message-ID: <20160518220759.GA25111@molb.org> On Mon, May 16, 2016 at 02:42PM +0200, Ingela Andin wrote: > On Sat, May 14, 2016 at 11:50AM -0700, Michael Truog wrote: > > I filed a bug for this at http://bugs.erlang.org/browse/ERL-144 due to it preventing the rebar3 bootstrap script from running. > > I could not reproduce this. I also could not reproduce this so far, so I opened two new reports: - http://bugs.erlang.org/browse/ERL-147 - http://bugs.erlang.org/browse/ERL-148 (not sure if ERL-144, ERL-147 and ERL-148 have the same solution or not) Regards, Johannes From t6sn7gt@REDACTED Thu May 19 00:55:44 2016 From: t6sn7gt@REDACTED (Donald Steven) Date: Wed, 18 May 2016 18:55:44 -0400 Subject: [erlang-questions] Pinging the great unknown In-Reply-To: <20160518211254.GA16712@carfax.org.uk> References: <154c59944fa-6fda-26d43@webprd-m47.mail.aol.com> <20160518211254.GA16712@carfax.org.uk> Message-ID: <573CF2F0.804@aim.com> Thanks Hugo, that's very helpful. Don On 5/18/2016 5:12 PM, Hugo Mills wrote: > On Wed, May 18, 2016 at 04:40:36PM -0400, Donald Steven wrote: >> Hi all, >> >> Simple newbie question for someone just learning. I'm getting the hang of sending and receiving messages work and pids, but how does a process ping a message into the great unknown that says basically: "Hello out there, I'm here, I exist..." if you don't know what if any processes are 'out there' to actually receive a message? > One approach: Process A (the receipient) registers itself with > register/2, using an atom as a name. Process B (the sender) uses the > same atom to look up the pid of A, with whereis/1. So, the initial > "connection" between the two comes from the programmer knowing that > process A registers under a particular name, and then telling process > B to use that name. There's a simple built-in registry (register/2, > whereis/1), or there are more complex third-party registries (like > gproc). > > Another approach: Process A starts. It starts process B, passing in > its own pid as a parameter. Alternatively, process C could have > started both A and B, getting the pid of A after it's running, and > passing that to B as a parameter on startup. > > The first approach is more typical, particularly in the case where > process A is a server of some kind. > > Hugo. > --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus From bchesneau@REDACTED Thu May 19 09:07:24 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 19 May 2016 07:07:24 +0000 Subject: [erlang-questions] erlang nif resource & thread safety In-Reply-To: <573C64F6.9000904@ericsson.com> References: <57276304.9070206@ericsson.com> <573C64F6.9000904@ericsson.com> Message-ID: thanks! the current behaviour is what i anted, though indeed having a way to attach a resource to one owner would be useful :) Anyway thans again for your answer. Beno?t On Wed, 18 May 2016 at 14:49, Sverker Eriksson < sverker.eriksson@REDACTED> wrote: > > > On 05/18/2016 02:13 PM, Benoit Chesneau wrote: > > > > On Mon, May 2, 2016 at 4:24 PM Sverker Eriksson < > sverker.eriksson@REDACTED> wrote: > >> A resource will be deallocated when >> >> * all terms created with enif_make_resource() have been garbage collected >> >> * AND the call to enif_alloc_resource() has been balanced by a call to >> enif_release_resource() >> >> * AND all calls to enif_keep_resource() have been balanced by an equal >> number of calls to enif_release_resource() >> >> >> >> On 05/02/2016 11:47 AM, Benoit Chesneau wrote: >> >> I had a look in the doc but I can't figure exactly how much a nif >> resource could live outside the process that created it, ie. how others >> processes can write/read to it and how to make sure the resource will be >> accessible to them. >> >> So I created a nif resource that have the following properties: >> - it is ref counted, each time a call is made to it, the counter is >> incremented and decremented when it return. >> >> A resource already has an internal ref counter, which is decremented by >> the GC and by enif_release_resource(). >> >> Adding your own ref counter is probably either redundant or just plain >> wrong. >> Every call to enif_release_resource() must be balanced by a prior call >> to either enif_alloc_resource() or enif_keep_resource(). You cannot >> release a resource that is still referred to by one of more terms. >> >> >> - the process that create it will live until the vm is closed >> >> A resource it not tied to the process that created it, >> it is tied to the terms that refer it. >> > > Related to that I recently came with the following question: > How to tie a nif resource to a owner process to make sure it's cleaned if > the process crash? > > Looking at the source of ETS it seems to use unexposed functions from > https://github.com/erlang/otp/blob/maint/erts/emulator/beam/erl_process.h > > I think I miss a C function like "nif_tie_resource(pid, somecalleback)" . > Or is there any code that already does that? > > - beno?t > > > > If the crashing process contains the last reference to your resource > then it will be destructed as part of the process termination cleanup > and you don't have to do anything. > > If you want your resource to be "cleaned" at process crash > even though it is still referred by other processes or ETS tables, > then there is no such callback today. > > You can accomplish that however by creating a second "owner" resource > with a pointer to the "payload" resource. Keep the owner resource term > in your owning process and let it clean the payload resource in its > destructor. > The destructor of the payload resource (if any) and all code doing > enif_get_resource must recognize and handle an already cleaned resource. > > > /Sverker, Erlang/OTP > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hellkvist@REDACTED Thu May 19 12:12:18 2016 From: hellkvist@REDACTED (Stefan Hellkvist) Date: Thu, 19 May 2016 12:12:18 +0200 Subject: [erlang-questions] How to detect or be alerted of faults in distributed Erlang systems? Message-ID: <159855E1-762D-489C-BB3D-4910E263C20E@gmail.com> Hello, Does anyone have any recommendations of tools or ways of working to share about how you detect or be alerted of faults on a global level in a distributed Erlang system? The reason why I ask is that I had a trivial fault in a gen_server where one of the handle_cast clauses had a missing {noreply, State} at the end of the clause, so whenever this message arrived at the gen_server the gen_server crashed because of the bad return value from the handle_cast clause. Unfortunately, due to the automatic restart of the gen_server, this went unnoticed for some time because all the actions in the clause was executing well and the message in particular was not very frequent, so the tests run on system level passed (although the restart did have some slight performance effects when occurring). This should?ve of course been caught in a unit tests but this clause was sadly not unit tested. The error was of course clearly reported in the report.log of the node in question but since there were so many nodes in the system and since I am obviously not aware of the process for collecting errors like this (and report them somewhere) I failed to see this in this particular report.log. Until today... So what I am looking for is advice or tools (perhaps it is even part of OTP?) on how to detect such failures on any node at some central point. All the nodes, even if they are all Erlang nodes, are not necessarily connected with every other node though, so it would be good to have a tool that does not depend on the nodes being connected. Stefan From zxq9@REDACTED Thu May 19 12:29:07 2016 From: zxq9@REDACTED (zxq9) Date: Thu, 19 May 2016 19:29:07 +0900 Subject: [erlang-questions] How to detect or be alerted of faults in distributed Erlang systems? In-Reply-To: <159855E1-762D-489C-BB3D-4910E263C20E@gmail.com> References: <159855E1-762D-489C-BB3D-4910E263C20E@gmail.com> Message-ID: <4137614.IIxb8Wi4Wh@burrito> On Thursday 19 May 2016 12:12:18 Stefan Hellkvist wrote: > Hello, > > Does anyone have any recommendations of tools or ways of working to share about how you detect or be alerted of faults on a global level in a distributed Erlang system? > > The reason why I ask is that I had a trivial fault in a gen_server where one of the handle_cast clauses had a missing {noreply, State} at the end of the clause, so whenever this message arrived at the gen_server the gen_server crashed because of the bad return value from the handle_cast clause. Unfortunately, due to the automatic restart of the gen_server, this went unnoticed for some time because all the actions in the clause was executing well and the message in particular was not very frequent, so the tests run on system level passed (although the restart did have some slight performance effects when occurring). > > This should?ve of course been caught in a unit tests but this clause was sadly not unit tested. The error was of course clearly reported in the report.log of the node in question but since there were so many nodes in the system and since I am obviously not aware of the process for collecting errors like this (and report them somewhere) I failed to see this in this particular report.log. Until today... > > So what I am looking for is advice or tools (perhaps it is even part of OTP?) on how to detect such failures on any node at some central point. All the nodes, even if they are all Erlang nodes, are not necessarily connected with every other node though, so it would be good to have a tool that does not depend on the nodes being connected. Hi, Stefan. I think you'll probably get a million good specific recommendations here -- but I'll explain the two most frequent general cases I've seen: 1- Your organization's sysop people already have some monitoring framework for errors and your system just isn't sending data to them. (Like Sentry or whatever.) 2- Your organization has some log analysis jobs that extract some aggregate usage history in addition to highlighting error conditions reported in log messages -- but errors produced by your application just aren't in the rules yet. For systems running in a pre-existing infrastructure or any significantly large system one (or both) of these has always been the case for me -- so after a discussion with a sysop the task becomes one of integration, not standing up a new service. For smaller systems, though, or totally self-contained ones it can be possible that you might not even see logs for months and have no way to analyze them. In this case (assuming the systems are networked, at least intermittently) having them pass at least error logs to a central logging service can be a big benefit (in this I'm thinking specifically of some meterological systems I once dealt with that were scattered around a large geographical area, but did actually have some low-bandwidth network access). As far as a sigle go-to recommendation ("Install The All New WhizzlePop Error Genie App (FOR ERLANG!!1!) and all your troubles will be a distant memory!")... I don't have one. There has always been some log analysis utility already sitting around somewhere, and I've only needed to make sure my system was integrating with it (a task I'll be dealing with next week all over again, in fact). -Craig From venkatesh.s@REDACTED Thu May 19 13:07:23 2016 From: venkatesh.s@REDACTED (Venkatesh Subramani) Date: Thu, 19 May 2016 16:37:23 +0530 Subject: [erlang-questions] Waht is Page & Match indicates in Tsung statistics? Also need explantion about tsung statistics & graph report Message-ID: Hi I am new for Tsung to load testing ejabberd chat server using XMPP. I'm doing the register process by using Register script file. In that, I have configured in request tag as [ error ][1] I have run the test for 100k users with interval time 0.008 for 15 minutes. I have the got the report as no match is 99932 and match is 68. I have referred many sites but not got satisfied on that. My clarification is - Why 68 is in Match counter? and What it indicates? - Also I need explanation for whole tsung-statistics and graph report that have been attached. - When No match condition is satisfied for all users. Such time, Page details is not coming in Main statistics. If condition is not satisfied, then page details is coming in Main statistics. So, i have confused a lot in page means. What is page indicates? - Regards, Venkatesh This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please delete it right away or notify the sender. [image: Inline image 3][image: Inline image 2] -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Graph report for 1lakh user-100516 (1).png Type: image/png Size: 573092 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Page clar in tsung-120516.png Type: image/png Size: 130027 bytes Desc: not available URL: From erlangworkshop@REDACTED Thu May 19 13:28:30 2016 From: erlangworkshop@REDACTED (Erlang Workshop) Date: Thu, 19 May 2016 13:28:30 +0200 Subject: [erlang-questions] [ANN] Final Call for Papers: Erlang Workshop 2016 -- Submission deadline (3 June) approaching Message-ID: <573DA35E.5000709@gmail.com> Apologies for any duplicates you may receive. CALL FOR PAPERS =============== Fifteenth ACM SIGPLAN Erlang Workshop ------------------------------ ----------------------------- Nara, Japan, September 23, 2016 Satellite event of the 21st ACM SIGPLAN International Conference on Functional Programming (ICFP 2016) September 18-24, 2016 The Erlang Workshop aims to bring together the open source, academic, and industrial communities of Erlang, to discuss technologies and languages related to Erlang. The Erlang model of concurrent programming has been widely emulated, for example by Akka in Scala, and even new programming languages were designed atop of the Erlang VM, such as Elixir. Therefore we would like to broaden the scope of the workshop to include systems like those mentioned above. The workshop will enable participants to familiarize themselves with recent developments on new techniques and tools, novel applications, draw lessons from users' experiences and identify research problems and common areas relevant to the practice of Erlang, Erlang-like languages, functional programming, distribution, concurrency etc. We invite three types of submissions. 1. Technical papers describing interesting contributions either in theoretical work or real world applications. Submission related to Erlang, Elixir, Akka, CloudHaskell, Occam, and functional programming are welcome and encouraged. Topics of interest include (but are not limited to): - virtual machine extensions and compilation techniques - implementations and interfaces of Erlang in/with other languages - new tools (profilers, tracers, debuggers, testing frameworks etc.) - language extensions - formal semantics, correctness and verification - testing Erlang programs - program analysis and transformation - Erlang-like languages and technologies - functional languages and multi-processing - concurrency in functional languages - functional languages and distributed computing - parallel programming - pattern based programming - Erlang in education The maximum length for technical papers is restricted to 12 pages. 2. Experience reports describing uses of Erlang in the "real-world", Erlang libraries for specific tasks, experiences from using Erlang in specific application domains, reusable programming idioms and elegant new ways of using Erlang to approach or solve a particular problem. The maximum length for the experience report is restricted to 2 pages. 3. Poster presentations describing topics related to the workshop goals. Each includes a maximum of 2 pages of the abstract and summary. Presentations in this category will be given an hour of shared simultaneous demonstration time. Workshop Co-Chairs ------------------ Melinda T?th, E?tv?s Lor?nd University, Hungary Scott Lystig Fritchie, Basho Japan KK Program Committee ----------------------------- (Note: the Workshop Co-Chairs are also committee members) Jamie Allen, Typesafe Laura M. Castro, University of A Coru?a, Spain Natalia Chechina, University of Glasgow Viktoria F?rd?s, Erlang Solutions Yosuke Hara, Rakuten, Inc. Kenji Rikitake, KRPEO Bruce Tate, iCanMakeItBetter Simon Thompson, University of Kent, UK Important Dates ----------------------- Submissions due: Friday, 3 June, 2016 Author notification: Friday, 8 July, 2016 Final copy due: Sunday, 31 July, 2016 Workshop date: September 23, 2016 Instructions to authors -------------------------------- Papers must be submitted online via EasyChair (via the "Erlang2016" event). The submission page is https://www.easychair.org/conferences/?conf=erlang2016 Submitted papers should be in portable document format (PDF), formatted using the ACM SIGPLAN style guidelines. Each submission must adhere to SIGPLAN's republication policy. Violation risks summary rejection of the offending submission. Accepted papers will be published by the ACM and will appear in the ACM Digital Library. Paper submissions will be considered for poster submission in the case they are not accepted as full papers. Venue & Registration Details ------------------------------------------ For registration, please see the ICFP 2016 web site at: http://conf.researchr.org/home/icfp-2016 Related Links -------------------- CFP: http://conf.researchr.org/track/icfp-2016/erlang-2016-papers ICFP 2016 web site: http://conf.researchr.org/home/icfp-2016 Past ACM SIGPLAN Erlang workshops: http://www.erlang.org/workshop/ Open Source Erlang: http://www.erlang.org/ EasyChair submission site: https://www.easychair.org/conferences/?conf=erlang2016 Author Information for SIGPLAN Conferences: http://www.sigplan.org/authorInformation.htm Atendee Information for SIGPLAN Events: http://www.sigplan.org/Resources/Policies/Anti-harassment -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Thu May 19 13:51:42 2016 From: roger@REDACTED (Roger Lipscombe) Date: Thu, 19 May 2016 12:51:42 +0100 Subject: [erlang-questions] How to detect or be alerted of faults in distributed Erlang systems? In-Reply-To: <159855E1-762D-489C-BB3D-4910E263C20E@gmail.com> References: <159855E1-762D-489C-BB3D-4910E263C20E@gmail.com> Message-ID: If a gen_server dies, error_logger is called. You need to find a way to get hold of those error reports in a timely fashion. In our system, we do this by using lager (https://github.com/basho/lager), which installs an error_logger hook and turns these error reports into error-level logging (alongside ordinary calls to lager:error). Then we use the lager_syslog backend to forward everything to our syslog / logstash / elastic search infrastructure, where we *could* slice and dice the reports for spotting these kinds of things. However, that's not what we *actually* do: we also have a custom lager backend that publishes error-level messages to a rabbit queue. The queue consumer groups the error reports together and then periodically emails them to a designated address. By regularly checking the relevant folder in my mail client, I can triage the error reports and file the appropriate bugs in our issue tracker. Obviously you could use your own error_logger hook to do this, rather than using lager. On 19 May 2016 at 11:12, Stefan Hellkvist wrote: > Hello, > > Does anyone have any recommendations of tools or ways of working to share about how you detect or be alerted of faults on a global level in a distributed Erlang system? > > The reason why I ask is that I had a trivial fault in a gen_server where one of the handle_cast clauses had a missing {noreply, State} at the end of the clause, so whenever this message arrived at the gen_server the gen_server crashed because of the bad return value from the handle_cast clause. Unfortunately, due to the automatic restart of the gen_server, this went unnoticed for some time because all the actions in the clause was executing well and the message in particular was not very frequent, so the tests run on system level passed (although the restart did have some slight performance effects when occurring). > > This should?ve of course been caught in a unit tests but this clause was sadly not unit tested. The error was of course clearly reported in the report.log of the node in question but since there were so many nodes in the system and since I am obviously not aware of the process for collecting errors like this (and report them somewhere) I failed to see this in this particular report.log. Until today... > > So what I am looking for is advice or tools (perhaps it is even part of OTP?) on how to detect such failures on any node at some central point. All the nodes, even if they are all Erlang nodes, are not necessarily connected with every other node though, so it would be good to have a tool that does not depend on the nodes being connected. > > Stefan > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mattevans123@REDACTED Thu May 19 17:16:43 2016 From: mattevans123@REDACTED (Matthew Evans) Date: Thu, 19 May 2016 11:16:43 -0400 Subject: [erlang-questions] ets with maps and/or select Message-ID: Hi, I want to create an ets table to store a large number of records (over 100,000). I then want to use ets:select/ets:match to search this table for records containing a specific value. The issue is that I do not know the position of this value in the record. For example take this record: R1#my_record{key = 123, value1 = 77, value2 = 88} R2#my_record{key = 321, value1 = 88, value2 = 77} R3#my_record{key = 999, value1 = 12, value2 = 73} I want to search the ets table to find any record containing value 77 (I could of course save these values in a list in each record, this is just an example). I know I can do an ets:foldl for this purpose, but I was wondering if there is a more efficient way? Could I for example save the values in a map, put the map in a record in the ets table and create a match_specification that I can use with an ets:select or ets:match? The map may look like: M = #{77 => ok, 88 => ok}.R4#my_record{key = 123, map = M}. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Thu May 19 17:45:51 2016 From: zxq9@REDACTED (zxq9) Date: Fri, 20 May 2016 00:45:51 +0900 Subject: [erlang-questions] ets with maps and/or select In-Reply-To: References: Message-ID: <4265828.cMAc8Oe7lB@changa> On 2016?5?19? ??? 11:16:43 Matthew Evans wrote: > Hi, > I want to create an ets table to store a large number of records (over 100,000). I then want to use ets:select/ets:match to search this table for records containing a specific value. The issue is that I do not know the position of this value in the record. For example take this record: > R1#my_record{key = 123, value1 = 77, value2 = 88} > R2#my_record{key = 321, value1 = 88, value2 = 77} > R3#my_record{key = 999, value1 = 12, value2 = 73} > I want to search the ets table to find any record containing value 77 (I could of course save these values in a list in each record, this is just an example). > I know I can do an ets:foldl for this purpose, but I was wondering if there is a more efficient way? > Could I for example save the values in a map, put the map in a record in the ets table and create a match_specification that I can use with an ets:select or ets:match? > The map may look like: > M = #{77 => ok, 88 => ok}.R4#my_record{key = 123, map = M}. This is the kind of thing you would build an index for. You could create a K/V structure (map, dict, another ETS table, whatever) of the form {Value, [AssociatedRecordKeys]}. The query steps then become: 1- Check the index 2- Directly pull the list of records stored in ETS associated with that value. This is, of course, doing half the work of maintaining an index yourself, which is a databasey thing to be doing. If its just this one task in your whole project then the overhead of maintaining the index and mental overhead of writing it is probably lower than including a database in your project. But if you have even one more situation that seems similar to this then the better use of your time would probably be to include a DB product that provides this kind of indexing out of the box and using that rather than building your ownDB-ish indexing thingy out of lower-level data structures/operations. I think. Maybe R19 has some magical operation I don't know about. ...? -Craig From list1@REDACTED Thu May 19 18:12:04 2016 From: list1@REDACTED (Grzegorz Junka) Date: Thu, 19 May 2016 16:12:04 +0000 Subject: [erlang-questions] Erlang and asm.js Message-ID: <54abded8-8e12-c7df-2bc5-ac0c7f2b7671@gjunka.com> Has anyone tried to port Erlang to asm.js so that it could run in the browser? I tried to search but it looks like only compiling Erlang code was attempted: http://luvv.ie/ They claim that any language that compiles with LLVM could be compiled to asm.js. Considering that HIPE compiles Erlang to a native code, theoretically both, BEAM and Erlang code could be compiled with LLVM to asm.js, and that would mean the ability to run any Erlang application together with BEAM/NIF-dependent libraries in the browser. Unless there are some specific assembler-based optimizations that couldn't be expressed in C easily? Grzegorz From ddosia@REDACTED Thu May 19 18:44:10 2016 From: ddosia@REDACTED (Daniil Churikov) Date: Thu, 19 May 2016 17:44:10 +0100 Subject: [erlang-questions] Database abstraction libraries In-Reply-To: References: Message-ID: Just spotted this question. You might find this[1] small library useful as well. [1] https://github.com/ddosia/mekao 2016-04-22 19:30 GMT+01:00 Andrew Berman : > Hey list, > > Are there any well supported database abstraction libraries out there? I > found a couple on Github (sqerl and sumo_db), but I wanted to see if there > was one that is popular that people are using in production under heavy > load. I'm not looking for an ORM (I had enough bad experience with it > using Hibernate in Java). > > Any experiences? > > Thanks, > > Andrew > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Thu May 19 18:47:42 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 19 May 2016 16:47:42 +0000 Subject: [erlang-questions] Database abstraction libraries In-Reply-To: References: Message-ID: oh also here is a Key/Value abstraction lib: https://github.com/refuge/rkvs - benoit On Thu, May 19, 2016 at 6:44 PM Daniil Churikov wrote: > Just spotted this question. You might find this[1] small library useful as > well. > > [1] https://github.com/ddosia/mekao > > 2016-04-22 19:30 GMT+01:00 Andrew Berman : > >> Hey list, >> >> Are there any well supported database abstraction libraries out there? I >> found a couple on Github (sqerl and sumo_db), but I wanted to see if there >> was one that is popular that people are using in production under heavy >> load. I'm not looking for an ORM (I had enough bad experience with it >> using Hibernate in Java). >> >> Any experiences? >> >> Thanks, >> >> Andrew >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Thu May 19 23:28:01 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 20 May 2016 00:28:01 +0300 Subject: [erlang-questions] monitor long_schedule and strange timeouts Message-ID: Once per hour we get strange situation on customer server. Log is getting full with messages like: 2016-05-20 02:06:52.204 <0.300.0> flu_sys_monitor:46 Monitor: {monitor,<0.27485.22>,long_schedule,[{timeout,2591},{in,{gen_server,loop,6}},{out,{gen_server,loop,6}}]} 2016-05-20 02:06:52.210 <0.300.0> flu_sys_monitor:46 Monitor: {monitor,<0.342.0>,long_schedule,[{timeout,2595},{in,{live_info_storage,loop_setter,1}},{out,{live_info_storage,loop_setter,1}}]} (loop_setter is a very small function that just reads message and set field in ets table) and many processes in system are getting stuck in different places like: {current_stacktrace,[{erts_internal,await_result,1,[]} Amount of erlang:statistics(total_active_tasks) is getting down from average 800 to 200 when such situation happens. I have two questions: 1) is erlang:statistics(total_active_tasks) about 800 a normal value or it is a very huge overload? System is working ok with such metric 2) are there any hints how to debug situation? Is it an external of something internal? If customer is running some task on this server, then why long schedules may appear? -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri May 20 02:56:07 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 20 May 2016 12:56:07 +1200 Subject: [erlang-questions] Erlang and asm.js In-Reply-To: <54abded8-8e12-c7df-2bc5-ac0c7f2b7671@gjunka.com> References: <54abded8-8e12-c7df-2bc5-ac0c7f2b7671@gjunka.com> Message-ID: On 20/05/16 4:12 AM, Grzegorz Junka wrote: > Has anyone tried to port Erlang to asm.js so that it could run in the > browser? Erlang can run without native code. C can be compiled to JavaScript using emscripten. The real problem, I imagine, is support for libraries. I suppose this would solve the problem of Erlang running too fast on too many platforms. From ok@REDACTED Fri May 20 05:11:52 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 20 May 2016 15:11:52 +1200 Subject: [erlang-questions] Database abstraction libraries In-Reply-To: References: Message-ID: <12f11e0c-c633-79a1-7ce7-2a00697ebb78@cs.otago.ac.nz> On 20/05/16 4:47 AM, Benoit Chesneau wrote: > oh also here is a Key/Value abstraction lib: > https://github.com/refuge/rkvs\ Hadn't come across that. Nice. From jinni.park@REDACTED Fri May 20 05:18:42 2016 From: jinni.park@REDACTED (Park, Sungjin) Date: Fri, 20 May 2016 12:18:42 +0900 Subject: [erlang-questions] Memory usage increase in OTP-18.2.1 Message-ID: I found that the memory usage's increased by about 20% when I upgraded erlang version from otp-r15b03 to otp-18.2.1. The green line's otp-18.2.1 and the blue line's otp-r15b03 in the attached graph. The vertical axis is residential memory used by beam in percentage. The green line starts a bit higher than the blue. This is ok but the gap increases as time goes by. erlang:memory/0 reveals memory usage details. == otp-r15b03 == 1> erlang:memory(). [{total,208909800}, {processes,69473238}, {processes_used,69388990}, {system,139436562}, {atom,924601}, {atom_used,900928}, {binary,1066312}, {code,22978624}, {ets,5108576}] == otp-18.2.1 == 1> erlang:memory(). [{total,250735008}, {processes,123869144}, {processes_used,123804656}, {system,126865864}, {atom,1033377}, {atom_used,1000716}, {binary,6331096}, {code,29690005}, {ets,5317136}] otp-18.2.1 uses about twice as much in processes and 6 times as much in binary. The system acts as a reverse proxy that receives http requests and forwards them to multiple backends. Nothing very special. Well, it's still affordable but I want to know that this is the normal price to pay to keep track of the otp pace. Or can there possibly be anything wrong in my code that I have to investigate further? -- Park, Sungjin -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Screen Shot 2016-05-20 at 12.01.22 PM.png Type: image/png Size: 42633 bytes Desc: not available URL: From hellkvist@REDACTED Fri May 20 07:32:48 2016 From: hellkvist@REDACTED (Stefan Hellkvist) Date: Fri, 20 May 2016 07:32:48 +0200 Subject: [erlang-questions] How to detect or be alerted of faults in distributed Erlang systems? In-Reply-To: References: <159855E1-762D-489C-BB3D-4910E263C20E@gmail.com> Message-ID: <585A26C8-194E-416F-8EFA-0C83AB300A4C@gmail.com> Thank you all for your suggestions and recommendations. I will look into WombatOAM and see what that is about. I do not have any existing, larger OAM system to integrate with so I will have to roll my own somehow. The nodes use lager and rabbitmq already so a solution similar to what Roger suggested might be what I will attempt. Thanks Stefan > On 19 May 2016, at 14:06, Zsolt Laky wrote: > > Hi Stefan, > > In addition what Roger mentioned, you may consider a monitoring tool like WombatOAM. It may be a big help in running a distributed environment. Out of the box, no need to implement sophisticated alert systems. > > Best, > > Zsolt >> On May 19, 2016, at 1:51 PM, Roger Lipscombe wrote: >> >> If a gen_server dies, error_logger is called. You need to find a way >> to get hold of those error reports in a timely fashion. In our system, >> we do this by using lager (https://github.com/basho/lager), which >> installs an error_logger hook and turns these error reports into >> error-level logging (alongside ordinary calls to lager:error). >> >> Then we use the lager_syslog backend to forward everything to our >> syslog / logstash / elastic search infrastructure, where we *could* >> slice and dice the reports for spotting these kinds of things. >> >> However, that's not what we *actually* do: we also have a custom lager >> backend that publishes error-level messages to a rabbit queue. The >> queue consumer groups the error reports together and then periodically >> emails them to a designated address. By regularly checking the >> relevant folder in my mail client, I can triage the error reports and >> file the appropriate bugs in our issue tracker. >> >> Obviously you could use your own error_logger hook to do this, rather >> than using lager. >> >> On 19 May 2016 at 11:12, Stefan Hellkvist wrote: >>> Hello, >>> >>> Does anyone have any recommendations of tools or ways of working to share about how you detect or be alerted of faults on a global level in a distributed Erlang system? >>> >>> The reason why I ask is that I had a trivial fault in a gen_server where one of the handle_cast clauses had a missing {noreply, State} at the end of the clause, so whenever this message arrived at the gen_server the gen_server crashed because of the bad return value from the handle_cast clause. Unfortunately, due to the automatic restart of the gen_server, this went unnoticed for some time because all the actions in the clause was executing well and the message in particular was not very frequent, so the tests run on system level passed (although the restart did have some slight performance effects when occurring). >>> >>> This should?ve of course been caught in a unit tests but this clause was sadly not unit tested. The error was of course clearly reported in the report.log of the node in question but since there were so many nodes in the system and since I am obviously not aware of the process for collecting errors like this (and report them somewhere) I failed to see this in this particular report.log. Until today... >>> >>> So what I am looking for is advice or tools (perhaps it is even part of OTP?) on how to detect such failures on any node at some central point. All the nodes, even if they are all Erlang nodes, are not necessarily connected with every other node though, so it would be good to have a tool that does not depend on the nodes being connected. >>> >>> Stefan >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From michal@REDACTED Fri May 20 08:00:20 2016 From: michal@REDACTED (=?utf-8?q?Micha=C5=82_Muska=C5=82a?=) Date: Thu, 19 May 2016 23:00:20 -0700 (PDT) Subject: [erlang-questions] High memory consumption inside docker In-Reply-To: References: Message-ID: Hello, It took me some time to finally get around to looking again at this. Running the instrument module turned out to be very helpful, thank you Lukas for this suggestion. I got very high usage for drv_tab, fd_tab and port_tab: {drv_tab,[{sizes,50331648,50331648,50331648}, {blocks,1,1,1}]}, {fd_tab,[{sizes,33554432,33554432,33554432},{blocks,1,1,1}]}, {port_tab,[{sizes,12582975,12582975,12582975}, {blocks,1,1,1}]} Running docker with decreased ulimit via the --ulimit nofile=1024:1024 flag seems to fix this issue. There is still increased binary usage, but it's not in any way as severe as this was. Thank you for your help, Micha?. On Mar 29 2016, at 6:44 pm, Lukas Larsson <lukas@REDACTED> wrote: > Hello, > > > > You want to start the emulator with the "+Mim true +Mis true" flags and then use instrument () to inspect what is going on. The most interesting for you should be something like: > > > > lists:reverse(lists:keysort(2,instrument:memory_status(types))). > > > > For example: > > > > erl +Mis true +Mim true > > Erlang/OTP 19 [DEVELOPMENT] [erts-8.0] [source-9946a17] [64-bit] [smp:16:16] [async-threads:10] [hipe] [kernel-poll:false] > > 1> lists:reverse(lists:keysort(2,instrument:memory_status(types))). > > [{timer_wheel,[{sizes,8391792,8391792,8391792}, > > {blocks,16,16,16}]}, > > {code,[{sizes,3463220,3463220,3469945},{blocks,84,84,84}]}, > > {proc_tab,[{sizes,3145791,3145791,3145791},{blocks,1,1,1}]}, > > {heap,[{sizes,1882920,1932208,3456120},{blocks,26,28,28}]}, > > {pre_alloc_data,[{sizes,1032303,1032303,1032303}, > > {blocks,17,17,17}]}, > > {port_tab,[{sizes,786495,786495,786495},{blocks,1,1,1}]}, > > {old_heap,[{sizes,679872,682880,2941656},{blocks,10,11,11}]}, > > {scheduler_data,[{sizes,673141,673141,673141}, > > {blocks,3,3,3}]}, > > {export_entry,[{sizes,462528,462528,462528}, > > {blocks,2628,2628,2628}]}, > > {literal,[{sizes,379648,379648,379648},{blocks,84,84,84}]}, > > {ethread_long_lived,[{sizes,326016,326016,326016}, > > {blocks,283,283,283}]}, > > {atom_entry,[{sizes,304840,304840,304840}, > > {blocks,7621,7621,7621}]}, > > {beam_register,[{sizes,264544,264544,264544}, > > {blocks,32,32,32}]}, > > {export_tab,[{sizes,162888,162888,162888}, > > {blocks,15,15,15}]}, > > {hipe_data,[{sizes,148464,148464,148464},{blocks,18,18,18}]}, > > {ethread_standard,[{sizes,133487,133487,133487}, > > {blocks,6,6,6}]}, > > {db_term,[{sizes,119568,119568,119568}, > > {blocks,388,388,388}]}, > > {atom_tab,[{sizes,112232,112232,121472},{blocks,10,10,10}]}, > > {atom_text,[{sizes,98328,98328,98328},{blocks,3,3,3}]}, > > {db_tabs,[{sizes,82023,82023,82023},{blocks,3,3,3}]}, > > {drv_internal,[{sizes,79352,79352,79608},{blocks,4,4,19}]}, > > {db_segment,[{sizes,51744,51744,53824},{blocks,19,19,...}]}, > > {fun_entry,[{sizes,51568,51568,...},{blocks,586,...}]}, > > {drv_binary,[{sizes,47774,...},{blocks,...}]}, > > {driver_event_state,[{sizes,...},{...}]}, > > {module_tab,[{...}|...]}, > > {module_entry,[...]}, > > {proc,...}, > > {...}|...] > > > > So in the above we can see that the timer wheel structures take the most memory and then code, proc tab etc etc. > > > > Lukas > > > > On Tue, Mar 29, 2016 at 2:01 PM, Micha? Muska?a <[michal@REDACTED](mailto:michal@REDACTED)> wrote: > >> Hello everybody, This is a followup from the elixir mailing list: There were several issues involved in that particular problem, but one of them might be of interest to the people here as well, as it's generally erlang-related rather than tied to elixir or any package in particular. The issue comes down to unusually high memory consumption for erlang, specifically the system part from erlang:memory/0. The output below shows values when running outside docker and inside: Outside docker: erl -eval 'io:format("~w~n", [erlang:memory()]), init:stop().' -noshell [{total,15182640},{processes,3799768},{processes_used,3797720},{system,1138287 2},{atom,194289},{atom_used,169621},{binary,78344},{code,3868184},{ets,229416} ] Inside docker's official erlang image (debian based) docker run --rm -it erlang:18-slim erl -eval 'io:format("~w~n", [erlang:memory()]), init:stop().' -noshell [{total,111303160},{processes,3799768},{processes_used,3797720},{system,107503 392},{atom,194289},{atom_used,169584},{binary,595608},{code,3860673},{ets,2394 64}] Inside minimal alpine linux erlang image docker run --rm -it msaraiva/erlang erl -eval 'io:format("~w~n", [erlang:memory()]), init:stop().' -noshell [{total,111432888},{processes,3748312},{processes_used,3746312},{system,107684 576},{atom,194289},{atom_used,169869},{binary,596568},{code,3847723},{ets,1852 96}] As you can see all the values except for system are very similar, with system being almost 10x as large. The issue is also consistent across different images. While outside docker a simple shell uses around 15MB, inside docker it grows to 80MB. I tried to investigate the issue further using recon, and it looks like the excessive memory is allocated with ll_alloc, although I have to admit I'm out of my depth here. I'll be thankful for any help or indications where I could dig deeper. Micha?. _______________________________________________ erlang-questions mailing list [erlang-questions@REDACTED](mailto:erlang-questions@REDACTED) > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri May 20 08:10:18 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 20 May 2016 09:10:18 +0300 Subject: [erlang-questions] Releases and Different OSs In-Reply-To: <1463577779.3602082.611473737.4788CABC@webmail.messagingengine.com> References: <1463577779.3602082.611473737.4788CABC@webmail.messagingengine.com> Message-ID: We are making something like that with our Flussonic. Erlang is compiled once on linux and then used in mac to create a package. -------------- next part -------------- An HTML attachment was scrubbed... URL: From list1@REDACTED Fri May 20 12:26:25 2016 From: list1@REDACTED (Grzegorz Junka) Date: Fri, 20 May 2016 10:26:25 +0000 Subject: [erlang-questions] Erlang and asm.js In-Reply-To: References: <54abded8-8e12-c7df-2bc5-ac0c7f2b7671@gjunka.com> Message-ID: On 20/05/2016 00:56, Richard A. O'Keefe wrote: > > On 20/05/16 4:12 AM, Grzegorz Junka wrote: >> Has anyone tried to port Erlang to asm.js so that it could run in the >> browser? > Erlang can run without native code. C can be compiled to JavaScript > using emscripten. The real problem, I imagine, is support for libraries. > > I suppose this would solve the problem of Erlang running too fast on > too many platforms. I didn't think running Erlang beam file on top of a virtual machine running on asm.js would be beneficial if both, the BEAM VM and the Erlang code could be compiled to asm.js. And which libraries do you require support for? If they are Erlang libraries and can be compiled to asm.js they can run alongside the main application. The only difficulty I can see is the scheduler which would need to work on JavaScript workers somehow instead of real CPU cores. But if they managed to run Ruby, Lua, Perl, Python, OpenGL, SQLite, Unreal Engine 3/4, Doom, and Dosbox on asm.js (among others) I don't see a technical reason why it wouldn't be possible somehow with Erlang, even if the scheduler had to use just the main underlying asm.js thread to juggle all its processes. Grzegorz From sverker.eriksson@REDACTED Fri May 20 12:32:57 2016 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Fri, 20 May 2016 12:32:57 +0200 Subject: [erlang-questions] How to disable parentheses matching in Erlang shell? In-Reply-To: References: <6013b959-58b0-a963-9478-d6845993f024@gjunka.com> <57379656.6010306@gmail.com> <5481da92-8dbb-750e-8ced-c5ffafd2e4ce@gjunka.com> <03e2c191-5810-2b29-6dea-38cafaa7d2cb@gjunka.com> <573B1858.1070203@ericsson.com> Message-ID: <573EE7D9.8040209@ericsson.com> Grzegorz, I don't think this problem has anything to do with your application. It think it has to do with the OS, the kind of terminals, where run_erl and to_erl are started, and their window widths. And it's not connected to parantheses matching per se but rather getting beam, run_erl and to_erl to agree on the window width and how to move to get the terminal to correctly move the cursor up and down. I tested with Ctrl+A and Ctrl+E and verified if it correctly jumped between start and end of the command line. I did another commit that you can try git fetch https://github.com/sverker/otp sverker/ttsl_drv-move-down But to get this right I think some work needs to be done to get to_erl to use the same terminal settings as run_erl for example. It's on the todo list. /Sverker, Erlang/OTP On 05/17/2016 07:55 PM, Grzegorz Junka wrote: > Sverker, I have a simple application which you could quickly compile > to be able to easily reproduce this issue. Just execute those steps > (one by one): > > mkdir test_sc > cd test_sc/ > git clone https://github.com/builderl/ex1_simple_cache.git > cd ex1_simple_cache/ > gmake get-deps > gmake dev > ./bin/init.esh > ./bin/start.esh > to_erl ../sc-1/shell/ > > This will pull an application, get all its dependencies, compile, > install, configure, start and connect to one of the running nodes > (there will be two running nodes from the same release). > > I hope that will be helpful but of course I am happy to recompile the > OTP from your branch and retry if needed. > > Grzegorz > > > On 17/05/2016 17:01, Grzegorz Junka wrote: >> :( >> >> As soon as I closed the bracket after typing {e, 5} from my example >> the cursor jumped some 10 columns to the right and then to the >> beginning of the line. That's how far it got (without the fix the >> cursor would jump incorrectly only after closing ']'). >> >> (sc-1@REDACTED)1> lists:sort([{a, 1}, {b, 2}, {c, 3}, >> {d, 4}, {e, 5} >> >> Grzegorz >> >> >> On 17/05/2016 13:10, Sverker Eriksson wrote: >>> Could you test this fix and see if it makes the cursor behave >>> when connected with to_erl/run_erl. >>> >>> git fetch https://github.com/sverker/otp sverker/ttsl_drv-move-down >>> >>> /Sverker, Erlang/OTP >>> >>> >>> On 05/15/2016 11:45 PM, Grzegorz Junka wrote: >>>> >>>> I just realized what is likely causing the problem. When I start >>>> the shell with erl and type in this testing string (with spaces so >>>> it's longer) it all works fine: >>>> >>>> lists:sort([{a, 1}, {b, 2}, {c, 3}, {d, 4}, {e, 5}, {f, 6}, {g, 7}, >>>> {h, 8}, {i, 9}, {j, 10}]). >>>> >>>> But I very rarely use Erlang shell in this way. Most of the time I >>>> am connecting to running Erlang nodes using to_erl specifying the >>>> shell's PID. Then it breaks, probably because my terminal size >>>> doesn't match the default terminal size with which the shell has >>>> been originally started. >>>> >>>> When I then try to type in the same example the shell will match >>>> each {} correctly but as soon as I type ']' to close the list the >>>> cursor jumps to the line above, then some other place when I type ')'. >>>> >>>> So it's not related to the environment but to the way I am using >>>> Erlang shell. Is there any solution to that? >>>> >>>> Grzegorz >>>> >>>> >>> >> > > From mononcqc@REDACTED Fri May 20 15:32:46 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 20 May 2016 09:32:46 -0400 Subject: [erlang-questions] Memory usage increase in OTP-18.2.1 In-Reply-To: References: Message-ID: <20160520133245.GC43164@fhebert-ltm2.internal.salesforce.com> On 05/20, Park, Sungjin wrote: >I found that the memory usage's increased by about 20% when I upgraded >erlang version from otp-r15b03 to otp-18.2.1. The green line's otp-18.2.1 >and the blue line's otp-r15b03 in the attached graph. The vertical axis is >residential memory used by beam in percentage. The green line starts a bit >higher than the blue. This is ok but the gap increases as time goes by. > Well, that's gonna be hard to pinpoint. What you're doing is going over 3 years of development and see that things changed. There's a lot of stuff that happened, and without knowing what your reverse proxy does or uses as a library it will be quite tricky to just say "oh yeah, for that pattern, 18 months ago, patch X caused this to bloat". I'm sure you can understand how tricky this is. First, let's look at the only piece of data we've got, the memory figures: >[{total, 208909800}, >[{total, 250735008}, > {processes, 69473238}, > {processes, 123869144}, > {processes_used, 69388990}, > {processes_used, 123804656}, > {binary, 1066312}, > {binary, 6331096}, > {code, 22978624}, > {code, 29690005}, > So it sounds like process memory went up, so did binary memory, and so did code memory. The most significant bump seems to come from processes though. That's as far as diagnostics can go. >The system acts as a reverse proxy that receives http requests and >forwards >them to multiple backends. Nothing very special. > There's a lot that can go on in a reverse proxy. >Well, it's still affordable but I want to know that this is the normal >price to pay to keep track of the otp pace. Or can there possibly be >anything wrong in my code that I have to investigate further? > Investigate further. There's so many ways to go forward with this: - try releases one at a time until you find which one exhibits the most drastic rise and look at the changelog - run manual garbage collections over the node and see if that fixes it; look in changelogs for changes to how GC works - did you need to update any libraries or dependencies when moving from R15 to 18? Check what changed there - Treat the issue as any other memory issue ignoring the version change and see if you can find anything. Most of the tips I have, I have written down in www.erlang-in-anger.com. Hopefully there's stuff in there that can help. The cost of moving forwards in Erlang is, in my experience, rather cheap compared to many platforms and languages, except when you're one of these people hitting a corner case hard (see: basho with scheduler collapse for a great example), but it is certainly going to be a cheaper cost when you manage to amortize it over many releases rather than introducing 3 years of change at once. Regards, Fred. From mikpelinux@REDACTED Fri May 20 16:31:55 2016 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Fri, 20 May 2016 16:31:55 +0200 Subject: [erlang-questions] monitor long_schedule and strange timeouts In-Reply-To: References: Message-ID: <22335.8155.361923.881447@gargle.gargle.HOWL> Max Lapshin writes: > Once per hour we get strange situation on customer server. > > Log is getting full with messages like: > > > 2016-05-20 02:06:52.204 <0.300.0> flu_sys_monitor:46 Monitor: > {monitor,<0.27485.22>,long_schedule,[{timeout,2591},{in,{gen_server,loop,6}},{out,{gen_server,loop,6}}]} > > 2016-05-20 02:06:52.210 <0.300.0> flu_sys_monitor:46 Monitor: > {monitor,<0.342.0>,long_schedule,[{timeout,2595},{in,{live_info_storage,loop_setter,1}},{out,{live_info_storage,loop_setter,1}}]} > > (loop_setter is a very small function that just reads message and set field > in ets table) > > and many processes in system are getting stuck in different places like: > > > {current_stacktrace,[{erts_internal,await_result,1,[]} > > > Amount of erlang:statistics(total_active_tasks) is getting down from > average 800 to 200 when such situation happens. > > > I have two questions: [snip] > 2) are there any hints how to debug situation? Is it an external of > something internal? If customer is running some task on this server, then > why long schedules may appear? A. The server is slowing down for whatever reason. Use normal OS-level monitoring tools to look for CPU, memory, or I/O hogs around the times your long schedules occur. A backup can slow down your file accesses, for instance. B. You may be using BIFs that perform a lot of work without yielding to the scheduler. Unless the context of the long_schedule events give enough hints as to what BIFs are involved, I'd use either gdb to dump the scheduler thread stacks at frequent intervals, or run a gprof- enabled VM and then analyze the gprof output. Other profiling tools like perf should also work. We have hit a number of these long-running BIFs, some have been fixed, but not all. A special case is listing directories with huge numbers of files in them: that can take very long time but consumes very little CPU. From list1@REDACTED Sat May 21 13:54:57 2016 From: list1@REDACTED (Grzegorz Junka) Date: Sat, 21 May 2016 11:54:57 +0000 Subject: [erlang-questions] How to disable parentheses matching in Erlang shell? In-Reply-To: <573EE7D9.8040209@ericsson.com> References: <6013b959-58b0-a963-9478-d6845993f024@gjunka.com> <57379656.6010306@gmail.com> <5481da92-8dbb-750e-8ced-c5ffafd2e4ce@gjunka.com> <03e2c191-5810-2b29-6dea-38cafaa7d2cb@gjunka.com> <573B1858.1070203@ericsson.com> <573EE7D9.8040209@ericsson.com> Message-ID: <28bd96dc-46a9-c81f-a710-63dffe851bb6@gjunka.com> Thanks Sverker, but the behaviour is the same as after the previous change. I understand it may not be possible to apply a quick and proper fix straight away. That's why I proposed to have at least the possibility of disabling this feature. It's not terribly important and slightly useful when it's working, but very annoying when it's not working. Maybe disable it by default in to_erl until it's properly fixed for example? And allow to enable it with some flag if some really requires it? Grzegorz On 20/05/2016 10:32, Sverker Eriksson wrote: > Grzegorz, > > I don't think this problem has anything to do > with your application. > > It think it has to do with the OS, the kind of terminals, > where run_erl and to_erl are started, and their window > widths. > > And it's not connected to parantheses matching per se > but rather getting beam, run_erl and to_erl to agree > on the window width and how to move to get the terminal > to correctly move the cursor up and down. I tested with > Ctrl+A and Ctrl+E and verified if it correctly jumped > between start and end of the command line. > > I did another commit that you can try > > git fetch https://github.com/sverker/otp sverker/ttsl_drv-move-down > > But to get this right I think some work needs to be done > to get to_erl to use the same terminal settings as run_erl > for example. It's on the todo list. > > /Sverker, Erlang/OTP > > > On 05/17/2016 07:55 PM, Grzegorz Junka wrote: >> Sverker, I have a simple application which you could quickly compile >> to be able to easily reproduce this issue. Just execute those steps >> (one by one): >> >> mkdir test_sc >> cd test_sc/ >> git clone https://github.com/builderl/ex1_simple_cache.git >> cd ex1_simple_cache/ >> gmake get-deps >> gmake dev >> ./bin/init.esh >> ./bin/start.esh >> to_erl ../sc-1/shell/ >> >> This will pull an application, get all its dependencies, compile, >> install, configure, start and connect to one of the running nodes >> (there will be two running nodes from the same release). >> >> I hope that will be helpful but of course I am happy to recompile the >> OTP from your branch and retry if needed. >> >> Grzegorz >> >> >> On 17/05/2016 17:01, Grzegorz Junka wrote: >>> :( >>> >>> As soon as I closed the bracket after typing {e, 5} from my example >>> the cursor jumped some 10 columns to the right and then to the >>> beginning of the line. That's how far it got (without the fix the >>> cursor would jump incorrectly only after closing ']'). >>> >>> (sc-1@REDACTED)1> lists:sort([{a, 1}, {b, 2}, {c, 3}, >>> {d, 4}, {e, 5} >>> >>> Grzegorz >>> >>> >>> On 17/05/2016 13:10, Sverker Eriksson wrote: >>>> Could you test this fix and see if it makes the cursor behave >>>> when connected with to_erl/run_erl. >>>> >>>> git fetch https://github.com/sverker/otp sverker/ttsl_drv-move-down >>>> >>>> /Sverker, Erlang/OTP >>>> >>>> >>>> On 05/15/2016 11:45 PM, Grzegorz Junka wrote: >>>>> >>>>> I just realized what is likely causing the problem. When I start >>>>> the shell with erl and type in this testing string (with spaces so >>>>> it's longer) it all works fine: >>>>> >>>>> lists:sort([{a, 1}, {b, 2}, {c, 3}, {d, 4}, {e, 5}, {f, 6}, {g, >>>>> 7}, {h, 8}, {i, 9}, {j, 10}]). >>>>> >>>>> But I very rarely use Erlang shell in this way. Most of the time I >>>>> am connecting to running Erlang nodes using to_erl specifying the >>>>> shell's PID. Then it breaks, probably because my terminal size >>>>> doesn't match the default terminal size with which the shell has >>>>> been originally started. >>>>> >>>>> When I then try to type in the same example the shell will match >>>>> each {} correctly but as soon as I type ']' to close the list the >>>>> cursor jumps to the line above, then some other place when I type >>>>> ')'. >>>>> >>>>> So it's not related to the environment but to the way I am using >>>>> Erlang shell. Is there any solution to that? >>>>> >>>>> Grzegorz >>>>> >>>>> >>>> >>> >> >> > From zzantozz@REDACTED Sun May 22 01:25:03 2016 From: zzantozz@REDACTED (Ryan Stewart) Date: Sat, 21 May 2016 23:25:03 +0000 Subject: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) In-Reply-To: References: <5736E1B4.80106@ninenines.eu> <20160516234917.GA3618@ferdmbp.local> Message-ID: I think you're still ignoring the ordering guarantees of supervision. If "top_sup" supervises [important_server_process, "worker_sup"], and "worker_sup" is a simple_one_for_one where you keep your transient/temporary workers, then "top_sup" will ensure that important_server_process doesn't get the shutdown signal until all workers are finished. It may not apply in this specific case, but I've found it invaluable. On Tue, May 17, 2016 at 3:09 PM Chandru < chandrashekhar.mullaparthi@REDACTED> wrote: > On 17 May 2016 at 00:49, Fred Hebert wrote: > >> On 05/16, Chandru wrote: >> >>> >>> No, it's not. The reason a terminate callback is provided in a gen_server >>> is so that a process can clean up when it terminates, not to delegate it >>> to >>> other processes. >>> >>> >> I'm gonna side with Lo?c here. The terminate callback is good for any >> process-local cleanup or optimistic work, but is by no means a safe way to >> terminate anything. > > >> >> For example, if you have many children to terminate and through some >> interleaving brutall_kill is triggered (or anyone calls exit(Pid, kill)), >> whatever work you wanted to do in terminate will be skipped by a >> non-trappable exit signal. >> > > Agreed, but how does adding the connection handling process (in Oliver's > use case) to a simple_one_for_one supervisor help? It doesn't give him > anything other than the illusion of being "supervised". > > Chandru > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Sun May 22 23:10:24 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Sun, 22 May 2016 22:10:24 +0100 Subject: [erlang-questions] simple_one_for_one supervisor - what happens at restart? (also: gen_tcp) In-Reply-To: References: <5736E1B4.80106@ninenines.eu> <20160516234917.GA3618@ferdmbp.local> Message-ID: On 22 May 2016 at 00:25, Ryan Stewart wrote: > I think you're still ignoring the ordering guarantees of supervision. If > "top_sup" supervises [important_server_process, "worker_sup"], and > "worker_sup" is a simple_one_for_one where you keep your > transient/temporary workers, then "top_sup" will ensure that > important_server_process doesn't get the shutdown signal until all workers > are finished. It may not apply in this specific case, but I've found it > invaluable. > Sure, it is useful, but I was discussing Oliver's specific question for his use case. Not the general merits of supervision trees. Chandru > On Tue, May 17, 2016 at 3:09 PM Chandru < > chandrashekhar.mullaparthi@REDACTED> wrote: > >> On 17 May 2016 at 00:49, Fred Hebert wrote: >> >>> On 05/16, Chandru wrote: >>> >>>> >>>> No, it's not. The reason a terminate callback is provided in a >>>> gen_server >>>> is so that a process can clean up when it terminates, not to delegate >>>> it to >>>> other processes. >>>> >>>> >>> I'm gonna side with Lo?c here. The terminate callback is good for any >>> process-local cleanup or optimistic work, but is by no means a safe way to >>> terminate anything. >> >> >>> >>> For example, if you have many children to terminate and through some >>> interleaving brutall_kill is triggered (or anyone calls exit(Pid, kill)), >>> whatever work you wanted to do in terminate will be skipped by a >>> non-trappable exit signal. >>> >> >> Agreed, but how does adding the connection handling process (in Oliver's >> use case) to a simple_one_for_one supervisor help? It doesn't give him >> anything other than the illusion of being "supervised". >> >> Chandru >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mkbucc@REDACTED Mon May 23 03:11:52 2016 From: mkbucc@REDACTED (Mark Bucciarelli) Date: Sun, 22 May 2016 21:11:52 -0400 Subject: [erlang-questions] erlfmt? In-Reply-To: References: <1461797208.3148043.591715937.1FAE1C3E@webmail.messagingengine.com> Message-ID: Thanks for the suggestions! I took a look at each, but none of the three options really gave me what I wanted. So I ended up wrapping erl_tidy in a way that gives me what I want: :%!erlfmt Here's the source, such as it is: https://github.com/mbucc/erlfmt. It was able to format 1,794 out of 1,795 source files under /usr/local/Cellar/erlang/18.2.1. It failed to format lib/erlang/lib/wx-1.6/src/gen/gl.erl, which is 971KB in size. Mark On Thu, Apr 28, 2016 at 3:09 AM, Roger Lipscombe wrote: > I use vimerl (https://github.com/jimenezrick/vimerl), which has an > escript included: > https://github.com/jimenezrick/vimerl/blob/master/indent/erlang_indent.erl > > I *believe* that it's equivalent to Emacs erlang-mode, but I don't > have Emacs installed :-P > > On 28 April 2016 at 07:24, Pierre Fenoll wrote: > > I made https://github.com/fenollp/erlang-formatter though it depends on > > OTP's Emacs erlang-mode. > > > > erlang-mode has been the de facto authority in terms of linting. > > However it has some issues, like when indenting typed record definitions. > > > > Dropping the dependency on Emacs would be great but again, which dev/CI > > machine doesn't have Emacs? > > > > On 28 Apr 2016, at 00:46, Tristan Sloughter wrote: > > > > Short answer: No. > > > > Long answer: No, but I'd love for someone with the time to clean up > > https://github.com/tsloughter/erl_tidy which is a rebar3 plugin named > `fmt` > > around erl_tidy. the erl_tidy code has issues with type specs and other > > newer syntax and will currently just dump out the AST for those. These > fixes > > would then, of course, need to be submitted upstream to OTP, but this > plugin > > provides a testing ground and hopefully motivation to someone out there > :) > > > > -- > > Tristan Sloughter > > t@REDACTED > > > > > > > > On Wed, Apr 27, 2016, at 05:37 PM, Mark Bucciarelli wrote: > > > > Hi, > > > > I'm wondering if anyone here uses or knows of a utility like gofmt for > > Erlang. If you're not familiar with gofmt, it simply reads source code > on > > stdin and writes formatted source code to stdout. > > > > I looked at erl_tidy but cannot see how to make it read stdin. > > > > Thanks, > > > > Mark > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Mon May 23 07:24:16 2016 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 23 May 2016 07:24:16 +0200 Subject: [erlang-questions] erlfmt? In-Reply-To: References: <1461797208.3148043.591715937.1FAE1C3E@webmail.messagingengine.com> Message-ID: <57429400.3060409@ericsson.com> Greetings, If you looked at https://github.com/ebengt/erlang_stdin_formatter I would be interested in suggestions as to where it was not sufficient. Bengt On 05/23/2016 03:11 AM, Mark Bucciarelli wrote: > Thanks for the suggestions! > > I took a look at each, but none of the three options really gave me > what I wanted. So I ended up wrapping erl_tidy in a way that gives me > what I want: > > :%!erlfmt > > Here's the source, such as it is: https://github.com/mbucc/erlfmt. > > It was able to format 1,794 out of 1,795 source files under > /usr/local/Cellar/erlang/18.2.1. It failed to format > lib/erlang/lib/wx-1.6/src/gen/gl.erl, which is 971KB in size. > > Mark > > On Thu, Apr 28, 2016 at 3:09 AM, Roger Lipscombe > > wrote: > > I use vimerl (https://github.com/jimenezrick/vimerl), which has an > escript included: > https://github.com/jimenezrick/vimerl/blob/master/indent/erlang_indent.erl > > I *believe* that it's equivalent to Emacs erlang-mode, but I don't > have Emacs installed :-P > > On 28 April 2016 at 07:24, Pierre Fenoll > wrote: > > I made https://github.com/fenollp/erlang-formatter though it > depends on > > OTP's Emacs erlang-mode. > > > > erlang-mode has been the de facto authority in terms of linting. > > However it has some issues, like when indenting typed record > definitions. > > > > Dropping the dependency on Emacs would be great but again, which > dev/CI > > machine doesn't have Emacs? > > > > On 28 Apr 2016, at 00:46, Tristan Sloughter > wrote: > > > > Short answer: No. > > > > Long answer: No, but I'd love for someone with the time to clean up > > https://github.com/tsloughter/erl_tidy which is a rebar3 plugin > named `fmt` > > around erl_tidy. the erl_tidy code has issues with type specs > and other > > newer syntax and will currently just dump out the AST for those. > These fixes > > would then, of course, need to be submitted upstream to OTP, but > this plugin > > provides a testing ground and hopefully motivation to someone > out there :) > > > > -- > > Tristan Sloughter > > t@REDACTED > > > > > > > > On Wed, Apr 27, 2016, at 05:37 PM, Mark Bucciarelli wrote: > > > > Hi, > > > > I'm wondering if anyone here uses or knows of a utility like > gofmt for > > Erlang. If you're not familiar with gofmt, it simply reads > source code on > > stdin and writes formatted source code to stdout. > > > > I looked at erl_tidy but cannot see how to make it read stdin. > > > > Thanks, > > > > Mark > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbike2000ru@REDACTED Mon May 23 12:10:54 2016 From: mbike2000ru@REDACTED (Dmitry) Date: Mon, 23 May 2016 10:10:54 +0000 (UTC) Subject: [erlang-questions] erl -name foo and hostname References: <1738749625.726843.1463998254145.JavaMail.yahoo.ref@mail.yahoo.com> Message-ID: <1738749625.726843.1463998254145.JavaMail.yahoo@mail.yahoo.com> Hi ?cat /etc/hosts127.0.0.1 ? ? ? ? ? ? ? localhost.localdomain localhost::1 ? ? ? ? ? ? localhost6.localdomain6 localhost6192.168.175.135 ? kazootest2.mydomain.local kazootest2 [root@REDACTED ~]# cat /etc/sysconfig/networkNETWORKING=yesNETWORKING_IPV6=noHOSTNAME=kazootest2.mydomain.local [root@REDACTED ~]# erl -name fooErlang R15B03 (erts-5.9.3.1) [source] [64-bit] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9.3.1 ?(abort with ^G)(foo@REDACTED)1> Why dous it shows kazootest2.localdomain? Any help appreciated -------------- next part -------------- An HTML attachment was scrubbed... URL: From list1@REDACTED Mon May 23 12:27:04 2016 From: list1@REDACTED (Grzegorz Junka) Date: Mon, 23 May 2016 10:27:04 +0000 Subject: [erlang-questions] erl -name foo and hostname In-Reply-To: <1738749625.726843.1463998254145.JavaMail.yahoo@mail.yahoo.com> References: <1738749625.726843.1463998254145.JavaMail.yahoo.ref@mail.yahoo.com> <1738749625.726843.1463998254145.JavaMail.yahoo@mail.yahoo.com> Message-ID: <1721a22e-7b8d-785c-376b-037bc6d053bf@gjunka.com> What does 'hostname' say? On 23/05/2016 10:10, Dmitry wrote: > Hi > > cat /etc/hosts > 127.0.0.1 localhost.localdomain localhost > ::1 localhost6.localdomain6 localhost6 > 192.168.175.135 kazootest2.mydomain.local kazootest2 > > [root@REDACTED ~]# cat /etc/sysconfig/network > NETWORKING=yes > NETWORKING_IPV6=no > HOSTNAME=kazootest2.mydomain.local > > [root@REDACTED ~]# erl -name foo > Erlang R15B03 (erts-5.9.3.1) [source] [64-bit] [async-threads:0] > [hipe] [kernel-poll:false] > > Eshell V5.9.3.1 (abort with ^G) > (foo@REDACTED)1> > > > Why dous it shows kazootest2.localdomain? > > Any help appreciated > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From hwinkel@REDACTED Mon May 23 17:44:10 2016 From: hwinkel@REDACTED (Holger Winkelmann) Date: Mon, 23 May 2016 17:44:10 +0200 (CEST) Subject: [erlang-questions] [ANN] erpc In-Reply-To: References: Message-ID: <1211912242.49380.1464018250091.JavaMail.zimbra@tpip.net> HI, > Yes, with different names. That way each application can have its own dedicated > connection(s) so that it doesn't suffer from sharing connections. For a single > named connection, you can also have multiple transport connections. Also, > connections are uni-directional so you have complete control over traffic > flows. I assume you mean the connection setup being uni-directional but the communication can be bi-directional? Cheers, Holger -------------- next part -------------- An HTML attachment was scrubbed... URL: From neerajsharma.live@REDACTED Tue May 24 17:15:14 2016 From: neerajsharma.live@REDACTED (Neeraj Sharma) Date: Tue, 24 May 2016 20:45:14 +0530 Subject: [erlang-questions] Erlang on RumpRun Unikernel Message-ID: Hi, I did an initial port of Erlang on the RumpRun unikernel ( https://github.com/rumpkernel/rumprun-packages/tree/master/erlang) in September last year. While the experience was enthralling, there were after thoughts which remained unanswered. I wonder what Erlang experts think regarding running Erlang on unikernel. RumpRun unikernel is an great project which (in my view) opened possibilities to design in some unique ways while shifting aware from the traditional operation system based design. There are some nuances like multi-threading instead of providing multi-process (no fork), but I Erlang does play nice (at least for the most part) with it. Needless to say the energy spent and my lack of understand on Erlang internals ensured that the project is suited for pet projects rather than production (or any serious use). My choice Erlang is a bit biased primarily it being my first experience to functional programming language and a long history of working in the telecommunications industry (though not using Erlang in production as much I'd wanted to). The language is awesome for many use cases though this email is primarily looking at its play with unikernels. The language blew me away with the ease and core language features for meeting complex requirements like (though not limited to) scalablity, availability and soft-real time behavior (not to mention the VMs capability to magically load the system resources evenly) which takes a lot of effort in implementing in traditional programming languages. My initial motivation to attempt the port was to look at role of Erlang (which pretty much does most of stuff a traditional OS+utils would provide to regular applications) in microservices architecture. In my opinion the choice of RumpRun unikernel makes a lot of sense in this respect rather than rely on traditional operating system architecture. Having said that, I would like to hear opinions from Erlang experts as to whether the marriage of Erlang with Unikernel has a bright future :) Thanks for your time, Neeraj -------------- next part -------------- An HTML attachment was scrubbed... URL: From list1@REDACTED Tue May 24 20:45:36 2016 From: list1@REDACTED (Grzegorz Junka) Date: Tue, 24 May 2016 18:45:36 +0000 Subject: [erlang-questions] Erlang on RumpRun Unikernel In-Reply-To: References: Message-ID: Sorry for my ignorance, how does it differ from http://erlangonxen.org/ (apart from its ability to run on bare metal without Xen)? Does rumprun support ARM processors/boards? IMHO both projects (Erlang on Xen / rumprun) suffer from a similar problem as IoT. They look great on paper as technologies, but it's hard to find an application in which they would be useful. They might be just waiting for that great idea which will allow them to break through and blow your mind. I could imagine an application which scales nearly linearly with the amount of nodes you allow it to use. When the capacity or speed needs to be increased, you just provision new nodes, each one a simple ARM module running Erlang, and you add it to the cluster. The application then uses the new node and distributes a bit of the running load to it. It's not impossible from a technical point of view but it's hard to imagine why a company would want to do that instead of moving the system to a new server with more CPU/RAM. In any case it's great that such an option exists and we can use it to try things. Anyway, thanks for bringing it up, I wouldn't have known Rumprun otherwise. Grzegorz On 24/05/2016 15:15, Neeraj Sharma wrote: > Hi, > > I did an initial port of Erlang on the RumpRun unikernel > (https://github.com/rumpkernel/rumprun-packages/tree/master/erlang) in > September last year. While the experience was enthralling, there were > after thoughts which remained unanswered. I wonder what Erlang experts > think regarding running Erlang on unikernel. RumpRun unikernel is an > great project which (in my view) opened possibilities to design in > some unique ways while shifting aware from the traditional operation > system based design. There are some nuances like multi-threading > instead of providing multi-process (no fork), but I Erlang does play > nice (at least for the most part) with it. Needless to say the energy > spent and my lack of understand on Erlang internals ensured that the > project is suited for pet projects rather than production (or any > serious use). > > My choice Erlang is a bit biased primarily it being my first > experience to functional programming language and a long history of > working in the telecommunications industry (though not using Erlang in > production as much I'd wanted to). The language is awesome for many > use cases though this email is primarily looking at its play with > unikernels. The language blew me away with the ease and core language > features for meeting complex requirements like (though not limited to) > scalablity, availability and soft-real time behavior (not to mention > the VMs capability to magically load the system resources evenly) > which takes a lot of effort in implementing in traditional programming > languages. > > My initial motivation to attempt the port was to look at role of > Erlang (which pretty much does most of stuff a traditional OS+utils > would provide to regular applications) in microservices architecture. > In my opinion the choice of RumpRun unikernel makes a lot of sense in > this respect rather than rely on traditional operating system > architecture. Having said that, I would like to hear opinions from > Erlang experts as to whether the marriage of Erlang with Unikernel has > a bright future :) > > > Thanks for your time, > Neeraj > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From radek@REDACTED Tue May 24 20:48:50 2016 From: radek@REDACTED (Radoslaw Gruchalski) Date: Tue, 24 May 2016 20:48:50 +0200 Subject: [erlang-questions] Erlang on RumpRun Unikernel In-Reply-To: References: Message-ID: And what would you do with the old server? Put in the box on a shelf? Horizontal scalability is a thing. ? Best regards, Radek Gruchalski radek@REDACTED de.linkedin.com/in/radgruchalski Confidentiality: This communication is intended for the above-named person and may be confidential and/or legally privileged. If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately. On May 24, 2016 at 8:45:44 PM, Grzegorz Junka (list1@REDACTED) wrote: Sorry for my ignorance, how does it differ from http://erlangonxen.org/ (apart from its ability to run on bare metal without Xen)? Does rumprun support ARM processors/boards? IMHO both projects (Erlang on Xen / rumprun) suffer from a similar problem as IoT. They look great on paper as technologies, but it's hard to find an application in which they would be useful. They might be just waiting for that great idea which will allow them to break through and blow your mind. I could imagine an application which scales nearly linearly with the amount of nodes you allow it to use. When the capacity or speed needs to be increased, you just provision new nodes, each one a simple ARM module running Erlang, and you add it to the cluster. The application then uses the new node and distributes a bit of the running load to it. It's not impossible from a technical point of view but it's hard to imagine why a company would want to do that instead of moving the system to a new server with more CPU/RAM. In any case it's great that such an option exists and we can use it to try things. Anyway, thanks for bringing it up, I wouldn't have known Rumprun otherwise. Grzegorz On 24/05/2016 15:15, Neeraj Sharma wrote: Hi, I did an initial port of Erlang on the RumpRun unikernel (https://github.com/rumpkernel/rumprun-packages/tree/master/erlang) in September last year. While the experience was enthralling, there were after thoughts which remained unanswered. I wonder what Erlang experts think regarding running Erlang on unikernel. RumpRun unikernel is an great project which (in my view) opened possibilities to design in some unique ways while shifting aware from the traditional operation system based design. There are some nuances like multi-threading instead of providing multi-process (no fork), but I Erlang does play nice (at least for the most part) with it. Needless to say the energy spent and my lack of understand on Erlang internals ensured that the project is suited for pet projects rather than production (or any serious use). My choice Erlang is a bit biased primarily it being my first experience to functional programming language and a long history of working in the telecommunications industry (though not using Erlang in production as much I'd wanted to). The language is awesome for many use cases though this email is primarily looking at its play with unikernels. The language blew me away with the ease and core language features for meeting complex requirements like (though not limited to) scalablity, availability and soft-real time behavior (not to mention the VMs capability to magically load the system resources evenly) which takes a lot of effort in implementing in traditional programming languages. My initial motivation to attempt the port was to look at role of Erlang (which pretty much does most of stuff a traditional OS+utils would provide to regular applications) in microservices architecture. In my opinion the choice of RumpRun unikernel makes a lot of sense in this respect rather than rely on traditional operating system architecture. Having said that, I would like to hear opinions from Erlang experts as to whether the marriage of Erlang with Unikernel has a bright future :) Thanks for your time, Neeraj _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mkbucc@REDACTED Tue May 24 21:35:54 2016 From: mkbucc@REDACTED (Mark Bucciarelli) Date: Tue, 24 May 2016 15:35:54 -0400 Subject: [erlang-questions] erlfmt? In-Reply-To: <57429400.3060409@ericsson.com> References: <1461797208.3148043.591715937.1FAE1C3E@webmail.messagingengine.com> <57429400.3060409@ericsson.com> Message-ID: Hi, I didn't look at that as I didn't find it before. I tried it, but only got comments out. Maybe I'm not using it right? ~/src/erlang_stdin_formatter/src$ cat erlfmt #!/usr/bin/env escript main([]) -> bepp:main([prog]). ~/src/erlang_stdin_formatter/src$ ./erlfmt < a_program.erl Thanks, Mark On Mon, May 23, 2016 at 1:24 AM, Bengt Kleberg wrote: > Greetings, > > If you looked at https://github.com/ebengt/erlang_stdin_formatter I would > be interested in suggestions as to where it was not sufficient. > > > Bengt > > > On 05/23/2016 03:11 AM, Mark Bucciarelli wrote: > > Thanks for the suggestions! > > I took a look at each, but none of the three options really gave me what I > wanted. So I ended up wrapping erl_tidy in a way that gives me what I want: > > :%!erlfmt > > Here's the source, such as it is: > https://github.com/mbucc/erlfmt. > > It was able to format 1,794 out of 1,795 source files under > /usr/local/Cellar/erlang/18.2.1. It failed to format > lib/erlang/lib/wx-1.6/src/gen/gl.erl, which is 971KB in size. > > Mark > > On Thu, Apr 28, 2016 at 3:09 AM, Roger Lipscombe > wrote: > >> I use vimerl (https://github.com/jimenezrick/vimerl), which has an >> escript included: >> https://github.com/jimenezrick/vimerl/blob/master/indent/erlang_indent.erl >> >> I *believe* that it's equivalent to Emacs erlang-mode, but I don't >> have Emacs installed :-P >> >> On 28 April 2016 at 07:24, Pierre Fenoll < >> pierrefenoll@REDACTED> wrote: >> > I made https://github.com/fenollp/erlang-formatter though it depends on >> > OTP's Emacs erlang-mode. >> > >> > erlang-mode has been the de facto authority in terms of linting. >> > However it has some issues, like when indenting typed record >> definitions. >> > >> > Dropping the dependency on Emacs would be great but again, which dev/CI >> > machine doesn't have Emacs? >> > >> > On 28 Apr 2016, at 00:46, Tristan Sloughter < >> t@REDACTED> wrote: >> > >> > Short answer: No. >> > >> > Long answer: No, but I'd love for someone with the time to clean up >> > https://github.com/tsloughter/erl_tidy which is a rebar3 plugin named >> `fmt` >> > around erl_tidy. the erl_tidy code has issues with type specs and other >> > newer syntax and will currently just dump out the AST for those. These >> fixes >> > would then, of course, need to be submitted upstream to OTP, but this >> plugin >> > provides a testing ground and hopefully motivation to someone out there >> :) >> > >> > -- >> > Tristan Sloughter >> > t@REDACTED >> > >> > >> > >> > On Wed, Apr 27, 2016, at 05:37 PM, Mark Bucciarelli wrote: >> > >> > Hi, >> > >> > I'm wondering if anyone here uses or knows of a utility like gofmt for >> > Erlang. If you're not familiar with gofmt, it simply reads source code >> on >> > stdin and writes formatted source code to stdout. >> > >> > I looked at erl_tidy but cannot see how to make it read stdin. >> > >> > Thanks, >> > >> > Mark >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From neerajsharma.live@REDACTED Tue May 24 22:02:04 2016 From: neerajsharma.live@REDACTED (Neeraj Sharma) Date: Wed, 25 May 2016 01:32:04 +0530 Subject: [erlang-questions] Erlang on RumpRun Unikernel In-Reply-To: References: Message-ID: On Wed, May 25, 2016 at 12:15 AM, Grzegorz Junka wrote: > > Sorry for my ignorance, how does it differ from http://erlangonxen.org/ (apart from its ability to run on bare metal without Xen)? Does rumprun support ARM processors/boards? There are a number of differences if you care about the internals, which may or many not interest all users. Having said that any comment I make here will have subjectivity and my bias towards RumpRun unikernel over the Ling (aka erlangonxen) project. I will give you my opinion and reasons for choosing RumpRun, but the official documentation for those projects can only do justice for any detailed comparison. Although dated I remember that Ling did a number of customization to get Erlang to work (one of them being playing around with the binary format beam). Additionally, the system software is to get only Erlang to work hence highly customized. The RumpRun unikernel on the other side is a generic library operating system which has a very strong focus towards POSIX compliance to allow existing code to run as much possible. Additionally, the drivers for RumpRun is from NetBSD which does infuse confidence. The Erlang port changes very little (for example uses the patch for running epmd in pure Erlang and such) to run it on RumpRun. I don't know how they compare in terms of performance, since I never benchmarked either so cannot comment on that. Some references for official documentation on the RumpRun and Rump Kernels project. * http://rumpkernel.org/ * https://github.com/rumpkernel/wiki/wiki * https://github.com/rumpkernel/rumprun * https://github.com/anttikantee/rumpkernel-book > > IMHO both projects (Erlang on Xen / rumprun) suffer from a similar problem as IoT. They look great on paper as technologies, but it's hard to find an application in which they would be useful. They might be just waiting for that great idea which will allow them to break through and blow your mind. > > I could imagine an application which scales nearly linearly with the amount of nodes you allow it to use. When the capacity or speed needs to be increased, you just provision new nodes, each one a simple ARM module running Erlang, and you add it to the cluster. The application then uses the new node and distributes a bit of the running load to it. > > It's not impossible from a technical point of view but it's hard to imagine why a company would want to do that instead of moving the system to a new server with more CPU/RAM. In any case it's great that such an option exists and we can use it to try things. Anyway, thanks for bringing it up, I wouldn't have known Rumprun otherwise. I agree that lack of such an showcase application does make things complicated and restrict adoption. -Neeraj From publicityifl@REDACTED Tue May 24 21:13:54 2016 From: publicityifl@REDACTED (publicityifl@REDACTED) Date: Tue, 24 May 2016 19:13:54 +0000 Subject: [erlang-questions] 1st CfP: IFL 2016 (28th Symposium on Implementation and Application of Functional Languages) Message-ID: <047d7b10d12bb1d93405339b5b50@google.com> Hello, Please, find below the first call for papers for IFL 2016. Please forward these to anyone you think may be interested. Apologies for any duplicates you may receive. best regards, Jurriaan Hage Publicity Chair of IFL --- IFL 2016 - Call for papers 28th SYMPOSIUM ON IMPLEMENTATION AND APPLICATION OF FUNCTIONAL LANGUAGES - IFL 2016 KU Leuven, Belgium In cooperation with ACM SIGPLAN August 31 - September 2, 2016 https://dtai.cs.kuleuven.be/events/ifl2016/ Scope The goal of the IFL symposia is to bring together researchers actively engaged in the implementation and application of functional and function-based programming languages. IFL 2016 will be a venue for researchers to present and discuss new ideas and concepts, work in progress, and publication-ripe results related to the implementation and application of functional languages and function-based programming. Peer-review Following the IFL tradition, IFL 2016 will use a post-symposium review process to produce the formal proceedings. All participants of IFL 2016 are invited to submit either a draft paper or an extended abstract describing work to be presented at the symposium. At no time may work submitted to IFL be simultaneously submitted to other venues; submissions must adhere to ACM SIGPLAN's republication policy: http://www.sigplan.org/Resources/Policies/Republication The submissions will be screened by the program committee chair to make sure they are within the scope of IFL, and will appear in the draft proceedings distributed at the symposium. Submissions appearing in the draft proceedings are not peer-reviewed publications. Hence, publications that appear only in the draft proceedings are not subject to the ACM SIGPLAN republication policy. After the symposium, authors will be given the opportunity to incorporate the feedback from discussions at the symposium and will be invited to submit a revised full article for the formal review process. From the revised submissions, the program committee will select papers for the formal proceedings considering their correctness, novelty, originality, relevance, significance, and clarity. The formal proceedings will appear in the International Conference Proceedings Series of the ACM Digital Library. Important dates August 1: Submission deadline draft papers August 3: Notification of acceptance for presentation August 5: Early registration deadline August 12: Late registration deadline August 22: Submission deadline for pre-symposium proceedings August 31 - September 2: IFL Symposium December 1: Submission deadline for post-symposium proceedings January 31, 2017: Notification of acceptance for post-symposium proceedings March 15, 2017: Camera-ready version for post-symposium proceedings Submission details Prospective authors are encouraged to submit papers or extended abstracts to be published in the draft proceedings and to present them at the symposium. All contributions must be written in English. Papers must use the new ACM two columns conference format, which can be found at: http://www.acm.org/publications/proceedings-template For the pre-symposium proceedings we adopt a 'weak' page limit of 12 pages. For the post-symposium proceedings the page limit of 12 pages is firm. Authors submit through EasyChair: https://easychair.org/conferences/?conf=ifl2016 Topics IFL welcomes submissions describing practical and theoretical work as well as submissions describing applications and tools in the context of functional programming. If you are not sure whether your work is appropriate for IFL 2016, please contact the PC chair at tom.schrijvers@REDACTED Topics of interest include, but are not limited to: - language concepts - type systems, type checking, type inferencing - compilation techniques - staged compilation - run-time function specialization - run-time code generation - partial evaluation - (abstract) interpretation - metaprogramming - generic programming - automatic program generation - array processing - concurrent/parallel programming - concurrent/parallel program execution - embedded systems - web applications - (embedded) domain specific languages - security - novel memory management techniques - run-time profiling performance measurements - debugging and tracing - virtual/abstract machine architectures - validation, verification of functional programs - tools and programming techniques - (industrial) applications Peter Landin Prize The Peter Landin Prize is awarded to the best paper presented at the symposium every year. The honored article is selected by the program committee based on the submissions received for the formal review process. The prize carries a cash award equivalent to 150 Euros. Programme committee Chair: Tom Schrijvers, KU Leuven, Belgium - Sandrine Blazy, University of Rennes 1, France - Laura Castro, University of A Coru??a, Spain - Jacques, Garrigue, Nagoya University, Japan - Clemens Grelck, University of Amsterdam, The Netherlands - Zoltan Horvath, Eotvos Lorand University, Hungary - Jan Martin Jansen, Netherlands Defence Academy, The Netherlands - Mauro Jaskelioff, CIFASIS/Universidad Nacional de Rosario, Argentina - Patricia Johann, Appalachian State University, USA - Wolfram Kahl, McMaster University, Canada - Pieter Koopman, Radboud University Nijmegen, The Netherlands - Shin-Cheng Mu, Academia Sinica, Taiwan - Henrik Nilsson, University of Nottingham, UK - Nikolaos Papaspyrou, National Technical University of Athens, Greece - Atze van der Ploeg, Chalmers University of Technology, Sweden - Matija Pretnar, University of Ljubljana, Slovenia - Tillmann Rendel, University of T??bingen, Germany - Christophe Scholliers, Universiteit Gent, Belgium - Sven-Bodo Scholz, Heriot-Watt University, UK - Melinda Toth, Eotvos Lorand University, Hungary - Meng Wang, University of Kent, UK - Jeremy Yallop, University of Cambridge, UK Venue The 28th IFL will be held in association with the Faculty of Computer Science, KU Leuven, Belgium. Leuven is centrally located in Belgium and can be easily reached from Brussels Airport by train (~15 minutes). The venue in the Arenberg Castle park can be reached by foot, bus or taxi from the city center. See the website for more information on the venue. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ericbmerritt@REDACTED Tue May 24 22:38:44 2016 From: ericbmerritt@REDACTED (Eric Merritt) Date: Tue, 24 May 2016 13:38:44 -0700 Subject: [erlang-questions] Erlang on RumpRun Unikernel In-Reply-To: References: Message-ID: Comparing this to erlangonxen is really comparing apples and oranges. Erlangonxen is not the erlang system as put out by Ericsson running on xen. Its a one-man reimplementation of the beam interpreter running on xen. Whether that is good or bad is an open question, but you can' t really compare it to what Neeraj is doing, which is the actual erlang system running as a microkernel. As to why a company would want it: Everything running on a server that isn't required by the service it is running is an opportunity for either failure or additional surface area to attack. If I can run, just my erlang service and nothing else but what it depends on directly and indirectly, that is a huge win for me. On Tue, May 24, 2016 at 11:45 AM, Grzegorz Junka wrote: > Sorry for my ignorance, how does it differ from http://erlangonxen.org/ > (apart from its ability to run on bare metal without Xen)? Does rumprun > support ARM processors/boards? > > IMHO both projects (Erlang on Xen / rumprun) suffer from a similar problem > as IoT. They look great on paper as technologies, but it's hard to find an > application in which they would be useful. They might be just waiting for > that great idea which will allow them to break through and blow your mind. > > I could imagine an application which scales nearly linearly with the amount > of nodes you allow it to use. When the capacity or speed needs to be > increased, you just provision new nodes, each one a simple ARM module > running Erlang, and you add it to the cluster. The application then uses the > new node and distributes a bit of the running load to it. > > It's not impossible from a technical point of view but it's hard to imagine > why a company would want to do that instead of moving the system to a new > server with more CPU/RAM. In any case it's great that such an option exists > and we can use it to try things. Anyway, thanks for bringing it up, I > wouldn't have known Rumprun otherwise. > > Grzegorz > > > On 24/05/2016 15:15, Neeraj Sharma wrote: > > Hi, > > I did an initial port of Erlang on the RumpRun unikernel > (https://github.com/rumpkernel/rumprun-packages/tree/master/erlang) in > September last year. While the experience was enthralling, there were after > thoughts which remained unanswered. I wonder what Erlang experts think > regarding running Erlang on unikernel. RumpRun unikernel is an great project > which (in my view) opened possibilities to design in some unique ways while > shifting aware from the traditional operation system based design. There are > some nuances like multi-threading instead of providing multi-process (no > fork), but I Erlang does play nice (at least for the most part) with it. > Needless to say the energy spent and my lack of understand on Erlang > internals ensured that the project is suited for pet projects rather than > production (or any serious use). > > My choice Erlang is a bit biased primarily it being my first experience to > functional programming language and a long history of working in the > telecommunications industry (though not using Erlang in production as much > I'd wanted to). The language is awesome for many use cases though this email > is primarily looking at its play with unikernels. The language blew me away > with the ease and core language features for meeting complex requirements > like (though not limited to) scalablity, availability and soft-real time > behavior (not to mention the VMs capability to magically load the system > resources evenly) which takes a lot of effort in implementing in traditional > programming languages. > > My initial motivation to attempt the port was to look at role of Erlang > (which pretty much does most of stuff a traditional OS+utils would provide > to regular applications) in microservices architecture. In my opinion the > choice of RumpRun unikernel makes a lot of sense in this respect rather than > rely on traditional operating system architecture. Having said that, I would > like to hear opinions from Erlang experts as to whether the marriage of > Erlang with Unikernel has a bright future :) > > > Thanks for your time, > Neeraj > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From chandrashekhar.mullaparthi@REDACTED Tue May 24 23:44:29 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Tue, 24 May 2016 22:44:29 +0100 Subject: [erlang-questions] [ANN] erpc In-Reply-To: <1211912242.49380.1464018250091.JavaMail.zimbra@tpip.net> References: <1211912242.49380.1464018250091.JavaMail.zimbra@tpip.net> Message-ID: Hi Holger, On 23 May 2016 at 16:44, Holger Winkelmann wrote: > HI, > > Yes, with different names. That way each application can have its own > dedicated connection(s) so that it doesn't suffer from sharing connections. > For a single named connection, you can also have multiple transport > connections. Also, connections are uni-directional so you have complete > control over traffic flows. > > I assume you mean the connection setup being uni-directional but the > communication can be bi-directional? > No, communication is uni-directional as well. Only the client side can send requests to the server side. If you want a bi-directional connection between two nodes A and B, you will have to configure a client connection on A to B, and vice versa. This is roughly the third time I'm trying to implement an efficient RPC mechanism for Erlang and I feel this is the most optimal way to design it. I'm still thinking of supporting bi-directional connections, but I haven't worked out the best way to do it - it may never happen. cheers, Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From lheadley@REDACTED Tue May 24 23:52:04 2016 From: lheadley@REDACTED (Lyn Headley) Date: Tue, 24 May 2016 14:52:04 -0700 Subject: [erlang-questions] Erlang on RumpRun Unikernel In-Reply-To: References: Message-ID: On Tue, May 24, 2016 at 11:45 AM, Grzegorz Junka wrote: > IMHO both projects (Erlang on Xen / rumprun) suffer from a similar problem > as IoT. They look great on paper as technologies, but it's hard to find an > application in which they would be useful. They might be just waiting for > that great idea which will allow them to break through and blow your mind. > erlang on xen has what I think is a great application: super-elastic clouds. Imagine a web site with no server. When it gets a request, it launches the server, and serves the request. Think how cheap this would be. From list1@REDACTED Tue May 24 23:54:33 2016 From: list1@REDACTED (Grzegorz Junka) Date: Tue, 24 May 2016 21:54:33 +0000 Subject: [erlang-questions] Erlang on RumpRun Unikernel In-Reply-To: References: Message-ID: <04ca3e23-53ce-9246-3ee1-50ef71e9d17f@gjunka.com> On 24/05/2016 21:52, Lyn Headley wrote: > On Tue, May 24, 2016 at 11:45 AM, Grzegorz Junka wrote: > >> IMHO both projects (Erlang on Xen / rumprun) suffer from a similar problem >> as IoT. They look great on paper as technologies, but it's hard to find an >> application in which they would be useful. They might be just waiting for >> that great idea which will allow them to break through and blow your mind. >> > erlang on xen has what I think is a great application: super-elastic > clouds. Imagine a web site with no server. When it gets a request, it > launches the server, and serves the request. Think how cheap this > would be. OK, nice idea, but where would be the web server running then? From radek@REDACTED Tue May 24 23:55:46 2016 From: radek@REDACTED (Radoslaw Gruchalski) Date: Tue, 24 May 2016 23:55:46 +0200 Subject: [erlang-questions] Erlang on RumpRun Unikernel In-Reply-To: References: Message-ID: People do that today, to a certain extent, with docker / containers. ? Best regards, Radek Gruchalski radek@REDACTED de.linkedin.com/in/radgruchalski Confidentiality: This communication is intended for the above-named person and may be confidential and/or legally privileged. If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately. On May 24, 2016 at 11:52:10 PM, Lyn Headley (lheadley@REDACTED) wrote: On Tue, May 24, 2016 at 11:45 AM, Grzegorz Junka wrote: > IMHO both projects (Erlang on Xen / rumprun) suffer from a similar problem > as IoT. They look great on paper as technologies, but it's hard to find an > application in which they would be useful. They might be just waiting for > that great idea which will allow them to break through and blow your mind. > erlang on xen has what I think is a great application: super-elastic clouds. Imagine a web site with no server. When it gets a request, it launches the server, and serves the request. Think how cheap this would be. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed May 25 06:59:21 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 25 May 2016 16:59:21 +1200 Subject: [erlang-questions] Erlang on RumpRun Unikernel In-Reply-To: References: Message-ID: <7fef00ce-5765-8b95-e7da-3b468d1881cb@cs.otago.ac.nz> On 25/05/16 8:38 AM, Eric Merritt wrote: > As to why a company would want it: Everything running on a server that > isn't required by the service it is running is an opportunity for > either failure or additional surface area to attack. If I can run, > just my erlang service and nothing else but what it depends on > directly and indirectly, that is a huge win for me. Also everything running on a server that isn't required is limiting the amount of memory available for useful work (paging is not your friend) and wasting electric power. From neerajsharma.live@REDACTED Wed May 25 07:11:57 2016 From: neerajsharma.live@REDACTED (Neeraj Sharma) Date: Wed, 25 May 2016 10:41:57 +0530 Subject: [erlang-questions] Can Erlang application load from within beam without external storage? Message-ID: Hi, I have been toying with the idea of packaging all the dependencies for running an Erlang application inside a custom binary which includes beam and application with its dependecies. Is this possible? The idea is to roll out Erlang + Rumprun image without need of an external storage to load the application or its dependencies. At present there is an image of Rumprun + beam and an additional iso generated out of the stock build with Erlang installation (though much more than what is required but something done at the time to get something working) and application beam files. Thanks, Neeraj From sergej.jurecko@REDACTED Wed May 25 07:15:15 2016 From: sergej.jurecko@REDACTED (=?utf-8?Q?Sergej_Jure=C4=8Dko?=) Date: Wed, 25 May 2016 07:15:15 +0200 Subject: [erlang-questions] Can Erlang application load from within beam without external storage? In-Reply-To: References: Message-ID: <0148DE6A-AD78-4183-B553-CBDDD9287545@gmail.com> This? http://erlang.org/doc/man/code.html#id104826 Sergej > On 25 May 2016, at 07:11, Neeraj Sharma wrote: > > Hi, > > I have been toying with the idea of packaging all the dependencies for > running an Erlang application inside a custom binary which includes > beam and application with its dependecies. Is this possible? > > The idea is to roll out Erlang + Rumprun image without need of an > external storage to load the application or its dependencies. At > present there is an image of Rumprun + beam and an additional iso > generated out of the stock build with Erlang installation (though much > more than what is required but something done at the time to get > something working) and application beam files. > > > Thanks, > Neeraj > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From neerajsharma.live@REDACTED Wed May 25 09:31:49 2016 From: neerajsharma.live@REDACTED (Neeraj Sharma) Date: Wed, 25 May 2016 13:01:49 +0530 Subject: [erlang-questions] Can Erlang application load from within beam without external storage? In-Reply-To: <0148DE6A-AD78-4183-B553-CBDDD9287545@gmail.com> References: <0148DE6A-AD78-4183-B553-CBDDD9287545@gmail.com> Message-ID: On Wed, May 25, 2016 at 10:45 AM, Sergej Jure?ko wrote: > This? > http://erlang.org/doc/man/code.html#id104826 > Interesting piece of information although I am afraid my requirement was not clear enough, so let me provide some additional details. My intention is to package Erlang VM, application and its dependencies (including OTP) into a single blob of binary which can run standalone. I am at very early stage to evaluate the possibilities here so things may appear vague at this point. Having said that ideally I'd like to retain the Erlang shell and given the nature of the build code reload is not required. I believe there are some projects like erllvm and Hipe but I dont have clear understanding of the same and wanted to know existence of any prior work which can be reused. Actually the single binary will also include the Rumprun unikernel, which at present is limited to Rumprun and Erlang beam VM, while the rest of the application (and dependencies) are stored on secondary media. An alternative path would be to evaluate the feasibility to bundle all the apps and dependencies as another blob (probably as zip as given in the link at the top) and a wrapper c code unpacks that into ram disk recreating the folder structure required by the Erlang VM. Ideally I would like to dwell on the first option before evaluating this one. Thanks, -Neeraj From list1@REDACTED Wed May 25 11:05:09 2016 From: list1@REDACTED (Grzegorz Junka) Date: Wed, 25 May 2016 09:05:09 +0000 Subject: [erlang-questions] Can Erlang application load from within beam without external storage? In-Reply-To: References: <0148DE6A-AD78-4183-B553-CBDDD9287545@gmail.com> Message-ID: <66b2e53d-2b9d-2d89-8e94-7592791dd972@gjunka.com> This is what Erlang releases are for! Albeit they don't produce a single binary, they are pretty straightforward to be moved around as a single file. Have a look at this example: mkdir test_sc cd test_sc/ git clone https://github.com/builderl/ex1_simple_cache.git cd ex1_simple_cache/ gmake get-deps gmake tgz This produces a single tgz file in tmp/ folder which contains the Erlang VM, erts and all required applications (and nothing more). It can be moved to any place since all paths used in it are relative. Once moved to the desired host/location you unpack, install and run it: mkdir myrelease cd myrelease tar -xzf ~/path/to/the/release.tgz ./bin/init.esh ./bin/start.esh to_erl ../sc-1/shell/ It's build with a normal makefile so bundling Rumprun unikernel to it should be a piece of cake. It's possible to create a single binary with all Erlang applications using the technique called escriptize http://blog.differentpla.net/blog/2015/10/21/rebar-escriptize but for it to work the Erlang runtime has to be already installed on the destination system. Grzegorz On 25/05/2016 07:31, Neeraj Sharma wrote: > On Wed, May 25, 2016 at 10:45 AM, Sergej Jure?ko > wrote: >> This? >> http://erlang.org/doc/man/code.html#id104826 >> > Interesting piece of information although I am afraid my requirement > was not clear enough, so let me provide some additional details. > > My intention is to package Erlang VM, application and its dependencies > (including OTP) into a single blob of binary which can run standalone. > I am at very early stage to evaluate the possibilities here so things > may appear vague at this point. Having said that ideally I'd like to > retain the Erlang shell and given the nature of the build code reload > is not required. I believe there are some projects like erllvm and > Hipe but I dont have clear understanding of the same and wanted to > know existence of any prior work which can be reused. Actually the > single binary will also include the Rumprun unikernel, which at > present is limited to Rumprun and Erlang beam VM, while the rest of > the application (and dependencies) are stored on secondary media. > > An alternative path would be to evaluate the feasibility to bundle all > the apps and dependencies as another blob (probably as zip as given in > the link at the top) and a wrapper c code unpacks that into ram disk > recreating the folder structure required by the Erlang VM. Ideally I > would like to dwell on the first option before evaluating this one. > > Thanks, > -Neeraj > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From neerajsharma.live@REDACTED Wed May 25 13:27:33 2016 From: neerajsharma.live@REDACTED (Neeraj Sharma) Date: Wed, 25 May 2016 16:57:33 +0530 Subject: [erlang-questions] Can Erlang application load from within beam without external storage? In-Reply-To: <66b2e53d-2b9d-2d89-8e94-7592791dd972@gjunka.com> References: <0148DE6A-AD78-4183-B553-CBDDD9287545@gmail.com> <66b2e53d-2b9d-2d89-8e94-7592791dd972@gjunka.com> Message-ID: On Wed, May 25, 2016 at 2:35 PM, Grzegorz Junka wrote: > This is what Erlang releases are for! Albeit they don't produce a single > binary, they are pretty straightforward to be moved around as a single file. > ... > > This produces a single tgz file in tmp/ folder which contains the Erlang VM, > erts and all required applications (and nothing more). It can be moved to > any place since all paths used in it are relative. Once moved to the desired > host/location you unpack, install and run it: > > > It's build with a normal makefile so bundling Rumprun unikernel to it should > be a piece of cake. > > It's possible to create a single binary with all Erlang applications using > the technique called escriptize > http://blog.differentpla.net/blog/2015/10/21/rebar-escriptize but for it to > work the Erlang runtime has to be already installed on the destination > system. > This would be useful when taking the second approach (I mentioned earlier) but it cannot be used to build standalone binary (the first option) and rolled-into-one bootable image with Rumprun unikernel. The idea is to run the resulting image directly on hypervisor (say Xen or KVM) without any storage attached, so the image is self sufficient with Rumprun unikernel, Erlang VM and the application. At present the application is read out of storage (specifically an ISO). Thanks, -Neeraj From roberto@REDACTED Wed May 25 14:00:57 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 25 May 2016 14:00:57 +0200 Subject: [erlang-questions] Directories in project root being created Message-ID: Dear list, When I run a release two directories get created for me: data.myapp@REDACTED log.myapp@REDACTED These directories are empty. Mnesia runs in memory only, and I've also set the directory to be temp: -mnesia schema_location ram -mnesia dir '"/tmp/mnesia"' So.. who is creating these directories? BTW they are empty. Thank you, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From k.petrauskas@REDACTED Wed May 25 14:12:56 2016 From: k.petrauskas@REDACTED (Karolis Petrauskas) Date: Wed, 25 May 2016 15:12:56 +0300 Subject: [erlang-questions] Directories in project root being created In-Reply-To: References: Message-ID: In my case, it was git://github.com/uwiger/setup.git. I solved this inconvenience by adding the following to the sys.config. {setup, [ {data_dir, "data"}, {log_dir, "log"} ]}, Karolis On Wed, May 25, 2016 at 3:00 PM, Roberto Ostinelli wrote: > Dear list, > When I run a release two directories get created for me: > > data.myapp@REDACTED > log.myapp@REDACTED > > These directories are empty. > > Mnesia runs in memory only, and I've also set the directory to be temp: > > -mnesia schema_location ram > -mnesia dir '"/tmp/mnesia"' > > So.. who is creating these directories? BTW they are empty. > > Thank you, > r. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From roberto@REDACTED Wed May 25 14:17:30 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Wed, 25 May 2016 14:17:30 +0200 Subject: [erlang-questions] Directories in project root being created In-Reply-To: References: Message-ID: ...and we have a winner. Thank you :) Do you happen to know what are these for? Can I just dump them in /tmp? -------------- next part -------------- An HTML attachment was scrubbed... URL: From k.petrauskas@REDACTED Wed May 25 14:45:45 2016 From: k.petrauskas@REDACTED (Karolis Petrauskas) Date: Wed, 25 May 2016 15:45:45 +0300 Subject: [erlang-questions] Directories in project root being created In-Reply-To: References: Message-ID: No, I don't. In my case they were empty all the time, so I just pointed them to existing folders in my release. Karolis On Wed, May 25, 2016 at 3:17 PM, Roberto Ostinelli wrote: > ...and we have a winner. Thank you :) > > Do you happen to know what are these for? Can I just dump them in /tmp? From mbike2000ru@REDACTED Wed May 25 15:53:23 2016 From: mbike2000ru@REDACTED (Dmitry) Date: Wed, 25 May 2016 13:53:23 +0000 (UTC) Subject: [erlang-questions] erl -name foo and hostname In-Reply-To: <1738749625.726843.1463998254145.JavaMail.yahoo@mail.yahoo.com> References: <1738749625.726843.1463998254145.JavaMail.yahoo.ref@mail.yahoo.com> <1738749625.726843.1463998254145.JavaMail.yahoo@mail.yahoo.com> Message-ID: <917605456.363863.1464184403367.JavaMail.yahoo@mail.yahoo.com> Hi here is my data: [root@REDACTED ~]# hostnamekazootest2.mydomain.local[root@REDACTED ~]# hostname -fkazootest2.mydomain.local[root@REDACTED ~]# erl -name fooErlang R15B03 (erts-5.9.3.1) [source] [64-bit] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9.3.1 ?(abort with ^G)(foo@REDACTED)1> On Monday, May 23, 2016 3:10 PM, Dmitry wrote: Hi ?cat /etc/hosts127.0.0.1 ? ? ? ? ? ? ? localhost.localdomain localhost::1 ? ? ? ? ? ? localhost6.localdomain6 localhost6192.168.175.135 ? kazootest2.mydomain.local kazootest2 [root@REDACTED ~]# cat /etc/sysconfig/networkNETWORKING=yesNETWORKING_IPV6=noHOSTNAME=kazootest2.mydomain.local [root@REDACTED ~]# erl -name fooErlang R15B03 (erts-5.9.3.1) [source] [64-bit] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9.3.1 ?(abort with ^G)(foo@REDACTED)1> Why dous it shows kazootest2.localdomain? Any help appreciated -------------- next part -------------- An HTML attachment was scrubbed... URL: From boris.muehmer@REDACTED Wed May 25 17:11:37 2016 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Wed, 25 May 2016 17:11:37 +0200 Subject: [erlang-questions] erl -name foo and hostname In-Reply-To: <917605456.363863.1464184403367.JavaMail.yahoo@mail.yahoo.com> References: <1738749625.726843.1463998254145.JavaMail.yahoo.ref@mail.yahoo.com> <1738749625.726843.1463998254145.JavaMail.yahoo@mail.yahoo.com> <917605456.363863.1464184403367.JavaMail.yahoo@mail.yahoo.com> Message-ID: Do You happen to have the "avahi" daemon running? Regards, Boris 2016-05-25 15:53 GMT+02:00 Dmitry : > Hi > > here is my data: > > [root@REDACTED ~]# hostname > kazootest2.mydomain.local > [root@REDACTED ~]# hostname -f > kazootest2.mydomain.local > [root@REDACTED ~]# erl -name foo > Erlang R15B03 (erts-5.9.3.1) [source] [64-bit] [async-threads:0] [hipe] > [kernel-poll:false] > > Eshell V5.9.3.1 (abort with ^G) > (foo@REDACTED)1> > > > > > On Monday, May 23, 2016 3:10 PM, Dmitry wrote: > > > Hi > > cat /etc/hosts > 127.0.0.1 localhost.localdomain localhost > ::1 localhost6.localdomain6 localhost6 > 192.168.175.135 kazootest2.mydomain.local kazootest2 > > [root@REDACTED ~]# cat /etc/sysconfig/network > NETWORKING=yes > NETWORKING_IPV6=no > HOSTNAME=kazootest2.mydomain.local > > [root@REDACTED ~]# erl -name foo > Erlang R15B03 (erts-5.9.3.1) [source] [64-bit] [async-threads:0] [hipe] > [kernel-poll:false] > > Eshell V5.9.3.1 (abort with ^G) > (foo@REDACTED)1> > > > Why dous it shows kazootest2.localdomain? > > Any help appreciated > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbike2000ru@REDACTED Wed May 25 17:30:37 2016 From: mbike2000ru@REDACTED (Dmitry) Date: Wed, 25 May 2016 15:30:37 +0000 (UTC) Subject: [erlang-questions] erl -name foo and hostname In-Reply-To: References: <1738749625.726843.1463998254145.JavaMail.yahoo.ref@mail.yahoo.com> <1738749625.726843.1463998254145.JavaMail.yahoo@mail.yahoo.com> <917605456.363863.1464184403367.JavaMail.yahoo@mail.yahoo.com> Message-ID: <1139543457.421170.1464190237184.JavaMail.yahoo@mail.yahoo.com> ps aux | grep avaroot ? ? 16011 ?0.0 ?0.0 103308 ? 852 pts/0 ? ?S+ ? 09:20 ? 0:00 grep ava[root@REDACTED ~]# no The AVAHI daemon is not running. On Wednesday, May 25, 2016 8:11 PM, Boris M?hmer wrote: Do You happen to have the "avahi" daemon running? Regards,Boris 2016-05-25 15:53 GMT+02:00 Dmitry : Hi here is my data: [root@REDACTED ~]# hostnamekazootest2.mydomain.local[root@REDACTED ~]# hostname -fkazootest2.mydomain.local[root@REDACTED ~]# erl -name fooErlang R15B03 (erts-5.9.3.1) [source] [64-bit] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9.3.1 ?(abort with ^G)(foo@REDACTED)1> On Monday, May 23, 2016 3:10 PM, Dmitry wrote: Hi ?cat /etc/hosts127.0.0.1 ? ? ? ? ? ? ? localhost.localdomain localhost::1 ? ? ? ? ? ? localhost6.localdomain6 localhost6192.168.175.135 ? kazootest2.mydomain.local kazootest2 [root@REDACTED ~]# cat /etc/sysconfig/networkNETWORKING=yesNETWORKING_IPV6=noHOSTNAME=kazootest2.mydomain.local [root@REDACTED ~]# erl -name fooErlang R15B03 (erts-5.9.3.1) [source] [64-bit] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9.3.1 ?(abort with ^G)(foo@REDACTED)1> Why dous it shows kazootest2.localdomain? Any help appreciated _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From boris.muehmer@REDACTED Wed May 25 17:51:36 2016 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Wed, 25 May 2016 17:51:36 +0200 Subject: [erlang-questions] erl -name foo and hostname In-Reply-To: <1139543457.421170.1464190237184.JavaMail.yahoo@mail.yahoo.com> References: <1738749625.726843.1463998254145.JavaMail.yahoo.ref@mail.yahoo.com> <1738749625.726843.1463998254145.JavaMail.yahoo@mail.yahoo.com> <917605456.363863.1464184403367.JavaMail.yahoo@mail.yahoo.com> <1139543457.421170.1464190237184.JavaMail.yahoo@mail.yahoo.com> Message-ID: Have You tried to ping "kazootest2.localdomain"? It would be interesting which IP shows up, if any at all. Maybe this gives a hint. Did You also have a look at "/etc/networks"? Regards, Boris 2016-05-25 17:30 GMT+02:00 Dmitry : > ps aux | grep ava > root 16011 0.0 0.0 103308 852 pts/0 S+ 09:20 0:00 grep ava > [root@REDACTED ~]# > > no The AVAHI daemon is not running. > > > On Wednesday, May 25, 2016 8:11 PM, Boris M?hmer > wrote: > > > Do You happen to have the "avahi" daemon running? > > > Regards, > Boris > > 2016-05-25 15:53 GMT+02:00 Dmitry : > > Hi > > here is my data: > > [root@REDACTED ~]# hostname > kazootest2.mydomain.local > [root@REDACTED ~]# hostname -f > kazootest2.mydomain.local > [root@REDACTED ~]# erl -name foo > Erlang R15B03 (erts-5.9.3.1) [source] [64-bit] [async-threads:0] [hipe] > [kernel-poll:false] > > Eshell V5.9.3.1 (abort with ^G) > (foo@REDACTED)1> > > > > > On Monday, May 23, 2016 3:10 PM, Dmitry wrote: > > > Hi > > cat /etc/hosts > 127.0.0.1 localhost.localdomain localhost > ::1 localhost6.localdomain6 localhost6 > 192.168.175.135 kazootest2.mydomain.local kazootest2 > > [root@REDACTED ~]# cat /etc/sysconfig/network > NETWORKING=yes > NETWORKING_IPV6=no > HOSTNAME=kazootest2.mydomain.local > > [root@REDACTED ~]# erl -name foo > Erlang R15B03 (erts-5.9.3.1) [source] [64-bit] [async-threads:0] [hipe] > [kernel-poll:false] > > Eshell V5.9.3.1 (abort with ^G) > (foo@REDACTED)1> > > > Why dous it shows kazootest2.localdomain? > > Any help appreciated > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbike2000ru@REDACTED Wed May 25 18:13:23 2016 From: mbike2000ru@REDACTED (Dmitry) Date: Wed, 25 May 2016 16:13:23 +0000 (UTC) Subject: [erlang-questions] erl -name foo and hostname In-Reply-To: References: <1738749625.726843.1463998254145.JavaMail.yahoo.ref@mail.yahoo.com> <1738749625.726843.1463998254145.JavaMail.yahoo@mail.yahoo.com> <917605456.363863.1464184403367.JavaMail.yahoo@mail.yahoo.com> <1139543457.421170.1464190237184.JavaMail.yahoo@mail.yahoo.com> Message-ID: <1576180469.438760.1464192803540.JavaMail.yahoo@mail.yahoo.com> /etc/init.d/network restartShutting down loopback interface: ? ? ? ? ? ? ? ? ? ? ? ? ?[ ?OK ?]Bringing up loopback interface: ? ? ? ? ? ? ? ? ? ? ? ? ? ?[ ?OK ?] [root@REDACTED ~]# ping kazootest2.localdomainping: unknown host kazootest2.localdomain [root@REDACTED ~]# cat /etc/networksdefault 0.0.0.0loopback 127.0.0.0link-local 169.254.0.0 On Wednesday, May 25, 2016 8:51 PM, Boris M?hmer wrote: Have You tried to ping "kazootest2.localdomain"??It would be interesting which IP shows up, if any at all. Maybe this gives a hint.Did You also have a look at "/etc/networks"?? Regards,Boris 2016-05-25 17:30 GMT+02:00 Dmitry : ps aux | grep avaroot ? ? 16011 ?0.0 ?0.0 103308 ? 852 pts/0 ? ?S+ ? 09:20 ? 0:00 grep ava[root@REDACTED ~]# no The AVAHI daemon is not running. On Wednesday, May 25, 2016 8:11 PM, Boris M?hmer wrote: Do You happen to have the "avahi" daemon running? Regards,Boris 2016-05-25 15:53 GMT+02:00 Dmitry : Hi here is my data: [root@REDACTED ~]# hostnamekazootest2.mydomain.local[root@REDACTED ~]# hostname -fkazootest2.mydomain.local[root@REDACTED ~]# erl -name fooErlang R15B03 (erts-5.9.3.1) [source] [64-bit] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9.3.1 ?(abort with ^G)(foo@REDACTED)1> On Monday, May 23, 2016 3:10 PM, Dmitry wrote: Hi ?cat /etc/hosts127.0.0.1 ? ? ? ? ? ? ? localhost.localdomain localhost::1 ? ? ? ? ? ? localhost6.localdomain6 localhost6192.168.175.135 ? kazootest2.mydomain.local kazootest2 [root@REDACTED ~]# cat /etc/sysconfig/networkNETWORKING=yesNETWORKING_IPV6=noHOSTNAME=kazootest2.mydomain.local [root@REDACTED ~]# erl -name fooErlang R15B03 (erts-5.9.3.1) [source] [64-bit] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9.3.1 ?(abort with ^G)(foo@REDACTED)1> Why dous it shows kazootest2.localdomain? Any help appreciated _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.ashmun@REDACTED Wed May 25 21:43:44 2016 From: john.ashmun@REDACTED (John Ashmun) Date: Wed, 25 May 2016 12:43:44 -0700 Subject: [erlang-questions] Can werl be detached? In-Reply-To: References: <6284265.hgQ7dGlg6e@burrito> Message-ID: I just found time to try what you suggested with the wx demo, but I got the same pop up Windows error message as when I try to run my release with -detached. I added ...\erts-5.9\bin to %Path% and tried again but it failed again with the same error message. Sent from my iPhone > On May 18, 2016, at 12:02 AM, Dan Gudmundsson wrote: > > At least this works as it should: > > werl -detached -run wx demo > > >> On Wed, May 18, 2016 at 3:48 AM John R. Ashmun wrote: >> >> >>> On Tue, May 17, 2016 at 6:32 PM, zxq9 wrote: >>> On Tuesday 17 May 2016 16:47:35 John R. Ashmun wrote: >>> > Perhaps my subject is not quite the correct question. Here's my context: >>> > >>> > I'm beginning to recreate in Erlang on Microsoft Windows 10 a public domain >>> > program I used to enjoy on my Commodore Amiga: mATC was a game in which >>> > the user acted as a military Air Traffic Controller. I am using a wx >>> > window to draw the player's map with aircraft data blocks overlaid. I >>> > don't have a use for the initial window that opens when werl is started. >>> >>> Awesome! >>> >>> > If I use this Command Prompt batch file, mATC.bat: >>> > >>> > werl -pa ebin -pa mATC_app\ebin -detached -boot mATC -config sys >>> > >>> > a Windows error message window appears that says: >>> > >>> > "Failed to execute C:\Program Files\erl5.9\erts5.9\bin\beam.smp.dll >>> > The system cannot find the file specified." >>> > >>> > The DLL is actually present at that location, and of course everything runs >>> > well when I don't use -detached. >>> >>> Hrm... that seems odd, but I have to admit I have no experience running >>> actual releases on Windows. What I usually do instead is have the runtime >>> installed on the Windows machine, unpack project code and kick things off >>> with an escript that builds and then launches the code I want to run. >>> >>> Changing from releases to from-source build->run may require a bit of >>> shuffling -- and may or may not be worth it depending on the project. >>> But this has been very effective and lightweight for my purposes. >>> >>> Like everything else on Windows there are a few quirks to making escripts >>> a click-to-run experience (specifying full path to "escript.exe" instead >>> of "escript" in your launcher, space for unheeded shebang, etc.). I've >>> wanted for a few years now to formalize and simplify a solution to this... >>> but real life gets in the way of the things I wish I could do as community >>> project work. :-( >>> >>> The escript-to-bootstrap approach has made my Erlang GUI experience on >>> Windows fairly painless -- I can develop on Linux or BSD and almost always >>> get away with relying on anything that works there (to include wx) working >>> pretty much the same way on Windows. >>> >>> I have not tried this on Windows 10 yet, but I assume it will continue to >>> work the same way it does on Windows 7 and 8.1. >>> >>> -Craig >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> I will look into escript, thanks. >> >> John >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Wed May 25 22:22:16 2016 From: dangud@REDACTED (Dan Gudmundsson) Date: Wed, 25 May 2016 20:22:16 +0000 Subject: [erlang-questions] Can werl be detached? In-Reply-To: References: <6284265.hgQ7dGlg6e@burrito> Message-ID: Hmm, It works fine for me on Windows 10 from msys2 with erlang in the path and from cmd shell without erlang in the path: C:\Users\familjen>"c:\Program Files (x86)\erl7.3\bin\werl" -run wx demo C:\Users\familjen>"c:\Program Files (x86)\erl7.3\bin\werl" -detached -run wx demo Oh noticed the version you are running... It also works for me with 17.0 i.e. erl-6.0 which is the oldest I have installed here at home. Why are you using an old version of erlang? Can you try a newer version and see if that works? On Wed, May 25, 2016 at 9:49 PM John Ashmun wrote: > I just found time to try what you suggested with the wx demo, but I got > the same pop up Windows error message as when I try to run my release with > -detached. > > I added ...\erts-5.9\bin to %Path% and tried again but it failed again > with the same error message. > > Sent from my iPhone > > On May 18, 2016, at 12:02 AM, Dan Gudmundsson wrote: > > At least this works as it should: > > werl -detached -run wx demo > > > On Wed, May 18, 2016 at 3:48 AM John R. Ashmun > wrote: > >> >> >> On Tue, May 17, 2016 at 6:32 PM, zxq9 wrote: >> >>> On Tuesday 17 May 2016 16:47:35 John R. Ashmun wrote: >>> > Perhaps my subject is not quite the correct question. Here's my >>> context: >>> > >>> > I'm beginning to recreate in Erlang on Microsoft Windows 10 a public >>> domain >>> > program I used to enjoy on my Commodore Amiga: mATC was a game in >>> which >>> > the user acted as a military Air Traffic Controller. I am using a wx >>> > window to draw the player's map with aircraft data blocks overlaid. I >>> > don't have a use for the initial window that opens when werl is >>> started. >>> >>> Awesome! >>> >>> > If I use this Command Prompt batch file, mATC.bat: >>> > >>> > werl -pa ebin -pa mATC_app\ebin -detached -boot mATC -config sys >>> > >>> > a Windows error message window appears that says: >>> > >>> > "Failed to execute C:\Program Files\erl5.9\erts5.9\bin\beam.smp.dll >>> > The system cannot find the file specified." >>> > >>> > The DLL is actually present at that location, and of course everything >>> runs >>> > well when I don't use -detached. >>> >>> Hrm... that seems odd, but I have to admit I have no experience running >>> actual releases on Windows. What I usually do instead is have the runtime >>> installed on the Windows machine, unpack project code and kick things off >>> with an escript that builds and then launches the code I want to run. >>> >>> Changing from releases to from-source build->run may require a bit of >>> shuffling -- and may or may not be worth it depending on the project. >>> But this has been very effective and lightweight for my purposes. >>> >>> Like everything else on Windows there are a few quirks to making escripts >>> a click-to-run experience (specifying full path to "escript.exe" instead >>> of "escript" in your launcher, space for unheeded shebang, etc.). I've >>> wanted for a few years now to formalize and simplify a solution to >>> this... >>> but real life gets in the way of the things I wish I could do as >>> community >>> project work. :-( >>> >>> The escript-to-bootstrap approach has made my Erlang GUI experience on >>> Windows fairly painless -- I can develop on Linux or BSD and almost >>> always >>> get away with relying on anything that works there (to include wx) >>> working >>> pretty much the same way on Windows. >>> >>> I have not tried this on Windows 10 yet, but I assume it will continue to >>> work the same way it does on Windows 7 and 8.1. >>> >>> -Craig >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> I will look into escript, thanks. >> >> John >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.goertzen@REDACTED Wed May 25 22:30:26 2016 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Wed, 25 May 2016 20:30:26 +0000 Subject: [erlang-questions] Can werl be detached? In-Reply-To: References: Message-ID: I have a windows app (also on R15) and this is the shortcut that launches it: Target: "C:\Program Files (x86)\myprogram\erlang\erts\bin\werl.exe" -detached -smp enable -run bootstrap" Start in: "C:\Program Files (x86)\myprogram\erlang\erts\bin" So, try changing the current directory first to erts/bin before running detached. On Tue, May 17, 2016 at 6:47 PM John R. Ashmun wrote: > Perhaps my subject is not quite the correct question. Here's my context: > > I'm beginning to recreate in Erlang on Microsoft Windows 10 a public > domain program I used to enjoy on my Commodore Amiga: mATC was a game in > which the user acted as a military Air Traffic Controller. I am using a wx > window to draw the player's map with aircraft data blocks overlaid. I > don't have a use for the initial window that opens when werl is started. > > http://erlang.org/doc/man/werl.html > > says, in part, "All flags except -oldshell work as they do for the erl > command." I don't have a UNIX-y > system available to me to learn how they work for the erl command. (The > erl command as an interactive program is iffy in a Windows Command Prompt > window.) > > In particular, the description of -detached in > > http://erlang.org/doc/man/erl.html > > would seem to be what I want: > > "Starts the Erlang runtime system detached from the system console. Useful > for running daemons and backgrounds processes. Implies -noinput." > > I want the Erlang VM to run my game without an Erlang shell's having > opened a window, and I want my game to be in control of its wx window. > > Further, following what I think I learned from "Erlang and OTP in Action", > I have an OTP application being started by a boot file created from a .rel > file. All of this is in the setting of an ancient Erlang release: R15B. > > If I use this Command Prompt batch file, mATC.bat: > > werl -pa ebin -pa mATC_app\ebin -detached -boot mATC -config sys > > a Windows error message window appears that says: > > "Failed to execute C:\Program Files\erl5.9\erts5.9\bin\beam.smp.dll > The system cannot find the file specified." > > The DLL is actually present at that location, and of course everything > runs well when I don't use -detached. > > Please advise. > > Regards, > John Ashmun > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.ashmun@REDACTED Wed May 25 23:07:26 2016 From: john.ashmun@REDACTED (John Ashmun) Date: Wed, 25 May 2016 14:07:26 -0700 Subject: [erlang-questions] Can werl be detached? In-Reply-To: References: Message-ID: <67B181D6-B6F9-410F-B0C1-57F2E043A138@gmail.com> That is indeed the knack I needed. Thanks! Sent from my iPhone > On May 25, 2016, at 1:30 PM, Daniel Goertzen wrote: > > I have a windows app (also on R15) and this is the shortcut that launches it: > > Target: "C:\Program Files (x86)\myprogram\erlang\erts\bin\werl.exe" -detached -smp enable -run bootstrap" > > Start in: "C:\Program Files (x86)\myprogram\erlang\erts\bin" > > So, try changing the current directory first to erts/bin before running detached. > > > >> On Tue, May 17, 2016 at 6:47 PM John R. Ashmun wrote: >> Perhaps my subject is not quite the correct question. Here's my context: >> >> I'm beginning to recreate in Erlang on Microsoft Windows 10 a public domain program I used to enjoy on my Commodore Amiga: mATC was a game in which the user acted as a military Air Traffic Controller. I am using a wx window to draw the player's map with aircraft data blocks overlaid. I don't have a use for the initial window that opens when werl is started. >> >> http://erlang.org/doc/man/werl.html >> >> says, in part, "All flags except -oldshell work as they do for the erl command." I don't have a UNIX-y system available to me to learn how they work for the erl command. (The erl command as an interactive program is iffy in a Windows Command Prompt window.) >> >> In particular, the description of -detached in >> >> http://erlang.org/doc/man/erl.html >> >> would seem to be what I want: >> >> "Starts the Erlang runtime system detached from the system console. Useful for running daemons and backgrounds processes. Implies -noinput." >> >> I want the Erlang VM to run my game without an Erlang shell's having opened a window, and I want my game to be in control of its wx window. >> >> Further, following what I think I learned from "Erlang and OTP in Action", I have an OTP application being started by a boot file created from a .rel file. All of this is in the setting of an ancient Erlang release: R15B. >> >> If I use this Command Prompt batch file, mATC.bat: >> >> werl -pa ebin -pa mATC_app\ebin -detached -boot mATC -config sys >> >> a Windows error message window appears that says: >> >> "Failed to execute C:\Program Files\erl5.9\erts5.9\bin\beam.smp.dll >> The system cannot find the file specified." >> >> The DLL is actually present at that location, and of course everything runs well when I don't use -detached. >> >> Please advise. >> >> Regards, >> John Ashmun >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.ashmun@REDACTED Wed May 25 23:20:07 2016 From: john.ashmun@REDACTED (John Ashmun) Date: Wed, 25 May 2016 14:20:07 -0700 Subject: [erlang-questions] Can werl be detached? In-Reply-To: References: <6284265.hgQ7dGlg6e@burrito> Message-ID: I'm running an old version of Erlang for the usual reasons: everything I've tried with it works fine & I have no experience upgrading Erlang installations, but lots of experience with other upgrade attempts that were disastrous. I have R16B01 on a laptop that I use to hold backup copies of my projects (& occasionally to determine whether what I write works on a different version of Erlang). I will try using -detached there & then upgrade it to R18 or whatever the latest stable release is & try it again using that version. BTW, the wx demo is going to help my guesswork with wx a lot. Sent from my iPhone > On May 25, 2016, at 1:22 PM, Dan Gudmundsson wrote: > > Hmm, > It works fine for me on Windows 10 from msys2 with erlang in the path > and from cmd shell without erlang in the path: > > C:\Users\familjen>"c:\Program Files (x86)\erl7.3\bin\werl" -run wx demo > > C:\Users\familjen>"c:\Program Files (x86)\erl7.3\bin\werl" -detached -run wx demo > > Oh noticed the version you are running... > > It also works for me with 17.0 i.e. erl-6.0 which is the oldest I have installed here at home. > > Why are you using an old version of erlang? > Can you try a newer version and see if that works? > > >> On Wed, May 25, 2016 at 9:49 PM John Ashmun wrote: >> I just found time to try what you suggested with the wx demo, but I got the same pop up Windows error message as when I try to run my release with -detached. >> >> I added ...\erts-5.9\bin to %Path% and tried again but it failed again with the same error message. >> >> Sent from my iPhone >> >>> On May 18, 2016, at 12:02 AM, Dan Gudmundsson wrote: >>> >>> At least this works as it should: >>> >>> werl -detached -run wx demo >>> >>> >>>> On Wed, May 18, 2016 at 3:48 AM John R. Ashmun wrote: >>>> >>>> >>>>> On Tue, May 17, 2016 at 6:32 PM, zxq9 wrote: >>>>> On Tuesday 17 May 2016 16:47:35 John R. Ashmun wrote: >>>>> > Perhaps my subject is not quite the correct question. Here's my context: >>>>> > >>>>> > I'm beginning to recreate in Erlang on Microsoft Windows 10 a public domain >>>>> > program I used to enjoy on my Commodore Amiga: mATC was a game in which >>>>> > the user acted as a military Air Traffic Controller. I am using a wx >>>>> > window to draw the player's map with aircraft data blocks overlaid. I >>>>> > don't have a use for the initial window that opens when werl is started. >>>>> >>>>> Awesome! >>>>> >>>>> > If I use this Command Prompt batch file, mATC.bat: >>>>> > >>>>> > werl -pa ebin -pa mATC_app\ebin -detached -boot mATC -config sys >>>>> > >>>>> > a Windows error message window appears that says: >>>>> > >>>>> > "Failed to execute C:\Program Files\erl5.9\erts5.9\bin\beam.smp.dll >>>>> > The system cannot find the file specified." >>>>> > >>>>> > The DLL is actually present at that location, and of course everything runs >>>>> > well when I don't use -detached. >>>>> >>>>> Hrm... that seems odd, but I have to admit I have no experience running >>>>> actual releases on Windows. What I usually do instead is have the runtime >>>>> installed on the Windows machine, unpack project code and kick things off >>>>> with an escript that builds and then launches the code I want to run. >>>>> >>>>> Changing from releases to from-source build->run may require a bit of >>>>> shuffling -- and may or may not be worth it depending on the project. >>>>> But this has been very effective and lightweight for my purposes. >>>>> >>>>> Like everything else on Windows there are a few quirks to making escripts >>>>> a click-to-run experience (specifying full path to "escript.exe" instead >>>>> of "escript" in your launcher, space for unheeded shebang, etc.). I've >>>>> wanted for a few years now to formalize and simplify a solution to this... >>>>> but real life gets in the way of the things I wish I could do as community >>>>> project work. :-( >>>>> >>>>> The escript-to-bootstrap approach has made my Erlang GUI experience on >>>>> Windows fairly painless -- I can develop on Linux or BSD and almost always >>>>> get away with relying on anything that works there (to include wx) working >>>>> pretty much the same way on Windows. >>>>> >>>>> I have not tried this on Windows 10 yet, but I assume it will continue to >>>>> work the same way it does on Windows 7 and 8.1. >>>>> >>>>> -Craig >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> I will look into escript, thanks. >>>> >>>> John >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbike2000ru@REDACTED Thu May 26 06:02:15 2016 From: mbike2000ru@REDACTED (Dmitry) Date: Thu, 26 May 2016 04:02:15 +0000 (UTC) Subject: [erlang-questions] erl -name foo and hostname In-Reply-To: <1576180469.438760.1464192803540.JavaMail.yahoo@mail.yahoo.com> References: <1738749625.726843.1463998254145.JavaMail.yahoo.ref@mail.yahoo.com> <1738749625.726843.1463998254145.JavaMail.yahoo@mail.yahoo.com> <917605456.363863.1464184403367.JavaMail.yahoo@mail.yahoo.com> <1139543457.421170.1464190237184.JavaMail.yahoo@mail.yahoo.com> <1576180469.438760.1464192803540.JavaMail.yahoo@mail.yahoo.com> Message-ID: <260463415.738951.1464235335482.JavaMail.yahoo@mail.yahoo.com> Actually what is interesting for me - if I do hostname = localhost.localdomain, erl -name foo will show localhost.localdomain. i am going to install DHCP/DNS and wil try to get hostname via DHCP On Wednesday, May 25, 2016 9:13 PM, Dmitry wrote: /etc/init.d/network restartShutting down loopback interface: ? ? ? ? ? ? ? ? ? ? ? ? ?[ ?OK ?]Bringing up loopback interface: ? ? ? ? ? ? ? ? ? ? ? ? ? ?[ ?OK ?] [root@REDACTED ~]# ping kazootest2.localdomainping: unknown host kazootest2.localdomain [root@REDACTED ~]# cat /etc/networksdefault 0.0.0.0loopback 127.0.0.0link-local 169.254.0.0 On Wednesday, May 25, 2016 8:51 PM, Boris M?hmer wrote: Have You tried to ping "kazootest2.localdomain"??It would be interesting which IP shows up, if any at all. Maybe this gives a hint.Did You also have a look at "/etc/networks"?? Regards,Boris 2016-05-25 17:30 GMT+02:00 Dmitry : ps aux | grep avaroot ? ? 16011 ?0.0 ?0.0 103308 ? 852 pts/0 ? ?S+ ? 09:20 ? 0:00 grep ava[root@REDACTED ~]# no The AVAHI daemon is not running. On Wednesday, May 25, 2016 8:11 PM, Boris M?hmer wrote: Do You happen to have the "avahi" daemon running? Regards,Boris 2016-05-25 15:53 GMT+02:00 Dmitry : Hi here is my data: [root@REDACTED ~]# hostnamekazootest2.mydomain.local[root@REDACTED ~]# hostname -fkazootest2.mydomain.local[root@REDACTED ~]# erl -name fooErlang R15B03 (erts-5.9.3.1) [source] [64-bit] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9.3.1 ?(abort with ^G)(foo@REDACTED)1> On Monday, May 23, 2016 3:10 PM, Dmitry wrote: Hi ?cat /etc/hosts127.0.0.1 ? ? ? ? ? ? ? localhost.localdomain localhost::1 ? ? ? ? ? ? localhost6.localdomain6 localhost6192.168.175.135 ? kazootest2.mydomain.local kazootest2 [root@REDACTED ~]# cat /etc/sysconfig/networkNETWORKING=yesNETWORKING_IPV6=noHOSTNAME=kazootest2.mydomain.local [root@REDACTED ~]# erl -name fooErlang R15B03 (erts-5.9.3.1) [source] [64-bit] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9.3.1 ?(abort with ^G)(foo@REDACTED)1> Why dous it shows kazootest2.localdomain? Any help appreciated _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From boris.muehmer@REDACTED Thu May 26 07:18:05 2016 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Thu, 26 May 2016 07:18:05 +0200 Subject: [erlang-questions] erl -name foo and hostname In-Reply-To: <260463415.738951.1464235335482.JavaMail.yahoo@mail.yahoo.com> References: <1738749625.726843.1463998254145.JavaMail.yahoo.ref@mail.yahoo.com> <1738749625.726843.1463998254145.JavaMail.yahoo@mail.yahoo.com> <917605456.363863.1464184403367.JavaMail.yahoo@mail.yahoo.com> <1139543457.421170.1464190237184.JavaMail.yahoo@mail.yahoo.com> <1576180469.438760.1464192803540.JavaMail.yahoo@mail.yahoo.com> <260463415.738951.1464235335482.JavaMail.yahoo@mail.yahoo.com> Message-ID: I haven't responded earlier, because I am currently struggling with my Ubuntu 16.04 system at home, which I only installed some days ago. This system is a default fresh desktop setup, without any changes so far. It was quite a surprise, that invoking erl with "-name" results in a crash with a dump file being created. I haven't seen such a behaviour for quite some time now. Well, this "new" problem currently keeps me distracted from Yours. Regards, Boris 2016-05-26 6:02 GMT+02:00 Dmitry : > Actually what is interesting for me - if I do hostname = > localhost.localdomain, > > erl -name foo will show localhost.localdomain. > > i am going to install DHCP/DNS and wil try to get hostname via DHCP > > > > > On Wednesday, May 25, 2016 9:13 PM, Dmitry wrote: > > > > > /etc/init.d/network restart > Shutting down loopback interface: [ OK ] > Bringing up loopback interface: [ OK ] > > > [root@REDACTED ~]# ping kazootest2.localdomain > ping: unknown host kazootest2.localdomain > > > [root@REDACTED ~]# cat /etc/networks > default 0.0.0.0 > loopback 127.0.0.0 > link-local 169.254.0.0 > > > > On Wednesday, May 25, 2016 8:51 PM, Boris M?hmer > wrote: > > > Have You tried to ping "kazootest2.localdomain"? > It would be interesting which IP shows up, if any at all. Maybe this gives > a hint. > Did You also have a look at "/etc/networks"? > > > Regards, > Boris > > > 2016-05-25 17:30 GMT+02:00 Dmitry : > > ps aux | grep ava > root 16011 0.0 0.0 103308 852 pts/0 S+ 09:20 0:00 grep ava > [root@REDACTED ~]# > > no The AVAHI daemon is not running. > > > On Wednesday, May 25, 2016 8:11 PM, Boris M?hmer > wrote: > > > Do You happen to have the "avahi" daemon running? > > > Regards, > Boris > > 2016-05-25 15:53 GMT+02:00 Dmitry : > > Hi > > here is my data: > > [root@REDACTED ~]# hostname > kazootest2.mydomain.local > [root@REDACTED ~]# hostname -f > kazootest2.mydomain.local > [root@REDACTED ~]# erl -name foo > Erlang R15B03 (erts-5.9.3.1) [source] [64-bit] [async-threads:0] [hipe] > [kernel-poll:false] > > Eshell V5.9.3.1 (abort with ^G) > (foo@REDACTED)1> > > > > > On Monday, May 23, 2016 3:10 PM, Dmitry wrote: > > > Hi > > cat /etc/hosts > 127.0.0.1 localhost.localdomain localhost > ::1 localhost6.localdomain6 localhost6 > 192.168.175.135 kazootest2.mydomain.local kazootest2 > > [root@REDACTED ~]# cat /etc/sysconfig/network > NETWORKING=yes > NETWORKING_IPV6=no > HOSTNAME=kazootest2.mydomain.local > > [root@REDACTED ~]# erl -name foo > Erlang R15B03 (erts-5.9.3.1) [source] [64-bit] [async-threads:0] [hipe] > [kernel-poll:false] > > Eshell V5.9.3.1 (abort with ^G) > (foo@REDACTED)1> > > > Why dous it shows kazootest2.localdomain? > > Any help appreciated > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From boris.muehmer@REDACTED Thu May 26 08:38:39 2016 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Thu, 26 May 2016 08:38:39 +0200 Subject: [erlang-questions] erl -name foo and hostname In-Reply-To: References: <1738749625.726843.1463998254145.JavaMail.yahoo.ref@mail.yahoo.com> <1738749625.726843.1463998254145.JavaMail.yahoo@mail.yahoo.com> <917605456.363863.1464184403367.JavaMail.yahoo@mail.yahoo.com> <1139543457.421170.1464190237184.JavaMail.yahoo@mail.yahoo.com> <1576180469.438760.1464192803540.JavaMail.yahoo@mail.yahoo.com> <260463415.738951.1464235335482.JavaMail.yahoo@mail.yahoo.com> Message-ID: Well my problem is resolved now: I just needed a missing host entry (with the proper IP and FQDN) on my Ubuntu 16.04 systems and it is running now. I also tested the connection between different machines, and they can "find" each other without any problems now. Regards, Boris 2016-05-26 7:18 GMT+02:00 Boris M?hmer : > I haven't responded earlier, because I am currently struggling with my > Ubuntu 16.04 system at home, which I only installed some days ago. This > system is a default fresh desktop setup, without any changes so far. It was > quite a surprise, that invoking erl with "-name" results in a crash with a > dump file being created. I haven't seen such a behaviour for quite some > time now. > > Well, this "new" problem currently keeps me distracted from Yours. > > > Regards, > Boris > > > 2016-05-26 6:02 GMT+02:00 Dmitry : > >> Actually what is interesting for me - if I do hostname = >> localhost.localdomain, >> >> erl -name foo will show localhost.localdomain. >> >> i am going to install DHCP/DNS and wil try to get hostname via DHCP >> >> >> >> >> On Wednesday, May 25, 2016 9:13 PM, Dmitry wrote: >> >> >> >> >> /etc/init.d/network restart >> Shutting down loopback interface: [ OK ] >> Bringing up loopback interface: [ OK ] >> >> >> [root@REDACTED ~]# ping kazootest2.localdomain >> ping: unknown host kazootest2.localdomain >> >> >> [root@REDACTED ~]# cat /etc/networks >> default 0.0.0.0 >> loopback 127.0.0.0 >> link-local 169.254.0.0 >> >> >> >> On Wednesday, May 25, 2016 8:51 PM, Boris M?hmer >> wrote: >> >> >> Have You tried to ping "kazootest2.localdomain"? >> It would be interesting which IP shows up, if any at all. Maybe this >> gives a hint. >> Did You also have a look at "/etc/networks"? >> >> >> Regards, >> Boris >> >> >> 2016-05-25 17:30 GMT+02:00 Dmitry : >> >> ps aux | grep ava >> root 16011 0.0 0.0 103308 852 pts/0 S+ 09:20 0:00 grep ava >> [root@REDACTED ~]# >> >> no The AVAHI daemon is not running. >> >> >> On Wednesday, May 25, 2016 8:11 PM, Boris M?hmer >> wrote: >> >> >> Do You happen to have the "avahi" daemon running? >> >> >> Regards, >> Boris >> >> 2016-05-25 15:53 GMT+02:00 Dmitry : >> >> Hi >> >> here is my data: >> >> [root@REDACTED ~]# hostname >> kazootest2.mydomain.local >> [root@REDACTED ~]# hostname -f >> kazootest2.mydomain.local >> [root@REDACTED ~]# erl -name foo >> Erlang R15B03 (erts-5.9.3.1) [source] [64-bit] [async-threads:0] [hipe] >> [kernel-poll:false] >> >> Eshell V5.9.3.1 (abort with ^G) >> (foo@REDACTED)1> >> >> >> >> >> On Monday, May 23, 2016 3:10 PM, Dmitry wrote: >> >> >> Hi >> >> cat /etc/hosts >> 127.0.0.1 localhost.localdomain localhost >> ::1 localhost6.localdomain6 localhost6 >> 192.168.175.135 kazootest2.mydomain.local kazootest2 >> >> [root@REDACTED ~]# cat /etc/sysconfig/network >> NETWORKING=yes >> NETWORKING_IPV6=no >> HOSTNAME=kazootest2.mydomain.local >> >> [root@REDACTED ~]# erl -name foo >> Erlang R15B03 (erts-5.9.3.1) [source] [64-bit] [async-threads:0] [hipe] >> [kernel-poll:false] >> >> Eshell V5.9.3.1 (abort with ^G) >> (foo@REDACTED)1> >> >> >> Why dous it shows kazootest2.localdomain? >> >> Any help appreciated >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> >> >> >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbike2000ru@REDACTED Thu May 26 13:07:01 2016 From: mbike2000ru@REDACTED (Dmitry) Date: Thu, 26 May 2016 11:07:01 +0000 (UTC) Subject: [erlang-questions] erl -name foo and hostname In-Reply-To: References: <1738749625.726843.1463998254145.JavaMail.yahoo.ref@mail.yahoo.com> <1738749625.726843.1463998254145.JavaMail.yahoo@mail.yahoo.com> <917605456.363863.1464184403367.JavaMail.yahoo@mail.yahoo.com> <1139543457.421170.1464190237184.JavaMail.yahoo@mail.yahoo.com> <1576180469.438760.1464192803540.JavaMail.yahoo@mail.yahoo.com> <260463415.738951.1464235335482.JavaMail.yahoo@mail.yahoo.com> Message-ID: <551724796.857981.1464260821900.JavaMail.yahoo@mail.yahoo.com> I found the cause: cat /etc/resolv.conf# Generated by NetworkManagerdomain localdomainsearch localdomainnameserver 192.168.175.2 After it I tested with DNS server (local server) - and everithing is ok. so DNS... Thanks You On Thursday, May 26, 2016 11:38 AM, Boris M?hmer wrote: Well my problem is resolved now: I just needed a missing host entry (with the proper IP and FQDN) on my Ubuntu 16.04 systems and it is running now. I also tested the connection between different machines, and they can "find" each other without any problems now. Regards, Boris 2016-05-26 7:18 GMT+02:00 Boris M?hmer : I haven't responded earlier, because I am currently struggling with my Ubuntu 16.04 system at home, which I only installed some days ago. This system is a default fresh desktop setup, without any changes so far. It was quite a surprise, that invoking erl with "-name" results in a crash with a dump file being created. I haven't seen such a behaviour for quite some time now. Well, this "new" problem currently keeps me distracted from Yours. Regards, Boris 2016-05-26 6:02 GMT+02:00 Dmitry : Actually what is interesting for me - if I do hostname = localhost.localdomain, erl -name foo will show localhost.localdomain. i am going to install DHCP/DNS and wil try to get hostname via DHCP On Wednesday, May 25, 2016 9:13 PM, Dmitry wrote: /etc/init.d/network restartShutting down loopback interface: ? ? ? ? ? ? ? ? ? ? ? ? ?[ ?OK ?]Bringing up loopback interface: ? ? ? ? ? ? ? ? ? ? ? ? ? ?[ ?OK ?] [root@REDACTED ~]# ping kazootest2.localdomainping: unknown host kazootest2.localdomain [root@REDACTED ~]# cat /etc/networksdefault 0.0.0.0loopback 127.0.0.0link-local 169.254.0.0 On Wednesday, May 25, 2016 8:51 PM, Boris M?hmer wrote: Have You tried to ping "kazootest2.localdomain"??It would be interesting which IP shows up, if any at all. Maybe this gives a hint.Did You also have a look at "/etc/networks"?? Regards,Boris 2016-05-25 17:30 GMT+02:00 Dmitry : ps aux | grep avaroot ? ? 16011 ?0.0 ?0.0 103308 ? 852 pts/0 ? ?S+ ? 09:20 ? 0:00 grep ava[root@REDACTED ~]# no The AVAHI daemon is not running. On Wednesday, May 25, 2016 8:11 PM, Boris M?hmer wrote: Do You happen to have the "avahi" daemon running? Regards,Boris 2016-05-25 15:53 GMT+02:00 Dmitry : Hi here is my data: [root@REDACTED ~]# hostnamekazootest2.mydomain.local[root@REDACTED ~]# hostname -fkazootest2.mydomain.local[root@REDACTED ~]# erl -name fooErlang R15B03 (erts-5.9.3.1) [source] [64-bit] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9.3.1 ?(abort with ^G)(foo@REDACTED)1> On Monday, May 23, 2016 3:10 PM, Dmitry wrote: Hi ?cat /etc/hosts127.0.0.1 ? ? ? ? ? ? ? localhost.localdomain localhost::1 ? ? ? ? ? ? localhost6.localdomain6 localhost6192.168.175.135 ? kazootest2.mydomain.local kazootest2 [root@REDACTED ~]# cat /etc/sysconfig/networkNETWORKING=yesNETWORKING_IPV6=noHOSTNAME=kazootest2.mydomain.local [root@REDACTED ~]# erl -name fooErlang R15B03 (erts-5.9.3.1) [source] [64-bit] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9.3.1 ?(abort with ^G)(foo@REDACTED)1> Why dous it shows kazootest2.localdomain? Any help appreciated _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Fri May 27 12:50:48 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 27 May 2016 12:50:48 +0200 Subject: [erlang-questions] rebar3 dependencies In-Reply-To: References: <1458343327.2161757.553505114.0E63C3AA@webmail.messagingengine.com> <56EC9CA1.80901@gmail.com> <1458347567.2174564.553539442.66BE7E14@webmail.messagingengine.com> <56ECA320.8050601@gmail.com> <1458349133.2179492.553551178.67554860@webmail.messagingengine.com> <56ECAD6C.7040905@gmail.com> <0B52FCB2-AC42-4C3F-ACC2-415ABE80A845@gmail.com> <20160319122425.GJ53561@ferdmbp.local> <56ED9C47.9000904@gmail.com> <1458421591.2408934.554047170.01DEDFFB@webmail.messagingengine.com> Message-ID: Dear list, Following up on this discussion. I've took on the task of maintaining the vendoring plugin started by Tristan (thank you for doing so and for your availability). I've started maintaining it now and have modified to meet my needs. I don't know how many of you vendor their dependencies, but this will allow you to do so. rebar3_vendor v0.3.0 has just been released and can be found here: https://hex.pm/packages/rebar3_vendor https://github.com/ostinelli/rebar3_vendor Best, r. On Fri, Mar 25, 2016 at 1:16 PM, Eric Meadows-J?nsson < eric.meadows.jonsson@REDACTED> wrote: > Hex.pm does not allow users to remove or overwrite published packages so > the issue that happened with npm cannot happen. Packages will only be > removed in very special circumstances, such as us being forced to do so for > legal reasons and even then we will of course not allow a new package to be > published with the removed's package name. > > Rebar and Mix will also add package checksums to the lock so if you don't > trust the Hex repository you are using you can at least trust the checksum > check. Additionally, over the next days I will work on improving and > documenting hex.pm's policies so that it will hopefully be clear how we > will act in circumstances such as these. > > On Wed, Mar 23, 2016 at 1:47 PM, Roberto Ostinelli > wrote: > >> On the subject on additional reasons to vendor dependencies: >> http://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/ >> >> BTW, not saying this can happen with hex.pm. >> >> Best, >> r. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > > -- > Eric Meadows-J?nsson > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Oliver.Korpilla@REDACTED Fri May 27 14:06:08 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Fri, 27 May 2016 14:06:08 +0200 Subject: [erlang-questions] Multi-node setup: Adding nodes with mnesia and gproc? Message-ID: Hello. We are trying to build a setup that will scale in an OpenStack cloud. Currently I as developer only know that the orchestrator will bring up additional instances as needed. I restructured my application in such a way that I have a central node that does not scale and provides services to all other nodes. It comes up first. Then at least one worker node is started which only starts one kind of worker, but possibly a lot of them - on demand. Me and my coworkers are new to the Erlang world and experimented a bit with gproc and mnesia when it comes to multi-node. The model we would be looking for is that a worker instance comes up and then gets access to all gproc global names and properties and also to the DB tables. So far our examples only worked if we connected the nodes first and then started gproc. Can gproc and mnesia accomodate nodes that were added to the cluster after they were started? Is anything special necessary? Thanks and regards, Oliver From Oliver.Korpilla@REDACTED Fri May 27 15:04:58 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Fri, 27 May 2016 15:04:58 +0200 Subject: [erlang-questions] Multi-node setup: Adding nodes with mnesia and gproc? In-Reply-To: References: , Message-ID: Hello, Ulf. And how about gproc? (You're the author, right? Love its feature set!) Thank you, Oliver ? Gesendet:?Freitag, 27. Mai 2016 um 14:32 Uhr Von:?"Ulf Wiger" An:?"Oliver Korpilla" Cc:?"erlang questions" Betreff:?Re: [erlang-questions] Multi-node setup: Adding nodes with mnesia and gproc? > On 27 May 2016, at 14:06, Oliver Korpilla wrote: > > Hello. > > We are trying to build a setup that will scale in an OpenStack cloud. Currently I as developer only know that the orchestrator will bring up additional instances as needed. > > I restructured my application in such a way that I have a central node that does not scale and provides services to all other nodes. It comes up first. Then at least one worker node is started which only starts one kind of worker, but possibly a lot of them - on demand. > > Me and my coworkers are new to the Erlang world and experimented a bit with gproc and mnesia when it comes to multi-node. The model we would be looking for is that a worker instance comes up and then gets access to all gproc global names and properties and also to the DB tables. So far our examples only worked if we connected the nodes first and then started gproc. > > Can gproc and mnesia accomodate nodes that were added to the cluster after they were started? Is anything special necessary? With mnesia, you can start the new nodes with the mnesia environment variable {extra_db_nodes, [CentralNode]}, where you could obviously add more nodes to the list. This will make mnesia fetch the schema from one of those nodes and make all tables available (remember, you don?t have to have a local copy of a table for it to be accessible). With this setting, mnesia will also automatically connect as it starts. Basically, you will always have to either manually connect, or instruct some application to do so via configuration. Using mnesia as indicated above is one way to do it. BR, Ulf W From community-manager@REDACTED Fri May 27 15:14:27 2016 From: community-manager@REDACTED (Bruce Yinhe) Date: Fri, 27 May 2016 15:14:27 +0200 Subject: [erlang-questions] New Erlang job openings Message-ID: Hi There are 23 Erlang-related job openings on https://erlangcentral.org/jobs. Subscribe to weekly Erlang job updates: https://erlangcentral.org/login/?action=register Publish an Erlang job opening: https://erlangcentral.org/jobs/add Software Engineer in Erlang - Erlang Solutions - London, UK http://erlangcentral.org/software-engineer-erlang-solutions-london/ Apply before: 2016-05-30 Erlang Backend Engineer - Tigertext - Santa Monica, CA, USA http://erlangcentral.org/erlang-backend-engineer-tigertext/ Apply before: 2016-05-30 Lead Erlang Engineer ? Relo Offered - SoCal Startup - Los Angeles, CA; Santa Monica, CA, USA https://erlangcentral.org/lead-erlang-engineer-relo-offered-socal-startup/ Apply before: 2016-05-30 Software Engineer ? Erlang/Functional Programming - Klarna - Stockholm, Sweden http://erlangcentral.org/software-engineer-erlang-functional-programming-klarna/ Apply before: 2016-05-30 Elixir Developer ? Contract - Darwin Recruitment - Billericay, UK http://erlangcentral.org/elixir-developer-contract-darwin-recruitment/ Apply before: 2016-05-30 Software Developer - LiveHelpNow - Willow Grove/Bethlehem, PA, USA http://erlangcentral.org/software-developer-livehelpnow/ Apply before: 2016-05-30 Erlang/Elixir Developer - Darwin Recruitment - London, UK http://erlangcentral.org/erlang-elixir-developer-darwin/ Apply before: 2016-05-30 Erlang and Elixir interns - Erlang Solutions - Krak?w, Poland http://erlangcentral.org/erlang-and-elixir-interns-erlang-solutions/ Apply before: 2016-05-30 Sr Application Security Engineer, Erlang - Machine Zone - Palo Alto, CA, USA http://erlangcentral.org/sr-application-security-engineer-erlang-machine-zone/ Apply before: 2016-05-30 Erlang Developer - Inaka - Buenos Aires, Argentina https://erlangcentral.org/erlang-developer-inaka/ Apply before: 2016-05-31 Erlang Developer - Sqor Sports - San Francisco, CA, USA https://erlangcentral.org/erlang-developer-sqor/ Apply before: 2016-06-29 Senior Software Developer in Erlang - DEK Technologies Sweden AB - Stockholm, Sweden http://erlangcentral.org/senior-software-developer-in-erlang-dek-technologies/ Apply before: 2016-06-29 Erlang Developer ? Contract - Darwin Recruitment - Billericay, UK http://erlangcentral.org/erlang-developer-contract-darwin-recruitment/ Apply before: 2016-06-29 Erlang Developer - Vocalink - London, UK http://erlangcentral.org/erlang-developer-vocalink/ Apply before: 2016-06-29 Senior Erlang Engineer - Grindr - Hollywood, CA, USA http://erlangcentral.org/senior-erlang-engineer-grindr/ Apply before: 2016-06-29 Erlang & Java Developer - Darwin Recruitment - London, UK http://erlangcentral.org/erlang-java-developer-darwin-recruitment/ Apply before: 2016-07-28 Platform and Data Engineer - Handy's culture deck - New York City, USA http://erlangcentral.org/platform-and-data-engineer-handy/ Apply before: 2016-07-30 Backend software engineer - Electic Imp - Any (remote); London, UK preferred, UK http://erlangcentral.org/backend-software-engineer-electric-imp/ Apply before: 2016-07-30 Erlang Developer - Krubera Group - Hertfordshire, UK https://erlangcentral.org/erlang-developer-krubera-group/ Apply before: 2016-11-04 Erlang Software Engineer - F Technologies - Dubai, UAE http://erlangcentral.org/erlang-software-engineer-f-technologies/ Apply before: 2016-12-30 Back-end Engineer - 2600hz - San Francisco, CA, USA http://erlangcentral.org/back-end-engineer-2600hz/ Apply before: 2017-08-30 Online Services Programmer - Creative Assembly - , UK http://erlangcentral.org/online-services-programmer-creative-assembly/ Apply before: 2016-08-30 Erlang Software Engineer - Cisco - Stockholm, Swedem http://erlangcentral.org/erlang-software-engineer-cisco/ Apply before: 2016-08-30 *Bruce Yinhe* Community Manager Industrial Erlang User Group +46 72 311 43 89 community-manager@REDACTED -- Visit our Erlang community site ErlangCentral.org | @ErlangCentral | Industrial Erlang User Group -------------- next part -------------- An HTML attachment was scrubbed... URL: From mattevans123@REDACTED Sun May 29 16:21:54 2016 From: mattevans123@REDACTED (Matthew Evans) Date: Sun, 29 May 2016 10:21:54 -0400 Subject: [erlang-questions] beam core file R17 Message-ID: Hi, This core was found on a live system (R17): 10:16:38:# erl Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] Eshell V6.2 (abort with ^G) 1> ........ May 28 22:18:56 [info ] plexxi kernel: [1235119.885465] beam.smp[2267] general protection ip:4b698a sp:7faeb6a7d650 error:0 in beam.smp[400000+1ac000] warning: Can't read pathname for load map: Input/output error.[Thread debugging using libthread_db enabled]Using host libthread_db library "/lib/libthread_db.so.1".Core was generated by `/usr/lib/erlang/erts-6.2/bin/beam.smp -K true -A 24 -P 350000 -- -root /usr/lib'.Program terminated with signal 11, Segmentation fault.#0 0x00000000004b698a in sweep_off_heap ()(gdb) bt#0 0x00000000004b698a in sweep_off_heap ()#1 0x00000000004b77d1 in do_minor ()#2 0x00000000004b8479 in erts_garbage_collect ()#3 0x00000000004e1374 in process_main ()#4 0x000000000048071d in sched_thread_func ()#5 0x0000000000549f89 in thr_wrapper ()#6 0x00007faeba32ba30 in start_thread () from /lib/libpthread.so.0#7 0x00007faeb9e8a53d in clone () from /lib/libc.so.6 (gdb) up -------------- next part -------------- An HTML attachment was scrubbed... URL: From t6sn7gt@REDACTED Sun May 29 17:00:34 2016 From: t6sn7gt@REDACTED (Donald Steven) Date: Sun, 29 May 2016 11:00:34 -0400 Subject: [erlang-questions] Reading a float from standard input Message-ID: <574B0412.4020404@aim.com> Hi all, (Newbie question) The following works: {X, _} = string:to_float(string:strip(io:get_line("X: "), right, $\n)), but is there a less cumbersome way to read a float from the standard input? Thanks. Don From Oliver.Korpilla@REDACTED Sun May 29 17:16:37 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Sun, 29 May 2016 17:16:37 +0200 Subject: [erlang-questions] Multi-node setup: Adding nodes with mnesia and gproc? In-Reply-To: <4D383DE8-C106-48BF-942D-258F22A226E6@gmail.com> References: , <4D383DE8-C106-48BF-942D-258F22A226E6@gmail.com> Message-ID: Hello, Ulf. Thanks for the help. I was able to let one node bring up the DB and the tables and the other access them, and I switched the names of distributed processes from node-local to global and the application still works. (I had to turn one direct-call API into RPC but not much more.) Ironically it already works for something close to an actual setup, but now I have to write a script that brings up all together for interacting with the test anti-product. WBR, Oliver ? ? Gesendet:?Freitag, 27. Mai 2016 um 16:36 Uhr Von:?"Ulf Wiger" An:?"Oliver Korpilla" Cc:?"erlang questions" Betreff:?Re: [erlang-questions] Multi-node setup: Adding nodes with mnesia and gproc? For the longest time, I?ve viewed the global name registration in gproc as ? ?experimental?, initially because gen_leader had no support for the dynamic adding/removal of nodes. Later versions of gen_leader have this, but the master branch of gproc is still a bit weak on merging. There is an ?uw-locks_leader? branch, which I believe has a better* approach to dynamic node handling and merge scenarios (including conflict resolution support), but it is a little bit behind the master branch. I plan to make that the new default eventually, but the merge was non-trivial, so I?ve put it on ice for the time being. BR, Ulf W * In all fairness, gen_leader is much more battle-tested than locks_leader. That of course means that with gen_leader, you can find out what the known issues are, whereas with locks_leader, you will have to discover them. The latter is of course much more fun! > On 27 May 2016, at 15:04, Oliver Korpilla wrote: > > Hello, Ulf. > > And how about gproc? (You're the author, right? Love its feature set!) > > Thank you, > Oliver > > > Gesendet: Freitag, 27. Mai 2016 um 14:32 Uhr > Von: "Ulf Wiger" > An: "Oliver Korpilla" > Cc: "erlang questions" > Betreff: Re: [erlang-questions] Multi-node setup: Adding nodes with mnesia and gproc? >> On 27 May 2016, at 14:06, Oliver Korpilla wrote: >> >> Hello. >> >> We are trying to build a setup that will scale in an OpenStack cloud. Currently I as developer only know that the orchestrator will bring up additional instances as needed. >> >> I restructured my application in such a way that I have a central node that does not scale and provides services to all other nodes. It comes up first. Then at least one worker node is started which only starts one kind of worker, but possibly a lot of them - on demand. >> >> Me and my coworkers are new to the Erlang world and experimented a bit with gproc and mnesia when it comes to multi-node. The model we would be looking for is that a worker instance comes up and then gets access to all gproc global names and properties and also to the DB tables. So far our examples only worked if we connected the nodes first and then started gproc. >> >> Can gproc and mnesia accomodate nodes that were added to the cluster after they were started? Is anything special necessary? > > With mnesia, you can start the new nodes with the mnesia environment variable {extra_db_nodes, [CentralNode]}, where you could obviously add more nodes to the list. This will make mnesia fetch the schema from one of those nodes and make all tables available (remember, you don?t have to have a local copy of a table for it to be accessible). With this setting, mnesia will also automatically connect as it starts. > > Basically, you will always have to either manually connect, or instruct some application to do so via configuration. Using mnesia as indicated above is one way to do it. > > BR, > Ulf W ? From gordeev.vladimir.v@REDACTED Sun May 29 17:53:11 2016 From: gordeev.vladimir.v@REDACTED (Vladimir Gordeev) Date: Sun, 29 May 2016 18:53:11 +0300 Subject: [erlang-questions] Trace everything Message-ID: I know maybe it sounds really bad, but I want to trace whole running erlang node. I want to get following information: * PidX spawns PidY * PidX died or exited * PidX sent message to PidY (message itself is ignored) Actually, nothing more. Maybe couple hardcoded patterns on messages, to count them differently. What are the best ways to achieve it, without degrading much overall performance? Is erlang:trace/3 okay for it? I've heard something about using lttng for Erlang -- is it was I need? -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Sun May 29 17:56:03 2016 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Sun, 29 May 2016 17:56:03 +0200 Subject: [erlang-questions] [erlang-bugs] beam core file R17 In-Reply-To: References: Message-ID: <22347.4371.901213.18449@gargle.gargle.HOWL> Matthew Evans writes: > Hi, > This core was found on a live system (R17): > > > > > > > > > 10:16:38:# erl > Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] > > > Eshell V6.2 (abort with ^G) > 1> > > ........ > > > > > > > > > May 28 22:18:56 [info ] plexxi kernel: [1235119.885465] beam.smp[2267] general protection ip:4b698a sp:7faeb6a7d650 error:0 in beam.smp[400000+1ac000] > warning: Can't read pathname for load map: Input/output error.[Thread debugging using libthread_db enabled]Using host libthread_db library "/lib/libthread_db.so.1".Core was generated by `/usr/lib/erlang/erts-6.2/bin/beam.smp -K true -A 24 -P 350000 -- -root /usr/lib'.Program terminated with signal 11, Segmentation fault.#0 0x00000000004b698a in sweep_off_heap ()(gdb) bt#0 0x00000000004b698a in sweep_off_heap ()#1 0x00000000004b77d1 in do_minor ()#2 0x00000000004b8479 in erts_garbage_collect ()#3 0x00000000004e1374 in process_main ()#4 0x000000000048071d in sched_thread_func ()#5 0x0000000000549f89 in thr_wrapper ()#6 0x00007faeba32ba30 in start_thread () from /lib/libpthread.so.0#7 0x00007faeb9e8a53d in clone () from /lib/libc.so.6 It looks like your beam.smp binary lacks debugging information, so we only know the general area where it crashed (sweep_off_heap() as called from do_minor()). Crashes here would usually be due to memory corruption, which could be caused by: - a bug in the VM - a bug in a NIF - a bug in HiPE - a bug in the C compiler used to compile the VM (I've seen that happen at least 3 times) - a HW error (though you'd then also find e.g. machine check events logged) If you want to debug this, you should first ensure that your beam.smp gets built and installed with full debugging information (just attach gdb, bt, and list to verify). You should also try without NIFs or native code, if those are used and you can configure them not to be. From mattevans123@REDACTED Sun May 29 18:04:16 2016 From: mattevans123@REDACTED (Matthew Evans) Date: Sun, 29 May 2016 12:04:16 -0400 Subject: [erlang-questions] [erlang-bugs] beam core file R17 In-Reply-To: <22347.4371.901213.18449@gargle.gargle.HOWL> References: <22347.4371.901213.18449@gargle.gargle.HOWL> Message-ID: Thanks, this software does use nifs Sent from my iPhone > On May 29, 2016, at 11:56 AM, Mikael Pettersson wrote: > > Matthew Evans writes: >> Hi, >> This core was found on a live system (R17): >> >> >> >> >> >> >> >> >> 10:16:38:# erl >> Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] >> >> >> Eshell V6.2 (abort with ^G) >> 1> >> >> ........ >> >> >> >> >> >> >> >> >> May 28 22:18:56 [info ] plexxi kernel: [1235119.885465] beam.smp[2267] general protection ip:4b698a sp:7faeb6a7d650 error:0 in beam.smp[400000+1ac000] >> warning: Can't read pathname for load map: Input/output error.[Thread debugging using libthread_db enabled]Using host libthread_db library "/lib/libthread_db.so.1".Core was generated by `/usr/lib/erlang/erts-6.2/bin/beam.smp -K true -A 24 -P 350000 -- -root /usr/lib'.Program terminated with signal 11, Segmentation fault.#0 0x00000000004b698a in sweep_off_heap ()(gdb) bt#0 0x00000000004b698a in sweep_off_heap ()#1 0x00000000004b77d1 in do_minor ()#2 0x00000000004b8479 in erts_garbage_collect ()#3 0x00000000004e1374 in process_main ()#4 0x000000000048071d in sched_thread_func ()#5 0x0000000000549f89 in thr_wrapper ()#6 0x00007faeba32ba30 in start_thread () from /lib/libpthread.so.0#7 0x00007faeb9e8a53d in clone () from /lib/libc.so.6 > > It looks like your beam.smp binary lacks debugging information, so we only know the general > area where it crashed (sweep_off_heap() as called from do_minor()). Crashes here would usually > be due to memory corruption, which could be caused by: > - a bug in the VM > - a bug in a NIF > - a bug in HiPE > - a bug in the C compiler used to compile the VM (I've seen that happen at least 3 times) > - a HW error (though you'd then also find e.g. machine check events logged) > > If you want to debug this, you should first ensure that your beam.smp gets built and installed > with full debugging information (just attach gdb, bt, and list to verify). You should also try > without NIFs or native code, if those are used and you can configure them not to be. From g@REDACTED Sun May 29 18:17:20 2016 From: g@REDACTED (Guilherme Andrade) Date: Sun, 29 May 2016 17:17:20 +0100 Subject: [erlang-questions] Reading a float from standard input In-Reply-To: <574B0412.4020404@aim.com> References: <574B0412.4020404@aim.com> Message-ID: On 29 May 2016 at 16:00, Donald Steven wrote: > The following works: {X, _} = string:to_float(string:strip(io:get_line("X: > "), right, $\n)), > > but is there a less cumbersome way to read a float from the standard input? > {ok, [F]} = io:fread("give me your floats: ", "~f") Cheers, > > Thanks. > > Don > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Guilherme -------------- next part -------------- An HTML attachment was scrubbed... URL: From t6sn7gt@REDACTED Sun May 29 18:39:43 2016 From: t6sn7gt@REDACTED (Donald Steven) Date: Sun, 29 May 2016 12:39:43 -0400 Subject: [erlang-questions] Reading a float from standard input In-Reply-To: References: <574B0412.4020404@aim.com> Message-ID: <574B1B4F.80706@aim.com> Thanks Guilherme! On 05/29/2016 12:17 PM, Guilherme Andrade wrote: > {ok, [F]} = io:fread("give me your floats: ", "~f") From garazdawi@REDACTED Sun May 29 21:23:15 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Sun, 29 May 2016 21:23:15 +0200 Subject: [erlang-questions] [erlang-bugs] beam core file R17 In-Reply-To: References: <22347.4371.901213.18449@gargle.gargle.HOWL> Message-ID: sweep_off_heap is run when collecting refc binaries (among other things), so if you by mistake have decremented the reference count of a binary in a nif too much, this error will happen when the GC is inspecting the binary. I'd recommend looking for something like that in any nifs you have. On Sun, May 29, 2016 at 6:04 PM, Matthew Evans wrote: > Thanks, this software does use nifs > > Sent from my iPhone > > > On May 29, 2016, at 11:56 AM, Mikael Pettersson > wrote: > > > > Matthew Evans writes: > >> Hi, > >> This core was found on a live system (R17): > >> > >> > >> > >> > >> > >> > >> > >> > >> 10:16:38:# erl > >> Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:8:8] [async-threads:10] > [hipe] [kernel-poll:false] > >> > >> > >> Eshell V6.2 (abort with ^G) > >> 1> > >> > >> ........ > >> > >> > >> > >> > >> > >> > >> > >> > >> May 28 22:18:56 [info ] plexxi kernel: [1235119.885465] > beam.smp[2267] general protection ip:4b698a sp:7faeb6a7d650 error:0 in > beam.smp[400000+1ac000] > >> warning: Can't read pathname for load map: Input/output error.[Thread > debugging using libthread_db enabled]Using host libthread_db library > "/lib/libthread_db.so.1".Core was generated by > `/usr/lib/erlang/erts-6.2/bin/beam.smp -K true -A 24 -P 350000 -- -root > /usr/lib'.Program terminated with signal 11, Segmentation fault.#0 > 0x00000000004b698a in sweep_off_heap ()(gdb) bt#0 0x00000000004b698a in > sweep_off_heap ()#1 0x00000000004b77d1 in do_minor ()#2 > 0x00000000004b8479 in erts_garbage_collect ()#3 0x00000000004e1374 in > process_main ()#4 0x000000000048071d in sched_thread_func ()#5 > 0x0000000000549f89 in thr_wrapper ()#6 0x00007faeba32ba30 in start_thread > () from /lib/libpthread.so.0#7 0x00007faeb9e8a53d in clone () from > /lib/libc.so.6 > > > > It looks like your beam.smp binary lacks debugging information, so we > only know the general > > area where it crashed (sweep_off_heap() as called from do_minor()). > Crashes here would usually > > be due to memory corruption, which could be caused by: > > - a bug in the VM > > - a bug in a NIF > > - a bug in HiPE > > - a bug in the C compiler used to compile the VM (I've seen that happen > at least 3 times) > > - a HW error (though you'd then also find e.g. machine check events > logged) > > > > If you want to debug this, you should first ensure that your beam.smp > gets built and installed > > with full debugging information (just attach gdb, bt, and list to > verify). You should also try > > without NIFs or native code, if those are used and you can configure > them not to be. > _______________________________________________ > erlang-bugs mailing list > erlang-bugs@REDACTED > http://erlang.org/mailman/listinfo/erlang-bugs > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mattevans123@REDACTED Mon May 30 03:32:58 2016 From: mattevans123@REDACTED (Matthew Evans) Date: Sun, 29 May 2016 21:32:58 -0400 Subject: [erlang-questions] [erlang-bugs] beam core file R17 In-Reply-To: References: , <22347.4371.901213.18449@gargle.gargle.HOWL>, , Message-ID: Thanks, Fortunately our latest release has moved all the NIF logic to a separate CNODE based process. I am leaning towards the NIF as the cause since the VM itself has proven to be very stable. Date: Sun, 29 May 2016 21:23:15 +0200 Subject: Re: [erlang-bugs] beam core file R17 From: garazdawi@REDACTED To: mattevans123@REDACTED CC: erlang-questions@REDACTED sweep_off_heap is run when collecting refc binaries (among other things), so if you by mistake have decremented the reference count of a binary in a nif too much, this error will happen when the GC is inspecting the binary. I'd recommend looking for something like that in any nifs you have. On Sun, May 29, 2016 at 6:04 PM, Matthew Evans wrote: Thanks, this software does use nifs Sent from my iPhone > On May 29, 2016, at 11:56 AM, Mikael Pettersson wrote: > > Matthew Evans writes: >> Hi, >> This core was found on a live system (R17): >> >> >> >> >> >> >> >> >> 10:16:38:# erl >> Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] >> >> >> Eshell V6.2 (abort with ^G) >> 1> >> >> ........ >> >> >> >> >> >> >> >> >> May 28 22:18:56 [info ] plexxi kernel: [1235119.885465] beam.smp[2267] general protection ip:4b698a sp:7faeb6a7d650 error:0 in beam.smp[400000+1ac000] >> warning: Can't read pathname for load map: Input/output error.[Thread debugging using libthread_db enabled]Using host libthread_db library "/lib/libthread_db.so.1".Core was generated by `/usr/lib/erlang/erts-6.2/bin/beam.smp -K true -A 24 -P 350000 -- -root /usr/lib'.Program terminated with signal 11, Segmentation fault.#0 0x00000000004b698a in sweep_off_heap ()(gdb) bt#0 0x00000000004b698a in sweep_off_heap ()#1 0x00000000004b77d1 in do_minor ()#2 0x00000000004b8479 in erts_garbage_collect ()#3 0x00000000004e1374 in process_main ()#4 0x000000000048071d in sched_thread_func ()#5 0x0000000000549f89 in thr_wrapper ()#6 0x00007faeba32ba30 in start_thread () from /lib/libpthread.so.0#7 0x00007faeb9e8a53d in clone () from /lib/libc.so.6 > > It looks like your beam.smp binary lacks debugging information, so we only know the general > area where it crashed (sweep_off_heap() as called from do_minor()). Crashes here would usually > be due to memory corruption, which could be caused by: > - a bug in the VM > - a bug in a NIF > - a bug in HiPE > - a bug in the C compiler used to compile the VM (I've seen that happen at least 3 times) > - a HW error (though you'd then also find e.g. machine check events logged) > > If you want to debug this, you should first ensure that your beam.smp gets built and installed > with full debugging information (just attach gdb, bt, and list to verify). You should also try > without NIFs or native code, if those are used and you can configure them not to be. _______________________________________________ erlang-bugs mailing list erlang-bugs@REDACTED http://erlang.org/mailman/listinfo/erlang-bugs -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon May 30 13:54:09 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 30 May 2016 14:54:09 +0300 Subject: [erlang-questions] speed of neotoma parser In-Reply-To: References: <55A810C00B7CDA48A5AB43C566064B41014C52EAD0@SE-EX019.groupinfra.com> Message-ID: This story ended with writing handmade parser. Seems that we've accidentially created some O(N^2) code in neotoma-generated parser, because small files are parsed quickly, but big files (50KB, 100KB and more) are parsed very slowly with our grammar. New handmade parser can parse in a linear time, something like 50 ms for a 200KB file when our neotoma grammar takes about 50 seconds or more. Sean have looked at our PEG grammar and confirmed that we have made anti-optimal grammar that creates bad parser (right, Sean?). I've looked at alternatives (it is hard to find anything as convenient as neotoma, I think that it is most convenient) and had to refuse from leex + yacc, because we have already made syntax that has dependency in lexical parsing from semantic context (I suppose that it has). So I've written in 2 days plain erlang code that just works on all our tests and all examples of user configs. Thanks again for all hints and help! -------------- next part -------------- An HTML attachment was scrubbed... URL: From tobias.ammann@REDACTED Mon May 30 14:53:08 2016 From: tobias.ammann@REDACTED (TA) Date: Mon, 30 May 2016 08:53:08 -0400 Subject: [erlang-questions] Erlang Certifications Message-ID: Hello, I have seen job adverts refer to an "Erlang Foundation" certification, but couldn't find anything recent about it online (only old pages on erlang.se). Is there a "standard" way to show your Erlang skills, e.g. certifications and (remote) courses? Where can I find out more (other than erlang.org/community)? Thanks Tobias -------------- next part -------------- An HTML attachment was scrubbed... URL: From richard.jonas@REDACTED Mon May 30 16:38:39 2016 From: richard.jonas@REDACTED (Richard Jonas) Date: Mon, 30 May 2016 16:38:39 +0200 Subject: [erlang-questions] bcrypt message queues Message-ID: Hey guys, we are implementing user registration and password checking functionality and using bcrypt for password hashing. During registration and also during login we need to compute the hashes of the passwords. The problem is that bcrypt is serializing requests even when I choose nif or port mechanism. https://github.com/chef/erlang-bcrypt With nif a message queue is instantly built when multiple users log in (compute hashes for match password). With a pool size+port implementation, the message queue building has just deferred a bit. Is there any alternative to creating bcrypt-like hashes? For me it seems that this library cannot be scaled well. Or have I missed something? -- Richard Jonas Erlang Solutions Hungary Kft Address: Riverpark Office K.32 K?zrakt?r street 32. 3/1. 1093 Budapest Hungary Phone/fax: +36-1-7000-654 -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Mon May 30 16:56:39 2016 From: dmkolesnikov@REDACTED (dmkolesnikov@REDACTED) Date: Mon, 30 May 2016 17:56:39 +0300 Subject: [erlang-questions] bcrypt message queues In-Reply-To: References: Message-ID: Hello, I've played with pbkdf2 function for same purpose. It is based on hmac from crypto library. https://github.com/fogfish/hash Someone, might disagree about pbkdf2 due to some of it's features but it works for me (tm) - Dmitry Sent from my iPhone > On 30 May 2016, at 17:38, Richard Jonas wrote: > > Hey guys, > > we are implementing user registration and password checking functionality and using bcrypt for password hashing. During registration and also during login we need to compute the hashes of the passwords. The problem is that bcrypt is serializing requests even when I choose nif or port mechanism. > > https://github.com/chef/erlang-bcrypt > > With nif a message queue is instantly built when multiple users log in (compute hashes for match password). With a pool size+port implementation, the message queue building has just deferred a bit. > > Is there any alternative to creating bcrypt-like hashes? For me it seems that this library cannot be scaled well. Or have I missed something? > > -- > Richard Jonas > Erlang Solutions Hungary Kft > > Address: > Riverpark Office K.32 > K?zrakt?r street 32. 3/1. > 1093 Budapest > Hungary > Phone/fax: > +36-1-7000-654 > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Mon May 30 18:47:10 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Mon, 30 May 2016 16:47:10 +0000 Subject: [erlang-questions] "-optional_callback" and older versions Message-ID: Hi, I am writing right now a behaviour that have an `-optional_callback` but compiling this module fails obviously with oldest versions: Warning: undefined callback function myfunction/2 (behaviour 'gen_somebehviour') is there a way to ignore this warning only? - benoit -------------- next part -------------- An HTML attachment was scrubbed... URL: From emeka_1978@REDACTED Mon May 30 19:28:37 2016 From: emeka_1978@REDACTED (emeka_1978@REDACTED) Date: Mon, 30 May 2016 20:28:37 +0300 Subject: [erlang-questions] How's it going Message-ID: <0000dd77fc63$6f36deb0$88c1187b$@yahoo.com> Hey, How are you doing? I've been thinking of you recently and I guess I found something useful for you, read more here Cheers, emeka_1978@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From puzza007@REDACTED Mon May 30 23:27:03 2016 From: puzza007@REDACTED (Paul Oliver) Date: Mon, 30 May 2016 21:27:03 +0000 Subject: [erlang-questions] bcrypt message queues In-Reply-To: References: Message-ID: Hi Richard, I have a fork that uses a pool of ports. It's being used in production and has been verified to process requests concurrently. https://github.com/puzza007/erlang-bcrypt Paul. On Tue, 31 May 2016, 02:45 Richard Jonas, < richard.jonas@REDACTED> wrote: > Hey guys, > > we are implementing user registration and password checking functionality > and using bcrypt for password hashing. During registration and also during > login we need to compute the hashes of the passwords. The problem is that > bcrypt is serializing requests even when I choose nif or port mechanism. > > https://github.com/chef/erlang-bcrypt > > With nif a message queue is instantly built when multiple users log in > (compute hashes for match password). With a pool size+port implementation, > the message queue building has just deferred a bit. > > Is there any alternative to creating bcrypt-like hashes? For me it seems > that this library cannot be scaled well. Or have I missed something? > > -- > Richard Jonas > Erlang Solutions Hungary Kft > > Address: > Riverpark Office K.32 > K?zrakt?r street 32. 3/1. > 1093 Budapest > Hungary > Phone/fax: > +36-1-7000-654 > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From john.ashmun@REDACTED Mon May 30 23:49:16 2016 From: john.ashmun@REDACTED (John R. Ashmun) Date: Mon, 30 May 2016 14:49:16 -0700 Subject: [erlang-questions] Can werl be detached? In-Reply-To: References: <6284265.hgQ7dGlg6e@burrito> Message-ID: Now, running under R18.2 and Windows 10, I get a popup error message window that says "Werl Failed to execute erl: The system cannot find the file specified." unless the current directory for the command prompt is "c:\Program Files\erl7.3\erts-7.3\bin". This is really similar to the restriction that Danial Goertzen pointed out about R15 albeit with an error message about a different file that's in ...\erts-(Vsn)\bin. Could werl be changed to look in the directory near its own location rather than the current directory? Further, it appears that erl.exe and werl.exe are each duplicated between ..\erl7.3\bin and ..erl7.3\erts-7.3\bin, for what purpose I am unsure, yet if werl were able to find itself, it could certainly find erl.exe. When the current directory is set to ..\erts-7.3\bin and I enter werl -detached -run wx demo the demo runs admirably, but after I close its window, a Windows process "erl" continues to run. Doubtless this is the detached VM, but how do I explain to naive game users its continuation and the need to stop it using, e.g., Task Manager? These startup questions have led me to the init module and its boot/1 and stop/0 & stop/1 functions. I think I would like my release's init process to call stop( ) when my game stops. Is there a handy example of the use of init:stop( )? Regards, John Ashmun On Wed, May 25, 2016 at 1:22 PM, Dan Gudmundsson wrote: > Hmm, > It works fine for me on Windows 10 from msys2 with erlang in the path > and from cmd shell without erlang in the path: > > C:\Users\familjen>"c:\Program Files (x86)\erl7.3\bin\werl" -run wx demo > > C:\Users\familjen>"c:\Program Files (x86)\erl7.3\bin\werl" -detached -run > wx demo > > Oh noticed the version you are running... > > It also works for me with 17.0 i.e. erl-6.0 which is the oldest I have > installed here at home. > > Why are you using an old version of erlang? > Can you try a newer version and see if that works? > > > On Wed, May 25, 2016 at 9:49 PM John Ashmun wrote: > >> I just found time to try what you suggested with the wx demo, but I got >> the same pop up Windows error message as when I try to run my release with >> -detached. >> >> I added ...\erts-5.9\bin to %Path% and tried again but it failed again >> with the same error message. >> >> Sent from my iPhone >> >> On May 18, 2016, at 12:02 AM, Dan Gudmundsson wrote: >> >> At least this works as it should: >> >> werl -detached -run wx demo >> >> >> On Wed, May 18, 2016 at 3:48 AM John R. Ashmun >> wrote: >> >>> >>> >>> On Tue, May 17, 2016 at 6:32 PM, zxq9 wrote: >>> >>>> On Tuesday 17 May 2016 16:47:35 John R. Ashmun wrote: >>>> > Perhaps my subject is not quite the correct question. Here's my >>>> context: >>>> > >>>> > I'm beginning to recreate in Erlang on Microsoft Windows 10 a public >>>> domain >>>> > program I used to enjoy on my Commodore Amiga: mATC was a game in >>>> which >>>> > the user acted as a military Air Traffic Controller. I am using a wx >>>> > window to draw the player's map with aircraft data blocks overlaid. I >>>> > don't have a use for the initial window that opens when werl is >>>> started. >>>> >>>> Awesome! >>>> >>>> > If I use this Command Prompt batch file, mATC.bat: >>>> > >>>> > werl -pa ebin -pa mATC_app\ebin -detached -boot mATC -config sys >>>> > >>>> > a Windows error message window appears that says: >>>> > >>>> > "Failed to execute C:\Program Files\erl5.9\erts5.9\bin\beam.smp.dll >>>> > The system cannot find the file specified." >>>> > >>>> > The DLL is actually present at that location, and of course >>>> everything runs >>>> > well when I don't use -detached. >>>> >>>> Hrm... that seems odd, but I have to admit I have no experience running >>>> actual releases on Windows. What I usually do instead is have the >>>> runtime >>>> installed on the Windows machine, unpack project code and kick things >>>> off >>>> with an escript that builds and then launches the code I want to run. >>>> >>>> Changing from releases to from-source build->run may require a bit of >>>> shuffling -- and may or may not be worth it depending on the project. >>>> But this has been very effective and lightweight for my purposes. >>>> >>>> Like everything else on Windows there are a few quirks to making >>>> escripts >>>> a click-to-run experience (specifying full path to "escript.exe" instead >>>> of "escript" in your launcher, space for unheeded shebang, etc.). I've >>>> wanted for a few years now to formalize and simplify a solution to >>>> this... >>>> but real life gets in the way of the things I wish I could do as >>>> community >>>> project work. :-( >>>> >>>> The escript-to-bootstrap approach has made my Erlang GUI experience on >>>> Windows fairly painless -- I can develop on Linux or BSD and almost >>>> always >>>> get away with relying on anything that works there (to include wx) >>>> working >>>> pretty much the same way on Windows. >>>> >>>> I have not tried this on Windows 10 yet, but I assume it will continue >>>> to >>>> work the same way it does on Windows 7 and 8.1. >>>> >>>> -Craig >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> >>> I will look into escript, thanks. >>> >>> John >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From dangud@REDACTED Tue May 31 08:25:26 2016 From: dangud@REDACTED (Dan Gudmundsson) Date: Tue, 31 May 2016 06:25:26 +0000 Subject: [erlang-questions] Can werl be detached? In-Reply-To: References: <6284265.hgQ7dGlg6e@burrito> Message-ID: On Mon, May 30, 2016 at 11:49 PM John R. Ashmun wrote: > Now, running under R18.2 and Windows 10, I get a popup error message > window that says > > "Werl > > Failed to execute erl: The system cannot find the file specified." > > unless the current directory for the command prompt is "c:\Program > Files\erl7.3\erts-7.3\bin". > > This is really similar to the restriction that Danial Goertzen pointed out > about R15 albeit with an error message about a different file that's in > ...\erts-(Vsn)\bin. Could werl be changed to look in the directory near > its own location rather than the current directory? Further, it appears > that erl.exe and werl.exe are each duplicated between ..\erl7.3\bin and > ..erl7.3\erts-7.3\bin, for what purpose I am unsure, yet if werl were able > to find itself, it could certainly find erl.exe. > > When the current directory is set to ..\erts-7.3\bin and I enter > > werl -detached -run wx demo > > the demo runs admirably, but after I close its window, a Windows process > "erl" continues to run. Doubtless this is the detached VM, but how do I > explain to naive game users its continuation and the need to stop it using, > e.g., Task Manager? > > These startup questions have led me to the init module and its boot/1 and > stop/0 & stop/1 functions. I think I would like my release's init process > to call stop( ) when my game stops. Is there a handy example of the use of > init:stop( )? > > Calling erlang:halt() will halt the erlang machine, but we can not call that for obvious reasons in wx demo. > Regards, > John Ashmun > > > On Wed, May 25, 2016 at 1:22 PM, Dan Gudmundsson wrote: > >> Hmm, >> It works fine for me on Windows 10 from msys2 with erlang in the path >> and from cmd shell without erlang in the path: >> >> C:\Users\familjen>"c:\Program Files (x86)\erl7.3\bin\werl" -run wx demo >> >> C:\Users\familjen>"c:\Program Files (x86)\erl7.3\bin\werl" -detached -run >> wx demo >> >> Oh noticed the version you are running... >> >> It also works for me with 17.0 i.e. erl-6.0 which is the oldest I have >> installed here at home. >> >> Why are you using an old version of erlang? >> Can you try a newer version and see if that works? >> >> >> On Wed, May 25, 2016 at 9:49 PM John Ashmun >> wrote: >> >>> I just found time to try what you suggested with the wx demo, but I got >>> the same pop up Windows error message as when I try to run my release with >>> -detached. >>> >>> I added ...\erts-5.9\bin to %Path% and tried again but it failed again >>> with the same error message. >>> >>> Sent from my iPhone >>> >>> On May 18, 2016, at 12:02 AM, Dan Gudmundsson wrote: >>> >>> At least this works as it should: >>> >>> werl -detached -run wx demo >>> >>> >>> On Wed, May 18, 2016 at 3:48 AM John R. Ashmun >>> wrote: >>> >>>> >>>> >>>> On Tue, May 17, 2016 at 6:32 PM, zxq9 wrote: >>>> >>>>> On Tuesday 17 May 2016 16:47:35 John R. Ashmun wrote: >>>>> > Perhaps my subject is not quite the correct question. Here's my >>>>> context: >>>>> > >>>>> > I'm beginning to recreate in Erlang on Microsoft Windows 10 a public >>>>> domain >>>>> > program I used to enjoy on my Commodore Amiga: mATC was a game in >>>>> which >>>>> > the user acted as a military Air Traffic Controller. I am using a wx >>>>> > window to draw the player's map with aircraft data blocks overlaid. >>>>> I >>>>> > don't have a use for the initial window that opens when werl is >>>>> started. >>>>> >>>>> Awesome! >>>>> >>>>> > If I use this Command Prompt batch file, mATC.bat: >>>>> > >>>>> > werl -pa ebin -pa mATC_app\ebin -detached -boot mATC -config sys >>>>> > >>>>> > a Windows error message window appears that says: >>>>> > >>>>> > "Failed to execute C:\Program Files\erl5.9\erts5.9\bin\beam.smp.dll >>>>> > The system cannot find the file specified." >>>>> > >>>>> > The DLL is actually present at that location, and of course >>>>> everything runs >>>>> > well when I don't use -detached. >>>>> >>>>> Hrm... that seems odd, but I have to admit I have no experience running >>>>> actual releases on Windows. What I usually do instead is have the >>>>> runtime >>>>> installed on the Windows machine, unpack project code and kick things >>>>> off >>>>> with an escript that builds and then launches the code I want to run. >>>>> >>>>> Changing from releases to from-source build->run may require a bit of >>>>> shuffling -- and may or may not be worth it depending on the project. >>>>> But this has been very effective and lightweight for my purposes. >>>>> >>>>> Like everything else on Windows there are a few quirks to making >>>>> escripts >>>>> a click-to-run experience (specifying full path to "escript.exe" >>>>> instead >>>>> of "escript" in your launcher, space for unheeded shebang, etc.). I've >>>>> wanted for a few years now to formalize and simplify a solution to >>>>> this... >>>>> but real life gets in the way of the things I wish I could do as >>>>> community >>>>> project work. :-( >>>>> >>>>> The escript-to-bootstrap approach has made my Erlang GUI experience on >>>>> Windows fairly painless -- I can develop on Linux or BSD and almost >>>>> always >>>>> get away with relying on anything that works there (to include wx) >>>>> working >>>>> pretty much the same way on Windows. >>>>> >>>>> I have not tried this on Windows 10 yet, but I assume it will continue >>>>> to >>>>> work the same way it does on Windows 7 and 8.1. >>>>> >>>>> -Craig >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>> >>>> I will look into escript, thanks. >>>> >>>> John >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Tue May 31 10:56:05 2016 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Tue, 31 May 2016 10:56:05 +0200 Subject: [erlang-questions] bcrypt message queues In-Reply-To: References: Message-ID: <574D51A5.8000106@ericsson.com> How heavy duty is this bcrypt? Does it take milliseconds to hash one little password? /Sverker On 05/30/2016 04:38 PM, Richard Jonas wrote: > Hey guys, > > we are implementing user registration and password checking functionality > and using bcrypt for password hashing. During registration and also during > login we need to compute the hashes of the passwords. The problem is that > bcrypt is serializing requests even when I choose nif or port mechanism. > > https://github.com/chef/erlang-bcrypt > > With nif a message queue is instantly built when multiple users log in > (compute hashes for match password). With a pool size+port implementation, > the message queue building has just deferred a bit. > > Is there any alternative to creating bcrypt-like hashes? For me it seems > that this library cannot be scaled well. Or have I missed something? > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From technion@REDACTED Tue May 31 11:30:37 2016 From: technion@REDACTED (Technion) Date: Tue, 31 May 2016 09:30:37 +0000 Subject: [erlang-questions] bcrypt message queues In-Reply-To: <574D51A5.8000106@ericsson.com> References: , <574D51A5.8000106@ericsson.com> Message-ID: Hi, If it takes less than "milliseconds", it's broken. Standard recommendations are 50ms. ________________________________ From: erlang-questions-bounces@REDACTED on behalf of Sverker Eriksson Sent: Tuesday, 31 May 2016 6:56:05 PM To: Richard Jonas Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] bcrypt message queues How heavy duty is this bcrypt? Does it take milliseconds to hash one little password? /Sverker On 05/30/2016 04:38 PM, Richard Jonas wrote: Hey guys, we are implementing user registration and password checking functionality and using bcrypt for password hashing. During registration and also during login we need to compute the hashes of the passwords. The problem is that bcrypt is serializing requests even when I choose nif or port mechanism. https://github.com/chef/erlang-bcrypt With nif a message queue is instantly built when multiple users log in (compute hashes for match password). With a pool size+port implementation, the message queue building has just deferred a bit. Is there any alternative to creating bcrypt-like hashes? For me it seems that this library cannot be scaled well. Or have I missed something? _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From richard.jonas@REDACTED Tue May 31 11:30:53 2016 From: richard.jonas@REDACTED (Richard Jonas) Date: Tue, 31 May 2016 11:30:53 +0200 Subject: [erlang-questions] bcrypt message queues In-Reply-To: <574D51A5.8000106@ericsson.com> References: <574D51A5.8000106@ericsson.com> Message-ID: I load tested with a vm (so the numbers are not relevant but the ratios). Oliver's fork spawns as many bcrypt_port (as a pool of ports) as many schedulers we have. Makes sense. The chef/erlang-bcrypt can spawn as many as we want (pool_size config key). I used round = 12, which took 42,000-60,000ms if we spawned 100 of workers on a single core (ok, bad idea, but load testing is such). With round = 4, it took 400-2100ms (median is 700ms) with the same number of workers. @Sverker: 1 password from shell took 5ms to hash. If I have time and possibility I embed bcrypt library into basho bench, and load test bcrypt that way. Because right now I load test it via cowboy rest api. So with round 4 we can say that other components will be the bottleneck, not the bcrypt. On Tue, May 31, 2016 at 10:56 AM, Sverker Eriksson < sverker.eriksson@REDACTED> wrote: > How heavy duty is this bcrypt? > Does it take milliseconds to hash one little password? > > /Sverker > > > On 05/30/2016 04:38 PM, Richard Jonas wrote: > > Hey guys, > > we are implementing user registration and password checking functionality > and using bcrypt for password hashing. During registration and also during > login we need to compute the hashes of the passwords. The problem is that > bcrypt is serializing requests even when I choose nif or port mechanism. > https://github.com/chef/erlang-bcrypt > > With nif a message queue is instantly built when multiple users log in > (compute hashes for match password). With a pool size+port implementation, > the message queue building has just deferred a bit. > > Is there any alternative to creating bcrypt-like hashes? For me it seems > that this library cannot be scaled well. Or have I missed something? > > > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > -- Richard Jonas Erlang Solutions Hungary Kft Address: Riverpark Office K.32 K?zrakt?r street 32. 3/1. 1093 Budapest Hungary Phone/fax: +36-1-7000-654 -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang.org@REDACTED Tue May 31 11:31:25 2016 From: erlang.org@REDACTED (Stanislaw Klekot) Date: Tue, 31 May 2016 11:31:25 +0200 Subject: [erlang-questions] bcrypt message queues In-Reply-To: <574D51A5.8000106@ericsson.com> References: <574D51A5.8000106@ericsson.com> Message-ID: <20160531093125.GA28976@jarowit.net> On Tue, May 31, 2016 at 10:56:05AM +0200, Sverker Eriksson wrote: > How heavy duty is this bcrypt? > Does it take milliseconds to hash one little password? Password hashes are *designed* to take long time to compute (e.g. a semi-traditional MD5-based crypt() is MD5 hash applied 1000 times, each time to the result of previous computation). This is to make precomputation attacks so much more costly. Having that said, https://github.com/chef/erlang-bcrypt has its internals sub-par, as it runs two processes (one for NIF and one for port driver; why port driver? there's no state to maintain between calls as far as I'm aware), and NIF is always called in one. This makes a great example of an unnecessary bottleneck. -- Stanislaw Klekot From bengt.kleberg@REDACTED Tue May 31 16:34:53 2016 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 31 May 2016 16:34:53 +0200 Subject: [erlang-questions] erlfmt? In-Reply-To: References: <1461797208.3148043.591715937.1FAE1C3E@webmail.messagingengine.com> <57429400.3060409@ericsson.com> Message-ID: <574DA10D.6030105@ericsson.com> There is a new version of the string_io module that handles the new io_request format (a unicode atom was added sometime). With this I think bepp (the formatter) now works. Note: to use bepp with '<', call stdin/0. bengt On 05/24/2016 09:35 PM, Mark Bucciarelli wrote: > Hi, > > I didn't look at that as I didn't find it before. > > I tried it, but only got comments out. Maybe I'm not using it right? > > ~/src/erlang_stdin_formatter/src$ cat erlfmt > > #!/usr/bin/env escript > > main([]) -> > > bepp:main([prog]). > > ~/src/erlang_stdin_formatter/src$ ./erlfmt < a_program.erl > > > Thanks, > > > Mark > > > On Mon, May 23, 2016 at 1:24 AM, Bengt Kleberg > > wrote: > > Greetings, > > If you looked at https://github.com/ebengt/erlang_stdin_formatter > I would be interested in suggestions as to where it was not > sufficient. > > > Bengt > > > On 05/23/2016 03:11 AM, Mark Bucciarelli wrote: >> Thanks for the suggestions! >> >> I took a look at each, but none of the three options really gave >> me what I wanted. So I ended up wrapping erl_tidy in a way that >> gives me what I want: >> >> :%!erlfmt >> >> Here's the source, such as it is: https://github.com/mbucc/erlfmt. >> >> It was able to format 1,794 out of 1,795 source files under >> /usr/local/Cellar/erlang/18.2.1. It failed to >> format lib/erlang/lib/wx-1.6/src/gen/gl.erl, which is 971KB in size. >> >> Mark >> >> On Thu, Apr 28, 2016 at 3:09 AM, Roger Lipscombe >> > wrote: >> >> I use vimerl (https://github.com/jimenezrick/vimerl), which >> has an >> escript included: >> https://github.com/jimenezrick/vimerl/blob/master/indent/erlang_indent.erl >> >> I *believe* that it's equivalent to Emacs erlang-mode, but I >> don't >> have Emacs installed :-P >> >> On 28 April 2016 at 07:24, Pierre Fenoll >> > wrote: >> > I made https://github.com/fenollp/erlang-formatter though >> it depends on >> > OTP's Emacs erlang-mode. >> > >> > erlang-mode has been the de facto authority in terms of >> linting. >> > However it has some issues, like when indenting typed >> record definitions. >> > >> > Dropping the dependency on Emacs would be great but again, >> which dev/CI >> > machine doesn't have Emacs? >> > >> > On 28 Apr 2016, at 00:46, Tristan Sloughter >> > wrote: >> > >> > Short answer: No. >> > >> > Long answer: No, but I'd love for someone with the time to >> clean up >> > https://github.com/tsloughter/erl_tidy which is a rebar3 >> plugin named `fmt` >> > around erl_tidy. the erl_tidy code has issues with type >> specs and other >> > newer syntax and will currently just dump out the AST for >> those. These fixes >> > would then, of course, need to be submitted upstream to >> OTP, but this plugin >> > provides a testing ground and hopefully motivation to >> someone out there :) >> > >> > -- >> > Tristan Sloughter >> > t@REDACTED >> > >> > >> > >> > On Wed, Apr 27, 2016, at 05:37 PM, Mark Bucciarelli wrote: >> > >> > Hi, >> > >> > I'm wondering if anyone here uses or knows of a utility >> like gofmt for >> > Erlang. If you're not familiar with gofmt, it simply reads >> source code on >> > stdin and writes formatted source code to stdout. >> > >> > I looked at erl_tidy but cannot see how to make it read stdin. >> > >> > Thanks, >> > >> > Mark >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: