From mononcqc@REDACTED Sat Sep 1 04:15:08 2018 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 31 Aug 2018 22:15:08 -0400 Subject: [erlang-questions] Unidirectional architectures in Erlang In-Reply-To: <20180831183629.GB4854@circlewave.net> References: <20180830233307.GE801@ferdmbp.local> <20180831183629.GB4854@circlewave.net> Message-ID: <20180901021507.GI801@ferdmbp.local> On 08/31, Jachym Holecek wrote: ># Fred Hebert 2018-08-30: >> This lets you test each section of the chain in isolation; then if all >> sections are well-behaved, you can infer that the whole chain should behave >> well. > >That sounds like a very dangerous conclusion to be making, personally. > >It's not unheard of that thousands of unit-tests "pass" and everything is >"ready" and then it all falls like a house of cards with a single naive >end-to-end flow. ;-) > Yes, this is why the original quote also strongly suggested having at least one end-to-end pipeline on top of it. It's just that you may have fewer details to worry about in the end-to-end pipeline when what you have to focus on is "is it all plumbed together right" rather than "are all elements of the whole chain correct and also plumbed together right". This is about scoping your tests, and defining which behaviours and properties of your code you want to validate. You'd want the unit tests to validate the minutiae of each step, but the integration suite to check that they're wired up correctly, for example. I've worked and seen lots of integration and system tests that, through obtuse combination of inputs, exercised very narrow behaviour of deeply-entrenched bits of code. Those tests are generally slow to set up, very fragile, and rarely deterministic. From joe@REDACTED Sat Sep 1 22:33:01 2018 From: joe@REDACTED (joe mcguckin) Date: Sat, 1 Sep 2018 13:33:01 -0700 Subject: [erlang-questions] BEAM in hardware Message-ID: <530ABE9C-1A6A-41DD-A21F-B6E1AEBE4D5F@via.net> Does anyone know if the BEAM vm has ever been implemented directly in hardware (e.g. in an FPGA)? Thanks, Joe Joe McGuckin ViaNet Communications joe@REDACTED 650-207-0372 cell 650-213-1302 office 650-969-2124 fax From joe@REDACTED Sat Sep 1 22:34:41 2018 From: joe@REDACTED (joe mcguckin) Date: Sat, 1 Sep 2018 13:34:41 -0700 Subject: [erlang-questions] Erlang performance for network VPN/firewall tasks? Message-ID: <806807B6-AF0B-47C7-B8E2-F0FDB6CC4583@via.net> Would ERLANG have good performance for implementing things like NAT or VPN?s ? Thanks, Joe Joe McGuckin ViaNet Communications joe@REDACTED 650-207-0372 cell 650-213-1302 office 650-969-2124 fax From frank.muller.erl@REDACTED Sat Sep 1 22:53:31 2018 From: frank.muller.erl@REDACTED (Frank Muller) Date: Sat, 1 Sep 2018 22:53:31 +0200 Subject: [erlang-questions] =?utf-8?q?ETS=E2=80=99s_performances_and_limit?= =?utf-8?q?s=3F?= Message-ID: Hi, Is ETS capable of handling 1 million key/value (or more)? My keys are 36B and all my values are local PIDs. My ETS is mostly read-only!!! What?s the largest ETS one has to deal with? Is there any limitations or point when the ETS?s perf degrades? I?m on CentOS7, Erlang 20.x & 21.x with 256GB of RAM. /Frank -------------- next part -------------- An HTML attachment was scrubbed... URL: From ameretat.reith@REDACTED Sat Sep 1 23:11:42 2018 From: ameretat.reith@REDACTED (Ameretat Reith) Date: Sun, 2 Sep 2018 01:41:42 +0430 Subject: [erlang-questions] Erlang performance for network VPN/firewall tasks? In-Reply-To: <806807B6-AF0B-47C7-B8E2-F0FDB6CC4583@via.net> References: <806807B6-AF0B-47C7-B8E2-F0FDB6CC4583@via.net> Message-ID: I don't know for NAT, but for VPN we managed to reach 700Mbps using tunctl and enacl poly1305-xsalsa20 on standard Xeon E3 servers. Good point is you will have great concurrency and your server won't break (as IPSec do) on high number of users, bad point is you have no control over some low-level optimizations or may be throttled by erlang itself; as it happened for upload speed to us because of UDP receive problems been talked recently in list. From dmytro.lytovchenko@REDACTED Sat Sep 1 23:06:59 2018 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Sat, 1 Sep 2018 23:06:59 +0200 Subject: [erlang-questions] BEAM in hardware In-Reply-To: <530ABE9C-1A6A-41DD-A21F-B6E1AEBE4D5F@via.net> References: <530ABE9C-1A6A-41DD-A21F-B6E1AEBE4D5F@via.net> Message-ID: There is no known hardware implementation because the original BEAM 158-instruction set is quite complex to load, parse and interpret, and the runtime part of hardware to support built-ins and data types would be pretty massive. But it might be possible if you simplify the opcode set, simplify the memory structure (to save on bit manipulations) similar to Python or Java memory model, and shrink it to something that is possible to hard-wire. So yes, some _hobby_ work on that was done by me without a significant result: it turned out to be massive amount of work, without a commercial project, and without any particular goal - it becomes really hard to focus on the topic and to keep going. Essentially I worked on compressed instructions and reduced instruction set (here https://github.com/kvakvs/E4VM ), after some success I left the project, tried implementing a normal BEAM VM in software ( https://github.com/kvakvs/ErlangRT ) and now trying another one for simplified memory model and compiling to machine code ( https://github.com/kvakvs/ErlangAoT ). On Sat, 1 Sep 2018 at 22:45, joe mcguckin wrote: > Does anyone know if the BEAM vm has ever been implemented directly in > hardware (e.g. in an FPGA)? > > Thanks, > > Joe > > > Joe McGuckin > ViaNet Communications > > joe@REDACTED > 650-207-0372 cell > 650-213-1302 office > 650-969-2124 fax > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthonym@REDACTED Sat Sep 1 23:09:29 2018 From: anthonym@REDACTED (ANTHONY MOLINARO) Date: Sat, 1 Sep 2018 14:09:29 -0700 Subject: [erlang-questions] =?utf-8?q?ETS=E2=80=99s_performances_and_limit?= =?utf-8?q?s=3F?= In-Reply-To: References: Message-ID: <393051E5-6357-4250-BA85-70BD55CFE7C1@alumni.caltech.edu> Hi, The simple answer is yes, but there?s some complications under scale. I?ve got a few ordered_sets with ~12 million entries in them which seem to work reasonably well under normal loads, but they tend to show abnormal performance under heavier concurrency. This is a known limitation which AFAIK has not been addressed yet (https://www.youtube.com/watch?v=40shUKSdh1A ) in the implementation of ordered_sets (I?d love to be told differently, but looking at the code it does not appear to have been changed in some time). I don?t believe that sets have the same issue. In all likelihood things will work fine for your use case, and you can test it out easily enough. HTH, -Anthony > On Sep 1, 2018, at 1:53 PM, Frank Muller wrote: > > Hi, > > Is ETS capable of handling 1 million key/value (or more)? My keys are 36B and all my values are local PIDs. My ETS is mostly read-only!!! > > What?s the largest ETS one has to deal with? > Is there any limitations or point when the ETS?s perf degrades? > > I?m on CentOS7, Erlang 20.x & 21.x with 256GB of RAM. > > /Frank > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfidelman@REDACTED Sat Sep 1 23:43:30 2018 From: mfidelman@REDACTED (Miles Fidelman) Date: Sat, 1 Sep 2018 17:43:30 -0400 Subject: [erlang-questions] BEAM in hardware In-Reply-To: References: <530ABE9C-1A6A-41DD-A21F-B6E1AEBE4D5F@via.net> Message-ID: On 9/1/18 5:06 PM, Dmytro Lytovchenko wrote: > There is no known hardware implementation because the original BEAM > 158-instruction set is quite complex to load, parse and interpret, and > the runtime part of hardware to support built-ins and data types would > be pretty massive. But it might be possible if you simplify the opcode > set, simplify the memory structure (to save on bit manipulations) > similar to Python or Java memory model, and shrink it to something > that is possible to hard-wire. Is it really any more complex than, implementing a CISC processor, or something funky like a GPU or the old BBN Butterfly?? For that matter, the old DG Nova had a pretty funky instruction set - looked a lot more like microcode than a traditional ISA. What about if you allow for microcode?? I remember working on microcode to extend a MIL-STD-1750 processor, to do pulse train manipulation, for electronic warfare applications (back in the day when 4mips was blinding fast).? Or what about the LISP machines? It doesn't seem out of the range of possibility - given a sufficiently resourced team.? Now whether it makes economic sense, is a separate question. Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra From scott_ribe@REDACTED Sun Sep 2 00:08:31 2018 From: scott_ribe@REDACTED (Scott Ribe) Date: Sat, 1 Sep 2018 16:08:31 -0600 Subject: [erlang-questions] BEAM in hardware In-Reply-To: References: <530ABE9C-1A6A-41DD-A21F-B6E1AEBE4D5F@via.net> Message-ID: > On Sep 1, 2018, at 3:43 PM, Miles Fidelman wrote: > > Or what about the LISP machines? Exactly. It could be done, but it makes no sense because of the economies of scale behind "standard architecture" CPUs. From dmytro.lytovchenko@REDACTED Sun Sep 2 00:30:14 2018 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Sun, 2 Sep 2018 00:30:14 +0200 Subject: [erlang-questions] BEAM in hardware In-Reply-To: References: <530ABE9C-1A6A-41DD-A21F-B6E1AEBE4D5F@via.net> Message-ID: On Sat, 1 Sep 2018 at 23:44, Miles Fidelman wrote: > > > On 9/1/18 5:06 PM, Dmytro Lytovchenko wrote: > > There is no known hardware implementation because the original BEAM > > 158-instruction set is quite complex to load, parse and interpret, and > > the runtime part of hardware to support built-ins and data types would > > be pretty massive. But it might be possible if you simplify the opcode > > set, simplify the memory structure (to save on bit manipulations) > > similar to Python or Java memory model, and shrink it to something > > that is possible to hard-wire. > > Is it really any more complex than, implementing a CISC processor, or > something funky like a GPU or the old BBN Butterfly? For that matter, > the old DG Nova had a pretty funky instruction set - looked a lot more > like microcode than a traditional ISA. > > That is my point. Of course anything that is implemented can be re-implemented, but there are resource limits to this complexity. To a large team with experience in chip design and production or even FPGA experience this might be totally possible, like "we just finished designing this beautiful Core i9 chip let's now take a break for a year and make a BEAM chip". But in real life such teams are rare, and I'm thinking of 1-3 person team and a _very_ limited budget, there's never enough time or working hands. Here to succeed one must take a lot of simplifications and considerations. > What about if you allow for microcode? I remember working on microcode > to extend a MIL-STD-1750 processor, to do pulse train manipulation, for > electronic warfare applications (back in the day when 4mips was blinding > fast). Or what about the LISP machines? > > It doesn't seem out of the range of possibility - given a sufficiently > resourced team. Now whether it makes economic sense, is a separate > question. > Making these two meet: The price of implementation vs. the available resources. > > Miles Fidelman > > > -- > In theory, there is no difference between theory and practice. > In practice, there is. .... Yogi Berra > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mfidelman@REDACTED Sun Sep 2 00:56:14 2018 From: mfidelman@REDACTED (Miles Fidelman) Date: Sat, 1 Sep 2018 18:56:14 -0400 Subject: [erlang-questions] BEAM in hardware In-Reply-To: References: <530ABE9C-1A6A-41DD-A21F-B6E1AEBE4D5F@via.net> Message-ID: <08ea46d4-77ab-c860-e520-931d3b9b0146@meetinghouse.net> On 9/1/18 6:30 PM, Dmytro Lytovchenko wrote: > > > On Sat, 1 Sep 2018 at 23:44, Miles Fidelman > > wrote: > > > > On 9/1/18 5:06 PM, Dmytro Lytovchenko wrote: > > There is no known hardware implementation because the original BEAM > > 158-instruction set is quite complex to load, parse and > interpret, and > > the runtime part of hardware to support built-ins and data types > would > > be pretty massive. But it might be possible if you simplify the > opcode > > set, simplify the memory structure (to save on bit manipulations) > > similar to Python or Java memory model, and shrink it to something > > that is possible to hard-wire. > > Is it really any more complex than, implementing a CISC processor, or > something funky like a GPU or the old BBN Butterfly?? For that > matter, > the old DG Nova had a pretty funky instruction set - looked a lot > more > like microcode than a traditional ISA. > > That is my point. Of course anything that is implemented can be > re-implemented, but there are resource limits to this complexity. To a > large team with experience in chip design and production or even FPGA > experience this might be totally possible, like "we just finished > designing this beautiful Core i9 chip let's now take a break for a > year and make a BEAM chip". But in real life such teams are rare, and > I'm thinking of 1-3 person team and a _very_ limited budget, there's > never enough time or working hands. Here to succeed one must take a > lot of simplifications and considerations. Of course, in the old days - when we were talking ECL chips, and wire-wrapped prototypes, a 2-3 person team could do a LOT.? (Back in the early 1980s, I was on a team that designed & built high-performance avionic computers.? We had a number of small projects, funded by internal IR&D money, and some small research grants from the Air Force. These days, when we're talking building chips, it's another story. Though... I wonder what could be accomplished if one started with a chip that supported writeable microcode - like a Core2.? Hmmm..... > What about if you allow for microcode?? I remember working on > microcode > to extend a MIL-STD-1750 processor, to do pulse train > manipulation, for > electronic warfare applications (back in the day when 4mips was > blinding > fast).? Or what about the LISP machines? > > It doesn't seem out of the range of possibility - given a > sufficiently > resourced team.? Now whether it makes economic sense, is a separate > question. > > > Making these two meet: The price of implementation vs. the available > resources. Might make a good PhD project, at a university that has access to military R&D funds. Might also be worthwhile to a company that needs high-performance, massively concurrent processing. Miles -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra -------------- next part -------------- An HTML attachment was scrubbed... URL: From v@REDACTED Sun Sep 2 01:29:36 2018 From: v@REDACTED (Valentin Micic) Date: Sun, 2 Sep 2018 01:29:36 +0200 Subject: [erlang-questions] =?windows-1252?q?ETS=92s_performances_and_limi?= =?windows-1252?q?ts=3F?= In-Reply-To: <393051E5-6357-4250-BA85-70BD55CFE7C1@alumni.caltech.edu> References: <393051E5-6357-4250-BA85-70BD55CFE7C1@alumni.caltech.edu> Message-ID: <6D456A9E-C872-4BB1-9BE3-78E0F8BFDD47@pharos-corp.com> On 01 Sep 2018, at 11:09 PM, ANTHONY MOLINARO wrote: > > I?ve got a few ordered_sets with ~12 million entries in them which seem to work reasonably well under normal loads, but they tend to show abnormal performance under heavier concurrency. > There is a good chance that performance degradation you are referring to may *not* be related to a "heavier concurrency" as much as it may be caused by a rate of inserts into a table of type "ordered_set". The manual indicates that ordered_set tables do behave differently "in come situations" (which I read as "insert operations when a table gets big"). Out of interest, could you please quantify "heavier concurrency" in terms of a number of processes running and interacting with the table? My experience is with an ETS table of type "set", hosting more than 190 million entries, and a steady insert rate (which may be updating existing, but also adding new entries) of 200 entries per second, with concurrent lookups from more than a hundred short-lived processes at any point in time (well, rather, most of the time). Apart from requiring some time to load all that data from disk to the ETS memory (System Start-Up), and needing some time to release all the memory upon ETS table destruction (System Shutdown), the performance is quite consistent. Kind regards V/ From ilya.khlopotov@REDACTED Sun Sep 2 04:34:45 2018 From: ilya.khlopotov@REDACTED (ILYA) Date: Sat, 01 Sep 2018 19:34:45 -0700 Subject: [erlang-questions] BEAM in hardware In-Reply-To: <530ABE9C-1A6A-41DD-A21F-B6E1AEBE4D5F@via.net> References: <530ABE9C-1A6A-41DD-A21F-B6E1AEBE4D5F@via.net> Message-ID: There was a project from Ericsson http://www.presentica.com/ppt-presentation/ecomp-an-erlang-processor BR, iilyak On September 1, 2018 1:33:01 PM PDT, joe mcguckin wrote: >Does anyone know if the BEAM vm has ever been implemented directly in >hardware (e.g. in an FPGA)? > >Thanks, > >Joe > > >Joe McGuckin >ViaNet Communications > >joe@REDACTED >650-207-0372 cell >650-213-1302 office >650-969-2124 fax > > > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Sun Sep 2 06:38:49 2018 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 1 Sep 2018 21:38:49 -0700 Subject: [erlang-questions] BEAM in hardware In-Reply-To: References: <530ABE9C-1A6A-41DD-A21F-B6E1AEBE4D5F@via.net> Message-ID: <461bad4b-3a23-1cc1-bdd7-7f1aeaafb66c@gmail.com> On 09/01/2018 07:34 PM, ILYA wrote: > There was a project from Ericsson > http://www.presentica.com/ppt-presentation/ecomp-an-erlang-processor That presentation was from the Erlang User Conference in 2000. Since then it has become related to two papers at: https://www.researchgate.net/publication/220760571 https://www.researchgate.net/publication/321538810 Best Regards, Michael From carlsson.richard@REDACTED Sun Sep 2 10:36:32 2018 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Sun, 2 Sep 2018 10:36:32 +0200 Subject: [erlang-questions] BEAM in hardware In-Reply-To: <530ABE9C-1A6A-41DD-A21F-B6E1AEBE4D5F@via.net> References: <530ABE9C-1A6A-41DD-A21F-B6E1AEBE4D5F@via.net> Message-ID: Not for the entire Beam, but here's a recent small scale attempt at going from Erlang down to Verilog: https://www.researchgate.net/publication/319364614_Distributed_memory_architecture_for_high-level_synthesis_of_embedded_controllers_from_Erlang /Richard Den l?r 1 sep. 2018 kl 22:45 skrev joe mcguckin : > Does anyone know if the BEAM vm has ever been implemented directly in > hardware (e.g. in an FPGA)? > > Thanks, > > Joe > > > Joe McGuckin > ViaNet Communications > > joe@REDACTED > 650-207-0372 cell > 650-213-1302 office > 650-969-2124 fax > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Sun Sep 2 11:52:14 2018 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Sun, 2 Sep 2018 11:52:14 +0200 Subject: [erlang-questions] =?utf-8?q?ETS=E2=80=99s_performances_and_limit?= =?utf-8?q?s=3F?= In-Reply-To: References: Message-ID: On Sat, Sep 1, 2018 at 10:53 PM, Frank Muller wrote: > Hi, > > Is ETS capable of handling 1 million key/value (or more)? My keys are 36B > and all my values are local PIDs. My ETS is mostly read-only!!! Yes, that's tiny. Using integers as keys would be best. > What?s the largest ETS one has to deal with? You're only limited by available RAM. We routinely handle tables with several 100s of million records in each, and we have many such tables in each node. > Is there any limitations or point when the ETS?s perf degrades? ETS tables with complex keys can degrade drastically under high insertion rates, but are Ok when mostly read with with occasional (say << 1k/s) insertions. > I?m on CentOS7, Erlang 20.x & 21.x with 256GB of RAM. You may want to disable transparent hugepages. There's also a default limit to the number of individual memory mappings per Linux process that you might run into with larger nodes, we did and it was a nightmare because it caused random unexpected VM deaths with no diagnostics of the real culprit. (Solution: bump the kernel's max_map_count, but be advised the limit is there because some memory map operations in the kernel don't scale well to large address spaces). /Mikael From mikpelinux@REDACTED Sun Sep 2 12:17:49 2018 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Sun, 2 Sep 2018 12:17:49 +0200 Subject: [erlang-questions] BEAM in hardware In-Reply-To: <530ABE9C-1A6A-41DD-A21F-B6E1AEBE4D5F@via.net> References: <530ABE9C-1A6A-41DD-A21F-B6E1AEBE4D5F@via.net> Message-ID: On Sat, Sep 1, 2018 at 10:33 PM, joe mcguckin wrote: > Does anyone know if the BEAM vm has ever been implemented directly in hardware (e.g. in an FPGA)? Not the BEAM that I know of, but Ericsson has had at least one project to implement an "Erlang processor" (via different presumably simpler VMs). Apart from the usual problems of synthesizing a processor, you also have to consider: - there's a lot of implicit state and semantics in the BEAM instructions, so you'd have to duplicate things like processes, dynamic memory allocation, garbage collection, exception handling, message passing, process links, etc - the BIFs, without which you can't do much, at a minimum you'd need networking support Microcoded HW implementations of LISP and Smalltalk were popular in the 70s and early 80s, but were quickly replaced by SW running on RISCs with higher performance, lower cost, and quicker updates. I'm not sure for what use case a HW Erlang makes sense (except as an intellectual exercise). From mfrench@REDACTED Sun Sep 2 12:52:00 2018 From: mfrench@REDACTED (Mike French) Date: Sun, 2 Sep 2018 10:52:00 +0000 Subject: [erlang-questions] BEAM in hardware In-Reply-To: References: <530ABE9C-1A6A-41DD-A21F-B6E1AEBE4D5F@via.net> Message-ID: It probably makes more sense to do a software implementation on a platform that is somewhat isomorphic to the Erlang abstractions of independent processes and message passing - maybe VxWorks. Whatever happened to the Parallella port? Mike -----Original Message----- From: erlang-questions-bounces@REDACTED [mailto:erlang-questions-bounces@REDACTED] On Behalf Of Mikael Pettersson Sent: Sunday, September 02, 2018 1:18 PM To: joe mcguckin Cc: erlang-questions Subject: Re: [erlang-questions] BEAM in hardware On Sat, Sep 1, 2018 at 10:33 PM, joe mcguckin wrote: > Does anyone know if the BEAM vm has ever been implemented directly in hardware (e.g. in an FPGA)? Not the BEAM that I know of, but Ericsson has had at least one project to implement an "Erlang processor" (via different presumably simpler VMs). Apart from the usual problems of synthesizing a processor, you also have to consider: - there's a lot of implicit state and semantics in the BEAM instructions, so you'd have to duplicate things like processes, dynamic memory allocation, garbage collection, exception handling, message passing, process links, etc - the BIFs, without which you can't do much, at a minimum you'd need networking support Microcoded HW implementations of LISP and Smalltalk were popular in the 70s and early 80s, but were quickly replaced by SW running on RISCs with higher performance, lower cost, and quicker updates. I'm not sure for what use case a HW Erlang makes sense (except as an intellectual exercise). _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From sigmastar@REDACTED Sun Sep 2 14:26:35 2018 From: sigmastar@REDACTED (Jesse Gumm) Date: Sun, 2 Sep 2018 07:26:35 -0500 Subject: [erlang-questions] ANN: erlPress_core V. 0.01 In-Reply-To: <1535647696.61235260@apps.rackspace.com> References: <1535647696.61235260@apps.rackspace.com> Message-ID: Way to go, Lloyd! I know you've been working your tail off to get this out, so I'm glad to see it finally make its way out. Cheers, dude! -Jesse On Thu, Aug 30, 2018, 11:48 AM wrote: > Generate PDFs from your Erlang applications. > > GitHub: writersglen:erlPress_core > > MIT license. > > erlPress_core modifies and extends the Hugh Watkins fork of Erlang > Erlguten originally developed by Joe Armstrong. > > Joe Armstrong set out an ambitious goal: "Better than TeX." > > http://erlang.org/euc/04/erlguten.pdf > > But development flagged. > > erlPress_core picks up the baton. With your creative and technical help, > our aspiration is to bring Joe's vision to reality and beyond. > > We see erlPress_core as the base for many as yet-to-be-developed Erlang > applications for creative print media publishing tools. > > To see current functionality, display .../pdf/galleys/ep_show_n_tell.pdf. > For source see ../src/tests/ep_show_n_tell.erl. > > erlPress_core is not yet the polished gem we envision. Your help in > buffing up rough edges and fleshing out functionality is heartily welcome. > > Among other things, Erlguten source contains still more hidden gems than > we've been able to dig out for lack of technical understanding or time. > We'd love to integrate them into erlPress_core. > > High on our help-wanted list: typefaces in erlPress_core are currently > limited to PostScript Type I. Support for OTF, TTF, and/or maybe TeX fonts > rank high on our road map. Your help here would be welcomed and much > appreciated. We have ideas how this can be done if you're interested. > > And more... we look forward to the creative applications that > erlPress_core will enable you to build. > > P.S. Thanks to Frank Muller and others who have encouraged and supported > development of erlPress_core. > > > > > > > ********************************************* > My books: > > THE GOSPEL OF ASHES > http://thegospelofashes.com > > Strength is not enough. Do they have the courage > and the cunning? Can they survive long enough to > save the lives of millions? > > FREEIN' PANCHO > http://freeinpancho.com > > A community of misfits help a troubled boy find his way > > AYA TAKEO > http://ayatakeo.com > > Star-crossed love, war and power in an alternative > universe > > Available through Amazon or by request from your > favorite bookstore > > > ********************************************** > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Sun Sep 2 15:57:46 2018 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Sun, 2 Sep 2018 09:57:46 -0400 Subject: [erlang-questions] BEAM in hardware In-Reply-To: References: <530ABE9C-1A6A-41DD-A21F-B6E1AEBE4D5F@via.net> Message-ID: <4EA8DA14-8672-4B8C-9A56-55241E9EA9C3@writersglen.com> Hi All, Some 30-odd years ago Chuck Moore, developer of Forth, turned his attention to chip design. He wrote his own chip-design CAD system in Forth and, subsequently designed and brought to silicon an impressive list of successful chips: https://en.m.wikipedia.org/wiki/Charles_H._Moore Chuck?s latest inspirations can be seen here: http://www.greenarraychips.com/ The key to Chuck?s genius: Pare the problem-at-hand down to it?s essence; seek starkest simplicity. This has kept him out of step with the computing mainstream throughout this career. Best wishes, LRP Sent from my iPad > On Sep 2, 2018, at 6:17 AM, Mikael Pettersson wrote: > >> On Sat, Sep 1, 2018 at 10:33 PM, joe mcguckin wrote: >> Does anyone know if the BEAM vm has ever been implemented directly in hardware (e.g. in an FPGA)? > > Not the BEAM that I know of, but Ericsson has had at least one project > to implement an "Erlang processor" (via different presumably simpler > VMs). > > Apart from the usual problems of synthesizing a processor, you also > have to consider: > - there's a lot of implicit state and semantics in the BEAM > instructions, so you'd have to duplicate things like processes, > dynamic memory allocation, garbage collection, exception handling, > message passing, process links, etc > - the BIFs, without which you can't do much, at a minimum you'd need > networking support > > Microcoded HW implementations of LISP and Smalltalk were popular in > the 70s and early 80s, but were quickly replaced by SW running on > RISCs with higher performance, lower cost, and quicker updates. > > I'm not sure for what use case a HW Erlang makes sense (except as an > intellectual exercise). > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From joe@REDACTED Sun Sep 2 21:58:33 2018 From: joe@REDACTED (Joe McGuckin) Date: Sun, 2 Sep 2018 12:58:33 -0700 Subject: [erlang-questions] BEAM in hardware In-Reply-To: <4EA8DA14-8672-4B8C-9A56-55241E9EA9C3@writersglen.com> References: <530ABE9C-1A6A-41DD-A21F-B6E1AEBE4D5F@via.net> <4EA8DA14-8672-4B8C-9A56-55241E9EA9C3@writersglen.com> Message-ID: Chuck Moore?s ?Moore Patent Portfolio? has to be licensed by anyone manufacturing Microprocessors. Moore?s patents cover such essential technologies like issuing more than one instruction per clock, multiple cores, embedded memory. Pretty smart guy. > On Sep 2, 2018, at 6:57 AM, Lloyd R. Prentice wrote: > > Hi All, > > Some 30-odd years ago Chuck Moore, developer of Forth, turned his attention to chip design. He wrote his own chip-design CAD system in Forth and, subsequently designed and brought to silicon an impressive list of successful chips: > > https://en.m.wikipedia.org/wiki/Charles_H._Moore > > Chuck?s latest inspirations can be seen here: > > http://www.greenarraychips.com/ > > The key to Chuck?s genius: Pare the problem-at-hand down to it?s essence; seek starkest simplicity. This has kept him out of step with the computing mainstream throughout this career. > > Best wishes, > > LRP > > Sent from my iPad > >> On Sep 2, 2018, at 6:17 AM, Mikael Pettersson wrote: >> >>> On Sat, Sep 1, 2018 at 10:33 PM, joe mcguckin wrote: >>> Does anyone know if the BEAM vm has ever been implemented directly in hardware (e.g. in an FPGA)? >> >> Not the BEAM that I know of, but Ericsson has had at least one project >> to implement an "Erlang processor" (via different presumably simpler >> VMs). >> >> Apart from the usual problems of synthesizing a processor, you also >> have to consider: >> - there's a lot of implicit state and semantics in the BEAM >> instructions, so you'd have to duplicate things like processes, >> dynamic memory allocation, garbage collection, exception handling, >> message passing, process links, etc >> - the BIFs, without which you can't do much, at a minimum you'd need >> networking support >> >> Microcoded HW implementations of LISP and Smalltalk were popular in >> the 70s and early 80s, but were quickly replaced by SW running on >> RISCs with higher performance, lower cost, and quicker updates. >> >> I'm not sure for what use case a HW Erlang makes sense (except as an >> intellectual exercise). >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From connolly.damien@REDACTED Mon Sep 3 13:27:04 2018 From: connolly.damien@REDACTED (Damien Towning) Date: Mon, 3 Sep 2018 21:27:04 +1000 Subject: [erlang-questions] Polling a CNODE Message-ID: Hi Everyone, I have a cnode written in C that performs some complex geometrical operations. The code is non blocking but I am wondering what the best way is to establish a keep alive poll back as return time for the cnode could be anywhere from a few seconds to hours. The cnode on the Erlang side is in elixir but I think this is really more an Erlang question than an Elixir one. I have something like this on the Erlang/Elixir side Nodex.Distributed.up() {conn, pid} = Cnode.start_link(%{exec_path: "bin/brep" , spawn_inactive_timeout: :infinity } ) Cnode.call(pid, msg ) And then something like this on the c code side else if ( erl_match(erl_format("blah"),operation) ) { std::vector para(3); ETERM *parameters = erl_element(2,emsg->msg); para = list_to_vectors( parameters ); cylinder(para[0],para[1],para[2]); response = erl_mk_int( geometry.currentIndex() ); erl_free_term(parameters); } My code does not block and I can regularly send messages back. I am just not sure what to send back as a regular wake up call. -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangsiri@REDACTED Mon Sep 3 15:44:11 2018 From: erlangsiri@REDACTED (Siri Hansen) Date: Mon, 3 Sep 2018 15:44:11 +0200 Subject: [erlang-questions] proper way to use the new logger In-Reply-To: <20180829160145.GA1867@valhala> References: <20180829160145.GA1867@valhala> Message-ID: Hi Nicolas, to set the module level in configuration, use the kernel parameter 'logger', and the term {module_level,Level,Modules}, for example, to set debug level for modules x, y and z: sys.config: [{kernel,[{logger,[{module_level,debug,[x,y,z]}]}]}]. Note that you must use the log macros for logging for this to work. Or, alternatively, add mfa=>{Module,Function,Arity} to the metadata for each log event. Unfortunately, there is no built-in way to get the module name only into each log message, but if you use the marcos (or add mfa yourself) you can get Module:Fun/Arity in your log messages by adding the atom 'mfa' to your formatter template, e.g. 1> logger:update_formatter_config(default,template,[time," ",level,", ",mfa,": ",msg,"\n"]). ok 2> logger:debug("hello world",#{mfa=>{x,f,1}}). 2018-09-03T15:37:35.577776+02:00 debug, x:f/1: hello world BR /siri Den ons. 29. aug. 2018 kl. 18:07 skrev Nicolas Martyanoff : > Hi, > > I have been trying to use the new logger application, and I am not sure > about > the way I am supposed to configure it for my needs (keep in mind that I am > an > Erlang beginner, hoping to use Erlang in production someday). > > During development of a "library", i.e. an OTP application used by the > program > I am working on, I need to see all logs related to the modules which are > part > of the OTP application. However I cannot see how to set the log level to > `debug` for these modules only (and not globally with `{logger_level, > debug}`). Ideally, there would be a way to configure it in a local > sys.config > to get this effect only during development, without having to do it > programmatically in every module I work on. > > More importantly, I can't seem to find a way to include the name of the > module > in log messages, either in the formatter template or anywhere else. Am I > supposed to add it manually to every single logger message I write ? > > I am just trying to find a simple way to add log instructions to the code I > write, with the possibility to control which ones are printed depending on > the > environment (local library development, application development, tests, > production...) using settings in sys.config. > > Thank you in advance. > > -- > Nicolas Martyanoff > http://snowsyn.net > khaelin@REDACTED > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rtrlists@REDACTED Mon Sep 3 18:56:56 2018 From: rtrlists@REDACTED (Robert Raschke) Date: Mon, 3 Sep 2018 18:56:56 +0200 Subject: [erlang-questions] Polling a CNODE In-Reply-To: References: Message-ID: I don't have the docs in front of me at the moment, but if I remember correctly you need to have the message receive loop running continuously, which handles the inter Erlang node pings for you. You can then monitor the C node from Elixir and get a message if a ping was not answered in a timely fashion. On Mon, 3 Sep 2018 13:27 Damien Towning, wrote: > Hi Everyone, > > I have a cnode written in C that performs some complex geometrical > operations. The code is non blocking but I am wondering what the best way > is to establish a keep alive poll back as return time for the cnode could > be anywhere from a few seconds to hours. The cnode on the Erlang side is in > elixir but I think this is really more an Erlang question than an Elixir > one. I have something like this > on the Erlang/Elixir side > > Nodex.Distributed.up() > {conn, pid} = Cnode.start_link(%{exec_path: "bin/brep" , > spawn_inactive_timeout: :infinity } ) > Cnode.call(pid, msg ) > > And then something like this on the c code side > > else if ( erl_match(erl_format("blah"),operation) ) { > std::vector para(3); > ETERM *parameters = erl_element(2,emsg->msg); > para = list_to_vectors( parameters ); > cylinder(para[0],para[1],para[2]); > response = erl_mk_int( geometry.currentIndex() ); > erl_free_term(parameters); > } > > My code does not block and I can regularly send messages back. I am just > not sure what to send back as a regular wake up call. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker@REDACTED Tue Sep 4 20:55:15 2018 From: sverker@REDACTED (Sverker Eriksson) Date: Tue, 4 Sep 2018 20:55:15 +0200 Subject: [erlang-questions] =?utf-8?q?ETS=E2=80=99s_performances_and_limit?= =?utf-8?q?s=3F?= In-Reply-To: <6D456A9E-C872-4BB1-9BE3-78E0F8BFDD47@pharos-corp.com> References: <393051E5-6357-4250-BA85-70BD55CFE7C1@alumni.caltech.edu> <6D456A9E-C872-4BB1-9BE3-78E0F8BFDD47@pharos-corp.com> Message-ID: <1536087315.20664.23.camel@erlang.org> On s?n, 2018-09-02 at 01:29 +0200, Valentin Micic wrote: > There is a good chance that performance degradation you are referring to may > *not* be related to a "heavier concurrency" as much as it may be caused by a > rate of inserts into a table of type "ordered_set". The manual indicates that > ordered_set tables do behave differently "in come situations" (which I read as > "insert operations when a table gets big"). > The paragraph containing "different behavior in some situations" is about the semantic differences for ordered_set due to keys compared with arithmetic == instead of matching =:= as the other table types. Performance wise ordered_set use AVL trees with O(log N) for lookup/insert/delete while the others use linear hashing with O(1). ordered_set does also not implement option 'write_concurrency'. Every mutating operation on an ordered_set will seize exclusive write lock on the entire table. This will most probably improve as we are expecting a very exciting pull request in a near future implementing 'write_concurrency' for ordered_set. /Sverker, Erlang/OTP @ Ericsson From frank.muller.erl@REDACTED Tue Sep 4 22:52:04 2018 From: frank.muller.erl@REDACTED (Frank Muller) Date: Tue, 4 Sep 2018 22:52:04 +0200 Subject: [erlang-questions] =?utf-8?q?ETS=E2=80=99s_performances_and_limit?= =?utf-8?q?s=3F?= In-Reply-To: <1536087315.20664.23.camel@erlang.org> References: <393051E5-6357-4250-BA85-70BD55CFE7C1@alumni.caltech.edu> <6D456A9E-C872-4BB1-9BE3-78E0F8BFDD47@pharos-corp.com> <1536087315.20664.23.camel@erlang.org> Message-ID: I?m only interested in ETS tables of type ?set? and their performances/limits. /Frank wrote > On s?n, 2018-09-02 at 01:29 +0200, Valentin Micic wrote: > > There is a good chance that performance degradation you are referring to > may > > *not* be related to a "heavier concurrency" as much as it may be caused > by a > > rate of inserts into a table of type "ordered_set". The manual indicates > that > > ordered_set tables do behave differently "in come situations" (which I > read as > > "insert operations when a table gets big"). > > > > The paragraph containing "different behavior in some situations" is about > the > semantic differences for ordered_set due to keys compared with arithmetic > == > instead of matching =:= as the other table types. > > > Performance wise ordered_set use AVL trees with O(log N) for > lookup/insert/delete while the others use linear hashing with O(1). > > ordered_set does also not implement option 'write_concurrency'. Every > mutating > operation on an ordered_set will seize exclusive write lock on the entire > table. > This will most probably improve as we are expecting a very exciting pull > request > in a near future implementing 'write_concurrency' for ordered_set. > > > /Sverker, Erlang/OTP @ Ericsson > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kuna.prime@REDACTED Wed Sep 5 01:02:21 2018 From: kuna.prime@REDACTED (Karlo Kuna) Date: Wed, 5 Sep 2018 01:02:21 +0200 Subject: [erlang-questions] erl_parse vs erl_syntax difference Message-ID: for example if we parse form (i'm omitting scan pass here) "-type a() :: tuple()." we get {ok,{attribute,1,type,{a,{type,1,tuple,any},[]}}} interesting part here is: {type,1,tuple,any} as far as i can see there no method how to construct valid tuple or map type annotations using erl_syntax. Specifically if one construct type annotation for tuple and use erl_syntax:revert/1 one gets: {type,1,tuple,any_size} any_size =/= any is this a bug?? -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Wed Sep 5 07:34:10 2018 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Wed, 5 Sep 2018 07:34:10 +0200 Subject: [erlang-questions] =?utf-8?q?ETS=E2=80=99s_performances_and_limit?= =?utf-8?q?s=3F?= In-Reply-To: References: <393051E5-6357-4250-BA85-70BD55CFE7C1@alumni.caltech.edu> <6D456A9E-C872-4BB1-9BE3-78E0F8BFDD47@pharos-corp.com> <1536087315.20664.23.camel@erlang.org> Message-ID: You may find this paper interesting. http://winsh.me/papers/erlang_workshop_2013.pdf regards, Chandru On Tue, 4 Sep 2018 at 22:52, Frank Muller wrote: > I?m only interested in ETS tables of type ?set? and their > performances/limits. > > /Frank > > wrote > >> On s?n, 2018-09-02 at 01:29 +0200, Valentin Micic wrote: >> > There is a good chance that performance degradation you are referring >> to may >> > *not* be related to a "heavier concurrency" as much as it may be caused >> by a >> > rate of inserts into a table of type "ordered_set". The manual >> indicates that >> > ordered_set tables do behave differently "in come situations" (which I >> read as >> > "insert operations when a table gets big"). >> > >> >> The paragraph containing "different behavior in some situations" is about >> the >> semantic differences for ordered_set due to keys compared with arithmetic >> == >> instead of matching =:= as the other table types. >> >> >> Performance wise ordered_set use AVL trees with O(log N) for >> lookup/insert/delete while the others use linear hashing with O(1). >> >> ordered_set does also not implement option 'write_concurrency'. Every >> mutating >> operation on an ordered_set will seize exclusive write lock on the entire >> table. >> This will most probably improve as we are expecting a very exciting pull >> request >> in a near future implementing 'write_concurrency' for ordered_set. >> >> >> /Sverker, Erlang/OTP @ Ericsson >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Wed Sep 5 15:19:04 2018 From: frank.muller.erl@REDACTED (Frank Muller) Date: Wed, 5 Sep 2018 15:19:04 +0200 Subject: [erlang-questions] Erlang & TLS Termination Message-ID: Hi guys Would like to hear from your experience(s) on using Erlang with TLS/SSL. The default Erlang stack doesn't perform well, Google says. Does anyone use Erlang in production behind: . Hitch: https://hitch-tls.org/ . Envoy: https://www.envoyproxy.io/ . HAProxy: http://www.haproxy.org/ . ??? Or directly using: . gen_ssl: how to make it scale? . fast_tls: https://github.com/processone/fast_tls . ??? Some of the solutions above are fast, some scale well ... but all with downsides (ex. Hitch adds 200kiB overhead per connection). We?re planning to go live with a fairly large number of secure connections: 45k - 75k. The sessions should be short (few seconds). Transferred data should range between 50KiB to 1miB max. Insights and/or feedbacks are very welcome. /Frank -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Wed Sep 5 15:52:27 2018 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 5 Sep 2018 09:52:27 -0400 Subject: [erlang-questions] Erlang & TLS Termination In-Reply-To: References: Message-ID: During my time at Heroku, we managed to make the Erlang TLS implementation (as a server) perform on par with Amazon's ELB stack -- it was something like 1-2ms (out of like 13-15ms) slower on the median case, but several seconds (if not minutes!) faster in the worst case, per connection, overall much more stable. We needed to patch a few things, but since around OTP-20, all of these have made it to upstream in OTP. The type of configuration used is described in a gist at https://gist.github.com/ferd/af9abf6b3600d2d7f08dba58fdfb514a -- the format changed in OTP-21, but the most important parts are all there. In short, it relies mostly on: - good configuration of cache (mostly disabling a bunch of them when you don't need disk cache) - good configuration of preferred cipher suites and ECCs (picking more complex ones than required slows things down -- i.e. picking a secp512 ECC instead of secp256 one iirc almost doubled the handshake time compared to AWS until we replicated their configurations) On Wed, Sep 5, 2018 at 9:19 AM, Frank Muller wrote: > Hi guys > > Would like to hear from your experience(s) on using Erlang with TLS/SSL. > The default Erlang stack doesn't perform well, Google says. > > Does anyone use Erlang in production behind: > > . Hitch: https://hitch-tls.org/ > . Envoy: https://www.envoyproxy.io/ > . HAProxy: http://www.haproxy.org/ > . ??? > > Or directly using: > . gen_ssl: how to make it scale? > . fast_tls: https://github.com/processone/fast_tls > . ??? > > Some of the solutions above are fast, some scale well ... but all with > downsides (ex. Hitch adds 200kiB overhead per connection). > > We?re planning to go live with a fairly large number of secure > connections: 45k - 75k. The sessions should be short (few seconds). > Transferred data should range between 50KiB to 1miB max. > > Insights and/or feedbacks are very welcome. > > /Frank > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Wed Sep 5 16:01:36 2018 From: frank.muller.erl@REDACTED (Frank Muller) Date: Wed, 5 Sep 2018 16:01:36 +0200 Subject: [erlang-questions] Erlang & TLS Termination In-Reply-To: References: Message-ID: Thanks Fred. Forgot to mention that I?ve already read your blog post about this. /Frank a ?crit : > During my time at Heroku, we managed to make the Erlang TLS implementation > (as a server) perform on par with Amazon's ELB stack -- it was something > like 1-2ms (out of like 13-15ms) slower on the median case, but several > seconds (if not minutes!) faster in the worst case, per connection, overall > much more stable. We needed to patch a few things, but since around OTP-20, > all of these have made it to upstream in OTP. > > The type of configuration used is described in a gist at > https://gist.github.com/ferd/af9abf6b3600d2d7f08dba58fdfb514a -- the > format changed in OTP-21, but the most important parts are all there. > > In short, it relies mostly on: > - good configuration of cache (mostly disabling a bunch of them when you > don't need disk cache) > - good configuration of preferred cipher suites and ECCs (picking more > complex ones than required slows things down -- i.e. picking a secp512 ECC > instead of secp256 one iirc almost doubled the handshake time compared to > AWS until we replicated their configurations) > > > On Wed, Sep 5, 2018 at 9:19 AM, Frank Muller > wrote: > >> Hi guys >> >> Would like to hear from your experience(s) on using Erlang with TLS/SSL. >> The default Erlang stack doesn't perform well, Google says. >> >> Does anyone use Erlang in production behind: >> >> . Hitch: https://hitch-tls.org/ >> . Envoy: https://www.envoyproxy.io/ >> . HAProxy: http://www.haproxy.org/ >> . ??? >> >> Or directly using: >> . gen_ssl: how to make it scale? >> . fast_tls: https://github.com/processone/fast_tls >> . ??? >> >> Some of the solutions above are fast, some scale well ... but all with >> downsides (ex. Hitch adds 200kiB overhead per connection). >> >> We?re planning to go live with a fairly large number of secure >> connections: 45k - 75k. The sessions should be short (few seconds). >> Transferred data should range between 50KiB to 1miB max. >> >> Insights and/or feedbacks are very welcome. >> >> /Frank >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dch@REDACTED Wed Sep 5 17:17:24 2018 From: dch@REDACTED (Dave Cottlehuber) Date: Wed, 05 Sep 2018 17:17:24 +0200 Subject: [erlang-questions] Erlang & TLS Termination In-Reply-To: References: Message-ID: <1536160644.476250.1497699104.6706720F@webmail.messagingengine.com> On Wed, 5 Sep 2018, at 15:19, Frank Muller wrote: > Hi guys > > Would like to hear from your experience(s) on using Erlang with > TLS/SSL.> The default Erlang stack doesn't perform well, Google says. > > Does anyone use Erlang in production behind: > > . Hitch: https://hitch-tls.org/ > . Envoy: https://www.envoyproxy.io/ > . HAProxy: http://www.haproxy.org/ > . ??? I?ve used haproxy to great effect handling all those messy acme/let?s encrypt renewals and load balancing connections across multiple servers for plain https, rabbitmq and socketio traffic. Roughly 50-80k https txns in and out per hour. The major win was matching up ibrowse connection settings with http1.1 pipelined connections to a 3rd party API. I think ferd?s tuning is a very similar outcome. the most important things in hindsight were - getting pipelining working to reuse TLS connections - observabilty - good logging - predictable failure modes I moved all services including DB and message brokers behind haproxy for these reasons. You should look very carefully at service discovery, and http2 support, in all of the situations you intend to proxy, not all of these projects have the same level of support. Developing your own heroku style LB would be interest g for a bigger organisation I think I needed something I could forget about. A+ Dave -------------- next part -------------- An HTML attachment was scrubbed... URL: From oliver.bollmann@REDACTED Wed Sep 5 23:00:06 2018 From: oliver.bollmann@REDACTED (Oliver Bollmann) Date: Wed, 5 Sep 2018 23:00:06 +0200 Subject: [erlang-questions] ssl psk since 20.3 failed Message-ID: <57620609-91a9-9131-0f10-352e4b4e9fa1@t-online.de> This works with 20.2.2 but since 20.3(21.x) it doesn't! Error in process <0.79.0> with exit value: {{badmatch,{error,{tls_alert,"handshake failure"}}}, ?[{client_server,init_connect,1,[{file,"client_server.erl"},{line,37}]}]} Any hints? -module(client_server). %%% Purpose: Example of SSL client and server using psk. -export([start/0, init_connect/1]). start() -> %% Start ssl application {ok,StartedApps} = application:ensure_all_started(ssl), %% Let the current process be the server that listens and accepts %% Listen {ok,LSock} = ssl:listen(0, mk_opts(listen)), {ok, {_,LPort}} = ssl:sockname(LSock), io:fwrite("Listen: port = ~w.~n", [LPort]), %% Spawn the client process that connects to the server spawn(?MODULE, init_connect, [LPort]), %% Accept {ok,ASock} = ssl:transport_accept(LSock), ok = ssl:ssl_accept(ASock), io:fwrite("Accept: accepted.~n"), ssl:send(ASock,"hello"), {error, closed} = ssl:recv(ASock,0), io:fwrite("Accept: detected closed.~n"), ssl:close(ASock), io:fwrite("Listen: closing and terminating.~n"), ssl:close(LSock), lists:foreach(fun application:stop/1, lists:reverse(StartedApps)). %% Client connect init_connect(LPort) -> {ok,Host} = inet:gethostname(), {ok,CSock} = ssl:connect(Host,LPort, mk_opts(connect)), io:fwrite("Connect: connected.~n"), {ok,Data} = ssl:recv(CSock,0), io:fwrite("Connect: got data: ~p~n", [Data]), io:fwrite("Connect: closing and terminating.~n"), ssl:close(CSock). mk_opts(listen) -> mk_opts("server"); mk_opts(connect) -> mk_opts("client"); mk_opts(Role) -> [{active, false}, {psk_identity,Role}, {user_lookup_fun,{fun lookup/3,list_to_binary(Role)}}, {versions,['tlsv1.2']}, {ciphers, [{dhe_psk,aes_256_gcm,null,sha384} ]} ]. lookup(psk,_,_) -> {ok,<<"psk">>}. -- Gr??e Oliver Bollmann -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Thu Sep 6 14:20:10 2018 From: erlang@REDACTED (Richard Bowker) Date: Thu, 6 Sep 2018 05:20:10 -0700 Subject: [erlang-questions] os:getenv regression Message-ID: <20180906052010.90D1E888@m0117567.ppops.net> An HTML attachment was scrubbed... URL: From rickard@REDACTED Thu Sep 6 14:29:06 2018 From: rickard@REDACTED (Rickard Green) Date: Thu, 6 Sep 2018 14:29:06 +0200 Subject: [erlang-questions] Patch package OTP 21.0.8 released Message-ID: <4f71933b-f2e1-4790-db55-ae299b677597@erlang.org> Patch Package: OTP 21.0.8 Git Tag: OTP-21.0.8 Date: 2018-09-06 Trouble Report Id: OTP-15275, OTP-15279, OTP-15280 Seq num: ERIERL-226 System: OTP Release: 21 Application: erts-10.0.7, kernel-6.0.1 Predecessor: OTP 21.0.7 Check out the git tag OTP-21.0.8, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- erts-10.0.7 ----------------------------------------------------- --------------------------------------------------------------------- The erts-10.0.7 application can be applied independently of other applications on a full OTP 21 installation. --- Fixed Bugs and Malfunctions --- OTP-15275 Application(s): erts Related Id(s): PR-1943 A process could get stuck in an infinite rescheduling loop between normal and dirty schedulers. This bug was introduced in ERTS version 10.0. Thanks to Maxim Fedorov for finding and fixing this issue. OTP-15279 Application(s): erts Related Id(s): ERIERL-226 Garbage collection of a distribution entry could cause an emulator crash if net_kernel had not brought previous connection attempts on it down properly. Full runtime dependencies of erts-10.0.7: kernel-6.0, sasl-3.0.1, stdlib-3.5 --------------------------------------------------------------------- --- kernel-6.0.1 ---------------------------------------------------- --------------------------------------------------------------------- The kernel-6.0.1 application can be applied independently of other applications on a full OTP 21 installation. --- Fixed Bugs and Malfunctions --- OTP-15280 Application(s): kernel Related Id(s): ERIERL-226, OTP-15279 Fixed bug in net_kernel that could cause an emulator crash if certain connection attempts failed. Bug exists since kernel-6.0 (OTP-21.0). Full runtime dependencies of kernel-6.0.1: erts-10.0, sasl-3.0, stdlib-3.5 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From raimo+erlang-questions@REDACTED Thu Sep 6 15:12:23 2018 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 6 Sep 2018 15:12:23 +0200 Subject: [erlang-questions] os:getenv regression In-Reply-To: <20180906052010.90D1E888@m0117567.ppops.net> References: <20180906052010.90D1E888@m0117567.ppops.net> Message-ID: <20180906131223.GA61806@erix.ericsson.se> My mail client does not like your mail client. I think the list policies say something about not HTML-only posts... On Thu, Sep 06, 2018 at 05:20:10AM -0700, Richard Bowker wrote: >
In OTP-20 os:getenv accepted either strings or binaries:

Erlang/OTP 20 [erts-9.3.3] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V9.3.3  (abort with ^G)
1> os:getenv("").
false
2> os:getenv(<<"">>).
false
3> 


however in OTP-21

Erlang/OTP 21 [erts-10.0.5] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1]

Eshell V10.0.5  (abort with ^G)
1> os:getenv("").
false
2> os:getenv(<<"">>).
** exception error: bad argument
     in function  os:get_env_var/1
        called as os:get_env_var(<<>>)
     in call from os:getenv/1 (os.erl, line 124)
3> 


Was this change deliberate?

Rich
-- / Raimo Niskanen, Erlang/OTP, Ericsson AB From wander4096@REDACTED Thu Sep 6 15:22:21 2018 From: wander4096@REDACTED (=?UTF-8?B?6ZmI5qKT56uL?=) Date: Thu, 6 Sep 2018 21:22:21 +0800 Subject: [erlang-questions] os:getenv regression In-Reply-To: <20180906131223.GA61806@erix.ericsson.se> References: <20180906052010.90D1E888@m0117567.ppops.net> <20180906131223.GA61806@erix.ericsson.se> Message-ID: Hi Richard, This change was introduced by this commit and I think it is by implementation. Best, tison. Raimo Niskanen ?2018?9?6??? ??9:12??? > My mail client does not like your mail client. > I think the list policies say something about not HTML-only posts... > > On Thu, Sep 06, 2018 at 05:20:10AM -0700, Richard Bowker wrote: > >
size="2" style="">In OTP-20 os:getenv > accepted either strings or binaries:

face="Arial, sans-serif">Erlang/OTP 20 [erts-9.3.3] [source] [64-bit] > [smp:16:16] [ds:16:16:10] [async-threads:10] [hipe] > [kernel-poll:false]

Eshell V9.3.3  (abort with ^G)
style="">1> > os:getenv("").
false
2> os:getenv(<<"">>).
style="">false
style="">3> 
style="font-family: Arial, sans-serif;">

DIV>
however in > OTP-21

Erlang/OTP 21 [erts-10.0.5] [source] [64-bit] [smp:16:16] > [ds:16:16:10] [async-threads:1]
face="Arial, sans-serif">
Eshell V10.0.5  (abort with ^G)
style="">1> > os:getenv("").
false
2> os:getenv(<<"">>).
style="">** exception error: bad > argument
  >    in function  os:get_env_var/1
style="">        called > as os:get_env_var(<<>>)
face="Arial, sans-se > rif">     in call from os:getenv/1 (os.erl, line > 124)
3> 


Was this change deliberate?
style="font-family: Arial, sans-serif;">
Rich
> > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Thu Sep 6 15:41:54 2018 From: ingela.andin@REDACTED (Ingela Andin) Date: Thu, 6 Sep 2018 15:41:54 +0200 Subject: [erlang-questions] ssl psk since 20.3 failed In-Reply-To: <57620609-91a9-9131-0f10-352e4b4e9fa1@t-online.de> References: <57620609-91a9-9131-0f10-352e4b4e9fa1@t-online.de> Message-ID: Humm ... I believe the this was broken by PR-1729, the solution feels familiar. I hope that the following patch covers all the cases. diff --git a/lib/ssl/src/ssl_handshake.erl b/lib/ssl/src/ssl_handshake.erl index 63996f5..4fbf463 100644 --- a/lib/ssl/src/ssl_handshake.erl +++ b/lib/ssl/src/ssl_handshake.erl @@ -1056,7 +1056,10 @@ select_curve(undefined, _, _) -> select_hashsign(_, _, KeyExAlgo, _, _Version) when KeyExAlgo == dh_anon; KeyExAlgo == ecdh_anon; KeyExAlgo == srp_anon; - KeyExAlgo == psk -> + KeyExAlgo == psk; + KeyExAlgo == dhe_psk; + KeyExAlgo == ecdhe_psk; + KeyExAlgo == rsa_psk -> {null, anon}; %% The signature_algorithms extension was introduced with TLS 1.2. Ignore it if we have %% negotiated a lower version. Regards Ingela Erlang/OTP team - Ericsson AB Den ons 5 sep. 2018 kl 23:00 skrev Oliver Bollmann < oliver.bollmann@REDACTED>: > This works with 20.2.2 but since 20.3(21.x) it doesn't! > > Error in process <0.79.0> with exit value: > {{badmatch,{error,{tls_alert,"handshake failure"}}}, > [{client_server,init_connect,1,[{file,"client_server.erl"},{line,37}]}]} > > Any hints? > > -module(client_server).%%% Purpose: Example of SSL client and server using psk.-export([start/0, init_connect/1]).start() -> > %% Start ssl application {ok, StartedApps} = application:ensure_all_started(ssl), > > %% Let the current process be the server that listens and accepts %% Listen {ok, LSock} = ssl:listen(0, mk_opts(listen)), > {ok, {_, LPort}} = ssl:sockname(LSock), > io:fwrite("Listen: port = ~w.~n", [LPort]), > > %% Spawn the client process that connects to the server spawn(?MODULE, init_connect, [LPort]), > > %% Accept {ok, ASock} = ssl:transport_accept(LSock), > ok = ssl:ssl_accept(ASock), > io:fwrite("Accept: accepted.~n"), > ssl:send(ASock, "hello"), > {error, closed} = ssl:recv(ASock, 0), > io:fwrite("Accept: detected closed.~n"), > ssl:close(ASock), > io:fwrite("Listen: closing and terminating.~n"), > ssl:close(LSock), > > lists:foreach(fun application:stop/1, lists:reverse(StartedApps)).%% Client connectinit_connect(LPort) -> > {ok, Host} = inet:gethostname(), > {ok, CSock} = ssl:connect(Host, LPort, mk_opts(connect)), > io:fwrite("Connect: connected.~n"), > {ok, Data} = ssl:recv(CSock, 0), > io:fwrite("Connect: got data: ~p~n", [Data]), > io:fwrite("Connect: closing and terminating.~n"), > ssl:close(CSock).mk_opts(listen) -> > mk_opts("server"); > mk_opts(connect) -> > mk_opts("client"); > mk_opts(Role) -> > [{active, false}, > {psk_identity,Role}, > {user_lookup_fun,{fun lookup/3,list_to_binary(Role)}}, > {versions,['tlsv1.2']}, > {ciphers, [{dhe_psk,aes_256_gcm,null,sha384} > ]} > ].lookup(psk,_,_) -> {ok,<<"psk">>}. > > -- > Gr??e > Oliver Bollmann > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Fri Sep 7 08:54:11 2018 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 7 Sep 2018 08:54:11 +0200 Subject: [erlang-questions] os:getenv regression In-Reply-To: <20180906081419.90D1DB1E@m0117566.ppops.net> References: <20180906081419.90D1DB1E@m0117566.ppops.net> Message-ID: <20180907065411.GA48701@erix.ericsson.se> On Thu, Sep 06, 2018 at 08:14:19AM -0700, Richard Bowker wrote: > I always forget to change the mode! :) You might have a dual mode that would allow text only clients to show a text only version of the mail. :-) > > In OTP-20 os:getenv accepted either strings or binaries: > > Erlang/OTP 20 [erts-9.3.3] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V9.3.3 (abort with ^G) > 1> os:getenv(""). > false > 2> os:getenv(<<"">>). > false > 3> > > > however in OTP-21 > > Erlang/OTP 21 [erts-10.0.5] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1] > > Eshell V10.0.5 (abort with ^G) > 1> os:getenv(""). > false > 2> os:getenv(<<"">>). > ** exception error: bad argument > in function os:get_env_var/1 > called as os:get_env_var(<<>>) > in call from os:getenv/1 (os.erl, line 124) > 3> > > > Was this change deliberate? os:getenv/1 has since forever been documented to accept a string() as its argument. So it seems that during a rewrite to OTP-21.0 the undocumented feature to also accept a binary or maybe even an iolist was removed. That feature was not documented, so therefore the change is according to some definition not a regression since the behaviour was never supported. :-) Note that the function is now documented to accept nonempty_string(), so therefore the behaviour to return false for the argument "" is now not supported. According to the new definition the function is allowed to crash for the argument "". I do not know if that was an intended change or not... Best Regards / Raimo Niskanen > > Rich > > --- raimo+erlang-questions@REDACTED wrote: > > From: Raimo Niskanen > To: Richard Bowker , > Subject: Re: [erlang-questions] os:getenv regression > Date: Thu, 6 Sep 2018 15:12:23 +0200 > > My mail client does not like your mail client. > I think the list policies say something about not HTML-only posts... > > On Thu, Sep 06, 2018 at 05:20:10AM -0700, Richard Bowker wrote: > >
In OTP-20 os:getenv accepted either strings or binaries:

Erlang/OTP 20 [erts-9.3.3] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V9.3.3  (abort with ^G)
1> os:getenv("").
false
2> os:getenv(<<"">>).
false
3> 


however in OTP-21

Erlang/OTP 21 [erts-10.0.5] [source] [64-bit] [smp:16:16] [ds:16:16:10] [async-threads:1]

Eshell V10.0.5  (abort with ^G)
1> os:getenv("").
false
2> os:getenv(<<"">>).
** exception error: bad argument
     in function  os:get_env_var/1
        called as os:get_env_var(<<>>)
     in call from os:getenv/1 (os.erl, line 12 > /FONT>
3> 


Was this change deliberate?

Rich
> > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From roger@REDACTED Fri Sep 7 12:11:17 2018 From: roger@REDACTED (Roger Lipscombe) Date: Fri, 7 Sep 2018 11:11:17 +0100 Subject: [erlang-questions] gen_udp:send blocks? Message-ID: I've got a gen_server that, in response to handle_cast, calls gen_udp:send. On one server, the call to gen_udp:send takes about 1500us. On a different server, the call takes about 75000us. Because of this, the process message queue is getting backed up, at the rate of about 120-250 messages/minute. The difference between the servers is obvious: one of them is in the same AWS region as the destination host; the other is the other side of the Atlantic (i.e. about 70ms away). My question is more about understanding the semantics of gen_udp:send: why is it blocking? If the buffer was full, I'd expect it to discard the message and return immediately. Regards, Roger. From v@REDACTED Fri Sep 7 13:37:58 2018 From: v@REDACTED (Valentin Micic) Date: Fri, 7 Sep 2018 13:37:58 +0200 Subject: [erlang-questions] gen_udp:send blocks? In-Reply-To: References: Message-ID: <75104AF3-3696-4413-B552-19B13E0AB2CF@pharos-corp.com> On 07 Sep 2018, at 12:11 PM, Roger Lipscombe wrote: > I've got a gen_server that, in response to handle_cast, calls > gen_udp:send. On one server, the call to gen_udp:send takes about > 1500us. On a different server, the call takes about 75000us. > > Because of this, the process message queue is getting backed up, at > the rate of about 120-250 messages/minute. > > The difference between the servers is obvious: one of them is in the > same AWS region as the destination host; the other is the other side > of the Atlantic (i.e. about 70ms away). > > My question is more about understanding the semantics of gen_udp:send: > why is it blocking? If the buffer was full, I'd expect it to discard > the message and return immediately. > > Regards, > Roger. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions UDP should behave the way you expected. May I ask if the caller of gen_udp:send is the owner (e.g. creator) of the socket? V/ From roger@REDACTED Fri Sep 7 15:00:01 2018 From: roger@REDACTED (Roger Lipscombe) Date: Fri, 7 Sep 2018 14:00:01 +0100 Subject: [erlang-questions] gen_udp:send blocks? In-Reply-To: <75104AF3-3696-4413-B552-19B13E0AB2CF@pharos-corp.com> References: <75104AF3-3696-4413-B552-19B13E0AB2CF@pharos-corp.com> Message-ID: On 7 September 2018 at 12:37, Valentin Micic wrote: > On 07 Sep 2018, at 12:11 PM, Roger Lipscombe wrote: >> My question is more about understanding the semantics of gen_udp:send: >> why is it blocking? If the buffer was full, I'd expect it to discard >> the message and return immediately. > > UDP should behave the way you expected. May I ask if the caller of gen_udp:send is the owner (e.g. creator) of the socket? Yes it is. I call gen_udp:open in Module:init/1, and gen_udp:send in Module:handle_cast/2. Oh. Here's a thought: we use a hostname as the destination in gen_udp:send. Could that be causing blocking? From dmkolesnikov@REDACTED Fri Sep 7 15:13:08 2018 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Fri, 7 Sep 2018 16:13:08 +0300 Subject: [erlang-questions] gen_udp:send blocks? In-Reply-To: References: <75104AF3-3696-4413-B552-19B13E0AB2CF@pharos-corp.com> Message-ID: <4FF61CF2-6C45-4E97-AF96-E0FDB2B8D91B@gmail.com> > On 7 Sep 2018, at 16.00, Roger Lipscombe wrote: > > Oh. Here's a thought: we use a hostname as the destination in > gen_udp:send. Could that be causing blocking? Usually, this causes DNS resolution, which takes some time. If you have short DNS TTL or disable cache it might cause visible delay. I think you need to check config of you DNS server and resolver as well. Best Regards, Dmitry -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From roger@REDACTED Fri Sep 7 15:20:19 2018 From: roger@REDACTED (Roger Lipscombe) Date: Fri, 7 Sep 2018 14:20:19 +0100 Subject: [erlang-questions] gen_udp:send blocks? In-Reply-To: <4FF61CF2-6C45-4E97-AF96-E0FDB2B8D91B@gmail.com> References: <75104AF3-3696-4413-B552-19B13E0AB2CF@pharos-corp.com> <4FF61CF2-6C45-4E97-AF96-E0FDB2B8D91B@gmail.com> Message-ID: On 7 September 2018 at 14:13, Dmitry Kolesnikov wrote: > > On 7 Sep 2018, at 16.00, Roger Lipscombe wrote: > > Oh. Here's a thought: we use a hostname as the destination in > gen_udp:send. Could that be causing blocking? > > > Usually, this causes DNS resolution, which takes some time. If you have > short DNS TTL or disable cache it might cause visible delay. I think you > need to check config of you DNS server and resolver as well. Yeah; this appears to be the problem. We're using consul, which has zero TTL (by default?). This means that each call to gen_udp:send resolves the name again. Based on the timings, it looks like this involves a trip across the Atlantic. We want a fairly short TTL (because otherwise how would health-check failover work?). I'll explicitly cache the IP address and refresh it after each N messages sent. From v@REDACTED Fri Sep 7 15:23:22 2018 From: v@REDACTED (Valentin Micic) Date: Fri, 7 Sep 2018 15:23:22 +0200 Subject: [erlang-questions] gen_udp:send blocks? In-Reply-To: References: <75104AF3-3696-4413-B552-19B13E0AB2CF@pharos-corp.com> Message-ID: On 07 Sep 2018, at 3:00 PM, Roger Lipscombe wrote: > On 7 September 2018 at 12:37, Valentin Micic wrote: >> On 07 Sep 2018, at 12:11 PM, Roger Lipscombe wrote: >>> My question is more about understanding the semantics of gen_udp:send: >>> why is it blocking? If the buffer was full, I'd expect it to discard >>> the message and return immediately. >> >> UDP should behave the way you expected. May I ask if the caller of gen_udp:send is the owner (e.g. creator) of the socket? > > Yes it is. I call gen_udp:open in Module:init/1, and gen_udp:send in > Module:handle_cast/2. > > Oh. Here's a thought: we use a hostname as the destination in > gen_udp:send. Could that be causing blocking? Well, if the name resolution is slow (e.g. you use some remote DNS server), this may cause some delay, but, as far as I know, local OS resolver would cache the name once it has been resolved. HOWEVER, if that delay caused gen_server message process queue to build up due to constant incoming traffic, than all bets are off -- when this number grows above hundred, this may cause the whole run-time to underperform, which puts more pressure one the gen_server's process message queue and so on. This could explain the behavior you observed. There's an easy way to check this -- just check the process message queue length for your gen_server (or any other process to that end), and you'll know :-) V/ From dmkolesnikov@REDACTED Fri Sep 7 15:28:24 2018 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Fri, 7 Sep 2018 16:28:24 +0300 Subject: [erlang-questions] gen_udp:send blocks? In-Reply-To: References: <75104AF3-3696-4413-B552-19B13E0AB2CF@pharos-corp.com> <4FF61CF2-6C45-4E97-AF96-E0FDB2B8D91B@gmail.com> Message-ID: <1D8307B8-2B28-4BA6-B8D4-6C4921C27567@gmail.com> > On 7 Sep 2018, at 16.20, Roger Lipscombe wrote: > > We want a fairly short TTL > (because otherwise how would health-check failover work?). I'll > explicitly cache the IP address and refresh it after each N messages > sent. I would implement a side process that re-resolves DNS every N seconds and then notifies your sender process with IP address. or change DNS settings to few seconds to benefit from OS feature. Keep in-mind that OTP has custom resolver. I?ve not hacked and do not know all details but it might have an explicit cache feature. It worth of checking. - Dmitry -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From roger@REDACTED Fri Sep 7 15:40:34 2018 From: roger@REDACTED (Roger Lipscombe) Date: Fri, 7 Sep 2018 14:40:34 +0100 Subject: [erlang-questions] gen_udp:send blocks? In-Reply-To: <1D8307B8-2B28-4BA6-B8D4-6C4921C27567@gmail.com> References: <75104AF3-3696-4413-B552-19B13E0AB2CF@pharos-corp.com> <4FF61CF2-6C45-4E97-AF96-E0FDB2B8D91B@gmail.com> <1D8307B8-2B28-4BA6-B8D4-6C4921C27567@gmail.com> Message-ID: On 7 September 2018 at 14:28, Dmitry Kolesnikov wrote: > I would implement a side process that re-resolves DNS every N seconds and > then notifies your sender process with IP address. That's a good suggestion, but we're already (as of 30 minutes ago) using pobox, so it's easier just to wedge it in there. If we weren't doing that, then your suggestion might be ideal. Thanks. > or change DNS settings to few seconds to benefit from OS feature. Ugh. That means talking to the ops team (just kidding!). I found https://www.consul.io/docs/guides/dns-cache.html, so I'll talk with them later about it. > Keep > in-mind that OTP has custom resolver. I?ve not hacked and do not know all > details but it might have an explicit cache feature. It worth of checking. Good point. I had a quick poke around in there a while ago. It's probably worth taking another look. From ingela.andin@REDACTED Fri Sep 7 22:54:02 2018 From: ingela.andin@REDACTED (Ingela Andin) Date: Fri, 7 Sep 2018 22:54:02 +0200 Subject: [erlang-questions] ssl psk since 20.3 failed In-Reply-To: References: <57620609-91a9-9131-0f10-352e4b4e9fa1@t-online.de> Message-ID: No problem :) I will include it in OTP-21.1 except rsa_psk uses certs so I removed the last one again. Have also tweaked the test cases so that this is test correctly. Regards Ingela Erlang/OTP team - Ericsson AB Den tors 6 sep. 2018 kl 15:41 skrev Ingela Andin : > > > Humm ... I believe the this was broken by PR-1729, the solution feels > familiar. I hope that the following patch covers all the cases. > > diff --git a/lib/ssl/src/ssl_handshake.erl b/lib/ssl/src/ssl_handshake.erl > index 63996f5..4fbf463 100644 > --- a/lib/ssl/src/ssl_handshake.erl > +++ b/lib/ssl/src/ssl_handshake.erl > @@ -1056,7 +1056,10 @@ select_curve(undefined, _, _) -> > select_hashsign(_, _, KeyExAlgo, _, _Version) when KeyExAlgo == dh_anon; > KeyExAlgo == ecdh_anon; > KeyExAlgo == srp_anon; > - KeyExAlgo == psk -> > + KeyExAlgo == psk; > + KeyExAlgo == dhe_psk; > + KeyExAlgo == ecdhe_psk; > + KeyExAlgo == rsa_psk -> > {null, anon}; > %% The signature_algorithms extension was introduced with TLS 1.2. Ignore > it if we have > %% negotiated a lower version. > > > Regards Ingela Erlang/OTP team - Ericsson AB > > > > > Den ons 5 sep. 2018 kl 23:00 skrev Oliver Bollmann < > oliver.bollmann@REDACTED>: > >> This works with 20.2.2 but since 20.3(21.x) it doesn't! >> >> Error in process <0.79.0> with exit value: >> {{badmatch,{error,{tls_alert,"handshake failure"}}}, >> [{client_server,init_connect,1,[{file,"client_server.erl"},{line,37}]}]} >> >> Any hints? >> >> -module(client_server).%%% Purpose: Example of SSL client and server using psk.-export([start/0, init_connect/1]).start() -> >> %% Start ssl application {ok, StartedApps} = application:ensure_all_started(ssl), >> >> %% Let the current process be the server that listens and accepts %% Listen {ok, LSock} = ssl:listen(0, mk_opts(listen)), >> {ok, {_, LPort}} = ssl:sockname(LSock), >> io:fwrite("Listen: port = ~w.~n", [LPort]), >> >> %% Spawn the client process that connects to the server spawn(?MODULE, init_connect, [LPort]), >> >> %% Accept {ok, ASock} = ssl:transport_accept(LSock), >> ok = ssl:ssl_accept(ASock), >> io:fwrite("Accept: accepted.~n"), >> ssl:send(ASock, "hello"), >> {error, closed} = ssl:recv(ASock, 0), >> io:fwrite("Accept: detected closed.~n"), >> ssl:close(ASock), >> io:fwrite("Listen: closing and terminating.~n"), >> ssl:close(LSock), >> >> lists:foreach(fun application:stop/1, lists:reverse(StartedApps)).%% Client connectinit_connect(LPort) -> >> {ok, Host} = inet:gethostname(), >> {ok, CSock} = ssl:connect(Host, LPort, mk_opts(connect)), >> io:fwrite("Connect: connected.~n"), >> {ok, Data} = ssl:recv(CSock, 0), >> io:fwrite("Connect: got data: ~p~n", [Data]), >> io:fwrite("Connect: closing and terminating.~n"), >> ssl:close(CSock).mk_opts(listen) -> >> mk_opts("server"); >> mk_opts(connect) -> >> mk_opts("client"); >> mk_opts(Role) -> >> [{active, false}, >> {psk_identity,Role}, >> {user_lookup_fun,{fun lookup/3,list_to_binary(Role)}}, >> {versions,['tlsv1.2']}, >> {ciphers, [{dhe_psk,aes_256_gcm,null,sha384} >> ]} >> ].lookup(psk,_,_) -> {ok,<<"psk">>}. >> >> -- >> Gr??e >> Oliver Bollmann >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Fri Sep 7 23:12:20 2018 From: ingela.andin@REDACTED (Ingela Andin) Date: Fri, 7 Sep 2018 23:12:20 +0200 Subject: [erlang-questions] Erlang & TLS Termination In-Reply-To: <1536160644.476250.1497699104.6706720F@webmail.messagingengine.com> References: <1536160644.476250.1497699104.6706720F@webmail.messagingengine.com> Message-ID: There are always lots of trade offs between security and efficiency. And we can never make all of them for you. Basic TLS Session handling I think is kind of flawed by design, we have tried to mitigate it. It seems for really powerful machines it is not enough and session table can be filled quicker than cleaned. We where thinking session tickets could be the solution and a workaround was to have a own session table callback as Fred suggested not saving any sessions. But TLS 1.3 deprecates both these ways and introduces a new way so TLS-1.2 session tickets are not so tempting any more. If people think they have no or a low benefit from the TLS-session tables maybe we could change it somehow. Regards Ingela Erlang(OTP team - Ericsson AB Den ons 5 sep. 20able18 kl 17:17 skrev Dave Cottlehuber : > On Wed, 5 Sep 2018, at 15:19, Frank Muller wrote: > > Hi guys > > > > Would like to hear from your experience(s) on using Erlang with TLS/SSL. > > The default Erlang stack doesn't perform well, Google says. > > > > Does anyone use Erlang in production behind: > > > > . Hitch: https://hitch-tls.org/ > > . Envoy: https://www.envoyproxy.io/ > > . HAProxy: http://www.haproxy.org/ > > . ??? > > I?ve used haproxy to great effect handling all those messy acme/let?s > encrypt renewals and load balancing connections across multiple servers for > plain https, rabbitmq and socketio traffic. Roughly 50-80k https txns in > and out per hour. The major win was matching up ibrowse connection settings > with http1.1 pipelined connections to a 3rd party API. I think ferd?s > tuning is a very similar outcome. > > the most important things in hindsight were > > - getting pipelining working to reuse TLS connections > - observabilty > - good logging > - predictable failure modes > > I moved all services including DB and message brokers behind haproxy for > these reasons. > > You should look very carefully at service discovery, and http2 support, in > all of the situations you intend to proxy, not all of these projects have > the same level of support. > > Developing your own heroku style LB would be interest g for a bigger > organisation I think I needed something I could forget about. > > A+ > Dave > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Sat Sep 8 06:45:43 2018 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Sat, 8 Sep 2018 00:45:43 -0400 Subject: [erlang-questions] erlpress_core follow-up Message-ID: <89AA3F42-0DF9-4104-BB14-2E859924DFB8@writersglen.com> Hello, A revered Erlang mentor pointed out that camel case is discouraged in Erlang circles. So I?ll soon change erlPress_core to erlpress_core on GitHub. But first I?d welcome input on few issues. 1. As stands, text resources and PDF output are hard-wired in code. image resources are tucked away in an obscure directory. These are particularly egregious problems when we specify erlpress_core as a dependency of a higher level application. So, question: what is the most appropriate way to allow the end user to configure source of text and image resources and destination of generated PDF files? 2. I?d like to resolve a few loose ends before the name change: - The CMark Markdown interpreter is not yet working as intended. - Code that deals with text that spills across panels and pages is incomplete. - Code that supports imposition, that is, stuff like labels, business cards, bookmarks, etc. is incomplete. Please let me know if other warts in the code come to light. 3. I also have ideas for rendering tables. I? do hope to get that in. 4. Anything else you?d like to see? Many thanks, LRP Sent from my iPad From frank.muller.erl@REDACTED Tue Sep 11 09:49:49 2018 From: frank.muller.erl@REDACTED (Frank Muller) Date: Tue, 11 Sep 2018 09:49:49 +0200 Subject: [erlang-questions] Prevent my crashing gen_server to produce any error message Message-ID: Hi, I found this old thread: is there a way to prevent a crashing process from sending a message to error_logger? http://erlang.org/pipermail/erlang-questions/2017-September/093407.html but I?m wondering if there is a documented, non-documented or even a hackish way to disable error reports for certain (gen_server) processes (Erlang 20.x or 21.x)? Thanks for you help/feedbacks /Frank -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangsiri@REDACTED Tue Sep 11 14:08:49 2018 From: erlangsiri@REDACTED (Siri Hansen) Date: Tue, 11 Sep 2018 14:08:49 +0200 Subject: [erlang-questions] Prevent my crashing gen_server to produce any error message In-Reply-To: References: Message-ID: With logger (OTP-21.x) you can use process metadata and a filter. For example, in your process' init function, call: logger:set_process_metadata(#{dont_log=>true}). Then add a filter which checks for this meta and stops the log - something like: logger:add_primary_filter(my_filter_id,{fun(#{meta:=#{dont_log:=true}}) -> stop; (_,_) -> ignore end,ok}). This will turn off all logs issued from the process itself, that is, the termination report from gen_server, the crash report from proc_lib, and any other log issued with the logger API. It will, however, not turn off the logs issued by the emulator after the process has crashed, i.e. the error report saying "Error in process <...> with exit reason ..." The reason for this is that the emulator has no knowledge of the process metadata, and can not include it in the log event. Therefore, to turn off these messages, you need a filter which can recognize the pid (which is still part of the metadata in this error report). But this is no problem if we're only talking about gen_server processes, since the emulator does not issue error reports for such processes. Best Regards /siri Den tir. 11. sep. 2018 kl. 09:50 skrev Frank Muller < frank.muller.erl@REDACTED>: > Hi, > > I found this old thread: is there a way to prevent a crashing process from > sending a message to error_logger? > > http://erlang.org/pipermail/erlang-questions/2017-September/093407.html > > but I?m wondering if there is a documented, non-documented or even a > hackish way to disable error reports for certain (gen_server) processes > (Erlang 20.x or 21.x)? > > Thanks for you help/feedbacks > /Frank > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Tue Sep 11 17:24:00 2018 From: frank.muller.erl@REDACTED (Frank Muller) Date: Tue, 11 Sep 2018 17:24:00 +0200 Subject: [erlang-questions] Prevent my crashing gen_server to produce any error message In-Reply-To: References: Message-ID: Hi Siri The filter helped a lot and less noise are shown on the console. But as you said, not everything is dropped and I can still see a badmatch crash report for example. Can you please explain us the PID idea inside the metadata as I?m not familiar with the new logger module? Thanks /Frank Le mar. 11 sept. 2018 ? 14:09, Siri Hansen a ?crit : > With logger (OTP-21.x) you can use process metadata and a filter. For > example, in your process' init function, call: > logger:set_process_metadata(#{dont_log=>true}). > > Then add a filter which checks for this meta and stops the log - something > like: > logger:add_primary_filter(my_filter_id,{fun(#{meta:=#{dont_log:=true}}) -> > stop; (_,_) -> ignore end,ok}). > > This will turn off all logs issued from the process itself, that is, the > termination report from gen_server, the crash report from proc_lib, and any > other log issued with the logger API. > > It will, however, not turn off the logs issued by the emulator after the > process has crashed, i.e. the error report saying "Error in process <...> > with exit reason ..." The reason for this is that the emulator has no > knowledge of the process metadata, and can not include it in the log event. > Therefore, to turn off these messages, you need a filter which can > recognize the pid (which is still part of the metadata in this error > report). But this is no problem if we're only talking about gen_server > processes, since the emulator does not issue error reports for such > processes. > > Best Regards > /siri > > > > > > Den tir. 11. sep. 2018 kl. 09:50 skrev Frank Muller < > frank.muller.erl@REDACTED>: > >> Hi, >> >> I found this old thread: is there a way to prevent a crashing process >> from sending a message to error_logger? >> >> http://erlang.org/pipermail/erlang-questions/2017-September/093407.html >> >> but I?m wondering if there is a documented, non-documented or even a >> hackish way to disable error reports for certain (gen_server) processes >> (Erlang 20.x or 21.x)? >> >> Thanks for you help/feedbacks >> /Frank >> > _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ledest@REDACTED Tue Sep 11 18:27:46 2018 From: ledest@REDACTED (Led) Date: Tue, 11 Sep 2018 19:27:46 +0300 Subject: [erlang-questions] Prevent my crashing gen_server to produce any error message In-Reply-To: References: Message-ID: > Can you please explain us the PID idea inside the metadata as I?m not > familiar with the new logger module? > > Thanks > /Frank > Maybe this: logger:add_primary_filter(my_filter_id,{fun(#{meta:=#{label:={proc_lib,crash}}}) -> stop; (_,_) -> ignore end,ok}). ? --- Led -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Wed Sep 12 06:08:13 2018 From: frank.muller.erl@REDACTED (Frank Muller) Date: Wed, 12 Sep 2018 06:08:13 +0200 Subject: [erlang-questions] Prevent my crashing gen_server to produce any error message In-Reply-To: References: Message-ID: Still seeing some supervisor crash report: (fah@REDACTED)1> =SUPERVISOR REPORT==== 12-Sep-2018::05:58:53.171292 === supervisor: {local,fah_sup} errorContext: child_terminated reason: {{badmatch,{<<"945a9769-070b-4635-b0f5-84c2105228f">>,76,45}}, [{fah,checkid,1,[{file,"src/fah.erl"},{line,210}]}, [...] {proc_lib,init_p_do_apply,3, [{file,"proc_lib.erl"},{line,249}]}]} offender: [{pid,<0.141.0>}, {id,fah}, {mfargs,{fah,start_link,[2,#Port<0.9>]}}, {restart_type,transient}, {shutdown,5000}, {child_type,worker}] I even tried to combine your and Siri?s filter, but no luck. /Frank Can you please explain us the PID idea inside the metadata as I?m not >> familiar with the new logger module? >> >> Thanks >> /Frank >> > > Maybe this: > > logger:add_primary_filter(my_filter_id,{fun(#{meta:=#{label:={proc_lib,crash}}}) > -> stop; (_,_) -> ignore end,ok}). > > ? > > --- > Led > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangsiri@REDACTED Wed Sep 12 12:41:19 2018 From: erlangsiri@REDACTED (Siri Hansen) Date: Wed, 12 Sep 2018 12:41:19 +0200 Subject: [erlang-questions] Prevent my crashing gen_server to produce any error message In-Reply-To: References: Message-ID: There are several different types of reports for a crashing process: 1) for gen* behaviours (gen_server, gen_event, gen_statem, ...), you get an error report issued by the crashing process just before it dies. For a gen_server, these messages start with " ** Generic server <...> terminating". These can be silenced with the process metadata + filter I suggested in my previous mail. 2) for gen* behaviours and other processes started with proc_lib, you get a crash report issued from the crashing process just before it dies. These reports (by default) have the heading == CRASH REPORT==, and the message starts with "crasher: ". These can be silenced with the process metadata + filter I suggested in my previous mail. 3) If the crashed process was part of a supervisor tree, a supervisor report is issued from the process' parent, that is, its supervisor. These reports (by default) have the heading == SUPERVISOR REPORT ==, and the message contains "errorContext: child_terminated". Since they are issued from a different process, these are not silenced by the suggested filter. 4) If the crashed process is not a gen* behaviour, and it is not started by proc_lib, then the emulator issues an error report which starts with "Error in process <...> with exit value:". These are not silenced by the suggested filter. These are the ones I can think of right now - and I assume that what you see are the supervisor reports only. Is this correct? How to silence the supervisor report depends on what you want to see from other processes. You can turn off supervisor reports with errorContext child_terminated completely for your system with this filter: logger:add_primary_filter(stop_sup_reports,{fun(#{log:=#{label:={supervisor,child_terminated}}},_) -> stop; (_,_) -> ignore end, ok}). or for one specific supervisor (fah_sup) with this one: logger:add_primary_filter(stop_sup_reports,{fun(#{msg:={report,#{label:={supervisor,child_terminated},report:=R}}},_) -> case proplists:get_value(supervisor,R) of {local,fah_sup} -> stop; _ -> ignore end; (_,_) -> ignore end, ok}). If the supervisor has other types of children, you might want to silence the reports related to one specific child id (fah) only. Then you can use this filter: logger:add_primary_filter(stop_sup_reports,{fun(#{msg:={report,#{label:={supervisor,child_terminated},report:=R}}},_) -> Child = proplists:get_value(offender,R), case proplists:get_value(id,Child) of fah -> stop; _ -> ignore end; (_,_) -> ignore end, ok}). Since I don't know much about what your system does and needs, these are only a few ideas. Another possibility can be to turn off all so called "SASL reports" completely - this would silence reports 2) and 3) for all of your system - if that is acceptable. The pid-idea from my first mail would only be needed for the emulator logs, 4) in my listing above, which I don't believe you see? >> Maybe this: >> >> logger:add_primary_filter(my_filter_id,{fun(#{meta:=#{label:={proc_lib,crash}}}) >> -> stop; (_,_) -> ignore end,ok}). >> >> ? >> >> I assume Len meant to put 'msg' instead of 'meta' here. That would work, but it would silence the crash reports, 2) in my listing above, from all processes, not only the specific ones. I hope this is of some help! Kind Regards /siri >> --- >> Led >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rickard@REDACTED Wed Sep 12 14:48:05 2018 From: rickard@REDACTED (Rickard Green) Date: Wed, 12 Sep 2018 14:48:05 +0200 Subject: [erlang-questions] Patch package OTP 21.0.9 released Message-ID: Patch Package: OTP 21.0.9 Git Tag: OTP-21.0.9 Date: 2018-09-12 Trouble Report Id: OTP-15289, OTP-15292, OTP-15296 Seq num: ERIERL-226, ERIERL-235, ERL-576 System: OTP Release: 21 Application: compiler-7.2.4, erts-10.0.8 Predecessor: OTP 21.0.8 Check out the git tag OTP-21.0.9, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- compiler-7.2.4 -------------------------------------------------- --------------------------------------------------------------------- The compiler-7.2.4 application can be applied independently of other applications on a full OTP 21 installation. --- Fixed Bugs and Malfunctions --- OTP-15292 Application(s): compiler Fix a regression in OTP-15204 that removed .beam file metadata that some external build tools relied on. Full runtime dependencies of compiler-7.2.4: crypto-3.6, erts-9.0, hipe-3.12, kernel-4.0, stdlib-2.5 --------------------------------------------------------------------- --- erts-10.0.8 ----------------------------------------------------- --------------------------------------------------------------------- The erts-10.0.8 application can be applied independently of other applications on a full OTP 21 installation. --- Fixed Bugs and Malfunctions --- OTP-15289 Application(s): erts Related Id(s): ERIERL-235, ERL-576, OTP-14943 As of ERTS version 10.0 (OTP 21.0) the erl_child_setup program, which creates port programs, ignores TERM signals. This setting was unintentionally inherited by port programs. Handling of TERM signals in port programs has now been restored to the default behavior. That is, terminate the process. OTP-15296 Application(s): erts Related Id(s): ERIERL-226, OTP-15279 The fix made for OTP-15279 in erts-10.07 (OTP-21.0.8) was not complete. It could cause a new connection attempt to be incorrectly aborted in certain cases. This fix will amend that flaw. Full runtime dependencies of erts-10.0.8: kernel-6.0, sasl-3.0.1, stdlib-3.5 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From frank.muller.erl@REDACTED Wed Sep 12 16:44:17 2018 From: frank.muller.erl@REDACTED (Frank Muller) Date: Wed, 12 Sep 2018 16:44:17 +0200 Subject: [erlang-questions] Prevent my crashing gen_server to produce any error message In-Reply-To: References: Message-ID: You made my day Siri. Working perfectly now and I?m able to suppress all reports from certain type of processes. The examples you provided should be added to the official doc. Thanks again!!! /Frank There are several different types of reports for a crashing process: > > 1) for gen* behaviours (gen_server, gen_event, gen_statem, ...), you get > an error report issued by the crashing process just before it dies. For a > gen_server, these messages start with " ** Generic server <...> > terminating". These can be silenced with the process metadata + filter I > suggested in my previous mail. > > 2) for gen* behaviours and other processes started with proc_lib, you get > a crash report issued from the crashing process just before it dies. These > reports (by default) have the heading == CRASH REPORT==, and the message > starts with "crasher: ". These can be silenced with the process metadata + > filter I suggested in my previous mail. > > 3) If the crashed process was part of a supervisor tree, a supervisor > report is issued from the process' parent, that is, its supervisor. These > reports (by default) have the heading == SUPERVISOR REPORT ==, and the > message contains "errorContext: child_terminated". Since they are issued > from a different process, these are not silenced by the suggested filter. > > 4) If the crashed process is not a gen* behaviour, and it is not started > by proc_lib, then the emulator issues an error report which starts with > "Error in process <...> with exit value:". These are not silenced by the > suggested filter. > > These are the ones I can think of right now - and I assume that what you > see are the supervisor reports only. Is this correct? > > How to silence the supervisor report depends on what you want to see from > other processes. You can turn off supervisor reports with errorContext > child_terminated completely for your system with this filter: > > logger:add_primary_filter(stop_sup_reports,{fun(#{log:=#{label:={supervisor,child_terminated}}},_) > -> stop; (_,_) -> ignore end, ok}). > > or for one specific supervisor (fah_sup) with this one: > > logger:add_primary_filter(stop_sup_reports,{fun(#{msg:={report,#{label:={supervisor,child_terminated},report:=R}}},_) > -> case proplists:get_value(supervisor,R) of {local,fah_sup} -> stop; _ -> > ignore end; (_,_) -> ignore end, ok}). > > If the supervisor has other types of children, you might want to silence > the reports related to one specific child id (fah) only. Then you can use > this filter: > > logger:add_primary_filter(stop_sup_reports,{fun(#{msg:={report,#{label:={supervisor,child_terminated},report:=R}}},_) > -> Child = proplists:get_value(offender,R), case > proplists:get_value(id,Child) of fah -> stop; _ -> ignore end; (_,_) -> > ignore end, ok}). > > Since I don't know much about what your system does and needs, these are > only a few ideas. Another possibility can be to turn off all so called > "SASL reports" completely - this would silence reports 2) and 3) for all of > your system - if that is acceptable. > > The pid-idea from my first mail would only be needed for the emulator > logs, 4) in my listing above, which I don't believe you see? > > > >>> Maybe this: >>> >>> logger:add_primary_filter(my_filter_id,{fun(#{meta:=#{label:={proc_lib,crash}}}) >>> -> stop; (_,_) -> ignore end,ok}). >>> >>> ? >>> >>> > I assume Len meant to put 'msg' instead of 'meta' here. That would work, > but it would silence the crash reports, 2) in my listing above, from all > processes, not only the specific ones. > > I hope this is of some help! > Kind Regards > /siri > > >>> > --- >>> Led >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From francesco@REDACTED Thu Sep 13 13:24:47 2018 From: francesco@REDACTED (Francesco Cesarini) Date: Thu, 13 Sep 2018 12:24:47 +0100 Subject: [erlang-questions] Erlang/Elixir talks at Functional Conf India Message-ID: <4f147b6a-33a9-b784-bcf0-f44871e8c8b3@erlang-solutions.com> Hi All, last year, Robert and I spoke and attended Functional Conf India in Bangalore. It was a great event with lots of interesting talks and attendees. We were blown away by the innovation happening in the functional programming space in general, and Erlang in particular. It is your turn this year. They are looking for proposals, and in some cases, might cover costs of flights and accommodation. They are also looking for tutorials, as they are hoping to run a one day bootcamp. If interested,? please submit your proposals here: https://confengine.com/functional-conf-2018/proposals ?before *Sep 15th*. Sooner the better. If you have any questions, read the Speaker guidelines and Review process ?and ask us. Happy to help! Francesco -------------- next part -------------- An HTML attachment was scrubbed... URL: From awsafrahman1704@REDACTED Thu Sep 13 23:47:49 2018 From: awsafrahman1704@REDACTED (Awsaf Rahman) Date: Thu, 13 Sep 2018 23:47:49 +0200 Subject: [erlang-questions] Running multiple beam files from an escript Message-ID: Hi all, I wrote a few microbenchmarks in Erlang to measure execution time. I have never used escripts but want to know how they work. Is it possible to run multiple beam files from an escript? For example, some condition decides which Beam file would be invoked? Regards Awsaf -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Fri Sep 14 09:13:52 2018 From: roger@REDACTED (Roger Lipscombe) Date: Fri, 14 Sep 2018 08:13:52 +0100 Subject: [erlang-questions] Running multiple beam files from an escript In-Reply-To: References: Message-ID: "rebar3 escriptize" -- http://www.rebar3.org/docs/commands#section-escriptize -- lets you combine multiple .erl files into a single escript. You can do the same with Elixir; see https://www.thegreatcodeadventure.com/executable-elixir-working-with-escripts/ Both of these take advantage of the fact that Erlang knows how to load BEAM files from a .ZIP file, and that .ZIP files can have "garbage" on the front. Said "garbage" being an entry point escript that invokes the main module from the ZIP. I experimented with it here: https://github.com/rlipscombe/escriptize -- but it's not maintained, so don't bother using it unless you need to support dependencies with NIFs/ports in them. On 13 September 2018 at 22:47, Awsaf Rahman wrote: > Hi all, > > I wrote a few microbenchmarks in Erlang to measure execution time. I have > never used escripts but want to know how they work. Is it possible to run > multiple beam files from an escript? For example, some condition decides > which Beam file would be invoked? > > Regards > Awsaf > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From amuhar3@REDACTED Fri Sep 14 09:11:28 2018 From: amuhar3@REDACTED (=?UTF-8?B?0JDQvdC90LAg0JzRg9GF0LDRgNGA0LDQvA==?=) Date: Fri, 14 Sep 2018 10:11:28 +0300 Subject: [erlang-questions] Use of custom function to define exometer entry Message-ID: Hello, Is it possible to use custom function to define entry ? Entry creation and subscription: exometer:re_register([system_load, Metric], {function, system_monitoring, highest_system_value, [Metric], proplist, [pid, value, process_name]}, []), exometer_report:subscribe(exometer_report_influxdb, [system_load, Metric], [pid, value, process_name], 10000) Configuration file: {exometer_core, [ {report, [ {reporters, [ {exometer_report_influxdb, [ {protocol, http}, {host, <<"127.0.0.1">>}, {port, 8086}, {db, <<"db">>}, {batch_window_size, 0}, {timestamping, true} ]} ]} ]}]} I send metrics values to influxdb. And I got an error : =ERROR REPORT==== 13-Sep-2018::14:39:17 === Loading of /home/anna/projects/test_project/_build/default/lib/hackney/ebin/hackney_tcp.beam failed: not_purged =ERROR REPORT==== 13-Sep-2018::14:39:07 === msg: "badarg\nKey = {key,exometer_report_influxdb,\n [system_load,memory],\n [pid,value,process_name],\n true,[]}\nTrace: [{exometer_report,report_values,2,\n [{file,\"/home/anna/projects/test_project/_build/default/lib/exometer_core/src/exometer_report.erl\"},\n {line,1550}]},\n {exometer_report,do_report,2,\n [{file,\"/home/anna/projects/test_project/_build/default/lib/exometer_core/src/exometer_report.erl\"},\n {line,1220}]},\n {exometer_report,handle_report,4,\n [{file,\"/home/anna/projects/test_project/_build/default/lib/exometer_core/src/exometer_report.erl\"},\n {line,1203}]},\n {exometer_report,handle_info,2,\n [{file,\"/home/anna/projects/test_project/_build/default/lib/exometer_core/src/exometer_report.erl\"},\n {line,1118}]},\n {gen_server,try_dispatch,4,[{file,\"gen_server.erl\"},{line,601}]},\n {gen_server,handle_msg,5,[{file,\"gen_server.erl\"},{line,667}]},\n {proc_lib,init_p_do_apply,3,[{file,\"proc_lib.erl\"},{line,247}]}]" options: [] -------------- next part -------------- An HTML attachment was scrubbed... URL: From hans.bolinder@REDACTED Fri Sep 14 09:58:26 2018 From: hans.bolinder@REDACTED (Hans Bolinder) Date: Fri, 14 Sep 2018 07:58:26 +0000 Subject: [erlang-questions] erl_parse vs erl_syntax difference In-Reply-To: References: Message-ID: [Karlo Kuna:] > for example if we parse form (i'm omitting scan pass here) > "-type a() :: tuple()." we get {ok,{attribute,1,type,{a,{type,1,tuple,any},[]}}} > > interesting part here is: > {type,1,tuple,any} > > as far as i can see there no method how to construct valid tuple or > map type annotations using erl_syntax. Specifically if one construct > type annotation for tuple and use erl_syntax:revert/1 one gets: > > {type,1,tuple,any_size} > > any_size =/= any > > is this a bug?? Thanks for pointing it out. A fix, see commit 1b91284, will be included in Erlang/OTP 21.1. Best regards, Hans Bolinder, Erlang/OTP team, Ericsson -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Fri Sep 14 12:07:07 2018 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Fri, 14 Sep 2018 12:07:07 +0200 Subject: [erlang-questions] Running multiple beam files from an escript In-Reply-To: References: Message-ID: Yes, escripts can include ZIP archives of beam and other files. I experimented a bit with that a while ago, resulting in "eld" (an Erlang linker) https://github.com/mikpe/eld (emphasis on experiment. Some of the build tools out there already support this feature, you should probably stick with those. On Fri, Sep 14, 2018 at 9:13 AM, Roger Lipscombe wrote: > "rebar3 escriptize" -- > http://www.rebar3.org/docs/commands#section-escriptize -- lets you combine > multiple .erl files into a single escript. You can do the same with Elixir; > see > https://www.thegreatcodeadventure.com/executable-elixir-working-with-escripts/ > > Both of these take advantage of the fact that Erlang knows how to load BEAM > files from a .ZIP file, and that .ZIP files can have "garbage" on the front. > Said "garbage" being an entry point escript that invokes the main module > from the ZIP. > > I experimented with it here: https://github.com/rlipscombe/escriptize -- but > it's not maintained, so don't bother using it unless you need to support > dependencies with NIFs/ports in them. > > On 13 September 2018 at 22:47, Awsaf Rahman > wrote: >> >> Hi all, >> >> I wrote a few microbenchmarks in Erlang to measure execution time. I have >> never used escripts but want to know how they work. Is it possible to run >> multiple beam files from an escript? For example, some condition decides >> which Beam file would be invoked? >> >> Regards >> Awsaf >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From kuna.prime@REDACTED Fri Sep 14 14:00:56 2018 From: kuna.prime@REDACTED (Karlo Kuna) Date: Fri, 14 Sep 2018 14:00:56 +0200 Subject: [erlang-questions] erl_parse vs erl_syntax difference In-Reply-To: References: Message-ID: this is great news thanks for quick fix On Fri, Sep 14, 2018 at 9:58 AM Hans Bolinder wrote: > [Karlo Kuna:] > > for example if we parse form (i'm omitting scan pass here) > > "-type a() :: tuple()." we get > {ok,{attribute,1,type,{a,{type,1,tuple,any},[]}}} > > > > interesting part here is: > > {type,1,tuple,any} > > > > as far as i can see there no method how to construct valid tuple or > > map type annotations using erl_syntax. Specifically if one construct > > type annotation for tuple and use erl_syntax:revert/1 one gets: > > > > {type,1,tuple,any_size} > > > > any_size =/= any > > > > is this a bug?? > > Thanks for pointing it out. A fix, see commit 1b91284, will be > included in Erlang/OTP 21.1. > > Best regards, > > Hans Bolinder, Erlang/OTP team, Ericsson > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Fri Sep 14 14:21:38 2018 From: frank.muller.erl@REDACTED (Frank Muller) Date: Fri, 14 Sep 2018 14:21:38 +0200 Subject: [erlang-questions] erl_parse vs erl_syntax difference In-Reply-To: References: Message-ID: Hi Hans When Erlang 21.1 will be release ? /Frank [Karlo Kuna:] > > for example if we parse form (i'm omitting scan pass here) > > "-type a() :: tuple()." we get > {ok,{attribute,1,type,{a,{type,1,tuple,any},[]}}} > > > > interesting part here is: > > {type,1,tuple,any} > > > > as far as i can see there no method how to construct valid tuple or > > map type annotations using erl_syntax. Specifically if one construct > > type annotation for tuple and use erl_syntax:revert/1 one gets: > > > > {type,1,tuple,any_size} > > > > any_size =/= any > > > > is this a bug?? > > Thanks for pointing it out. A fix, see commit 1b91284, will be > included in Erlang/OTP 21.1. > > Best regards, > > Hans Bolinder, Erlang/OTP team, Ericsson > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hans.bolinder@REDACTED Fri Sep 14 16:15:40 2018 From: hans.bolinder@REDACTED (Hans Bolinder) Date: Fri, 14 Sep 2018 14:15:40 +0000 Subject: [erlang-questions] erl_parse vs erl_syntax difference In-Reply-To: References: , Message-ID: Hi, > When Erlang 21.1 will be release ? 2018-09-26. BR -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Sat Sep 15 10:22:49 2018 From: frank.muller.erl@REDACTED (Frank Muller) Date: Sat, 15 Sep 2018 10:22:49 +0200 Subject: [erlang-questions] inets rewrite as NIF? Message-ID: Hi Any plan for this? If yes, for which release? I?m asking because switching to 21.0 gave us a tremendous boost in perf (efile multi-pollset rewrite). Thanks for the hard work. /Frank -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Mon Sep 17 01:43:03 2018 From: mjtruog@REDACTED (Michael Truog) Date: Sun, 16 Sep 2018 16:43:03 -0700 Subject: [erlang-questions] [ANN] CloudI 1.7.4 Released! Message-ID: Download 1.7.4 from https://osdn.net/dl/cloudi/cloudi-1.7.4.tar.gz CloudI (https://cloudi.org/) is a "universal integrator" using an Erlang core to provide fault-tolerance with efficiency and scalability. The CloudI API provides a minimal interface to communicate among services so programming language agnostic and protocol agnostic integration can occur.? CloudI currently integrates with the programming languages: C/C++, Elixir, Erlang, Go, Haskell, Java, JavaScript/node.js, OCaml, PHP, Perl, Python, and Ruby,? Many reusable services are included that rely on the CloudI service bus. The details for this release are below: ? * Added CloudI Service API services_status function to provide ??? current uptime and availability information for any CloudI service ??? (using Erlang monotonic time) ? * Added CloudI Service API code_status function to provide ??? information about the installation and any runtime changes of ??? the CloudI service files on the filesystem ? * ZeroMQ support was removed and SSH connectivity was added to ??? cloudi_service_router as a better alternative ? * cloudi_crdt had small additions and an important bugfix ? * Added JSON support to cloudi_service_api_requests ? * Fixed Javascript CloudI API service termination (no delay now) ? * Added support in Python, Python/C and Java CloudI APIs for ??? non-member (static) functions (passed to the subscribe function) ? * Erlang/OTP 21.x compatibility ? * Haskell CloudI API dependencies were updated ? * Bugs were fixed and other improvements were added ??? (see the ChangeLog for more detail) The CloudI mailing list moved to OSDN, use the link below to subscribe: https://lists.osdn.me/mailman/listinfo/cloudi-questions My email address for CloudI communication (related to source code) is now mjtruog@REDACTED instead of mjtruog@REDACTED .? The gmail email address will continue to function, but all references to my email address in CloudI source code and documentation have been changed to mjtruog@REDACTED . Please mention any problems, issues, or ideas! Thanks, Michael SHA256 CHECKSUMS cloudi-1.7.4.tar.bz2 (9676512 bytes) b491b642db811a4e1394d8b488ce2cb3e839587be0d02c3c611bb434ef1bb249 cloudi-1.7.4.tar.gz (12064562 bytes) 0f23fd2f3e016428094a1eae192dd3c85c8895a3f284dffbb80b859b1241e97b From bchesneau@REDACTED Mon Sep 17 10:55:13 2018 From: bchesneau@REDACTED (Benoit Chesneau) Date: Mon, 17 Sep 2018 10:55:13 +0200 Subject: [erlang-questions] [ANN] Erlang rocksdb 0.21.0 released Message-ID: erlang rocksdb 0.21.0 a binding of #rocksdb for #erlang and #elixirlang applications has been released with support of latest rocksdb 5.15.10 and the addition of erlang and bitset merge operators [1] , prefix extractors and some other enhancement and fixes. Full changelog: erlang-rocksdb 0.21.0, released on 2018/09/16 ================================================ - bump to rocksdb 5.15.10 - add: erlang merge operator - add: erlang bitset merge operator - add: rocksdb:batch_data_size/1 - add: prefix_transform option Project Gitlab: https://gitlab.com/barrel-db/erlang-rocksdb The project is available on hex.pm: https://hex.pm/packages/rocksdb To get started: https://gitlab.com/barrel-db/erlang-rocksdb/wikis/Getting-started If you have any issue or question, please ask me directlty or via a ticket at https://gitlab.com/barrel-db/erlang-rocksdb/issues Enjoy! Beno?t [1] merge operator : https://gitlab.com/barrel-db/erlang-rocksdb/wikis/Erlang-Merge-Operator [2] prefix extractors: https://gitlab.com/barrel-db/erlang-rocksdb/wikis/Prefix-Seek -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Tue Sep 18 09:39:29 2018 From: lukas@REDACTED (Lukas Larsson) Date: Tue, 18 Sep 2018 09:39:29 +0200 Subject: [erlang-questions] inets rewrite as NIF? In-Reply-To: References: Message-ID: On Sat, Sep 15, 2018 at 10:23 AM Frank Muller wrote: > Hi > > Any plan for this? > Yes If yes, for which release? > As with most SW development the future is hard to predict, but we aim to have it done by OTP-22. It is a huge re-write and keeping it backwards compatible, bug free and with similar or better performance is going to be difficult. > > I?m asking because switching to 21.0 gave us a tremendous boost in perf > (efile multi-pollset rewrite). > > That's great! > Thanks for the hard work. > > /Frank > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hans.bolinder@REDACTED Tue Sep 18 12:05:01 2018 From: hans.bolinder@REDACTED (Hans Bolinder) Date: Tue, 18 Sep 2018 10:05:01 +0000 Subject: [erlang-questions] io:format/1+2 in OTP 21 doesn't take iodat() anymore In-Reply-To: <68EB9EEA-E2E7-4BA9-B184-0B13860AD7C0@alind.io> References: <68EB9EEA-E2E7-4BA9-B184-0B13860AD7C0@alind.io> Message-ID: [Thu Jul 12 10:24:48 CEST 2018 Adam Lindberg wrote:] > Haven't found any information anywhere in the changelogs about this, > but there seems to be a regression in the way io:format/1+2 works. > > It used to be possible to call it with a mixed IO list of binaries and > strings (iodata()), but this is no longer possible: > > 14> io:format(["foo", "bar"]). > foobarok > 15> io:format([<<"foo">>, "bar"]). > ** exception error: bad argument > in function io:format/3 > called as io:format(<0.63.0>,[<<"foo">>,"bar"],[]) > > It seems it was never documented, as the format was always 'atom() | > string() | binary()', but in practice 'iodata()' was allowed. Is this > change intentional or is it a bug? The change is a consequence of some refactoring (OTP-14983, option 'chars_limit'). Thanks for pointing it out; I didn't notice the change. We'll make your example work in Erlang/OTP 21.1. See commit 2d04d2b. And we'll consider introducing better runtime checking of the Format argument in Erlang/OTP 22.0. The reason is that control sequences are ignored if Format is, for instance, a list of binaries. Best regards, Hans Bolinder, Erlang/OTP team, Ericsson -------------- next part -------------- An HTML attachment was scrubbed... URL: From john@REDACTED Tue Sep 18 15:36:06 2018 From: john@REDACTED (John =?ISO-8859-1?Q?H=F6gberg?=) Date: Tue, 18 Sep 2018 15:36:06 +0200 Subject: [erlang-questions] Patch package OTP 20.3.8.9 released Message-ID: <1537277766.10064.3.camel@erlang.org> Patch Package:???????????OTP 20.3.8.9 Git Tag:?????????????????OTP-20.3.8.9 Date:????????????????????2018-09-18 Trouble Report Id:???????OTP-15292 Seq num:???????????????? System:??????????????????OTP Release:?????????????????20 Application:?????????????compiler-7.1.5.2 Predecessor:?????????????OTP 20.3.8.8 ?Check out the git tag OTP-20.3.8.9, and build a full OTP system ?including documentation. Apply one or more applications from this ?build as patches to your installation using the 'otp_patch_apply' ?tool. For information on install requirements, see descriptions for ?each application version below. ?--------------------------------------------------------------------- ?--- compiler-7.1.5.2 ------------------------------------------------ ?--------------------------------------------------------------------- ?The compiler-7.1.5.2 application can be applied independently of ?other applications on a full OTP 20 installation. ?--- Fixed Bugs and Malfunctions --- ? OTP-15292????Application(s): compiler ???????????????Fix a regression in OTP-15204 that removed .beam file ???????????????metadata that some external build tools relied on. ?Full runtime dependencies of compiler-7.1.5.2: crypto-3.6, erts-9.0, ?hipe-3.12, kernel-4.0, stdlib-2.5 ?--------------------------------------------------------------------- ?--------------------------------------------------------------------- ?--------------------------------------------------------------------- From dieter@REDACTED Tue Sep 18 13:30:38 2018 From: dieter@REDACTED (dieter@REDACTED) Date: Tue, 18 Sep 2018 11:30:38 +0000 Subject: [erlang-questions] integrate rebar3 release and systemd In-Reply-To: References: Message-ID: <047aefa2732b6c0b276e545b80dcbdb8@afterlogic.edis.at> Hi all, I spent the last two days for finding a way to integrate systemd and an erlang release, created with rebar3. Here are my findings, maybe they are useful for some of you as well. My goal was to have the release to be started automatically at boot time, and to be restarted if it totally crashes. I use Ubuntu 18. Internal application/process restarts are handled inside erlang. I started with a release $ rebar3 as prod tar and installed that into /opt/patchbox. Then I took systemd.erl from Max Lapshin (many thanks for this!) from his gist https://gist.github.com/maxlapshin/01773f0fca706acdcb4acb77d91d78bb (https://gist.github.com/maxlapshin/01773f0fca706acdcb4acb77d91d78bb) Here I added a function for writing a PID file, and integrated this with ready() and watchdog(), so by calling systemd:start_link(PidFile) from my top supervisor after a successful startup, the PID file is created and READY is sent to systemd. (And the WATCHDOG is started). write_pid(PidFile) -> {ok, F} = file:open(PidFile, [write, raw]), ok = file:write(F, os:getpid()), ok = file:close(F). init(PidFile) -> write_pid(PidFile), erlang:send_after( 100, self(), ready), erlang:send_after(60000, self(), watchdog), {ok, state}. handle_info(ready, State) -> ready(), {noreply, State}; Finally, the configuration file for systemd is this one, I saved it in /etc/systemd/system/patchbox.service [Unit] Description=Patchbox After=network-online.target Wants=network-online.target [Service] # The start script forks the erlang VM, the means that the systemd's PID # will soon be gone, and the READY message will come from an orphan Type=forking User=patchbox Group=patchbox # Allow root permission for Pre commands (mkdir, chown, etc..) PermissionsStartOnly=true ExecStartPre=/bin/mkdir -p /run/patchbox ExecStartPre=/bin/chown patchbox:patchbox /run/patchbox # Call rebar3's start script ExecStart=/opt/patchbox/bin/patchbox start Restart=on-failure TimeoutStartSec=300s # Expect WATCHDOG at least every 2 minutes WatchdogSec=120s # Allow READY and WATCHDOG from other PIDs than the original NotifyAccess=all # systemd expects the services' PID here PIDFile=/run/patchbox/patchbox.pid [Install] WantedBy=multi-user.target Kind regards, Dieter -------------- next part -------------- An HTML attachment was scrubbed... URL: From Tobias.Schlager@REDACTED Wed Sep 19 11:15:43 2018 From: Tobias.Schlager@REDACTED (Tobias Schlager) Date: Wed, 19 Sep 2018 09:15:43 +0000 Subject: [erlang-questions] integrate rebar3 release and systemd In-Reply-To: <047aefa2732b6c0b276e545b80dcbdb8@afterlogic.edis.at> References: , <047aefa2732b6c0b276e545b80dcbdb8@afterlogic.edis.at> Message-ID: <12F2115FD1CCEE4294943B2608A18FA30289E519B8@MAIL01.win.lbaum.eu> Hi Dieter, we've been using Erlang releases with Systemd on CentOS for quite some time now. No critics here, I just want to seize the opportunity and point out the solution we use: We compile Erlang/OTP ourselves using the '--enable-systemd' option. We package epmd as a separate RPM with a proper service file. In our own service files we do not rely on the rebar start scripts. Instead, we are using erlexec directly, e.g. $ systemctl cat epmd.service [Unit] Description=Erlang Port Mapper Daemon After=network-online.target Wants=network-online.target [Service] ExecStart=/usr/bin/epmd Restart=always RestartSec=2 Type=simple StandardOutput=journal StandardError=journal [Install] WantedBy=multi-user.target $ systemctl cat xxx.service [Unit] Description=xxx After=epmd.service network-online.target Wants=epmd.service [Service] Environment=BINDIR=/usr/lib64/xxx/erts-9.1/bin Environment=COOKIE=xxx Environment=VERSION=??? Environment=EMU=beam Environment=ERL_MAX_PORTS=32768 Environment=PROGNAME=xxx Environment=ROOTDIR=/usr/lib64/xxx ExecStart=/usr/lib64/xxx/erts-9.1/bin/erlexec -boot ${ROOTDIR}/releases/${VERSION}/xxx -mode embedded -config ${ROOTDIR}/releases/${VERSION}/sys.config -name xxx -setcookie ${COOKIE} -noinput +K true ExecStop=/usr/lib64/xxx/erts-9.1/bin/escript ${BINDIR}/nodetool -name xxx -setcookie ${COOKIE} stop ExecStop=/bin/sh -c 'while $(kill -0 $MAINPID 2>/dev/null); do sleep 1; done' WorkingDirectory=/usr/lib64/xxx Restart=on-failure RestartSec=2 [Install] WantedBy=multi-user.target Regards Tobias From dieter@REDACTED Wed Sep 19 12:19:02 2018 From: dieter@REDACTED (dieter@REDACTED) Date: Wed, 19 Sep 2018 10:19:02 +0000 Subject: [erlang-questions] integrate rebar3 release and systemd In-Reply-To: <12F2115FD1CCEE4294943B2608A18FA30289E519B8@MAIL01.win.lbaum.eu> References: , <047aefa2732b6c0b276e545b80dcbdb8@afterlogic.edis.at> <12F2115FD1CCEE4294943B2608A18FA30289E519B8@MAIL01.win.lbaum.eu> Message-ID: Hi Tobias, thanks a lot for your example, I appreciate it very much! In fact, I wanted to ask in my first email for other solutions and feedback, but then hit the send button too early.. My release is a standalone application, so a separate epmd-systemd unit is not important for me. But you probably have a distributed solution with several releases/nodes per machine where this approach clearly is superior. Kind regards, Dieter Am Mi., Sept. 19, 2018 11:16 schrieb Tobias Schlager : Hi Dieter, we've been using Erlang releases with Systemd on CentOS for quite some time now. No critics here, I just want to seize the opportunity and point out the solution we use: We compile Erlang/OTP ourselves using the '--enable-systemd' option. We package epmd as a separate RPM with a proper service file. In our own service files we do not rely on the rebar start scripts. Instead, we are using erlexec directly, e.g. $ systemctl cat epmd.service [Unit] Description=Erlang Port Mapper Daemon After=network-online.target Wants=network-online.target [Service] ExecStart=/usr/bin/epmd Restart=always RestartSec=2 Type=simple StandardOutput=journal StandardError=journal [Install] WantedBy=multi-user.target $ systemctl cat xxx.service [Unit] Description=xxx After=epmd.service network-online.target Wants=epmd.service [Service] Environment=BINDIR=/usr/lib64/xxx/erts-9.1/bin Environment=COOKIE=xxx Environment=VERSION=??? Environment=EMU=beam Environment=ERL_MAX_PORTS=32768 Environment=PROGNAME=xxx Environment=ROOTDIR=/usr/lib64/xxx ExecStart=/usr/lib64/xxx/erts-9.1/bin/erlexec -boot ${ROOTDIR}/releases/${VERSION}/xxx -mode embedded -config ${ROOTDIR}/releases/${VERSION}/sys.config -name xxx -setcookie ${COOKIE} -noinput +K true ExecStop=/usr/lib64/xxx/erts-9.1/bin/escript ${BINDIR}/nodetool -name xxx -setcookie ${COOKIE} stop ExecStop=/bin/sh -c 'while $(kill -0 $MAINPID 2>/dev/null); do sleep 1; done' WorkingDirectory=/usr/lib64/xxx Restart=on-failure RestartSec=2 [Install] WantedBy=multi-user.target Regards Tobias -------------- next part -------------- An HTML attachment was scrubbed... URL: From lutz.behnke@REDACTED Wed Sep 19 13:08:27 2018 From: lutz.behnke@REDACTED (Lutz Behnke) Date: Wed, 19 Sep 2018 13:08:27 +0200 Subject: [erlang-questions] Return of problematic values in gen_* functions error tuples Message-ID: <9f33b795-50f0-2a85-bcd0-d206f7f48b08@informatik.haw-hamburg.de> Hi there, I have question I have been wondering about for some time, but which is bugging me right now: Why are the offending values not returned as part of the tuple when signaling an error in virtually all of the stdlib function? Example: The gen_statem:parse_event_result/8 will terminate the state machine with {bad_return_from_state_function,badargs}, but not include the offending value. IMO it would help a lot in determining the cause of the fault. This is similar in a wide range of functions in all the gen_* modules as well as ets functions. I think I remember some other cases throughout the stdlib, but don't have any ready examples. Sometimes the functions just throw a fault with {error, badargs}. Why not include the bad argument? e.g. {bad_return_from_state_function, {badargs, TheBadTerm}} from gen_statem:parse_event_result/8 or ets:lookup/3 could return {bardarg, Key} I know that it will break the API to a certain extend, but will patches to change this be considered for inclusion in the mainly sources? I have looked at the gen_statem code and am considering a fork to help me debug. mfg lutz -- Lutz Behnke Hochschule f?r Angewandte Wissenschaften Hamburg, Labor f?r Allgemeine Informatik, phone: +49 40 42875-8156 mailto:lutz.behnke@REDACTED fax : +49 40 2803770 http://users.informatik.haw-hamburg.de/~sage Berliner Tor 7, 20099 Hamburg, Germany From dm.klionsky@REDACTED Wed Sep 19 13:45:36 2018 From: dm.klionsky@REDACTED (Dmitry Klionsky) Date: Wed, 19 Sep 2018 14:45:36 +0300 Subject: [erlang-questions] Return of problematic values in gen_* functions error tuples In-Reply-To: <9f33b795-50f0-2a85-bcd0-d206f7f48b08@informatik.haw-hamburg.de> References: <9f33b795-50f0-2a85-bcd0-d206f7f48b08@informatik.haw-hamburg.de> Message-ID: <961f978d-2b6f-47b6-1815-17e1525e6653@gmail.com> >> Why are the offending values not returned as part of the tuple when signaling an error in virtually all of the stdlib function? The values might be huge. On 09/19/2018 02:08 PM, Lutz Behnke wrote: > Hi there, > > I have question I have been wondering about for some time, but which > is bugging me right now: > > ????Why are the offending values not returned as part of the tuple > ??????? when signaling an error in virtually all of the stdlib function? > > Example: The gen_statem:parse_event_result/8 will terminate the state > machine with {bad_return_from_state_function,badargs}, but not include > the offending value. > IMO it would help a lot in determining the cause of the fault. > This is similar in a wide range of functions in all the gen_* modules > as well as ets functions. I think I remember some other cases > throughout the stdlib, but don't have any ready examples. > Sometimes the functions just throw a fault with {error, badargs}. > > Why not include the bad argument? > ?e.g. {bad_return_from_state_function, {badargs, TheBadTerm}} > ? from gen_statem:parse_event_result/8 > or > ?ets:lookup/3 could return {bardarg, Key} > > > I know that it will break the API to a certain extend, but will > patches to change this be considered for inclusion in the mainly > sources? I have looked at the gen_statem code and am considering a > fork to help me debug. > > mfg lutz > -- BR, Dmitry From raimo+erlang-questions@REDACTED Wed Sep 19 14:03:02 2018 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 19 Sep 2018 14:03:02 +0200 Subject: [erlang-questions] Return of problematic values in gen_* functions error tuples In-Reply-To: <9f33b795-50f0-2a85-bcd0-d206f7f48b08@informatik.haw-hamburg.de> References: <9f33b795-50f0-2a85-bcd0-d206f7f48b08@informatik.haw-hamburg.de> Message-ID: <20180919120302.GA95392@erix.ericsson.se> On Wed, Sep 19, 2018 at 01:08:27PM +0200, Lutz Behnke wrote: > Hi there, > > I have question I have been wondering about for some time, but which is > bugging me right now: > > Why are the offending values not returned as part of the tuple > when signaling an error in virtually all of the stdlib function? > > Example: The gen_statem:parse_event_result/8 will terminate the state > machine with {bad_return_from_state_function,badargs}, but not include > the offending value. Bad example on your part, I think. That particular exception is of class 'error' with reason {bad_return_from_state_function,Result} where Result is the term that the state function returned that is: the offending value. So if the state machine terminates with that reason there is a state function that has returned 'badargs' instead of an event_handler_result(StateType). Which state function that returned 'badargs' is unfortunately lost since the code has returned from the function, but the current state is recorded in the error report, which indirectly points out the offending state function. > IMO it would help a lot in determining the cause of the fault. > This is similar in a wide range of functions in all the gen_* modules as > well as ets functions. I think I remember some other cases throughout > the stdlib, but don't have any ready examples. > Sometimes the functions just throw a fault with {error, badargs}. That would be "the function just throws an exception of class 'error' with reason 'badarg'", just for the record. > > Why not include the bad argument? > e.g. {bad_return_from_state_function, {badargs, TheBadTerm}} > from gen_statem:parse_event_result/8 > or > ets:lookup/3 could return {bardarg, Key} That would be ets:lookup_element/3, and yes, it throws an exception of class 'error' with reason just 'badarg'. That is a standard reason, and introducing {badarg,Info} would probably surprise lots of debug tools. If you get the stack trace from that exception you will see that the arguments are in the first element, so therefore it redundant to have any part of the arguments in the exception reason. gen_statem throws a more specific reason that does not use the well known 'badarg' reason, partly because in your example it is not an argument to a function that is bad - it is a return value, and partly to introduce a more describing keyword for which it is not unexpected to have supplementary info. > > > I know that it will break the API to a certain extend, but will patches > to change this be considered for inclusion in the mainly sources? I have > looked at the gen_statem code and am considering a fork to help me debug. Changing return values of API functions is not backwards compatible, so that would most probably not be considered. Changing an exception reason is less sensitive, but changing the toplevel term in this case 'badarg' into a tuple is rather likely to break code that catches and inspects the reason. If the documentation says that the reason is something like {broad_reason,Description} the Description could probably be changed since code that follows the documentation could only match for {broad_reason,_} and would not break. In short we can change what is not documented. (with lots of exceptions, unfortunately) Improvement suggestions of gen_statem's exception reasons are welcome, but I have already put some effort into making them useful, so I am a bit surprised that you bring gen_statem up as the first module to improve... And I do not agree with the example you gave - have you got better examples or can you please clarify? Best Regards / Raimo Niskanen > > mfg lutz > > -- > Lutz Behnke > Hochschule f?r Angewandte Wissenschaften Hamburg, > Labor f?r Allgemeine Informatik, > > phone: +49 40 42875-8156 mailto:lutz.behnke@REDACTED > fax : +49 40 2803770 http://users.informatik.haw-hamburg.de/~sage > Berliner Tor 7, 20099 Hamburg, Germany > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From lutz.behnke@REDACTED Wed Sep 19 17:10:51 2018 From: lutz.behnke@REDACTED (Lutz Behnke) Date: Wed, 19 Sep 2018 17:10:51 +0200 Subject: [erlang-questions] Return of problematic values in gen_* functions error tuples In-Reply-To: <20180919120302.GA95392@erix.ericsson.se> References: <9f33b795-50f0-2a85-bcd0-d206f7f48b08@informatik.haw-hamburg.de> <20180919120302.GA95392@erix.ericsson.se> Message-ID: <98c127a5-4a79-0399-29d3-21d7ac40a943@informatik.haw-hamburg.de> Hi Raimo, you are right. I misread the code in gen_statem.erl:1314. I thought the actual result is thrown away (by using '_ ->....'), but it is available in Result. So that has helped me find the spot to continue debugging my own code (which should not throw/return badarg AFAIK). Ok, I understand the argument of the parameter being in the stack trace. But I seem to run in to situations where crossing the process boundary will invalidate the stack trace. Ah! I have just understood that the whole problem was of my very own creation (and has been for some time, due to a misunderstanding of the proper use of the throw primitive.) Thank you for shining a light at the sordid underbelly of my code. I will now proceed to duplicate methods and concepts from the way gen_statem handles errors.... :-) Thx mfg lutz Am 19.09.2018 um 14:03 schrieb Raimo Niskanen: > On Wed, Sep 19, 2018 at 01:08:27PM +0200, Lutz Behnke wrote: >> Hi there, >> >> I have question I have been wondering about for some time, but which is >> bugging me right now: >> >> Why are the offending values not returned as part of the tuple >> when signaling an error in virtually all of the stdlib function? >> >> Example: The gen_statem:parse_event_result/8 will terminate the state >> machine with {bad_return_from_state_function,badargs}, but not include >> the offending value. > > Bad example on your part, I think. > > That particular exception is of class 'error' with reason > {bad_return_from_state_function,Result} where Result is the term > that the state function returned that is: the offending value. > > So if the state machine terminates with that reason there is > a state function that has returned 'badargs' instead of > an event_handler_result(StateType). > > Which state function that returned 'badargs' is unfortunately lost since > the code has returned from the function, but the current state is recorded > in the error report, which indirectly points out the offending state > function. > >> IMO it would help a lot in determining the cause of the fault. >> This is similar in a wide range of functions in all the gen_* modules as >> well as ets functions. I think I remember some other cases throughout >> the stdlib, but don't have any ready examples. >> Sometimes the functions just throw a fault with {error, badargs}. > > That would be "the function just throws an exception of class 'error' > with reason 'badarg'", just for the record. > >> >> Why not include the bad argument? >> e.g. {bad_return_from_state_function, {badargs, TheBadTerm}} >> from gen_statem:parse_event_result/8 >> or >> ets:lookup/3 could return {bardarg, Key} > > That would be ets:lookup_element/3, and yes, it throws an exception of > class 'error' with reason just 'badarg'. > > That is a standard reason, and introducing {badarg,Info} would probably > surprise lots of debug tools. > > If you get the stack trace from that exception you will see that > the arguments are in the first element, so therefore it redundant > to have any part of the arguments in the exception reason. > > gen_statem throws a more specific reason that does not use the well known > 'badarg' reason, partly because in your example it is not an argument to a > function that is bad - it is a return value, and partly to introduce a > more describing keyword for which it is not unexpected to have > supplementary info. > >> >> >> I know that it will break the API to a certain extend, but will patches >> to change this be considered for inclusion in the mainly sources? I have >> looked at the gen_statem code and am considering a fork to help me debug. > > Changing return values of API functions is not backwards compatible, so > that would most probably not be considered. > > Changing an exception reason is less sensitive, but changing the toplevel > term in this case 'badarg' into a tuple is rather likely to break code that > catches and inspects the reason. If the documentation says that the reason > is something like {broad_reason,Description} the Description could > probably be changed since code that follows the documentation could only > match for {broad_reason,_} and would not break. > > In short we can change what is not documented. > (with lots of exceptions, unfortunately) > > Improvement suggestions of gen_statem's exception reasons are welcome, but > I have already put some effort into making them useful, so I am a bit > surprised that you bring gen_statem up as the first module to improve... > And I do not agree with the example you gave - have you got better examples > or can you please clarify? > > Best Regards > / Raimo Niskanen > > >> >> mfg lutz >> >> -- >> Lutz Behnke >> Hochschule f?r Angewandte Wissenschaften Hamburg, >> Labor f?r Allgemeine Informatik, >> >> phone: +49 40 42875-8156 mailto:lutz.behnke@REDACTED >> fax : +49 40 2803770 http://users.informatik.haw-hamburg.de/~sage >> Berliner Tor 7, 20099 Hamburg, Germany >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -- Lutz Behnke Hochschule f?r Angewandte Wissenschaften Hamburg, Labor f?r Allgemeine Informatik, phone: +49 40 42875-8156 mailto:lutz.behnke@REDACTED fax : +49 40 2803770 http://users.informatik.haw-hamburg.de/~sage Berliner Tor 7, 20099 Hamburg, Germany From michael.coquard@REDACTED Wed Sep 19 17:19:25 2018 From: michael.coquard@REDACTED (=?UTF-8?Q?Micha=c3=abl_COQUARD?=) Date: Wed, 19 Sep 2018 17:19:25 +0200 Subject: [erlang-questions] Dirty CPU scheduler and pure Erlang code Message-ID: <1e66070d-41d1-0f62-4e9c-97346d9ba10a@ac-nancy-metz.fr> Hi, I try to understand how the processes are scheduled on the dirty schedulers. As I understand, the dirty schedulers are meant to execute uninterruptible native code to avoid blocking the standard schedulers (and other Erlang processes) for a long time. I noticed the majority of all memory allocations are scheduled on the dirty schedulers: mmap(), munmap() and mremap() calls from the mseg_alloc and sys_alloc Now, consider the following code: -module(foo). -compile([export_all,nowarn_export_all]). loop_reverse_erl() -> ??? L = lists:seq(1,1000000), ??? loop_reverse_erl(L). loop_reverse_erl(L) -> ??? my_reverse(L,[]), ??? loop_reverse_erl(L). % vanilla reverse written in pure Erlang from efficiency_guide.erl my_reverse([H|T], Acc) -> my_reverse(T, [H|Acc]); my_reverse([], Acc) -> Acc. When I launch the loop (foo:loop_reverse_erl().) from the shell, I see an activity on one of a dirty scheduler thread (with top -H -p ) The dirty scheduler doesn't do anything on a simple infinite loop like this loop_reverse_erl(L) -> ??? loop_reverse_erl(L). I was expecting only native functions (BIFs / NIFs) are executed on the dirty schedulers. Is it the expected behavior ? If yes, which conditions trigger the choice of the "dirty" or "normal" scheduler ? OS: Linux RHEL 7 Erlang version: OTP/21 erts-10.0 (same on OTP/20) Michael From gomoripeti@REDACTED Wed Sep 19 17:32:50 2018 From: gomoripeti@REDACTED (=?UTF-8?B?UGV0aSBHw7Ztw7ZyaQ==?=) Date: Wed, 19 Sep 2018 17:32:50 +0200 Subject: [erlang-questions] Dirty CPU scheduler and pure Erlang code In-Reply-To: <1e66070d-41d1-0f62-4e9c-97346d9ba10a@ac-nancy-metz.fr> References: <1e66070d-41d1-0f62-4e9c-97346d9ba10a@ac-nancy-metz.fr> Message-ID: Hi Michael, With the module `msacc` you can get more info about what the dirty scheduler is busy with. For example it can do garbage collection too. On Wed, Sep 19, 2018 at 5:19 PM, Micha?l COQUARD < michael.coquard@REDACTED> wrote: > Hi, > > I try to understand how the processes are scheduled on the dirty > schedulers. > > As I understand, the dirty schedulers are meant to execute uninterruptible > native code to avoid blocking the standard schedulers (and other Erlang > processes) for a long time. > > I noticed the majority of all memory allocations are scheduled on the > dirty schedulers: mmap(), munmap() and mremap() calls from the mseg_alloc > and sys_alloc > > Now, consider the following code: > -module(foo). > -compile([export_all,nowarn_export_all]). > > loop_reverse_erl() -> > L = lists:seq(1,1000000), > loop_reverse_erl(L). > > loop_reverse_erl(L) -> > my_reverse(L,[]), > loop_reverse_erl(L). > > % vanilla reverse written in pure Erlang from efficiency_guide.erl > my_reverse([H|T], Acc) -> my_reverse(T, [H|Acc]); > my_reverse([], Acc) -> Acc. > > When I launch the loop (foo:loop_reverse_erl().) from the shell, I see an > activity on one of a dirty scheduler thread (with top -H -p beam.smp>) > > The dirty scheduler doesn't do anything on a simple infinite loop like this > loop_reverse_erl(L) -> > loop_reverse_erl(L). > > I was expecting only native functions (BIFs / NIFs) are executed on the > dirty schedulers. > > Is it the expected behavior ? If yes, which conditions trigger the choice > of the "dirty" or "normal" scheduler ? > > OS: Linux RHEL 7 > Erlang version: OTP/21 erts-10.0 (same on OTP/20) > > Michael > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jordan.day@REDACTED Wed Sep 19 20:32:13 2018 From: jordan.day@REDACTED (Jordan Day) Date: Wed, 19 Sep 2018 13:32:13 -0500 Subject: [erlang-questions] NIFs and lists of resources Message-ID: Apologies that this is more of a C question than an Erlang question? I?m working on a NIF accessing a vendor?s C API and wondering what the correct approach is to handling a function from which I?d like to return a list of resources. While my functions handling individual resources seem to be working as expected (calling the destructors when the item is GC?d on the erlang side), I?m not seeing that happen with a function where I?m returning a list of resources. In code where I?d normally do something like vendor_template templates[num_templates]; fill_templates(templates); ... return templates; I?d like each vendor_template struct to become a resource I can pass around and have GC?d when I?m done with them, on a per-item basis. I presumed I could do the normal resource-related calls and then return an erlang list, but I?m not seeing the destructor function called when the list should be getting GC?d on the erlang side. Example: vendor_template *templates = enif_alloc_resource(TEMPLATE_TYPE, sizeof(vendor_template) * num_templates); ERL_NIF_TERM template_res[num_templates]; fill_templates(templates); for (int i = 0; i < num_templates; i++) { template_res[i] = enif_make_resource(env, &(templates[i])); enif_release_resource(&(templates[i])); } ERL_NIF_TERM template_list = enif_make_list_from_array(env, template_res, num_templates); return template_list; The returned list of resources seem to be correct, as I can use their individual references in subsequent NIF calls, but when the returned list is discarded, I never see the destructor function called. Any tips on what I might be doing wrong? Does the enif_make_list_from_array/2 call ?break? this? If so, how can I return my list of resources? Regards, Jordan -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker@REDACTED Wed Sep 19 21:23:33 2018 From: sverker@REDACTED (Sverker Eriksson) Date: Wed, 19 Sep 2018 21:23:33 +0200 Subject: [erlang-questions] NIFs and lists of resources In-Reply-To: References: Message-ID: <1537385013.4112.36.camel@erlang.org> Each call to enif_alloc_resource() returns ONE resource object. enif_make_resource expects a pointer returned from enif_alloc_resource(), and only your &templates [0] is such a pointer. So, you probable want to call enif_alloc_resource one time for every resource in your array/list. /Sverker On ons, 2018-09-19 at 13:32 -0500, Jordan Day wrote: > Apologies that this is more of a C question than an Erlang question? > > I?m working on a NIF accessing a vendor?s C API and wondering what the correct > approach is to handling a function from which I?d like to return a list of > resources. While my functions handling individual resources seem to be working > as expected (calling the destructors when the item is GC?d on the erlang > side), I?m not seeing that happen with a function where I?m returning a list > of resources. > > In code where I?d normally do something like > vendor_template templates[num_templates]; > fill_templates(templates); > ... > return templates; > > > I?d like each vendor_template struct to become a resource I can pass around > and have GC?d when I?m done with them, on a per-item basis. I presumed I could > do the normal resource-related calls and then return an erlang list, but I?m > not seeing the destructor function called when the list should be getting GC?d > on the erlang side. > > Example: > > vendor_template *templates = enif_alloc_resource(TEMPLATE_TYPE, > sizeof(vendor_template) * num_templates); > ERL_NIF_TERM template_res[num_templates]; > fill_templates(templates); > for (int i = 0; i < num_templates; i++) { > ? template_res[i] = enif_make_resource(env, &(templates[i])); > ? enif_release_resource(&(templates[i])); > } > > ERL_NIF_TERM template_list = enif_make_list_from_array(env, template_res, > num_templates); > return template_list; > > The returned list of resources seem to be correct, as I can use their > individual references in subsequent NIF calls, but when the returned list is > discarded, I never see the destructor function called. > > Any tips on what I might be doing wrong? Does the enif_make_list_from_array/2 > call ?break? this? If so, how can I return my list of resources? > > Regards, > Jordan > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.coquard@REDACTED Thu Sep 20 09:06:42 2018 From: michael.coquard@REDACTED (=?UTF-8?Q?Micha=c3=abl_COQUARD?=) Date: Thu, 20 Sep 2018 09:06:42 +0200 Subject: [erlang-questions] Dirty CPU scheduler and pure Erlang code In-Reply-To: References: <1e66070d-41d1-0f62-4e9c-97346d9ba10a@ac-nancy-metz.fr> Message-ID: <1a0974da-7e24-8499-5a01-f7a0a707528c@ac-nancy-metz.fr> Hi Peti, Thank you for your quick response, this is exactly what I was looking for. It was the garbage collector activity. Michael Le 19/09/2018 ? 17:32, Peti G?m?ri a ?crit?: > Hi Michael, > > With the module `msacc` you can get more info about what the dirty scheduler is busy with. For example it can do garbage collection too. > > On Wed, Sep 19, 2018 at 5:19 PM, Micha?l COQUARD > wrote: > > Hi, > > I try to understand how the processes are scheduled on the dirty schedulers. > > As I understand, the dirty schedulers are meant to execute uninterruptible native code to avoid blocking the standard schedulers (and other > Erlang processes) for a long time. > > I noticed the majority of all memory allocations are scheduled on the dirty schedulers: mmap(), munmap() and mremap() calls from the mseg_alloc > and sys_alloc > > Now, consider the following code: > -module(foo). > -compile([export_all,nowarn_export_all]). > > loop_reverse_erl() -> > ??? L = lists:seq(1,1000000), > ??? loop_reverse_erl(L). > > loop_reverse_erl(L) -> > ??? my_reverse(L,[]), > ??? loop_reverse_erl(L). > > % vanilla reverse written in pure Erlang from efficiency_guide.erl > my_reverse([H|T], Acc) -> my_reverse(T, [H|Acc]); > my_reverse([], Acc) -> Acc. > > When I launch the loop (foo:loop_reverse_erl().) from the shell, I see an activity on one of a dirty scheduler thread (with top -H -p beam.smp>) > > The dirty scheduler doesn't do anything on a simple infinite loop like this > loop_reverse_erl(L) -> > ??? loop_reverse_erl(L). > > I was expecting only native functions (BIFs / NIFs) are executed on the dirty schedulers. > > Is it the expected behavior ? If yes, which conditions trigger the choice of the "dirty" or "normal" scheduler ? > > OS: Linux RHEL 7 > Erlang version: OTP/21 erts-10.0 (same on OTP/20) > > Michael > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Sep 20 13:28:05 2018 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 20 Sep 2018 13:28:05 +0200 Subject: [erlang-questions] Dirty CPU scheduler and pure Erlang code In-Reply-To: <1e66070d-41d1-0f62-4e9c-97346d9ba10a@ac-nancy-metz.fr> References: <1e66070d-41d1-0f62-4e9c-97346d9ba10a@ac-nancy-metz.fr> Message-ID: On Wed, Sep 19, 2018 at 5:27 PM Micha?l COQUARD < michael.coquard@REDACTED> wrote: > > I was expecting only native functions (BIFs / NIFs) are executed on the > dirty schedulers. > > Any work that is lengthy and not cooperative is better served with a task on the dirty scheduler. That way, we avoid blocking the scheduler thread which would mean we would stall processes. Long-running garbage collections are one such thing, so they can get backgrounded on the dirty scheduler. Once the GC is done, the process is moved back onto a "normal" scheduler. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Thu Sep 20 13:34:14 2018 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 20 Sep 2018 13:34:14 +0200 Subject: [erlang-questions] Return of problematic values in gen_* functions error tuples In-Reply-To: <98c127a5-4a79-0399-29d3-21d7ac40a943@informatik.haw-hamburg.de> References: <9f33b795-50f0-2a85-bcd0-d206f7f48b08@informatik.haw-hamburg.de> <20180919120302.GA95392@erix.ericsson.se> <98c127a5-4a79-0399-29d3-21d7ac40a943@informatik.haw-hamburg.de> Message-ID: <20180920113414.GA12874@erix.ericsson.se> On Wed, Sep 19, 2018 at 05:10:51PM +0200, Lutz Behnke wrote: > Hi Raimo, : > > Thank you for shining a light at the sordid underbelly of my code. I > will now proceed to duplicate methods and concepts from the way > gen_statem handles errors.... :-) Glad to be of assistance! ;-) > > Thx > > mfg lutz > > -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From sverker@REDACTED Thu Sep 20 15:04:51 2018 From: sverker@REDACTED (Sverker Eriksson) Date: Thu, 20 Sep 2018 15:04:51 +0200 Subject: [erlang-questions] Patch package OTP 19.3.6.11 released Message-ID: <1537448691.4112.47.camel@erlang.org> Patch Package:???????????OTP 19.3.6.11 Git Tag:?????????????????OTP-19.3.6.11 Date:????????????????????2018-09-20 Trouble Report Id:???????OTP-14609 Seq num:?????????????????ERIERL-238 System:??????????????????OTP Release:?????????????????19 Application:?????????????erts-8.3.5.6 Predecessor:?????????????OTP 19.3.6.10 ?Check out the git tag OTP-19.3.6.11, and build a full OTP system ?including documentation. Apply one or more applications from this ?build as patches to your installation using the 'otp_patch_apply' ?tool. For information on install requirements, see descriptions for ?each application version below. ?--------------------------------------------------------------------- ?--- erts-8.3.5.6 ---------------------------------------------------- ?--------------------------------------------------------------------- ?Note! The erts-8.3.5.6 application can *not* be applied independently ???????of other applications on an arbitrary OTP 19 installation. ???????On a full OTP 19 installation, also the following runtime ???????dependency has to be satisfied: ???????-- sasl-3.0.1 (first satisfied in OTP 19.1) ?--- Fixed Bugs and Malfunctions --- ? OTP-14609????Application(s): erts ???????????????Related Id(s): ERIERL-238 ???????????????Fixed small memory leak that could occur when sending ???????????????to a terminating port. ?Full runtime dependencies of erts-8.3.5.6: kernel-5.0, sasl-3.0.1, ?stdlib-3.0 ?--------------------------------------------------------------------- ?--------------------------------------------------------------------- ?--------------------------------------------------------------------- From tuncer.ayaz@REDACTED Thu Sep 20 19:29:59 2018 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Thu, 20 Sep 2018 17:29:59 +0000 Subject: [erlang-questions] [ANN] Triq community fork Message-ID: We are pleased to announce the community fork of Triq. Triq (Term Reductive Invariant Questant) is an Apache licensed QuickCheck library for Erlang. It is a continuation and community fork of https://github.com/krestenkrab/triq. This fork is in *no way* affiliated with or a product of Trifork. | Homepage | https://triq.gitlab.io | Hex.pm | https://hex.pm/packages/triq | Team | https://gitlab.com/triq/triq/project_members If you use Triq and/or want to contribute, we do welcome new team members. It's a community fork after all. * Writing QuickCheck properties with Triq To write properties with Triq, you just include triq.hrl: -include_lib("triq/include/triq.hrl"). Modules compiled with the triq.hrl header auto-export all functions named prop_* and have a function added, called check/0, which runs triq:check/1 on all the properties in the module. Further, adding the attribute -triq(eunit) will generate EUnit tests for all properties, turning your module into a regular EUnit test suite. If you use erlang.mk, you will typically want to use the built-in Triq plugin (https://erlang.mk/guide/triq.html) to check properties. Otherwise, we highly recommend letting Triq generate EUnit tests, thus arriving at a demo module like this: -module(triq_demo). -include_lib("triq/include/triq.hrl"). -triq(eunit). prop_append() -> ?FORALL({Xs,Ys},{list(int()),list(int())}, lists:reverse(Xs++Ys) == lists:reverse(Ys) ++ lists:reverse(Xs)). Now, all you have to do is run rebar3 eunit: $ rebar3 eunit -v ===> Verifying dependencies... ===> Compiling triq_demo ===> Performing EUnit tests... ======================== EUnit ======================== file "triq_demo.app" application 'triq_demo' triq_demo:5: append_test_ (module 'triq_demo')...[0.262 s] ok [done in 0.269 s] [done in 0.274 s] ======================================================= Test passed. If you use -triq({eunit, [{runs, N}]}), then Triq will do N runs for each property in the module, which is equivalent to calling triq:check(Module,N). This can be useful to make Triq try more (or less) cases than the default. For advanced features, please consult the API docs. * Obtaining Triq Triq is available via Hex.pm and its git repository. If your build tool supports it, we suggest to use the Hex.pm package. | rebar.config | {deps, [triq]} | erlang.mk | DEPS = triq | mix.exs | {:triq, "~> 1.*"} From t@REDACTED Fri Sep 21 02:30:28 2018 From: t@REDACTED (Tristan Sloughter) Date: Thu, 20 Sep 2018 18:30:28 -0600 Subject: [erlang-questions] [ANN] Rebar3 3.7.0-rc1 Message-ID: <1537489828.3271982.1515457520.38A4A6EA@webmail.messagingengine.com> Thanks to funding from the Erlang Software Foundation (currently still the IEUG) for this feature development on rebar3, I'm happy to announce rebar3 3.7.0-rc1, available at https://github.com/erlang/rebar3/releases/tag/3.7.0-rc1. We have released this version as a release candidate (rc) because it contains fundamental changes to rebar3's package handling. This should be a backwards compatible change for users, but significant internally. Even if you don't make use of the new features, we would like to receive your feedback to ensure nothing has broken for your everyday use. The main new features in this release: * Support for hex v2. This is a more efficient registry implementation which is updated per-package only as needed, which should save memory, disk space, and allow for faster start times. * Support for multiple repository fetching. Repositories are defined as a list and searched sequentially for the first match. This allows the development and overlaying of private indexes and mirrors * Multiple repos includes private Hex organizations (https://hex.pm/pricing) which are defined the same way as they are for mix users. See rebar3_hex (https://github.com/tsloughter/rebar3_hex) for instructions on how to authenticate against private organizations and working with repositories Additionally, Elixir packages are no longer filtered from the registry. I'm in the process of updating the plugin, https://github.com/barrel-db/rebar3_elixir_compile, for these changes to provide much smoother support for all mix packages in rebar3 projects. There will also be an updated resource plugin API to be stabilized soon enough. Tristan From frank.muller.erl@REDACTED Fri Sep 21 08:40:00 2018 From: frank.muller.erl@REDACTED (Frank Muller) Date: Fri, 21 Sep 2018 08:40:00 +0200 Subject: [erlang-questions] Throttling HTTP requests Message-ID: Hi guys We have an HTTP web service based Cowboy which is receiving more and more traffic. We would like to implement a throttling mechanism per IP address to limit how many request/sec the same IP address can make. Is there any library out there for that? Or any (simple yet efficient) algorithm to implement? Ideas & feedabacks are very welcome. Thanks in advance. /Frank -------------- next part -------------- An HTML attachment was scrubbed... URL: From Tobias.Schlager@REDACTED Fri Sep 21 09:02:35 2018 From: Tobias.Schlager@REDACTED (Tobias Schlager) Date: Fri, 21 Sep 2018 07:02:35 +0000 Subject: [erlang-questions] Throttling HTTP requests In-Reply-To: References: Message-ID: <12F2115FD1CCEE4294943B2608A18FA30289E51B98@MAIL01.win.lbaum.eu> Hi Frank, I've implemented a simple form of rate limiting in this [1] little project (module http2smtp_rate). The module is probably not what you need but the idea behind it, could be used as a starting point. The basic idea is the following: Everytime a request comes in, the request process makes a gen_server call to this server (you could also spawn the limiter processes per source IP address for sharding). This request will increment and return a counter (specific for this IP address). If the counter exceeds the limit, the request gets terminated immediately. The server has an internal timer that fires once per desired interval and resets all counters. Regards Tobias [1] https://github.com/lindenbaum/http2smtp -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Fri Sep 21 09:15:28 2018 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Fri, 21 Sep 2018 10:15:28 +0300 Subject: [erlang-questions] Throttling HTTP requests In-Reply-To: References: Message-ID: Hi, Global throttling might be challenging. You would need to run a proxy and define throttling policies there. I would discard this option unless you are talking about large service. Let?s consider local throttling policies, where each api nodes maintain counters independently each other. It was not clear do you want to reject request or put it into queue so that customers are still served but they latency grows. There are few easy solutions, which does not require any Erlang * Nginx * Linux traffic shaping If you keen to implement this feature in Erlang then please consider a circuit breaker pattern. You can create a fuse per IP address or sub network to reject a traffic. Best Regards, Dmitry > On 21 Sep 2018, at 9.40, Frank Muller wrote: > > Hi guys > > We have an HTTP web service based Cowboy which is receiving more and more traffic. > > We would like to implement a throttling mechanism per IP address to limit how many request/sec the same IP address can make. > > Is there any library out there for that? > Or any (simple yet efficient) algorithm to implement? > > Ideas & feedabacks are very welcome. > > Thanks in advance. > > /Frank > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: Message signed with OpenPGP URL: From roger@REDACTED Fri Sep 21 09:19:30 2018 From: roger@REDACTED (Roger Lipscombe) Date: Fri, 21 Sep 2018 08:19:30 +0100 Subject: [erlang-questions] Throttling HTTP requests In-Reply-To: References: Message-ID: I've not tried, but it looks like you could use https://github.com/uwiger/jobs and write some cowboy middleware that returns 429. Or do what we do: since our HTTP-stuff is fronted by nginx (for SSL termination and upstream service routing), we used https://www.nginx.com/blog/rate-limiting-nginx/ On 21 September 2018 at 07:40, Frank Muller wrote: > Hi guys > > We have an HTTP web service based Cowboy which is receiving more and more > traffic. > > We would like to implement a throttling mechanism per IP address to limit > how many request/sec the same IP address can make. > > Is there any library out there for that? > Or any (simple yet efficient) algorithm to implement? > > Ideas & feedabacks are very welcome. > > Thanks in advance. > > /Frank > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From k.petrauskas@REDACTED Fri Sep 21 09:32:33 2018 From: k.petrauskas@REDACTED (Karolis Petrauskas) Date: Fri, 21 Sep 2018 10:32:33 +0300 Subject: [erlang-questions] Throttling HTTP requests In-Reply-To: References: Message-ID: I was trying to use https://github.com/uwiger/jobs for rate limiting in my recent project. With the spike pattern, this library was not performing. I had to accept 50k requests in ~10-20s, queue them and then handle them at a limited rate (10tps). The rate was limited way more, than configured (1tps instead of 10tps, etc). Then I switched to a custom gen_server with timer:sleep/1 and queue in the process inbox. Karolis On Fri, Sep 21, 2018 at 10:19 AM Roger Lipscombe wrote: > I've not tried, but it looks like you could use > https://github.com/uwiger/jobs and write some cowboy middleware that > returns 429. > > Or do what we do: since our HTTP-stuff is fronted by nginx (for SSL > termination and upstream service routing), we used > https://www.nginx.com/blog/rate-limiting-nginx/ > > On 21 September 2018 at 07:40, Frank Muller > wrote: > >> Hi guys >> >> We have an HTTP web service based Cowboy which is receiving more and more >> traffic. >> >> We would like to implement a throttling mechanism per IP address to limit >> how many request/sec the same IP address can make. >> >> Is there any library out there for that? >> Or any (simple yet efficient) algorithm to implement? >> >> Ideas & feedabacks are very welcome. >> >> Thanks in advance. >> >> /Frank >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gerhard@REDACTED Fri Sep 21 11:00:12 2018 From: gerhard@REDACTED (Gerhard Lazu) Date: Fri, 21 Sep 2018 10:00:12 +0100 Subject: [erlang-questions] Throttling HTTP requests In-Reply-To: References: Message-ID: Last time I had to do something similar I went with Cloudflare. It's been a few years now and it's still the cheaper option with no management overhad. If delegating this functionality to a SaaS is an option for you, it's worth considering. On 21 Sep 2018, 07:40 +0100, Frank Muller , wrote: > Hi guys > > We have an HTTP web service based Cowboy which is receiving more and more traffic. > > We would like to implement a throttling mechanism per IP address to limit how many request/sec the same IP address can make. > > Is there any library out there for that? > Or any (simple yet efficient) algorithm to implement? > > Ideas & feedabacks are very welcome. > > Thanks in advance. > > /Frank > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ruan.pienaar@REDACTED Fri Sep 21 11:12:55 2018 From: ruan.pienaar@REDACTED (Ruan Pienaar) Date: Fri, 21 Sep 2018 10:12:55 +0100 Subject: [erlang-questions] Throttling HTTP requests In-Reply-To: References: Message-ID: spent a few hours writing a erlang library for rate limiting within the vm. https://github.com/ruanpienaar/tempo On Fri, Sep 21, 2018 at 7:40 AM Frank Muller wrote: > Hi guys > > We have an HTTP web service based Cowboy which is receiving more and more > traffic. > > We would like to implement a throttling mechanism per IP address to limit > how many request/sec the same IP address can make. > > Is there any library out there for that? > Or any (simple yet efficient) algorithm to implement? > > Ideas & feedabacks are very welcome. > > Thanks in advance. > > /Frank > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- *Erlang Solutions cares about your data and privacy; please find all details about the basis for communicating with you and the way we process your data in our **Privacy Policy* *.You can update your email preferences or opt-out from receiving Marketing emails here .* *Ruan Pienaar* *+44 7500 708 814* -------------- next part -------------- An HTML attachment was scrubbed... URL: From paulperegud@REDACTED Fri Sep 21 13:07:04 2018 From: paulperegud@REDACTED (Paul Peregud) Date: Fri, 21 Sep 2018 11:07:04 +0000 Subject: [erlang-questions] Throttling HTTP requests In-Reply-To: References: Message-ID: https://github.com/jlouis/safetyvalve/ is worth recommending. On Fri, Sep 21, 2018 at 9:14 AM Ruan Pienaar < ruan.pienaar@REDACTED> wrote: > spent a few hours writing a erlang library for rate limiting within the vm. > https://github.com/ruanpienaar/tempo > > > > On Fri, Sep 21, 2018 at 7:40 AM Frank Muller > wrote: > >> Hi guys >> >> We have an HTTP web service based Cowboy which is receiving more and more >> traffic. >> >> We would like to implement a throttling mechanism per IP address to limit >> how many request/sec the same IP address can make. >> >> Is there any library out there for that? >> Or any (simple yet efficient) algorithm to implement? >> >> Ideas & feedabacks are very welcome. >> >> Thanks in advance. >> >> /Frank >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > -- > *Erlang Solutions cares about your data and privacy; please find all > details about the basis for communicating with you and the way we process > your data in our **Privacy Policy* > *.You can update > your email preferences or opt-out from receiving Marketing emails here > .* > > *Ruan Pienaar* > *+44 7500 708 814* > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Best regards, Paul Peregud +48602112091 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Fri Sep 21 14:45:20 2018 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 21 Sep 2018 08:45:20 -0400 Subject: [erlang-questions] Throttling HTTP requests In-Reply-To: References: Message-ID: The tricky question is about how much overload you might be expecting. The thing is that accepting an HTTP request and parsing it in order to know where it goes is a relatively costly operation (compared to just paying $2, getting a few thousand AWS instances, and throwing pre-built garbage queries at a server somewhere). If what you're trying to do is absorb and track HTTP traffic to put a cap on even costlier operations at the back-end (or on servers upstream) such as costly processing or DB queries, then you don't necessarily need to be super fast; you just need to be cheaper than the big costly operations you're trying to protect against. There are various strategies there depending on how slow you can afford to be, many of which have been described here: - using a gen_server to track and count (riskier on large servers as it may represent a single point of contention) - Jobs, which can be augmented with a lot of probes - Locks taken and handled with ETS - Circuit breakers to look at various metrics to force input to stall when becoming unstable I've written a blog post that tracks most mechanism available for overload handling and flow control there: https://ferd.ca/handling-overload.html And at Heroku, we also developed a locking library (also in the blog post above) named canal_lock (https://github.com/heroku/canal_lock) which was aimed at being able to do lock-management for backend services where the number of available nodes changes and adapts over time. Still, if you can afford it, circuit breakers or frameworks like jobs do a good there since they can adapt with a lot of local-only information (and approaches such as https://en.wikipedia.org/wiki/Additive_increase/multiplicative_decrease for self-regulation. Also look at CoDel and other algorithms) But all of this won't necessarily go far if what you have is overload traffic that is too costly to handle directly in terms of "parsing all the HTTP Requests is already too much CPU". Then you need to move to cheaper means: track connections or per-domain traffic through SNI data rather than plain HTTP requests, and impose limits there. Turn on options such as TCP_DEFER_ACCEPT which tell the kernel not to return you an 'accepted' socket until it has data waiting on the line, which saves on context switching and prevents idle connections from going up to your app (but won't do much if you're using TLS or the PROXY protocol since they expect to send info down the line right away). When that won't work, you'll need to go down to the OS-level. You can tweak the size of ACK and SYN queues for indiscriminate filtering, or use packet filtering rules where the kernel itself can do quick inspection and dropping of data you cannot accept based on some data they contain. If you're at that level though, you may need a second NIC that is privately accessible just in order to be able to go set and modify these rules while the public NIC is under stress. If that doesn't work, then you need to look at a broader scope in terms of what your data center can do, but I figure this gets far from what you needed here. In any case, the critical bit is that your system can be modeled as a queuing system; if you want to be able to take control over its stability, you must have the ability to handle validation and expulsion of overflow tasks faster than they can arrive. If you can't do that, you will not control what it is that you drop in terms of overflow. Any point of asynchrony in your system can act as an open valve where a task is reported to go fast when it goes sit into a growing queue down somewhere, so you have to be careful and make sure that your control and filtering at the edge reflects your actual bottlenecks and loads down the line. On Fri, Sep 21, 2018 at 2:40 AM Frank Muller wrote: > Hi guys > > We have an HTTP web service based Cowboy which is receiving more and more > traffic. > > We would like to implement a throttling mechanism per IP address to limit > how many request/sec the same IP address can make. > > Is there any library out there for that? > Or any (simple yet efficient) algorithm to implement? > > Ideas & feedabacks are very welcome. > > Thanks in advance. > > /Frank > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fchschneider@REDACTED Fri Sep 21 16:43:37 2018 From: fchschneider@REDACTED (Frans Schneider) Date: Fri, 21 Sep 2018 16:43:37 +0200 Subject: [erlang-questions] wxErlang modifying event handling Message-ID: Hi list, Does anybody know how to redirect an event, such as an update_ui event, from the main frame to the control which has focus. I want the behavior of the global editing menu be under the control of the wxTextCtrl which has the current focus. For this to work, the update_ui event should be processed by the text control's, instead of the main frame's event manager. wxWidgets' wxEventHandler has the ProcessEvent function for this, but that is missing in wxErlang. Thanks, Frans From g@REDACTED Sat Sep 22 15:01:45 2018 From: g@REDACTED (Guilherme Andrade) Date: Sat, 22 Sep 2018 14:01:45 +0100 Subject: [erlang-questions] Throttling HTTP requests In-Reply-To: References: Message-ID: Hello Frank, If you're looking at imposing some amount of fairness besides a rate limit, aequitas might be useful for you: https://github.com/g-andrade/aequitas It attempts to detect outliers in consumers of resources and throttle them (in your use case, these could be IP addresses performing an unusually large number of requests.) It can also enforce a collective rate limit. On Fri, 21 Sep 2018 at 07:40, Frank Muller wrote: > Hi guys > > We have an HTTP web service based Cowboy which is receiving more and more > traffic. > > We would like to implement a throttling mechanism per IP address to limit > how many request/sec the same IP address can make. > > Is there any library out there for that? > Or any (simple yet efficient) algorithm to implement? > > Ideas & feedabacks are very welcome. > > Thanks in advance. > > /Frank > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Guilherme -------------- next part -------------- An HTML attachment was scrubbed... URL: From awsafrahman1704@REDACTED Sun Sep 23 00:08:52 2018 From: awsafrahman1704@REDACTED (Awsaf Rahman) Date: Sun, 23 Sep 2018 00:08:52 +0200 Subject: [erlang-questions] Memory usage Message-ID: Hello all, I want to profile a few Erlang programs I wrote. Basically, I want to find out the following: 1) CPU and memory usage during compilation 2) CPU and memory usage during execution Is there anyway I can do this? Kind regards Awsaf -------------- next part -------------- An HTML attachment was scrubbed... URL: From paulperegud@REDACTED Sun Sep 23 20:05:01 2018 From: paulperegud@REDACTED (Paul Peregud) Date: Sun, 23 Sep 2018 20:05:01 +0200 Subject: [erlang-questions] Memory usage In-Reply-To: References: Message-ID: To profile CPU time, you can use fprof (part of OTP distribution). Why would you want to profile CPU usage during compilation? Are you planning working on Erlang compilator (erlc)? On Sun, Sep 23, 2018 at 12:12 AM Awsaf Rahman wrote: > Hello all, > > I want to profile a few Erlang programs I wrote. Basically, I want to find > out the following: > 1) CPU and memory usage during compilation > 2) CPU and memory usage during execution > > Is there anyway I can do this? > > Kind regards > Awsaf > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Best regards, Paul Peregud +48602112091 -------------- next part -------------- An HTML attachment was scrubbed... URL: From vasdeveloper@REDACTED Mon Sep 24 02:30:35 2018 From: vasdeveloper@REDACTED (Theepan) Date: Mon, 24 Sep 2018 06:00:35 +0530 Subject: [erlang-questions] ErlGuten Message-ID: Hi Team, I see many forks of ErlGuten. Which one is the latest, most stable and widely used? https://github.com/richcarl/erlguten https://github.com/CarlWright/NGerlguten https://github.com/hwatkins/erlguten/commits/master Best, Theepan -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Mon Sep 24 03:06:28 2018 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Sun, 23 Sep 2018 21:06:28 -0400 Subject: [erlang-questions] ErlGuten In-Reply-To: References: Message-ID: <7AB3CEDA-34AF-41E2-BEE9-2B27274FF680@writersglen.com> Hi Thepan, I?ve undertaken a major revision of ErlGuten called erlPress_core? announced it on Erlang questions several weeks ago. It?s available from GitHub as Writersglen/erlPress_core. I think you?ll find it much more accessible than the ErlGutens. I?ve since been polishing up the code and adding a few new features. After the launch I learned that camel case is not conventional usage when it comes to the names of Erlang applications and libraries. So I plan to release the updated version as erlpress_core. With smooth sailing I hope to release sometime this week or next. All the best, Lloyd Sent from my iPad > On Sep 23, 2018, at 8:30 PM, Theepan wrote: > > > Hi Team, > > I see many forks of ErlGuten. Which one is the latest, most stable and widely used? > > https://github.com/richcarl/erlguten > https://github.com/CarlWright/NGerlguten > https://github.com/hwatkins/erlguten/commits/master > > Best, > Theepan > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From heturing@REDACTED Mon Sep 24 09:26:03 2018 From: heturing@REDACTED (=?UTF-8?B?44Kr44Kr44Kt?=) Date: Mon, 24 Sep 2018 16:26:03 +0900 Subject: [erlang-questions] An erlang exercise which confused me Message-ID: I am reading programming erlang (2nd Ed.) and get a little confused by the exercise 16-6. Here is the question Twits are exactly 140 bytes long. Write a random-access twit storage module called twit_store.erl that exports the following: init(K) allocates space for K twits. store(N, Buf) stores twit number N(1,...K) with data Buf (a 140-byte binary) in the store. fetch(N) fetches the data for twit number N. I am confused by the meaning of init(K). I try to use function file:allocate but always get a result of {error, einval}. Could anyone help me? -------------- next part -------------- An HTML attachment was scrubbed... URL: From wojtek@REDACTED Mon Sep 24 10:53:05 2018 From: wojtek@REDACTED (Wojciech Ziniewicz) Date: Mon, 24 Sep 2018 10:53:05 +0200 Subject: [erlang-questions] How to perform running code vs. beam files integrity check Message-ID: Hello, We develop an application on a highly regulated market. Some regulators force us to protect the running code from memory modification attacks. Consider following attack: - the app is running and all modules are loaded - attacker gains access to RAM, scans it and modifies a value in the memory (or a function) so the the running code differs from the code that has been loaded during initialization - the app continues operation without noticing that code has been modified - a state where two different apps are located on a single machine: the one in RAM and the one on the disk I'm looking for *any* measures provided by erlang vm/tooling that would help mitigating this attack. The beam_lib provides tools for verifying the integrity of beam files but some kind of access to the running code would be required to close the loop here. Thanks WZ -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Mon Sep 24 13:04:29 2018 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 24 Sep 2018 13:04:29 +0200 Subject: [erlang-questions] An erlang exercise which confused me In-Reply-To: References: Message-ID: twit_store:init(N) just allocates space for K twits. Note that I said *nothing* about how or where the twits get stored - it might be in the file system, or in a database, or in a process - it's up to you to choose. Seems like you've chose to store the twits in a file. {error,einval} means that some argument to file:allocate is incorrect, you will have to investigate why this is the case. Cheers /Joe On Mon, Sep 24, 2018 at 9:26 AM, ??? wrote: > I am reading programming erlang (2nd Ed.) and get a little confused by the > exercise 16-6. > > Here is the question > > Twits are exactly 140 bytes long. Write a random-access twit storage module > called twit_store.erl that exports the following: init(K) allocates space > for K twits. store(N, Buf) stores twit number N(1,...K) with data Buf (a > 140-byte binary) in the store. fetch(N) fetches the data for twit number N. > > I am confused by the meaning of init(K). I try to use function file:allocate > but always get a result of {error, einval}. Could anyone help me? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From vasdeveloper@REDACTED Mon Sep 24 21:59:27 2018 From: vasdeveloper@REDACTED (Theepan) Date: Tue, 25 Sep 2018 01:29:27 +0530 Subject: [erlang-questions] ErlGuten In-Reply-To: <7AB3CEDA-34AF-41E2-BEE9-2B27274FF680@writersglen.com> References: <7AB3CEDA-34AF-41E2-BEE9-2B27274FF680@writersglen.com> Message-ID: Hi Lloyd, Thank you for your response. I have some quick clarifications -- * What are the major improvements made on the source ErlGuten? Did you improve support to different image formats? Any critical bugs fixed? * Is erlPress_core used in commercial applications, specifically of high throughput types? * What is the minimum erlang/OTP version required? Best, Theepan On Mon, Sep 24, 2018 at 6:36 AM Lloyd R. Prentice wrote: > Hi Thepan, > > I?ve undertaken a major revision of ErlGuten called erlPress_core? > announced it on Erlang questions several weeks ago. It?s available from > GitHub as Writersglen/erlPress_core. I think you?ll find it much more > accessible than the ErlGutens. > > I?ve since been polishing up the code and adding a few new features. After > the launch I learned that camel case is not conventional usage when it > comes to the names of Erlang applications and libraries. So I plan to > release the updated version as erlpress_core. With smooth sailing I hope to > release sometime this week or next. > > All the best, > > Lloyd > > Sent from my iPad > > On Sep 23, 2018, at 8:30 PM, Theepan wrote: > > > Hi Team, > > I see many forks of ErlGuten. Which one is the latest, most stable and > widely used? > > https://github.com/richcarl/erlguten > https://github.com/CarlWright/NGerlguten > https://github.com/hwatkins/erlguten/commits/master > > Best, > Theepan > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Mon Sep 24 22:33:28 2018 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Mon, 24 Sep 2018 22:33:28 +0200 Subject: [erlang-questions] How to perform running code vs. beam files integrity check In-Reply-To: References: Message-ID: Given the premise that the attacker is able to read and write the Erlang VM's memory, you've already lost. You could do something like: 1. Snapshot the process's text pages (VM, libc, other mapped libraries) and store copies on a separate, secured system. 2. Augment the VM to report where it loads .beam files, then via an external tool (e.g. ptrace-based) snapshot those too as modules get loaded. (Note: loaded modules have a very different representation to the .beam originals). 3. At suitable times, using again an external tool, re-read the VM's memory and compare with the snapshots stored elsewhere. This will detect unexpected code modifications. You cannot ask the VM to validate itself, since the attacker can modify and neutralize that code. However, equally bad may be if the attacker modifies data, but that is less easy to detect since data often is meant to be modified; e.g., how can one determine if modifications to an ETS table are intentional or not? For real security you really need to prevent the attacker from ever gaining access to the application's RAM. If you're deploying on Linux or *BSD, I would consider locking down the OS hosting the Erlang VM, e.g. by disabling ptrace, /dev/mem, and similar mechanisms, and by restricting logins to only secure terminals. Oh, and disable Erlang RPC. You may also consider deploying on a lockstep OS/HW platform (2 or more systems executing in lockstep, the HW compares results and detects if one starts to differ), but I don't know if they are commercially available any more. I think Tandem used to make such systems, mainly for fault-tolerance. On Mon, Sep 24, 2018 at 10:53 AM, Wojciech Ziniewicz wrote: > Hello, > > We develop an application on a highly regulated market. Some regulators > force us to protect the running code from memory modification attacks. > Consider following attack: > - the app is running and all modules are loaded > - attacker gains access to RAM, scans it and modifies a value in the memory > (or a function) so the the running code differs from the code that has been > loaded during initialization > - the app continues operation without noticing that code has been modified > - a state where two different apps are located on a single machine: the one > in RAM and the one on the disk > > I'm looking for *any* measures provided by erlang vm/tooling that would help > mitigating this attack. > > The beam_lib provides tools for verifying the integrity of beam files but > some kind of access to the running code would be required to close the loop > here. > > Thanks > WZ > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From lloyd@REDACTED Mon Sep 24 23:26:15 2018 From: lloyd@REDACTED (lloyd@REDACTED) Date: Mon, 24 Sep 2018 17:26:15 -0400 (EDT) Subject: [erlang-questions] ErlGuten In-Reply-To: References: <7AB3CEDA-34AF-41E2-BEE9-2B27274FF680@writersglen.com> Message-ID: <1537824375.310720011@apps.rackspace.com> Hi Theepan, 1. If you look at ErlGuten source, you'll see that function documentation is fairly minimal and many parameters have single character names with little to no documentation. This makes maintenance and revision very difficult. I consider the core ErlGuten modules diamonds-in-the-rough. I also found the high-level page make-up functions provided by Hugh and Carl limited with respect to professional page make-up and, to me at least, a bit confusing. But erlpress_core owes a deep debt and much gratitude to the ErlGuten lineage. erlpress_core is based on Joe's font-handling, justification, and XML parsing code with few if any changes. But it provides map structures to represent nearly all of the PDF objects represented in eg_pdf_op.erl. These map structures incorporate many default parameters so generating PDF documents is syntactically simple and consistent without sacrificing expressive flexibility. You can easily customize display by instantiating map parameters with your own values. You can see most of the PDF objects currently supported by erlpress_core, including boxed text, justification options, and various line and text objects, demonstrated in ep_show_n_tell.pdf and the source in ep_show_n_tell.erl. If you look at the map definitions of the various PDF objects, you'll see how to customize display. At some point if would be good to re-write the base Erlguten modules with more explicit function and parameter documentation, but I'm not up to that at this point. But, anyone looking for a challenge is free to jump in. I'm currently working on tables and text flow across pages (think reports and book chapters). Hope to deliver these features plus much code polishing in the next release coming "real soon now." High on my wish/to-do list are: -- Easy-to_use page-grid design functions -- Imposition (printing multiple page impressions on a single sheet of paper stock) -- Articles and beads (think text jumping across columns and pages) -- Easy-to-use and expressive page make-up functions -- Example templates for various print formats -- Markdown input -- Support of TTF and OTF fonts Joe Armstrong expressed the following goal when he first announced ErlGuten: "Better than TeX." That's a tall order, yet to be realized. My hope is that erlpress will move the ball further down the field. 2. I haven't done anything with image formats beyond what you'll find in eg_pdf_image.erl. I'd welcome work in that direction. 3. erlpress_core is still at Version 0.01. It's just out. I'm working toward sufficient functionality and stability to support a web application that we have currently under development. 4. I developed erlpress_core on Erlang/OTP 19. It does require support for maps. I see the erlpress_core library as a valuable library for embedding PDF generation into Erlang applications and as the basis for many exciting Erlang-based print production tools. So, Treepan, I appreciate your interest and would welcome your involvement in testing and feature development. I would jump for joy if some Erlang guru were to step forward take on the TTF/OTF support issue. I've done some research and have a few ideas on how expanded font support might be accomplished, but have too little time over the next several months to dig into it. Incidentally, in my previous post my iPad decided it was smarter than me and erroneously corrected my GitHUb address.it should be writersglen/erlPress_core. My intention is to bag the camel case in my next release with luck in a week or two so it should look like writersglen/erlpress_core. All the best, Lloyd -----Original Message----- From: "Theepan" Sent: Monday, September 24, 2018 3:59pm To: lloyd@REDACTED Cc: "Erlang Questions Mailing List" , "Joe Armstrong" Subject: Re: [erlang-questions] ErlGuten Hi Lloyd, Thank you for your response. I have some quick clarifications -- * What are the major improvements made on the source ErlGuten? Did you improve support to different image formats? Any critical bugs fixed? * Is erlPress_core used in commercial applications, specifically of high throughput types? * What is the minimum erlang/OTP version required? Best, Theepan On Mon, Sep 24, 2018 at 6:36 AM Lloyd R. Prentice wrote: > Hi Thepan, > > I?ve undertaken a major revision of ErlGuten called erlPress_core? > announced it on Erlang questions several weeks ago. It?s available from > GitHub as Writersglen/erlPress_core. I think you?ll find it much more > accessible than the ErlGutens. > > I?ve since been polishing up the code and adding a few new features. After > the launch I learned that camel case is not conventional usage when it > comes to the names of Erlang applications and libraries. So I plan to > release the updated version as erlpress_core. With smooth sailing I hope to > release sometime this week or next. > > All the best, > > Lloyd > > Sent from my iPad > > On Sep 23, 2018, at 8:30 PM, Theepan wrote: > > > Hi Team, > > I see many forks of ErlGuten. Which one is the latest, most stable and > widely used? > > https://github.com/richcarl/erlguten > https://github.com/CarlWright/NGerlguten > https://github.com/hwatkins/erlguten/commits/master > > Best, > Theepan > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From eric.pailleau@REDACTED Mon Sep 24 23:30:17 2018 From: eric.pailleau@REDACTED (Eric Pailleau) Date: Mon, 24 Sep 2018 23:30:17 +0200 Subject: [erlang-questions] How to perform running code vs. beam files integrity check In-Reply-To: References: Message-ID: <79vrpjajeo93m7je4lar89r7.1537824617691@email.android.com> Hi, considering that Erlang was invented for code change at runtime, and two versions of same module can run at same time in different processes... Hard to know if a difference is an attack or not. This imply to give up this feature for your app. An attack could change code for a single process and recover original module code between two checks. Erlang has no security. ---- Wojciech Ziniewicz a ?crit ---- >Hello, > >We develop an application on a highly regulated market. Some regulators >force us to protect the running code from memory modification attacks. >Consider following attack: >- the app is running and all modules are loaded >- attacker gains access to RAM, scans it and modifies a value in the memory >(or a function) so the the running code differs from the code that has been >loaded during initialization >- the app continues operation without noticing that code has been modified >- a state where two different apps are located on a single machine: the >one in RAM and the one on the disk > >I'm looking for *any* measures provided by erlang vm/tooling that would >help mitigating this attack. > >The beam_lib provides tools for verifying the integrity of beam files but >some kind of access to the running code would be required to close the loop >here. > >Thanks >WZ > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vasdeveloper@REDACTED Tue Sep 25 01:15:47 2018 From: vasdeveloper@REDACTED (Theepan) Date: Tue, 25 Sep 2018 04:45:47 +0530 Subject: [erlang-questions] ErlGuten In-Reply-To: <1537824375.310720011@apps.rackspace.com> References: <7AB3CEDA-34AF-41E2-BEE9-2B27274FF680@writersglen.com> <1537824375.310720011@apps.rackspace.com> Message-ID: Thanks Lloyd for your detailed explanation on what has been done, and your vision for the library. However we are already using ErlGuten in one of our production systems, and the issues is mainly with the images. I will definitely keep an eye on the development of erlpress_core in the future, as you seem enthusiastic about making it a lively library. The issues we have are: ** When some JPEG image is embedded into the PDF, it turns out dark in the generated PDF file ** When some PNG file is embedded, it takes too long (unto 5 minutes) to generate the PDF file. Debugging shows that the delay is on defilter(Method, Line1, Line2, Offset, Width, Iter) of eg_pdf_image module. Did you come acrsoss this? Best, Theepan On Tue, Sep 25, 2018 at 2:56 AM wrote: > Hi Theepan, > > 1. If you look at ErlGuten source, you'll see that function documentation > is fairly minimal and many parameters have single character names with > little to no documentation. This makes maintenance and revision very > difficult. I consider the core ErlGuten modules diamonds-in-the-rough. > > I also found the high-level page make-up functions provided by Hugh and > Carl limited with respect to professional page make-up and, to me at least, > a bit confusing. > > But erlpress_core owes a deep debt and much gratitude to the ErlGuten > lineage. > > erlpress_core is based on Joe's font-handling, justification, and XML > parsing code with few if any changes. But it provides map structures to > represent nearly all of the PDF objects represented in eg_pdf_op.erl. These > map structures incorporate many default parameters so generating PDF > documents is syntactically simple and consistent without sacrificing > expressive flexibility. You can easily customize display by instantiating > map parameters with your own values. > > You can see most of the PDF objects currently supported by erlpress_core, > including boxed text, justification options, and various line and text > objects, demonstrated in ep_show_n_tell.pdf and the source in > ep_show_n_tell.erl. > > If you look at the map definitions of the various PDF objects, you'll see > how to customize display. > > At some point if would be good to re-write the base Erlguten modules with > more explicit function and parameter documentation, but I'm not up to that > at this point. But, anyone looking for a challenge is free to jump in. > > I'm currently working on tables and text flow across pages (think reports > and book chapters). Hope to deliver these features plus much code polishing > in the next release coming "real soon now." > > High on my wish/to-do list are: > > -- Easy-to_use page-grid design functions > -- Imposition (printing multiple page impressions on a single sheet of > paper stock) > -- Articles and beads (think text jumping across columns and pages) > -- Easy-to-use and expressive page make-up functions > -- Example templates for various print formats > -- Markdown input > -- Support of TTF and OTF fonts > > Joe Armstrong expressed the following goal when he first announced > ErlGuten: > > "Better than TeX." > > That's a tall order, yet to be realized. My hope is that erlpress will > move the ball further down the field. > > 2. I haven't done anything with image formats beyond what you'll find in > eg_pdf_image.erl. I'd welcome work in that direction. > > 3. erlpress_core is still at Version 0.01. It's just out. I'm working > toward sufficient functionality and stability to support a web application > that we have currently under development. > > 4. I developed erlpress_core on Erlang/OTP 19. It does require support for > maps. > > I see the erlpress_core library as a valuable library for embedding PDF > generation into Erlang applications and as the basis for many exciting > Erlang-based print production tools. > > So, Treepan, I appreciate your interest and would welcome your involvement > in testing and feature development. > > I would jump for joy if some Erlang guru were to step forward take on the > TTF/OTF support issue. I've done some research and have a few ideas on how > expanded font support might be accomplished, but have too little time over > the next several months to dig into it. > > Incidentally, in my previous post my iPad decided it was smarter than me > and erroneously corrected my GitHUb address.it should be > writersglen/erlPress_core. > > My intention is to bag the camel case in my next release with luck in a > week or two so it should look like writersglen/erlpress_core. > > All the best, > > Lloyd > > > -----Original Message----- > From: "Theepan" > Sent: Monday, September 24, 2018 3:59pm > To: lloyd@REDACTED > Cc: "Erlang Questions Mailing List" , "Joe > Armstrong" > Subject: Re: [erlang-questions] ErlGuten > > Hi Lloyd, > > Thank you for your response. I have some quick clarifications -- > > * What are the major improvements made on the source ErlGuten? Did you > improve support to different image formats? Any critical bugs fixed? > * Is erlPress_core used in commercial applications, specifically of high > throughput types? > * What is the minimum erlang/OTP version required? > > Best, > Theepan > > > > > > > On Mon, Sep 24, 2018 at 6:36 AM Lloyd R. Prentice > wrote: > > > Hi Thepan, > > > > I?ve undertaken a major revision of ErlGuten called erlPress_core? > > announced it on Erlang questions several weeks ago. It?s available from > > GitHub as Writersglen/erlPress_core. I think you?ll find it much more > > accessible than the ErlGutens. > > > > I?ve since been polishing up the code and adding a few new features. > After > > the launch I learned that camel case is not conventional usage when it > > comes to the names of Erlang applications and libraries. So I plan to > > release the updated version as erlpress_core. With smooth sailing I hope > to > > release sometime this week or next. > > > > All the best, > > > > Lloyd > > > > Sent from my iPad > > > > On Sep 23, 2018, at 8:30 PM, Theepan wrote: > > > > > > Hi Team, > > > > I see many forks of ErlGuten. Which one is the latest, most stable and > > widely used? > > > > https://github.com/richcarl/erlguten > > https://github.com/CarlWright/NGerlguten > > https://github.com/hwatkins/erlguten/commits/master > > > > Best, > > Theepan > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vasdeveloper@REDACTED Tue Sep 25 01:27:29 2018 From: vasdeveloper@REDACTED (Theepan) Date: Tue, 25 Sep 2018 04:57:29 +0530 Subject: [erlang-questions] ErlGuten In-Reply-To: References: <7AB3CEDA-34AF-41E2-BEE9-2B27274FF680@writersglen.com> <1537824375.310720011@apps.rackspace.com> Message-ID: Wanted to add to the issues the reasons -- ** When some JPEG image is embedded into the PDF, it turns out dark in the generated PDF file [Happens due to CYMK color profile.] ** When some PNG file is embedded, it takes too long (unto 5 minutes) to generate the PDF file. Debugging shows that the delay is on defilter(Method , Line1, Line2, Offset, Width, Iter) of eg_pdf_image module.[Happens due to Alpha channel presence] On Tue, Sep 25, 2018 at 4:45 AM Theepan wrote: > Thanks Lloyd for your detailed explanation on what has been done, and your > vision for the library. However we are already using ErlGuten in one of our > production systems, and the issues is mainly with the images. > > I will definitely keep an eye on the development of erlpress_core in the > future, as you seem enthusiastic about making it a lively library. > > The issues we have are: > > ** When some JPEG image is embedded into the PDF, it turns out dark in the > generated PDF file > ** When some PNG file is embedded, it takes too long (unto 5 minutes) to > generate the PDF file. Debugging shows that the delay is on defilter( > Method, Line1, Line2, Offset, Width, Iter) of eg_pdf_image module. > > Did you come acrsoss this? > > Best, > Theepan > > > > > > On Tue, Sep 25, 2018 at 2:56 AM wrote: > >> Hi Theepan, >> >> 1. If you look at ErlGuten source, you'll see that function documentation >> is fairly minimal and many parameters have single character names with >> little to no documentation. This makes maintenance and revision very >> difficult. I consider the core ErlGuten modules diamonds-in-the-rough. >> >> I also found the high-level page make-up functions provided by Hugh and >> Carl limited with respect to professional page make-up and, to me at least, >> a bit confusing. >> >> But erlpress_core owes a deep debt and much gratitude to the ErlGuten >> lineage. >> >> erlpress_core is based on Joe's font-handling, justification, and XML >> parsing code with few if any changes. But it provides map structures to >> represent nearly all of the PDF objects represented in eg_pdf_op.erl. These >> map structures incorporate many default parameters so generating PDF >> documents is syntactically simple and consistent without sacrificing >> expressive flexibility. You can easily customize display by instantiating >> map parameters with your own values. >> >> You can see most of the PDF objects currently supported by erlpress_core, >> including boxed text, justification options, and various line and text >> objects, demonstrated in ep_show_n_tell.pdf and the source in >> ep_show_n_tell.erl. >> >> If you look at the map definitions of the various PDF objects, you'll see >> how to customize display. >> >> At some point if would be good to re-write the base Erlguten modules with >> more explicit function and parameter documentation, but I'm not up to that >> at this point. But, anyone looking for a challenge is free to jump in. >> >> I'm currently working on tables and text flow across pages (think reports >> and book chapters). Hope to deliver these features plus much code polishing >> in the next release coming "real soon now." >> >> High on my wish/to-do list are: >> >> -- Easy-to_use page-grid design functions >> -- Imposition (printing multiple page impressions on a single sheet of >> paper stock) >> -- Articles and beads (think text jumping across columns and pages) >> -- Easy-to-use and expressive page make-up functions >> -- Example templates for various print formats >> -- Markdown input >> -- Support of TTF and OTF fonts >> >> Joe Armstrong expressed the following goal when he first announced >> ErlGuten: >> >> "Better than TeX." >> >> That's a tall order, yet to be realized. My hope is that erlpress will >> move the ball further down the field. >> >> 2. I haven't done anything with image formats beyond what you'll find in >> eg_pdf_image.erl. I'd welcome work in that direction. >> >> 3. erlpress_core is still at Version 0.01. It's just out. I'm working >> toward sufficient functionality and stability to support a web application >> that we have currently under development. >> >> 4. I developed erlpress_core on Erlang/OTP 19. It does require support >> for maps. >> >> I see the erlpress_core library as a valuable library for embedding PDF >> generation into Erlang applications and as the basis for many exciting >> Erlang-based print production tools. >> >> So, Treepan, I appreciate your interest and would welcome your >> involvement in testing and feature development. >> >> I would jump for joy if some Erlang guru were to step forward take on the >> TTF/OTF support issue. I've done some research and have a few ideas on how >> expanded font support might be accomplished, but have too little time over >> the next several months to dig into it. >> >> Incidentally, in my previous post my iPad decided it was smarter than me >> and erroneously corrected my GitHUb address.it should be >> writersglen/erlPress_core. >> >> My intention is to bag the camel case in my next release with luck in a >> week or two so it should look like writersglen/erlpress_core. >> >> All the best, >> >> Lloyd >> >> >> -----Original Message----- >> From: "Theepan" >> Sent: Monday, September 24, 2018 3:59pm >> To: lloyd@REDACTED >> Cc: "Erlang Questions Mailing List" , "Joe >> Armstrong" >> Subject: Re: [erlang-questions] ErlGuten >> >> Hi Lloyd, >> >> Thank you for your response. I have some quick clarifications -- >> >> * What are the major improvements made on the source ErlGuten? Did you >> improve support to different image formats? Any critical bugs fixed? >> * Is erlPress_core used in commercial applications, specifically of high >> throughput types? >> * What is the minimum erlang/OTP version required? >> >> Best, >> Theepan >> >> >> >> >> >> >> On Mon, Sep 24, 2018 at 6:36 AM Lloyd R. Prentice >> wrote: >> >> > Hi Thepan, >> > >> > I?ve undertaken a major revision of ErlGuten called erlPress_core? >> > announced it on Erlang questions several weeks ago. It?s available from >> > GitHub as Writersglen/erlPress_core. I think you?ll find it much more >> > accessible than the ErlGutens. >> > >> > I?ve since been polishing up the code and adding a few new features. >> After >> > the launch I learned that camel case is not conventional usage when it >> > comes to the names of Erlang applications and libraries. So I plan to >> > release the updated version as erlpress_core. With smooth sailing I >> hope to >> > release sometime this week or next. >> > >> > All the best, >> > >> > Lloyd >> > >> > Sent from my iPad >> > >> > On Sep 23, 2018, at 8:30 PM, Theepan wrote: >> > >> > >> > Hi Team, >> > >> > I see many forks of ErlGuten. Which one is the latest, most stable and >> > widely used? >> > >> > https://github.com/richcarl/erlguten >> > https://github.com/CarlWright/NGerlguten >> > https://github.com/hwatkins/erlguten/commits/master >> > >> > Best, >> > Theepan >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From awsafrahman1704@REDACTED Mon Sep 24 23:59:00 2018 From: awsafrahman1704@REDACTED (Awsaf Rahman) Date: Mon, 24 Sep 2018 23:59:00 +0200 Subject: [erlang-questions] Memory usage In-Reply-To: References: Message-ID: I am trying to compare Erlang and Haskell based on compile time, execution time, memory usage and cpu usage for a set of 10 programs. I am not fluent in either of them and this is my first time trying benchmarking. On Sun, Sep 23, 2018 at 8:05 PM Paul Peregud wrote: > To profile CPU time, you can use fprof (part of OTP distribution). > > Why would you want to profile CPU usage during compilation? Are you > planning working on Erlang compilator (erlc)? > > On Sun, Sep 23, 2018 at 12:12 AM Awsaf Rahman > wrote: > >> Hello all, >> >> I want to profile a few Erlang programs I wrote. Basically, I want to >> find out the following: >> 1) CPU and memory usage during compilation >> 2) CPU and memory usage during execution >> >> Is there anyway I can do this? >> >> Kind regards >> Awsaf >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > -- > Best regards, > Paul Peregud > +48602112091 > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenneth.lundin@REDACTED Tue Sep 25 08:08:31 2018 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Tue, 25 Sep 2018 08:08:31 +0200 Subject: [erlang-questions] How to perform running code vs. beam files integrity check In-Reply-To: <79vrpjajeo93m7je4lar89r7.1537824617691@email.android.com> References: <79vrpjajeo93m7je4lar89r7.1537824617691@email.android.com> Message-ID: On Mon, Sep 24, 2018, 23:30 Eric Pailleau wrote: > > Hi, considering that Erlang was invented for code change at runtime, and > two versions of same module can run at same time in different processes... > Hard to know if a difference is an attack or not. > This imply to give up this feature for your app. > An attack could change code for a single process and recover original > module code between two checks. > Erlang has no security. > You claim that Erlang has no security, but that does not make Erlang less > secure than any other language or runtime if the attacker can manipulate > RAM for the running user space application? > I think it is better to concentrate on not letting an attacker have access to RAM. /Kenneth > > ---- Wojciech Ziniewicz a ?crit ---- > > Hello, > > We develop an application on a highly regulated market. Some regulators > force us to protect the running code from memory modification attacks. > Consider following attack: > - the app is running and all modules are loaded > - attacker gains access to RAM, scans it and modifies a value in the > memory (or a function) so the the running code differs from the code that > has been loaded during initialization > - the app continues operation without noticing that code has been modified > - a state where two different apps are located on a single machine: the > one in RAM and the one on the disk > > I'm looking for *any* measures provided by erlang vm/tooling that would > help mitigating this attack. > > The beam_lib provides tools for verifying the integrity of beam files but > some kind of access to the running code would be required to close the loop > here. > > Thanks > WZ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Tue Sep 25 08:39:20 2018 From: eric.pailleau@REDACTED (Eric Pailleau) Date: Tue, 25 Sep 2018 08:39:20 +0200 Subject: [erlang-questions] How to perform running code vs. beam files integrity check In-Reply-To: References: <79vrpjajeo93m7je4lar89r7.1537824617691@email.android.com> Message-ID: <6od8chn8tbn9v46pclsvs2nk.1537857560314@email.android.com> Sure. And not access to node. -setcookie option in ps is a serious security issue for example. ---- Kenneth Lundin a ?crit ---- >On Mon, Sep 24, 2018, 23:30 Eric Pailleau wrote: > >> >> Hi, considering that Erlang was invented for code change at runtime, and >> two versions of same module can run at same time in different processes... >> Hard to know if a difference is an attack or not. >> This imply to give up this feature for your app. >> An attack could change code for a single process and recover original >> module code between two checks. >> Erlang has no security. >> You claim that Erlang has no security, but that does not make Erlang less >> secure than any other language or runtime if the attacker can manipulate >> RAM for the running user space application? >> > >I think it is better to concentrate on not letting an attacker have access >to RAM. > >/Kenneth > >> > > >> ---- Wojciech Ziniewicz a ?crit ---- >> >> Hello, >> >> We develop an application on a highly regulated market. Some regulators >> force us to protect the running code from memory modification attacks. >> Consider following attack: >> - the app is running and all modules are loaded >> - attacker gains access to RAM, scans it and modifies a value in the >> memory (or a function) so the the running code differs from the code that >> has been loaded during initialization >> - the app continues operation without noticing that code has been modified >> - a state where two different apps are located on a single machine: the >> one in RAM and the one on the disk >> >> I'm looking for *any* measures provided by erlang vm/tooling that would >> help mitigating this attack. >> >> The beam_lib provides tools for verifying the integrity of beam files but >> some kind of access to the running code would be required to close the loop >> here. >> >> Thanks >> WZ >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From raoknz@REDACTED Tue Sep 25 13:35:42 2018 From: raoknz@REDACTED (Richard O'Keefe) Date: Tue, 25 Sep 2018 23:35:42 +1200 Subject: [erlang-questions] Memory usage In-Reply-To: References: Message-ID: The kind of comparison you want is fraught with difficulty. There are, for example, several different Haskell compilers. GHC, of course, is *the* Haskell compiler, but UHC is still a contender, and LHC is coming along. If you are happy with Haskell 98, JHC is still around. You cannot actually benchmark *Haskell*, only a particular Haskell compiler version with certain options on a particular (kind of) machine running a particular operating system and libraries. So let's stick with GHC. Depending on exactly what you have, GHC has *at least* two different back ends. It also has more optimisation options than any sane programmer wants to think about. The effects of something like -ffull-laziness can be anything between very welcome and spectacularly unhelpful. Understanding performance in a non- strict language is tricky, and as a tip for beginners, see if you understand the difference between foldl and foldl' . And of course, GHC is actively maintained. I'm using Ubuntu Linux 18.04 (Bionic Beaver) and the latest version of GHC available from the Ubuntu repository is 8.0.2. The current stable release is 8.4.3. The current release in beta is 8.6.1. There have been four releases of GHC so far this year. Do they all have the same performance for all programs? Erlang is also very actively maintained, and also has at least two back ends (BEAM and HiPE). How big are the 10 programs and how long do you expect them to run? On Tue, 25 Sep 2018 at 16:25, Awsaf Rahman wrote: > I am trying to compare Erlang and Haskell based on compile time, execution > time, memory usage and cpu usage for a set of 10 programs. I am not fluent > in either of them and this is my first time trying benchmarking. > > On Sun, Sep 23, 2018 at 8:05 PM Paul Peregud > wrote: > >> To profile CPU time, you can use fprof (part of OTP distribution). >> >> Why would you want to profile CPU usage during compilation? Are you >> planning working on Erlang compilator (erlc)? >> >> On Sun, Sep 23, 2018 at 12:12 AM Awsaf Rahman >> wrote: >> >>> Hello all, >>> >>> I want to profile a few Erlang programs I wrote. Basically, I want to >>> find out the following: >>> 1) CPU and memory usage during compilation >>> 2) CPU and memory usage during execution >>> >>> Is there anyway I can do this? >>> >>> Kind regards >>> Awsaf >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> -- >> Best regards, >> Paul Peregud >> +48602112091 >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Tue Sep 25 14:08:56 2018 From: frank.muller.erl@REDACTED (Frank Muller) Date: Tue, 25 Sep 2018 14:08:56 +0200 Subject: [erlang-questions] Pretty print an XML? Message-ID: Hi guys Is there a way to pretty print a valid XML string loaded from file? /Frank -------------- next part -------------- An HTML attachment was scrubbed... URL: From dm.klionsky@REDACTED Tue Sep 25 14:25:52 2018 From: dm.klionsky@REDACTED (Dmitry Klionsky) Date: Tue, 25 Sep 2018 15:25:52 +0300 Subject: [erlang-questions] Pretty print an XML? In-Reply-To: References: Message-ID: <67308f6e-0684-104b-34be-3d0d77b0c33f@gmail.com> $ cat data.xml | xmllint --format - On 09/25/2018 03:08 PM, Frank Muller wrote: > Hi guys > > Is there a way to pretty print a valid XML string loaded from file? > > /Frank > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- BR, Dmitry -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Tue Sep 25 14:46:58 2018 From: vances@REDACTED (Vance Shipley) Date: Tue, 25 Sep 2018 18:16:58 +0530 Subject: [erlang-questions] Pretty print an XML? In-Reply-To: References: Message-ID: On Tue, Sep 25, 2018 at 5:39 PM Frank Muller wrote: > Is there a way to pretty print a valid XML string loaded from file? Unfortunately with xmerl you have to sprinkle stuff like this in yourself: #xmlText{value="\n "} -- -Vance From elbrujohalcon@REDACTED Tue Sep 25 14:48:33 2018 From: elbrujohalcon@REDACTED (Brujo Benavides) Date: Tue, 25 Sep 2018 09:48:33 -0300 Subject: [erlang-questions] Pretty print an XML? In-Reply-To: References: Message-ID: Or you can use the xml simplifier ;) Brujo Benavides > On 25 Sep 2018, at 09:46, Vance Shipley wrote: > > On Tue, Sep 25, 2018 at 5:39 PM Frank Muller wrote: >> Is there a way to pretty print a valid XML string loaded from file? > > Unfortunately with xmerl you have to sprinkle stuff like this in > yourself: #xmlText{value="\n "} > > > -- > -Vance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From w.a.de.jong@REDACTED Tue Sep 25 15:06:49 2018 From: w.a.de.jong@REDACTED (Willem de Jong) Date: Tue, 25 Sep 2018 15:06:49 +0200 Subject: [erlang-questions] Pretty print an XML? In-Reply-To: References: Message-ID: It may not work for all XML documents (it was not really intended as a general XML pretty printer), but you can try erlsom_lib:prettyPrint(String). It is part of erlsom ( https://github.com/willemdj/erlsom). regards, Willem Op di 25 sep. 2018 om 14:48 schreef Brujo Benavides : > Or you can use the xml simplifier > > ;) > > ------------------------------ > *Brujo Benavides * > > > > On 25 Sep 2018, at 09:46, Vance Shipley wrote: > > On Tue, Sep 25, 2018 at 5:39 PM Frank Muller > wrote: > > Is there a way to pretty print a valid XML string loaded from file? > > > Unfortunately with xmerl you have to sprinkle stuff like this in > yourself: #xmlText{value="\n "} > > > -- > -Vance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From henrik.x.nord@REDACTED Tue Sep 25 15:29:39 2018 From: henrik.x.nord@REDACTED (Henrik Nord X) Date: Tue, 25 Sep 2018 13:29:39 +0000 Subject: [erlang-questions] Patch package OTP 21.1 released Message-ID: <1537882179.4914.34.camel@ericsson.com> Erlang/OTP 21.1 is the first service release for the 21 major release with new features, improvements as well as a few features! Highlights SSH: ?The cipher 'chacha20-poly1305@REDACTED' is now??supported if OpenSSL 1.1.1 or higher is used as cryptolib. ?The key exchange methods 'curve25519-sha256@REDACTED', 'curve25519- sha256' and 'curve448-sha512' are implemented.? ?They all depends on that OpenSSL 1.1.1 or higher used as cryptolib. Crypto: Typing in crypto and public_key applications are reworked and corrected. New chapter on Algorithm details in the User's guide ERTS, Kernel: The socket options recvtos, recvttl, recvtclass and pktoptions have been implemented in the socket modules.? Se documentation in gen_tcp, gen_udp and inet modules for more info! For more details see: http://erlang.org/download/otp_src_21.1.readme Pre built versions for Windows can be fetched here: http://erlang.org/download/otp_win32_21.1.exe http://erlang.org/download/otp_win64_21.1.exe Online documentation can be browsed here: http://erlang.org/documentation/doc-10.1/doc The Erlang/OTP source can also be found at GitHub on the official Erlang repository, Here: OTP-21.1 Please report any new issues via Erlang/OTPs public issue tracker https://bugs.erlang.org ? System Dashboard - Erlang Programming Language bugs.erlang.org Welcome to Erlang Programming Language. What is bugs.erlang.org? Here you report bugs in Erlang/OTP, suggestions for new features and improvements. We want to thank all of those who sent us patches, suggestions and bug reports! ? Thank you! ? The Erlang/OTP Team at Ericsson From sean.hinde@REDACTED Tue Sep 25 16:19:59 2018 From: sean.hinde@REDACTED (Sean Hinde) Date: Tue, 25 Sep 2018 16:19:59 +0200 Subject: [erlang-questions] ErlGuten In-Reply-To: References: <7AB3CEDA-34AF-41E2-BEE9-2B27274FF680@writersglen.com> <1537824375.310720011@apps.rackspace.com> Message-ID: <7C188AE2-A0E3-4E06-B541-C6BEB38C2957@mac.com> I hit the same issue. I guess you are using one of the versions with lots of lists:nth. Following find my patch on the version I was using. You can find my branch at: https://github.com/proactively/erlguten/tree/png-performance No time right now to make nice pull requests - I will eventually =============================================== diff --git a/src/eg_pdf_image.erl b/src/eg_pdf_image.erl index 1a97e0b..817f1b8 100644 --- a/src/eg_pdf_image.erl +++ b/src/eg_pdf_image.erl @@ -431,11 +431,11 @@ extractScanLines(Width,Decompressed) -> extractLine(ScanLines,Width,<< >>) -> AlmostDone = lists:reverse(ScanLines), - [ {0, string:chars(0,Width-1)} | AlmostDone]; + [ {0, list_to_binary(string:chars(0,Width-1))} | AlmostDone]; extractLine(ScanLines,Width,Image) -> LineSize = Width - 1, << Method:8, Line:LineSize/binary-unit:8, Rest/binary>> = Image, - extractLine([{Method, binary_to_list(Line) } | ScanLines ], Width, Rest). + extractLine([{Method, Line} | ScanLines ], Width, Rest). %% @doc Remove the filter on all the bytes in the scan lines. @@ -448,19 +448,26 @@ processLine([{_Method, _Line1}], _Iter, _Offset, Results)-> A = lists:flatten( lists:reverse(Results) ), list_to_binary( A ); processLine([{_, Line1},{Method, Line2} | Remainder], Iter, Offset, Results) -> - {ok, Unfiltered} = defilter(Method, Line1, Line2,Offset,length(Line1),Iter), + {ok, Unfiltered} = defilter(Method, Line1, Line2,Offset,size(Line1),Iter), processLine([{Method, Unfiltered } | Remainder], Iter, Offset, [Unfiltered | Results] ). %% @doc Taking two lines of stream defilter the 2nd with the previously defiltered 1st line. defilter(Method, Line1, Line2, Offset, Width, Iter) when Iter =< Width -> NewVal = case Iter =< Offset of - true -> filter(lists:nth(Iter,Line2), 0, lists:nth(Iter,Line1), 0, Method); - false -> filter(lists:nth(Iter,Line2), lists:nth(Iter-Offset,Line2), lists:nth(Iter,Line1), lists:nth(Iter-Offset,Line1), Method) + true -> filter(binary:at(Line2, Iter - 1), 0, + binary:at(Line1, Iter - 1), 0, Method); + false -> filter(binary:at(Line2, Iter - 1), + binary:at(Line2, Iter - Offset - 1), + binary:at(Line1, Iter - 1), + binary:at(Line1, Iter - Offset - 1), Method) end, L2 = case Iter == Width of - true -> lists:flatten([lists:sublist(Line2,Iter - 1),NewVal]); - false -> lists:flatten([lists:sublist(Line2,Iter - 1),NewVal,lists:nthtail(Iter, Line2)]) + true -> Start = binary:part(Line2, 0, Iter - 1), + <>; + false -> Start = binary:part(Line2, 0, Iter - 1), + End = binary:part(Line2, Iter, size(Line2) - Iter), + <> end, defilter(Method, Line1, L2 ,Offset, Width, Iter+1); defilter(_Method, _Line1, Line2, _Offset, _Width, _Iter) -> @@ -513,8 +520,7 @@ inflate_stream(Data) -> Decompressed = zlib:inflate(Z, Data), ok = zlib:inflateEnd(Z), zlib:close(Z), - F = fun(A, B) -> <> end, - MergedBinaries = lists:foldr(F, <<>>, Decompressed), + MergedBinaries = iolist_to_binary(Decompressed), {ok,MergedBinaries}. %% @doc Compress a bit stream using the zlib/deflate algorithm =============================================== > On 25 Sep 2018, at 01:15, Theepan wrote: > > Thanks Lloyd for your detailed explanation on what has been done, and your vision for the library. However we are already using ErlGuten in one of our production systems, and the issues is mainly with the images. > > I will definitely keep an eye on the development of erlpress_core in the future, as you seem enthusiastic about making it a lively library. > > The issues we have are: > > ** When some JPEG image is embedded into the PDF, it turns out dark in the generated PDF file > ** When some PNG file is embedded, it takes too long (unto 5 minutes) to generate the PDF file. Debugging shows that the delay is on defilter(Method, Line1, Line2, Offset, Width, Iter) of eg_pdf_image module. > > Did you come acrsoss this? > > Best, > Theepan > > > > > > On Tue, Sep 25, 2018 at 2:56 AM > wrote: > Hi Theepan, > > 1. If you look at ErlGuten source, you'll see that function documentation is fairly minimal and many parameters have single character names with little to no documentation. This makes maintenance and revision very difficult. I consider the core ErlGuten modules diamonds-in-the-rough. > > I also found the high-level page make-up functions provided by Hugh and Carl limited with respect to professional page make-up and, to me at least, a bit confusing. > > But erlpress_core owes a deep debt and much gratitude to the ErlGuten lineage. > > erlpress_core is based on Joe's font-handling, justification, and XML parsing code with few if any changes. But it provides map structures to represent nearly all of the PDF objects represented in eg_pdf_op.erl. These map structures incorporate many default parameters so generating PDF documents is syntactically simple and consistent without sacrificing expressive flexibility. You can easily customize display by instantiating map parameters with your own values. > > You can see most of the PDF objects currently supported by erlpress_core, including boxed text, justification options, and various line and text objects, demonstrated in ep_show_n_tell.pdf and the source in ep_show_n_tell.erl. > > If you look at the map definitions of the various PDF objects, you'll see how to customize display. > > At some point if would be good to re-write the base Erlguten modules with more explicit function and parameter documentation, but I'm not up to that at this point. But, anyone looking for a challenge is free to jump in. > > I'm currently working on tables and text flow across pages (think reports and book chapters). Hope to deliver these features plus much code polishing in the next release coming "real soon now." > > High on my wish/to-do list are: > > -- Easy-to_use page-grid design functions > -- Imposition (printing multiple page impressions on a single sheet of paper stock) > -- Articles and beads (think text jumping across columns and pages) > -- Easy-to-use and expressive page make-up functions > -- Example templates for various print formats > -- Markdown input > -- Support of TTF and OTF fonts > > Joe Armstrong expressed the following goal when he first announced ErlGuten: > > "Better than TeX." > > That's a tall order, yet to be realized. My hope is that erlpress will move the ball further down the field. > > 2. I haven't done anything with image formats beyond what you'll find in eg_pdf_image.erl. I'd welcome work in that direction. > > 3. erlpress_core is still at Version 0.01. It's just out. I'm working toward sufficient functionality and stability to support a web application that we have currently under development. > > 4. I developed erlpress_core on Erlang/OTP 19. It does require support for maps. > > I see the erlpress_core library as a valuable library for embedding PDF generation into Erlang applications and as the basis for many exciting Erlang-based print production tools. > > So, Treepan, I appreciate your interest and would welcome your involvement in testing and feature development. > > I would jump for joy if some Erlang guru were to step forward take on the TTF/OTF support issue. I've done some research and have a few ideas on how expanded font support might be accomplished, but have too little time over the next several months to dig into it. > > Incidentally, in my previous post my iPad decided it was smarter than me and erroneously corrected my GitHUb address.it should be writersglen/erlPress_core. > > My intention is to bag the camel case in my next release with luck in a week or two so it should look like writersglen/erlpress_core. > > All the best, > > Lloyd > > > -----Original Message----- > From: "Theepan" > > Sent: Monday, September 24, 2018 3:59pm > To: lloyd@REDACTED > Cc: "Erlang Questions Mailing List" >, "Joe Armstrong" > > Subject: Re: [erlang-questions] ErlGuten > > Hi Lloyd, > > Thank you for your response. I have some quick clarifications -- > > * What are the major improvements made on the source ErlGuten? Did you > improve support to different image formats? Any critical bugs fixed? > * Is erlPress_core used in commercial applications, specifically of high > throughput types? > * What is the minimum erlang/OTP version required? > > Best, > Theepan > > > > > > > On Mon, Sep 24, 2018 at 6:36 AM Lloyd R. Prentice > > wrote: > > > Hi Thepan, > > > > I?ve undertaken a major revision of ErlGuten called erlPress_core? > > announced it on Erlang questions several weeks ago. It?s available from > > GitHub as Writersglen/erlPress_core. I think you?ll find it much more > > accessible than the ErlGutens. > > > > I?ve since been polishing up the code and adding a few new features. After > > the launch I learned that camel case is not conventional usage when it > > comes to the names of Erlang applications and libraries. So I plan to > > release the updated version as erlpress_core. With smooth sailing I hope to > > release sometime this week or next. > > > > All the best, > > > > Lloyd > > > > Sent from my iPad > > > > On Sep 23, 2018, at 8:30 PM, Theepan > wrote: > > > > > > Hi Team, > > > > I see many forks of ErlGuten. Which one is the latest, most stable and > > widely used? > > > > https://github.com/richcarl/erlguten > > https://github.com/CarlWright/NGerlguten > > https://github.com/hwatkins/erlguten/commits/master > > > > Best, > > Theepan > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Tue Sep 25 18:51:10 2018 From: lloyd@REDACTED (lloyd@REDACTED) Date: Tue, 25 Sep 2018 12:51:10 -0400 (EDT) Subject: [erlang-questions] ErlGuten In-Reply-To: References: <7AB3CEDA-34AF-41E2-BEE9-2B27274FF680@writersglen.com> <1537824375.310720011@apps.rackspace.com> Message-ID: <1537894270.37611727@apps.rackspace.com> Hi Theepan, Other than display in my demo PDF, I haven't worked much with images. The issues you've run across seem serious. Getting at the root of the problems will likely involve a deep dive into both the PDF reference manuals and core ErlGuten source code. As I noted in my earlier email, ErlGuten source code is very challenging. It would benefit significantly from re-write, possibly refactoring in some cases and definitely systematic documentation of functions and parameters. I'm not confident that my skills are up to this. My take is that it would require some amount of community effort to put ErlGuten/erlpress_core on a solid, maintainable base. I would dearly love to see this happen. Is there a CS major or intern out there with time to take on the challenges? Joe showed us the way and I, for one, am deeply grateful. Given that PDF is a significant standard in the print media world, Erlang deserves to have a world-class library and suite of tools to generate PDF documents and decode them back into Erlang structures. This will take community effort. All the best, Lloyd -----Original Message----- From: "Theepan" Sent: Monday, September 24, 2018 7:27pm To: "Lloyd Prentice" Cc: "Erlang Questions Mailing List" , "Joe Armstrong" Subject: Re: [erlang-questions] ErlGuten Wanted to add to the issues the reasons -- ** When some JPEG image is embedded into the PDF, it turns out dark in the generated PDF file [Happens due to CYMK color profile.] ** When some PNG file is embedded, it takes too long (unto 5 minutes) to generate the PDF file. Debugging shows that the delay is on defilter(Method , Line1, Line2, Offset, Width, Iter) of eg_pdf_image module.[Happens due to Alpha channel presence] On Tue, Sep 25, 2018 at 4:45 AM Theepan wrote: > Thanks Lloyd for your detailed explanation on what has been done, and your > vision for the library. However we are already using ErlGuten in one of our > production systems, and the issues is mainly with the images. > > I will definitely keep an eye on the development of erlpress_core in the > future, as you seem enthusiastic about making it a lively library. > > The issues we have are: > > ** When some JPEG image is embedded into the PDF, it turns out dark in the > generated PDF file > ** When some PNG file is embedded, it takes too long (unto 5 minutes) to > generate the PDF file. Debugging shows that the delay is on defilter( > Method, Line1, Line2, Offset, Width, Iter) of eg_pdf_image module. > > Did you come acrsoss this? > > Best, > Theepan > > > > > > On Tue, Sep 25, 2018 at 2:56 AM wrote: > >> Hi Theepan, >> >> 1. If you look at ErlGuten source, you'll see that function documentation >> is fairly minimal and many parameters have single character names with >> little to no documentation. This makes maintenance and revision very >> difficult. I consider the core ErlGuten modules diamonds-in-the-rough. >> >> I also found the high-level page make-up functions provided by Hugh and >> Carl limited with respect to professional page make-up and, to me at least, >> a bit confusing. >> >> But erlpress_core owes a deep debt and much gratitude to the ErlGuten >> lineage. >> >> erlpress_core is based on Joe's font-handling, justification, and XML >> parsing code with few if any changes. But it provides map structures to >> represent nearly all of the PDF objects represented in eg_pdf_op.erl. These >> map structures incorporate many default parameters so generating PDF >> documents is syntactically simple and consistent without sacrificing >> expressive flexibility. You can easily customize display by instantiating >> map parameters with your own values. >> >> You can see most of the PDF objects currently supported by erlpress_core, >> including boxed text, justification options, and various line and text >> objects, demonstrated in ep_show_n_tell.pdf and the source in >> ep_show_n_tell.erl. >> >> If you look at the map definitions of the various PDF objects, you'll see >> how to customize display. >> >> At some point if would be good to re-write the base Erlguten modules with >> more explicit function and parameter documentation, but I'm not up to that >> at this point. But, anyone looking for a challenge is free to jump in. >> >> I'm currently working on tables and text flow across pages (think reports >> and book chapters). Hope to deliver these features plus much code polishing >> in the next release coming "real soon now." >> >> High on my wish/to-do list are: >> >> -- Easy-to_use page-grid design functions >> -- Imposition (printing multiple page impressions on a single sheet of >> paper stock) >> -- Articles and beads (think text jumping across columns and pages) >> -- Easy-to-use and expressive page make-up functions >> -- Example templates for various print formats >> -- Markdown input >> -- Support of TTF and OTF fonts >> >> Joe Armstrong expressed the following goal when he first announced >> ErlGuten: >> >> "Better than TeX." >> >> That's a tall order, yet to be realized. My hope is that erlpress will >> move the ball further down the field. >> >> 2. I haven't done anything with image formats beyond what you'll find in >> eg_pdf_image.erl. I'd welcome work in that direction. >> >> 3. erlpress_core is still at Version 0.01. It's just out. I'm working >> toward sufficient functionality and stability to support a web application >> that we have currently under development. >> >> 4. I developed erlpress_core on Erlang/OTP 19. It does require support >> for maps. >> >> I see the erlpress_core library as a valuable library for embedding PDF >> generation into Erlang applications and as the basis for many exciting >> Erlang-based print production tools. >> >> So, Treepan, I appreciate your interest and would welcome your >> involvement in testing and feature development. >> >> I would jump for joy if some Erlang guru were to step forward take on the >> TTF/OTF support issue. I've done some research and have a few ideas on how >> expanded font support might be accomplished, but have too little time over >> the next several months to dig into it. >> >> Incidentally, in my previous post my iPad decided it was smarter than me >> and erroneously corrected my GitHUb address.it should be >> writersglen/erlPress_core. >> >> My intention is to bag the camel case in my next release with luck in a >> week or two so it should look like writersglen/erlpress_core. >> >> All the best, >> >> Lloyd >> >> >> -----Original Message----- >> From: "Theepan" >> Sent: Monday, September 24, 2018 3:59pm >> To: lloyd@REDACTED >> Cc: "Erlang Questions Mailing List" , "Joe >> Armstrong" >> Subject: Re: [erlang-questions] ErlGuten >> >> Hi Lloyd, >> >> Thank you for your response. I have some quick clarifications -- >> >> * What are the major improvements made on the source ErlGuten? Did you >> improve support to different image formats? Any critical bugs fixed? >> * Is erlPress_core used in commercial applications, specifically of high >> throughput types? >> * What is the minimum erlang/OTP version required? >> >> Best, >> Theepan >> >> >> >> >> >> >> On Mon, Sep 24, 2018 at 6:36 AM Lloyd R. Prentice >> wrote: >> >> > Hi Thepan, >> > >> > I?ve undertaken a major revision of ErlGuten called erlPress_core? >> > announced it on Erlang questions several weeks ago. It?s available from >> > GitHub as Writersglen/erlPress_core. I think you?ll find it much more >> > accessible than the ErlGutens. >> > >> > I?ve since been polishing up the code and adding a few new features. >> After >> > the launch I learned that camel case is not conventional usage when it >> > comes to the names of Erlang applications and libraries. So I plan to >> > release the updated version as erlpress_core. With smooth sailing I >> hope to >> > release sometime this week or next. >> > >> > All the best, >> > >> > Lloyd >> > >> > Sent from my iPad >> > >> > On Sep 23, 2018, at 8:30 PM, Theepan wrote: >> > >> > >> > Hi Team, >> > >> > I see many forks of ErlGuten. Which one is the latest, most stable and >> > widely used? >> > >> > https://github.com/richcarl/erlguten >> > https://github.com/CarlWright/NGerlguten >> > https://github.com/hwatkins/erlguten/commits/master >> > >> > Best, >> > Theepan >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> >> >> From vasdeveloper@REDACTED Tue Sep 25 20:45:13 2018 From: vasdeveloper@REDACTED (Theepan) Date: Wed, 26 Sep 2018 00:15:13 +0530 Subject: [erlang-questions] ErlGuten In-Reply-To: <7C188AE2-A0E3-4E06-B541-C6BEB38C2957@mac.com> References: <7AB3CEDA-34AF-41E2-BEE9-2B27274FF680@writersglen.com> <1537824375.310720011@apps.rackspace.com> <7C188AE2-A0E3-4E06-B541-C6BEB38C2957@mac.com> Message-ID: Hi Sean, I used GraphicsMagic to remove the alpha channel before passing onto ErlGuten. It ensures that the code not taking that inefficient path. It works very well. I hope ErlGuten needs to be improved for image embedding support and performance using port. Also when the alpha channel is removed, eg_pdg gen_server can be made to work for parallel PDF generation by removing the local name registration. Best, Theepan On Tue, Sep 25, 2018 at 7:50 PM Sean Hinde wrote: > I hit the same issue. I guess you are using one of the versions with lots > of lists:nth. > > Following find my patch on the version I was using. > > You can find my branch at: > > https://github.com/proactively/erlguten/tree/png-performance > > No time right now to make nice pull requests - I will eventually > > =============================================== > > diff --git a/src/eg_pdf_image.erl b/src/eg_pdf_image.erl > index 1a97e0b..817f1b8 100644 > --- a/src/eg_pdf_image.erl > +++ b/src/eg_pdf_image.erl > @@ -431,11 +431,11 @@ extractScanLines(Width,Decompressed) -> > > extractLine(ScanLines,Width,<< >>) -> > AlmostDone = lists:reverse(ScanLines), > - [ {0, string:chars(0,Width-1)} | AlmostDone]; > + [ {0, list_to_binary(string:chars(0,Width-1))} | AlmostDone]; > extractLine(ScanLines,Width,Image) -> > LineSize = Width - 1, > << Method:8, Line:LineSize/binary-unit:8, Rest/binary>> = Image, > - extractLine([{Method, binary_to_list(Line) } | ScanLines ], Width, > Rest). > + extractLine([{Method, Line} | ScanLines ], Width, Rest). > > %% @doc Remove the filter on all the bytes in the scan lines. > > @@ -448,19 +448,26 @@ processLine([{_Method, _Line1}], _Iter, _Offset, > Results)-> > A = lists:flatten( lists:reverse(Results) ), > list_to_binary( A ); > processLine([{_, Line1},{Method, Line2} | Remainder], Iter, Offset, > Results) -> > - {ok, Unfiltered} = defilter(Method, Line1, > Line2,Offset,length(Line1),Iter), > + {ok, Unfiltered} = defilter(Method, Line1, > Line2,Offset,size(Line1),Iter), > processLine([{Method, Unfiltered } | Remainder], Iter, Offset, > [Unfiltered | Results] ). > > %% @doc Taking two lines of stream defilter the 2nd with the previously > defiltered 1st line. > > defilter(Method, Line1, Line2, Offset, Width, Iter) when Iter =< Width -> > NewVal = case Iter =< Offset of > - true -> filter(lists:nth(Iter,Line2), 0, lists:nth(Iter,Line1), 0, > Method); > - false -> filter(lists:nth(Iter,Line2), lists:nth(Iter-Offset,Line2), > lists:nth(Iter,Line1), lists:nth(Iter-Offset,Line1), Method) > + true -> filter(binary:at(Line2, Iter - 1), 0, > + binary:at(Line1, Iter - 1), 0, Method); > + false -> filter(binary:at(Line2, Iter - 1), > + binary:at(Line2, Iter - Offset - 1), > + binary:at(Line1, Iter - 1), > + binary:at(Line1, Iter - Offset - 1), Method) > end, > L2 = case Iter == Width of > - true -> lists:flatten([lists:sublist(Line2,Iter - 1),NewVal]); > - false -> lists:flatten([lists:sublist(Line2,Iter - > 1),NewVal,lists:nthtail(Iter, Line2)]) > + true -> Start = binary:part(Line2, 0, Iter - 1), > + <>; > + false -> Start = binary:part(Line2, 0, Iter - 1), > + End = binary:part(Line2, Iter, size(Line2) - Iter), > + <> > end, > defilter(Method, Line1, L2 ,Offset, Width, Iter+1); > defilter(_Method, _Line1, Line2, _Offset, _Width, _Iter) -> > @@ -513,8 +520,7 @@ inflate_stream(Data) -> > Decompressed = zlib:inflate(Z, Data), > ok = zlib:inflateEnd(Z), > zlib:close(Z), > - F = fun(A, B) -> <> end, > - MergedBinaries = lists:foldr(F, <<>>, Decompressed), > + MergedBinaries = iolist_to_binary(Decompressed), > {ok,MergedBinaries}. > > %% @doc Compress a bit stream using the zlib/deflate algorithm > > =============================================== > > On 25 Sep 2018, at 01:15, Theepan wrote: > > Thanks Lloyd for your detailed explanation on what has been done, and your > vision for the library. However we are already using ErlGuten in one of our > production systems, and the issues is mainly with the images. > > I will definitely keep an eye on the development of erlpress_core in the > future, as you seem enthusiastic about making it a lively library. > > The issues we have are: > > ** When some JPEG image is embedded into the PDF, it turns out dark in the > generated PDF file > ** When some PNG file is embedded, it takes too long (unto 5 minutes) to > generate the PDF file. Debugging shows that the delay is on defilter( > Method, Line1, Line2, Offset, Width, Iter) of eg_pdf_image module. > > Did you come acrsoss this? > > Best, > Theepan > > > > > > On Tue, Sep 25, 2018 at 2:56 AM wrote: > >> Hi Theepan, >> >> 1. If you look at ErlGuten source, you'll see that function documentation >> is fairly minimal and many parameters have single character names with >> little to no documentation. This makes maintenance and revision very >> difficult. I consider the core ErlGuten modules diamonds-in-the-rough. >> >> I also found the high-level page make-up functions provided by Hugh and >> Carl limited with respect to professional page make-up and, to me at least, >> a bit confusing. >> >> But erlpress_core owes a deep debt and much gratitude to the ErlGuten >> lineage. >> >> erlpress_core is based on Joe's font-handling, justification, and XML >> parsing code with few if any changes. But it provides map structures to >> represent nearly all of the PDF objects represented in eg_pdf_op.erl. These >> map structures incorporate many default parameters so generating PDF >> documents is syntactically simple and consistent without sacrificing >> expressive flexibility. You can easily customize display by instantiating >> map parameters with your own values. >> >> You can see most of the PDF objects currently supported by erlpress_core, >> including boxed text, justification options, and various line and text >> objects, demonstrated in ep_show_n_tell.pdf and the source in >> ep_show_n_tell.erl. >> >> If you look at the map definitions of the various PDF objects, you'll see >> how to customize display. >> >> At some point if would be good to re-write the base Erlguten modules with >> more explicit function and parameter documentation, but I'm not up to that >> at this point. But, anyone looking for a challenge is free to jump in. >> >> I'm currently working on tables and text flow across pages (think reports >> and book chapters). Hope to deliver these features plus much code polishing >> in the next release coming "real soon now." >> >> High on my wish/to-do list are: >> >> -- Easy-to_use page-grid design functions >> -- Imposition (printing multiple page impressions on a single sheet of >> paper stock) >> -- Articles and beads (think text jumping across columns and pages) >> -- Easy-to-use and expressive page make-up functions >> -- Example templates for various print formats >> -- Markdown input >> -- Support of TTF and OTF fonts >> >> Joe Armstrong expressed the following goal when he first announced >> ErlGuten: >> >> "Better than TeX." >> >> That's a tall order, yet to be realized. My hope is that erlpress will >> move the ball further down the field. >> >> 2. I haven't done anything with image formats beyond what you'll find in >> eg_pdf_image.erl. I'd welcome work in that direction. >> >> 3. erlpress_core is still at Version 0.01. It's just out. I'm working >> toward sufficient functionality and stability to support a web application >> that we have currently under development. >> >> 4. I developed erlpress_core on Erlang/OTP 19. It does require support >> for maps. >> >> I see the erlpress_core library as a valuable library for embedding PDF >> generation into Erlang applications and as the basis for many exciting >> Erlang-based print production tools. >> >> So, Treepan, I appreciate your interest and would welcome your >> involvement in testing and feature development. >> >> I would jump for joy if some Erlang guru were to step forward take on the >> TTF/OTF support issue. I've done some research and have a few ideas on how >> expanded font support might be accomplished, but have too little time over >> the next several months to dig into it. >> >> Incidentally, in my previous post my iPad decided it was smarter than me >> and erroneously corrected my GitHUb address.it should be >> writersglen/erlPress_core. >> >> My intention is to bag the camel case in my next release with luck in a >> week or two so it should look like writersglen/erlpress_core. >> >> All the best, >> >> Lloyd >> >> >> -----Original Message----- >> From: "Theepan" >> Sent: Monday, September 24, 2018 3:59pm >> To: lloyd@REDACTED >> Cc: "Erlang Questions Mailing List" , "Joe >> Armstrong" >> Subject: Re: [erlang-questions] ErlGuten >> >> Hi Lloyd, >> >> Thank you for your response. I have some quick clarifications -- >> >> * What are the major improvements made on the source ErlGuten? Did you >> improve support to different image formats? Any critical bugs fixed? >> * Is erlPress_core used in commercial applications, specifically of high >> throughput types? >> * What is the minimum erlang/OTP version required? >> >> Best, >> Theepan >> >> >> >> >> >> >> On Mon, Sep 24, 2018 at 6:36 AM Lloyd R. Prentice >> wrote: >> >> > Hi Thepan, >> > >> > I?ve undertaken a major revision of ErlGuten called erlPress_core? >> > announced it on Erlang questions several weeks ago. It?s available from >> > GitHub as Writersglen/erlPress_core. I think you?ll find it much more >> > accessible than the ErlGutens. >> > >> > I?ve since been polishing up the code and adding a few new features. >> After >> > the launch I learned that camel case is not conventional usage when it >> > comes to the names of Erlang applications and libraries. So I plan to >> > release the updated version as erlpress_core. With smooth sailing I >> hope to >> > release sometime this week or next. >> > >> > All the best, >> > >> > Lloyd >> > >> > Sent from my iPad >> > >> > On Sep 23, 2018, at 8:30 PM, Theepan wrote: >> > >> > >> > Hi Team, >> > >> > I see many forks of ErlGuten. Which one is the latest, most stable and >> > widely used? >> > >> > https://github.com/richcarl/erlguten >> > https://github.com/CarlWright/NGerlguten >> > https://github.com/hwatkins/erlguten/commits/master >> > >> > Best, >> > Theepan >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> >> >> _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vasdeveloper@REDACTED Tue Sep 25 20:55:53 2018 From: vasdeveloper@REDACTED (Theepan) Date: Wed, 26 Sep 2018 00:25:53 +0530 Subject: [erlang-questions] ErlGuten In-Reply-To: <1537894270.37611727@apps.rackspace.com> References: <7AB3CEDA-34AF-41E2-BEE9-2B27274FF680@writersglen.com> <1537824375.310720011@apps.rackspace.com> <1537894270.37611727@apps.rackspace.com> Message-ID: Hi Lloyd, You are right. PDF generation library is crucial to most of the commercial applications. The code base is not very complicated, but then again we need somebody who has time and willingness to take on the work. Please find what have written to Sean on what improvements can be crucial short term. Wish work arounds and short fixes we could achieve the required scemantics, performance and throughput out of ErlGuten. Best, Theepan On Tue, Sep 25, 2018 at 10:21 PM wrote: > Hi Theepan, > > Other than display in my demo PDF, I haven't worked much with images. > > The issues you've run across seem serious. Getting at the root of the > problems will likely involve a deep dive into both the PDF reference > manuals and core ErlGuten source code. > > As I noted in my earlier email, ErlGuten source code is very challenging. > It would benefit significantly from re-write, possibly refactoring in some > cases and definitely systematic documentation of functions and parameters. > > I'm not confident that my skills are up to this. My take is that it would > require some amount of community effort to put ErlGuten/erlpress_core on a > solid, maintainable base. I would dearly love to see this happen. > > Is there a CS major or intern out there with time to take on the > challenges? > > Joe showed us the way and I, for one, am deeply grateful. Given that PDF > is a significant standard in the print media world, Erlang deserves to have > a world-class library and suite of tools to generate PDF documents and > decode them back into Erlang structures. > > This will take community effort. > > All the best, > > Lloyd > > > > > > -----Original Message----- > From: "Theepan" > Sent: Monday, September 24, 2018 7:27pm > To: "Lloyd Prentice" > Cc: "Erlang Questions Mailing List" , "Joe > Armstrong" > Subject: Re: [erlang-questions] ErlGuten > > Wanted to add to the issues the reasons -- > > ** When some JPEG image is embedded into the PDF, it turns out dark in the > generated PDF file [Happens due to CYMK color profile.] > ** When some PNG file is embedded, it takes too long (unto 5 minutes) to > generate the PDF file. Debugging shows that the delay is on defilter(Method > , Line1, Line2, Offset, Width, Iter) of eg_pdf_image module.[Happens due to > Alpha channel presence] > > > > On Tue, Sep 25, 2018 at 4:45 AM Theepan wrote: > > > Thanks Lloyd for your detailed explanation on what has been done, and > your > > vision for the library. However we are already using ErlGuten in one of > our > > production systems, and the issues is mainly with the images. > > > > I will definitely keep an eye on the development of erlpress_core in the > > future, as you seem enthusiastic about making it a lively library. > > > > The issues we have are: > > > > ** When some JPEG image is embedded into the PDF, it turns out dark in > the > > generated PDF file > > ** When some PNG file is embedded, it takes too long (unto 5 minutes) to > > generate the PDF file. Debugging shows that the delay is on defilter( > > Method, Line1, Line2, Offset, Width, Iter) of eg_pdf_image module. > > > > Did you come acrsoss this? > > > > Best, > > Theepan > > > > > > > > > > > > On Tue, Sep 25, 2018 at 2:56 AM wrote: > > > >> Hi Theepan, > >> > >> 1. If you look at ErlGuten source, you'll see that function > documentation > >> is fairly minimal and many parameters have single character names with > >> little to no documentation. This makes maintenance and revision very > >> difficult. I consider the core ErlGuten modules diamonds-in-the-rough. > >> > >> I also found the high-level page make-up functions provided by Hugh and > >> Carl limited with respect to professional page make-up and, to me at > least, > >> a bit confusing. > >> > >> But erlpress_core owes a deep debt and much gratitude to the ErlGuten > >> lineage. > >> > >> erlpress_core is based on Joe's font-handling, justification, and XML > >> parsing code with few if any changes. But it provides map structures to > >> represent nearly all of the PDF objects represented in eg_pdf_op.erl. > These > >> map structures incorporate many default parameters so generating PDF > >> documents is syntactically simple and consistent without sacrificing > >> expressive flexibility. You can easily customize display by > instantiating > >> map parameters with your own values. > >> > >> You can see most of the PDF objects currently supported by > erlpress_core, > >> including boxed text, justification options, and various line and text > >> objects, demonstrated in ep_show_n_tell.pdf and the source in > >> ep_show_n_tell.erl. > >> > >> If you look at the map definitions of the various PDF objects, you'll > see > >> how to customize display. > >> > >> At some point if would be good to re-write the base Erlguten modules > with > >> more explicit function and parameter documentation, but I'm not up to > that > >> at this point. But, anyone looking for a challenge is free to jump in. > >> > >> I'm currently working on tables and text flow across pages (think > reports > >> and book chapters). Hope to deliver these features plus much code > polishing > >> in the next release coming "real soon now." > >> > >> High on my wish/to-do list are: > >> > >> -- Easy-to_use page-grid design functions > >> -- Imposition (printing multiple page impressions on a single sheet of > >> paper stock) > >> -- Articles and beads (think text jumping across columns and pages) > >> -- Easy-to-use and expressive page make-up functions > >> -- Example templates for various print formats > >> -- Markdown input > >> -- Support of TTF and OTF fonts > >> > >> Joe Armstrong expressed the following goal when he first announced > >> ErlGuten: > >> > >> "Better than TeX." > >> > >> That's a tall order, yet to be realized. My hope is that erlpress will > >> move the ball further down the field. > >> > >> 2. I haven't done anything with image formats beyond what you'll find in > >> eg_pdf_image.erl. I'd welcome work in that direction. > >> > >> 3. erlpress_core is still at Version 0.01. It's just out. I'm working > >> toward sufficient functionality and stability to support a web > application > >> that we have currently under development. > >> > >> 4. I developed erlpress_core on Erlang/OTP 19. It does require support > >> for maps. > >> > >> I see the erlpress_core library as a valuable library for embedding PDF > >> generation into Erlang applications and as the basis for many exciting > >> Erlang-based print production tools. > >> > >> So, Treepan, I appreciate your interest and would welcome your > >> involvement in testing and feature development. > >> > >> I would jump for joy if some Erlang guru were to step forward take on > the > >> TTF/OTF support issue. I've done some research and have a few ideas on > how > >> expanded font support might be accomplished, but have too little time > over > >> the next several months to dig into it. > >> > >> Incidentally, in my previous post my iPad decided it was smarter than me > >> and erroneously corrected my GitHUb address.it should be > >> writersglen/erlPress_core. > >> > >> My intention is to bag the camel case in my next release with luck in a > >> week or two so it should look like writersglen/erlpress_core. > >> > >> All the best, > >> > >> Lloyd > >> > >> > >> -----Original Message----- > >> From: "Theepan" > >> Sent: Monday, September 24, 2018 3:59pm > >> To: lloyd@REDACTED > >> Cc: "Erlang Questions Mailing List" , "Joe > >> Armstrong" > >> Subject: Re: [erlang-questions] ErlGuten > >> > >> Hi Lloyd, > >> > >> Thank you for your response. I have some quick clarifications -- > >> > >> * What are the major improvements made on the source ErlGuten? Did you > >> improve support to different image formats? Any critical bugs fixed? > >> * Is erlPress_core used in commercial applications, specifically of high > >> throughput types? > >> * What is the minimum erlang/OTP version required? > >> > >> Best, > >> Theepan > >> > >> > >> > >> > >> > >> > >> On Mon, Sep 24, 2018 at 6:36 AM Lloyd R. Prentice < > lloyd@REDACTED> > >> wrote: > >> > >> > Hi Thepan, > >> > > >> > I?ve undertaken a major revision of ErlGuten called erlPress_core? > >> > announced it on Erlang questions several weeks ago. It?s available > from > >> > GitHub as Writersglen/erlPress_core. I think you?ll find it much more > >> > accessible than the ErlGutens. > >> > > >> > I?ve since been polishing up the code and adding a few new features. > >> After > >> > the launch I learned that camel case is not conventional usage when it > >> > comes to the names of Erlang applications and libraries. So I plan to > >> > release the updated version as erlpress_core. With smooth sailing I > >> hope to > >> > release sometime this week or next. > >> > > >> > All the best, > >> > > >> > Lloyd > >> > > >> > Sent from my iPad > >> > > >> > On Sep 23, 2018, at 8:30 PM, Theepan wrote: > >> > > >> > > >> > Hi Team, > >> > > >> > I see many forks of ErlGuten. Which one is the latest, most stable and > >> > widely used? > >> > > >> > https://github.com/richcarl/erlguten > >> > https://github.com/CarlWright/NGerlguten > >> > https://github.com/hwatkins/erlguten/commits/master > >> > > >> > Best, > >> > Theepan > >> > > >> > > >> > _______________________________________________ > >> > erlang-questions mailing list > >> > erlang-questions@REDACTED > >> > http://erlang.org/mailman/listinfo/erlang-questions > >> > > >> > > >> > >> > >> > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Tue Sep 25 21:42:34 2018 From: lloyd@REDACTED (lloyd@REDACTED) Date: Tue, 25 Sep 2018 15:42:34 -0400 (EDT) Subject: [erlang-questions] ErlGuten + a short survey In-Reply-To: References: <7AB3CEDA-34AF-41E2-BEE9-2B27274FF680@writersglen.com> <1537824375.310720011@apps.rackspace.com> <1537894270.37611727@apps.rackspace.com> Message-ID: <1537904554.681631612@apps.rackspace.com> Hi folks, As Theepan says, "PDF generation library is crucial to most of the commercial applications." If you are using ErlGuten, please help inform my work on erlpress_core: 1. What are you using it for? 2. What features do you wish it has but doesn't? Many thanks, Lloyd -----Original Message----- From: "Theepan" Sent: Tuesday, September 25, 2018 2:55pm To: "Lloyd Prentice" Cc: "Erlang Questions Mailing List" , "Joe Armstrong" Subject: Re: [erlang-questions] ErlGuten Hi Lloyd, You are right. PDF generation library is crucial to most of the commercial applications. The code base is not very complicated, but then again we need somebody who has time and willingness to take on the work. Please find what have written to Sean on what improvements can be crucial short term. Wish work arounds and short fixes we could achieve the required scemantics, performance and throughput out of ErlGuten. Best, Theepan On Tue, Sep 25, 2018 at 10:21 PM wrote: > Hi Theepan, > > Other than display in my demo PDF, I haven't worked much with images. > > The issues you've run across seem serious. Getting at the root of the > problems will likely involve a deep dive into both the PDF reference > manuals and core ErlGuten source code. > > As I noted in my earlier email, ErlGuten source code is very challenging. > It would benefit significantly from re-write, possibly refactoring in some > cases and definitely systematic documentation of functions and parameters. > > I'm not confident that my skills are up to this. My take is that it would > require some amount of community effort to put ErlGuten/erlpress_core on a > solid, maintainable base. I would dearly love to see this happen. > > Is there a CS major or intern out there with time to take on the > challenges? > > Joe showed us the way and I, for one, am deeply grateful. Given that PDF > is a significant standard in the print media world, Erlang deserves to have > a world-class library and suite of tools to generate PDF documents and > decode them back into Erlang structures. > > This will take community effort. > > All the best, > > Lloyd > > > > > > -----Original Message----- > From: "Theepan" > Sent: Monday, September 24, 2018 7:27pm > To: "Lloyd Prentice" > Cc: "Erlang Questions Mailing List" , "Joe > Armstrong" > Subject: Re: [erlang-questions] ErlGuten > > Wanted to add to the issues the reasons -- > > ** When some JPEG image is embedded into the PDF, it turns out dark in the > generated PDF file [Happens due to CYMK color profile.] > ** When some PNG file is embedded, it takes too long (unto 5 minutes) to > generate the PDF file. Debugging shows that the delay is on defilter(Method > , Line1, Line2, Offset, Width, Iter) of eg_pdf_image module.[Happens due to > Alpha channel presence] > > > > On Tue, Sep 25, 2018 at 4:45 AM Theepan wrote: > > > Thanks Lloyd for your detailed explanation on what has been done, and > your > > vision for the library. However we are already using ErlGuten in one of > our > > production systems, and the issues is mainly with the images. > > > > I will definitely keep an eye on the development of erlpress_core in the > > future, as you seem enthusiastic about making it a lively library. > > > > The issues we have are: > > > > ** When some JPEG image is embedded into the PDF, it turns out dark in > the > > generated PDF file > > ** When some PNG file is embedded, it takes too long (unto 5 minutes) to > > generate the PDF file. Debugging shows that the delay is on defilter( > > Method, Line1, Line2, Offset, Width, Iter) of eg_pdf_image module. > > > > Did you come acrsoss this? > > > > Best, > > Theepan > > > > > > > > > > > > On Tue, Sep 25, 2018 at 2:56 AM wrote: > > > >> Hi Theepan, > >> > >> 1. If you look at ErlGuten source, you'll see that function > documentation > >> is fairly minimal and many parameters have single character names with > >> little to no documentation. This makes maintenance and revision very > >> difficult. I consider the core ErlGuten modules diamonds-in-the-rough. > >> > >> I also found the high-level page make-up functions provided by Hugh and > >> Carl limited with respect to professional page make-up and, to me at > least, > >> a bit confusing. > >> > >> But erlpress_core owes a deep debt and much gratitude to the ErlGuten > >> lineage. > >> > >> erlpress_core is based on Joe's font-handling, justification, and XML > >> parsing code with few if any changes. But it provides map structures to > >> represent nearly all of the PDF objects represented in eg_pdf_op.erl. > These > >> map structures incorporate many default parameters so generating PDF > >> documents is syntactically simple and consistent without sacrificing > >> expressive flexibility. You can easily customize display by > instantiating > >> map parameters with your own values. > >> > >> You can see most of the PDF objects currently supported by > erlpress_core, > >> including boxed text, justification options, and various line and text > >> objects, demonstrated in ep_show_n_tell.pdf and the source in > >> ep_show_n_tell.erl. > >> > >> If you look at the map definitions of the various PDF objects, you'll > see > >> how to customize display. > >> > >> At some point if would be good to re-write the base Erlguten modules > with > >> more explicit function and parameter documentation, but I'm not up to > that > >> at this point. But, anyone looking for a challenge is free to jump in. > >> > >> I'm currently working on tables and text flow across pages (think > reports > >> and book chapters). Hope to deliver these features plus much code > polishing > >> in the next release coming "real soon now." > >> > >> High on my wish/to-do list are: > >> > >> -- Easy-to_use page-grid design functions > >> -- Imposition (printing multiple page impressions on a single sheet of > >> paper stock) > >> -- Articles and beads (think text jumping across columns and pages) > >> -- Easy-to-use and expressive page make-up functions > >> -- Example templates for various print formats > >> -- Markdown input > >> -- Support of TTF and OTF fonts > >> > >> Joe Armstrong expressed the following goal when he first announced > >> ErlGuten: > >> > >> "Better than TeX." > >> > >> That's a tall order, yet to be realized. My hope is that erlpress will > >> move the ball further down the field. > >> > >> 2. I haven't done anything with image formats beyond what you'll find in > >> eg_pdf_image.erl. I'd welcome work in that direction. > >> > >> 3. erlpress_core is still at Version 0.01. It's just out. I'm working > >> toward sufficient functionality and stability to support a web > application > >> that we have currently under development. > >> > >> 4. I developed erlpress_core on Erlang/OTP 19. It does require support > >> for maps. > >> > >> I see the erlpress_core library as a valuable library for embedding PDF > >> generation into Erlang applications and as the basis for many exciting > >> Erlang-based print production tools. > >> > >> So, Treepan, I appreciate your interest and would welcome your > >> involvement in testing and feature development. > >> > >> I would jump for joy if some Erlang guru were to step forward take on > the > >> TTF/OTF support issue. I've done some research and have a few ideas on > how > >> expanded font support might be accomplished, but have too little time > over > >> the next several months to dig into it. > >> > >> Incidentally, in my previous post my iPad decided it was smarter than me > >> and erroneously corrected my GitHUb address.it should be > >> writersglen/erlPress_core. > >> > >> My intention is to bag the camel case in my next release with luck in a > >> week or two so it should look like writersglen/erlpress_core. > >> > >> All the best, > >> > >> Lloyd > >> > >> > >> -----Original Message----- > >> From: "Theepan" > >> Sent: Monday, September 24, 2018 3:59pm > >> To: lloyd@REDACTED > >> Cc: "Erlang Questions Mailing List" , "Joe > >> Armstrong" > >> Subject: Re: [erlang-questions] ErlGuten > >> > >> Hi Lloyd, > >> > >> Thank you for your response. I have some quick clarifications -- > >> > >> * What are the major improvements made on the source ErlGuten? Did you > >> improve support to different image formats? Any critical bugs fixed? > >> * Is erlPress_core used in commercial applications, specifically of high > >> throughput types? > >> * What is the minimum erlang/OTP version required? > >> > >> Best, > >> Theepan > >> > >> > >> > >> > >> > >> > >> On Mon, Sep 24, 2018 at 6:36 AM Lloyd R. Prentice < > lloyd@REDACTED> > >> wrote: > >> > >> > Hi Thepan, > >> > > >> > I?ve undertaken a major revision of ErlGuten called erlPress_core? > >> > announced it on Erlang questions several weeks ago. It?s available > from > >> > GitHub as Writersglen/erlPress_core. I think you?ll find it much more > >> > accessible than the ErlGutens. > >> > > >> > I?ve since been polishing up the code and adding a few new features. > >> After > >> > the launch I learned that camel case is not conventional usage when it > >> > comes to the names of Erlang applications and libraries. So I plan to > >> > release the updated version as erlpress_core. With smooth sailing I > >> hope to > >> > release sometime this week or next. > >> > > >> > All the best, > >> > > >> > Lloyd > >> > > >> > Sent from my iPad > >> > > >> > On Sep 23, 2018, at 8:30 PM, Theepan wrote: > >> > > >> > > >> > Hi Team, > >> > > >> > I see many forks of ErlGuten. Which one is the latest, most stable and > >> > widely used? > >> > > >> > https://github.com/richcarl/erlguten > >> > https://github.com/CarlWright/NGerlguten > >> > https://github.com/hwatkins/erlguten/commits/master > >> > > >> > Best, > >> > Theepan > >> > > >> > > >> > _______________________________________________ > >> > erlang-questions mailing list > >> > erlang-questions@REDACTED > >> > http://erlang.org/mailman/listinfo/erlang-questions > >> > > >> > > >> > >> > >> > > > From sean.hinde@REDACTED Tue Sep 25 21:42:47 2018 From: sean.hinde@REDACTED (Sean Hinde) Date: Tue, 25 Sep 2018 21:42:47 +0200 Subject: [erlang-questions] ErlGuten In-Reply-To: References: <7AB3CEDA-34AF-41E2-BEE9-2B27274FF680@writersglen.com> <1537824375.310720011@apps.rackspace.com> <7C188AE2-A0E3-4E06-B541-C6BEB38C2957@mac.com> Message-ID: <1D63E4FC-A4E1-4EF0-BE74-4AB69B25E9B1@mac.com> Yeah. Erlang is not normally seen as the ultimate language for that kind of image processing. For my application I need the alpha channel, so that's me. If it becomes a bigger perf problem I would imagine some integration with libpng is in order. I don't see why the alpha channel should require a named process, but I've not got very far with my usage. Will check - thanks for the heads up And the code isn't actually that complex - it's mostly just that PDF itself is fairly complex Sean > On 25 Sep 2018, at 20:45, Theepan wrote: > > Hi Sean, > > I used GraphicsMagic to remove the alpha channel before passing onto ErlGuten. It ensures that the code not taking that inefficient path. It works very well. > > I hope ErlGuten needs to be improved for image embedding support and performance using port. Also when the alpha channel is removed, eg_pdg gen_server can be made to work for parallel PDF generation by removing the local name registration. > > Best, > Theepan > > > > On Tue, Sep 25, 2018 at 7:50 PM Sean Hinde > wrote: > I hit the same issue. I guess you are using one of the versions with lots of lists:nth. > > Following find my patch on the version I was using. > > You can find my branch at: > > https://github.com/proactively/erlguten/tree/png-performance > > No time right now to make nice pull requests - I will eventually > > =============================================== > > diff --git a/src/eg_pdf_image.erl b/src/eg_pdf_image.erl > index 1a97e0b..817f1b8 100644 > --- a/src/eg_pdf_image.erl > +++ b/src/eg_pdf_image.erl > @@ -431,11 +431,11 @@ extractScanLines(Width,Decompressed) -> > > extractLine(ScanLines,Width,<< >>) -> > AlmostDone = lists:reverse(ScanLines), > - [ {0, string:chars(0,Width-1)} | AlmostDone]; > + [ {0, list_to_binary(string:chars(0,Width-1))} | AlmostDone]; > extractLine(ScanLines,Width,Image) -> > LineSize = Width - 1, > << Method:8, Line:LineSize/binary-unit:8, Rest/binary>> = Image, > - extractLine([{Method, binary_to_list(Line) } | ScanLines ], Width, Rest). > + extractLine([{Method, Line} | ScanLines ], Width, Rest). > > %% @doc Remove the filter on all the bytes in the scan lines. > > @@ -448,19 +448,26 @@ processLine([{_Method, _Line1}], _Iter, _Offset, Results)-> > A = lists:flatten( lists:reverse(Results) ), > list_to_binary( A ); > processLine([{_, Line1},{Method, Line2} | Remainder], Iter, Offset, Results) -> > - {ok, Unfiltered} = defilter(Method, Line1, Line2,Offset,length(Line1),Iter), > + {ok, Unfiltered} = defilter(Method, Line1, Line2,Offset,size(Line1),Iter), > processLine([{Method, Unfiltered } | Remainder], Iter, Offset, [Unfiltered | Results] ). > > %% @doc Taking two lines of stream defilter the 2nd with the previously defiltered 1st line. > > defilter(Method, Line1, Line2, Offset, Width, Iter) when Iter =< Width -> > NewVal = case Iter =< Offset of > - true -> filter(lists:nth(Iter,Line2), 0, lists:nth(Iter,Line1), 0, Method); > - false -> filter(lists:nth(Iter,Line2), lists:nth(Iter-Offset,Line2), lists:nth(Iter,Line1), lists:nth(Iter-Offset,Line1), Method) > + true -> filter(binary:at(Line2, Iter - 1), 0, > + binary:at(Line1, Iter - 1), 0, Method); > + false -> filter(binary:at(Line2, Iter - 1), > + binary:at(Line2, Iter - Offset - 1), > + binary:at(Line1, Iter - 1), > + binary:at(Line1, Iter - Offset - 1), Method) > end, > L2 = case Iter == Width of > - true -> lists:flatten([lists:sublist(Line2,Iter - 1),NewVal]); > - false -> lists:flatten([lists:sublist(Line2,Iter - 1),NewVal,lists:nthtail(Iter, Line2)]) > + true -> Start = binary:part(Line2, 0, Iter - 1), > + <>; > + false -> Start = binary:part(Line2, 0, Iter - 1), > + End = binary:part(Line2, Iter, size(Line2) - Iter), > + <> > end, > defilter(Method, Line1, L2 ,Offset, Width, Iter+1); > defilter(_Method, _Line1, Line2, _Offset, _Width, _Iter) -> > @@ -513,8 +520,7 @@ inflate_stream(Data) -> > Decompressed = zlib:inflate(Z, Data), > ok = zlib:inflateEnd(Z), > zlib:close(Z), > - F = fun(A, B) -> <> end, > - MergedBinaries = lists:foldr(F, <<>>, Decompressed), > + MergedBinaries = iolist_to_binary(Decompressed), > {ok,MergedBinaries}. > > %% @doc Compress a bit stream using the zlib/deflate algorithm > > =============================================== > >> On 25 Sep 2018, at 01:15, Theepan > wrote: >> >> Thanks Lloyd for your detailed explanation on what has been done, and your vision for the library. However we are already using ErlGuten in one of our production systems, and the issues is mainly with the images. >> >> I will definitely keep an eye on the development of erlpress_core in the future, as you seem enthusiastic about making it a lively library. >> >> The issues we have are: >> >> ** When some JPEG image is embedded into the PDF, it turns out dark in the generated PDF file >> ** When some PNG file is embedded, it takes too long (unto 5 minutes) to generate the PDF file. Debugging shows that the delay is on defilter(Method, Line1, Line2, Offset, Width, Iter) of eg_pdf_image module. >> >> Did you come acrsoss this? >> >> Best, >> Theepan >> >> >> >> >> >> On Tue, Sep 25, 2018 at 2:56 AM > wrote: >> Hi Theepan, >> >> 1. If you look at ErlGuten source, you'll see that function documentation is fairly minimal and many parameters have single character names with little to no documentation. This makes maintenance and revision very difficult. I consider the core ErlGuten modules diamonds-in-the-rough. >> >> I also found the high-level page make-up functions provided by Hugh and Carl limited with respect to professional page make-up and, to me at least, a bit confusing. >> >> But erlpress_core owes a deep debt and much gratitude to the ErlGuten lineage. >> >> erlpress_core is based on Joe's font-handling, justification, and XML parsing code with few if any changes. But it provides map structures to represent nearly all of the PDF objects represented in eg_pdf_op.erl. These map structures incorporate many default parameters so generating PDF documents is syntactically simple and consistent without sacrificing expressive flexibility. You can easily customize display by instantiating map parameters with your own values. >> >> You can see most of the PDF objects currently supported by erlpress_core, including boxed text, justification options, and various line and text objects, demonstrated in ep_show_n_tell.pdf and the source in ep_show_n_tell.erl. >> >> If you look at the map definitions of the various PDF objects, you'll see how to customize display. >> >> At some point if would be good to re-write the base Erlguten modules with more explicit function and parameter documentation, but I'm not up to that at this point. But, anyone looking for a challenge is free to jump in. >> >> I'm currently working on tables and text flow across pages (think reports and book chapters). Hope to deliver these features plus much code polishing in the next release coming "real soon now." >> >> High on my wish/to-do list are: >> >> -- Easy-to_use page-grid design functions >> -- Imposition (printing multiple page impressions on a single sheet of paper stock) >> -- Articles and beads (think text jumping across columns and pages) >> -- Easy-to-use and expressive page make-up functions >> -- Example templates for various print formats >> -- Markdown input >> -- Support of TTF and OTF fonts >> >> Joe Armstrong expressed the following goal when he first announced ErlGuten: >> >> "Better than TeX." >> >> That's a tall order, yet to be realized. My hope is that erlpress will move the ball further down the field. >> >> 2. I haven't done anything with image formats beyond what you'll find in eg_pdf_image.erl. I'd welcome work in that direction. >> >> 3. erlpress_core is still at Version 0.01. It's just out. I'm working toward sufficient functionality and stability to support a web application that we have currently under development. >> >> 4. I developed erlpress_core on Erlang/OTP 19. It does require support for maps. >> >> I see the erlpress_core library as a valuable library for embedding PDF generation into Erlang applications and as the basis for many exciting Erlang-based print production tools. >> >> So, Treepan, I appreciate your interest and would welcome your involvement in testing and feature development. >> >> I would jump for joy if some Erlang guru were to step forward take on the TTF/OTF support issue. I've done some research and have a few ideas on how expanded font support might be accomplished, but have too little time over the next several months to dig into it. >> >> Incidentally, in my previous post my iPad decided it was smarter than me and erroneously corrected my GitHUb address.it should be writersglen/erlPress_core. >> >> My intention is to bag the camel case in my next release with luck in a week or two so it should look like writersglen/erlpress_core. >> >> All the best, >> >> Lloyd >> >> >> -----Original Message----- >> From: "Theepan" > >> Sent: Monday, September 24, 2018 3:59pm >> To: lloyd@REDACTED >> Cc: "Erlang Questions Mailing List" >, "Joe Armstrong" > >> Subject: Re: [erlang-questions] ErlGuten >> >> Hi Lloyd, >> >> Thank you for your response. I have some quick clarifications -- >> >> * What are the major improvements made on the source ErlGuten? Did you >> improve support to different image formats? Any critical bugs fixed? >> * Is erlPress_core used in commercial applications, specifically of high >> throughput types? >> * What is the minimum erlang/OTP version required? >> >> Best, >> Theepan >> >> >> >> >> >> >> On Mon, Sep 24, 2018 at 6:36 AM Lloyd R. Prentice > >> wrote: >> >> > Hi Thepan, >> > >> > I?ve undertaken a major revision of ErlGuten called erlPress_core? >> > announced it on Erlang questions several weeks ago. It?s available from >> > GitHub as Writersglen/erlPress_core. I think you?ll find it much more >> > accessible than the ErlGutens. >> > >> > I?ve since been polishing up the code and adding a few new features. After >> > the launch I learned that camel case is not conventional usage when it >> > comes to the names of Erlang applications and libraries. So I plan to >> > release the updated version as erlpress_core. With smooth sailing I hope to >> > release sometime this week or next. >> > >> > All the best, >> > >> > Lloyd >> > >> > Sent from my iPad >> > >> > On Sep 23, 2018, at 8:30 PM, Theepan > wrote: >> > >> > >> > Hi Team, >> > >> > I see many forks of ErlGuten. Which one is the latest, most stable and >> > widely used? >> > >> > https://github.com/richcarl/erlguten >> > https://github.com/CarlWright/NGerlguten >> > https://github.com/hwatkins/erlguten/commits/master >> > >> > Best, >> > Theepan >> > >> > >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmytro.lytovchenko@REDACTED Tue Sep 25 22:06:09 2018 From: dmytro.lytovchenko@REDACTED (Dmytro Lytovchenko) Date: Tue, 25 Sep 2018 22:06:09 +0200 Subject: [erlang-questions] ErlGuten In-Reply-To: <1537894270.37611727@apps.rackspace.com> References: <7AB3CEDA-34AF-41E2-BEE9-2B27274FF680@writersglen.com> <1537824375.310720011@apps.rackspace.com> <1537894270.37611727@apps.rackspace.com> Message-ID: I am doing a prettifying pass now: - Adding function specs - Converting text specs to normal -spec specs - Adding named types where they make sense - Replacing tabs with spaces - Running Dialyzer on it, and fixing types so it makes sense - ... that sort of thing. Can't promise when it will be finished, a day or two, or 3 weeks. You'll have a pull request when its done. Will be a MASSIVE changeset but no actual logic affected. On Tue, 25 Sep 2018 at 18:51, wrote: > Hi Theepan, > > Other than display in my demo PDF, I haven't worked much with images. > > The issues you've run across seem serious. Getting at the root of the > problems will likely involve a deep dive into both the PDF reference > manuals and core ErlGuten source code. > > As I noted in my earlier email, ErlGuten source code is very challenging. > It would benefit significantly from re-write, possibly refactoring in some > cases and definitely systematic documentation of functions and parameters. > > I'm not confident that my skills are up to this. My take is that it would > require some amount of community effort to put ErlGuten/erlpress_core on a > solid, maintainable base. I would dearly love to see this happen. > > Is there a CS major or intern out there with time to take on the > challenges? > > Joe showed us the way and I, for one, am deeply grateful. Given that PDF > is a significant standard in the print media world, Erlang deserves to have > a world-class library and suite of tools to generate PDF documents and > decode them back into Erlang structures. > > This will take community effort. > > All the best, > > Lloyd > > > > > > -----Original Message----- > From: "Theepan" > Sent: Monday, September 24, 2018 7:27pm > To: "Lloyd Prentice" > Cc: "Erlang Questions Mailing List" , "Joe > Armstrong" > Subject: Re: [erlang-questions] ErlGuten > > Wanted to add to the issues the reasons -- > > ** When some JPEG image is embedded into the PDF, it turns out dark in the > generated PDF file [Happens due to CYMK color profile.] > ** When some PNG file is embedded, it takes too long (unto 5 minutes) to > generate the PDF file. Debugging shows that the delay is on defilter(Method > , Line1, Line2, Offset, Width, Iter) of eg_pdf_image module.[Happens due to > Alpha channel presence] > > > > On Tue, Sep 25, 2018 at 4:45 AM Theepan wrote: > > > Thanks Lloyd for your detailed explanation on what has been done, and > your > > vision for the library. However we are already using ErlGuten in one of > our > > production systems, and the issues is mainly with the images. > > > > I will definitely keep an eye on the development of erlpress_core in the > > future, as you seem enthusiastic about making it a lively library. > > > > The issues we have are: > > > > ** When some JPEG image is embedded into the PDF, it turns out dark in > the > > generated PDF file > > ** When some PNG file is embedded, it takes too long (unto 5 minutes) to > > generate the PDF file. Debugging shows that the delay is on defilter( > > Method, Line1, Line2, Offset, Width, Iter) of eg_pdf_image module. > > > > Did you come acrsoss this? > > > > Best, > > Theepan > > > > > > > > > > > > On Tue, Sep 25, 2018 at 2:56 AM wrote: > > > >> Hi Theepan, > >> > >> 1. If you look at ErlGuten source, you'll see that function > documentation > >> is fairly minimal and many parameters have single character names with > >> little to no documentation. This makes maintenance and revision very > >> difficult. I consider the core ErlGuten modules diamonds-in-the-rough. > >> > >> I also found the high-level page make-up functions provided by Hugh and > >> Carl limited with respect to professional page make-up and, to me at > least, > >> a bit confusing. > >> > >> But erlpress_core owes a deep debt and much gratitude to the ErlGuten > >> lineage. > >> > >> erlpress_core is based on Joe's font-handling, justification, and XML > >> parsing code with few if any changes. But it provides map structures to > >> represent nearly all of the PDF objects represented in eg_pdf_op.erl. > These > >> map structures incorporate many default parameters so generating PDF > >> documents is syntactically simple and consistent without sacrificing > >> expressive flexibility. You can easily customize display by > instantiating > >> map parameters with your own values. > >> > >> You can see most of the PDF objects currently supported by > erlpress_core, > >> including boxed text, justification options, and various line and text > >> objects, demonstrated in ep_show_n_tell.pdf and the source in > >> ep_show_n_tell.erl. > >> > >> If you look at the map definitions of the various PDF objects, you'll > see > >> how to customize display. > >> > >> At some point if would be good to re-write the base Erlguten modules > with > >> more explicit function and parameter documentation, but I'm not up to > that > >> at this point. But, anyone looking for a challenge is free to jump in. > >> > >> I'm currently working on tables and text flow across pages (think > reports > >> and book chapters). Hope to deliver these features plus much code > polishing > >> in the next release coming "real soon now." > >> > >> High on my wish/to-do list are: > >> > >> -- Easy-to_use page-grid design functions > >> -- Imposition (printing multiple page impressions on a single sheet of > >> paper stock) > >> -- Articles and beads (think text jumping across columns and pages) > >> -- Easy-to-use and expressive page make-up functions > >> -- Example templates for various print formats > >> -- Markdown input > >> -- Support of TTF and OTF fonts > >> > >> Joe Armstrong expressed the following goal when he first announced > >> ErlGuten: > >> > >> "Better than TeX." > >> > >> That's a tall order, yet to be realized. My hope is that erlpress will > >> move the ball further down the field. > >> > >> 2. I haven't done anything with image formats beyond what you'll find in > >> eg_pdf_image.erl. I'd welcome work in that direction. > >> > >> 3. erlpress_core is still at Version 0.01. It's just out. I'm working > >> toward sufficient functionality and stability to support a web > application > >> that we have currently under development. > >> > >> 4. I developed erlpress_core on Erlang/OTP 19. It does require support > >> for maps. > >> > >> I see the erlpress_core library as a valuable library for embedding PDF > >> generation into Erlang applications and as the basis for many exciting > >> Erlang-based print production tools. > >> > >> So, Treepan, I appreciate your interest and would welcome your > >> involvement in testing and feature development. > >> > >> I would jump for joy if some Erlang guru were to step forward take on > the > >> TTF/OTF support issue. I've done some research and have a few ideas on > how > >> expanded font support might be accomplished, but have too little time > over > >> the next several months to dig into it. > >> > >> Incidentally, in my previous post my iPad decided it was smarter than me > >> and erroneously corrected my GitHUb address.it should be > >> writersglen/erlPress_core. > >> > >> My intention is to bag the camel case in my next release with luck in a > >> week or two so it should look like writersglen/erlpress_core. > >> > >> All the best, > >> > >> Lloyd > >> > >> > >> -----Original Message----- > >> From: "Theepan" > >> Sent: Monday, September 24, 2018 3:59pm > >> To: lloyd@REDACTED > >> Cc: "Erlang Questions Mailing List" , "Joe > >> Armstrong" > >> Subject: Re: [erlang-questions] ErlGuten > >> > >> Hi Lloyd, > >> > >> Thank you for your response. I have some quick clarifications -- > >> > >> * What are the major improvements made on the source ErlGuten? Did you > >> improve support to different image formats? Any critical bugs fixed? > >> * Is erlPress_core used in commercial applications, specifically of high > >> throughput types? > >> * What is the minimum erlang/OTP version required? > >> > >> Best, > >> Theepan > >> > >> > >> > >> > >> > >> > >> On Mon, Sep 24, 2018 at 6:36 AM Lloyd R. Prentice < > lloyd@REDACTED> > >> wrote: > >> > >> > Hi Thepan, > >> > > >> > I?ve undertaken a major revision of ErlGuten called erlPress_core? > >> > announced it on Erlang questions several weeks ago. It?s available > from > >> > GitHub as Writersglen/erlPress_core. I think you?ll find it much more > >> > accessible than the ErlGutens. > >> > > >> > I?ve since been polishing up the code and adding a few new features. > >> After > >> > the launch I learned that camel case is not conventional usage when it > >> > comes to the names of Erlang applications and libraries. So I plan to > >> > release the updated version as erlpress_core. With smooth sailing I > >> hope to > >> > release sometime this week or next. > >> > > >> > All the best, > >> > > >> > Lloyd > >> > > >> > Sent from my iPad > >> > > >> > On Sep 23, 2018, at 8:30 PM, Theepan wrote: > >> > > >> > > >> > Hi Team, > >> > > >> > I see many forks of ErlGuten. Which one is the latest, most stable and > >> > widely used? > >> > > >> > https://github.com/richcarl/erlguten > >> > https://github.com/CarlWright/NGerlguten > >> > https://github.com/hwatkins/erlguten/commits/master > >> > > >> > Best, > >> > Theepan > >> > > >> > > >> > _______________________________________________ > >> > erlang-questions mailing list > >> > erlang-questions@REDACTED > >> > http://erlang.org/mailman/listinfo/erlang-questions > >> > > >> > > >> > >> > >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Tue Sep 25 22:12:22 2018 From: lloyd@REDACTED (lloyd@REDACTED) Date: Tue, 25 Sep 2018 16:12:22 -0400 (EDT) Subject: [erlang-questions] ErlGuten In-Reply-To: References: <7AB3CEDA-34AF-41E2-BEE9-2B27274FF680@writersglen.com> <1537824375.310720011@apps.rackspace.com> <1537894270.37611727@apps.rackspace.com> Message-ID: <1537906342.335620521@apps.rackspace.com> Yeah Dmytro! All the best, Lloyd -----Original Message----- From: "Dmytro Lytovchenko" Sent: Tuesday, September 25, 2018 4:06pm To: lloyd@REDACTED Cc: "Erlang/OTP discussions" Subject: Re: [erlang-questions] ErlGuten I am doing a prettifying pass now: - Adding function specs - Converting text specs to normal -spec specs - Adding named types where they make sense - Replacing tabs with spaces - Running Dialyzer on it, and fixing types so it makes sense - ... that sort of thing. Can't promise when it will be finished, a day or two, or 3 weeks. You'll have a pull request when its done. Will be a MASSIVE changeset but no actual logic affected. On Tue, 25 Sep 2018 at 18:51, wrote: > Hi Theepan, > > Other than display in my demo PDF, I haven't worked much with images. > > The issues you've run across seem serious. Getting at the root of the > problems will likely involve a deep dive into both the PDF reference > manuals and core ErlGuten source code. > > As I noted in my earlier email, ErlGuten source code is very challenging. > It would benefit significantly from re-write, possibly refactoring in some > cases and definitely systematic documentation of functions and parameters. > > I'm not confident that my skills are up to this. My take is that it would > require some amount of community effort to put ErlGuten/erlpress_core on a > solid, maintainable base. I would dearly love to see this happen. > > Is there a CS major or intern out there with time to take on the > challenges? > > Joe showed us the way and I, for one, am deeply grateful. Given that PDF > is a significant standard in the print media world, Erlang deserves to have > a world-class library and suite of tools to generate PDF documents and > decode them back into Erlang structures. > > This will take community effort. > > All the best, > > Lloyd > > > > > > -----Original Message----- > From: "Theepan" > Sent: Monday, September 24, 2018 7:27pm > To: "Lloyd Prentice" > Cc: "Erlang Questions Mailing List" , "Joe > Armstrong" > Subject: Re: [erlang-questions] ErlGuten > > Wanted to add to the issues the reasons -- > > ** When some JPEG image is embedded into the PDF, it turns out dark in the > generated PDF file [Happens due to CYMK color profile.] > ** When some PNG file is embedded, it takes too long (unto 5 minutes) to > generate the PDF file. Debugging shows that the delay is on defilter(Method > , Line1, Line2, Offset, Width, Iter) of eg_pdf_image module.[Happens due to > Alpha channel presence] > > > > On Tue, Sep 25, 2018 at 4:45 AM Theepan wrote: > > > Thanks Lloyd for your detailed explanation on what has been done, and > your > > vision for the library. However we are already using ErlGuten in one of > our > > production systems, and the issues is mainly with the images. > > > > I will definitely keep an eye on the development of erlpress_core in the > > future, as you seem enthusiastic about making it a lively library. > > > > The issues we have are: > > > > ** When some JPEG image is embedded into the PDF, it turns out dark in > the > > generated PDF file > > ** When some PNG file is embedded, it takes too long (unto 5 minutes) to > > generate the PDF file. Debugging shows that the delay is on defilter( > > Method, Line1, Line2, Offset, Width, Iter) of eg_pdf_image module. > > > > Did you come acrsoss this? > > > > Best, > > Theepan > > > > > > > > > > > > On Tue, Sep 25, 2018 at 2:56 AM wrote: > > > >> Hi Theepan, > >> > >> 1. If you look at ErlGuten source, you'll see that function > documentation > >> is fairly minimal and many parameters have single character names with > >> little to no documentation. This makes maintenance and revision very > >> difficult. I consider the core ErlGuten modules diamonds-in-the-rough. > >> > >> I also found the high-level page make-up functions provided by Hugh and > >> Carl limited with respect to professional page make-up and, to me at > least, > >> a bit confusing. > >> > >> But erlpress_core owes a deep debt and much gratitude to the ErlGuten > >> lineage. > >> > >> erlpress_core is based on Joe's font-handling, justification, and XML > >> parsing code with few if any changes. But it provides map structures to > >> represent nearly all of the PDF objects represented in eg_pdf_op.erl. > These > >> map structures incorporate many default parameters so generating PDF > >> documents is syntactically simple and consistent without sacrificing > >> expressive flexibility. You can easily customize display by > instantiating > >> map parameters with your own values. > >> > >> You can see most of the PDF objects currently supported by > erlpress_core, > >> including boxed text, justification options, and various line and text > >> objects, demonstrated in ep_show_n_tell.pdf and the source in > >> ep_show_n_tell.erl. > >> > >> If you look at the map definitions of the various PDF objects, you'll > see > >> how to customize display. > >> > >> At some point if would be good to re-write the base Erlguten modules > with > >> more explicit function and parameter documentation, but I'm not up to > that > >> at this point. But, anyone looking for a challenge is free to jump in. > >> > >> I'm currently working on tables and text flow across pages (think > reports > >> and book chapters). Hope to deliver these features plus much code > polishing > >> in the next release coming "real soon now." > >> > >> High on my wish/to-do list are: > >> > >> -- Easy-to_use page-grid design functions > >> -- Imposition (printing multiple page impressions on a single sheet of > >> paper stock) > >> -- Articles and beads (think text jumping across columns and pages) > >> -- Easy-to-use and expressive page make-up functions > >> -- Example templates for various print formats > >> -- Markdown input > >> -- Support of TTF and OTF fonts > >> > >> Joe Armstrong expressed the following goal when he first announced > >> ErlGuten: > >> > >> "Better than TeX." > >> > >> That's a tall order, yet to be realized. My hope is that erlpress will > >> move the ball further down the field. > >> > >> 2. I haven't done anything with image formats beyond what you'll find in > >> eg_pdf_image.erl. I'd welcome work in that direction. > >> > >> 3. erlpress_core is still at Version 0.01. It's just out. I'm working > >> toward sufficient functionality and stability to support a web > application > >> that we have currently under development. > >> > >> 4. I developed erlpress_core on Erlang/OTP 19. It does require support > >> for maps. > >> > >> I see the erlpress_core library as a valuable library for embedding PDF > >> generation into Erlang applications and as the basis for many exciting > >> Erlang-based print production tools. > >> > >> So, Treepan, I appreciate your interest and would welcome your > >> involvement in testing and feature development. > >> > >> I would jump for joy if some Erlang guru were to step forward take on > the > >> TTF/OTF support issue. I've done some research and have a few ideas on > how > >> expanded font support might be accomplished, but have too little time > over > >> the next several months to dig into it. > >> > >> Incidentally, in my previous post my iPad decided it was smarter than me > >> and erroneously corrected my GitHUb address.it should be > >> writersglen/erlPress_core. > >> > >> My intention is to bag the camel case in my next release with luck in a > >> week or two so it should look like writersglen/erlpress_core. > >> > >> All the best, > >> > >> Lloyd > >> > >> > >> -----Original Message----- > >> From: "Theepan" > >> Sent: Monday, September 24, 2018 3:59pm > >> To: lloyd@REDACTED > >> Cc: "Erlang Questions Mailing List" , "Joe > >> Armstrong" > >> Subject: Re: [erlang-questions] ErlGuten > >> > >> Hi Lloyd, > >> > >> Thank you for your response. I have some quick clarifications -- > >> > >> * What are the major improvements made on the source ErlGuten? Did you > >> improve support to different image formats? Any critical bugs fixed? > >> * Is erlPress_core used in commercial applications, specifically of high > >> throughput types? > >> * What is the minimum erlang/OTP version required? > >> > >> Best, > >> Theepan > >> > >> > >> > >> > >> > >> > >> On Mon, Sep 24, 2018 at 6:36 AM Lloyd R. Prentice < > lloyd@REDACTED> > >> wrote: > >> > >> > Hi Thepan, > >> > > >> > I?ve undertaken a major revision of ErlGuten called erlPress_core? > >> > announced it on Erlang questions several weeks ago. It?s available > from > >> > GitHub as Writersglen/erlPress_core. I think you?ll find it much more > >> > accessible than the ErlGutens. > >> > > >> > I?ve since been polishing up the code and adding a few new features. > >> After > >> > the launch I learned that camel case is not conventional usage when it > >> > comes to the names of Erlang applications and libraries. So I plan to > >> > release the updated version as erlpress_core. With smooth sailing I > >> hope to > >> > release sometime this week or next. > >> > > >> > All the best, > >> > > >> > Lloyd > >> > > >> > Sent from my iPad > >> > > >> > On Sep 23, 2018, at 8:30 PM, Theepan wrote: > >> > > >> > > >> > Hi Team, > >> > > >> > I see many forks of ErlGuten. Which one is the latest, most stable and > >> > widely used? > >> > > >> > https://github.com/richcarl/erlguten > >> > https://github.com/CarlWright/NGerlguten > >> > https://github.com/hwatkins/erlguten/commits/master > >> > > >> > Best, > >> > Theepan > >> > > >> > > >> > _______________________________________________ > >> > erlang-questions mailing list > >> > erlang-questions@REDACTED > >> > http://erlang.org/mailman/listinfo/erlang-questions > >> > > >> > > >> > >> > >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From frank.muller.erl@REDACTED Tue Sep 25 22:15:15 2018 From: frank.muller.erl@REDACTED (Frank Muller) Date: Tue, 25 Sep 2018 22:15:15 +0200 Subject: [erlang-questions] Patch package OTP 21.1 released In-Reply-To: <1537882179.4914.34.camel@ericsson.com> References: <1537882179.4914.34.camel@ericsson.com> Message-ID: On Ubuntu 16.04 LTS: $ kerl build 21.1 21.1 [...] ./configure -with-dynamic-trace=systemtap --enable-m64-build --enable-dirty-schedulers --enable-lock-check [...] checking for compiler flags for loadable drivers... -m64 -O2 -I/usr/local/include -I/home/frank/.kerl/builds/21.1/otp_src_21.1/erts/x86_64-unknown-linux-gnu -fno-tree-copyrename -D_GNU_SOURCE -fPIC checking for linker for loadable drivers... gcc checking for linker flags for loadable drivers... -m64 -shared -Wl,-Bsymbolic checking for 'runtime library path' linker flag... -Wl,-R checking for a compiler that handles jumptables... gcc checking whether the code model is small... yes checking for dtrace... no configure: error: No dtrace utility found. configure: error: /home/frank/.kerl/builds/21.1/otp_src_21.1/erts/configure failed for erts Why it is looking for Dtrace on Linux (i?ve specified systemtap)? Didn?t face this issue with 21.0 on the same machine. /Frank Erlang/OTP 21.1 is the first service release for the 21 major release > with new features, improvements as well as a few features! > > Highlights > SSH: > The cipher 'chacha20-poly1305@REDACTED' is now supported if > OpenSSL 1.1.1 or higher is used as cryptolib. > The key exchange methods 'curve25519-sha256@REDACTED', 'curve25519- > sha256' and 'curve448-sha512' are implemented. > They all depends on that OpenSSL 1.1.1 or higher used as cryptolib. > Crypto: > Typing in crypto and public_key applications are reworked and > corrected. > New chapter on Algorithm details in the User's guide > ERTS, Kernel: > The socket options recvtos, recvttl, recvtclass and pktoptions have > been implemented in the socket modules. > Se documentation in gen_tcp, gen_udp and inet modules for more info! > > > For more details see: > http://erlang.org/download/otp_src_21.1.readme > > Pre built versions for Windows can be fetched here: > http://erlang.org/download/otp_win32_21.1.exe > http://erlang.org/download/otp_win64_21.1.exe > > Online documentation can be browsed here: > http://erlang.org/documentation/doc-10.1/doc > > The Erlang/OTP source can also be found at GitHub on the official > Erlang repository, Here: OTP-21.1 > > > Please report any new issues via Erlang/OTPs public issue tracker > https://bugs.erlang.org > System Dashboard - Erlang Programming Language bugs.erlang.org > Welcome to Erlang Programming Language. What is bugs.erlang.org? Here > you report bugs in Erlang/OTP, suggestions for new features and > improvements. > We want to thank all of those who sent us patches, suggestions and bug > reports! > > Thank you! > > The Erlang/OTP Team at Ericsson > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Tue Sep 25 23:56:01 2018 From: lloyd@REDACTED (lloyd@REDACTED) Date: Tue, 25 Sep 2018 17:56:01 -0400 (EDT) Subject: [erlang-questions] How to extract string between XML tags Message-ID: <1537912561.49523738@apps.rackspace.com> Hello, By now I should know how to do this. But I've fumbled for more time than I have to find an elegant solution. Can anyone show a better way? Example string: "Firstname" % NOTE: could be any valid tag My kludge: extract_text(TaggedText) -> Split = re:split(TaggedText, "<"), Split2 = lists:nth(2, Split), Split3 = binary_to_list(Split2), Split4 = re:split(Split3, ">"), Split5 = lists:nth(2, Split4), binary_to_list(Split5). Surely there's a better way. Many thanks, LRP ********************************************* My books: THE GOSPEL OF ASHES http://thegospelofashes.com Strength is not enough. Do they have the courage and the cunning? Can they survive long enough to save the lives of millions? FREEIN' PANCHO http://freeinpancho.com A community of misfits help a troubled boy find his way AYA TAKEO http://ayatakeo.com Star-crossed love, war and power in an alternative universe Available through Amazon or by request from your favorite bookstore ********************************************** From hugo@REDACTED Wed Sep 26 00:06:30 2018 From: hugo@REDACTED (Hugo Mills) Date: Tue, 25 Sep 2018 22:06:30 +0000 Subject: [erlang-questions] How to extract string between XML tags In-Reply-To: <1537912561.49523738@apps.rackspace.com> References: <1537912561.49523738@apps.rackspace.com> Message-ID: <20180925220630.GD5429@carfax.org.uk> On Tue, Sep 25, 2018 at 05:56:01PM -0400, lloyd@REDACTED wrote: > Hello, > > By now I should know how to do this. But I've fumbled for more time than I have to find an elegant solution. > > Can anyone show a better way? > > Example string: "Firstname" % NOTE: could be any valid tag > > My kludge: > > extract_text(TaggedText) -> > Split = re:split(TaggedText, "<"), > Split2 = lists:nth(2, Split), > Split3 = binary_to_list(Split2), > Split4 = re:split(Split3, ">"), > Split5 = lists:nth(2, Split4), > binary_to_list(Split5). > > Surely there's a better way. XML isn't a regular language, so (in the general case) you can't(*) use regexes and simple string splitting to parse XML correctly. If you've got a very constrained input, where you know that it's going to conform to specific patterns that you can match on, then you might get away with it, but if that's not the case, you're barking up the wrong tree with any kind of regex. The solution is to swallow the pain and use a proper XML library. I've had good results with erlsom in my own projects, but there's several other erlang XML libs out there, with various benefits and issues. I'm sure others will weigh in with their experiences with those. Hugo. (*) By "can't", I don't mean "it's just too painful". I mean "it's provably not possible to do it right in all cases". -- Hugo Mills | "There's a Martian war machine outside -- they want hugo@REDACTED carfax.org.uk | to talk to you about a cure for the common cold." http://carfax.org.uk/ | PGP: E2AB1DE4 | Stephen Franklin, Babylon 5 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From mononcqc@REDACTED Wed Sep 26 00:20:31 2018 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 25 Sep 2018 18:20:31 -0400 Subject: [erlang-questions] How to extract string between XML tags In-Reply-To: <1537912561.49523738@apps.rackspace.com> References: <1537912561.49523738@apps.rackspace.com> Message-ID: <20180925222030.GC13554@ferdmbp.local> On 09/25, lloyd@REDACTED wrote: >Hello, > >By now I should know how to do this. But I've fumbled for more time than I have to find an elegant solution. > >Can anyone show a better way? > >Example string: "Firstname" % NOTE: could be any valid tag > >My kludge: > >extract_text(TaggedText) -> > Split = re:split(TaggedText, "<"), > Split2 = lists:nth(2, Split), > Split3 = binary_to_list(Split2), > Split4 = re:split(Split3, ">"), > Split5 = lists:nth(2, Split4), > binary_to_list(Split5). > >Surely there's a better way. > The classic answer: https://stackoverflow.com/a/1732454/35344 The nice non-ridiculous one: You will want to use an XML parser to parse XML. Regular expressions are usually not the proper structure. Let's take this as an example: 1> Str = "aaaabbbbbcccccdddcccbbbbbbb". "aaaabbbbbcccccdddcccbbbbbbb" 2> rr(xmerl). [xmerl_event,xmerl_fun_states,xmerl_scanner,xmlAttribute, xmlComment,xmlContext,xmlDecl,xmlDocument,xmlElement, xmlNamespace,xmlNode,xmlNsNode,xmlObj,xmlPI,xmlText] 3> {XML, _} = xmerl_scan:string(Str). {#xmlElement{ name = a,expanded_name = a,nsinfo = [], namespace = #xmlNamespace{default = [],nodes = []}, parents = [],pos = 1,attributes = [], content = [#xmlText{ parents = [{a,1}], pos = 1,language = [],value = "aaaa",type = text}, #xmlElement{ name = b,expanded_name = b,nsinfo = [], namespace = #xmlNamespace{default = [],nodes = []}, parents = [{a,1}], pos = 2,attributes = [], content = [#xmlText{ parents = [{b,2},{a,1}], pos = 1,language = [],value = "bbbbb",type = text}, #xmlElement{ name = c,expanded_name = c,nsinfo = [], namespace = #xmlNamespace{...}, parents = [...],...}, #xmlText{ parents = [{b,2},{a,...}], pos = 3,language = [], value = [...],...}], language = [],xmlbase = "/Users/ferd", elementdef = undeclared}, #xmlText{ parents = [{a,1}], pos = 3,language = [],value = "bbbb",type = text}], language = [],xmlbase = "/Users/ferd", elementdef = undeclared}, []} This gives you a parsed XML document. You can use xpath to access nodes, if you want. XPath defines a syntax to query the insides of XML documents as strings: https://en.wikipedia.org/wiki/XPath For example, the /a/b/c string would mean 'within the root document /, find the node a, and then go find node b in there, and go find node c'. This fives something like this: 8> xmerl_xpath:string("/a/b/c", XML). [#xmlElement{ name = c,expanded_name = c,nsinfo = [], namespace = #xmlNamespace{default = [],nodes = []}, parents = [{b,2},{a,1}], pos = 2,attributes = [], content = [#xmlText{ parents = [{c,2},{b,2},{a,1}], pos = 1,language = [],value = "ccccc",type = text}, #xmlElement{ name = d,expanded_name = d,nsinfo = [], namespace = #xmlNamespace{default = [],nodes = []}, parents = [{c,2},{b,2},{a,1}], pos = 2,attributes = [], content = [#xmlText{ parents = [{d,2},{c,2},{b,2},{a,...}], pos = 1,language = [],value = "ddd",type = text}], language = [],xmlbase = "/Users/ferd", elementdef = undeclared}, #xmlText{ parents = [{c,2},{b,2},{a,1}], pos = 3,language = [],value = "ccc",type = text}], language = [],xmlbase = "/Users/ferd", elementdef = undeclared}] You can see that the XML node has 3 entries in it: a text node (#xmlText with a value "cccc", #xmlElement which has the name 'd' (so the tag), and another text node. You can then go and dig within `` by adding to the xpath: 9> xmerl_xpath:string("/a/b/c/d", XML). [#xmlElement{name = d,expanded_name = d,nsinfo = [], namespace = #xmlNamespace{default = [],nodes = []}, parents = [{c,2},{b,2},{a,1}], pos = 2,attributes = [], content = [#xmlText{parents = [{d,2},{c,2},{b,2},{a,1}], pos = 1,language = [],value = "ddd",type = text}], language = [],xmlbase = "/Users/ferd", elementdef = undeclared}] And the sole node contained there is the one with the content that is #xmlText.content = "ddd". If you want to extract the text, you can use the `text()` xpath qualifier: 18> xmerl_xpath:string("/a/b/c/text()", XML). [#xmlText{parents = [{c,2},{b,2},{a,1}], pos = 1,language = [],value = "ccccc",type = text}, #xmlText{parents = [{c,2},{b,2},{a,1}], pos = 3,language = [],value = "ccc",type = text}] 19> xmerl_xpath:string("/a/b/c/d/text()", XML). [#xmlText{parents = [{d,2},{c,2},{b,2},{a,1}], pos = 1,language = [],value = "ddd",type = text}] The xmerl structure is kind of cumbersome, but when you have to handle more complex documents, a real parser with niceties like xpath can do wonders to handle documents as a logical structure rather than as a group of tokens to wrangle. Regards, Fred. From lloyd@REDACTED Wed Sep 26 00:57:30 2018 From: lloyd@REDACTED (lloyd@REDACTED) Date: Tue, 25 Sep 2018 18:57:30 -0400 (EDT) Subject: [erlang-questions] How to extract string between XML tags In-Reply-To: <20180925222030.GC13554@ferdmbp.local> References: <1537912561.49523738@apps.rackspace.com> <20180925222030.GC13554@ferdmbp.local> Message-ID: <1537916250.014723660@apps.rackspace.com> Thanks all, This is definitely useful info for another aspect of my project. But perhaps there's a simpler solution to my immediate problem if I pose my question more specifically. Goal: Add tables to erlpress_core. Problem: Extract cell info from html tables: table() -> "
Firstname Lastname Age
Jill Smith 50
Eve Jackson 94
". My function ep_parse_table/1 gives me: [[["Firstname"], ["Lastname"], ["Age"]], [["Jill"],["Smith"],["50"]], [["Eve"],["Jackson"],["94"]]] Is it reasonable to require xmerl as a dependency of erlpress_core and go through the many transformations required to extract cell data for every cell in a, perhaps, large table? How likely is that a user will include full-fledged XML data in table cells? If so then maybe we need to suck up the pain. Or, maybe we just specify that XML data is not permitted in tables submitted to erlpress_core. Any thoughts? All the best, L. -----Original Message----- From: "Fred Hebert" Sent: Tuesday, September 25, 2018 6:20pm To: lloyd@REDACTED Cc: "Erlang/OTP discussions" Subject: Re: [erlang-questions] How to extract string between XML tags On 09/25, lloyd@REDACTED wrote: >Hello, > >By now I should know how to do this. But I've fumbled for more time than I have to find an elegant solution. > >Can anyone show a better way? > >Example string: "Firstname" % NOTE: could be any valid tag > >My kludge: > >extract_text(TaggedText) -> > Split = re:split(TaggedText, "<"), > Split2 = lists:nth(2, Split), > Split3 = binary_to_list(Split2), > Split4 = re:split(Split3, ">"), > Split5 = lists:nth(2, Split4), > binary_to_list(Split5). > >Surely there's a better way. > The classic answer: https://stackoverflow.com/a/1732454/35344 The nice non-ridiculous one: You will want to use an XML parser to parse XML. Regular expressions are usually not the proper structure. Let's take this as an example: 1> Str = "aaaabbbbbcccccdddcccbbbbbbb". "aaaabbbbbcccccdddcccbbbbbbb" 2> rr(xmerl). [xmerl_event,xmerl_fun_states,xmerl_scanner,xmlAttribute, xmlComment,xmlContext,xmlDecl,xmlDocument,xmlElement, xmlNamespace,xmlNode,xmlNsNode,xmlObj,xmlPI,xmlText] 3> {XML, _} = xmerl_scan:string(Str). {#xmlElement{ name = a,expanded_name = a,nsinfo = [], namespace = #xmlNamespace{default = [],nodes = []}, parents = [],pos = 1,attributes = [], content = [#xmlText{ parents = [{a,1}], pos = 1,language = [],value = "aaaa",type = text}, #xmlElement{ name = b,expanded_name = b,nsinfo = [], namespace = #xmlNamespace{default = [],nodes = []}, parents = [{a,1}], pos = 2,attributes = [], content = [#xmlText{ parents = [{b,2},{a,1}], pos = 1,language = [],value = "bbbbb",type = text}, #xmlElement{ name = c,expanded_name = c,nsinfo = [], namespace = #xmlNamespace{...}, parents = [...],...}, #xmlText{ parents = [{b,2},{a,...}], pos = 3,language = [], value = [...],...}], language = [],xmlbase = "/Users/ferd", elementdef = undeclared}, #xmlText{ parents = [{a,1}], pos = 3,language = [],value = "bbbb",type = text}], language = [],xmlbase = "/Users/ferd", elementdef = undeclared}, []} This gives you a parsed XML document. You can use xpath to access nodes, if you want. XPath defines a syntax to query the insides of XML documents as strings: https://en.wikipedia.org/wiki/XPath For example, the /a/b/c string would mean 'within the root document /, find the node a, and then go find node b in there, and go find node c'. This fives something like this: 8> xmerl_xpath:string("/a/b/c", XML). [#xmlElement{ name = c,expanded_name = c,nsinfo = [], namespace = #xmlNamespace{default = [],nodes = []}, parents = [{b,2},{a,1}], pos = 2,attributes = [], content = [#xmlText{ parents = [{c,2},{b,2},{a,1}], pos = 1,language = [],value = "ccccc",type = text}, #xmlElement{ name = d,expanded_name = d,nsinfo = [], namespace = #xmlNamespace{default = [],nodes = []}, parents = [{c,2},{b,2},{a,1}], pos = 2,attributes = [], content = [#xmlText{ parents = [{d,2},{c,2},{b,2},{a,...}], pos = 1,language = [],value = "ddd",type = text}], language = [],xmlbase = "/Users/ferd", elementdef = undeclared}, #xmlText{ parents = [{c,2},{b,2},{a,1}], pos = 3,language = [],value = "ccc",type = text}], language = [],xmlbase = "/Users/ferd", elementdef = undeclared}] You can see that the XML node has 3 entries in it: a text node (#xmlText with a value "cccc", #xmlElement which has the name 'd' (so the tag), and another text node. You can then go and dig within `` by adding to the xpath: 9> xmerl_xpath:string("/a/b/c/d", XML). [#xmlElement{name = d,expanded_name = d,nsinfo = [], namespace = #xmlNamespace{default = [],nodes = []}, parents = [{c,2},{b,2},{a,1}], pos = 2,attributes = [], content = [#xmlText{parents = [{d,2},{c,2},{b,2},{a,1}], pos = 1,language = [],value = "ddd",type = text}], language = [],xmlbase = "/Users/ferd", elementdef = undeclared}] And the sole node contained there is the one with the content that is #xmlText.content = "ddd". If you want to extract the text, you can use the `text()` xpath qualifier: 18> xmerl_xpath:string("/a/b/c/text()", XML). [#xmlText{parents = [{c,2},{b,2},{a,1}], pos = 1,language = [],value = "ccccc",type = text}, #xmlText{parents = [{c,2},{b,2},{a,1}], pos = 3,language = [],value = "ccc",type = text}] 19> xmerl_xpath:string("/a/b/c/d/text()", XML). [#xmlText{parents = [{d,2},{c,2},{b,2},{a,1}], pos = 1,language = [],value = "ddd",type = text}] The xmerl structure is kind of cumbersome, but when you have to handle more complex documents, a real parser with niceties like xpath can do wonders to handle documents as a logical structure rather than as a group of tokens to wrangle. Regards, Fred. From mononcqc@REDACTED Wed Sep 26 01:38:33 2018 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 25 Sep 2018 19:38:33 -0400 Subject: [erlang-questions] How to extract string between XML tags In-Reply-To: <1537916250.014723660@apps.rackspace.com> References: <1537912561.49523738@apps.rackspace.com> <20180925222030.GC13554@ferdmbp.local> <1537916250.014723660@apps.rackspace.com> Message-ID: <20180925233832.GD13554@ferdmbp.local> On 09/25, lloyd@REDACTED wrote: >Thanks all, > >This is definitely useful info for another aspect of my project. > >But perhaps there's a simpler solution to my immediate problem if I pose my question more specifically. > >Goal: Add tables to erlpress_core. > >Problem: Extract cell info from html tables: > >table() -> > ... > >My function ep_parse_table/1 gives me: > >[[["Firstname"], > ["Lastname"], > ["Age"]], > [["Jill"],["Smith"],["50"]], > [["Eve"],["Jackson"],["94"]]] > {XML, _} = xmerl_parse:string(table()), Rows = xmerl_xpath:string("/table/tr", XML), >From there: [[[ Text || #xmlText{value=Text} <- Col] || #xmlElement{content=Col} <- Cols] || #xmlElement{content=Cols} <- Rows]. Gives: [[["Firstname"],["Lastname"],["Age"]], [["Jill"],["Smith"],["50"]], [["Eve"],["Jackson"],["94"]]] If you want to keep the node's type: [[[ {Name,Text} || #xmlText{value=Text} <- Col] || #xmlElement{content=Col, name=Name} <- Cols] || #xmlElement{content=Cols} <- Rows]. Gives: [[[{th,"Firstname"}],[{th,"Lastname"}],[{th,"Age"}]], [[{td,"Jill"}],[{td,"Smith"}],[{td,"50"}]], [[{td,"Eve"}],[{td,"Jackson"}],[{td,"94"}]]] This is a bit obtuse due to using list comprehensions, I haven't taken the time to clean the code up. >Is it reasonable to require xmerl as a dependency of erlpress_core and >go through the many transformations required to extract cell data for >every cell in a, perhaps, large table? > It's not that bad, considering xmerl is part of the standard library, but that's a fair concern anyway. >How likely is that a user will include full-fledged XML data in table cells? > The sad thing is that XML is _simpler_ to parse than HTML and all its variants (because they are less strict, they allow for more stuff to happen). The question though is what is the syntax you aim to support? Should people be able to style text using tags like , , , , and so on? Or do you expect literal text always? What you accept or refuse defines what you can deal with. >If so then maybe we need to suck up the pain. > >Or, maybe we just specify that XML data is not permitted in tables >submitted to erlpress_core. > >Any thoughts? > You could say XML data is not supported. That does not prevent you from using the XML parser rather than writing your own. For example, what does someone do when they want to use the '' string from within the table to avoid breaking your own parser? What's the escape sequence? Using XML as a parser, you get it for free: > is > and < is <: 72> xmerl_scan:string(" bf<td>aaa"). {#xmlElement{... content = [#xmlText{value = " bfaaa" ...}], ...}, []} You can see the resulting string being " bfaaa" despite already being in a element. No confusion to be had. If you don't use the parser, you have to come up with these rules yourself, and implement them properly. That's a lot of work :) From lloyd@REDACTED Wed Sep 26 05:17:51 2018 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Tue, 25 Sep 2018 23:17:51 -0400 Subject: [erlang-questions] How to extract string between XML tags In-Reply-To: <20180925233832.GD13554@ferdmbp.local> References: <1537912561.49523738@apps.rackspace.com> <20180925222030.GC13554@ferdmbp.local> <1537916250.014723660@apps.rackspace.com> <20180925233832.GD13554@ferdmbp.local> Message-ID: Hi Fred, This is gorgeous! in it will go. Thank you so much. All the best, Lloyd Sent from my iPad > On Sep 25, 2018, at 7:38 PM, Fred Hebert wrote: > >> On 09/25, lloyd@REDACTED wrote: >> Thanks all, >> >> This is definitely useful info for another aspect of my project. >> >> But perhaps there's a simpler solution to my immediate problem if I pose my question more specifically. >> >> Goal: Add tables to erlpress_core. >> >> Problem: Extract cell info from html tables: >> >> table() -> >> ... >> >> My function ep_parse_table/1 gives me: >> >> [[["Firstname"], >> ["Lastname"], >> ["Age"]], >> [["Jill"],["Smith"],["50"]], >> [["Eve"],["Jackson"],["94"]]] >> > > {XML, _} = xmerl_parse:string(table()), > Rows = xmerl_xpath:string("/table/tr", XML), > > > From there: > > [[[ Text || #xmlText{value=Text} <- Col] > || #xmlElement{content=Col} <- Cols] > || #xmlElement{content=Cols} <- Rows]. > > Gives: > > [[["Firstname"],["Lastname"],["Age"]], > [["Jill"],["Smith"],["50"]], > [["Eve"],["Jackson"],["94"]]] > > If you want to keep the node's type: > > [[[ {Name,Text} || #xmlText{value=Text} <- Col] > || #xmlElement{content=Col, name=Name} <- Cols] > || #xmlElement{content=Cols} <- Rows]. > > Gives: > > [[[{th,"Firstname"}],[{th,"Lastname"}],[{th,"Age"}]], > [[{td,"Jill"}],[{td,"Smith"}],[{td,"50"}]], > [[{td,"Eve"}],[{td,"Jackson"}],[{td,"94"}]]] > > This is a bit obtuse due to using list comprehensions, I haven't taken the time to clean the code up. > >> Is it reasonable to require xmerl as a dependency of erlpress_core and go through the many transformations required to extract cell data for every cell in a, perhaps, large table? >> > > It's not that bad, considering xmerl is part of the standard library, but that's a fair concern anyway. > >> How likely is that a user will include full-fledged XML data in table cells? >> > > The sad thing is that XML is _simpler_ to parse than HTML and all its variants (because they are less strict, they allow for more stuff to happen). > > The question though is what is the syntax you aim to support? Should people be able to style text using tags like , , , , and so on? Or do you expect literal text always? What you accept or refuse defines what you can deal with. > >> If so then maybe we need to suck up the pain. >> >> Or, maybe we just specify that XML data is not permitted in tables submitted to erlpress_core. >> >> Any thoughts? >> > > You could say XML data is not supported. That does not prevent you from using the XML parser rather than writing your own. > > For example, what does someone do when they want to use the '' string from within the table to avoid breaking your own parser? What's the escape sequence? Using XML as a parser, you get it for free: > is > and < is <: > > 72> xmerl_scan:string(" bf<td>aaa"). > {#xmlElement{... > content = [#xmlText{value = " bfaaa" ...}], > ...}, > []} > > You can see the resulting string being " bfaaa" despite already being in a element. No confusion to be had. > > If you don't use the parser, you have to come up with these rules yourself, and implement them properly. That's a lot of work :) > > From essen@REDACTED Wed Sep 26 09:04:21 2018 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Wed, 26 Sep 2018 09:04:21 +0200 Subject: [erlang-questions] ErlGuten + a short survey In-Reply-To: <1537904554.681631612@apps.rackspace.com> References: <7AB3CEDA-34AF-41E2-BEE9-2B27274FF680@writersglen.com> <1537824375.310720011@apps.rackspace.com> <1537894270.37611727@apps.rackspace.com> <1537904554.681631612@apps.rackspace.com> Message-ID: <69bc7e83-936b-1f03-0b78-759d5a184349@ninenines.eu> As you know I will at some point[0] want to make Asciidoc produce PDF files so I may end up wanting to use it. However from what I've seen so far I'm not sure it would be very helpful. My main concern is that from the Asciidoc I get an AST that has little pagination information attached. I would therefore need to go from that Asciidoc-AST to a sort of PDF-AST which comes with pagination information and can then be used to generate the PDF using the PDF drawing primitives. I have not seen code to automatically paginate in erlguten/erlpress_core but maybe I missed it. Then there's all the concerns about table of contents, figure listings and whatnot. Writing the PDF itself could be done by erlguten/erlpress_core, but it could also just be building an iolist on the fly like I do for man pages[1]. This would be similar to what projects like PHP's fpdf library are doing. Writing the PDF is not a big concern either way since Asciidoc has limited capabilities compared to PDF. In other words, as far as I can tell so far, erlguten seems to solve a different problem than the one I am mainly concerned with. But I've only done a cursory examination of erlguten/erlpress_core so it's possible that I missed something. [0] This or next Christmas. Probably. [1] https://git.ninenines.eu/asciideck.git/tree/src/asciideck_to_manpage.erl On 09/25/2018 09:42 PM, lloyd@REDACTED wrote: > Hi folks, > > As Theepan says, "PDF generation library is crucial to most of the commercial > applications." > > If you are using ErlGuten, please help inform my work on erlpress_core: > > 1. What are you using it for? > 2. What features do you wish it has but doesn't? > > Many thanks, > > Lloyd > > -----Original Message----- > From: "Theepan" > Sent: Tuesday, September 25, 2018 2:55pm > To: "Lloyd Prentice" > Cc: "Erlang Questions Mailing List" , "Joe Armstrong" > Subject: Re: [erlang-questions] ErlGuten > > Hi Lloyd, > > You are right. PDF generation library is crucial to most of the commercial > applications. The code base is not very complicated, but then again we need > somebody who has time and willingness to take on the work. Please find what > have written to Sean on what improvements can be crucial short term. > > Wish work arounds and short fixes we could achieve the required scemantics, > performance and throughput out of ErlGuten. > > Best, > Theepan > > > > On Tue, Sep 25, 2018 at 10:21 PM wrote: > >> Hi Theepan, >> >> Other than display in my demo PDF, I haven't worked much with images. >> >> The issues you've run across seem serious. Getting at the root of the >> problems will likely involve a deep dive into both the PDF reference >> manuals and core ErlGuten source code. >> >> As I noted in my earlier email, ErlGuten source code is very challenging. >> It would benefit significantly from re-write, possibly refactoring in some >> cases and definitely systematic documentation of functions and parameters. >> >> I'm not confident that my skills are up to this. My take is that it would >> require some amount of community effort to put ErlGuten/erlpress_core on a >> solid, maintainable base. I would dearly love to see this happen. >> >> Is there a CS major or intern out there with time to take on the >> challenges? >> >> Joe showed us the way and I, for one, am deeply grateful. Given that PDF >> is a significant standard in the print media world, Erlang deserves to have >> a world-class library and suite of tools to generate PDF documents and >> decode them back into Erlang structures. >> >> This will take community effort. >> >> All the best, >> >> Lloyd >> >> >> >> >> >> -----Original Message----- >> From: "Theepan" >> Sent: Monday, September 24, 2018 7:27pm >> To: "Lloyd Prentice" >> Cc: "Erlang Questions Mailing List" , "Joe >> Armstrong" >> Subject: Re: [erlang-questions] ErlGuten >> >> Wanted to add to the issues the reasons -- >> >> ** When some JPEG image is embedded into the PDF, it turns out dark in the >> generated PDF file [Happens due to CYMK color profile.] >> ** When some PNG file is embedded, it takes too long (unto 5 minutes) to >> generate the PDF file. Debugging shows that the delay is on defilter(Method >> , Line1, Line2, Offset, Width, Iter) of eg_pdf_image module.[Happens due to >> Alpha channel presence] >> >> >> >> On Tue, Sep 25, 2018 at 4:45 AM Theepan wrote: >> >>> Thanks Lloyd for your detailed explanation on what has been done, and >> your >>> vision for the library. However we are already using ErlGuten in one of >> our >>> production systems, and the issues is mainly with the images. >>> >>> I will definitely keep an eye on the development of erlpress_core in the >>> future, as you seem enthusiastic about making it a lively library. >>> >>> The issues we have are: >>> >>> ** When some JPEG image is embedded into the PDF, it turns out dark in >> the >>> generated PDF file >>> ** When some PNG file is embedded, it takes too long (unto 5 minutes) to >>> generate the PDF file. Debugging shows that the delay is on defilter( >>> Method, Line1, Line2, Offset, Width, Iter) of eg_pdf_image module. >>> >>> Did you come acrsoss this? >>> >>> Best, >>> Theepan >>> >>> >>> >>> >>> >>> On Tue, Sep 25, 2018 at 2:56 AM wrote: >>> >>>> Hi Theepan, >>>> >>>> 1. If you look at ErlGuten source, you'll see that function >> documentation >>>> is fairly minimal and many parameters have single character names with >>>> little to no documentation. This makes maintenance and revision very >>>> difficult. I consider the core ErlGuten modules diamonds-in-the-rough. >>>> >>>> I also found the high-level page make-up functions provided by Hugh and >>>> Carl limited with respect to professional page make-up and, to me at >> least, >>>> a bit confusing. >>>> >>>> But erlpress_core owes a deep debt and much gratitude to the ErlGuten >>>> lineage. >>>> >>>> erlpress_core is based on Joe's font-handling, justification, and XML >>>> parsing code with few if any changes. But it provides map structures to >>>> represent nearly all of the PDF objects represented in eg_pdf_op.erl. >> These >>>> map structures incorporate many default parameters so generating PDF >>>> documents is syntactically simple and consistent without sacrificing >>>> expressive flexibility. You can easily customize display by >> instantiating >>>> map parameters with your own values. >>>> >>>> You can see most of the PDF objects currently supported by >> erlpress_core, >>>> including boxed text, justification options, and various line and text >>>> objects, demonstrated in ep_show_n_tell.pdf and the source in >>>> ep_show_n_tell.erl. >>>> >>>> If you look at the map definitions of the various PDF objects, you'll >> see >>>> how to customize display. >>>> >>>> At some point if would be good to re-write the base Erlguten modules >> with >>>> more explicit function and parameter documentation, but I'm not up to >> that >>>> at this point. But, anyone looking for a challenge is free to jump in. >>>> >>>> I'm currently working on tables and text flow across pages (think >> reports >>>> and book chapters). Hope to deliver these features plus much code >> polishing >>>> in the next release coming "real soon now." >>>> >>>> High on my wish/to-do list are: >>>> >>>> -- Easy-to_use page-grid design functions >>>> -- Imposition (printing multiple page impressions on a single sheet of >>>> paper stock) >>>> -- Articles and beads (think text jumping across columns and pages) >>>> -- Easy-to-use and expressive page make-up functions >>>> -- Example templates for various print formats >>>> -- Markdown input >>>> -- Support of TTF and OTF fonts >>>> >>>> Joe Armstrong expressed the following goal when he first announced >>>> ErlGuten: >>>> >>>> "Better than TeX." >>>> >>>> That's a tall order, yet to be realized. My hope is that erlpress will >>>> move the ball further down the field. >>>> >>>> 2. I haven't done anything with image formats beyond what you'll find in >>>> eg_pdf_image.erl. I'd welcome work in that direction. >>>> >>>> 3. erlpress_core is still at Version 0.01. It's just out. I'm working >>>> toward sufficient functionality and stability to support a web >> application >>>> that we have currently under development. >>>> >>>> 4. I developed erlpress_core on Erlang/OTP 19. It does require support >>>> for maps. >>>> >>>> I see the erlpress_core library as a valuable library for embedding PDF >>>> generation into Erlang applications and as the basis for many exciting >>>> Erlang-based print production tools. >>>> >>>> So, Treepan, I appreciate your interest and would welcome your >>>> involvement in testing and feature development. >>>> >>>> I would jump for joy if some Erlang guru were to step forward take on >> the >>>> TTF/OTF support issue. I've done some research and have a few ideas on >> how >>>> expanded font support might be accomplished, but have too little time >> over >>>> the next several months to dig into it. >>>> >>>> Incidentally, in my previous post my iPad decided it was smarter than me >>>> and erroneously corrected my GitHUb address.it should be >>>> writersglen/erlPress_core. >>>> >>>> My intention is to bag the camel case in my next release with luck in a >>>> week or two so it should look like writersglen/erlpress_core. >>>> >>>> All the best, >>>> >>>> Lloyd >>>> >>>> >>>> -----Original Message----- >>>> From: "Theepan" >>>> Sent: Monday, September 24, 2018 3:59pm >>>> To: lloyd@REDACTED >>>> Cc: "Erlang Questions Mailing List" , "Joe >>>> Armstrong" >>>> Subject: Re: [erlang-questions] ErlGuten >>>> >>>> Hi Lloyd, >>>> >>>> Thank you for your response. I have some quick clarifications -- >>>> >>>> * What are the major improvements made on the source ErlGuten? Did you >>>> improve support to different image formats? Any critical bugs fixed? >>>> * Is erlPress_core used in commercial applications, specifically of high >>>> throughput types? >>>> * What is the minimum erlang/OTP version required? >>>> >>>> Best, >>>> Theepan >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Mon, Sep 24, 2018 at 6:36 AM Lloyd R. Prentice < >> lloyd@REDACTED> >>>> wrote: >>>> >>>>> Hi Thepan, >>>>> >>>>> I?ve undertaken a major revision of ErlGuten called erlPress_core? >>>>> announced it on Erlang questions several weeks ago. It?s available >> from >>>>> GitHub as Writersglen/erlPress_core. I think you?ll find it much more >>>>> accessible than the ErlGutens. >>>>> >>>>> I?ve since been polishing up the code and adding a few new features. >>>> After >>>>> the launch I learned that camel case is not conventional usage when it >>>>> comes to the names of Erlang applications and libraries. So I plan to >>>>> release the updated version as erlpress_core. With smooth sailing I >>>> hope to >>>>> release sometime this week or next. >>>>> >>>>> All the best, >>>>> >>>>> Lloyd >>>>> >>>>> Sent from my iPad >>>>> >>>>> On Sep 23, 2018, at 8:30 PM, Theepan wrote: >>>>> >>>>> >>>>> Hi Team, >>>>> >>>>> I see many forks of ErlGuten. Which one is the latest, most stable and >>>>> widely used? >>>>> >>>>> https://github.com/richcarl/erlguten >>>>> https://github.com/CarlWright/NGerlguten >>>>> https://github.com/hwatkins/erlguten/commits/master >>>>> >>>>> Best, >>>>> Theepan >>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>>> >>>> >>>> >> >> >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin https://ninenines.eu From sean.hinde@REDACTED Wed Sep 26 11:17:26 2018 From: sean.hinde@REDACTED (Sean Hinde) Date: Wed, 26 Sep 2018 11:17:26 +0200 Subject: [erlang-questions] ErlGuten + a short survey In-Reply-To: <69bc7e83-936b-1f03-0b78-759d5a184349@ninenines.eu> References: <7AB3CEDA-34AF-41E2-BEE9-2B27274FF680@writersglen.com> <1537824375.310720011@apps.rackspace.com> <1537894270.37611727@apps.rackspace.com> <1537904554.681631612@apps.rackspace.com> <69bc7e83-936b-1f03-0b78-759d5a184349@ninenines.eu> Message-ID: <3DDE7568-70A8-4E0A-A8B4-99F1E6187116@mac.com> > > My main concern is that from the Asciidoc I get an AST that has little pagination information attached. I would therefore need to go from that Asciidoc-AST to a sort of PDF-AST which comes with pagination information and can then be used to generate the PDF using the PDF drawing primitives. I have not seen code to automatically paginate in erlguten/erlpress_core but maybe I missed it. Then there's all the concerns about table of contents, figure listings and whatnot. The table generation code in eg_table.erl does auto pagination and IIRC it was not much more than keeping track of used vertical and pushing a new page when there wasn't enough left. The rich text mechanism lets you know how many lines in each paragraph From frank.muller.erl@REDACTED Wed Sep 26 17:56:14 2018 From: frank.muller.erl@REDACTED (Frank Muller) Date: Wed, 26 Sep 2018 17:56:14 +0200 Subject: [erlang-questions] Patch package OTP 21.1 released In-Reply-To: References: <1537882179.4914.34.camel@ericsson.com> Message-ID: 21.1 successfully complied on macOS. Anyone else?s facing the same issue as me on Linux? /Frank On Ubuntu 16.04 LTS: > > $ kerl build 21.1 21.1 > [...] > ./configure -with-dynamic-trace=systemtap --enable-m64-build > --enable-dirty-schedulers --enable-lock-check > [...] > checking for compiler flags for loadable drivers... -m64 -O2 > -I/usr/local/include > -I/home/frank/.kerl/builds/21.1/otp_src_21.1/erts/x86_64-unknown-linux-gnu > -fno-tree-copyrename -D_GNU_SOURCE -fPIC > checking for linker for loadable drivers... gcc > checking for linker flags for loadable drivers... -m64 -shared > -Wl,-Bsymbolic > checking for 'runtime library path' linker flag... -Wl,-R > checking for a compiler that handles jumptables... gcc > checking whether the code model is small... yes > checking for dtrace... no > configure: error: No dtrace utility found. > configure: error: > /home/frank/.kerl/builds/21.1/otp_src_21.1/erts/configure failed for erts > > Why it is looking for Dtrace on Linux (i?ve specified systemtap)? > > Didn?t face this issue with 21.0 on the same machine. > > > /Frank > > > Erlang/OTP 21.1 is the first service release for the 21 major release >> with new features, improvements as well as a few features! >> >> Highlights >> SSH: >> The cipher 'chacha20-poly1305@REDACTED' is now supported if >> OpenSSL 1.1.1 or higher is used as cryptolib. >> The key exchange methods 'curve25519-sha256@REDACTED', 'curve25519- >> sha256' and 'curve448-sha512' are implemented. >> They all depends on that OpenSSL 1.1.1 or higher used as cryptolib. >> Crypto: >> Typing in crypto and public_key applications are reworked and >> corrected. >> New chapter on Algorithm details in the User's guide >> ERTS, Kernel: >> The socket options recvtos, recvttl, recvtclass and pktoptions have >> been implemented in the socket modules. >> Se documentation in gen_tcp, gen_udp and inet modules for more info! >> >> >> For more details see: >> http://erlang.org/download/otp_src_21.1.readme >> >> Pre built versions for Windows can be fetched here: >> http://erlang.org/download/otp_win32_21.1.exe >> http://erlang.org/download/otp_win64_21.1.exe >> >> Online documentation can be browsed here: >> http://erlang.org/documentation/doc-10.1/doc >> >> The Erlang/OTP source can also be found at GitHub on the official >> Erlang repository, Here: OTP-21.1 >> >> >> Please report any new issues via Erlang/OTPs public issue tracker >> https://bugs.erlang.org >> System Dashboard - Erlang Programming Language bugs.erlang.org >> Welcome to Erlang Programming Language. What is bugs.erlang.org? Here >> you report bugs in Erlang/OTP, suggestions for new features and >> improvements. >> We want to thank all of those who sent us patches, suggestions and bug >> reports! >> >> Thank you! >> >> The Erlang/OTP Team at Ericsson >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Wed Sep 26 18:31:54 2018 From: lloyd@REDACTED (lloyd@REDACTED) Date: Wed, 26 Sep 2018 12:31:54 -0400 (EDT) Subject: [erlang-questions] ErlGuten + a short survey In-Reply-To: <69bc7e83-936b-1f03-0b78-759d5a184349@ninenines.eu> References: <7AB3CEDA-34AF-41E2-BEE9-2B27274FF680@writersglen.com> <1537824375.310720011@apps.rackspace.com> <1537894270.37611727@apps.rackspace.com> <1537904554.681631612@apps.rackspace.com> <69bc7e83-936b-1f03-0b78-759d5a184349@ninenines.eu> Message-ID: <1537979514.42845570@apps.rackspace.com> Hi Lo?c, If you print out ep_show_n_tell.pdf you'll see page numbers at the bottom of each page alternating bottom flush left and flush right depending upon verso or recto page. Pagination is specified as follows: ep_pagination:first_page(PDF, Job), % ****** Page1 ...which prints: Page 1 ep_pagination:next_page(PDF, Job), % ****** Page 2 ...which prints: Page 2 ...etc. If you want a page with no pagination, then simply don't include these calls in your PDF source. Here's the code: first_page(PDF, Job) -> eg_pdf:set_pagesize(PDF,a4), eg_pdf:set_author(PDF,"LRP"), eg_pdf:set_title(PDF, "Test Elements"), eg_pdf:set_subject(PDF,"erlPrest work-in-progress"), eg_pdf:set_keywords(PDF,"Erlang, PDF, erlPress"), eg_pdf:new_page(PDF), eg_pdf:set_page(PDF,1), pageno(PDF, Job), ok. next_page(PDF, Job) -> CurrentPage = eg_pdf:get_page_no(PDF), NextPage = CurrentPage + 1, eg_pdf:new_page(PDF), eg_pdf:set_page(PDF, NextPage), pageno(PDF, Job) ok. You can also devise your own pagination styles using this function: pageno(PDF, Job)-> {Font, Size} = maps:get(page_no_font, Job), eg_pdf:begin_text(PDF), eg_pdf:set_font(PDF,Font, Size), A = eg_pdf:get_page_no(PDF), Str = "Page " ++ eg_pdf_op:n2s(A), Width = eg_pdf:get_string_width(PDF,"Times-Roman", 11, Str), case A rem 2 of 0 -> eg_pdf:set_text_pos(PDF, 100, 50); 1 -> eg_pdf:set_text_pos(PDF, 510 - Width, 50) end, eg_pdf:text(PDF, "Page " ++ eg_pdf_op:n2s(A)), eg_pdf:end_text(PDF), ok. As to "table of contents, figure listings and whatnot," erlpress_core is just that, a core library upon which much can be built. And it is in its infancy. I do intend, when I can get to it, to support TOCs, figure listings, etc.-- my vision/fantasy is to create a versatile book production tool. And, while I'm at it, maybe even a magazine production tool. Who knows? But, you are right. There is much work to be done. It could be done much faster through community involvement and collaboration. But I understand that you have your own fish to fry. I am, however, enormously grateful for the outstanding work you have done with Cowboy and your dedication to documentation excellence. All the best, Lloyd -----Original Message----- From: "Lo?c Hoguin" Sent: Wednesday, September 26, 2018 3:04am To: lloyd@REDACTED Cc: "Erlang Questions Mailing List" Subject: Re: [erlang-questions] ErlGuten + a short survey As you know I will at some point[0] want to make Asciidoc produce PDF files so I may end up wanting to use it. However from what I've seen so far I'm not sure it would be very helpful. My main concern is that from the Asciidoc I get an AST that has little pagination information attached. I would therefore need to go from that Asciidoc-AST to a sort of PDF-AST which comes with pagination information and can then be used to generate the PDF using the PDF drawing primitives. I have not seen code to automatically paginate in erlguten/erlpress_core but maybe I missed it. Then there's all the concerns about table of contents, figure listings and whatnot. Writing the PDF itself could be done by erlguten/erlpress_core, but it could also just be building an iolist on the fly like I do for man pages[1]. This would be similar to what projects like PHP's fpdf library are doing. Writing the PDF is not a big concern either way since Asciidoc has limited capabilities compared to PDF. In other words, as far as I can tell so far, erlguten seems to solve a different problem than the one I am mainly concerned with. But I've only done a cursory examination of erlguten/erlpress_core so it's possible that I missed something. [0] This or next Christmas. Probably. [1] https://git.ninenines.eu/asciideck.git/tree/src/asciideck_to_manpage.erl On 09/25/2018 09:42 PM, lloyd@REDACTED wrote: > Hi folks, > > As Theepan says, "PDF generation library is crucial to most of the commercial > applications." > > If you are using ErlGuten, please help inform my work on erlpress_core: > > 1. What are you using it for? > 2. What features do you wish it has but doesn't? > > Many thanks, > > Lloyd > > -----Original Message----- > From: "Theepan" > Sent: Tuesday, September 25, 2018 2:55pm > To: "Lloyd Prentice" > Cc: "Erlang Questions Mailing List" , "Joe Armstrong" > Subject: Re: [erlang-questions] ErlGuten > > Hi Lloyd, > > You are right. PDF generation library is crucial to most of the commercial > applications. The code base is not very complicated, but then again we need > somebody who has time and willingness to take on the work. Please find what > have written to Sean on what improvements can be crucial short term. > > Wish work arounds and short fixes we could achieve the required scemantics, > performance and throughput out of ErlGuten. > > Best, > Theepan > > > > On Tue, Sep 25, 2018 at 10:21 PM wrote: > >> Hi Theepan, >> >> Other than display in my demo PDF, I haven't worked much with images. >> >> The issues you've run across seem serious. Getting at the root of the >> problems will likely involve a deep dive into both the PDF reference >> manuals and core ErlGuten source code. >> >> As I noted in my earlier email, ErlGuten source code is very challenging. >> It would benefit significantly from re-write, possibly refactoring in some >> cases and definitely systematic documentation of functions and parameters. >> >> I'm not confident that my skills are up to this. My take is that it would >> require some amount of community effort to put ErlGuten/erlpress_core on a >> solid, maintainable base. I would dearly love to see this happen. >> >> Is there a CS major or intern out there with time to take on the >> challenges? >> >> Joe showed us the way and I, for one, am deeply grateful. Given that PDF >> is a significant standard in the print media world, Erlang deserves to have >> a world-class library and suite of tools to generate PDF documents and >> decode them back into Erlang structures. >> >> This will take community effort. >> >> All the best, >> >> Lloyd >> >> >> >> >> >> -----Original Message----- >> From: "Theepan" >> Sent: Monday, September 24, 2018 7:27pm >> To: "Lloyd Prentice" >> Cc: "Erlang Questions Mailing List" , "Joe >> Armstrong" >> Subject: Re: [erlang-questions] ErlGuten >> >> Wanted to add to the issues the reasons -- >> >> ** When some JPEG image is embedded into the PDF, it turns out dark in the >> generated PDF file [Happens due to CYMK color profile.] >> ** When some PNG file is embedded, it takes too long (unto 5 minutes) to >> generate the PDF file. Debugging shows that the delay is on defilter(Method >> , Line1, Line2, Offset, Width, Iter) of eg_pdf_image module.[Happens due to >> Alpha channel presence] >> >> >> >> On Tue, Sep 25, 2018 at 4:45 AM Theepan wrote: >> >>> Thanks Lloyd for your detailed explanation on what has been done, and >> your >>> vision for the library. However we are already using ErlGuten in one of >> our >>> production systems, and the issues is mainly with the images. >>> >>> I will definitely keep an eye on the development of erlpress_core in the >>> future, as you seem enthusiastic about making it a lively library. >>> >>> The issues we have are: >>> >>> ** When some JPEG image is embedded into the PDF, it turns out dark in >> the >>> generated PDF file >>> ** When some PNG file is embedded, it takes too long (unto 5 minutes) to >>> generate the PDF file. Debugging shows that the delay is on defilter( >>> Method, Line1, Line2, Offset, Width, Iter) of eg_pdf_image module. >>> >>> Did you come acrsoss this? >>> >>> Best, >>> Theepan >>> >>> >>> >>> >>> >>> On Tue, Sep 25, 2018 at 2:56 AM wrote: >>> >>>> Hi Theepan, >>>> >>>> 1. If you look at ErlGuten source, you'll see that function >> documentation >>>> is fairly minimal and many parameters have single character names with >>>> little to no documentation. This makes maintenance and revision very >>>> difficult. I consider the core ErlGuten modules diamonds-in-the-rough. >>>> >>>> I also found the high-level page make-up functions provided by Hugh and >>>> Carl limited with respect to professional page make-up and, to me at >> least, >>>> a bit confusing. >>>> >>>> But erlpress_core owes a deep debt and much gratitude to the ErlGuten >>>> lineage. >>>> >>>> erlpress_core is based on Joe's font-handling, justification, and XML >>>> parsing code with few if any changes. But it provides map structures to >>>> represent nearly all of the PDF objects represented in eg_pdf_op.erl. >> These >>>> map structures incorporate many default parameters so generating PDF >>>> documents is syntactically simple and consistent without sacrificing >>>> expressive flexibility. You can easily customize display by >> instantiating >>>> map parameters with your own values. >>>> >>>> You can see most of the PDF objects currently supported by >> erlpress_core, >>>> including boxed text, justification options, and various line and text >>>> objects, demonstrated in ep_show_n_tell.pdf and the source in >>>> ep_show_n_tell.erl. >>>> >>>> If you look at the map definitions of the various PDF objects, you'll >> see >>>> how to customize display. >>>> >>>> At some point if would be good to re-write the base Erlguten modules >> with >>>> more explicit function and parameter documentation, but I'm not up to >> that >>>> at this point. But, anyone looking for a challenge is free to jump in. >>>> >>>> I'm currently working on tables and text flow across pages (think >> reports >>>> and book chapters). Hope to deliver these features plus much code >> polishing >>>> in the next release coming "real soon now." >>>> >>>> High on my wish/to-do list are: >>>> >>>> -- Easy-to_use page-grid design functions >>>> -- Imposition (printing multiple page impressions on a single sheet of >>>> paper stock) >>>> -- Articles and beads (think text jumping across columns and pages) >>>> -- Easy-to-use and expressive page make-up functions >>>> -- Example templates for various print formats >>>> -- Markdown input >>>> -- Support of TTF and OTF fonts >>>> >>>> Joe Armstrong expressed the following goal when he first announced >>>> ErlGuten: >>>> >>>> "Better than TeX." >>>> >>>> That's a tall order, yet to be realized. My hope is that erlpress will >>>> move the ball further down the field. >>>> >>>> 2. I haven't done anything with image formats beyond what you'll find in >>>> eg_pdf_image.erl. I'd welcome work in that direction. >>>> >>>> 3. erlpress_core is still at Version 0.01. It's just out. I'm working >>>> toward sufficient functionality and stability to support a web >> application >>>> that we have currently under development. >>>> >>>> 4. I developed erlpress_core on Erlang/OTP 19. It does require support >>>> for maps. >>>> >>>> I see the erlpress_core library as a valuable library for embedding PDF >>>> generation into Erlang applications and as the basis for many exciting >>>> Erlang-based print production tools. >>>> >>>> So, Treepan, I appreciate your interest and would welcome your >>>> involvement in testing and feature development. >>>> >>>> I would jump for joy if some Erlang guru were to step forward take on >> the >>>> TTF/OTF support issue. I've done some research and have a few ideas on >> how >>>> expanded font support might be accomplished, but have too little time >> over >>>> the next several months to dig into it. >>>> >>>> Incidentally, in my previous post my iPad decided it was smarter than me >>>> and erroneously corrected my GitHUb address.it should be >>>> writersglen/erlPress_core. >>>> >>>> My intention is to bag the camel case in my next release with luck in a >>>> week or two so it should look like writersglen/erlpress_core. >>>> >>>> All the best, >>>> >>>> Lloyd >>>> >>>> >>>> -----Original Message----- >>>> From: "Theepan" >>>> Sent: Monday, September 24, 2018 3:59pm >>>> To: lloyd@REDACTED >>>> Cc: "Erlang Questions Mailing List" , "Joe >>>> Armstrong" >>>> Subject: Re: [erlang-questions] ErlGuten >>>> >>>> Hi Lloyd, >>>> >>>> Thank you for your response. I have some quick clarifications -- >>>> >>>> * What are the major improvements made on the source ErlGuten? Did you >>>> improve support to different image formats? Any critical bugs fixed? >>>> * Is erlPress_core used in commercial applications, specifically of high >>>> throughput types? >>>> * What is the minimum erlang/OTP version required? >>>> >>>> Best, >>>> Theepan >>>> >>>> >>>> >>>> >>>> >>>> >>>> On Mon, Sep 24, 2018 at 6:36 AM Lloyd R. Prentice < >> lloyd@REDACTED> >>>> wrote: >>>> >>>>> Hi Thepan, >>>>> >>>>> I?ve undertaken a major revision of ErlGuten called erlPress_core? >>>>> announced it on Erlang questions several weeks ago. It?s available >> from >>>>> GitHub as Writersglen/erlPress_core. I think you?ll find it much more >>>>> accessible than the ErlGutens. >>>>> >>>>> I?ve since been polishing up the code and adding a few new features. >>>> After >>>>> the launch I learned that camel case is not conventional usage when it >>>>> comes to the names of Erlang applications and libraries. So I plan to >>>>> release the updated version as erlpress_core. With smooth sailing I >>>> hope to >>>>> release sometime this week or next. >>>>> >>>>> All the best, >>>>> >>>>> Lloyd >>>>> >>>>> Sent from my iPad >>>>> >>>>> On Sep 23, 2018, at 8:30 PM, Theepan wrote: >>>>> >>>>> >>>>> Hi Team, >>>>> >>>>> I see many forks of ErlGuten. Which one is the latest, most stable and >>>>> widely used? >>>>> >>>>> https://github.com/richcarl/erlguten >>>>> https://github.com/CarlWright/NGerlguten >>>>> https://github.com/hwatkins/erlguten/commits/master >>>>> >>>>> Best, >>>>> Theepan >>>>> >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>>> >>>> >>>> >> >> >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin https://ninenines.eu From alex@REDACTED Thu Sep 27 00:28:46 2018 From: alex@REDACTED (alex@REDACTED) Date: Wed, 26 Sep 2018 18:28:46 -0400 Subject: [erlang-questions] Patch package OTP 21.1 released In-Reply-To: References: <1537882179.4914.34.camel@ericsson.com> Message-ID: <19fc6dbbb5a4f7b2086ca41bd08abfc4@smtp.hushmail.com> I don't have any issues configuring and compiling in Linux, but then again (other than my CFLAGS options) I practically use the default options.? In your case, based on your email, probably the issue is that you missed one hyphen (-with-dynamic-trace=systemtap) when configuring systemtrap and the configuration I suppose defaulted to dtrace. Cheers, Alex On 9/26/18 11:56 AM, Frank Muller wrote: > 21.1 successfully complied on macOS. > Anyone else?s facing the same issue as me on Linux? > > /Frank > > On Ubuntu 16.04 LTS: > > $ kerl build 21.1 21.1 > [...] > ./configure -with-dynamic-trace=systemtap --enable-m64-build > --enable-dirty-schedulers --enable-lock-check > [...] > checking for compiler flags for loadable drivers... -m64 -O2 > -I/usr/local/include > -I/home/frank/.kerl/builds/21.1/otp_src_21.1/erts/x86_64-unknown-linux-gnu > ??-fno-tree-copyrename ?-D_GNU_SOURCE -fPIC > checking for linker for loadable drivers... gcc > checking for linker flags for loadable drivers... -m64 -shared > -Wl,-Bsymbolic > checking for 'runtime library path' linker flag... -Wl,-R > checking for a compiler that handles jumptables... gcc > checking whether the code model is small... yes > checking for dtrace... no > configure: error: No dtrace utility found. > configure: error: > /home/frank/.kerl/builds/21.1/otp_src_21.1/erts/configure failed > for erts > > Why it is looking for Dtrace on Linux (i?ve specified systemtap)? > > Didn?t face this issue with 21.0 on the same machine. > > > /Frank > > > Erlang/OTP 21.1 is the first service release for the 21 major > release > with new features, improvements as well as a few features! > > Highlights > SSH: > ?The cipher 'chacha20-poly1305@REDACTED > ' is now??supported if > OpenSSL 1.1.1 or higher is used as cryptolib. > ?The key exchange methods 'curve25519-sha256@REDACTED > ', 'curve25519- > sha256' and 'curve448-sha512' are implemented. > ?They all depends on that OpenSSL 1.1.1 or higher used as > cryptolib. > Crypto: > Typing in crypto and public_key applications are reworked and > corrected. > New chapter on Algorithm details in the User's guide > ERTS, Kernel: > The socket options recvtos, recvttl, recvtclass and pktoptions > have > been implemented in the socket modules. > Se documentation in gen_tcp, gen_udp and inet modules for more > info! > > > For more details see: > http://erlang.org/download/otp_src_21.1.readme > > Pre built versions for Windows can be fetched here: > http://erlang.org/download/otp_win32_21.1.exe > http://erlang.org/download/otp_win64_21.1.exe > > Online documentation can be browsed here: > http://erlang.org/documentation/doc-10.1/doc > > The Erlang/OTP source can also be found at GitHub on the official > Erlang repository, Here: OTP-21.1 > > > Please report any new issues via Erlang/OTPs public issue tracker > https://bugs.erlang.org > ? System Dashboard - Erlang Programming Language > bugs.erlang.org > Welcome to Erlang Programming Language. What is > bugs.erlang.org ? Here > you report bugs in Erlang/OTP, suggestions for new features and > improvements. > We want to thank all of those who sent us patches, suggestions > and bug > reports! > > Thank you! > > The Erlang/OTP Team at Ericsson > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From soverdor@REDACTED Thu Sep 27 01:21:58 2018 From: soverdor@REDACTED (Sam Overdorf) Date: Wed, 26 Sep 2018 16:21:58 -0700 Subject: [erlang-questions] Patch package OTP 21.1 released In-Reply-To: <19fc6dbbb5a4f7b2086ca41bd08abfc4@smtp.hushmail.com> References: <1537882179.4914.34.camel@ericsson.com> <19fc6dbbb5a4f7b2086ca41bd08abfc4@smtp.hushmail.com> Message-ID: I just tried this version and have the following problem. I run my programs with "escript" I have a ".erlang" to set additional code paths. My ".erlang" is not getting executed so my erlang programs are not found. "erl" seems to work just fine. Thanks, Sam Overdorf soverdor@REDACTED On Wed, Sep 26, 2018 at 3:29 PM alex@REDACTED wrote: > > I don't have any issues configuring and compiling in Linux, but then again (other than my CFLAGS options) I practically use the default options. In your case, based on your email, probably the issue is that you missed one hyphen (-with-dynamic-trace=systemtap) when configuring systemtrap and the configuration I suppose defaulted to dtrace. > > Cheers, > Alex > > > On 9/26/18 11:56 AM, Frank Muller wrote: > > 21.1 successfully complied on macOS. > Anyone else?s facing the same issue as me on Linux? > > /Frank > >> On Ubuntu 16.04 LTS: >> >> $ kerl build 21.1 21.1 >> [...] >> ./configure -with-dynamic-trace=systemtap --enable-m64-build --enable-dirty-schedulers --enable-lock-check >> [...] >> checking for compiler flags for loadable drivers... -m64 -O2 -I/usr/local/include -I/home/frank/.kerl/builds/21.1/otp_src_21.1/erts/x86_64-unknown-linux-gnu -fno-tree-copyrename -D_GNU_SOURCE -fPIC >> checking for linker for loadable drivers... gcc >> checking for linker flags for loadable drivers... -m64 -shared -Wl,-Bsymbolic >> checking for 'runtime library path' linker flag... -Wl,-R >> checking for a compiler that handles jumptables... gcc >> checking whether the code model is small... yes >> checking for dtrace... no >> configure: error: No dtrace utility found. >> configure: error: /home/frank/.kerl/builds/21.1/otp_src_21.1/erts/configure failed for erts >> >> Why it is looking for Dtrace on Linux (i?ve specified systemtap)? >> >> Didn?t face this issue with 21.0 on the same machine. >> >> >> /Frank >> >> >>> Erlang/OTP 21.1 is the first service release for the 21 major release >>> with new features, improvements as well as a few features! >>> >>> Highlights >>> SSH: >>> The cipher 'chacha20-poly1305@REDACTED' is now supported if >>> OpenSSL 1.1.1 or higher is used as cryptolib. >>> The key exchange methods 'curve25519-sha256@REDACTED', 'curve25519- >>> sha256' and 'curve448-sha512' are implemented. >>> They all depends on that OpenSSL 1.1.1 or higher used as cryptolib. >>> Crypto: >>> Typing in crypto and public_key applications are reworked and >>> corrected. >>> New chapter on Algorithm details in the User's guide >>> ERTS, Kernel: >>> The socket options recvtos, recvttl, recvtclass and pktoptions have >>> been implemented in the socket modules. >>> Se documentation in gen_tcp, gen_udp and inet modules for more info! >>> >>> >>> For more details see: >>> http://erlang.org/download/otp_src_21.1.readme >>> >>> Pre built versions for Windows can be fetched here: >>> http://erlang.org/download/otp_win32_21.1.exe >>> http://erlang.org/download/otp_win64_21.1.exe >>> >>> Online documentation can be browsed here: >>> http://erlang.org/documentation/doc-10.1/doc >>> >>> The Erlang/OTP source can also be found at GitHub on the official >>> Erlang repository, Here: OTP-21.1 >>> >>> >>> Please report any new issues via Erlang/OTPs public issue tracker >>> https://bugs.erlang.org >>> System Dashboard - Erlang Programming Language bugs.erlang.org >>> Welcome to Erlang Programming Language. What is bugs.erlang.org? Here >>> you report bugs in Erlang/OTP, suggestions for new features and >>> improvements. >>> We want to thank all of those who sent us patches, suggestions and bug >>> reports! >>> >>> Thank you! >>> >>> The Erlang/OTP Team at Ericsson >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From alex@REDACTED Thu Sep 27 05:15:57 2018 From: alex@REDACTED (alex@REDACTED) Date: Wed, 26 Sep 2018 23:15:57 -0400 Subject: [erlang-questions] Patch package OTP 21.1 released In-Reply-To: References: <1537882179.4914.34.camel@ericsson.com> <19fc6dbbb5a4f7b2086ca41bd08abfc4@smtp.hushmail.com> Message-ID: If this was working before, and assuming the Erlang expressions contained are still valid... I'd make sure the HOME system variable hasn't changed by checking on the command-line ($ echo $HOME) and verifying this within the Erlang shell (init:get_argument(home).). Cheers, Alex On 9/26/18 7:21 PM, Sam Overdorf wrote: > I just tried this version and have the following problem. > I run my programs with "escript" I have a ".erlang" to set additional > code paths. > My ".erlang" is not getting executed so my erlang programs are not found. > "erl" seems to work just fine. > > Thanks, > Sam Overdorf > soverdor@REDACTED > > > > On Wed, Sep 26, 2018 at 3:29 PM alex@REDACTED wrote: >> I don't have any issues configuring and compiling in Linux, but then again (other than my CFLAGS options) I practically use the default options. In your case, based on your email, probably the issue is that you missed one hyphen (-with-dynamic-trace=systemtap) when configuring systemtrap and the configuration I suppose defaulted to dtrace. >> >> Cheers, >> Alex >> >> >> On 9/26/18 11:56 AM, Frank Muller wrote: >> >> 21.1 successfully complied on macOS. >> Anyone else?s facing the same issue as me on Linux? >> >> /Frank >> >>> On Ubuntu 16.04 LTS: >>> >>> $ kerl build 21.1 21.1 >>> [...] >>> ./configure -with-dynamic-trace=systemtap --enable-m64-build --enable-dirty-schedulers --enable-lock-check >>> [...] >>> checking for compiler flags for loadable drivers... -m64 -O2 -I/usr/local/include -I/home/frank/.kerl/builds/21.1/otp_src_21.1/erts/x86_64-unknown-linux-gnu -fno-tree-copyrename -D_GNU_SOURCE -fPIC >>> checking for linker for loadable drivers... gcc >>> checking for linker flags for loadable drivers... -m64 -shared -Wl,-Bsymbolic >>> checking for 'runtime library path' linker flag... -Wl,-R >>> checking for a compiler that handles jumptables... gcc >>> checking whether the code model is small... yes >>> checking for dtrace... no >>> configure: error: No dtrace utility found. >>> configure: error: /home/frank/.kerl/builds/21.1/otp_src_21.1/erts/configure failed for erts >>> >>> Why it is looking for Dtrace on Linux (i?ve specified systemtap)? >>> >>> Didn?t face this issue with 21.0 on the same machine. >>> >>> >>> /Frank >>> >>> >>>> Erlang/OTP 21.1 is the first service release for the 21 major release >>>> with new features, improvements as well as a few features! >>>> >>>> Highlights >>>> SSH: >>>> The cipher 'chacha20-poly1305@REDACTED' is now supported if >>>> OpenSSL 1.1.1 or higher is used as cryptolib. >>>> The key exchange methods 'curve25519-sha256@REDACTED', 'curve25519- >>>> sha256' and 'curve448-sha512' are implemented. >>>> They all depends on that OpenSSL 1.1.1 or higher used as cryptolib. >>>> Crypto: >>>> Typing in crypto and public_key applications are reworked and >>>> corrected. >>>> New chapter on Algorithm details in the User's guide >>>> ERTS, Kernel: >>>> The socket options recvtos, recvttl, recvtclass and pktoptions have >>>> been implemented in the socket modules. >>>> Se documentation in gen_tcp, gen_udp and inet modules for more info! >>>> >>>> >>>> For more details see: >>>> http://erlang.org/download/otp_src_21.1.readme >>>> >>>> Pre built versions for Windows can be fetched here: >>>> http://erlang.org/download/otp_win32_21.1.exe >>>> http://erlang.org/download/otp_win64_21.1.exe >>>> >>>> Online documentation can be browsed here: >>>> http://erlang.org/documentation/doc-10.1/doc >>>> >>>> The Erlang/OTP source can also be found at GitHub on the official >>>> Erlang repository, Here: OTP-21.1 >>>> >>>> >>>> Please report any new issues via Erlang/OTPs public issue tracker >>>> https://bugs.erlang.org >>>> System Dashboard - Erlang Programming Language bugs.erlang.org >>>> Welcome to Erlang Programming Language. What is bugs.erlang.org? Here >>>> you report bugs in Erlang/OTP, suggestions for new features and >>>> improvements. >>>> We want to thank all of those who sent us patches, suggestions and bug >>>> reports! >>>> >>>> Thank you! >>>> >>>> The Erlang/OTP Team at Ericsson >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Thu Sep 27 08:29:50 2018 From: frank.muller.erl@REDACTED (Frank Muller) Date: Thu, 27 Sep 2018 08:29:50 +0200 Subject: [erlang-questions] Patch package OTP 21.1 released In-Reply-To: <19fc6dbbb5a4f7b2086ca41bd08abfc4@smtp.hushmail.com> References: <1537882179.4914.34.camel@ericsson.com> <19fc6dbbb5a4f7b2086ca41bd08abfc4@smtp.hushmail.com> Message-ID: Thx Alex. Added the missing hyphen but still getting the same error. Any other idea? I don't have any issues configuring and compiling in Linux, but then again > (other than my CFLAGS options) I practically use the default options. In > your case, based on your email, probably the issue is that you missed one > hyphen (-with-dynamic-trace=systemtap) when configuring systemtrap and > the configuration I suppose defaulted to dtrace. > > Cheers, > Alex > > > On 9/26/18 11:56 AM, Frank Muller wrote: > > 21.1 successfully complied on macOS. > Anyone else?s facing the same issue as me on Linux? > > /Frank > > On Ubuntu 16.04 LTS: >> >> $ kerl build 21.1 21.1 >> [...] >> ./configure -with-dynamic-trace=systemtap --enable-m64-build >> --enable-dirty-schedulers --enable-lock-check >> [...] >> checking for compiler flags for loadable drivers... -m64 -O2 >> -I/usr/local/include >> -I/home/frank/.kerl/builds/21.1/otp_src_21.1/erts/x86_64-unknown-linux-gnu >> -fno-tree-copyrename -D_GNU_SOURCE -fPIC >> checking for linker for loadable drivers... gcc >> checking for linker flags for loadable drivers... -m64 -shared >> -Wl,-Bsymbolic >> checking for 'runtime library path' linker flag... -Wl,-R >> checking for a compiler that handles jumptables... gcc >> checking whether the code model is small... yes >> checking for dtrace... no >> configure: error: No dtrace utility found. >> configure: error: >> /home/frank/.kerl/builds/21.1/otp_src_21.1/erts/configure failed for erts >> >> Why it is looking for Dtrace on Linux (i?ve specified systemtap)? >> >> Didn?t face this issue with 21.0 on the same machine. >> >> >> /Frank >> >> >> Erlang/OTP 21.1 is the first service release for the 21 major release >>> with new features, improvements as well as a few features! >>> >>> Highlights >>> SSH: >>> The cipher 'chacha20-poly1305@REDACTED' is now supported if >>> OpenSSL 1.1.1 or higher is used as cryptolib. >>> The key exchange methods 'curve25519-sha256@REDACTED', 'curve25519- >>> sha256' and 'curve448-sha512' are implemented. >>> They all depends on that OpenSSL 1.1.1 or higher used as cryptolib. >>> Crypto: >>> Typing in crypto and public_key applications are reworked and >>> corrected. >>> New chapter on Algorithm details in the User's guide >>> ERTS, Kernel: >>> The socket options recvtos, recvttl, recvtclass and pktoptions have >>> been implemented in the socket modules. >>> Se documentation in gen_tcp, gen_udp and inet modules for more info! >>> >>> >>> For more details see: >>> http://erlang.org/download/otp_src_21.1.readme >>> >>> Pre built versions for Windows can be fetched here: >>> http://erlang.org/download/otp_win32_21.1.exe >>> http://erlang.org/download/otp_win64_21.1.exe >>> >>> Online documentation can be browsed here: >>> http://erlang.org/documentation/doc-10.1/doc >>> >>> The Erlang/OTP source can also be found at GitHub on the official >>> Erlang repository, Here: OTP-21.1 >>> >>> >>> Please report any new issues via Erlang/OTPs public issue tracker >>> https://bugs.erlang.org >>> System Dashboard - Erlang Programming Language bugs.erlang.org >>> Welcome to Erlang Programming Language. What is bugs.erlang.org? Here >>> you report bugs in Erlang/OTP, suggestions for new features and >>> improvements. >>> We want to thank all of those who sent us patches, suggestions and bug >>> reports! >>> >>> Thank you! >>> >>> The Erlang/OTP Team at Ericsson >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Thu Sep 27 09:46:35 2018 From: lukas@REDACTED (Lukas Larsson) Date: Thu, 27 Sep 2018 09:46:35 +0200 Subject: [erlang-questions] Patch package OTP 21.1 released In-Reply-To: References: <1537882179.4914.34.camel@ericsson.com> Message-ID: the dtrace tool on Linux is used to convert .d files (that dtrace uses) to systemtap files. Our configure checks require this tool to be present when configuring with dynamic-trace=systemtap. This is not something that has changed in between 21.0 and 21.1. Are you sure that nothing has changed in your environment? On Tue, Sep 25, 2018 at 10:15 PM Frank Muller wrote: > On Ubuntu 16.04 LTS: > > $ kerl build 21.1 21.1 > [...] > ./configure -with-dynamic-trace=systemtap --enable-m64-build > --enable-dirty-schedulers --enable-lock-check > [...] > checking for compiler flags for loadable drivers... -m64 -O2 > -I/usr/local/include > -I/home/frank/.kerl/builds/21.1/otp_src_21.1/erts/x86_64-unknown-linux-gnu > -fno-tree-copyrename -D_GNU_SOURCE -fPIC > checking for linker for loadable drivers... gcc > checking for linker flags for loadable drivers... -m64 -shared > -Wl,-Bsymbolic > checking for 'runtime library path' linker flag... -Wl,-R > checking for a compiler that handles jumptables... gcc > checking whether the code model is small... yes > checking for dtrace... no > configure: error: No dtrace utility found. > configure: error: > /home/frank/.kerl/builds/21.1/otp_src_21.1/erts/configure failed for erts > > Why it is looking for Dtrace on Linux (i?ve specified systemtap)? > > Didn?t face this issue with 21.0 on the same machine. > > /Frank > > > Erlang/OTP 21.1 is the first service release for the 21 major release >> with new features, improvements as well as a few features! >> >> Highlights >> SSH: >> The cipher 'chacha20-poly1305@REDACTED' is now supported if >> OpenSSL 1.1.1 or higher is used as cryptolib. >> The key exchange methods 'curve25519-sha256@REDACTED', 'curve25519- >> sha256' and 'curve448-sha512' are implemented. >> They all depends on that OpenSSL 1.1.1 or higher used as cryptolib. >> Crypto: >> Typing in crypto and public_key applications are reworked and >> corrected. >> New chapter on Algorithm details in the User's guide >> ERTS, Kernel: >> The socket options recvtos, recvttl, recvtclass and pktoptions have >> been implemented in the socket modules. >> Se documentation in gen_tcp, gen_udp and inet modules for more info! >> >> >> For more details see: >> http://erlang.org/download/otp_src_21.1.readme >> >> Pre built versions for Windows can be fetched here: >> http://erlang.org/download/otp_win32_21.1.exe >> http://erlang.org/download/otp_win64_21.1.exe >> >> Online documentation can be browsed here: >> http://erlang.org/documentation/doc-10.1/doc >> >> The Erlang/OTP source can also be found at GitHub on the official >> Erlang repository, Here: OTP-21.1 >> >> >> Please report any new issues via Erlang/OTPs public issue tracker >> https://bugs.erlang.org >> System Dashboard - Erlang Programming Language bugs.erlang.org >> Welcome to Erlang Programming Language. What is bugs.erlang.org? Here >> you report bugs in Erlang/OTP, suggestions for new features and >> improvements. >> We want to thank all of those who sent us patches, suggestions and bug >> reports! >> >> Thank you! >> >> The Erlang/OTP Team at Ericsson >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From frank.muller.erl@REDACTED Thu Sep 27 18:43:33 2018 From: frank.muller.erl@REDACTED (Frank Muller) Date: Thu, 27 Sep 2018 18:43:33 +0200 Subject: [erlang-questions] Patch package OTP 21.1 released In-Reply-To: References: <1537882179.4914.34.camel@ericsson.com> Message-ID: Hi Lukas The following package was removed after a system auto-update. This fixed the issue: $ sudo apt-get install systemtap-sdt-dev Thanks again guys. /Frank the dtrace tool on Linux is used to convert .d files (that dtrace uses) to > systemtap files. Our configure checks require this tool to be present when > configuring with dynamic-trace=systemtap. This is not something that has > changed in between 21.0 and 21.1. Are you sure that nothing has changed in > your environment? > > On Tue, Sep 25, 2018 at 10:15 PM Frank Muller > wrote: > >> On Ubuntu 16.04 LTS: >> >> $ kerl build 21.1 21.1 >> [...] >> ./configure -with-dynamic-trace=systemtap --enable-m64-build >> --enable-dirty-schedulers --enable-lock-check >> [...] >> checking for compiler flags for loadable drivers... -m64 -O2 >> -I/usr/local/include >> -I/home/frank/.kerl/builds/21.1/otp_src_21.1/erts/x86_64-unknown-linux-gnu >> -fno-tree-copyrename -D_GNU_SOURCE -fPIC >> checking for linker for loadable drivers... gcc >> checking for linker flags for loadable drivers... -m64 -shared >> -Wl,-Bsymbolic >> checking for 'runtime library path' linker flag... -Wl,-R >> checking for a compiler that handles jumptables... gcc >> checking whether the code model is small... yes >> checking for dtrace... no >> configure: error: No dtrace utility found. >> configure: error: >> /home/frank/.kerl/builds/21.1/otp_src_21.1/erts/configure failed for erts >> >> Why it is looking for Dtrace on Linux (i?ve specified systemtap)? >> >> Didn?t face this issue with 21.0 on the same machine. >> >> /Frank >> >> >> Erlang/OTP 21.1 is the first service release for the 21 major release >>> with new features, improvements as well as a few features! >>> >>> Highlights >>> SSH: >>> The cipher 'chacha20-poly1305@REDACTED' is now supported if >>> OpenSSL 1.1.1 or higher is used as cryptolib. >>> The key exchange methods 'curve25519-sha256@REDACTED', 'curve25519- >>> sha256' and 'curve448-sha512' are implemented. >>> They all depends on that OpenSSL 1.1.1 or higher used as cryptolib. >>> Crypto: >>> Typing in crypto and public_key applications are reworked and >>> corrected. >>> New chapter on Algorithm details in the User's guide >>> ERTS, Kernel: >>> The socket options recvtos, recvttl, recvtclass and pktoptions have >>> been implemented in the socket modules. >>> Se documentation in gen_tcp, gen_udp and inet modules for more info! >>> >>> >>> For more details see: >>> http://erlang.org/download/otp_src_21.1.readme >>> >>> Pre built versions for Windows can be fetched here: >>> http://erlang.org/download/otp_win32_21.1.exe >>> http://erlang.org/download/otp_win64_21.1.exe >>> >>> Online documentation can be browsed here: >>> http://erlang.org/documentation/doc-10.1/doc >>> >>> The Erlang/OTP source can also be found at GitHub on the official >>> Erlang repository, Here: OTP-21.1 >>> >>> >>> Please report any new issues via Erlang/OTPs public issue tracker >>> https://bugs.erlang.org >>> System Dashboard - Erlang Programming Language bugs.erlang.org >>> Welcome to Erlang Programming Language. What is bugs.erlang.org? Here >>> you report bugs in Erlang/OTP, suggestions for new features and >>> improvements. >>> We want to thank all of those who sent us patches, suggestions and bug >>> reports! >>> >>> Thank you! >>> >>> The Erlang/OTP Team at Ericsson >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From soverdor@REDACTED Fri Sep 28 00:15:56 2018 From: soverdor@REDACTED (Sam Overdorf) Date: Thu, 27 Sep 2018 15:15:56 -0700 Subject: [erlang-questions] Patch package OTP 21.1 released In-Reply-To: References: <1537882179.4914.34.camel@ericsson.com> Message-ID: I forgot to mention that I am using windows 64bit. otp 20.3 works ok, otp 21.0 and 21.1 do not work using "escript". "erl" works just fine. so it must be something to do with "escript". Thanks, Sam Overdorf soverdor@REDACTED On Thu, Sep 27, 2018 at 9:43 AM Frank Muller wrote: > > Hi Lukas > > The following package was removed after a system auto-update. This fixed the issue: > > $ sudo apt-get install systemtap-sdt-dev > > > Thanks again guys. > > > /Frank > > >> the dtrace tool on Linux is used to convert .d files (that dtrace uses) to systemtap files. Our configure checks require this tool to be present when configuring with dynamic-trace=systemtap. This is not something that has changed in between 21.0 and 21.1. Are you sure that nothing has changed in your environment? >> >> On Tue, Sep 25, 2018 at 10:15 PM Frank Muller wrote: >>> >>> On Ubuntu 16.04 LTS: >>> >>> $ kerl build 21.1 21.1 >>> [...] >>> ./configure -with-dynamic-trace=systemtap --enable-m64-build --enable-dirty-schedulers --enable-lock-check >>> [...] >>> checking for compiler flags for loadable drivers... -m64 -O2 -I/usr/local/include -I/home/frank/.kerl/builds/21.1/otp_src_21.1/erts/x86_64-unknown-linux-gnu -fno-tree-copyrename -D_GNU_SOURCE -fPIC >>> checking for linker for loadable drivers... gcc >>> checking for linker flags for loadable drivers... -m64 -shared -Wl,-Bsymbolic >>> checking for 'runtime library path' linker flag... -Wl,-R >>> checking for a compiler that handles jumptables... gcc >>> checking whether the code model is small... yes >>> checking for dtrace... no >>> configure: error: No dtrace utility found. >>> configure: error: /home/frank/.kerl/builds/21.1/otp_src_21.1/erts/configure failed for erts >>> >>> Why it is looking for Dtrace on Linux (i?ve specified systemtap)? >>> >>> Didn?t face this issue with 21.0 on the same machine. >>> >>> /Frank >>> >>> >>>> Erlang/OTP 21.1 is the first service release for the 21 major release >>>> with new features, improvements as well as a few features! >>>> >>>> Highlights >>>> SSH: >>>> The cipher 'chacha20-poly1305@REDACTED' is now supported if >>>> OpenSSL 1.1.1 or higher is used as cryptolib. >>>> The key exchange methods 'curve25519-sha256@REDACTED', 'curve25519- >>>> sha256' and 'curve448-sha512' are implemented. >>>> They all depends on that OpenSSL 1.1.1 or higher used as cryptolib. >>>> Crypto: >>>> Typing in crypto and public_key applications are reworked and >>>> corrected. >>>> New chapter on Algorithm details in the User's guide >>>> ERTS, Kernel: >>>> The socket options recvtos, recvttl, recvtclass and pktoptions have >>>> been implemented in the socket modules. >>>> Se documentation in gen_tcp, gen_udp and inet modules for more info! >>>> >>>> >>>> For more details see: >>>> http://erlang.org/download/otp_src_21.1.readme >>>> >>>> Pre built versions for Windows can be fetched here: >>>> http://erlang.org/download/otp_win32_21.1.exe >>>> http://erlang.org/download/otp_win64_21.1.exe >>>> >>>> Online documentation can be browsed here: >>>> http://erlang.org/documentation/doc-10.1/doc >>>> >>>> The Erlang/OTP source can also be found at GitHub on the official >>>> Erlang repository, Here: OTP-21.1 >>>> >>>> >>>> Please report any new issues via Erlang/OTPs public issue tracker >>>> https://bugs.erlang.org >>>> System Dashboard - Erlang Programming Language bugs.erlang.org >>>> Welcome to Erlang Programming Language. What is bugs.erlang.org? Here >>>> you report bugs in Erlang/OTP, suggestions for new features and >>>> improvements. >>>> We want to thank all of those who sent us patches, suggestions and bug >>>> reports! >>>> >>>> Thank you! >>>> >>>> The Erlang/OTP Team at Ericsson >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From dgud@REDACTED Fri Sep 28 07:03:14 2018 From: dgud@REDACTED (Dan Gudmundsson) Date: Fri, 28 Sep 2018 07:03:14 +0200 Subject: [erlang-questions] Patch package OTP 21.1 released In-Reply-To: References: <1537882179.4914.34.camel@ericsson.com> Message-ID: Sam you will have to manually load .erlang from your escript if want that functionality now. - Changed the default behaviour of .erlang loading: .erlang is no longer loaded from the current directory. c:erlangrc(PathList) can be used to search and load an .erlang file from user specified directories. escript, erlc, dialyzer and typer no longer load an .erlang at all. *** POTENTIAL INCOMPATIBILITY *** Own Id: OTP-14439 On Fri, Sep 28, 2018 at 12:09 AM Sam Overdorf wrote: > I forgot to mention that I am using windows 64bit. > otp 20.3 works ok, > otp 21.0 and 21.1 do not work using "escript". > "erl" works just fine. so it must be something to do with "escript". > > Thanks, > Sam Overdorf > soverdor@REDACTED > > > > > On Thu, Sep 27, 2018 at 9:43 AM Frank Muller > wrote: > > > > Hi Lukas > > > > The following package was removed after a system auto-update. This fixed > the issue: > > > > $ sudo apt-get install systemtap-sdt-dev > > > > > > Thanks again guys. > > > > > > /Frank > > > > > >> the dtrace tool on Linux is used to convert .d files (that dtrace uses) > to systemtap files. Our configure checks require this tool to be present > when configuring with dynamic-trace=systemtap. This is not something that > has changed in between 21.0 and 21.1. Are you sure that nothing has changed > in your environment? > >> > >> On Tue, Sep 25, 2018 at 10:15 PM Frank Muller < > frank.muller.erl@REDACTED> wrote: > >>> > >>> On Ubuntu 16.04 LTS: > >>> > >>> $ kerl build 21.1 21.1 > >>> [...] > >>> ./configure -with-dynamic-trace=systemtap --enable-m64-build > --enable-dirty-schedulers --enable-lock-check > >>> [...] > >>> checking for compiler flags for loadable drivers... -m64 -O2 > -I/usr/local/include > -I/home/frank/.kerl/builds/21.1/otp_src_21.1/erts/x86_64-unknown-linux-gnu > -fno-tree-copyrename -D_GNU_SOURCE -fPIC > >>> checking for linker for loadable drivers... gcc > >>> checking for linker flags for loadable drivers... -m64 -shared > -Wl,-Bsymbolic > >>> checking for 'runtime library path' linker flag... -Wl,-R > >>> checking for a compiler that handles jumptables... gcc > >>> checking whether the code model is small... yes > >>> checking for dtrace... no > >>> configure: error: No dtrace utility found. > >>> configure: error: > /home/frank/.kerl/builds/21.1/otp_src_21.1/erts/configure failed for erts > >>> > >>> Why it is looking for Dtrace on Linux (i?ve specified systemtap)? > >>> > >>> Didn?t face this issue with 21.0 on the same machine. > >>> > >>> /Frank > >>> > >>> > >>>> Erlang/OTP 21.1 is the first service release for the 21 major release > >>>> with new features, improvements as well as a few features! > >>>> > >>>> Highlights > >>>> SSH: > >>>> The cipher 'chacha20-poly1305@REDACTED' is now supported if > >>>> OpenSSL 1.1.1 or higher is used as cryptolib. > >>>> The key exchange methods 'curve25519-sha256@REDACTED', > 'curve25519- > >>>> sha256' and 'curve448-sha512' are implemented. > >>>> They all depends on that OpenSSL 1.1.1 or higher used as cryptolib. > >>>> Crypto: > >>>> Typing in crypto and public_key applications are reworked and > >>>> corrected. > >>>> New chapter on Algorithm details in the User's guide > >>>> ERTS, Kernel: > >>>> The socket options recvtos, recvttl, recvtclass and pktoptions have > >>>> been implemented in the socket modules. > >>>> Se documentation in gen_tcp, gen_udp and inet modules for more info! > >>>> > >>>> > >>>> For more details see: > >>>> http://erlang.org/download/otp_src_21.1.readme > >>>> > >>>> Pre built versions for Windows can be fetched here: > >>>> http://erlang.org/download/otp_win32_21.1.exe > >>>> http://erlang.org/download/otp_win64_21.1.exe > >>>> > >>>> Online documentation can be browsed here: > >>>> http://erlang.org/documentation/doc-10.1/doc > >>>> > >>>> The Erlang/OTP source can also be found at GitHub on the official > >>>> Erlang repository, Here: OTP-21.1 > >>>> > >>>> > >>>> Please report any new issues via Erlang/OTPs public issue tracker > >>>> https://bugs.erlang.org > >>>> System Dashboard - Erlang Programming Language bugs.erlang.org > >>>> Welcome to Erlang Programming Language. What is bugs.erlang.org? Here > >>>> you report bugs in Erlang/OTP, suggestions for new features and > >>>> improvements. > >>>> We want to thank all of those who sent us patches, suggestions and bug > >>>> reports! > >>>> > >>>> Thank you! > >>>> > >>>> The Erlang/OTP Team at Ericsson > >>>> _______________________________________________ > >>>> erlang-questions mailing list > >>>> erlang-questions@REDACTED > >>>> http://erlang.org/mailman/listinfo/erlang-questions > >>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From petergi@REDACTED Fri Sep 28 10:11:05 2018 From: petergi@REDACTED (Peter J Etheridge) Date: Fri, 28 Sep 2018 18:11:05 +1000 Subject: [erlang-questions] how to mnesia:write/1 in separate module? Message-ID: <209c47caa59097043b29bd87509ba5024424bbff@mail.iinet.net.au> Dear Friends, A record from module_A is to be written into an mnesia table in module_B. What is your preferred method? What steps might a novice overlook? Perhaps a link to an example? Thank you in advance,Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Fri Sep 28 11:00:37 2018 From: lukas@REDACTED (Lukas Larsson) Date: Fri, 28 Sep 2018 11:00:37 +0200 Subject: [erlang-questions] Celebrating 20 years as Open Source with a new blog! In-Reply-To: References: Message-ID: Hello Everyone! It has now been almost 6 months since we started this blog and have since then posted 14 articles! For those that don't regularly check it, I just wanted to let you know that Bj?rn just finished a great series of blog posts about the new SSA format that will be used by the Erlang compiler in OTP-22. Check it out! Lukas On Tue, Apr 17, 2018 at 6:01 PM Kenneth Lundin wrote: > This year it is 20 years since Erlang/OTP was released as Open Source and > we thought it was a good idea to celebrate with a new blog from the OTP > team. > > Check out http://blog.erlang.org > > /Regards Kenneth, Erlang/OTP Ericsson > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From walter.weinmann@REDACTED Fri Sep 28 11:30:12 2018 From: walter.weinmann@REDACTED (Walter Weinmann) Date: Fri, 28 Sep 2018 11:30:12 +0200 Subject: [erlang-questions] Celebrating 20 years as Open Source with a new blog! In-Reply-To: References: Message-ID: Hi Lukas It would be great if there would be a way to subscribe to an automatic notification when there are new blog entries. Walter On Fri, 28 Sep 2018 at 11:01, Lukas Larsson wrote: > Hello Everyone! > > It has now been almost 6 months since we started this blog and have since > then posted 14 articles! > > For those that don't regularly check it, I just wanted to let you know > that Bj?rn just finished a great series of blog posts about the new SSA > format that will be used by the Erlang compiler in OTP-22. Check it out! > > Lukas > > On Tue, Apr 17, 2018 at 6:01 PM Kenneth Lundin wrote: > >> This year it is 20 years since Erlang/OTP was released as Open Source and >> we thought it was a good idea to celebrate with a new blog from the OTP >> team. >> >> Check out http://blog.erlang.org >> >> /Regards Kenneth, Erlang/OTP Ericsson >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- ___________________________________________________________________________ *Walter Weinmann* ___________________________________________________________________________ Obertorplatz 4 CH-4130 Rheinfelden Tel +41 (0)61 841 06 10 ___________________________________________________________________________ Schulstrasse 1 CH-6037 Root Tel +41 (0)41 530 39 70 ___________________________________________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Sep 28 11:31:47 2018 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Fri, 28 Sep 2018 11:31:47 +0200 Subject: [erlang-questions] Celebrating 20 years as Open Source with a new blog! In-Reply-To: References: Message-ID: <16fac1cd-518a-e71a-62bc-0bbd4a4ebbc5@ninenines.eu> RSS? http://blog.erlang.org/feed.xml On 09/28/2018 11:30 AM, Walter Weinmann wrote: > Hi Lukas > > It would be great if there would be a way to subscribe to an automatic > notification when there are new blog entries. > > Walter > > On Fri, 28 Sep 2018 at 11:01, Lukas Larsson > wrote: > > Hello Everyone! > > It has now been almost 6 months since we started this blog and have > since then posted 14 articles! > > For those that don't regularly check it, I just wanted to let you > know that Bj?rn just finished a great series of blog posts about the > new SSA format that will be used by the Erlang compiler in OTP-22. > Check it out! > > Lukas > > On Tue, Apr 17, 2018 at 6:01 PM Kenneth Lundin > wrote: > > This year it is 20 years since Erlang/OTP was released as Open > Source and > we thought it was a good idea to celebrate with a new blog from > the OTP team. > > Check out http://blog.erlang.org > > /Regards Kenneth, Erlang/OTP Ericsson > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > > ___________________________________________________________________________ > *Walter Weinmann* > ___________________________________________________________________________ > Obertorplatz 4 > CH-4130 Rheinfelden > Tel +41 (0)61 841 06 10 > ___________________________________________________________________________ > Schulstrasse 1 > CH-6037 Root > Tel +41 (0)41 530 39 70 > ___________________________________________________________________________ > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin https://ninenines.eu From lukas@REDACTED Fri Sep 28 11:58:22 2018 From: lukas@REDACTED (Lukas Larsson) Date: Fri, 28 Sep 2018 11:58:22 +0200 Subject: [erlang-questions] Celebrating 20 years as Open Source with a new blog! In-Reply-To: <16fac1cd-518a-e71a-62bc-0bbd4a4ebbc5@ninenines.eu> References: <16fac1cd-518a-e71a-62bc-0bbd4a4ebbc5@ninenines.eu> Message-ID: Yes, I also use rss. If you want it as an e-mail, there are free services that send e-mails when an rss feed changes. On Fri, Sep 28, 2018 at 11:31 AM Lo?c Hoguin wrote: > RSS? http://blog.erlang.org/feed.xml > > On 09/28/2018 11:30 AM, Walter Weinmann wrote: > > Hi Lukas > > > > It would be great if there would be a way to subscribe to an automatic > > notification when there are new blog entries. > > > > Walter > > > > On Fri, 28 Sep 2018 at 11:01, Lukas Larsson > > wrote: > > > > Hello Everyone! > > > > It has now been almost 6 months since we started this blog and have > > since then posted 14 articles! > > > > For those that don't regularly check it, I just wanted to let you > > know that Bj?rn just finished a great series of blog posts about the > > new SSA format that will be used by the Erlang compiler in OTP-22. > > Check it out! > > > > Lukas > > > > On Tue, Apr 17, 2018 at 6:01 PM Kenneth Lundin > > wrote: > > > > This year it is 20 years since Erlang/OTP was released as Open > > Source and > > we thought it was a good idea to celebrate with a new blog from > > the OTP team. > > > > Check out http://blog.erlang.org > > > > /Regards Kenneth, Erlang/OTP Ericsson > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > -- > > > > > ___________________________________________________________________________ > > *Walter Weinmann* > > > ___________________________________________________________________________ > > Obertorplatz 4 > > CH-4130 Rheinfelden > > Tel +41 (0)61 841 06 10 > > > ___________________________________________________________________________ > > Schulstrasse 1 > > CH-6037 Root > > Tel +41 (0)41 530 39 70 > > > ___________________________________________________________________________ > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Lo?c Hoguin > https://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex@REDACTED Sat Sep 29 03:01:26 2018 From: alex@REDACTED (alex@REDACTED) Date: Fri, 28 Sep 2018 21:01:26 -0400 Subject: [erlang-questions] Celebrating 20 years as Open Source with a new blog! In-Reply-To: References: <16fac1cd-518a-e71a-62bc-0bbd4a4ebbc5@ninenines.eu> Message-ID: <7f939d4afbcfa0ce9fe5364aed6ad7e1@smtp.hushmail.com> It would also be great if there was a web link from within erlang.org... just saying... On 9/28/18 5:58 AM, Lukas Larsson wrote: > Yes, I also use rss. If you want it as an e-mail, there are free > services that send e-mails when an rss?feed changes. > > On Fri, Sep 28, 2018 at 11:31 AM Lo?c Hoguin > wrote: > > RSS? http://blog.erlang.org/feed.xml > > On 09/28/2018 11:30 AM, Walter Weinmann wrote: > > Hi Lukas > > > > It would be great if there would be a way to subscribe to an > automatic > > notification when there are new blog entries. > > > > Walter > > > > On Fri, 28 Sep 2018 at 11:01, Lukas Larsson > > >> wrote: > > > >? ? ?Hello Everyone! > > > >? ? ?It has now been almost 6 months since we started this blog > and have > >? ? ?since then posted 14 articles! > > > >? ? ?For those that don't regularly check it, I just wanted to > let you > >? ? ?know that Bj?rn just finished a great series of blog posts > about the > >? ? ?new SSA format that will be used by the Erlang compiler in > OTP-22. > >? ? ?Check it out! > > > >? ? ?Lukas > > > >? ? ?On Tue, Apr 17, 2018 at 6:01 PM Kenneth Lundin > > >? ? ?>> wrote: > > > >? ? ? ? ?This year it is 20 years since Erlang/OTP was released > as Open > >? ? ? ? ?Source and > >? ? ? ? ?we thought it was a good idea to celebrate with a new > blog from > >? ? ? ? ?the OTP team. > > > >? ? ? ? ?Check out http://blog.erlang.org > > > >? ? ? ? ?/Regards Kenneth, Erlang/OTP Ericsson > >? ? ? ? ?_______________________________________________ > >? ? ? ? ?erlang-questions mailing list > > erlang-questions@REDACTED > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > >? ? ?_______________________________________________ > >? ? ?erlang-questions mailing list > > erlang-questions@REDACTED > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > -- > > > > > ___________________________________________________________________________ > > *Walter Weinmann* > > > ___________________________________________________________________________ > > Obertorplatz 4 > > CH-4130 Rheinfelden > > Tel +41 (0)61 841 06 10 > > > ___________________________________________________________________________ > > Schulstrasse 1 > > CH-6037 Root > > Tel +41 (0)41 530 39 70 > > > ___________________________________________________________________________ > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Lo?c Hoguin > https://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Sat Sep 29 11:13:10 2018 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sat, 29 Sep 2018 11:13:10 +0200 Subject: [erlang-questions] How to extract string between XML tags In-Reply-To: <1537912561.49523738@apps.rackspace.com> References: <1537912561.49523738@apps.rackspace.com> Message-ID: <992532f0-f511-cc5d-28be-13b35215f6fa@wanadoo.fr> hello, sorry did not see this question before. A simple regexp is possible "<\/?[^>]{1,}>" re:replace("title bold","<\/?[^>]{1,}>","", [global, {return, list}]). "title bold" Le 25/09/2018 ? 23:56, lloyd@REDACTED a ?crit?: > Hello, > > By now I should know how to do this. But I've fumbled for more time than I have to find an elegant solution. > > Can anyone show a better way? > > Example string: "Firstname" % NOTE: could be any valid tag > > My kludge: > > extract_text(TaggedText) -> > Split = re:split(TaggedText, "<"), > Split2 = lists:nth(2, Split), > Split3 = binary_to_list(Split2), > Split4 = re:split(Split3, ">"), > Split5 = lists:nth(2, Split4), > binary_to_list(Split5). > > Surely there's a better way. > > Many thanks, > > LRP > > ********************************************* > My books: > > THE GOSPEL OF ASHES > http://thegospelofashes.com > > Strength is not enough. Do they have the courage > and the cunning? Can they survive long enough to > save the lives of millions? > > FREEIN' PANCHO > http://freeinpancho.com > > A community of misfits help a troubled boy find his way > > AYA TAKEO > http://ayatakeo.com > > Star-crossed love, war and power in an alternative > universe > > Available through Amazon or by request from your > favorite bookstore > > > ********************************************** > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From eric.pailleau@REDACTED Sat Sep 29 11:18:14 2018 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sat, 29 Sep 2018 11:18:14 +0200 Subject: [erlang-questions] How to extract string between XML tags In-Reply-To: <992532f0-f511-cc5d-28be-13b35215f6fa@wanadoo.fr> References: <1537912561.49523738@apps.rackspace.com> <992532f0-f511-cc5d-28be-13b35215f6fa@wanadoo.fr> Message-ID: <92df261e-36a4-7292-9401-38027186b85c@wanadoo.fr> Hello, BTW "]{1,}>" works too, no need to escape / (Perl reflex :) ...) Le 29/09/2018 ? 11:13, PAILLEAU Eric a ?crit?: > hello, > sorry did not see this question before. > > A simple regexp is possible "<\/?[^>]{1,}>" > > re:replace("title bold","<\/?[^>]{1,}>","", [global, > {return, list}]). > "title bold" > > > > Le 25/09/2018 ? 23:56, lloyd@REDACTED a ?crit?: >> Hello, >> >> By now I should know how to do this. But I've fumbled for more time >> than I have to find an elegant solution. >> >> Can anyone show a better way? >> >> Example string: "Firstname"? % NOTE: could be any valid tag >> >> My kludge: >> >> extract_text(TaggedText) -> >> ?? Split = re:split(TaggedText, "<"), >> ?? Split2 = lists:nth(2, Split), >> ?? Split3 = binary_to_list(Split2), >> ?? Split4 = re:split(Split3, ">"), >> ?? Split5 = lists:nth(2, Split4), >> ?? binary_to_list(Split5). >> >> Surely there's a better way. >> >> Many thanks, >> >> LRP >> >> ********************************************* >> My books: >> >> THE GOSPEL OF ASHES >> http://thegospelofashes.com >> >> Strength is not enough. Do they have the courage >> and the cunning? Can they survive long enough to >> save the lives of millions? >> >> FREEIN' PANCHO >> http://freeinpancho.com >> >> A community of misfits help a troubled boy find his way >> >> AYA TAKEO >> http://ayatakeo.com >> >> Star-crossed love, war and power in an alternative >> universe >> >> Available through Amazon or by request from your >> favorite bookstore >> >> >> ********************************************** >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From eric.pailleau@REDACTED Sat Sep 29 12:53:55 2018 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sat, 29 Sep 2018 12:53:55 +0200 Subject: [erlang-questions] [ANN] geas 2.3.0 (Erlang 21.1) Message-ID: Hi Geas 2.3.0 has been released ! Geas is a tool detecting the runnable official Erlang release window for your project. Geas will tell you also : - what are the offending functions in the beam/source files that reduce the available window. - if some beam files are compiled native. - the installed patches and recommend patches that should be installed depending your code. Geas is available as a module, erlang.mk and rebar 2/3 plugins. Changelog : - Update for Erlang 21.1 detection and database - Geas is now available on hex.pm (geas_rebar3 plugin later) - Geas project is now using Travis CI build status. https://github.com/crownedgrouse/geas https://github.com/crownedgrouse/geas/releases/tag/2.3.0 Cheers ! Eric From eckard.brauer@REDACTED Sat Sep 29 13:30:29 2018 From: eckard.brauer@REDACTED (Eckard Brauer) Date: Sat, 29 Sep 2018 13:30:29 +0200 Subject: [erlang-questions] How to extract string between XML tags In-Reply-To: <92df261e-36a4-7292-9401-38027186b85c@wanadoo.fr> References: <1537912561.49523738@apps.rackspace.com> <992532f0-f511-cc5d-28be-13b35215f6fa@wanadoo.fr> <92df261e-36a4-7292-9401-38027186b85c@wanadoo.fr> Message-ID: <20180929133029.0b0510b2@gmx.de> Hello, just another (a beginner's) question probably leading away from the initial point: If I use T = re:replace("title bold", "<\([^>]\+\)>\(.*\)]\+\)>", "\\1 \\2 \\3", [global,{return, list}]). how could I check that T is of the form "X Y X"? Am Sat, 29 Sep 2018 11:18:14 +0200 schrieb PAILLEAU Eric : > Hello, > BTW "]{1,}>" works too, no need to escape / (Perl > reflex :) ...) > > > Le 29/09/2018 ? 11:13, PAILLEAU Eric a ?crit?: > [...] > [...] > [...] > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- :) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: Digitale Signatur von OpenPGP URL: From lloyd@REDACTED Sat Sep 29 17:11:19 2018 From: lloyd@REDACTED (Lloyd R. Prentice) Date: Sat, 29 Sep 2018 11:11:19 -0400 Subject: [erlang-questions] How to extract string between XML tags In-Reply-To: <992532f0-f511-cc5d-28be-13b35215f6fa@wanadoo.fr> References: <1537912561.49523738@apps.rackspace.com> <992532f0-f511-cc5d-28be-13b35215f6fa@wanadoo.fr> Message-ID: Thanks, Eric! Best wishes, Lloyd Sent from my iPad > On Sep 29, 2018, at 5:13 AM, PAILLEAU Eric wrote: > > hello, > sorry did not see this question before. > > A simple regexp is possible "<\/?[^>]{1,}>" > > re:replace("title bold","<\/?[^>]{1,}>","", [global, {return, list}]). > "title bold" > > > >> Le 25/09/2018 ? 23:56, lloyd@REDACTED a ?crit : >> Hello, >> By now I should know how to do this. But I've fumbled for more time than I have to find an elegant solution. >> Can anyone show a better way? >> Example string: "Firstname" % NOTE: could be any valid tag >> My kludge: >> extract_text(TaggedText) -> >> Split = re:split(TaggedText, "<"), >> Split2 = lists:nth(2, Split), >> Split3 = binary_to_list(Split2), >> Split4 = re:split(Split3, ">"), >> Split5 = lists:nth(2, Split4), >> binary_to_list(Split5). >> Surely there's a better way. >> Many thanks, >> LRP >> ********************************************* >> My books: >> THE GOSPEL OF ASHES >> http://thegospelofashes.com >> Strength is not enough. Do they have the courage >> and the cunning? Can they survive long enough to >> save the lives of millions? >> FREEIN' PANCHO >> http://freeinpancho.com >> A community of misfits help a troubled boy find his way >> AYA TAKEO >> http://ayatakeo.com >> Star-crossed love, war and power in an alternative >> universe >> Available through Amazon or by request from your >> favorite bookstore >> ********************************************** >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From hugo@REDACTED Sat Sep 29 17:16:58 2018 From: hugo@REDACTED (Hugo Mills) Date: Sat, 29 Sep 2018 15:16:58 +0000 Subject: [erlang-questions] How to extract string between XML tags In-Reply-To: References: <1537912561.49523738@apps.rackspace.com> <992532f0-f511-cc5d-28be-13b35215f6fa@wanadoo.fr> Message-ID: <20180929151658.GI5429@carfax.org.uk> Note that this only works if there's no nested tags of the same type. For example, it'll get this wrong: Part of a nested tag... (And there's *no* regex that can get this right in general) Hugo. On Sat, Sep 29, 2018 at 11:11:19AM -0400, Lloyd R. Prentice wrote: > Thanks, Eric! > > Best wishes, > > Lloyd > > Sent from my iPad > > > On Sep 29, 2018, at 5:13 AM, PAILLEAU Eric wrote: > > > > hello, > > sorry did not see this question before. > > > > A simple regexp is possible "<\/?[^>]{1,}>" > > > > re:replace("title bold","<\/?[^>]{1,}>","", [global, {return, list}]). > > "title bold" > > > > > > > >> Le 25/09/2018 ? 23:56, lloyd@REDACTED a ?crit : > >> Hello, > >> By now I should know how to do this. But I've fumbled for more time than I have to find an elegant solution. > >> Can anyone show a better way? > >> Example string: "Firstname" % NOTE: could be any valid tag > >> My kludge: > >> extract_text(TaggedText) -> > >> Split = re:split(TaggedText, "<"), > >> Split2 = lists:nth(2, Split), > >> Split3 = binary_to_list(Split2), > >> Split4 = re:split(Split3, ">"), > >> Split5 = lists:nth(2, Split4), > >> binary_to_list(Split5). > >> Surely there's a better way. > >> Many thanks, > >> LRP > >> ********************************************* > >> My books: > >> THE GOSPEL OF ASHES > >> http://thegospelofashes.com > >> Strength is not enough. Do they have the courage > >> and the cunning? Can they survive long enough to > >> save the lives of millions? > >> FREEIN' PANCHO > >> http://freeinpancho.com > >> A community of misfits help a troubled boy find his way > >> AYA TAKEO > >> http://ayatakeo.com > >> Star-crossed love, war and power in an alternative > >> universe > >> Available through Amazon or by request from your > >> favorite bookstore > >> ********************************************** > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Hugo Mills | IoT: The S stands for Security. hugo@REDACTED carfax.org.uk | http://carfax.org.uk/ | PGP: E2AB1DE4 | -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From eric.pailleau@REDACTED Sat Sep 29 17:23:06 2018 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sat, 29 Sep 2018 17:23:06 +0200 Subject: [erlang-questions] How to extract string between XML tags In-Reply-To: <20180929151658.GI5429@carfax.org.uk> References: <1537912561.49523738@apps.rackspace.com> <992532f0-f511-cc5d-28be-13b35215f6fa@wanadoo.fr> <20180929151658.GI5429@carfax.org.uk> Message-ID: <4ab83605-b462-38c2-2722-57c5a5690f23@wanadoo.fr> Hello Hugo, it works with anything, as this solution is not to catch data in tags, but to remove tags. Even with unbalanced crapy html. 1> re:replace("Part of a nested tag...","<\/?[^>]{1,}>","", [global, {return, list}]). "Part of a nested tag..." Le 29/09/2018 ? 17:16, Hugo Mills a ?crit?: > Note that this only works if there's no nested tags of the same > type. For example, it'll get this wrong: > > Part of a nested tag... > > (And there's *no* regex that can get this right in general) > > Hugo. > > On Sat, Sep 29, 2018 at 11:11:19AM -0400, Lloyd R. Prentice wrote: >> Thanks, Eric! >> >> Best wishes, >> >> Lloyd >> >> Sent from my iPad >> >>> On Sep 29, 2018, at 5:13 AM, PAILLEAU Eric wrote: >>> >>> hello, >>> sorry did not see this question before. >>> >>> A simple regexp is possible "<\/?[^>]{1,}>" >>> >>> re:replace("title bold","<\/?[^>]{1,}>","", [global, {return, list}]). >>> "title bold" >>> >>> >>> >>>> Le 25/09/2018 ? 23:56, lloyd@REDACTED a ?crit : >>>> Hello, >>>> By now I should know how to do this. But I've fumbled for more time than I have to find an elegant solution. >>>> Can anyone show a better way? >>>> Example string: "Firstname" % NOTE: could be any valid tag >>>> My kludge: >>>> extract_text(TaggedText) -> >>>> Split = re:split(TaggedText, "<"), >>>> Split2 = lists:nth(2, Split), >>>> Split3 = binary_to_list(Split2), >>>> Split4 = re:split(Split3, ">"), >>>> Split5 = lists:nth(2, Split4), >>>> binary_to_list(Split5). >>>> Surely there's a better way. >>>> Many thanks, >>>> LRP >>>> ********************************************* >>>> My books: >>>> THE GOSPEL OF ASHES >>>> http://thegospelofashes.com >>>> Strength is not enough. Do they have the courage >>>> and the cunning? Can they survive long enough to >>>> save the lives of millions? >>>> FREEIN' PANCHO >>>> http://freeinpancho.com >>>> A community of misfits help a troubled boy find his way >>>> AYA TAKEO >>>> http://ayatakeo.com >>>> Star-crossed love, war and power in an alternative >>>> universe >>>> Available through Amazon or by request from your >>>> favorite bookstore >>>> ********************************************** >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -- ---------------------------------------- Eric PAILLEAU | eric@REDACTED ---------------------------------------- From eric.pailleau@REDACTED Sat Sep 29 17:59:34 2018 From: eric.pailleau@REDACTED (PAILLEAU Eric) Date: Sat, 29 Sep 2018 17:59:34 +0200 Subject: [erlang-questions] How to extract string between XML tags In-Reply-To: <20180929133029.0b0510b2@gmx.de> References: <1537912561.49523738@apps.rackspace.com> <992532f0-f511-cc5d-28be-13b35215f6fa@wanadoo.fr> <92df261e-36a4-7292-9401-38027186b85c@wanadoo.fr> <20180929133029.0b0510b2@gmx.de> Message-ID: <2729f103-604f-9a34-3826-b827e52f275b@wanadoo.fr> Hello, if the question is to be sure that tags are correctly balanced, it is better to use xmerl parser like Fred proposed. I see an issue in your regexp "<\([^>]\+\)>\(.*\)]\+\)>" (.*) will catch anything including tags (I mean also < ) use instead "<\([^>]\+\)>\([^<]+\)]\+\)>" i.e anything that is not a tag start. but for instance it will not work on nested tags : 1> re:replace("title bold","<\([^>]\+\)>\([^<]+\)]\+\)>","\\1 \\2 \\3",[global,{return, list}]). "title b bold b" note that could be rewritten also to : 2> A = re:replace("title bold","<\([^>]\+\)>\([^<]+\)]\+\)>","\\1 \\2 \\1",[global,{return, list}]). "title b bold b" 3> B = re:replace("title bold","<\([^>]\+\)>\([^<]+\)]\+\)>","\\1 \\2 \\1",[global,{return, list}]). "title b bold b" As \\1 MUST BE equal to \\3 4> A = B. should be ok. Exemple with a single tag 43> A = re:replace("title","<\([^>]\+\)>\([^<]+\)]\+\)>","\\1 \\2 \\3",[global,{return, list}]). "th title th" 44> B = re:replace("title","<\([^>]\+\)>\([^<]+\)]\+\)>","\\1 \\2 \\3",[global,{return, list}]). "th title th" 45> A = B. "th title th" But with unbalanced tag fails: 48> A = re:replace("title","<\([^>]\+\)>\([^<]+\)]\+\)>","\\1 \\2 \\3",[global,{return, list}]). "th title b" 49> B = re:replace("title","<\([^>]\+\)>\([^<]+\)]\+\)>","\\1 \\2 \\1",[global,{return, list}]). "th title th" 50> A=B. ** exception error: no match of right hand side value "th title th" Regards Le 29/09/2018 ? 13:30, Eckard Brauer a ?crit?: > Hello, > > just another (a beginner's) question probably leading away from the > initial point: > > If I use > > T = re:replace("title bold", > "<\([^>]\+\)>\(.*\)]\+\)>", > "\\1 \\2 \\3", > [global,{return, list}]). > > how could I check that T is of the form "X Y X"? > > > > Am Sat, 29 Sep 2018 11:18:14 +0200 > schrieb PAILLEAU Eric : > >> Hello, >> BTW "]{1,}>" works too, no need to escape / (Perl >> reflex :) ...) >> >> >> Le 29/09/2018 ? 11:13, PAILLEAU Eric a ?crit?: >> [...] >> [...] >> [...] >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From eckard.brauer@REDACTED Sat Sep 29 20:29:41 2018 From: eckard.brauer@REDACTED (Eckard Brauer) Date: Sat, 29 Sep 2018 20:29:41 +0200 Subject: [erlang-questions] How to extract string between XML tags In-Reply-To: <2729f103-604f-9a34-3826-b827e52f275b@wanadoo.fr> References: <1537912561.49523738@apps.rackspace.com> <992532f0-f511-cc5d-28be-13b35215f6fa@wanadoo.fr> <92df261e-36a4-7292-9401-38027186b85c@wanadoo.fr> <20180929133029.0b0510b2@gmx.de> <2729f103-604f-9a34-3826-b827e52f275b@wanadoo.fr> Message-ID: <20180929202941.289c451b@gmx.de> Thanks for the response. yes, indeed the first idea was to check balancing, and I'm aware of xmerl and the regex issue. Thanks for the hints, it's because "[^<]" wouldn't match nested tags as you wrote below. So my first idea was to try X ++ Y ++ X = re:replace(...) what of course didn't work. I know of the limitations of REs, just wanted to check if there's a way to ga as with Prolog's append/2 (append([X,Y,X], [a,b,a]).) As I wrote, I'm just playing around with Erlang. Most of my work consists of either C(++) or shell programming with only limited practice of other languages (e.g. Prolog, very little Lisp, Python, PHP). So I'm here only for learning so far, and glad for any help I can get. Thanks again, best regards Eckard Am Sat, 29 Sep 2018 17:59:34 +0200 schrieb PAILLEAU Eric : > Hello, > if the question is to be sure that tags are correctly balanced, it is > better to use xmerl parser like Fred proposed. > > I see an issue in your regexp > "<\([^>]\+\)>\(.*\)]\+\)>" (.*) will catch anything > including tags (I mean also < ) > > use instead > "<\([^>]\+\)>\([^<]+\)]\+\)>" i.e anything that is not a tag > start. > > but for instance it will not work on nested tags : > > 1> re:replace("title > bold","<\([^>]\+\)>\([^<]+\)]\+\)>","\\1 \\2 > \\3",[global,{return, list}]). > "title b bold b" > > note that could be rewritten also to : > [...] > bold","<\([^>]\+\)>\([^<]+\)]\+\)>","\\1 \\2 > \\1",[global,{return, list}]). > "title b bold b" > > [...] > bold","<\([^>]\+\)>\([^<]+\)]\+\)>","\\1 \\2 > \\1",[global,{return, list}]). > "title b bold b" > > As \\1 MUST BE equal to \\3 > [...] > should be ok. > > Exemple with a single tag > > [...] > re:replace("title","<\([^>]\+\)>\([^<]+\)]\+\)>","\\1 > \\2 \\3",[global,{return, list}]). > "th title th" > [...] > re:replace("title","<\([^>]\+\)>\([^<]+\)]\+\)>","\\1 > \\2 \\3",[global,{return, list}]). > "th title th" > [...] > "th title th" > > But with unbalanced tag fails: > > [...] > re:replace("title","<\([^>]\+\)>\([^<]+\)]\+\)>","\\1 > \\2 \\3",[global,{return, list}]). > "th title b" > [...] > re:replace("title","<\([^>]\+\)>\([^<]+\)]\+\)>","\\1 > \\2 \\1",[global,{return, list}]). > "th title th" > [...] > ** exception error: no match of right hand side value "th title th" > > Regards > > > Le 29/09/2018 ? 13:30, Eckard Brauer a ?crit?: > [...] > [...] > [...] > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- :) -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: Digitale Signatur von OpenPGP URL: