From gustav.simonsson@REDACTED Sat Jun 1 01:05:14 2013 From: gustav.simonsson@REDACTED (Gustav Simonsson) Date: Sat, 1 Jun 2013 01:05:14 +0200 Subject: [erlang-questions] Preallocate memory In-Reply-To: References: Message-ID: Hi, Afaik there is no flag for the Erlang VM to tell it to preallocate memory. There is however the +hms flag to increase the default heap size of Erlang processes. You may want to look at the +hmbs and +a flags [1] as well depending on how your Erlang application is constructed. There is also a ton of flags for the Erlang allocators [2] which could be useful. The most efficient way however, if you want to limit the RAM usage of your Java app, is probably to configure the JVM with the -Xmx flag [3], or use something like ulimit [4] or cgroups [5] to limit the JVM. Cheers, Gustav [1] http://www.erlang.org/doc/man/erl.html [2] http://www.erlang.org/doc/man/erts_alloc.html#flags [3] http://docs.oracle.com/javase/7/docs/technotes/tools/solaris/java.html) [4] http://ss64.com/bash/ulimit.html [5] https://www.kernel.org/doc/Documentation/cgroups/memory.txt On Fri, May 31, 2013 at 4:36 PM, Matthew Evans wrote: > Hi Group > > What is the best way (if it's possible) to preallocate memory on startup > for the Erlang VM? On our setup we have a low priority Java app that we > don't want to consume RAM that is needed for the higher priority Erlang VM. > > Thanks > > Matt > > Sent from my iPhone > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brum76@REDACTED Sat Jun 1 01:29:28 2013 From: brum76@REDACTED (Radu Brumariu) Date: Fri, 31 May 2013 19:29:28 -0400 Subject: [erlang-questions] erlang/C embedded db suggestion Message-ID: Hi, I have a project where an application reads bits from a database using a C library and makes some decisions based on the data available there. At the same time, the same database is periodically updated by an Erlang app. It is highly likely that both the C lib and the Erlang app would have the library open at the same time, however the C lib would be read only, whereas Erlang process would open it for write as well. Any suggestions on what the database might be ? Something like term storage ( bdb ) would work for my needs, but bdb cannot be opened in parallel. Same goes for LevelDB, KyotoCabinet. The only that I have found ( so far ) is sqlite(but I don't need sql). I am thinking that an alternative would be to have a raw file mmap-ped by the C library, but I would much rather use something that has some primitives already built. Also, I am considering doing IPC via a unix/tcp socket with the Erlang process, but I'd rather use a db/file ( provides natural fallback if the process does not run, or other comm issues). For the time being, the Erlang app and the C lib would be on the same box. Thanks, Radu -------------- next part -------------- An HTML attachment was scrubbed... URL: From xramtsov@REDACTED Sat Jun 1 03:11:06 2013 From: xramtsov@REDACTED (Evgeniy Khramtsov) Date: Sat, 01 Jun 2013 11:11:06 +1000 Subject: [erlang-questions] STUN server In-Reply-To: References: Message-ID: <51A94A2A.2020303@gmail.com> On 01.06.2013 01:00, Igor Karymov wrote: > You may take a look here: > https://github.com/processone/ejabberd/tree/master/src/stun > It's a separate project now: https://github.com/processone/stun -- Regards, Evgeniy Khramtsov, ProcessOne. xmpp:xram@REDACTED From bchesneau@REDACTED Sat Jun 1 10:43:28 2013 From: bchesneau@REDACTED (Benoit Chesneau) Date: Sat, 1 Jun 2013 10:43:28 +0200 Subject: [erlang-questions] STUN server In-Reply-To: <51A94A2A.2020303@gmail.com> References: <51A94A2A.2020303@gmail.com> Message-ID: On Sat, Jun 1, 2013 at 3:11 AM, Evgeniy Khramtsov wrote: > On 01.06.2013 01:00, Igor Karymov wrote: >> >> You may take a look here: >> https://github.com/processone/ejabberd/tree/master/src/stun >> > > It's a separate project now: https://github.com/processone/stun > Awesome! Thanks :) - benoit From bryan@REDACTED Sat Jun 1 17:34:36 2013 From: bryan@REDACTED (Bryan Hughes) Date: Sat, 01 Jun 2013 08:34:36 -0700 Subject: [erlang-questions] Are nested records copied? Message-ID: <51AA148C.6060505@go-factory.net> Howdy! I am working on section of code that will be dealing with a large number of records in a relatively tight loop and have a question in how Erlang constructs a nested record. If I define a person and manager record: 1> rd(person, {name, age}). person 2> rd(manager, {person, dept}). manager Then create a person record: 3> P = #person{name = bryan, age = 45}. #person{name = bryan,age = 45} And then a manager record: 4> M = #manager{person = P, dept = foo}. #manager{person = #person{name = bryan,age = 45},dept = foo} When the manager record is created, is the nested person record P referenced by the manager record M, or is it copied? Thanks, Bryan -- Bryan Hughes *Go Factory* http://www.go-factory.net /"Internet Class, Enterprise Grade"/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bryan@REDACTED Sat Jun 1 17:48:43 2013 From: bryan@REDACTED (Bryan Hughes) Date: Sat, 01 Jun 2013 08:48:43 -0700 Subject: [erlang-questions] Best implementation for a clustered connected client list ? In-Reply-To: <78DED4DA-C92D-4580-B3EA-185B9A06AD88@gmail.com> References: <96E60454-C631-498F-9325-232FD571EBFA@gmail.com> <481A2AF6-FDCE-4805-8AFD-EA6F5A2C9FF8@gmail.com> <78DED4DA-C92D-4580-B3EA-185B9A06AD88@gmail.com> Message-ID: <51AA17DB.7000609@go-factory.net> Hi Morgan, Have you taken a look at Basho's Riak Core (its open source) - they have solved nicely the consistent hashing mapping to vnodes that allow clusters to dynamically change in size while being stable. http://basho.com/where-to-start-with-riak-core/ We are looking at using it to implement distribution in our solution. Haven't dove into yet, so can speak to what is involved in adopting their solution. Cheers, Bryan On 5/31/13 3:52 AM, Morgan Segalis wrote: > Hi Dmitry, > > I have though about consistent hashing, > > The only issue is that consistent hashing will work if we have a fixed > number of cluster, If we add dynamically another cluster, the hash > won't gives me the same cluster... > I might be wrong... > > Actually right now I have a gateway, which will choose the cluster on > which there is the less number of connected clients, and redirect the > client to this one. Working like a load balancer among other things > the gateway does. > > Best regards, > Morgan. > > > Le 31 mai 2013 ? 12:30, Dmitry Kolesnikov > a ?crit : > >> Hello, >> >> You current implementation starts to suffer from performance due to >> large number of messages to discover process location. >> You have to define a formal rule about "id" and its relation to node >> where processes exists. Essentially I am talking about consistent >> hashing. >> >> To be honest, I am not getting what is wrong with ETS and gproc. I am >> using a similar approach for my cluster management. I am using P2P >> methodology >> where local tables get sync periodically + updates on local table is >> replicated to cluster members. Each node is capable to observe the >> status of cluster members. Once node is disconnected the table is >> clean-up. However, I am using that approach for "internal" processes. >> "Client connection" are not distributed globally. >> >> Best Regards, >> Dmitry >> >> On May 31, 2013, at 1:12 PM, Morgan Segalis > > wrote: >> >>> Hi, >>> >>> Actually right now, I have this implementation : >>> >>> Each node has a clientpool >>> Each client pool manage an ets table, where all client are inserted >>> / removed / searched >>> >>> If an user A sends a message to user B, it will check first the >>> local ets, if not found, will ask to all nodes if they found user B >>> in their ets table, if user B is connected to another node, this >>> node will return the process pid. >>> >>> Advantage of this implementation : no synchronization required if >>> one node goes down and back up again... >>> Disadvantage of this implementation : I guess the number of message >>> / sec is much higher of number of connection/disconnection / sec >>> >>> >>> Le 31 mai 2013 ? 00:36, Chris Hicks >> > a ?crit : >>> >>>> Please keep in mind that this is the result of a total of about 5 >>>> seconds of thinking. >>>> >>>> You could have a coordinator on each node which is responsible for >>>> communicating with the coordinators on all of the other connected >>>> nodes. Your ETS entries would need to be expanded to keep track of >>>> the node that the user is connected on as well. The coordinators >>>> track the joining/leaving of the cluster of all other nodes and >>>> will purge the ETS table of any entries that belong to any recently >>>> downed node. As long as you don't have hard real-time requirements, >>>> which if you do you're using the wrong tool anyway, then you can >>>> come up with a bunch of ways to group together updates between >>>> coordinators to make sure they don't get overloaded. >>>> >>>> Without a lot more details on the exact sort of metrics your system >>>> needs to be able to handle it's all really just a guessing game, in >>>> the end. >>>> >>>> >>>> On Thu, May 30, 2013 at 8:38 AM, Morgan Segalis >>> > wrote: >>>> >>>> Hi everyone, >>>> >>>> I'm currently looking for better ways to do a clustered >>>> connected client list. >>>> >>>> On every server I have an ets table, in which I insert / delete >>>> / search the process of each client connecting to the server. >>>> >>>> Each row in the ets table is for now a simple {"id", >>>> } >>>> >>>> I have tried the gproc module from Ulf Wiger, but when a >>>> cluster goes down, everything goes wrong... (especially if it >>>> is the elected leader). >>>> >>>> If a cluster goes down, other clusters should consider that >>>> every client connected on the said cluster are actually not >>>> connected (even if is just a simple connection error between >>>> clusters). >>>> If it goes back online, back on the nodes() list, other >>>> clusters should consider clients on this cluster back online. >>>> >>>> What would be in your opinion the best way to do that ? >>>> >>>> It is a messaging system, so it has to handle massive message >>>> passing through processes. >>>> >>>> Thank you for your help. >>>> >>>> Morgan. >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Bryan Hughes *Go Factory* http://www.go-factory.net /"Internet Class, Enterprise Grade"/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Sat Jun 1 17:50:59 2013 From: bob@REDACTED (Bob Ippolito) Date: Sat, 1 Jun 2013 08:50:59 -0700 Subject: [erlang-questions] Are nested records copied? In-Reply-To: <51AA148C.6060505@go-factory.net> References: <51AA148C.6060505@go-factory.net> Message-ID: It's not copied in this case, nested records aren't flattened or anything like that. Records are just tuples after all. On Saturday, June 1, 2013, Bryan Hughes wrote: > Howdy! > > I am working on section of code that will be dealing with a large number > of records in a relatively tight loop and have a question in how Erlang > constructs a nested record. > > If I define a person and manager record: > > 1> rd(person, {name, age}). > person > 2> rd(manager, {person, dept}). > manager > > Then create a person record: > > 3> P = #person{name = bryan, age = 45}. > #person{name = bryan,age = 45} > > And then a manager record: > > 4> M = #manager{person = P, dept = foo}. > #manager{person = #person{name = bryan,age = 45},dept = foo} > > When the manager record is created, is the nested person record P > referenced by the manager record M, or is it copied? > > Thanks, > Bryan > > -- > > Bryan Hughes > *Go Factory* > http://www.go-factory.net > > *"Internet Class, Enterprise Grade"* > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bryan@REDACTED Sat Jun 1 17:55:32 2013 From: bryan@REDACTED (Bryan Hughes) Date: Sat, 01 Jun 2013 08:55:32 -0700 Subject: [erlang-questions] Are nested records copied? In-Reply-To: References: <51AA148C.6060505@go-factory.net> Message-ID: <51AA1974.6030403@go-factory.net> Thanks Bob - that is what I was thinking/hoping. On 6/1/13 8:50 AM, Bob Ippolito wrote: > It's not copied in this case, nested records aren't flattened or > anything like that. Records are just tuples after all. > > On Saturday, June 1, 2013, Bryan Hughes wrote: > > Howdy! > > I am working on section of code that will be dealing with a large > number of records in a relatively tight loop and have a question > in how Erlang constructs a nested record. > > If I define a person and manager record: > > 1> rd(person, {name, age}). > person > 2> rd(manager, {person, dept}). > manager > > Then create a person record: > > 3> P = #person{name = bryan, age = 45}. > #person{name = bryan,age = 45} > > And then a manager record: > > 4> M = #manager{person = P, dept = foo}. > #manager{person = #person{name = bryan,age = 45},dept = foo} > > When the manager record is created, is the nested person record P > referenced by the manager record M, or is it copied? > > Thanks, > Bryan > > -- > > Bryan Hughes > *Go Factory* > http://www.go-factory.net > > /"Internet Class, Enterprise Grade"/ > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Sun Jun 2 04:15:53 2013 From: felixgallo@REDACTED (Felix Gallo) Date: Sat, 1 Jun 2013 19:15:53 -0700 Subject: [erlang-questions] speeding the development process for OTP nodes Message-ID: I'm developing a medium-sized erlang node comprised of several interacting applications. In doing so, I'm finding myself staring at the output of ./rebar compile && ./rebar generate && ./rel/bin/mynode/bin/mynode console quite a bit more than is perhaps wanted. I thought that https://github.com/rustyio/sync might be the answer to all of my problems, as it advertises hot code reloading on file changes. However, I haven't yet coaxed it into working inside the ./rel/bin/mynode/bin/mynode console environment; I suspect it's detecting the code changes, and recompiling the code, but it's not redeploying into the release, so doesn't reload the beams. So then I thought that maybe it's dumb to include the reltool build in the development cycle, maybe the pro erlang way is to just run erl by hand like our forefathers did before us. But when I give that a shot, as an OTP neophyte, I have this lurking feeling that I am aiming a gun at my foot. For those of us who don't believe in emacs, what's the one true optimal way to tighten up the development loop here? F. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremy@REDACTED Sun Jun 2 04:26:20 2013 From: jeremy@REDACTED (Jeremy Ong) Date: Sat, 1 Jun 2013 19:26:20 -0700 Subject: [erlang-questions] speeding the development process for OTP nodes In-Reply-To: References: Message-ID: I believe you said it yourself. Don't use reltool in your development cycle. Reltool is for generating releases and going through the whole rigamarole for small iterations is overkill. Write a simple shell script to launch erl with all your code paths and start rustyio/sync for development. Before you release or perform a code upgrade, loop in rebar generate at that point. On Sat, Jun 1, 2013 at 7:15 PM, Felix Gallo wrote: > I'm developing a medium-sized erlang node comprised of several interacting > applications. In doing so, I'm finding myself staring at the output of > > ./rebar compile && ./rebar generate && ./rel/bin/mynode/bin/mynode console > > quite a bit more than is perhaps wanted. > > I thought that https://github.com/rustyio/sync might be the answer to all > of my problems, as it advertises hot code reloading on file changes. > However, I haven't yet coaxed it into working inside the > ./rel/bin/mynode/bin/mynode console environment; I suspect it's detecting > the code changes, and recompiling the code, but it's not redeploying into > the release, so doesn't reload the beams. > > So then I thought that maybe it's dumb to include the reltool build in the > development cycle, maybe the pro erlang way is to just run erl by hand like > our forefathers did before us. But when I give that a shot, as an OTP > neophyte, I have this lurking feeling that I am aiming a gun at my foot. > > For those of us who don't believe in emacs, what's the one true optimal > way to tighten up the development loop here? > > F. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Sun Jun 2 04:39:02 2013 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 01 Jun 2013 19:39:02 -0700 Subject: [erlang-questions] speeding the development process for OTP nodes In-Reply-To: References: Message-ID: <51AAB046.2070004@gmail.com> The only problem you can run into is application dependencies not being started automatically, and issues related to that (i.e., related to not having a real release to use for application startup). There is some code that can help you here https://github.com/okeuday/reltool_util, if you need it. On 06/01/2013 07:26 PM, Jeremy Ong wrote: > I believe you said it yourself. Don't use reltool in your development cycle. Reltool is for generating releases and going through the whole rigamarole for small iterations is overkill. Write a simple shell script to launch erl with all your code paths and start rustyio/sync for development. Before you release or perform a code upgrade, loop in rebar generate at that point. > > > On Sat, Jun 1, 2013 at 7:15 PM, Felix Gallo > wrote: > > I'm developing a medium-sized erlang node comprised of several interacting applications. In doing so, I'm finding myself staring at the output of > > ./rebar compile && ./rebar generate && ./rel/bin/mynode/bin/mynode console > > quite a bit more than is perhaps wanted. > > I thought that https://github.com/rustyio/sync might be the answer to all of my problems, as it advertises hot code reloading on file changes. However, I haven't yet coaxed it into working inside the ./rel/bin/mynode/bin/mynode console environment; I suspect it's detecting the code changes, and recompiling the code, but it's not redeploying into the release, so doesn't reload the beams. > > So then I thought that maybe it's dumb to include the reltool build in the development cycle, maybe the pro erlang way is to just run erl by hand like our forefathers did before us. But when I give that a shot, as an OTP neophyte, I have this lurking feeling that I am aiming a gun at my foot. > > For those of us who don't believe in emacs, what's the one true optimal way to tighten up the development loop here? > > F. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Sun Jun 2 06:19:36 2013 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 01 Jun 2013 21:19:36 -0700 Subject: [erlang-questions] [ANN] CloudI 1.2.2 Released! Message-ID: <51AAC7D8.7080902@gmail.com> Download 1.2.2 from http://sourceforge.net/projects/cloudi/files/latest/download What is CloudI? A Cloud at the lowest level! CloudI (http://cloudi.org) is a "universal integrator" using an Erlang core, with its actor model, to pursue efficiency, fault-tolerance and scalability. The CloudI API provides a minimal interface to communicate among actors that are called services, so programming language agnostic, database agnostic, and messaging bus agnostic integration can occur. CloudI currently integrates with the programming languages Erlang, C/C++, Java, Python, and Ruby, the databases PostgreSQL, MySQL, couchdb, memcached, and tokyotyrant, the messaging bus ZeroMQ and the internal CloudI service bus. HTTP is also supported for easy integration with cowboy. If anyone is willing to get involved, don't hesitate to contact me or start looking at the code. Thank you for all the suggestions and questions so far! This release finished up some performance issues based on loadtesting and profiling. The latest loadtest results are summarized at https://github.com/okeuday/CloudI/raw/master/src/tests/http_req/loadtest/results_v1_2_2/201306_summary.pdf . Please note that the 1.2.2 release only currently works with R16 (a problem with the old reltool prevents R15 usage right now), tell me if that is a problem. The gory details are below: * Performance was improved further for external services with the addition of UNIX domain socket support as the "local" protocol. The python_c CloudI API now has minimal latency. The Erlang, C/C++, and Java CloudI APIs now have similarly low latency (20-40ms latency mean max, 130-170ms latency max sample, for 20k connections at 10k req/s with the "local" protocol). Now all the CloudI API implementations have been modified to send the request response before an exception is thrown to avoid imposing any extra request latency. * v1 UUIDs now have the correct variant bits set. This fix makes the UUIDs backwards incompatible (if older UUIDs have been stored in a database). * Prevously, external service buffer sizes where assigned after being doubled. Now, that is no longer the case. If that is a concern, make sure to double your configured buffer size (specified in the external service configuration). * Problems with the CloudI database services have now been fixed. It is now true that the cloudi_service_init/3 self() process is the Dispatcher Erlang process used in future service function calls. That will remain true in the future to maintain consistency with OTP behaviours. However, it is mainly for proper link/monitor creation, it is still best to use cloudi_service:self/1 when retrieving the service's Erlang process. * Added the "cloudi" module for Erlang usage of CloudI services within external Erlang processes. * Added the "cloudi_x_" scope (i.e., namespace) prefix to all external CloudI dependencies so that it is easier to include CloudI with other Erlang applications. This change will impact any Erlang applications that currently use CloudI and CloudI external dependencies, but have not yet included their own version of the external dependencies. * Added proper support for loading Erlang applications and Erlang releases that include a CloudI service module. * Added newest/oldest destination refresh method options. * Now external service configuration can use the "default" atom for either the protocol and/or the buffer size. The default protocol is the new "local" protocol and the default buffer size is currently 16384 bytes (Linux localhost (inet) MTU) but may change in the future based on further testing. Please mention any problems, issues, or ideas! Thanks, Michael -------------- next part -------------- An HTML attachment was scrubbed... URL: From yashgt@REDACTED Sun Jun 2 10:19:53 2013 From: yashgt@REDACTED (Yash Ganthe) Date: Sun, 2 Jun 2013 13:49:53 +0530 Subject: [erlang-questions] Can Mnesia replace conventional RDBMS? Message-ID: The Mnesia documentation says: largest possible mnesia table (for now) is 4Gb A table can be distributed across nodes so that part of it remains on one node and part on another. In a distributed sense, does this mean 4GB per table per node, or 4GB per table across nodes? Is there a way by which Mnesia can be used in a system that will store about 100GB of data in some tables and support ACID properties just like popular RDBMS's like MySQL/Oracle? Thanks, Yash -------------- next part -------------- An HTML attachment was scrubbed... URL: From rapsey@REDACTED Sun Jun 2 10:50:06 2013 From: rapsey@REDACTED (Rapsey) Date: Sun, 2 Jun 2013 10:50:06 +0200 Subject: [erlang-questions] Can Mnesia replace conventional RDBMS? In-Reply-To: References: Message-ID: I believe it's 4gb per fragment. Mnesia sounds great at first, but it has some difficult operational issues. The biggest one is records. Once you set your schema, it will be extremely challenging to change. Records are inflexible. So if you need to change your schema, you need to change your record definition. If you change your record definition, the database and your code needs to be updated at the same time. I don't see how that is possible without taking your entire service offline. Sergej On Sun, Jun 2, 2013 at 10:19 AM, Yash Ganthe wrote: > The Mnesia documentation says: > largest possible mnesia table (for now) is 4Gb > > A table can be distributed across nodes so that part of it remains on one > node and part on another. In a distributed sense, does this mean 4GB per > table per node, or 4GB per table across nodes? > > Is there a way by which Mnesia can be used in a system that will store about > 100GB of data in some tables and support ACID properties just like popular > RDBMS's like MySQL/Oracle? > > Thanks, > Yash > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From attila.r.nohl@REDACTED Sun Jun 2 10:59:58 2013 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Sun, 2 Jun 2013 10:59:58 +0200 Subject: [erlang-questions] speeding the development process for OTP nodes In-Reply-To: References: Message-ID: 2013/6/2 Felix Gallo : [...] > For those of us who don't believe in emacs, what's the one true optimal way > to tighten up the development loop here? http://www.lysator.liu.se/~tab/erlang/reshd/ This can be used from vim too: a shell script (using an expect script) connects to the TCP port and executes (eventually) compile:file/2. Then this script was set to the makeprg and worked great, under 5 seconds from typing :make to get the code in the system. Of course, if you modify headers, you'll have to manually recompile and load the including modules, but that happens quite rarely. From khandrish@REDACTED Sun Jun 2 17:39:10 2013 From: khandrish@REDACTED (Chris Hicks) Date: Sun, 2 Jun 2013 08:39:10 -0700 Subject: [erlang-questions] Can Mnesia replace conventional RDBMS? In-Reply-To: References: Message-ID: Also, Mnesia focuses on the CA in CAP, and doesn't behave very nicely when you get a split cluster that you then have to bring back into line. The pain of changing the records can be mitigated somewhat by having a single base record type that you store that's basically just a key/value pair (-record(document, {doc_id, doc}) for example) with your changing data held in the value part...but that makes a lot of operations just a bit more complicated than they otherwise would be and you have to do a lot more work yourself. If you have a system design which isn't going to change and is going to remain on a single server Mnesia might just work for you, but if you're looking at any sort of distributed or changing application architecture you might want to consider another solution. On Sun, Jun 2, 2013 at 1:19 AM, Yash Ganthe wrote: > The Mnesia documentation says: > largest possible mnesia table (for now) is 4Gb > > A table can be distributed across nodes so that part of it remains on one > node and part on another. In a distributed sense, does this mean 4GB per > table per node, or 4GB per table across nodes? > > Is there a way by which Mnesia can be used in a system that will store > about 100GB of data in some tables and support ACID properties just like > popular RDBMS's like MySQL/Oracle? > > Thanks, > Yash > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Sun Jun 2 20:02:29 2013 From: ulf@REDACTED (Ulf Wiger) Date: Sun, 2 Jun 2013 20:02:29 +0200 Subject: [erlang-questions] Can Mnesia replace conventional RDBMS? In-Reply-To: References: Message-ID: <8CE0A99D-35E3-4976-99DD-380A5DA1777E@feuerlabs.com> I don't think the documentation says that (at least I can't find it). If you keep data in ram_copies or disc_copies, and run a 64-bit VM, tables can be larger than that. How much data you can have in practice will e.g. depend on your tolerance for long startup times (disc_copies must be loaded into RAM at startup). Disc_only_copies are in fact limited to 2 GB, and you _really_ don't want to exceed that, as mnesia doesn't have a nice way of handling errors at that level - the result will likely be inconsistencies in the database. And even if you fragment your disc_only_copies tables, Dets repair can result in unacceptably long restart times. You need to measure and consider your requirements. Mnesia is a very different beast from Oracle et al. It shines as a low-latency RAM database (with optional persistency), esp. in applications where the access patterns are well known at design-time. Oracle, for example, sacrifices response time for throughput, and adds overhead with e.g. query optimization, which is a good deal if your queries are ad-hoc or you have tons of data and prefer to defer search strategies to the database itself. BR, Ulf W On 2 Jun 2013, at 10:19, Yash Ganthe wrote: > The Mnesia documentation says: > largest possible mnesia table (for now) is 4Gb > > A table can be distributed across nodes so that part of it remains on one node and part on another. In a distributed sense, does this mean 4GB per table per node, or 4GB per table across nodes? > > Is there a way by which Mnesia can be used in a system that will store about 100GB of data in some tables and support ACID properties just like popular RDBMS's like MySQL/Oracle? > > Thanks, > Yash > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From tom@REDACTED Sun Jun 2 20:33:33 2013 From: tom@REDACTED (Tom Samplonius) Date: Sun, 2 Jun 2013 11:33:33 -0700 Subject: [erlang-questions] Can Mnesia replace conventional RDBMS? In-Reply-To: References: Message-ID: <6CD6E191-105C-47B1-8041-EE3AE8B8B0A9@samplonius.org> On Jun 2, 2013, at 1:19 AM, Yash Ganthe wrote: > Is there a way by which Mnesia can be used in a system that will store about 100GB of data in some tables and support ACID properties just like popular RDBMS's like MySQL/Oracle? > You may want to look at the Riak database. It is mostly erlang. While not really relational, it is pretty flexible. Tom > Thanks, > Yash > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From andrew.pennebaker@REDACTED Sun Jun 2 21:22:51 2013 From: andrew.pennebaker@REDACTED (Andrew Pennebaker) Date: Sun, 2 Jun 2013 15:22:51 -0400 Subject: [erlang-questions] Literal support for unary (1#XXXXXX...) Message-ID: Could we extend the # integer syntax to support base 1? It would be cool if 1#11111111 worked, instead of doing this: $ erl Erlang R15B03 (erts-5.9.3.1) [source] [64-bit] [smp:2:2] [async-threads:0] [hipe] [kernel-poll:false] [dtrace] Eshell V5.9.3.1 (abort with ^G) 1> 2#1000. 8 2> 1#11111111. * 1: illegal base '1' 2> #11111111. * 1: syntax error before: 11111111 To be sure, base 1 isn't used that often in practice. But neither is base 3, 13, or 37, so it seems a little strange why the range of base literals is [2, 36]. If we apply the principle of "least surprise", I think it would be a good idea to also accept base 1. At worst, this would add one more complexity step; if we can't use the current base conversion algorithm for base 1, then adding one more match case and summing the number of 1's to follow would be trivial enough to implement. What do you think? -- Cheers, Andrew Pennebaker www.yellosoft.us -------------- next part -------------- An HTML attachment was scrubbed... URL: From masklinn@REDACTED Sun Jun 2 21:26:37 2013 From: masklinn@REDACTED (Masklinn) Date: Sun, 2 Jun 2013 21:26:37 +0200 Subject: [erlang-questions] Literal support for unary (1#XXXXXX...) In-Reply-To: References: Message-ID: On 2013-06-02, at 21:22 , Andrew Pennebaker wrote: > Could we extend the # integer syntax to support base 1? It would be cool if > 1#11111111 worked, instead of doing this: > > $ erl > Erlang R15B03 (erts-5.9.3.1) [source] [64-bit] [smp:2:2] [async-threads:0] > [hipe] [kernel-poll:false] [dtrace] > > Eshell V5.9.3.1 (abort with ^G) > 1> 2#1000. > 8 > 2> 1#11111111. > * 1: illegal base '1' > 2> #11111111. > * 1: syntax error before: 11111111 > > To be sure, base 1 isn't used that often in practice. But neither is base > 3, 13, or 37, so it seems a little strange why the range of base literals > is [2, 36]. If we apply the principle of "least surprise", I think it would > be a good idea to also accept base 1. Why would base1 be limited to the tally mark "1" though? Technically any symbol can be used as the tally mark. From bourinov@REDACTED Sun Jun 2 21:26:08 2013 From: bourinov@REDACTED (Max Bourinov) Date: Sun, 2 Jun 2013 23:26:08 +0400 Subject: [erlang-questions] Literal support for unary (1#XXXXXX...) In-Reply-To: References: Message-ID: Hi Andrew, How base 1 works? Could you explain to me please? Is it number or digits after dash? Best regards, Max Best regards, Max On Sun, Jun 2, 2013 at 11:22 PM, Andrew Pennebaker < andrew.pennebaker@REDACTED> wrote: > Could we extend the # integer syntax to support base 1? It would be cool > if 1#11111111 worked, instead of doing this: > > $ erl > Erlang R15B03 (erts-5.9.3.1) [source] [64-bit] [smp:2:2] [async-threads:0] > [hipe] [kernel-poll:false] [dtrace] > > Eshell V5.9.3.1 (abort with ^G) > 1> 2#1000. > 8 > 2> 1#11111111. > * 1: illegal base '1' > 2> #11111111. > * 1: syntax error before: 11111111 > > To be sure, base 1 isn't used that often in practice. But neither is base > 3, 13, or 37, so it seems a little strange why the range of base literals > is [2, 36]. If we apply the principle of "least surprise", I think it would > be a good idea to also accept base 1. > > At worst, this would add one more complexity step; if we can't use the > current base conversion algorithm for base 1, then adding one more match > case and summing the number of 1's to follow would be trivial enough to > implement. > > What do you think? > > -- > Cheers, > > Andrew Pennebaker > www.yellosoft.us > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Sun Jun 2 21:44:27 2013 From: kostis@REDACTED (Kostis Sagonas) Date: Sun, 02 Jun 2013 21:44:27 +0200 Subject: [erlang-questions] Literal support for unary (1#XXXXXX...) In-Reply-To: References: Message-ID: <51ABA09B.3000507@cs.ntua.gr> On 06/02/2013 09:22 PM, Andrew Pennebaker wrote: > Could we extend the # integer syntax to support base 1? It would be cool > if 1#11111111 worked, instead of doing this: > > $ erl > Erlang R15B03 (erts-5.9.3.1) [source] [64-bit] [smp:2:2] > [async-threads:0] [hipe] [kernel-poll:false] [dtrace] > > Eshell V5.9.3.1 (abort with ^G) > 1> 2#1000. > 8 > 2> 1#11111111. > * 1: illegal base '1' > 2> #11111111. > * 1: syntax error before: 11111111 > > To be sure, base 1 isn't used that often in practice. But neither is > base 3, 13, or 37, so it seems a little strange why the range of base > literals is [2, 36]. If we apply the principle of "least surprise", I > think it would be a good idea to also accept base 1. > > At worst, this would add one more complexity step; if we can't use the > current base conversion algorithm for base 1, then adding one more match > case and summing the number of 1's to follow would be trivial enough to > implement. > > What do you think? Is this a troll question? Numbers B#XXXXXX come with the implicit constraint that the X's are in the range 0..B-1 (in some notation). What is B#BBBBBB supposed to mean (for any base, not just 1)? These numbers do not make any sense. Kostis From andrew.pennebaker@REDACTED Sun Jun 2 21:40:08 2013 From: andrew.pennebaker@REDACTED (Andrew Pennebaker) Date: Sun, 2 Jun 2013 15:40:08 -0400 Subject: [erlang-questions] Literal support for unary (1#XXXXXX...) In-Reply-To: References: Message-ID: > How base 1 works? Could you explain to me please? Is it number or digits > after dash? > Typically, the value of a base 1 is the number of 1's. So three in base ten would be 1#111. http://en.wikipedia.org/wiki/Base_1 As Masklinn points out, the digits used don't really matter because we're just evaluating the length. It could be X's, 3's, or chicken scratches, zeroes, capital I's, or a mix of 1#AlPhAbEt SoUP. Though I think for clarity, we should select a single permissible character and stick to it. Something like tally marks. -- Cheers, Andrew Pennebaker www.yellosoft.us -------------- next part -------------- An HTML attachment was scrubbed... URL: From john_re@REDACTED Sun Jun 2 23:29:08 2013 From: john_re@REDACTED (giovanni_re) Date: Sun, 02 Jun 2013 14:29:08 -0700 Subject: [erlang-questions] Erlang bugs - Beginners Guide to Erlang (equanimity.nl) - ycombinator Message-ID: <1370208548.2495.140661238905521.215F2DC8@webmail.messagingengine.com> A Beginners Guide to Erlang (equanimity.nl) 113 points by wardb 2 days ago | 7 comments https://news.ycombinator.com/item?id=5798254 http://blog.equanimity.nl/blog/2013/05/29/a-beginners-guide-to-erlang/ --- andrewparker 2 days ago | link This is a great page, but it assumes that I (the reader) am already certain I want to invest time learning Erlang. I think it would be valuable if the author would open the page with a brief paragraph on why it's worth investing the time to learn Erlang and some of its tradeoffs vs other languages. reply 501 1 day ago | link Collating all the tradeoffs and bugbears would benefit both the existing community and newcomers. Things that bug me that immediately come to mind: * No public bug tracker * epmd's security (or lack thereof) * Can't insist that epmd be started separately (`erl -no_epmd` won't start epmd but it also won't start epmd's gen_server) * Can't swap out epmd because while `erl -epmd_module foo` is there, `net_kernel:epmd_module()` is barely used. (Although I don't think it's interface is documented anyway) * No built-in way to hook UNIX signals * While the documentation itself is pretty good, it's presentation is lacking and it's difficult to quickly correct mistakes as you run into them * It can be difficult to reason about when a shared binary will be garbage collected * OS packaging (I'm thinking of Debian/Ubuntu) of Erlang and Erlang apps tends to be more harmful than helpful (old packages, namespace conflicts, etc) --- Join the BerkeleyTIP-Global mail list - http://groups.google.com/group/BerkTIPGlobal. All Freedom SW, HW & Culture. From bourinov@REDACTED Sun Jun 2 23:36:35 2013 From: bourinov@REDACTED (Max Bourinov) Date: Mon, 3 Jun 2013 01:36:35 +0400 Subject: [erlang-questions] Literal support for unary (1#XXXXXX...) In-Reply-To: References: Message-ID: Andrew, What would be usage for base 1 except "least surprise"? Best regards, Max On Sun, Jun 2, 2013 at 11:40 PM, Andrew Pennebaker < andrew.pennebaker@REDACTED> wrote: > > How base 1 works? Could you explain to me please? Is it number or digits >> after dash? >> > > Typically, the value of a base 1 is the number of 1's. So three in base > ten would be 1#111. > > http://en.wikipedia.org/wiki/Base_1 > > As Masklinn points out, the digits used don't really matter because we're > just evaluating the length. It could be X's, 3's, or chicken scratches, > zeroes, capital I's, or a mix of 1#AlPhAbEt SoUP. Though I think for > clarity, we should select a single permissible character and stick to it. > Something like tally marks. > > -- > Cheers, > > Andrew Pennebaker > www.yellosoft.us > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From john_re@REDACTED Sun Jun 2 23:52:39 2013 From: john_re@REDACTED (giovanni_re) Date: Sun, 02 Jun 2013 14:52:39 -0700 Subject: [erlang-questions] Erlang concurrency - On Go (dehora.net) - ycombinator Message-ID: <1370209959.18911.140661238911453.0FD1EE0C@webmail.messagingengine.com> On Go (dehora.net) http://dehora.net/journal/2013/06/01/on-go/ 139 points by ishbits 19 hours ago | 98 comments https://news.ycombinator.com/item?id=5806970 2 erlang mentions: ----- graue 15 hours ago | link The lack of exceptions just seems like a total bear to me. I wrote C for 10 years and did tons of stuff like: Open file "foobar.dat". If that fails, abort with this error. Read the first 4 bytes into x. If that fails, abort with a different error. Seek to byte x. If that fails, abort with yet another error. and so on, and so on, over and over again. Python's exceptions are a huge improvement over this pattern of timidly asking permission to do anything. The fact is there are so, so many occasions where you want to abort a computation if one of the many steps along the way goes wrong. Something like Haskell's Maybe monad is a good way of attacking the problem too. But Go has neither. It seems to just offer the bad old clumsy C way and say, "Deal with it." To those who have written real Go programs, I'm honestly wondering: how is this not a pain in the ass? reply MetaCosm 15 hours ago | link Unlike C, Go isn't overloading what is returned, it has an extra param, and the language has been baked to handle doing like initialization and checks in single line if statements. It does force the developer who might actually have a clue what the exception is and how to fix it to handle it, but IMHO this is a good thing. Go forces lots of things like this (not using an import, won't compile; not using a variable, won't compile). Honestly, compared to the exception hellscapes I have had to deal with in Java and C++ --- it seems like the path of least surprise. Which incidentally has been my favorite things about Go, the low number of surprises. A lot of using Go in real work has gone against my expectations. There are a lot of things I initially saw as huge warts (lack of exceptions, generics and import versions), but I liked channels enough (Erlang background) to give it a shot. So far, I have been delighted by using it as a stack (build cycle, deploy method, terse C'ish syntax). ----- hcarvalhoalves 17 hours ago | link Everybody compares Go and Python. How is it so? Go compiles to a binary, implements static typing, got rid of classes, got rid of exceptions, and added lots of special purpose keywords on top of reinventing C's syntax. The only similarity I see is the "import" statement. reply jlgreco 17 hours ago | link They both approximately match up on mental effort /friction/hassle for me, which largely seems to be coincidence since they do have dissimilar feature sets. For example, it use to be that I would use Python when I thought that C (my language of choice) was going to be too much of a hassle; now I use Go when I think that C will be too much hassle. I think this accounts for a large portion of the Go<->Python talk you see. reply SiVal 15 hours ago | link When app(lication)s meant desktop apps, most of us used C as an app dev language. Over the years, the platform changed and most app developers switched to Java and C# and, more recently, to Python and Ruby. Those who still use C these days mostly use it to deal with unusually demanding constraints (performance, memory, etc.) They use C when only C will do, which usually is not for apps. C is not the default app programming language; it's a special-purpose language of sorts. Go is not very competitive with C at what people mostly use C for these days: tightest constraints and maximum customization. Go is very competitive with Python (& Ruby), though, as a high-productivity app development language, being almost as productive as those two but with much higher performance, lower memory requirements, and no need to install a big runtime. It's biggest shortcoming, vis-a-vis Py & Ruby, is the immaturity of its environment (libraries, toolchains, web frameworks, etc) due to its relative newness. As that changes, many will switch from P&R to Go, but fewer will switch from C to Go, because most who don't absolutely need C have already switched away from it. reply pjmlp 13 hours ago | link What I always find interesting is that the Python and Ruby developers are willing to loose abstraction power to delve down in Go, while they would be able to keep it if they would move to PyPy, JVM/.NET based languages or FP languages with native compilers instead. reply coldtea 9 hours ago | link Yes, but in return they get something more mature than PyPy, something with less hassle that bringing along a VM (JVM/.NET), and something less alien and marginal than FP languages with native compilers. They might not get the best technical experience, but they DO get: 1) mostly imperative (and most LIKE it that way), 2) nice concurrency support, 3) a lot of niceties (first class functions, implicit interfaces, maps, etc), 4) quite full batteries included, 5) lots of other kids using it 6) regular success posts on HN 7) nice, and mostly predictable, reasoning about speed and memory reply pjmlp 58 minutes ago | link > Yes, but in return they get something more mature than PyPy, something with less hassle that bringing along a VM (JVM/.NET), and something less alien and marginal than FP languages with native compilers. There are commercial native compilers for Java and .NET if you are willing to pay for them. Language != Implementation As for what one gets, 1) Offered by C++, JVM, .NET languages, D, Rust, ... 2) Scala, Haskell, OCaml, F#, D, Rust, Erlang, .... 3) Almost any modern language with exception of C and Java. 4) Java and .NET have more batteries 5) Yes, because big boys have better tools 6) Fortune 500 companies don't use languages based on HN posts 7) Tooling still behind what big boys use for other languages reply MetaCosm 20 minutes ago | link You kinda made his point for him didn't you? He said all these are benefits of Go, and you pointed out he would need to use lots of different languages to get them elsewhere. Notice the lack of overlap in your own points(1-4), that right there is the crux of the issue. Beyond that, on (5) I simply respond... what? I think you are flat out wrong about (6) and how they pick languages (I contract for them), they "hear" node.js is super fast and will double output and want to use it. (7) is part of what Go was designed to enable, building tools for it is AWESOME (Go AST!). If anything you made a killer post, maybe just not for the reasons you think. --- Join the BerkeleyTIP-Global mail list - http://groups.google.com/group/BerkTIPGlobal. All Freedom SW, HW & Culture. From yashgt@REDACTED Mon Jun 3 08:01:27 2013 From: yashgt@REDACTED (yashgt@REDACTED) Date: Mon, 03 Jun 2013 06:01:27 +0000 Subject: [erlang-questions] How to upgrade the Mnesia record structure? Message-ID: <51ac3149.e982310a.5ec1.5708@mx.google.com> An HTML attachment was scrubbed... URL: From ulf@REDACTED Mon Jun 3 09:02:19 2013 From: ulf@REDACTED (Ulf Wiger) Date: Mon, 3 Jun 2013 09:02:19 +0200 Subject: [erlang-questions] How to upgrade the Mnesia record structure? In-Reply-To: <51ac3149.e982310a.5ec1.5708@mx.google.com> References: <51ac3149.e982310a.5ec1.5708@mx.google.com> Message-ID: You should absolutely update all existing records in the table. You do this with mnesia:transform_table(Tab, Fun, NewAttributeList) Note that the manual says: "Fun is a function which takes a record of the old type and returns a transformed record of the new type. The Fun argument can also be the atom ignore, it indicates that only the meta data about the table will be updated. Usage of ignore is not recommended but included as a possibility for the user do to his own transform." The reason for the 'ignore' trick is that it is sometimes not feasible to transform the entire table within a transaction while user requests are held off waiting for the operation to complete. One can then instead transform the table definition and spawn a process that transforms one record at a time. This would of course mean that any code that reads from the table must be prepared to get an old record, and transform it on the fly to the latest record format. This is not terribly difficult to accomplish*, but requires some forethought. For various reasons like the above, it is a very good idea to have your own code access mnesia via a database API that *you* control. An alternative is to always use mnesia:activity/2 (even for dirty access), making it possible to introduce these workarounds in your own access module. That is, never use mnesia:transaction/1 - use mnesia:activity/2 instead. * At least as long as you mainly read objects. If you are e.g. using QLC, it becomes a non-trivial task. BR, Ulf W On 3 Jun 2013, at 08:01, yashgt@REDACTED wrote: > If a record is defined as: > > -record(shop, {item, quantity, cost}). > > and several records are stored in an Mnesia table, and a few months later the structure is changed to : > -record(shop, {item, quantity, source, company, cost}). > > Do we need to migrate ALL existing records in the DB to include the additional 2 elements in the tuple? I know that a record is actually a tuple. If we leave the existing records untouched, will the database get messed becaues of new records coming in with the new structure? > > Thanks, > Yash > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From mihai@REDACTED Mon Jun 3 09:33:59 2013 From: mihai@REDACTED (Mihai Balea) Date: Mon, 3 Jun 2013 10:33:59 +0300 Subject: [erlang-questions] speeding the development process for OTP nodes In-Reply-To: References: Message-ID: <352FEDB4-E5AE-48D3-A70A-88F907E5FD12@hates.ms> I have a slightly modified version of sync that attempts to work around this issue, you can find it here: https://github.com/sbalea/sync.git It's not bulletproof, which is why I haven't yet submitted a PR for it, but it seems to work in the majority of my use cases. Mihai On Jun 2, 2013, at 5:15 AM, Felix Gallo wrote: > I'm developing a medium-sized erlang node comprised of several interacting applications. In doing so, I'm finding myself staring at the output of > > ./rebar compile && ./rebar generate && ./rel/bin/mynode/bin/mynode console > > quite a bit more than is perhaps wanted. > > I thought that https://github.com/rustyio/sync might be the answer to all of my problems, as it advertises hot code reloading on file changes. However, I haven't yet coaxed it into working inside the ./rel/bin/mynode/bin/mynode console environment; I suspect it's detecting the code changes, and recompiling the code, but it's not redeploying into the release, so doesn't reload the beams. > > So then I thought that maybe it's dumb to include the reltool build in the development cycle, maybe the pro erlang way is to just run erl by hand like our forefathers did before us. But when I give that a shot, as an OTP neophyte, I have this lurking feeling that I am aiming a gun at my foot. > > For those of us who don't believe in emacs, what's the one true optimal way to tighten up the development loop here? > > F. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From turinglife@REDACTED Mon Jun 3 08:46:34 2013 From: turinglife@REDACTED (turinglife) Date: Mon, 3 Jun 2013 14:46:34 +0800 Subject: [erlang-questions] a problem about "file:path_eval([".", "/root"], ".erlang"): error on line 1: 1: syntax error before:"!!!!! Message-ID: <201306031446317717252@gmail.com> Hi friends, When I try to install erlang on my CentOS platform, I encountered a problem. "make" is always not success. illustrated by the below information. =ERROR REPORT==== 26-May-2013::21:03:18 === file:path_eval([".","/root"],".erlang"): error on line 1: 1: syntax error before: make[2]: Leaving directory `/otp_src_R14B04/erts/start_scripts' make[1]: Leaving directory `/otp_src_R14B04/erts' make[1]: Entering directory `/otp_src_R14B04/lib/dialyzer' === Entering application dialyzer So, I would like to ask why? what reasons lead to this error? Thank you very much! Best regards, turinglife turinglife -------------- next part -------------- An HTML attachment was scrubbed... URL: From jinni.park@REDACTED Mon Jun 3 11:34:47 2013 From: jinni.park@REDACTED (Park, Sungjin) Date: Mon, 3 Jun 2013 18:34:47 +0900 Subject: [erlang-questions] gen_server doesn't respond Message-ID: Guys, I'm encountering cases when gen_server doesn't respond. In such cases, erlang:process_info/1 shows the process state 'waiting' with some messages in the message_queue. The messages have not been consumed for tens of minutes at least. Anybody has any idean about this? -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Mon Jun 3 12:34:49 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Mon, 3 Jun 2013 11:34:49 +0100 Subject: [erlang-questions] gen_server doesn't respond In-Reply-To: References: Message-ID: Does the callback module execute any selective receives inside handlers perhaps? If not, are you sure the message has actually arrived in that process' mailbox, can you see unhandled messages in the message queue (I.e., length > 0 whilst waiting)? On 3 Jun 2013, at 10:34, "Park, Sungjin" wrote: > Guys, > > I'm encountering cases when gen_server doesn't respond. > In such cases, erlang:process_info/1 shows the process state 'waiting' with some messages in the message_queue. The messages have not been consumed for tens of minutes at least. Anybody has any idean about this? > > > -- > Park, Sungjin > ------------------------------------------------------------------------------------------------------------------- > Peculiar travel suggestions are dancing lessons from god. > -- The Books of Bokonon > ------------------------------------------------------------------------------------------------------------------- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From aaron.l.france@REDACTED Mon Jun 3 12:39:48 2013 From: aaron.l.france@REDACTED (Aaron France) Date: Mon, 3 Jun 2013 12:39:48 +0200 Subject: [erlang-questions] gen_server doesn't respond In-Reply-To: References: Message-ID: Does it make any synchronous calls to other gen_servers? On Mon, Jun 3, 2013 at 12:34 PM, Tim Watson wrote: > Does the callback module execute any selective receives inside handlers > perhaps? If not, are you sure the message has actually arrived in that > process' mailbox, can you see unhandled messages in the message queue > (I.e., length > 0 whilst waiting)? > > On 3 Jun 2013, at 10:34, "Park, Sungjin" wrote: > > > Guys, > > > > I'm encountering cases when gen_server doesn't respond. > > In such cases, erlang:process_info/1 shows the process state 'waiting' > with some messages in the message_queue. The messages have not been > consumed for tens of minutes at least. Anybody has any idean about this? > > > > > > -- > > Park, Sungjin > > > ------------------------------------------------------------------------------------------------------------------- > > Peculiar travel suggestions are dancing lessons from god. > > -- The Books of Bokonon > > > ------------------------------------------------------------------------------------------------------------------- > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From olav@REDACTED Mon Jun 3 12:59:28 2013 From: olav@REDACTED (Olav Frengstad) Date: Mon, 3 Jun 2013 12:59:28 +0200 Subject: [erlang-questions] Erlang package managers? In-Reply-To: <37FE68BE-1FE3-4A98-95E6-14C42A15C230@gmail.com> References: <5adc0079-b2b9-4c8b-85f6-c06086dab778@googlegroups.com> <37FE68BE-1FE3-4A98-95E6-14C42A15C230@gmail.com> Message-ID: Sorry for late reply, Steven: As Tim pointed out, that means an OTP level change. The distinction i was making was is that there are two different tasks to solve: - Having a way to fetch dependencies (we have rebar, ERL_LIBS, ...), - Having a way to do some action (compile, test, run, ...) with the correct dependencies. I don't think there is much need for version constraints in applications _before_ there is multi-version support in the tools we use. However storing information on how to build an application (compiler opts, dependencies, OTP requirements) in the .app would be a step in the right direction - that is something for a different thread though. Olav 2013/5/31 Tim Watson : > Hi Steve, > > On 30 May 2013, at 00:29, Steve Davis wrote: >> However, it seems to me that ERL_LIBS works pretty good for me... except where we have "deps" on particular versions. >> > > Yeah, that's the problem with ERL_LIBS - otherwise, it works great. > >> I'm throwing it out that if our apps were reliably semantically versioned (www.semver.org) and that the .app file were updated to recognize versions in >> {application, _, {applications, [...]}} > > That's an OTP level change though right? I think the Olav was trying to address some of the issues around dependency handling without assuming changes to core Erlang/OTP or rebar. > >> >> ...where atom was "latest" and {atom, version} were some .ez or other ERL_LIBS folder... >> >> ...would that not solve a whole raft of the issues here? >> >> Interested in the responses :) >> > > Possibly it might help for some tools, but rebar doesn't use the application file currently. Even if you did get this feature into OTP, such that the code loader recognised the versions of things you're attempting to work with specifically, you wouldn't get around all the issues. What about applications where dependencies share dependencies on different major versions, i.e., you need rabbitmq-client + webmachine + mochiweb2 but also raik + webmachine + mochiweb 3 or whatever? How do you pick the right one to download, to compiler, to use? And what about dependencies that aren't packaged so that rebar can build them? Do you fork those repositories and maintain rebar support all yourself? What about needing to build multiple times against, say, multiple versions of OTP for testing. What about when a dependency has native code that needs to be built and you need to ensure that it is built for the same arch/platform as the emulator it's getting loaded into, assuming you've got mu > ltiple OTP installs? And so on... > > Some of the things I wrote in that blog post (that Olav references in his project's README) were intended to address, or at least think about, those kinds of issues. I've since moved back to plain old rebar, with a few plugins to customise the dependency handling, since I had neither the time nor the appetite to try and solve all the various problems. Now I'm left with annoyances, but nothing that prevents me from getting stuff done. I wish Olav the best of luck with his tool though, and will be keeping a close eye on it. > > Cheers, > Tim > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Med Vennlig Hilsen Olav Frengstad Systemutvikler // FWT +47 920 42 090 From ransomr@REDACTED Mon Jun 3 18:11:16 2013 From: ransomr@REDACTED (Ransom Richardson) Date: Mon, 3 Jun 2013 16:11:16 +0000 Subject: [erlang-questions] Handling Crash Reports at scale Message-ID: Are there tools/procedures that are recommended for processing crash reports from a service running at scale? Currently we have a limited deployment and I look through all of the crash reports by hand. Some are very useful for finding actual bugs in our code. But other crashes are the result of client's sending bad data, strange timing issues (mostly not in our code), etc and are not actionable. As we prepare to scale up our service, I'm wondering how to continue to get the value from the interesting crash reports without having to look through all of the uninteresting ones. I haven't found rb to be very useful for finding the new/interesting crashes. Are there effective ways that peopler are using it? Are there other tools for parsing and grouping crash reports to make it easy to find new/interesting ones? thanks, Ransom -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Jun 3 18:42:04 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 3 Jun 2013 20:42:04 +0400 Subject: [erlang-questions] Handling Crash Reports at scale In-Reply-To: References: Message-ID: Never, never, listen never redirect error_logger to email! I have understood that I'm wrong when I got 28000 of emails in mailbox and had to write IMAP script. Use http://airbrake.io/ for collecting your errors =) From puzza007@REDACTED Mon Jun 3 18:58:33 2013 From: puzza007@REDACTED (Paul Oliver) Date: Mon, 3 Jun 2013 12:58:33 -0400 Subject: [erlang-questions] Handling Crash Reports at scale In-Reply-To: References: Message-ID: Or http://riemann.io/ On Mon, Jun 3, 2013 at 12:42 PM, Max Lapshin wrote: > Never, never, listen never redirect error_logger to email! > > I have understood that I'm wrong when I got 28000 of emails in mailbox > and had to write IMAP script. > > Use http://airbrake.io/ for collecting your errors =) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthonym@REDACTED Mon Jun 3 19:29:54 2013 From: anthonym@REDACTED (ANTHONY MOLINARO) Date: Mon, 3 Jun 2013 10:29:54 -0700 Subject: [erlang-questions] Handling Crash Reports at scale In-Reply-To: References: Message-ID: <24F2ADC1-368D-4856-8BAB-AC5660F3033F@alumni.caltech.edu> I'd recommend addressing the other crashes with "defensive code". By capturing and categorizing certain types of errors and clearing those from your logs you'll be able to better see true errors. I usually have a two phase approach. First phase, I capture and log the types of errors. Second, once I see certain types are regular I replace those with metrics (via mondemand). Once you have a stream of errors you can then monitor the rates, plus uncaught errors will make it into your logs which should remain very sparse. -Anthony On Jun 3, 2013, at 9:11 AM, Ransom Richardson wrote: > Are there tools/procedures that are recommended for processing crash reports from a service running at scale? > > Currently we have a limited deployment and I look through all of the crash reports by hand. Some are very useful for finding actual bugs in our code. But other crashes are the result of client's sending bad data, strange timing issues (mostly not in our code), etc and are not actionable. As we prepare to scale up our service, I'm wondering how to continue to get the value from the interesting crash reports without having to look through all of the uninteresting ones. > > I haven't found rb to be very useful for finding the new/interesting crashes. Are there effective ways that peopler are using it? > > Are there other tools for parsing and grouping crash reports to make it easy to find new/interesting ones? > > thanks, > Ransom > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthonym@REDACTED Mon Jun 3 19:34:30 2013 From: anthonym@REDACTED (ANTHONY MOLINARO) Date: Mon, 3 Jun 2013 10:34:30 -0700 Subject: [erlang-questions] Are nested records copied? In-Reply-To: <51AA1974.6030403@go-factory.net> References: <51AA148C.6060505@go-factory.net> <51AA1974.6030403@go-factory.net> Message-ID: <747A58A0-2CEC-42A6-AA1D-CFC7FCE631F9@alumni.caltech.edu> BTW, you'll probably learn a lot by checking out this video from the last Erlang factory http://www.erlang-factory.com/conference/SFBay2013/speakers/ErikStenman as it gives some good examples of what is happening in the heap and stack of a process and talks about term sharing and how it works. -Anthony On Jun 1, 2013, at 8:55 AM, Bryan Hughes wrote: > Thanks Bob - that is what I was thinking/hoping. > > On 6/1/13 8:50 AM, Bob Ippolito wrote: >> It's not copied in this case, nested records aren't flattened or anything like that. Records are just tuples after all. >> >> On Saturday, June 1, 2013, Bryan Hughes wrote: >> Howdy! >> >> I am working on section of code that will be dealing with a large number of records in a relatively tight loop and have a question in how Erlang constructs a nested record. >> >> If I define a person and manager record: >> >> 1> rd(person, {name, age}). >> person >> 2> rd(manager, {person, dept}). >> manager >> >> Then create a person record: >> >> 3> P = #person{name = bryan, age = 45}. >> #person{name = bryan,age = 45} >> >> And then a manager record: >> >> 4> M = #manager{person = P, dept = foo}. >> #manager{person = #person{name = bryan,age = 45},dept = foo} >> >> When the manager record is created, is the nested person record P referenced by the manager record M, or is it copied? >> >> Thanks, >> Bryan >> >> -- >> Bryan Hughes >> Go Factory >> http://www.go-factory.net >> >> "Internet Class, Enterprise Grade" >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mloftis@REDACTED Mon Jun 3 23:22:44 2013 From: mloftis@REDACTED (Michael Loftis) Date: Mon, 3 Jun 2013 14:22:44 -0700 Subject: [erlang-questions] Handling Crash Reports at scale In-Reply-To: References: Message-ID: On Mon, Jun 3, 2013 at 9:42 AM, Max Lapshin wrote: > Never, never, listen never redirect error_logger to email! s/error_logger/any error logging/ I worked at a web host for a number of years. Quick way to completely break your app is to have every error email you. Summaries, fine, but immediate emails every time will bite you! > > I have understood that I'm wrong when I got 28000 of emails in mailbox > and had to write IMAP script. > > Use http://airbrake.io/ for collecting your errors =) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- "Genius might be described as a supreme capacity for getting its possessors into trouble of all kinds." -- Samuel Butler From jleivent@REDACTED Tue Jun 4 03:03:21 2013 From: jleivent@REDACTED (Jonathan Leivent) Date: Mon, 03 Jun 2013 21:03:21 -0400 Subject: [erlang-questions] ets vs. SMP memory effects Message-ID: <51AD3CD9.8090904@comcast.net> After reading the on-line doc on ets, I'm a bit confused about its guarantees. I understand that it guarantees atomicity and isolation for operations on a single object. However, what about a sequence of update operations on different objects performed by the same process? Are the results of these operations guaranteed to be visible to other processes in only the order in which they occur? Or, might SMP-style weakly-consistent memory - or even compiler/hardware operation re-ordering - cause the updates to appear in a different order to other processes? -- Jonathan From jinni.park@REDACTED Tue Jun 4 03:54:59 2013 From: jinni.park@REDACTED (Park, Sungjin) Date: Tue, 4 Jun 2013 10:54:59 +0900 Subject: [erlang-questions] gen_server doesn't respond In-Reply-To: References: Message-ID: Shell input/output logs below. ============================ 4> process_info(pid(0,453,4757)). [{current_function,{*gen,do_call,4*}}, {initial_call,{proc_lib,init_p,5}}, {status,*waiting*}, {message_queue_len,1}, {messages,[*{'$gen_call',{<0.26172.4841>,* * #Ref<0.0.2946.93178>},* * state}*]}, {links,[<0.25337.4839>]}, {dictionary,[{'$ancestors',[<0.452.4757>,<0.424.0>, <0.423.0>,ranch_sup,<0.420.0>]}, {'$initial_call',{my_session,init,1}}]}, {trap_exit,true}, {error_handler,error_handler}, {priority,normal}, {group_leader,<0.419.0>}, {total_heap_size,1364}, {heap_size,987}, {stack_size,32}, {reductions,1487}, {garbage_collection,[{min_bin_vheap_size,46368}, {min_heap_size,233}, {fullsweep_after,65535}, {minor_gcs,8}]}, {suspending,[]}] ============================= As you can see the process has a message {..., state} in it's message queue but still in waiting state. It doesn't respond to gen_server:call(Pid, state) from this point on. The process_info shows that the process is now in gen:do_call/4 which is not under my control. The module has all the fallback handlers for handle_call/3, handle_cast/2, handle_info/2 etc, so that every message be matched. There are a couple of places in my project where gen_server:call/2 used against this process. I suppose that the calls should timeout in 5 sec by default as is stated in the gen_server specification. So there should be no such case that the process stuck indefinitely. On Mon, Jun 3, 2013 at 7:39 PM, Aaron France wrote: > Does it make any synchronous calls to other gen_servers? > > > On Mon, Jun 3, 2013 at 12:34 PM, Tim Watson wrote: > >> Does the callback module execute any selective receives inside handlers >> perhaps? If not, are you sure the message has actually arrived in that >> process' mailbox, can you see unhandled messages in the message queue >> (I.e., length > 0 whilst waiting)? >> >> On 3 Jun 2013, at 10:34, "Park, Sungjin" wrote: >> >> > Guys, >> > >> > I'm encountering cases when gen_server doesn't respond. >> > In such cases, erlang:process_info/1 shows the process state 'waiting' >> with some messages in the message_queue. The messages have not been >> consumed for tens of minutes at least. Anybody has any idean about this? >> > >> > >> > -- >> > Park, Sungjin >> > >> ------------------------------------------------------------------------------------------------------------------- >> > Peculiar travel suggestions are dancing lessons from god. >> > -- The Books of Bokonon >> > >> ------------------------------------------------------------------------------------------------------------------- >> > _______________________________________________ >> > erlang-questions mailing list >> > erlang-questions@REDACTED >> > http://erlang.org/mailman/listinfo/erlang-questions >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue Jun 4 04:09:34 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 4 Jun 2013 14:09:34 +1200 Subject: [erlang-questions] Are nested records copied? In-Reply-To: <51AA148C.6060505@go-factory.net> References: <51AA148C.6060505@go-factory.net> Message-ID: <00B0F046-B61D-4CE9-884F-3F583C5A5F74@cs.otago.ac.nz> On 2/06/2013, at 3:34 AM, Bryan Hughes wrote: > When the manager record is created, is the nested person record P referenced by the manager record M, or is it copied? Just pointed to. A record with N fields takes exactly as much space as a tuple with N+1 elements (because it *is* a tuple with N+1 elements), no matter what those elements are. From ok@REDACTED Tue Jun 4 04:43:13 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 4 Jun 2013 14:43:13 +1200 Subject: [erlang-questions] Literal support for unary (1#XXXXXX...) In-Reply-To: References: Message-ID: On 3/06/2013, at 7:22 AM, Andrew Pennebaker wrote: > To be sure, base 1 isn't used that often in practice. But neither is base 3, 13, or 37, so it seems a little strange why the range of base literals is [2, 36]. If we apply the principle of "least surprise", I think it would be a good idea to also accept base 1. One problem is that base 1 intrinsically and necessarily violates the principle of least surprise. In base n, the legal digits are 0..(n-1). So in base 1 the only legal digit should be 0 (and the only expressible number zero). But *instead* the only legal digit is 1 (and it is dubious whether zero is/should be expressible at all.) To allow zero to be expressible in base 1, perhaps we should take /1#[01]+/ as the syntax for base 1? Aside from "coolness" (hello, Avery!), what's the need for this? From max.lapshin@REDACTED Tue Jun 4 09:32:34 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 4 Jun 2013 11:32:34 +0400 Subject: [erlang-questions] Handling Crash Reports at scale In-Reply-To: References: Message-ID: Ok, what about rienman.io? Does it collapse errors into single email? From jinni.park@REDACTED Tue Jun 4 10:12:00 2013 From: jinni.park@REDACTED (Park, Sungjin) Date: Tue, 4 Jun 2013 17:12:00 +0900 Subject: [erlang-questions] gen_server doesn't respond In-Reply-To: References: Message-ID: The code contains no explicit gen_server:call to other process. There are, - lager:log calls - proc_lib:spawn_link calls - timer:send_after calls - gen_server:cast calls Can any of the above calls cause blocking? ---------- Forwarded message ---------- From: Max Lapshin Date: Tue, Jun 4, 2013 at 12:48 PM Subject: Re: [erlang-questions] gen_server doesn't respond To: "Park, Sungjin" Your process is sleeping in gen_server:call to other process > > Suppose that you are trying to call yourself. -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Tue Jun 4 10:13:53 2013 From: sergej.jurecko@REDACTED (Sergej Jurecko) Date: Tue, 4 Jun 2013 10:13:53 +0200 Subject: [erlang-questions] gen_server doesn't respond In-Reply-To: References: Message-ID: What version of erlang are you using? Are you using any NIFs? Sergej Jurecko On Jun 4, 2013, at 10:12 AM, Park, Sungjin wrote: > The code contains no explicit gen_server:call to other process. > There are, > - lager:log calls > - proc_lib:spawn_link calls > - timer:send_after calls > - gen_server:cast calls > > Can any of the above calls cause blocking? > > ---------- Forwarded message ---------- > From: Max Lapshin > Date: Tue, Jun 4, 2013 at 12:48 PM > Subject: Re: [erlang-questions] gen_server doesn't respond > To: "Park, Sungjin" > > > Your process is sleeping in gen_server:call to other process > Suppose that you are trying to call yourself. > > > > -- > Park, Sungjin > ------------------------------------------------------------------------------------------------------------------- > Peculiar travel suggestions are dancing lessons from god. > -- The Books of Bokonon > ------------------------------------------------------------------------------------------------------------------- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jinni.park@REDACTED Tue Jun 4 10:15:15 2013 From: jinni.park@REDACTED (Park, Sungjin) Date: Tue, 4 Jun 2013 17:15:15 +0900 Subject: [erlang-questions] gen_server doesn't respond In-Reply-To: References: Message-ID: I'm running OTP R15B03 without any NIF. On Tue, Jun 4, 2013 at 5:13 PM, Sergej Jurecko wrote: > What version of erlang are you using? Are you using any NIFs? > > > Sergej Jurecko > > On Jun 4, 2013, at 10:12 AM, Park, Sungjin wrote: > > The code contains no explicit gen_server:call to other process. > There are, > - lager:log calls > - proc_lib:spawn_link calls > - timer:send_after calls > - gen_server:cast calls > > Can any of the above calls cause blocking? > > ---------- Forwarded message ---------- > From: Max Lapshin > Date: Tue, Jun 4, 2013 at 12:48 PM > Subject: Re: [erlang-questions] gen_server doesn't respond > To: "Park, Sungjin" > > > Your process is sleeping in gen_server:call to other process >> >> Suppose that you are trying to call yourself. > > > > -- > Park, Sungjin > > ------------------------------------------------------------------------------------------------------------------- > Peculiar travel suggestions are dancing lessons from god. > -- The Books of Bokonon > > ------------------------------------------------------------------------------------------------------------------- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From jinni.park@REDACTED Tue Jun 4 10:19:45 2013 From: jinni.park@REDACTED (Park, Sungjin) Date: Tue, 4 Jun 2013 17:19:45 +0900 Subject: [erlang-questions] gen_server doesn't respond In-Reply-To: References: Message-ID: OTP R15B03-1 to be precise without any NIF. On Tue, Jun 4, 2013 at 5:15 PM, Park, Sungjin wrote: > I'm running OTP R15B03 without any NIF. > > > On Tue, Jun 4, 2013 at 5:13 PM, Sergej Jurecko wrote: > >> What version of erlang are you using? Are you using any NIFs? >> >> >> Sergej Jurecko >> >> On Jun 4, 2013, at 10:12 AM, Park, Sungjin wrote: >> >> The code contains no explicit gen_server:call to other process. >> There are, >> - lager:log calls >> - proc_lib:spawn_link calls >> - timer:send_after calls >> - gen_server:cast calls >> >> Can any of the above calls cause blocking? >> >> ---------- Forwarded message ---------- >> From: Max Lapshin >> Date: Tue, Jun 4, 2013 at 12:48 PM >> Subject: Re: [erlang-questions] gen_server doesn't respond >> To: "Park, Sungjin" >> >> >> Your process is sleeping in gen_server:call to other process >>> >>> Suppose that you are trying to call yourself. >> >> >> >> -- >> Park, Sungjin >> >> ------------------------------------------------------------------------------------------------------------------- >> Peculiar travel suggestions are dancing lessons from god. >> -- The Books of Bokonon >> >> ------------------------------------------------------------------------------------------------------------------- >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> > > > -- > Park, Sungjin > > ------------------------------------------------------------------------------------------------------------------- > Peculiar travel suggestions are dancing lessons from god. > -- The Books of Bokonon > > ------------------------------------------------------------------------------------------------------------------- > -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Tue Jun 4 10:45:40 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Tue, 4 Jun 2013 11:45:40 +0300 Subject: [erlang-questions] gen_server doesn't respond In-Reply-To: References: Message-ID: <18FB305D-6645-4EB0-91D5-B44A3E0C0FD6@gmail.com> Hello, > {current_function,{gen,do_call,4}} Your process made a call to another process with infinite timeout. The process do not respond because you got a deadlock. You have to check: * no circular dependencies on process call * the parent supervisor is not called from process init * no infinite gen_server:call is used at your api. - Dmitry On Jun 4, 2013, at 4:54 AM, "Park, Sungjin" wrote: > Shell input/output logs below. > > ============================ > 4> process_info(pid(0,453,4757)). > [{current_function,{gen,do_call,4}}, > {initial_call,{proc_lib,init_p,5}}, > {status,waiting}, > {message_queue_len,1}, > {messages,[{'$gen_call',{<0.26172.4841>, > #Ref<0.0.2946.93178>}, > state}]}, > {links,[<0.25337.4839>]}, > {dictionary,[{'$ancestors',[<0.452.4757>,<0.424.0>, > <0.423.0>,ranch_sup,<0.420.0>]}, > {'$initial_call',{my_session,init,1}}]}, > {trap_exit,true}, > {error_handler,error_handler}, > {priority,normal}, > {group_leader,<0.419.0>}, > {total_heap_size,1364}, > {heap_size,987}, > {stack_size,32}, > {reductions,1487}, > {garbage_collection,[{min_bin_vheap_size,46368}, > {min_heap_size,233}, > {fullsweep_after,65535}, > {minor_gcs,8}]}, > {suspending,[]}] > ============================= > > As you can see the process has a message {..., state} in it's message queue but still in waiting state. It doesn't respond to gen_server:call(Pid, state) from this point on. The process_info shows that the process is now in gen:do_call/4 which is not under my control. The module has all the fallback handlers for handle_call/3, handle_cast/2, handle_info/2 etc, so that every message be matched. There are a couple of places in my project where gen_server:call/2 used against this process. I suppose that the calls should timeout in 5 sec by default as is stated in the gen_server specification. So there should be no such case that the process stuck indefinitely. > > > > On Mon, Jun 3, 2013 at 7:39 PM, Aaron France wrote: > Does it make any synchronous calls to other gen_servers? > > > On Mon, Jun 3, 2013 at 12:34 PM, Tim Watson wrote: > Does the callback module execute any selective receives inside handlers perhaps? If not, are you sure the message has actually arrived in that process' mailbox, can you see unhandled messages in the message queue (I.e., length > 0 whilst waiting)? > > On 3 Jun 2013, at 10:34, "Park, Sungjin" wrote: > > > Guys, > > > > I'm encountering cases when gen_server doesn't respond. > > In such cases, erlang:process_info/1 shows the process state 'waiting' with some messages in the message_queue. The messages have not been consumed for tens of minutes at least. Anybody has any idean about this? > > > > > > -- > > Park, Sungjin > > ------------------------------------------------------------------------------------------------------------------- > > Peculiar travel suggestions are dancing lessons from god. > > -- The Books of Bokonon > > ------------------------------------------------------------------------------------------------------------------- > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Park, Sungjin > ------------------------------------------------------------------------------------------------------------------- > Peculiar travel suggestions are dancing lessons from god. > -- The Books of Bokonon > ------------------------------------------------------------------------------------------------------------------- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From attila.r.nohl@REDACTED Tue Jun 4 11:03:58 2013 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Tue, 4 Jun 2013 11:03:58 +0200 Subject: [erlang-questions] gen_server doesn't respond In-Reply-To: References: Message-ID: 2013/6/4 Park, Sungjin : > Shell input/output logs below. > > ============================ > 4> process_info(pid(0,453,4757)). > [{current_function,{gen,do_call,4}}, > {initial_call,{proc_lib,init_p,5}}, > {status,waiting}, > {message_queue_len,1}, > {messages,[{'$gen_call',{<0.26172.4841>, > #Ref<0.0.2946.93178>}, > state}]}, > {links,[<0.25337.4839>]}, > {dictionary,[{'$ancestors',[<0.452.4757>,<0.424.0>, > <0.423.0>,ranch_sup,<0.420.0>]}, > {'$initial_call',{my_session,init,1}}]}, > {trap_exit,true}, > {error_handler,error_handler}, > {priority,normal}, > {group_leader,<0.419.0>}, > {total_heap_size,1364}, > {heap_size,987}, > {stack_size,32}, > {reductions,1487}, > {garbage_collection,[{min_bin_vheap_size,46368}, > {min_heap_size,233}, > {fullsweep_after,65535}, > {minor_gcs,8}]}, > {suspending,[]}] > ============================= > > As you can see the process has a message {..., state} in it's message queue > but still in waiting state. It doesn't respond to gen_server:call(Pid, > state) from this point on. The process_info shows that the process is now > in gen:do_call/4 which is not under my control. The module has all the > fallback handlers for handle_call/3, handle_cast/2, handle_info/2 etc, so > that every message be matched. There are a couple of places in my project > where gen_server:call/2 used against this process. I suppose that the calls > should timeout in 5 sec by default as is stated in the gen_server > specification. So there should be no such case that the process stuck > indefinitely. Check process_info(Pid, backtrace) to see that which gen_server:call (in your gen_server) is blocked. Then check that gen_server, maybe that calls back to your gen_server or something like that. From max.lapshin@REDACTED Tue Jun 4 11:05:16 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 4 Jun 2013 13:05:16 +0400 Subject: [erlang-questions] gen_server doesn't respond In-Reply-To: <18FB305D-6645-4EB0-91D5-B44A3E0C0FD6@gmail.com> References: <18FB305D-6645-4EB0-91D5-B44A3E0C0FD6@gmail.com> Message-ID: Well, you show your output and it tells us that your process is deadlocked in some remote call. You can show here your code and I'm sure that after 5 seconds anyone will show you at least one place where this call is originated from. From ulf@REDACTED Tue Jun 4 11:18:41 2013 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 4 Jun 2013 11:18:41 +0200 Subject: [erlang-questions] ANN: gproc_pool + some performance tidbits Message-ID: I pushed a new gproc component called gproc_pool the other day. The main idea, apart from wanting to see how well it would work, was that I wanted to be able to register servers with gproc and then have an efficient way of pooling between them. A benefit of using gproc throughout is that the registration objects serve as a 'footprint' for each process - by listing the gproc entities for each process, you can tell a lot about its purpose. The way gproc_pool works is that: 1. You define a pool, by naming it, and optionally specifying its size (gproc_pool:new(Pool) | gproc_pool:new(Pool, Type, Options)) 2. You add worker names to the pool (gproc_pool:add_worker(Pool, Name)) 3. Your servers each connect to a given name (gproc_pool:connect_worker(Pool, Name)) 4. Users pick a worker for each request (gproc_pool:pick(Pool)) My little test code indicates that the different load-balancing strategies perform a bit differently: (https://github.com/uwiger/gproc/blob/master/src/gproc_pool.erl#L843) (Create a pool, add 6 workers and iterate 100k times, incrementing a gproc counter for each iteration.) 3> gproc_pool:test(100000,round_robin,[]). worker stats (848): [{a,16667},{b,16667},{c,16667},{d,16667},{e,16666},{f,16666}] {2801884,ok} 4> gproc_pool:test(100000,hash,[]). worker stats (848): [{a,16744},{b,16716},{c,16548},{d,16594},{e,16749},{f,16649}] {1891517,ok} 5> gproc_pool:test(100000,random,[]). worker stats (848): [{a,16565},{b,16542},{c,16613},{d,16872},{e,16727},{f,16681}] {3701011,ok} 6> gproc_pool:test(100000,direct,[]). worker stats (848): [{a,16667},{b,16667},{c,16667},{d,16667},{e,16666},{f,16666}] {1766639,ok} 11> gproc_pool:test(100000,claim,[]). worker stats (848): [{a,100000},{b,0},{c,0},{d,0},{e,0},{f,0}] {7569425,ok} The worker stats show how evenly the workers were selected, and the {Time, ok} comes from timer:tc/3, i.e. Time/100000 is the per-iteration cost: round_robin: 28 usec (maintain a 'current' counter, modulo Size) hash: 19 usec (gproc_pool:pick(Pool, Val), hash on Val) random: 37 usec (pick a random worker, using crypto:rand_uniform/2) direct: 18 usec (gproc_pool:pick(Pool, N), where N modulo Size selects worker) claim: 76 usec (claim the first available worker, apply a fun, then release) I think the per-selection cost is acceptable as-is, but could perhaps be improved (esp. the 'random' strategy is surprisingly expensive). All the selection work is done in the caller's process, BTW - no communication with the gproc or gproc_pool servers (except for admin tasks). The 'claim' strategy is also surprisingly expensive. I believe it's because I'm using gproc:select/3 to find the first free worker. Note also that it results in an extremely uneven distribution. That's obviously because the test run claims the first available worker and then releases it before iterating - it's always going to select the first worker.) https://github.com/uwiger/gproc/blob/master/doc/gproc_pool.md Feedback welcome, be it with performance tips, usability tips, or other. BR, Ulf W Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From kenneth.lundin@REDACTED Tue Jun 4 13:52:19 2013 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Tue, 4 Jun 2013 13:52:19 +0200 Subject: [erlang-questions] ets vs. SMP memory effects In-Reply-To: <51AD3CD9.8090904@comcast.net> References: <51AD3CD9.8090904@comcast.net> Message-ID: ETS tables can be seen as global data within the Erlang VM. If a process makes updates A, B and C, the updates will become visible in that order for other processes. Of course other processes might come in and see the updates in between A, B, C as well. There is no transaction support in ETS where several updates can be done as one atomic operation. To achieve that you need to use Mnesia. /Kenneth Erlang/OTP Ericsson On Tue, Jun 4, 2013 at 3:03 AM, Jonathan Leivent wrote: > After reading the on-line doc on ets, I'm a bit confused about its > guarantees. I understand that it guarantees atomicity and isolation for > operations on a single object. However, what about a sequence of update > operations on different objects performed by the same process? Are the > results of these operations guaranteed to be visible to other processes in > only the order in which they occur? Or, might SMP-style weakly-consistent > memory - or even compiler/hardware operation re-ordering - cause the > updates to appear in a different order to other processes? > > -- Jonathan > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peerst@REDACTED Tue Jun 4 13:48:04 2013 From: peerst@REDACTED (Peer Stritzinger) Date: Tue, 4 Jun 2013 13:48:04 +0200 Subject: [erlang-questions] Handling Crash Reports at scale References: <24F2ADC1-368D-4856-8BAB-AC5660F3033F@alumni.caltech.edu> Message-ID: I'm not sure what you mean by defensive code, but just to be sure: You should never let error reporting influence how you handle faults in your software. Defensive code is a code smell in Erlang. What you coucld do is to have a error logger handler that matches certain often occuring errors and ignores it. Even better it should count them (the count of something unineresting is often interesting). For counting and other metrics you might want to have a look at folsom https://github.com/boundary/folsom -- Peer On 2013-06-03 17:29:54 +0000, ANTHONY MOLINARO said: > I'd recommend addressing the other crashes with "defensive code". ?By > capturing and categorizing certain types of errors and clearing those > from your logs you'll be able to better see true errors. ?I usually > have a two phase approach. ?First phase, I capture and log the types of > errors. ?Second, once I see certain types ?are regular I replace those > with metrics (via mondemand). ?Once you have a stream of errors you can > then monitor the rates, plus uncaught errors will make it into your > logs which should remain very sparse. > > -Anthony > > On Jun 3, 2013, at 9:11 AM, Ransom Richardson wrote: > Are there tools/procedures that are recommended for processing crash > reports from a service running at scale? > > Currently we have a limited deployment and I look through all of the > crash reports by hand. Some are very useful for finding actual bugs in > our code. But other crashes are the result of client's sending bad > data, strange timing issues (mostly not in our code), etc and are not > actionable. As we prepare to scale up our service, I'm wondering how to > continue to get the value from the interesting crash reports without > having to look through all of the uninteresting ones.? > > I haven't found rb to be very useful for finding the new/interesting > crashes. Are there effective ways that peopler are using it? > > Are there other tools for parsing and grouping crash reports to make it > easy to find new/interesting ones? > > thanks, > Ransom > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jinni.park@REDACTED Tue Jun 4 13:57:30 2013 From: jinni.park@REDACTED (Park, Sungjin) Date: Tue, 4 Jun 2013 20:57:30 +0900 Subject: [erlang-questions] gen_server doesn't respond In-Reply-To: References: <18FB305D-6645-4EB0-91D5-B44A3E0C0FD6@gmail.com> Message-ID: Ok, I'll prepare to share the code in some way. In the meantime, just to get some hint can any of below be potentially block or synchronous at least to make the system very slow? - lager:log - proc_lib:spawn_link - timer:send_after - gen_server:cast - erlang:garbage_collect -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Tue Jun 4 14:11:10 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 4 Jun 2013 16:11:10 +0400 Subject: [erlang-questions] gen_server doesn't respond In-Reply-To: References: <18FB305D-6645-4EB0-91D5-B44A3E0C0FD6@gmail.com> Message-ID: No. You are missing something else in _your_ code. The problem is in it. While you will "prepare" your code, you will loose it. From jinni.park@REDACTED Tue Jun 4 14:27:04 2013 From: jinni.park@REDACTED (Park, Sungjin) Date: Tue, 4 Jun 2013 21:27:04 +0900 Subject: [erlang-questions] gen_server doesn't respond In-Reply-To: References: <18FB305D-6645-4EB0-91D5-B44A3E0C0FD6@gmail.com> Message-ID: Thanks for your concern Max. I might have missed some points in the explanation though. There were more processes not responding when it happened. Actually most of the processes across all 4 nodes in the cluster got stuck including ranch_conns_sup etc. etc. Strange enough, the system top had been decent like under 10% both cpu and memory. I also think that I might be too lucky to get exact answer by sharing just this amount of information. I just want to get some hint from rough ideas or similar experiences. Thanks. On Tue, Jun 4, 2013 at 9:11 PM, Max Lapshin wrote: > No. You are missing something else in _your_ code. The problem is in it. > > While you will "prepare" your code, you will loose it. > -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergej.jurecko@REDACTED Tue Jun 4 14:30:19 2013 From: sergej.jurecko@REDACTED (Sergej Jurecko) Date: Tue, 4 Jun 2013 14:30:19 +0200 Subject: [erlang-questions] gen_server doesn't respond In-Reply-To: References: <18FB305D-6645-4EB0-91D5-B44A3E0C0FD6@gmail.com> Message-ID: Are you absolutely sure you don't use any NIFs? What applications do you use besides your own? Because it sounds like the scheduler locks that NIFs cause. Or are you dependent on hard drive performance maybe? Erlang might have been waiting for I/O. Sergej On Jun 4, 2013, at 2:27 PM, Park, Sungjin wrote: > Thanks for your concern Max. > > I might have missed some points in the explanation though. > There were more processes not responding when it happened. > Actually most of the processes across all 4 nodes in the cluster got stuck including ranch_conns_sup etc. etc. Strange enough, the system top had been decent like under 10% both cpu and memory. I also think that I might be too lucky to get exact answer by sharing just this amount of information. I just want to get some hint from rough ideas or similar experiences. Thanks. > > > > On Tue, Jun 4, 2013 at 9:11 PM, Max Lapshin wrote: > No. You are missing something else in _your_ code. The problem is in it. > > While you will "prepare" your code, you will loose it. > > > > -- > Park, Sungjin > ------------------------------------------------------------------------------------------------------------------- > Peculiar travel suggestions are dancing lessons from god. > -- The Books of Bokonon > ------------------------------------------------------------------------------------------------------------------- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From garry@REDACTED Tue Jun 4 15:08:49 2013 From: garry@REDACTED (Garry Hodgson) Date: Tue, 04 Jun 2013 09:08:49 -0400 Subject: [erlang-questions] low level packet access from erlang In-Reply-To: <20130403212407.GA25006@ioctl> References: <515C6E13.1050406@research.att.com> <20130403212407.GA25006@ioctl> Message-ID: <51ADE6E1.4030205@research.att.com> On 04/03/2013 05:24 PM, Michael Santos wrote: > > I wrote an Erlang tun/tap interface on top of procket: > > https://github.com/msantos/tunctl i've been playing with tunctl, but can't seem to get it past initial eperm problems in create(). i'm guessing i missed some kind of setup step, but i don't know what. i set the capabilities as per instructions: --> getcap /usr/local/lib/erlang/erts-5.9.2/bin/beam.smp /usr/local/lib/erlang/erts-5.9.2/bin/beam /usr/local/lib/erlang/erts-5.9.2/bin/beam.smp = cap_net_admin+ep /usr/local/lib/erlang/erts-5.9.2/bin/beam = cap_net_admin+ep running as root, when i try tuncer:create() with or without args, i get: (r3@REDACTED)2> tuncer:create( <<"tun0">> ). ** exception exit: {badmatch,{error,eperm}} in function tuncer:init/1 in call from gen_server:init_it/6 (gen_server.erl, line 304) in call from proc_lib:init_p_do_apply/3 (proc_lib.erl, line 227) (r3@REDACTED)3> =CRASH REPORT==== 4-Jun-2013::08:28:13 === crasher: initial call: tuncer:init/1 pid: <0.64.0> registered_name: [] exception exit: {{badmatch,{error,eperm}}, [{tuncer,init,1,[]}, {gen_server,init_it,6, any idea what i may be missing? -- Garry Hodgson AT&T Chief Security Office (CSO) "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." From garry@REDACTED Tue Jun 4 15:10:05 2013 From: garry@REDACTED (Garry Hodgson) Date: Tue, 04 Jun 2013 09:10:05 -0400 Subject: [erlang-questions] low level packet access from erlang In-Reply-To: <51ADE6E1.4030205@research.att.com> References: <515C6E13.1050406@research.att.com> <20130403212407.GA25006@ioctl> <51ADE6E1.4030205@research.att.com> Message-ID: <51ADE72D.8090309@research.att.com> forgot to mention, i'm running erlang R15B02 on CentOs 6.2 On 06/04/2013 09:08 AM, Garry Hodgson wrote: > On 04/03/2013 05:24 PM, Michael Santos wrote: >> >> I wrote an Erlang tun/tap interface on top of procket: >> >> https://github.com/msantos/tunctl > i've been playing with tunctl, but can't seem to get > it past initial eperm problems in create(). i'm guessing > i missed some kind of setup step, but i don't know what. > > i set the capabilities as per instructions: > > --> getcap /usr/local/lib/erlang/erts-5.9.2/bin/beam.smp > /usr/local/lib/erlang/erts-5.9.2/bin/beam > /usr/local/lib/erlang/erts-5.9.2/bin/beam.smp = cap_net_admin+ep > /usr/local/lib/erlang/erts-5.9.2/bin/beam = cap_net_admin+ep > > running as root, when i try tuncer:create() with or without args, i get: > > (r3@REDACTED)2> tuncer:create( <<"tun0">> ). > ** exception exit: {badmatch,{error,eperm}} > in function tuncer:init/1 > in call from gen_server:init_it/6 (gen_server.erl, line 304) > in call from proc_lib:init_p_do_apply/3 (proc_lib.erl, line 227) > (r3@REDACTED)3> > =CRASH REPORT==== 4-Jun-2013::08:28:13 === > crasher: > initial call: tuncer:init/1 > pid: <0.64.0> > registered_name: [] > exception exit: {{badmatch,{error,eperm}}, > [{tuncer,init,1,[]}, > {gen_server,init_it,6, > > any idea what i may be missing? > -- Garry Hodgson AT&T Chief Security Office (CSO) "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." From jesper.louis.andersen@REDACTED Tue Jun 4 15:15:24 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 4 Jun 2013 15:15:24 +0200 Subject: [erlang-questions] Handling Crash Reports at scale In-Reply-To: References: <24F2ADC1-368D-4856-8BAB-AC5660F3033F@alumni.caltech.edu> Message-ID: <041DF843-551C-4BBF-8FA1-D2DD84D81C92@erlang-solutions.com> I often start out by writing the code naively. The naive way of writing code is to assume that everything goes fine and there are nothing that could go wrong. Then the first crash happens and you go into the code with the knowledge of "this can happen, what do I wanna do?". In many cases the error is on the callers side, so just return a correct error back to him. In some cases you need to fix your own code because your naive approach didn't work. And so on. Fixing this error cleans up the error log for that case. In effect, you are building your software on the good path only and then you sporadically handle the few error cases that will occur in practice. The code you add is only defensive if you are in doubt it will ever get executed. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen On Jun 4, 2013, at 1:48 PM, Peer Stritzinger wrote: > I'm not sure what you mean by defensive code, but just to be sure: > > You should never let error reporting influence how you handle faults in your software. Defensive code is a code smell in Erlang. > > What you coucld do is to have a error logger handler that matches certain often occuring errors and ignores it. Even better it should count them (the count of something unineresting is often interesting). > > For counting and other metrics you might want to have a look at folsom https://github.com/boundary/folsom > > -- Peer > > > On 2013-06-03 17:29:54 +0000, ANTHONY MOLINARO said: > > I'd recommend addressing the other crashes with "defensive code". By capturing and categorizing certain types of errors and clearing those from your logs you'll be able to better see true errors. I usually have a two phase approach. First phase, I capture and log the types of errors. Second, once I see certain types are regular I replace those with metrics (via mondemand). Once you have a stream of errors you can then monitor the rates, plus uncaught errors will make it into your logs which should remain very sparse. > > -Anthony > > On Jun 3, 2013, at 9:11 AM, Ransom Richardson wrote: > Are there tools/procedures that are recommended for processing crash reports from a service running at scale? > > Currently we have a limited deployment and I look through all of the crash reports by hand. Some are very useful for finding actual bugs in our code. But other crashes are the result of client's sending bad data, strange timing issues (mostly not in our code), etc and are not actionable. As we prepare to scale up our service, I'm wondering how to continue to get the value from the interesting crash reports without having to look through all of the uninteresting ones. > > I haven't found rb to be very useful for finding the new/interesting crashes. Are there effective ways that peopler are using it? > > Are there other tools for parsing and grouping crash reports to make it easy to find new/interesting ones? > > thanks, > Ransom > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From eric.boyer@REDACTED Tue Jun 4 15:34:09 2013 From: eric.boyer@REDACTED (Eric Boyer) Date: Tue, 4 Jun 2013 09:34:09 -0400 Subject: [erlang-questions] SSL Errors with R16B In-Reply-To: References: <323c2652-d8de-42b9-b267-7f1f2d853b27@googlegroups.com> Message-ID: Aaron, You might want to make sure that you have the ASN.1 module code loaded. I have had an issue before where my build didn't include it and SSL would give those tls_alert issues. R15B worked fine, but when it wasn't in R16 it would give errors. Eric On Fri, May 31, 2013 at 5:48 AM, Aaron France wrote: > Hi, > > The code works fine on 15B, I know how to use the post. Please read what I > am typing :) > > When I say "it doesn't work", I of course mean the error in the OP. > > Regards, > Aaron > > > On Fri, May 31, 2013 at 11:47 AM, Bogdan Andu wrote: > >> what does tit mean "it doesn't work for me"? >> >> you can POST a Body of data to an Url like this: >> >> httpc:request(post, {" >> https://region-b.geo-1.identity.hpcloudsvc.com:35357/", >> [{"Content-Length", ContentLength}], >> ContentType, >> Body}, >> [], []). >> >> you have to make the Body and compute the length of the Body yourself. >> >> Best Regards and good luck >> >> Bogdan >> >> >> On Fri, May 31, 2013 at 12:30 PM, Aaron France wrote: >> >>> Right, but it doesn't work for me. It's also an endpoint where you need >>> to post/get w/ specific bodies, perhaps that's getting in the way? >>> >>> >>> On Fri, May 31, 2013 at 11:25 AM, Bogdan Andu wrote: >>> >>>> it works! >>>> >>>> >>>> On Fri, May 31, 2013 at 12:17 PM, Aaron France < >>>> aaron.l.france@REDACTED> wrote: >>>> >>>>> What's this telling you? >>>>> >>>>> >>>>> On Fri, May 31, 2013 at 11:02 AM, Bogdan Andu wrote: >>>>> >>>>>> >>>>>> Tried this: >>>>>> >>>>>> $ erl >>>>>> Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:2:2] >>>>>> [async-threads:10] [kernel-poll:false] >>>>>> >>>>>> >>>>>> Eshell V5.10.1 (abort with ^G) >>>>>> 1> inest:start(). >>>>>> ** exception error: undefined function inest:start/0 >>>>>> 2> inets:start(). >>>>>> ok >>>>>> 3> ssl:start(). >>>>>> ok >>>>>> 4> httpc:request(" >>>>>> http://region-b.geo-1.identity.hpcloudsvc.com:35357/"). >>>>>> {error,socket_closed_remotely} >>>>>> 5> httpc:request(" >>>>>> https://region-b.geo-1.identity.hpcloudsvc.com:35357/"). >>>>>> >>>>>> {ok,{{"HTTP/1.1",200,"OK"}, >>>>>> [{"cache-control","no-cache"}, >>>>>> {"date","Fri, 31 May 2013 08:59:44 GMT"}, >>>>>> >>>>>> {"pragma","no-cache"}, >>>>>> {"server","Apache-Coyote/1.1"}, >>>>>> {"content-length","1080"}, >>>>>> {"content-type","application/json"}, >>>>>> {"expires","-1"}], >>>>>> "{\n \"versions\" : {\n \"otherAttributes\" : {\n },\n >>>>>> \"version\" : [ {\n \"id\" : \"v1.0\",\n \"otherAttributes\" : >>>>>> {\n },\n \"status\" : \"DEPRECATED\",\n \"updated\" : >>>>>> \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : \"v1.1\",\n >>>>>> \"link\" : [ {\n \"href\" : \" >>>>>> https://docs.hpcloud.com/api/identity\",\n \"rel\" : >>>>>> \"describedby\",\n \"type\" : \"text/html\"\n } ],\n >>>>>> \"otherAttributes\" : {\n },\n \"status\" : \"CURRENT\",\n >>>>>> \"updated\" : \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : >>>>>> \"v2.0\",\n \"link\" : [ {\n \"href\" : \" >>>>>> https://docs.hpcloud.com/api/identity\",\n \"rel\" : >>>>>> \"describedby\",\n \"type\" : \"text/html\"\n } ],\n >>>>>> \"otherAttributes\" : {\n },\n \"status\" : \"CURRENT\",\n >>>>>> \"updated\" : \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : >>>>>> \"v3.0\",\n \"link\" : [ {\n \"href\" : \" >>>>>> https://docs.hpcloud.com/api/identity\",\n \"rel\" : >>>>>> \"describedby\",\n \"type\" : \"text/html\"\n } ],\n >>>>>> \"otherAttributes\" : {\n },\n \"status\" : \"BETA\",\n >>>>>> \"updated\" : \"2013-04-25T08:59:45.110Z\"\n } ]\n }\n}"}} >>>>>> 6> >>>>>> >>>>>> >>>>>> >>>>>> On Fri, May 31, 2013 at 10:28 AM, Aaron France < >>>>>> aaron.l.france@REDACTED> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I'm using httpc, so I tried: httpc:set_options([{socket, >>>>>>> {verify, verify_none}}]). and I still receive the same error. >>>>>>> >>>>>>> That's correct? If not; let me know. Thanks! >>>>>>> >>>>>>> Regards, >>>>>>> Aaron >>>>>>> >>>>>>> >>>>>>> On Fri, May 31, 2013 at 9:19 AM, Bogdan Andu wrote: >>>>>>> >>>>>>>> may be the ssl stack changed a bit and some defaults parameters. >>>>>>>> >>>>>>>> you may try to pass the ssl option {verify, verify_none} to >>>>>>>> ssl:connect function and see what happens. >>>>>>>> >>>>>>>> Bogdan >>>>>>>> >>>>>>>> >>>>>>>> On Thu, May 30, 2013 at 10:17 PM, Aaron France < >>>>>>>> aaron.l.france@REDACTED> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> But why is it working on R15B? Is the ssl code basically broken on >>>>>>>>> R15B? >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, May 30, 2013 at 9:12 PM, wrote: >>>>>>>>> >>>>>>>>>> Hello, >>>>>>>>>> >>>>>>>>>> verify the output of the command: >>>>>>>>>> >>>>>>>>>> $ openssl s_client -connect >>>>>>>>>> region-b.geo-1.identity.hpcloudsvc.com:35357 >>>>>>>>>> ogdan >>>>>>>>>> it seems that there is a certificate self signed. >>>>>>>>>> >>>>>>>>>> May be a solution is to disable certificate verification on peer >>>>>>>>>> and see what happens. >>>>>>>>>> >>>>>>>>>> Bogdan >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thursday, May 30, 2013 5:43:04 PM UTC+3, Aaron France wrote: >>>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> I'm trying to make HTTPS calls to an API. >>>>>>>>>>> >>>>>>>>>>> My stack trace is: >>>>>>>>>>> >>>>>>>>>>> SSL: certify: ssl_handshake.erl:263:Fatal error: certificate >>>>>>>>>>> unknown >>>>>>>>>>> ** exception error: no match of right hand side value >>>>>>>>>>> {error, >>>>>>>>>>> {failed_connect, >>>>>>>>>>> [{to_address, >>>>>>>>>>> {"region-b.geo-1.identity.** >>>>>>>>>>> hpcloudsvc.com >>>>>>>>>>> ",35357}}, >>>>>>>>>>> {inet,[inet],{tls_alert,"**certificate >>>>>>>>>>> unknown"}}]}} >>>>>>>>>>> in function herp_identity:login/3 (src/herp_identity.erl, >>>>>>>>>>> line 9) >>>>>>>>>>> >>>>>>>>>>> I've looked at some other issues which point it to being a >>>>>>>>>>> regression in R16B and I've reverted to R15 and the issue is non-existent. >>>>>>>>>>> >>>>>>>>>>> Any tips on getting around this or will it require a patch in >>>>>>>>>>> R16B? The bug is still evident in HEAD so possibly not enough data has been >>>>>>>>>>> gathered about it. >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Aaron >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> erlang-questions mailing list >>>>>>>>>> erlang-questions@REDACTED >>>>>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From puzza007@REDACTED Tue Jun 4 15:35:30 2013 From: puzza007@REDACTED (Paul Oliver) Date: Tue, 4 Jun 2013 09:35:30 -0400 Subject: [erlang-questions] Handling Crash Reports at scale In-Reply-To: References: Message-ID: Yeah, it can. It's basically a stream processor for service events. Very flexible. It's worth watching the Monitorama talk on the website. On Tue, Jun 4, 2013 at 3:32 AM, Max Lapshin wrote: > Ok, what about rienman.io? Does it collapse errors into single email? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aaron.l.france@REDACTED Tue Jun 4 15:57:57 2013 From: aaron.l.france@REDACTED (Aaron France) Date: Tue, 4 Jun 2013 14:57:57 +0100 Subject: [erlang-questions] SSL Errors with R16B In-Reply-To: References: <323c2652-d8de-42b9-b267-7f1f2d853b27@googlegroups.com> Message-ID: Hi, Yes, sorry if you didn't see the above e-mails but it turns out I didn't have the asn1 module in my release and thus it wasn't loaded. I must ask; is there a better way to understand what applications I need to include before getting into situations like this? Regards, Aaron On Tue, Jun 4, 2013 at 2:34 PM, Eric Boyer wrote: > Aaron, > > You might want to make sure that you have the ASN.1 module code loaded. I > have had an issue before where my build didn't include it and SSL would > give those tls_alert issues. R15B worked fine, but when it wasn't in R16 it > would give errors. > > Eric > > > On Fri, May 31, 2013 at 5:48 AM, Aaron France wrote: > >> Hi, >> >> The code works fine on 15B, I know how to use the post. Please read what >> I am typing :) >> >> When I say "it doesn't work", I of course mean the error in the OP. >> >> Regards, >> Aaron >> >> >> On Fri, May 31, 2013 at 11:47 AM, Bogdan Andu wrote: >> >>> what does tit mean "it doesn't work for me"? >>> >>> you can POST a Body of data to an Url like this: >>> >>> httpc:request(post, {" >>> https://region-b.geo-1.identity.hpcloudsvc.com:35357/", >>> [{"Content-Length", ContentLength}], >>> ContentType, >>> Body}, >>> [], []). >>> >>> you have to make the Body and compute the length of the Body yourself. >>> >>> Best Regards and good luck >>> >>> Bogdan >>> >>> >>> On Fri, May 31, 2013 at 12:30 PM, Aaron France >> > wrote: >>> >>>> Right, but it doesn't work for me. It's also an endpoint where you need >>>> to post/get w/ specific bodies, perhaps that's getting in the way? >>>> >>>> >>>> On Fri, May 31, 2013 at 11:25 AM, Bogdan Andu wrote: >>>> >>>>> it works! >>>>> >>>>> >>>>> On Fri, May 31, 2013 at 12:17 PM, Aaron France < >>>>> aaron.l.france@REDACTED> wrote: >>>>> >>>>>> What's this telling you? >>>>>> >>>>>> >>>>>> On Fri, May 31, 2013 at 11:02 AM, Bogdan Andu wrote: >>>>>> >>>>>>> >>>>>>> Tried this: >>>>>>> >>>>>>> $ erl >>>>>>> Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:2:2] >>>>>>> [async-threads:10] [kernel-poll:false] >>>>>>> >>>>>>> >>>>>>> Eshell V5.10.1 (abort with ^G) >>>>>>> 1> inest:start(). >>>>>>> ** exception error: undefined function inest:start/0 >>>>>>> 2> inets:start(). >>>>>>> ok >>>>>>> 3> ssl:start(). >>>>>>> ok >>>>>>> 4> httpc:request(" >>>>>>> http://region-b.geo-1.identity.hpcloudsvc.com:35357/"). >>>>>>> {error,socket_closed_remotely} >>>>>>> 5> httpc:request(" >>>>>>> https://region-b.geo-1.identity.hpcloudsvc.com:35357/"). >>>>>>> >>>>>>> {ok,{{"HTTP/1.1",200,"OK"}, >>>>>>> [{"cache-control","no-cache"}, >>>>>>> {"date","Fri, 31 May 2013 08:59:44 GMT"}, >>>>>>> >>>>>>> {"pragma","no-cache"}, >>>>>>> {"server","Apache-Coyote/1.1"}, >>>>>>> {"content-length","1080"}, >>>>>>> {"content-type","application/json"}, >>>>>>> {"expires","-1"}], >>>>>>> "{\n \"versions\" : {\n \"otherAttributes\" : {\n >>>>>>> },\n \"version\" : [ {\n \"id\" : \"v1.0\",\n >>>>>>> \"otherAttributes\" : {\n },\n \"status\" : >>>>>>> \"DEPRECATED\",\n \"updated\" : \"2012-06-20T08:59:45.110Z\"\n }, >>>>>>> {\n \"id\" : \"v1.1\",\n \"link\" : [ {\n \"href\" : \" >>>>>>> https://docs.hpcloud.com/api/identity\",\n \"rel\" : >>>>>>> \"describedby\",\n \"type\" : \"text/html\"\n } ],\n >>>>>>> \"otherAttributes\" : {\n },\n \"status\" : \"CURRENT\",\n >>>>>>> \"updated\" : \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : >>>>>>> \"v2.0\",\n \"link\" : [ {\n \"href\" : \" >>>>>>> https://docs.hpcloud.com/api/identity\",\n \"rel\" : >>>>>>> \"describedby\",\n \"type\" : \"text/html\"\n } ],\n >>>>>>> \"otherAttributes\" : {\n },\n \"status\" : \"CURRENT\",\n >>>>>>> \"updated\" : \"2012-06-20T08:59:45.110Z\"\n }, {\n \"id\" : >>>>>>> \"v3.0\",\n \"link\" : [ {\n \"href\" : \" >>>>>>> https://docs.hpcloud.com/api/identity\",\n \"rel\" : >>>>>>> \"describedby\",\n \"type\" : \"text/html\"\n } ],\n >>>>>>> \"otherAttributes\" : {\n },\n \"status\" : \"BETA\",\n >>>>>>> \"updated\" : \"2013-04-25T08:59:45.110Z\"\n } ]\n }\n}"}} >>>>>>> 6> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, May 31, 2013 at 10:28 AM, Aaron France < >>>>>>> aaron.l.france@REDACTED> wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I'm using httpc, so I tried: httpc:set_options([{socket, >>>>>>>> {verify, verify_none}}]). and I still receive the same error. >>>>>>>> >>>>>>>> That's correct? If not; let me know. Thanks! >>>>>>>> >>>>>>>> Regards, >>>>>>>> Aaron >>>>>>>> >>>>>>>> >>>>>>>> On Fri, May 31, 2013 at 9:19 AM, Bogdan Andu wrote: >>>>>>>> >>>>>>>>> may be the ssl stack changed a bit and some defaults parameters. >>>>>>>>> >>>>>>>>> you may try to pass the ssl option {verify, verify_none} to >>>>>>>>> ssl:connect function and see what happens. >>>>>>>>> >>>>>>>>> Bogdan >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, May 30, 2013 at 10:17 PM, Aaron France < >>>>>>>>> aaron.l.france@REDACTED> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> But why is it working on R15B? Is the ssl code basically broken >>>>>>>>>> on R15B? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Thu, May 30, 2013 at 9:12 PM, wrote: >>>>>>>>>> >>>>>>>>>>> Hello, >>>>>>>>>>> >>>>>>>>>>> verify the output of the command: >>>>>>>>>>> >>>>>>>>>>> $ openssl s_client -connect >>>>>>>>>>> region-b.geo-1.identity.hpcloudsvc.com:35357 >>>>>>>>>>> ogdan >>>>>>>>>>> it seems that there is a certificate self signed. >>>>>>>>>>> >>>>>>>>>>> May be a solution is to disable certificate verification on peer >>>>>>>>>>> and see what happens. >>>>>>>>>>> >>>>>>>>>>> Bogdan >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Thursday, May 30, 2013 5:43:04 PM UTC+3, Aaron France wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> I'm trying to make HTTPS calls to an API. >>>>>>>>>>>> >>>>>>>>>>>> My stack trace is: >>>>>>>>>>>> >>>>>>>>>>>> SSL: certify: ssl_handshake.erl:263:Fatal error: certificate >>>>>>>>>>>> unknown >>>>>>>>>>>> ** exception error: no match of right hand side value >>>>>>>>>>>> {error, >>>>>>>>>>>> {failed_connect, >>>>>>>>>>>> [{to_address, >>>>>>>>>>>> {"region-b.geo-1.identity.** >>>>>>>>>>>> hpcloudsvc.com >>>>>>>>>>>> ",35357}}, >>>>>>>>>>>> {inet,[inet],{tls_alert,"**certificate >>>>>>>>>>>> unknown"}}]}} >>>>>>>>>>>> in function herp_identity:login/3 (src/herp_identity.erl, >>>>>>>>>>>> line 9) >>>>>>>>>>>> >>>>>>>>>>>> I've looked at some other issues which point it to being a >>>>>>>>>>>> regression in R16B and I've reverted to R15 and the issue is non-existent. >>>>>>>>>>>> >>>>>>>>>>>> Any tips on getting around this or will it require a patch in >>>>>>>>>>>> R16B? The bug is still evident in HEAD so possibly not enough data has been >>>>>>>>>>>> gathered about it. >>>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> Aaron >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> erlang-questions mailing list >>>>>>>>>>> erlang-questions@REDACTED >>>>>>>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Tue Jun 4 16:27:25 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 4 Jun 2013 18:27:25 +0400 Subject: [erlang-questions] Handling Crash Reports at scale In-Reply-To: References: Message-ID: It is important to remember that remote user is usually waiting for something like http 500 or 403, not just closing socket. So, just "let-it-crash" is not always suitable way to write software. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jinni.park@REDACTED Tue Jun 4 16:27:35 2013 From: jinni.park@REDACTED (Sungjin Park) Date: Tue, 4 Jun 2013 23:27:35 +0900 Subject: [erlang-questions] gen_server doesn't respond In-Reply-To: References: <18FB305D-6645-4EB0-91D5-B44A3E0C0FD6@gmail.com> Message-ID: I'm using ranch, lager and goldrush due to lager's dependency but no nif of my own. The system leaves quite a lot of logs, approx one 100mb file every 5min or so. Each server has 200-300k tcp connections. R15B03-1 running on top of centos 5.2 maybe. Sent from my iPad On 2013. 6. 4., at ?? 9:30, Sergej Jurecko wrote: > Are you absolutely sure you don't use any NIFs? What applications do you use besides your own? Because it sounds like the scheduler locks that NIFs cause. > Or are you dependent on hard drive performance maybe? Erlang might have been waiting for I/O. > > > Sergej > > On Jun 4, 2013, at 2:27 PM, Park, Sungjin wrote: > >> Thanks for your concern Max. >> >> I might have missed some points in the explanation though. >> There were more processes not responding when it happened. >> Actually most of the processes across all 4 nodes in the cluster got stuck including ranch_conns_sup etc. etc. Strange enough, the system top had been decent like under 10% both cpu and memory. I also think that I might be too lucky to get exact answer by sharing just this amount of information. I just want to get some hint from rough ideas or similar experiences. Thanks. >> >> >> >> On Tue, Jun 4, 2013 at 9:11 PM, Max Lapshin wrote: >>> No. You are missing something else in _your_ code. The problem is in it. >>> >>> While you will "prepare" your code, you will loose it. >> >> >> >> -- >> Park, Sungjin >> ------------------------------------------------------------------------------------------------------------------- >> Peculiar travel suggestions are dancing lessons from god. >> -- The Books of Bokonon >> ------------------------------------------------------------------------------------------------------------------- >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From f355@REDACTED Tue Jun 4 16:37:03 2013 From: f355@REDACTED (Konstantin Tcepliaev) Date: Tue, 04 Jun 2013 18:37:03 +0400 Subject: [erlang-questions] gen_server doesn't respond In-Reply-To: References: <18FB305D-6645-4EB0-91D5-B44A3E0C0FD6@gmail.com> Message-ID: <1320001370356623@web17h.yandex.ru> An HTML attachment was scrubbed... URL: From jon@REDACTED Tue Jun 4 16:37:45 2013 From: jon@REDACTED (Jon Schneider) Date: Tue, 4 Jun 2013 15:37:45 +0100 Subject: [erlang-questions] gen_server doesn't respond In-Reply-To: References: <18FB305D-6645-4EB0-91D5-B44A3E0C0FD6@gmail.com> Message-ID: <6db5206a7a811d7b43ae8ec5c556a1e2.squirrel@mail.jschneider.net> I'm an Erlang newbie so can't really help from that angle but you could use strace on the Erlang(beam) processes not forgetting threads to see what they're up to. With a bit of work you could dump all but the last X of strace output and then spot what changes when things go wrong. Could you be exercising bug in the network stack or getting it er constipated for want of a better term ? You could try another OS I suggest NOT another Linux. Jon From jinni.park@REDACTED Tue Jun 4 16:45:42 2013 From: jinni.park@REDACTED (Park, Sungjin) Date: Tue, 4 Jun 2013 23:45:42 +0900 Subject: [erlang-questions] gen_server doesn't respond In-Reply-To: <6db5206a7a811d7b43ae8ec5c556a1e2.squirrel@mail.jschneider.net> References: <18FB305D-6645-4EB0-91D5-B44A3E0C0FD6@gmail.com> <6db5206a7a811d7b43ae8ec5c556a1e2.squirrel@mail.jschneider.net> Message-ID: Konstantin, Jon, unfortunately the nodes are already restarted to restore the service that I can't take erlang:process_info(Pid, backtrace) nor strace. I'll keep this in mind next time it happens. Big thanks. On Tue, Jun 4, 2013 at 11:37 PM, Jon Schneider wrote: > I'm an Erlang newbie so can't really help from that angle but you could > use strace on the Erlang(beam) processes not forgetting threads to see > what they're up to. With a bit of work you could dump all but the last X > of strace output and then spot what changes when things go wrong. > > Could you be exercising bug in the network stack or getting it er > constipated for want of a better term ? You could try another OS I suggest > NOT another Linux. > > Jon > > > -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Tue Jun 4 16:54:38 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 4 Jun 2013 16:54:38 +0200 Subject: [erlang-questions] gen_server doesn't respond In-Reply-To: <1320001370356623@web17h.yandex.ru> References: <18FB305D-6645-4EB0-91D5-B44A3E0C0FD6@gmail.com> <1320001370356623@web17h.yandex.ru> Message-ID: <8BDA1D39-8322-42B1-A741-4FE0043262D1@erlang-solutions.com> On Jun 4, 2013, at 4:37 PM, Konstantin Tcepliaev wrote: > As it was already said before, try issuing erlang:process_info(Pid, backtrace) on your process. It will return one large human-readable (well, almost) binary which might give you a clue on what's going on. > erlang:process_info(Pid, current_stacktrace) is also worth looking at. It is the same information, but a little more readable as a term. The current function is not that interesting, but a couple of frames up in the stack is the culprit. My guess is a deadlock because you have a "call ring" A -> B -> A or A -> B -> C -> A or something such. Or perhaps even A -> A, which I have seen a couple of times. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen From jon@REDACTED Tue Jun 4 16:56:14 2013 From: jon@REDACTED (Jon Schneider) Date: Tue, 4 Jun 2013 15:56:14 +0100 Subject: [erlang-questions] gen_server doesn't respond In-Reply-To: References: <18FB305D-6645-4EB0-91D5-B44A3E0C0FD6@gmail.com> <6db5206a7a811d7b43ae8ec5c556a1e2.squirrel@mail.jschneider.net> Message-ID: Another thing is assuming NIFs are always shared objects you'd see them mapped in /proc//maps . Probably worth keeping a copy of proc/maps and possibly one or two other proc things before and after something interesting happens. Jon From peerst@REDACTED Tue Jun 4 17:03:48 2013 From: peerst@REDACTED (Peer Stritzinger) Date: Tue, 4 Jun 2013 17:03:48 +0200 Subject: [erlang-questions] Handling Crash Reports at scale References: Message-ID: On 2013-06-04 14:27:25 +0000, Max Lapshin said: > It is important to remember that remote user is usually waiting for > something like http 500 or 403, not just closing socket. > > So, just "let-it-crash" is not always suitable way to write software. Well "let it crash" is for unexpected errors. In your eample HTTP 500 error seems to be the right thing that your webserver should handle right on the top level when any handler does crash. HTTP 403 and other errors are a different story. Besides that I agree with Jespers way of adding error handling sparsely while development. The nice thing is if you have the crashes happening you no longer need to write error handling code that you can't trigger in a real situation. -- Peer From michael.santos@REDACTED Tue Jun 4 17:30:54 2013 From: michael.santos@REDACTED (Michael Santos) Date: Tue, 4 Jun 2013 11:30:54 -0400 Subject: [erlang-questions] low level packet access from erlang In-Reply-To: <51ADE6E1.4030205@research.att.com> References: <515C6E13.1050406@research.att.com> <20130403212407.GA25006@ioctl> <51ADE6E1.4030205@research.att.com> Message-ID: <20130604153054.GA31551@ioctl> On Tue, Jun 04, 2013 at 09:08:49AM -0400, Garry Hodgson wrote: > On 04/03/2013 05:24 PM, Michael Santos wrote: > > > >I wrote an Erlang tun/tap interface on top of procket: > > > >https://github.com/msantos/tunctl > i've been playing with tunctl, but can't seem to get > it past initial eperm problems in create(). i'm guessing > i missed some kind of setup step, but i don't know what. > > i set the capabilities as per instructions: > > --> getcap /usr/local/lib/erlang/erts-5.9.2/bin/beam.smp > /usr/local/lib/erlang/erts-5.9.2/bin/beam > /usr/local/lib/erlang/erts-5.9.2/bin/beam.smp = cap_net_admin+ep > /usr/local/lib/erlang/erts-5.9.2/bin/beam = cap_net_admin+ep > > running as root, when i try tuncer:create() with or without args, i get: > > (r3@REDACTED)2> tuncer:create( <<"tun0">> ). > ** exception exit: {badmatch,{error,eperm}} > in function tuncer:init/1 > in call from gen_server:init_it/6 (gen_server.erl, line 304) > in call from proc_lib:init_p_do_apply/3 (proc_lib.erl, line 227) > (r3@REDACTED)3> > =CRASH REPORT==== 4-Jun-2013::08:28:13 === > crasher: > initial call: tuncer:init/1 > pid: <0.64.0> > registered_name: [] > exception exit: {{badmatch,{error,eperm}}, > [{tuncer,init,1,[]}, > {gen_server,init_it,6, > > any idea what i may be missing? Looks like you have everything set up correctly. Just to make sure, I tried creating a tun device as root: ~/src/erlang/tunctl(master)$ sudo ./start.sh Erlang R16B01 (erts-5.10.2) [source-e72043e] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] Eshell V5.10.2 (abort with ^G) 1> tuncer:create( <<"tun0">> ). {ok,<0.35.0>} 2> $ ip addr <...> 5: tun0: mtu 1500 qdisc noop state DOWN qlen 500 link/ether 4a:1c:12:7e:4d:56 brd ff:ff:ff:ff:ff:ff You can try creating the tun device manually: $ sudo ./start.sh 1> {ok, FD} = procket:dev("net/tun"). {ok,9} 2> procket:ioctl(FD, 1074025674, <<"tun0", 0:96, 1:2/native-integer-unit:8, 0:112>>). {ok,<<116,117,110,48,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0, 0,0,0,0,0,...>>} Are you able to create a tun device using other utilities? This should work: ip tuntap add mode tun foo Is selinux enabled? From anthonym@REDACTED Tue Jun 4 18:31:56 2013 From: anthonym@REDACTED (ANTHONY MOLINARO) Date: Tue, 4 Jun 2013 09:31:56 -0700 Subject: [erlang-questions] Handling Crash Reports at scale In-Reply-To: References: <24F2ADC1-368D-4856-8BAB-AC5660F3033F@alumni.caltech.edu> Message-ID: Well, I almost didn't use the term defensive programming, but to some handling any fault would be considered defensive. As an example, lets say I'm parsing the value of an HTTP cookie. The first pass I just assume the cookie is always good, and code for the correct case (least defensive). If the cookie always parses correctly, I'm done, but lets say the parsing code start failing and I see exceptions in my logs. I then want to determine is the parsing failing because the parsing is broken (a software defect) or because of random corruption of the HTTP stream. So usually I'll put enough defensive code (e.g., code which captures exceptions, instead of 'letting it fail'), to log the error cases, and allow processing to continue. If the logging shows a defect it can be fixed, if it shows this is general random corruption it can be left in place or replaced with a counter. I use mondemand for counters because I can get graphs easily and have systems which use the data streams to alert based on specific changes. I think that defensive code is necessary in many cases to handle faults at the extremities of your system (and sometimes internally depending on the boundaries). So calling defensive code a code smell seems to overly generalize it. -Anthony On Jun 4, 2013, at 4:48 AM, Peer Stritzinger wrote: > I'm not sure what you mean by defensive code, but just to be sure: > > You should never let error reporting influence how you handle faults in your software. Defensive code is a code smell in Erlang. > > What you coucld do is to have a error logger handler that matches certain often occuring errors and ignores it. Even better it should count them (the count of something unineresting is often interesting). > > For counting and other metrics you might want to have a look at folsom https://github.com/boundary/folsom > > -- Peer > > > On 2013-06-03 17:29:54 +0000, ANTHONY MOLINARO said: > > I'd recommend addressing the other crashes with "defensive code". By capturing and categorizing certain types of errors and clearing those from your logs you'll be able to better see true errors. I usually have a two phase approach. First phase, I capture and log the types of errors. Second, once I see certain types are regular I replace those with metrics (via mondemand). Once you have a stream of errors you can then monitor the rates, plus uncaught errors will make it into your logs which should remain very sparse. > > -Anthony > > On Jun 3, 2013, at 9:11 AM, Ransom Richardson wrote: > Are there tools/procedures that are recommended for processing crash reports from a service running at scale? > > Currently we have a limited deployment and I look through all of the crash reports by hand. Some are very useful for finding actual bugs in our code. But other crashes are the result of client's sending bad data, strange timing issues (mostly not in our code), etc and are not actionable. As we prepare to scale up our service, I'm wondering how to continue to get the value from the interesting crash reports without having to look through all of the uninteresting ones. > > I haven't found rb to be very useful for finding the new/interesting crashes. Are there effective ways that peopler are using it? > > Are there other tools for parsing and grouping crash reports to make it easy to find new/interesting ones? > > thanks, > Ransom > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthonym@REDACTED Tue Jun 4 18:52:25 2013 From: anthonym@REDACTED (ANTHONY MOLINARO) Date: Tue, 4 Jun 2013 09:52:25 -0700 Subject: [erlang-questions] ANN: gproc_pool + some performance tidbits In-Reply-To: References: Message-ID: <043F5F0A-9BB4-4AE6-ABFC-9BBD035DB478@alumni.caltech.edu> Hi Ulf, Have you done any concurrent tests? I only ask because I've seen our own pooling code (https://github.com/openx/gen_server_pool) have issues under load. Now in our case it's because of a single gen_server acting as a dispatch layer, which should not be the case for gproc as IIRC it uses ets to provide for fast concurrent access (something also done in a novel way by https://github.com/ferd/dispcount/ which I keep meaning to try out), but I'd be curious to know if you've done any concurrent testing which shows that. I think the number of pool implementations in erlang has probably finally surpassed the number of json parsers ;) -Anthony On Jun 4, 2013, at 2:18 AM, Ulf Wiger wrote: > > I pushed a new gproc component called gproc_pool the other day. > > The main idea, apart from wanting to see how well it would work, was that I wanted to be able to register servers with gproc and then have an efficient way of pooling between them. A benefit of using gproc throughout is that the registration objects serve as a 'footprint' for each process - by listing the gproc entities for each process, you can tell a lot about its purpose. > > The way gproc_pool works is that: > 1. You define a pool, by naming it, and optionally specifying its size > (gproc_pool:new(Pool) | gproc_pool:new(Pool, Type, Options)) > 2. You add worker names to the pool > (gproc_pool:add_worker(Pool, Name)) > 3. Your servers each connect to a given name > (gproc_pool:connect_worker(Pool, Name)) > 4. Users pick a worker for each request (gproc_pool:pick(Pool)) > > My little test code indicates that the different load-balancing strategies perform a bit differently: > > (https://github.com/uwiger/gproc/blob/master/src/gproc_pool.erl#L843) > > (Create a pool, add 6 workers and iterate 100k times, > incrementing a gproc counter for each iteration.) > > 3> gproc_pool:test(100000,round_robin,[]). > worker stats (848): > [{a,16667},{b,16667},{c,16667},{d,16667},{e,16666},{f,16666}] > {2801884,ok} > 4> gproc_pool:test(100000,hash,[]). > worker stats (848): > [{a,16744},{b,16716},{c,16548},{d,16594},{e,16749},{f,16649}] > {1891517,ok} > 5> gproc_pool:test(100000,random,[]). > worker stats (848): > [{a,16565},{b,16542},{c,16613},{d,16872},{e,16727},{f,16681}] > {3701011,ok} > 6> gproc_pool:test(100000,direct,[]). > worker stats (848): > [{a,16667},{b,16667},{c,16667},{d,16667},{e,16666},{f,16666}] > {1766639,ok} > 11> gproc_pool:test(100000,claim,[]). > worker stats (848): > [{a,100000},{b,0},{c,0},{d,0},{e,0},{f,0}] > {7569425,ok} > > > The worker stats show how evenly the workers were selected, > and the {Time, ok} comes from timer:tc/3, i.e. Time/100000 is the per-iteration cost: > > round_robin: 28 usec (maintain a 'current' counter, modulo Size) > hash: 19 usec (gproc_pool:pick(Pool, Val), hash on Val) > random: 37 usec (pick a random worker, using crypto:rand_uniform/2) > direct: 18 usec (gproc_pool:pick(Pool, N), where N modulo Size selects worker) > claim: 76 usec (claim the first available worker, apply a fun, then release) > > I think the per-selection cost is acceptable as-is, but could perhaps be improved (esp. the 'random' strategy is surprisingly expensive). All the selection work is done in the caller's process, BTW - no communication with the gproc or gproc_pool servers (except for admin tasks). > > The 'claim' strategy is also surprisingly expensive. I believe it's because I'm using gproc:select/3 to find the first free worker. Note also that it results in an extremely uneven distribution. That's obviously because the test run claims the first available worker and then releases it before iterating - it's always going to select the first worker.) > > https://github.com/uwiger/gproc/blob/master/doc/gproc_pool.md > > Feedback welcome, be it with performance tips, usability tips, or other. > > BR, > Ulf W > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Tue Jun 4 21:24:21 2013 From: ulf@REDACTED (Ulf Wiger) Date: Tue, 4 Jun 2013 21:24:21 +0200 Subject: [erlang-questions] ANN: gproc_pool + some performance tidbits In-Reply-To: <043F5F0A-9BB4-4AE6-ABFC-9BBD035DB478@alumni.caltech.edu> References: <043F5F0A-9BB4-4AE6-ABFC-9BBD035DB478@alumni.caltech.edu> Message-ID: <2F0CE77B-6960-435E-A363-682D9B306EAE@feuerlabs.com> On 4 Jun 2013, at 18:52, ANTHONY MOLINARO wrote: > Hi Ulf, > > Have you done any concurrent tests? I only ask because I've seen our own pooling code (https://github.com/openx/gen_server_pool) have issues under load. Now in our case > it's because of a single gen_server acting as a dispatch layer, which should not be the > case for gproc as IIRC it uses ets to provide for fast concurrent access (something also > done in a novel way by https://github.com/ferd/dispcount/ which I keep meaning to try > out), but I'd be curious to know if you've done any concurrent testing which shows that. I hadn't, but did so now. Spawning N clients, which run 1000 iterations each, on e.g. a round_robin pool: N Avg usec/iteration 1 37 10 250 100 1630 1000 18813 Of course, this was a pretty nasty test, with all processes banging away at the pool as fast as they possibly could. If you want frequent mutex conflicts, that's probably as good a way as any to provoke them. When I insert a random sleep (0-50 ms) between each iteration, time each pick request and collect the averages, 100 concurrent workers pay on average 50 usec per selection. For 1000 concurrent workers, the average rises to 60 usec. The corresponding average for the hash pool and 1000 concurrent workers is 20 usec. (All on my Macbook Air) > I think the number of pool implementations in erlang has probably finally surpassed > the number of json parsers ;) Well, that tends to happen with fun and reasonably well-bounded problems. ;) BR, Ulf W > > -Anthony > > On Jun 4, 2013, at 2:18 AM, Ulf Wiger wrote: > >> >> I pushed a new gproc component called gproc_pool the other day. >> >> The main idea, apart from wanting to see how well it would work, was that I wanted to be able to register servers with gproc and then have an efficient way of pooling between them. A benefit of using gproc throughout is that the registration objects serve as a 'footprint' for each process - by listing the gproc entities for each process, you can tell a lot about its purpose. >> >> The way gproc_pool works is that: >> 1. You define a pool, by naming it, and optionally specifying its size >> (gproc_pool:new(Pool) | gproc_pool:new(Pool, Type, Options)) >> 2. You add worker names to the pool >> (gproc_pool:add_worker(Pool, Name)) >> 3. Your servers each connect to a given name >> (gproc_pool:connect_worker(Pool, Name)) >> 4. Users pick a worker for each request (gproc_pool:pick(Pool)) >> >> My little test code indicates that the different load-balancing strategies perform a bit differently: >> >> (https://github.com/uwiger/gproc/blob/master/src/gproc_pool.erl#L843) >> >> (Create a pool, add 6 workers and iterate 100k times, >> incrementing a gproc counter for each iteration.) >> >> 3> gproc_pool:test(100000,round_robin,[]). >> worker stats (848): >> [{a,16667},{b,16667},{c,16667},{d,16667},{e,16666},{f,16666}] >> {2801884,ok} >> 4> gproc_pool:test(100000,hash,[]). >> worker stats (848): >> [{a,16744},{b,16716},{c,16548},{d,16594},{e,16749},{f,16649}] >> {1891517,ok} >> 5> gproc_pool:test(100000,random,[]). >> worker stats (848): >> [{a,16565},{b,16542},{c,16613},{d,16872},{e,16727},{f,16681}] >> {3701011,ok} >> 6> gproc_pool:test(100000,direct,[]). >> worker stats (848): >> [{a,16667},{b,16667},{c,16667},{d,16667},{e,16666},{f,16666}] >> {1766639,ok} >> 11> gproc_pool:test(100000,claim,[]). >> worker stats (848): >> [{a,100000},{b,0},{c,0},{d,0},{e,0},{f,0}] >> {7569425,ok} >> >> >> The worker stats show how evenly the workers were selected, >> and the {Time, ok} comes from timer:tc/3, i.e. Time/100000 is the per-iteration cost: >> >> round_robin: 28 usec (maintain a 'current' counter, modulo Size) >> hash: 19 usec (gproc_pool:pick(Pool, Val), hash on Val) >> random: 37 usec (pick a random worker, using crypto:rand_uniform/2) >> direct: 18 usec (gproc_pool:pick(Pool, N), where N modulo Size selects worker) >> claim: 76 usec (claim the first available worker, apply a fun, then release) >> >> I think the per-selection cost is acceptable as-is, but could perhaps be improved (esp. the 'random' strategy is surprisingly expensive). All the selection work is done in the caller's process, BTW - no communication with the gproc or gproc_pool servers (except for admin tasks). >> >> The 'claim' strategy is also surprisingly expensive. I believe it's because I'm using gproc:select/3 to find the first free worker. Note also that it results in an extremely uneven distribution. That's obviously because the test run claims the first available worker and then releases it before iterating - it's always going to select the first worker.) >> >> https://github.com/uwiger/gproc/blob/master/doc/gproc_pool.md >> >> Feedback welcome, be it with performance tips, usability tips, or other. >> >> BR, >> Ulf W >> >> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >> http://feuerlabs.com >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From pablo.platt@REDACTED Tue Jun 4 21:30:47 2013 From: pablo.platt@REDACTED (pablo platt) Date: Tue, 4 Jun 2013 22:30:47 +0300 Subject: [erlang-questions] ANN: gproc_pool + some performance tidbits In-Reply-To: <2F0CE77B-6960-435E-A363-682D9B306EAE@feuerlabs.com> References: <043F5F0A-9BB4-4AE6-ABFC-9BBD035DB478@alumni.caltech.edu> <2F0CE77B-6960-435E-A363-682D9B306EAE@feuerlabs.com> Message-ID: What's the use case for workers in the pool? Is it only for distributing a task or also for implementing a pool of DB connections like https://github.com/devinus/poolboy ? Why workers has names? I know I can just give them names such as 0,1,2... but trying to understand the rational. As always, I'm sure this functionality will be a major part in my server like everything else in gproc, even if I still don't know why ;) Thanks On Tue, Jun 4, 2013 at 10:24 PM, Ulf Wiger wrote: > > On 4 Jun 2013, at 18:52, ANTHONY MOLINARO wrote: > > Hi Ulf, > > Have you done any concurrent tests? I only ask because I've seen our own > pooling code (https://github.com/openx/gen_server_pool) have issues under > load. Now in our case > it's because of a single gen_server acting as a dispatch layer, which > should not be the > case for gproc as IIRC it uses ets to provide for fast concurrent access > (something also > done in a novel way by https://github.com/ferd/dispcount/ which I keep > meaning to try > out), but I'd be curious to know if you've done any concurrent testing > which shows that. > > > I hadn't, but did so now. > > Spawning N clients, which run 1000 iterations each, on e.g. a round_robin > pool: > > N Avg usec/iteration > 1 37 > 10 250 > 100 1630 > 1000 18813 > > Of course, this was a pretty nasty test, with all processes banging away > at the pool as fast as they possibly could. If you want frequent mutex > conflicts, that's probably as good a way as any to provoke them. > > When I insert a random sleep (0-50 ms) between each iteration, time each > pick request and collect the averages, 100 concurrent workers pay on > average 50 usec per selection. For 1000 concurrent workers, the average > rises to 60 usec. > > The corresponding average for the hash pool and 1000 concurrent workers is > 20 usec. > > (All on my Macbook Air) > > > I think the number of pool implementations in erlang has probably finally > surpassed > the number of json parsers ;) > > > Well, that tends to happen with fun and reasonably well-bounded problems. > ;) > > BR, > Ulf W > > > -Anthony > > On Jun 4, 2013, at 2:18 AM, Ulf Wiger wrote: > > > I pushed a new gproc component called gproc_pool the other day. > > The main idea, apart from wanting to see how well it would work, was that > I wanted to be able to register servers with gproc and then have an > efficient way of pooling between them. A benefit of using gproc throughout > is that the registration objects serve as a 'footprint' for each process - > by listing the gproc entities for each process, you can tell a lot about > its purpose. > > The way gproc_pool works is that: > 1. You define a pool, by naming it, and optionally specifying its size > (gproc_pool:new(Pool) | gproc_pool:new(Pool, Type, Options)) > 2. You add worker names to the pool > (gproc_pool:add_worker(Pool, Name)) > 3. Your servers each connect to a given name > (gproc_pool:connect_worker(Pool, Name)) > 4. Users pick a worker for each request (gproc_pool:pick(Pool)) > > My little test code indicates that the different load-balancing strategies > perform a bit differently: > > (https://github.com/uwiger/gproc/blob/master/src/gproc_pool.erl#L843) > > (Create a pool, add 6 workers and iterate 100k times, > incrementing a gproc counter for each iteration.) > > 3> gproc_pool:test(100000,round_robin,[]). > worker stats (848): > [{a,16667},{b,16667},{c,16667},{d,16667},{e,16666},{f,16666}] > {2801884,ok} > 4> gproc_pool:test(100000,hash,[]). > worker stats (848): > [{a,16744},{b,16716},{c,16548},{d,16594},{e,16749},{f,16649}] > {1891517,ok} > 5> gproc_pool:test(100000,random,[]). > worker stats (848): > [{a,16565},{b,16542},{c,16613},{d,16872},{e,16727},{f,16681}] > {3701011,ok} > 6> gproc_pool:test(100000,direct,[]). > worker stats (848): > [{a,16667},{b,16667},{c,16667},{d,16667},{e,16666},{f,16666}] > {1766639,ok} > 11> gproc_pool:test(100000,claim,[]). > worker stats (848): > [{a,100000},{b,0},{c,0},{d,0},{e,0},{f,0}] > {7569425,ok} > > > The worker stats show how evenly the workers were selected, > and the {Time, ok} comes from timer:tc/3, i.e. Time/100000 is the > per-iteration cost: > > round_robin: 28 usec (maintain a 'current' counter, modulo Size) > hash: 19 usec (gproc_pool:pick(Pool, Val), hash on Val) > random: 37 usec (pick a random worker, using crypto:rand_uniform/2) > direct: 18 usec (gproc_pool:pick(Pool, N), where N modulo Size selects > worker) > claim: 76 usec (claim the first available worker, apply a fun, then > release) > > I think the per-selection cost is acceptable as-is, but could perhaps be > improved (esp. the 'random' strategy is surprisingly expensive). All the > selection work is done in the caller's process, BTW - no communication with > the gproc or gproc_pool servers (except for admin tasks). > > The 'claim' strategy is also surprisingly expensive. I believe it's > because I'm using gproc:select/3 to find the first free worker. Note also > that it results in an extremely uneven distribution. That's obviously > because the test run claims the first available worker and then releases it > before iterating - it's always going to select the first worker.) > > https://github.com/uwiger/gproc/blob/master/doc/gproc_pool.md > > Feedback welcome, be it with performance tips, usability tips, or other. > > BR, > Ulf W > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peerst@REDACTED Tue Jun 4 23:04:18 2013 From: peerst@REDACTED (Peer Stritzinger) Date: Tue, 4 Jun 2013 23:04:18 +0200 Subject: [erlang-questions] Handling Crash Reports at scale References: <24F2ADC1-368D-4856-8BAB-AC5660F3033F@alumni.caltech.edu> Message-ID: We obviously mean something different by defensive coding. Apparently there is a range of definitions for the word: From: http://c2.com/cgi/wiki?DefensiveProgramming "Defensive programming defends against the currently impossible." To: http://en.wikipedia.org/wiki/Defensive_programming defines it as basically "Secure Programming" a context in which I didn't hear it before (their first example strcpy to a fixed buffer vs. strncpy is a quite bad example, ist rather uncorrect programs vs. correct programs) When I used the word I certainly didn't mean checking at the user/world facing borders of systems. Also I didn't mean using assert() in C and pattern matching to have a implicit assert in Erlang. In the case I meant it its: {ok, F} = file:open("/var/log/mylogfile", [append]) %% not defensive, since its a bug in my system if this doesn't work vs. case file:open("/var/log/mylogfile", [append]) of %% defensively handling potential buggy behaviour {ok, F} -> proceed_normally(F); {error, Reason} -> dont_know_what_to_do_now_really_trying_to_fix_the_situation_somehow() end Might not be the perfect example. But by all means check your inputs, but if it is a "this can't happen" situation feel free to crash the subsystem. If this does show up in your logs, either fix the bug or update your misconception of "this can't happen" if it occurs every 5minutes ;-) <-- handle this and count it if it fills your logs otherwise -- Peer On 2013-06-04 16:31:56 +0000, ANTHONY MOLINARO said: > Well, I almost didn't use the term defensive programming, but to some > handling any fault would be considered defensive. > > As an example, lets say I'm parsing the value of an HTTP cookie. > > The first pass I just assume the cookie is always good, and code for > the correct case (least defensive). > If the cookie always parses correctly, I'm done, but lets say the > parsing code start failing and I see exceptions in my logs. ?I then > want to determine is the parsing failing because the parsing is broken > (a software defect) or because of random corruption of the HTTP stream. > ?So usually I'll put enough defensive code (e.g., code which captures > exceptions, instead of 'letting it fail'), to log the error cases, and > allow processing to continue. > If the logging shows a defect it can be fixed, if it shows this is > general random corruption it can be left in place or replaced with a > counter. > > I use mondemand for counters because I can get graphs easily and have > systems which use the data streams to alert based on specific changes. > > I think that defensive code is necessary in many cases to handle faults > at the extremities of your system (and sometimes internally depending > on the boundaries). ?So calling defensive code a code smell seems to > overly generalize it. > > -Anthony > > On Jun 4, 2013, at 4:48 AM, Peer Stritzinger wrote: > I'm not sure what you mean by defensive code, but just to be sure: > > You should never let error reporting influence how you handle faults in > your software. Defensive code is a code smell in Erlang. > > What you coucld do is to have a error logger handler that matches > certain often occuring errors and ignores it. Even better it should > count them (the count of something unineresting is often interesting). > > For counting and other metrics you might want to have a look at folsom > https://github.com/boundary/folsom > > -- Peer > > > On 2013-06-03 17:29:54 +0000, ANTHONY MOLINARO said: > > I'd recommend addressing the other crashes with "defensive code". ?By > capturing and categorizing certain types of errors and clearing those > from your logs you'll be able to better see true errors. ?I usually > have a two phase approach. ?First phase, I capture and log the types of > errors. ?Second, once I see certain types ?are regular I replace those > with metrics (via mondemand). ?Once you have a stream of errors you can > then monitor the rates, plus uncaught errors will make it into your > logs which should remain very sparse. > > -Anthony > > On Jun 3, 2013, at 9:11 AM, Ransom Richardson wrote: > Are there tools/procedures that are recommended for processing crash > reports from a service running at scale? > > Currently we have a limited deployment and I look through all of the > crash reports by hand. Some are very useful for finding actual bugs in > our code. But other crashes are the result of client's sending bad > data, strange timing issues (mostly not in our code), etc and are not > actionable. As we prepare to scale up our service, I'm wondering how to > continue to get the value from the interesting crash reports without > having to look through all of the uninteresting ones.? > > I haven't found rb to be very useful for finding the new/interesting > crashes. Are there effective ways that peopler are using it? > > Are there other tools for parsing and grouping crash reports to make it > easy to find new/interesting ones? > > thanks, > Ransom > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From garry@REDACTED Wed Jun 5 01:07:45 2013 From: garry@REDACTED (Garry Hodgson) Date: Tue, 04 Jun 2013 19:07:45 -0400 Subject: [erlang-questions] low level packet access from erlang In-Reply-To: <20130604153054.GA31551@ioctl> References: <515C6E13.1050406@research.att.com> <20130403212407.GA25006@ioctl> <51ADE6E1.4030205@research.att.com> <20130604153054.GA31551@ioctl> Message-ID: <51AE7341.4090200@research.att.com> On 06/04/2013 11:30 AM, Michael Santos wrote: > Looks like you have everything set up correctly. Just to make sure, I > tried creating a tun device as root: > > ~/src/erlang/tunctl(master)$ sudo ./start.sh > Erlang R16B01 (erts-5.10.2) [source-e72043e] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V5.10.2 (abort with ^G) > 1> tuncer:create( <<"tun0">> ). > {ok,<0.35.0>} > 2> > hmmm...that gives me the aforementioned error. > You can try creating the tun device manually: > > $ sudo ./start.sh > > 1> {ok, FD} = procket:dev("net/tun"). > {ok,9} --> erl -pa /usr/local/sut/sut/deps/pkt/ebin -pa /usr/local/sut/sut/ebin -pa /usr/local/sut/sut/deps/procket/ebin Erlang R15B02 (erts-5.9.2) [source] [64-bit] [smp:8:8] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.9.2 (abort with ^G) 1> procket:dev("net/tun"). {error,eperm} --> ls -l /dev/net/tun crw-rw-rw- 1 root root 10, 200 Jun 3 17:23 /dev/net/tun --> getcap /usr/local/lib/erlang/erts-5.9.2/bin/beam /usr/local/lib/erlang/erts-5.9.2/bin/beam.smp /usr/local/lib/erlang/erts-5.9.2/bin/beam = cap_net_admin+ep /usr/local/lib/erlang/erts-5.9.2/bin/beam.smp = cap_net_admin+ep > > 2> procket:ioctl(FD, 1074025674, <<"tun0", 0:96, 1:2/native-integer-unit:8, 0:112>>). > {ok,<<116,117,110,48,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0, > 0,0,0,0,0,...>>} > > Are you able to create a tun device using other utilities? This should > work: > > ip tuntap add mode tun foo that does not: --> ip tuntap add mode tun foo Object "tuntap" is unknown, try "ip help". but i can create tun using tunctl: --> tunctl -n -u garry -g garry Set 'tun0' persistent and owned by uid 1234 gid 1234 > > Is selinux enabled? i believe not: --> /usr/sbin/getenforce Disabled puzzling. -- Garry Hodgson AT&T Chief Security Office (CSO) "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." From garry@REDACTED Wed Jun 5 01:49:43 2013 From: garry@REDACTED (Garry Hodgson) Date: Tue, 04 Jun 2013 19:49:43 -0400 Subject: [erlang-questions] low level packet access from erlang In-Reply-To: <51AE7341.4090200@research.att.com> References: <515C6E13.1050406@research.att.com> <20130403212407.GA25006@ioctl> <51ADE6E1.4030205@research.att.com> <20130604153054.GA31551@ioctl> <51AE7341.4090200@research.att.com> Message-ID: <51AE7D17.2050807@research.att.com> ha! success! r3:init: procket got {ok,13} r3:init:got socket #Port<0.1538> r3:init: dev is <0.58.0> r3:init: ip address is "135.207.243.86" poking around just now, i noticed that while i had a sudoers entry, the permissions on my procket were not what you specified in README: [root@REDACTED ~]# ls -l /usr/local/sut/sut/deps/procket/priv/procket -rwxr-xr-x 1 root root 11712 Jun 4 13:37 /usr/local/sut/sut/deps/procket/priv/procket so i changed them: [root@REDACTED ~]# chmod 4750 /usr/local/sut/sut/deps/procket/priv/procket [root@REDACTED ~]# ls -l /usr/local/sut/sut/deps/procket/priv/procket -rwsr-x--- 1 root root 11712 Jun 4 13:37 /usr/local/sut/sut/deps/procket/priv/procket and your tests worked: (r3@REDACTED)2> tuncer:create( <<"tun9">> ). {ok,<0.64.0>} (r3@REDACTED)3> procket:dev("net/tun"). {ok,16} as did my code: r3:init: procket got {ok,13} r3:init:got socket #Port<0.1538> r3:init: dev is <0.58.0> r3:init: ip address is "135.207.243.86" i think i recall doing this before, but it may have been on a different machine. and it's odd that it helped, given that i've been running as root. in any case, i appear to be unstuck. i'll likely have more questions, but i'm good for now. thanks for your help, and your willingness to be helpful. this is gonna be fun. On 06/04/2013 07:07 PM, Garry Hodgson wrote: > On 06/04/2013 11:30 AM, Michael Santos wrote: >> Looks like you have everything set up correctly. Just to make sure, I >> tried creating a tun device as root: >> >> ~/src/erlang/tunctl(master)$ sudo ./start.sh >> Erlang R16B01 (erts-5.10.2) [source-e72043e] [smp:2:2] >> [async-threads:10] [hipe] [kernel-poll:false] >> Eshell V5.10.2 (abort with ^G) >> 1> tuncer:create( <<"tun0">> ). >> {ok,<0.35.0>} >> 2> > hmmm...that gives me the aforementioned error. >> You can try creating the tun device manually: >> >> $ sudo ./start.sh >> 1> {ok, FD} = procket:dev("net/tun"). >> {ok,9} > --> erl -pa /usr/local/sut/sut/deps/pkt/ebin -pa > /usr/local/sut/sut/ebin -pa /usr/local/sut/sut/deps/procket/ebin > Erlang R15B02 (erts-5.9.2) [source] [64-bit] [smp:8:8] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.9.2 (abort with ^G) > 1> procket:dev("net/tun"). > {error,eperm} > > --> ls -l /dev/net/tun > crw-rw-rw- 1 root root 10, 200 Jun 3 17:23 /dev/net/tun > --> getcap /usr/local/lib/erlang/erts-5.9.2/bin/beam > /usr/local/lib/erlang/erts-5.9.2/bin/beam.smp > /usr/local/lib/erlang/erts-5.9.2/bin/beam = cap_net_admin+ep > /usr/local/lib/erlang/erts-5.9.2/bin/beam.smp = cap_net_admin+ep > >> 2> procket:ioctl(FD, 1074025674, <<"tun0", 0:96, >> 1:2/native-integer-unit:8, 0:112>>). >> {ok,<<116,117,110,48,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0, >> 0,0,0,0,0,...>>} >> >> Are you able to create a tun device using other utilities? This should >> work: >> >> ip tuntap add mode tun foo > > that does not: > > --> ip tuntap add mode tun foo > Object "tuntap" is unknown, try "ip help". > > but i can create tun using tunctl: > > --> tunctl -n -u garry -g garry > Set 'tun0' persistent and owned by uid 1234 gid 1234 > >> >> Is selinux enabled? > i believe not: > > --> /usr/sbin/getenforce > Disabled > > puzzling. > -- Garry Hodgson AT&T Chief Security Office (CSO) "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." From ulf@REDACTED Wed Jun 5 07:47:32 2013 From: ulf@REDACTED (Ulf Wiger) Date: Wed, 5 Jun 2013 07:47:32 +0200 Subject: [erlang-questions] ANN: gproc_pool + some performance tidbits In-Reply-To: References: <043F5F0A-9BB4-4AE6-ABFC-9BBD035DB478@alumni.caltech.edu> <2F0CE77B-6960-435E-A363-682D9B306EAE@feuerlabs.com> Message-ID: <462D5AF9-3792-49DD-8520-149A3977016D@feuerlabs.com> On 4 Jun 2013, at 21:30, pablo platt wrote: > What's the use case for workers in the pool? > Is it only for distributing a task or also for implementing a pool of DB connections like https://github.com/devinus/poolboy ? I believe it *is* fairly similar to poolboy, but I thought it would be consistent with the gproc philosophy to have a pool concept in gproc, since: - One of the things you need to do in a worker pool implementation is to keep track of the worker processes, and gproc is good at this - A benefit of using gproc is that you can get some query/debugging/monitoring capabilities for free. For example, after setting up my test pool (gproc_pool:setup_test_pool/3), I can use the following stock gproc function: 2> gproc_pool:setup_test_pool(mypool,round_robin,[]). add_worker(mypool, a) -> 1; Ws = [{a,1}] add_worker(mypool, b) -> 2; Ws = [{a,1},{b,2}] add_worker(mypool, c) -> 3; Ws = [{a,1},{b,2},{c,3}] add_worker(mypool, d) -> 4; Ws = [{a,1},{b,2},{c,3},{d,4}] add_worker(mypool, e) -> 5; Ws = [{a,1},{b,2},{c,3},{d,4},{e,5}] add_worker(mypool, f) -> 6; Ws = [{a,1},{b,2},{c,3},{d,4},{e,5},{f,6}] [true,true,true,true,true,true] 3> gproc:in info/1 info/2 init/1 3> catch gproc:info(self()). [{gproc,[{{n,l,[gproc_pool,mypool,1,a]},0}, {{n,l,[gproc_pool,mypool,2,b]},0}, {{n,l,[gproc_pool,mypool,3,c]},0}, {{n,l,[gproc_pool,mypool,4,d]},0}, {{n,l,[gproc_pool,mypool,5,e]},0}, {{n,l,[gproc_pool,mypool,6,f]},0}]}, {current_function,{erl_eval,do_apply,6}}, {initial_call,{erlang,apply,2}}, {status,running}, {message_queue_len,0}, ?] Thus, from the 'gproc footprint' of the process, I can readily tell that it's a worker in the pool 'mypool' (even if I'm not familiar with the gproc_pool concept, I can guess from convention that the first part of the name is a module name). The whole idea of gproc was in fact to provide a single set of patterns that I saw appearing in many different places in our code, in lots of different implementations. So in a sense, practically everything that gproc provides is stuff that people have implemented before, in reasonably similar ways. :) Hopefully with gproc, some user code can become simpler, more debuggable and a bit more uniform. > Why workers has names? > I know I can just give them names such as 0,1,2... but trying to understand the rational. I thought it was a useful layer of abstraction. The performance of the pool is somewhat dependent on the spread of workers across the available slots (especially if the pool is half-full, and hashing or random selection is used). The workers themselves only need to know what they are to call themselves as they connect to the pool. Whoever manages the pool can control the positioning of each worker. > As always, I'm sure this functionality will be a major part in my server like everything else in gproc, > even if I still don't know why ;) Haha! This reminds me of the first design review meeting at Ericsson where gproc's predecessor sysproc was up for review. The chairman of the meeting said "I guess we'll approve it, even though I don't understand what it's for". :) It was a good decision, I thought? BR, Ulf W > > Thanks > > > > > On Tue, Jun 4, 2013 at 10:24 PM, Ulf Wiger wrote: > > On 4 Jun 2013, at 18:52, ANTHONY MOLINARO wrote: > >> Hi Ulf, >> >> Have you done any concurrent tests? I only ask because I've seen our own pooling code (https://github.com/openx/gen_server_pool) have issues under load. Now in our case >> it's because of a single gen_server acting as a dispatch layer, which should not be the >> case for gproc as IIRC it uses ets to provide for fast concurrent access (something also >> done in a novel way by https://github.com/ferd/dispcount/ which I keep meaning to try >> out), but I'd be curious to know if you've done any concurrent testing which shows that. > > I hadn't, but did so now. > > Spawning N clients, which run 1000 iterations each, on e.g. a round_robin pool: > > N Avg usec/iteration > 1 37 > 10 250 > 100 1630 > 1000 18813 > > Of course, this was a pretty nasty test, with all processes banging away at the pool as fast as they possibly could. If you want frequent mutex conflicts, that's probably as good a way as any to provoke them. > > When I insert a random sleep (0-50 ms) between each iteration, time each pick request and collect the averages, 100 concurrent workers pay on average 50 usec per selection. For 1000 concurrent workers, the average rises to 60 usec. > > The corresponding average for the hash pool and 1000 concurrent workers is 20 usec. > > (All on my Macbook Air) > > >> I think the number of pool implementations in erlang has probably finally surpassed >> the number of json parsers ;) > > Well, that tends to happen with fun and reasonably well-bounded problems. ;) > > BR, > Ulf W > >> >> -Anthony >> >> On Jun 4, 2013, at 2:18 AM, Ulf Wiger wrote: >> >>> >>> I pushed a new gproc component called gproc_pool the other day. >>> >>> The main idea, apart from wanting to see how well it would work, was that I wanted to be able to register servers with gproc and then have an efficient way of pooling between them. A benefit of using gproc throughout is that the registration objects serve as a 'footprint' for each process - by listing the gproc entities for each process, you can tell a lot about its purpose. >>> >>> The way gproc_pool works is that: >>> 1. You define a pool, by naming it, and optionally specifying its size >>> (gproc_pool:new(Pool) | gproc_pool:new(Pool, Type, Options)) >>> 2. You add worker names to the pool >>> (gproc_pool:add_worker(Pool, Name)) >>> 3. Your servers each connect to a given name >>> (gproc_pool:connect_worker(Pool, Name)) >>> 4. Users pick a worker for each request (gproc_pool:pick(Pool)) >>> >>> My little test code indicates that the different load-balancing strategies perform a bit differently: >>> >>> (https://github.com/uwiger/gproc/blob/master/src/gproc_pool.erl#L843) >>> >>> (Create a pool, add 6 workers and iterate 100k times, >>> incrementing a gproc counter for each iteration.) >>> >>> 3> gproc_pool:test(100000,round_robin,[]). >>> worker stats (848): >>> [{a,16667},{b,16667},{c,16667},{d,16667},{e,16666},{f,16666}] >>> {2801884,ok} >>> 4> gproc_pool:test(100000,hash,[]). >>> worker stats (848): >>> [{a,16744},{b,16716},{c,16548},{d,16594},{e,16749},{f,16649}] >>> {1891517,ok} >>> 5> gproc_pool:test(100000,random,[]). >>> worker stats (848): >>> [{a,16565},{b,16542},{c,16613},{d,16872},{e,16727},{f,16681}] >>> {3701011,ok} >>> 6> gproc_pool:test(100000,direct,[]). >>> worker stats (848): >>> [{a,16667},{b,16667},{c,16667},{d,16667},{e,16666},{f,16666}] >>> {1766639,ok} >>> 11> gproc_pool:test(100000,claim,[]). >>> worker stats (848): >>> [{a,100000},{b,0},{c,0},{d,0},{e,0},{f,0}] >>> {7569425,ok} >>> >>> >>> The worker stats show how evenly the workers were selected, >>> and the {Time, ok} comes from timer:tc/3, i.e. Time/100000 is the per-iteration cost: >>> >>> round_robin: 28 usec (maintain a 'current' counter, modulo Size) >>> hash: 19 usec (gproc_pool:pick(Pool, Val), hash on Val) >>> random: 37 usec (pick a random worker, using crypto:rand_uniform/2) >>> direct: 18 usec (gproc_pool:pick(Pool, N), where N modulo Size selects worker) >>> claim: 76 usec (claim the first available worker, apply a fun, then release) >>> >>> I think the per-selection cost is acceptable as-is, but could perhaps be improved (esp. the 'random' strategy is surprisingly expensive). All the selection work is done in the caller's process, BTW - no communication with the gproc or gproc_pool servers (except for admin tasks). >>> >>> The 'claim' strategy is also surprisingly expensive. I believe it's because I'm using gproc:select/3 to find the first free worker. Note also that it results in an extremely uneven distribution. That's obviously because the test run claims the first available worker and then releases it before iterating - it's always going to select the first worker.) >>> >>> https://github.com/uwiger/gproc/blob/master/doc/gproc_pool.md >>> >>> Feedback welcome, be it with performance tips, usability tips, or other. >>> >>> BR, >>> Ulf W >>> >>> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >>> http://feuerlabs.com >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> > > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From pablo.platt@REDACTED Wed Jun 5 12:34:20 2013 From: pablo.platt@REDACTED (pablo platt) Date: Wed, 5 Jun 2013 13:34:20 +0300 Subject: [erlang-questions] ANN: gproc_pool + some performance tidbits In-Reply-To: <462D5AF9-3792-49DD-8520-149A3977016D@feuerlabs.com> References: <043F5F0A-9BB4-4AE6-ABFC-9BBD035DB478@alumni.caltech.edu> <2F0CE77B-6960-435E-A363-682D9B306EAE@feuerlabs.com> <462D5AF9-3792-49DD-8520-149A3977016D@feuerlabs.com> Message-ID: Thanks for the explanation. On Wed, Jun 5, 2013 at 8:47 AM, Ulf Wiger wrote: > > On 4 Jun 2013, at 21:30, pablo platt wrote: > > What's the use case for workers in the pool? > Is it only for distributing a task or also for implementing a pool of DB > connections like https://github.com/devinus/poolboy ? > > > > I believe it *is* fairly similar to poolboy, but I thought it would be > consistent with the gproc philosophy to have a pool concept in gproc, since: > > - One of the things you need to do in a worker pool implementation is to > keep track of the worker processes, and gproc is good at this > > - A benefit of using gproc is that you can get some > query/debugging/monitoring capabilities for free. For example, after > setting up my test pool (gproc_pool:setup_test_pool/3), I can use the > following stock gproc function: > > 2> gproc_pool:setup_test_pool(mypool,round_robin,[]). > add_worker(mypool, a) -> 1; Ws = [{a,1}] > add_worker(mypool, b) -> 2; Ws = [{a,1},{b,2}] > add_worker(mypool, c) -> 3; Ws = [{a,1},{b,2},{c,3}] > add_worker(mypool, d) -> 4; Ws = [{a,1},{b,2},{c,3},{d,4}] > add_worker(mypool, e) -> 5; Ws = [{a,1},{b,2},{c,3},{d,4},{e,5}] > add_worker(mypool, f) -> 6; Ws = [{a,1},{b,2},{c,3},{d,4},{e,5},{f,6}] > [true,true,true,true,true,true] > 3> gproc:in > info/1 info/2 init/1 > 3> catch gproc:info(self()). > [{gproc,[{{n,l,[gproc_pool,mypool,1,a]},0}, > {{n,l,[gproc_pool,mypool,2,b]},0}, > {{n,l,[gproc_pool,mypool,3,c]},0}, > {{n,l,[gproc_pool,mypool,4,d]},0}, > {{n,l,[gproc_pool,mypool,5,e]},0}, > {{n,l,[gproc_pool,mypool,6,f]},0}]}, > {current_function,{erl_eval,do_apply,6}}, > {initial_call,{erlang,apply,2}}, > {status,running}, > {message_queue_len,0}, > ?] > > Thus, from the 'gproc footprint' of the process, I can readily tell that > it's a worker in the pool 'mypool' (even if I'm not familiar with the > gproc_pool concept, I can guess from convention that the first part of the > name is a module name). > > The whole idea of gproc was in fact to provide a single set of patterns > that I saw appearing in many different places in our code, in lots of > different implementations. So in a sense, practically everything that gproc > provides is stuff that people have implemented before, in reasonably > similar ways. :) Hopefully with gproc, some user code can become simpler, > more debuggable and a bit more uniform. > > Why workers has names? > I know I can just give them names such as 0,1,2... but trying to > understand the rational. > > > I thought it was a useful layer of abstraction. > > The performance of the pool is somewhat dependent on the spread of workers > across the available slots (especially if the pool is half-full, and > hashing or random selection is used). The workers themselves only need to > know what they are to call themselves as they connect to the pool. Whoever > manages the pool can control the positioning of each worker. > > > As always, I'm sure this functionality will be a major part in my server > like everything else in gproc, > even if I still don't know why ;) > > > Haha! This reminds me of the first design review meeting at Ericsson where > gproc's predecessor sysproc was up for review. The chairman of the meeting > said "I guess we'll approve it, even though I don't understand what it's > for". :) > > It was a good decision, I thought? > > BR, > Ulf W > > > > Thanks > > > > > On Tue, Jun 4, 2013 at 10:24 PM, Ulf Wiger wrote: > >> >> On 4 Jun 2013, at 18:52, ANTHONY MOLINARO wrote: >> >> Hi Ulf, >> >> Have you done any concurrent tests? I only ask because I've seen our own >> pooling code (https://github.com/openx/gen_server_pool) have issues >> under load. Now in our case >> it's because of a single gen_server acting as a dispatch layer, which >> should not be the >> case for gproc as IIRC it uses ets to provide for fast concurrent access >> (something also >> done in a novel way by https://github.com/ferd/dispcount/ which I keep >> meaning to try >> out), but I'd be curious to know if you've done any concurrent testing >> which shows that. >> >> >> I hadn't, but did so now. >> >> Spawning N clients, which run 1000 iterations each, on e.g. a round_robin >> pool: >> >> N Avg usec/iteration >> 1 37 >> 10 250 >> 100 1630 >> 1000 18813 >> >> Of course, this was a pretty nasty test, with all processes banging away >> at the pool as fast as they possibly could. If you want frequent mutex >> conflicts, that's probably as good a way as any to provoke them. >> >> When I insert a random sleep (0-50 ms) between each iteration, time each >> pick request and collect the averages, 100 concurrent workers pay on >> average 50 usec per selection. For 1000 concurrent workers, the average >> rises to 60 usec. >> >> The corresponding average for the hash pool and 1000 concurrent workers >> is 20 usec. >> >> (All on my Macbook Air) >> >> >> I think the number of pool implementations in erlang has probably finally >> surpassed >> the number of json parsers ;) >> >> >> Well, that tends to happen with fun and reasonably well-bounded problems. >> ;) >> >> BR, >> Ulf W >> >> >> -Anthony >> >> On Jun 4, 2013, at 2:18 AM, Ulf Wiger wrote: >> >> >> I pushed a new gproc component called gproc_pool the other day. >> >> The main idea, apart from wanting to see how well it would work, was that >> I wanted to be able to register servers with gproc and then have an >> efficient way of pooling between them. A benefit of using gproc throughout >> is that the registration objects serve as a 'footprint' for each process - >> by listing the gproc entities for each process, you can tell a lot about >> its purpose. >> >> The way gproc_pool works is that: >> 1. You define a pool, by naming it, and optionally specifying its size >> (gproc_pool:new(Pool) | gproc_pool:new(Pool, Type, Options)) >> 2. You add worker names to the pool >> (gproc_pool:add_worker(Pool, Name)) >> 3. Your servers each connect to a given name >> (gproc_pool:connect_worker(Pool, Name)) >> 4. Users pick a worker for each request (gproc_pool:pick(Pool)) >> >> My little test code indicates that the different load-balancing >> strategies perform a bit differently: >> >> (https://github.com/uwiger/gproc/blob/master/src/gproc_pool.erl#L843) >> >> (Create a pool, add 6 workers and iterate 100k times, >> incrementing a gproc counter for each iteration.) >> >> 3> gproc_pool:test(100000,round_robin,[]). >> worker stats (848): >> [{a,16667},{b,16667},{c,16667},{d,16667},{e,16666},{f,16666}] >> {2801884,ok} >> 4> gproc_pool:test(100000,hash,[]). >> worker stats (848): >> [{a,16744},{b,16716},{c,16548},{d,16594},{e,16749},{f,16649}] >> {1891517,ok} >> 5> gproc_pool:test(100000,random,[]). >> worker stats (848): >> [{a,16565},{b,16542},{c,16613},{d,16872},{e,16727},{f,16681}] >> {3701011,ok} >> 6> gproc_pool:test(100000,direct,[]). >> worker stats (848): >> [{a,16667},{b,16667},{c,16667},{d,16667},{e,16666},{f,16666}] >> {1766639,ok} >> 11> gproc_pool:test(100000,claim,[]). >> worker stats (848): >> [{a,100000},{b,0},{c,0},{d,0},{e,0},{f,0}] >> {7569425,ok} >> >> >> The worker stats show how evenly the workers were selected, >> and the {Time, ok} comes from timer:tc/3, i.e. Time/100000 is the >> per-iteration cost: >> >> round_robin: 28 usec (maintain a 'current' counter, modulo Size) >> hash: 19 usec (gproc_pool:pick(Pool, Val), hash on Val) >> random: 37 usec (pick a random worker, using crypto:rand_uniform/2) >> direct: 18 usec (gproc_pool:pick(Pool, N), where N modulo Size selects >> worker) >> claim: 76 usec (claim the first available worker, apply a fun, then >> release) >> >> I think the per-selection cost is acceptable as-is, but could perhaps be >> improved (esp. the 'random' strategy is surprisingly expensive). All the >> selection work is done in the caller's process, BTW - no communication with >> the gproc or gproc_pool servers (except for admin tasks). >> >> The 'claim' strategy is also surprisingly expensive. I believe it's >> because I'm using gproc:select/3 to find the first free worker. Note also >> that it results in an extremely uneven distribution. That's obviously >> because the test run claims the first available worker and then releases it >> before iterating - it's always going to select the first worker.) >> >> https://github.com/uwiger/gproc/blob/master/doc/gproc_pool.md >> >> Feedback welcome, be it with performance tips, usability tips, or other. >> >> BR, >> Ulf W >> >> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >> http://feuerlabs.com >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. >> http://feuerlabs.com >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. > http://feuerlabs.com > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.santos@REDACTED Wed Jun 5 15:23:08 2013 From: michael.santos@REDACTED (Michael Santos) Date: Wed, 5 Jun 2013 09:23:08 -0400 Subject: [erlang-questions] low level packet access from erlang In-Reply-To: <51AE7D17.2050807@research.att.com> References: <515C6E13.1050406@research.att.com> <20130403212407.GA25006@ioctl> <51ADE6E1.4030205@research.att.com> <20130604153054.GA31551@ioctl> <51AE7341.4090200@research.att.com> <51AE7D17.2050807@research.att.com> Message-ID: <20130605132308.GA32186@ioctl> On Tue, Jun 04, 2013 at 07:49:43PM -0400, Garry Hodgson wrote: > ha! success! Awesome. > r3:init: procket got {ok,13} > r3:init:got socket #Port<0.1538> > r3:init: dev is <0.58.0> > r3:init: ip address is "135.207.243.86" > > poking around just now, i noticed that while i had > a sudoers entry, the permissions on my procket > were not what you specified in README: > > [root@REDACTED ~]# ls -l /usr/local/sut/sut/deps/procket/priv/procket > -rwxr-xr-x 1 root root 11712 Jun 4 13:37 > /usr/local/sut/sut/deps/procket/priv/procket > > so i changed them: > > [root@REDACTED ~]# chmod 4750 /usr/local/sut/sut/deps/procket/priv/procket > [root@REDACTED ~]# ls -l /usr/local/sut/sut/deps/procket/priv/procket > -rwsr-x--- 1 root root 11712 Jun 4 13:37 > /usr/local/sut/sut/deps/procket/priv/procket > > and your tests worked: > > (r3@REDACTED)2> tuncer:create( <<"tun9">> ). > {ok,<0.64.0>} > (r3@REDACTED)3> procket:dev("net/tun"). > {ok,16} > > as did my code: > > r3:init: procket got {ok,13} > r3:init:got socket #Port<0.1538> > r3:init: dev is <0.58.0> > r3:init: ip address is "135.207.243.86" > > i think i recall doing this before, but it may have been on a different > machine. and it's odd that it helped, given that i've been running > as root. That's really weird. I changed the code to be "smarter" about calling sudo, I'll have to check if I broke something. > in any case, i appear to be unstuck. i'll likely have more > questions, but i'm good for now. thanks for your help, > and your willingness to be helpful. > > this is gonna be fun. Ask away and if you have any comments/suggestions, please let me know! > On 06/04/2013 07:07 PM, Garry Hodgson wrote: > >On 06/04/2013 11:30 AM, Michael Santos wrote: > >>Looks like you have everything set up correctly. Just to make sure, I > >>tried creating a tun device as root: > >> > >> ~/src/erlang/tunctl(master)$ sudo ./start.sh > >> Erlang R16B01 (erts-5.10.2) [source-e72043e] [smp:2:2] > >>[async-threads:10] [hipe] [kernel-poll:false] > >> Eshell V5.10.2 (abort with ^G) > >> 1> tuncer:create( <<"tun0">> ). > >> {ok,<0.35.0>} > >> 2> > >hmmm...that gives me the aforementioned error. > >>You can try creating the tun device manually: > >> > >> $ sudo ./start.sh > >> 1> {ok, FD} = procket:dev("net/tun"). > >> {ok,9} > >--> erl -pa /usr/local/sut/sut/deps/pkt/ebin -pa > >/usr/local/sut/sut/ebin -pa /usr/local/sut/sut/deps/procket/ebin > >Erlang R15B02 (erts-5.9.2) [source] [64-bit] [smp:8:8] > >[async-threads:0] [hipe] [kernel-poll:false] > > > >Eshell V5.9.2 (abort with ^G) > >1> procket:dev("net/tun"). > >{error,eperm} > > > >--> ls -l /dev/net/tun > >crw-rw-rw- 1 root root 10, 200 Jun 3 17:23 /dev/net/tun > >--> getcap /usr/local/lib/erlang/erts-5.9.2/bin/beam > >/usr/local/lib/erlang/erts-5.9.2/bin/beam.smp > >/usr/local/lib/erlang/erts-5.9.2/bin/beam = cap_net_admin+ep > >/usr/local/lib/erlang/erts-5.9.2/bin/beam.smp = cap_net_admin+ep > > > >> 2> procket:ioctl(FD, 1074025674, <<"tun0", 0:96, > >>1:2/native-integer-unit:8, 0:112>>). > >>{ok,<<116,117,110,48,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0, > >> 0,0,0,0,0,...>>} > >> > >>Are you able to create a tun device using other utilities? This should > >>work: > >> > >> ip tuntap add mode tun foo > > > >that does not: > > > >--> ip tuntap add mode tun foo > >Object "tuntap" is unknown, try "ip help". > > > >but i can create tun using tunctl: > > > >--> tunctl -n -u garry -g garry > >Set 'tun0' persistent and owned by uid 1234 gid 1234 > > > >> > >>Is selinux enabled? > >i believe not: > > > >--> /usr/sbin/getenforce > >Disabled > > > >puzzling. > > > > > -- > Garry Hodgson > AT&T Chief Security Office (CSO) > > "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." > From garry@REDACTED Wed Jun 5 15:36:05 2013 From: garry@REDACTED (Garry Hodgson) Date: Wed, 05 Jun 2013 09:36:05 -0400 Subject: [erlang-questions] low level packet access from erlang In-Reply-To: <20130605132308.GA32186@ioctl> References: <515C6E13.1050406@research.att.com> <20130403212407.GA25006@ioctl> <51ADE6E1.4030205@research.att.com> <20130604153054.GA31551@ioctl> <51AE7341.4090200@research.att.com> <51AE7D17.2050807@research.att.com> <20130605132308.GA32186@ioctl> Message-ID: <51AF3EC5.9030303@research.att.com> On 06/05/2013 09:23 AM, Michael Santos wrote: > That's really weird. I changed the code to be "smarter" about calling > sudo, I'll have to check if I broke something. right? especially because i had the sudoers entries in place, and i was already root. go figure. -- Garry Hodgson AT&T Chief Security Office (CSO) "This e-mail and any files transmitted with it are AT&T property, are confidential, and are intended solely for the use of the individual or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please notify the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited." From jleivent@REDACTED Wed Jun 5 18:31:42 2013 From: jleivent@REDACTED (Jonathan Leivent) Date: Wed, 05 Jun 2013 12:31:42 -0400 Subject: [erlang-questions] mnesia question: best way to index on combined/nested fields? Message-ID: <51AF67EE.2010002@comcast.net> I would like to set up an mnesia table on a record type that contains two data fields, such that there are indexes on each independent field as well as on the tuple of both fields - actually, the tuple of both fields should be the key. The table represents the results of a concurrent 2-D cross product calculation where the most common query is on individual elements, but occasionally there are queries on individual rows and/or columns. Is there a way to do this without replicating fields? For example, suppose the record type is: -record(foo, {x, y}). It seems like I would instead need to use a different record type augmented with redundant fields, like this: -record(bar, {foo::#foo{}, x, y}). where R#bar.x is always kept the same as R#bar.foo#foo.x (similarly for y). Then I can create the table: mnesia:create_table(bar, [{index, [x, y]},...]). so that a whole foo record is the key, while (copies of) x and y are indexed separately. That's not that bad, but something I see in the mnesia doc suggests that this might not be the best way to handle such a case. The particular doc line is in the mnesia:create_table section under the index attribute, and it says: "{index, Intlist}, where Intlist is a list of attribute names (atoms) or record fields for which Mnesia shall build and maintain an extra index table." In particular the way it says both "attribute names (atoms)" and "record fields" separately. Does that mean something like the following is possible, where the table indexes on nested record fields?: -record(baz, {foo::#foo{}}). %% no redundant fields mnesia:create_table(baz, [{index, [#baz.foo#foo.x, #baz.foo#foo.y]},...]). Or, does that "or" in that line of doc really mean "also known as" - in which case I guess I do need to use redundant fields... It would really be nice to be able to do something more direct, like the following, where the key is explicitly defined as the tuple of fields x and y, so that they can also be indexed separately: mnesia:create_table(foo, [{key, {x, y}}, {index, [x, y]}, ...]). -- Jonathan From robert.virding@REDACTED Wed Jun 5 22:48:53 2013 From: robert.virding@REDACTED (Robert Virding) Date: Wed, 5 Jun 2013 21:48:53 +0100 (BST) Subject: [erlang-questions] gen_server doesn't respond In-Reply-To: <18FB305D-6645-4EB0-91D5-B44A3E0C0FD6@gmail.com> Message-ID: <1738567306.139934343.1370465333194.JavaMail.root@erlang-solutions.com> Funny discussion this. In this case I think Dmitry hit the nail on the head when he said that {current_function,{ gen,do_call,4 }} exactly describes what your problem is. The gen_server is suspended waiting in a gen_server:call to another server which is not replying. (gen_server:call -> gen:call/4 -> gen:do_call/4) With a little basic tracing (sys:trace) you should be able to work out where/why the server is calling another server and why it is hanging. Robert ----- Original Message ----- > From: "Dmitry Kolesnikov" > To: "Sungjin Park" > Cc: "erlang-questions" > Sent: Tuesday, 4 June, 2013 10:45:40 AM > Subject: Re: [erlang-questions] gen_server doesn't respond > Hello, > > {current_function,{ gen,do_call,4 }} > > Your process made a call to another process with infinite timeout. > The process do not respond because you got a deadlock. > You have to check: > * no circular dependencies on process call > * the parent supervisor is not called from process init > * no infinite gen_server:call is used at your api. > - Dmitry > On Jun 4, 2013, at 4:54 AM, "Park, Sungjin" < jinni.park@REDACTED > > wrote: > > Shell input/output logs below. > > > ============================ > > > 4> process_info(pid(0,453,4757)). > > > [{current_function,{ gen,do_call,4 }}, > > > {initial_call,{proc_lib,init_p,5}}, > > > {status, waiting }, > > > {message_queue_len,1}, > > > {messages,[ {'$gen_call',{<0.26172.4841>, > > > #Ref<0.0.2946.93178>}, > > > state} ]}, > > > {links,[<0.25337.4839>]}, > > > {dictionary,[{'$ancestors',[<0.452.4757>,<0.424.0>, > > > <0.423.0>,ranch_sup,<0.420.0>]}, > > > {'$initial_call',{my_session,init,1}}]}, > > > {trap_exit,true}, > > > {error_handler,error_handler}, > > > {priority,normal}, > > > {group_leader,<0.419.0>}, > > > {total_heap_size,1364}, > > > {heap_size,987}, > > > {stack_size,32}, > > > {reductions,1487}, > > > {garbage_collection,[{min_bin_vheap_size,46368}, > > > {min_heap_size,233}, > > > {fullsweep_after,65535}, > > > {minor_gcs,8}]}, > > > {suspending,[]}] > > > ============================= > > > As you can see the process has a message {..., state} in it's > > message > > queue but still in waiting state. It doesn't respond to > > gen_server:call(Pid, state) from this point on. The process_info > > shows that the process is now in gen:do_call/4 which is not under > > my > > control. The module has all the fallback handlers for > > handle_call/3, > > handle_cast/2, handle_info/2 etc, so that every message be matched. > > There are a couple of places in my project where gen_server:call/2 > > used against this process. I suppose that the calls should timeout > > in 5 sec by default as is stated in the gen_server specification. > > So > > there should be no such case that the process stuck indefinitely. > > > On Mon, Jun 3, 2013 at 7:39 PM, Aaron France < > > aaron.l.france@REDACTED > wrote: > > > > Does it make any synchronous calls to other gen_servers? > > > > > > On Mon, Jun 3, 2013 at 12:34 PM, Tim Watson < > > > watson.timothy@REDACTED > wrote: > > > > > > > Does the callback module execute any selective receives inside > > > > handlers perhaps? If not, are you sure the message has actually > > > > arrived in that process' mailbox, can you see unhandled > > > > messages > > > > in > > > > the message queue (I.e., length > 0 whilst waiting)? > > > > > > > > > > On 3 Jun 2013, at 10:34, "Park, Sungjin" < jinni.park@REDACTED > > > > > > > > > wrote: > > > > > > > > > > > Guys, > > > > > > > > > > > > > > > > > > > > > > I'm encountering cases when gen_server doesn't respond. > > > > > > > > > > > In such cases, erlang:process_info/1 shows the process state > > > > > 'waiting' with some messages in the message_queue. The > > > > > messages > > > > > have not been consumed for tens of minutes at least. Anybody > > > > > has > > > > > any idean about this? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > Park, Sungjin > > > > > > > > > > > ------------------------------------------------------------------------------------------------------------------- > > > > > > > > > > > Peculiar travel suggestions are dancing lessons from god. > > > > > > > > > > > -- The Books of Bokonon > > > > > > > > > > > ------------------------------------------------------------------------------------------------------------------- > > > > > > > > > > > _______________________________________________ > > > > > > > > > > > erlang-questions mailing list > > > > > > > > > > > erlang-questions@REDACTED > > > > > > > > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > > _______________________________________________ > > > > > > > > > > erlang-questions mailing list > > > > > > > > > > erlang-questions@REDACTED > > > > > > > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > -- > > > Park, Sungjin > > > ------------------------------------------------------------------------------------------------------------------- > > > Peculiar travel suggestions are dancing lessons from god. > > > -- The Books of Bokonon > > > ------------------------------------------------------------------------------------------------------------------- > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul@REDACTED Wed Jun 5 22:59:20 2013 From: paul@REDACTED (Paul Rubin) Date: Wed, 5 Jun 2013 13:59:20 -0700 Subject: [erlang-questions] handling crash on db connect Message-ID: I'm using gar1t's redis driver and find that if the redis server is running, redis:connect() causes the client to crash. try/catch doesn't handle it, so I think it's linking some other process which then crashes and propagates the exit. What's the right way to handle the crash, in order to report some reasonable error and/or do a sleep/retry loop? One obvious way is trapexit() and explicitly handle the exit signal, but I have the impression that this would be overly drastic in Erlang. Connection failure to a db should be a fairly routine error condition, so I feel like there's something I'm missing. Any advice? Thanks. Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul@REDACTED Wed Jun 5 22:59:54 2013 From: paul@REDACTED (Paul Rubin) Date: Wed, 5 Jun 2013 13:59:54 -0700 Subject: [erlang-questions] handling crash on db connect In-Reply-To: References: Message-ID: On Wed, Jun 5, 2013 at 1:59 PM, Paul Rubin wrote: > I'm using gar1t's redis driver and find that if the redis server is > running, > That was supposed to say, if the redis server is NOT running. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fredrikelinder@REDACTED Thu Jun 6 05:43:49 2013 From: fredrikelinder@REDACTED (Fredrik Linder) Date: Wed, 5 Jun 2013 20:43:49 -0700 (PDT) Subject: [erlang-questions] NIF and timing Message-ID: <186519ab-06cf-41e6-8258-e8c8e61ae537@googlegroups.com> Hello First, thanks for your help on previous questions :-) I have a NIF implementation that have a peculiar timing behavior (the NIF code is pretty simple, doing enif_get_tuple, enif_get_atom, enif_get_resource and some O(1) operations). The median time is 39 us (microseconds) 15666/16079 calls take < 100 us 16049/16079 calls take < 1000 us 16052/16079 calls take < 10000 us 16067/16079 calls take < 100000 us I'm running a list comprehension that gives me the time taken for each call: [begin A = os:timestamp(), doit(), io:format("~w ", [now:time_diff(os:timestamp(), A)]) end || _ <- lists:seq(1, 16#10000)]. I'm running R16B on a 4 core VM. Of course some lag due to process switching etc would be expected, but 12 of the 16k calls take more than 0.1 seconds to run. That seems way to much to me. (I get the same behavior when not doing the io:format/2 call). Any clue why? Thank you /Fredrik -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Thu Jun 6 11:57:46 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 6 Jun 2013 11:57:46 +0200 Subject: [erlang-questions] handling crash on db connect In-Reply-To: References: Message-ID: <2FEF06FD-78CC-4F13-AB52-27434F0564D9@erlang-solutions.com> On Jun 5, 2013, at 10:59 PM, Paul Rubin wrote: > I'm using gar1t's redis driver and find that if the redis server is running, redis:connect() causes the client to crash. try/catch doesn't handle it, so I think it's linking some other process which then crashes and propagates the exit. > > What's the right way to handle the crash, in order to report some reasonable error and/or do a sleep/retry loop? One obvious way is trapexit() and explicitly handle the exit signal, but I have the impression that this would be overly drastic in Erlang. Connection failure to a db should be a fairly routine error condition, so I feel like there's something I'm missing. > > Any advice? Thanks. > Rewrite the driver? It makes sense to return some kind of error tuple in this case because it is expected behaviour that you cannot establish a connection. Then you can crash on ok = redis:connect(?) which is better. Of course this is a matter of style, but I much prefer attempts at connections to be able to fail like this. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen > Paul > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From freza@REDACTED Thu Jun 6 12:54:59 2013 From: freza@REDACTED (Jachym Holecek) Date: Thu, 6 Jun 2013 06:54:59 -0400 Subject: [erlang-questions] handling crash on db connect In-Reply-To: <2FEF06FD-78CC-4F13-AB52-27434F0564D9@erlang-solutions.com> References: <2FEF06FD-78CC-4F13-AB52-27434F0564D9@erlang-solutions.com> Message-ID: <20130606105459.GA7415@circlewave.net> # Jesper Louis Andersen 2013-06-06: > On Jun 5, 2013, at 10:59 PM, Paul Rubin wrote: > > > I'm using gar1t's redis driver and find that if the redis server is running, redis:connect() causes the client to crash. try/catch doesn't handle it, so I think it's linking some other process which then crashes and propagates the exit. > > > > What's the right way to handle the crash, in order to report some reasonable error and/or do a sleep/retry loop? One obvious way is trapexit() and explicitly handle the exit signal, but I have the impression that this would be overly drastic in Erlang. Connection failure to a db should be a fairly routine error condition, so I feel like there's something I'm missing. > > > > Any advice? Thanks. > > > > Rewrite the driver? It makes sense to return some kind of error tuple in this case because it is expected behaviour that you cannot establish a connection. Then you can crash on > > ok = redis:connect(?) > > which is better. Of course this is a matter of style, but I much prefer attempts at connections to be able to fail like this. Yes. Probably good to have 'redis:connect(...)' retry connect periodically and send you a message when it's up -- not only connect succeeded but ideally some heartbeat roundtrip completed with reasonable latency too. When connection goes down, send a message about it and go back to reconnect state. Keep doing heartbeats every now and then while connection is up so you spot failures proactively. Can additionally leverage user traffic for link monitoring (ex- cessive request timeouts -- probably not good, whether it's failing link or overloaded peer you probably want to go away for a while). You could push this extra complexity to user code, but then you'll likely find that all users want it. Can't trust those dodgy networks after all, they always fail, sometimes in funny ways. BR, -- Jachym From jesper.louis.andersen@REDACTED Thu Jun 6 14:14:49 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 6 Jun 2013 14:14:49 +0200 Subject: [erlang-questions] handling crash on db connect In-Reply-To: <20130606105459.GA7415@circlewave.net> References: <2FEF06FD-78CC-4F13-AB52-27434F0564D9@erlang-solutions.com> <20130606105459.GA7415@circlewave.net> Message-ID: On Jun 6, 2013, at 12:54 PM, Jachym Holecek wrote: > Yes. Probably good to have 'redis:connect(...)' retry connect periodically and > send you a message when it's up -- not only connect succeeded but ideally some > heartbeat roundtrip completed with reasonable latency too. When connection > goes down, send a message about it and go back to reconnect state. Keep doing > heartbeats every now and then while connection is up so you spot failures > proactively. Can additionally leverage user traffic for link monitoring (ex- > cessive request timeouts -- probably not good, whether it's failing link > or overloaded peer you probably want to go away for a while). This is an interesting question in general. Do you want your API to be "connection" oriented and do a single db:connect, which is essentially a single-threaded way of working. Or do you provide an application handling a "service" which maintains an ongoing connection to the database, reconnecting if needed and so on. I think i prefer both. In some situations, one over the other is preferred. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen From zhengsyao@REDACTED Thu Jun 6 14:34:18 2013 From: zhengsyao@REDACTED (=?utf-8?Q?=22Siyao_Zheng=28=E9=83=91=E6=80=9D=E9=81=A5=29=22?=) Date: Thu, 6 Jun 2013 20:34:18 +0800 Subject: [erlang-questions] Send Port around distributed Erlang nodes Message-ID: Hi all, It is possible to send a port id to other node, but the receiving node can't operate on this port by the received port id. According to the spec of erlang:send/2, in order to send message to this port, the port must be registered with a name, but messages can be send to remote directly. Is it an intended design ? Do you think it is necessary to allow the following scene be possible: node A listens on a socket S, sends S to node B, and B accepts this S? Is there any considerations about this ability? Cheers Siyao Zheng From ttom.kelly@REDACTED Thu Jun 6 15:31:15 2013 From: ttom.kelly@REDACTED (tom kelly) Date: Thu, 6 Jun 2013 14:31:15 +0100 Subject: [erlang-questions] run-to-completion scheduling Message-ID: Hello List, Our partner company is investigating the Intel Data Plane Development Kit, with its run-to-completion scheduling. I don't know so much about the Erlang VM or scheduler so I'd be grateful for any insights, can our Erlang product that runs on their system get any of the optimisations available from the DPDK? Is it a simple matter of recompiling erlang on the target environment or would it involve some unfeasible redesigning it instead? Like I say, I don't know so much about this part of Erlang. So no need to be gentle if the question is nonsensical ;-) Thanks, //TTom. From bchesneau@REDACTED Thu Jun 6 15:53:57 2013 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 6 Jun 2013 15:53:57 +0200 Subject: [erlang-questions] packet_size option of decode_packet Message-ID: The doc says : {packet_size, integer()} Sets the max allowed size of the packet body. If the packet header indicates that the length of the packet is longer than the max allowed length, the packet is considered invalid. Default is 0 which means no size limit. But I wonder what does it means for http parsing. If I do {ok, Data} = gen_tcp:recv(Socket, 0, infinity), decode_packet(http_bin, Data, [{packet_size, 4096}]). What would happen if bytes_size(Data) > 4096 but the request line itself only take 1024 bytes ? - benoit From bchesneau@REDACTED Thu Jun 6 16:04:56 2013 From: bchesneau@REDACTED (Benoit Chesneau) Date: Thu, 6 Jun 2013 16:04:56 +0200 Subject: [erlang-questions] packet_size option of decode_packet In-Reply-To: References: Message-ID: ok sorry for the noise. Did a quick manual test and it seems to use that size for the maximum length of a request line which is good: 2> Test1. <<"GET / HTTP/1.1\r\nXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"...>> 3> erlang:decode_packet(http_bin, Test1, [{packet_size, 4092}]). {ok,{http_request,'GET',{abs_path,<<"/">>},{1,1}}, <<"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"...>>} 4> byte_size(Test1). 8208 5> erlang:decode_packet(http_bin, Test1, [{packet_size, 8}]). {error,invalid} 6> erlang:decode_packet(http_bin, Test1, [{packet_size, 16}]). {ok,{http_request,'GET',{abs_path,<<"/">>},{1,1}}, <<"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"...>>} - benoit On Thu, Jun 6, 2013 at 3:53 PM, Benoit Chesneau wrote: > The doc says : > > {packet_size, integer()} > Sets the max allowed size of the packet body. If the packet header > indicates that the length of the packet is longer than the max allowed > length, the packet is considered invalid. Default is 0 which means no > size limit. > > > But I wonder what does it means for http parsing. > > If I do > > {ok, Data} = gen_tcp:recv(Socket, 0, infinity), > decode_packet(http_bin, Data, [{packet_size, 4096}]). > > > What would happen if bytes_size(Data) > 4096 but the request line > itself only take 1024 bytes ? > > - benoit From ulf@REDACTED Thu Jun 6 16:47:04 2013 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 6 Jun 2013 16:47:04 +0200 Subject: [erlang-questions] run-to-completion scheduling In-Reply-To: References: Message-ID: I doubt that you can improve Erlang's performance itself with DPDK, but you can use Erlang as an orchestration layer on top of it, and use NIFs to link the two. Resource binaries in Erlang exist specifically so that you can reference e.g. data plane resources as if they were erlang objects. Fair warning: Run-to-completion scheduling works nicely for embarrassingly parallel problems, but sucks for orchestation problems. So Erlang and DPDK together can make for a good combo. BR, Ulf W Ulf Wiger, Feuerlabs, Inc. http://www.feuerlabs.com 6 jun 2013 kl. 15:31 skrev tom kelly : > Hello List, > > Our partner company is investigating the Intel Data Plane Development > Kit, with its run-to-completion scheduling. I don't know so much about > the Erlang VM or scheduler so I'd be grateful for any insights, can > our Erlang product that runs on their system get any of the > optimisations available from the DPDK? > > Is it a simple matter of recompiling erlang on the target environment > or would it involve some unfeasible redesigning it instead? > > Like I say, I don't know so much about this part of Erlang. So no need > to be gentle if the question is nonsensical ;-) > > Thanks, > //TTom. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From paul@REDACTED Thu Jun 6 17:38:07 2013 From: paul@REDACTED (Paul Rubin) Date: Thu, 6 Jun 2013 08:38:07 -0700 Subject: [erlang-questions] handling crash on db connect In-Reply-To: References: <2FEF06FD-78CC-4F13-AB52-27434F0564D9@erlang-solutions.com> <20130606105459.GA7415@circlewave.net> Message-ID: Jesper Louis Andersen wrote: Rewrite the driver? It makes sense to return some kind of error tuple in this case because it is expected behaviour that you cannot establish a connection. Then you can crash on ok = redis:connect(?) Thanks. It's odd that redis:connect() returns {ok, Connection} in the success case, which had me expecting it to return an error result in the failure case instead of crashing. I wonder if I could just be making some dumb error, and/or found a bug in the driver, rather than intended behavior. I've opened a github issue at https://github.com/gar1t/erlang-redis/issues/1 Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Thu Jun 6 17:58:44 2013 From: g@REDACTED (Garrett Smith) Date: Thu, 6 Jun 2013 10:58:44 -0500 Subject: [erlang-questions] handling crash on db connect In-Reply-To: References: <2FEF06FD-78CC-4F13-AB52-27434F0564D9@erlang-solutions.com> <20130606105459.GA7415@circlewave.net> Message-ID: Hi Paul, Sorry I'm not used to seeing questions about my Redis bindings graduate to the main list :) On Thu, Jun 6, 2013 at 10:38 AM, Paul Rubin wrote: > Jesper Louis Andersen wrote: > > Rewrite the driver? It makes sense to return some kind of error tuple in > this case because it is expected behaviour that you cannot establish a > connection. Then you can crash on > > ok = redis:connect(?) redis:connect is a wrapper for the process start_link. The return value is standard for an OTP process: either {ok, Pid} or {error, Reason}. > Thanks. It's odd that redis:connect() returns {ok, Connection} in the > success case, which had me expecting it to return an error result in the > failure case instead of crashing. > > I wonder if I could just be making some dumb error, and/or found a bug in > the driver, rather than intended behavior. Not a dumb error at all -- but neither a bug. This is by design and is pretty common for OTP processes. In particular, risky code gets executed in the context of the process (for the sake of proper isolation) and calling processes need to trap exit to deal with problems, or just let it all crash and get restarted by the supervisor. The question of how to handle connection problems can be tricky. I typically bake this into a "connection handler" type process that indeed traps exit and then figures out what to do -- other simply let's the client process exit propagate up to the supervisor. I'll typically have a retry logic that waits for a period of time after failures, logging attempts, errors, etc. This is obviously application specific (retry policies, logging, etc) and IMO doesn't belong in the lower level redis client process. Btw, this higher level process can also serve as a general wrapper to the Redis client, exposing an application specific interface. I can provide an example of this type of process -- or maybe something like this would be appropriate as a utility within the library. Garrett From paul@REDACTED Thu Jun 6 18:35:45 2013 From: paul@REDACTED (Paul Rubin) Date: Thu, 6 Jun 2013 09:35:45 -0700 Subject: [erlang-questions] handling crash on db connect In-Reply-To: References: <2FEF06FD-78CC-4F13-AB52-27434F0564D9@erlang-solutions.com> <20130606105459.GA7415@circlewave.net> Message-ID: On Thu, Jun 6, 2013 at 8:58 AM, Garrett Smith wrote: > Hi Paul, > Sorry I'm not used to seeing questions about my Redis bindings > graduate to the main list :) > Hey, thanks for the reply. I asked on the list first because I didn't want to bug you on github until I was more confident that I wasn't doing something silly. > redis:connect is a wrapper for the process start_link. The return > value is standard for an OTP process: either {ok, Pid} or {error, > Reason}. > Yes, this is what I expected, but in fact when I say A = redis:connect(). when the redis server is not running, I get a crash instead of {error,Reason}. Is that what's supposed to happen? I'm still not clear on this, as it conflicts with what you say further down. What's the point of returning {ok, Pid} instead of just Pid in the non-error case, if there's no possibility of returning anything without ok, in the error case? > Not a dumb error at all -- but neither a bug. This is by design and is > pretty common for OTP processes. In particular, risky code gets > executed in the context of the process (for the sake of proper > isolation) and calling processes need to trap exit to deal with > problems, or just let it all crash and get restarted by the > supervisor. > OK, I guess I can trap exit, but I had thought of trapexit itself as being risky and generally best left to the OTP supervision libraries except for special circumstances, and a database being down is relatively normal. The "let it crash" approach would be {ok, Pid} = redis:connect(). which would crash with a pattern match failure in the case of an error return. > > The question of how to handle connection problems can be tricky. I > typically bake this into a "connection handler" type process that > indeed traps exit and then figures out what to do -- other simply > let's the client process exit propagate up to the supervisor. I'll > typically have a retry logic that waits for a period of time after > failures, logging attempts, errors, etc. > Yes, that seems like the right thing, retry every few seconds until the db is back. I just hadn't thought of trapping exit as part of it (as opposed to just checking for error value). I actually do have a separate gen_server making a persistent connection in its init/1 and holding onto it, and then other parts of my program call this gen_server which in turn makes the redis call. When the init crashes, my top supervisor restarts the gen_server immediately, this repeats until MaxR runs out, and the whole VM crashes. It came as a shock that a fairly routine error case could cause this to happen. I find myself wishing for a general additional OTP supervision strategy (one_for_one_delay, say) that on crash would attempt restart no more than once per retry period (e.g. 1 second). I had kind of thought that was what MaxT does, but I guess not. > > I can provide an example of this type of process -- or maybe something > like this would be appropriate as a utility within the library This would be great, thanks! Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Thu Jun 6 18:49:29 2013 From: g@REDACTED (Garrett Smith) Date: Thu, 6 Jun 2013 11:49:29 -0500 Subject: [erlang-questions] handling crash on db connect In-Reply-To: References: <2FEF06FD-78CC-4F13-AB52-27434F0564D9@erlang-solutions.com> <20130606105459.GA7415@circlewave.net> Message-ID: On Thu, Jun 6, 2013 at 11:35 AM, Paul Rubin wrote: > On Thu, Jun 6, 2013 at 8:58 AM, Garrett Smith wrote: >> >> Hi Paul, >> Sorry I'm not used to seeing questions about my Redis bindings >> graduate to the main list :) > > > Hey, thanks for the reply. I asked on the list first because I didn't want > to bug you on github until I was more confident that I wasn't doing > something silly. > >> >> redis:connect is a wrapper for the process start_link. The return >> value is standard for an OTP process: either {ok, Pid} or {error, >> Reason}. > > > Yes, this is what I expected, but in fact when I say > A = redis:connect(). > when the redis server is not running, I get a crash instead of > {error,Reason}. Is that what's supposed to happen? I'm still not clear on > this, as it conflicts with what you say further down. What's the point of > returning {ok, Pid} instead of just Pid in the non-error case, if there's no > possibility of returning anything without ok, in the error case? This is an excellent question! The result {ok, Pid} simply means that the process has started and initialized. You'll get {error, Reason} if there was a problem starting the process (e.g. you're using a registered name that is already running). If you're curious about the API, take a look around the gen_server docs, in particular the start_link call: http://erlang.org/doc/man/gen_server.html#start_link-3 In the case of this Redis client, the process has started, but it crashed. Because this is a start_link (emphasis on link) your calling process also crashes because of the link. A call to start, e.g. would not trigger this -- you'd get {ok, Pid} return value, and then the process Pid would immediately terminate. I didn't implement any of the start functions for the client process. It's arguably bad form to not implement parallel start functions to each start_link -- but the cases where start_link is *wrong* are quite rare (none occur to me actually :) >> Not a dumb error at all -- but neither a bug. This is by design and is >> pretty common for OTP processes. In particular, risky code gets >> executed in the context of the process (for the sake of proper >> isolation) and calling processes need to trap exit to deal with >> problems, or just let it all crash and get restarted by the >> supervisor. > > > OK, I guess I can trap exit, but I had thought of trapexit itself as being > risky and generally best left to the OTP supervision libraries except for > special circumstances, and a database being down is relatively normal. The > "let it crash" approach would be > {ok, Pid} = redis:connect(). > which would crash with a pattern match failure in the case of an error > return. Yes, though it's not the end of the world to handle crashed processes explicitly -- supervisors have extremely naive recovery policies and it's not uncommon to have to implement something fancier. In the case of server connections, e.g. you almost always want to wait for some period before retrying, lest you DoS your servers :) >> >> >> The question of how to handle connection problems can be tricky. I >> typically bake this into a "connection handler" type process that >> indeed traps exit and then figures out what to do -- other simply >> let's the client process exit propagate up to the supervisor. I'll >> typically have a retry logic that waits for a period of time after >> failures, logging attempts, errors, etc. > > > Yes, that seems like the right thing, retry every few seconds until the db > is back. I just hadn't thought of trapping exit as part of it (as opposed > to just checking for error value). I actually do have a separate gen_server > making a persistent connection in its init/1 and holding onto it, and then > other parts of my program call this gen_server which in turn makes the redis > call. When the init crashes, my top supervisor restarts the gen_server > immediately, this repeats until MaxR runs out, and the whole VM crashes. It > came as a shock that a fairly routine error case could cause this to happen. > I find myself wishing for a general additional OTP supervision strategy > (one_for_one_delay, say) that on crash would attempt restart no more than > once per retry period (e.g. 1 second). I had kind of thought that was what > MaxT does, but I guess not. >> >> >> I can provide an example of this type of process -- or maybe something >> like this would be appropriate as a utility within the library > > > This would be great, thanks! I'll do this -- thanks for pointing this out! This is a pretty central problem that I glossed over. I'll update the github issue when it's pushed. Garrett From g@REDACTED Thu Jun 6 19:02:05 2013 From: g@REDACTED (Garrett Smith) Date: Thu, 6 Jun 2013 12:02:05 -0500 Subject: [erlang-questions] handling crash on db connect In-Reply-To: References: <2FEF06FD-78CC-4F13-AB52-27434F0564D9@erlang-solutions.com> <20130606105459.GA7415@circlewave.net> Message-ID: On Thu, Jun 6, 2013 at 11:35 AM, Paul Rubin wrote: > Yes, that seems like the right thing, retry every few seconds until the db > is back. I just hadn't thought of trapping exit as part of it (as opposed > to just checking for error value). I actually do have a separate gen_server > making a persistent connection in its init/1 and holding onto it, and then > other parts of my program call this gen_server which in turn makes the redis > call. When the init crashes, my top supervisor restarts the gen_server > immediately, this repeats until MaxR runs out, and the whole VM crashes. It > came as a shock that a fairly routine error case could cause this to happen. > I find myself wishing for a general additional OTP supervision strategy > (one_for_one_delay, say) that on crash would attempt restart no more than > once per retry period (e.g. 1 second). I had kind of thought that was what > MaxT does, but I guess not. I missed this part in my previous reply and I think it deserves some comment. What you observed I think is a very healthy problem -- a surprising catastrophic failure! This is a thing of beauty because it calls attention to a serious problem: you're relying on something that suddenly isn't working. Rather than lure you into a false sense of confidence, Erlang's default answer is to STOP. Now what? Dunno, but it got your attention :) If you start to look at your Erlang applications as ecosystems of independent services, you can start to think about shoring up each service to improve its availability, performance, etc. -- just as one might in a service oriented architecture. In the case of your Redis service, you want something that advocates for your Redis DB availability. That advocate (an Erlang process) can deal with connections, retries, error handling, etc. as an intelligent facade to the underlying Redis client process (which can some and go at any time depending on network, Redis server availability, etc.) I'll highlight this idea in the example I provide for this connection problem. Garrett From paul@REDACTED Thu Jun 6 21:01:28 2013 From: paul@REDACTED (Paul Rubin) Date: Thu, 6 Jun 2013 12:01:28 -0700 Subject: [erlang-questions] handling crash on db connect In-Reply-To: References: <2FEF06FD-78CC-4F13-AB52-27434F0564D9@erlang-solutions.com> <20130606105459.GA7415@circlewave.net> Message-ID: On Thu, Jun 6, 2013 at 9:49 AM, Garrett Smith wrote: > >> redis:connect is a wrapper for the process start_link.... > The result {ok, Pid} simply means that the process has started and > initialized. Hey, thanks again, and aha, ok, this makes sense in terms of how start_link behaves. It still comes as a surprise that redis:connect creates such a link though. My program (typical for Erlang) accepts 1000's of client connections with a separate process for each one. It uses redis for user authentication, so a new connection arrives, there is a redis lookup, and then the connection stays open "forever" (weeks, months...) without accessing redis again. If redis is down, I'd expect new authentication attempts to fail, but old connections to keep running as long as they don't try to talk to redis. Redis is never supposed to be down so I could imagine having this program run for a long time without ever encountering the issue. I don't have an automatic test for handling redis being down (I guess I should write one). I noticed this with a manual test, but I thought part of the idea of OTP was to handle problems gracefully, including software bugs resulting from error conditions that weren't thought of during testing. I didn't implement any of the start functions for the client process. > It's arguably bad form to not implement parallel start functions to > each start_link -- but the cases where start_link is *wrong* are quite > rare (none occur to me actually :) > start_link seems wrong to me in this case, but I still find OTP somewhat murky so I can't claim to have that good intuition about what it should or shouldn't do. > > Yes, though it's not the end of the world to handle crashed processes > explicitly -- supervisors have extremely naive recovery policies and > it's not uncommon to have to implement something fancier. I wonder if what's really going on here is a gap in the available supervision strategies. I was a bit surprised to learn from the supervisor docs, "To prevent a supervisor from getting into an infinite loop of child process terminations and restarts, a *maximum restart frequency* is defined using two integer values MaxR and MaxT. If more than MaxR restarts occur within MaxT seconds, the supervisor terminates all child processes and then itself." I had somehow thought that if MaxR restarts happened in MaxT seconds, the supervisor would just sleep until MaxT seconds had passed, then start retrying again (i.e. limit the frequency rather than the absolute count). It does seem to me there should be an option for something like that. Restarts due to external process failures seem like a common situation to want to deal with, so (in my naive expectation) I'd think OTP should implement an infinite sleep/retry loop as one of its restart strategies. > In the case of server connections, e.g. you almost always want to wait for > some > period before retrying, lest you DoS your servers :) > Yes. This is the common situation referred to above ;) > > I'll do this -- thanks for pointing this out! This is a pretty central > problem that I glossed over. I'll update the github issue when it's pushed Great! Thanks! (re followup message): What you observed I think is a very healthy problem -- a surprising catastrophic failure! This is a thing of beauty because it calls attention to a serious problem: you're relying on something that suddenly isn't working. Rather than lure you into a false sense of confidence, Erlang's default answer is to STOP. Now what? Dunno, but it got your attention :) OK, that is another case of my expectations being wrong, but it calls into question why I went to the trouble of learning Erlang in the first place ;) Just about all other languages stop the program when something fails, but I thought the idea of Erlang was to keep going no matter what! Of course I want it to get my attention, but I thought that it did that by writing "CRASH REPORT" into the sasl log in ALL CAPS so I could grep for it the next morning, instead of blowing up my entire service when a non-essential part of the program hits a glitch. The whole idea of supervision, hot code patches, appups, relups, etc. is to be able to write programs that are just NEVER DOWN, even if the code has bugs. Anything that makes the VM crash for reasons short of hardware meltdown seems wrong from that point of view. If you start to look at your Erlang applications as ecosystems of independent services, you can start to think about shoring up each service to improve its availability, performance, etc. -- just as one might in a service oriented architecture. In the case of your Redis service, you want something that advocates for your Redis DB availability. That advocate (an Erlang process) can deal with connections, retries, error handling, etc. as an intelligent facade to the underlying Redis client process (which can some and go at any time depending on network, Redis server availability, etc.) Of course I'd expect to have to shore up the individual services as problems with them arose and patterns got identified, but in the meanwhile I'd expect a service being down to just mean I couldn't use it. Like if Github is down for some reason, I'd expect to get a connection error if I try to access it from the office, but I wouldn't expect Github's outage to set off a demolition for the whole building my office is in. That's sort of what start_link does, from what I gather. The failure path should instead be isolated by default, propagating only to the parts of the code that actually depend on the things that failed. So it seems to me, using start_link in a client isn't right, unless the fates of the two interacting processes are actually entwined in a way that neither one could conceivably run without the other. Other independent processes should be able to keep running. Regards Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Thu Jun 6 21:28:21 2013 From: g@REDACTED (Garrett Smith) Date: Thu, 6 Jun 2013 14:28:21 -0500 Subject: [erlang-questions] handling crash on db connect In-Reply-To: References: <2FEF06FD-78CC-4F13-AB52-27434F0564D9@erlang-solutions.com> <20130606105459.GA7415@circlewave.net> Message-ID: On Thu, Jun 6, 2013 at 2:01 PM, Paul Rubin wrote: > On Thu, Jun 6, 2013 at 9:49 AM, Garrett Smith wrote: > >> I didn't implement any of the start functions for the client process. >> It's arguably bad form to not implement parallel start functions to >> each start_link -- but the cases where start_link is *wrong* are quite >> rare (none occur to me actually :) > > start_link seems wrong to me in this case, but I still find OTP somewhat > murky so I can't claim to have that good intuition about what it should or > shouldn't do. My fault -- I didn't make this clear at all! In fact, when there's a connect error from the client, you *will* get an error result (e.g. {error,{connect,econnrefused}}). However, because the calling process is linked, you never see it -- you crash. I'll add the start functions to redis_client and use those instead with redis:connect (which is just a facade anyway -- arguably you should use redis_client:start/start_link directly). This will be much less surprising :) Garrett From davidnwelton@REDACTED Thu Jun 6 21:48:44 2013 From: davidnwelton@REDACTED (David Welton) Date: Thu, 6 Jun 2013 21:48:44 +0200 Subject: [erlang-questions] erlexec debugging Message-ID: Hi guys, I've stumbled into what's feeling like a rat's nest with erlexec: https://github.com/saleyn/erlexec/issues/11 And I'm beginning to wonder if I'm the only one out there really putting it through its paces... Anyone out there using this code? Do you rely on it to kill stuff in more complex situations, like, say, where the child process catches SIGTERM and shuts down, or needs SIGKILL'ing or something like that? erlexec s a nice bit of code because it covers a huge weakness with core Erlang, which is pretty lacking in the handling-external-processes department. I'll be working on it some tomorrow, but other eyeballs would be welcome! -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From paul@REDACTED Thu Jun 6 23:24:50 2013 From: paul@REDACTED (Paul Rubin) Date: Thu, 6 Jun 2013 14:24:50 -0700 Subject: [erlang-questions] handling crash on db connect In-Reply-To: References: <2FEF06FD-78CC-4F13-AB52-27434F0564D9@erlang-solutions.com> <20130606105459.GA7415@circlewave.net> Message-ID: On Thu, Jun 6, 2013 at 12:28 PM, Garrett Smith wrote: > I'll add the start functions to redis_client and use those instead > with redis:connect (which is just a facade anyway -- arguably you > should use redis_client:start/start_link directly). > This will be much less surprising :) > Oh great, thanks! Maybe it's best to support both interfaces, to not break existing clients. Nobody on irc knew anything about this though. I may separately try writing a generic sleep/retry supervisor as a way of understanding OTP better. I've been wanting to read through the OTP sources but have been putting it off. This is a good reason :). Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Fri Jun 7 00:16:14 2013 From: g@REDACTED (Garrett Smith) Date: Thu, 6 Jun 2013 17:16:14 -0500 Subject: [erlang-questions] handling crash on db connect In-Reply-To: References: <2FEF06FD-78CC-4F13-AB52-27434F0564D9@erlang-solutions.com> <20130606105459.GA7415@circlewave.net> Message-ID: Paul, I made some tweaks, which I think are the Right Thing, thanks to your feedback! - redis:connect is a pass-through to new redis_client:start functions - Added redis:connect_link, which conforms to the old behavior Here's an example process, which covers the points I made earlier wrt a connection-manager/front-end to an underlying client process: https://github.com/gar1t/erlang-redis/blob/master/examples/dbservice/src/dbservice.erl How to use the example is here: https://github.com/gar1t/erlang-redis/blob/master/examples/dbservice/README.md On Thu, Jun 6, 2013 at 2:01 PM, Paul Rubin wrote: > On Thu, Jun 6, 2013 at 9:49 AM, Garrett Smith wrote: >> >> >> redis:connect is a wrapper for the process start_link.... >> The result {ok, Pid} simply means that the process has started and >> initialized. > > > Hey, thanks again, and aha, ok, this makes sense in terms of how start_link > behaves. It still comes as a surprise that redis:connect creates such a > link though. My program (typical for Erlang) accepts 1000's of client > connections with a separate process for each one. It uses redis for user > authentication, so a new connection arrives, there is a redis lookup, and > then the connection stays open "forever" (weeks, months...) without > accessing redis again. If redis is down, I'd expect new authentication > attempts to fail, but old connections to keep running as long as they don't > try to talk to redis. Redis is never supposed to be down so I could imagine > having this program run for a long time without ever encountering the issue. > I don't have an automatic test for handling redis being down (I guess I > should write one). I noticed this with a manual test, but I thought part of > the idea of OTP was to handle problems gracefully, including software bugs > resulting from error conditions that weren't thought of during testing. But if it *can* be down, it *will* be down, at some point :) >> I didn't implement any of the start functions for the client process. >> It's arguably bad form to not implement parallel start functions to >> each start_link -- but the cases where start_link is *wrong* are quite >> rare (none occur to me actually :) > > > start_link seems wrong to me in this case, but I still find OTP somewhat > murky so I can't claim to have that good intuition about what it should or > shouldn't do. Good use case for a start function, which we have now :) >> Yes, though it's not the end of the world to handle crashed processes >> explicitly -- supervisors have extremely naive recovery policies and >> it's not uncommon to have to implement something fancier. > > I wonder if what's really going on here is a gap in the available > supervision strategies. I was a bit surprised to learn from the supervisor > docs, > > "To prevent a supervisor from getting into an infinite loop of child process > terminations and restarts, a maximum restart frequency is defined using two > integer values MaxR and MaxT. If more than MaxR restarts occur within MaxT > seconds, the supervisor terminates all child processes and then itself." > > I had somehow thought that if MaxR restarts happened in MaxT seconds, the > supervisor would just sleep until MaxT seconds had passed, then start > retrying again (i.e. limit the frequency rather than the absolute count). > It does seem to me there should be an option for something like that. > Restarts due to external process failures seem like a common situation to > want to deal with, so (in my naive expectation) I'd think OTP should > implement an infinite sleep/retry loop as one of its restart strategies. There's a general lack-of-appetite to make supervisors any more complicated than they are -- but this topic comes up once in a while. Take a look at the dbservice application and the way a standard supervisor is used with a retry delay in the supervised process -- I use this pattern all the time and its prove pretty effective, for these applications. >> In the case of server connections, e.g. you almost always want to wait for >> some >> period before retrying, lest you DoS your servers :) > > > Yes. This is the common situation referred to above ;) >> >> >> I'll do this -- thanks for pointing this out! This is a pretty central >> problem that I glossed over. I'll update the github issue when it's >> pushed > > Great! Thanks! Thank you -- great input/feedback! Garrett From paul@REDACTED Fri Jun 7 00:39:41 2013 From: paul@REDACTED (Paul Rubin) Date: Thu, 6 Jun 2013 15:39:41 -0700 Subject: [erlang-questions] handling crash on db connect In-Reply-To: References: <2FEF06FD-78CC-4F13-AB52-27434F0564D9@erlang-solutions.com> <20130606105459.GA7415@circlewave.net> Message-ID: Thanks! I will look into switching my program to use dbservice. I should probably put any further comments/issues on github to avoid clogging the mailing list further. Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From samuelrivas@REDACTED Fri Jun 7 08:08:05 2013 From: samuelrivas@REDACTED (Samuel) Date: Fri, 7 Jun 2013 08:08:05 +0200 Subject: [erlang-questions] a problem about "file:path_eval([".", "/root"], ".erlang"): error on line 1: 1: syntax error before:"!!!!! In-Reply-To: <201306031446317717252@gmail.com> References: <201306031446317717252@gmail.com> Message-ID: You seem to have a broken .erlang file somewhere, possibly /root/.erlang. The runtime tries to evaluate that file when you start the vm and in you case it complains about bad syntax. Regards -- Samuela From davidnwelton@REDACTED Fri Jun 7 10:01:45 2013 From: davidnwelton@REDACTED (David Welton) Date: Fri, 7 Jun 2013 10:01:45 +0200 Subject: [erlang-questions] gen_retry? Re: handling crash on db connect Message-ID: > The question of how to handle connection problems can be tricky. I > typically bake this into a "connection handler" type process that > indeed traps exit and then figures out what to do -- other simply > let's the client process exit propagate up to the supervisor. I'll > typically have a retry logic that waits for a period of time after > failures, logging attempts, errors, etc. I've chatted about this a bit with the folks on #erlang, and while it seems there are some reasons that it's tricky, I wonder if it would be possible to make a behavior that deals with situations like this, where the code: * deals with some resource that might go away. * should not actually crash. * should know that it is in a degraded state. * should have a strategy to attempt to reconnect, like, say, exponential backoff with a max wait time. This is particularly relevant to database drivers, which is where I encountered the problem. For instance, with a web site, you don't want the whole thing to just fall over because the database is gone, you want it to return a nice message to the user apologizing for the problem, and keep trying to connect. -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From vincent.dephily@REDACTED Fri Jun 7 10:50:27 2013 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Fri, 07 Jun 2013 10:50:27 +0200 Subject: [erlang-questions] dialyzer and #record vs tuple Message-ID: <3822354.5RfTPR0ESt@moltowork> Hi List, I'm doing a long-overdue set of dialyzer checks on our software, and have a few occurences of the following problem: > % Define a record with a constrained field > -record(foo, {bar = 0 :: integer()}). > > % Use the record, works as expected > new_foo(Id) -> #foo{bar=Id}. > > % Use a similar-looking tuple, dialyzer complains about > % Record construction #foo{bar::true} violates the declared type of > % field bar::integer() > default_options() -> [{foo, true}]. I understand why dialyzer gets confused, since it can't (?) differenciate between a "raw" tuple and a record. But at the semantic level, they really are different things, so changing the record definition to allow booleans doesn't really make sense. How would you handle that kind of problem ? So far I have resorted to changing either the name of the record or the tag/arity of the tuple, but it generally feels wrong and inelegant, and later commits might inadvertantly reintroduce the natural-but-clashing tuple. -- Vincent de Phily From kostis@REDACTED Fri Jun 7 11:08:18 2013 From: kostis@REDACTED (Kostis Sagonas) Date: Fri, 07 Jun 2013 12:08:18 +0300 Subject: [erlang-questions] dialyzer and #record vs tuple In-Reply-To: <3822354.5RfTPR0ESt@moltowork> References: <3822354.5RfTPR0ESt@moltowork> Message-ID: <51B1A302.40406@cs.ntua.gr> On 06/07/13 11:50, Vincent de Phily wrote: > Hi List, > > I'm doing a long-overdue set of dialyzer checks on our software, and have a > few occurences of the following problem: > >> % Define a record with a constrained field >> -record(foo, {bar = 0 :: integer()}). >> >> % Use the record, works as expected >> new_foo(Id) -> #foo{bar=Id}. >> >> % Use a similar-looking tuple, dialyzer complains about >> % Record construction #foo{bar::true} violates the declared type of >> % field bar::integer() >> default_options() -> [{foo, true}]. > > I understand why dialyzer gets confused, since it can't (?) differenciate > between a "raw" tuple and a record. But at the semantic level, they really are > different things, so changing the record definition to allow booleans doesn't > really make sense. > > How would you handle that kind of problem ? So far I have resorted to changing > either the name of the record or the tag/arity of the tuple, but it generally > feels wrong and inelegant, and later commits might inadvertantly reintroduce > the natural-but-clashing tuple. There is no perfect `solution' to this . Dialyzer works at a level where records have been expanded into tuples and thus records and tuples are indistinguishable between them. Furthermore, it makes the assumption that within a module any tuple with a record tag and with a size that matches some typed record declaration refers to the declared types. So you have two options: 1. Use different tags for records and tuples e.g. foo_rec vs. foo (as you write) 2. Do not declare types for records that for some reason or another you also want to have similar looking tuples (as you write, declaring the "union" of the types does not make much sense). From the two options, I personally very much prefer the first. Kostis From vincent.dephily@REDACTED Fri Jun 7 11:23:01 2013 From: vincent.dephily@REDACTED (Vincent de Phily) Date: Fri, 07 Jun 2013 02:23:01 -0700 (PDT) Subject: [erlang-questions] dialyzer and #record vs tuple In-Reply-To: <51B1A302.40406@cs.ntua.gr> References: <3822354.5RfTPR0ESt@moltowork> <51B1A302.40406@cs.ntua.gr> Message-ID: <2417658.JFCBhiOC2b@moltowork> On Friday 07 June 2013 12:08:18 Kostis Sagonas wrote: > There is no perfect `solution' to this . > > Dialyzer works at a level where records have been expanded into tuples > and thus records and tuples are indistinguishable between them. > Furthermore, it makes the assumption that within a module any tuple with > a record tag and with a size that matches some typed record declaration > refers to the declared types. So you have two options: > > 1. Use different tags for records and tuples e.g. foo_rec vs. foo (as > you write) > > 2. Do not declare types for records that for some reason or another > you also want to have similar looking tuples (as you write, declaring > the "union" of the types does not make much sense). > > From the two options, I personally very much prefer the first. Thanks for confirming this, pity there's no nice way of handling that problem but I'll stick with option 1 for now. Maybe when frames arrive in erlang it'll fix that issue :) -- Vincent de Phily From sverker.eriksson@REDACTED Fri Jun 7 14:12:12 2013 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Fri, 7 Jun 2013 14:12:12 +0200 Subject: [erlang-questions] NIF and timing In-Reply-To: <186519ab-06cf-41e6-8258-e8c8e61ae537@googlegroups.com> References: <186519ab-06cf-41e6-8258-e8c8e61ae537@googlegroups.com> Message-ID: <51B1CE1C.7090306@erix.ericsson.se> Fredrik Linder wrote: > Hello > > First, thanks for your help on previous questions :-) > > I have a NIF implementation that have a peculiar timing behavior (the NIF > code is pretty simple, doing enif_get_tuple, enif_get_atom, > enif_get_resource and some O(1) operations). > > The median time is 39 us (microseconds) > 15666/16079 calls take < 100 us > 16049/16079 calls take < 1000 us > 16052/16079 calls take < 10000 us > 16067/16079 calls take < 100000 us > > I'm running a list comprehension that gives me the time taken for each call: > > [begin A = os:timestamp(), doit(), io:format("~w ", > [now:time_diff(os:timestamp(), A)]) end || _ <- lists:seq(1, 16#10000)]. > > I'm running R16B on a 4 core VM. > > Of course some lag due to process switching etc would be expected, but 12 > of the 16k calls take more than 0.1 seconds to run. That seems way to much > to me. (I get the same behavior when not doing the io:format/2 call). > > Any clue why? > > If your NIF builds terms that triggers a garbage collection (fills up the heap), that GC will be done before the NIF returns returns to its caller. /Sverker Erlang/OTP Ericsson From mononcqc@REDACTED Fri Jun 7 14:54:34 2013 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 7 Jun 2013 08:54:34 -0400 Subject: [erlang-questions] handling crash on db connect In-Reply-To: References: <2FEF06FD-78CC-4F13-AB52-27434F0564D9@erlang-solutions.com> <20130606105459.GA7415@circlewave.net> Message-ID: <20130607125433.GA54252@ferdmbp.local> Ah this is an interesting bit that I wanted to respond to. I'll do so inline. On 06/06, Paul Rubin wrote: > > OK, that is another case of my expectations being wrong, but it calls into > question why I went to the trouble of learning Erlang in the first place > ;) Just about all other languages stop the program when something fails, > but I thought the idea of Erlang was to keep going no matter what! Of > course I want it to get my attention, but I thought that it did that by > writing "CRASH REPORT" into the sasl log in ALL CAPS so I could grep for it > the next morning, instead of blowing up my entire service when a > non-essential part of the program hits a glitch. The whole idea of > supervision, hot code patches, appups, relups, etc. is to be able to write > programs that are just NEVER DOWN, even if the code has bugs. Anything > that makes the VM crash for reasons short of hardware meltdown seems wrong > from that point of view. > Erlang's idea is not to keep going no matter what, it's to keep going even though there are individual failures. This doesn't mean that any and all amount of failing is equal. Would you expect your node to blindly try to start on its own to infinity without ever booting if the problem was that the code cannot be found/loaded? What if you didn't even call a supervisor at the top of your start function for the application behaviour? It *is* a useful feature to be able to tell a supervisor to give up after a while. You can do so by specifying the MaxR and MaxT according to what you want. If you never want it to die, you could always use a MaxR of 999999999 over a MaxT of 1 second and hope the code can't fail fast enough and get the behaviour you want. You may kill the node due to too much logging and going out of memory on your error logger, though. Moreover, once you have divided your system into OTP applications, you can choose to know which one is vital or not. Each OTP application can be started in 3 ways: temporary, transient, permanent, either by doing it manually in application:start(Name, Type), or in the config for releases in systools or reltool. - permanent: if the app terminates, the entire system is taken down, excluding manual termination of the app with application:stop/1. - transient: if the app terminates for reason 'normal', that's ok. Any other reason for termination shuts down the entire system. - temporary: the application is allowed to stop for any reason. It will be reported, but nothing bad will happen. Note that OTP will not restart all applications indefinitely. It will just stop running them (restarting apps is their top supervisor's responsibility). If your node can really live without an app, you may want to make it 'temporary'. Nothing in there says "OTP shuts down the node all the time" nearly as much as the configuration you give it. > If you start to look at your Erlang applications as ecosystems of > independent services, you can start to think about shoring up each > service to improve its availability, performance, etc. -- just as one > might in a service oriented architecture. In the case of your Redis > service, you want something that advocates for your Redis DB > availability. That advocate (an Erlang process) can deal with > connections, retries, error handling, etc. as an intelligent facade to > the underlying Redis client process (which can some and go at any time > depending on network, Redis server availability, etc.) > > Of course I'd expect to have to shore up the individual services as > problems with them arose and patterns got identified, but in the meanwhile > I'd expect a service being down to just mean I couldn't use it. Like if > Github is down for some reason, I'd expect to get a connection error if I > try to access it from the office, but I wouldn't expect Github's outage to > set off a demolition for the whole building my office is in. That's sort > of what start_link does, from what I gather. The failure path should > instead be isolated by default, propagating only to the parts of the code > that actually depend on the things that failed. So it seems to me, using > start_link in a client isn't right, unless the fates of the two interacting > processes are actually entwined in a way that neither one could conceivably > run without the other. Other independent processes should be able to keep > running. start_link is synchronous for one simple reason: so you can know that whatever was started in the supervision tree is generally available when it's spawned. This is useful within a single app for the case where, for example, you have internal dependencies: __________[app_supersup] / | [app_ets_sup] [app_sup] | | | | | [ets tables ] [app_workers] In this case, synchronous supervisors mean that app_ets_sup is started and has started its ETS tables when app_sup comes up, and I can safely assume my resources are there without any tricky code that polls for their existence in the app_workers' init functions. Similarly, these guarantees generally extend to other applications (because they also use supervisors), so I know that when I started 'gproc' or 'poolboy', I should be able to rely on them internally, without repeatedly looking for some registered process and hoping it comes up soon. It reduces the complexity of my own code. Now when we look at external dependencies (services, databases, etc.) we have many ways to design things, and here's part of the problem you had: Requiring a connection to be established for 'init' to work (and hence for start_link to return successfully) implicitly means that other follower applications should be able to expect the connection to be there when they boot. This may make sense, if for example, you're going on a loopback interface that should not be broken (UDP, TCP, who cares) because it's local. These do not need an 'advocate'. If you expect that the connection can logically be impossible to establish and that you should expect failures, then by all means do not return it as part of init as a guarantee. Just guarantee that a manager for the connection will be available to tell you whether it's online or not. Yes, this means you'll need an asynchronous start, but you'll most likely need a 'reconnect' piece of code in your gen_server any way because you expect the connection to fail at some point. So instead of doing: init(Args) -> Opts = parse_args(Args), {ok, Port}?= connect(Opts), {ok, #state{sock=Port, opts=Opts}}. [...] handle_info(reconnect, S = #state{sock=undefined, opts=Opts}) -> case connect(Opts) of {ok, New} -> {noreply, S#state{sock=New}}; _ -> self() ! reconnect, {noreply, S} end; You do: init(Args) -> Opts = parse_args(Args), %% you could try connecting here anyway, for a best effort thing %% but be ready to not have a connection. self() ! reconnect, {ok, #state{sock=undefined, opts=Opts}}. [...] handle_info(reconnect, S = #state{sock=undefined, opts=Opts}) -> case connect(Opts) of {ok, New} -> {noreply, S#state{sock=New}}; _ -> self() ! reconnect, {noreply, S} end; And otherwise the code doesn't really change, because you expected failures anyway. You just changed the kind of guarantees you gave to applications and processes started after yours from "the connection is available" to "the connection manager is available". Then document that somewhere, either in a README.md or EDoc or anything else. This 'failure path' isolation is yours to do, because you, as the application developer, have the knowledge to know what is a good error or not to break on. You may, for example, want to crash if you cannot have permissions to edit files on disk while you expect to. Now putting more control back into your hands does mean you have more responsibilities and more ways to make mistakes, but it is a very small change that basically lets you start your application as permanent instead of temporary. Most of the hard decision making where you and I will generally make mistakes will be when code calls your connection manager: Am I able to get a connection? Am I able to send something over the connection or did it fail? Did I get the value I expected out of it? Did the connection fail after I received part of my answer? Did it just time out? Not dealing with these right will have the potential to kill your own app, even though the connection manager is there and runs well, no matter if it were able to have the connection in its init or not. That does not change depending on how supervisors work. This answer got to be much longer than I originally anticipated. I hope this helps, though! Regards, Fred. From raimo+erlang-questions@REDACTED Fri Jun 7 15:10:31 2013 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 7 Jun 2013 15:10:31 +0200 Subject: [erlang-questions] Send Port around distributed Erlang nodes In-Reply-To: References: Message-ID: <20130607131031.GB5653@erix.ericsson.se> On Thu, Jun 06, 2013 at 08:34:18PM +0800, "Siyao Zheng(???)" wrote: > Hi all, > > It is possible to send a port id to other node, but the receiving node can't operate on this port by the received port id. According to the spec of erlang:send/2, in order to send message to this port, the port must be registered with a name, but messages can be send to remote directly. Is it an intended design ? Do you think it is necessary to allow the following scene be possible: node A listens on a socket S, sends S to node B, and B accepts this S? Is there any considerations about this ability? That must be a documentation oversight. You can not send to a remote port. When sending to a remotely registered name it is only process names that are looked up on the remote node. > > Cheers > Siyao Zheng > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From mail@REDACTED Fri Jun 7 15:12:59 2013 From: mail@REDACTED (Alexander Alexeev) Date: Fri, 7 Jun 2013 17:12:59 +0400 Subject: [erlang-questions] How to upgrade the Mnesia record structure? In-Reply-To: <51ac3149.e982310a.5ec1.5708@mx.google.com> References: <51ac3149.e982310a.5ec1.5708@mx.google.com> Message-ID: <20130607171259.08641f5a@portege> On Mon, 03 Jun 2013 06:01:27 +0000 yashgt@REDACTED wrote: > If a record is defined as: > > -record(shop, {item, quantity, cost}). > > and several records are stored in an Mnesia table, and a few months > later the structure is changed to : -record(shop, {item, quantity, > source, company, cost}). > > Do we need to migrate ALL existing records in the DB to include the > additional 2 elements in the tuple? I know that a record is actually > a tuple. If we leave the existing records untouched, will the > database get messed becaues of new records coming in with the new > structure? > > Thanks, > Yash You can also provide a special "extendable" field (call it a backpack) of type proplist. In your case initial record would have been: -record(show, {item, quantity, cost, backpack}). Then when you would need to store source and company you could write: #show{ item = Item, quantity = Quantity, cost = Cost, backpack = [{source, Source},{company, Company}] } From yendris.cruz@REDACTED Fri Jun 7 16:22:49 2013 From: yendris.cruz@REDACTED (Yendris Cruz Mendoza) Date: Fri, 07 Jun 2013 10:22:49 -0400 Subject: [erlang-questions] about GIS Message-ID: Hi , is possible that exist any open source software about GIS implemented in Erlang?..for example, Web Map Services, Tile Map Services or Web processing services especifications. Sorry my English...regards From jleivent@REDACTED Fri Jun 7 17:57:16 2013 From: jleivent@REDACTED (Jonathan Leivent) Date: Fri, 07 Jun 2013 11:57:16 -0400 Subject: [erlang-questions] dialyzer and #record vs tuple In-Reply-To: <51B1A302.40406@cs.ntua.gr> References: <3822354.5RfTPR0ESt@moltowork> <51B1A302.40406@cs.ntua.gr> Message-ID: <51B202DC.8000804@comcast.net> I have a similar problem with Dialyzer (which I run all the time using Emacs flymake mode), but on match specs of records with typed fields. In this case, Dialyzer sees atoms like '$1' and '_' as field values, which don't match the record field types. It will then invariably complain with "Warning: Function ... has no local return" for functions that contain the resulting match specs. This happens whether the match specs are composed by hand or by ets:fun2ms/ms_transform.hrl magic. To prevent this, I add the following: -ifdef(dialyzer). dialyzer_fake_fun2ms(F) when is_function(F, 1) -> [{ok, [ok], [ok]}]. -define(msify(X), dialyzer_fake_fun2ms(X)). -else. -define(msify(X), ets:fun2ms(X)). -endif. Then put the functions being transformed within ?msify macro calls, like so: -record(foo, {a :: integer(), b :: integer(), c :: integer()}). get_foos() -> ets:select(foos, ?msify(fun (#foo{a=0, b=X, c=Y}) when X > 5 -> Y end)). Then run Dialyzer (but not the compiler) with -Ddialyzer. -- Jonathan Leivent From davidnwelton@REDACTED Fri Jun 7 18:20:27 2013 From: davidnwelton@REDACTED (David Welton) Date: Fri, 7 Jun 2013 18:20:27 +0200 Subject: [erlang-questions] erlexec debugging In-Reply-To: References: Message-ID: After a lot of hacking in several git branches, I have something that seems to work https://github.com/saleyn/erlexec/pull/12 If anyone wants to have a look-see, it'd be appreciated. -- David N. Welton http://www.welton.it/davidw/ http://www.dedasys.com/ From dszoboszlay@REDACTED Fri Jun 7 20:46:41 2013 From: dszoboszlay@REDACTED (=?utf-8?Q?Szoboszlay_D=C3=A1niel?=) Date: Fri, 07 Jun 2013 19:46:41 +0100 Subject: [erlang-questions] dialyzer and #record vs tuple In-Reply-To: <2417658.JFCBhiOC2b@moltowork> References: <3822354.5RfTPR0ESt@moltowork> <51B1A302.40406@cs.ntua.gr> <2417658.JFCBhiOC2b@moltowork> Message-ID: Hi, You may also try to remove the type declaration from the record but declare a new type that specializes your now generic record type, like this: -record(foo, {bar = 0}). -type foo() :: #foo{bar :: integer()}. -spec new_foo(Id :: integer()) -> foo(). You will have to write some -spec lines, but personally I'd do that anyway... Daniel On Fri, 07 Jun 2013 10:23:01 +0100, Vincent de Phily wrote: > On Friday 07 June 2013 12:08:18 Kostis Sagonas wrote: >> There is no perfect `solution' to this . >> >> Dialyzer works at a level where records have been expanded into tuples >> and thus records and tuples are indistinguishable between them. >> Furthermore, it makes the assumption that within a module any tuple with >> a record tag and with a size that matches some typed record declaration >> refers to the declared types. So you have two options: >> >> 1. Use different tags for records and tuples e.g. foo_rec vs. foo (as >> you write) >> >> 2. Do not declare types for records that for some reason or another >> you also want to have similar looking tuples (as you write, declaring >> the "union" of the types does not make much sense). >> >> From the two options, I personally very much prefer the first. > > Thanks for confirming this, pity there's no nice way of handling that > problem > but I'll stick with option 1 for now. > > Maybe when frames arrive in erlang it'll fix that issue :) From watson.timothy@REDACTED Fri Jun 7 21:32:44 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 7 Jun 2013 20:32:44 +0100 Subject: [erlang-questions] handling crash on db connect In-Reply-To: References: <2FEF06FD-78CC-4F13-AB52-27434F0564D9@erlang-solutions.com> <20130606105459.GA7415@circlewave.net> Message-ID: On 6 Jun 2013, at 20:01, Paul Rubin wrote: > > I wonder if what's really going on here is a gap in the available supervision strategies. I was a bit surprised to learn from the supervisor docs, > > "To prevent a supervisor from getting into an infinite loop of child process terminations and restarts, a maximum restart frequency is defined using two integer values MaxR and MaxT. If more than MaxR restarts occur within MaxT seconds, the supervisor terminates all child processes and then itself." > > I had somehow thought that if MaxR restarts happened in MaxT seconds, the supervisor would just sleep until MaxT seconds had passed, then start retrying again (i.e. limit the frequency rather than the absolute count). It does seem to me there should be an option for something like that. Whilst this is orthogonal to the points covered already, it might be worth mentioning that RabbitMQ uses its own copy of supervisor, imaginatively named supervisor2, which offers this feature under the name "delayed restarts". A child restart configured with {Type, Delay} will use Type to handle restarts unless it hits the maximum restart intensity, at which point it will try again after Delay. With Type set to permanent for example, that basically means the supervisor will keep trying to restart forever. We use this in some plugins to provide a reconnect/retry delay option. At some point we could submit a patch to OTP, if there's any appetite for incorporating it. Cheers, Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: From tcury@REDACTED Fri Jun 7 22:15:39 2013 From: tcury@REDACTED (Tiago Cury) Date: Fri, 7 Jun 2013 17:15:39 -0300 Subject: [erlang-questions] erl_tar:add continue on error Message-ID: <50D49D74-A1FA-4CF6-B210-1A7D157011A3@ymail.com> Hi, I'm using erl_tar:add to create a backup from my server, but when it finds a kind of file type like socket it returns immediately with a tuple {error, ?reason?} and aborts the entire process. Is there a way to erl_tar:add ignore these errors (skip just the bad file) and continue the operation? From lcastro@REDACTED Sat Jun 8 01:29:37 2013 From: lcastro@REDACTED (Laura M. Castro) Date: Sat, 8 Jun 2013 01:29:37 +0200 Subject: [erlang-questions] ACM SIGPLAN Erlang Workshop 2013 Final Call For Papers Message-ID: Hello, This is the Final Call for Papers for the Twelfth ACM SIGPLAN Erlang Workshop. Apologies for any duplicates you may receive. CALL FOR PAPERS ================= Twelfth ACM SIGPLAN Erlang Workshop ----------------------------------------------------------- Boston, Massachusetts, September 28, 2013 Satellite event of the 18th ACM SIGPLAN International Conference on Functional Programming (ICFP 2013) September 25-27, 2013 Erlang is a concurrent, distributed functional programming language aimed at systems with requirements of massive concurrency, soft real time response, fault tolerance, and high availability. It has been available as open source for 15 years, creating a community that actively contributes to its already existing rich set of libraries and applications. Originally created for telecom applications, its usage has spread to other domains including e-commerce, banking, databases, and computer telephony and messaging. Erlang programs are today among the largest applications written in any functional programming language. These applications offer new opportunities to evaluate functional programming and functional programming methods on a very large scale and suggest new problems for the research community to solve. This workshop will bring together the open source, academic, and industrial programming communities of Erlang. It will enable participants to familiarize themselves with recent developments on new techniques and tools tailored to Erlang, novel applications, draw lessons from users' experiences and identify research problems and common areas relevant to the practice of Erlang and functional programming. We invite three types of submissions. 1. Technical papers describing language extensions, critical discussions of the status quo, formal semantics of language constructs, program analysis and transformation, virtual machine extensions and compilation techniques, implementations and interfaces of Erlang in/with other languages, and new tools (profilers, tracers, debuggers, testing frameworks, etc.). The maximum length for technical papers is restricted to 12 pages. 2. Practice and application papers describing uses of Erlang in the "real-world", Erlang libraries for specific tasks, experiences from using Erlang in specific application domains, reusable programming idioms and elegant new ways of using Erlang to approach or solve a particular problem. The maximum length for the practice and application papers is restricted to 12 pages. Note that this is a maximum length; we welcome shorter papers also, and the program committee will evaluate all papers on an equal basis independent of their lengths. 3. Poster presentations describing topics related to the workshop goals. Each includes a maximum of 2 pages of the abstract and summary. Presentations in this category will be given an hour of shared simultaneous demonstration time. Workshop Chair ----------------------- Steve Vinoski, Basho Technologies, USA Program Chair ------------------- Laura M. Castro, University of A Coru?a, Spain Program Committee ----------------------------- (Note: the Workshop and Program Chairs are also committee members) Lars-Ake Fredlund, Universidad Polit?cnica de Madrid, Spain Kevin Hammond, University of St. Andrews, UK Torben Hoffman, Erlang Solutions Limited, UK Zolt?n Horv?th, E?tv?s Lor?nd University, Hungary Kenneth Lundin, Ericsson AB, Sweden Micka?l R?mond, ProcessOne, France Kenji Rikitake, Basho Japan KK, Japan Simon Thompson, University of Kent, UK Important Dates ----------------------- Submission deadline: Fri June 14, 2013 Author notification: Thu July 11, 2013 Final submission for the publisher: Thu July 25, 2013 Workshop date (tentative, subject to change): September 28, 2013 Instructions to authors -------------------------------- Papers must be submitted online via EasyChair (via the "Erlang2013" event). The submission page is https://www.easychair.org/conferences/?conf=erlang2013 Submitted papers should be in portable document format (PDF), formatted using the ACM SIGPLAN style guidelines. Each submission must adhere to SIGPLAN's republication policy. Violation risks summary rejection of the offending submission. Accepted papers will be published by the ACM and will appear in the ACM Digital Library. Paper submissions will be considered for poster submission in the case they are not accepted as full papers. Venue & Registration Details ------------------------------------------ For registration, please see the ICFP 2013 web site at: http://icfpconference.org/icfp2013/ Related Links -------------------- ICFP 2013 web site: http://www.icfpconference.org/icfp2013/ Past ACM SIGPLAN Erlang workshops: http://www.erlang.org/workshop/ Open Source Erlang: http://www.erlang.org/ EasyChair submission site: https://www.easychair.org/account/signin.cgi?conf=erlang2013 Author Information for SIGPLAN Conferences: http://www.sigplan.org/authorInformation.htm -- Laura M. Castro From jinni.park@REDACTED Sat Jun 8 04:56:54 2013 From: jinni.park@REDACTED (Park, Sungjin) Date: Sat, 8 Jun 2013 11:56:54 +0900 Subject: [erlang-questions] gen_server:call/2 exceptions? Message-ID: When does gen_server:call/2 throw an exception as {'EXIT', normal}? -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From peerst@REDACTED Sat Jun 8 09:41:15 2013 From: peerst@REDACTED (Peer Stritzinger) Date: Sat, 8 Jun 2013 09:41:15 +0200 Subject: [erlang-questions] erl_tar:add continue on error References: <50D49D74-A1FA-4CF6-B210-1A7D157011A3@ymail.com> Message-ID: On 2013-06-07 20:15:39 +0000, Tiago Cury said: > Hi, I'm using erl_tar:add to create a backup from my server, but when > it finds a kind of file type like socket it returns > immediately with a tuple {error, > reason > } and aborts the entire process. > > Is there a way to erl_tar:add ignore these errors (skip just the bad > file) and continue the operation? You could use filelib:fold_files to recurse over the tree calling erl_tar:add on what it findes. According to its documentation it only recurses over regular files. Cheers -- Peer From robert.virding@REDACTED Sat Jun 8 17:41:25 2013 From: robert.virding@REDACTED (Robert Virding) Date: Sat, 8 Jun 2013 16:41:25 +0100 (BST) Subject: [erlang-questions] gen_server:call/2 exceptions? In-Reply-To: Message-ID: <1939197026.142865311.1370706085698.JavaMail.root@erlang-solutions.com> IIRC it can if the server to which it has sent the call terminates with the reason 'normal' before sending a reply. Robert ----- Original Message ----- > From: "Sungjin Park" > To: "erlang-questions" > Sent: Saturday, 8 June, 2013 4:56:54 AM > Subject: [erlang-questions] gen_server:call/2 exceptions? > When does gen_server:call/2 throw an exception as {'EXIT', normal}? > -- > Park, Sungjin > ------------------------------------------------------------------------------------------------------------------- > Peculiar travel suggestions are dancing lessons from god. > -- The Books of Bokonon > ------------------------------------------------------------------------------------------------------------------- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From jinni.park@REDACTED Sat Jun 8 18:20:06 2013 From: jinni.park@REDACTED (Park, Sungjin) Date: Sun, 9 Jun 2013 01:20:06 +0900 Subject: [erlang-questions] gen_server:call/2 exceptions? In-Reply-To: <1939197026.142865311.1370706085698.JavaMail.root@erlang-solutions.com> References: <1939197026.142865311.1370706085698.JavaMail.root@erlang-solutions.com> Message-ID: Then there is no way to catch it, right? On Sun, Jun 9, 2013 at 12:41 AM, Robert Virding < robert.virding@REDACTED> wrote: > IIRC it can if the server to which it has sent the call terminates with > the reason 'normal' before sending a reply. > > Robert > > ------------------------------ > > *From: *"Sungjin Park" > *To: *"erlang-questions" > *Sent: *Saturday, 8 June, 2013 4:56:54 AM > *Subject: *[erlang-questions] gen_server:call/2 exceptions? > > > When does gen_server:call/2 throw an exception as {'EXIT', normal}? > > -- > Park, Sungjin > > ------------------------------------------------------------------------------------------------------------------- > Peculiar travel suggestions are dancing lessons from god. > -- The Books of Bokonon > > ------------------------------------------------------------------------------------------------------------------- > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Sat Jun 8 18:24:29 2013 From: robert.virding@REDACTED (Robert Virding) Date: Sat, 8 Jun 2013 17:24:29 +0100 (BST) Subject: [erlang-questions] gen_server:call/2 exceptions? In-Reply-To: Message-ID: <954400754.142897608.1370708669768.JavaMail.root@erlang-solutions.com> Again IIRC it the gen_server:call itself which generates an exception when it detects that the server has dies. If so just doing catch gen_server:call (...) will stop the calling process from crashing. You will then need a case to check if you got a reply or the server died. Robert ----- Original Message ----- > From: "Sungjin Park" > To: "erlang-questions" > Sent: Saturday, 8 June, 2013 6:20:06 PM > Subject: Re: [erlang-questions] gen_server:call/2 exceptions? > Then there is no way to catch it, right? > On Sun, Jun 9, 2013 at 12:41 AM, Robert Virding < > robert.virding@REDACTED > wrote: > > IIRC it can if the server to which it has sent the call terminates > > with the reason 'normal' before sending a reply. > > > Robert > > > > From: "Sungjin Park" < jinni.park@REDACTED > > > > > > > To: "erlang-questions" < erlang-questions@REDACTED > > > > > > > Sent: Saturday, 8 June, 2013 4:56:54 AM > > > > > > Subject: [erlang-questions] gen_server:call/2 exceptions? > > > > > > When does gen_server:call/2 throw an exception as {'EXIT', > > > normal}? > > > > > > -- > > > > > > Park, Sungjin > > > > > > ------------------------------------------------------------------------------------------------------------------- > > > > > > Peculiar travel suggestions are dancing lessons from god. > > > > > > -- The Books of Bokonon > > > > > > ------------------------------------------------------------------------------------------------------------------- > > > > > > _______________________________________________ > > > > > > erlang-questions mailing list > > > > > > erlang-questions@REDACTED > > > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Park, Sungjin > ------------------------------------------------------------------------------------------------------------------- > Peculiar travel suggestions are dancing lessons from god. > -- The Books of Bokonon > ------------------------------------------------------------------------------------------------------------------- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From tmr@REDACTED Sat Jun 8 21:22:57 2013 From: tmr@REDACTED (Tomas Morstein) Date: Sat, 8 Jun 2013 12:22:57 -0700 (PDT) Subject: [erlang-questions] erlang/C embedded db suggestion In-Reply-To: References: Message-ID: I am not sure if it is what you're looking for, but if you need an embedded database for Erlang, I would suggest you this one: https://github.com/ztmr/egtm EGTM is Erlang/OTP binding for FIS GT.M (http://en.wikipedia.org/wiki/GT.M) implementation of the M-technology (http://en.wikipedia.org/wiki/MUMPS). The version published on GitHub is purely NIF-based. You can configure it so that it will start a few slave Erlang worker-nodes but due to Erlang RPC performance penalty, the single-node NIF is much more faster if you use it properly. Also, GT.M itself has built-in multi-site (replication) support, with no dependency on Erlang stuff. Another nice thing is that you can crunch your data directly in the M language if needed -- it can minimize the amount of NIF calls in some kinds of applications. We are about to release a new EGTM version (we use it in production, but simply haven't merged it to the public repo) which also supports an I/O based data exchange between Erlang and M functions. There is also IODB product which is an OOP extension of EGTM which allows you to map complex Erlang structures to GT.M globals (trees). But I guess that's not what you're looking for. Feel free to contact me if interested in more details! Tom On Saturday, June 1, 2013 1:29:28 AM UTC+2, Radu Brumariu wrote: > > Hi, > I have a project where an application reads bits from a database using a C > library and makes some decisions based on the data available there. At the > same time, the same database is periodically updated by an Erlang app. > > It is highly likely that both the C lib and the Erlang app would have the > library open at the same time, however the C lib would be read only, > whereas Erlang process would open it for write as well. > > Any suggestions on what the database might be ? Something like term > storage ( bdb ) would work for my needs, but bdb cannot be opened in > parallel. Same goes for LevelDB, KyotoCabinet. The only that I have found ( > so far ) is sqlite(but I don't need sql). I am thinking that an alternative > would be to have a raw file mmap-ped by the C library, but I would much > rather use something that has some primitives already built. > > Also, I am considering doing IPC via a unix/tcp socket with the Erlang > process, but I'd rather use a db/file ( provides natural fallback if the > process does not run, or other comm issues). > > For the time being, the Erlang app and the C lib would be on the same box. > > Thanks, > Radu > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tcury@REDACTED Sat Jun 8 21:27:53 2013 From: tcury@REDACTED (Tiago Cury) Date: Sat, 8 Jun 2013 16:27:53 -0300 Subject: [erlang-questions] erl_tar:add continue on error In-Reply-To: References: <50D49D74-A1FA-4CF6-B210-1A7D157011A3@ymail.com> Message-ID: I wrote some records to a file and the surprise ?. Air11:Backuper tcury$ file /tmp/a.ea /tmp/a.ea: DBase 3 data file with memo(s) (1634009600 records) LOLLLLLLL On Jun 8, 2013, at 4:41 AM, Peer Stritzinger wrote: > On 2013-06-07 20:15:39 +0000, Tiago Cury said: > >> Hi, I'm using erl_tar:add to create a backup from my server, but when it finds a kind of file type like socket it returns >> immediately with a tuple {error, reason >> } and aborts the entire process. >> Is there a way to erl_tar:add ignore these errors (skip just the bad file) and continue the operation? > > You could use filelib:fold_files to recurse over the tree calling erl_tar:add on what it findes. > > According to its documentation it only recurses over regular files. > > Cheers > -- Peer > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mjtruog@REDACTED Sun Jun 9 00:46:56 2013 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 08 Jun 2013 15:46:56 -0700 Subject: [erlang-questions] [ANN] CloudI 1.2.3 Released! Message-ID: <51B3B460.4060008@gmail.com> Download 1.2.3 from http://sourceforge.net/projects/cloudi/files/latest/download What is CloudI? A Cloud at the lowest level! CloudI (http://cloudi.org/) is a "universal integrator" using an Erlang core, with its actor model, to pursue efficiency, fault-tolerance and scalability. The CloudI API provides a minimal interface to communicate among actors that are called services, so programming language agnostic, database agnostic, and messaging bus agnostic integration can occur. CloudI currently integrates with the programming languages Erlang, C/C++, Java, Python, and Ruby, the databases PostgreSQL, MySQL, couchdb, memcached, and tokyotyrant, the messaging bus ZeroMQ and the internal CloudI service bus. HTTP is also supported for easy integration with cowboy. If anyone is willing to get involved, don't hesitate to contact me or start looking at the code. Thank you for all the suggestions and questions so far! This release provides some minor fixes after the 1.2.2 release. The 1.2.2 loadtest results are summarized at https://github.com/okeuday/CloudI/raw/master/src/tests/http_req/loadtest/results_v1_2_2/201306_summary.pdf . The details are below: June 8, 2013 Version 1.2.3 (beta) released. * Added internal service purging when services are removed. The service module, or service application modules are purged after the service has been stopped, and the service applications (if any were used) has been stopped. The functionality works with modules, applications and releases (within a .script file). When applications are stopped and purged, only applications which are not dependencies for other running applications are removed. * Focused on bugs/fixes, mainly for OSX. The external service "local" protocol configuration option did not work properly on some installations of OSX, so the default CloudI configuration file now uses the "tcp" protocol configuration option (to be the same as releases before 1.2.2). The problem with the "local" protocol configuration option may be limited to slower machines (the problem is caused by execution spending too much time within a socket accept function call within an Erlang NIF). The "local" protocol configuration option on OSX will be addressed in a later release. Please mention any problems, issues, or ideas! Thanks, Michael From jinni.park@REDACTED Sun Jun 9 07:49:07 2013 From: jinni.park@REDACTED (Park, Sungjin) Date: Sun, 9 Jun 2013 14:49:07 +0900 Subject: [erlang-questions] gen_server:call/2 exceptions? In-Reply-To: <954400754.142897608.1370708669768.JavaMail.root@erlang-solutions.com> References: <954400754.142897608.1370708669768.JavaMail.root@erlang-solutions.com> Message-ID: Got it! Thank you. On Sun, Jun 9, 2013 at 1:24 AM, Robert Virding < robert.virding@REDACTED> wrote: > Again IIRC it the gen_server:call itself which generates an exception when > it detects that the server has dies. If so just doing > > catch gen_server:call(...) > > will stop the calling process from crashing. You will then need a case to > check if you got a reply or the server died. > > Robert > > ------------------------------ > > *From: *"Sungjin Park" > *To: *"erlang-questions" > *Sent: *Saturday, 8 June, 2013 6:20:06 PM > *Subject: *Re: [erlang-questions] gen_server:call/2 exceptions? > > > Then there is no way to catch it, right? > > > On Sun, Jun 9, 2013 at 12:41 AM, Robert Virding < > robert.virding@REDACTED> wrote: > >> IIRC it can if the server to which it has sent the call terminates with >> the reason 'normal' before sending a reply. >> >> Robert >> >> ------------------------------ >> >> *From: *"Sungjin Park" >> *To: *"erlang-questions" >> *Sent: *Saturday, 8 June, 2013 4:56:54 AM >> *Subject: *[erlang-questions] gen_server:call/2 exceptions? >> >> >> When does gen_server:call/2 throw an exception as {'EXIT', normal}? >> >> -- >> Park, Sungjin >> >> ------------------------------------------------------------------------------------------------------------------- >> Peculiar travel suggestions are dancing lessons from god. >> -- The Books of Bokonon >> >> ------------------------------------------------------------------------------------------------------------------- >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> > > > -- > Park, Sungjin > > ------------------------------------------------------------------------------------------------------------------- > Peculiar travel suggestions are dancing lessons from god. > -- The Books of Bokonon > > ------------------------------------------------------------------------------------------------------------------- > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From bryan@REDACTED Sun Jun 9 23:35:50 2013 From: bryan@REDACTED (Bryan Hughes) Date: Sun, 09 Jun 2013 14:35:50 -0700 Subject: [erlang-questions] [ANN] Erlang plugin for IntelliJ IDEA version 0.4 In-Reply-To: References: Message-ID: <51B4F536.7020708@go-factory.net> Hey Everyone, I have been using Sergey's plugin for a few months now and it is really quite nice. If you are a huge fan like I am of Jetbrains IDE's, then it is a must have. I develop in Objective-C (iPhone), Java (Android) and Erlang and it is nice to have everything under one roof in a robust IDE (sadly I lost my Emacs skills a while back). Understand that it is a work in progress, but is pretty impressive. I have personally contributed to his work and would urge any of you who find the plugin valuable to support him. If you go to... http://ignatov.github.io/intellij-erlang/ and scroll down a ways, there a link if you would like to make a donation. Cheers, Bryan On 1/21/13 12:21 AM, Max Bourinov wrote: > Hi Sergey, > > I just tried your plugin and it looks very promising. I will give it a > try in my next project. Thank you for your work! > > p.s.: I am myself used to use idea from 2004, but last year switched > to Sublime Text 2. > > Best regards, > Max > > > > On Mon, Jan 21, 2013 at 11:29 AM, Sergey Ignatov > wrote: > > Hi all, > > Today I'm happy to announce the new version of Erlang plugin for > IntelliJ IDEA. The new version is the biggest improvement of our > project. > > The highlight of the new version: > - Rebar run configurations (also includes Rebar-based Eunit > integration) > - Erlang types support > - External and local documentation for OTP functions and modules > - Function parameter info context help > - Upgraded structure view (shows exported and inner functions) > - Go to action for all Erlang definitions > - Improved BIFs completion > - And many others, please check out release notes: > https://github.com/ignatov/intellij-erlang#version-04 > > Plugin page in IntelliJ repository: > http://plugins.jetbrains.com/plugin/?pluginId=7083 > Source code: https://github.com/ignatov/intellij-erlang > > If you are interesting in this plugin, feel free to use the issue > tracker: https://github.com/ignatov/intellij-erlang/issues > > Stay tuned. > > Cheers, > Sergey Ignatov > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Bryan Hughes *Go Factory* http://www.go-factory.net /"Internet Class, Enterprise Grade"/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From alex.arnon@REDACTED Mon Jun 10 09:56:15 2013 From: alex.arnon@REDACTED (Alex Arnon) Date: Mon, 10 Jun 2013 10:56:15 +0300 Subject: [erlang-questions] [ANN] Erlang plugin for IntelliJ IDEA version 0.4 In-Reply-To: <51B4F536.7020708@go-factory.net> References: <51B4F536.7020708@go-factory.net> Message-ID: +1 On Mon, Jun 10, 2013 at 12:35 AM, Bryan Hughes wrote: > Hey Everyone, > > I have been using Sergey's plugin for a few months now and it is really > quite nice. If you are a huge fan like I am of Jetbrains IDE's, then it is > a must have. I develop in Objective-C (iPhone), Java (Android) and Erlang > and it is nice to have everything under one roof in a robust IDE (sadly I > lost my Emacs skills a while back). > > Understand that it is a work in progress, but is pretty impressive. I > have personally contributed to his work and would urge any of you who find > the plugin valuable to support him. > > If you go to... > > http://ignatov.github.io/intellij-erlang/ > > and scroll down a ways, there a link if you would like to make a donation. > > Cheers, > Bryan > > > On 1/21/13 12:21 AM, Max Bourinov wrote: > > Hi Sergey, > > I just tried your plugin and it looks very promising. I will give it a > try in my next project. Thank you for your work! > > p.s.: I am myself used to use idea from 2004, but last year switched to > Sublime Text 2. > > Best regards, > Max > > > > On Mon, Jan 21, 2013 at 11:29 AM, Sergey Ignatov wrote: > >> Hi all, >> >> Today I'm happy to announce the new version of Erlang plugin for >> IntelliJ IDEA. The new version is the biggest improvement of our >> project. >> >> The highlight of the new version: >> - Rebar run configurations (also includes Rebar-based Eunit integration) >> - Erlang types support >> - External and local documentation for OTP functions and modules >> - Function parameter info context help >> - Upgraded structure view (shows exported and inner functions) >> - Go to action for all Erlang definitions >> - Improved BIFs completion >> - And many others, please check out release notes: >> https://github.com/ignatov/intellij-erlang#version-04 >> >> Plugin page in IntelliJ repository: >> http://plugins.jetbrains.com/plugin/?pluginId=7083 >> Source code: https://github.com/ignatov/intellij-erlang >> >> If you are interesting in this plugin, feel free to use the issue >> tracker: https://github.com/ignatov/intellij-erlang/issues >> >> Stay tuned. >> >> Cheers, >> Sergey Ignatov >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > -- > > Bryan Hughes > *Go Factory* > http://www.go-factory.net > > *"Internet Class, Enterprise Grade"* > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jozsef.berces@REDACTED Mon Jun 10 10:56:05 2013 From: jozsef.berces@REDACTED (=?iso-8859-1?Q?J=F3zsef_B=E9rces?=) Date: Mon, 10 Jun 2013 08:56:05 +0000 Subject: [erlang-questions] async threads Message-ID: <7460EBDDCF52084A849D0F271CE059B8138BBD@ESESSMB101.ericsson.se> Hi, Is there a way to increase the async threads once the VM is running? I have a VM running already and but I do not want to have outage by restarting it. But it seems it waits for IO too much and I would like to see if increasing the async threads improves anything. Thanks, Jozsef -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Mon Jun 10 11:01:19 2013 From: bchesneau@REDACTED (Benoit Chesneau) Date: Mon, 10 Jun 2013 11:01:19 +0200 Subject: [erlang-questions] [ANN] hackney 0.4.1 released Message-ID: Just released hackney version 0.4.1, Hackney is an HTTP client library in Erlang, compatible with erlang R15 and sup Main features of Hackney are: - no message passing: response is directly streamed to the current process and a state is kept in a `#client{}' record. - binary streams - SSL support - Keepalive handling - basic authentication - stream the response and the requests - multipart support (stramed or not) - chunked encoding support - Can send files using the sendfile API - Chunked encoding support - Optional socket pool - Used parse transform for shorcut methods calls: `hackney:get("https://friendpaste.com")' You can find a description of latest changes here: https://github.com/benoitc/hackney/blob/master/NEWS.md#041---20130610 or by looking at the commits logs: https://github.com/benoitc/hackney/compare/0.4.0...0.4.1 Enjoy :) - benoit From jesper.louis.andersen@REDACTED Mon Jun 10 11:13:01 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 10 Jun 2013 11:13:01 +0200 Subject: [erlang-questions] async threads In-Reply-To: <7460EBDDCF52084A849D0F271CE059B8138BBD@ESESSMB101.ericsson.se> References: <7460EBDDCF52084A849D0F271CE059B8138BBD@ESESSMB101.ericsson.se> Message-ID: <68AEACB9-3F7D-42A1-818A-645B2E555F94@erlang-solutions.com> On Jun 10, 2013, at 10:56 AM, J?zsef B?rces wrote: > Hi, > > Is there a way to increase the async threads once the VM is running? > I have a VM running already and but I do not want to have outage by restarting it. But it seems it waits for IO too much and I would like to see if increasing the async threads improves anything. If you are talking about "iowait" in the linux kernel, then my guess would be "no improvement". Increasing async threads means that disk I/O won't block schedulers as much which can be beneficial if you have other processes that can do meaningful work without hitting the disk. But if you are ending up on the disk anyhow, then chances are you are just going to make things worse if there are more disk access. I would use a tool to look at threads on the kernel level and look how many threads are currently blocked waiting for disk I/O. Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen From jesper.louis.andersen@REDACTED Mon Jun 10 11:14:15 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 10 Jun 2013 11:14:15 +0200 Subject: [erlang-questions] NIF and timing In-Reply-To: <51B1CE1C.7090306@erix.ericsson.se> References: <186519ab-06cf-41e6-8258-e8c8e61ae537@googlegroups.com> <51B1CE1C.7090306@erix.ericsson.se> Message-ID: On Jun 7, 2013, at 2:12 PM, Sverker Eriksson wrote: > If your NIF builds terms that triggers a garbage collection (fills up the heap), that GC will be done before the NIF returns returns to its caller. > How does this fit in with long-running NIFs blocking the scheduler and messing up the scheduler migration code? Jesper Louis Andersen Erlang Solutions Ltd., Copenhagen From max.lapshin@REDACTED Mon Jun 10 11:49:09 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 10 Jun 2013 13:49:09 +0400 Subject: [erlang-questions] async threads In-Reply-To: <68AEACB9-3F7D-42A1-818A-645B2E555F94@erlang-solutions.com> References: <7460EBDDCF52084A849D0F271CE059B8138BBD@ESESSMB101.ericsson.se> <68AEACB9-3F7D-42A1-818A-645B2E555F94@erlang-solutions.com> Message-ID: Flussonic is currently running on some installations and sending more than 4 GBit/s of video from disk to the network. Of course, no hardware RAID is used, because it cannot work with such loads. It is a common situation when one of 16 attached HDD is used on 100%. I have decided to make following configuration: 160 async threads with small stack size and set of gen_servers that can block access to one hdd. When it is required to read a chunk from disk, web process asks this queue locker to allow to read from disk and if during 5 seconds, access isn't granted, user gets HTTP 503 which means that this disk is overloaded. From dima@REDACTED Mon Jun 10 12:39:55 2013 From: dima@REDACTED (Dmitry Vasilev) Date: Mon, 10 Jun 2013 12:39:55 +0200 Subject: [erlang-questions] [ANN] ErlPort 1.0.0alpha released Message-ID: <51B5ACFB.7000408@hlabs.org> Hi all, I've just released ErlPort version 1.0.0alpha. ErlPort is a library for Erlang which helps connect Erlang to a number of other programming languages (currently supported Python and Ruby). Apart from using ErlPort as a tool to call Python/Ruby functions from Erlang it also can be used as a middleware for Python/Ruby instances. Check http://erlport.org for more details. The new version of ErlPort is basically a complete redesign and rewrite of the project. The main changes in this version are: - Redesigned as Erlang application - Added support for all recent (2.5 and higher) versions of Python - Added support for all recent (1.8.6 and higher) versions of Ruby - Added support for custom data types The following is an example Python session with Erlang shell to give you a feeling how ErlPort works: 1> {ok, P} = python:start(). {ok,<0.34.0>} 2> python:call(P, sys, 'version.__str__', []). <<"2.7.3 (default, Aug 1 2012, 05:14:39) \n[GCC 4.6.3]">> 3> python:call(P, operator, add, [2, 2]). 4 4> python:stop(P). ok -- Dmitry Vasiliev http://hlabs.org http://twitter.com/hdima From yoursurrogategod@REDACTED Mon Jun 10 14:08:53 2013 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Mon, 10 Jun 2013 08:08:53 -0400 Subject: [erlang-questions] Would this function be tail-recursive? Message-ID: Hi all, it's been some time since I've touched Erlang. I found this code and would like to know if it would be tail-recursive: http://bin.cakephp.org/view/2084302499 Would loop need to be called at the end of the method or is it fine if it's called just before end? -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Jun 10 14:10:50 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 10 Jun 2013 16:10:50 +0400 Subject: [erlang-questions] [ANN] hackney 0.4.1 released In-Reply-To: References: Message-ID: what for do you use parse_transform for hackney:get ? Why you haven't just made a shortcut? From masklinn@REDACTED Mon Jun 10 14:23:53 2013 From: masklinn@REDACTED (Masklinn) Date: Mon, 10 Jun 2013 14:23:53 +0200 Subject: [erlang-questions] Would this function be tail-recursive? In-Reply-To: References: Message-ID: <55D38927-5FEF-40E5-B68D-A877C0238AC8@masklinn.net> On 2013-06-10, at 14:08 , Yves S. Garret wrote: > Hi all, it's been some time since I've touched Erlang. I found this code > and would > like to know if it would be tail-recursive: > http://bin.cakephp.org/view/2084302499 > > Would loop need to be called at the end of the method or is it fine if it's > called just before end? This function is tail-recursive yes: there's nothing to evaluate after the recursive `loop` call so it's in "tail" position. From bchesneau@REDACTED Mon Jun 10 15:13:11 2013 From: bchesneau@REDACTED (Benoit Chesneau) Date: Mon, 10 Jun 2013 15:13:11 +0200 Subject: [erlang-questions] [ANN] hackney 0.4.1 released In-Reply-To: References: Message-ID: On Mon, Jun 10, 2013 at 2:10 PM, Max Lapshin wrote: > what for do you use parse_transform for hackney:get ? Why you haven't > just made a shortcut? If I don't use parse transforms I would have to do a shortcut for each methods. Methods supported are: https://github.com/benoitc/hackney/blob/master/include/hackney.hrl#L1 It was easier to do like this. Why? - benoit From n.oxyde@REDACTED Mon Jun 10 15:51:47 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Mon, 10 Jun 2013 15:51:47 +0200 Subject: [erlang-questions] [ANN] hackney 0.4.1 released In-Reply-To: References: Message-ID: <52DCF3BB-B057-42F5-9EA1-B8B1A0A7592B@gmail.com> Hello Benoit, Why use a parse transform for such a tiny amount of boilerplate when you could have used a single macro taking a method name and generating the shortcut function? Shortcuts work in the shell, parse transforms don't. Regards, -- Anthony Ramine Le 10 juin 2013 ? 15:13, Benoit Chesneau a ?crit : > On Mon, Jun 10, 2013 at 2:10 PM, Max Lapshin wrote: >> what for do you use parse_transform for hackney:get ? Why you haven't >> just made a shortcut? > > If I don't use parse transforms I would have to do a shortcut for each > methods. Methods supported are: > > https://github.com/benoitc/hackney/blob/master/include/hackney.hrl#L1 > > It was easier to do like this. Why? > > - benoit > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From bchesneau@REDACTED Mon Jun 10 15:59:10 2013 From: bchesneau@REDACTED (Benoit Chesneau) Date: Mon, 10 Jun 2013 15:59:10 +0200 Subject: [erlang-questions] [ANN] hackney 0.4.1 released In-Reply-To: <52DCF3BB-B057-42F5-9EA1-B8B1A0A7592B@gmail.com> References: <52DCF3BB-B057-42F5-9EA1-B8B1A0A7592B@gmail.com> Message-ID: On Mon, Jun 10, 2013 at 3:51 PM, Anthony Ramine wrote: > Hello Benoit, > > Why use a parse transform for such a tiny amount of boilerplate when you could have used a single macro taking a method name and generating the shortcut function? > > Shortcuts work in the shell, parse transforms don't. Parse transforms was the only thing coming in my mind at that time. Anything that could work in the shell would be better indeed. How would work this macro? - benoit From n.oxyde@REDACTED Mon Jun 10 16:23:04 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Mon, 10 Jun 2013 16:23:04 +0200 Subject: [erlang-questions] [ANN] hackney 0.4.1 released In-Reply-To: References: <52DCF3BB-B057-42F5-9EA1-B8B1A0A7592B@gmail.com> Message-ID: -define(SHORTCUT(Method), Method(URL) -> hackney:request(Method, URL)). ?SHORTCUT(get). To be honest I would just write each shortcut without any trick. -- Anthony Ramine Le 10 juin 2013 ? 15:59, Benoit Chesneau a ?crit : > On Mon, Jun 10, 2013 at 3:51 PM, Anthony Ramine wrote: >> Hello Benoit, >> >> Why use a parse transform for such a tiny amount of boilerplate when you could have used a single macro taking a method name and generating the shortcut function? >> >> Shortcuts work in the shell, parse transforms don't. > > Parse transforms was the only thing coming in my mind at that time. > Anything that could work in the shell would be better indeed. How > would work this macro? > > - benoit From jinni.park@REDACTED Tue Jun 11 03:40:06 2013 From: jinni.park@REDACTED (Park, Sungjin) Date: Tue, 11 Jun 2013 10:40:06 +0900 Subject: [erlang-questions] Fwd: gen_server doesn't respond In-Reply-To: References: <18FB305D-6645-4EB0-91D5-B44A3E0C0FD6@gmail.com> <1738567306.139934343.1370465333194.JavaMail.root@erlang-solutions.com> Message-ID: I succeeded in reproducing the problem throughout three days expedition and about to share the experience now. All the gen_server:call/4 in the stacktrace are found to be calls to lager actually. The lager behaves asynchronously at first but goes synchronous at some limit. Once it goes synchronous there are hundreds of thousands of process waiting for reply. It makes the system hang. Actually the vm and shell is still responsive but the processes hang and any new process attempting to leave a log hangs also. I was using lager with {async, 100} and 1000 rolling files of 100MB each. I suppose shifting those files would have taken quite some time for lager that it triggered lager's synchronous mode. What made the problem more probable is that there was a cron scheduler designed to copy the log files regularly to a different directory, whose schedule could overlap with lager's log file shifting schedule eventually to make things worse. I solved the problem by reducing number of rolling files and changing the cron scheduler to move files rather than to copy and to run more often to prevent burst. One thing I hope there be an option for lager not to go synchronous but just drop logs at some limit. For my kind of system, synchronous logging is simply unusable. On Thu, Jun 6, 2013 at 5:48 AM, Robert Virding < robert.virding@REDACTED> wrote: > Funny discussion this. In this case I think Dmitry hit the nail on the > head when he said that > > {current_function,{*gen,do_call,4*}} > > exactly describes what your problem is. The gen_server is suspended > waiting in a gen_server:call to another server which is not replying. > (gen_server:call -> gen:call/4 -> gen:do_call/4) With a little basic > tracing (sys:trace) you should be able to work out where/why the server is > calling another server and why it is hanging. > > Robert > > ------------------------------ > > *From: *"Dmitry Kolesnikov" > *To: *"Sungjin Park" > *Cc: *"erlang-questions" > *Sent: *Tuesday, 4 June, 2013 10:45:40 AM > > *Subject: *Re: [erlang-questions] gen_server doesn't respond > > Hello, > > {current_function,{*gen,do_call,4*}} > > Your process made a call to another process with infinite timeout. > > The process do not respond because you got a deadlock. > You have to check: > * no circular dependencies on process call > * the parent supervisor is not called from process init > * no infinite gen_server:call is used at your api. > > - Dmitry > > On Jun 4, 2013, at 4:54 AM, "Park, Sungjin" wrote: > > Shell input/output logs below. > > ============================ > 4> process_info(pid(0,453,4757)). > [{current_function,{*gen,do_call,4*}}, > {initial_call,{proc_lib,init_p,5}}, > {status,*waiting*}, > {message_queue_len,1}, > {messages,[*{'$gen_call',{<0.26172.4841>,* > * #Ref<0.0.2946.93178>},* > * state}*]}, > {links,[<0.25337.4839>]}, > {dictionary,[{'$ancestors',[<0.452.4757>,<0.424.0>, > <0.423.0>,ranch_sup,<0.420.0>]}, > {'$initial_call',{my_session,init,1}}]}, > {trap_exit,true}, > {error_handler,error_handler}, > {priority,normal}, > {group_leader,<0.419.0>}, > {total_heap_size,1364}, > {heap_size,987}, > {stack_size,32}, > {reductions,1487}, > {garbage_collection,[{min_bin_vheap_size,46368}, > {min_heap_size,233}, > {fullsweep_after,65535}, > {minor_gcs,8}]}, > {suspending,[]}] > ============================= > > As you can see the process has a message {..., state} in it's message > queue but still in waiting state. It doesn't respond to > gen_server:call(Pid, state) from this point on. The process_info shows > that the process is now in gen:do_call/4 which is not under my control. The > module has all the fallback handlers for handle_call/3, handle_cast/2, > handle_info/2 etc, so that every message be matched. There are a couple of > places in my project where gen_server:call/2 used against this process. I > suppose that the calls should timeout in 5 sec by default as is stated in > the gen_server specification. So there should be no such case that the > process stuck indefinitely. > > > > On Mon, Jun 3, 2013 at 7:39 PM, Aaron France wrote: > >> Does it make any synchronous calls to other gen_servers? >> >> >> On Mon, Jun 3, 2013 at 12:34 PM, Tim Watson wrote: >> >>> Does the callback module execute any selective receives inside handlers >>> perhaps? If not, are you sure the message has actually arrived in that >>> process' mailbox, can you see unhandled messages in the message queue >>> (I.e., length > 0 whilst waiting)? >>> >>> On 3 Jun 2013, at 10:34, "Park, Sungjin" wrote: >>> >>> > Guys, >>> > >>> > I'm encountering cases when gen_server doesn't respond. >>> > In such cases, erlang:process_info/1 shows the process state 'waiting' >>> with some messages in the message_queue. The messages have not been >>> consumed for tens of minutes at least. Anybody has any idean about this? >>> > >>> > >>> > -- >>> > Park, Sungjin >>> > >>> ------------------------------------------------------------------------------------------------------------------- >>> > Peculiar travel suggestions are dancing lessons from god. >>> > -- The Books of Bokonon >>> > >>> ------------------------------------------------------------------------------------------------------------------- >>> > _______________________________________________ >>> > erlang-questions mailing list >>> > erlang-questions@REDACTED >>> > http://erlang.org/mailman/listinfo/erlang-questions >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> > > > -- > Park, Sungjin > > ------------------------------------------------------------------------------------------------------------------- > Peculiar travel suggestions are dancing lessons from god. > -- The Books of Bokonon > > ------------------------------------------------------------------------------------------------------------------- > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From khandrish@REDACTED Tue Jun 11 05:13:46 2013 From: khandrish@REDACTED (Chris Hicks) Date: Mon, 10 Jun 2013 20:13:46 -0700 Subject: [erlang-questions] Getting chunked responses with ibrowse Message-ID: Hello all, I've been playing around with ibrowse, trying to get chunked responses from a server and the behavior I'm seeing isn't at all what I expect given the documentation. When I send the following request: ibrowse:send_req(URL, [], get, [], [{stream_to, self()}]). I get the headers back from the server immediately...and then no messages after that, until the timeout message anyway. From what the documentation says, I'd expect to just continue to get messages "as data arrives on the socket" but this apparently isn't the case. If I call stream_next(Request_id) nothing happens at all. However, if I instead change the call to: ibrowse:send_req(URL, [], get, [], [{stream_to, {self(), once}}]). I get the headers, and then when I call stream_next(Request_id) I suddenly begin to get the packets the server is sending. Can anyone explain to me why I am seeing what I am seeing? The use case I am trying to fulfill is that I connect to a server and get a set of data back that describes the state of the server as a json object. On any changes, which might not happen for a very long time (days/months), a new set of data is streamed to the client. I would like to have one process receiving multiple streams, from multiple servers, handling these status changes. Is this possible, or will I need to spin up workers for each stream and have them wait for new data, providing it to that central process? -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Tue Jun 11 09:27:26 2013 From: bchesneau@REDACTED (Benoit Chesneau) Date: Tue, 11 Jun 2013 09:27:26 +0200 Subject: [erlang-questions] [ANN] hackney 0.4.1 released In-Reply-To: References: <52DCF3BB-B057-42F5-9EA1-B8B1A0A7592B@gmail.com> Message-ID: On Mon, Jun 10, 2013 at 4:23 PM, Anthony Ramine wrote: > -define(SHORTCUT(Method), Method(URL) -> hackney:request(Method, URL)). > ?SHORTCUT(get). > > To be honest I would just write each shortcut without any trick. > > Well It doesn't provide the same syntax here where yo can have ` hackney:get(..)` or `hackney:post(...)` The main reason I'm doing a parse transform is to not have to worry about adding a method or handling different cases, for example: hackney:get(URL) but also: hackney:get(URL, Headers) or: hackney:get(Url, Headers, Options) and do that for each method. Most of pseudo rest client don't handle all methods. But I can indeed write them manually, not a big deal. Will do that on next release. In the mean time if there is a way to automatized that, that would be good. - ben From thomas@REDACTED Tue Jun 11 13:59:21 2013 From: thomas@REDACTED (Thomas Allen) Date: Tue, 11 Jun 2013 04:59:21 -0700 Subject: [erlang-questions] [ANN] hackney 0.4.1 released In-Reply-To: References: <52DCF3BB-B057-42F5-9EA1-B8B1A0A7592B@gmail.com> Message-ID: <1370951961.15049.140661242532653.1FB0A7D4@webmail.messagingengine.com> Hi Benoit, On Tue, Jun 11, 2013, at 12:27 AM, Benoit Chesneau wrote: > The main reason I'm doing a parse transform is to not have to worry > about adding a method or handling different cases, for example: > > hackney:get(URL) > > but also: > > hackney:get(URL, Headers) > > or: > > hackney:get(Url, Headers, Options) > > > and do that for each method. In that case, I'd think the macro approach is probably the least error-prone and easiest to maintain. Yours in bikeshedding, Thomas From andrew.pennebaker@REDACTED Tue Jun 11 16:25:10 2013 From: andrew.pennebaker@REDACTED (Andrew Pennebaker) Date: Tue, 11 Jun 2013 10:25:10 -0400 Subject: [erlang-questions] Bundle rebar with Erlang/OTP packages Message-ID: A nice feature of Ruby is the built-in RubyGems system, an easy way for developers to package, release, and install each other's Ruby modules. Erlang suffers from a lack of a de facto package system. Most recommend rebar, but it's not that easy to setup, and most of the Erlang tutorials don't say how to use it. I think it would greatly improve the Erlang community if we bundled rebar with the official Erlang/OTP packages, so that developers could package, release, and install each other's modules with out of the box Erlang, no additional tools required. -- Cheers, Andrew Pennebaker www.yellosoft.us -------------- next part -------------- An HTML attachment was scrubbed... URL: From lemenkov@REDACTED Tue Jun 11 16:34:47 2013 From: lemenkov@REDACTED (Peter Lemenkov) Date: Tue, 11 Jun 2013 18:34:47 +0400 Subject: [erlang-questions] Bundle rebar with Erlang/OTP packages In-Reply-To: References: Message-ID: 2013/6/11 Andrew Pennebaker : > Erlang suffers from a lack of a de facto package system. Most recommend > rebar, but it's not that easy to setup, and most of the Erlang tutorials > don't say how to use it. > > I think it would greatly improve the Erlang community if we bundled rebar > with the official Erlang/OTP packages, so that developers could package, > release, and install each other's modules with out of the box Erlang, no > additional tools required. Actually you don't need to build and install rebar. It's already available in all major Linux distributions' repositories - Debian, Ubuntu, RHEL, Fedora. -- With best regards, Peter Lemenkov. From andrew.pennebaker@REDACTED Tue Jun 11 16:59:13 2013 From: andrew.pennebaker@REDACTED (Andrew Pennebaker) Date: Tue, 11 Jun 2013 10:59:13 -0400 Subject: [erlang-questions] Bundle rebar with Erlang/OTP packages In-Reply-To: References: Message-ID: That's nice to know! What about Windows and Mac? For some reason, rebar isn't in stable Homebrew yet, only in head. On Tue, Jun 11, 2013 at 10:34 AM, Peter Lemenkov wrote: > 2013/6/11 Andrew Pennebaker : > > > Erlang suffers from a lack of a de facto package system. Most recommend > > rebar, but it's not that easy to setup, and most of the Erlang tutorials > > don't say how to use it. > > > > I think it would greatly improve the Erlang community if we bundled rebar > > with the official Erlang/OTP packages, so that developers could package, > > release, and install each other's modules with out of the box Erlang, no > > additional tools required. > > Actually you don't need to build and install rebar. It's already > available in all major Linux distributions' repositories - Debian, > Ubuntu, RHEL, Fedora. > > -- > With best regards, Peter Lemenkov. > -- Cheers, Andrew Pennebaker www.yellosoft.us -------------- next part -------------- An HTML attachment was scrubbed... URL: From lemenkov@REDACTED Tue Jun 11 17:05:34 2013 From: lemenkov@REDACTED (Peter Lemenkov) Date: Tue, 11 Jun 2013 19:05:34 +0400 Subject: [erlang-questions] Bundle rebar with Erlang/OTP packages In-Reply-To: References: Message-ID: 2013/6/11 Andrew Pennebaker : > That's nice to know! > > What about Windows and Mac? For some reason, rebar isn't in stable Homebrew > yet, only in head. I personally don't care about Windows users since I don't see any real reason to use Erlang on Windows. Regarding Mac OS X - indeed you should invest your time into enhancing package system for that platform. -- With best regards, Peter Lemenkov. From egil@REDACTED Tue Jun 11 17:39:02 2013 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Tue, 11 Jun 2013 17:39:02 +0200 Subject: [erlang-questions] R16B01 Release Candidate 1 on Github Message-ID: <51B74496.50102@erlang.org> Hi, Things are dropping in place and getting stable so we have pushed a R16B01 release candidate to GitHub. https://github.com/erlang/otp/tree/OTP_R16B01_RC1 git fetch git://github.com/erlang/otp.git OTP_R16B01_RC1 Some minor things are still not ready yet, but we are getting there. Take it for a spin and see if we have missed something critical. // Bj?rn-Egil Erlang/OTP From moxford@REDACTED Tue Jun 11 18:04:17 2013 From: moxford@REDACTED (Mike Oxford) Date: Tue, 11 Jun 2013 09:04:17 -0700 Subject: [erlang-questions] erlang/C embedded db suggestion In-Reply-To: References: Message-ID: BDB would work well for this. G'luck! -mox -------------- next part -------------- An HTML attachment was scrubbed... URL: From jleivent@REDACTED Tue Jun 11 18:07:01 2013 From: jleivent@REDACTED (Jonathan Leivent) Date: Tue, 11 Jun 2013 12:07:01 -0400 Subject: [erlang-questions] Bundle rebar with Erlang/OTP packages In-Reply-To: References: Message-ID: <51B74B25.6020303@comcast.net> On 06/11/2013 10:34 AM, Peter Lemenkov wrote: > 2013/6/11 Andrew Pennebaker : >... > Actually you don't need to build and install rebar. It's already > available in all major Linux distributions' repositories - Debian, > Ubuntu, RHEL, Fedora. > > -- > With best regards, Peter Lemenkov. I'm using Linux Mint Debian Edition, and using Synaptic to search for "rebar" only finds the Erlang plugin for Vim (because it mentions rebar in its features comment). I also tried command-line apt search rebar - and got nothing. I then tried the same search on an Ubuntu 12.10 machine using Ubuntu's software sources gui, and still got nothing. Hmmm... searching in packages.debian.org finds hits only in jessie and sid - not squeeze or wheezy. A search of packages.ubuntu.com only shows it under Raring (13.04). -- Jonathan From andrew.pennebaker@REDACTED Tue Jun 11 18:21:33 2013 From: andrew.pennebaker@REDACTED (Andrew Pennebaker) Date: Tue, 11 Jun 2013 12:21:33 -0400 Subject: [erlang-questions] Bundle rebar with Erlang/OTP packages In-Reply-To: References: Message-ID: That's a horrible position. Windows isn't my favorite OS either, but it only hurts the Erlang commumity to dismiss support for it. There are plenty of services using a Windows stack, and if we don't care about Windows support, Windows developers will continue to use programming languages that DO care, like Haskell. On Jun 11, 2013 11:05 AM, "Peter Lemenkov" wrote: > 2013/6/11 Andrew Pennebaker : > > That's nice to know! > > > > What about Windows and Mac? For some reason, rebar isn't in stable > Homebrew > > yet, only in head. > > I personally don't care about Windows users since I don't see any real > reason to use Erlang on Windows. Regarding Mac OS X - indeed you > should invest your time into enhancing package system for that > platform. > -- > With best regards, Peter Lemenkov. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Tue Jun 11 19:52:22 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Tue, 11 Jun 2013 19:52:22 +0200 Subject: [erlang-questions] [ANN] hackney 0.4.1 released In-Reply-To: <1370951961.15049.140661242532653.1FB0A7D4@webmail.messagingengine.com> References: <52DCF3BB-B057-42F5-9EA1-B8B1A0A7592B@gmail.com> <1370951961.15049.140661242532653.1FB0A7D4@webmail.messagingengine.com> Message-ID: <07DF404C-2853-4144-BDE9-35E78D4A3751@gmail.com> Got a better idea, with macros: https://github.com/benoitc/hackney/pull/26 -- Anthony Ramine Le 11 juin 2013 ? 13:59, Thomas Allen a ?crit : > Hi Benoit, > > On Tue, Jun 11, 2013, at 12:27 AM, Benoit Chesneau wrote: >> The main reason I'm doing a parse transform is to not have to worry >> about adding a method or handling different cases, for example: >> >> hackney:get(URL) >> >> but also: >> >> hackney:get(URL, Headers) >> >> or: >> >> hackney:get(Url, Headers, Options) >> >> >> and do that for each method. > > In that case, I'd think the macro approach is probably the least > error-prone and easiest to maintain. > > Yours in bikeshedding, > Thomas > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jeremy@REDACTED Tue Jun 11 20:34:18 2013 From: jeremy@REDACTED (Jeremy Ong) Date: Tue, 11 Jun 2013 11:34:18 -0700 Subject: [erlang-questions] Bundle rebar with Erlang/OTP packages In-Reply-To: References: Message-ID: My 2 cents. Package management and rebar come up all the time on the listserv and it will continue to come up until a viable long term solution is in place. I think the thing to fix is the underlying issue of application level namespacing. The existing dependency management system is an afterthought. If application A requires (X, v1.0) and application B requires (X, v1.1), and you want A and B to coexist, you're going to have issues. Ideally, erlang would know that when application A calls a module from X, it references the v1.0 one etc. In short, the place to start thinking about this should not be any peripheral system but incorporating it into the underlying language. On Tue, Jun 11, 2013 at 9:21 AM, Andrew Pennebaker < andrew.pennebaker@REDACTED> wrote: > That's a horrible position. Windows isn't my favorite OS either, but it > only hurts the Erlang commumity to dismiss support for it. There are plenty > of services using a Windows stack, and if we don't care about Windows > support, Windows developers will continue to use programming languages that > DO care, like Haskell. > On Jun 11, 2013 11:05 AM, "Peter Lemenkov" wrote: > >> 2013/6/11 Andrew Pennebaker : >> > That's nice to know! >> > >> > What about Windows and Mac? For some reason, rebar isn't in stable >> Homebrew >> > yet, only in head. >> >> I personally don't care about Windows users since I don't see any real >> reason to use Erlang on Windows. Regarding Mac OS X - indeed you >> should invest your time into enhancing package system for that >> platform. >> -- >> With best regards, Peter Lemenkov. >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From darkit@REDACTED Tue Jun 11 20:17:41 2013 From: darkit@REDACTED (Max Grigoriev) Date: Tue, 11 Jun 2013 21:17:41 +0300 Subject: [erlang-questions] Cowboy REST POST and custom status code Message-ID: How can I set status code in my REST POST handler. Now I'm doing this make_post(Req, State) -> lager:info("post"), Resp = cowboy_req:set_resp_body(<<"HELLO POST\n">>, Req), {ok, Resp3} = cowboy_req:reply(201, Resp), {true, Resp3, State}. But I receive such errors: 1:43:10.510 [info] [nonode@REDACTED#pbshare_logic_registration#<0.205.0>] [handle_info#35] :Exit Logic from <0.204.0> Reason: {function_clause,[{cowboy_req,reply,[204,[],<<>>,{http_req,#Port<0.6355>,ranch_tcp,keepalive,<0.204.0>,<<"POST">>,'HTTP/1.1',{{127,0,0,1},50024},<<"localhost">>,undefined,8080,<<"/rest/registration/make">>,[<<"make">>],<<>>,undefined,[],[{<<"user-agent">>,<<"curl/7.29.0">>},{<<"host">>,<<"localhost:8080">>},{<<"accept">>,<<" */* ">>},{<<"content-type">>,<<"application/json">>},{<<"content-length">>,<<"14">>}],[{<<"content-length">>,14},{<<"expect">>,undefined},{<<"content-length">>,14},{<<"content-type">>,{<<"application">>,<<"json">>,[]}},{<<"if-modified-since">>,undefined},{<<"if-none-match">>,undefined},{<<"if-unmodified-since">>,undefined},{<<"if-match">>,undefined},{<<"accept">>,[{{<<" *">>,<<"*">>,[]},1000,[]}]}],undefined,[{charset,undefined},{media_type,{<<"application">>,<<"json">>,[]}}],done,undefined,<<>>,false,done,[],<<>>,undefined}],[{file,"src/cowboy_req.erl"},{line,948}]},{cowboy_rest,respond,3,[{file,"src/cowboy_rest.erl"},{line,1085}]},{cowboy_rest,upgrade,4,[{file,"src/cowboy_rest.erl"},{line,75}]},{cowboy_protocol,execute,4,[{file,"src/cowboy_protocol.erl"},{line,523}]}]} 21:43:10.510 [error] [Undefined#Undefined#emulator] [Undefined#Undefined] :Error in process <0.204.0> with exit value: {function_clause,[{cowboy_req,reply,[204,[],<<0 bytes>>,{http_req,#Port<0.6355>,ranch_tcp,keepalive,<0.204.0>,<<4 bytes>>,'HTTP/1.1',{{127,0,0,1},50024},<<9 bytes>>,undefined,8080,<<23 bytes>>,[<<4 bytes>>],<<0 bytes>>,undefined,[],[{<<10 bytes>>,<<11 bytes>>},{<<4 bytes>>,<<14 bytes>>},{<<6 bytes>>,<<3 bytes>>},{<<12 bytes>>,<<16 bytes>>},{<<14 bytes>>,<<2 bytes>>}],[{<<14 bytes>>,14},{<<6 bytes>>,undefined},{<<14 bytes>>,14},{<<12 bytes>>,{<<11 bytes>>,<<4 bytes>>,[]}},{<<17 bytes>>,undefined},{<<13 bytes>>,undefined},{<<19 bytes>>,undefined},{<<8 bytes>>,undefined... -------------- next part -------------- An HTML attachment was scrubbed... URL: From aaron.l.france@REDACTED Tue Jun 11 21:25:17 2013 From: aaron.l.france@REDACTED (Aaron France) Date: Tue, 11 Jun 2013 21:25:17 +0200 Subject: [erlang-questions] R16B01 Release Candidate 1 on Github In-Reply-To: <51B74496.50102@erlang.org> References: <51B74496.50102@erlang.org> Message-ID: Hi, Is there a change list anywhere? On Tue, Jun 11, 2013 at 5:39 PM, Bj?rn-Egil Dahlberg wrote: > Hi, > > Things are dropping in place and getting stable so we have pushed a R16B01 > release candidate to GitHub. > > https://github.com/erlang/otp/**tree/OTP_R16B01_RC1 > > git fetch git://github.com/erlang/otp.**gitOTP_R16B01_RC1 > > Some minor things are still not ready yet, but we are getting there. > > Take it for a spin and see if we have missed something critical. > > // Bj?rn-Egil > Erlang/OTP > > > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Tue Jun 11 21:30:14 2013 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 11 Jun 2013 12:30:14 -0700 Subject: [erlang-questions] Bundle rebar with Erlang/OTP packages In-Reply-To: References: Message-ID: <51B77AC6.1080401@gmail.com> I have a script, to move dependencies into their own namespace, so that they are isolated, such that it is possible for separate versions of X to coexist. I do not consider the solution ideal, but it seems best given the current constraints, when you want to avoid module/application conflicts with other application/module dependencies. The script is here: https://github.com/okeuday/reltool_util/blob/master/scope I see the namespace concern as more of a OTP/reltool concern. The problems with rebar are often reported, but development of rebar has been haphazard and misguided. The rebar development does not change based on complaints, and so far no one has been willing to take the time to rewrite it. I do hope that a better solution presents itself, but I don't believe anyone is holding their breath for rebar to change after the previous years of instability. Some simple things: rebar could have a develop branch so that master doesn't break every-other-commit rebar could have tags more frequently, so that testing occurs with more confidence rebar could avoid a binary distribution, so that it could not remain an opaque binary that can not be included in a normal repository due to security concerns There are many other problems, but they stray into specifics... these basic problems have never been addressed since rebar was first created. On 06/11/2013 11:34 AM, Jeremy Ong wrote: > My 2 cents. > > Package management and rebar come up all the time on the listserv and it will continue to come up until a viable long term solution is in place. I think the thing to fix is the underlying issue of application level namespacing. The existing dependency management system is an afterthought. If application A requires (X, v1.0) and application B requires (X, v1.1), and you want A and B to coexist, you're going to have issues. > > Ideally, erlang would know that when application A calls a module from X, it references the v1.0 one etc. > > In short, the place to start thinking about this should not be any peripheral system but incorporating it into the underlying language. > > > On Tue, Jun 11, 2013 at 9:21 AM, Andrew Pennebaker > wrote: > > That's a horrible position. Windows isn't my favorite OS either, but it only hurts the Erlang commumity to dismiss support for it. There are plenty of services using a Windows stack, and if we don't care about Windows support, Windows developers will continue to use programming languages that DO care, like Haskell. > > On Jun 11, 2013 11:05 AM, "Peter Lemenkov" > wrote: > > 2013/6/11 Andrew Pennebaker >: > > That's nice to know! > > > > What about Windows and Mac? For some reason, rebar isn't in stable Homebrew > > yet, only in head. > > I personally don't care about Windows users since I don't see any real > reason to use Erlang on Windows. Regarding Mac OS X - indeed you > should invest your time into enhancing package system for that > platform. > -- > With best regards, Peter Lemenkov. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Tue Jun 11 21:35:11 2013 From: bchesneau@REDACTED (Benoit Chesneau) Date: Tue, 11 Jun 2013 21:35:11 +0200 Subject: [erlang-questions] [ANN] hackney 0.4.1 released In-Reply-To: <07DF404C-2853-4144-BDE9-35E78D4A3751@gmail.com> References: <52DCF3BB-B057-42F5-9EA1-B8B1A0A7592B@gmail.com> <1370951961.15049.140661242532653.1FB0A7D4@webmail.messagingengine.com> <07DF404C-2853-4144-BDE9-35E78D4A3751@gmail.com> Message-ID: On Tue, Jun 11, 2013 at 7:52 PM, Anthony Ramine wrote: > Got a better idea, with macros: > > https://github.com/benoitc/hackney/pull/26 > Thanks! I actually like the idea. I will test it asap. Maybe another way (and simpler) would be having all methods set in their own module. but anyway will play with it :) - benoit > -- > Anthony Ramine > > Le 11 juin 2013 ? 13:59, Thomas Allen a ?crit : > >> Hi Benoit, >> >> On Tue, Jun 11, 2013, at 12:27 AM, Benoit Chesneau wrote: >>> The main reason I'm doing a parse transform is to not have to worry >>> about adding a method or handling different cases, for example: >>> >>> hackney:get(URL) >>> >>> but also: >>> >>> hackney:get(URL, Headers) >>> >>> or: >>> >>> hackney:get(Url, Headers, Options) >>> >>> >>> and do that for each method. >> >> In that case, I'd think the macro approach is probably the least >> error-prone and easiest to maintain. >> >> Yours in bikeshedding, >> Thomas >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >Thanks! From ok@REDACTED Wed Jun 12 03:03:40 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 12 Jun 2013 13:03:40 +1200 Subject: [erlang-questions] Bundle rebar with Erlang/OTP packages In-Reply-To: References: Message-ID: <396FB668-27AA-47BD-8423-2CC8D45A5893@cs.otago.ac.nz> On 12/06/2013, at 2:34 AM, Peter Lemenkov wrote: > > Actually you don't need to build and install rebar. It's already > available in all major Linux distributions' repositories - Debian, > Ubuntu, RHEL, Fedora. That helps me on Mac OS X and Solaris how? (Oh, and I keep an OpenBSD VM around for testing too...) From jinni.park@REDACTED Wed Jun 12 03:12:46 2013 From: jinni.park@REDACTED (Park, Sungjin) Date: Wed, 12 Jun 2013 10:12:46 +0900 Subject: [erlang-questions] Can I prevent lager from changing to sync mode? Message-ID: I want it never fall to sync mode. Just dropping some logs is ok. -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Jun 12 06:26:25 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 12 Jun 2013 06:26:25 +0200 Subject: [erlang-questions] Cowboy REST POST and custom status code In-Reply-To: References: Message-ID: <51B7F871.3000708@ninenines.eu> Cowboy will set the status code automatically. POST requests will do a 201 if the resource didn't exist before and if a URL to the resource is provided (201 requires a Location header): make_post(Req, State) -> {{true, URL}, Req, State}. If the resource did exist before and you return the above, a 303 will be used. If you only return true, Cowboy will do a 200 or 204 depending on whether a response body has been set. On 06/11/2013 08:17 PM, Max Grigoriev wrote: > How can I set status code in my REST POST handler. > > Now I'm doing this > > |make_post(Req, State) -> > lager:info("post"), > Resp = cowboy_req:set_resp_body(<<"HELLO POST\n">>, Req), > {ok, Resp3} = cowboy_req:reply(201, Resp), > {true, Resp3, State}. > | > > But I receive such errors: > > 1:43:10.510 [info] [nonode@REDACTED#pbshare_logic_registration#<0.205.0>] > [handle_info#35] :Exit Logic from <0.204.0> Reason: > {function_clause,[{cowboy_req,reply,[204,[],<<>>,{http_req,#Port<0.6355>,ranch_tcp,keepalive,<0.204.0>,<<"POST">>,'HTTP/1.1',{{127,0,0,1},50024},<<"localhost">>,undefined,8080,<<"/rest/registration/make">>,[<<"make">>],<<>>,undefined,[],[{<<"user-agent">>,<<"curl/7.29.0">>},{<<"host">>,<<"localhost:8080">>},{<<"accept">>,<<"///">>},{<<"content-type">>,<<"application/json">>},{<<"content-length">>,<<"14">>}],[{<<"content-length">>,14},{<<"expect">>,undefined},{<<"content-length">>,14},{<<"content-type">>,{<<"application">>,<<"json">>,[]}},{<<"if-modified-since">>,undefined},{<<"if-none-match">>,undefined},{<<"if-unmodified-since">>,undefined},{<<"if-match">>,undefined},{<<"accept">>,[{{<<"/">>,<<"/">>,[]},1000,[]}]}],undefined,[{charset,undefined},{media_type,{<<"application">>,<<"json">>,[]}}],done,undefined,<<>>,false,done,[],<<>>,undefined}],[{file,"src/cowboy_req.erl"},{line,948}]},{cowboy_rest,respond,3,[{file,"src/cowboy_rest.erl"},{line,1085}]},{cowboy_rest,upgrade,4,[{file,"s rc/cowboy_rest.erl"},{line,75}]},{cowboy_protocol,execute,4,[{file,"src/cowboy_protocol.erl"},{line,523}]}]} > 21:43:10.510 [error] [Undefined#Undefined#emulator] > [Undefined#Undefined] :Error in process <0.204.0> with exit value: > {function_clause,[{cowboy_req,reply,[204,[],<<0 > bytes>>,{http_req,#Port<0.6355>,ranch_tcp,keepalive,<0.204.0>,<<4 > bytes>>,'HTTP/1.1',{{127,0,0,1},50024},<<9 bytes>>,undefined,8080,<<23 > bytes>>,[<<4 bytes>>],<<0 bytes>>,undefined,[],[{<<10 bytes>>,<<11 > bytes>>},{<<4 bytes>>,<<14 bytes>>},{<<6 bytes>>,<<3 bytes>>},{<<12 > bytes>>,<<16 bytes>>},{<<14 bytes>>,<<2 bytes>>}],[{<<14 > bytes>>,14},{<<6 bytes>>,undefined},{<<14 bytes>>,14},{<<12 > bytes>>,{<<11 bytes>>,<<4 bytes>>,[]}},{<<17 bytes>>,undefined},{<<13 > bytes>>,undefined},{<<19 bytes>>,undefined},{<<8 bytes>>,undefined... > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From darkit@REDACTED Wed Jun 12 08:08:47 2013 From: darkit@REDACTED (Max Grigoriev) Date: Tue, 11 Jun 2013 23:08:47 -0700 (PDT) Subject: [erlang-questions] Cowboy REST POST and custom status code In-Reply-To: <51B7F871.3000708@ninenines.eu> References: <51B7F871.3000708@ninenines.eu> Message-ID: <145808d6-0d9d-4ce6-8fab-7347a956ab95@googlegroups.com> 201 is just an example. We have legacy system with different components and I'm trying to change one of them to erlang. And what I really need to set different status codes and what I see it's impossible without cowboy code changes :( On Wednesday, June 12, 2013 7:26:25 AM UTC+3, Lo?c Hoguin wrote: > > Cowboy will set the status code automatically. > > POST requests will do a 201 if the resource didn't exist before and if a > URL to the resource is provided (201 requires a Location header): > > make_post(Req, State) -> > {{true, URL}, Req, State}. > > If the resource did exist before and you return the above, a 303 will be > used. > > If you only return true, Cowboy will do a 200 or 204 depending on > whether a response body has been set. > > On 06/11/2013 08:17 PM, Max Grigoriev wrote: > > How can I set status code in my REST POST handler. > > > > Now I'm doing this > > > > |make_post(Req, State) -> > > lager:info("post"), > > Resp = cowboy_req:set_resp_body(<<"HELLO POST\n">>, Req), > > {ok, Resp3} = cowboy_req:reply(201, Resp), > > {true, Resp3, State}. > > | > > > > But I receive such errors: > > > > 1:43:10.510 [info] [nonode@REDACTED#pbshare_logic_registration#<0.205.0>] > > [handle_info#35] :Exit Logic from <0.204.0> Reason: > > > {function_clause,[{cowboy_req,reply,[204,[],<<>>,{http_req,#Port<0.6355>,ranch_tcp,keepalive,<0.204.0>,<<"POST">>,'HTTP/1.1',{{127,0,0,1},50024},<<"localhost">>,undefined,8080,<<"/rest/registration/make">>,[<<"make">>],<<>>,undefined,[],[{<<"user-agent">>,<<"curl/7.29.0">>},{<<"host">>,<<"localhost:8080">>},{<<"accept">>,<<"///">>},{<<"content-type">>,<<"application/json">>},{<<"content-length">>,<<"14">>}],[{<<"content-length">>,14},{<<"expect">>,undefined},{<<"content-length">>,14},{<<"content-type">>,{<<"application">>,<<"json">>,[]}},{<<"if-modified-since">>,undefined},{<<"if-none-match">>,undefined},{<<"if-unmodified-since">>,undefined},{<<"if-match">>,undefined},{<<"accept">>,[{{<<"/">>,<<"/">>,[]},1000,[]}]}],undefined,[{charset,undefined},{media_type,{<<"application">>,<<"json">>,[]}}],done,undefined,<<>>,false,done,[],<<>>,undefined}],[{file,"src/cowboy_req.erl"},{line,948}]},{cowboy_rest,respond,3,[{file,"src/cowboy_rest.erl"},{line,1085}]},{cowboy_rest,upgrade,4,[{file,"s > > rc/cowboy_rest.erl"},{line,75}]},{cowboy_protocol,execute,4,[{file,"src/cowboy_protocol.erl"},{line,523}]}]} > > > 21:43:10.510 [error] [Undefined#Undefined#emulator] > > [Undefined#Undefined] :Error in process <0.204.0> with exit value: > > {function_clause,[{cowboy_req,reply,[204,[],<<0 > > bytes>>,{http_req,#Port<0.6355>,ranch_tcp,keepalive,<0.204.0>,<<4 > > bytes>>,'HTTP/1.1',{{127,0,0,1},50024},<<9 bytes>>,undefined,8080,<<23 > > bytes>>,[<<4 bytes>>],<<0 bytes>>,undefined,[],[{<<10 bytes>>,<<11 > > bytes>>},{<<4 bytes>>,<<14 bytes>>},{<<6 bytes>>,<<3 bytes>>},{<<12 > > bytes>>,<<16 bytes>>},{<<14 bytes>>,<<2 bytes>>}],[{<<14 > > bytes>>,14},{<<6 bytes>>,undefined},{<<14 bytes>>,14},{<<12 > > bytes>>,{<<11 bytes>>,<<4 bytes>>,[]}},{<<17 bytes>>,undefined},{<<13 > > bytes>>,undefined},{<<19 bytes>>,undefined},{<<8 bytes>>,undefined... > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-q...@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-q...@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From darkit@REDACTED Wed Jun 12 08:15:18 2013 From: darkit@REDACTED (Max Grigoriev) Date: Tue, 11 Jun 2013 23:15:18 -0700 (PDT) Subject: [erlang-questions] Cowboy REST POST and custom status code In-Reply-To: References: Message-ID: <85928944-6216-4b8c-a2d7-089bda791182@googlegroups.com> Thanks to Slava Yurin His advise: make_post(Req, State) -> ..., {ok, Reply} = cowboy_req:reply(201, [], <<"HELLO POST\n">>, Req), {halt, Reply, State}. On Tuesday, June 11, 2013 9:17:41 PM UTC+3, Max Grigoriev wrote: > > How can I set status code in my REST POST handler. > > Now I'm doing this > > make_post(Req, State) -> > lager:info("post"), > Resp = cowboy_req:set_resp_body(<<"HELLO POST\n">>, Req), > {ok, Resp3} = cowboy_req:reply(201, Resp), > {true, Resp3, State}. > > But I receive such errors: > > 1:43:10.510 [info] [nonode@REDACTED#pbshare_logic_registration#<0.205.0>] > [handle_info#35] :Exit Logic from <0.204.0> Reason: > {function_clause,[{cowboy_req,reply,[204,[],<<>>,{http_req,#Port<0.6355>,ranch_tcp,keepalive,<0.204.0>,<<"POST">>,'HTTP/1.1',{{127,0,0,1},50024},<<"localhost">>,undefined,8080,<<"/rest/registration/make">>,[<<"make">>],<<>>,undefined,[],[{<<"user-agent">>,<<"curl/7.29.0">>},{<<"host">>,<<"localhost:8080">>},{<<"accept">>,<<" > */* > ">>},{<<"content-type">>,<<"application/json">>},{<<"content-length">>,<<"14">>}],[{<<"content-length">>,14},{<<"expect">>,undefined},{<<"content-length">>,14},{<<"content-type">>,{<<"application">>,<<"json">>,[]}},{<<"if-modified-since">>,undefined},{<<"if-none-match">>,undefined},{<<"if-unmodified-since">>,undefined},{<<"if-match">>,undefined},{<<"accept">>,[{{<<" > *">>,<<"*">>,[]},1000,[]}]}],undefined,[{charset,undefined},{media_type,{<<"application">>,<<"json">>,[]}}],done,undefined,<<>>,false,done,[],<<>>,undefined}],[{file,"src/cowboy_req.erl"},{line,948}]},{cowboy_rest,respond,3,[{file,"src/cowboy_rest.erl"},{line,1085}]},{cowboy_rest,upgrade,4,[{file,"src/cowboy_rest.erl"},{line,75}]},{cowboy_protocol,execute,4,[{file,"src/cowboy_protocol.erl"},{line,523}]}]} > 21:43:10.510 [error] [Undefined#Undefined#emulator] [Undefined#Undefined] > :Error in process <0.204.0> with exit value: > {function_clause,[{cowboy_req,reply,[204,[],<<0 > bytes>>,{http_req,#Port<0.6355>,ranch_tcp,keepalive,<0.204.0>,<<4 > bytes>>,'HTTP/1.1',{{127,0,0,1},50024},<<9 bytes>>,undefined,8080,<<23 > bytes>>,[<<4 bytes>>],<<0 bytes>>,undefined,[],[{<<10 bytes>>,<<11 > bytes>>},{<<4 bytes>>,<<14 bytes>>},{<<6 bytes>>,<<3 bytes>>},{<<12 > bytes>>,<<16 bytes>>},{<<14 bytes>>,<<2 bytes>>}],[{<<14 bytes>>,14},{<<6 > bytes>>,undefined},{<<14 bytes>>,14},{<<12 bytes>>,{<<11 bytes>>,<<4 > bytes>>,[]}},{<<17 bytes>>,undefined},{<<13 bytes>>,undefined},{<<19 > bytes>>,undefined},{<<8 bytes>>,undefined... > -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Wed Jun 12 09:24:44 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 12 Jun 2013 08:24:44 +0100 Subject: [erlang-questions] why is mmap so darn difficult Message-ID: <57542809-682D-42F0-BBF0-91C67EBD5B62@gmail.com> This is part plea, part question. The short version is that what I really want is for OTP to provide file:mmap. Along with file:sendfile, this would allow me to rely heavily on the OS to deal with getting data onto the disk, paging in and out and delivering it to clients (i.e., sockets) with a minimum of copying and bringing data into user space. The long version is that I'm quite happy to roll my sleeves up and write a driver to handle mmap, but I'm unconvinced that there's a practical way to do this without having to copy (see, for example, emmap, which does indeed copy), and that kinda defeats the object. In fact, since NIF calls taking longer than a millisecond or so can get the schedulers in a tizzy, my instinct is that a driver would be a better choice, but my question is whether or not one can avoid copying - yes I know that you can pass a binary from erlang to a driver without copying, but that's not quite the same thing. And even if I did manage that, presumably file:sendfile would no longer be of much use. Any pointers here would be much appreciated, though proper mmap support would obviously be ideal. From a bit of quality time with google, it seems I'm not the only person frustrated by this. Cheers, Tim -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 235 bytes Desc: Message signed with OpenPGP using GPGMail URL: From max.lapshin@REDACTED Wed Jun 12 09:32:42 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 12 Jun 2013 11:32:42 +0400 Subject: [erlang-questions] why is mmap so darn difficult In-Reply-To: <57542809-682D-42F0-BBF0-91C67EBD5B62@gmail.com> References: <57542809-682D-42F0-BBF0-91C67EBD5B62@gmail.com> Message-ID: Tim, you are messing a bit about drivers and nifs. There is absolutely no problem in implementing mmap for read in erlang. I have done so in erlyvideo, but refused. Mmap is absolutely unpredictable. Access to memory can take several nanoseconds or several seconds. It is impossible to build a good service when you have such fluctuations. There is no problem in using file:pread and gen_tcp:send. At least, erlyvideo can send several gigabits of video without mmap. It would be impossible to achieve it with mmap, because you will not be able to make it stable. So, with mmap: you don't need it. It is a bad technology. From watson.timothy@REDACTED Wed Jun 12 09:59:42 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 12 Jun 2013 08:59:42 +0100 Subject: [erlang-questions] why is mmap so darn difficult In-Reply-To: References: <57542809-682D-42F0-BBF0-91C67EBD5B62@gmail.com> Message-ID: Hi Max! On 12 Jun 2013, at 08:32, Max Lapshin wrote: > Tim, you are messing a bit about drivers and nifs. > > There is absolutely no problem in implementing mmap for read in > erlang. I have done so in erlyvideo, but refused. It's both reads and writes that I want to handle. Think long (ish) term storage of data that gets written to sequentially, with multi reader who also read sequentially from a fixed point (so a seek to find the initial position, followed by more sequential reads). > > Mmap is absolutely unpredictable. Access to memory can take several > nanoseconds or several seconds. It is impossible to build a > good service when you have such fluctuations. > Interesting. That's not a situation I've come across before, but regardless, it is amortized costs that interest me on the whole. I can quite happily suffer the odd individual operation being slow, providing the overall inputs rates remain high. > There is no problem in using file:pread and gen_tcp:send. Sure, but it is fast writes that I care about. Do you think mmap is going to be problematic in that regard also? I certainly know of other uses of mmap (outside of Erlang) that claim writer rates of around 250Khz. I'm unconvinced I'll be able to replicate anything like that using basic file operations. Of course I could use an embedded database I suppose - that's something I've considered and have selected a couple of candidates - but then I loose the ability to use file:sendfile to push the data downstream later on. Perhaps this latter concern isn't a massive issue - since I've not benchmarked, I have no idea - but I need an embedded database with the minimum possible memory footprint, so I can't have something where all the keys need to fit in memory. The one that has come up as a choice has a NIF based driver too, which I'm totally unconvinced by, so I'll have to write my own driver anyway. Sigh... > At least, erlyvideo can send several gigabits of video without mmap. It would be > impossible to achieve it with mmap, because you will not be able to > make it stable. > Ok, so according to this, for readers file:pread + normal socket APIs can achieve a high enough throughput. Now the problem is solely with writes. Erlyvideo does not, presumably, read gigabits of video from a socket, put them on disk for later reading *and* deliver them in 'as close to realtime as possible' to another live reader (socket) with minimal delay? > So, with mmap: you don't need it. It is a bad technology. I'll definitely take that experience you've had with it into consideration. Thanks. Tim From max.lapshin@REDACTED Wed Jun 12 10:02:29 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 12 Jun 2013 12:02:29 +0400 Subject: [erlang-questions] why is mmap so darn difficult In-Reply-To: References: <57542809-682D-42F0-BBF0-91C67EBD5B62@gmail.com> Message-ID: Of course, mmap is not a fast write. Look at mongodb: it becomes absolutely unpredictable database, when you need to write data due to mmap. You never know how much time write will take. So, file:pwrite, file:pread are ok. Don't be afraid of them. From watson.timothy@REDACTED Wed Jun 12 10:35:33 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 12 Jun 2013 09:35:33 +0100 Subject: [erlang-questions] why is mmap so darn difficult In-Reply-To: References: <57542809-682D-42F0-BBF0-91C67EBD5B62@gmail.com> Message-ID: <1056D6F0-6E3D-4FBF-ABE9-E2420862F84C@gmail.com> On 12 Jun 2013, at 09:02, Max Lapshin wrote: > Of course, mmap is not a fast write. > I think the idea was not to flush from application code, but rely on the OS pagecache to gain some performance benefits at the cost of reliability (since without fsync, data loss is possible). > Look at mongodb: it becomes absolutely unpredictable database, when > you need to write data due to mmap. > > You never know how much time write will take. > Fascinating observation. But again, isn't that only a problem for individual writes? I'm not arguing for one thing or another here, just trying to understand the variables. If I want to write to write a stream to disk as fast as possible, isn't letting the OS deal with paging/caching/flushing likely to offer better overall throughput? > So, file:pwrite, file:pread are ok. Don't be afraid of them. We already have code that does regular reads/writes but the usage pattern is currently quite random, and throughput isn't in the 100s of Khz, though that's not entirely due to file I/O either. Part of the problem space I'm looking at is that the data store here might be terra-bytes large, so the assumption is/was that mmap would offer real benefits for that situation, as it appears to do for e.g., varnish, apache-kafka, etc. There are other possibilities of course, such as embedding http://symas.com/mdb/ - here is a product using mmap that claims phenomenally high throughput for concurrent readers and writers. Of course it's perhaps not so easy to take data out of that and send it to a client with sendfile, but perhaps *regular* sockets writes are fine. There is a driver for that database going around, but it's a NIF and therefore I'm not convinced it's usable due to the effect a long call can have on scheduling. Of course a linked in driver can share binaries with the emulator and process things out of bound using the async thread pool, then send them back to any process in the emulator (without copying) using erl_drv_send_term. Perhaps that's a better choice than trying to go low-level and use mmap directly, but it does raise the question why that software is able to get such good "headline rates" using mmap, when others (such as mongodb, cited) do not. Cheers, Tim From max.lapshin@REDACTED Wed Jun 12 10:42:03 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 12 Jun 2013 12:42:03 +0400 Subject: [erlang-questions] why is mmap so darn difficult In-Reply-To: <1056D6F0-6E3D-4FBF-ABE9-E2420862F84C@gmail.com> References: <57542809-682D-42F0-BBF0-91C67EBD5B62@gmail.com> <1056D6F0-6E3D-4FBF-ABE9-E2420862F84C@gmail.com> Message-ID: I really don't know when mmap can be a good choice. I thought that video streaming, when you need to read about half of gigabyte per second from disk, repack it and write to network, is the good place for mmap. It was a mistake. If you get slow mmap disk read, than whole erlang sticks and you get PHP-like performance when whole thread is blocked for one client. Perhaps in embedded mmap would be better when you want to save memory and have 10 requests per day, but on high load mmap doesn't help. From watson.timothy@REDACTED Wed Jun 12 11:41:23 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 12 Jun 2013 10:41:23 +0100 Subject: [erlang-questions] why is mmap so darn difficult In-Reply-To: References: <57542809-682D-42F0-BBF0-91C67EBD5B62@gmail.com> <1056D6F0-6E3D-4FBF-ABE9-E2420862F84C@gmail.com> Message-ID: <7BC1729C-4602-4F6F-B96E-FCE0402DCCC5@gmail.com> Max, On 12 Jun 2013, at 09:42, Max Lapshin wrote: > I thought that video streaming, when you need to read about half of > gigabyte per second from disk, repack it and write to network, is the > good place for mmap. > It was a mistake. If you get slow mmap disk read, than whole erlang > sticks and you get PHP-like performance when whole thread is blocked > for one client. > Isn't that because the pread call you used was in a NIF, and therefore could block the scheulder(s) no? My thought's about the approach were initially somewhat different. The write operations were just file:write, whereas there would be no pread, but rather data would be delivered to the socket using sendfile/5, which would utilise the async thread pool if it's enabled. > Perhaps in embedded mmap would be better when you want to save memory > and have 10 requests per day, but on high load mmap doesn't help. I'd like to handle writing hundreds of thousands of input events per second and reading at various different speeds, but supporting (reads of) up to tens of thousands per second. -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Wed Jun 12 11:43:22 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 12 Jun 2013 13:43:22 +0400 Subject: [erlang-questions] why is mmap so darn difficult In-Reply-To: <7BC1729C-4602-4F6F-B96E-FCE0402DCCC5@gmail.com> References: <57542809-682D-42F0-BBF0-91C67EBD5B62@gmail.com> <1056D6F0-6E3D-4FBF-ABE9-E2420862F84C@gmail.com> <7BC1729C-4602-4F6F-B96E-FCE0402DCCC5@gmail.com> Message-ID: Of course, you will not be able to handle such load with mmap, because you will suffer from blocking whole VM on mmap page faults. Add more async threads and profile application. Don't waste your time on mmap. From Christopher.Phillips@REDACTED Wed Jun 12 15:51:41 2013 From: Christopher.Phillips@REDACTED (Phillips, Christopher) Date: Wed, 12 Jun 2013 13:51:41 +0000 Subject: [erlang-questions] erlang-questions Digest, Vol 117, Issue 3 In-Reply-To: Message-ID: I'm not looking too closely at the errors, so this may not be all you need, but, all of the REST callbacks can return a tuple of {halt, Response, State}, which is the correct idiom to use when you want to stop the REST handler from continuing (so in your code you'd return {halt, Resp3, State}). That said, you may want to look at the REST handler's flow logic; there is a path that will automatically return a 201 if you opt to use it (I don't recall it offhand, and it may be that migrating to use it is more effort than just halting with a 201). If the former doesn't address everything, and the latter isn't feasible (or you have questions about it), consider signing up for and asking on the 99s mailing list, extend@REDACTED On 6/12/13 4:35 AM, "erlang-questions-request@REDACTED" wrote: >How can I set status code in my REST POST handler. > >Now I'm doing this > >make_post(Req, State) -> > lager:info("post"), > Resp = cowboy_req:set_resp_body(<<"HELLO POST\n">>, Req), > {ok, Resp3} = cowboy_req:reply(201, Resp), > {true, Resp3, State}. > >But I receive such errors: > >1:43:10.510 [info] [nonode@REDACTED#pbshare_logic_registration#<0.205.0>] >[handle_info#35] :Exit Logic from <0.204.0> Reason: >{function_clause,[{cowboy_req,reply,[204,[],<<>>,{http_req,#Port<0.6355>,r >anch_tcp,keepalive,<0.204.0>,<<"POST">>,'HTTP/1.1',{{127,0,0,1},50024},<<" >localhost">>,undefined,8080,<<"/rest/registration/make">>,[<<"make">>],<<> >>,undefined,[],[{<<"user-agent">>,<<"curl/7.29.0">>},{<<"host">>,<<"localh >>ost:8080">>},{<<"accept">>,<<" >*/* >">>},{<<"content-type">>,<<"application/json">>},{<<"content-length">>,<<" >14">>}],[{<<"content-length">>,14},{<<"expect">>,undefined},{<<"content-le >ngth">>,14},{<<"content-type">>,{<<"application">>,<<"json">>,[]}},{<<"if- >modified-since">>,undefined},{<<"if-none-match">>,undefined},{<<"if-unmodi >fied-since">>,undefined},{<<"if-match">>,undefined},{<<"accept">>,[{{<<" >*">>,<<"*">>,[]},1000,[]}]}],undefined,[{charset,undefined},{media_type,{< ><"application">>,<<"json">>,[]}}],done,undefined,<<>>,false,done,[],<<>>,u >ndefined}],[{file,"src/cowboy_req.erl"},{line,948}]},{cowboy_rest,respond, >3,[{file,"src/cowboy_rest.erl"},{line,1085}]},{cowboy_rest,upgrade,4,[{fil >e,"src/cowboy_rest.erl"},{line,75}]},{cowboy_protocol,execute,4,[{file,"sr >c/cowboy_protocol.erl"},{line,523}]}]} >21:43:10.510 [error] [Undefined#Undefined#emulator] [Undefined#Undefined] >:Error in process <0.204.0> with exit value: >{function_clause,[{cowboy_req,reply,[204,[],<<0 >bytes>>,{http_req,#Port<0.6355>,ranch_tcp,keepalive,<0.204.0>,<<4 >bytes>>,'HTTP/1.1',{{127,0,0,1},50024},<<9 bytes>>,undefined,8080,<<23 >bytes>>,[<<4 bytes>>],<<0 bytes>>,undefined,[],[{<<10 bytes>>,<<11 >bytes>>},{<<4 bytes>>,<<14 bytes>>},{<<6 bytes>>,<<3 bytes>>},{<<12 >bytes>>,<<16 bytes>>},{<<14 bytes>>,<<2 bytes>>}],[{<<14 bytes>>,14},{<<6 >bytes>>,undefined},{<<14 bytes>>,14},{<<12 bytes>>,{<<11 bytes>>,<<4 >bytes>>,[]}},{<<17 bytes>>,undefined},{<<13 bytes>>,undefined},{<<19 >bytes>>,undefined},{<<8 bytes>>,undefined... From Christopher.Phillips@REDACTED Wed Jun 12 15:53:44 2013 From: Christopher.Phillips@REDACTED (Phillips, Christopher) Date: Wed, 12 Jun 2013 13:53:44 +0000 Subject: [erlang-questions] Cowboy REST POST and custom status code Message-ID: I'm not looking too closely at the errors, so this may not be all you need, but, all of the REST callbacks can return a tuple of {halt, Response, State}, which is the correct idiom to use when you want to stop the REST handler from continuing (so in your code you'd return {halt, Resp3, State}). That said, you may want to look at the REST handler's flow logic; there is a path that will automatically return a 201 if you opt to use it (I don't recall it offhand, and it may be that migrating to use it is more effort than just halting with a 201). If the former doesn't address everything, and the latter isn't feasible (or you have questions about it), consider signing up for and asking on the 99s mailing list, extend@REDACTED. On 6/12/13 4:35 AM, "erlang-questions-request@REDACTED" > wrote: How can I set status code in my REST POST handler. Now I'm doing this make_post(Req, State) -> lager:info("post"), Resp = cowboy_req:set_resp_body(<<"HELLO POST\n">>, Req), {ok, Resp3} = cowboy_req:reply(201, Resp), {true, Resp3, State}. But I receive such errors: 1:43:10.510 [info] [nonode@REDACTED#pbshare_logic_registration#<0.205.0>] [handle_info#35] :Exit Logic from <0.204.0> Reason: {function_clause,[{cowboy_req,reply,[204,[],<<>>,{http_req,#Port<0.6355>,r anch_tcp,keepalive,<0.204.0>,<<"POST">>,'HTTP/1.1',{{127,0,0,1},50024},<<" localhost">>,undefined,8080,<<"/rest/registration/make">>,[<<"make">>],<<> ,undefined,[],[{<<"user-agent">>,<<"curl/7.29.0">>},{<<"host">>,<<"localh ost:8080">>},{<<"accept">>,<<" */* ">>},{<<"content-type">>,<<"application/json">>},{<<"content-length">>,<<" 14">>}],[{<<"content-length">>,14},{<<"expect">>,undefined},{<<"content-le ngth">>,14},{<<"content-type">>,{<<"application">>,<<"json">>,[]}},{<<"if- modified-since">>,undefined},{<<"if-none-match">>,undefined},{<<"if-unmodi fied-since">>,undefined},{<<"if-match">>,undefined},{<<"accept">>,[{{<<" *">>,<<"*">>,[]},1000,[]}]}],undefined,[{charset,undefined},{media_type,{< <"application">>,<<"json">>,[]}}],done,undefined,<<>>,false,done,[],<<>>,u ndefined}],[{file,"src/cowboy_req.erl"},{line,948}]},{cowboy_rest,respond, 3,[{file,"src/cowboy_rest.erl"},{line,1085}]},{cowboy_rest,upgrade,4,[{fil e,"src/cowboy_rest.erl"},{line,75}]},{cowboy_protocol,execute,4,[{file,"sr c/cowboy_protocol.erl"},{line,523}]}]} 21:43:10.510 [error] [Undefined#Undefined#emulator] [Undefined#Undefined] :Error in process <0.204.0> with exit value: {function_clause,[{cowboy_req,reply,[204,[],<<0 bytes>>,{http_req,#Port<0.6355>,ranch_tcp,keepalive,<0.204.0>,<<4 bytes>>,'HTTP/1.1',{{127,0,0,1},50024},<<9 bytes>>,undefined,8080,<<23 bytes>>,[<<4 bytes>>],<<0 bytes>>,undefined,[],[{<<10 bytes>>,<<11 bytes>>},{<<4 bytes>>,<<14 bytes>>},{<<6 bytes>>,<<3 bytes>>},{<<12 bytes>>,<<16 bytes>>},{<<14 bytes>>,<<2 bytes>>}],[{<<14 bytes>>,14},{<<6 bytes>>,undefined},{<<14 bytes>>,14},{<<12 bytes>>,{<<11 bytes>>,<<4 bytes>>,[]}},{<<17 bytes>>,undefined},{<<13 bytes>>,undefined},{<<19 bytes>>,undefined},{<<8 bytes>>,undefined... -------------- next part -------------- An HTML attachment was scrubbed... URL: From egil@REDACTED Wed Jun 12 16:07:27 2013 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Wed, 12 Jun 2013 16:07:27 +0200 Subject: [erlang-questions] R16B01 Release Candidate 1 on Github In-Reply-To: References: <51B74496.50102@erlang.org> Message-ID: <51B8809F.6070902@erlang.org> On 2013-06-11 21:25, Aaron France wrote: > Hi, > > Is there a change list anywhere? I've attached a listing of current items in maint-branch. // Bj?rn-Egil > > > On Tue, Jun 11, 2013 at 5:39 PM, Bj?rn-Egil Dahlberg > wrote: > > Hi, > > Things are dropping in place and getting stable so we have pushed > a R16B01 release candidate to GitHub. > > https://github.com/erlang/otp/tree/OTP_R16B01_RC1 > > git fetch git://github.com/erlang/otp.git > OTP_R16B01_RC1 > > Some minor things are still not ready yet, but we are getting there. > > Take it for a spin and see if we have missed something critical. > > // Bj?rn-Egil > Erlang/OTP > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- Patch Id: erl_%ID%_otp_beam_%PLATFORM% Label: Complete release Date: 2013-06-12 Trouble Report Id: OTP-9610, OTP-10085, OTP-10126, OTP-10271, OTP-10279, OTP-10450, OTP-10494, OTP-10497, OTP-10604, OTP-10607, OTP-10648, OTP-10689, OTP-10855, OTP-10856, OTP-10857, OTP-10873, OTP-10876, OTP-10893, OTP-10894, OTP-10899, OTP-10901, OTP-10902, OTP-10903, OTP-10904, OTP-10905, OTP-10909, OTP-10910, OTP-10912, OTP-10914, OTP-10918, OTP-10920, OTP-10929, OTP-10930, OTP-10931, OTP-10934, OTP-10938, OTP-10939, OTP-10941, OTP-10945, OTP-10947, OTP-10948, OTP-10949, OTP-10951, OTP-10955, OTP-10956, OTP-10972, OTP-10975, OTP-10979, OTP-10980, OTP-10983, OTP-10984, OTP-10985, OTP-10986, OTP-10988, OTP-10990, OTP-10991, OTP-10992, OTP-10993, OTP-10996, OTP-10997, OTP-10999, OTP-11002, OTP-11003, OTP-11004, OTP-11005, OTP-11007, OTP-11008, OTP-11009, OTP-11012, OTP-11013, OTP-11014, OTP-11016, OTP-11017, OTP-11019, OTP-11024, OTP-11025, OTP-11026, OTP-11027, OTP-11028, OTP-11030, OTP-11031, OTP-11035, OTP-11039, OTP-11040, OTP-11041, OTP-11042, OTP-11044, OTP-11045, OTP-11046, OTP-11050, OTP-11051, OTP-11052, OTP-11053, OTP-11057, OTP-11058, OTP-11060, OTP-11064, OTP-11069, OTP-11071, OTP-11074, OTP-11075, OTP-11076, OTP-11077, OTP-11078, OTP-11079, OTP-11080, OTP-11084, OTP-11085, OTP-11086, OTP-11087, OTP-11088, OTP-11092, OTP-11095, OTP-11096, OTP-11097, OTP-11098, OTP-11099, OTP-11100, OTP-11101, OTP-11103, OTP-11104, OTP-11106, OTP-11107, OTP-11108, OTP-11111, OTP-11115, OTP-11117, OTP-11119, OTP-11120, OTP-11122, OTP-11125, OTP-11126, OTP-11127, OTP-11129, OTP-11130, OTP-11134, OTP-11136, OTP-11137, OTP-11138, OTP-11140, OTP-11142, OTP-11143, OTP-11144, OTP-11145, OTP-11146, OTP-11153, OTP-11154, OTP-11158, OTP-11159, OTP-11163, OTP-11164 Seq num: OTP-10336, OTP-10336, OTP-10562, OTP-10692, OTP-10832, OTP-10907, OTP-11007, OTP-11101, seq12212, seq12227, seq12267, seq12304, seq12313, seq12339, seq12342 System: otp Release: %RELEASE% Operating System: %OS% Architecture: %ARCHITECTURE% Erlang machine: BEAM Application: asn1-2.0.2, common_test-1.7.2, compiler-4.9.2, cosEvent-2.1.14, cosFileTransfer-1.1.15, cosNotification-1.1.20, crypto-3.0, debugger-3.2.11, dialyzer-2.6.1, diameter-1.4.2, erl_interface-3.7.13, erts-5.10.2, et-1.4.4.4, hipe-3.10.2, ic-4.3.2, inets-5.9.5, kernel-2.16.2, megaco-3.17.0.1, mnesia-4.9, observer-1.3.1, odbc-2.10.16, orber-3.6.26.1, os_mon-2.2.12, percept-0.8.8.1, public_key-0.19, reltool-0.6.4, runtime_tools-1.8.11, sasl-2.3.2, snmp-4.24, ssh-2.1.7, ssl-5.3, stdlib-1.19.2, test_server-3.6.2, tools-2.6.11 Files: all Description: --- REMAINING ISSUES ---------------------------------------------------- OTP-10085 The ticket with label 'ct_telnet:expect needs new option that ignores newlines' has status 'cancelled'. [otp_team_mw] OTP-10689 The ticket with label 'inet:gethostbyname to lowercase in case of searching inet_db' has status 'open'. [otp_team_ps] OTP-11142 The ticket with label 'Interruptible crypto functions' has status 'open'. [otp_team_ps] OTP-11164 The ticket with label 'Add the +sfwi system flag' has status 'open'. [rickard] erl_interface Scheduled for release, but not mentioned in any release note ic Scheduled for release, but not mentioned in any release note jinterface (Test suite and/or vsn.mk modified) sasl Version should be 2.3.2, but is 2.4.1 wx Modified, but version number missing in upcoming-versions -> 61613ac (Dan Gudmundsson) - Merge branch 'dgud/testcase_fixes' into maint -> c1fdb08 (YAMAMOTO Takashi) - reduce bashism -> 35fb992 (Dan Gudmundsson) - wx: Fix failing testcases --- HIGHLIGHTS ---------------------------------------------------------- OTP-10279 == erts == Support for migration of memory carriers between memory allocator instances has been introduced. For more information see the documentation of the +Macul command line argument. When enabling this feature on the fix_alloc allocator, a different strategy for management of fix blocks will be used. The information returned from erlang:system_info({allocator, A}), and erlang:system_info({allocator_sizes, A}) will be slightly different when this feature has been enabled. An mbcs_pool tuple will be present giving information about abandoned carriers, and in the fix_alloc case no fix_types tuple will be present. OTP-11009 == ssl public_key crypto common_test dialyzer ssh stdlib snmp inets == Integrate elliptic curve contribution from Andreas Schultz In order to be able to support elliptic curve cipher suites in SSL/TLS, additions to handle elliptic curve infrastructure has been added to public_key and crypto. This also has resulted in a rewrite of the crypto API to gain consistency and remove unnecessary overhead. All OTP applications using crypto has been updated to use the new API. Impact: Elliptic curve cryptography (ECC) offers equivalent security with smaller key sizes than other public key algorithms. Smaller key sizes result in savings for power, memory, bandwidth, and computational cost that make ECC especially attractive for constrained environments. OTP-11159 == erts == Lift static limitation (FD_SETSIZE) for file descriptors on Mac OS X. (Thanks to Anthony Ramine) --- POTENTIAL INCOMPATIBILITIES ----------------------------------------- OTP-11077 == erts == A new algorithm for management of the process, and port tables has been introduced. The new algorithm ensures that both insert and delete operations can be made in O(1) time complexity. Previously used algorithm either caused insert or delete to be O(N). The new algorithm will also ensure that reuse of identifiers will be less frequent than when the old algorithm was used. Previously used algorithm ensured that the latest created identifier compared as the largest when comparing two identifiers of the same type that had been created on the same node as long as no identifiers had been reused. Since identifiers can be reused quite fast, one has never been able to rely on this property. Due to the introduction of this new algorithm this property will not hold even if no identifiers has been reused yet. This could be considered as an incompatibility. Due to this potential incompatibility, it will still be possible to enable the old algorithm for some time. The command line argument +P legacy will enable the old algorithm on the process table, and +Q legacy will do the same for the port table. These command line arguments are however deprecated as of their introduction and have been scheduled for removal in OTP-R18. --- asn1 ---------------------------------------------------------------- OTP-11016 Postscript files no longer needed for the generation of PDF files have been removed. OTP-11111 Fix some Makefile rules that didn't support silent rules. Thanks to Anthony Ramine. OTP-11134 PER/UPER: A semi-constrained INTEGER with a non-zero lower bound would be incorrectly decoded. This bug was introduced in R16. PER/UPER: Given INTEGER (10..MAX, ...), attempting to decode any integer below 10 would cause the encoder to enter an infinite loop. PER/UPER: For a type with an extensible SIZE constraint, sizes outside of the root range were incorrectly encoded. Given a constraint such as (SIZE (5, ...)), encoding a size less than 5 would fail (PER/UPER). Similarly, for BER decoding would fail. PER: The encoder did not align a known multiplier string (such as IA5String) of length 16 bits (exactly) to an octet boundary. In rare circumstances, DEFAULT values for the UPER backend could be wrongly encoded. OTP-11153 UPER: The compiler would crash when compiling an ENUMERATED having more than 63 extended values. PER/UPER: A SEQUENCE with more 64 extended values could not be decoded. OTP-11154 When decoding a SEQUENCE defined inline inside a an extension addition group, the record named generated by the decoding code would not match the name in the generated .hrl file. --- common_test --------------------------------------------------------- OTP-10126 A design flaw in the generic connection handling in Common Test made it impossible to implement a connection handler that could map multiple connection names (i.e. configuration variable aliases) to single connection pids. This problem has been solved. OTP-10494 If it could not be decided which test case a certain log printout belonged to, the common test framework log was earlier used. Such printouts are now instead sent to unexpected_io.log.html in test_server so that there is only one place to look for "missing" printouts. OTP-10648 If a telnet connection is hanging, then a call to ct_telnet:close/1 will time out after 5 seconds and the connection process is brutally killed. In some cases the connection would not be unregistered and attempts at opening a new connection with the same name would make common_test try to reuse the same connection since it believed that it was still alive. This has been corrected - a killed connection is now always unregistered. OTP-10855 Test performance has been improved by means of a cache for the top level HTML index logs (all_runs.html and index.html, in the logdir directory). This solves problems with slow start up times and test execution times increasing with the number of ct_run directories stored in logdir. The cached index entries are stored in RAM during test execution and are saved to file in logdir (for faster start up times) whenever a test run finishes. OTP-10856 The '-force_stop' flag to use with time-limited repeats of test runs can now be used with a new 'skip_rest' option which causes the rest of the test cases in the ongoing test job to be skipped when the time limit is reached. E.g. 'ct_run -spec xxx -duration 010000 -force_stop skip_rest' OTP-10857 Testing of the test specification functionality has been improved and a couple of minor bugs have been discovered and corrected. OTP-10902 Make cover smarter about finding source from beam. In particular, search using the source path in module_info if the current heuristic fails. OTP-10920 Add a variant of ct_slave:start/2 that starts a node with specified options on the local host. OTP-11009 Integrate elliptic curve contribution from Andreas Schultz In order to be able to support elliptic curve cipher suites in SSL/TLS, additions to handle elliptic curve infrastructure has been added to public_key and crypto. This also has resulted in a rewrite of the crypto API to gain consistency and remove unnecessary overhead. All OTP applications using crypto has been updated to use the new API. Impact: Elliptic curve cryptography (ECC) offers equivalent security with smaller key sizes than other public key algorithms. Smaller key sizes result in savings for power, memory, bandwidth, and computational cost that make ECC especially attractive for constrained environments. OTP-11016 Postscript files no longer needed for the generation of PDF files have been removed. OTP-11044 A link is added from the red error printout in a test case log (for a failed test case) to the full error description at the end of the log. The reason for this is that the error description in the red field is sometimes truncated at 50 characters in order to keep the log as short and easy to read as possible. OTP-11046 Links to the top level index files in some HTML footers had disappeared. This error has been corrected. Also, a problem with the suite overview log file not being closed properly has been solved. OTP-11052 Common Test would, in case of timetrap error, print a warning in the log if end_per_testcase wasn't implemented in the suite, even though it's an optional function. This printout has been removed. OTP-11095 A new option 'no_prompt_check' is added to ct_telnet:expect/3. If this option is used, ct_telnet will not wait for a prompt or a newline before attempting to match the given pattern. --- compiler ------------------------------------------------------------ OTP-10939 Compiling functions with complex boolean operations in guards could be very slow. (Thanks to Magnus Muller for reporting this issue.) OTP-11005 Fix optimization of some binary comprehensions. Thanks to Anthony Ramine. OTP-11069 Use a set to store ref registers in beam_receive. Thanks to Anthony Ramine. OTP-11119 Certain guard expressions used in a receive statement could cause the compiler to crash. OTP-11129 Fix renaming of bs_put_string instructions. Thanks to Anthony Ramine. --- cosEvent ------------------------------------------------------------ OTP-11016 Postscript files no longer needed for the generation of PDF files have been removed. --- cosFileTransfer ----------------------------------------------------- OTP-11016 Postscript files no longer needed for the generation of PDF files have been removed. --- cosNotification ----------------------------------------------------- OTP-11016 Postscript files no longer needed for the generation of PDF files have been removed. --- crypto -------------------------------------------------------------- OTP-11009 Integrate elliptic curve contribution from Andreas Schultz In order to be able to support elliptic curve cipher suites in SSL/TLS, additions to handle elliptic curve infrastructure has been added to public_key and crypto. This also has resulted in a rewrite of the crypto API to gain consistency and remove unnecessary overhead. All OTP applications using crypto has been updated to use the new API. Impact: Elliptic curve cryptography (ECC) offers equivalent security with smaller key sizes than other public key algorithms. Smaller key sizes result in savings for power, memory, bandwidth, and computational cost that make ECC especially attractive for constrained environments. OTP-11058 Fixed a spelling mistake in crypto docs. Thanks to Klaus Trainer --- debugger ------------------------------------------------------------ OTP-10899 A new checkbox has been added. When it is checked, the range set by the erl flag +pc is used for determining when to print lists of integers as strings. When it is unchecked, integer lists are never printed as strings. A minor incompatibility: settings saved by Erlang R16B01 or later cannot be read by Erlang R16B or earlier. OTP-11041 Erlang source files with non-ASCII characters are now encoded in UTF-8 (instead of latin1). --- dialyzer ------------------------------------------------------------ OTP-10918 Include module, function and arity in Dialyzer's "overlapping domain" warnings. Thanks to Magnus Henoch. OTP-10996 Improve Dialyzer output for scan errors. Thanks to Magnus Henoch. OTP-11009 Integrate elliptic curve contribution from Andreas Schultz In order to be able to support elliptic curve cipher suites in SSL/TLS, additions to handle elliptic curve infrastructure has been added to public_key and crypto. This also has resulted in a rewrite of the crypto API to gain consistency and remove unnecessary overhead. All OTP applications using crypto has been updated to use the new API. Impact: Elliptic curve cryptography (ECC) offers equivalent security with smaller key sizes than other public key algorithms. Smaller key sizes result in savings for power, memory, bandwidth, and computational cost that make ECC especially attractive for constrained environments. OTP-11027 Bitstring type inference and duplicate module error message fixes. Thanks to Stavros Aronis. OTP-11041 Erlang source files with non-ASCII characters are now encoded in UTF-8 (instead of latin1). OTP-11057 A bug that made it impossible to do any analyses from the GUI has been fixed. --- diameter ------------------------------------------------------------ OTP-9610 Allow peer connections to be shared between Erlang nodes for the purpose of sending outgoing requests. A diameter_app(3) pick_peer/4 callback gets a list of remote candidates as argument, allowing a callback on one node to select a transport connection established on another node. The service_opt() share_peers controls the extent to which local connections are shared with remote nodes. The service_opt() use_shared_peers controls the extent to which connections shared from remote nodes are utilized on the local node. OTP-10893 Allow listening diameter_{tcp,sctp} transports to be configured with remote addresses. Option 'accept' allows remote addresses to be configured as tuples or regular expressions. Remote addresses are matched against the configured values at connection establishment, any non-matching address causing the connection to be aborted. OTP-10972 Detect more transport_opt() configuration errors at diameter:add_transport/2. Many errors would previously not be detected until transport start, diameter:add_transport/2 returning 'ok' but transport connections failing to be established. An error tuple is now returned. OTP-10986 Make explicit local address configuration optional in diameter_tcp:start/3. The default address (as determined by gen_tcp) is now used when a local address is not explicitly configured. OTP-11007 Fix handling of 5014 (INVALID_AVP_LENGTH) errors. This was in some cases reported as 3009 (INVALID_AVP_BITS). Note that the correction is partially implemented in modules generated by diameterc(1): a dictionary file must be recompiled for the correction to apply to any messages it defines. OTP-11014 Fix faulty capitalization in release notes. Diameter = the protocol.
diameter = the Erlang application. OTP-11017 Improve handling of unrecognized service options. Such options were silently ignored by diameter:start_service/2. An error tuple is now returned. OTP-11019 Fix watchdog memory leak. Entries were not removed from a service-specific ets table, causing them to be orphaned at connection reestablishment for listening transports, and diameter:remove_transport/2 for both listening and connecting transports. The fault was introduced by OTP-10692 in diameter-1.4.1 (R16B). OTP-11026 Fix decode failure on AVP Length < 8. The failure caused the message in question to be discarded. OTP-11045 Respect Host-IP-Address configuration. Addresses returned from a transport module were always used to populate Host-IP-Address AVP's in an outgoing CER/CEA, which precluded the sending of a VIP address. Transport addresses are now only used if Host-IP-Address is unspecified. OTP-11050 Don't send default Inband-Security-Id in CER/CEA. RFC 6733 recommends against the use of Inband-Security-Id. Only send a value that differs from the default, NO_INBAND_SECURITY = 0. OTP-11051 Fix mkdir race. Install could fail if examples/code and examples/dict were created in parallel. Noticed on FreeBSD. OTP-11060 Make spawn options for request processes configurable. OTP-11087 Fix recognition of 5001 on mandatory AVP's. An AVP setting the M-bit was not regarded as erroneous if it was defined in the dictionary in question and its container (message or Grouped AVP) had an 'AVP' field. It's now regarded as a 5001 error (AVP_UNSUPPORTED), as in the case that the AVP is not defined. Note that the correction is partially implemented in modules generated by diameterc(1): a dictionary file must be recompiled for the correction to apply to any messages it defines. OTP-11092 Fix setting of Failed-AVP on handle_request {answer_message, 5xxx} return. Failed-AVP was never in the outgoing answer-message. It is now set with the AVP from the first entry with the specified Result-Code in the errors field of the incoming diameter_packet, if found. OTP-11115 Fix watchdog function_clause A listening transport on a service that allowed multiple connections to the same peer could result in a function_clause error in module diameter_watchdog. The resulting crash was harmless but unseemly. Thanks to Aleksander Nycz. OTP-11127 Fix population of Failed-AVP. In cases in which diameter populated this AVP, many values were sent instead of one as suggested by RFC 6733. This was partially corrected by OTP-11007. --- erts ---------------------------------------------------------------- OTP-10271 Replaced the lock protecting gathering of garbage collection statistics with a lock-free solution. OTP-10279 Support for migration of memory carriers between memory allocator instances has been introduced. For more information see the documentation of the +Macul command line argument. When enabling this feature on the fix_alloc allocator, a different strategy for management of fix blocks will be used. The information returned from erlang:system_info({allocator, A}), and erlang:system_info({allocator_sizes, A}) will be slightly different when this feature has been enabled. An mbcs_pool tuple will be present giving information about abandoned carriers, and in the fix_alloc case no fix_types tuple will be present. OTP-10497 A bug in prim_inet has been corrected. If the port owner was killed at a bad time while closing the socket port the port could become orphaned hence causing port and socket leaking. Reported by Fred Herbert, Dmitry Belyaev and others. OTP-10941 Compilation fixes for NetBSD. Thanks to YAMAMOTO Takashi. OTP-10984 Fixed a race condition when using delayed_write when writing to a file which would cause the same data to be written multiple times. OTP-10997 Fix small memory leak from tracing with option meta. OTP-11002 Correct typo in erlsrv usage. Thanks to Bryan Hunter OTP-11003 ct_run: delete unused function. Thanks to Tuncer Ayaz. OTP-11008 Change specs for spawn_opt to use the process_level() type declaration instead of re-defining it in various places. Thanks to Kostis Sagonas. OTP-11012 Corrections to run_erl/to_erl handshake behaviour. OTP-11016 Postscript files no longer needed for the generation of PDF files have been removed. OTP-11024 Fix typo in type: erlang:process_info_item(). Thanks to Andrew Tunnell-Jones. OTP-11041 Erlang source files with non-ASCII characters are now encoded in UTF-8 (instead of latin1). OTP-11064 Fix src/dest overlap issue in ttsl driver. Thanks to Steve Vinovski. OTP-11074 Optimization of simultaneous inet_db operations on the same socket by using a lock free implementation. OTP-11075 The high_msgq_watermark and low_msgq_watermark inet socket options introduced in OTP-R16A could only be set on TCP sockets. These options are now generic and can be set on all types of sockets. OTP-11076 When sending to a port using erlang:send(Port, Msg, [nosuspend]), the send operation was performed synchronously. This bug has now been fixed. OTP-11077 A new algorithm for management of the process, and port tables has been introduced. The new algorithm ensures that both insert and delete operations can be made in O(1) time complexity. Previously used algorithm either caused insert or delete to be O(N). The new algorithm will also ensure that reuse of identifiers will be less frequent than when the old algorithm was used. Previously used algorithm ensured that the latest created identifier compared as the largest when comparing two identifiers of the same type that had been created on the same node as long as no identifiers had been reused. Since identifiers can be reused quite fast, one has never been able to rely on this property. Due to the introduction of this new algorithm this property will not hold even if no identifiers has been reused yet. This could be considered as an incompatibility. Due to this potential incompatibility, it will still be possible to enable the old algorithm for some time. The command line argument +P legacy will enable the old algorithm on the process table, and +Q legacy will do the same for the port table. These command line arguments are however deprecated as of their introduction and have been scheduled for removal in OTP-R18. OTP-11080 When converting a faulty binary to a list with unicode:characters_to_list, the error return value could contain a faulty "rest", i.e. the io_list of characters that could not be converted was wrong. This happened only if input was a sub binary and conversion was from utf8. This is now corrected. OTP-11084 Runtime system could crash when reporting stale driver_select(). OTP-11085 Fix lock order violation for memory instrumentation (+Mim, +Mis, +Mit). OTP-11086 Fixed some compilation warnings on miscellaneous platforms. Thanks to Anthony Ramine. OTP-11088 Support wide characters in the shell through wcwidth(). Thanks to Anthony Ramine. Reported by Lo?c Hoguin. OTP-11096 Fixed issue when flushing i/o during shutdown on windows where the Erlang VM would sometime hang due to a race condition. OTP-11097 Fixed issue where repeated calls to erlang:nodes() could cause unnecessary contention in the dist_table lock. OTP-11098 Added total used memory for each process in erlang crash dumps. OTP-11106 Properly guard WIDE_TAG use with HAVE_WCWIDTH in ttsl_drv. Thanks to Anthony Ramine OTP-11111 Fix some Makefile rules that didn't support silent rules. Thanks to Anthony Ramine. OTP-11125 Added support for hipe on Raspberry Pi (armv6l). Thanks to Klaus Alfert. OTP-11137 Fix receive support in erl_eval with a BEAM module. Thanks to Anthony Ramine. OTP-11146 erlang:now() could suddenly jump ~24 days into the future on Windows. This is now corrected. Thanks to Garret Smith for reporting and testing fixes. OTP-11158 Remove 'query' from the list of reserved words in docs. Thanks to Matthias Endler and Lo?c Hoguin. OTP-11159 Lift static limitation (FD_SETSIZE) for file descriptors on Mac OS X. (Thanks to Anthony Ramine) OTP-11163 erlang:term_to_binary will now cost an appropriate amount of reductions and will interrupt (yield) for reschedule if the term is big. This avoids too long schedules when term_to_binary is used. Impact: Programs running term_to_binary on large terms will run more smothly, but rescheduling will impact the single process performance of the BIF. Single threaded benchmarks will show degraded performance of the BIF when called with very large terms, while general system behaviour will be improved. The overhead for allowing restart and reduction counting also degrades local performance of the BIF with between 5% and 10% even for small terms. --- et ------------------------------------------------------------------ OTP-11039 Use erlang:demonitor(Ref, [flush]) where applicable. Thanks to Lo?c Hoguin. OTP-11078 Rename and document lists:zf/2 as lists:filtermap/2. Thanks to Anthony Ramine. --- hipe ---------------------------------------------------------------- OTP-10904 Fix the title of hipe_app documentation page. Thanks to Lo?c Hoguin. OTP-10985 Fix native code compiler crash involving bs_match_string. Thanks to Kostis Sagonas. OTP-11031 Loosen the assumptions of code that handles escaping functions. Thanks to Kostis Sagonas --- inets --------------------------------------------------------------- OTP-10934 Fix http_request:http_headers/1 to send content-length when length is zero. Thanks to CA Meijer. OTP-11009 Integrate elliptic curve contribution from Andreas Schultz In order to be able to support elliptic curve cipher suites in SSL/TLS, additions to handle elliptic curve infrastructure has been added to public_key and crypto. This also has resulted in a rewrite of the crypto API to gain consistency and remove unnecessary overhead. All OTP applications using crypto has been updated to use the new API. Impact: Elliptic curve cryptography (ECC) offers equivalent security with smaller key sizes than other public key algorithms. Smaller key sizes result in savings for power, memory, bandwidth, and computational cost that make ECC especially attractive for constrained environments. OTP-11122 Fix {stream, {self, once}} in httpc to work as expected. Thanks to Masatake Daimon --- kernel -------------------------------------------------------------- OTP-10497 A bug in prim_inet has been corrected. If the port owner was killed at a bad time while closing the socket port the port could become orphaned hence causing port and socket leaking. Reported by Fred Herbert, Dmitry Belyaev and others. OTP-10910 Add application:ensure_started/1,2. It is equivavlent to application:start/1,2 except it returns ok for already started applications. OTP-11040 Optimize communication with file io server. Thanks to Anthony Ramine. OTP-11041 Erlang source files with non-ASCII characters are now encoded in UTF-8 (instead of latin1). OTP-11074 Optimization of simultaneous inet_db operations on the same socket by using a lock free implementation. OTP-11075 The high_msgq_watermark and low_msgq_watermark inet socket options introduced in OTP-R16A could only be set on TCP sockets. These options are now generic and can be set on all types of sockets. OTP-11104 Fix deep list argument error under Windows in os:cmd/1. Thanks to Aleksandr Vinokurov . --- megaco -------------------------------------------------------------- OTP-11016 Postscript files no longer needed for the generation of PDF files have been removed. --- mnesia -------------------------------------------------------------- OTP-11030 If mnesia:clear_table/2 was called during a table load on that table, the schema record was written to the table instead of clearing table. OTP-11103 Optimize index creation for Mnesia set tables. Thanks to Nick Marino. --- observer ------------------------------------------------------------ OTP-10604 -- The new Memory field from a crash dump is now presented by crashdump viewer, both in the process overview and in the process detail page. -- A summary of blocks- and carriers sizes is added to the allocator information page in the crashdump viewer. OTP-10894 Some bugs related to calculation of CPU/scheduler utilization in observer are corrected. Current function for a process is accepted to be 'undefined' when running hipe. OTP-10929 Use "open" as default browser for crashdump viewer on Mac OS X. Thanks to Magnus Henoch. OTP-10931 Fix observer table viewer crash on formatting improper lists. Thanks to Andrey Tsirulev OTP-11016 Postscript files no longer needed for the generation of PDF files have been removed. OTP-11136 Add processes state view in observer. Thanks to Eric Pailleau. --- odbc ---------------------------------------------------------------- OTP-10993 Fix a 64bit related bug in odbcserver. Thanks to Satoshi Kinoshita. OTP-11016 Postscript files no longer needed for the generation of PDF files have been removed. OTP-11126 Fix checking for odbc in standard locations when "with-odbc" flag present. Thanks to Alexey Saltanov. --- orber --------------------------------------------------------------- OTP-11016 Postscript files no longer needed for the generation of PDF files have been removed. --- os_mon -------------------------------------------------------------- OTP-10941 Compilation fixes for NetBSD. Thanks to YAMAMOTO Takashi. OTP-10945 Fixed disksup:get_disk_data for SUSv3, specifically OS X ML. Thanks to Sriram Melkote. --- otp ----------------------------------------------------------------- OTP-11117 Properly ignore os_mon and otp_mibs generated includes --- percept ------------------------------------------------------------- OTP-11016 Postscript files no longer needed for the generation of PDF files have been removed. --- public_key ---------------------------------------------------------- OTP-10873 Add support for ISO oids 1.3.14.3.2.29 and 1.3.14.3.2.27 that are somtimes used instead of the PKCS defined oids 1.2.840.113549.1.1.5 and 1.2.840.10040.4.3. Add function pkix_sign_types:/1 that translates oids to to algorithm atoms ex: > public_key:pkix_sign_types({1,3,14,3,2,29}). {sha,rsa} OTP-11009 Integrate elliptic curve contribution from Andreas Schultz In order to be able to support elliptic curve cipher suites in SSL/TLS, additions to handle elliptic curve infrastructure has been added to public_key and crypto. This also has resulted in a rewrite of the crypto API to gain consistency and remove unnecessary overhead. All OTP applications using crypto has been updated to use the new API. Impact: Elliptic curve cryptography (ECC) offers equivalent security with smaller key sizes than other public key algorithms. Smaller key sizes result in savings for power, memory, bandwidth, and computational cost that make ECC especially attractive for constrained environments. --- reltool ------------------------------------------------------------- OTP-10988 Reltool used to fail if an application was mentioned in the config file which was not found in the file system, even if the application was explicitly excluded in the config. This has been changed and will only produce a warning. If the application is not explicitly excluded it will still cause reltool to fail. Thanks to H?kan Mattsson! OTP-11099 Fix possibly "not owner" error while file copy with reltool. Thanks to Alexey Saltanov. OTP-11137 Fix receive support in erl_eval with a BEAM module. Thanks to Anthony Ramine. --- runtime_tools ------------------------------------------------------- OTP-10894 Some bugs related to calculation of CPU/scheduler utilization in observer are corrected. Current function for a process is accepted to be 'undefined' when running hipe. OTP-11041 Erlang source files with non-ASCII characters are now encoded in UTF-8 (instead of latin1). --- sasl ---------------------------------------------------------------- OTP-11137 Fix receive support in erl_eval with a BEAM module. Thanks to Anthony Ramine. --- snmp ---------------------------------------------------------------- OTP-11004 Fix SNMP gitignore files. Thanks to Anthony Ramine. OTP-11009 Integrate elliptic curve contribution from Andreas Schultz In order to be able to support elliptic curve cipher suites in SSL/TLS, additions to handle elliptic curve infrastructure has been added to public_key and crypto. This also has resulted in a rewrite of the crypto API to gain consistency and remove unnecessary overhead. All OTP applications using crypto has been updated to use the new API. Impact: Elliptic curve cryptography (ECC) offers equivalent security with smaller key sizes than other public key algorithms. Smaller key sizes result in savings for power, memory, bandwidth, and computational cost that make ECC especially attractive for constrained environments. OTP-11016 Postscript files no longer needed for the generation of PDF files have been removed. OTP-11101 [snmp/agent] Introduced the mib-server data module behaviour. OTP-11107 [snmp/agent] Introduced a documented mib-storage behaviour. This will allow users to implement alternative storage methods. --- ssh ----------------------------------------------------------------- OTP-10975 ssh:daemon will get feeded with an argument even if it is not a valid expression. OTP-10983 Properly ignore everything in lib/ssh/doc/html/. Thanks to Anthony Ramine. OTP-11009 Integrate elliptic curve contribution from Andreas Schultz In order to be able to support elliptic curve cipher suites in SSL/TLS, additions to handle elliptic curve infrastructure has been added to public_key and crypto. This also has resulted in a rewrite of the crypto API to gain consistency and remove unnecessary overhead. All OTP applications using crypto has been updated to use the new API. Impact: Elliptic curve cryptography (ECC) offers equivalent security with smaller key sizes than other public key algorithms. Smaller key sizes result in savings for power, memory, bandwidth, and computational cost that make ECC especially attractive for constrained environments. --- ssl ----------------------------------------------------------------- OTP-10450 Add support for PSK (Pre Shared Key) and SRP (Secure Remote Password) chipher suits, thanks to Andreas Schultz. OTP-10905 Honor the versions option to ssl:connect and ssl:listen. OTP-10909 Next protocol negotiation with reused sessions will now succeed OTP-10955 Fix SSL Next Protocol Negotiation documentation. Thanks to Julien Barbot. OTP-10980 Fix ssl_connection to support reading proxy/chain certificates. Thanks to Valentin Kuznetsov. OTP-11009 Integrate elliptic curve contribution from Andreas Schultz In order to be able to support elliptic curve cipher suites in SSL/TLS, additions to handle elliptic curve infrastructure has been added to public_key and crypto. This also has resulted in a rewrite of the crypto API to gain consistency and remove unnecessary overhead. All OTP applications using crypto has been updated to use the new API. Impact: Elliptic curve cryptography (ECC) offers equivalent security with smaller key sizes than other public key algorithms. Smaller key sizes result in savings for power, memory, bandwidth, and computational cost that make ECC especially attractive for constrained environments. --- stdlib -------------------------------------------------------------- OTP-10938 Delete obsolete note about simple-one-for-one supervisor. Thanks to Magnus Henoch. OTP-10951 When selecting encoding of a script written in Erlang (escript) the optional directive on the second line is now recognized. OTP-10990 The Erlang scanner no longer accepts floating point numbers in the input string. OTP-10992 The function erl_parse:abstract/2 has been documented. OTP-11009 Integrate elliptic curve contribution from Andreas Schultz In order to be able to support elliptic curve cipher suites in SSL/TLS, additions to handle elliptic curve infrastructure has been added to public_key and crypto. This also has resulted in a rewrite of the crypto API to gain consistency and remove unnecessary overhead. All OTP applications using crypto has been updated to use the new API. Impact: Elliptic curve cryptography (ECC) offers equivalent security with smaller key sizes than other public key algorithms. Smaller key sizes result in savings for power, memory, bandwidth, and computational cost that make ECC especially attractive for constrained environments. OTP-11013 Added sys:get_state/1,2 and sys:replace_state/2,3. Thanks to Steve Vinoski. OTP-11025 Optimizations to gen mechanism. Thanks to Lo?c Hoguin. OTP-11035 Optimizations to gen.erl. Thanks to Lo?c Hoguin. OTP-11039 Use erlang:demonitor(Ref, [flush]) where applicable. Thanks to Lo?c Hoguin. OTP-11041 Erlang source files with non-ASCII characters are now encoded in UTF-8 (instead of latin1). OTP-11042 Fix rest_for_one and one_for_all restarting a child not terminated. Thanks to James Fish. OTP-11053 Fix excessive CPU consumption of timer_server. Thanks to Aliaksey Kandratsenka. OTP-11078 Rename and document lists:zf/2 as lists:filtermap/2. Thanks to Anthony Ramine. OTP-11079 Fixed an inconsistent state in epp. Thanks to Anthony Ramine OTP-11080 When converting a faulty binary to a list with unicode:characters_to_list, the error return value could contain a faulty "rest", i.e. the io_list of characters that could not be converted was wrong. This happened only if input was a sub binary and conversion was from utf8. This is now corrected. OTP-11100 The type hook_function() has been corrected in erl_pp, the Erlang Pretty Printer. The printing of invalid forms, e.g. record field types, has also been fixed. It has been broken since R16B. (Thanks to Tomáš Janoušek.) OTP-11108 c:ls(File) will now print File, similar to ls(1) in Unix. The error messages have also been improved. (Thanks to Bengt Kleberg.) OTP-11137 Fix receive support in erl_eval with a BEAM module. Thanks to Anthony Ramine. OTP-11140 Support callback attributes in erl_pp. Thanks to Anthony Ramine. OTP-11143 Improve erl_lint performance. Thanks to Jos? Valim. --- test_server --------------------------------------------------------- OTP-10607 Some unused code related to remote targets is removed, and documentation is updated. OTP-10856 The '-force_stop' flag to use with time-limited repeats of test runs can now be used with a new 'skip_rest' option which causes the rest of the test cases in the ongoing test job to be skipped when the time limit is reached. E.g. 'ct_run -spec xxx -duration 010000 -force_stop skip_rest' OTP-10991 A bug in test_server_gl caused io requests containing invalid data (i.e. not unicode:chardata()) to hang, since no io reply was sent. This has been corrected. OTP-11052 Common Test would, in case of timetrap error, print a warning in the log if end_per_testcase wasn't implemented in the suite, even though it's an optional function. This printout has been removed. --- tools --------------------------------------------------------------- OTP-10902 Make cover smarter about finding source from beam. In particular, search using the source path in module_info if the current heuristic fails. OTP-10930 Remove Flymake dependency in erlang-pkg.el. Thanks to Magnus Henoch. OTP-10979 When cover:stop(Node) was called on a non-existing node, a process waiting for cover data from the node would hang forever. This has been corrected. OTP-10999 Erlang-mode: Add autoload cookies for file extension associations. Thanks to Magnus Henoch. OTP-11016 Postscript files no longer needed for the generation of PDF files have been removed. OTP-11028 Fix a race condition when there're several applications in apps directory. Thanks to Manuel Rubio. OTP-11144 New option for eprof, 'set_on_spawn'. This option was previously always on and is also the default. From mloftis@REDACTED Wed Jun 12 15:15:23 2013 From: mloftis@REDACTED (Michael Loftis) Date: Wed, 12 Jun 2013 06:15:23 -0700 Subject: [erlang-questions] why is mmap so darn difficult In-Reply-To: <7BC1729C-4602-4F6F-B96E-FCE0402DCCC5@gmail.com> References: <57542809-682D-42F0-BBF0-91C67EBD5B62@gmail.com> <1056D6F0-6E3D-4FBF-ABE9-E2420862F84C@gmail.com> <7BC1729C-4602-4F6F-B96E-FCE0402DCCC5@gmail.com> Message-ID: On Wed, Jun 12, 2013 at 2:41 AM, Tim Watson wrote: > Max, > > On 12 Jun 2013, at 09:42, Max Lapshin wrote: > > I thought that video streaming, when you need to read about half of > gigabyte per second from disk, repack it and write to network, is the > good place for mmap. > It was a mistake. If you get slow mmap disk read, than whole erlang > sticks and you get PHP-like performance when whole thread is blocked > for one client. > > > Isn't that because the pread call you used was in a NIF, and therefore could > block the scheulder(s) no? My thought's about the approach were initially > somewhat different. The write operations were just file:write, whereas there > would be no pread, but rather data would be delivered to the socket using > sendfile/5, which would utilise the async thread pool if it's enabled. > > Perhaps in embedded mmap would be better when you want to save memory > and have 10 requests per day, but on high load mmap doesn't help. > > > I'd like to handle writing hundreds of thousands of input events per second > and reading at various different speeds, but supporting (reads of) up to > tens of thousands per second. > mmap would still need to run in a thread to make things consistently faster. Whenever a miss happens without being in a thread you're going down the slow path of a page fault. If this is in a main erlang VM thread, that thread is just stalled out and doing no useful work while this happens, and then requests stack up behind it. The thread eventually becomes available again and hopefully services a lot of those backlogged requests before being hit with another page fault to/from disk. If the page faults happen often enough the whole thing comes apart really quickly. This is why file:* would, should, be faster. The problem isn't in the "everything is going well" case, it's when the slow path hits, it doesn't block just that one request, it blocks all possible work on that entire thread with mmap, partly because you can't even guess if something is available or not, you just have to hope/pray it is. With erlang async i/o threads, the i/o thread is the one that gets blocked on a read or a write. Zero copy read/write could help some, but your overwhelming latency is going to be in those slow hits. You still benefit from the OSes disk cache even without mmap. -- "Genius might be described as a supreme capacity for getting its possessors into trouble of all kinds." -- Samuel Butler From watson.timothy@REDACTED Wed Jun 12 19:38:02 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 12 Jun 2013 18:38:02 +0100 Subject: [erlang-questions] why is mmap so darn difficult In-Reply-To: References: <57542809-682D-42F0-BBF0-91C67EBD5B62@gmail.com> <1056D6F0-6E3D-4FBF-ABE9-E2420862F84C@gmail.com> <7BC1729C-4602-4F6F-B96E-FCE0402DCCC5@gmail.com> Message-ID: <4AA125AF-98B1-4CA0-91B6-E2109D1757C6@gmail.com> Hi Michael, On 12 Jun 2013, at 14:15, Michael Loftis wrote: > mmap would still need to run in a thread to make things consistently > faster. Whenever a miss happens without being in a thread you're > going down the slow path of a page fault. If this is in a main erlang > VM thread, that thread is just stalled out and doing no useful work > while this happens, and then requests stack up behind it. The thread > eventually becomes available again and hopefully services a lot of > those backlogged requests before being hit with another page fault > to/from disk. If the page faults happen often enough the whole thing > comes apart really quickly. This is why file:* would, should, be > faster. I get that part, which was why I though that using the async thread pool might help. Using file:pwrite doesn't appear to offer particularly good performance, even though I'm performing sequential writes (i.e., always appending to the end of the file). It took a full 103519.239ms to get just over 5Gb worth of 100k chunks onto the disk, and that's with [raw, binary, delayed_write] to boot. That's roughly 52429 entries in 104 seconds, for a write speed of 504Hz, which is completely pants. I know of write speeds in the order of 10s of Khz are possible with a combination of prim_file:* instead of file:* and various other tricks, but that's still nowhere near fast enough and the examples I'm thinking of are really designed for more random access patterns. Another thing to bear in mind here is that my writes are almost entirely sequential - just appending to a file. Reads are also sequential in nature, though there is an initial seek required to get to the starting position, but after adjusting for the initial offset, all reads are sequential. Currently, I've managed to flush data to a connected socket using sendfile and can get around 100Khz for a single consumer. Sadly adding multiple consumers of the same file and attempting to use file:sendfile seems to introduce a lot of variance into the results. Using a combination of file:pread and gen_tcp:send however, produced very poor results for me, despite the sequential access. > The problem isn't in the "everything is going well" case, it's when > the slow path hits, it doesn't block just that one request, it blocks > all possible work on that entire thread with mmap, partly because you > can't even guess if something is available or not, you just have to > hope/pray it is. Yes indeed, that's why I don't like the idea of using NIFs for things like this - I prefer drivers for the most part really. Hopefully my opinion will change when we (eventually) get 'native processes' in R19 (or whenever). > With erlang async i/o threads, the i/o thread is the > one that gets blocked on a read or a write. Exactly. I can have 1024 (ish) thread pools threads sucking up potential delays, meanwhile I don't have to worry about scheduler threads getting 'stuck' as it were. Yes of course there will be additional costs involved, due to context switches and whatnot, but still. > Zero copy read/write > could help some, but your overwhelming latency is going to be in those > slow hits. You still benefit from the OSes disk cache even without > mmap. Sure, that's certainly true. Cheers, Tim From watson.timothy@REDACTED Wed Jun 12 19:44:41 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 12 Jun 2013 18:44:41 +0100 Subject: [erlang-questions] why is mmap so darn difficult In-Reply-To: References: <57542809-682D-42F0-BBF0-91C67EBD5B62@gmail.com> <1056D6F0-6E3D-4FBF-ABE9-E2420862F84C@gmail.com> <7BC1729C-4602-4F6F-B96E-FCE0402DCCC5@gmail.com> Message-ID: Hi Max, On 12 Jun 2013, at 10:43, Max Lapshin wrote: > Of course, you will not be able to handle such load with mmap, because > you will suffer from blocking whole VM on mmap page faults. > Yes of course you're right about that, with mmap running in a NIF. I attempted to use both emmap and the mmap driver you wrote for erlyvideo and found the resulting behaviour highly variable. > Add more async threads and profile application. Don't waste your time on mmap. I'm starting to wonder whether attempting to provide very high speed concurrent reads + writes in Erlang is going to work at all. I might spend some time looking at lmdb instead, since it claims some extremely high numbers. I was hoping to avoid writing a driver though, but ho hum... Cheers, Tim From tcury@REDACTED Wed Jun 12 18:59:36 2013 From: tcury@REDACTED (Tiago Cury) Date: Wed, 12 Jun 2013 13:59:36 -0300 Subject: [erlang-questions] erl_tar:add continue on error In-Reply-To: References: <50D49D74-A1FA-4CF6-B210-1A7D157011A3@ymail.com> Message-ID: <7FA4BC38-D002-4E8E-B5E0-B8DF15B46BC1@ymail.com> Good. I'll try this approach. Thanks ;-) On Jun 8, 2013, at 4:41 AM, Peer Stritzinger wrote: > On 2013-06-07 20:15:39 +0000, Tiago Cury said: > >> Hi, I'm using erl_tar:add to create a backup from my server, but when it finds a kind of file type like socket it returns >> immediately with a tuple {error, reason >> } and aborts the entire process. >> Is there a way to erl_tar:add ignore these errors (skip just the bad file) and continue the operation? > > You could use filelib:fold_files to recurse over the tree calling erl_tar:add on what it findes. > > According to its documentation it only recurses over regular files. > > Cheers > -- Peer > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From max.lapshin@REDACTED Wed Jun 12 20:47:29 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Wed, 12 Jun 2013 22:47:29 +0400 Subject: [erlang-questions] why is mmap so darn difficult In-Reply-To: References: <57542809-682D-42F0-BBF0-91C67EBD5B62@gmail.com> <1056D6F0-6E3D-4FBF-ABE9-E2420862F84C@gmail.com> <7BC1729C-4602-4F6F-B96E-FCE0402DCCC5@gmail.com> Message-ID: mmap is faster than file when you are reading from vfs cache. It is faster on, lets say 5%, or even 15%. But speed of microbenchmarks is nothing when you run real service. You need to make your software controllable. For example, erlyvideo makes gen_server:call to a special queue before each disk read. It is slower, but it makes server much more stable. It makes many other different checks like process_info(Pid, message_queue_len) to make things more stable and server behaviour smooth. If you chase for speed of microbenchmarks, you can build a server that runs excellently on your laptop, but begins to block on production. This is my lesson from my experience. On Wed, Jun 12, 2013 at 9:44 PM, Tim Watson wrote: > Hi Max, > > On 12 Jun 2013, at 10:43, Max Lapshin wrote: > >> Of course, you will not be able to handle such load with mmap, because >> you will suffer from blocking whole VM on mmap page faults. >> > > Yes of course you're right about that, with mmap running in a NIF. I attempted to use both emmap and the mmap driver you wrote for erlyvideo and found the resulting behaviour highly variable. > >> Add more async threads and profile application. Don't waste your time on mmap. > > I'm starting to wonder whether attempting to provide very high speed concurrent reads + writes in Erlang is going to work at all. I might spend some time looking at lmdb instead, since it claims some extremely high numbers. I was hoping to avoid writing a driver though, but ho hum... > > Cheers, > Tim From watson.timothy@REDACTED Wed Jun 12 21:31:16 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 12 Jun 2013 20:31:16 +0100 Subject: [erlang-questions] why is mmap so darn difficult In-Reply-To: References: <57542809-682D-42F0-BBF0-91C67EBD5B62@gmail.com> <1056D6F0-6E3D-4FBF-ABE9-E2420862F84C@gmail.com> <7BC1729C-4602-4F6F-B96E-FCE0402DCCC5@gmail.com> Message-ID: <5F37F59C-0CCC-47DE-8161-CCB7F40B3096@gmail.com> On 12 Jun 2013, at 19:47, Max Lapshin wrote: > For example, erlyvideo makes gen_server:call to a special queue before > each disk read. It is slower, but it makes server much more stable. > It makes many other different checks like process_info(Pid, > message_queue_len) to make things more stable and server behaviour > smooth. > > If you chase for speed of microbenchmarks, you can build a server that > runs excellently on your laptop, but begins to block on production. > > This is my lesson from my experience. Yes indeed, we've all done things like that. Of course it would be nice to have both, i.e., something that's fast and consistently reliable. I'm willing to sacrifice throughput where that's necessary of course, and the business of forcing reads/writes through a funnel is not a bad thing either. I'd just like that funnel to be a bit quicker is all. ;) Thanks again for your helpful feedback - knowing about your experiences has definitely given me plenty of pause for thought, which is *always* a good thing. Cheers, Tim From tcury@REDACTED Wed Jun 12 21:30:33 2013 From: tcury@REDACTED (Tiago Cury) Date: Wed, 12 Jun 2013 16:30:33 -0300 Subject: [erlang-questions] exception when no match found Message-ID: <12C5A5CC-5B2E-40CF-B754-DE39D84744BE@ymail.com> I think that erlang would just bypass the entire case..of instead of throw an exception when no match found. It's can economize more lines of code. From mononcqc@REDACTED Wed Jun 12 22:33:28 2013 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 12 Jun 2013 13:33:28 -0700 Subject: [erlang-questions] exception when no match found In-Reply-To: <12C5A5CC-5B2E-40CF-B754-DE39D84744BE@ymail.com> References: <12C5A5CC-5B2E-40CF-B754-DE39D84744BE@ymail.com> Message-ID: <20130612203327.GA90819@ferdair.local> Well I think it shouldn't. Crashes can economize me headaches by being very loud things instead of silent buggers that I wonder why the hell they don't get to run and stuff after them fails. On 06/12, Tiago Cury wrote: > I think that erlang would just bypass the entire case..of instead of throw an exception when no match found. It's can economize more lines of code. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From essen@REDACTED Wed Jun 12 22:43:20 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 12 Jun 2013 22:43:20 +0200 Subject: [erlang-questions] exception when no match found In-Reply-To: <12C5A5CC-5B2E-40CF-B754-DE39D84744BE@ymail.com> References: <12C5A5CC-5B2E-40CF-B754-DE39D84744BE@ymail.com> Message-ID: <51B8DD68.2080300@ninenines.eu> (Sorry, failed to Reply All.) A case is an expression, it returns a value. You can't have implicit clauses because the compiler can't guess what this value should be. On 06/12/2013 09:30 PM, Tiago Cury wrote: > I think that erlang would just bypass the entire case..of instead of throw an exception when no match found. It's can economize more lines of code. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From mediocregopher@REDACTED Thu Jun 13 02:28:47 2013 From: mediocregopher@REDACTED (Brian Picciano) Date: Wed, 12 Jun 2013 20:28:47 -0400 Subject: [erlang-questions] High lock contention on dist_tables In-Reply-To: References: <43187.1366743687@snookles.snookles.com> Message-ID: Thanks for the heads up Lukas! Sorry I stopped responding, we ended up solving the problem (for now) by drastically cutting down on inter-node communication in another way, and this thread got lost in my inbox, but I really appreciate the follow-up! On Thu, May 16, 2013 at 3:43 AM, Lukas Larsson wrote: > Hello Brian, > > Just letting you know that I have just merged a fix which changes the > rwlock I mentioned before to an rlock. This should reduce the contention > which you are seeing if it was caused by many calls to erlang:nodes(). > > Lukas > > > On Tue, Apr 23, 2013 at 9:01 PM, Scott Lystig Fritchie < > fritchie@REDACTED> wrote: > >> Brian Picciano wrote: >> >> bp> We have a pool of 3 erlang nodes, all on different servers. Every >> bp> afternoon, without fail, we start seeing lots of messages between >> bp> the nodes start having really high latency, on the order of tens of >> bp> seconds. [...] >> >> Brian, it's probably worthwhile to continue chasing the 'lcnt' avenue >> as you've been corresponding with Lukas... >> >> ... but at the same time, I also wonder about "tens of seconds". My gut >> says that such delays would require some amazingly high lock contention >> rates. Something that can cause such messaging delays much more easily >> is network congestion/packet loss that triggers TCP slow start. Many >> Linux kernels have the RTO_min value at one second, which is the amount >> of time to wait before entering slow start state. >> >> If network packet loss is a problem, this blog posting can explain one >> reason why it's happening: >> http://www.snookles.com/slf-blog/2012/01/05/tcp-incast-what-is-it/ >> >> -Scott >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yashgt@REDACTED Thu Jun 13 09:23:45 2013 From: yashgt@REDACTED (yashgt@REDACTED) Date: Thu, 13 Jun 2013 07:23:45 +0000 Subject: [erlang-questions] Unloading all modules from ERL session Message-ID: <51b97396.515ce00a.0a84.ffffce40@mx.google.com> An HTML attachment was scrubbed... URL: From gustav.simonsson@REDACTED Thu Jun 13 09:30:01 2013 From: gustav.simonsson@REDACTED (Gustav Simonsson) Date: Thu, 13 Jun 2013 09:30:01 +0200 Subject: [erlang-questions] Unloading all modules from ERL session In-Reply-To: <51b97396.515ce00a.0a84.ffffce40@mx.google.com> References: <51b97396.515ce00a.0a84.ffffce40@mx.google.com> Message-ID: Hi, If this is for development, you may want to look at these two tools: https://github.com/oinksoft/reloader https://github.com/rustyio/sync Cheers, Gustav On Thu, Jun 13, 2013 at 9:23 AM, wrote: > Hi, > > If an Erlang session on command prompt is running and I call a function > from a module, the module gets loaded. If I realize that there is a bug, > and while leaving the erl session running, change the code and recompile > the module and amny other modules, the older version of the module remains > loaded in teh erl session. > > How do I unload all loaded modules so that the newly compiled modules get > loaded the next time I invoke the function in the erl session, the new code > takes effect? > > Thanks, > Yash > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.goertzen@REDACTED Thu Jun 13 17:46:20 2013 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Thu, 13 Jun 2013 10:46:20 -0500 Subject: [erlang-questions] Unloading all modules from ERL session In-Reply-To: <51b97396.515ce00a.0a84.ffffce40@mx.google.com> References: <51b97396.515ce00a.0a84.ffffce40@mx.google.com> Message-ID: If it is just 1 or 2 modules, reload those modules with... 1> l(mymodule). {module,mymodule} 2> Dan. On Thu, Jun 13, 2013 at 2:23 AM, wrote: > Hi, > > If an Erlang session on command prompt is running and I call a function > from a module, the module gets loaded. If I realize that there is a bug, > and while leaving the erl session running, change the code and recompile > the module and amny other modules, the older version of the module remains > loaded in teh erl session. > > How do I unload all loaded modules so that the newly compiled modules get > loaded the next time I invoke the function in the erl session, the new code > takes effect? > > Thanks, > Yash > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lcastro@REDACTED Thu Jun 13 18:36:25 2013 From: lcastro@REDACTED (Laura M. Castro) Date: Thu, 13 Jun 2013 18:36:25 +0200 Subject: [erlang-questions] [Deadline extension] ACM SIGPLAN Erlang Workshop 2013 Final Call For Papers Message-ID: Hello, Responding to popular demand, we announce an extension for the Twelfth ACM SIGPLAN Erlang Workshop. New deadline is June 21st. Apologies for any duplicates you may receive. CALL FOR PAPERS ================= Twelfth ACM SIGPLAN Erlang Workshop ----------------------------------------------------------- Boston, Massachusetts, September 28, 2013 Satellite event of the 18th ACM SIGPLAN International Conference on Functional Programming (ICFP 2013) September 25-27, 2013 Erlang is a concurrent, distributed functional programming language aimed at systems with requirements of massive concurrency, soft real time response, fault tolerance, and high availability. It has been available as open source for 15 years, creating a community that actively contributes to its already existing rich set of libraries and applications. Originally created for telecom applications, its usage has spread to other domains including e-commerce, banking, databases, and computer telephony and messaging. Erlang programs are today among the largest applications written in any functional programming language. These applications offer new opportunities to evaluate functional programming and functional programming methods on a very large scale and suggest new problems for the research community to solve. This workshop will bring together the open source, academic, and industrial programming communities of Erlang. It will enable participants to familiarize themselves with recent developments on new techniques and tools tailored to Erlang, novel applications, draw lessons from users' experiences and identify research problems and common areas relevant to the practice of Erlang and functional programming. We invite three types of submissions. 1. Technical papers describing language extensions, critical discussions of the status quo, formal semantics of language constructs, program analysis and transformation, virtual machine extensions and compilation techniques, implementations and interfaces of Erlang in/with other languages, and new tools (profilers, tracers, debuggers, testing frameworks, etc.). The maximum length for technical papers is restricted to 12 pages. 2. Practice and application papers describing uses of Erlang in the "real-world", Erlang libraries for specific tasks, experiences from using Erlang in specific application domains, reusable programming idioms and elegant new ways of using Erlang to approach or solve a particular problem. The maximum length for the practice and application papers is restricted to 12 pages. Note that this is a maximum length; we welcome shorter papers also, and the program committee will evaluate all papers on an equal basis independent of their lengths. 3. Poster presentations describing topics related to the workshop goals. Each includes a maximum of 2 pages of the abstract and summary. Presentations in this category will be given an hour of shared simultaneous demonstration time. Workshop Chair ----------------------- Steve Vinoski, Basho Technologies, USA Program Chair ------------------- Laura M. Castro, University of A Coru?a, Spain Program Committee ----------------------------- (Note: the Workshop and Program Chairs are also committee members) Lars-Ake Fredlund, Universidad Polit?cnica de Madrid, Spain Kevin Hammond, University of St. Andrews, UK Torben Hoffman, Erlang Solutions Limited, UK Zolt?n Horv?th, E?tv?s Lor?nd University, Hungary Kenneth Lundin, Ericsson AB, Sweden Micka?l R?mond, ProcessOne, France Kenji Rikitake, Basho Japan KK, Japan Simon Thompson, University of Kent, UK Important Dates ----------------------- Submission deadline: Fri June 14, 2013 [EXTENDED: Fri June 21, 2013] Author notification: Thu July 11, 2013 Final submission for the publisher: Thu July 25, 2013 Workshop date (tentative, subject to change): September 28, 2013 Instructions to authors -------------------------------- Papers must be submitted online via EasyChair (via the "Erlang2013" event). The submission page is https://www.easychair.org/conferences/?conf=erlang2013 Submitted papers should be in portable document format (PDF), formatted using the ACM SIGPLAN style guidelines. Each submission must adhere to SIGPLAN's republication policy. Violation risks summary rejection of the offending submission. Accepted papers will be published by the ACM and will appear in the ACM Digital Library. Paper submissions will be considered for poster submission in the case they are not accepted as full papers. Venue & Registration Details ------------------------------------------ For registration, please see the ICFP 2013 web site at: http://icfpconference.org/icfp2013/ Related Links -------------------- ICFP 2013 web site: http://www.icfpconference.org/icfp2013/ Past ACM SIGPLAN Erlang workshops: http://www.erlang.org/workshop/ Open Source Erlang: http://www.erlang.org/ EasyChair submission site: https://www.easychair.org/account/signin.cgi?conf=erlang2013 Author Information for SIGPLAN Conferences: http://www.sigplan.org/authorInformation.htm -- Laura M. Castro From q3.development@REDACTED Thu Jun 13 18:06:04 2013 From: q3.development@REDACTED (QDev) Date: Thu, 13 Jun 2013 17:06:04 +0100 Subject: [erlang-questions] driver_create_port overhead and driver_caller Message-ID: <4765DCE4-9840-4187-B1A5-D7DD143320AF@btinternet.com> I have linked in driver which deals with embedded library, can make calls some in outputv, some using driver_async, but now have to support some other long running operations requiring thread affinity and (more importantly) need to receive multiple commands/actions from Erlang client code associated with long running transaction in library API. Currently considering to spawn (or pool) some new thread for these (long running tasks) and using ErlDrvMutex and simple queue/array data structure to communicate with it from outputv, but there is also affinity between Erlang callers process id and long running task thread (since caller originates with erlang:send/2 direct to port, which is accessed via ets table), because replies are sent via driver_send_term so need to also store client's process id in ErlDrvTermData and use driver_monitor_process to ensure that if client dies, work is rolled back. IIRC driver_caller not thread safe, so need to only use this in outputv is that still correct? Now, just getting used to driver API changes in R16 (haven't written driver since R13B) and have seen driver_create_port. This seems to create new port running driver code, so wonder if this is better approach for these long running tasks, and what are benefits/advantage of doing this? Thanks QD From andrew@REDACTED Thu Jun 13 19:43:19 2013 From: andrew@REDACTED (Andrew Thompson) Date: Thu, 13 Jun 2013 13:43:19 -0400 Subject: [erlang-questions] Can I prevent lager from changing to sync mode? In-Reply-To: References: Message-ID: <20130613174319.GR31341@hijacked.us> On Wed, Jun 12, 2013 at 10:12:46AM +0900, Park, Sungjin wrote: > I want it never fall to sync mode. > Just dropping some logs is ok. Right now, for anything other than error_logger messages, lager will not drop any messages, ever. The sync/async modes are just for mailbox size regulation. Andrew From ransomr@REDACTED Thu Jun 13 21:36:50 2013 From: ransomr@REDACTED (Ransom Richardson) Date: Thu, 13 Jun 2013 19:36:50 +0000 Subject: [erlang-questions] child_setup high CPU usage Message-ID: I'm seeing an issue where under what should be light load I see two child_setup processes spawned by Erlang using 100% CPU (combined) and making my machine unresponsive for a number of seconds. I recently updated to R16B and never saw this issue on previous releases. This happens when processing files sent over ssl from a client. My code uploads the files to S3 (using erlcloud, which uses httpc). If the client uploads only a few files it works fine. But if the client uploads about 6 files in a row then I see the behavior where the two child_setup processes consume 100% CPU. I would expect the only CPU intensive parts of this operation to be the calls to digest and MAC the S3 request. I'm not aware of any calls made to the shell in this scenario. I plan to debug further, but would be interested if anyone else has seen this or has ideas as to what could be causing this or how best to start looking. thanks, Ransom -------------- next part -------------- An HTML attachment was scrubbed... URL: From ransomr@REDACTED Thu Jun 13 22:04:01 2013 From: ransomr@REDACTED (Ransom Richardson) Date: Thu, 13 Jun 2013 20:04:01 +0000 Subject: [erlang-questions] child_setup high CPU usage In-Reply-To: References: Message-ID: To answer at least part of my own question, I think the issue was cause by increasing the ulimit for nofile to 1000000. Here's the child_setup line: /usr/local/lib/erlang/erts-5.10.1/bin/child_setup false . exec /bin/sh -s unix:cmd 2>&1 3:999999 50:1 56:0 - ________________________________ From: erlang-questions-bounces@REDACTED on behalf of Ransom Richardson Sent: Thursday, June 13, 2013 3:36 PM To: erlang-questions@REDACTED Subject: [erlang-questions] child_setup high CPU usage I'm seeing an issue where under what should be light load I see two child_setup processes spawned by Erlang using 100% CPU (combined) and making my machine unresponsive for a number of seconds. I recently updated to R16B and never saw this issue on previous releases. This happens when processing files sent over ssl from a client. My code uploads the files to S3 (using erlcloud, which uses httpc). If the client uploads only a few files it works fine. But if the client uploads about 6 files in a row then I see the behavior where the two child_setup processes consume 100% CPU. I would expect the only CPU intensive parts of this operation to be the calls to digest and MAC the S3 request. I'm not aware of any calls made to the shell in this scenario. I plan to debug further, but would be interested if anyone else has seen this or has ideas as to what could be causing this or how best to start looking. thanks, Ransom -------------- next part -------------- An HTML attachment was scrubbed... URL: From darkit@REDACTED Thu Jun 13 22:19:42 2013 From: darkit@REDACTED (Max Grigoriev) Date: Thu, 13 Jun 2013 23:19:42 +0300 Subject: [erlang-questions] Couple Riaks question Message-ID: - Should I create a connection pool or it's OK to create {ok, Pid} = riakc_pb_socket:start_link("127.0.0.1", 8087) on each web request It's very strange and pity that erlang native client doesn't support all features Java one has: http://docs.basho.com/riak/latest/references/Client-Libraries/ - Am I right that Link walking is implemented as Map/Reduce and it uses full scan? - How do you connect to riak nodes (I mean how to decide to which node connet) or just put Riak cluster behind HAProxy and that's all Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.charles.davis@REDACTED Fri Jun 14 02:29:40 2013 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 13 Jun 2013 17:29:40 -0700 (PDT) Subject: [erlang-questions] Unloading all modules from ERL session In-Reply-To: <51b97396.515ce00a.0a84.ffffce40@mx.google.com> References: <51b97396.515ce00a.0a84.ffffce40@mx.google.com> Message-ID: <554fd84a-b7f0-4a5e-af1a-b6b069bd65a2@googlegroups.com> Why not make:all([load]). ??? e.g. I have this in my user_default... make() -> {ok, WorkingDir} = file:get_cwd(), case file:read_file_info("Emakefile") of {ok, _} -> io:format("Emake ~p~n", [WorkingDir]); {error, _} -> io:format("~p~n", [WorkingDir]) end, make:all([load]). -------------- next part -------------- An HTML attachment was scrubbed... URL: From rr@REDACTED Fri Jun 14 03:35:34 2013 From: rr@REDACTED (Rick Reed) Date: Thu, 13 Jun 2013 18:35:34 -0700 Subject: [erlang-questions] child_setup high CPU usage In-Reply-To: References: Message-ID: <51BA7366.30406@whatsapp.com> We use this patch: https://github.com/reedr/otp/commit/b3420b3632c94d4c4869e88f057ca659aa513aea to use closefrom(3) (which is supported on FreeBSD). Rr On 6/13/13 1:04 PM, Ransom Richardson wrote: > To answer at least part of my own question, I think the issue was cause by increasing the ulimit for nofile to 1000000. Here's the child_setup line: > > /usr/local/lib/erlang/erts-5.10.1/bin/child_setup false . exec /bin/sh -s unix:cmd 2>&1 3:999999 50:1 56:0 - > > > ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ > *From:* erlang-questions-bounces@REDACTED on behalf of Ransom Richardson > *Sent:* Thursday, June 13, 2013 3:36 PM > *To:* erlang-questions@REDACTED > *Subject:* [erlang-questions] child_setup high CPU usage > I'm seeing an issue where under what should be light load I see two child_setup processes spawned by Erlang using 100% CPU (combined) and making my machine unresponsive for a number of seconds. > > I recently updated to R16B and never saw this issue on previous releases. > > This happens when processing files sent over ssl from a client. My code uploads the files to S3 (using erlcloud, which uses httpc). If the client uploads only a few files it works fine. But if the client uploads about 6 files in a row then I see the behavior where the two child_setup processes consume 100% CPU. I would expect the only CPU intensive parts of this operation to be the calls to digest and MAC the S3 request. I'm not aware of any calls made to the shell in this scenario. > > I plan to debug further, but would be interested if anyone else has seen this or has ideas as to what could be causing this or how best to start looking. > > thanks, > Ransom > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From jleivent@REDACTED Fri Jun 14 03:38:15 2013 From: jleivent@REDACTED (Jonathan Leivent) Date: Thu, 13 Jun 2013 21:38:15 -0400 Subject: [erlang-questions] Compiler list comprehension optimizations Message-ID: <51BA7407.3060307@comcast.net> I have some questions about compiler optimizations on list comprehensions. The efficiency guide says that list comprehensions that are "neither assigned to a variable, nor passed to another function, nor returned" will be optimized by the compiler to avoid the list construction. However, Dialyzer sometimes complains about such list comprehensions with "Expression produces a value of type ['ok'], but this value is unmatched". So I have developed the habit of assigning to "_", as with: foo(...) -> ..., _ = [bar(X) || ...], .... After re-reading that line in the efficiency guide, it seems like this assignment might actually defeat the above compiler optimization. Does it? Or does the compiler still manage to do the optimization? If this does prevent the optimization, is there a way to get the optimization and prevent the Dialyzer warning while still using a list comprehension? Another question about list comprehension optimizations: If a list being iterated over is generated from a function that just builds the list element-wise, can the compiler inline that function so that the intermediate list isn't built? For instance, if the function is something like the very useful list of all non-empty tails of a list: alltails(L=[_|T]) -> [L|alltails(T)]; alltails([]) -> []. [... || T <- alltails(L), ...] can the compiler generate code that avoids actually building the list of all tails of L? Obviously, one could write alltails in (reversed) tail-recursive form, but I would think that if the compiler has any chance of inlining alltails into that list comprehension to remove the intermediate list, the best chance would be with the simpler non-tail-recursive form. -- Jonathan From n.oxyde@REDACTED Fri Jun 14 09:48:10 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Fri, 14 Jun 2013 09:48:10 +0200 Subject: [erlang-questions] Compiler list comprehension optimizations In-Reply-To: <51BA7407.3060307@comcast.net> References: <51BA7407.3060307@comcast.net> Message-ID: Hello Jonathan, Replied inline. Regards, -- Anthony Ramine Le 14 juin 2013 ? 03:38, Jonathan Leivent a ?crit : > I have some questions about compiler optimizations on list comprehensions. > > The efficiency guide says that list comprehensions that are "neither assigned to a variable, nor passed to another function, nor returned" will be optimized by the compiler to avoid the list construction. > > However, Dialyzer sometimes complains about such list comprehensions with "Expression produces a value of type ['ok'], but this value is unmatched". So I have developed the habit of assigning to "_", as with: > > foo(...) -> > ..., > _ = [bar(X) || ...], > .... > > After re-reading that line in the efficiency guide, it seems like this assignment might actually defeat the above compiler optimization. Does it? Or does the compiler still manage to do the optimization? It doesn't, the compiler knows that "_" is not a variable and thus that the value of the expression ins't used. The match is there solely to silence the Dialyzer warning, to tell him you know what you are doing. > If this does prevent the optimization, is there a way to get the optimization and prevent the Dialyzer warning while still using a list comprehension? > > Another question about list comprehension optimizations: If a list being iterated over is generated from a function that just builds the list element-wise, can the compiler inline that function so that the intermediate list isn't built? For instance, if the function is something like the very useful list of all non-empty tails of a list: > > alltails(L=[_|T]) -> [L|alltails(T)]; > alltails([]) -> []. > > [... || T <- alltails(L), ...] > > can the compiler generate code that avoids actually building the list of all tails of L? > > Obviously, one could write alltails in (reversed) tail-recursive form, but I would think that if the compiler has any chance of inlining alltails into that list comprehension to remove the intermediate list, the best chance would be with the simpler non-tail-recursive form. If you tell the compiler to inline alltails/1, it could do some optimisations but I don't think it is smart enough, at the very least it won't inline anything if you don't tell it so. From thomasl_erlang@REDACTED Fri Jun 14 11:16:45 2013 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Fri, 14 Jun 2013 02:16:45 -0700 (PDT) Subject: [erlang-questions] Compiler list comprehension optimizations In-Reply-To: <51BA7407.3060307@comcast.net> References: <51BA7407.3060307@comcast.net> Message-ID: <1371201405.33856.YahooMailNeo@web140103.mail.bf1.yahoo.com> First, I don't think this is done today.? Second, it's a difficult issue in the general case. Unlike Haskell, Erlang is a strict language, so if you're a stickler for having the program behave the same before and after optimization, you will see that, in the general case, you run a risk of getting different answers if you reorder computations that way. For example, a different exception or different side effects could be generated in the new program. Ensuring the optimization is safe can quickly get intractable to a compiler. (So in practice it might be applied only in rare or simplistic cases.) With that said, I think it could be interesting to investigate this optimization further, perhaps simply by being more relaxed about such issues. It can obviously yield great results in Haskell. Best, Thomas >________________________________ > From: Jonathan Leivent > > >Another question about list comprehension optimizations:? If a list? > >being iterated over is generated from a function that just builds the >list element-wise, can the compiler inline that function so that the >intermediate list isn't built? ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve@REDACTED Fri Jun 14 12:12:46 2013 From: steve@REDACTED (Steve Strong) Date: Fri, 14 Jun 2013 12:12:46 +0200 Subject: [erlang-questions] Records / Proplists / JSON / BSON Message-ID: Hi, We have built a number of projects recently that have a mongodb backend, an HTML / javascript frontend and Erlang in the middle - pretty standard stuff. One of the things the we end up repeating over and over is mapping data from BSON (the mongo format) to Records (our preferred in-memory format for Erlang) and JSON (to send / receive from the browser). To add to the mix, we also like using proplists in configuration files, so have mappings from those to records as well. On the last project I finally got sick of doing it by hand, so knocked up a fairly simple parse transform to take the records (with their type specifications) and generate the mapping code, which has resulted in being able to do things like (note - pseudocode only!): Foo = build_my_record(), mongo:insert(collection, mapper:record_to_bson(Foo)), web_socket:send(Client, mapper:record_to_json(Foo)), receive {client, Response} -> do_stuff_with(mapper:json_to_record(foo, Response) end This has worked very well, and handles about 80% of the types we throw at it. The sorts of types that it doesn't deal with are unions and tuples, e.g. -record(bla, { metadata :: x() | y(), ratio :: {integer(), integer()} }). Due to the value the simple version has had to us, I'm about to embark on a re-work that is going to aim to handle (pretty much) any type you can throw at it, and give full two-directional fidelity on the conversions (e.g., you can assert that Data == xxx_to_record(record_to_xxx(Data)) ). So, a couple of questions: 1. Does such a thing already exist? I'm no fan of re-inventing the wheel :) 2. If not, would anyone be interested in it being open-sourced? 3. If 2., then does anyone have opinions on the functionality / API etc? Interested in any feedback, Cheers, Steve -- Steve Strong Sent with Sparrow (http://www.sparrowmailapp.com/?sig) -------------- next part -------------- An HTML attachment was scrubbed... URL: From q3.development@REDACTED Fri Jun 14 12:19:07 2013 From: q3.development@REDACTED (QDev) Date: Fri, 14 Jun 2013 11:19:07 +0100 Subject: [erlang-questions] driver_create_port overhead and driver_caller In-Reply-To: <4765DCE4-9840-4187-B1A5-D7DD143320AF@btinternet.com> References: <4765DCE4-9840-4187-B1A5-D7DD143320AF@btinternet.com> Message-ID: <415F5924-DB56-45DE-ABD7-C56405030B56@btinternet.com> Is there any advice from OTP team on best way to handle long running task in driver that needs to do ongoing communication with peer Erlang process? Can Erlang process calls outputv and use mutex to talk to async thread (or manual spawned thread) be best? Is locking in outpuv not risk to schedulers if outputv take too long, like with NIF calls? Is there other way - would it be better in outputv to put binary/ErlIOVec directly in driver queue with driver_enqv or driver_enq_bin but then port data lock has to be use to synchronise reads/writes to driver queue anyway. That means more contention if many concurrent threads are working no? So I was thinking better to use mutex per long running thread instead. Also, is it considered necessary to pool worker thread manually? In previous releases, using async thread pool could have detrimental effect on I/O performance - is this still the case? Because if using async thread pool is fine now, maybe easier to do that instead of pooling threads manually, though problem of signalling is still there. Thanks QD On 13 Jun 2013, at 17:06, QDev wrote: > I have linked in driver which deals with embedded library, can make calls some in outputv, some using driver_async, but now have to support some other long running operations requiring thread affinity and (more importantly) need to receive multiple commands/actions from Erlang client code associated with long running transaction in library API. Currently considering to spawn (or pool) some new thread for these (long running tasks) and using ErlDrvMutex and simple queue/array data structure to communicate with it from outputv, but there is also affinity between Erlang callers process id and long running task thread (since caller originates with erlang:send/2 direct to port, which is accessed via ets table), because replies are sent via driver_send_term so need to also store client's process id in ErlDrvTermData and use driver_monitor_process to ensure that if client dies, work is rolled back. IIRC driver_caller not thread safe, so need to only use this in outputv is that still > correct? > > Now, just getting used to driver API changes in R16 (haven't written driver since R13B) and have seen driver_create_port. This seems to create new port running driver code, so wonder if this is better approach for these long running tasks, and what are benefits/advantage of doing this? > > Thanks > QD > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From steve@REDACTED Fri Jun 14 12:34:28 2013 From: steve@REDACTED (Steve Strong) Date: Fri, 14 Jun 2013 12:34:28 +0200 Subject: [erlang-questions] Records / Proplists / JSON / BSON In-Reply-To: <476411371205902@web29f.yandex.ru> References: <476411371205902@web29f.yandex.ru> Message-ID: <2A08C381DCD343CFAFAE62E4B7E3E9A7@srstrong.com> My plan for the actual JSON encode / decode is for the mapper to produce a structure that's compatible with jsx (https://github.com/talentdeficit/jsx) - that's the encoder / decoder that we currently use. If we go the open-source route, I'd probably aim to make that pluggable so that folk can use their preferred JSON encoder. Cheers, Steve -- Steve Strong Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Friday, 14 June 2013 at 12:31, Slava Yurin wrote: > See https://github.com/iskra/jsonx. Maybe it decoder/encoder will help you. > > 14.06.2013, 17:13, "Steve Strong" : > > Hi, > > > > We have built a number of projects recently that have a mongodb backend, an HTML / javascript frontend and Erlang in the middle - pretty standard stuff. One of the things the we end up repeating over and over is mapping data from BSON (the mongo format) to Records (our preferred in-memory format for Erlang) and JSON (to send / receive from the browser). To add to the mix, we also like using proplists in configuration files, so have mappings from those to records as well. > > > > On the last project I finally got sick of doing it by hand, so knocked up a fairly simple parse transform to take the records (with their type specifications) and generate the mapping code, which has resulted in being able to do things like (note - pseudocode only!): > > > > Foo = build_my_record(), > > mongo:insert(collection, mapper:record_to_bson(Foo)), > > web_socket:send(Client, mapper:record_to_json(Foo)), > > > > receive > > {client, Response} -> > > do_stuff_with(mapper:json_to_record(foo, Response) > > end > > > > This has worked very well, and handles about 80% of the types we throw at it. The sorts of types that it doesn't deal with are unions and tuples, e.g. > > > > -record(bla, { > > metadata :: x() | y(), > > ratio :: {integer(), integer()} > > }). > > > > Due to the value the simple version has had to us, I'm about to embark on a re-work that is going to aim to handle (pretty much) any type you can throw at it, and give full two-directional fidelity on the conversions (e.g., you can assert that Data == xxx_to_record(record_to_xxx(Data)) ). > > > > So, a couple of questions: > > > > 1. Does such a thing already exist? I'm no fan of re-inventing the wheel :) > > 2. If not, would anyone be interested in it being open-sourced? > > 3. If 2., then does anyone have opinions on the functionality / API etc? > > > > Interested in any feedback, > > > > Cheers, > > > > Steve > > -- > > Steve Strong > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > > > > > > > , > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From prof3ta@REDACTED Fri Jun 14 13:11:40 2013 From: prof3ta@REDACTED (Roberto Aloi) Date: Fri, 14 Jun 2013 13:11:40 +0200 Subject: [erlang-questions] Unloading all modules from ERL session In-Reply-To: <554fd84a-b7f0-4a5e-af1a-b6b069bd65a2@googlegroups.com> References: <51b97396.515ce00a.0a84.ffffce40@mx.google.com> <554fd84a-b7f0-4a5e-af1a-b6b069bd65a2@googlegroups.com> Message-ID: In Erlang, two and only two versions of the same module can coexist in the run time system (an old version and a new version). Processes running the old version of the code will keep running that version (even if you load a new version of the module) until they make a fully qualified call to the module itself (I.e. M:F(A)). If you add a third version of the code in the system (e.g. you load a module twice) the "very old" version gets purged and those processes executing the obsolete version get killed. You can also use code:purge and code:soft_purge to "force" this behaviour. Look at the "code" module for more options. On 14 Jun 2013 02:29, "Steve Davis" wrote: > Why not make:all([load]). > > ??? > > e.g. I have this in my user_default... > > make() -> > {ok, WorkingDir} = file:get_cwd(), > case file:read_file_info("Emakefile") of > {ok, _} -> io:format("Emake ~p~n", [WorkingDir]); > {error, _} -> io:format("~p~n", [WorkingDir]) > end, > make:all([load]). > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve@REDACTED Fri Jun 14 14:16:54 2013 From: steve@REDACTED (Steve Strong) Date: Fri, 14 Jun 2013 14:16:54 +0200 Subject: [erlang-questions] Records / Proplists / JSON / BSON In-Reply-To: <1371211608.5363.140661243880897.756414FC@webmail.messagingengine.com> References: <476411371205902@web29f.yandex.ru> <2A08C381DCD343CFAFAE62E4B7E3E9A7@srstrong.com> <1371211608.5363.140661243880897.756414FC@webmail.messagingengine.com> Message-ID: Hi Thomas, The first view projects where we did the mapping "by hand" were exactly using record_info, and for very simple records it works just fine. The problems that we had with that approach is that we don't have any type information, so it makes it hard to get some types (atoms, timestamps) to roundtrip to JSON and back with full type fidelity. For example, you have to convert atoms to strings to pass to the browser but on the way back it's impossible to know whether the string you're receiving from the browser should stay a string to be turned back into an atom. From a parse transform with type information, all these transformations can be done automatically, and nested structures can be handled easily. If type information was available at runtime, that would be great but unfortunately the only way to get that is via the parse_transform mechanisms. Cheers, Steve -- Steve Strong Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Friday, 14 June 2013 at 14:06, Thomas Allen wrote: > Hi Steve, > > I've done some similar things, except rather than using a parse > transform to do this (I do not like parse transforms except as a last > resort, I find macros a more explicit way to express compile-time > things) I simply store my record info in application env when my > application starts, and define functions that access these values: > db:fields(RecName), etc. Then, I have higher level functions that use > this metadata. It's easy to test, and I reason it's pretty fast, with > application env being backed by ETS (it's never been a bottleneck). > > %% in .hrl > -define(DB_FIELDS(Model), {Model, record_info(fields, Model)}). > > %% in app:start/0: > application:set_env(db, model_fields, > [?DB_FIELDS(blog_post), > ?DB_FIELDS(contact_message)]), > > %% in record utils > fields() -> > {ok, Fields} = application:get_env(db, model_fields), > Fields. > > fields(Model) -> > proplists:get_value(Model, fields()). > > %% etc. > > Maybe others here with more experience can correct me, but I think some > Erlang developers are too eager to introduce parse transforms to solve > simple problems. > > Thomas > > > On Fri, Jun 14, 2013, at 03:34 AM, Steve Strong wrote: > > My plan for the actual JSON encode / decode is for the mapper to produce > > a structure that's compatible with jsx > > (https://github.com/talentdeficit/jsx) - that's the encoder / decoder > > that we currently use. If we go the open-source route, I'd probably aim > > to make that pluggable so that folk can use their preferred JSON encoder. > > > > Cheers, > > > > Steve > > > > -- > > Steve Strong > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > > > > > On Friday, 14 June 2013 at 12:31, Slava Yurin wrote: > > > > > See https://github.com/iskra/jsonx. Maybe it decoder/encoder will help you. > > > > > > 14.06.2013, 17:13, "Steve Strong" : > > > > Hi, > > > > > > > > We have built a number of projects recently that have a mongodb backend, an HTML / javascript frontend and Erlang in the middle - pretty standard stuff. One of the things the we end up repeating over and over is mapping data from BSON (the mongo format) to Records (our preferred in-memory format for Erlang) and JSON (to send / receive from the browser). To add to the mix, we also like using proplists in configuration files, so have mappings from those to records as well. > > > > > > > > On the last project I finally got sick of doing it by hand, so knocked up a fairly simple parse transform to take the records (with their type specifications) and generate the mapping code, which has resulted in being able to do things like (note - pseudocode only!): > > > > > > > > Foo = build_my_record(), > > > > mongo:insert(collection, mapper:record_to_bson(Foo)), > > > > web_socket:send(Client, mapper:record_to_json(Foo)), > > > > > > > > receive > > > > {client, Response} -> > > > > do_stuff_with(mapper:json_to_record(foo, Response) > > > > end > > > > > > > > This has worked very well, and handles about 80% of the types we throw at it. The sorts of types that it doesn't deal with are unions and tuples, e.g. > > > > > > > > -record(bla, { > > > > metadata :: x() | y(), > > > > ratio :: {integer(), integer()} > > > > }). > > > > > > > > Due to the value the simple version has had to us, I'm about to embark on a re-work that is going to aim to handle (pretty much) any type you can throw at it, and give full two-directional fidelity on the conversions (e.g., you can assert that Data == xxx_to_record(record_to_xxx(Data)) ). > > > > > > > > So, a couple of questions: > > > > > > > > 1. Does such a thing already exist? I'm no fan of re-inventing the wheel :) > > > > 2. If not, would anyone be interested in it being open-sourced? > > > > 3. If 2., then does anyone have opinions on the functionality / API etc? > > > > > > > > Interested in any feedback, > > > > > > > > Cheers, > > > > > > > > Steve > > > > -- > > > > Steve Strong > > > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > > > > > > > > > > > > > > > , > > > > _______________________________________________ > > > > erlang-questions mailing list > > > > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > > > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas@REDACTED Fri Jun 14 14:06:48 2013 From: thomas@REDACTED (Thomas Allen) Date: Fri, 14 Jun 2013 05:06:48 -0700 Subject: [erlang-questions] Records / Proplists / JSON / BSON In-Reply-To: <2A08C381DCD343CFAFAE62E4B7E3E9A7@srstrong.com> References: <476411371205902@web29f.yandex.ru> <2A08C381DCD343CFAFAE62E4B7E3E9A7@srstrong.com> Message-ID: <1371211608.5363.140661243880897.756414FC@webmail.messagingengine.com> Hi Steve, I've done some similar things, except rather than using a parse transform to do this (I do not like parse transforms except as a last resort, I find macros a more explicit way to express compile-time things) I simply store my record info in application env when my application starts, and define functions that access these values: db:fields(RecName), etc. Then, I have higher level functions that use this metadata. It's easy to test, and I reason it's pretty fast, with application env being backed by ETS (it's never been a bottleneck). %% in .hrl -define(DB_FIELDS(Model), {Model, record_info(fields, Model)}). %% in app:start/0: application:set_env(db, model_fields, [?DB_FIELDS(blog_post), ?DB_FIELDS(contact_message)]), %% in record utils fields() -> {ok, Fields} = application:get_env(db, model_fields), Fields. fields(Model) -> proplists:get_value(Model, fields()). %% etc. Maybe others here with more experience can correct me, but I think some Erlang developers are too eager to introduce parse transforms to solve simple problems. Thomas On Fri, Jun 14, 2013, at 03:34 AM, Steve Strong wrote: > My plan for the actual JSON encode / decode is for the mapper to produce > a structure that's compatible with jsx > (https://github.com/talentdeficit/jsx) - that's the encoder / decoder > that we currently use. If we go the open-source route, I'd probably aim > to make that pluggable so that folk can use their preferred JSON encoder. > > Cheers, > > Steve > > -- > Steve Strong > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > > On Friday, 14 June 2013 at 12:31, Slava Yurin wrote: > > > See https://github.com/iskra/jsonx. Maybe it decoder/encoder will help you. > > > > 14.06.2013, 17:13, "Steve Strong" : > > > Hi, > > > > > > We have built a number of projects recently that have a mongodb backend, an HTML / javascript frontend and Erlang in the middle - pretty standard stuff. One of the things the we end up repeating over and over is mapping data from BSON (the mongo format) to Records (our preferred in-memory format for Erlang) and JSON (to send / receive from the browser). To add to the mix, we also like using proplists in configuration files, so have mappings from those to records as well. > > > > > > On the last project I finally got sick of doing it by hand, so knocked up a fairly simple parse transform to take the records (with their type specifications) and generate the mapping code, which has resulted in being able to do things like (note - pseudocode only!): > > > > > > Foo = build_my_record(), > > > mongo:insert(collection, mapper:record_to_bson(Foo)), > > > web_socket:send(Client, mapper:record_to_json(Foo)), > > > > > > receive > > > {client, Response} -> > > > do_stuff_with(mapper:json_to_record(foo, Response) > > > end > > > > > > This has worked very well, and handles about 80% of the types we throw at it. The sorts of types that it doesn't deal with are unions and tuples, e.g. > > > > > > -record(bla, { > > > metadata :: x() | y(), > > > ratio :: {integer(), integer()} > > > }). > > > > > > Due to the value the simple version has had to us, I'm about to embark on a re-work that is going to aim to handle (pretty much) any type you can throw at it, and give full two-directional fidelity on the conversions (e.g., you can assert that Data == xxx_to_record(record_to_xxx(Data)) ). > > > > > > So, a couple of questions: > > > > > > 1. Does such a thing already exist? I'm no fan of re-inventing the wheel :) > > > 2. If not, would anyone be interested in it being open-sourced? > > > 3. If 2., then does anyone have opinions on the functionality / API etc? > > > > > > Interested in any feedback, > > > > > > Cheers, > > > > > > Steve > > > -- > > > Steve Strong > > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > > > > > > > > > > > , > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From thomas@REDACTED Fri Jun 14 14:34:38 2013 From: thomas@REDACTED (Thomas Allen) Date: Fri, 14 Jun 2013 05:34:38 -0700 Subject: [erlang-questions] Records / Proplists / JSON / BSON In-Reply-To: References: <476411371205902@web29f.yandex.ru> <2A08C381DCD343CFAFAE62E4B7E3E9A7@srstrong.com> <1371211608.5363.140661243880897.756414FC@webmail.messagingengine.com> Message-ID: <1371213278.10831.140661243892421.22B444A3@webmail.messagingengine.com> That makes sense. My only hesitation there is that I might eventually need the ability to embed more information than type specifications allow me to: Leaning on type specs feels like a dead end. Thomas On Fri, Jun 14, 2013, at 05:16 AM, Steve Strong wrote: > Hi Thomas, > > The first view projects where we did the mapping "by hand" were exactly > using record_info, and for very simple records it works just fine. > > The problems that we had with that approach is that we don't have any > type information, so it makes it hard to get some types (atoms, > timestamps) to roundtrip to JSON and back with full type fidelity. For > example, you have to convert atoms to strings to pass to the browser but > on the way back it's impossible to know whether the string you're > receiving from the browser should stay a string to be turned back into an > atom. From a parse transform with type information, all these > transformations can be done automatically, and nested structures can be > handled easily. > > If type information was available at runtime, that would be great but > unfortunately the only way to get that is via the parse_transform > mechanisms. > > Cheers, > > Steve > > -- > Steve Strong > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > > On Friday, 14 June 2013 at 14:06, Thomas Allen wrote: > > > Hi Steve, > > > > I've done some similar things, except rather than using a parse > > transform to do this (I do not like parse transforms except as a last > > resort, I find macros a more explicit way to express compile-time > > things) I simply store my record info in application env when my > > application starts, and define functions that access these values: > > db:fields(RecName), etc. Then, I have higher level functions that use > > this metadata. It's easy to test, and I reason it's pretty fast, with > > application env being backed by ETS (it's never been a bottleneck). > > > > %% in .hrl > > -define(DB_FIELDS(Model), {Model, record_info(fields, Model)}). > > > > %% in app:start/0: > > application:set_env(db, model_fields, > > [?DB_FIELDS(blog_post), > > ?DB_FIELDS(contact_message)]), > > > > %% in record utils > > fields() -> > > {ok, Fields} = application:get_env(db, model_fields), > > Fields. > > > > fields(Model) -> > > proplists:get_value(Model, fields()). > > > > %% etc. > > > > Maybe others here with more experience can correct me, but I think some > > Erlang developers are too eager to introduce parse transforms to solve > > simple problems. > > > > Thomas > > > > > > On Fri, Jun 14, 2013, at 03:34 AM, Steve Strong wrote: > > > My plan for the actual JSON encode / decode is for the mapper to produce > > > a structure that's compatible with jsx > > > (https://github.com/talentdeficit/jsx) - that's the encoder / decoder > > > that we currently use. If we go the open-source route, I'd probably aim > > > to make that pluggable so that folk can use their preferred JSON encoder. > > > > > > Cheers, > > > > > > Steve > > > > > > -- > > > Steve Strong > > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > > > > > > > > On Friday, 14 June 2013 at 12:31, Slava Yurin wrote: > > > > > > > See https://github.com/iskra/jsonx. Maybe it decoder/encoder will help you. > > > > > > > > 14.06.2013, 17:13, "Steve Strong" : > > > > > Hi, > > > > > > > > > > We have built a number of projects recently that have a mongodb backend, an HTML / javascript frontend and Erlang in the middle - pretty standard stuff. One of the things the we end up repeating over and over is mapping data from BSON (the mongo format) to Records (our preferred in-memory format for Erlang) and JSON (to send / receive from the browser). To add to the mix, we also like using proplists in configuration files, so have mappings from those to records as well. > > > > > > > > > > On the last project I finally got sick of doing it by hand, so knocked up a fairly simple parse transform to take the records (with their type specifications) and generate the mapping code, which has resulted in being able to do things like (note - pseudocode only!): > > > > > > > > > > Foo = build_my_record(), > > > > > mongo:insert(collection, mapper:record_to_bson(Foo)), > > > > > web_socket:send(Client, mapper:record_to_json(Foo)), > > > > > > > > > > receive > > > > > {client, Response} -> > > > > > do_stuff_with(mapper:json_to_record(foo, Response) > > > > > end > > > > > > > > > > This has worked very well, and handles about 80% of the types we throw at it. The sorts of types that it doesn't deal with are unions and tuples, e.g. > > > > > > > > > > -record(bla, { > > > > > metadata :: x() | y(), > > > > > ratio :: {integer(), integer()} > > > > > }). > > > > > > > > > > Due to the value the simple version has had to us, I'm about to embark on a re-work that is going to aim to handle (pretty much) any type you can throw at it, and give full two-directional fidelity on the conversions (e.g., you can assert that Data == xxx_to_record(record_to_xxx(Data)) ). > > > > > > > > > > So, a couple of questions: > > > > > > > > > > 1. Does such a thing already exist? I'm no fan of re-inventing the wheel :) > > > > > 2. If not, would anyone be interested in it being open-sourced? > > > > > 3. If 2., then does anyone have opinions on the functionality / API etc? > > > > > > > > > > Interested in any feedback, > > > > > > > > > > Cheers, > > > > > > > > > > Steve > > > > > -- > > > > > Steve Strong > > > > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > > > > > > > > > > > > > > > > > > > , > > > > > _______________________________________________ > > > > > erlang-questions mailing list > > > > > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > > > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > > erlang-questions mailing list > > > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > > > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > > > > From lars@REDACTED Fri Jun 14 15:06:33 2013 From: lars@REDACTED (Lars Herbach) Date: Fri, 14 Jun 2013 13:06:33 +0000 Subject: [erlang-questions] If Condition vs. Multiple Function Clauses Message-ID: Hi List, I'm currently working myself through the "?tudes for Erlang" book [1] and in exercise it wks's to write a recursive function, calculating the greatest common divisor for two numbers N and M. The suggested solution is a single gcd/2 function with an If condition and recursion: gcd(M, N) -> if M == N -> M; M > N -> gcd(M - N, N; true -> gcd(M, N - M) end. I by myself took another way, working with multiple function clauses (did I name it right?): gcd(M, N) when M == N -> M; gcd(M, N) when M > N -> gcd(M - N, N); gcd(M, N) -> gcd(M, N - M). Now I've got two questions about that: 1) Is my solution still recursive, since I practically call different functions? 2) Are there any benefits in regards of efficiancy for the first solution? Thanks, Lars. [1]: http://shop.oreilly.com/product/0636920030034.do -------------- next part -------------- An HTML attachment was scrubbed... URL: From freza@REDACTED Fri Jun 14 16:04:23 2013 From: freza@REDACTED (Jachym Holecek) Date: Fri, 14 Jun 2013 10:04:23 -0400 Subject: [erlang-questions] If Condition vs. Multiple Function Clauses In-Reply-To: References: Message-ID: <20130614140423.GA2409@circlewave.net> # Lars Herbach 2013-06-14: > I'm currently working myself through the "?tudes for Erlang" book [1] and in > exercise it wks's to write a recursive function, calculating the greatest > common divisor for two numbers N and M. The suggested solution is a single > gcd/2 function with an If condition and recursion: > > gcd(M, N) -> > if M == N -> M; > M > N -> gcd(M - N, N; > true -> gcd(M, N - M) > end. Not entirely correct: what happens if you invoke 'gcd(-3, 9)' for example? Same applies to the version below. > I by myself took another way, working with multiple function clauses (did I > name it right?): Yes, that's correct naming. Note that multiple clauses are just a convenient way to write things, 'gcd/2' is still just one function. It's instructive to have a look at the result of 'erlc +to_core xxx.erl', that gives you Core Erlang translation of your source -- a slightly more lowlevel representation with less sugar on top and more uniformity. > gcd(M, N) when M == N -> > M; > gcd(M, N) when M > N -> > gcd(M - N, N); > gcd(M, N) -> > gcd(M, N - M). I'd prefer this style myself as well, and from history of this list I think many people would agree. 'If' expressions are quite useful, but in different use cases -- some number of simple checks on a few unrelated values, when you feel splitting the checks to another mini-function doesn't improve clarity (and/or you can't be bothered to invent yet another function name ;-). > Now I've got two questions about that: > 1) Is my solution still recursive, since I practically call different > functions? Certainly recursive, tail-recursive even (doesn't matter in this case though, you have as much stack space as you like -- only relevant consideration is 'bounded' vs. 'unbounded'). You're not calling different function, there's only one 'gcd/2' in your code with a case analysis in it -- just like in the first case. At this level ("what does my code compile down to") the result of running 'erlc +to_asm xxx.erl' is useful -- that gives you BEAM assembly dump, quite readable even if you don't exactly know what some/most instructions mean. > 2) Are there any benefits in regards of efficiancy for the first solution? No, at least in this simple case they compile down to almost the same ASM, I'd expect that to be true pretty much all the time. Recommended to optimize for clarity and simplicity. In the systems Erlang is typically used for efficiency comes from entirely different places than messing about with individual lines, expressions or functions. Unless you have a loadtest proving the contrary for your particular project ;-), and even then optimization wouldn't probably take more than an afternoon, so no reason to worry. Focusing on more global dataflows, load distribution, points of contention, safety of various internal protocols, load mitigation, fault isolation + propagation etc. is better use of your time, but also a bit of an advanced topic -- not sure existing literature covers it much? (Recent discussion between Max Lapshin and Tim Watson illustrates the really important considerations very well, for instance. There were many excellent posts by others as well over time -- sorry, couldn't possibly excavate all the links. :-/) But for completeness -- Here's Erlang efficiency guide: http://www.erlang.org/doc/efficiency_guide/introduction.html HTH, -- Jachym From jleivent@REDACTED Fri Jun 14 16:56:22 2013 From: jleivent@REDACTED (Jonathan Leivent) Date: Fri, 14 Jun 2013 10:56:22 -0400 Subject: [erlang-questions] Compiler list comprehension optimizations In-Reply-To: References: <51BA7407.3060307@comcast.net> Message-ID: <51BB2F16.8070406@comcast.net> On 06/14/2013 03:48 AM, Anthony Ramine wrote: > ... > It doesn't, the compiler knows that "_" is not a variable and thus that the value of the expression ins't used. The match is there solely to silence the Dialyzer warning, to tell him you know what you are doing. Thanks - that's nice to know. -- Jonathan From jleivent@REDACTED Fri Jun 14 20:03:48 2013 From: jleivent@REDACTED (Jonathan Leivent) Date: Fri, 14 Jun 2013 14:03:48 -0400 Subject: [erlang-questions] Compiler list comprehension optimizations In-Reply-To: <1371201405.33856.YahooMailNeo@web140103.mail.bf1.yahoo.com> References: <51BA7407.3060307@comcast.net> <1371201405.33856.YahooMailNeo@web140103.mail.bf1.yahoo.com> Message-ID: <51BB5B04.4090408@comcast.net> On 06/14/2013 05:16 AM, Thomas Lindgren wrote: > First, I don't think this is done today. > > Second, it's a difficult issue in the general case. Unlike Haskell, Erlang is a strict language, so if you're a stickler for having the program behave the same before and after optimization, you will see that, in the general case, you run a risk of getting different answers if you reorder computations that way. For example, a different exception or different side effects could be generated in the new program. Ensuring the optimization is safe can quickly get intractable to a compiler. (So in practice it might be applied only in rare or simplistic cases.) > > With that said, I think it could be interesting to investigate this optimization further, perhaps simply by being more relaxed about such issues. It can obviously yield great results in Haskell. > > Best, > Thomas OK - I'm trying to develop a feel for when list comprehensions can be used to make code more readable without fear that they also might make that code slower and/or more wasteful of memory. I obviously wouldn't want the Erlang compiler to introduce an optimization that re-ordered side-effects - I just assumed it would be able to tell when functions are side-effect-free in many simple cases (and presume that they might have side effects in the more complex cases). Maybe the particular case of iterating over tails of a list is itself useful enough to warrant an addition to the Erlang list comprehension syntax - maybe something like: [... || Tail <+ List, ...] That would, for example, make code that generates all 2-combinations of elements from a list very easy to read: [{X, Y} || [X|Rest] <+ List, Y <- Rest] One could also imagine a more general case, with an initial expression, continue condition, and next expression, kind of like a for loop common to other languages: [... || I := 0; I < 100; I + 1, ...] -- Jonathan From lars@REDACTED Fri Jun 14 21:32:49 2013 From: lars@REDACTED (Lars Herbach) Date: Fri, 14 Jun 2013 19:32:49 +0000 Subject: [erlang-questions] If Condition vs. Multiple Function Clauses In-Reply-To: <20130614140423.GA2409@circlewave.net> References: <20130614140423.GA2409@circlewave.net> Message-ID: Thank you very much! I'm currently diving into Erlang, coming from Python and C and you really helped me getting a feeling for the language. I BTW really enjoy the atmosphere and tone in this mailing list, I know much worse. Thanks for that. -Lars On Jun 14, 2013, at 16:04, Jachym Holecek wrote > > HTH, > -- Jachym From ingela.andin@REDACTED Fri Jun 14 23:51:16 2013 From: ingela.andin@REDACTED (Ingela Andin) Date: Fri, 14 Jun 2013 23:51:16 +0200 Subject: [erlang-questions] SSL/TLS transport customization Message-ID: Due to an issue that came up on the EUC, I just want to clearify that the Erlang ssl application now (even before R16B01) will let you customize the transport protocol in such a way that you do not need to use the inet-driver. From the documentation: "transportoption() = {cb_info, {CallbackModule::atom(), DataTag::atom(), ClosedTag::atom(), ErrTag:atom()}} - defaults to {gen_tcp, tcp, tcp_closed, tcp_error}. Can be used to customize the transport layer. The callback module must implement a reliable transport protocol and behave as gen_tcp and in addition have functions corresponding to inet:setopts/2, inet:getopts/2, inet:peername/1, inet:sockname/1 and inet:port/1. The callback gen_tcp is treated specially and will call inet directly." We plan to formalize the API and extend the documentation of it, but it is there now and can be used. Regards Ingela Erlang/OTP team - Ericsson AB -------------- next part -------------- An HTML attachment was scrubbed... URL: From nem@REDACTED Sat Jun 15 00:17:23 2013 From: nem@REDACTED (Geoff Cant) Date: Fri, 14 Jun 2013 15:17:23 -0700 Subject: [erlang-questions] SSL/TLS transport customization In-Reply-To: References: Message-ID: <01517416-B138-4E4A-B14F-8BA666C95542@erlang.geek.nz> Will this support be something like a new stdlib module akin to (the API of) https://github.com/extend/ranch/blob/master/src/ranch_transport.erl ? (maybe call them stream_transport and dgram_transport) Once something like this existed, I'd be highly tempted to write all my libraries to use stream_transport - it'd save re-inventing it all the time. Hopefully the following people would be able to delete some code too: * Ranch https://github.com/extend/ranch/blob/master/src/ranch_transport.erl https://github.com/extend/ranch/blob/master/src/ranch_tcp.erl https://github.com/extend/ranch/blob/master/src/ranch_ssl.erl * Hackney https://github.com/benoitc/hackney/blob/master/src/hackney_tcp_transport.erl https://github.com/benoitc/hackney/blob/master/src/hackney_ssl_transport.erl * Mochiweb https://github.com/mochi/mochiweb/blob/master/src/mochiweb_socket.erl * Inets ;p https://github.com/erlang/otp/blob/master/lib/inets/src/http_lib/http_transport.erl * lhttpc https://github.com/esl/lhttpc/blob/master/src/lhttpc_sock.erl * Misultin https://github.com/ostinelli/misultin/blob/master/src/misultin_socket.erl * RabbitMQ (and common library) https://github.com/mkurkov/rabbit_common/blob/master/src/rabbit_net.erl * Stomperl https://github.com/mplatov/stomperl/blob/master/src/socket_wrapper.erl * Open Server Platform https://github.com/ranok/open-server-platform/blob/master/src/osp_socket.erl I could go on :) Cheers, -- Geoff Cant On 2013-06-14, at 14:51 , Ingela Andin wrote: > Due to an issue that came up on the EUC, I just want to clearify that the > Erlang ssl application now (even before R16B01) will let you customize the > transport protocol in such a way that you do not need to use the > inet-driver. From the documentation: > > "transportoption() = {cb_info, {CallbackModule::atom(), DataTag::atom(), > ClosedTag::atom(), ErrTag:atom()}} - defaults to {gen_tcp, tcp, tcp_closed, > tcp_error}. Can be used to customize the transport layer. The callback > module must implement a reliable transport protocol and behave as gen_tcp > and in addition have functions corresponding to inet:setopts/2, > inet:getopts/2, inet:peername/1, inet:sockname/1 and inet:port/1. The > callback gen_tcp is treated specially and will call inet directly." > > We plan to formalize the API and extend the documentation of it, but it is > there now and can be used. > > Regards Ingela Erlang/OTP team - Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From nem@REDACTED Sat Jun 15 00:35:42 2013 From: nem@REDACTED (Geoff Cant) Date: Fri, 14 Jun 2013 15:35:42 -0700 Subject: [erlang-questions] SSL/TLS transport customization In-Reply-To: <01517416-B138-4E4A-B14F-8BA666C95542@erlang.geek.nz> References: <01517416-B138-4E4A-B14F-8BA666C95542@erlang.geek.nz> Message-ID: <13F20E50-E3F6-4164-8C08-1A0D939ECE6B@erlang.geek.nz> On 2013-06-14, at 15:17 , Geoff Cant wrote: > Will this support be something like a new stdlib module akin to (the API of) https://github.com/extend/ranch/blob/master/src/ranch_transport.erl ? (maybe call them stream_transport and dgram_transport) > In a future glorious utopian OTP we'd also have EEP29[1] abstract patterns, so an implementation of stream_transport could provide the right receive patterns for {active, _} messages. Anyone want a place on my Xmas (or holiday of your choice) card list[2]? :) -- Geoff Cant [1] EEP29: http://www.erlang.org/eeps/eep-0029.html [2] This year's list currently stands at: Steve Vinoski for sys:get_state and sys:replace_state :D From steven.charles.davis@REDACTED Sat Jun 15 02:32:40 2013 From: steven.charles.davis@REDACTED (Steve Davis) Date: Fri, 14 Jun 2013 17:32:40 -0700 (PDT) Subject: [erlang-questions] Unloading all modules from ERL session In-Reply-To: References: <51b97396.515ce00a.0a84.ffffce40@mx.google.com> <554fd84a-b7f0-4a5e-af1a-b6b069bd65a2@googlegroups.com> Message-ID: <3fd4e198-33a2-4441-bafe-9de939ce463b@googlegroups.com> Hi Rob, Good point and awesomely true, and your advice provides invaluable insight into the caution needed when releasing new code to long-running systems in prod (and the need for code change callback implementations). However, I've found that during development I rarely need that level of control, so make:all([load]) is more than enough for most purposes in dev that I have faced. "as simple as possible, but no simpler" I guess the trick is to decide what is simple enough for the purpose. /s On Friday, June 14, 2013 6:11:40 AM UTC-5, Roberto Aloi wrote: > > In Erlang, two and only two versions of the same module can coexist in the > run time system (an old version and a new version). Processes running the > old version of the code will keep running that version (even if you load a > new version of the module) until they make a fully qualified call to the > module itself (I.e. M:F(A)). If you add a third version of the code in the > system (e.g. you load a module twice) the "very old" version gets purged > and those processes executing the obsolete version get killed. You can also > use code:purge and code:soft_purge to "force" this behaviour. Look at the > "code" module for more options. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.charles.davis@REDACTED Sat Jun 15 02:40:06 2013 From: steven.charles.davis@REDACTED (Steve Davis) Date: Fri, 14 Jun 2013 17:40:06 -0700 (PDT) Subject: [erlang-questions] If Condition vs. Multiple Function Clauses In-Reply-To: References: Message-ID: <137be0d1-e1cd-4ab9-af1b-5f9707a2f860@googlegroups.com> I strongly (personally) dislike use of "if" syntax, but only because of the counterintuitive "true" evaluation. What if the language were to return the atom "else" instead of "true"? I doubt that would be considered "better" but it would sure make things more readable... /s On Friday, June 14, 2013 8:06:33 AM UTC-5, Lars Herbach wrote: > > Hi List, > I'm currently working myself through the "?tudes for Erlang" book [1] and > in exercise it wks's to write a recursive function, calculating the > greatest common divisor for two numbers N and M. The suggested solution is > a single gcd/2 function with an If condition and recursion: > > gcd(M, N) -> > if M == N -> M; > M > N -> gcd(M - N, N; > true -> gcd(M, N - M) > end. > > I by myself took another way, working with multiple function clauses > (did I name it right?): > > gcd(M, N) when M == N -> > M; > gcd(M, N) when M > N -> > gcd(M - N, N); > gcd(M, N) -> > gcd(M, N - M). > > Now I've got two questions about that: > 1) Is my solution still recursive, since I practically call different > functions? > 2) Are there any benefits in regards of efficiancy for the first solution? > > Thanks, > Lars. > > > [1]: http://shop.oreilly.com/product/0636920030034.do > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mahesh@REDACTED Sat Jun 15 10:43:00 2013 From: mahesh@REDACTED (Mahesh Paolini-Subramanya) Date: Sat, 15 Jun 2013 10:43:00 +0200 Subject: [erlang-questions] If Condition vs. Multiple Function Clauses In-Reply-To: <137be0d1-e1cd-4ab9-af1b-5f9707a2f860@googlegroups.com> References: <137be0d1-e1cd-4ab9-af1b-5f9707a2f860@googlegroups.com> Message-ID: <243427C0-0D51-4114-8FAF-6537AD50DEE7@dieswaytoofast.com> Via LYSE and ROK - and with the forlorn hope that this doesn't start up again :-) --> It may be more FAMILIAR, but that doesn't mean 'else' is a good thing. I know that writing '; true ->' is a very easy way to get 'else' in Erlang, but we have a couple of decades of psychology-of-programming results to show that it's a bad idea. I have started to replace: by if X > Y -> a() if X > Y -> a() ; true -> b() ; X =< Y -> b() end end if X > Y -> a() if X > Y -> a() ; X < Y -> b() ; X < Y -> b() ; true -> c() ; X ==Y -> c() end end which I find mildly annoying when _writing_ the code but enormously helpful when _reading_ it. Speaking for myself, I've pretty much gotten to the point where I don't notice it. Heck, I barely notice syntax anymore. About the best analogy I can come up with is spoken languages (adjectives after nouns in Italian? The Horror! The Horror!). It may seem strange at first, but you don't even notice it after a few days? Cheers Mahesh Paolini-Subramanya That Tall Bald Indian Guy... Google+ | Blog | Twitter | LinkedIn On Jun 15, 2013, at 2:40 AM, Steve Davis wrote: > I strongly (personally) dislike use of "if" syntax, but only because of the counterintuitive "true" evaluation. > > What if the language were to return the atom "else" instead of "true"? > > I doubt that would be considered "better" but it would sure make things more readable... > > /s > > On Friday, June 14, 2013 8:06:33 AM UTC-5, Lars Herbach wrote: > Hi List, > I'm currently working myself through the "?tudes for Erlang" book [1] and in exercise it wks's to write a recursive function, calculating the greatest common divisor for two numbers N and M. The suggested solution is a single gcd/2 function with an If condition and recursion: > > gcd(M, N) -> > if M == N -> M; > M > N -> gcd(M - N, N; > true -> gcd(M, N - M) > end. > > I by myself took another way, working with multiple function clauses (did I name it right?): > > gcd(M, N) when M == N -> > M; > gcd(M, N) when M > N -> > gcd(M - N, N); > gcd(M, N) -> > gcd(M, N - M). > > Now I've got two questions about that: > 1) Is my solution still recursive, since I practically call different functions? > 2) Are there any benefits in regards of efficiancy for the first solution? > > Thanks, > Lars. > > > [1]: http://shop.oreilly.com/product/0636920030034.do > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Sat Jun 15 10:51:12 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Sat, 15 Jun 2013 10:51:12 +0200 Subject: [erlang-questions] If Condition vs. Multiple Function Clauses In-Reply-To: <0B9298A1-A9E6-40CD-9EBF-242EDCBC89DC@gmail.com> References: <137be0d1-e1cd-4ab9-af1b-5f9707a2f860@googlegroups.com> <0B9298A1-A9E6-40CD-9EBF-242EDCBC89DC@gmail.com> Message-ID: <4637ED23-1BF7-4DFD-9F07-EEF93D82F1BD@gmail.com> Failed to reply to all? -- Anthony Ramine Le 15 juin 2013 ? 10:29, Anthony Ramine a ?crit : > Hello Steve, > > No thanks. > > If something is to be done, it should be to allow a literal 'otherwise' atom in the last 'if' clause pattern. In Haskell, otherwise is an alias of true just for that reason. > > Regards, > > -- > Anthony Ramine > > Le 15 juin 2013 ? 02:40, Steve Davis a ?crit : > >> I strongly (personally) dislike use of "if" syntax, but only because of the counterintuitive "true" evaluation. >> >> What if the language were to return the atom "else" instead of "true"? >> >> I doubt that would be considered "better" but it would sure make things more readable... >> >> /s >> >> On Friday, June 14, 2013 8:06:33 AM UTC-5, Lars Herbach wrote: >> Hi List, >> I'm currently working myself through the "?tudes for Erlang" book [1] and in exercise it wks's to write a recursive function, calculating the greatest common divisor for two numbers N and M. The suggested solution is a single gcd/2 function with an If condition and recursion: >> >> gcd(M, N) -> >> if M == N -> M; >> M > N -> gcd(M - N, N; >> true -> gcd(M, N - M) >> end. >> >> I by myself took another way, working with multiple function clauses (did I name it right?): >> >> gcd(M, N) when M == N -> >> M; >> gcd(M, N) when M > N -> >> gcd(M - N, N); >> gcd(M, N) -> >> gcd(M, N - M). >> >> Now I've got two questions about that: >> 1) Is my solution still recursive, since I practically call different functions? >> 2) Are there any benefits in regards of efficiancy for the first solution? >> >> Thanks, >> Lars. >> >> >> [1]: http://shop.oreilly.com/product/0636920030034.do >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From steven.charles.davis@REDACTED Sat Jun 15 15:30:10 2013 From: steven.charles.davis@REDACTED (Steve Davis) Date: Sat, 15 Jun 2013 08:30:10 -0500 Subject: [erlang-questions] If Condition vs. Multiple Function Clauses In-Reply-To: References: <137be0d1-e1cd-4ab9-af1b-5f9707a2f860@googlegroups.com> Message-ID: Yep, that would be a better, more consistent way to go... if M == N -> a(); M > N -> b(); _ -> c() end. :-) ty Erik /s On Jun 15, 2013, at 2:42 AM, Erik S?e S?rensen wrote: > Me too. One might even consider using "_" for such a catch-all. > (A reason in favour of that, apart from the syntactic similarity with "case"-catch-alls: 'else' is a valid expression like 'true', '_' isn't and therefore doesn't confuse by the apparent overlap.) > > Den 15/06/2013 02.40 skrev "Steve Davis" : > I strongly (personally) dislike use of "if" syntax, but only because of the counterintuitive "true" evaluation. > > What if the language were to return the atom "else" instead of "true"? > > I doubt that would be considered "better" but it would sure make things more readable... > > /s > > On Friday, June 14, 2013 8:06:33 AM UTC-5, Lars Herbach wrote: > Hi List, > I'm currently working myself through the "?tudes for Erlang" book [1] and in exercise it wks's to write a recursive function, calculating the greatest common divisor for two numbers N and M. The suggested solution is a single gcd/2 function with an If condition and recursion: > > gcd(M, N) -> > if M == N -> M; > M > N -> gcd(M - N, N; > true -> gcd(M, N - M) > end. > > I by myself took another way, working with multiple function clauses (did I name it right?): > > gcd(M, N) when M == N -> > M; > gcd(M, N) when M > N -> > gcd(M - N, N); > gcd(M, N) -> > gcd(M, N - M). > > Now I've got two questions about that: > 1) Is my solution still recursive, since I practically call different functions? > 2) Are there any benefits in regards of efficiancy for the first solution? > > Thanks, > Lars. > > > [1]: http://shop.oreilly.com/product/0636920030034.do > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yoursurrogategod@REDACTED Sat Jun 15 15:57:20 2013 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Sat, 15 Jun 2013 09:57:20 -0400 Subject: [erlang-questions] What happens with atoms across different systems? Message-ID: Hello, I'm reading the book Erlang and OTP in Action and came across this part: "In Erlang, an atom is a special kind of string constant that is identified only by the characters in the string, so that two atoms are always considered to be exactly the same if they have the same character representation. But internally, these strings are stored in a table and are referred to by the table index, so that checking atoms for equivalence at runtime amounts to comparing two small integers; and each time you use an atom, it takes up only one word of memory. (The index number used for any particular atom is automatically assigned at runtime and can vary from one run of the system to the next; there is no way, and no need, for the user to know this.)" Now, this got me thinking. What if I have two systems that send messages to one another. I update one and include a new atom that I'm using. This atom gets sent to system #2 that cannot recognize it just yet. Does that mean that that message will sit in the receiving thread's queue until it can recognize it or can be handled/thrown away? As a follow-up question, does it make sense to use atoms in this fashion? Thanks in advance, --Yves -------------- next part -------------- An HTML attachment was scrubbed... URL: From jdavid.eisenberg@REDACTED Sat Jun 15 16:18:13 2013 From: jdavid.eisenberg@REDACTED (J David Eisenberg) Date: Sat, 15 Jun 2013 07:18:13 -0700 Subject: [erlang-questions] If Condition vs. Multiple Function Clauses Message-ID: > > Message: 8 > Date: Fri, 14 Jun 2013 13:06:33 +0000 > From: Lars Herbach > To: "erlang-questions@REDACTED" > Subject: [erlang-questions] If Condition vs. Multiple Function Clauses > Message-ID: > Content-Type: text/plain; charset="iso-8859-1" > > Hi List, > I'm currently working myself through the "?tudes for Erlang" book [1] and in exercise it wks's to write a recursive function, calculating the greatest common divisor for two numbers N and M. The suggested solution is a single gcd/2 function with an If condition and recursion: > > gcd(M, N) -> > if M == N -> M; > M > N -> gcd(M - N, N; > true -> gcd(M, N - M) > end. > > I by myself took another way, working with multiple function clauses (did I name it right?): > > gcd(M, N) when M == N -> > M; > gcd(M, N) when M > N -> > gcd(M - N, N); > gcd(M, N) -> > gcd(M, N - M). > You're right. Your solution is much more in the spirit of Erlang. I wrote the ?tude with "if" because I wanted to have an example of its use somewhere in the book. I will update the book to put in your solution as an alternate answer, with some words about using guards in preference to "if".. > Now I've got two questions about that: > 1) Is my solution still recursive, since I practically call different functions? > 2) Are there any benefits in regards of efficiancy for the first solution? > > Thanks, > Lars. > > > [1]: http://shop.oreilly.com/product/0636920030034.do From yoursurrogategod@REDACTED Sat Jun 15 17:17:01 2013 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Sat, 15 Jun 2013 11:17:01 -0400 Subject: [erlang-questions] What are the differences between beam files produced by c(...) and erlc? Message-ID: This is the code that I'm working with just for fun: -module(my_module). -export([pie/0]). pie() -> 3.14. Now, in the Erlang shell, I do c(my_module). and get a .beam file that's 540 bytes. Ok, no problem. Now, if I run erlc my_module.erl, I get the same .beam file that 604 bytes! Why? Shouldn't they be the same? If not, what are the differences? I'm running CentOS 6.4 32-bit and Erlang version R16B. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bob@REDACTED Sat Jun 15 17:46:04 2013 From: bob@REDACTED (Bob Ippolito) Date: Sat, 15 Jun 2013 08:46:04 -0700 Subject: [erlang-questions] What happens with atoms across different systems? In-Reply-To: References: Message-ID: Sending the message will create the atom on the other side if it doesn't already exist. The semantics of how long it sits in the receiver's mailbox depends only on the code running in that process. On Saturday, June 15, 2013, Yves S. Garret wrote: > Hello, > > I'm reading the book Erlang and OTP in Action and came across this part: > "In Erlang, an atom is a special kind of string constant that is > identified only by the > characters in the string, so that two atoms are always considered to be > exactly the > same if they have the same character representation. But internally, these > strings > are stored in a table and are referred to by the table index, so that > checking atoms > for equivalence at runtime amounts to comparing two small integers; and > each time > you use an atom, it takes up only one word of memory. (The index number > used for > any particular atom is automatically assigned at runtime and can vary from > one run > of the system to the next; there is no way, and no need, for the user to > know this.)" > > Now, this got me thinking. What if I have two systems that send messages > to one > another. I update one and include a new atom that I'm using. This atom > gets sent > to system #2 that cannot recognize it just yet. Does that mean that that > message > will sit in the receiving thread's queue until it can recognize it or can > be > handled/thrown away? > > As a follow-up question, does it make sense to use atoms in this fashion? > > Thanks in advance, > --Yves > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sat Jun 15 18:12:44 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 15 Jun 2013 20:12:44 +0400 Subject: [erlang-questions] What are the differences between beam files produced by c(...) and erlc? In-Reply-To: References: Message-ID: Post both of them here in binary form. I suppose, that there differ some not-important chunks like debug_info. From yoursurrogategod@REDACTED Sat Jun 15 18:20:13 2013 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Sat, 15 Jun 2013 12:20:13 -0400 Subject: [erlang-questions] What are the differences between beam files produced by c(...) and erlc? In-Reply-To: References: Message-ID: On Sat, Jun 15, 2013 at 12:12 PM, Max Lapshin wrote: > Post both of them here in binary form. I suppose, that there differ > some not-important chunks like debug_info. > Ok, here they are attached. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: c_hello_world.beam Type: application/octet-stream Size: 576 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: erlc_hello_world.beam Type: application/octet-stream Size: 640 bytes Desc: not available URL: From max.lapshin@REDACTED Sat Jun 15 18:27:59 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 15 Jun 2013 20:27:59 +0400 Subject: [erlang-questions] What are the differences between beam files produced by c(...) and erlc? In-Reply-To: References: Message-ID: 1> beam_lib:info("c_hello_world.beam"). [{file,"c_hello_world.beam"}, {module,hello_world}, {chunks,[{"Atom",20,61}, {"Code",92,79}, {"StrT",180,0}, {"ImpT",188,40}, {"ExpT",236,40}, {"LitT",284,38}, {"LocT",332,4}, {"Attr",344,40}, {"CInf",392,141}, {"Abst",544,0}, {"Line",552,22}]}] 2> beam_lib:info("erlc_hello_world.beam"). [{file,"erlc_hello_world.beam"}, {module,hello_world}, {chunks,[{"Atom",20,61}, {"Code",92,79}, {"StrT",180,0}, {"ImpT",188,40}, {"ExpT",236,40}, {"LitT",284,38}, {"LocT",332,4}, {"Attr",344,40}, {"CInf",392,205}, {"Abst",608,0}, {"Line",616,22}]}] CInf are different. 6> {ok, {hello_world,[{"CInf",CHelloWorld}]}} = beam_lib:chunks("c_hello_world.beam", ["CInf"]). {ok,{hello_world,[{"CInf", <<131,108,0,0,0,4,104,2,100,0,7,111,112,116,105,111, 110,115,106,104,2,100,...>>}]}} 7> erlang:binary_to_term(CHelloWorld). [{options,[]}, {version,"4.9.1"}, {time,{2013,6,15,16,19,23}}, {source,"/home/ysg/Documents/Development/tweann/source/hello_world.erl"}] 8> {ok, {hello_world,[{"CInf",ERLCHelloWorld}]}} = beam_lib:chunks("erlc_hello_world.beam", ["CInf"]). {ok,{hello_world,[{"CInf", <<131,108,0,0,0,4,104,2,100,0,7,111,112,116,105,111, 110,115,108,0,0,0,...>>}]}} 9> erlang:binary_to_term(ERLCHelloWorld). [{options,[{outdir,"/home/ysg/Documents/Development/tweann/source"}]}, {version,"4.9.1"}, {time,{2013,6,15,16,18,33}}, {source,"/home/ysg/Documents/Development/tweann/source/hello_world.erl"}] So, here is the difference: erlc variant has different compile options and include outdir. From yubao.liu@REDACTED Sat Jun 15 18:07:58 2013 From: yubao.liu@REDACTED (yubao.liu@REDACTED) Date: Sat, 15 Jun 2013 09:07:58 -0700 (PDT) Subject: [erlang-questions] how to complete module name and function name in emacs erlang-shell-mode? Message-ID: <1371312478.78941.YahooMailIosMobile@web162904.mail.bf1.yahoo.com> Hi,

I use the emacs extension shipped in latest erlang release under Mac OS, in terminal, i can press TAB key to complete module name and function name in erl command, but with emacs in the erlang shell window showed by pressing C-c C-z, the auto complete feature is gone.

i can use M-x term to run erl, it can auto complete, but this buffer is separate from the "erlang IDE", for example, compiling erlang code in a buffer doesn't reflect in this separate erl instance.

Distel might be helpful, but it looks no update for a long time, haven't tried it.

Regards,
Yubao Liu


Sent from Yahoo! Mail for iPhone -------------- next part -------------- An HTML attachment was scrubbed... URL: From tcury@REDACTED Sat Jun 15 21:27:08 2013 From: tcury@REDACTED (Tiago Cury) Date: Sat, 15 Jun 2013 16:27:08 -0300 Subject: [erlang-questions] mnesia:select returning wrong number of columns Message-ID: <41A3E3DE-A85C-4955-ACE1-6D01082CA0F3@ymail.com> -record(user, {id, name, password}). F = fun() -> User = #user{_ = '$1', name= '$2', password= '$3'}, mnesia:select(user, [{User, [], ['$1','$2','$3']}]) end. mnesia:transaction(F). {atomic,["admin"]} Why is it returning just one column other than three ($1,$2,$3) ? From calleja.justin@REDACTED Sat Jun 15 22:56:34 2013 From: calleja.justin@REDACTED (Justin Calleja) Date: Sat, 15 Jun 2013 22:56:34 +0200 Subject: [erlang-questions] eaop Message-ID: Hi all, In case anyone is interested, there is a parse transform project to do compile time AOP in Erlang: https://github.com/justin-calleja/eaop Not sure if I'm suppose to post emails like this here but I have seen similar emails. Please let me know if I'm not. I was going to send this email after adding more documentation but I'm not sure when I'll be getting round to doing that... there is some documentation in the wiki pages on github. This is just a pet project I wrote because I needed more functionality from: http://sourceforge.net/projects/erlaop/ for another prototype project. (However, eaop is a complete re-write of ErlAOP. eaop re-implements the features in ErlAOP using the erl_syntax module and adds a couple of features of its own). cheers, Justin -------------- next part -------------- An HTML attachment was scrubbed... URL: From tcury@REDACTED Sat Jun 15 23:21:45 2013 From: tcury@REDACTED (Tiago Cury) Date: Sat, 15 Jun 2013 18:21:45 -0300 Subject: [erlang-questions] mnesia:select returning wrong number of columns In-Reply-To: <41A3E3DE-A85C-4955-ACE1-6D01082CA0F3@ymail.com> References: <41A3E3DE-A85C-4955-ACE1-6D01082CA0F3@ymail.com> Message-ID: <18E5893B-F58D-4618-95D8-3F7B90008C70@ymail.com> I missed []. The correct line is: F = fun() -> User = #user{_='$1',name='$2',password='$3'},mnesia:select(user, [{User, [], [['$1','$2','$3']]}]) end. On Jun 15, 2013, at 4:27 PM, Tiago Cury wrote: > -record(user, {id, name, password}). > > F = fun() -> User = #user{_ = '$1', name= '$2', password= '$3'}, mnesia:select(user, [{User, [], ['$1','$2','$3']}]) end. > mnesia:transaction(F). > > {atomic,["admin"]} > > Why is it returning just one column other than three ($1,$2,$3) ? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Sat Jun 15 14:21:25 2013 From: ok@REDACTED (ok@REDACTED) Date: Sun, 16 Jun 2013 00:21:25 +1200 Subject: [erlang-questions] Compiler list comprehension optimizations In-Reply-To: <51BB5B04.4090408@comcast.net> References: <51BA7407.3060307@comcast.net> <1371201405.33856.YahooMailNeo@web140103.mail.bf1.yahoo.com> <51BB5B04.4090408@comcast.net> Message-ID: <7c70418ea0bbaa919c3ed2befba9be70.squirrel@chasm.otago.ac.nz> > On 06/14/2013 05:16 AM, Thomas Lindgren wrote: > > One could also imagine a more general case, with an initial expression, > continue condition, and next expression, kind of like a for loop common > to other languages: > > [... || I := 0; I < 100; I + 1, ...] We have seen [... where I = 0 then I+1 while I < 100] proposed, but that's not actually as general as a functional language should be. What's needed is a general 'unfold' proposal, and we've had that for a while too. Semicolons in Erlang always have either the actual or the metaphorical force of "OR", so they make no sense in this context. From ok@REDACTED Sat Jun 15 14:33:16 2013 From: ok@REDACTED (ok@REDACTED) Date: Sun, 16 Jun 2013 00:33:16 +1200 Subject: [erlang-questions] If Condition vs. Multiple Function Clauses In-Reply-To: <137be0d1-e1cd-4ab9-af1b-5f9707a2f860@googlegroups.com> References: <137be0d1-e1cd-4ab9-af1b-5f9707a2f860@googlegroups.com> Message-ID: <1f56bd31c939e8da35becf5ac428bda9.squirrel@chasm.otago.ac.nz> > I strongly (personally) dislike use of "if" syntax, but only because of > the > counterintuitive "true" evaluation. What's counterintuitive about it? It's instantly familiar to any Lisp programmer (except for being spelled 'true' instead of 'T') or to any Prolog programmer. > > What if the language were to return the atom "else" instead of "true"? Now that _would_ be unintuitive. In any case, 'if' has no truck with values of any kind, it is driven by guards that succeed or fail, not by expressions that have values. >> gcd(M, N) -> >> if M == N -> M; >> M > N -> gcd(M - N, N; >> true -> gcd(M, N - M) >> end. That is indeed a common Erlang style, but in a language that uses both commas and semicolons so heavily, and especially in a language that is lacking an 'elif' or 'else' keyword, I find it advisible to put semicolons in a place where commas never go, namely at the beginning of lines. It is also easier to read if the arrows are aligned. gcd(M, N) -> if M > N -> gcd(M - N, N) ; N < M -> gcd(M, N - M) ; true -> M end. >> >> I by myself took another way, working with multiple function clauses >> (did I name it right?): >> >> gcd(M, N) when M == N -> >> M; >> gcd(M, N) when M > N -> >> gcd(M - N, N); >> gcd(M, N) -> >> gcd(M, N - M). >> >> Now I've got two questions about that: >> 1) Is my solution still recursive, since I practically call different >> functions? Yes your solution is still recursive. In all essentials, it is the *same* solution. No, you are *NOT* calling different functions. There is only one gcd/2 function, and you are calling *it* (not *them*). In fact you *can't* call a clause. >> 2) Are there any benefits in regards of efficiency for the first >> solution? If you really want to know, measure them. But it would not be surprising if the compiler generated essentially the same code. Anyone who cared about efficiency that much would be using a M rem N rather than M - N, or would be using a binary gcd. First you write something that is obviously correct. Then you test it and fix the mistakes. Then you can measure it, and consider a rewrite if it is too slow. From publicityifl@REDACTED Sun Jun 16 20:58:47 2013 From: publicityifl@REDACTED (publicityifl@REDACTED) Date: Sun, 16 Jun 2013 11:58:47 -0700 (PDT) Subject: [erlang-questions] Call for Papers IFL 2013 Message-ID: <51be0ae7.ee49b40a.672c.ffff9bde@mx.google.com> Hello, Please, find below the second call for papers for IFL 2013. Please forward these to anyone you think may be interested. Apologies for any duplicates you may receive. best regards, Jurriaan Hage Publicity Chair of IFL CALL FOR PAPERS 25th SYMPOSIUM ON IMPLEMENTATION AND APPLICATION OF FUNCTIONAL LANGUAGES - IFL 2013 RADBOUD UNIVERSITY NIJMEGEN, THE NETHERLANDS ACM In-Cooperation / ACM SIGPLAN AUGUST 28 - 30 2013 "Landgoed Holthurnsche Hof" http://ifl2013.cs.ru.nl We are proud to announce that the 25th edition of the IFL series returns to its roots at the Radboud University Nijmegen in the Netherlands. The symposium is held from 28th to 30th of August 2013. Scope ----- The goal of the IFL symposia is to bring together researchers actively engaged in the implementation and application of functional and function-based programming languages. IFL 2013 will be a venue for researchers to present and discuss new ideas and concepts, work in progress, and publication-ripe results related to the implementation and application of functional languages and function-based programming. Following the IFL tradition, IFL 2013 will use a post-symposium review process to produce the formal proceedings which will be published in the ACM Digital Library. All participants of IFL 2013 are invited to submit either a draft paper or an extended abstract describing work to be presented at the symposium. At no time may work submitted to IFL be simultaneously submitted to other venues; submissions must adhere to ACM SIGPLAN's republication policy: http://www.sigplan.org/Resources/Policies/Republication The submissions will be screened by the program committee chair to make sure they are within the scope of IFL, and will appear in the draft proceedings distributed at the symposium. Submissions appearing in the draft proceedings are not peer-reviewed publications. Hence, publications that appear only in the draft proceedings do not count as publication for the ACM SIGPLAN republication policy. After the symposium, authors will be given the opportunity to incorporate the feedback from discussions at the symposium and will be invited to submit a revised full article for the formal review process. From the revised submissions, the program committee will select papers for the formal proceedings considering their correctness, novelty, originality, relevance, significance, and clarity. Invited Speaker --------------- Lennart Augustsson, currently employed by the Standard Chartered Bank, well-known for his work on Haskell, parallel Haskell, Cayenne, and Bluespec, is the invited speaker of IFL 2013. He will be talking about practical applications of functional programming. Submission Details ------------------ Submission deadline draft papers: July 31 Notification of acceptance for presentation: August 2 Early registration deadline: August 7 Late registration deadline: August 14 Submission deadline for pre-symposium proceedings: August 21 25th IFL Symposium: August 28-30 Submission deadline for post-symposium proceedings: November 11 Notification of acceptance for post-symposium proceedings: December 18 Camera-ready version for post-symposium proceedings: February 3 2014 Prospective authors are encouraged to submit papers or extended abstracts to be published in the draft proceedings and to present them at the symposium. All contributions must be written in English. Papers must adhere to the standard ACM two columns conference format. For the pre-symposium proceedings we adopt a 'weak' page limit of 12 pages. For the post-symposium proceedings the page limit of 12 pages is firm. A suitable document template for LaTeX can be found at: http://www.acm.org/sigs/sigplan/authorInformation.htm Papers are to be submitted via the conference's EasyChair submission page: https://www.easychair.org/conferences/?conf=ifl2013 Topics ------ IFL welcomes submissions describing practical and theoretical work as well as submissions describing applications and tools in the context of functional programming. If you are not sure whether your work is appropriate for IFL 2013, please contact the PC chair at rinus@REDACTED Topics of interest include, but are not limited to: ? language concepts ? type systems, type checking, type inferencing ? compilation techniques ? staged compilation ? run-time function specialization ? run-time code generation ? partial evaluation ? (abstract) interpretation ? metaprogramming ? generic programming ? automatic program generation ? array processing ? concurrent/parallel programming ? concurrent/parallel program execution ? embedded systems ? web applications ? (embedded) domain specific languages ? security ? novel memory management techniques ? run-time profiling performance measurements ? debugging and tracing ? virtual/abstract machine architectures ? validation, verification of functional programs ? tools and programming techniques ? (industrial) applications Peter Landin Prize ------------------ The Peter Landin Prize is awarded to the best paper presented at the symposium every year. The honoured article is selected by the program committee based on the submissions received for the formal review process. The prize carries a cash award equivalent to 150 Euros. Programme committee ------------------- ? Thomas Arts, Quviq, Gothenburg, Sweden ? Andrew Butterfield, Trinity College, Dublin, Ireland ? Edwin Brady, University of St. Andrews, UK ? Clemens Grelck, University of Amsterdam, Netherlands ? Adam Granicz, IntelliFactory, Budapest, Hungary ? Jeremy Gibbons, University of Oxford, UK ? Fritz Henglein, University of Copenhagen, Denmark ? Stephan Herhut, Intel Labs, Santa Clara, US ? Ralf Hinze (co-chair), University of Oxford, UK ? Zolt?n Horv?th, E?tv?s Lor?nd University, Budapest, Hungary ? Zhenjiang Hu, University of Tokyo, Japan ? Mauro Jaskelioff, Universidad Nacional de Rosario, Argentina ? Johan Jeuring, University of Utrecht, Netherlands ? Rita Loogen, University of Marburg, Germany ? Marco T. Moraz?n, Seton Hall University, New Jersey, US ? Dominic Orchard, University of Cambridge, UK ? Rinus Plasmeijer (chair), Radboud University Nijmegen, Netherlands ? Tim Sheard, Portland State University, US ? Sam Tobin-Hochstadt, Northeastern University / Indiana University, US ? Peter Thiemann, University of Freiburg, Germany ? Simon Thompson, University of Kent, UK Venue ----- The 25th IFL is organized by the Radboud University Nijmegen, Model Based Software Development Department at the Nijmegen Institute for Computing and Information Sciences. The event is held in the Landgoed ?Holthurnsche Hof?, a rural estate in the woodlands surrounding Nijmegen. It can be reached quickly and easily by public transport. From jleivent@REDACTED Sun Jun 16 21:14:12 2013 From: jleivent@REDACTED (Jonathan Leivent) Date: Sun, 16 Jun 2013 15:14:12 -0400 Subject: [erlang-questions] mnesia question - fastest way to tell if ordered_set table is empty or not Message-ID: <51BE0E84.7010207@comcast.net> What is the fastest way, avoiding locking, to tell if an ordered_set mnesia table is empty or not? I'm assuming that if the table is not an ordered set, then: mnesia:activity(ets, fun mnesia:first/1, [Table]) == '$end_of_table' is the fastest. However, if the table is an ordered set, this seems like it could take O(logN) time. Would this be faster?: mnesia:activity(ets, fun mnesia:table_info/2, [Table, size]) == 0 Is there a faster alternative? -- Jonathan From chandrashekhar.mullaparthi@REDACTED Sun Jun 16 23:24:35 2013 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Sun, 16 Jun 2013 22:24:35 +0100 Subject: [erlang-questions] Getting chunked responses with ibrowse In-Reply-To: References: Message-ID: Hi Chris, Can you turn on ibrowse tracing and send me the trace output? What you are trying to do should be possible, though I'm not sure if you can have a timeout value that large (days/months). Chandru On 11 June 2013 04:13, Chris Hicks wrote: > Hello all, > > I've been playing around with ibrowse, trying to get chunked responses > from a server and the behavior I'm seeing isn't at all what I expect given > the documentation. > > When I send the following request: > > ibrowse:send_req(URL, [], get, [], [{stream_to, self()}]). > > I get the headers back from the server immediately...and then no messages > after that, until the timeout message anyway. From what the documentation > says, I'd expect to just continue to get messages "as data arrives on the > socket" but this apparently isn't the case. > > If I call stream_next(Request_id) nothing happens at all. However, if I > instead change the call to: > > ibrowse:send_req(URL, [], get, [], [{stream_to, {self(), once}}]). > > I get the headers, and then when I call stream_next(Request_id) I suddenly > begin to get the packets the server is sending. > > Can anyone explain to me why I am seeing what I am seeing? > > The use case I am trying to fulfill is that I connect to a server and get > a set of data back that describes the state of the server as a json object. > On any changes, which might not happen for a very long time (days/months), > a new set of data is streamed to the client. I would like to have one > process receiving multiple streams, from multiple servers, handling these > status changes. Is this possible, or will I need to spin up workers for > each stream and have them wait for new data, providing it to that central > process? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Sun Jun 16 23:25:38 2013 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Sun, 16 Jun 2013 22:25:38 +0100 Subject: [erlang-questions] mnesia question - fastest way to tell if ordered_set table is empty or not In-Reply-To: <51BE0E84.7010207@comcast.net> References: <51BE0E84.7010207@comcast.net> Message-ID: Have you tried this? mnesia:table_info(Table_name, size). Chandru On 16 June 2013 20:14, Jonathan Leivent wrote: > What is the fastest way, avoiding locking, to tell if an ordered_set > mnesia table is empty or not? I'm assuming that if the table is not an > ordered set, then: > > mnesia:activity(ets, fun mnesia:first/1, [Table]) == '$end_of_table' > > is the fastest. However, if the table is an ordered set, this seems like > it could take O(logN) time. > > Would this be faster?: > > mnesia:activity(ets, fun mnesia:table_info/2, [Table, size]) == 0 > > Is there a faster alternative? > > -- Jonathan > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Sun Jun 16 23:48:21 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 17 Jun 2013 09:48:21 +1200 Subject: [erlang-questions] Records / Proplists / JSON / BSON In-Reply-To: <1371213278.10831.140661243892421.22B444A3@webmail.messagingengine.com> References: <476411371205902@web29f.yandex.ru> <2A08C381DCD343CFAFAE62E4B7E3E9A7@srstrong.com> <1371211608.5363.140661243880897.756414FC@webmail.messagingengine.com> <1371213278.10831.140661243892421.22B444A3@webmail.messagingengine.com> Message-ID: <1F0CBAAA-1207-44A5-8337-5D07A2A6FDF4@cs.otago.ac.nz> On 15/06/2013, at 12:34 AM, Thomas Allen wrote: > That makes sense. My only hesitation there is that I might eventually > need the ability to embed more information than type specifications > allow me to: Leaning on type specs feels like a dead end. > There's another approach. Define your own "record" language, and generate the Erlang code (*including* -record declarations) from that. Your "language" could just a data structure like {record, foo, [ {Field,Type,...}, ... }. which could be enriched with whatever hints you want for JSON, BSON, or whatever else takes your fancy. From jleivent@REDACTED Sun Jun 16 23:49:41 2013 From: jleivent@REDACTED (Jonathan Leivent) Date: Sun, 16 Jun 2013 17:49:41 -0400 Subject: [erlang-questions] mnesia question - fastest way to tell if ordered_set table is empty or not In-Reply-To: References: <51BE0E84.7010207@comcast.net> Message-ID: <51BE32F5.9050804@comcast.net> On 06/16/2013 05:25 PM, Chandru wrote: > Have you tried this? > > mnesia:table_info(Table_name, size). > > Chandru That's the alternative I mentioned below, just with an ets activity wrapped around it. I just don't know if table_info-size is going to get the answer fast in all cases. For instance, if it actually has to iterate to count the records. Or if it collects more info than just size, and then returns only size because that's all that's being asked for. Or if it has to wait for write locks on the table to get dropped, even though it is wrapped in an ets activity. I forgot to mention that the table is local_content, if that makes any difference. > > On 16 June 2013 20:14, Jonathan Leivent wrote: > >> What is the fastest way, avoiding locking, to tell if an ordered_set >> mnesia table is empty or not? I'm assuming that if the table is not an >> ordered set, then: >> >> mnesia:activity(ets, fun mnesia:first/1, [Table]) == '$end_of_table' >> >> is the fastest. However, if the table is an ordered set, this seems like >> it could take O(logN) time. >> >> Would this be faster?: >> >> mnesia:activity(ets, fun mnesia:table_info/2, [Table, size]) == 0 >> >> Is there a faster alternative? >> >> -- Jonathan From tcury@REDACTED Mon Jun 17 00:16:23 2013 From: tcury@REDACTED (Tiago Cury) Date: Sun, 16 Jun 2013 19:16:23 -0300 Subject: [erlang-questions] erl_tar:add continue on error In-Reply-To: <7FA4BC38-D002-4E8E-B5E0-B8DF15B46BC1@ymail.com> References: <50D49D74-A1FA-4CF6-B210-1A7D157011A3@ymail.com> <7FA4BC38-D002-4E8E-B5E0-B8DF15B46BC1@ymail.com> Message-ID: <70C1D9DB-9C94-4CB8-9C7F-A82EB1CAC23B@ymail.com> Just to share: https://github.com/curysoft/Backuper From ok@REDACTED Mon Jun 17 01:19:31 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 17 Jun 2013 11:19:31 +1200 Subject: [erlang-questions] If Condition vs. Multiple Function Clauses In-Reply-To: References: <137be0d1-e1cd-4ab9-af1b-5f9707a2f860@googlegroups.com> Message-ID: On 16/06/2013, at 1:30 AM, Steve Davis wrote: > Yep, that would be a better, more consistent way to go... > > if M == N -> a(); > M > N -> b(); > _ -> c() > end. Let's see, this says "Should the test M == N succeed, compute a(). Otherwise, should the test M >N succeed, compute b(). Otherwise, take a new variable that has never been given any value whatsoever; should that variable, contrary to all sane expectation, somehow magically turn out to be bound to a succeeding test, compute c()." Just how is the "_" supposed to get a value here? By the way, anyone who is really suffering from the lack of 'else' can just -define(else, true). and then if M < N -> c() ; M > N -> b() ; ?else -> a() end and be done with it, although putting M == N there instead would make it more obvious to one's readers, should one _have_ readers. From zambal@REDACTED Mon Jun 17 07:01:21 2013 From: zambal@REDACTED (Vincent Siliakus) Date: Sun, 16 Jun 2013 22:01:21 -0700 (PDT) Subject: [erlang-questions] Records / Proplists / JSON / BSON In-Reply-To: <1F0CBAAA-1207-44A5-8337-5D07A2A6FDF4@cs.otago.ac.nz> References: <476411371205902@web29f.yandex.ru> <2A08C381DCD343CFAFAE62E4B7E3E9A7@srstrong.com> <1371211608.5363.140661243880897.756414FC@webmail.messagingengine.com> <1371213278.10831.140661243892421.22B444A3@webmail.messagingengine.com> <1F0CBAAA-1207-44A5-8337-5D07A2A6FDF4@cs.otago.ac.nz> Message-ID: Op zondag 16 juni 2013 23:48:21 UTC+2 schreef Richard A. O'Keefe het volgende: > > > There's another approach. > > Define your own "record" language, > and generate the Erlang code (*including* -record declarations) from that. > > And if you go down that route, be sure to first checkout the Piqi project > at : > http://piqi.org -vincent -------------- next part -------------- An HTML attachment was scrubbed... URL: From jinni.park@REDACTED Mon Jun 17 10:17:14 2013 From: jinni.park@REDACTED (Park, Sungjin) Date: Mon, 17 Jun 2013 17:17:14 +0900 Subject: [erlang-questions] Can I prevent lager from changing to sync mode? In-Reply-To: <20130613174319.GR31341@hijacked.us> References: <20130613174319.GR31341@hijacked.us> Message-ID: I badly want that feature though... On Fri, Jun 14, 2013 at 2:43 AM, Andrew Thompson wrote: > On Wed, Jun 12, 2013 at 10:12:46AM +0900, Park, Sungjin wrote: > > I want it never fall to sync mode. > > Just dropping some logs is ok. > > Right now, for anything other than error_logger messages, lager will not > drop any messages, ever. The sync/async modes are just for mailbox size > regulation. > > Andrew > -- Park, Sungjin ------------------------------------------------------------------------------------------------------------------- Peculiar travel suggestions are dancing lessons from god. -- The Books of Bokonon ------------------------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve@REDACTED Mon Jun 17 10:29:44 2013 From: steve@REDACTED (Steve Strong) Date: Mon, 17 Jun 2013 10:29:44 +0200 Subject: [erlang-questions] Records / Proplists / JSON / BSON In-Reply-To: References: <476411371205902@web29f.yandex.ru> <2A08C381DCD343CFAFAE62E4B7E3E9A7@srstrong.com> <1371211608.5363.140661243880897.756414FC@webmail.messagingengine.com> <1371213278.10831.140661243892421.22B444A3@webmail.messagingengine.com> <1F0CBAAA-1207-44A5-8337-5D07A2A6FDF4@cs.otago.ac.nz> Message-ID: <453B7A2683804A30A2690DAC4FD7062B@srstrong.com> Thanks for the pointers to piqi, it looks close to what I was looking to build. However, I've been playing a little and can't seem to express some of the constructs that we regularly use, for example we use tuples quite a lot for simple things such as ratios, so I may have a record such as: -record(foo, { ratio :: {integer(), integer()}, %% other stuff here }). piqi certainly doesn't have support for these out of the box, but I'm also struggling to see how I can define one as an extension. I'm trying to provide a mapping from an Erlang tuple to a piqi list; it wouldn't be perfect, since piqi doesn't know the list should be precisely 2 long, but it would be good enough. I can get piqi to make calls to custom code where I can do the tuple <-> list mapping, but the generated .hrl file ends up with the primitive piqi type (so list(integer()) ) rather than my erlang type ({integer(), integer()}). This would likely upset Dialyzer quite a lot ;) . Any piqi gurus out there have any hints? Cheers, Steve -- Steve Strong Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Monday, 17 June 2013 at 07:01, Vincent Siliakus wrote: > Op zondag 16 juni 2013 23:48:21 UTC+2 schreef Richard A. O'Keefe het volgende: > > > > There's another approach. > > > > Define your own "record" language, > > and generate the Erlang code (*including* -record declarations) from that. > > > And if you go down that route, be sure to first checkout the Piqi project at : > http://piqi.org > > -vincent > > > > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Mon Jun 17 10:31:53 2013 From: mjtruog@REDACTED (Michael Truog) Date: Mon, 17 Jun 2013 01:31:53 -0700 Subject: [erlang-questions] Can I prevent lager from changing to sync mode? In-Reply-To: References: <20130613174319.GR31341@hijacked.us> Message-ID: <51BEC979.1030405@gmail.com> If you wanted, there is a logger here which drops messages if the source process log frequency hits every 10 microseconds within a 10 second window. That prevents a misbehaving process from causing excessive logging, such that the misbehaving process is ignored during the rest of the 10 second window. https://github.com/okeuday/CloudI/blob/master/src/lib/cloudi_core/src/cloudi_logger.erl On 06/17/2013 01:17 AM, Park, Sungjin wrote: > I badly want that feature though... > > > > > On Fri, Jun 14, 2013 at 2:43 AM, Andrew Thompson > wrote: > > On Wed, Jun 12, 2013 at 10:12:46AM +0900, Park, Sungjin wrote: > > I want it never fall to sync mode. > > Just dropping some logs is ok. > > Right now, for anything other than error_logger messages, lager will not > drop any messages, ever. The sync/async modes are just for mailbox size > regulation. > > Andrew > > > > > -- > Park, Sungjin > ------------------------------------------------------------------------------------------------------------------- > Peculiar travel suggestions are dancing lessons from god. > -- The Books of Bokonon > ------------------------------------------------------------------------------------------------------------------- > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Mon Jun 17 12:42:57 2013 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Mon, 17 Jun 2013 11:42:57 +0100 Subject: [erlang-questions] mnesia question - fastest way to tell if ordered_set table is empty or not In-Reply-To: <51BE32F5.9050804@comcast.net> References: <51BE0E84.7010207@comcast.net> <51BE32F5.9050804@comcast.net> Message-ID: On 16 June 2013 22:49, Jonathan Leivent wrote: > On 06/16/2013 05:25 PM, Chandru wrote: > >> Have you tried this? >> >> mnesia:table_info(Table_name, size). >> >> Chandru >> > > That's the alternative I mentioned below, just with an ets activity > wrapped around it. I just don't know if table_info-size is going to get > the answer fast in all cases. For instance, if it actually has to iterate > to count the records. Or if it collects more info than just size, and then > returns only size because that's all that's being asked for. Or if it has > to wait for write locks on the table to get dropped, even though it is > wrapped in an ets activity. > > I forgot to mention that the table is local_content, if that makes any > difference. Yes, sorry I didn't notice that. AFAIK, table info such as size is metadata which is maintained for each table, and not something which is determined each time you issue the command. Chandru > > > >> On 16 June 2013 20:14, Jonathan Leivent wrote: >> >> What is the fastest way, avoiding locking, to tell if an ordered_set >>> mnesia table is empty or not? I'm assuming that if the table is not an >>> ordered set, then: >>> >>> mnesia:activity(ets, fun mnesia:first/1, [Table]) == '$end_of_table' >>> >>> is the fastest. However, if the table is an ordered set, this seems like >>> it could take O(logN) time. >>> >>> Would this be faster?: >>> >>> mnesia:activity(ets, fun mnesia:table_info/2, [Table, size]) == 0 >>> >>> Is there a faster alternative? >>> >>> -- Jonathan >>> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dch@REDACTED Mon Jun 17 13:03:12 2013 From: dch@REDACTED (Dave Cottlehuber) Date: Mon, 17 Jun 2013 13:03:12 +0200 Subject: [erlang-questions] about GIS In-Reply-To: References: Message-ID: On 7 June 2013 16:22, Yendris Cruz Mendoza wrote: > Hi , is possible that exist any open source software about GIS implemented > in Erlang?..for example, Web Map Services, Tile Map Services or Web > processing services especifications. Sorry my English...regards Apache CouchDB, with @vmx's GeoCouch[2] extension, might be what you're after. [1]: http://couchdb.apache.org/ [2]: https://github.com/couchbase/geocouch/ Feel free to ask more questions on the couchdb mailing list. A+ Dave From post@REDACTED Mon Jun 17 13:01:51 2013 From: post@REDACTED (sm) Date: Mon, 17 Jun 2013 13:01:51 +0200 Subject: [erlang-questions] edoc generation Message-ID: <51BEEC9F.1050000@gothensitz.com> I've just ran edoc on a collection of erlang source files with the command: erl -noshell -run edoc_run application 'app' '"src"' '[{dir, "src"}]' The source files contain the erlang library. A few libraries gave this result: src/tls_record.erl, function is_acceptable_version/1: at line 477: multiple @spec tag. edoc: skipping source file 'src/tls_record.erl': {'EXIT',error}. src/orber_ifr_aliasdef.erl: at line 61: syntax error before: '?tcheck' edoc: skipping source file 'src/orber_ifr_aliasdef.erl': {'EXIT',error}. src/orber_ifr_idltype.erl: at line 46: syntax error before: '?tcheck' edoc: skipping source file 'src/orber_ifr_idltype.erl': {'EXIT',error}. src/mnesia_power_SUITE.erl: at line 37: syntax error before: mnesia edoc: skipping source file 'src/mnesia_power_SUITE.erl': {'EXIT',error}. src/ssl.erl, function start/0: at line 80: multiple @spec tag. edoc: skipping source file 'src/ssl.erl': {'EXIT',error}. src/distr_startup_SUITE.erl: at line 68: syntax error before: Nodes edoc: skipping source file 'src/distr_startup_SUITE.erl': {'EXIT',error}. src/orber_ifr_arraydef.erl: at line 53: syntax error before: '?tcheck' edoc: skipping source file 'src/orber_ifr_arraydef.erl': {'EXIT',error}. src/diameter_capx.erl: at line 120: syntax error before: ';' edoc: skipping source file 'src/diameter_capx.erl': {'EXIT',error}. src/orber_ifr_interfacedef.erl: at line 85: syntax error before: '?tcheck' edoc: skipping source file 'src/orber_ifr_interfacedef.erl': {'EXIT',error}. src/tls.erl, function start/0: at line 88: multiple @spec tag. edoc: skipping source file 'src/tls.erl': {'EXIT',error}. src/orber_ifr_operationdef.erl: at line 72: syntax error before: '?tcheck' edoc: skipping source file 'src/orber_ifr_operationdef.erl': {'EXIT',error}. src/dialyzer_utils.erl, function pp_hook/3: at line 456: @spec arity does not match. edoc: skipping source file 'src/dialyzer_utils.erl': {'EXIT',error}. 2543- fatal: {endtag_does_not_match,{was,code,should_have_been,ocde}} src/ct_gen_conn.erl, function start/4: at line 61: error in XML parser: {fatal,{{endtag_does_not_match, {was,code,should_have_been,ocde}}, {file,file_name_unknown}, {line,86},{col,61}}}. edoc: skipping source file 'src/ct_gen_conn.erl': {'EXIT',error}. src/ct_framework.erl, function add_data_dir/2: at line 1331: @spec arity does not match. edoc: skipping source file 'src/ct_framework.erl': {'EXIT',error}. src/orber_ifr_constantdef.erl: at line 65: syntax error before: '?tcheck' edoc: skipping source file 'src/orber_ifr_constantdef.erl': {'EXIT',error}. src/orber_ifr_fixeddef.erl: at line 48: syntax error before: '?tcheck' edoc: skipping source file 'src/orber_ifr_fixeddef.erl': {'EXIT',error}. src/public_key.erl, function pem_entry_decode/1: at line 86: @spec arity does not match. edoc: skipping source file 'src/public_key.erl': {'EXIT',error}. src/orber_ifr_exceptiondef.erl: at line 64: syntax error before: '?tcheck' edoc: skipping source file 'src/orber_ifr_exceptiondef.erl': {'EXIT',error}. src/diameter_dbg.erl: at line 116: syntax error before: '.' edoc: skipping source file 'src/diameter_dbg.erl': {'EXIT',error}. src/orber_ifr_repository.erl: at line 65: syntax error before: '?tcheck' edoc: skipping source file 'src/orber_ifr_repository.erl': {'EXIT',error}. src/zip.erl: at line 216: syntax error before: do_openzip_open edoc: skipping source file 'src/zip.erl': {'EXIT',error}. src/ct_run.erl, function run_test/1: at line 872: syntax error before: '--' edoc: skipping source file 'src/ct_run.erl': {'EXIT',error}. src/ct_util.erl, function start/0: at line 80: @spec arity does not match. edoc: skipping source file 'src/ct_util.erl': {'EXIT',error}. src/orber_ifr_enumdef.erl: at line 61: syntax error before: '?tcheck' edoc: skipping source file 'src/orber_ifr_enumdef.erl': {'EXIT',error}. edoc: skipping source file 'src/calendar.erl': {'EXIT',{{badmatch,{atom,354,'?SECONDS_PER_DAY'}},[{edoc_specs,d2e,1,[{file,[101,100|...]},{line,339}]},{edoc_specs,type,2,[{file,[101|...]},{line,65}]},{edoc_extract,'-get_tags/6-lc$^0/1-0-',2,[{file,[...]},{line,...}]},{edoc_extract,get_tags,6,[{file,...},{...}]},{edoc_extract,get_tags,6,[{...}|...]},{edoc_extract,source1,5,[...]},{edoc_doclet,source,9,...},{lists,foldl,...}]}}. src/edoc_types.erl, function is_predefined/2: at line 32: file not found: edoc_types.hrl edoc: skipping source file 'src/edoc_types.erl': {'EXIT',error}. 2543- fatal: {endtag_does_not_match,{was,p,should_have_been,'N'}} src/ct_repeat.erl, in module header: at line 20: error in XML parser: {fatal, {{endtag_does_not_match,{was,p,should_have_been,'N'}}, {file,file_name_unknown},{line,27},{col,33}}}. edoc: skipping source file 'src/ct_repeat.erl': {'EXIT',error}. src/ct_logs.erl, function init/2: at line 78: @spec arity does not match. edoc: skipping source file 'src/ct_logs.erl': {'EXIT',error}. src/corba_request.erl: at line 278: syntax error before: '?query_check' edoc: skipping source file 'src/corba_request.erl': {'EXIT',error}. src/orber_ifr_typecode.erl: at line 67: syntax error before: '?tcheck' edoc: skipping source file 'src/orber_ifr_typecode.erl': {'EXIT',error}. src/orber_ifr_stringdef.erl: at line 48: syntax error before: '?tcheck' edoc: skipping source file 'src/orber_ifr_stringdef.erl': {'EXIT',error}. src/orber_ifr_attributedef.erl: at line 64: syntax error before: '?tcheck' edoc: skipping source file 'src/orber_ifr_attributedef.erl': {'EXIT',error}. src/orber_ifr_wstringdef.erl: at line 46: syntax error before: '?tcheck' edoc: skipping source file 'src/orber_ifr_wstringdef.erl': {'EXIT',error}. src/crypto.erl, function hmac/3: at line 267: multiple @spec tag. src/crypto.erl, function hmac/3: at line 266: multiple @spec tag. src/crypto.erl, function hmac/3: at line 265: multiple @spec tag. src/crypto.erl, function hmac/3: at line 264: multiple @spec tag. src/crypto.erl, function hmac/3: at line 263: multiple @spec tag. edoc: skipping source file 'src/crypto.erl': {'EXIT',error}. src/edoc_doclet.erl, function run/2: at line 38: file not found: edoc_doclet.hrl edoc: skipping source file 'src/edoc_doclet.erl': {'EXIT',error}. src/ssl_manager.erl, function register_session/3: at line 156: multiple @spec tag. edoc: skipping source file 'src/ssl_manager.erl': {'EXIT',error}. src/megaco_ber_bin_drv_encoder.erl: at line 277: head mismatch edoc: skipping source file 'src/megaco_ber_bin_drv_encoder.erl': {'EXIT',error}. src/orber_ifr_sequencedef.erl: at line 53: syntax error before: '?tcheck' edoc: skipping source file 'src/orber_ifr_sequencedef.erl': {'EXIT',error}. src/ssl_pkix_db.erl, function cache_pem_file/2: at line 118: multiple @spec tag. edoc: skipping source file 'src/ssl_pkix_db.erl': {'EXIT',error}. src/orber_ifr_uniondef.erl: at line 67: syntax error before: '?tcheck' edoc: skipping source file 'src/orber_ifr_uniondef.erl': {'EXIT',error}. src/orber_ifr_structdef.erl: at line 60: syntax error before: '?tcheck' edoc: skipping source file 'src/orber_ifr_structdef.erl': {'EXIT',error}. src/orber_ifr_primitivedef.erl: at line 46: syntax error before: '?tcheck' edoc: skipping source file 'src/orber_ifr_primitivedef.erl': {'EXIT',error}. src/orber_ifr_moduledef.erl: at line 64: syntax error before: '?tcheck' edoc: skipping source file 'src/orber_ifr_moduledef.erl': {'EXIT',error}. edoc: error in doclet 'edoc_doclet': {'EXIT',error}. edoc: edoc terminated abnormally: error. I've placed all the source file in one directory. I've also tested the some libray files in there orginal directory with the same results. How can I resolve this? SM. From steven.charles.davis@REDACTED Mon Jun 17 15:06:29 2013 From: steven.charles.davis@REDACTED (Steve Davis) Date: Mon, 17 Jun 2013 08:06:29 -0500 Subject: [erlang-questions] If Condition vs. Multiple Function Clauses In-Reply-To: References: <137be0d1-e1cd-4ab9-af1b-5f9707a2f860@googlegroups.com> Message-ID: <52D268D1-0C79-4A75-9D1D-5E171797851A@gmail.com> Hi Richard, Yep, you changed my mind (again). I wasn't thinking about this correctly. Thanks very much for your replies, Steve On Jun 16, 2013, at 6:19 PM, "Richard A. O'Keefe" wrote: > > On 16/06/2013, at 1:30 AM, Steve Davis wrote: > >> Yep, that would be a better, more consistent way to go... >> >> if M == N -> a(); >> M > N -> b(); >> _ -> c() >> end. > > Let's see, this says > "Should the test M == N succeed, compute a(). > Otherwise, should the test M >N succeed, compute b(). > Otherwise, take a new variable that has never been given > any value whatsoever; should that variable, contrary to > all sane expectation, somehow magically turn out to be > bound to a succeeding test, compute c()." > > Just how is the "_" supposed to get a value here? > > By the way, anyone who is really suffering from the lack of 'else' > can just > -define(else, true). > and then > if M < N -> c() > ; M > N -> b() > ; ?else -> a() > end > and be done with it, although putting M == N there instead would > make it more obvious to one's readers, should one _have_ readers. > From yashgt@REDACTED Mon Jun 17 15:48:47 2013 From: yashgt@REDACTED (yashgt@REDACTED) Date: Mon, 17 Jun 2013 13:48:47 +0000 Subject: [erlang-questions] Sending environment values for rebar-packaged apps Message-ID: <51bf13d5.0256310a.375c.ffffe1af@mx.google.com> An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Mon Jun 17 15:58:47 2013 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 17 Jun 2013 15:58:47 +0200 Subject: [erlang-questions] Sending environment values for rebar-packaged apps In-Reply-To: <51bf13d5.0256310a.375c.ffffe1af@mx.google.com> References: <51bf13d5.0256310a.375c.ffffe1af@mx.google.com> Message-ID: <1371477527.4753.37.camel@sekic1152.rnd.ki.sw.ericsson.se> Greetings, "erl - key1 value1 key2 value2 ..." bengt On Mon, 2013-06-17 at 13:48 +0000, yashgt@REDACTED wrote: > Hi, > > If I have to launch a node with certain environment values I usually > do: > "erl -pa ebin -run appname -env [{myvar, 1234}]" > > If I package this as a node using rebar, I can start the node using: > sh rel/mynode/bin/mynode start > > But how do I set values for the app-specific enviroment vars? I do not > want to set them in a file as the values will change for every > invocation and hence I want to send them on the command prompt. > > Thanks, > Yash > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From yashgt@REDACTED Mon Jun 17 16:15:55 2013 From: yashgt@REDACTED (Yash Ganthe) Date: Mon, 17 Jun 2013 07:15:55 -0700 (PDT) Subject: [erlang-questions] Sending environment values for rebar-packaged apps In-Reply-To: <1371477527.4753.37.camel@sekic1152.rnd.ki.sw.ericsson.se> References: <51bf13d5.0256310a.375c.ffffe1af@mx.google.com> <1371477527.4753.37.camel@sekic1152.rnd.ki.sw.ericsson.se> Message-ID: This is when I start my node using erl. My question is about when I start the node as a package(without using erl). On Jun 17, 6:58?pm, Bengt Kleberg wrote: > Greetings, > > "erl - key1 value1 key2 value2 ..." > > bengt > > > > > > > > > > On Mon, 2013-06-17 at 13:48 +0000, yas...@REDACTED wrote: > > Hi, > > > If I have to launch a node with certain environment values I usually > > do: > > "erl -pa ebin -run appname -env [{myvar, 1234}]" > > > If I package this as a node using rebar, I can start the node using: > > sh rel/mynode/bin/mynode start > > > But how do I set values for the app-specific enviroment vars? I do not > > want to set them in a file as the values will change for every > > invocation and hence I want to send them on the command prompt. > > > Thanks, > > Yash > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questi...@REDACTED > >http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions From f355@REDACTED Mon Jun 17 16:33:45 2013 From: f355@REDACTED (Konstantin Tcepliaev) Date: Mon, 17 Jun 2013 18:33:45 +0400 Subject: [erlang-questions] Sending environment values for rebar-packaged apps In-Reply-To: References: <51bf13d5.0256310a.375c.ffffe1af@mx.google.com> <1371477527.4753.37.camel@sekic1152.rnd.ki.sw.ericsson.se> Message-ID: <1059961371479625@web18g.yandex.ru> Hi, Have you tried specifying the same command line flags after "start"? Like this: sh rel/mynode/bin/mynode start - key1 value1 key2 value2 ... I think that should work: https://github.com/basho/rebar/blob/master/priv/templates/simplenode.runner#L115 Cheers, -- Konstantin 17.06.2013, 18:16, "Yash Ganthe" : > This is when I start my node using erl. > > My question is about when I start the node as a package(without using > erl). > > On Jun 17, 6:58?pm, Bengt Kleberg wrote: > >> ?Greetings, >> >> ?"erl - key1 value1 key2 value2 ..." >> >> ?bengt >> >> ?On Mon, 2013-06-17 at 13:48 +0000, yas...@REDACTED wrote: >>> ?Hi, >>> ?If I have to launch a node with certain environment values I usually >>> ?do: >>> ?"erl -pa ebin -run appname -env [{myvar, 1234}]" >>> ?If I package this as a node using rebar, I can start the node using: >>> ?sh rel/mynode/bin/mynode start >>> ?But how do I set values for the app-specific enviroment vars? I do not >>> ?want to set them in a file as the values will change for every >>> ?invocation and hence I want to send them on the command prompt. >>> ?Thanks, >>> ?Yash >>> ?_______________________________________________ >>> ?erlang-questions mailing list >>> ?erlang-questi...@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >> ?_______________________________________________ >> ?erlang-questions mailing list >> ?erlang-questi...@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From francesco@REDACTED Mon Jun 17 19:08:37 2013 From: francesco@REDACTED (Francesco Cesarini) Date: Mon, 17 Jun 2013 18:08:37 +0100 Subject: [erlang-questions] FP Days 2013 (Cambridge, UK) Call for Speakers closing soon In-Reply-To: References: Message-ID: <51BF4295.9090305@erlang-solutions.com> Hi All, We had a few Erlang talks at this conference last year, alongside some really impressive names in FP. For anyone wanting to visit Cambridge, I can warmly recommend submitting either a talk or a tutorial. Francesco -------- Original Message -------- Subject: Francesco, FP Days 2013 (Cambridge, UK) Call for Speakers closing soon Date: Mon, 17 Jun 2013 09:00:59 -0400 From: Mark Dalgarno To: Francesco Cesarini Francesco, FP Days 2013 (Cambridge, UK) Call for Speakers closing soon The Call for Speakers for October's FP Days 2013 conference closes this week. Submission deadline: *Friday 21st June* We are seeking high-quality session proposals covering any aspect of Functional Programming in any language. Hands-on sessions, experience reports, tutorials, panels and other interactive sessions are particularly encouraged although more theoretical sessions are also welcom e. We welcome sessions from 45 to 180 minutes. This year's conference runs from 24th-25th October and will be held at a new, larger venue - Murray Edwards College, Cambridge. In addition to paying no conference fees, being a speaker gives you a unique opportunity to present your viewpoint to our audience. Please visit the FP Days 2013 session submission page for more information. Register your interest and see who else is tracking the event on our Lanyrd page: http://lanyrd.com/2013/fpdays/ Software Acumen, St. Johns Innovation Centre, Cowley Road, Cambridge, Cambridgeshire CB4 0WS, UNITED KINGDOM Unsubscribe | Change Subscriber Options -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremy@REDACTED Tue Jun 18 05:33:52 2013 From: jeremy@REDACTED (Jeremy Ong) Date: Mon, 17 Jun 2013 20:33:52 -0700 Subject: [erlang-questions] erlang-bcrypt status Message-ID: Hi all, The highest profile implementation of the bcrypt algorithm is currently the smarkets fork of erlang-bcrypt at https://github.com/smarkets/erlang-bcrypt. However, activity on this fork has not existed for a year now and I think we should reach a consensus on where this library should be officially maintained so everybody's patches can coalesce cohesively. The issues I see now are build problems on various nixes, a very outdated bundled rebar, and a potentially thread safety issue mentioned here: https://github.com/smarkets/erlang-bcrypt/pull/9/files. I am willing to do this and the fork would be actively maintained at the company I work at (Quark Games). However, I want to open up this opportunity in case somebody has already done significant work on the library and wants to curate it that I am unaware of. Cheers, Jeremy -------------- next part -------------- An HTML attachment was scrubbed... URL: From aaron.l.france@REDACTED Tue Jun 18 09:02:12 2013 From: aaron.l.france@REDACTED (Aaron France) Date: Tue, 18 Jun 2013 08:02:12 +0100 Subject: [erlang-questions] erlang-bcrypt status In-Reply-To: References: Message-ID: Hi, My advice would be to fork it, merge the patches where applicable and maintain and advertise that fork. If all goes well, the original maintainers will want that fork. Regards, Aaron On Tue, Jun 18, 2013 at 4:33 AM, Jeremy Ong wrote: > Hi all, > > The highest profile implementation of the bcrypt algorithm is currently > the smarkets fork of erlang-bcrypt at > https://github.com/smarkets/erlang-bcrypt. > > However, activity on this fork has not existed for a year now and I think > we should reach a consensus on where this library should be officially > maintained so everybody's patches can coalesce cohesively. > > The issues I see now are build problems on various nixes, a very outdated > bundled rebar, and a potentially thread safety issue mentioned here: > https://github.com/smarkets/erlang-bcrypt/pull/9/files. > > I am willing to do this and the fork would be actively maintained at the > company I work at (Quark Games). However, I want to open up this > opportunity in case somebody has already done significant work on the > library and wants to curate it that I am unaware of. > > Cheers, > Jeremy > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Tue Jun 18 09:58:47 2013 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 18 Jun 2013 09:58:47 +0200 Subject: [erlang-questions] edoc generation In-Reply-To: <51BEEC9F.1050000@gothensitz.com> References: <51BEEC9F.1050000@gothensitz.com> Message-ID: <51C01337.6020503@gmail.com> On 2013-06-17 13:01 , sm wrote: > I've just ran edoc on a collection of erlang source files with the > command: erl -noshell -run edoc_run application 'app' '"src"' '[{dir, > "src"}]' > > The source files contain the erlang library. > A few libraries gave this result: > > src/tls_record.erl, function is_acceptable_version/1: at line 477: > multiple @spec tag. > edoc: skipping source file 'src/tls_record.erl': {'EXIT',error}. > src/orber_ifr_aliasdef.erl: at line 61: syntax error before: '?tcheck' ... > edoc: error in doclet 'edoc_doclet': {'EXIT',error}. > edoc: edoc terminated abnormally: error. > > > I've placed all the source file in one directory. I've also tested the > some libray files in there orginal directory with the same results. It looks like some of the OTP applications contain broken edoc annotations (and in some cases, weird uses of macros that make it impossible for edoc to parse the sources). But the OTP applications are still mostly documented through hand-written SGML files, so it's not surprising that they haven't all been cleaned up to work with edoc. > How can I resolve this? Either by patching the OTP applications or by not running edoc on them in the first place. Why do you want to do this, when you can get the full OTP documentation from www.erlang.org? /Richard From z@REDACTED Tue Jun 18 10:41:47 2013 From: z@REDACTED (Danil Zagoskin) Date: Tue, 18 Jun 2013 12:41:47 +0400 Subject: [erlang-questions] edoc generation In-Reply-To: <51BEEC9F.1050000@gothensitz.com> References: <51BEEC9F.1050000@gothensitz.com> Message-ID: There are many errors caused by macros. You need to turn on preprocessing in EDoc. It will also process includes, so you need to pass include dirs too. See it in my project (edoc is run in parse_transform): https://github.com/stolen/autohelp/blob/master/src/autohelp.erl#L46 2013/6/17 sm > I've just ran edoc on a collection of erlang source files with the > command: erl -noshell -run edoc_run application 'app' '"src"' '[{dir, > "src"}]' > > The source files contain the erlang library. > A few libraries gave this result: > > src/tls_record.erl, function is_acceptable_version/1: at line 477: > multiple @spec tag. > edoc: skipping source file 'src/tls_record.erl': {'EXIT',error}. > src/orber_ifr_aliasdef.erl: at line 61: syntax error before: '?tcheck' > edoc: skipping source file 'src/orber_ifr_aliasdef.erl': {'EXIT',error}. > src/orber_ifr_idltype.erl: at line 46: syntax error before: '?tcheck' > edoc: skipping source file 'src/orber_ifr_idltype.erl': {'EXIT',error}. > src/mnesia_power_SUITE.erl: at line 37: syntax error before: mnesia > edoc: skipping source file 'src/mnesia_power_SUITE.erl': {'EXIT',error}. > src/ssl.erl, function start/0: at line 80: multiple @spec tag. > edoc: skipping source file 'src/ssl.erl': {'EXIT',error}. > src/distr_startup_SUITE.erl: at line 68: syntax error before: Nodes > edoc: skipping source file 'src/distr_startup_SUITE.erl': {'EXIT',error}. > src/orber_ifr_arraydef.erl: at line 53: syntax error before: '?tcheck' > edoc: skipping source file 'src/orber_ifr_arraydef.erl': {'EXIT',error}. > src/diameter_capx.erl: at line 120: syntax error before: ';' > edoc: skipping source file 'src/diameter_capx.erl': {'EXIT',error}. > src/orber_ifr_interfacedef.**erl: at line 85: syntax error before: > '?tcheck' > edoc: skipping source file 'src/orber_ifr_interfacedef.**erl': > {'EXIT',error}. > src/tls.erl, function start/0: at line 88: multiple @spec tag. > edoc: skipping source file 'src/tls.erl': {'EXIT',error}. > src/orber_ifr_operationdef.**erl: at line 72: syntax error before: > '?tcheck' > edoc: skipping source file 'src/orber_ifr_operationdef.**erl': > {'EXIT',error}. > src/dialyzer_utils.erl, function pp_hook/3: at line 456: @spec arity does > not match. > edoc: skipping source file 'src/dialyzer_utils.erl': {'EXIT',error}. > 2543- fatal: {endtag_does_not_match,{was,**code,should_have_been,ocde}} > src/ct_gen_conn.erl, function start/4: at line 61: error in XML parser: > {fatal,{{endtag_does_not_**match, {was,code,should_have_been,**ocde}}, > {file,file_name_unknown}, {line,86},{col,61}}}. > edoc: skipping source file 'src/ct_gen_conn.erl': {'EXIT',error}. > src/ct_framework.erl, function add_data_dir/2: at line 1331: @spec arity > does not match. > edoc: skipping source file 'src/ct_framework.erl': {'EXIT',error}. > src/orber_ifr_constantdef.erl: at line 65: syntax error before: '?tcheck' > edoc: skipping source file 'src/orber_ifr_constantdef.**erl': > {'EXIT',error}. > src/orber_ifr_fixeddef.erl: at line 48: syntax error before: '?tcheck' > edoc: skipping source file 'src/orber_ifr_fixeddef.erl': {'EXIT',error}. > src/public_key.erl, function pem_entry_decode/1: at line 86: @spec arity > does not match. > edoc: skipping source file 'src/public_key.erl': {'EXIT',error}. > src/orber_ifr_exceptiondef.**erl: at line 64: syntax error before: > '?tcheck' > edoc: skipping source file 'src/orber_ifr_exceptiondef.**erl': > {'EXIT',error}. > src/diameter_dbg.erl: at line 116: syntax error before: '.' > edoc: skipping source file 'src/diameter_dbg.erl': {'EXIT',error}. > src/orber_ifr_repository.erl: at line 65: syntax error before: '?tcheck' > edoc: skipping source file 'src/orber_ifr_repository.erl'**: > {'EXIT',error}. > src/zip.erl: at line 216: syntax error before: do_openzip_open > edoc: skipping source file 'src/zip.erl': {'EXIT',error}. > src/ct_run.erl, function run_test/1: at line 872: syntax error before: '--' > edoc: skipping source file 'src/ct_run.erl': {'EXIT',error}. > src/ct_util.erl, function start/0: at line 80: @spec arity does not match. > edoc: skipping source file 'src/ct_util.erl': {'EXIT',error}. > src/orber_ifr_enumdef.erl: at line 61: syntax error before: '?tcheck' > edoc: skipping source file 'src/orber_ifr_enumdef.erl': {'EXIT',error}. > edoc: skipping source file 'src/calendar.erl': > {'EXIT',{{badmatch,{atom,354,'**?SECONDS_PER_DAY'}},[{edoc_** > specs,d2e,1,[{file,[101,100|..**.]},{line,339}]},{edoc_specs,** > type,2,[{file,[101|...]},{**line,65}]},{edoc_extract,'-** > get_tags/6-lc$^0/1-0-',2,[{**file,[...]},{line,...}]},{** > edoc_extract,get_tags,6,[{**file,...},{...}]},{edoc_** > extract,get_tags,6,[{...}|...]**},{edoc_extract,source1,5,[...** > ]},{edoc_doclet,source,9,...},**{lists,foldl,...}]}}. > src/edoc_types.erl, function is_predefined/2: at line 32: file not found: > edoc_types.hrl > edoc: skipping source file 'src/edoc_types.erl': {'EXIT',error}. > 2543- fatal: {endtag_does_not_match,{was,p,**should_have_been,'N'}} > src/ct_repeat.erl, in module header: at line 20: error in XML parser: > {fatal, {{endtag_does_not_match,{was,**p,should_have_been,'N'}}, > {file,file_name_unknown},{**line,27},{col,33}}}. > edoc: skipping source file 'src/ct_repeat.erl': {'EXIT',error}. > src/ct_logs.erl, function init/2: at line 78: @spec arity does not match. > edoc: skipping source file 'src/ct_logs.erl': {'EXIT',error}. > src/corba_request.erl: at line 278: syntax error before: '?query_check' > edoc: skipping source file 'src/corba_request.erl': {'EXIT',error}. > src/orber_ifr_typecode.erl: at line 67: syntax error before: '?tcheck' > edoc: skipping source file 'src/orber_ifr_typecode.erl': {'EXIT',error}. > src/orber_ifr_stringdef.erl: at line 48: syntax error before: '?tcheck' > edoc: skipping source file 'src/orber_ifr_stringdef.erl': {'EXIT',error}. > src/orber_ifr_attributedef.**erl: at line 64: syntax error before: > '?tcheck' > edoc: skipping source file 'src/orber_ifr_attributedef.**erl': > {'EXIT',error}. > src/orber_ifr_wstringdef.erl: at line 46: syntax error before: '?tcheck' > edoc: skipping source file 'src/orber_ifr_wstringdef.erl'**: > {'EXIT',error}. > src/crypto.erl, function hmac/3: at line 267: multiple @spec tag. > src/crypto.erl, function hmac/3: at line 266: multiple @spec tag. > src/crypto.erl, function hmac/3: at line 265: multiple @spec tag. > src/crypto.erl, function hmac/3: at line 264: multiple @spec tag. > src/crypto.erl, function hmac/3: at line 263: multiple @spec tag. > edoc: skipping source file 'src/crypto.erl': {'EXIT',error}. > src/edoc_doclet.erl, function run/2: at line 38: file not found: > edoc_doclet.hrl > edoc: skipping source file 'src/edoc_doclet.erl': {'EXIT',error}. > src/ssl_manager.erl, function register_session/3: at line 156: multiple > @spec tag. > edoc: skipping source file 'src/ssl_manager.erl': {'EXIT',error}. > src/megaco_ber_bin_drv_**encoder.erl: at line 277: head mismatch > edoc: skipping source file 'src/megaco_ber_bin_drv_**encoder.erl': > {'EXIT',error}. > src/orber_ifr_sequencedef.erl: at line 53: syntax error before: '?tcheck' > edoc: skipping source file 'src/orber_ifr_sequencedef.**erl': > {'EXIT',error}. > src/ssl_pkix_db.erl, function cache_pem_file/2: at line 118: multiple > @spec tag. > edoc: skipping source file 'src/ssl_pkix_db.erl': {'EXIT',error}. > src/orber_ifr_uniondef.erl: at line 67: syntax error before: '?tcheck' > edoc: skipping source file 'src/orber_ifr_uniondef.erl': {'EXIT',error}. > src/orber_ifr_structdef.erl: at line 60: syntax error before: '?tcheck' > edoc: skipping source file 'src/orber_ifr_structdef.erl': {'EXIT',error}. > src/orber_ifr_primitivedef.**erl: at line 46: syntax error before: > '?tcheck' > edoc: skipping source file 'src/orber_ifr_primitivedef.**erl': > {'EXIT',error}. > src/orber_ifr_moduledef.erl: at line 64: syntax error before: '?tcheck' > edoc: skipping source file 'src/orber_ifr_moduledef.erl': {'EXIT',error}. > edoc: error in doclet 'edoc_doclet': {'EXIT',error}. > edoc: edoc terminated abnormally: error. > > > I've placed all the source file in one directory. I've also tested the > some libray files in there orginal directory with the same results. > > How can I resolve this? > > SM. > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -- --------------------------------------------- ????? ???????? | +7 906 064 20 47 | z@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From postsm@REDACTED Tue Jun 18 12:25:52 2013 From: postsm@REDACTED (sm) Date: Tue, 18 Jun 2013 12:25:52 +0200 Subject: [erlang-questions] edoc generation In-Reply-To: <51C01337.6020503@gmail.com> References: <51C01337.6020503@gmail.com> Message-ID: <51C035B0.7020504@gothensitz.com> Regarding the edoc generation, I've looked at the erlang documentation and realized that I need an erlang API list of functions to incorporate into a programming editor. I created a small file with three functions and the calltip work just fine with the editor. the full OTP documentation found in www.erlang.org has an PDF or HTML as a file format. I have two choices: 1. Tweak the html file produced by edoc_run, 2. or go to the source and just extract the spec and type information. The html produced by edoc_run has less clutter than the OTP documentation. I can change the css and change the layout, then copy the text from a web browser into an editor for further processing. In the past I created an xsl stylesheet for the xml files, then transformed the xml file into layout format I needed, but inconsistancies showed up. sm From watson.timothy@REDACTED Tue Jun 18 13:39:31 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 18 Jun 2013 12:39:31 +0100 Subject: [erlang-questions] driver_create_port overhead and driver_caller In-Reply-To: <415F5924-DB56-45DE-ABD7-C56405030B56@btinternet.com> References: <4765DCE4-9840-4187-B1A5-D7DD143320AF@btinternet.com> <415F5924-DB56-45DE-ABD7-C56405030B56@btinternet.com> Message-ID: <9F3E1381-FFD1-4A2A-A0B2-CB99A2452BFC@gmail.com> On 14 Jun 2013, at 11:19, QDev wrote: > Is there any advice from OTP team on best way to handle long running task in driver that needs to do ongoing communication with peer Erlang process? Can Erlang process calls outputv and use mutex to talk to async thread (or manual spawned thread) be best? I've seen driver code that worked that way which was pretty efficient. > Is locking in outpuv not risk to schedulers if outputv take too long, like with NIF calls? If I understand it correctly, yes, this is a risk. Use the async thread pool or pool/spawn your own threads to compensate. > Is there other way - would it be better in outputv to put binary/ErlIOVec directly in driver queue with driver_enqv or driver_enq_bin but then port data lock has to be use to synchronise reads/writes to driver queue anyway. That means more contention if many concurrent threads are working no? So I was thinking better to use mutex per long running thread instead. I've not used the driver queue much in the past, so not sure about that. > Also, is it considered necessary to pool worker thread manually? In previous releases, using async thread pool could have detrimental effect on I/O performance - is this still the case? ISTR reading that the async threads used by drivers are now separate from those used by the I/O drv internally. Cheers, Tim From yashgt@REDACTED Tue Jun 18 13:39:55 2013 From: yashgt@REDACTED (yashgt@REDACTED) Date: Tue, 18 Jun 2013 11:39:55 +0000 Subject: [erlang-questions] Packaged node: environment variables not accessible Message-ID: <51c04722.4546e00a.22d3.5e77@mx.google.com> An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Tue Jun 18 13:40:16 2013 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Tue, 18 Jun 2013 13:40:16 +0200 Subject: [erlang-questions] Configuring rebar across directories In-Reply-To: <51a609ab.e982310a.5ec1.ffffcd05@mx.google.com> References: <51a609ab.e982310a.5ec1.ffffcd05@mx.google.com> Message-ID: On Wed, May 29, 2013 at 3:58 PM, wrote: > I have > myapp > /src > myapp.erl > myapp_sup.erl > /include > and > mymodule > /src > mymodule.erl > /include > mymodule.hrl > > As you can see, I have developed a module which is outside the > structure of the myapp. If a module in myapp has to use the header > files of mymodule, what do I need to do in the rebar.config of > myapp? > > Currently, when I compile myapp, it gives an error that the > mymoduel.hrl cannot be found. > > My intention is to develop mymodule as a shared module across > several apps. The usual way is to make mymodule part of ERL_LIBS and then -include_lib("mymodule/include/mymodule.hrl"). You can either configure the ERL_LIBS env var or {lib_dirs, []} in rebar.config. Many projects put libs or apps in a separate apps/ directory and configure the top level rebar.config accordingly. In your case that would be: {lib_dirs, ["apps"]}. {sub_dirs, ["apps/myapp", "apps/mymodule"]}. Alternatively you can modify erl_opts (.erl compile opts) in rebar.config to add extra include paths as follows, but this would be unconventional and might be less flexible: {erl_opts, [{i, "../mymodule/include"}]}. For more help see compile(3)[1] or run 'rebar help compile'. [1] http://www.erlang.org/doc/man/compile.html From syupei@REDACTED Tue Jun 18 13:45:31 2013 From: syupei@REDACTED (Peter) Date: Tue, 18 Jun 2013 04:45:31 -0700 (PDT) Subject: [erlang-questions] why my test case of socket are always slower much than c++ Message-ID: <02ddd507-5be9-49b9-9a01-4073bc80e142@googlegroups.com> i test for the socket rec/sed speed, i used c++?Erlang and golang, and the result like this: so, i want know, why always slow when erlang as a rec-server. is use wrong? my source code see attachment, and my test case like this: the erlang server: erl +K true server:start(8080, 200) the erlang client: erl +K true client:start(192.168.0.1, 8080, 200, 1000*10000) the cpp server: ./server 192.168.0.1 8080 200 the cpp client: ./client 192.168.0.1 8080 10000000 200 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: client.erl Type: application/octet-stream Size: 737 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: server.erl Type: application/octet-stream Size: 1232 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: client.cpp Type: text/x-c++src Size: 1728 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: server.cpp Type: text/x-c++src Size: 1818 bytes Desc: not available URL: From g@REDACTED Tue Jun 18 14:50:13 2013 From: g@REDACTED (Garrett Smith) Date: Tue, 18 Jun 2013 07:50:13 -0500 Subject: [erlang-questions] Chicago User Group - this Thurs Message-ID: If you're in the Chicago area this Thursday and want an injection of Erlang, there's a get together! http://www.meetup.com/ErlangChicago/events/124274192/ Garrett From lukas@REDACTED Tue Jun 18 15:35:55 2013 From: lukas@REDACTED (Lukas Larsson) Date: Tue, 18 Jun 2013 15:35:55 +0200 Subject: [erlang-questions] driver_create_port overhead and driver_caller In-Reply-To: <9F3E1381-FFD1-4A2A-A0B2-CB99A2452BFC@gmail.com> References: <4765DCE4-9840-4187-B1A5-D7DD143320AF@btinternet.com> <415F5924-DB56-45DE-ABD7-C56405030B56@btinternet.com> <9F3E1381-FFD1-4A2A-A0B2-CB99A2452BFC@gmail.com> Message-ID: On Tue, Jun 18, 2013 at 1:39 PM, Tim Watson wrote: > On 14 Jun 2013, at 11:19, QDev wrote: > > > Is there other way - would it be better in outputv to put > binary/ErlIOVec directly in driver queue with driver_enqv or driver_enq_bin > but then port data lock has to be use to synchronise reads/writes to driver > queue anyway. That means more contention if many concurrent threads are > working no? So I was thinking better to use mutex per long running thread > instead. > > I've not used the driver queue much in the past, so not sure about that. > The good thing about using the driver queue is that if there is data and a flush callback is implemented you can make sure that data is handled before closing the port. So that you don't loose things which you might need. If you do notice that you get a lot of contentions on the pdl you can of course roll your own more fine grained locking and just stick a byte of data in the driver queue in order to trigger flushes. > > Also, is it considered necessary to pool worker thread manually? In > previous releases, using async thread pool could have detrimental effect on > I/O performance - is this still the case? > > ISTR reading that the async threads used by drivers are now separate from > those used by the I/O drv internally. > File I/O uses the same async threads as drivers. In fact the I/O drivers do not use any functionality which is not available to any other driver. So if you do have long running jobs in the async pool it could interfere with the latency of file I/O. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Tue Jun 18 16:00:12 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 18 Jun 2013 15:00:12 +0100 Subject: [erlang-questions] driver_create_port overhead and driver_caller In-Reply-To: References: <4765DCE4-9840-4187-B1A5-D7DD143320AF@btinternet.com> <415F5924-DB56-45DE-ABD7-C56405030B56@btinternet.com> <9F3E1381-FFD1-4A2A-A0B2-CB99A2452BFC@gmail.com> Message-ID: <25D72E15-36C7-43E5-B1E7-97A8FE90EFEE@gmail.com> On 18 Jun 2013, at 14:35, Lukas Larsson wrote: > On Tue, Jun 18, 2013 at 1:39 PM, Tim Watson wrote: > On 14 Jun 2013, at 11:19, QDev wrote: > > > Is there other way - would it be better in outputv to put binary/ErlIOVec directly in driver queue with driver_enqv or driver_enq_bin but then port data lock has to be use to synchronise reads/writes to driver queue anyway. That means more contention if many concurrent threads are working no? So I was thinking better to use mutex per long running thread instead. > > I've not used the driver queue much in the past, so not sure about that. > > The good thing about using the driver queue is that if there is data and a flush callback is implemented you can make sure that data is handled before closing the port. So that you don't loose things which you might need. If you do notice that you get a lot of contentions on the pdl you can of course roll your own more fine grained locking and just stick a byte of data in the driver queue in order to trigger flushes. > That sounds like workable approach, will look into it. > > > Also, is it considered necessary to pool worker thread manually? In previous releases, using async thread pool could have detrimental effect on I/O performance - is this still the case? > > ISTR reading that the async threads used by drivers are now separate from those used by the I/O drv internally. > > File I/O uses the same async threads as drivers. In fact the I/O drivers do not use any functionality which is not available to any other driver. So if you do have long running jobs in the async pool it could interfere with the latency of file I/O. > Long running jobs will use own thread pool then. Thanks! QD -------------- next part -------------- An HTML attachment was scrubbed... URL: From watson.timothy@REDACTED Tue Jun 18 16:02:15 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 18 Jun 2013 15:02:15 +0100 Subject: [erlang-questions] driver_create_port overhead and driver_caller In-Reply-To: References: <4765DCE4-9840-4187-B1A5-D7DD143320AF@btinternet.com> <415F5924-DB56-45DE-ABD7-C56405030B56@btinternet.com> <9F3E1381-FFD1-4A2A-A0B2-CB99A2452BFC@gmail.com> Message-ID: On 18 Jun 2013, at 14:35, Lukas Larsson wrote: > > The good thing about using the driver queue is that if there is data and a flush callback is implemented you can make sure that data is handled before closing the port. So that you don't loose things which you might need. If you do notice that you get a lot of contentions on the pdl you can of course roll your own more fine grained locking and just stick a byte of data in the driver queue in order to trigger flushes. > Oh, that's nifty, I like it. Is there any benefit to doing that versus creating a pipe and using driver_select to signal/check? > File I/O uses the same async threads as drivers. In fact the I/O drivers do not use any functionality which is not available to any other driver. So if you do have long running jobs in the async pool it could interfere with the latency of file I/O. > Hmn, ok that's for clearing that up - it's important to be aware of the risks there. Cheers, Tim From alessandro.sivieri@REDACTED Tue Jun 18 16:25:00 2013 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Tue, 18 Jun 2013 16:25:00 +0200 Subject: [erlang-questions] Compilation options for the VM: optimizations Message-ID: Hi all, I have done a couple of little experiments by adding compilation options (these CFLAGS [1]) on a Raspberry Pi when compiling the Erlang VM, but for what I can see the code seems to spend *more* time when executed: I tried a couple of really silly algorithms (such as prime numbers calculation) and the optimized version spends a few seconds more on computation than the one without any CFLAGS specified. Ideas why? [1] CFLAGS='-O2 -march=armv6zk -mtune=arm1176jzf-s -mfpu=vfp -ftree-vectorize -ffast-math -mfloat-abi=hard' -- Sivieri Alessandro alessandro.sivieri@REDACTED http://sivieri.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From carlsson.richard@REDACTED Tue Jun 18 16:56:28 2013 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Tue, 18 Jun 2013 16:56:28 +0200 Subject: [erlang-questions] What happens with atoms across different systems? In-Reply-To: References: Message-ID: <51C0751C.7030901@gmail.com> On 2013-06-15 15:57 , Yves S. Garret wrote: > Hello, > > I'm reading the book Erlang and OTP in Action and came across this > part: "In Erlang, an atom is a special kind of string constant that > is identified only by the characters in the string, so that two atoms > are always considered to be exactly the same if they have the same > character representation. But internally, these strings are stored in > a table and are referred to by the table index, so that checking > atoms for equivalence at runtime amounts to comparing two small > integers; and each time you use an atom, it takes up only one word of > memory. (The index number used for any particular atom is > automatically assigned at runtime and can vary from one run of the > system to the next; there is no way, and no need, for the user to > know this.)" > > Now, this got me thinking. What if I have two systems that send > messages to one another. I update one and include a new atom that > I'm using. This atom gets sent to system #2 that cannot recognize it > just yet. Does that mean that that message will sit in the receiving > thread's queue until it can recognize it or can be handled/thrown > away? The atom tables on the two nodes are completely separate, and apart from pre-defined atoms needed for the runtime system, the indexes used will depend on in what order atoms are added (which mostly depends on the order in which code is loaded). Hence, 'foo' can be entry 4711 on one node and entry 12345 on another node. But the runtime representation of an atom (a single word containing the index and a type tag) is local to the node, and these indexes are never passed between nodes. Instead, when a node sends a term to another node, the Erlang distribution protocol keeps a cache which assigns a temporary number to the atoms that have been sent, so the first time an atom is seen on the wire, it is passed as a string plus the cache entry number, and then for repeated occurrences only the number is passed, to save bandwidth. The receiving node then converts the incoming atoms to its own local representation. This all happens before the message is passed on to the actual receiving process, so by the time it ends up in the message queue, the atoms already exist. If the receiving node hasn't yet loaded any code that will accept messages tagged with the new atoms, the messages will stay in the mailbox until the code is loaded (as long as you don't have a catch-all clause that will swallow any currently unknown messages). > As a follow-up question, does it make sense to use atoms in this > fashion? Sure, it's being done all the time. /Richard From lukas@REDACTED Tue Jun 18 16:57:44 2013 From: lukas@REDACTED (Lukas Larsson) Date: Tue, 18 Jun 2013 16:57:44 +0200 Subject: [erlang-questions] driver_create_port overhead and driver_caller In-Reply-To: References: <4765DCE4-9840-4187-B1A5-D7DD143320AF@btinternet.com> <415F5924-DB56-45DE-ABD7-C56405030B56@btinternet.com> <9F3E1381-FFD1-4A2A-A0B2-CB99A2452BFC@gmail.com> Message-ID: On Tue, Jun 18, 2013 at 4:02 PM, Tim Watson wrote: > On 18 Jun 2013, at 14:35, Lukas Larsson wrote: > > > > The good thing about using the driver queue is that if there is data and > a flush callback is implemented you can make sure that data is handled > before closing the port. So that you don't loose things which you might > need. If you do notice that you get a lot of contentions on the pdl you can > of course roll your own more fine grained locking and just stick a byte of > data in the driver queue in order to trigger flushes. > > > > Oh, that's nifty, I like it. Is there any benefit to doing that versus > creating a pipe and using driver_select to signal/check? > Not sure. What will happen is the port will remain in the system (but all links will be triggered and you cannot interact with it from erlang) and you can continue doing ready_output/ready_input until you clear the driver queue. When the queue is cleared the normal stop behaviour happens. IMO it sometimes makes for nicer code as you don't have to worry about the port being tugged away from under your feet when you have data in the queue which has to be delivered. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From juanjo@REDACTED Tue Jun 18 17:02:35 2013 From: juanjo@REDACTED (Juan Jose Comellas) Date: Tue, 18 Jun 2013 12:02:35 -0300 Subject: [erlang-questions] Configuring rebar across directories In-Reply-To: References: <51a609ab.e982310a.5ec1.ffffcd05@mx.google.com> Message-ID: Tuncer, is there a reason why the rebar lib_dirs configuration setting only allows relative paths? Right now the function that expands these paths looks like this: expand_lib_dirs([], _Root, Acc) -> Acc; expand_lib_dirs([Dir | Rest], Root, Acc) -> Apps = filelib:wildcard(filename:join([Dir, "*", "ebin"])), FqApps = [filename:join([Root, A]) || A <- Apps], expand_lib_dirs(Rest, Root, Acc ++ FqApps). Changing it to something like this would resolve this problem: expand_lib_dirs([], _Root, Acc) -> Acc; expand_lib_dirs([Dir | Rest], Root, Acc) -> Apps = filelib:wildcard(filename:join([Dir, "*", "ebin"])), FqApps = case filename:pathtype(Dir) of absolute -> Apps; relative -> [filename:join([Root, A]) || A <- Apps] end, expand_lib_dirs(Rest, Root, Acc ++ FqApps). Would this addition be accepted into rebar? I haven't created a pull request for this because I don't know what the rationale was for making these paths relative. Juanjo On Tue, Jun 18, 2013 at 8:40 AM, Tuncer Ayaz wrote: > On Wed, May 29, 2013 at 3:58 PM, wrote: > > I have > > myapp > > /src > > myapp.erl > > myapp_sup.erl > > /include > > and > > mymodule > > /src > > mymodule.erl > > /include > > mymodule.hrl > > > > As you can see, I have developed a module which is outside the > > structure of the myapp. If a module in myapp has to use the header > > files of mymodule, what do I need to do in the rebar.config of > > myapp? > > > > Currently, when I compile myapp, it gives an error that the > > mymoduel.hrl cannot be found. > > > > My intention is to develop mymodule as a shared module across > > several apps. > > The usual way is to make mymodule part of ERL_LIBS and then > -include_lib("mymodule/include/mymodule.hrl"). > > You can either configure the ERL_LIBS env var or {lib_dirs, []} in > rebar.config. Many projects put libs or apps in a separate apps/ > directory and configure the top level rebar.config accordingly. > In your case that would be: > {lib_dirs, ["apps"]}. > {sub_dirs, ["apps/myapp", "apps/mymodule"]}. > > Alternatively you can modify erl_opts (.erl compile opts) in > rebar.config to add extra include paths as follows, but this would be > unconventional and might be less flexible: > {erl_opts, [{i, "../mymodule/include"}]}. > > For more help see compile(3)[1] or run 'rebar help compile'. > > [1] http://www.erlang.org/doc/man/compile.html > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From juanjo@REDACTED Tue Jun 18 17:08:58 2013 From: juanjo@REDACTED (Juan Jose Comellas) Date: Tue, 18 Jun 2013 12:08:58 -0300 Subject: [erlang-questions] erlang-bcrypt status In-Reply-To: References: Message-ID: We've been using the Opscode fork of this library at https://github.com/opscode/erlang-bcrypt as our source and I have moved some critical fixes that were sitting as pull requests in the Smarkets repository to the Opscode one. Juanjo On Tue, Jun 18, 2013 at 12:33 AM, Jeremy Ong wrote: > Hi all, > > The highest profile implementation of the bcrypt algorithm is currently > the smarkets fork of erlang-bcrypt at > https://github.com/smarkets/erlang-bcrypt. > > However, activity on this fork has not existed for a year now and I think > we should reach a consensus on where this library should be officially > maintained so everybody's patches can coalesce cohesively. > > The issues I see now are build problems on various nixes, a very outdated > bundled rebar, and a potentially thread safety issue mentioned here: > https://github.com/smarkets/erlang-bcrypt/pull/9/files. > > I am willing to do this and the fork would be actively maintained at the > company I work at (Quark Games). However, I want to open up this > opportunity in case somebody has already done significant work on the > library and wants to curate it that I am unaware of. > > Cheers, > Jeremy > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.goertzen@REDACTED Tue Jun 18 17:41:28 2013 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Tue, 18 Jun 2013 10:41:28 -0500 Subject: [erlang-questions] why my test case of socket are always slower much than c++ Message-ID: A few points: - You have nodelay turned on for the Erlang code but not the C++ code; this will slow things down. - You specify many other tuning options. I suggest removing them all unless you are absolutely certain a default is causing you problems. - On the receive side, {active, once} might be faster. For most uses it is the best way to do socket receives in Erlang. Cheers, Dan. On Tue, Jun 18, 2013 at 6:45 AM, Peter wrote: > i test for the socket rec/sed speed, > > i used c++?Erlang and golang, and the result like this: > > > > > so, i want know, why always slow when erlang as a rec-server. is use > wrong? > > my source code see attachment, and my test case like this: > > the erlang server: > erl +K true > server:start(8080, 200) > > the erlang client: > erl +K true > client:start(192.168.0.1, 8080, 200, 1000*10000) > > the cpp server: > ./server 192.168.0.1 8080 200 > > the cpp client: > ./client 192.168.0.1 8080 10000000 200 > > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tuncer.ayaz@REDACTED Tue Jun 18 17:41:22 2013 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Tue, 18 Jun 2013 17:41:22 +0200 Subject: [erlang-questions] Configuring rebar across directories In-Reply-To: References: <51a609ab.e982310a.5ec1.ffffcd05@mx.google.com> Message-ID: On Tue, Jun 18, 2013 at 5:02 PM, Juan Jose Comellas wrote: > Tuncer, is there a reason why the rebar lib_dirs configuration > setting only allows relative paths? Right now the function that > expands these paths looks like this: I'm not aware of any reason and I think you're the first to request support for absolute paths. > expand_lib_dirs([], _Root, Acc) -> > Acc; > expand_lib_dirs([Dir | Rest], Root, Acc) -> > Apps = filelib:wildcard(filename:join([Dir, "*", "ebin"])), > FqApps = [filename:join([Root, A]) || A <- Apps], > expand_lib_dirs(Rest, Root, Acc ++ FqApps). > > Changing it to something like this would resolve this problem: > > expand_lib_dirs([], _Root, Acc) -> > Acc; > expand_lib_dirs([Dir | Rest], Root, Acc) -> > Apps = filelib:wildcard(filename:join([Dir, "*", "ebin"])), > FqApps = case filename:pathtype(Dir) of > absolute -> Apps; > relative -> [filename:join([Root, A]) || A <- Apps] > end, > expand_lib_dirs(Rest, Root, Acc ++ FqApps). > > Would this addition be accepted into rebar? I haven't created a pull > request for this because I don't know what the rationale was for > making these paths relative. This looks reasonable to me, but I can't speak for Dave. From essen@REDACTED Tue Jun 18 18:22:49 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 18 Jun 2013 18:22:49 +0200 Subject: [erlang-questions] why my test case of socket are always slower much than c++ In-Reply-To: References: Message-ID: <51C08959.40407@ninenines.eu> On 06/18/2013 05:41 PM, Daniel Goertzen wrote: > - On the receive side, {active, once} might be faster. For most uses it > is the best way to do socket receives in Erlang. Not true. Active mode and recv are equivalent in speed when you don't know how much data you want. It used to be that recv was faster than active but this got optimized in one of the R15 releases. The reason it is generally the best way is that you usually want to receive other kinds of messages at the same time. On the other hand, if you know how much data you need, then a recv(Size) will be better because it won't return until it gets it all. Receiving a 8MB file in 8 calls is obviously better than receiving it in 8000 messages. As for why the test case has these results, the answer is simple. The case is testing sequential throughput. Erlang is optimized for concurrent low latency. Not quite the same. Erlang can do throughput though, if you increase the recv size. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From derekbrown121@REDACTED Tue Jun 18 19:05:38 2013 From: derekbrown121@REDACTED (Derek Brown) Date: Tue, 18 Jun 2013 13:05:38 -0400 Subject: [erlang-questions] Etudes for Erlang discussion/study meetup in NYC every Tuesday Message-ID: Tonight will be the second of the series, based around the Etudes for Erlang book (http://chimera.labs.oreilly.com/books/1234000000726). Details can be found at http://www.meetup.com/Erlang-NYC/ Derek -------------- next part -------------- An HTML attachment was scrubbed... URL: From dizzyd@REDACTED Tue Jun 18 22:43:44 2013 From: dizzyd@REDACTED (Dave Smith) Date: Tue, 18 Jun 2013 14:43:44 -0600 Subject: [erlang-questions] Configuring rebar across directories In-Reply-To: References: <51a609ab.e982310a.5ec1.ffffcd05@mx.google.com> Message-ID: Yes, seems reasonable, Juan. I don't believe it would break anything. D. On Tue, Jun 18, 2013 at 9:41 AM, Tuncer Ayaz wrote: > On Tue, Jun 18, 2013 at 5:02 PM, Juan Jose Comellas wrote: > > Tuncer, is there a reason why the rebar lib_dirs configuration > > setting only allows relative paths? Right now the function that > > expands these paths looks like this: > > I'm not aware of any reason and I think you're the first to request > support for absolute paths. > > > expand_lib_dirs([], _Root, Acc) -> > > Acc; > > expand_lib_dirs([Dir | Rest], Root, Acc) -> > > Apps = filelib:wildcard(filename:join([Dir, "*", "ebin"])), > > FqApps = [filename:join([Root, A]) || A <- Apps], > > expand_lib_dirs(Rest, Root, Acc ++ FqApps). > > > > Changing it to something like this would resolve this problem: > > > > expand_lib_dirs([], _Root, Acc) -> > > Acc; > > expand_lib_dirs([Dir | Rest], Root, Acc) -> > > Apps = filelib:wildcard(filename:join([Dir, "*", "ebin"])), > > FqApps = case filename:pathtype(Dir) of > > absolute -> Apps; > > relative -> [filename:join([Root, A]) || A <- Apps] > > end, > > expand_lib_dirs(Rest, Root, Acc ++ FqApps). > > > > Would this addition be accepted into rebar? I haven't created a pull > > request for this because I don't know what the rationale was for > > making these paths relative. > > This looks reasonable to me, but I can't speak for Dave. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From emmiller@REDACTED Wed Jun 19 00:58:01 2013 From: emmiller@REDACTED (Evan Miller) Date: Tue, 18 Jun 2013 17:58:01 -0500 Subject: [erlang-questions] Chicago User Group - this Thurs In-Reply-To: References: Message-ID: I thoroughly object to the fact that Garrett plans to fraternize with Scala freaks afterwards. However, the Meetup itself should be a rollicking good time. It will answer, among other questions: When is a date not a date? What's fault-tolerant, distributed, and injectable into your ARM? Does Garrett Smith ever terminate an Erlang statement with a comma, or does he refactor it first? Is Joe Armstrong's bathroom generally too hot, too cold, or just right? Plus questions from the audience. The venue (1871) is in the River North area, pretty easy to get to from public transit. However, parking can be a PsychoB, if you know what I mean. Evan On Tue, Jun 18, 2013 at 8:50 AM, Garrett Smith wrote: > If you're in the Chicago area this Thursday and want an injection of > Erlang, there's a get together! > > http://www.meetup.com/ErlangChicago/events/124274192/ > > Garrett > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Evan Miller http://www.evanmiller.org/ From essen@REDACTED Wed Jun 19 01:02:41 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Wed, 19 Jun 2013 01:02:41 +0200 Subject: [erlang-questions] Chicago User Group - this Thurs In-Reply-To: References: Message-ID: <51C0E711.7040906@ninenines.eu> On 06/19/2013 12:58 AM, Evan Miller wrote: > I thoroughly object to the fact that Garrett plans to fraternize with > Scala freaks afterwards. However, the Meetup itself should be a > rollicking good time. It will answer, among other questions: When is a > date not a date? What's fault-tolerant, distributed, and injectable > into your ARM? Does Garrett Smith ever terminate an Erlang statement > with a comma, or does he refactor it first? Is Joe Armstrong's > bathroom generally too hot, too cold, or just right? Plus questions The first rule of the Erlang community is: Never talk about Joe's bathroom. Answering that question would just spill the beans on the secrets behind Erlang's awesomeness. > from the audience. > > The venue (1871) is in the River North area, pretty easy to get to > from public transit. However, parking can be a PsychoB, if you know > what I mean. > > Evan > > > On Tue, Jun 18, 2013 at 8:50 AM, Garrett Smith wrote: >> If you're in the Chicago area this Thursday and want an injection of >> Erlang, there's a get together! >> >> http://www.meetup.com/ErlangChicago/events/124274192/ >> >> Garrett >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From seth@REDACTED Wed Jun 19 07:00:29 2013 From: seth@REDACTED (Seth Falcon) Date: Tue, 18 Jun 2013 22:00:29 -0700 Subject: [erlang-questions] erlang-bcrypt status In-Reply-To: References: Message-ID: We are using erlang-bcrypt as part of our Erlang based Chef server implementation "erchef" and initially created our fork to address some build issues. We'll do our best to review and apply pull requests for that repo and provide sane tags for builds. Cheers, + seth -- Seth Falcon | Development Lead | Opscode -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremy@REDACTED Wed Jun 19 08:17:58 2013 From: jeremy@REDACTED (Jeremy Ong) Date: Tue, 18 Jun 2013 23:17:58 -0700 Subject: [erlang-questions] erlang-bcrypt status In-Reply-To: References: Message-ID: Hey Seth, Awesome and glad to see that it's found a new home. Cheers, Jeremy On Tue, Jun 18, 2013 at 10:00 PM, Seth Falcon wrote: > We are using erlang-bcrypt as part of our Erlang based Chef server > implementation "erchef" and initially created our fork to address some > build issues. We'll do our best to review and apply pull requests for that > repo and provide sane tags for builds. > > Cheers, > > + seth > > -- > Seth Falcon | Development Lead | Opscode > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Wed Jun 19 08:34:00 2013 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Wed, 19 Jun 2013 08:34:00 +0200 Subject: [erlang-questions] edoc generation In-Reply-To: <51C035B0.7020504@gothensitz.com> References: <51C01337.6020503@gmail.com> <51C035B0.7020504@gothensitz.com> Message-ID: <20130619063400.GB11902@erix.ericsson.se> On Tue, Jun 18, 2013 at 12:25:52PM +0200, sm wrote: > Regarding the edoc generation, > > I've looked at the erlang documentation and realized that I need an > erlang API list of functions to incorporate into a programming > editor. > > I created a small file with three functions and the calltip work > just fine with the editor. the full OTP documentation found in > www.erlang.org has an PDF or HTML as a file format. > > I have two choices: > > 1. Tweak the html file produced by edoc_run, > 2. or go to the source and just extract the spec and type information. There might exist a third alternative. In the documentation tarball, and also online there are index files, one per application, that is used by e.g the search engine at http://www.erlang.org/erldoc, and maybe other search engines. These are created in the normal documentation build and are in Erlang term format, have a look inside any. To whip up an escript that outputs any format you like should be dead easy and probably use file:consult/1. Here are two: http://www.erlang.org/documentation/doc-5.10.1/erts-5.10.1/doc/html/erts.eix http://www.erlang.org/documentation/doc-5.10.1/lib/stdlib-1.19.1/doc/html/stdlib.eix Their format should be fairly stable and is used to map from application, module and function to html filenames and anchors. Therefore they contain a documented function list for each module. > > The html produced by edoc_run has less clutter than the OTP > documentation. I can change the css and change the layout, then copy > the text from a web browser into an editor for further processing. > > In the past I created an xsl stylesheet for the xml files, > then transformed the xml file into layout format I needed, but > inconsistancies showed up. > > sm > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From karol.urbanski@REDACTED Wed Jun 19 11:54:58 2013 From: karol.urbanski@REDACTED (Karol Urbanski) Date: Wed, 19 Jun 2013 11:54:58 +0200 Subject: [erlang-questions] R16B01 binary packages for CentOS, Debian, Mac, Ubuntu and Windows Message-ID: <20130619095457.GA24995@dex.pl.erlangsystems.com> Hello, You can find our binary packages for the newest Erlang release (R16B01) at http://www.erlang-solutions.com/downloads/download-erlang-otp The systems we provide the packages for are CentOS 6, Mac OS X Snow Leopard and newer, Debian 6.0.3 and 7.0.0, Ubuntu 13.04 + 12.10 + 12.04 and Windows. More to come soon! Best regards, Karol Urbanski From kenneth.lundin@REDACTED Wed Jun 19 15:26:08 2013 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Wed, 19 Jun 2013 15:26:08 +0200 Subject: [erlang-questions] OTP R16B01 has been released Message-ID: Erlang/OTP R16B01 has been released. OTP R16B01 is a service release with mostly a number of small corrections and user contributions. But there are some new functions worth mentioning as well, here are some of them: - Migration of memory carriers between allocator instances, when enabled it will result in reduced memory footprint when the memory load is unevenly distributed between scheduler specific allocator instances. - Interruptible term_to_binary, (thanks to Erik Stenman) - Added the +sfwi Scheduler Forced Wakeup Interval - All crypto functions except the block chipers will now chunk large input into several calls to avoid blocking the scheduler for too long. - Elliptic curve support in crypto, ssl and ssh(thanks to Andreas Schultz) - Lift static limitation (FD_SETSIZE) for file descriptors on Mac OS X. (thanks to Anthony Ramine) - Removed a lock around GC statistics counter - Many small improvements and in Diameter - 60 user contributions You can find the README file with more detailed info at http://www.erlang.org/download/otp_src_R16B01.readme You can download the full source distribution from http://www.erlang.org/download/otp_src_R16B01.tar.gz http://www.erlang.org/download/otp_src_R16B01.readme (this file) Note: To unpack the TAR archive you need a GNU TAR compatible program. For installation instructions please read the README that is part of the distribution. You can also find this release at the official Erlang/OTP Git-repository at Github here: https://github.com/erlang/otp tagged *OTP_R16B01* The Windows binary distribution can be downloaded from http://www.erlang.org/download/otp_win32_R16B01.exe http://www.erlang.org/download/otp_win64_R16B01.exe On-line documentation can be found at http://www.erlang.org/doc/. You can also download the complete HTML documentation or the Unix manual files http://www.erlang.org/download/otp_doc_html_R16B01.tar.gz http://www.erlang.org/download/otp_doc_man_R16B01.tar.gz We also want to thank those that sent us patches, suggestions and bug reports, The Erlang/OTP Team at Ericsson -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Wed Jun 19 15:42:11 2013 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 19 Jun 2013 09:42:11 -0400 Subject: [erlang-questions] OTP R16B01 has been released In-Reply-To: References: Message-ID: <20130619134210.GA34756@ferdair.local> I've updated erlang-history to go with this release, if any reader feels like getting a little bit more shell persistence: https://github.com/ferd/erlang-history Thanks for the relase! Regards, Fred. On 06/19, Kenneth Lundin wrote: > Erlang/OTP R16B01 has been released. > > OTP R16B01 is a service release with mostly a number of small corrections > and user contributions. But there are some new functions worth mentioning > as well, here are some of them: > > > - Migration of memory carriers between allocator instances, when enabled > it will result in reduced memory footprint when the memory load is unevenly > distributed between scheduler specific allocator instances. > - Interruptible term_to_binary, (thanks to Erik Stenman) > - Added the +sfwi Scheduler Forced Wakeup Interval > - All crypto functions except the block chipers will now chunk large > input into several calls to avoid blocking the scheduler for too long. > - Elliptic curve support in crypto, ssl and ssh(thanks to Andreas > Schultz) > - Lift static limitation (FD_SETSIZE) for file descriptors on Mac OS X. > (thanks to Anthony Ramine) > - Removed a lock around GC statistics counter > - Many small improvements and in Diameter > - 60 user contributions > > You can find the README file with more detailed info at > http://www.erlang.org/download/otp_src_R16B01.readme > > You can download the full source distribution from > http://www.erlang.org/download/otp_src_R16B01.tar.gz > http://www.erlang.org/download/otp_src_R16B01.readme (this file) > > Note: To unpack the TAR archive you need a GNU TAR compatible program. > > For installation instructions please read the README that is part of > the distribution. > > You can also find this release at the official Erlang/OTP Git-repository at > Github here: > https://github.com/erlang/otp tagged *OTP_R16B01* > > The Windows binary distribution can be downloaded from > > http://www.erlang.org/download/otp_win32_R16B01.exe > http://www.erlang.org/download/otp_win64_R16B01.exe > > > On-line documentation can be found at http://www.erlang.org/doc/. > You can also download the complete HTML documentation or the Unix manual > files > > http://www.erlang.org/download/otp_doc_html_R16B01.tar.gz > http://www.erlang.org/download/otp_doc_man_R16B01.tar.gz > > We also want to thank those that sent us patches, suggestions and bug > reports, > > The Erlang/OTP Team at Ericsson > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jleivent@REDACTED Wed Jun 19 20:23:49 2013 From: jleivent@REDACTED (Jonathan Leivent) Date: Wed, 19 Jun 2013 14:23:49 -0400 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially Message-ID: <51C1F735.5040706@comcast.net> Suppose one has multiple conditions that are not guards that need to be tested sequentially (meaning, only test condition N+1 after condition N tests false) in a function. Using a case or if in a simplistic way would then produce considerable nesting, especially as the number of conditions increases above 2 or 3. Is there some way in Erlang to write such code without either breaking up the function clause or using high degrees of nesting? The closest I can come is something like: fun(...) -> C1 = cond1(...), C2 = C1 orelse cond2(...), C3 = C2 orelse cond3(...), if C1 -> ...; C2 -> ...; C3 -> ...; true -> ... end. Which works, but looks rather cryptic. It also seems like it is doing all of the branching at least twice. I guess this also implies the question: why does Erlang require the conditions in an if statement to be guards? -- Jonathan From dmkolesnikov@REDACTED Wed Jun 19 23:50:20 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Thu, 20 Jun 2013 00:50:20 +0300 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <51C1F735.5040706@comcast.net> References: <51C1F735.5040706@comcast.net> Message-ID: <64976996-9E66-4162-B9FD-37048BCA22F7@gmail.com> Hello, I am solving this problem in following way: run() -> maybe_cond1(cond1(), ?). maybe_cond1(false, ?) -> maybe_cond2(cond2(), ?); maybe_cond1(true, ?) -> % do cond1. maybe_cond2(false, ?) -> maybe_cond2(cond3(), ?); maybe_cond2(true, ?) -> % do cond2. maybe_cond3(false, ?) -> % do failure maybe_cond3(true, ?) -> % do cond3. this approach allows you to have as many conditions as you like. The code can be simplified by using lists:fold and closers e.g. lists:foldl(fun assert/2, false, [fun cond1/0, fun cond2/0, fun cond3/0]). assert(_, true) -> true; assert(Fun, false) -> Fun(). cond1() -> case is_cond1() of true -> % do condition 1 true; false -> false end. ? but I do prefer the first approach. Best Regards, Dmitry On Jun 19, 2013, at 9:23 PM, Jonathan Leivent wrote: > Suppose one has multiple conditions that are not guards that need to be tested sequentially (meaning, only test condition N+1 after condition N tests false) in a function. Using a case or if in a simplistic way would then produce considerable nesting, especially as the number of conditions increases above 2 or 3. > > Is there some way in Erlang to write such code without either breaking up the function clause or using high degrees of nesting? > > The closest I can come is something like: > > fun(...) -> > C1 = cond1(...), > C2 = C1 orelse cond2(...), > C3 = C2 orelse cond3(...), > if C1 -> ...; > C2 -> ...; > C3 -> ...; > true -> ... > end. > > Which works, but looks rather cryptic. It also seems like it is doing all of the branching at least twice. > > I guess this also implies the question: why does Erlang require the conditions in an if statement to be guards? > > -- Jonathan > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From siraaj@REDACTED Thu Jun 20 01:57:37 2013 From: siraaj@REDACTED (Siraaj Khandkar) Date: Wed, 19 Jun 2013 19:57:37 -0400 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <64976996-9E66-4162-B9FD-37048BCA22F7@gmail.com> References: <51C1F735.5040706@comcast.net> <64976996-9E66-4162-B9FD-37048BCA22F7@gmail.com> Message-ID: <51C24571.9000700@khandkar.net> On 06/19/2013 05:50 PM, Dmitry Kolesnikov wrote: > Hello, > > I am solving this problem in following way: > > run() -> > maybe_cond1(cond1(), ?). > > maybe_cond1(false, ?) -> > maybe_cond2(cond2(), ?); > maybe_cond1(true, ?) -> > % do cond1. > > maybe_cond2(false, ?) -> > maybe_cond2(cond3(), ?); > maybe_cond2(true, ?) -> > % do cond2. > > maybe_cond3(false, ?) -> > % do failure > maybe_cond3(true, ?) -> > % do cond3. > > this approach allows you to have as many conditions as you like. > The code can be simplified by using lists:fold and closers e.g. > > lists:foldl(fun assert/2, false, [fun cond1/0, fun cond2/0, fun cond3/0]). > > assert(_, true) -> > true; > assert(Fun, false) -> > Fun(). > > cond1() -> > case is_cond1() of > true -> > % do condition 1 > true; > false -> > false > end. > > ? > > but I do prefer the first approach. I like the fold approach, but a bit more specialized: fold_until_error([], X) -> {ok, X}; fold_until_error([F|Fs], X) -> case F(X) of {ok, Y} -> fold_until_error(Fs, Y); {error, _}=E -> E end. do_stuff(Stuff) -> Conditions = [ fun c1/1 , ... , fun cn/1 ], case fold_until_error(Conditions, Stuff) of {ok, Data} -> ...; {error, Reason} -> ... end. One can, of course, use a boolean() instead of ok|error tuples, but I prefer it because you can know exactly what did not pass and for what reason [1]. [1]: http://existentialtype.wordpress.com/2011/03/15/boolean-blindness/ > On Jun 19, 2013, at 9:23 PM, Jonathan Leivent wrote: > >> Suppose one has multiple conditions that are not guards that need to be tested sequentially (meaning, only test condition N+1 after condition N tests false) in a function. Using a case or if in a simplistic way would then produce considerable nesting, especially as the number of conditions increases above 2 or 3. >> >> Is there some way in Erlang to write such code without either breaking up the function clause or using high degrees of nesting? >> >> The closest I can come is something like: >> >> fun(...) -> >> C1 = cond1(...), >> C2 = C1 orelse cond2(...), >> C3 = C2 orelse cond3(...), >> if C1 -> ...; >> C2 -> ...; >> C3 -> ...; >> true -> ... >> end. >> >> Which works, but looks rather cryptic. It also seems like it is doing all of the branching at least twice. >> >> I guess this also implies the question: why does Erlang require the conditions in an if statement to be guards? >> >> -- Jonathan From thomas.burdick@REDACTED Thu Jun 20 02:37:04 2013 From: thomas.burdick@REDACTED (Tom Burdick) Date: Wed, 19 Jun 2013 19:37:04 -0500 Subject: [erlang-questions] OTP R16B01 has been released In-Reply-To: References: Message-ID: I noticed today after updating that public_key now requires asn1, what gives? Eshell V5.10.2 (abort with ^G) 1> application:start(public_key). {error,{not_started,asn1}} Tom On Wed, Jun 19, 2013 at 8:26 AM, Kenneth Lundin wrote: > Erlang/OTP R16B01 has been released. > > OTP R16B01 is a service release with mostly a number of small corrections > and user contributions. But there are some new functions worth mentioning > as well, here are some of them: > > > - Migration of memory carriers between allocator instances, when > enabled it will result in reduced memory footprint when the memory load is > unevenly distributed between scheduler specific allocator instances. > - Interruptible term_to_binary, (thanks to Erik Stenman) > - Added the +sfwi Scheduler Forced Wakeup Interval > - All crypto functions except the block chipers will now chunk large > input into several calls to avoid blocking the scheduler for too long. > - Elliptic curve support in crypto, ssl and ssh(thanks to Andreas > Schultz) > - Lift static limitation (FD_SETSIZE) for file descriptors on Mac OS > X. (thanks to Anthony Ramine) > - Removed a lock around GC statistics counter > - Many small improvements and in Diameter > - 60 user contributions > > You can find the README file with more detailed info at > http://www.erlang.org/download/otp_src_R16B01.readme > > You can download the full source distribution from > http://www.erlang.org/download/otp_src_R16B01.tar.gz > http://www.erlang.org/download/otp_src_R16B01.readme (this file) > > Note: To unpack the TAR archive you need a GNU TAR compatible program. > > For installation instructions please read the README that is part of > the distribution. > > You can also find this release at the official Erlang/OTP Git-repository > at Github here: > https://github.com/erlang/otp tagged *OTP_R16B01* > > The Windows binary distribution can be downloaded from > > http://www.erlang.org/download/otp_win32_R16B01.exe > http://www.erlang.org/download/otp_win64_R16B01.exe > > > On-line documentation can be found at http://www.erlang.org/doc/. > You can also download the complete HTML documentation or the Unix manual > files > > http://www.erlang.org/download/otp_doc_html_R16B01.tar.gz > http://www.erlang.org/download/otp_doc_man_R16B01.tar.gz > > We also want to thank those that sent us patches, suggestions and bug > reports, > > The Erlang/OTP Team at Ericsson > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nem@REDACTED Thu Jun 20 02:48:09 2013 From: nem@REDACTED (Geoff Cant) Date: Wed, 19 Jun 2013 17:48:09 -0700 Subject: [erlang-questions] OTP R16B01 has been released In-Reply-To: References: Message-ID: With each new release that this happens, I remember that I have not yet submitted a patch to the application module to include the lines I put in all my apps: https://github.com/archaelus/erlang-factory-2012-example/blob/master/src/ef_app.erl#L25 and https://github.com/archaelus/erlang-factory-2012-example/blob/master/src/ef_app.erl#L70-L79 These lines together allow you to specify your direct dependencies in your .app file, and use 'AppMod:start().' or 'erl -s AppMod' to do a recursive start of your app and all its transitive dependencies. Interestingly, with correct OTP .app files, 'application' has enough information to start my app and all its dependencies. Currently 'application' is a bit of a jerk - it tells me the exact app that it could have started for me, that is a missing start dependency for the app I am trying to start. These lines may make development a little easier for you right now, and either I, or someone soon to be on my famed Holiday Card List, will hopefully fix this wart before R16B02. Anyone have suggestions for a function name for this feature? application:recursive_start/2? application:start_recursively/2? application:run/2? Cheers, -Geoff (an OTP app developer with sub-optimal laziness) On 2013-06-19, at 17:37 , Tom Burdick wrote: > I noticed today after updating that public_key now requires asn1, what > gives? > > Eshell V5.10.2 (abort with ^G) > 1> application:start(public_key). > {error,{not_started,asn1}} > > Tom > > > On Wed, Jun 19, 2013 at 8:26 AM, Kenneth Lundin wrote: > >> Erlang/OTP R16B01 has been released. >> >> OTP R16B01 is a service release with mostly a number of small corrections >> and user contributions. But there are some new functions worth mentioning >> as well, here are some of them: >> >> >> - Migration of memory carriers between allocator instances, when >> enabled it will result in reduced memory footprint when the memory load is >> unevenly distributed between scheduler specific allocator instances. >> - Interruptible term_to_binary, (thanks to Erik Stenman) >> - Added the +sfwi Scheduler Forced Wakeup Interval >> - All crypto functions except the block chipers will now chunk large >> input into several calls to avoid blocking the scheduler for too long. >> - Elliptic curve support in crypto, ssl and ssh(thanks to Andreas >> Schultz) >> - Lift static limitation (FD_SETSIZE) for file descriptors on Mac OS >> X. (thanks to Anthony Ramine) >> - Removed a lock around GC statistics counter >> - Many small improvements and in Diameter >> - 60 user contributions >> >> You can find the README file with more detailed info at >> http://www.erlang.org/download/otp_src_R16B01.readme >> >> You can download the full source distribution from >> http://www.erlang.org/download/otp_src_R16B01.tar.gz >> http://www.erlang.org/download/otp_src_R16B01.readme (this file) >> >> Note: To unpack the TAR archive you need a GNU TAR compatible program. >> >> For installation instructions please read the README that is part of >> the distribution. >> >> You can also find this release at the official Erlang/OTP Git-repository >> at Github here: >> https://github.com/erlang/otp tagged *OTP_R16B01* >> >> The Windows binary distribution can be downloaded from >> >> http://www.erlang.org/download/otp_win32_R16B01.exe >> http://www.erlang.org/download/otp_win64_R16B01.exe >> >> >> On-line documentation can be found at http://www.erlang.org/doc/. >> You can also download the complete HTML documentation or the Unix manual >> files >> >> http://www.erlang.org/download/otp_doc_html_R16B01.tar.gz >> http://www.erlang.org/download/otp_doc_man_R16B01.tar.gz >> >> We also want to thank those that sent us patches, suggestions and bug >> reports, >> >> The Erlang/OTP Team at Ericsson >> From ok@REDACTED Thu Jun 20 04:15:23 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 20 Jun 2013 14:15:23 +1200 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <51C1F735.5040706@comcast.net> References: <51C1F735.5040706@comcast.net> Message-ID: <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> On 20/06/2013, at 6:23 AM, Jonathan Leivent wrote: > Suppose one has multiple conditions that are not guards that need to be tested sequentially (meaning, only test condition N+1 after condition N tests false) in a function. Ah hah! You want the long-ago-proposed 'cond' form. pick([{Test,Act}|Rest]) -> case Test() of true -> Act() ; false -> f(Rest) end. > fun(...) -> > C1 = cond1(...), > C2 = C1 orelse cond2(...), > C3 = C2 orelse cond3(...), > if C1 -> ...; > C2 -> ...; > C3 -> ...; > true -> ... > end. fun(...) -> pick([ {fun () -> cond1... end, fun () -> action1... end}, {fun () -> cond2... end, fun () -> action2... end}, {fun () -> cond3... end, fun () -> action3... end}, {fun () -> true end, fun () -> action4... end}). Choose your own name for this. However, I cannot help feeling that in each specific instance of this there would be something better to write instead. So how about showing us a real example? > > I guess this also implies the question: why does Erlang require the conditions in an if statement to be guards? Because back when Erlang was invented, there were no conditions that were not guard tests *only*. For example, X > Y was not an expression. There were no andalso or orelse. Ah, the good old days, before Erlang tried to turn into something else but got stuck half way. I really must finish writing up the '-guard' declaration EEP. From thomasl_erlang@REDACTED Thu Jun 20 07:07:55 2013 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Wed, 19 Jun 2013 22:07:55 -0700 (PDT) Subject: [erlang-questions] OTP R16B01 has been released In-Reply-To: References: Message-ID: <1371704875.22487.YahooMailNeo@web140103.mail.bf1.yahoo.com> Sounds great. Unless there are good reasons to keep the current behavior (anyone?), how about just naming your patch start/1? The current behavior, should one need it, might be renamed to start_only/1. Maybe add a boolean function to check if an application has been started. Anything else? Best, Thomas >________________________________ > From: Geoff Cant > >Interestingly, with correct OTP .app files, 'application' has enough information to start my app and all its dependencies. Currently 'application' is a bit of a jerk - it tells me the exact app that it could have started for me, that is a missing start dependency for the app I am trying to start. > >These lines may make development a little easier for you right now, and either I, or someone soon to be on my famed Holiday Card List, will hopefully fix this wart before R16B02. > > >Anyone have suggestions for a function name for this feature? application:recursive_start/2? application:start_recursively/2? application:run/2? > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From 249505968@REDACTED Thu Jun 20 07:57:55 2013 From: 249505968@REDACTED (=?gb18030?B?99L30Q==?=) Date: Thu, 20 Jun 2013 13:57:55 +0800 Subject: [erlang-questions] Is export always better than complie(export_all)? Message-ID: I'm working in a program as 10,000 line code. Most of the time the function are changing. Like and some new function or change some old. The program is not so big that we have no document to describe our module. And all the team member should read all of the code. And I usually use compile expert_all because I feel it make the work easier to handle But one of my teammate said that I must never use export_all. It's that necessary to use export() instead of compile(export_all)? And what's the reason to use export? -------------- next part -------------- An HTML attachment was scrubbed... URL: From goofan.su@REDACTED Thu Jun 20 08:07:48 2013 From: goofan.su@REDACTED (goofansu) Date: Thu, 20 Jun 2013 14:07:48 +0800 Subject: [erlang-questions] =?utf-8?Q?=E5=9B=9E=E5=A4=8D=EF=BC=9A_?= Is export always better than complie(export_all)? In-Reply-To: References: Message-ID: <972B426827324F4D8D0B5313DD300E06@gmail.com> Hi, Exported functions are interfaces to the outsides. Once these functions are used heavily, it is difficult to change a function. For example, add/remove a parameter or change the function's name. -- goofansu ??? Sparrow (http://www.sparrowmailapp.com/?sig) ? 2013?6?20???????1:57??? ??? > I'm working in a program as 10,000 line code. > Most of the time the function are changing. > Like and some new function or change some old. > The program is not so big that we have no document to describe our module. > And all the team member should read all of the code. > And I usually use compile expert_all because I feel it make the work easier to handle > But one of my teammate said that I must never use export_all. > It's that necessary to use export() instead of compile(export_all)? > And what's the reason to use export? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Thu Jun 20 08:10:02 2013 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 20 Jun 2013 08:10:02 +0200 Subject: [erlang-questions] Is export always better than complie(export_all)? In-Reply-To: References: Message-ID: <1371708602.4758.8.camel@sekic1152.rnd.ki.sw.ericsson.se> Greetings, When you use export_all, all functions in the module have to remain backwards compatible. Or you have to go over all other lines of code since they might use any function you change. With -export(). only exported functions need to be backwards compatible. Or trigger a check when you change them. bengt On Thu, 2013-06-20 at 13:57 +0800, ?? wrote: > I'm working in a program as 10,000 line code. > Most of the time the function are changing. > Like and some new function or change some old. > The program is not so big that we have no document to describe our > module. > And all the team member should read all of the code. > And I usually use compile expert_all because I feel it make the work > easier to handle > But one of my teammate said that I must never use export_all. > It's that necessary to use export() instead of compile(export_all)? > And what's the reason to use export? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From 249505968@REDACTED Thu Jun 20 08:10:17 2013 From: 249505968@REDACTED (=?gb18030?B?99L30Q==?=) Date: Thu, 20 Jun 2013 14:10:17 +0800 Subject: [erlang-questions] =?gb18030?b?u9i4tKO6ICBJc2V4cG9ydCBhbHdheXMg?= =?gb18030?q?better_than_complie=28export=5Fall=29=3F?= Message-ID: I know the usage of -expert(). But why it so important? Why should we always use -expert() but not compile(expert_all)? What's the benefit? ------------------ ???? ------------------ ???: "goofansu"; ????: 2013?6?20?(???) ??2:07 ???: "??"<249505968@REDACTED>; ??: "erlang-questions"; ??: ??? [erlang-questions] Isexport always better than complie(export_all)? Hi, Exported functions are interfaces to the outsides. Once these functions are used heavily, it is difficult to change a function. For example, add/remove a parameter or change the function's name. -- goofansu ??? Sparrow ? 2013?6?20???????1:57??? ??? I'm working in a program as 10,000 line code. Most of the time the function are changing. Like and some new function or change some old. The program is not so big that we have no document to describe our module. And all the team member should read all of the code. And I usually use compile expert_all because I feel it make the work easier to handle But one of my teammate said that I must never use export_all. It's that necessary to use export() instead of compile(export_all)? And what's the reason to use export? _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Thu Jun 20 08:30:05 2013 From: vances@REDACTED (Vance Shipley) Date: Thu, 20 Jun 2013 12:00:05 +0530 Subject: [erlang-questions] =?utf-8?b?5Zue5aSN77yaICAgSXNleHBvcnQgYWx3YXlz?= =?utf-8?q?_better_than_complie=28export=5Fall=29=3F?= In-Reply-To: References: Message-ID: <20130620063005.GA43700@aluminium.motivity.ca> On Thu, Jun 20, 2013 at 02:10:17PM +0800, ?? wrote: } I know the usage of -expert(). } But why it so important? } Why should we always use -expert() but not compile(expert_all)? } What's the benefit? If you haven't cleanly seperated the public interfaces to your modules from their internal mechanisms you haven't designed your program, you've only implemented it. You are well on your way to spaghetti code which will be a nightmare to work with. -- -Vance From ok@REDACTED Thu Jun 20 08:31:45 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 20 Jun 2013 18:31:45 +1200 Subject: [erlang-questions] =?utf-8?b?5Zue5aSN77yaICBJcyBleHBvcnQgYWx3YXlz?= =?utf-8?q?_better_than_complie=28export=5Fall=29=3F?= In-Reply-To: <972B426827324F4D8D0B5313DD300E06@gmail.com> References: <972B426827324F4D8D0B5313DD300E06@gmail.com> Message-ID: <2EE3931D-AD23-48CD-B254-3C7E5130AAE1@cs.otago.ac.nz> > ? 2013?6?20???????1:57??? ??? > >> I'm working in a program as 10,000 line code. >> Most of the time the function are changing. So what's new? It was ever thus. >> And all the team member should read all of the code. >> And I usually use compile expert_all because I feel it make the work easier to handle For whom? The more you export, the harder it is for someone to find the one function they need. The more you export, the more others will depend on exactly what the current version does, so the less you will be able to change. You should at all times have a *design* that says what services a module is *intended* to provide to the rest of the system. You MUST export the minimal set of functions that are necessary for *outside* code to use those services. You MAY export some 'convenience' functions defined in terms of the core functions. You SHOULD NOT export anything else. How often I've read "look, here are these neat undocumented features of System X version N", followed a few years later by "Those wicked builders of System X broke all my programs in version N+1 when the undocumented functions changed/went away." >> But one of my teammate said that I must never use export_all. He's wrong too. When debugging, it is useful to be able to refer to anything in a module. But you can do that with compiler command line switches/option lists. >> It's that necessary to use export() instead of compile(export_all)? >> And what's the reason to use export? To make it clear to yourself, to the system, and to other people which functions are part of the interface and may be safely called outside, and which other functions (by omission) are *not*, and may be changed without anyone else having any right to complain. If you export everything, you are promising everyone else who uses your module that you'll try very hard to keep *all* of those functions' interfaces unchanged for a rather long time -- as the project counts time. From ok@REDACTED Thu Jun 20 08:36:34 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 20 Jun 2013 18:36:34 +1200 Subject: [erlang-questions] =?utf-8?b?5Zue5aSN77yaICBJcyBleHBvcnQgYWx3YXlz?= =?utf-8?q?_better_than_complie=28export=5Fall=29=3F?= In-Reply-To: <2EE3931D-AD23-48CD-B254-3C7E5130AAE1@cs.otago.ac.nz> References: <972B426827324F4D8D0B5313DD300E06@gmail.com> <2EE3931D-AD23-48CD-B254-3C7E5130AAE1@cs.otago.ac.nz> Message-ID: <4C906E36-B16C-4D45-8779-087E6A0BB8D4@cs.otago.ac.nz> I should point out that the advice "use -export, not export_all, and use it as little as you can" is _exactly_ the same as the advice "declare every function and top level variable 'static' if you possibly can" given to C and C++ programmers, and for the same reason, and the same as the advice given to Java programmers "prefer private to protected, protected to (package), and package to public; don't make a method or a field public unless you _have_ to in order to get the job done". There is nothing specific to Erlang here. The whole _point_ of modules is to hide as much as you can. See just about any good software engineering book. From 249505968@REDACTED Thu Jun 20 08:43:23 2013 From: 249505968@REDACTED (=?gb18030?B?99L30Q==?=) Date: Thu, 20 Jun 2013 14:43:23 +0800 Subject: [erlang-questions] =?gb18030?b?u9i4tKO6ICC72Li0o7ogIElzIGV4cG9y?= =?gb18030?q?t_always_better_than_complie=28export=5Fall=29=3F?= Message-ID: hmm.... In fact, the module I use compile(expert_all) not like the big module give to others. just for cut the big file into small one. none of others will use the module, it just coupling to another module. It just like some private function in some bigger module.(I know it's better to place all in one module, but too many old code just did like that) And the program I design are always change. Becase it's just like some private function in big module . The function are always change, too. And the change may out of my control..... Everytime change the code I have to fix the export... ------------------ ???? ------------------ ???: "Richard A. O'Keefe"; ????: 2013?6?20?(???) ??2:31 ???: "goofansu"; ??: "??"<249505968@REDACTED>; "erlang-questions"; ??: Re: [erlang-questions] ??? Is export always better than complie(export_all)? > ? 2013?6?20???????1:57??? ??? > >> I'm working in a program as 10,000 line code. >> Most of the time the function are changing. So what's new? It was ever thus. >> And all the team member should read all of the code. >> And I usually use compile expert_all because I feel it make the work easier to handle For whom? The more you export, the harder it is for someone to find the one function they need. The more you export, the more others will depend on exactly what the current version does, so the less you will be able to change. You should at all times have a *design* that says what services a module is *intended* to provide to the rest of the system. You MUST export the minimal set of functions that are necessary for *outside* code to use those services. You MAY export some 'convenience' functions defined in terms of the core functions. You SHOULD NOT export anything else. How often I've read "look, here are these neat undocumented features of System X version N", followed a few years later by "Those wicked builders of System X broke all my programs in version N+1 when the undocumented functions changed/went away." >> But one of my teammate said that I must never use export_all. He's wrong too. When debugging, it is useful to be able to refer to anything in a module. But you can do that with compiler command line switches/option lists. >> It's that necessary to use export() instead of compile(export_all)? >> And what's the reason to use export? To make it clear to yourself, to the system, and to other people which functions are part of the interface and may be safely called outside, and which other functions (by omission) are *not*, and may be changed without anyone else having any right to complain. If you export everything, you are promising everyone else who uses your module that you'll try very hard to keep *all* of those functions' interfaces unchanged for a rather long time -- as the project counts time. -------------- next part -------------- An HTML attachment was scrubbed... URL: From 249505968@REDACTED Thu Jun 20 08:54:37 2013 From: 249505968@REDACTED (=?gb18030?B?99L30Q==?=) Date: Thu, 20 Jun 2013 14:54:37 +0800 Subject: [erlang-questions] =?gb18030?b?u9i4tKO6ICBJcyBleHBvcnQgYWx3YXlz?= =?gb18030?q?_better_than_complie=28export=5Fall=29=3F?= Message-ID: Yep, I can find this word in any book. I just want to know the reason. I use expert_all because most of the time no one will use it. And I don't like just follow some word in book. I think all rule have its realm. Thank you. I get the reason why my teammate just force me to do that. And now I'm going to change my code..... ------------------ Original ------------------ From: "Richard A. O'Keefe"; Date: Thu, Jun 20, 2013 02:36 PM To: "goofansu"; Cc: "??"<249505968@REDACTED>; "erlang-questions"; Subject: Re: [erlang-questions] ??? Is export always better than complie(export_all)? I should point out that the advice "use -export, not export_all, and use it as little as you can" is _exactly_ the same as the advice "declare every function and top level variable 'static' if you possibly can" given to C and C++ programmers, and for the same reason, and the same as the advice given to Java programmers "prefer private to protected, protected to (package), and package to public; don't make a method or a field public unless you _have_ to in order to get the job done". There is nothing specific to Erlang here. The whole _point_ of modules is to hide as much as you can. See just about any good software engineering book. -------------- next part -------------- An HTML attachment was scrubbed... URL: From send2philip@REDACTED Thu Jun 20 10:06:40 2013 From: send2philip@REDACTED (Philip Clarke) Date: Thu, 20 Jun 2013 09:06:40 +0100 Subject: [erlang-questions] reading data from a utf-8 encoded file Message-ID: Hi, I am trying to get the basics of reading data from a utf-8 file. >From the docs ( http://www.erlang.org/doc/apps/stdlib/unicode_usage.html#id62290), it says: "It is slightly confusing that a file opened with e.g. file:open(Name,[read,{encoding,utf8}]), cannot be properly read using file:read(File,N) but you have to use the io module to retrieve the Unicode data from it." So I tested this out by writing some unicode to a file: Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] Eshell V5.10.1 (abort with ^G) 1> {ok, InputDevice} = file:open("/tmp/test.utf8", [write, {encoding, unicode}]). {ok,<0.35.0>} 2> io:put_chars(InputDevice, <<"??????"/utf8>>). ok 3> file:close(InputDevice). and then read this back using io:get_line/2 4> {ok, OutputDevice} = file:open("/tmp/test.utf8", [read, {encoding, unicode}]). {ok,<0.39.0>} 5> io:get_line(OutputDevice, ""). "??????" 6> file:close(OutputDevice). So far so good, but I also get the same result when I read in the file using file:read_line/1 7> f(). ok 8> {ok, InputDevice} = file:open("/tmp/test.utf8", [read, {encoding, unicode}]). {ok,<0.44.0>} 9> file:read_line(InputDevice). {ok,"??????"} 10> file:close(InputDevice). So is it really wrong to use file:read_line/1 ? It seems to give the correct result. I suspect that file:read_line/1 is just reading in a list of bytes from the file. The unicode string in the example is represented by the bytes show here: 11> unicode:characters_to_binary("??????"). <<208,174,208,189,208,184,208,186,208,190,208,180>> Which (using unicode:characters_to_list) translates to: [1070,1085,1080,1082,1086,1076] = "??????" Another reason I ask this question is that it seems wrong to use io:get_line/2 as this requires a value for a Prompt which is not used when reading from a file. Thanks in advance Philip -------------- next part -------------- An HTML attachment was scrubbed... URL: From thebullno1@REDACTED Thu Jun 20 10:23:29 2013 From: thebullno1@REDACTED (Bach Le) Date: Thu, 20 Jun 2013 16:23:29 +0800 Subject: [erlang-questions] Callback modules, 3rd party libraries and pooling Message-ID: Recently, while trying to understand OTP better, I looked into the significance of the "modules" list in a child spec. Apparently, it is used to suspend processes during upgrade. 'dynamic' is a special case and a get_modules sys message will be sent to the process. Two problems come to my mind: 1) Many 3rd party libraries only provide start_link functions and many times, the module with the start_link function is not the correct callback module. epgsql for example, provide pgsql:connect but the actual callback module is pgsql_connection. To be able to provide the correct module, library users have to look into the source code. Moreover, specifying the callback module or even child spec yourself sounds like an abstraction leak to me. IMO, it is time to make it common practice to provide a child_spec function which returns child spec for embedding in another application. 2) All pooling libraries that I know of (hottub, pooler, poolboy) assume that the module in the worker's MFA is the callback module, which is "wrong". Besides what I said in 1), there are also times when I want to run some custom initialization with a pooled resource after starting it. In which case, the module which contains the custom start_link function is assumed to be the callback module. poolboy, for example, only accepts start_link/1 so a wrapper is a must for many libraries. While most database drivers are quite stable so they don't need to be upgraded too often, connections are mostly transient resources so restarting the pool is enough, this practice is still kind of problematic. What are your thoughts on this? -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Thu Jun 20 12:48:20 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 20 Jun 2013 12:48:20 +0200 Subject: [erlang-questions] OTP R16B01 has been released In-Reply-To: References: Message-ID: <51C2DDF4.2050103@ninenines.eu> Congratulations! It seems crypto:sha/1 is now deprecated. I think it's to favor crypto:hash/2, but the details are obscure there as the readme only talks about a "new API". Can we get more info on what's been deprecated and when was crypto:hash/2 or others added? On 06/19/2013 03:26 PM, Kenneth Lundin wrote: > Erlang/OTP R16B01 has been released. > > OTP R16B01 is a service release with mostly a number of small > corrections and user contributions. But there are some new functions > worth mentioning as well, here are some of them: > > * Migration of memory carriers between allocator instances, when > enabled it will result in reduced memory footprint when the memory > load is unevenly distributed between scheduler specific allocator > instances. > * Interruptible term_to_binary, (thanks to Erik Stenman) > * Added the +sfwi Scheduler Forced Wakeup Interval > * All crypto functions except the block chipers will now chunk large > input into several calls to avoid blocking the scheduler for too long. > * Elliptic curve support in crypto, ssl and ssh(thanks to Andreas > Schultz) > * Lift static limitation (FD_SETSIZE) for file descriptors on Mac OS > X. (thanks to Anthony Ramine) > * Removed a lock around GC statistics counter > * Many small improvements and in Diameter > * 60 user contributions > > You can find the README file with more detailed info at > http://www.erlang.org/download/otp_src_R16B01.readme > > > You can download the full source distribution from > http://www.erlang.org/download/otp_src_R16B01.tar.gz > http://www.erlang.org/download/otp_src_R16B01.readme (this file) > > Note: To unpack the TAR archive you need a GNU TAR compatible program. > > For installation instructions please read the README that is part of > the distribution. > > You can also find this release at the official Erlang/OTP Git-repository > at Github here: > https://github.com/erlang/otp tagged _OTP_R16B01_ > > The Windows binary distribution can be downloaded from > > http://www.erlang.org/download/otp_win32_R16B01.exe > http://www.erlang.org/download/otp_win64_R16B01.exe > > > On-line documentation can be found at http://www.erlang.org/doc/. > You can also download the complete HTML documentation or the Unix manual > files > > http://www.erlang.org/download/otp_doc_html_R16B01.tar.gz > http://www.erlang.org/download/otp_doc_man_R16B01.tar.gz > > We also want to thank those that sent us patches, suggestions and bug > reports, > > The Erlang/OTP Team at Ericsson > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From kenneth.lundin@REDACTED Thu Jun 20 13:32:01 2013 From: kenneth.lundin@REDACTED (Kenneth Lundin) Date: Thu, 20 Jun 2013 13:32:01 +0200 Subject: [erlang-questions] OTP R16B01 has been released In-Reply-To: References: Message-ID: The requirement of asn.1 from public_key was introduced already in R16B but without that beeing reflected in the app file. We had several reports from people having trouble with for example SSL in R16B because asn.1 was missing in their system and the error indication was not obvious. Now it is unfortunate that application start complains about asn.1 not beeing started when asn1 not needs to be started since it is only a library , it does not start any processes. The same goes for public_key it does not start any processes and there is no need to start it. The distinction between library applications and "living" applications should be addressed in the application module. /Kenneth , Erlang/OTP Ericsson Den 20 jun 2013 02:37 skrev "Tom Burdick" : > I noticed today after updating that public_key now requires asn1, what > gives? > > Eshell V5.10.2 (abort with ^G) > 1> application:start(public_key). > {error,{not_started,asn1}} > > Tom > > > On Wed, Jun 19, 2013 at 8:26 AM, Kenneth Lundin wrote: > >> Erlang/OTP R16B01 has been released. >> >> OTP R16B01 is a service release with mostly a number of small corrections >> and user contributions. But there are some new functions worth mentioning >> as well, here are some of them: >> >> >> - Migration of memory carriers between allocator instances, when >> enabled it will result in reduced memory footprint when the memory load is >> unevenly distributed between scheduler specific allocator instances. >> - Interruptible term_to_binary, (thanks to Erik Stenman) >> - Added the +sfwi Scheduler Forced Wakeup Interval >> - All crypto functions except the block chipers will now chunk large >> input into several calls to avoid blocking the scheduler for too long. >> - Elliptic curve support in crypto, ssl and ssh(thanks to Andreas >> Schultz) >> - Lift static limitation (FD_SETSIZE) for file descriptors on Mac OS >> X. (thanks to Anthony Ramine) >> - Removed a lock around GC statistics counter >> - Many small improvements and in Diameter >> - 60 user contributions >> >> You can find the README file with more detailed info at >> http://www.erlang.org/download/otp_src_R16B01.readme >> >> You can download the full source distribution from >> http://www.erlang.org/download/otp_src_R16B01.tar.gz >> http://www.erlang.org/download/otp_src_R16B01.readme (this file) >> >> Note: To unpack the TAR archive you need a GNU TAR compatible program. >> >> For installation instructions please read the README that is part of >> the distribution. >> >> You can also find this release at the official Erlang/OTP Git-repository >> at Github here: >> https://github.com/erlang/otp tagged *OTP_R16B01* >> >> The Windows binary distribution can be downloaded from >> >> http://www.erlang.org/download/otp_win32_R16B01.exe >> http://www.erlang.org/download/otp_win64_R16B01.exe >> >> >> On-line documentation can be found at http://www.erlang.org/doc/. >> You can also download the complete HTML documentation or the Unix manual >> files >> >> http://www.erlang.org/download/otp_doc_html_R16B01.tar.gz >> http://www.erlang.org/download/otp_doc_man_R16B01.tar.gz >> >> We also want to thank those that sent us patches, suggestions and bug >> reports, >> >> The Erlang/OTP Team at Ericsson >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From markb@REDACTED Thu Jun 20 13:59:36 2013 From: markb@REDACTED (markb@REDACTED) Date: Thu, 20 Jun 2013 20:59:36 +0900 (JST) Subject: [erlang-questions] OpenCL binding test failure Message-ID: <3307329.1371729576628.JavaMail.markb@parkcity.ne.jp> Greetings, I'm having trouble with the OpenCL bindings by TonyRog (https://github.com/tonyrog/cl). Machine is a Mac Mini running OS X 10.8.4, with Erlang R16B01 (tried with both 64-bit and 32-bit builds). $ erl -pa cl/ebin Erlang R16B01 (erts-5.10.2) [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] Eshell V5.10.2 (abort with ^G) 1> cl_basic:test(). PlatformInfo: [{profile,"FULL_PROFILE"}, {version,"OpenCL 1.2 (Apr 25 2013 18:32:06)"}, {name,"Apple"}, {vendor,"Apple"}, {extensions,"cl_APPLE_SetMemObjectDestructor cl_APPLE_ContextLoggingFunctions cl_APPLE_clut cl_APPLE_query_kernel_names cl_APPLE_gl_sharing cl_khr_gl_event"}] Device: {device_t,429392760,<<>>} DeviceInfo: [{type,[cpu]}, {vendor_id,4294967295}, <> Sampler2Info: [{reference_count,1}, {context,{context_t,429392848,<<>>}}, {normalized_coords,false}, {addressing_mode,repeat}, {filter_mode,linear}] Program: {program_t,429647808,<<>>} Trace/BPT trap: 5 I also tried the fix described elsewhere to add "-framework Foundation" to LIBCARBON in erts/configure.in when building Erlang, but same result. The failure also occurs with the binary Erlang builds. Would appreciate any assistance. From essen@REDACTED Thu Jun 20 16:15:34 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Thu, 20 Jun 2013 16:15:34 +0200 Subject: [erlang-questions] [ANN] Cowboy 0.8.6 and Ranch 0.8.4 (R16B01 compatible) Message-ID: <51C30E86.8050305@ninenines.eu> Hello, Just a heads up, a new version of Cowboy and Ranch has been pushed that fixes compilation and documentation for R16B01: you now need the asn1 application to be started when using SSL. They will be started automatically if they weren't before, but you will need them in your release (done automatically with relx). Other interesting changes for Cowboy include: * Support for compression of Websocket frames (thanks Ali Sabil) * Initial support for SPDY Both are experimental features and undocumented at this point, but feedback is more than welcome. See the changelog for more information. * https://github.com/extend/cowboy * https://github.com/extend/ranch Enjoy! -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From thomas.burdick@REDACTED Thu Jun 20 16:51:12 2013 From: thomas.burdick@REDACTED (Tom Burdick) Date: Thu, 20 Jun 2013 09:51:12 -0500 Subject: [erlang-questions] OTP R16B01 has been released In-Reply-To: References: Message-ID: I like the idea given above of having an intelligent start function. It obviously already knows which applications need to be started, why bother telling me instead of just starting them? If the current behaviour is required application:start_all(myapp) would be cool. It'd save me some hassles. Tom On Thu, Jun 20, 2013 at 6:32 AM, Kenneth Lundin wrote: > The requirement of asn.1 from public_key was introduced already in R16B > but without that beeing reflected in the app file. > We had several reports from people having trouble with for example SSL in > R16B because asn.1 was missing in their system and the error indication was > not obvious. > > Now it is unfortunate that application start complains about asn.1 not > beeing started when asn1 not needs to be started since it is only a library > , it does not start any processes. The same goes for public_key it does not > start any processes and there is no need to start it. > The distinction between library applications and "living" applications > should be addressed in the application module. > > /Kenneth , Erlang/OTP Ericsson > Den 20 jun 2013 02:37 skrev "Tom Burdick" : > > I noticed today after updating that public_key now requires asn1, what >> gives? >> >> Eshell V5.10.2 (abort with ^G) >> 1> application:start(public_key). >> {error,{not_started,asn1}} >> >> Tom >> >> >> On Wed, Jun 19, 2013 at 8:26 AM, Kenneth Lundin > > wrote: >> >>> Erlang/OTP R16B01 has been released. >>> >>> OTP R16B01 is a service release with mostly a number of small >>> corrections and user contributions. But there are some new functions worth >>> mentioning as well, here are some of them: >>> >>> >>> - Migration of memory carriers between allocator instances, when >>> enabled it will result in reduced memory footprint when the memory load is >>> unevenly distributed between scheduler specific allocator instances. >>> - Interruptible term_to_binary, (thanks to Erik Stenman) >>> - Added the +sfwi Scheduler Forced Wakeup Interval >>> - All crypto functions except the block chipers will now chunk large >>> input into several calls to avoid blocking the scheduler for too long. >>> - Elliptic curve support in crypto, ssl and ssh(thanks to Andreas >>> Schultz) >>> - Lift static limitation (FD_SETSIZE) for file descriptors on Mac OS >>> X. (thanks to Anthony Ramine) >>> - Removed a lock around GC statistics counter >>> - Many small improvements and in Diameter >>> - 60 user contributions >>> >>> You can find the README file with more detailed info at >>> http://www.erlang.org/download/otp_src_R16B01.readme >>> >>> You can download the full source distribution from >>> http://www.erlang.org/download/otp_src_R16B01.tar.gz >>> http://www.erlang.org/download/otp_src_R16B01.readme (this file) >>> >>> Note: To unpack the TAR archive you need a GNU TAR compatible program. >>> >>> For installation instructions please read the README that is part of >>> the distribution. >>> >>> You can also find this release at the official Erlang/OTP >>> Git-repository at Github here: >>> https://github.com/erlang/otp tagged *OTP_R16B01* >>> >>> The Windows binary distribution can be downloaded from >>> >>> http://www.erlang.org/download/otp_win32_R16B01.exe >>> http://www.erlang.org/download/otp_win64_R16B01.exe >>> >>> >>> On-line documentation can be found at http://www.erlang.org/doc/. >>> You can also download the complete HTML documentation or the Unix manual >>> files >>> >>> http://www.erlang.org/download/otp_doc_html_R16B01.tar.gz >>> http://www.erlang.org/download/otp_doc_man_R16B01.tar.gz >>> >>> We also want to thank those that sent us patches, suggestions and bug >>> reports, >>> >>> The Erlang/OTP Team at Ericsson >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Thu Jun 20 17:00:01 2013 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 20 Jun 2013 11:00:01 -0400 Subject: [erlang-questions] OTP R16B01 has been released In-Reply-To: References: Message-ID: <20130620150000.GA93763@ferdair.local> I've started working on one already. If all goes well with the tests, I should have a mail in erlang-patches later today On 06/20, Tom Burdick wrote: > I like the idea given above of having an intelligent start function. It > obviously already knows which applications need to be started, why bother > telling me instead of just starting them? > > If the current behaviour is required application:start_all(myapp) would be > cool. It'd save me some hassles. > > Tom > > > On Thu, Jun 20, 2013 at 6:32 AM, Kenneth Lundin wrote: > > > The requirement of asn.1 from public_key was introduced already in R16B > > but without that beeing reflected in the app file. > > We had several reports from people having trouble with for example SSL in > > R16B because asn.1 was missing in their system and the error indication was > > not obvious. > > > > Now it is unfortunate that application start complains about asn.1 not > > beeing started when asn1 not needs to be started since it is only a library > > , it does not start any processes. The same goes for public_key it does not > > start any processes and there is no need to start it. > > The distinction between library applications and "living" applications > > should be addressed in the application module. > > > > /Kenneth , Erlang/OTP Ericsson > > Den 20 jun 2013 02:37 skrev "Tom Burdick" : > > > > I noticed today after updating that public_key now requires asn1, what > >> gives? > >> > >> Eshell V5.10.2 (abort with ^G) > >> 1> application:start(public_key). > >> {error,{not_started,asn1}} > >> > >> Tom > >> > >> > >> On Wed, Jun 19, 2013 at 8:26 AM, Kenneth Lundin >> > wrote: > >> > >>> Erlang/OTP R16B01 has been released. > >>> > >>> OTP R16B01 is a service release with mostly a number of small > >>> corrections and user contributions. But there are some new functions worth > >>> mentioning as well, here are some of them: > >>> > >>> > >>> - Migration of memory carriers between allocator instances, when > >>> enabled it will result in reduced memory footprint when the memory load is > >>> unevenly distributed between scheduler specific allocator instances. > >>> - Interruptible term_to_binary, (thanks to Erik Stenman) > >>> - Added the +sfwi Scheduler Forced Wakeup Interval > >>> - All crypto functions except the block chipers will now chunk large > >>> input into several calls to avoid blocking the scheduler for too long. > >>> - Elliptic curve support in crypto, ssl and ssh(thanks to Andreas > >>> Schultz) > >>> - Lift static limitation (FD_SETSIZE) for file descriptors on Mac OS > >>> X. (thanks to Anthony Ramine) > >>> - Removed a lock around GC statistics counter > >>> - Many small improvements and in Diameter > >>> - 60 user contributions > >>> > >>> You can find the README file with more detailed info at > >>> http://www.erlang.org/download/otp_src_R16B01.readme > >>> > >>> You can download the full source distribution from > >>> http://www.erlang.org/download/otp_src_R16B01.tar.gz > >>> http://www.erlang.org/download/otp_src_R16B01.readme (this file) > >>> > >>> Note: To unpack the TAR archive you need a GNU TAR compatible program. > >>> > >>> For installation instructions please read the README that is part of > >>> the distribution. > >>> > >>> You can also find this release at the official Erlang/OTP > >>> Git-repository at Github here: > >>> https://github.com/erlang/otp tagged *OTP_R16B01* > >>> > >>> The Windows binary distribution can be downloaded from > >>> > >>> http://www.erlang.org/download/otp_win32_R16B01.exe > >>> http://www.erlang.org/download/otp_win64_R16B01.exe > >>> > >>> > >>> On-line documentation can be found at http://www.erlang.org/doc/. > >>> You can also download the complete HTML documentation or the Unix manual > >>> files > >>> > >>> http://www.erlang.org/download/otp_doc_html_R16B01.tar.gz > >>> http://www.erlang.org/download/otp_doc_man_R16B01.tar.gz > >>> > >>> We also want to thank those that sent us patches, suggestions and bug > >>> reports, > >>> > >>> The Erlang/OTP Team at Ericsson > >>> > >>> _______________________________________________ > >>> erlang-questions mailing list > >>> erlang-questions@REDACTED > >>> http://erlang.org/mailman/listinfo/erlang-questions > >>> > >>> > >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From jleivent@REDACTED Thu Jun 20 17:29:40 2013 From: jleivent@REDACTED (Jonathan Leivent) Date: Thu, 20 Jun 2013 11:29:40 -0400 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> References: <51C1F735.5040706@comcast.net> <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> Message-ID: <51C31FE4.4030100@comcast.net> On 06/19/2013 10:15 PM, Richard A. O'Keefe wrote: > > On 20/06/2013, at 6:23 AM, Jonathan Leivent wrote: > >> Suppose one has multiple conditions that are not guards that need to be tested sequentially (meaning, only test condition N+1 after condition N tests false) in a function. > > Ah hah! You want the long-ago-proposed 'cond' form. > > pick([{Test,Act}|Rest]) -> > case Test() > of true -> Act() > ; false -> f(Rest) > end. >> fun(...) -> >> C1 = cond1(...), >> C2 = C1 orelse cond2(...), >> C3 = C2 orelse cond3(...), >> if C1 -> ...; >> C2 -> ...; >> C3 -> ...; >> true -> ... >> end. > > fun(...) -> > pick([ > {fun () -> cond1... end, > fun () -> action1... end}, > {fun () -> cond2... end, > fun () -> action2... end}, > {fun () -> cond3... end, > fun () -> action3... end}, > {fun () -> true end, > fun () -> action4... end}). > > Choose your own name for this. I was hoping for something that more syntactically resembles a classic if/elsif/else block. Without closures. I started playing with some macros, and came up with this: -define(IF(C), try (C) andalso begin). -define(ELSIF(C), true end orelse (C) andalso begin). -define(ELSE, true end orelse begin). -define(ENDIF, true end catch {if_result, IF_RESULT} -> IF_RESULT end). -define(BREAK(R), throw({if_result, R})). foo(X) -> ?IF(cond1(X)) expr1, expr2, ?BREAK(expr3), ?ELSIF(cond2(X)) expr4, expr5, ?BREAK(expr6), ?ELSIF(cond3(X)) expr7, expr8, ?BREAK(expr9), ?ELSE expr0, expra, ?BREAK(exprb), ?ENDIF. One downside is that emacs isn't smart enough to indent this correctly, since it doesn't expand macros. Also, to play better with other things, including other such ?IF statements in the same scope, the ?ENDIF macro shouldn't bind a variable - but I haven't figured out how to make that not happen. > > However, I cannot help feeling that in each specific instance > of this there would be something better to write instead. So > how about showing us a real example? Anything that would have worked in an if statement were the conditions just guards. In the case I'm interested in, the conditions are predicates on an opaque type - so they can't be guards. >> >> I guess this also implies the question: why does Erlang require the conditions in an if statement to be guards? > > Because back when Erlang was invented, there were no conditions that > were not guard tests *only*. For example, X > Y was not an expression. > There were no andalso or orelse. OK - I will then add a word to the above question: why does Erlang *still* require the conditions in an if statement to be guards? How hard is it to remove this restriction? -- Jonathan From n.oxyde@REDACTED Thu Jun 20 17:42:27 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Thu, 20 Jun 2013 17:42:27 +0200 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <51C31FE4.4030100@comcast.net> References: <51C1F735.5040706@comcast.net> <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> <51C31FE4.4030100@comcast.net> Message-ID: <96590E43-C204-44DA-A9B7-2DD17C01FBFB@gmail.com> Because changing it at this point would be weird. And because guards have special semantics and can't fail. Supporting expressions now would mean weird semantics like in comprehensions, where you can't distinguish a guard from an expression and know that they are compiled differently, or breaking existing code such as "if length(X) =:= 3 -> ... ; ... end": if X is not a list and that expression is evaluated with guard semantics, the first clause is skipped, whereas if it is evaluated with expression semantics, the code crashes. If I understood things correctly, Erlang is supposed to have a cond Clauses end expression ?try to evaluate "cond" in Erlang, it's a syntax error because cond is a reserved keyword?, which is the same as if but with tests compiled with expression semantics. Maybe it should be implemented for real. Regards, -- Anthony Ramine Le 20 juin 2013 ? 17:29, Jonathan Leivent a ?crit : > OK - I will then add a word to the above question: why does Erlang *still* require the conditions in an if statement to be guards? > > How hard is it to remove this restriction? From jleivent@REDACTED Thu Jun 20 18:27:25 2013 From: jleivent@REDACTED (Jonathan Leivent) Date: Thu, 20 Jun 2013 12:27:25 -0400 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <96590E43-C204-44DA-A9B7-2DD17C01FBFB@gmail.com> References: <51C1F735.5040706@comcast.net> <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> <51C31FE4.4030100@comcast.net> <96590E43-C204-44DA-A9B7-2DD17C01FBFB@gmail.com> Message-ID: <51C32D6D.9000700@comcast.net> On 06/20/2013 11:42 AM, Anthony Ramine wrote: > Because changing it at this point would be weird. And because guards have special semantics and can't fail. > > Supporting expressions now would mean weird semantics like in comprehensions, where you can't distinguish a guard from an expression and know that they are compiled differently, or breaking existing code such as "if length(X) =:= 3 -> ... ; ... end": if X is not a list and that expression is evaluated with guard semantics, the first clause is skipped, whereas if it is evaluated with expression semantics, the code crashes. > OK - that is surprising. That certainly changes my thinking about if statements. Is this particular semantic difference between guards and general conditions ever expressed in the reference manual? If so, I certainly missed it. If I had read some code that just said: if length(X) =:= 3 -> ... I would have inferred (incorrectly) that it would be an error for X to not be a list. > If I understood things correctly, Erlang is supposed to have a cond Clauses end expression ?try to evaluate "cond" in Erlang, it's a syntax error because cond is a reserved keyword?, which is the same as if but with tests compiled with expression semantics. Maybe it should be implemented for real. Perhaps that is a remnant of "the long-ago-proposed 'cond' form" that Richard mentioned. -- Jonathan From erlang@REDACTED Thu Jun 20 18:30:28 2013 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 20 Jun 2013 18:30:28 +0200 Subject: [erlang-questions] Is export always better than complie(export_all)? In-Reply-To: References: Message-ID: export_all should only be used when debugging In production you should only explicitly export the functions you need. By exporting a subset of the functions: - your programs are easier to understand. - the compiler could perform better optimisations - the type checker (dialyzer) can infer better types /Joe On Thu, Jun 20, 2013 at 7:57 AM, ?? <249505968@REDACTED> wrote: > I'm working in a program as 10,000 line code. > Most of the time the function are changing. > Like and some new function or change some old. > The program is not so big that we have no document to describe our module. > And all the team member should read all of the code. > And I usually use compile expert_all because I feel it make the work > easier to handle > But one of my teammate said that I must never use export_all. > It's that necessary to use export() instead of compile(export_all)? > And what's the reason to use export? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jleivent@REDACTED Thu Jun 20 19:33:11 2013 From: jleivent@REDACTED (Jonathan Leivent) Date: Thu, 20 Jun 2013 13:33:11 -0400 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <51C32D6D.9000700@comcast.net> References: <51C1F735.5040706@comcast.net> <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> <51C31FE4.4030100@comcast.net> <96590E43-C204-44DA-A9B7-2DD17C01FBFB@gmail.com> <51C32D6D.9000700@comcast.net> Message-ID: <51C33CD7.1090109@comcast.net> > OK - that is surprising. That certainly changes my thinking about if > statements. Is this particular semantic difference between guards and > general conditions ever expressed in the reference manual? If so, I > certainly missed it. On closer inspection: "If an arithmetic expression, a boolean expression, a short-circuit expression, or a call to a guard BIF fails (because of invalid arguments), the entire guard fails. " Before understanding this semantic peculiarity of guards, the parenthetical portion of the above statement in the reference manual didn't make much of an impression on me. Now, I see it is conveying this semantics. Actually, when I read that sentence alone, it still doesn't quite do the trick. It's the following sentence: "If the guard was part of a guard sequence, the next guard in the sequence (that is, the guard following the next semicolon) will be evaluated." that completes the meaning by disambiguating "fails" as "evaluates to false" instead of "throws an exception". Although, still, I'd probably make the mistake of typing "length(0)." into the interpreter, seeing the error, and think that this experience overrules what I might have otherwise inferred from the manual. To further confuse matters (or, maybe just me), many guards are is_type tests - which would seem to be useful, but are not, as protection for other guards in the same guard expression: if is_list(X), length(X) =:= 3 -> ... Previous to this discussion, the reference manual statement: "The reason for restricting the set of valid expressions is that evaluation of a guard expression must be guaranteed to be free of side effects." was the one that I based my (mis)understanding of the guard/condition distinction on. I could understand why that would be an important restriction for guards on things like function and receive clauses, but not so much on if statements - or, at least, why there would not be any Erlang statement that would allow testing a sequence of arbitrary conditions. Doesn't guard semantics violate the Principle of Least Astonishment? I'd be astonished if I was alone in thinking that... -- Jonathan From Christopher.Phillips@REDACTED Thu Jun 20 19:54:56 2013 From: Christopher.Phillips@REDACTED (Phillips, Christopher) Date: Thu, 20 Jun 2013 17:54:56 +0000 Subject: [erlang-questions] Distributed Application Without Priority? Message-ID: We have a system currently that has two nodes that a distributed application runs across. We've talked about what it would take to make it run on both, but for now it's easier to just fail over when one node goes down. We have no real need for priority in this. We really don't care which the app runs on, just that it's running somewhere. Put another way; I'd like to support failover, while preventing take over. I, first, wondered if there was a mechanism that supported this; everything I see for distributed applications describes listing them in priority order, rather than without a priority. I can see at least one way I might achieve preventing takeover, but it seems like if it was unintentional it may have some unexpected side effects; namely, make it so node 1 lists node 2 as higher preference, and node 2 lists node 1 as higher preference, which would achieve, to my best understanding, what I want (a quick smoke test showed it working like I'd expect), but I'm not sure what risks it might carry with it. My second question is, does anyone know what issues this might cause if we tried running things this way? -------------- next part -------------- An HTML attachment was scrubbed... URL: From ulf@REDACTED Thu Jun 20 20:51:14 2013 From: ulf@REDACTED (Ulf Wiger) Date: Thu, 20 Jun 2013 20:51:14 +0200 Subject: [erlang-questions] Distributed Application Without Priority? In-Reply-To: References: Message-ID: <81091BE6-6556-4478-8D87-EA412CF9B866@feuerlabs.com> If you check the docs for application:load/2, it says: "Nodes is a list of node names where the application may run, in priority from left to right. Node names can be grouped using tuples to indicate that they have the same priority. Example: Nodes = [cp1@REDACTED, {cp2@REDACTED, cp3@REDACTED}]" In the example above, cp2@REDACTED and cp3@REDACTED have the same priority. Takeover happens either when explicitly ordered when a higher-priority node appears. BR, Ulf W Ulf Wiger, Feuerlabs, Inc. http://feuerlabs.com 20 jun 2013 kl. 19:54 skrev "Phillips, Christopher" : > > > We have a system currently that has two nodes that a distributed application runs across. We've talked about what it would take to make it run on both, but for now it's easier to just fail over when one node goes down. > > We have no real need for priority in this. We really don't care which the app runs on, just that it's running somewhere. Put another way; I'd like to support failover, while preventing take over. > > I, first, wondered if there was a mechanism that supported this; everything I see for distributed applications describes listing them in priority order, rather than without a priority. > > I can see at least one way I might achieve preventing takeover, but it seems like if it was unintentional it may have some unexpected side effects; namely, make it so node 1 lists node 2 as higher preference, and node 2 lists node 1 as higher preference, which would achieve, to my best understanding, what I want (a quick smoke test showed it working like I'd expect), but I'm not sure what risks it might carry with it. My second question is, does anyone know what issues this might cause if we tried running things this way? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Fri Jun 21 00:07:46 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Fri, 21 Jun 2013 00:07:46 +0200 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <51C33CD7.1090109@comcast.net> References: <51C1F735.5040706@comcast.net> <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> <51C31FE4.4030100@comcast.net> <96590E43-C204-44DA-A9B7-2DD17C01FBFB@gmail.com> <51C32D6D.9000700@comcast.net> <51C33CD7.1090109@comcast.net> Message-ID: <7F1C7362-3A79-401F-B6FB-FEA199CAA913@gmail.com> I don't believe they do in places where guards are the only inhabitant. I do believe they violate it in comprehensions; call these two functions with [foo]: F = fun (L) -> [ X || X <- L, X rem 2 =:= 0 ], G = fun (L) -> [ X || X <- L, _ = X rem 2 =:= 0 ]. Regards, PS: Sorry for the double mail Jonathan, failed to reply to all. -- Anthony Ramine Le 20 juin 2013 ? 19:33, Jonathan Leivent a ?crit : > >> OK - that is surprising. That certainly changes my thinking about if >> statements. Is this particular semantic difference between guards and >> general conditions ever expressed in the reference manual? If so, I >> certainly missed it. > > On closer inspection: > > "If an arithmetic expression, a boolean expression, a short-circuit expression, or a call to a guard BIF fails (because of invalid arguments), the entire guard fails. " > > Before understanding this semantic peculiarity of guards, the parenthetical portion of the above statement in the reference manual didn't make much of an impression on me. Now, I see it is conveying this semantics. Actually, when I read that sentence alone, it still doesn't quite do the trick. It's the following sentence: > > "If the guard was part of a guard sequence, the next guard in the sequence (that is, the guard following the next semicolon) will be evaluated." > > that completes the meaning by disambiguating "fails" as "evaluates to false" instead of "throws an exception". Although, still, I'd probably make the mistake of typing "length(0)." into the interpreter, seeing the error, and think that this experience overrules what I might have otherwise inferred from the manual. To further confuse matters (or, maybe just me), many guards are is_type tests - which would seem to be useful, but are not, as protection for other guards in the same guard expression: > > if is_list(X), length(X) =:= 3 -> ... > > Previous to this discussion, the reference manual statement: "The reason for restricting the set of valid expressions is that evaluation of a guard expression must be guaranteed to be free of side effects." was the one that I based my (mis)understanding of the guard/condition distinction on. I could understand why that would be an important restriction for guards on things like function and receive clauses, but not so much on if statements - or, at least, why there would not be any Erlang statement that would allow testing a sequence of arbitrary conditions. > > Doesn't guard semantics violate the Principle of Least Astonishment? > > I'd be astonished if I was alone in thinking that... > > -- Jonathan > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Fri Jun 21 01:01:49 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 21 Jun 2013 11:01:49 +1200 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <51C31FE4.4030100@comcast.net> References: <51C1F735.5040706@comcast.net> <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> <51C31FE4.4030100@comcast.net> Message-ID: On 21/06/2013, at 3:29 AM, Jonathan Leivent wrote: > ?ELSE > expr0, > expra, > ?BREAK(exprb), > ?ENDIF. The ?BREAK is rather horrible; to my mind, far more horrible than nice clean obvious closures. I wrote: > >> >> However, I cannot help feeling that in each specific instance >> of this there would be something better to write instead. So >> how about showing us a real example? > > Anything that would have worked in an if statement were the conditions just guards. In the case I'm interested in, the conditions are predicates on an opaque type - so they can't be guards. That's interesting, but I doubt that anyone ever thought you _could_ have written your code as a simple 'if'. *How about showing us a real example?* As in, actual code. Real examples often show opportunities for restructuring in completely different ways. Also, I'm having a hard time imagining so many sequentially tested conditions that case C1 of true -> A1 ; false -> case C2 of true -> A2 ; false -> ... end end is really hard to write. It's certainly what cond C1 -> A1 ; C2 -> C2 ; ... end would compile into, if it existed. >> Because back when Erlang was invented, there were no conditions that >> were not guard tests *only*. For example, X > Y was not an expression. >> There were no andalso or orelse. > > OK - I will then add a word to the above question: why does Erlang *still* require the conditions in an if statement to be guards? > > How hard is it to remove this restriction? Expressions and guards have radically different semantics. It was a really bad mistake when people started making them look more similar. For example, tuple_size(T) div N =:= M doesn't raise an exception when N is zero, it just *fails*. Guards are part of pattern matching; in order to get efficient pattern matching code it is important that the compiler should be free to reorder matching and testing any way it likes; in particular G1, G2 need not be tested in the order G1 then G2 and often shouldn't be tested in that order. But if these could be called to user defined functions, such reordering could break things. It would probably be easier to design a new Erlang-like language compiling to the same VM than to *coherently* unify guards and expressions. From jleivent@REDACTED Fri Jun 21 01:21:15 2013 From: jleivent@REDACTED (Jonathan Leivent) Date: Thu, 20 Jun 2013 19:21:15 -0400 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <7F1C7362-3A79-401F-B6FB-FEA199CAA913@gmail.com> References: <51C1F735.5040706@comcast.net> <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> <51C31FE4.4030100@comcast.net> <96590E43-C204-44DA-A9B7-2DD17C01FBFB@gmail.com> <51C32D6D.9000700@comcast.net> <51C33CD7.1090109@comcast.net> <7F1C7362-3A79-401F-B6FB-FEA199CAA913@gmail.com> Message-ID: <51C38E6B.7010601@comcast.net> On 06/20/2013 06:07 PM, Anthony Ramine wrote: > I don't believe they do in places where guards are the only inhabitant. > > I do believe they violate it in comprehensions; call these two functions with [foo]: > > F = fun (L) -> [ X || X <- L, X rem 2 =:= 0 ], > G = fun (L) -> [ X || X <- L, _ = X rem 2 =:= 0 ]. > > Regards, > You omitted the "end"s - but you obviously didn't mean to do that. OK - so you just blew my mind. What is happening here? Let me see if I can explain it: In F, the "X rem 2" guard fails because X is not an integer - but it is in a guard context, so no error is produced, the guard just evaluates to false, and F([foo]) returns []. In G, the "X rem 2" is in the rhs of an "assignment" - really a match expression: that is no longer a guard context, so G([foo]) produces an error instead. Does that sound right? Personally, I would stay away from using this false-not-error-on-bad-type behavior of guards that are not patterns. It's easy enough to use the is_type BIFs to make the meaning explicit and far less astonishing: H = fun (L) -> [X || X <- L, is_integer(X), X rem 2 =:= 0] end. But, then I'm writing Erlang code that I expect to be read and understood by non-Erlang programmers. Is Dialyzer (or some other lint-like Erlang tool) customizable? If so, I'd like to add a customization that flags such cases for my own use. -- Jonathan From ok@REDACTED Fri Jun 21 01:45:15 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 21 Jun 2013 11:45:15 +1200 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <51C32D6D.9000700@comcast.net> References: <51C1F735.5040706@comcast.net> <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> <51C31FE4.4030100@comcast.net> <96590E43-C204-44DA-A9B7-2DD17C01FBFB@gmail.com> <51C32D6D.9000700@comcast.net> Message-ID: <478F57D8-3CF2-4FC6-B210-C7FEA0F9802E@cs.otago.ac.nz> On 21/06/2013, at 4:27 AM, Jonathan Leivent wrote: > OK - that is surprising. That certainly changes my thinking about if statements. Is this particular semantic difference between guards and general conditions ever expressed in the reference manual? If so, I certainly missed it. Yes it is. Section 7.24 "Guard Sequences", where guards are described. http://www.erlang.org/doc/reference_manual/expressions.html#id80042 At the end, there is this sentence: If an arithmetic expression, a boolean expression, a short-circuit expression, or a call to a guard BIF fails (because of invalid arguments), the entire guard fails . If the guard was part of a guard sequence, the next guard in the sequence (that is, the guard following the next semicolon) will be evaluated. It's also explained clearly in the Erlang books I've read. From n.oxyde@REDACTED Fri Jun 21 02:34:08 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Fri, 21 Jun 2013 02:34:08 +0200 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <455D5380-42F9-4B27-9F11-133B62D96B85@gmail.com> References: <51C1F735.5040706@comcast.net> <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> <51C31FE4.4030100@comcast.net> <96590E43-C204-44DA-A9B7-2DD17C01FBFB@gmail.com> <51C32D6D.9000700@comcast.net> <51C33CD7.1090109@comcast.net> <7F1C7362-3A79-401F-B6FB-FEA199CAA913@gmail.com> <51C38E6B.7010601@comcast.net> <455D5380-42F9-4B27-9F11-133B62D96B85@gmail.com> Message-ID: Err, sorry again Jonathan. Forwarding to list. -- Anthony Ramine Le 21 juin 2013 ? 02:33, Anthony Ramine a ?crit : > Indeed, sorry. > >> OK - so you just blew my mind. What is happening here? Let me see if I can explain it: >> >> In F, the "X rem 2" guard fails because X is not an integer - but it is in a guard context, so no error is produced, the guard just evaluates to false, and F([foo]) returns []. >> >> In G, the "X rem 2" is in the rhs of an "assignment" - really a match expression: that is no longer a guard context, so G([foo]) produces an error instead. >> >> Does that sound right? > > Indeed, a match is an expression which is not a guard, thus the filter is compiled with expression semantics. > >> Personally, I would stay away from using this false-not-error-on-bad-type behavior of guards that are not patterns. It's easy enough to use the is_type BIFs to make the meaning explicit and far less astonishing: >> >> H = fun (L) -> [X || X <- L, is_integer(X), X rem 2 =:= 0] end. >> >> But, then I'm writing Erlang code that I expect to be read and understood by non-Erlang programmers. >> >> Is Dialyzer (or some other lint-like Erlang tool) customizable? If so, I'd like to add a customization that flags such cases for my own use. > > What could be done is to compile every filter as an expression and introduce a new kind of expression, something like "when Guard end", which would compile Guard with guard semantics. > > What's good: > > Guard semantics usable everywhere, with full power and disjunction and conjunctions, something not doable in current comprehensions. > > Syntax clues about what has guard semantics and what has expression semantics. > > What's bad: > > Breaking backwards compatibility. Huge problem. So huge I wouldn't be surprised and I would rather expect that the semantics comprehension filters will never change. > > Nevertheless, I do think it is feasible in multiple OTP versions, progressively. > > Regards, From jleivent@REDACTED Fri Jun 21 02:34:43 2013 From: jleivent@REDACTED (Jonathan Leivent) Date: Thu, 20 Jun 2013 20:34:43 -0400 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: References: <51C1F735.5040706@comcast.net> <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> <51C31FE4.4030100@comcast.net> Message-ID: <51C39FA3.5020907@comcast.net> On 06/20/2013 07:01 PM, Richard A. O'Keefe wrote: > > On 21/06/2013, at 3:29 AM, Jonathan Leivent wrote: > >> ?ELSE >> expr0, >> expra, >> ?BREAK(exprb), >> ?ENDIF. > > The ?BREAK is rather horrible; to my mind, far more horrible than nice > clean obvious closures. I never claimed it was a good solution. Rather, it's the lack of an unrestricted sequential multi-conditional statement that made me consider such things. > > I wrote: >> >>> >>> However, I cannot help feeling that in each specific instance >>> of this there would be something better to write instead. So >>> how about showing us a real example? >> >> Anything that would have worked in an if statement were the conditions just guards. In the case I'm interested in, the conditions are predicates on an opaque type - so they can't be guards. > > That's interesting, but I doubt that anyone ever thought > you _could_ have written your code as a simple 'if'. > *How about showing us a real example?* > As in, actual code. I have a data structure that is implemented by a module called "proposal" - part of a distributed agreement protocol. There is a receive in a separate module that is a gen_server behavior that needs to examine proposals in msgs using API predicates exported from the proposal module. So, the code would be, if "if" allowed: loop(State) -> receive {propose, From, Proposal} -> if proposal:round(Proposal) < State#state.current_round -> ...; proposal:is_subsumed_by(Proposal, State#state.last_proposal) -> ...; etc. I'm sure you get the idea. > > Real examples often show opportunities for restructuring in > completely different ways. The problem is, once you have multiple conditions that are not guards, and that you want to examine sequentially, what simple structure can you come up with in Erlang? > > Also, I'm having a hard time imagining so many sequentially > tested conditions that > > case C1 > of true -> A1 > ; false -> > case C2 > of true -> A2 > ; false -> > ... > end > end > > is really hard to write. It's certainly what > cond C1 -> A1 ; C2 -> C2 ; ... end > would compile into, if it existed. Oh - certainly - there's only 4 in my current case, but as the design expands, there could easily be more. But, 4 is plenty already ugly nestingwise. I'm probably only doing a mediocre job of justifying an if/elsif/else construct. > > >>> Because back when Erlang was invented, there were no conditions that >>> were not guard tests *only*. For example, X > Y was not an expression. >>> There were no andalso or orelse. >> >> OK - I will then add a word to the above question: why does Erlang *still* require the conditions in an if statement to be guards? >> >> How hard is it to remove this restriction? > > Expressions and guards have radically different semantics. > It was a really bad mistake when people started making them > look more similar. > > For example, tuple_size(T) div N =:= M doesn't raise an > exception when N is zero, it just *fails*. > > Guards are part of pattern matching; in order to get efficient > pattern matching code it is important that the compiler should > be free to reorder matching and testing any way it likes; in > particular G1, G2 need not be tested in the order G1 then G2 > and often shouldn't be tested in that order. But if these > could be called to user defined functions, such reordering > could break things. > > It would probably be easier to design a new Erlang-like language > compiling to the same VM than to *coherently* unify guards and > expressions. OK - now that I understand how different guards are, I would not at all suggest that if statements should be modified to allow normal conditional expressions. However, why was that "cond" expression dropped or never fully implemented? -- Jonathan From n.oxyde@REDACTED Fri Jun 21 02:35:22 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Fri, 21 Jun 2013 02:35:22 +0200 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <478F57D8-3CF2-4FC6-B210-C7FEA0F9802E@cs.otago.ac.nz> References: <51C1F735.5040706@comcast.net> <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> <51C31FE4.4030100@comcast.net> <96590E43-C204-44DA-A9B7-2DD17C01FBFB@gmail.com> <51C32D6D.9000700@comcast.net> <478F57D8-3CF2-4FC6-B210-C7FEA0F9802E@cs.otago.ac.nz> Message-ID: <12F74DB5-1F10-473F-BDC3-62816B3D54F8@gmail.com> Guard semantics are not mentioned in the documentation about comprehension filters though. And I would rather have it removed than documented, but that's another story which I tell in another mail. -- Anthony Ramine Le 21 juin 2013 ? 01:45, Richard A. O'Keefe a ?crit : > > On 21/06/2013, at 4:27 AM, Jonathan Leivent wrote: >> OK - that is surprising. That certainly changes my thinking about if statements. Is this particular semantic difference between guards and general conditions ever expressed in the reference manual? If so, I certainly missed it. > > Yes it is. > > Section 7.24 "Guard Sequences", where guards are described. > > http://www.erlang.org/doc/reference_manual/expressions.html#id80042 > > At the end, there is this sentence: > > If an arithmetic expression, a boolean expression, a short-circuit > expression, or a call to a guard BIF fails (because of invalid > arguments), the entire guard fails . If the guard was part of > a guard sequence, the next guard in the sequence (that is, the > guard following the next semicolon) will be evaluated. > > > It's also explained clearly in the Erlang books I've read. > From ok@REDACTED Fri Jun 21 03:34:53 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 21 Jun 2013 13:34:53 +1200 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <12F74DB5-1F10-473F-BDC3-62816B3D54F8@gmail.com> References: <51C1F735.5040706@comcast.net> <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> <51C31FE4.4030100@comcast.net> <96590E43-C204-44DA-A9B7-2DD17C01FBFB@gmail.com> <51C32D6D.9000700@comcast.net> <478F57D8-3CF2-4FC6-B210-C7FEA0F9802E@cs.otago.ac.nz> <12F74DB5-1F10-473F-BDC3-62816B3D54F8@gmail.com> Message-ID: <4D87DAF1-3E82-434B-A431-72853DB881EC@cs.otago.ac.nz> On 21/06/2013, at 12:35 PM, Anthony Ramine wrote: > Guard semantics are not mentioned in the documentation about comprehension filters though. Fair enough. And I'm not the least little bit happy about the fact that comprehension filters aren't guards. I would have expected '[' '||' ']' where ::= ',' | '<-' | '=' | We can tolerate a thing being one way (X < Y is only a guard, and if X or Y raises an exception, it just fails) or all the other way (guards and expressions are identical), but a mixture is bound to confuse. From mjtruog@REDACTED Fri Jun 21 03:37:02 2013 From: mjtruog@REDACTED (Michael Truog) Date: Thu, 20 Jun 2013 18:37:02 -0700 Subject: [erlang-questions] [ANN] CloudI 1.2.4 Released! Message-ID: <51C3AE3E.1060802@gmail.com> Download 1.2.4 from http://sourceforge.net/projects/cloudi/files/latest/download What is CloudI? A Cloud at the lowest level! CloudI (http://cloudi.org/) is a "universal integrator" using an Erlang core, with its actor model, to pursue efficiency, fault-tolerance and scalability. The CloudI API provides a minimal interface to communicate among actors that are called services, so programming language agnostic, database agnostic, and messaging bus agnostic integration can occur. CloudI currently integrates with the programming languages Erlang, C/C++, Java, Python, and Ruby, the databases PostgreSQL, MySQL, couchdb, memcached, and tokyotyrant, the messaging bus ZeroMQ and the internal CloudI service bus. HTTP is also supported for easy integration with cowboy. If anyone is willing to get involved, don't hesitate to contact me or start looking at the code. Thank you for all the bug-reports, suggestions and questions so far! This release is mainly for the fixes for R16B01 compilation, but there are other fixes (and websockets). The details are below: June 20, 2013 Version 1.2.4 (beta) released. * The external service "local" (i.e., "default") protocol configuration option has been fixed and was tested on OSX successfully. The problem was with the unix domain socket initialization, so it doesn't affect the loadtest results done for the 1.2.2 release (see https://github.com/okeuday/CloudI/raw/master/src/tests/http_req/loadtest/results_v1_2_2/201306_summary.pdf). The default configuration external service protocol configuration, as well as the quickstart remain with the "tcp" option, to avoid blocking people that are not concerned about the efficiency benefits and give more time for testing it. * WebSockets support was added to the cloudi_service_http_cowboy internal service. To handle websocket connections, just use the "{use_websockets, true}" configuration option, as shown for port 6464 within the default configuration. The example is at: https://github.com/okeuday/CloudI/raw/master/src/tests/http_req/public_html/websockets.html . * Various fixes were added for the R16B01 release (crypto application usage) * cowboy was updated to 0.8.6 * The cloudi_service_api module was added so that the CloudI Service API can be accessed within the same VM with Erlang function calls (the service providing JSON-RPC and HTTP integration for the CloudI Service API had a name change from cloudi_service_api to cloudi_service_api_requests) * Two examples of different ways to release Erlang applications with CloudI exist in the examples directory as hello_world1 and hello_world2. hello_world1 shows how to let CloudI run your Erlang source code as a service, while hello_world2 shows you a basic Erlang release that with external source code that has CloudI as a dependency. * Infrequent use cases of the internal service init function required some fixes to prevent errnoeous usage. The functions send_sync and recv_async should not be used within an internal service init function, similar to the restriction on gen_server:call for the gen_server OTP/Erlang behaviour (you can always send a message to self() and handle it within the cloudi_service_handle_info/3 to utilize these functions). Please mention any problems, issues, or ideas! Thanks, Michael From ok@REDACTED Fri Jun 21 04:25:22 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 21 Jun 2013 14:25:22 +1200 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <51C39FA3.5020907@comcast.net> References: <51C1F735.5040706@comcast.net> <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> <51C31FE4.4030100@comcast.net> <51C39FA3.5020907@comcast.net> Message-ID: <7B6C2A52-CBEA-4439-B886-788F780D913A@cs.otago.ac.nz> On 21/06/2013, at 12:34 PM, Jonathan Leivent wrote: > I have a data structure that is implemented by a module called "proposal" - part of a distributed agreement protocol. There is a receive in a separate module that is a gen_server behavior that needs to examine proposals in msgs using API predicates exported from the proposal module. So, the code would be, if "if" allowed: > > loop(State) -> > receive > {propose, From, Proposal} -> > if proposal:round(Proposal) < State#state.current_round -> ...; > proposal:is_subsumed_by(Proposal, > State#state.last_proposal) -> ...; > > etc. I'm sure you get the idea. "Getting the idea" is not the point. "Seeing the details" is. > The problem is, once you have multiple conditions that are not guards, and that you want to examine sequentially, what simple structure can you come up with in Erlang? Well, for one thing, it might be an idea to rip the branching structure out of the receive. loop(State) -> receive {propose, From, Proposal} -> handle_proposal(Proposal, State, From) ; ... end. handle_proposal(Proposal, State, From) -> Proposal_Round = proposal:round(Proposal), if Proposal_Round < State#state.current_round -> handle_old_proposal(Proposal, State, From) ; true -> handle_new_proposal(Proposal, State, From) end. handle_new_proposal(Proposal, State, From) -> case proposal:is_subsumed_by(Proposal, State#state.last_proposal) of true -> handle_subsumed_proposal(Proposal, State, From) ; false -> handle_interesting_proposal(Proposal, State, From) end. And that this point, I stop, because my experience tells me "BEWARE OF BOOLEANS!" If the case where Y subsumes X is interesting, how about the case where X subsumes Y? Why isn't this something like case proposal:relative_generality(Proposal, State#state.last_proposal) of identical -> ; more_general -> ; more_specific -> ; incomparable -> end What is common between the various branches? Could the proposal: module be restructured to provide additional/alternative queries that make this code easier to express? > However, why was that "cond" expression dropped or never fully implemented? I've often wondered that myself. From jleivent@REDACTED Fri Jun 21 05:41:06 2013 From: jleivent@REDACTED (Jonathan Leivent) Date: Thu, 20 Jun 2013 23:41:06 -0400 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <7B6C2A52-CBEA-4439-B886-788F780D913A@cs.otago.ac.nz> References: <51C1F735.5040706@comcast.net> <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> <51C31FE4.4030100@comcast.net> <51C39FA3.5020907@comcast.net> <7B6C2A52-CBEA-4439-B886-788F780D913A@cs.otago.ac.nz> Message-ID: <51C3CB52.70906@comcast.net> On 06/20/2013 10:25 PM, Richard A. O'Keefe wrote: > ... >> The problem is, once you have multiple conditions that are not guards, and that you want to examine sequentially, what simple structure can you come up with in Erlang? > > Well, for one thing, it might be an idea to rip the branching structure > out of the receive. Yeah, sure. But, I'm forced to do that, instead of deciding to do it if and when it's a good way to modularize the code. In this particular case, I'd like to see the full conditional all at once, compactly, without obfuscating plumbing, preferably in the receive statement with the other protocol messages. I think it would just help me think clearly about this key top level of the protocol's decision tree. Again, I haven't done a good job justifying the existence of unrestricted sequential conditional statements. But, considering their ubiquity in virtually all other programming languages, do you really think Erlang's omission of such a construct is a "feature"? Although, you might not be trying to argue that. Anyway, I appreciate the help with finding an alternative coding scheme - but if it doesn't exhibit the compactness I was hoping for - that I can visualize in an Erlang-like pseudocode - then this isn't really helping here. -- Jonathan From siraaj@REDACTED Fri Jun 21 06:38:50 2013 From: siraaj@REDACTED (Siraaj Khandkar) Date: Fri, 21 Jun 2013 00:38:50 -0400 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <51C3CB52.70906@comcast.net> References: <51C1F735.5040706@comcast.net> <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> <51C31FE4.4030100@comcast.net> <51C39FA3.5020907@comcast.net> <7B6C2A52-CBEA-4439-B886-788F780D913A@cs.otago.ac.nz> <51C3CB52.70906@comcast.net> Message-ID: <9258C196-18D1-4EC7-90C1-FC82BB556671@khandkar.net> On Jun 20, 2013, at 23:41, Jonathan Leivent wrote: > On 06/20/2013 10:25 PM, Richard A. O'Keefe wrote: >> ... >>> The problem is, once you have multiple conditions that are not guards, and that you want to examine sequentially, what simple structure can you come up with in Erlang? >> >> Well, for one thing, it might be an idea to rip the branching structure >> out of the receive. > > Yeah, sure. But, I'm forced to do that, instead of deciding to do it if and when it's a good way to modularize the code. In this particular case, I'd like to see the full conditional all at once, compactly, without obfuscating plumbing, preferably in the receive statement with the other protocol messages. I think it would just help me think clearly about this key top level of the protocol's decision tree. > > Again, I haven't done a good job justifying the existence of unrestricted sequential conditional statements. You can fold over an unrestricted list of sequentially ordered closures. Other than being different from if-then-else, why do you find that approach so unappealing? > But, considering their ubiquity in virtually all other programming languages, do you really think Erlang's omission of such a construct is a "feature"? > > Although, you might not be trying to argue that. > > Anyway, I appreciate the help with finding an alternative coding scheme - but if it doesn't exhibit the compactness I was hoping for - that I can visualize in an Erlang-like pseudocode - then this isn't really helping here. > > -- Jonathan From ok@REDACTED Fri Jun 21 07:04:13 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 21 Jun 2013 17:04:13 +1200 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <51C3CB52.70906@comcast.net> References: <51C1F735.5040706@comcast.net> <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> <51C31FE4.4030100@comcast.net> <51C39FA3.5020907@comcast.net> <7B6C2A52-CBEA-4439-B886-788F780D913A@cs.otago.ac.nz> <51C3CB52.70906@comcast.net> Message-ID: <3ED3C4FC-2C69-45EC-9799-87D7F60AC094@cs.otago.ac.nz> On 21/06/2013, at 3:41 PM, Jonathan Leivent wrote: >> Well, for one thing, it might be an idea to rip the branching structure >> out of the receive. > > Yeah, sure. But, I'm forced to do that, instead of deciding to do it if and when it's a good way to modularize the code. Someone has a blog where they looked at how to write Erlang code so that it's _obviously_ right, and ripping things into smaller pieces was part of it. If you can't name it, you don't understand it. > In this particular case, I'd like to see the full conditional all at once, compactly, without obfuscating plumbing, preferably in the receive statement with the other protocol messages. "compactly" and "in the receive statement" is a contradiction. I've seen too many Erlang state loops with giant functions stretching over multiple pages. I'm not talking about *adding* obfuscation, I'm talking about *removing* it. > I think it would just help me think clearly about this key top level of the protocol's decision tree. > > Again, I haven't done a good job justifying the existence of unrestricted sequential conditional statements. But, considering their ubiquity in virtually all other programming languages, do you really think Erlang's omission of such a construct is a "feature"? But we *STILL* haven't seen any *ACTUAL* code to tell whether anyone other than you would experience a problem with the code, or whether there might be a completely different way to structure the code. No, I don't think that it's a feature. Arguably, Erlang would have been better off without 'if' entirely. Let's face it "everybody else does it" is not a good argument for anything. "Virtually all other programming languages" have bounded size integers. "Virtually all other programming languages" rely *massively* on assignment statements. You and I have now spent more time talking about this than it would take to just write the wretched code and get on with our lives. In a way I *do* want to say that a language that won't let you code automatically like "virtually all other programming languages" is an opportunity to try to think differently about what you're doing. That doesn't mean the old way is _wrong_; maybe the language really is deficient. But an opportunity to think differently is to be taken. I was about to say "Any time I find myself yearning for a Fortran feature, I stop and think", but actually, Fortran 2008 has finally picked up the 'let' feature of (most) functional languages -- although in typical Fortran fashion it's called ASSSOCIATE. Without seeing the actual code, nobody else can tell whether 'compactness' is really a virtue in this case, or whether it gets in the way of 'clarity'. I do repeat that more than 40 years after meeting formal logic, I love logic, but am extremely suspicious of Boolean expressions in functional languages. Most Boolean-valued functions that have been discussed in this mailing list turned out to be things that would be better conceived some other way (like a Haskell Maybe or Either or something with more than two possibilities). How can I tell whether this is the case in your problem without seeing the code? (Yes, I know, old Johnny One-Note harping on a single string.) > > Anyway, I appreciate the help with finding an alternative coding scheme - but if it doesn't exhibit the compactness I was hoping for - that I can visualize in an Erlang-like pseudocode - then this isn't really helping here. I have to leave in order to get to my father-in-law's birthday party. This is what I have by way of patches to erl_parse.yrl so far. 37c37,40 < if_expr if_clause if_clauses case_expr cr_clause cr_clauses receive_expr --- > if_expr if_clause if_clauses > case_expr cr_clause cr_clauses > cond_expr cond_clause cond_clauses > receive_expr 56c59 < 'after' 'begin' 'case' 'try' 'catch' 'end' 'fun' 'if' 'of' 'receive' 'when' --- > 'after' 'begin' 'case' 'try' 'catch' 'end' 'fun' 'cond' 'if' 'of' 'receive' 'when' 277a281 > expr_max -> cond_expr : '$1'. 387a392,398 > cond_expr -> 'cond' cond_clauses 'end' : mk_cond($2). > > cond_clauses -> cond_clause : ['$1']. > cond_clauses -> cond_clause ';' cond_clauses : ['$1' | '$3']. > > cond_clause -> expr clause_body : {clause,?line('$1'),'$1','$2'}. > 1103a1115,1124 > > mk_cond([{clause,Line,Expr,Body}]) -> > {'case',Line,Expr,[ > {clause,Line,[{atom,Line,true}],[],Body}]}. > mk_cond([{clause,Line,Expr,Body}|Rest]) -> > [{clause,Next,_,_}|_] = Rest, > {'case',Line,Expr,[ > {clause,Line,[{atom,Line,true}],[],Body}, > {clause,Next,[{atom,Line,false}],[],[mk_cond(Rest)]}]}. erlc is happy with this, but it has had no other testing. From siraaj@REDACTED Fri Jun 21 07:23:29 2013 From: siraaj@REDACTED (Siraaj Khandkar) Date: Fri, 21 Jun 2013 01:23:29 -0400 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <3ED3C4FC-2C69-45EC-9799-87D7F60AC094@cs.otago.ac.nz> References: <51C1F735.5040706@comcast.net> <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> <51C31FE4.4030100@comcast.net> <51C39FA3.5020907@comcast.net> <7B6C2A52-CBEA-4439-B886-788F780D913A@cs.otago.ac.nz> <51C3CB52.70906@comcast.net> <3ED3C4FC-2C69-45EC-9799-87D7F60AC094@cs.otago.ac.nz> Message-ID: <7DB5F0AD-3461-41A8-995A-94991837EC94@khandkar.net> On Jun 21, 2013, at 1:04, "Richard A. O'Keefe" wrote: > > On 21/06/2013, at 3:41 PM, Jonathan Leivent wrote: >>> Well, for one thing, it might be an idea to rip the branching structure >>> out of the receive. >> >> Yeah, sure. But, I'm forced to do that, instead of deciding to do it if and when it's a good way to modularize the code. > > Someone has a blog where they looked at how to write Erlang code > so that it's _obviously_ right, and ripping things into smaller > pieces was part of it. If you can't name it, you don't understand it. http://www.gar1t.com/blog/2012/06/10/solving-embarrassingly-obvious-problems-in-erlang/ > I do repeat that more than 40 years after meeting formal logic, > I love logic, but am extremely suspicious of Boolean expressions in > functional languages. Most Boolean-valued functions that have been > discussed in this mailing list turned out to be things that would be > better conceived some other way (like a Haskell Maybe or Either or > something with more than two possibilities). I _really_ liked the following two posts, by Rob Harper, discussing this issue: http://existentialtype.wordpress.com/2011/03/15/dont-mention-equality/ http://existentialtype.wordpress.com/2011/03/15/boolean-blindness/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Fri Jun 21 10:21:12 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Fri, 21 Jun 2013 10:21:12 +0200 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <4D87DAF1-3E82-434B-A431-72853DB881EC@cs.otago.ac.nz> References: <51C1F735.5040706@comcast.net> <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> <51C31FE4.4030100@comcast.net> <96590E43-C204-44DA-A9B7-2DD17C01FBFB@gmail.com> <51C32D6D.9000700@comcast.net> <478F57D8-3CF2-4FC6-B210-C7FEA0F9802E@cs.otago.ac.nz> <12F74DB5-1F10-473F-BDC3-62816B3D54F8@gmail.com> <4D87DAF1-3E82-434B-A431-72853DB881EC@cs.otago.ac.nz> Message-ID: <959684F3-5D32-489D-B7BC-5F1C13452020@gmail.com> Hello Richard, What we currently have: qual -> pat '<-' expr. qual -> bin_pat '<=' expr. qual -> expr. I would keep it as is but compile every filter as an expression and the following grammar rule: expr -> 'when' guard 'end'. And then I would also add two generators which do not skip elements but crash instead: qual -> pat '<:-' expr. qual -> bin_pat '<:=' expr. I do like the different direction you took though. Allowing only guards mean matches can be handled differently, thus one could write X = foo() instead of begin X = foo(), true end. Regards, -- Anthony Ramine Le 21 juin 2013 ? 03:34, Richard A. O'Keefe a ?crit : > > On 21/06/2013, at 12:35 PM, Anthony Ramine wrote: > >> Guard semantics are not mentioned in the documentation about comprehension filters though. > > Fair enough. And I'm not the least little bit happy about the fact > that comprehension filters aren't guards. > > I would have expected > '[' '||' ']' > where > ::= ',' > | '<-' > | '=' > | > > We can tolerate a thing being one way (X < Y is only a guard, and if X or Y > raises an exception, it just fails) or all the other way (guards and > expressions are identical), but a mixture is bound to confuse. > From n.oxyde@REDACTED Fri Jun 21 10:22:37 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Fri, 21 Jun 2013 10:22:37 +0200 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <3ED3C4FC-2C69-45EC-9799-87D7F60AC094@cs.otago.ac.nz> References: <51C1F735.5040706@comcast.net> <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> <51C31FE4.4030100@comcast.net> <51C39FA3.5020907@comcast.net> <7B6C2A52-CBEA-4439-B886-788F780D913A@cs.otago.ac.nz> <51C3CB52.70906@comcast.net> <3ED3C4FC-2C69-45EC-9799-87D7F60AC094@cs.otago.ac.nz> Message-ID: I'll make a proper implementation of it this week-end. I would have done it earlier if you had written an EEP :) -- Anthony Ramine Le 21 juin 2013 ? 07:04, Richard A. O'Keefe a ?crit : > This is what I have by way of patches to erl_parse.yrl so far. From ulf@REDACTED Fri Jun 21 11:05:14 2013 From: ulf@REDACTED (Ulf Wiger) Date: Fri, 21 Jun 2013 11:05:14 +0200 Subject: [erlang-questions] gproc branch for resolving netsplits Message-ID: <903355C7-1A2C-4045-A656-6E3615F63E4A@feuerlabs.com> I've pushed an experimental branch for resolving netsplits in gproc. I'd like to go over the code a bit more and do more testing, but since it's time for the annual Swedish heathen rituals (Midsummer), I expect it won't happen today. :P https://github.com/uwiger/gproc/tree/split-brain If anyone feels adventurous, I'd certainly appreciate some feedback. In particular, I'd like to know if anyone really needs the protocol between gproc_dist instances to be backward compatible for upgrade purposes. I could fix that, but it would certainly be nicer to skip it. :) BR, Ulf W The idea is that you add a deconflict method for each global registration (default method is 'exit_all'). This can be done using either gproc:reg(Key, Value, [Deconflict | OtherAttrs]), or gproc:set_attributes(Key, [Deconflict | OtherAttrs]), where Deconflict = gproc:deconflict(Method) Method :: exit_all | smallest_pid | largest_pid (More methods could be envisioned, esp. {Module, Function}). From the edoc source: %% @spec deconflict(Type) -> {gproc_deconflict, Type}. %% @doc Attribute controlling reconciliation after split-brain. %% %% Certain network failures can result in gproc nodes becoming separated from %% each other - a so-called 'split-brain' scenario. When the nodes reconnect, %% there may be conflicting registrations in the gproc database. This function %% instructs gproc on how to resolve such conflicts. %% %% The following deconflict methods are supported %% %% * `exit_all' - send an `exit(Pid, {gproc_conflict, Key})' signal to all %% involved pids, and remove the registrations %% * `smallest_pid' - send an `exit(Pid, {gproc_conflict, Key})' signal to the %% smallest pid, and remove the corresponding registration. %% * `largest_pid' - send an `exit(Pid, {gproc_conflict, Key})' signal to the %% largest pid, and remove the corresponding registration. Ulf Wiger, Co-founder & Developer Advocate, Feuerlabs Inc. http://feuerlabs.com From jesper.louis.andersen@REDACTED Fri Jun 21 11:38:58 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 21 Jun 2013 11:38:58 +0200 Subject: [erlang-questions] [HEADS UP] Emysql driver Message-ID: Hi Erlangers, Henning Diedrich and I have been working a bit on the Emysql driver, https://github.com/Eonblast/Emysql cleaning it up and removing old cruft from it. Over the last month or so we have reached a point where 3-4 emysql branches were merged back into the above path. It has been running in production at a customer site, Issuu, for about 1 month without any problems to report and we believe it is fairly stable. NOTE: There is a backwards-incompatible change in `master`. We removed the encoding support of automatic conversion of binaries into utf-8. This means that client *users* are now responsible for encoding data properly. On the other hand, this means that the driver won't try to encode binary BLOB data as utf-8 anymore :) Branches of interest: `master`. Try it out. It contains the merger. Tags of interest: `0.2.16`. This tag is just *before* the merge, so if you need time to adapt to the new setup, you can lock to this tag and obtain the old behaviour. J. Erlang Solutions Ltd, Copenhagen. -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Fri Jun 21 11:46:14 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 21 Jun 2013 13:46:14 +0400 Subject: [erlang-questions] [HEADS UP] Emysql driver In-Reply-To: References: Message-ID: Jesper, thanks a lot! It is always great when projects are unforked on github! From origincheung@REDACTED Fri Jun 21 17:21:49 2013 From: origincheung@REDACTED (OriginCheung) Date: Fri, 21 Jun 2013 23:21:49 +0800 Subject: [erlang-questions] About HAMT & VList Message-ID: I read this file ( http://www.erlang-factory.com/upload/presentations/468/EUC_Hashes2011.pdf) on yesterday. Read HAMT & VList. So, how cat implement them? thanks for help! OCheung -------------- next part -------------- An HTML attachment was scrubbed... URL: From ransomr@REDACTED Fri Jun 21 17:44:34 2013 From: ransomr@REDACTED (Ransom Richardson) Date: Fri, 21 Jun 2013 15:44:34 +0000 Subject: [erlang-questions] iPhone unable to connect over SSL after upgrading to R16B01 Message-ID: Not sure if this is a known issue but I didn't see anything about it when I searched. After upgrading to R16B01 our iPhone client can no longer connect to the service over SSL. The same build works fine when deployed on a machine running R16B. If we specify {ciphers, [{rsa,aes_256_cbc,sha256}]}, then it is able to connect successfully However, if we specify: {ciphers, [{rsa,aes_256_cbc,sha256}, {ecdhe_rsa,aes_256_cbc,sha}]}, then the connection fails. I didn't debug further, but did notice this following in tls_handshake.erl, which seems like it might be related: %% TODO: need to take supported Curves into Account when selecting the CipherSuite.... %% if whe have an ECDSA cert with an unsupported curve, we need to drop ECDSA ciphers -------------- next part -------------- An HTML attachment was scrubbed... URL: From tcury@REDACTED Sat Jun 22 00:15:23 2013 From: tcury@REDACTED (Tiago Cury) Date: Fri, 21 Jun 2013 19:15:23 -0300 Subject: [erlang-questions] test Message-ID: <48A6281A-F54A-404A-8D2E-6BD589A90A4D@ymail.com> test From tcury@REDACTED Sat Jun 22 00:16:48 2013 From: tcury@REDACTED (Tiago Cury) Date: Fri, 21 Jun 2013 19:16:48 -0300 Subject: [erlang-questions] reltool and env vars Message-ID: <8042B6AD-E6F1-46F1-B40E-E3CB21B0BB1D@ymail.com> Hi, how can I configure the retool to add these environment variables to my release startup file? INSTANCE=backuper1 PORT=8080 TX_PORT=8081 thanks, Tiago. From tcury@REDACTED Sat Jun 22 01:41:20 2013 From: tcury@REDACTED (Tiago Cury) Date: Fri, 21 Jun 2013 20:41:20 -0300 Subject: [erlang-questions] reltool and env vars In-Reply-To: <8042B6AD-E6F1-46F1-B40E-E3CB21B0BB1D@ymail.com> References: <8042B6AD-E6F1-46F1-B40E-E3CB21B0BB1D@ymail.com> Message-ID: FIXED: I put these variables in my ebin/application.app file. ;-) And about the mnesia dir ? On Jun 21, 2013, at 7:16 PM, Tiago Cury wrote: > Hi, how can I configure the retool to add these environment variables to my release startup file? > > INSTANCE=backuper1 > PORT=8080 > TX_PORT=8081 > > thanks, Tiago. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From eric@REDACTED Sat Jun 22 03:05:35 2013 From: eric@REDACTED (Eric Moritz) Date: Fri, 21 Jun 2013 21:05:35 -0400 Subject: [erlang-questions] reltool and env vars In-Reply-To: References: <8042B6AD-E6F1-46F1-B40E-E3CB21B0BB1D@ymail.com> Message-ID: You can use reltool's overlay/template functionality to generate a .config file that your startup script can use Riak does this if you want to see how it is done. https://github.com/basho/riak/blob/master/rel/reltool.config#L83 Somewhere in Erlang.orgs docs there is a description of all these options but I'm having trouble finding it on my phone. On Jun 21, 2013 7:41 PM, "Tiago Cury" wrote: > FIXED: I put these variables in my ebin/application.app file. ;-) > > And about the mnesia dir ? > On Jun 21, 2013, at 7:16 PM, Tiago Cury wrote: > > > Hi, how can I configure the retool to add these environment variables to > my release startup file? > > > > INSTANCE=backuper1 > > PORT=8080 > > TX_PORT=8081 > > > > thanks, Tiago. > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From derekbrown121@REDACTED Fri Jun 21 03:58:09 2013 From: derekbrown121@REDACTED (Derek Brown) Date: Thu, 20 Jun 2013 21:58:09 -0400 Subject: [erlang-questions] Seeking speakers for NYC Erlang Meetup Message-ID: Looking for speakers on any topic Erlang-related at our NYC meetup: http://www.meetup.com/Erlang-NYC/ Thanks, Derek -------------- next part -------------- An HTML attachment was scrubbed... URL: From donpedrothird@REDACTED Sat Jun 22 10:48:59 2013 From: donpedrothird@REDACTED (John Doe) Date: Sat, 22 Jun 2013 12:48:59 +0400 Subject: [erlang-questions] dirty_update_counters and mnesia_frag Message-ID: It looks like atm it is impossible to use dirty_update_counters in mnesia_activity with AccessMod == mnesia_frag (and with any other access modules too) as this function is not forwarded to callback access module. I for one don't think there would be any potential problems with dirty_update_counters in frag_tables context if it was possible to use this combination at all. Why call to this function does not forwarded to access module, and are there any workarounds besides manual sharding? -------------- next part -------------- An HTML attachment was scrubbed... URL: From donpedrothird@REDACTED Sat Jun 22 12:08:05 2013 From: donpedrothird@REDACTED (Evgeny M) Date: Sat, 22 Jun 2013 03:08:05 -0700 (PDT) Subject: [erlang-questions] dirty_update_counters and mnesia_frag In-Reply-To: References: Message-ID: <48fa42fd-e952-4648-81c1-012b244571ca@googlegroups.com> Follow up: I copied key_to_frag_name and its' dependencies from mnesia_frag.erl to my project. Now I generate frag table names in my code and use dirty_update_counters on a generated table name, bypassing mnesia_frag callback altogether. I use this for dirty_update_counters only and call mnesia:activity(..., mnesia_frag) for everything else, like selects etc. Could there be any problems in such workflow? There are no other write operations besides dirty_update_counters and clear_table. ???????, 22 ???? 2013 ?., 12:48:59 UTC+4 ???????????? Evgeny M ???????: > > It looks like atm it is impossible to use dirty_update_counters in > mnesia_activity with AccessMod == mnesia_frag (and with any other access > modules too) as this function is not forwarded to callback access module. > I for one don't think there would be any potential problems with > dirty_update_counters in frag_tables context if it was possible to use this > combination at all. > Why call to this function does not forwarded to access module, and are > there any workarounds besides manual sharding? > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Sat Jun 22 14:18:53 2013 From: bchesneau@REDACTED (Benoit Chesneau) Date: Sat, 22 Jun 2013 14:18:53 +0200 Subject: [erlang-questions] crypto deprecation warnings in latest release Message-ID: I see a lot of deprecation warnings in the crypto module with the latest release of Erlang. (crypto:sha_mac/2 -> crypto:hmac/3, crypto_md5_* -> crypto:hash/2 ...) . Is there a way to handle this changes at compile time to use latest functions when possible and old functions in legacy releases? With the help of rebar I did such change: https://github.com/refuge/erlang-oauth/commit/2ab4a24c3f7db95b9ab43099ba1fa876a072e955 But I'm wondering if there is a better solution. Also able to handle more legacy versions. Any idea? - benoit From tcury@REDACTED Sat Jun 22 17:27:43 2013 From: tcury@REDACTED (Tiago Cury) Date: Sat, 22 Jun 2013 12:27:43 -0300 Subject: [erlang-questions] yaws & windows Message-ID: Hi, I did an application with yaws embedded and works fine on unix, but now I need to use on Windows platforms and I saw that yaws is not friend of windows (there is some bash scripts and posix C code). What's are the alternatives? From eriksoe@REDACTED Sat Jun 22 17:40:33 2013 From: eriksoe@REDACTED (=?ISO-8859-1?Q?Erik_S=F8e_S=F8rensen?=) Date: Sat, 22 Jun 2013 17:40:33 +0200 Subject: [erlang-questions] yaws & windows In-Reply-To: References: Message-ID: You can download yaws for Windows. For embedded yaws, use those Windows files as a basis for your release - reltool allows you to specify a distribution root directory. This has worked for me. (I can find the details if necessary.) Hi, I did an application with yaws embedded and works fine on unix, but now I need to use on Windows platforms and I saw that yaws is not friend of windows (there is some bash scripts and posix C code). What's are the alternatives? _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From tcury@REDACTED Sat Jun 22 17:45:02 2013 From: tcury@REDACTED (Tiago Cury) Date: Sat, 22 Jun 2013 12:45:02 -0300 Subject: [erlang-questions] yaws & windows In-Reply-To: References: Message-ID: Looks good. Do you have these details? Will help me so much. thanks Tiago On Jun 22, 2013, at 12:40 PM, Erik S?e S?rensen wrote: > You can download yaws for Windows. > For embedded yaws, use those Windows files as a basis for your release - reltool allows you to specify a distribution root directory. > This has worked for me. > (I can find the details if necessary.) > Hi, I did an application with yaws embedded and works fine on unix, but now I need to use on Windows platforms and I saw that yaws is not friend of windows (there is some bash scripts and posix C code). > > What's are the alternatives? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From anthonym@REDACTED Sat Jun 22 18:40:54 2013 From: anthonym@REDACTED (ANTHONY MOLINARO) Date: Sat, 22 Jun 2013 09:40:54 -0700 Subject: [erlang-questions] crypto deprecation warnings in latest release In-Reply-To: References: Message-ID: Something similar had to happen in mochiweb for R15 (where a bug in gen_tcp caused issues), it ended up looking similar https://github.com/mochi/mochiweb/commit/92b8f5bd10ce69d40005c8e78e52db9003f0189d So I think you came up with the best solution. Luckily after a few years you'll be able to remove the special cases :) -Anthony On Jun 22, 2013, at 5:18 AM, Benoit Chesneau wrote: > I see a lot of deprecation warnings in the crypto module with the > latest release of Erlang. (crypto:sha_mac/2 -> crypto:hmac/3, > crypto_md5_* -> crypto:hash/2 ...) . > > Is there a way to handle this changes at compile time to use latest > functions when possible and old functions in legacy releases? > > > With the help of rebar I did such change: > > https://github.com/refuge/erlang-oauth/commit/2ab4a24c3f7db95b9ab43099ba1fa876a072e955 > > But I'm wondering if there is a better solution. Also able to handle > more legacy versions. > > Any idea? > > - benoit > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Sat Jun 22 22:12:56 2013 From: bchesneau@REDACTED (Benoit Chesneau) Date: Sat, 22 Jun 2013 22:12:56 +0200 Subject: [erlang-questions] [ann] econfig 0.4. released Message-ID: I just released econfig 0.4.0. econfig is a simple configuration handler in Erlang. For now it only supports ini file. New features are: - add econfig:start/0 and econfig:stop/0 functions to start and stop easily econfig in tests or on the shell. - add `econfig:section/1` function to get a list of all sections. - add `econfig:prefix` function to get al l sections starting with Prefix - add `econfig:cfg2list/2` to retrieve all the configuration as a proplist - add `econfig:cfg2list/3` to retrieves all the config as a proplist and group sections by key: eg. 3> econfig:cfg2list(test). [{"http \"private\"",[{"listen","192.168.1.2:8080"}]}, {"http \"default\"",[{"listen","80"}]}, {"http \"ssl\"", [{"listen","443"}, {"ssl","on"}, {"ssl_certficate","cert.pem"}, {"ssl_certficate_key","cert.key"}]}] 4> econfig:cfg2list(test, " "). [{"http", [{"\"private\"",[{"listen","192.168.1.2:8080"}]}, {"\"default\"",[{"listen","80"}]}, {"\"ssl\"", [{"listen","443"}, {"ssl","on"}, {"ssl_certficate","cert.pem"}, {"ssl_certficate_key","cert.key"}]}]}] - add `econfig:open_config/2` and `econfig:open_config/3` to open or create a config file and register it. Full changelog: https://github.com/benoitc/econfig/compare/0.3.0...0.4.0 Coming features are: - support for the toml specification (https://github.com/mojombo/toml) - support for json config - allows distribution of the ocnfig over different machines/VMs. Any feedback is appreciated :) Enjoy! - benoit From bchesneau@REDACTED Sat Jun 22 23:52:20 2013 From: bchesneau@REDACTED (Benoit Chesneau) Date: Sat, 22 Jun 2013 23:52:20 +0200 Subject: [erlang-questions] crypto deprecation warnings in latest release In-Reply-To: References: Message-ID: I see :) Thanks! On Sat, Jun 22, 2013 at 6:40 PM, ANTHONY MOLINARO wrote: > Something similar had to happen in mochiweb for R15 (where a bug in gen_tcp > caused issues), it ended up looking similar > > https://github.com/mochi/mochiweb/commit/92b8f5bd10ce69d40005c8e78e52db9003f0189d > > So I think you came up with the best solution. Luckily after a few years > you'll be able to remove the special > cases :) > > -Anthony > > On Jun 22, 2013, at 5:18 AM, Benoit Chesneau wrote: > > I see a lot of deprecation warnings in the crypto module with the > latest release of Erlang. (crypto:sha_mac/2 -> crypto:hmac/3, > crypto_md5_* -> crypto:hash/2 ...) . > > Is there a way to handle this changes at compile time to use latest > functions when possible and old functions in legacy releases? > > > With the help of rebar I did such change: > > https://github.com/refuge/erlang-oauth/commit/2ab4a24c3f7db95b9ab43099ba1fa876a072e955 > > But I'm wondering if there is a better solution. Also able to handle > more legacy versions. > > Any idea? > > - benoit > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From ben@REDACTED Sat Jun 22 14:55:37 2013 From: ben@REDACTED (Ben Butler-Cole) Date: Sat, 22 Jun 2013 13:55:37 +0100 Subject: [erlang-questions] Changing output file for erlc Message-ID: Hello Is it possible to get erlc to change the file that it sends its output to? I gather that the directory can be changed with `-o`, but I can't see any way to change the file. Equivalently, is it possible to get it to write to stdout? I would like to do this in order to use it with the redo build tool ( http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/introduction-to-redo.html ). Thanks -Ben -------------- next part -------------- An HTML attachment was scrubbed... URL: From tcury@REDACTED Sun Jun 23 05:53:28 2013 From: tcury@REDACTED (Tiago Cury) Date: Sun, 23 Jun 2013 00:53:28 -0300 Subject: [erlang-questions] yaws & windows In-Reply-To: References: Message-ID: Yeahhh, I compiled my app with embedded yaws on Windows. thanks a lot ;-) On Jun 22, 2013, at 12:45 PM, Tiago Cury wrote: > Looks good. Do you have these details? Will help me so much. thanks Tiago > > On Jun 22, 2013, at 12:40 PM, Erik S?e S?rensen wrote: > >> You can download yaws for Windows. >> For embedded yaws, use those Windows files as a basis for your release - reltool allows you to specify a distribution root directory. >> This has worked for me. >> (I can find the details if necessary.) >> Hi, I did an application with yaws embedded and works fine on unix, but now I need to use on Windows platforms and I saw that yaws is not friend of windows (there is some bash scripts and posix C code). >> >> What's are the alternatives? >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From pablo.platt@REDACTED Sun Jun 23 00:41:34 2013 From: pablo.platt@REDACTED (pablo platt) Date: Sun, 23 Jun 2013 01:41:34 +0300 Subject: [erlang-questions] Using avconv from erlang Message-ID: Hi, I'm trying to transcode a live RTMP stream with avconv from erlang. How do I need to use open_port to get output from avconv? The following doesn't work: start() -> spawn(audio, start_avconv, []). start_avconv() -> Port = open_port({spawn, "avconv -i rtmp://127.0.0.1/audio/test -f mp3 -"}, [binary, stream]), read(Port, <<>>). read(Port, _Data) -> receive {Port, {data, NewData}} -> io:format("~p~n", [NewData]), read(Port, <<>>); M -> io:format("~p~n", [M]) end. Is there an erlang project that helps working with audio and video streams? Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From bourinov@REDACTED Sun Jun 23 11:24:43 2013 From: bourinov@REDACTED (Max Bourinov) Date: Sun, 23 Jun 2013 11:24:43 +0200 Subject: [erlang-questions] Using avconv from erlang In-Reply-To: References: Message-ID: Hi Pablo, check this out: https://github.com/erlyvideo Best regards, Max On Sun, Jun 23, 2013 at 12:41 AM, pablo platt wrote: > Hi, > > I'm trying to transcode a live RTMP stream with avconv from erlang. > How do I need to use open_port to get output from avconv? > > The following doesn't work: > > start() -> > spawn(audio, start_avconv, []). > > start_avconv() -> > Port = open_port({spawn, "avconv -i rtmp://127.0.0.1/audio/test -f mp3 -"}, [binary, stream]), > read(Port, <<>>). > > read(Port, _Data) -> > receive > {Port, {data, NewData}} -> > io:format("~p~n", [NewData]), > read(Port, <<>>); > M -> > io:format("~p~n", [M]) > end. > > Is there an erlang project that helps working with audio and video streams? > > Thanks > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pablo.platt@REDACTED Sun Jun 23 11:31:28 2013 From: pablo.platt@REDACTED (pablo platt) Date: Sun, 23 Jun 2013 12:31:28 +0300 Subject: [erlang-questions] Using avconv from erlang In-Reply-To: References: Message-ID: @Max erlyvideo has interesting stuff in it but I couldn't find docs or active community. There are several versions and names (erlyvideo, flussonic) and it is not clear which one is active and open sourced. Thanks for the suggestion but for me erlyvideo is not a true open source project like gproc, mochiweb, rabbitmq, cowboy and many other great projects. On Sun, Jun 23, 2013 at 12:24 PM, Max Bourinov wrote: > Hi Pablo, > > check this out: https://github.com/erlyvideo > > Best regards, > Max > > > > On Sun, Jun 23, 2013 at 12:41 AM, pablo platt wrote: > >> Hi, >> >> I'm trying to transcode a live RTMP stream with avconv from erlang. >> How do I need to use open_port to get output from avconv? >> >> The following doesn't work: >> >> start() -> >> spawn(audio, start_avconv, []). >> >> start_avconv() -> >> Port = open_port({spawn, "avconv -i rtmp://127.0.0.1/audio/test -f mp3 -"}, [binary, stream]), >> read(Port, <<>>). >> >> read(Port, _Data) -> >> receive >> {Port, {data, NewData}} -> >> io:format("~p~n", [NewData]), >> read(Port, <<>>); >> M -> >> io:format("~p~n", [M]) >> end. >> >> Is there an erlang project that helps working with audio and video streams? >> >> Thanks >> >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yoursurrogategod@REDACTED Sun Jun 23 15:05:30 2013 From: yoursurrogategod@REDACTED (Yves S. Garret) Date: Sun, 23 Jun 2013 09:05:30 -0400 Subject: [erlang-questions] Question about how to better use the debugger Message-ID: Hi all, I've been trying to debug this method in module hello_world (just to learn): fac(0) -> 1; fac(N) when N > 0, is_integer(N) -> N * fac(N - 1). % break-point I have a break-point on the last line (where the comment is). Now, when I go through the motions of debugging, it works fine. I can see N being decremented as it should be. However, when I get to fac(0), the method instantly returns to the caller (the shell). In other debuggers (Visual Studio C#, for example), I'd have the option to go back up the entire recursive tree and see the multiplication and the results of the multiplication being done. Can I do the same here? If so, what am I missing? Is there a better strategy than what I'm using in debugging? -------------- next part -------------- An HTML attachment was scrubbed... URL: From prof3ta@REDACTED Sun Jun 23 15:08:27 2013 From: prof3ta@REDACTED (Roberto Aloi) Date: Sun, 23 Jun 2013 15:08:27 +0200 Subject: [erlang-questions] Question about how to better use the debugger In-Reply-To: References: Message-ID: You could "trace" the function: http://aloiroberto.wordpress.com/2009/02/23/tracing-erlang-functions/ On 23 Jun 2013 15:05, "Yves S. Garret" wrote: > Hi all, > > I've been trying to debug this method in module hello_world (just to > learn): > fac(0) -> > 1; > fac(N) when N > 0, is_integer(N) -> > N * fac(N - 1). % break-point > > I have a break-point on the last line (where the comment is). Now, when I > go > through the motions of debugging, it works fine. I can see N being > decremented as > it should be. However, when I get to fac(0), the method instantly returns > to the > caller (the shell). In other debuggers (Visual Studio C#, for example), > I'd have the > option to go back up the entire recursive tree and see the multiplication > and the > results of the multiplication being done. Can I do the same here? If so, > what am I > missing? Is there a better strategy than what I'm using in debugging? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mloftis@REDACTED Sun Jun 23 18:09:47 2013 From: mloftis@REDACTED (Michael Loftis) Date: Sun, 23 Jun 2013 09:09:47 -0700 Subject: [erlang-questions] Using avconv from erlang In-Reply-To: References: Message-ID: Yes erlyvideo was renamed flussonic, and also closed source now. They moved away from github at some point (not sure when, just noticed the other day myself). You can't download it without a license anymore after the split point as far as I'm aware. On Sun, Jun 23, 2013 at 2:31 AM, pablo platt wrote: > @Max > > erlyvideo has interesting stuff in it but I couldn't find docs or active > community. > There are several versions and names (erlyvideo, flussonic) and it is not > clear which one is active and open sourced. > > Thanks for the suggestion but for me erlyvideo is not a true open source > project like gproc, mochiweb, rabbitmq, cowboy and many other great > projects. > > > > On Sun, Jun 23, 2013 at 12:24 PM, Max Bourinov wrote: >> >> Hi Pablo, >> >> check this out: https://github.com/erlyvideo >> >> Best regards, >> Max >> >> >> >> On Sun, Jun 23, 2013 at 12:41 AM, pablo platt >> wrote: >>> >>> Hi, >>> >>> I'm trying to transcode a live RTMP stream with avconv from erlang. >>> How do I need to use open_port to get output from avconv? >>> >>> The following doesn't work: >>> >>> start() -> >>> spawn(audio, start_avconv, []). >>> >>> start_avconv() -> >>> Port = open_port({spawn, "avconv -i rtmp://127.0.0.1/audio/test -f >>> mp3 -"}, [binary, stream]), >>> read(Port, <<>>). >>> >>> read(Port, _Data) -> >>> receive >>> {Port, {data, NewData}} -> >>> io:format("~p~n", [NewData]), >>> read(Port, <<>>); >>> M -> >>> io:format("~p~n", [M]) >>> end. >>> >>> Is there an erlang project that helps working with audio and video >>> streams? >>> >>> Thanks >>> >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- "Genius might be described as a supreme capacity for getting its possessors into trouble of all kinds." -- Samuel Butler From ericbmerritt@REDACTED Sun Jun 23 19:06:46 2013 From: ericbmerritt@REDACTED (Eric Merritt) Date: Sun, 23 Jun 2013 10:06:46 -0700 Subject: [erlang-questions] turning off opaque types underspecified warning Message-ID: If you add an opaque type to a module that just points to a simple term (or a tuple etc) as follows: -opaque hello() :: term(). You get the following warning: src/elixir.erl:15: Warning: opaque type hello() is underspecified and therefore meaningless >From the standpoint of Dialyzer this makes sense. However, types are used as much to communication information to a programmer as they are to communicate information to Dialyzer. So while the above is meaningless to Dialyzer it could very well convey meaning and be useful to the human reading the code. So the big question is: is there anyway to disable this warning? and the secondary question is: does it really make sense for this warning to exist in the first place? From siraaj@REDACTED Sun Jun 23 20:28:09 2013 From: siraaj@REDACTED (Siraaj Khandkar) Date: Sun, 23 Jun 2013 14:28:09 -0400 Subject: [erlang-questions] turning off opaque types underspecified warning In-Reply-To: References: Message-ID: <51C73E39.1070402@khandkar.net> On 06/23/2013 01:06 PM, Eric Merritt wrote: > If you add an opaque type to a module that just points to a simple > term (or a tuple etc) as follows: > > -opaque hello() :: term(). > > You get the following warning: > > src/elixir.erl:15: Warning: opaque type hello() is underspecified > and therefore meaningless > > > From the standpoint of Dialyzer this makes sense. However, types are > used as much to communication information to a programmer as they are > to communicate information to Dialyzer. So while the above is > meaningless to Dialyzer it could very well convey meaning and be > useful to the human reading the code. It does read as quite meaningless to me as a human reader. What use-case do you envision? The best alternative I can imagine is to parametrize is, something like this: -opaque hidden(A) :: A. -spec of_string(A) -> hidden(A) when A :: string(). -spec of_int(A) -> hidden(A) when A :: integer(). That would mean something to me as a human reader. Mind you, this is off the top of my head - I did not try to Dialyze it :) > > So the big question is: is there anyway to disable this warning? > > and the secondary question is: does it really make sense for this > warning to exist in the first place? It makes sense to me, but I'm no Dialyzer or success-typing expert. My intuition is mainly influenced by Hindley?Milner type systems. From max.lapshin@REDACTED Sun Jun 23 20:34:01 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 23 Jun 2013 22:34:01 +0400 Subject: [erlang-questions] Using avconv from erlang In-Reply-To: References: Message-ID: > > > > To make it clear: last versions of erlyvideo are not > fully opensourced because there haven't been any patches from users during > 4 years. So there is no use for me to loose money. You > > If one cannot find documentation for erlyvideo, then I will not even > comment it. Also there is published code for old versions which is working. > > About topic: you have written the code that will send data to erlang, but > only first chink. There are many other problems, but this is most > important: you need to add loop there. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From talklittle@REDACTED Sun Jun 23 21:06:18 2013 From: talklittle@REDACTED (Andrew Shu) Date: Sun, 23 Jun 2013 12:06:18 -0700 (PDT) Subject: [erlang-questions] iPhone unable to connect over SSL after upgrading to R16B01 In-Reply-To: References: Message-ID: <5f2a8a61-7cad-4896-a2b5-be02df1ad1ad@googlegroups.com> THANK YOU for posting this! This being my first time using Cowboy HTTPS and secure websockets, I was going crazy trying to figure out why SSL wasn't working via Chromium on Linux, while curl seemed to handle the self-signed certificates okay. It wouldn't have occurred to me that it could be an Erlang bug. After reverting to R16B, and removing all traces of R16B01, everything seems working. I wasted a lot of time swapping out SSL certificates to no avail. I think sticking with R16B is the best, or only, solution for now. I had been getting a Chromium gray error screen with "ERR_SSL_CLIENT_AUTH_SIGNATURE_FAILED". Not the usual "this certificate is not trusted" red screen. Firefox choked too. Curl seemed ok, strangely enough. Erlang shell was printing "SSL: certify: tls_connection.erl:2275:Fatal error: unknown ca" with some certificates. Swapping certs may have removed the error sometimes (I wasn't checking this carefully), but the Chromium error remained. A real CA-signed cert didn't help. This was happening even with the Cowboy SSL example at https://github.com/extend/cowboy/tree/master/examples/ssl_hello_world Again, thanks for indicating that this is an Erlang regression. R16B seems to work fine compared to R16B01. On Friday, June 21, 2013 11:44:34 AM UTC-4, Ransom Richardson wrote: > > Not sure if this is a known issue but I didn't see anything about it > when I searched. > > After upgrading to R16B01 our iPhone client can no longer connect to the > service over SSL. The same build works fine when deployed on a machine > running R16B. > > If we specify > > {ciphers, [{rsa,aes_256_cbc,sha256}]}, > > then it is able to connect successfully > > However, if we specify: > > {ciphers, [{rsa,aes_256_cbc,sha256}, {ecdhe_rsa,aes_256_cbc,sha}]}, > > then the connection fails. > > I didn't debug further, but did notice this following in > tls_handshake.erl, which seems like it might be related: > > %% TODO: need to take supported Curves into Account when selecting > the CipherSuite.... > %% if whe have an ECDSA cert with an unsupported curve, we need > to drop ECDSA ciphers > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pablo.platt@REDACTED Sun Jun 23 23:53:14 2013 From: pablo.platt@REDACTED (pablo platt) Date: Mon, 24 Jun 2013 00:53:14 +0300 Subject: [erlang-questions] Using avconv from erlang In-Reply-To: References: Message-ID: I can find the minimal erlyvideo documentation you published but you tried real hard to make it impossible to use erlyvideo. Last time I checked the forum was inactive and info was a mix of English and Russian. Good luck with your Flussonic product. I do have a loop in my code. The Erlang community need a "real" open source media server. On Sun, Jun 23, 2013 at 9:34 PM, Max Lapshin wrote: > >> > > >> To make it clear: last versions of erlyvideo are not >> fully opensourced because there haven't been any patches from users during >> 4 years. So there is no use for me to loose money. > > > You > >> >> If one cannot find documentation for erlyvideo, then I will not even >> comment it. Also there is published code for old versions which is working. >> >> About topic: you have written the code that will send data to erlang, but >> only first chink. There are many other problems, but this is most >> important: you need to add loop there. >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Mon Jun 24 00:17:05 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 24 Jun 2013 10:17:05 +1200 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <959684F3-5D32-489D-B7BC-5F1C13452020@gmail.com> References: <51C1F735.5040706@comcast.net> <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> <51C31FE4.4030100@comcast.net> <96590E43-C204-44DA-A9B7-2DD17C01FBFB@gmail.com> <51C32D6D.9000700@comcast.net> <478F57D8-3CF2-4FC6-B210-C7FEA0F9802E@cs.otago.ac.nz> <12F74DB5-1F10-473F-BDC3-62816B3D54F8@gmail.com> <4D87DAF1-3E82-434B-A431-72853DB881EC@cs.otago.ac.nz> <959684F3-5D32-489D-B7BC-5F1C13452020@gmail.com> Message-ID: On 21/06/2013, at 8:21 PM, Anthony Ramine wrote: > qual -> pat '<-' expr. > qual -> bin_pat '<=' expr. > qual -> expr. > > And then I would also add two generators which do not skip elements but crash instead: > > qual -> pat '<:-' expr. > qual -> bin_pat '<:=' expr. I don't understand what these are supposed to do. Would you care to say more? > > I do like the different direction you took though. Allowing only guards mean matches can be handled differently, thus one could write X = foo() instead of begin X = foo(), true end. I've been using X <- [foo()]. From alessandro.sivieri@REDACTED Mon Jun 24 01:19:29 2013 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Mon, 24 Jun 2013 01:19:29 +0200 Subject: [erlang-questions] erl_driver and select() usage Message-ID: Hi all, I have used the driver_select() function in my Erlang drivers, on sockets or other unix file types, and it worked without problems. Now, if I am using an existing library that already provides its own asynchronous function, which is called every time a certain hardware event happens, can I use this function to read from the driver? Would it be the same thing as using the driver_select() and invoking the reading part from the ready_input() function provided in the driver definition? -- Sivieri Alessandro alessandro.sivieri@REDACTED http://sivieri.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From yousuke.hara@REDACTED Mon Jun 24 04:21:42 2013 From: yousuke.hara@REDACTED (Yousuke Hara) Date: Mon, 24 Jun 2013 11:21:42 +0900 Subject: [erlang-questions] [ANN] LeoFS v0.14.3 Released! Message-ID: Hello, We have released LeoFS v0.14.3. LeoFS is an object storage system shaped for the Web. In the series of LeoFS v0.14, we improved performance (the cache mechanism and the coordinator) and fixed several bugs. You can see the release notes, which is here: < https://github.com/leo-project/leofs/blob/master/releasenotes/leofs-0.14.md> LeoFS rooms the improvement of scalability, so we're planning to implement "multi data-center replica placement" with v1.0.0 in the end of next month. If you have any questions and having interest in LeoFS, please feel free to contact us. * Repository: https://github.com/leo-project/leofs * Website: http://www.leofs.org/ * Blog: http://www.leofs.org/blog/index.html * Twitter: https://twitter.com/LeoFastStorage Thanks, Yosuke -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Jun 24 05:43:47 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 24 Jun 2013 07:43:47 +0400 Subject: [erlang-questions] erl_driver and select() usage In-Reply-To: References: Message-ID: It is good if you can find some file descriptor that is responsible for your hardware events. If no then launch it in separate thread -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Mon Jun 24 09:00:05 2013 From: ingela.andin@REDACTED (Ingela Andin) Date: Mon, 24 Jun 2013 09:00:05 +0200 Subject: [erlang-questions] iPhone unable to connect over SSL after upgrading to R16B01 In-Reply-To: <5f2a8a61-7cad-4896-a2b5-be02df1ad1ad@googlegroups.com> References: <5f2a8a61-7cad-4896-a2b5-be02df1ad1ad@googlegroups.com> Message-ID: Hi! Thank you for the example, I will try to reproduce the problem and fix it. A workaround is to specify the cipher suites to: {ciphers, [{dhe_rsa,aes_256_cbc,sha256}, {dhe_dss,aes_256_cbc,sha256}, {rsa,aes_256_cbc,sha256}, {dhe_rsa,aes_128_cbc,sha256}, {dhe_dss,aes_128_cbc,sha256}, {rsa,aes_128_cbc,sha256}, {dhe_rsa,aes_256_cbc,sha}, {dhe_dss,aes_256_cbc,sha}, {rsa,aes_256_cbc,sha}, {dhe_rsa,'3des_ede_cbc',sha}, {dhe_dss,'3des_ede_cbc',sha}, {rsa,'3des_ede_cbc',sha}, {dhe_rsa,aes_128_cbc,sha}, {dhe_dss,aes_128_cbc,sha}, {rsa,aes_128_cbc,sha}, {rsa,rc4_128,sha}, {rsa,rc4_128,md5}, {dhe_rsa,des_cbc,sha}, {rsa,des_cbc,sha}]} so that elliptic-curve ciphers will not be negotiated. The elliptic curve support in R16B01 is limited to named elliptic curves and, the TODO spotted seems like a likely candidate for the problem, I am so sorry we missed that when integrating the Elliptic curve contribution. Anyhow maybe the fix will be supporting all type of curves instead of doing the "TODO" I do not think there is so much work left to do that. Regards Ingela Erlang/OTP team - Ericsson AB 2013/6/23 Andrew Shu > THANK YOU for posting this! This being my first time using Cowboy HTTPS > and secure websockets, I was going crazy trying to figure out why SSL > wasn't working via Chromium on Linux, while curl seemed to handle the > self-signed certificates okay. It wouldn't have occurred to me that it > could be an Erlang bug. > > After reverting to R16B, and removing all traces of R16B01, everything > seems working. > I wasted a lot of time swapping out SSL certificates to no avail. I think > sticking with R16B is the best, or only, solution for now. > > I had been getting a Chromium gray error screen with > "ERR_SSL_CLIENT_AUTH_SIGNATURE_FAILED". Not the usual "this certificate is > not trusted" red screen. > Firefox choked too. Curl seemed ok, strangely enough. > > Erlang shell was printing "SSL: certify: tls_connection.erl:2275:Fatal > error: unknown ca" with some certificates. Swapping certs may have removed > the error sometimes (I wasn't checking this carefully), but the Chromium > error remained. A real CA-signed cert didn't help. > > This was happening even with the Cowboy SSL example at > https://github.com/extend/cowboy/tree/master/examples/ssl_hello_world > > Again, thanks for indicating that this is an Erlang regression. R16B seems > to work fine compared to R16B01. > > > On Friday, June 21, 2013 11:44:34 AM UTC-4, Ransom Richardson wrote: >> >> Not sure if this is a known issue but I didn't see anything about it >> when I searched. >> >> After upgrading to R16B01 our iPhone client can no longer connect to >> the service over SSL. The same build works fine when deployed on a machine >> running R16B. >> >> If we specify >> >> {ciphers, [{rsa,aes_256_cbc,sha256}]}, >> >> then it is able to connect successfully >> >> However, if we specify: >> >> {ciphers, [{rsa,aes_256_cbc,sha256}, {ecdhe_rsa,aes_256_cbc,sha}]}, >> >> then the connection fails. >> >> I didn't debug further, but did notice this following in >> tls_handshake.erl, which seems like it might be related: >> >> %% TODO: need to take supported Curves into Account when selecting >> the CipherSuite.... >> %% if whe have an ECDSA cert with an unsupported curve, we need >> to drop ECDSA ciphers >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gustav.simonsson@REDACTED Mon Jun 24 10:07:06 2013 From: gustav.simonsson@REDACTED (Gustav Simonsson) Date: Mon, 24 Jun 2013 10:07:06 +0200 Subject: [erlang-questions] Changing output file for erlc In-Reply-To: References: Message-ID: Hi, This prints the compiled beam of foo.erl to stdout and then redirects it to a file: erl -noshell -eval '{ok, _, Bin, _} = compile:file("foo", [binary, return]), file:write_file("/dev/fd/1", Bin), init:stop().' > foo.beam Cheers, Gustav Simonsson On Sat, Jun 22, 2013 at 2:55 PM, Ben Butler-Cole wrote: > Hello > > Is it possible to get erlc to change the file that it sends its output to? > I gather that the directory can be changed with `-o`, but I can't see any > way to change the file. Equivalently, is it possible to get it to write to > stdout? > > I would like to do this in order to use it with the redo build tool ( > http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/introduction-to-redo.html > ). > > Thanks > -Ben > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Mon Jun 24 11:26:56 2013 From: ingela.andin@REDACTED (Ingela Andin) Date: Mon, 24 Jun 2013 11:26:56 +0200 Subject: [erlang-questions] iPhone unable to connect over SSL after upgrading to R16B01 In-Reply-To: References: <5f2a8a61-7cad-4896-a2b5-be02df1ad1ad@googlegroups.com> Message-ID: Hi! This was happening even with the Cowboy SSL example at > https://github.com/extend/cowboy/tree/master/examples/ssl_hello_world > > > I am not sure what you where doing here, but I set up an openssl server with the certs from this exampel and then connected with R16B01 getting the cipher suite: "ECDHE-RSA-AES256-SHA384" or {ecdhe_rsa,aes_256_cbc,sha384}} in Erlang terminology. Same result if I use an erlang server. This however does not rule out that there is no situation where elliptic curve does not work as expected. So if either of you having this problem, or anyone else that has seen something like it, can provide a way of reproducing the problem that would great. Regards Ingela Erlang/OTP team - Ericsson AB -------------- next part -------------- An HTML attachment was scrubbed... URL: From alessandro.sivieri@REDACTED Mon Jun 24 11:30:47 2013 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Mon, 24 Jun 2013 11:30:47 +0200 Subject: [erlang-questions] erl_driver and select() usage In-Reply-To: References: Message-ID: Well, the asynchronous function of the library that I want to use actually uses select() in its implementation... so in a way it is already doing what I usually do with driver_select(). On Mon, Jun 24, 2013 at 5:43 AM, Max Lapshin wrote: > It is good if you can find some file descriptor that is responsible for > your hardware events. > If no then launch it in separate thread > -- Sivieri Alessandro alessandro.sivieri@REDACTED http://sivieri.wordpress.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Mon Jun 24 14:35:11 2013 From: ingela.andin@REDACTED (Ingela Andin) Date: Mon, 24 Jun 2013 14:35:11 +0200 Subject: [erlang-questions] iPhone unable to connect over SSL after upgrading to R16B01 In-Reply-To: <5f2a8a61-7cad-4896-a2b5-be02df1ad1ad@googlegroups.com> References: <5f2a8a61-7cad-4896-a2b5-be02df1ad1ad@googlegroups.com> Message-ID: Hi again, 2013/6/23 Andrew Shu > THANK YOU for posting this! This being my first time using Cowboy HTTPS > and secure websockets, I was going crazy trying to figure out why SSL > wasn't working via Chromium on Linux, while curl seemed to handle the > self-signed certificates okay. It wouldn't have occurred to me that it > could be an Erlang bug. > > After reverting to R16B, and removing all traces of R16B01, everything > seems working. > I wasted a lot of time swapping out SSL certificates to no avail. I think > sticking with R16B is the best, or only, solution for now. > > I had been getting a Chromium gray error screen with > "ERR_SSL_CLIENT_AUTH_SIGNATURE_FAILED". Not the usual "this certificate is > not trusted" red screen. > Firefox choked too. Curl seemed ok, strangely enough. > > Ok just make sure I run an openssl client against the erlang server too, with the cowboy example (as you try to connect to an erlang-server with other clients). This works too. So it seems openssl and curl can connect to the erlang-server but not chrom and firefox? And the connection fails due to that the client sends and alert. So atleas this problem seems not to be related to ECDSA certificates. (The missed TODO) . The other clients could also have issues with ecc-cipher suites, you could try to setting up an openssl server using openssl s_server -accept 4433 -CAfile ca.crt -cert server.crt -key server.key and trying the clients to see if they can connect with ecc-ciphers. The following command must return elliptic curve ciper cuites ECDH* ECDSA* > openssl ciphers Regards Ingela Erlang/OTP team - Ericsson AB -------------- next part -------------- An HTML attachment was scrubbed... URL: From n.oxyde@REDACTED Mon Jun 24 14:45:43 2013 From: n.oxyde@REDACTED (Anthony Ramine) Date: Mon, 24 Jun 2013 14:45:43 +0200 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <128B61D3-29AE-4BC2-8583-9DC60A878C88@gmail.com> References: <51C1F735.5040706@comcast.net> <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> <51C31FE4.4030100@comcast.net> <96590E43-C204-44DA-A9B7-2DD17C01FBFB@gmail.com> <51C32D6D.9000700@comcast.net> <478F57D8-3CF2-4FC6-B210-C7FEA0F9802E@cs.otago.ac.nz> <12F74DB5-1F10-473F-BDC3-62816B3D54F8@gmail.com> <4D87DAF1-3E82-434B-A431-72853DB881EC@cs.otago.ac.nz> <959684F3-5D32-489D-B7BC-5F1C13452020@gmail.com> <128B61D3-29AE-4BC2-8583-9DC60A878C88@gmail.com> Message-ID: Forgot to Cc the list, again. -- Anthony Ramine Le 24 juin 2013 ? 13:24, Anthony Ramine a ?crit : > Hello Richard, > > List = [foo], > [] = [ X || {bar,X} <- List ], > {'EXIT',_} = (catch [ X || {bar,X} <:- List ]). > > Basically, the <:- generator doesn't have a lc([_|T]) -> lc(T) skipping clause. > > Regards, From ddosia@REDACTED Mon Jun 24 14:51:47 2013 From: ddosia@REDACTED (Daniil Churikov) Date: Mon, 24 Jun 2013 05:51:47 -0700 (PDT) Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <51C1F735.5040706@comcast.net> References: <51C1F735.5040706@comcast.net> Message-ID: Hello Jonathan, hello list. In _some_ cases to avoid such validation nesting I use try/catch and pattern matching: try v1 = proplists:get_value(p1, L), v2 = proplists:get_value(p2, L), v3 = proplists:get_value(p3, L) catch _:_ -> %% handle exception here and return %% or re-throw it false end But you should clearly understand what code you will wrap with try/catch. It is possible hide some errors inside this block, so i try not not wrap complex logic inside, only some input validators. -------------- next part -------------- An HTML attachment was scrubbed... URL: From talklittle@REDACTED Mon Jun 24 16:49:07 2013 From: talklittle@REDACTED (Andrew Shu) Date: Mon, 24 Jun 2013 10:49:07 -0400 Subject: [erlang-questions] iPhone unable to connect over SSL after upgrading to R16B01 In-Reply-To: References: <5f2a8a61-7cad-4896-a2b5-be02df1ad1ad@googlegroups.com> Message-ID: Hi Ingela thanks for troubleshooting this. I ran the openssl server and pointed Chromium at https://localhost:4433, using the same certs from the Cowboy example. It works, so I don't think the web browser client is the issue. Here's the long HTML printout by openssl server, when I hit https://localhost:4433 using Chromium: s_server -accept 4433 -CAfile cowboy-ca.crt -cert server.crt -key server.key -www Ciphers supported in s_server binary TLSv1/SSLv3:ECDHE-RSA-AES256-GCM-SHA384TLSv1/SSLv3:ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1/SSLv3:ECDHE-RSA-AES256-SHA384 TLSv1/SSLv3:ECDHE-ECDSA-AES256-SHA384 TLSv1/SSLv3:ECDHE-RSA-AES256-SHA TLSv1/SSLv3:ECDHE-ECDSA-AES256-SHA TLSv1/SSLv3:SRP-DSS-AES-256-CBC-SHA TLSv1/SSLv3:SRP-RSA-AES-256-CBC-SHA TLSv1/SSLv3:DHE-DSS-AES256-GCM-SHA384TLSv1/SSLv3:DHE-RSA-AES256-GCM-SHA384 TLSv1/SSLv3:DHE-RSA-AES256-SHA256 TLSv1/SSLv3:DHE-DSS-AES256-SHA256 TLSv1/SSLv3:DHE-RSA-AES256-SHA TLSv1/SSLv3:DHE-DSS-AES256-SHA TLSv1/SSLv3:DHE-RSA-CAMELLIA256-SHA TLSv1/SSLv3:DHE-DSS-CAMELLIA256-SHA TLSv1/SSLv3:ECDH-RSA-AES256-GCM-SHA384TLSv1/SSLv3:ECDH-ECDSA-AES256-GCM-SHA384 TLSv1/SSLv3:ECDH-RSA-AES256-SHA384 TLSv1/SSLv3:ECDH-ECDSA-AES256-SHA384 TLSv1/SSLv3:ECDH-RSA-AES256-SHA TLSv1/SSLv3:ECDH-ECDSA-AES256-SHA TLSv1/SSLv3:AES256-GCM-SHA384 TLSv1/SSLv3:AES256-SHA256 TLSv1/SSLv3:AES256-SHA TLSv1/SSLv3:CAMELLIA256-SHA TLSv1/SSLv3:PSK-AES256-CBC-SHA TLSv1/SSLv3:ECDHE-RSA-DES-CBC3-SHA TLSv1/SSLv3:ECDHE-ECDSA-DES-CBC3-SHA TLSv1/SSLv3:SRP-DSS-3DES-EDE-CBC-SHA TLSv1/SSLv3:SRP-RSA-3DES-EDE-CBC-SHA TLSv1/SSLv3:EDH-RSA-DES-CBC3-SHA TLSv1/SSLv3:EDH-DSS-DES-CBC3-SHA TLSv1/SSLv3:ECDH-RSA-DES-CBC3-SHA TLSv1/SSLv3:ECDH-ECDSA-DES-CBC3-SHA TLSv1/SSLv3:DES-CBC3-SHA TLSv1/SSLv3:PSK-3DES-EDE-CBC-SHA TLSv1/SSLv3:ECDHE-RSA-AES128-GCM-SHA256 TLSv1/SSLv3:ECDHE-ECDSA-AES128-GCM-SHA256TLSv1/SSLv3:ECDHE-RSA-AES128-SHA256 TLSv1/SSLv3:ECDHE-ECDSA-AES128-SHA256TLSv1/SSLv3:ECDHE-RSA-AES128-SHA TLSv1/SSLv3:ECDHE-ECDSA-AES128-SHA TLSv1/SSLv3:SRP-DSS-AES-128-CBC-SHA TLSv1/SSLv3:SRP-RSA-AES-128-CBC-SHA TLSv1/SSLv3:DHE-DSS-AES128-GCM-SHA256 TLSv1/SSLv3:DHE-RSA-AES128-GCM-SHA256TLSv1/SSLv3:DHE-RSA-AES128-SHA256 TLSv1/SSLv3:DHE-DSS-AES128-SHA256 TLSv1/SSLv3:DHE-RSA-AES128-SHA TLSv1/SSLv3:DHE-DSS-AES128-SHA TLSv1/SSLv3:DHE-RSA-SEED-SHA TLSv1/SSLv3:DHE-DSS-SEED-SHA TLSv1/SSLv3:DHE-RSA-CAMELLIA128-SHA TLSv1/SSLv3:DHE-DSS-CAMELLIA128-SHA TLSv1/SSLv3:ECDH-RSA-AES128-GCM-SHA256 TLSv1/SSLv3:ECDH-ECDSA-AES128-GCM-SHA256TLSv1/SSLv3:ECDH-RSA-AES128-SHA256 TLSv1/SSLv3:ECDH-ECDSA-AES128-SHA256 TLSv1/SSLv3:ECDH-RSA-AES128-SHA TLSv1/SSLv3:ECDH-ECDSA-AES128-SHA TLSv1/SSLv3:AES128-GCM-SHA256 TLSv1/SSLv3:AES128-SHA256 TLSv1/SSLv3:AES128-SHA TLSv1/SSLv3:SEED-SHA TLSv1/SSLv3:CAMELLIA128-SHA TLSv1/SSLv3:PSK-AES128-CBC-SHA TLSv1/SSLv3:ECDHE-RSA-RC4-SHA TLSv1/SSLv3:ECDHE-ECDSA-RC4-SHA TLSv1/SSLv3:ECDH-RSA-RC4-SHA TLSv1/SSLv3:ECDH-ECDSA-RC4-SHA TLSv1/SSLv3:RC4-SHA TLSv1/SSLv3:RC4-MD5 TLSv1/SSLv3:PSK-RC4-SHA TLSv1/SSLv3:EDH-RSA-DES-CBC-SHA TLSv1/SSLv3:EDH-DSS-DES-CBC-SHA TLSv1/SSLv3:DES-CBC-SHA TLSv1/SSLv3:EXP-EDH-RSA-DES-CBC-SHA TLSv1/SSLv3:EXP-EDH-DSS-DES-CBC-SHA TLSv1/SSLv3:EXP-DES-CBC-SHA TLSv1/SSLv3:EXP-RC2-CBC-MD5 TLSv1/SSLv3:EXP-RC4-MD5 --- Ciphers common between both SSL end points: ECDHE-ECDSA-AES256-SHA ECDHE-RSA-AES256-SHA DHE-RSA-CAMELLIA256-SHA DHE-DSS-CAMELLIA256-SHA DHE-RSA-AES256-SHA DHE-DSS-AES256-SHA ECDH-RSA-AES256-SHA ECDH-ECDSA-AES256-SHA CAMELLIA256-SHA AES256-SHA ECDHE-ECDSA-RC4-SHA ECDHE-ECDSA-AES128-SHA ECDHE-RSA-RC4-SHA ECDHE-RSA-AES128-SHA DHE-RSA-CAMELLIA128-SHA DHE-DSS-CAMELLIA128-SHA DHE-RSA-AES128-SHA DHE-DSS-AES128-SHA ECDH-RSA-RC4-SHA ECDH-RSA-AES128-SHA ECDH-ECDSA-RC4-SHA ECDH-ECDSA-AES128-SHA SEED-SHA CAMELLIA128-SHA RC4-SHA RC4-MD5 AES128-SHA ECDHE-ECDSA-DES-CBC3-SHA ECDHE-RSA-DES-CBC3-SHA EDH-RSA-DES-CBC3-SHA EDH-DSS-DES-CBC3-SHA ECDH-RSA-DES-CBC3-SHA ECDH-ECDSA-DES-CBC3-SHA DES-CBC3-SHA --- New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-SHA SSL-Session: Protocol : TLSv1.1 Cipher : ECDHE-RSA-AES256-SHA Session-ID: Session-ID-ctx: 01000000 Master-Key: 9D707F79ED6FCF06935AAEC6C52E1E42642B2900EC790BC3BBC602F6DD19619220C3C800E173D75313D83DA6053E6786 Key-Arg : None PSK identity: None PSK identity hint: None SRP username: None Start Time: 1372085057 Timeout : 300 (sec) Verify return code: 0 (ok) --- 0 items in the session cache 0 client connects (SSL_connect()) 0 client renegotiates (SSL_connect()) 0 client connects that finished 1 server accepts (SSL_accept()) 0 server renegotiates (SSL_accept()) 1 server accepts that finished 0 session cache hits 1 session cache misses 0 session cache timeouts 0 callback cache hits 0 cache full overflows (128 allowed) --- no client certificate available On Mon, Jun 24, 2013 at 8:35 AM, Ingela Andin wrote: > Hi again, > > 2013/6/23 Andrew Shu > >> THANK YOU for posting this! This being my first time using Cowboy HTTPS >> and secure websockets, I was going crazy trying to figure out why SSL >> wasn't working via Chromium on Linux, while curl seemed to handle the >> self-signed certificates okay. It wouldn't have occurred to me that it >> could be an Erlang bug. >> >> After reverting to R16B, and removing all traces of R16B01, everything >> seems working. >> I wasted a lot of time swapping out SSL certificates to no avail. I think >> sticking with R16B is the best, or only, solution for now. >> >> I had been getting a Chromium gray error screen with >> "ERR_SSL_CLIENT_AUTH_SIGNATURE_FAILED". Not the usual "this certificate is >> not trusted" red screen. >> Firefox choked too. Curl seemed ok, strangely enough. >> >> > Ok just make sure I run an openssl client against the erlang server too, > with the cowboy example (as you > try to connect to an erlang-server with other clients). This works too. > So it seems openssl and curl can connect to the erlang-server but not chrom > and firefox? And the connection fails due to that the client sends and > alert. So atleas this problem seems not to be related to ECDSA > certificates. (The missed TODO) . > > The other clients could also have issues with ecc-cipher suites, you could > try to setting up an openssl server using > > openssl s_server -accept 4433 -CAfile ca.crt -cert server.crt -key > server.key > > and trying the clients to see if they can connect with ecc-ciphers. > > The following command must return elliptic curve ciper cuites ECDH* > ECDSA* > > > openssl ciphers > > > Regards Ingela Erlang/OTP team - Ericsson AB > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tcury@REDACTED Mon Jun 24 20:07:35 2013 From: tcury@REDACTED (Tiago Cury) Date: Mon, 24 Jun 2013 15:07:35 -0300 Subject: [erlang-questions] rebar dev Message-ID: <5629E2BB-953B-44AD-958D-D4D906AE677C@ymail.com> Is there a command in rebar to start a dev (foreground erl with all deps) environment ? From jeremy@REDACTED Mon Jun 24 20:11:56 2013 From: jeremy@REDACTED (Jeremy Ong) Date: Mon, 24 Jun 2013 11:11:56 -0700 Subject: [erlang-questions] rebar dev In-Reply-To: <5629E2BB-953B-44AD-958D-D4D906AE677C@ymail.com> References: <5629E2BB-953B-44AD-958D-D4D906AE677C@ymail.com> Message-ID: Just do it yourself: erl -pa deps/*/ebin Make it a local script if you do it a lot. On Mon, Jun 24, 2013 at 11:07 AM, Tiago Cury wrote: > Is there a command in rebar to start a dev (foreground erl with all deps) > environment ? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tcury@REDACTED Mon Jun 24 20:13:37 2013 From: tcury@REDACTED (Tiago Cury) Date: Mon, 24 Jun 2013 15:13:37 -0300 Subject: [erlang-questions] rebar dev In-Reply-To: References: <5629E2BB-953B-44AD-958D-D4D906AE677C@ymail.com> Message-ID: <2E3C29D2-1F9D-4875-9D71-C4474B064704@ymail.com> Ok, thanks. On Jun 24, 2013, at 3:11 PM, Jeremy Ong wrote: > Just do it yourself: > > erl -pa deps/*/ebin > > Make it a local script if you do it a lot. > > > On Mon, Jun 24, 2013 at 11:07 AM, Tiago Cury wrote: > Is there a command in rebar to start a dev (foreground erl with all deps) environment ? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul.james.barry@REDACTED Mon Jun 24 20:49:22 2013 From: paul.james.barry@REDACTED (Paul Barry) Date: Mon, 24 Jun 2013 19:49:22 +0100 Subject: [erlang-questions] Erlang jobs in Ireland Message-ID: Hi all. It's not often that the word "Erlang" makes it into a mainstream news story in Ireland (anywhere?), so here you go: http://www.rte.ie/news/business/2013/0624/458515-aol-ireland-jobs/ I would suggest this is, indeed, a very good sign. Cheers. Paul. -- Paul Barry, w: http://paulbarry.itcarlow.ie - e: paul.barry@REDACTED Lecturer, Computer Networking: Institute of Technology, Carlow, Ireland. -------------- next part -------------- An HTML attachment was scrubbed... URL: From seancribbs@REDACTED Mon Jun 24 23:43:57 2013 From: seancribbs@REDACTED (Sean Cribbs) Date: Mon, 24 Jun 2013 16:43:57 -0500 Subject: [erlang-questions] rebar dev In-Reply-To: <2E3C29D2-1F9D-4875-9D71-C4474B064704@ymail.com> References: <5629E2BB-953B-44AD-958D-D4D906AE677C@ymail.com> <2E3C29D2-1F9D-4875-9D71-C4474B064704@ymail.com> Message-ID: There's also the "shell" command, but it tends to have some strange interactions with the tty. On Mon, Jun 24, 2013 at 1:13 PM, Tiago Cury wrote: > Ok, thanks. > > On Jun 24, 2013, at 3:11 PM, Jeremy Ong wrote: > > Just do it yourself: > > erl -pa deps/*/ebin > > Make it a local script if you do it a lot. > > > On Mon, Jun 24, 2013 at 11:07 AM, Tiago Cury wrote: > >> Is there a command in rebar to start a dev (foreground erl with all deps) >> environment ? >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Tue Jun 25 00:31:03 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 25 Jun 2013 00:31:03 +0200 Subject: [erlang-questions] Using avconv from erlang In-Reply-To: References: Message-ID: <51C8C8A7.2000408@ninenines.eu> On 06/23/2013 11:53 PM, pablo platt wrote: > The Erlang community need a "real" open source media server. This is a bit of a harsh thing to say. If there was really a need from Erlang users, then perhaps the erlyvideo project would have received more patches. It can be guessed that the majority of the erlyvideo users aren't Erlang developers and therefore wouldn't contribute code back. So the decision to make it closed source makes sense. Now if one day there's a new need for an open source media server written in Erlang, surely alternatives will come out and Erlyvideo might want to open source it again. But there's no evidence that this will happen anytime soon. I'm all for open source personally, but it's no silver bullet. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From tuncer.ayaz@REDACTED Tue Jun 25 00:48:31 2013 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Tue, 25 Jun 2013 00:48:31 +0200 Subject: [erlang-questions] rebar dev In-Reply-To: References: <5629E2BB-953B-44AD-958D-D4D906AE677C@ymail.com> <2E3C29D2-1F9D-4875-9D71-C4474B064704@ymail.com> Message-ID: On Mon, Jun 24, 2013 at 11:43 PM, Sean Cribbs wrote: > There's also the "shell" command, but it tends to have some strange > interactions with the tty. What specifically? I've been prototyping fixes for at least two issues. So, knowing what's the most pressing problem will help me prioritize. > On Mon, Jun 24, 2013 at 1:13 PM, Tiago Cury wrote: >> >> Ok, thanks. >> >> On Jun 24, 2013, at 3:11 PM, Jeremy Ong wrote: >> >> Just do it yourself: >> >> erl -pa deps/*/ebin >> >> Make it a local script if you do it a lot. >> >> >> On Mon, Jun 24, 2013 at 11:07 AM, Tiago Cury wrote: >>> >>> Is there a command in rebar to start a dev (foreground erl with >>> all deps) environment ? From tcury@REDACTED Tue Jun 25 01:01:45 2013 From: tcury@REDACTED (Tiago Cury) Date: Mon, 24 Jun 2013 20:01:45 -0300 Subject: [erlang-questions] rebar dev In-Reply-To: References: <5629E2BB-953B-44AD-958D-D4D906AE677C@ymail.com> <2E3C29D2-1F9D-4875-9D71-C4474B064704@ymail.com> Message-ID: <8973BBC2-D3D3-45CB-A04E-3417888C3C5F@ymail.com> I tried rebar shell, Look: Air11:Backuper tcury$ ./rebar shell ==> jsx (shell) NOTICE: Using experimental 'shell' command ==> ibrowse (shell) NOTICE: Using experimental 'shell' command ==> yaws (shell) NOTICE: Using experimental 'shell' command ==> meck (shell) NOTICE: Using experimental 'shell' command ==> erlcloud (shell) NOTICE: Using experimental 'shell' command ==> Backuper (shell) NOTICE: Using experimental 'shell' command Erlang R16B (erts-5.10.1) [source-05f1189] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] =ERROR REPORT==== 24-Jun-2013::20:00:31 === driver_select(0x0000000011c82648, 0, ERL_DRV_READ ERL_DRV_USE, 1) by tty_sl (tty_sl -c -e) driver #Port<0.5953> stealing control of fd=0 from input driver fd (0/1) #Port<0.505> Eshell V5.10.1 (abort with ^G) 1> Eshell V5.10.1 (abort with ^G) 1> q(). ok 2> {error_logger,{{2013,6,24},{20,0,56}},"~s~n",["driver_select(0x0000000011c809d0, 0, ERL_DRV_READ ERL_DRV_WRITE, 0) by fd (0/1) driver #Port<0.505> stealing control of fd=0 from input driver tty_sl (tty_sl -c -e) #Port<0.5953> \n"]} From tristan.sloughter@REDACTED Tue Jun 25 01:08:53 2013 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Mon, 24 Jun 2013 18:08:53 -0500 Subject: [erlang-questions] rebar dev In-Reply-To: <8973BBC2-D3D3-45CB-A04E-3417888C3C5F@ymail.com> References: <5629E2BB-953B-44AD-958D-D4D906AE677C@ymail.com> <2E3C29D2-1F9D-4875-9D71-C4474B064704@ymail.com> <8973BBC2-D3D3-45CB-A04E-3417888C3C5F@ymail.com> Message-ID: Yup, rebar has commands like 'shell' and 'ct' and used to have 'dialyzer'. All have never worked as you'd expect, causing everyone to still use Makefiles. Without those 3 commands at least, plus consistent compiling of c_src would be nice..., rebar is just a slightly better erlc and still requires Make. On Mon, Jun 24, 2013 at 6:01 PM, Tiago Cury wrote: > I tried rebar shell, Look: > > Air11:Backuper tcury$ ./rebar shell > ==> jsx (shell) > NOTICE: Using experimental 'shell' command > ==> ibrowse (shell) > NOTICE: Using experimental 'shell' command > ==> yaws (shell) > NOTICE: Using experimental 'shell' command > ==> meck (shell) > NOTICE: Using experimental 'shell' command > ==> erlcloud (shell) > NOTICE: Using experimental 'shell' command > ==> Backuper (shell) > NOTICE: Using experimental 'shell' command > Erlang R16B (erts-5.10.1) [source-05f1189] [64-bit] [smp:2:2] > [async-threads:10] [hipe] [kernel-poll:false] > > > =ERROR REPORT==== 24-Jun-2013::20:00:31 === > > driver_select(0x0000000011c82648, 0, ERL_DRV_READ ERL_DRV_USE, 1) by > tty_sl (tty_sl -c -e) driver #Port<0.5953> stealing control of fd=0 from > input driver fd (0/1) #Port<0.505> > > > Eshell V5.10.1 (abort with ^G) > > 1> Eshell V5.10.1 (abort with ^G) > 1> q(). > ok > 2> > {error_logger,{{2013,6,24},{20,0,56}},"~s~n",["driver_select(0x0000000011c809d0, > 0, ERL_DRV_READ ERL_DRV_WRITE, 0) by fd (0/1) driver #Port<0.505> stealing > control of fd=0 from input driver tty_sl (tty_sl -c -e) #Port<0.5953> \n"]} > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dizzyd@REDACTED Tue Jun 25 01:18:46 2013 From: dizzyd@REDACTED (Dave Smith) Date: Mon, 24 Jun 2013 17:18:46 -0600 Subject: [erlang-questions] rebar dev In-Reply-To: References: <5629E2BB-953B-44AD-958D-D4D906AE677C@ymail.com> <2E3C29D2-1F9D-4875-9D71-C4474B064704@ymail.com> <8973BBC2-D3D3-45CB-A04E-3417888C3C5F@ymail.com> Message-ID: Care to elaborate on what "consistent compiling of c_src" is all about? Also, rebar still has ct support. :) D. On Mon, Jun 24, 2013 at 5:08 PM, Tristan Sloughter < tristan.sloughter@REDACTED> wrote: > Yup, rebar has commands like 'shell' and 'ct' and used to have 'dialyzer'. > All have never worked as you'd expect, causing everyone to still use > Makefiles. Without those 3 commands at least, plus consistent compiling of > c_src would be nice..., rebar is just a slightly better erlc and still > requires Make. > > > On Mon, Jun 24, 2013 at 6:01 PM, Tiago Cury wrote: > >> I tried rebar shell, Look: >> >> Air11:Backuper tcury$ ./rebar shell >> ==> jsx (shell) >> NOTICE: Using experimental 'shell' command >> ==> ibrowse (shell) >> NOTICE: Using experimental 'shell' command >> ==> yaws (shell) >> NOTICE: Using experimental 'shell' command >> ==> meck (shell) >> NOTICE: Using experimental 'shell' command >> ==> erlcloud (shell) >> NOTICE: Using experimental 'shell' command >> ==> Backuper (shell) >> NOTICE: Using experimental 'shell' command >> Erlang R16B (erts-5.10.1) [source-05f1189] [64-bit] [smp:2:2] >> [async-threads:10] [hipe] [kernel-poll:false] >> >> >> =ERROR REPORT==== 24-Jun-2013::20:00:31 === >> >> driver_select(0x0000000011c82648, 0, ERL_DRV_READ ERL_DRV_USE, 1) by >> tty_sl (tty_sl -c -e) driver #Port<0.5953> stealing control of fd=0 from >> input driver fd (0/1) #Port<0.505> >> >> >> Eshell V5.10.1 (abort with ^G) >> >> 1> Eshell V5.10.1 (abort with ^G) >> 1> q(). >> ok >> 2> >> {error_logger,{{2013,6,24},{20,0,56}},"~s~n",["driver_select(0x0000000011c809d0, >> 0, ERL_DRV_READ ERL_DRV_WRITE, 0) by fd (0/1) driver #Port<0.505> stealing >> control of fd=0 from input driver tty_sl (tty_sl -c -e) #Port<0.5953> \n"]} >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tristan.sloughter@REDACTED Tue Jun 25 01:23:54 2013 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Mon, 24 Jun 2013 18:23:54 -0500 Subject: [erlang-questions] rebar dev In-Reply-To: References: <5629E2BB-953B-44AD-958D-D4D906AE677C@ymail.com> <2E3C29D2-1F9D-4875-9D71-C4474B064704@ymail.com> <8973BBC2-D3D3-45CB-A04E-3417888C3C5F@ymail.com> Message-ID: Right, I mentioned 'shell and 'ct' as being commands that still exist, unlike 'dialyze'. The compiling of C code has been inconsistent in that a version of rebar from time 1 will not work, than updating to time 2 works and then stops working again at time 3. A point that makes it important to always bundle rebar escript with your source if you have C since most likely no other will work, but a problem then arises when that project is a dep of another and the rebar used in the parent doesn't work (too old, too new, who knows). On Mon, Jun 24, 2013 at 6:18 PM, Dave Smith wrote: > Care to elaborate on what "consistent compiling of c_src" is all about? > > Also, rebar still has ct support. :) > > D. > > > On Mon, Jun 24, 2013 at 5:08 PM, Tristan Sloughter < > tristan.sloughter@REDACTED> wrote: > >> Yup, rebar has commands like 'shell' and 'ct' and used to have >> 'dialyzer'. All have never worked as you'd expect, causing everyone to >> still use Makefiles. Without those 3 commands at least, plus consistent >> compiling of c_src would be nice..., rebar is just a slightly better erlc >> and still requires Make. >> >> >> On Mon, Jun 24, 2013 at 6:01 PM, Tiago Cury wrote: >> >>> I tried rebar shell, Look: >>> >>> Air11:Backuper tcury$ ./rebar shell >>> ==> jsx (shell) >>> NOTICE: Using experimental 'shell' command >>> ==> ibrowse (shell) >>> NOTICE: Using experimental 'shell' command >>> ==> yaws (shell) >>> NOTICE: Using experimental 'shell' command >>> ==> meck (shell) >>> NOTICE: Using experimental 'shell' command >>> ==> erlcloud (shell) >>> NOTICE: Using experimental 'shell' command >>> ==> Backuper (shell) >>> NOTICE: Using experimental 'shell' command >>> Erlang R16B (erts-5.10.1) [source-05f1189] [64-bit] [smp:2:2] >>> [async-threads:10] [hipe] [kernel-poll:false] >>> >>> >>> =ERROR REPORT==== 24-Jun-2013::20:00:31 === >>> >>> driver_select(0x0000000011c82648, 0, ERL_DRV_READ ERL_DRV_USE, 1) by >>> tty_sl (tty_sl -c -e) driver #Port<0.5953> stealing control of fd=0 from >>> input driver fd (0/1) #Port<0.505> >>> >>> >>> Eshell V5.10.1 (abort with ^G) >>> >>> 1> Eshell V5.10.1 (abort with ^G) >>> 1> q(). >>> ok >>> 2> >>> {error_logger,{{2013,6,24},{20,0,56}},"~s~n",["driver_select(0x0000000011c809d0, >>> 0, ERL_DRV_READ ERL_DRV_WRITE, 0) by fd (0/1) driver #Port<0.505> stealing >>> control of fd=0 from input driver tty_sl (tty_sl -c -e) #Port<0.5953> \n"]} >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tcury@REDACTED Tue Jun 25 01:29:55 2013 From: tcury@REDACTED (Tiago Cury) Date: Mon, 24 Jun 2013 20:29:55 -0300 Subject: [erlang-questions] rebar dev In-Reply-To: References: <5629E2BB-953B-44AD-958D-D4D906AE677C@ymail.com> <2E3C29D2-1F9D-4875-9D71-C4474B064704@ymail.com> <8973BBC2-D3D3-45CB-A04E-3417888C3C5F@ymail.com> Message-ID: <406D3BD4-D65B-407C-B7AE-558542464636@ymail.com> Erlang adoption could be bigger if it had a standard tool chain for building and package mgmt (retrieve and dependency). Look at the universe of java: ant, maven, buidr, gant, ivy, sbt, etc? etc? etc?. On Jun 24, 2013, at 8:18 PM, Dave Smith wrote: > Care to elaborate on what "consistent compiling of c_src" is all about? > > Also, rebar still has ct support. :) > > D. > > On Mon, Jun 24, 2013 at 5:08 PM, Tristan Sloughter wrote: > Yup, rebar has commands like 'shell' and 'ct' and used to have 'dialyzer'. All have never worked as you'd expect, causing everyone to still use Makefiles. Without those 3 commands at least, plus consistent compiling of c_src would be nice..., rebar is just a slightly better erlc and still requires Make. > > > On Mon, Jun 24, 2013 at 6:01 PM, Tiago Cury wrote: > I tried rebar shell, Look: > > Air11:Backuper tcury$ ./rebar shell > ==> jsx (shell) > NOTICE: Using experimental 'shell' command > ==> ibrowse (shell) > NOTICE: Using experimental 'shell' command > ==> yaws (shell) > NOTICE: Using experimental 'shell' command > ==> meck (shell) > NOTICE: Using experimental 'shell' command > ==> erlcloud (shell) > NOTICE: Using experimental 'shell' command > ==> Backuper (shell) > NOTICE: Using experimental 'shell' command > Erlang R16B (erts-5.10.1) [source-05f1189] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] > > > =ERROR REPORT==== 24-Jun-2013::20:00:31 === > driver_select(0x0000000011c82648, 0, ERL_DRV_READ ERL_DRV_USE, 1) by tty_sl (tty_sl -c -e) driver #Port<0.5953> stealing control of fd=0 from input driver fd (0/1) #Port<0.505> > > Eshell V5.10.1 (abort with ^G) > 1> Eshell V5.10.1 (abort with ^G) > 1> q(). > ok > 2> {error_logger,{{2013,6,24},{20,0,56}},"~s~n",["driver_select(0x0000000011c809d0, 0, ERL_DRV_READ ERL_DRV_WRITE, 0) by fd (0/1) driver #Port<0.505> stealing control of fd=0 from input driver tty_sl (tty_sl -c -e) #Port<0.5953> \n"]} > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue Jun 25 04:07:00 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 25 Jun 2013 14:07:00 +1200 Subject: [erlang-questions] style question - best way to test multiple non-guard conditions sequentially In-Reply-To: <7DB5F0AD-3461-41A8-995A-94991837EC94@khandkar.net> References: <51C1F735.5040706@comcast.net> <7399173B-E72B-409C-BC0F-EDF737CAAB1A@cs.otago.ac.nz> <51C31FE4.4030100@comcast.net> <51C39FA3.5020907@comcast.net> <7B6C2A52-CBEA-4439-B886-788F780D913A@cs.otago.ac.nz> <51C3CB52.70906@comcast.net> <3ED3C4FC-2C69-45EC-9799-87D7F60AC094@cs.otago.ac.nz> <7DB5F0AD-3461-41A8-995A-94991837EC94@khandkar.net> Message-ID: On 21/06/2013, at 5:23 PM, Siraaj Khandkar wrote: > http://www.gar1t.com/blog/2012/06/10/solving-embarrassingly-obvious-problems-in-erlang/ > http://existentialtype.wordpress.com/2011/03/15/dont-mention-equality/ > http://existentialtype.wordpress.com/2011/03/15/boolean-blindness/ I had those in mind, so thanks for finding them. There are a number of bug prevention tricks that are unreasonably effective. Compare class Mutex { ... bool trylock(); }; if (my_mutex->trylock()) { // is the lock held here or not? } with class Mutex { enum Status {This_Thread_Has_It, Another_Thread_Has_It}; ... Status trylock(); }; if (my_mutex->trylock() == Mutex::This_Thread_Has_It) { // we don't need to think here; we _know_. } And then in functional languages, so often we find that there's more information to return in at least one of the cases. Harper points out the benefits of case List of [] -> ... ; [Head|Tail] -> ... end compared with if null(List) -> ... ; true -> Head = hd(List), tl = tl(List), ... end I think there's a felt difference (not necessarily a fundamental difference) between languages with pattern matching -- where constructing "rich answers", returning them, and dispatching on them *feels* cheap -- and languages without -- where none of these is true. For example, even though I'm suspicious of Booleans, my Smalltalk code is riddled with #ifTrue:ifFalse:. (About 6% of the SLOC, in fact.) A good deal of that has to do with the fact that you can't return multiple answers at the same time from a Smalltalk method without creating a new object. (The fact that Smalltalk didn't have var parameters was excusable in an experimental language; the way Java made their absence a religious dogma is not. One of the things C# got right.) From garry@REDACTED Tue Jun 25 00:30:43 2013 From: garry@REDACTED (Garry Watkins) Date: Mon, 24 Jun 2013 18:30:43 -0400 Subject: [erlang-questions] Novice Question - What are the differences between liberts.a and liberts_r.a? Message-ID: What are the differences between liberts.a and liberts_r.a? -- Garry Watkins Managing Director DynaFOCUS, LLC (843) 608-8004 (google voice) (843) 276-6808 (cell) -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Tue Jun 25 08:16:16 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 25 Jun 2013 10:16:16 +0400 Subject: [erlang-questions] Novice Question - What are the differences between liberts.a and liberts_r.a? In-Reply-To: References: Message-ID: thread safety, perhaps? From watson.timothy@REDACTED Tue Jun 25 10:22:36 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 25 Jun 2013 09:22:36 +0100 Subject: [erlang-questions] rebar dev In-Reply-To: <406D3BD4-D65B-407C-B7AE-558542464636@ymail.com> References: <5629E2BB-953B-44AD-958D-D4D906AE677C@ymail.com> <2E3C29D2-1F9D-4875-9D71-C4474B064704@ymail.com> <8973BBC2-D3D3-45CB-A04E-3417888C3C5F@ymail.com> <406D3BD4-D65B-407C-B7AE-558542464636@ymail.com> Message-ID: On 25 Jun 2013, at 00:29, Tiago Cury wrote: > Erlang adoption could be bigger if it had a standard tool chain for building and package mgmt (retrieve and dependency). Look at the universe of java: ant, maven, buidr, gant, ivy, sbt, etc? etc? etc?. And how exactly is that (list) a standard tool chain? From stas@REDACTED Tue Jun 25 11:05:51 2013 From: stas@REDACTED (Stanislav Sedov) Date: Tue, 25 Jun 2013 02:05:51 -0700 Subject: [erlang-questions] rebar dev In-Reply-To: References: <5629E2BB-953B-44AD-958D-D4D906AE677C@ymail.com> <2E3C29D2-1F9D-4875-9D71-C4474B064704@ymail.com> <8973BBC2-D3D3-45CB-A04E-3417888C3C5F@ymail.com> <406D3BD4-D65B-407C-B7AE-558542464636@ymail.com> Message-ID: <4BE76861-AED5-4E2E-92A0-3036D836AE06@freebsd.org> On Jun 25, 2013, at 1:22 AM, Tim Watson wrote: > On 25 Jun 2013, at 00:29, Tiago Cury wrote: > >> Erlang adoption could be bigger if it had a standard tool chain for building and package mgmt (retrieve and dependency). Look at the universe of java: ant, maven, buidr, gant, ivy, sbt, etc? etc? etc?. > > And how exactly is that (list) a standard tool chain? > I believe that was . -- ST4096-RIPE From baliulia@REDACTED Tue Jun 25 11:49:09 2013 From: baliulia@REDACTED (=?UTF-8?B?SWduYXMgVnnFoW5pYXVza2Fz?=) Date: Tue, 25 Jun 2013 11:49:09 +0200 Subject: [erlang-questions] turning off opaque types underspecified warning In-Reply-To: References: Message-ID: <51C96795.4080906@gmail.com> On 06/23/2013 07:06 PM, Eric Merritt wrote: > So the big question is: is there anyway to disable this warning? Unlikely from the manual, but maybe `-Wno_opaque` would help? I agree that in general it's a pretty weird warning to give, it does make sense to introduce these types for better documented code. -- Ignas From watson.timothy@REDACTED Tue Jun 25 12:49:11 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 25 Jun 2013 11:49:11 +0100 Subject: [erlang-questions] rebar dev In-Reply-To: <4BE76861-AED5-4E2E-92A0-3036D836AE06@freebsd.org> References: <5629E2BB-953B-44AD-958D-D4D906AE677C@ymail.com> <2E3C29D2-1F9D-4875-9D71-C4474B064704@ymail.com> <8973BBC2-D3D3-45CB-A04E-3417888C3C5F@ymail.com> <406D3BD4-D65B-407C-B7AE-558542464636@ymail.com> <4BE76861-AED5-4E2E-92A0-3036D836AE06@freebsd.org> Message-ID: <59F9FC10-315F-4509-8F05-F88C51C1E6D8@gmail.com> On 25 Jun 2013, at 10:05, Stanislav Sedov wrote: >> >>> Erlang adoption could be bigger if it had a standard tool chain for building and package mgmt (retrieve and dependency). Look at the universe of java: ant, maven, buidr, gant, ivy, sbt, etc? etc? etc?. >> >> And how exactly is that (list) a standard tool chain? >> > > I believe that was . Hehe... Being a brit, I should've spotted that! :) From bosky101@REDACTED Tue Jun 25 13:24:07 2013 From: bosky101@REDACTED (Bhasker Kode) Date: Tue, 25 Jun 2013 16:54:07 +0530 Subject: [erlang-questions] Un-identifiable SSL issue when upgrading from R 15B-03 to 16A, 16B, 16B-01 Message-ID: Ping! I'm posting my experience upgrading from an Ubuntu 11.04 image R15B-02 to a more recent release. tl;dr https://gist.github.com/bosky101/5844535 I began with R16B01, R16B, R16A, R15B-03 respectively and was only able to run the expected TLS behaviour in R15-03. I tested them on different instances of the ubuntu 11.04 image. Various OpenSSL made no difference. Here is a module to test the same with accompanying notes https://gist.github.com/bosky101/5844535 It has constants for ?USE_VERIFY_PEER, ?USE_SSL_ACCEPT. You may have to pass ?CERT,?KEY,?CA as well. Considering that under the same circumstances - only R15B-03 worked - I wonder if this is a regression issue. Will be happy to pass ssh access if it helps. ~Bosky | Verbs.im | @bhaskerkode -------------- next part -------------- An HTML attachment was scrubbed... URL: From bmktuwien@REDACTED Tue Jun 25 13:51:03 2013 From: bmktuwien@REDACTED (BM Kim) Date: Tue, 25 Jun 2013 13:51:03 +0200 Subject: [erlang-questions] Cowboy (Erlang) VS Haskell (Warp) Message-ID: <51C98427.6040402@gmail.com> Hi folks, First of all, I want to apologise for my poor english skills, since english is not my first language, but I'll try my best to formulate my quesions as clear as possible. Second, I've just begun to learn erlang, so if I'm asking obvious "noob" questions I apologise for that too in advance... Anywho, now to my actual question: I am planning to write a high-performance server application in erlang, which will primarily handle HTTP requests. After some reseach with google, I narrowed down my choices to cowboy, misultin and mochiweb and decided to go with the cowboy library first... Looking at some tutorials, I've quickly built a small server capable of serving static files and was eager to see first benchmark-results... I've also built a small Haskell server using Warp library to compare it with erlang's cowboy... But my first impression was, that my cowboy server is much much slower than expected when serving static-files and after some research I found a presentation of the cowboy's author claiming that cowboy shouldn't be used for serving static-files. So I modified the server code, so that it replies to every request with in-memory 4Kb binary blob and compared it with my haskell warp server serving 4kb static file... this is my simple cowboy's http handler: ---------------------------------------------------------------------- blob() -> [<<0:8>> || _ <- lists:seq(1,4096)]. init({tcp, http}, Req, _Opts) -> {ok, Req, []}. handle(Req, State) -> {ok, Req2} = cowboy_req:reply(200, [], blob(), Req), {ok, Req2, State}. terminate(_Reason, _Req, _State) -> ok. ----------------------------------------------------------------------- I've tested both cowboy server and warp server with weighttp -k -c200 -n10000 http://localhost:8888/4kblob.txt on my i5-2540M thinkpad laptop running ubuntu 12.10. Haskell code was compiled via GHC 7.6 and my erlang VM is R16B. ---------------------------------------------------------------------- Warp: finished in 0 sec, 279 millisec and 52 microsec, 35835 req/s, 147331 kbyte/s ----------------------------------------------------------------------- ----------------------------------------------------------------------- Cowboy (serving actually in-memory blob): finished in 1 sec, 683 millisec and 264 microsec, 5940 req/s, 24447 kbyte/s ----------------------------------------------------------------------- I've noticed, that when testing the cowboy-server the cpu usage is so much higher than when testing the warp-server... What do you guys think could be the problem here? Can't imagine that erlang server is that much slower than haskell server... I aplogise again, if the question is forumalted chaotic, if you need more details just let me know... I am trying to look into this on my own, but since I am still an erlang noob, some helpful tips/advices from the experts will be much much appreciated!!! Thanks in advance, --cheers, bmk From qoocku@REDACTED Tue Jun 25 14:05:08 2013 From: qoocku@REDACTED (=?UTF-8?B?RGFtaWFuIERvYnJvY3p5xYRza2k=?=) Date: Tue, 25 Jun 2013 14:05:08 +0200 Subject: [erlang-questions] Cowboy (Erlang) VS Haskell (Warp) In-Reply-To: <51C98427.6040402@gmail.com> References: <51C98427.6040402@gmail.com> Message-ID: <51C98774.3080009@gmail.com> W dniu 25.06.2013 13:51, BM Kim pisze: > Hi folks, > > First of all, I want to apologise for my poor english skills, > since english is not my first language, but I'll try my best > to formulate my quesions as clear as possible. > > Second, I've just begun to learn erlang, so if I'm asking > obvious "noob" questions I apologise for that too in advance... > > Anywho, now to my actual question: > > I am planning to write a high-performance server application in erlang, > which will primarily handle HTTP requests. After some reseach with google, > I narrowed down my choices to cowboy, misultin and mochiweb and decided > to go with the cowboy library first... > > Looking at some tutorials, I've quickly built a small server capable of > serving static files and was eager to see first benchmark-results... > I've also built a small Haskell server using Warp library to compare it > with erlang's cowboy... > > But my first impression was, that my cowboy server is much much slower than > expected when serving static-files and after some research I found a presentation > of the cowboy's author claiming that cowboy shouldn't be used for serving > static-files. So I modified the server code, so that it replies to every > request with in-memory 4Kb binary blob and compared it with my haskell warp > server serving 4kb static file... > > this is my simple cowboy's http handler: > > ---------------------------------------------------------------------- > > blob() -> > [<<0:8>> || _ <- lists:seq(1,4096)]. First, try to replace blob/0 function with this: blob() -> <<0:(4096*8)>>. Then, restart the test and report ;) -- D. > init({tcp, http}, Req, _Opts) -> > {ok, Req, []}. > > > handle(Req, State) -> > {ok, Req2} = cowboy_req:reply(200, [], blob(), Req), > {ok, Req2, State}. > > > terminate(_Reason, _Req, _State) -> > ok. > > ----------------------------------------------------------------------- > > I've tested both cowboy server and warp server with > > weighttp -k -c200 -n10000 http://localhost:8888/4kblob.txt > > on my i5-2540M thinkpad laptop running ubuntu 12.10. > Haskell code was compiled via GHC 7.6 and my erlang VM is R16B. > > > ---------------------------------------------------------------------- > Warp: > > finished in 0 sec, 279 millisec and 52 microsec, 35835 req/s, 147331 kbyte/s > > ----------------------------------------------------------------------- > > ----------------------------------------------------------------------- > Cowboy (serving actually in-memory blob): > > finished in 1 sec, 683 millisec and 264 microsec, 5940 req/s, 24447 kbyte/s > ----------------------------------------------------------------------- > > I've noticed, that when testing the cowboy-server the cpu usage is so much > higher than when testing the warp-server... > What do you guys think could be the problem here? Can't imagine that erlang > server is that much slower than haskell server... > I aplogise again, if the question is forumalted chaotic, if you need more > details just let me know... > > I am trying to look into this on my own, but since I am still an erlang noob, some > helpful tips/advices from the experts will be much much appreciated!!! > > Thanks in advance, > > --cheers, > bmk > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ingela.andin@REDACTED Tue Jun 25 14:54:18 2013 From: ingela.andin@REDACTED (Ingela Andin) Date: Tue, 25 Jun 2013 14:54:18 +0200 Subject: [erlang-questions] Un-identifiable SSL issue when upgrading from R 15B-03 to 16A, 16B, 16B-01 In-Reply-To: References: Message-ID: It works fine for me on both R16B01 and R16B .... (I edited out the real hostnames) Erlang R16B01 (erts-5.10.2) [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] Eshell V5.10.2 (abort with ^G) 1> c(test). {ok,test} 2> test:start(). <0.54.0> <0.54.0> listening on 8443 use_verify_peer:false use_accept_timeout:false 3> test:client("XXX"). New connection loopdata Accepted! Ready to receive data Socket received data... <<"GET / HTTP/1.1\r\nHost:XXX:8443\r\n">> client got back <<"{\"ok\":true}">> 4> loopdata Accepted! Ready to receive data4> Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:8:8] [async-threads:10] [hipe] [kernel-poll:false] Eshell V5.10.1 (abort with ^G) 1> c(test). test.erl:124: Warning: variable 'Port' is unused {ok,test} 2> test:start(). <0.54.0> <0.54.0> listening on 8443 use_verify_peer:false use_accept_timeout:false 3> test:client("XXX"). New connection loopdata Accepted! Ready to receive data Socket received data... <<"GET / HTTP/1.1\r\nHost:XXX:8443\r\n">> client got back <<"{\"ok\":true}">> loopdata Accepted! Ready to receive data4> Regards Ingela Erlang/OTP team - Ericsson AB 2013/6/25 Bhasker Kode > Ping! > > I'm posting my experience upgrading from an Ubuntu 11.04 image R15B-02 to > a more recent release. > tl;dr https://gist.github.com/bosky101/5844535 > > I began with R16B01, R16B, R16A, R15B-03 respectively and was only able to > run the expected TLS behaviour in R15-03. > I tested them on different instances of the ubuntu 11.04 image. Various > OpenSSL made no difference. > > Here is a module to test the same with accompanying notes > https://gist.github.com/bosky101/5844535 > It has constants for ?USE_VERIFY_PEER, ?USE_SSL_ACCEPT. You may have to > pass ?CERT,?KEY,?CA as well. > > Considering that under the same circumstances - only R15B-03 worked - I > wonder if this is a regression issue. > Will be happy to pass ssh access if it helps. > > ~Bosky | Verbs.im | @bhaskerkode > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From garry@REDACTED Tue Jun 25 14:48:12 2013 From: garry@REDACTED (Garry Watkins) Date: Tue, 25 Jun 2013 08:48:12 -0400 Subject: [erlang-questions] Novice Question - What are the differences between liberts.a and liberts_r.a? In-Reply-To: References: Message-ID: <9670CD47-BED3-4EC6-BF43-927179E728D2@dynafocus.com> But which one is thread safe? The ei libraries use underscore St to indicate single threaded. Sent from my iPhone > On Jun 25, 2013, at 2:16 AM, Max Lapshin wrote: > > thread safety, perhaps? From max.lapshin@REDACTED Tue Jun 25 15:25:16 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 25 Jun 2013 17:25:16 +0400 Subject: [erlang-questions] Cowboy (Erlang) VS Haskell (Warp) In-Reply-To: <51C98774.3080009@gmail.com> References: <51C98427.6040402@gmail.com> <51C98774.3080009@gmail.com> Message-ID: You are making wrong tests. Erlyvideo can stream many gigabits of video via Cowboy server. Is it a performance? Is it bigger than 100 mbit of text messages? Problem with high performance is not only in primitive benchmarks. It is also about smoothness of work. What will happen with your server sending millions of messages if one gen_server in it will get blocked for a long disk read? Are you ready for crash due to OOM. If you want to spend time, than start writing in Erlang, because Erlang will allow you to write multicore software that can perform self-monitoring of internal loading via regulating message flows. It is a very important thing to understand: in Erlang cross-objects function calls are serialized into messages. Other languages offer you only direct function call. And only after you have some load, start measuring and collecting statistics: what is slow and what is not. From bosky101@REDACTED Tue Jun 25 15:20:16 2013 From: bosky101@REDACTED (Bhasker Kode) Date: Tue, 25 Jun 2013 18:50:16 +0530 Subject: [erlang-questions] Un-identifiable SSL issue when upgrading from R 15B-03 to 16A, 16B, 16B-01 In-Reply-To: References: Message-ID: Did you run this on a mac? Interestingly it worked for me on my desktop( mac osx lion), but not in the environment where R15-03 worked (ubuntu 11.04). ~B On Tue, Jun 25, 2013 at 6:24 PM, Ingela Andin wrote: > It works fine for me on both R16B01 and R16B .... (I edited out the real > hostnames) > > Erlang R16B01 (erts-5.10.2) [smp:4:4] [async-threads:10] [hipe] > [kernel-poll:false] > > Eshell V5.10.2 (abort with ^G) > 1> c(test). > {ok,test} > 2> test:start(). > <0.54.0> > > <0.54.0> listening on 8443 use_verify_peer:false use_accept_timeout:false > 3> test:client("XXX"). > > New connection > loopdata Accepted! Ready to receive data > Socket received data... > <<"GET / HTTP/1.1\r\nHost:XXX:8443\r\n">> > > client got back > <<"{\"ok\":true}">> > 4> > > loopdata Accepted! Ready to receive data4> > > > Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:8:8] [async-threads:10] > [hipe] [kernel-poll:false] > > Eshell V5.10.1 (abort with ^G) > 1> c(test). > test.erl:124: Warning: variable 'Port' is unused > {ok,test} > 2> test:start(). > <0.54.0> > > <0.54.0> listening on 8443 use_verify_peer:false use_accept_timeout:false > 3> test:client("XXX"). > > New connection > loopdata Accepted! Ready to receive data > Socket received data... > <<"GET / HTTP/1.1\r\nHost:XXX:8443\r\n">> > > client got back > <<"{\"ok\":true}">> > > loopdata Accepted! Ready to receive data4> > > > Regards Ingela Erlang/OTP team - Ericsson AB > > > > > 2013/6/25 Bhasker Kode > >> Ping! >> >> I'm posting my experience upgrading from an Ubuntu 11.04 image R15B-02 to >> a more recent release. >> tl;dr https://gist.github.com/bosky101/5844535 >> >> I began with R16B01, R16B, R16A, R15B-03 respectively and was only able >> to run the expected TLS behaviour in R15-03. >> I tested them on different instances of the ubuntu 11.04 image. Various >> OpenSSL made no difference. >> >> Here is a module to test the same with accompanying notes >> https://gist.github.com/bosky101/5844535 >> It has constants for ?USE_VERIFY_PEER, ?USE_SSL_ACCEPT. You may have to >> pass ?CERT,?KEY,?CA as well. >> >> Considering that under the same circumstances - only R15B-03 worked - I >> wonder if this is a regression issue. >> Will be happy to pass ssh access if it helps. >> >> ~Bosky | Verbs.im | @bhaskerkode >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mloftis@REDACTED Tue Jun 25 15:58:22 2013 From: mloftis@REDACTED (Michael Loftis) Date: Tue, 25 Jun 2013 06:58:22 -0700 Subject: [erlang-questions] Novice Question - What are the differences between liberts.a and liberts_r.a? In-Reply-To: <9670CD47-BED3-4EC6-BF43-927179E728D2@dynafocus.com> References: <9670CD47-BED3-4EC6-BF43-927179E728D2@dynafocus.com> Message-ID: _r suffix on a library is the accepted unix practice to indicate a library is reentrant, aka thread safe. On Tuesday, June 25, 2013, Garry Watkins wrote: > > But which one is thread safe? The ei libraries use underscore St to > indicate single threaded. > > Sent from my iPhone > > > On Jun 25, 2013, at 2:16 AM, Max Lapshin > > wrote: > > > > thread safety, perhaps? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- "Genius might be described as a supreme capacity for getting its possessors into trouble of all kinds." -- Samuel Butler -------------- next part -------------- An HTML attachment was scrubbed... URL: From ttom.kelly@REDACTED Tue Jun 25 16:26:22 2013 From: ttom.kelly@REDACTED (tom kelly) Date: Tue, 25 Jun 2013 15:26:22 +0100 Subject: [erlang-questions] spawn_link - zombie Message-ID: Hi List, I'm debugging a weird problem here on a customer site. We have an application that accepts tcp connections then kicks off a gen_server to implement some session logic, part of which is starting up a socket handler, which is also a gen_server, that provides a customised interface to the tcp connection. So after the init stage we have two linked processes. In my test environment both processes get tidied up correctly, when we get a tcp_closed from the connection or when we tell the session process to shut down. But on this customer site I'm seeing thousands of zombie socket_handlers, and they're causing us to hit a system_limit when we get new connections. I haven't managed to reproduce this state in my test environment yet but suspect it's happening in the init phase. Not sure if I've reproduced the cause of my problem here but I have this simple testcase that might help me if I can understand it. This spawns a process that spawns a process, if the parent process lives a while (comment in the 2nd sleep) then everything's as expected with two linked processes, with the parent linked to the shell process. But in this case if the parent terminates straight away the child will survive as an unlinked zombie, even though it was started with spawn_link, where I'd expect it to terminate too. Is this a bug or are my expectations incorrect? And any explanation much appreciated! 70> 70> self(). <0.126.0> 71> 71> 71> spawn_link(fun() -> io:format("~n~nP1: ~p~n~n",[self()]), spawn_link(fun() -> io:format("~n~nP2: ~p~n~n",[self()]), timer:sleep(300000) end)%, %timer:sleep(300000) end). P1: <0.137.0> <0.137.0> P2: <0.138.0> 72> 72> 72> process_info(erlang:list_to_pid("<0.137.0>"),links). undefined 73> 73> 73> process_info(erlang:list_to_pid("<0.138.0>"),links). {links,[]} 74> 74> 74> //TTom. From steve@REDACTED Tue Jun 25 16:33:05 2013 From: steve@REDACTED (Steve Strong) Date: Tue, 25 Jun 2013 16:33:05 +0200 Subject: [erlang-questions] spawn_link - zombie In-Reply-To: References: Message-ID: <9498BCF790DC40CD9C2DF9D6ABBAB0F8@srstrong.com> Your parent process (<0.137.0>) is exiting normally, which doesn't cause linked processes to exit. If the final line in the parent was something like "exit(fail)", then I'd expect <0.138.0> to terminate as well. The error handling section in http://www.erlang.org/doc/reference_manual/processes.html describes this. Cheers, Steve -- Steve Strong Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Tuesday, 25 June 2013 at 16:26, tom kelly wrote: > Hi List, > > I'm debugging a weird problem here on a customer site. We have an > application that accepts tcp connections then kicks off a gen_server > to implement some session logic, part of which is starting up a socket > handler, which is also a gen_server, that provides a customised > interface to the tcp connection. So after the init stage we have two > linked processes. > > In my test environment both processes get tidied up correctly, when we > get a tcp_closed from the connection or when we tell the session > process to shut down. But on this customer site I'm seeing thousands > of zombie socket_handlers, and they're causing us to hit a > system_limit when we get new connections. I haven't managed to > reproduce this state in my test environment yet but suspect it's > happening in the init phase. > > Not sure if I've reproduced the cause of my problem here but I have > this simple testcase that might help me if I can understand it. This > spawns a process that spawns a process, if the parent process lives a > while (comment in the 2nd sleep) then everything's as expected with > two linked processes, with the parent linked to the shell process. But > in this case if the parent terminates straight away the child will > survive as an unlinked zombie, even though it was started with > spawn_link, where I'd expect it to terminate too. > > Is this a bug or are my expectations incorrect? And any explanation > much appreciated! > > > 70> > 70> self(). > <0.126.0> > 71> > 71> > 71> spawn_link(fun() -> io:format("~n~nP1: ~p~n~n",[self()]), > spawn_link(fun() -> io:format("~n~nP2: > ~p~n~n",[self()]), > timer:sleep(300000) > end)%, > %timer:sleep(300000) > end). > > > P1: <0.137.0> > > <0.137.0> > > > P2: <0.138.0> > > 72> > 72> > 72> process_info(erlang:list_to_pid("<0.137.0>"),links). > undefined > 73> > 73> > 73> process_info(erlang:list_to_pid("<0.138.0>"),links). > {links,[]} > 74> > 74> > 74> > > > //TTom. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bosky101@REDACTED Tue Jun 25 16:25:19 2013 From: bosky101@REDACTED (Bhasker Kode) Date: Tue, 25 Jun 2013 19:55:19 +0530 Subject: [erlang-questions] Un-identifiable SSL issue when upgrading from R 15B-03 to 16A, 16B, 16B-01 In-Reply-To: References: Message-ID: Something with the environment perhaps then. Thanks. ~B On Tue, Jun 25, 2013 at 7:25 PM, Ingela Andin wrote: > HI again! > > 2013/6/25 Bhasker Kode > >> Also, can you try with a valid cacertfile ? >> >> I'm guessing this has to do with certificate chaining ( used in any >> production environment, but usually ignored in dev environments ) >> >> > > It works with a valid certificate too! > > > Regards Ingela Erlang/OTP team - Ericsson AB > > > >> ~B >> >> On Tue, Jun 25, 2013 at 6:50 PM, Bhasker Kode wrote: >> >>> Did you run this on a mac? >>> >>> Interestingly it worked for me on my desktop( mac osx lion), but not in >>> the environment where R15-03 worked (ubuntu 11.04). >>> >>> ~B >>> >>> >>> On Tue, Jun 25, 2013 at 6:24 PM, Ingela Andin wrote: >>> >>>> It works fine for me on both R16B01 and R16B .... (I edited out the >>>> real hostnames) >>>> >>>> Erlang R16B01 (erts-5.10.2) [smp:4:4] [async-threads:10] [hipe] >>>> [kernel-poll:false] >>>> >>>> Eshell V5.10.2 (abort with ^G) >>>> 1> c(test). >>>> {ok,test} >>>> 2> test:start(). >>>> <0.54.0> >>>> >>>> <0.54.0> listening on 8443 use_verify_peer:false >>>> use_accept_timeout:false >>>> 3> test:client("XXX"). >>>> >>>> New connection >>>> loopdata Accepted! Ready to receive data >>>> Socket received data... >>>> <<"GET / HTTP/1.1\r\nHost:XXX:8443\r\n">> >>>> >>>> client got back >>>> <<"{\"ok\":true}">> >>>> 4> >>>> >>>> loopdata Accepted! Ready to receive data4> >>>> >>>> >>>> Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:8:8] >>>> [async-threads:10] [hipe] [kernel-poll:false] >>>> >>>> Eshell V5.10.1 (abort with ^G) >>>> 1> c(test). >>>> test.erl:124: Warning: variable 'Port' is unused >>>> {ok,test} >>>> 2> test:start(). >>>> <0.54.0> >>>> >>>> <0.54.0> listening on 8443 use_verify_peer:false >>>> use_accept_timeout:false >>>> 3> test:client("XXX"). >>>> >>>> New connection >>>> loopdata Accepted! Ready to receive data >>>> Socket received data... >>>> <<"GET / HTTP/1.1\r\nHost:XXX:8443\r\n">> >>>> >>>> client got back >>>> <<"{\"ok\":true}">> >>>> >>>> loopdata Accepted! Ready to receive data4> >>>> >>>> >>>> Regards Ingela Erlang/OTP team - Ericsson AB >>>> >>>> >>>> >>>> >>>> 2013/6/25 Bhasker Kode >>>> >>>>> Ping! >>>>> >>>>> I'm posting my experience upgrading from an Ubuntu 11.04 image R15B-02 >>>>> to a more recent release. >>>>> tl;dr https://gist.github.com/bosky101/5844535 >>>>> >>>>> I began with R16B01, R16B, R16A, R15B-03 respectively and was only >>>>> able to run the expected TLS behaviour in R15-03. >>>>> I tested them on different instances of the ubuntu 11.04 image. >>>>> Various OpenSSL made no difference. >>>>> >>>>> Here is a module to test the same with accompanying notes >>>>> https://gist.github.com/bosky101/5844535 >>>>> It has constants for ?USE_VERIFY_PEER, ?USE_SSL_ACCEPT. You may have >>>>> to pass ?CERT,?KEY,?CA as well. >>>>> >>>>> Considering that under the same circumstances - only R15B-03 worked - >>>>> I wonder if this is a regression issue. >>>>> Will be happy to pass ssh access if it helps. >>>>> >>>>> ~Bosky | Verbs.im | @bhaskerkode >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Tue Jun 25 16:49:18 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Tue, 25 Jun 2013 16:49:18 +0200 Subject: [erlang-questions] Un-identifiable SSL issue when upgrading from R 15B-03 to 16A, 16B, 16B-01 In-Reply-To: References: Message-ID: <51C9ADEE.1070401@ninenines.eu> Are you using the Erlang packages from apt-get? Those are notoriously broken in many devilish ways. On 06/25/2013 03:20 PM, Bhasker Kode wrote: > Did you run this on a mac? > > Interestingly it worked for me on my desktop( mac osx lion), but not in > the environment where R15-03 worked (ubuntu 11.04). > > ~B > > On Tue, Jun 25, 2013 at 6:24 PM, Ingela Andin > wrote: > > It works fine for me on both R16B01 and R16B .... (I edited out the > real hostnames) > > Erlang R16B01 (erts-5.10.2) [smp:4:4] [async-threads:10] [hipe] > [kernel-poll:false] > > Eshell V5.10.2 (abort with ^G) > 1> c(test). > {ok,test} > 2> test:start(). > <0.54.0> > > <0.54.0> listening on 8443 use_verify_peer:false > use_accept_timeout:false > 3> test:client("XXX"). > > New connection > loopdata Accepted! Ready to receive data > Socket received data... > <<"GET / HTTP/1.1\r\nHost:XXX:8443\r\n">> > > client got back > <<"{\"ok\":true}">> > 4> > > loopdata Accepted! Ready to receive data4> > > > Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:8:8] > [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V5.10.1 (abort with ^G) > 1> c(test). > test.erl:124: Warning: variable 'Port' is unused > {ok,test} > 2> test:start(). > <0.54.0> > > <0.54.0> listening on 8443 use_verify_peer:false > use_accept_timeout:false > 3> test:client("XXX"). > > New connection > loopdata Accepted! Ready to receive data > Socket received data... > <<"GET / HTTP/1.1\r\nHost:XXX:8443\r\n">> > > client got back > <<"{\"ok\":true}">> > > loopdata Accepted! Ready to receive data4> > > > Regards Ingela Erlang/OTP team - Ericsson AB > > > > > 2013/6/25 Bhasker Kode > > > Ping! > > I'm posting my experience upgrading from an Ubuntu 11.04 image > R15B-02 to a more recent release. > tl;dr https://gist.github.com/bosky101/5844535 > > I began with R16B01, R16B, R16A, R15B-03 respectively and was > only able to run the expected TLS behaviour in R15-03. > I tested them on different instances of the ubuntu 11.04 image. > Various OpenSSL made no difference. > > Here is a module to test the same with accompanying notes > https://gist.github.com/bosky101/5844535 > It has constants for ?USE_VERIFY_PEER, ?USE_SSL_ACCEPT. You may > have to pass ?CERT,?KEY,?CA as well. > > Considering that under the same circumstances - only R15B-03 > worked - I wonder if this is a regression issue. > Will be happy to pass ssh access if it helps. > > ~Bosky | Verbs.im | @bhaskerkode > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From tcury@REDACTED Tue Jun 25 17:08:36 2013 From: tcury@REDACTED (Tiago Cury) Date: Tue, 25 Jun 2013 12:08:36 -0300 Subject: [erlang-questions] rebar dev In-Reply-To: <59F9FC10-315F-4509-8F05-F88C51C1E6D8@gmail.com> References: <5629E2BB-953B-44AD-958D-D4D906AE677C@ymail.com> <2E3C29D2-1F9D-4875-9D71-C4474B064704@ymail.com> <8973BBC2-D3D3-45CB-A04E-3417888C3C5F@ymail.com> <406D3BD4-D65B-407C-B7AE-558542464636@ymail.com> <4BE76861-AED5-4E2E-92A0-3036D836AE06@freebsd.org> <59F9FC10-315F-4509-8F05-F88C51C1E6D8@gmail.com> Message-ID: <88BD3E76-9337-4FAA-9BC1-8133EA308E6F@ymail.com> - package build, build phases, dependency retrieval. With rebar, if your app depends on a non-rebar app their dependencies are not retrieved. On Jun 25, 2013, at 7:49 AM, Tim Watson wrote: > On 25 Jun 2013, at 10:05, Stanislav Sedov wrote: >>> >>>> Erlang adoption could be bigger if it had a standard tool chain for building and package mgmt (retrieve and dependency). Look at the universe of java: ant, maven, buidr, gant, ivy, sbt, etc? etc? etc?. >>> >>> And how exactly is that (list) a standard tool chain? >>> >> >> I believe that was . > > Hehe... Being a brit, I should've spotted that! :) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From watson.timothy@REDACTED Tue Jun 25 17:36:47 2013 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 25 Jun 2013 16:36:47 +0100 Subject: [erlang-questions] rebar dev In-Reply-To: <88BD3E76-9337-4FAA-9BC1-8133EA308E6F@ymail.com> References: <5629E2BB-953B-44AD-958D-D4D906AE677C@ymail.com> <2E3C29D2-1F9D-4875-9D71-C4474B064704@ymail.com> <8973BBC2-D3D3-45CB-A04E-3417888C3C5F@ymail.com> <406D3BD4-D65B-407C-B7AE-558542464636@ymail.com> <4BE76861-AED5-4E2E-92A0-3036D836AE06@freebsd.org> <59F9FC10-315F-4509-8F05-F88C51C1E6D8@gmail.com> <88BD3E76-9337-4FAA-9BC1-8133EA308E6F@ymail.com> Message-ID: On 25 Jun 2013, at 16:08, Tiago Cury wrote: > - package build, build phases, dependency retrieval. > > With rebar, if your app depends on a non-rebar app their dependencies are not retrieved. > I'm well aware of these issues - they are the reasons that I wrote the rebar_alien_plugin, rebar_alt_deps and rebar_dist_plugin packages on github. I agree it would be better if there was a standardised approach, but IMO the community would be best served by 1. agreeing *how* this is *supposed* to work in practise 2. patching rebar only if necessary 3. performing all the /custom/ actions in a plugin 4. seeing whether anyone bothers to adopt it If (4) is successful, then there's maybe an argument for putting the changes into rebar itself. The *major* problem here is lack of traction and (more importantly) agreement in the Erlang open source community. A tool can't actually solve that issue. Maven became popular once it was released and enough people who were fed up with Ant decided to jump ship. Ivy came along and some jumped back. Then the rest, etc etc. And just for the record, I have spent many hours discussing this with Tuncer, Dizzy, Eric Merrit and the Erlware guys and others too. So far there have been lots of ideas and quite a few very reasonable arguments for keeping the status quo. Feel free to propose something concrete! :) Cheers, Tim From ingela.andin@REDACTED Tue Jun 25 17:48:48 2013 From: ingela.andin@REDACTED (Ingela Andin) Date: Tue, 25 Jun 2013 17:48:48 +0200 Subject: [erlang-questions] Un-identifiable SSL issue when upgrading from R 15B-03 to 16A, 16B, 16B-01 In-Reply-To: References: Message-ID: Hi! 2013/6/25 Bhasker Kode > Going off the forums. > > One more thing, can you try with ?USE_VERIFY_PEER , ?USE_ACCEPT_TIMEOUT > both to true ? > > Yes works too, I would be very disappointed if it did not ;) Regards Ingela Erlang/OTP team - Ericsson AB ~B > > On Tue, Jun 25, 2013 at 7:24 PM, Ingela Andin wrote: > >> Hi! >> >> >> 2013/6/25 Bhasker Kode >> >>> Did you run this on a mac? >>> >>> Interestingly it worked for me on my desktop( mac osx lion), but not in >>> the environment where R15-03 worked (ubuntu 11.04). >>> >>> >> >> I tested it on Ubuntu 12.04.2 and sles-10. So maybe something is off with >> the environment. >> >> >> Regards Ingela Erlang/OTP - Ericsson AB >> >> >>> ~B >>> >>> >>> On Tue, Jun 25, 2013 at 6:24 PM, Ingela Andin wrote: >>> >>>> It works fine for me on both R16B01 and R16B .... (I edited out the >>>> real hostnames) >>>> >>>> Erlang R16B01 (erts-5.10.2) [smp:4:4] [async-threads:10] [hipe] >>>> [kernel-poll:false] >>>> >>>> Eshell V5.10.2 (abort with ^G) >>>> 1> c(test). >>>> {ok,test} >>>> 2> test:start(). >>>> <0.54.0> >>>> >>>> <0.54.0> listening on 8443 use_verify_peer:false >>>> use_accept_timeout:false >>>> 3> test:client("XXX"). >>>> >>>> New connection >>>> loopdata Accepted! Ready to receive data >>>> Socket received data... >>>> <<"GET / HTTP/1.1\r\nHost:XXX:8443\r\n">> >>>> >>>> client got back >>>> <<"{\"ok\":true}">> >>>> 4> >>>> >>>> loopdata Accepted! Ready to receive data4> >>>> >>>> >>>> Erlang R16B (erts-5.10.1) [source] [64-bit] [smp:8:8] >>>> [async-threads:10] [hipe] [kernel-poll:false] >>>> >>>> Eshell V5.10.1 (abort with ^G) >>>> 1> c(test). >>>> test.erl:124: Warning: variable 'Port' is unused >>>> {ok,test} >>>> 2> test:start(). >>>> <0.54.0> >>>> >>>> <0.54.0> listening on 8443 use_verify_peer:false >>>> use_accept_timeout:false >>>> 3> test:client("XXX"). >>>> >>>> New connection >>>> loopdata Accepted! Ready to receive data >>>> Socket received data... >>>> <<"GET / HTTP/1.1\r\nHost:XXX:8443\r\n">> >>>> >>>> client got back >>>> <<"{\"ok\":true}">> >>>> >>>> loopdata Accepted! Ready to receive data4> >>>> >>>> >>>> Regards Ingela Erlang/OTP team - Ericsson AB >>>> >>>> >>>> >>>> >>>> 2013/6/25 Bhasker Kode >>>> >>>>> Ping! >>>>> >>>>> I'm posting my experience upgrading from an Ubuntu 11.04 image R15B-02 >>>>> to a more recent release. >>>>> tl;dr https://gist.github.com/bosky101/5844535 >>>>> >>>>> I began with R16B01, R16B, R16A, R15B-03 respectively and was only >>>>> able to run the expected TLS behaviour in R15-03. >>>>> I tested them on different instances of the ubuntu 11.04 image. >>>>> Various OpenSSL made no difference. >>>>> >>>>> Here is a module to test the same with accompanying notes >>>>> https://gist.github.com/bosky101/5844535 >>>>> It has constants for ?USE_VERIFY_PEER, ?USE_SSL_ACCEPT. You may have >>>>> to pass ?CERT,?KEY,?CA as well. >>>>> >>>>> Considering that under the same circumstances - only R15B-03 worked - >>>>> I wonder if this is a regression issue. >>>>> Will be happy to pass ssh access if it helps. >>>>> >>>>> ~Bosky | Verbs.im | @bhaskerkode >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Tue Jun 25 19:34:27 2013 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Tue, 25 Jun 2013 18:34:27 +0100 Subject: [erlang-questions] Un-identifiable SSL issue when upgrading from R 15B-03 to 16A, 16B, 16B-01 In-Reply-To: <51C9ADEE.1070401@ninenines.eu> References: <51C9ADEE.1070401@ninenines.eu> Message-ID: <20130625173427.GA5630@precise.local> On Tue, Jun 25, 2013 at 04:49:18PM +0200, Lo?c Hoguin wrote: > Are you using the Erlang packages from apt-get? Those are > notoriously broken in many devilish ways. Can you elaborate on that? For me packages from debian experimental seem to work pretty OK, for more than two years, on many environments. If there are concrete issues, they could be addressed IMO. Motiejus From essen@REDACTED Tue Jun 25 19:43:07 2013 From: essen@REDACTED (=?windows-1252?Q?Lo=EFc_Hoguin?=) Date: Tue, 25 Jun 2013 19:43:07 +0200 Subject: [erlang-questions] Un-identifiable SSL issue when upgrading from R 15B-03 to 16A, 16B, 16B-01 In-Reply-To: <20130625173427.GA5630@precise.local> References: <51C9ADEE.1070401@ninenines.eu> <20130625173427.GA5630@precise.local> Message-ID: <51C9D6AB.7030402@ninenines.eu> On 06/25/2013 07:34 PM, Motiejus Jak?tys wrote: > On Tue, Jun 25, 2013 at 04:49:18PM +0200, Lo?c Hoguin wrote: >> Are you using the Erlang packages from apt-get? Those are >> notoriously broken in many devilish ways. > > Can you elaborate on that? For me packages from debian experimental seem > to work pretty OK, for more than two years, on many environments. > > If there are concrete issues, they could be addressed IMO. I can't, I'm not an Ubuntu or Debian user. All I know is that when a user has issues and is using the packages from these distros, the solution is always to install the ESL ones or compile from source. I'm not sure anyone actually bothered to look at how exactly they are broken. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From bmktuwien@REDACTED Tue Jun 25 19:38:25 2013 From: bmktuwien@REDACTED (BM Kim) Date: Tue, 25 Jun 2013 17:38:25 +0000 (UTC) Subject: [erlang-questions] Cowboy (Erlang) VS Haskell (Warp) References: <51C98427.6040402@gmail.com> <51C98774.3080009@gmail.com> Message-ID: Damian Dobroczy?ski gmail.com> writes: > > W dniu 25.06.2013 13:51, BM Kim pisze: > > Hi folks, > > > > First of all, I want to apologise for my poor english skills, > > since english is not my first language, but I'll try my best > > to formulate my quesions as clear as possible. > > > > Second, I've just begun to learn erlang, so if I'm asking > > obvious "noob" questions I apologise for that too in advance... > > > > Anywho, now to my actual question: > > > > I am planning to write a high-performance server application in erlang, > > which will primarily handle HTTP requests. After some reseach with google, > > I narrowed down my choices to cowboy, misultin and mochiweb and decided > > to go with the cowboy library first... > > > > Looking at some tutorials, I've quickly built a small server capable of > > serving static files and was eager to see first benchmark-results... > > I've also built a small Haskell server using Warp library to compare it > > with erlang's cowboy... > > > > But my first impression was, that my cowboy server is much much slower than > > expected when serving static-files and after some research I found a presentation > > of the cowboy's author claiming that cowboy shouldn't be used for serving > > static-files. So I modified the server code, so that it replies to every > > request with in-memory 4Kb binary blob and compared it with my haskell warp > > server serving 4kb static file... > > > > this is my simple cowboy's http handler: > > > > ---------------------------------------------------------------------- > > > > blob() -> > > [<<0:8>> || _ <- lists:seq(1,4096)]. > > First, try to replace blob/0 function with this: > > blob() -> <<0:(4096*8)>>. > > Then, restart the test and report ;) > Hi, Thank you very much for pointing out the obvious mistake... After correcting it, I got improvement from 5940 req/s to 8650 req/s... But still much slower than the haskell warp-server, which has throughput of 38000 req/s... But I have another question regarding blob/0. Is it going to be evaluated only once (like GHC would do) since it is a pure expression? I'm not so sure, since erlang is not pure and any function can have side-effects which you can't mark as with the IO monad in Haskell... From tuncer.ayaz@REDACTED Tue Jun 25 20:27:23 2013 From: tuncer.ayaz@REDACTED (Tuncer Ayaz) Date: Tue, 25 Jun 2013 20:27:23 +0200 Subject: [erlang-questions] rebar dev In-Reply-To: <8973BBC2-D3D3-45CB-A04E-3417888C3C5F@ymail.com> References: <5629E2BB-953B-44AD-958D-D4D906AE677C@ymail.com> <2E3C29D2-1F9D-4875-9D71-C4474B064704@ymail.com> <8973BBC2-D3D3-45CB-A04E-3417888C3C5F@ymail.com> Message-ID: On Tue, Jun 25, 2013 at 1:01 AM, Tiago Cury wrote: > I tried rebar shell, Look: Hi Tiago, thanks for the report. > Air11:Backuper tcury$ ./rebar shell > ==> jsx (shell) > NOTICE: Using experimental 'shell' command > ==> ibrowse (shell) > NOTICE: Using experimental 'shell' command > ==> yaws (shell) > NOTICE: Using experimental 'shell' command > ==> meck (shell) > NOTICE: Using experimental 'shell' command > ==> erlcloud (shell) > NOTICE: Using experimental 'shell' command > ==> Backuper (shell) > NOTICE: Using experimental 'shell' command This is wrong and should be fixed in my unpublished branch. Are you interested in testing one or more rough patches? > Erlang R16B (erts-5.10.1) [source-05f1189] [64-bit] [smp:2:2] > [async-threads:10] [hipe] [kernel-poll:false] > > > =ERROR REPORT==== 24-Jun-2013::20:00:31 === > driver_select(0x0000000011c82648, > 0, ERL_DRV_READ ERL_DRV_USE, 1) by tty_sl (tty_sl -c -e) driver > #Port<0.5953> stealing control of fd=0 from input driver fd (0/1) > #Port<0.505> > > Eshell V5.10.1 (abort with ^G) > 1> Eshell V5.10.1 (abort with ^G) > 1> q(). > ok > 2> > {error_logger,{{2013,6,24},{20,0,56}},"~s~n",["driver_select(0x0000000011c809d0, > 0, ERL_DRV_READ ERL_DRV_WRITE, 0) by fd (0/1) driver #Port<0.505> > stealing control of fd=0 from input driver tty_sl (tty_sl -c -e) > #Port<0.5953> \n"]} Here's a ticket with all known issues and possible solutions: https://github.com/rebar/rebar/issues/106 From sergej.jurecko@REDACTED Tue Jun 25 20:30:22 2013 From: sergej.jurecko@REDACTED (Sergej Jurecko) Date: Tue, 25 Jun 2013 20:30:22 +0200 Subject: [erlang-questions] Cowboy (Erlang) VS Haskell (Warp) In-Reply-To: References: <51C98427.6040402@gmail.com> <51C98774.3080009@gmail.com> Message-ID: <61361B31-320E-424F-BB57-05FE1E6FE0AF@gmail.com> On Jun 25, 2013, at 7:38 PM, BM Kim wrote: > Damian Dobroczy?ski gmail.com> writes: > >> >> W dniu 25.06.2013 13:51, BM Kim pisze: >>> Hi folks, >>> >>> First of all, I want to apologise for my poor english skills, >>> since english is not my first language, but I'll try my best >>> to formulate my quesions as clear as possible. >>> >>> Second, I've just begun to learn erlang, so if I'm asking >>> obvious "noob" questions I apologise for that too in advance... >>> >>> Anywho, now to my actual question: >>> >>> I am planning to write a high-performance server application in erlang, >>> which will primarily handle HTTP requests. After some reseach with > google, >>> I narrowed down my choices to cowboy, misultin and mochiweb and decided >>> to go with the cowboy library first... >>> >>> Looking at some tutorials, I've quickly built a small server capable of >>> serving static files and was eager to see first benchmark-results... >>> I've also built a small Haskell server using Warp library to compare it >>> with erlang's cowboy... >>> >>> But my first impression was, that my cowboy server is much much slower > than >>> expected when serving static-files and after some research I found a > presentation >>> of the cowboy's author claiming that cowboy shouldn't be used for > serving >>> static-files. So I modified the server code, so that it replies to every >>> request with in-memory 4Kb binary blob and compared it with my haskell > warp >>> server serving 4kb static file... >>> >>> this is my simple cowboy's http handler: >>> >>> ---------------------------------------------------------------------- >>> >>> blob() -> >>> [<<0:8>> || _ <- lists:seq(1,4096)]. >> >> First, try to replace blob/0 function with this: >> >> blob() -> <<0:(4096*8)>>. >> >> Then, restart the test and report ;) >> > > > Hi, > > Thank you very much for pointing out the obvious mistake... > After correcting it, I got improvement from 5940 req/s to 8650 req/s... > > But still much slower than the haskell warp-server, which has throughput > of 38000 req/s... > > But I have another question regarding blob/0. Is it going to be evaluated > only once (like GHC would do) since it is a pure expression? I'm not > so sure, since erlang is not pure and any function can have side-effects > which you can't mark as with the IO monad in Haskell... > If you only care about performance use Haskell or c. Sergej From hartem@REDACTED Tue Jun 25 20:36:36 2013 From: hartem@REDACTED (Artem Harutyunyan) Date: Tue, 25 Jun 2013 11:36:36 -0700 Subject: [erlang-questions] Core Backend Developer opportunity in Silicon Valley Message-ID: Hi, We are looking for a seasoned backend developer who likes to program in C and Erlang: https://www.qualys.com/careers/?jvi=oAMBXfwE,Job Please let me know if you are interested. Best regards, Artem. From essen@REDACTED Tue Jun 25 20:38:55 2013 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 25 Jun 2013 20:38:55 +0200 Subject: [erlang-questions] Cowboy (Erlang) VS Haskell (Warp) In-Reply-To: References: <51C98427.6040402@gmail.com> <51C98774.3080009@gmail.com> Message-ID: <51C9E3BF.3020006@ninenines.eu> On 06/25/2013 07:38 PM, BM Kim wrote: > Damian Dobroczy?ski gmail.com> writes: > >> >> W dniu 25.06.2013 13:51, BM Kim pisze: >>> Hi folks, >>> >>> First of all, I want to apologise for my poor english skills, >>> since english is not my first language, but I'll try my best >>> to formulate my quesions as clear as possible. >>> >>> Second, I've just begun to learn erlang, so if I'm asking >>> obvious "noob" questions I apologise for that too in advance... >>> >>> Anywho, now to my actual question: >>> >>> I am planning to write a high-performance server application in erlang, >>> which will primarily handle HTTP requests. After some reseach with > google, >>> I narrowed down my choices to cowboy, misultin and mochiweb and decided >>> to go with the cowboy library first... >>> >>> Looking at some tutorials, I've quickly built a small server capable of >>> serving static files and was eager to see first benchmark-results... >>> I've also built a small Haskell server using Warp library to compare it >>> with erlang's cowboy... >>> >>> But my first impression was, that my cowboy server is much much slower > than >>> expected when serving static-files and after some research I found a > presentation >>> of the cowboy's author claiming that cowboy shouldn't be used for > serving >>> static-files. So I modified the server code, so that it replies to every >>> request with in-memory 4Kb binary blob and compared it with my haskell > warp >>> server serving 4kb static file... >>> >>> this is my simple cowboy's http handler: >>> >>> ---------------------------------------------------------------------- >>> >>> blob() -> >>> [<<0:8>> || _ <- lists:seq(1,4096)]. >> >> First, try to replace blob/0 function with this: >> >> blob() -> <<0:(4096*8)>>. >> >> Then, restart the test and report ;) >> > > > Hi, > > Thank you very much for pointing out the obvious mistake... > After correcting it, I got improvement from 5940 req/s to 8650 req/s... > > But still much slower than the haskell warp-server, which has throughput > of 38000 req/s... That's not surprising at all, you are performing the same thing exactly all the time, so of course Haskell is going to be fast at this. Same goes for JIT enabled environments like Java. The JIT can easily compile it to machine code once and be done with it. You're not actually testing the HTTP server, or even the language performance, you are testing the ability of the platform to optimize one operation to death. > But I have another question regarding blob/0. Is it going to be evaluated > only once (like GHC would do) since it is a pure expression? I'm not > so sure, since erlang is not pure and any function can have side-effects > which you can't mark as with the IO monad in Haskell... Erlang doesn't do that. Closest is at compile time when A = 1 + 1 becomes A = 2 in the compiled file, but it's only done for a very small subset of all expressions. Erlang shines not in synthetic benchmarks, but in production, when thousands of clients connect to your server and expect their requests to arrive as quick as if they were alone on the server. Erlang is optimized for latency, and this latency will be the same regardless of there being only one user or ten thousands. Your benchmark on the other hand is evaluating throughput. Throughput is boring, and not really useful for Web applications. (See Max' email for more details on that.) -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From bmktuwien@REDACTED Tue Jun 25 20:54:43 2013 From: bmktuwien@REDACTED (BM Kim) Date: Tue, 25 Jun 2013 20:54:43 +0200 Subject: [erlang-questions] Cowboy (Erlang) VS Haskell (Warp) In-Reply-To: <51C9E3BF.3020006@ninenines.eu> References: <51C98427.6040402@gmail.com> <51C98774.3080009@gmail.com> <51C9E3BF.3020006@ninenines.eu> Message-ID: <51C9E773.1080809@gmail.com> On 06/25/2013 08:38 PM, Lo?c Hoguin wrote: [...] >>>> >>>> blob() -> >>>> [<<0:8>> || _ <- lists:seq(1,4096)]. >>> >>> First, try to replace blob/0 function with this: >>> >>> blob() -> <<0:(4096*8)>>. >>> >>> Then, restart the test and report ;) >>> >> >> >> Hi, >> >> Thank you very much for pointing out the obvious mistake... >> After correcting it, I got improvement from 5940 req/s to 8650 req/s... >> >> But still much slower than the haskell warp-server, which has throughput >> of 38000 req/s... > > That's not surprising at all, you are performing the same thing exactly all the time, so of course Haskell is going to be fast at this. Same goes > for JIT enabled environments like Java. The JIT can easily compile it to machine code once and be done with it. > > You're not actually testing the HTTP server, or even the language performance, you are testing the ability of the platform to optimize one operation > to death. > >> But I have another question regarding blob/0. Is it going to be evaluated >> only once (like GHC would do) since it is a pure expression? I'm not >> so sure, since erlang is not pure and any function can have side-effects >> which you can't mark as with the IO monad in Haskell... > > Erlang doesn't do that. Closest is at compile time when A = 1 + 1 becomes A = 2 in the compiled file, but it's only done for a very small subset of > all expressions. > > Erlang shines not in synthetic benchmarks, but in production, when thousands of clients connect to your server and expect their requests to arrive > as quick as if they were alone on the server. Erlang is optimized for latency, and this latency will be the same regardless of there being only one > user or ten thousands. > > Your benchmark on the other hand is evaluating throughput. Throughput is boring, and not really useful for Web applications. (See Max' email for > more details on that.) > Thank you so much, for your helpful and insightful explanation, which gives me a new perspective on my current benchmark-strategy and expectations... If it is not too much trouble, can you me some further advice regarding erlang performance in general and using the cowboy library "efficiently", if there are some issues/tricks which are not documented in the manual but I should be aware of? by the way: can you recommend some opensource erlang projects incorporating the cowboy library that I can learn from? Again thank you for your time and advices! -- bmk From essen@REDACTED Tue Jun 25 21:21:11 2013 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 25 Jun 2013 21:21:11 +0200 Subject: [erlang-questions] Cowboy (Erlang) VS Haskell (Warp) In-Reply-To: <51C9E773.1080809@gmail.com> References: <51C98427.6040402@gmail.com> <51C98774.3080009@gmail.com> <51C9E3BF.3020006@ninenines.eu> <51C9E773.1080809@gmail.com> Message-ID: <51C9EDA7.1090706@ninenines.eu> On 06/25/2013 08:54 PM, BM Kim wrote: > On 06/25/2013 08:38 PM, Lo?c Hoguin wrote: > > [...] >>>>> >>>>> blob() -> >>>>> [<<0:8>> || _ <- lists:seq(1,4096)]. >>>> >>>> First, try to replace blob/0 function with this: >>>> >>>> blob() -> <<0:(4096*8)>>. >>>> >>>> Then, restart the test and report ;) >>>> >>> >>> >>> Hi, >>> >>> Thank you very much for pointing out the obvious mistake... >>> After correcting it, I got improvement from 5940 req/s to 8650 req/s... >>> >>> But still much slower than the haskell warp-server, which has throughput >>> of 38000 req/s... >> >> That's not surprising at all, you are performing the same thing exactly all the time, so of course Haskell is going to be fast at this. Same goes >> for JIT enabled environments like Java. The JIT can easily compile it to machine code once and be done with it. >> >> You're not actually testing the HTTP server, or even the language performance, you are testing the ability of the platform to optimize one operation >> to death. >> >>> But I have another question regarding blob/0. Is it going to be evaluated >>> only once (like GHC would do) since it is a pure expression? I'm not >>> so sure, since erlang is not pure and any function can have side-effects >>> which you can't mark as with the IO monad in Haskell... >> >> Erlang doesn't do that. Closest is at compile time when A = 1 + 1 becomes A = 2 in the compiled file, but it's only done for a very small subset of >> all expressions. >> >> Erlang shines not in synthetic benchmarks, but in production, when thousands of clients connect to your server and expect their requests to arrive >> as quick as if they were alone on the server. Erlang is optimized for latency, and this latency will be the same regardless of there being only one >> user or ten thousands. >> >> Your benchmark on the other hand is evaluating throughput. Throughput is boring, and not really useful for Web applications. (See Max' email for >> more details on that.) >> > > Thank you so much, for your helpful and insightful explanation, which gives me a new > perspective on my current benchmark-strategy and expectations... > If it is not too much trouble, can you me some further advice regarding erlang > performance in general and using the cowboy library "efficiently", if there are some > issues/tricks which are not documented in the manual but I should be aware of? The tricks are documented in either the Cowboy manual or the Ranch manual (for things relating to sockets). But there aren't many of them and you shouldn't start with these questions. Ask yourself whether the project enables you to do what you need instead. There's no doubt that Erlang fits your needs if it's a Web project. The only gotcha would be that you want to use a NIF for image manipulation (there isn't any pure Erlang lib anyway at this time), and you want to use the OpenCL NIFs for any heavy computation. You can also use Erlang to handle clusters of Python instances or any other platforms, like has been done by many people (see CloudI for example, if you have that kind of need). > by the way: can you recommend some opensource erlang projects incorporating the > cowboy library that I can learn from? I'd look at n2o, axiom, ChicagoBoss, or other frameworks, in that order. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From mjtruog@REDACTED Tue Jun 25 21:52:20 2013 From: mjtruog@REDACTED (okeuday) Date: Tue, 25 Jun 2013 12:52:20 -0700 (PDT) Subject: [erlang-questions] Cowboy (Erlang) VS Haskell (Warp) In-Reply-To: References: <51C98427.6040402@gmail.com> <51C98774.3080009@gmail.com> Message-ID: When I use their benchmark here: https://github.com/yesodweb/benchmarks (as mentioned here: http://www.yesodweb.com/blog/2011/03/preliminary-warp-cross-language-benchmarks) to test elli, here: https://github.com/knutin/elli I get 99000 req/s. The (httperf) test of CloudI's http_req erlang service using cowboy gives 13358 req/s. cowboy has more features, so that can explain the extra average latency which limits throughput. If you want to understand why their benchmark isn't decent, read this: http://www.mnot.net/blog/2011/05/18/http_benchmark_rules So, if you want something faster in Erlang, you could use ellis, however, keep in mind their testing isn't long enough to be meaningful (due to garbage collection and other impacts on performance). On Tuesday, June 25, 2013 10:38:25 AM UTC-7, BM Kim wrote: > > Damian Dobroczy?ski gmail.com> writes: > > > > > W dniu 25.06.2013 13:51, BM Kim pisze: > > > Hi folks, > > > > > > First of all, I want to apologise for my poor english skills, > > > since english is not my first language, but I'll try my best > > > to formulate my quesions as clear as possible. > > > > > > Second, I've just begun to learn erlang, so if I'm asking > > > obvious "noob" questions I apologise for that too in advance... > > > > > > Anywho, now to my actual question: > > > > > > I am planning to write a high-performance server application in erlang, > > > which will primarily handle HTTP requests. After some reseach with > google, > > > I narrowed down my choices to cowboy, misultin and mochiweb and decided > > > to go with the cowboy library first... > > > > > > Looking at some tutorials, I've quickly built a small server capable of > > > serving static files and was eager to see first benchmark-results... > > > I've also built a small Haskell server using Warp library to compare it > > > with erlang's cowboy... > > > > > > But my first impression was, that my cowboy server is much much slower > than > > > expected when serving static-files and after some research I found a > presentation > > > of the cowboy's author claiming that cowboy shouldn't be used for > serving > > > static-files. So I modified the server code, so that it replies to > every > > > request with in-memory 4Kb binary blob and compared it with my haskell > warp > > > server serving 4kb static file... > > > > > > this is my simple cowboy's http handler: > > > > > > ---------------------------------------------------------------------- > > > > > > blob() -> > > > [<<0:8>> || _ <- lists:seq(1,4096)]. > > > > First, try to replace blob/0 function with this: > > > > blob() -> <<0:(4096*8)>>. > > > > Then, restart the test and report ;) > > > > > Hi, > > Thank you very much for pointing out the obvious mistake... > After correcting it, I got improvement from 5940 req/s to 8650 req/s... > > But still much slower than the haskell warp-server, which has throughput > of 38000 req/s... > > But I have another question regarding blob/0. Is it going to be evaluated > only once (like GHC would do) since it is a pure expression? I'm not > so sure, since erlang is not pure and any function can have side-effects > which you can't mark as with the IO monad in Haskell... > > _______________________________________________ > erlang-questions mailing list > erlang-q...@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Tue Jun 25 22:06:57 2013 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Tue, 25 Jun 2013 22:06:57 +0200 Subject: [erlang-questions] Cowboy (Erlang) VS Haskell (Warp) In-Reply-To: References: <51C98427.6040402@gmail.com> <51C98774.3080009@gmail.com> Message-ID: <51C9F861.8080005@ninenines.eu> You'd also end up with tons of unsupervised processes, which isn't a good thing when you want to see what's wrong in your production system later on (something I last did today and do quite regularly when consulting). That's the biggest difference between the two, Cowboy follows OTP principles, elli doesn't. Unfortunately supervision does have a cost when you need to accept many short-lived processes quickly, but we have done a lot of work in Ranch to reduce that cost as much as possible. You could always replace Ranch with something that doesn't supervise if you really need to optimize to death, but so far even the handful of companies that I know that use it for ad bidding and other high frequency purposes haven't had the need to do that. In short: removing supervision only looks good on benchmarks. On 06/25/2013 09:52 PM, okeuday wrote: > When I use their benchmark here: > https://github.com/yesodweb/benchmarks > (as mentioned here: > http://www.yesodweb.com/blog/2011/03/preliminary-warp-cross-language-benchmarks) > to test elli, here: > https://github.com/knutin/elli > > I get 99000 req/s. The (httperf) test of CloudI's http_req erlang > service using cowboy gives 13358 req/s. cowboy has more features, so > that can explain the extra average latency which limits throughput. > > If you want to understand why their benchmark isn't decent, read this: > http://www.mnot.net/blog/2011/05/18/http_benchmark_rules > > So, if you want something faster in Erlang, you could use ellis, > however, keep in mind their testing isn't long enough to be meaningful > (due to garbage collection and other impacts on performance). > > > On Tuesday, June 25, 2013 10:38:25 AM UTC-7, BM Kim wrote: > > Damian Dobroczy?ski gmail.com > writes: > > > > > W dniu 25.06.2013 13:51, BM Kim pisze: > > > Hi folks, > > > > > > First of all, I want to apologise for my poor english skills, > > > since english is not my first language, but I'll try my best > > > to formulate my quesions as clear as possible. > > > > > > Second, I've just begun to learn erlang, so if I'm asking > > > obvious "noob" questions I apologise for that too in advance... > > > > > > Anywho, now to my actual question: > > > > > > I am planning to write a high-performance server application in > erlang, > > > which will primarily handle HTTP requests. After some reseach with > google, > > > I narrowed down my choices to cowboy, misultin and mochiweb and > decided > > > to go with the cowboy library first... > > > > > > Looking at some tutorials, I've quickly built a small server > capable of > > > serving static files and was eager to see first > benchmark-results... > > > I've also built a small Haskell server using Warp library to > compare it > > > with erlang's cowboy... > > > > > > But my first impression was, that my cowboy server is much much > slower > than > > > expected when serving static-files and after some research I > found a > presentation > > > of the cowboy's author claiming that cowboy shouldn't be used for > serving > > > static-files. So I modified the server code, so that it replies > to every > > > request with in-memory 4Kb binary blob and compared it with my > haskell > warp > > > server serving 4kb static file... > > > > > > this is my simple cowboy's http handler: > > > > > > > ---------------------------------------------------------------------- > > > > > > blob() -> > > > [<<0:8>> || _ <- lists:seq(1,4096)]. > > > > First, try to replace blob/0 function with this: > > > > blob() -> <<0:(4096*8)>>. > > > > Then, restart the test and report ;) > > > > > Hi, > > Thank you very much for pointing out the obvious mistake... > After correcting it, I got improvement from 5940 req/s to 8650 req/s... > > But still much slower than the haskell warp-server, which has throughput > of 38000 req/s... > > But I have another question regarding blob/0. Is it going to be > evaluated > only once (like GHC would do) since it is a pure expression? I'm not > so sure, since erlang is not pure and any function can have side-effects > which you can't mark as with the IO monad in Haskell... > > _______________________________________________ > erlang-questions mailing list > erlang-q...@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From knutin@REDACTED Wed Jun 26 11:23:26 2013 From: knutin@REDACTED (Knut Nesheim) Date: Wed, 26 Jun 2013 11:23:26 +0200 Subject: [erlang-questions] Cowboy (Erlang) VS Haskell (Warp) In-Reply-To: <51C9F861.8080005@ninenines.eu> References: <51C98427.6040402@gmail.com> <51C98774.3080009@gmail.com> <51C9F861.8080005@ninenines.eu> Message-ID: I'm a bit curious to your comments about unsupervised processes in Elli and Elli not following the OTP principles. Why do you think unsupervised request processes is a bad thing? They are short-lived and you cannot restart them in any meaningful way. It is true that Elli does not hook any of the processes it creates into a supervisor. As a user of Elli, you include the Elli "server" inside your supervision tree. This server starts linked acceptor processes which accepts a new connection, then handles the request (and keep-alive) before they die and the Elli server starts a new acceptor. If any of these processes exits abnormally, Elli knows about it. If you wish to find these processes to debug something, you can use elli:get_acceptors(ElliPid) or process_info(ElliPid, links). Mochiweb and Yaws does more or less the same. Knut On Tue, Jun 25, 2013 at 10:06 PM, Lo?c Hoguin wrote: > You'd also end up with tons of unsupervised processes, which isn't a good > thing when you want to see what's wrong in your production system later on > (something I last did today and do quite regularly when consulting). That's > the biggest difference between the two, Cowboy follows OTP principles, elli > doesn't. Unfortunately supervision does have a cost when you need to accept > many short-lived processes quickly, but we have done a lot of work in Ranch > to reduce that cost as much as possible. You could always replace Ranch > with something that doesn't supervise if you really need to optimize to > death, but so far even the handful of companies that I know that use it for > ad bidding and other high frequency purposes haven't had the need to do > that. > > In short: removing supervision only looks good on benchmarks. > > > On 06/25/2013 09:52 PM, okeuday wrote: > >> When I use their benchmark here: >> https://github.com/yesodweb/**benchmarks >> (as mentioned here: >> http://www.yesodweb.com/blog/**2011/03/preliminary-warp-** >> cross-language-benchmarks >> ) >> to test elli, here: >> https://github.com/knutin/elli >> >> I get 99000 req/s. The (httperf) test of CloudI's http_req erlang >> service using cowboy gives 13358 req/s. cowboy has more features, so >> that can explain the extra average latency which limits throughput. >> >> If you want to understand why their benchmark isn't decent, read this: >> http://www.mnot.net/blog/2011/**05/18/http_benchmark_rules >> >> So, if you want something faster in Erlang, you could use ellis, >> however, keep in mind their testing isn't long enough to be meaningful >> (due to garbage collection and other impacts on performance). >> >> >> On Tuesday, June 25, 2013 10:38:25 AM UTC-7, BM Kim wrote: >> >> Damian Dobroczy?ski gmail.com > >> writes: >> >> > >> > W dniu 25.06.2013 13:51, BM Kim pisze: >> > > Hi folks, >> > > >> > > First of all, I want to apologise for my poor english skills, >> > > since english is not my first language, but I'll try my best >> > > to formulate my quesions as clear as possible. >> > > >> > > Second, I've just begun to learn erlang, so if I'm asking >> > > obvious "noob" questions I apologise for that too in advance... >> > > >> > > Anywho, now to my actual question: >> > > >> > > I am planning to write a high-performance server application in >> erlang, >> > > which will primarily handle HTTP requests. After some reseach >> with >> google, >> > > I narrowed down my choices to cowboy, misultin and mochiweb and >> decided >> > > to go with the cowboy library first... >> > > >> > > Looking at some tutorials, I've quickly built a small server >> capable of >> > > serving static files and was eager to see first >> benchmark-results... >> > > I've also built a small Haskell server using Warp library to >> compare it >> > > with erlang's cowboy... >> > > >> > > But my first impression was, that my cowboy server is much much >> slower >> than >> > > expected when serving static-files and after some research I >> found a >> presentation >> > > of the cowboy's author claiming that cowboy shouldn't be used for >> serving >> > > static-files. So I modified the server code, so that it replies >> to every >> > > request with in-memory 4Kb binary blob and compared it with my >> haskell >> warp >> > > server serving 4kb static file... >> > > >> > > this is my simple cowboy's http handler: >> > > >> > > >> ------------------------------**------------------------------** >> ---------- >> > > >> > > blob() -> >> > > [<<0:8>> || _ <- lists:seq(1,4096)]. >> > >> > First, try to replace blob/0 function with this: >> > >> > blob() -> <<0:(4096*8)>>. >> > >> > Then, restart the test and report ;) >> > >> >> >> Hi, >> >> Thank you very much for pointing out the obvious mistake... >> After correcting it, I got improvement from 5940 req/s to 8650 >> req/s... >> >> But still much slower than the haskell warp-server, which has >> throughput >> of 38000 req/s... >> >> But I have another question regarding blob/0. Is it going to be >> evaluated >> only once (like GHC would do) since it is a pure expression? I'm not >> so sure, since erlang is not pure and any function can have >> side-effects >> which you can't mark as with the IO monad in Haskell... >> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-q...@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> >> > >> >> >> >> >> ______________________________**_________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/**listinfo/erlang-questions >> >> > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > ______________________________**_________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/**listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed Jun 26 14:48:19 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 26 Jun 2013 14:48:19 +0200 Subject: [erlang-questions] Un-identifiable SSL issue when upgrading from R 15B-03 to 16A, 16B, 16B-01 In-Reply-To: <20130625173427.GA5630@precise.local> References: <51C9ADEE.1070401@ninenines.eu> <20130625173427.GA5630@precise.local> Message-ID: On Tue, Jun 25, 2013 at 7:34 PM, Motiejus Jak?tys wrote: > Can you elaborate on that? For me packages from debian experimental seem > to work pretty OK, for more than two years, on many environments. > > If there are concrete issues, they could be addressed IMO. > Two ways in which they are horribly broken: 1. They split up the OTP release into different parts. For instance, you need erlang-crypto in order to get the crypto application. It is not installed by default. Usually this leads to poor newbie users asking why they can't do crypto:start() to test their SSL stuff and so on. It would have been easier just to package everything into a single package. 2. erl -man lists won't work since man-pages are installed in odd places, or are not installed at all. 3. In older Ubuntu versions, there were symlinks which interfered with release generation so you could not generate a release. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ivan@REDACTED Wed Jun 26 16:19:37 2013 From: ivan@REDACTED (Ivan Uemlianin) Date: Wed, 26 Jun 2013 15:19:37 +0100 Subject: [erlang-questions] Erlang and/or MPI Message-ID: <51CAF879.1080700@llaisdy.com> Dear All I am being given access to an HPC (High Performance Computing) cluster. Some technical details are here [1] but basically it seems to be lots of multi-core (12, 16, 64) computers glued together by something called Infiniband. I have ideas of using the platform for speech recognition: processing lots of data files, and training up large hidden Markov models or dynamic Bayesian networks. Much of the standard open-source software in this area is in C (e.g., Sphinx, Kaldi). I do have long-term pipe-dreams/fantasies of writing similar stuff in erlang. Induction to the HPC Wales platform includes classes on MPI and OpenMP. So, my question is: what is the best (or, "are the good") ways to use erlang in such an environment? I've looked around on the web and found a lot of heated debate, and one "benchmark". One 2011 blog post [2] is informative, and there is a relevant-looking 2008 research project [3]. Are there special best practices for working on a cluster of multi-core computers, or is it a case of "write idiomatic erlang and everything will be fine"? With thanks and best wishes Ivan [1] http://hpcwales.co.uk/sites/default/files/hpcwales/HPCWales_technical_specification_0512.pdf [2] http://jlouisramblings.blogspot.co.uk/2011/07/erlangs-parallelism-is-not-parallelism.html [3] http://www.researchgate.net/publication/221211398_High-performance_technical_computing_with_erlang -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From andra.dinu@REDACTED Wed Jun 26 16:32:59 2013 From: andra.dinu@REDACTED (Andra Dinu) Date: Wed, 26 Jun 2013 15:32:59 +0100 (BST) Subject: [erlang-questions] Talk submissions open for Berlin Erlang Factory Lite on 16 October Message-ID: <2115118894.14104713.1372257179263.JavaMail.root@erlang-solutions.com> After the success in Munich, the Erlang Factory Lite returns to Germany for the second time this year on 16 October. If you have an idea for a talk, do not hesitate to share it with the world. Talk submission is open until 15 August. Submit early to avoid dissapointment. http://www.erlang-factory.com/conference/Berlin2013 Thanks, Andra -- Andra Dinu Community & Social ERLANG SOLUTIONS LTD New Loom House 101 Back Church Lane London, E1 1LU United Kingdom Tel +44(0)2076550344 Mob +44(0)7983484387 www.erlang-solutions.com From mjtruog@REDACTED Wed Jun 26 16:36:51 2013 From: mjtruog@REDACTED (Michael Truog) Date: Wed, 26 Jun 2013 07:36:51 -0700 Subject: [erlang-questions] Erlang and/or MPI In-Reply-To: <51CAF879.1080700@llaisdy.com> References: <51CAF879.1080700@llaisdy.com> Message-ID: <51CAFC83.7080402@gmail.com> Hi Ivan, You should need an Infiniband driver for Erlang to avoid normal sockets use, unless you use MPI from within Erlang through a NIF or port driver. The main problem is that an Erlang cluster is limited to roughly 100-200 nodes because of the fully connected network topology it makes (due to the net tick time of 60 seconds). You can always make a different topology to avoid the problem, but with your cluster size, you may not need to. Otherwise normal Erlang use shouldn't be a problem. Best Regards, Michael On 06/26/2013 07:19 AM, Ivan Uemlianin wrote: > Dear All > > I am being given access to an HPC (High Performance Computing) cluster. Some technical details are here [1] but basically it seems to be lots of multi-core (12, 16, 64) computers glued together by something called Infiniband. > > I have ideas of using the platform for speech recognition: processing lots of data files, and training up large hidden Markov models or dynamic Bayesian networks. Much of the standard open-source software in this area is in C (e.g., Sphinx, Kaldi). I do have long-term pipe-dreams/fantasies of writing similar stuff in erlang. > > Induction to the HPC Wales platform includes classes on MPI and OpenMP. > > So, my question is: what is the best (or, "are the good") ways to use erlang in such an environment? > > I've looked around on the web and found a lot of heated debate, and one "benchmark". One 2011 blog post [2] is informative, and there is a relevant-looking 2008 research project [3]. > > Are there special best practices for working on a cluster of multi-core computers, or is it a case of "write idiomatic erlang and everything will be fine"? > > With thanks and best wishes > > Ivan > > > [1] http://hpcwales.co.uk/sites/default/files/hpcwales/HPCWales_technical_specification_0512.pdf > > [2] http://jlouisramblings.blogspot.co.uk/2011/07/erlangs-parallelism-is-not-parallelism.html > > [3] http://www.researchgate.net/publication/221211398_High-performance_technical_computing_with_erlang > > From juanjo@REDACTED Wed Jun 26 16:38:53 2013 From: juanjo@REDACTED (Juan Jose Comellas) Date: Wed, 26 Jun 2013 11:38:53 -0300 Subject: [erlang-questions] Un-identifiable SSL issue when upgrading from R 15B-03 to 16A, 16B, 16B-01 In-Reply-To: References: <51C9ADEE.1070401@ninenines.eu> <20130625173427.GA5630@precise.local> Message-ID: The Erlang packages that come with Ubuntu are normally outdated and at one time had the debug symbols removed, but the Debian packages are normally of a very good quality and have never exhibited any of the problems you mentioned. As I use Ubuntu, what I normally do is build the Debian packages on the different Ubuntu versions that I use (i.e. latest LTS and latest release). At one time Alessandro Sivieri was maintaining a PPA ( https://launchpad.net/~scattino/+archive/ppa) with the latest Erlang packages there, but it seems he stopped doing this a while ago. If there is enough interest I could start maintaining something similar to that. On Wed, Jun 26, 2013 at 9:48 AM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > > On Tue, Jun 25, 2013 at 7:34 PM, Motiejus Jak?tys wrote: > >> Can you elaborate on that? For me packages from debian experimental seem >> to work pretty OK, for more than two years, on many environments. >> >> If there are concrete issues, they could be addressed IMO. >> > > Two ways in which they are horribly broken: > > 1. They split up the OTP release into different parts. For instance, you > need erlang-crypto in order to get the crypto application. It is not > installed by default. Usually this leads to poor newbie users asking why > they can't do crypto:start() to test their SSL stuff and so on. It would > have been easier just to package everything into a single package. > > 2. erl -man lists won't work since man-pages are installed in odd places, > or are not installed at all. > > 3. In older Ubuntu versions, there were symlinks which interfered with > release generation so you could not generate a release. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Wed Jun 26 16:48:36 2013 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Wed, 26 Jun 2013 16:48:36 +0200 Subject: [erlang-questions] Cowboy (Erlang) VS Haskell (Warp) In-Reply-To: References: <51C98427.6040402@gmail.com> <51C98774.3080009@gmail.com> <51C9F861.8080005@ninenines.eu> Message-ID: <51CAFF44.6080501@ninenines.eu> They are not short lived. Spawning unsupervised processes is only fine if you spawn the process to do exactly one thing and that thing cannot get the process stuck (otherwise you got an invisible leak). And you say it yourself, your processes do keep-alive. So they can be open for quite a while. Now if a request handler gets stuck for one reason or another, and the process is supervised, it takes about 5 minutes to find what's wrong because the OTP tools and libraries do all the work for you in figuring it out. You don't have to think, you already know how to deal with issues. If the process isn't supervised, you cut yourself from most of the documented tools and techniques, for example none of your processes will show up in observer. You also force your users to figure out the alternative ways to debug things instead of what they use everywhere else (see Fred's posts [1] and [2]). You have to write custom code to extract metrics, logging will work in unexpected ways, etc. And the worse part is that you essentially reimplemented a supervisor in your Elli server process. Yesterday I was debugging an issue and found out file:consult/1 calls were getting stuck (bug report incoming, still testing things). If I was using Elli, I would have cursed your name many times because of all the time I'd have lost trying to understand how to debug this thing. Following OTP principles is one of Cowboy's most important feature, and this will soon be improved again by making even request processes special processes that you can debug like you would a gen_server. At this point everything will be a special process. This is especially important since the Web continues moving toward long-running connections with Websocket, SPDY and HTTP/2.0. [1] http://ferd.ca/poll-results-erlang-maintenance.html [2] http://ferd.ca/code-janitor-nobody-s-dream-everyone-s-job-and-how-erlang-can-help.html On 06/26/2013 11:23 AM, Knut Nesheim wrote: > I'm a bit curious to your comments about unsupervised processes in Elli > and Elli not following the OTP principles. Why do you think unsupervised > request processes is a bad thing? They are short-lived and you cannot > restart them in any meaningful way. > > It is true that Elli does not hook any of the processes it creates into > a supervisor. As a user of Elli, you include the Elli "server" inside > your supervision tree. This server starts linked acceptor processes > which accepts a new connection, then handles the request (and > keep-alive) before they die and the Elli server starts a new acceptor. > If any of these processes exits abnormally, Elli knows about it. If you > wish to find these processes to debug something, you can use > elli:get_acceptors(ElliPid) or process_info(ElliPid, links). Mochiweb > and Yaws does more or less the same. > > Knut > > > On Tue, Jun 25, 2013 at 10:06 PM, Lo?c Hoguin > wrote: > > You'd also end up with tons of unsupervised processes, which isn't a > good thing when you want to see what's wrong in your production > system later on (something I last did today and do quite regularly > when consulting). That's the biggest difference between the two, > Cowboy follows OTP principles, elli doesn't. Unfortunately > supervision does have a cost when you need to accept many > short-lived processes quickly, but we have done a lot of work in > Ranch to reduce that cost as much as possible. You could always > replace Ranch with something that doesn't supervise if you really > need to optimize to death, but so far even the handful of companies > that I know that use it for ad bidding and other high frequency > purposes haven't had the need to do that. > > In short: removing supervision only looks good on benchmarks. > > > On 06/25/2013 09:52 PM, okeuday wrote: > > When I use their benchmark here: > https://github.com/yesodweb/__benchmarks > > (as mentioned here: > http://www.yesodweb.com/blog/__2011/03/preliminary-warp-__cross-language-benchmarks > ) > to test elli, here: > https://github.com/knutin/elli > > I get 99000 req/s. The (httperf) test of CloudI's http_req erlang > service using cowboy gives 13358 req/s. cowboy has more > features, so > that can explain the extra average latency which limits throughput. > > If you want to understand why their benchmark isn't decent, read > this: > http://www.mnot.net/blog/2011/__05/18/http_benchmark_rules > > > So, if you want something faster in Erlang, you could use ellis, > however, keep in mind their testing isn't long enough to be > meaningful > (due to garbage collection and other impacts on performance). > > > On Tuesday, June 25, 2013 10:38:25 AM UTC-7, BM Kim wrote: > > Damian Dobroczy?ski gmail.com > > writes: > > > > > W dniu 25.06.2013 13 :51, BM Kim pisze: > > > Hi folks, > > > > > > First of all, I want to apologise for my poor english > skills, > > > since english is not my first language, but I'll try > my best > > > to formulate my quesions as clear as possible. > > > > > > Second, I've just begun to learn erlang, so if I'm asking > > > obvious "noob" questions I apologise for that too in > advance... > > > > > > Anywho, now to my actual question: > > > > > > I am planning to write a high-performance server > application in > erlang, > > > which will primarily handle HTTP requests. After some > reseach with > google, > > > I narrowed down my choices to cowboy, misultin and > mochiweb and > decided > > > to go with the cowboy library first... > > > > > > Looking at some tutorials, I've quickly built a small > server > capable of > > > serving static files and was eager to see first > benchmark-results... > > > I've also built a small Haskell server using Warp > library to > compare it > > > with erlang's cowboy... > > > > > > But my first impression was, that my cowboy server is > much much > slower > than > > > expected when serving static-files and after some > research I > found a > presentation > > > of the cowboy's author claiming that cowboy shouldn't > be used for > serving > > > static-files. So I modified the server code, so that > it replies > to every > > > request with in-memory 4Kb binary blob and compared it > with my > haskell > warp > > > server serving 4kb static file... > > > > > > this is my simple cowboy's http handler: > > > > > > > > ------------------------------__------------------------------__---------- > > > > > > blob() -> > > > [<<0:8>> || _ <- lists:seq(1,4096)]. > > > > First, try to replace blob/0 function with this: > > > > blob() -> <<0:(4096*8)>>. > > > > Then, restart the test and report ;) > > > > > Hi, > > Thank you very much for pointing out the obvious mistake... > After correcting it, I got improvement from 5940 req/s to > 8650 req/s... > > But still much slower than the haskell warp-server, which > has throughput > of 38000 req/s... > > But I have another question regarding blob/0. Is it going to be > evaluated > only once (like GHC would do) since it is a pure > expression? I'm not > so sure, since erlang is not pure and any function can have > side-effects > which you can't mark as with the IO monad in Haskell... > > _________________________________________________ > erlang-questions mailing list > erlang-q...@REDACTED > http://erlang.org/mailman/__listinfo/erlang-questions > > > > > > > > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/__listinfo/erlang-questions > > > > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/__listinfo/erlang-questions > > > -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From jesper.louis.andersen@REDACTED Wed Jun 26 19:20:18 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 26 Jun 2013 19:20:18 +0200 Subject: [erlang-questions] Erlang and/or MPI In-Reply-To: <51CAF879.1080700@llaisdy.com> References: <51CAF879.1080700@llaisdy.com> Message-ID: On Wed, Jun 26, 2013 at 4:19 PM, Ivan Uemlianin wrote: > http://jlouisramblings.**blogspot.co.uk/2011/07/** > erlangs-parallelism-is-not-**parallelism.html > As the author of that blog post, I would caution you to use Erlang for this kind of work. First, it is the kind of work where you have a small computational kernel which has to run as fast as possible. This is exactly the kind of problems where Erlang usually does not shine, due to the language being interpreted and not focused on fast floating point calculations. Erlang would be good as a higher-level coordinator in such systems, but the low-level code is probably better written in a compiled language with good FP performance: C, Common Lisp, Haskell, Ocaml, and so on. Second, you would need to get to Infiniband from within Erlang. MPI and OpenMP already provides the tooling for this, so are you out to solve a problem, or are you curious and want to do research on how to make Erlang shine in this situation? NIFs are hard to pull off because you need to cooperate with the Erlang schedulers. Third, the problems for which Erlang is best is different from your problem: providing low-latency stable operation even in the advent of spurious errors. And isolating bad code from good code [1]. [1] Aside: This is a strength of Erlang. A badly written process is usually not able to take down the node as a whole, due to memory space isolation. In large shared-memory-style programs, one bad library can easily sabotage the data on which another part of the program operates. -------------- next part -------------- An HTML attachment was scrubbed... URL: From desired.mta@REDACTED Wed Jun 26 19:39:25 2013 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Wed, 26 Jun 2013 18:39:25 +0100 Subject: [erlang-questions] Un-identifiable SSL issue when upgrading from R 15B-03 to 16A, 16B, 16B-01 In-Reply-To: References: <51C9ADEE.1070401@ninenines.eu> <20130625173427.GA5630@precise.local> Message-ID: On Wed, Jun 26, 2013 at 1:48 PM, Jesper Louis Andersen wrote: > > On Tue, Jun 25, 2013 at 7:34 PM, Motiejus Jak?tys > wrote: >> >> Can you elaborate on that? For me packages from debian experimental seem >> to work pretty OK, for more than two years, on many environments. >> >> If there are concrete issues, they could be addressed IMO. > > > Two ways in which they are horribly broken: > > 1. They split up the OTP release into different parts. For instance, you > need erlang-crypto in order to get the crypto application. It is not > installed by default. Usually this leads to poor newbie users asking why > they can't do crypto:start() to test their SSL stuff and so on. It would > have been easier just to package everything into a single package. Everyone should install 'erlang', which depends on erlang-crypto. Not `erlang-base` or anything like that. `apt-get install erlang`. > 2. erl -man lists won't work since man-pages are installed in odd places, or > are not installed at all. Always works/worked for me. In fact, this is one of the main reason that man pages work out of the box only when I use the debian package. With kerl you have to set paths; I do not allow myself using plain sudo make install. Contrary, with esl-erlang, check out `erl -man file`. The last thing you will get is file(3erl) (last I tried). I figured out that this is erlang-mode fault, it placed a symlink in some weird place. I am not emacs fan, so did not take time to report it. Standard apt-get install erlang && apt-get purge erlang-mode makes it perfect. > 3. In older Ubuntu versions, there were symlinks which interfered with > release generation so you could not generate a release. Never tried anything older than R14B02; did not have any issues with everything after that. However, like Juan correctly pointed out, packages are dated. -- Motiejus Jak?tys From desired.mta@REDACTED Wed Jun 26 23:33:58 2013 From: desired.mta@REDACTED (Motiejus =?utf-8?Q?Jak=C5=A1tys?=) Date: Wed, 26 Jun 2013 22:33:58 +0100 Subject: [erlang-questions] Erlang packaging (was: Un-identifiable SSL issue when upgrading from) R 15B-03 to 16A, 16B, 16B-01 In-Reply-To: References: <51C9ADEE.1070401@ninenines.eu> <20130625173427.GA5630@precise.local> Message-ID: <20130626213358.GA4642@precise.local> On Wed, Jun 26, 2013 at 06:39:25PM +0100, Motiejus Jak?tys wrote: > On Wed, Jun 26, 2013 at 1:48 PM, Jesper Louis Andersen > wrote: > > > > On Tue, Jun 25, 2013 at 7:34 PM, Motiejus Jak?tys > > wrote: > >> > >> Can you elaborate on that? For me packages from debian experimental seem > >> to work pretty OK, for more than two years, on many environments. > >> > >> If there are concrete issues, they could be addressed IMO. > > > > > > Two ways in which they are horribly broken: > > > > 1. They split up the OTP release into different parts. For instance, you > > need erlang-crypto in order to get the crypto application. It is not > > installed by default. Usually this leads to poor newbie users asking why > > they can't do crypto:start() to test their SSL stuff and so on. It would > > have been easier just to package everything into a single package. > > Everyone should install 'erlang', which depends on erlang-crypto. Not > `erlang-base` or anything like that. `apt-get install erlang`. In fact, this is one of the things I really like about Erlang packaging. It is very easy to get full erlang installation by installing erlang. erlang-nox on headless servers (no wx, gs etc). Or pick erlang-base + what-ever-you-need. Sure, if user installs something like couchdb, which installs subset of erlang, that will not work. And shouldn't. OpenWRT follows the same principle: you can install base package which has net_kernel, erts and some more basic stuff in about ~2 MB (when I tried), and then add anything you like/need. I played with mnesia distributed mode on my laptop and home router (tp-link with 8 megs of flash and 16 megs of ram). Installation time? 15 minutes. 14 of which took to figure out how easy it is (and possible), and 1 minute for installation. Of course, for "real" systems you would cross-compile it, generate a release and then go crazy optimizing it. However, something between one-click-desktop-installation (easy!) and manual cross-compilation (advanced!) is very useful and convenient in some circumstances. For instance, showing off master-master DB replication between a crappy home router and a laptop on Friday night in a CS dorm. Motiejus From tcury@REDACTED Wed Jun 26 23:44:37 2013 From: tcury@REDACTED (Tiago Cury) Date: Wed, 26 Jun 2013 18:44:37 -0300 Subject: [erlang-questions] erl_tar improvements Message-ID: Ability to enable/disable directory recursion through option for erl_tar:add and erl_tar:extract. From max.lapshin@REDACTED Wed Jun 26 23:52:25 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 27 Jun 2013 01:52:25 +0400 Subject: [erlang-questions] erl_tar improvements In-Reply-To: References: Message-ID: Tiago, I've just opened 30 minutes ago this file and was going to add this feature. Have you already prepared patch or I should do it? From tcury@REDACTED Thu Jun 27 00:00:32 2013 From: tcury@REDACTED (Tiago Cury) Date: Wed, 26 Jun 2013 19:00:32 -0300 Subject: [erlang-questions] erl_tar improvements In-Reply-To: References: Message-ID: <8E8DEA8C-9EE4-4B5A-B014-63DF72C2B975@ymail.com> LOL. I did some workaround in my APP, not in erl_tar source. Please go forward. I saw that erl_tar:add always enter over directories adding it's files and you cannot disable it. One or more non regular files (sockets and pipes) can abort the entire process. Too, I saw that erl_tar:extract with a directory inside the FileList option extract just that directory. Should be better if there is an option to extract that directory and it's files like gnu tar. On Jun 26, 2013, at 6:52 PM, Max Lapshin wrote: > Tiago, I've just opened 30 minutes ago this file and was going to add > this feature. > > Have you already prepared patch or I should do it? From max.lapshin@REDACTED Thu Jun 27 00:24:12 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Thu, 27 Jun 2013 02:24:12 +0400 Subject: [erlang-questions] erl_tar improvements In-Reply-To: <8E8DEA8C-9EE4-4B5A-B014-63DF72C2B975@ymail.com> References: <8E8DEA8C-9EE4-4B5A-B014-63DF72C2B975@ymail.com> Message-ID: I have met another problem with recursive directories. I've written a tool that creates debian packages and wanted to use erl_tar. dpkg requires that directories must exist in tar archive and erl_tar cannot add only directory, it starts adding files inside and forget to add inner directories. So, this archive becomes unuseable for dpkg. I think that I need to make a patch and try to push it to mainline. From tcury@REDACTED Thu Jun 27 00:58:16 2013 From: tcury@REDACTED (Tiago Cury) Date: Wed, 26 Jun 2013 19:58:16 -0300 Subject: [erlang-questions] erl_tar improvements In-Reply-To: References: <8E8DEA8C-9EE4-4B5A-B014-63DF72C2B975@ymail.com> Message-ID: <2A856D55-5228-455E-B976-BD27B19F70B3@ymail.com> Sure, can we clone the erlang source and send pull requests? If yes, I think that we can work on it together! On Jun 26, 2013, at 7:24 PM, Max Lapshin wrote: > I have met another problem with recursive directories. > > I've written a tool that creates debian packages and wanted to use > erl_tar. dpkg requires that directories must exist in tar archive and > erl_tar cannot add only directory, it starts adding files inside and > forget to add inner directories. > So, this archive becomes unuseable for dpkg. > > I think that I need to make a patch and try to push it to mainline. From ok@REDACTED Thu Jun 27 03:26:52 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 27 Jun 2013 13:26:52 +1200 Subject: [erlang-questions] Erlang and/or MPI In-Reply-To: References: <51CAF879.1080700@llaisdy.com> Message-ID: On 27/06/2013, at 5:20 AM, Jesper Louis Andersen wrote: > > On Wed, Jun 26, 2013 at 4:19 PM, Ivan Uemlianin wrote: > http://jlouisramblings.blogspot.co.uk/2011/07/erlangs-parallelism-is-not-parallelism.html > > First, it is the kind of work where you have a small computational kernel which has to run as fast as possible. This is exactly the kind of problems where Erlang usually does not shine, due to the language being interpreted and not focused on fast floating point calculations. Except for commands typed at the shell, Erlang is compiled, not interpreted. And there is a native code generator; c(File, [native]) does the trick if native code generation was enabled at build time. And the native code generator exploits -spec declarations (and some type inference) to avoid boxing floats some of the time. So Erlang might well make good sense as a language for prototyping an application, even a floating-point one, to the point where you can do some benchmarking and find out where the time is really going. > Erlang would be good as a higher-level coordinator in such systems, but the low-level code is probably better written in a compiled language with good FP performance: C, Common Lisp, Haskell, Ocaml, and so on. My experience has been that Fortran is *still* the King if you want serious floating-point performance, and modern Fortran is a serious modular programming language. GNU Fortran has _nearly_ caught up with the 2003 standard, which saw some serious additions to the language. Coarrays are part of the 2008 standard, and gfortran support for them is still incomplete, which is a pity, because they look extremely useful for this kind of problem. I haven't personally used the Intel Fortran compiler, but I've used the Intel C compiler, and Intel's Parallel Studio _looks_ impressive. I see the HPCWales systems have the Intel compilers... What's often an issue in floating-point calculations is algorithm design, and good facilities for managing abstractions may be more important than raw floating-point speed. Haskell is pretty good here, and there are some interesting libraries, like the beginnings of support for Geometric Algebra. There's certainly an HMM library (version 0.04, so be warned). > Second, you would need to get to Infiniband from within Erlang. http://erlang.org/pipermail/erlang-questions/2009-July/045299.html There's another idea which has often been applied in Haskell, and that's writing a program in a high level language that _generates_ low level code. From hc@REDACTED Thu Jun 27 07:39:15 2013 From: hc@REDACTED (H.C. v. Stockhausen) Date: Thu, 27 Jun 2013 07:39:15 +0200 Subject: [erlang-questions] ssl {active, once} {packet, 4} Message-ID: Hi, this is my first post to this group. Hello everyone. I am writing a client that connects to a server via ssl:connect. Upon connection the server returns a . Thereafter the client sends commands. The protocol has a 4 byte header that specifies the payload length followed by the xml payload. Here's an extract of my initial attempt that works: %%%%%%%%%%%%%%%%%%%%%%%% ... -behaviour(gen_server). -define(HEADER_SIZE, 4). ... init([]) -> {ok, Socket} = ssl:connect( , , [ binary, {active, false}, {keepalive, true}, {certfile, }, {keyfile, } ]), {ok, #state{socket=Socket}, 0}. ... handle_info(timeout, State) -> {ok, _Greeting} = recv(State#state.socket), {noreply, State}. ... % internal helpers send(Socket, Msg) -> Length = string:len(Msg) + ?HEADER_SIZE, Data = <>, ok = ssl:send(Socket, Data), ok. recv(Socket) -> {ok, <>} = ssl:recv(Socket, ?HEADER_SIZE), {ok, Data} = ssl:recv(Socket, Length-?HEADER_SIZE), Response = binary_to_list(Data), {ok, Response}. %%%%%%%%%%%%%%%%%%%%%%%% I have two questions. 1) As you can see init()/1 has a 0 timeout to trigger handle_info(timeout, State) which then fetches and discard the greeting message. Shouldn't I be able to just set {active, once} instead and handle the initial server response in a matching handle_info clause? I tried but it's never triggered. All I receive after a while is {ssl_closed,{sslsocket,new_ssl,<0.65.0>}}. 2) I manually build and unwrap the header/payload but isn't that what {packet, 4} is for? I also tried that but the server does not respond, as incidentally was the case when I initially didn't set the header properly. I realise that there are a lot of list_to_binary_to_list_to... conversions but that's another issue, isn't it? Is this how {active, once} and {packet, 4} should work or am I missing something, such as differences when it comes to ssl or client vs. server behaviour? Best regards, Hans Christian From zerthurd@REDACTED Thu Jun 27 08:47:03 2013 From: zerthurd@REDACTED (Maxim Treskin) Date: Thu, 27 Jun 2013 13:47:03 +0700 Subject: [erlang-questions] Erlang blog platform Message-ID: Hello Could you please recommend any Erlang blog platform? Years ago I wrote Metalkia: https://github.com/metachord/metalkia_web (see rebar deps) using Nitrogen and Riak but now I see it should be totally rewriten to use PostgreSQL or MySQL. Now http://erlang-russian.org based on this platform. May be someone wrote best blog platform with commentaries in Erlang? -- Max Treskin -------------- next part -------------- An HTML attachment was scrubbed... URL: From yashgt@REDACTED Thu Jun 27 09:38:30 2013 From: yashgt@REDACTED (Yash Ganthe) Date: Thu, 27 Jun 2013 13:08:30 +0530 Subject: [erlang-questions] (no subject) Message-ID: ---------- Forwarded message ---------- From: Yash Ganthe Date: Thu, Jun 27, 2013 at 12:54 PM Subject: Running multiple instances of same rebar packaged node To: "?? Ganthe (yashgt@REDACTED)" ** ** rebar@REDACTED,erlang-questions@REDACTED**** ** ** Hi,**** ** ** A node packaged using rebar to run as a standalone application uses a file named vm.args in which the name of the node is specified. When the node is instantiated on command prompt, the runner script reads the name of the node from vm.args and uses it for starting the node.**** ** ** If I have to start multiple instances of the same node on the same server, there is a problem because the name of the node clashes with previously invoke node.**** ** ** Is there a way around it?**** ** ** Thanks,**** Yash**** DISCLAIMER ========== This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dmkolesnikov@REDACTED Thu Jun 27 10:09:58 2013 From: dmkolesnikov@REDACTED (Dmitry Kolesnikov) Date: Thu, 27 Jun 2013 11:09:58 +0300 Subject: [erlang-questions] (no subject) In-Reply-To: References: Message-ID: <9E8351E7-DB28-4611-835C-08ADAD94B697@gmail.com> Hello, Yes multiple options exists: 1) use vars.config to parametrize releases but you need to assemble multiple release (tarball) 2) patch bootstrap code for you needs Best Regards, Dmitry On Jun 27, 2013, at 10:38 AM, Yash Ganthe wrote: > > > ---------- Forwarded message ---------- > From: Yash Ganthe > Date: Thu, Jun 27, 2013 at 12:54 PM > Subject: Running multiple instances of same rebar packaged node > To: "?? Ganthe (yashgt@REDACTED)" > > > > > rebar@REDACTED,erlang-questions@REDACTED > > > > Hi, > > > > A node packaged using rebar to run as a standalone application uses a file named vm.args in which the name of the node is specified. When the node is instantiated on command prompt, the runner script reads the name of the node from vm.args and uses it for starting the node. > > > > If I have to start multiple instances of the same node on the same server, there is a problem because the name of the node clashes with previously invoke node. > > > > Is there a way around it? > > > > Thanks, > > Yash > > DISCLAIMER ========== This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From knutin@REDACTED Thu Jun 27 11:17:00 2013 From: knutin@REDACTED (Knut Nesheim) Date: Thu, 27 Jun 2013 11:17:00 +0200 Subject: [erlang-questions] Cowboy (Erlang) VS Haskell (Warp) In-Reply-To: <51CAFF44.6080501@ninenines.eu> References: <51C98427.6040402@gmail.com> <51C98774.3080009@gmail.com> <51C9F861.8080005@ninenines.eu> <51CAFF44.6080501@ninenines.eu> Message-ID: I agree that having a web request process be "debuggable" using the sys module can be quite useful. So far no users of Elli has told me of such a wish. For myself and the users of Elli I know of, process_info/2 has been very helpful in debugging the web request processes. If you want a webserver that strictly follows the OTP principles, maybe you should use Cowboy. There are both pros and cons with doing so and as is pretty obvious, people have different views of the tradeoffs. Some might think it is an absolute requirement to strictly follow the OTP principles, others might think there are cases when it doesn't make much sense. Elli is not meant to be everything to everybody. If you find it hard to debug Elli, cursing my name is misplacing your anger and won't help you achieve anything. I happily help anybody who comes to me with questions. Knut On Wed, Jun 26, 2013 at 4:48 PM, Lo?c Hoguin wrote: > They are not short lived. Spawning unsupervised processes is only fine if > you spawn the process to do exactly one thing and that thing cannot get the > process stuck (otherwise you got an invisible leak). And you say it > yourself, your processes do keep-alive. So they can be open for quite a > while. > > Now if a request handler gets stuck for one reason or another, and the > process is supervised, it takes about 5 minutes to find what's wrong > because the OTP tools and libraries do all the work for you in figuring it > out. You don't have to think, you already know how to deal with issues. > > If the process isn't supervised, you cut yourself from most of the > documented tools and techniques, for example none of your processes will > show up in observer. You also force your users to figure out the > alternative ways to debug things instead of what they use everywhere else > (see Fred's posts [1] and [2]). You have to write custom code to extract > metrics, logging will work in unexpected ways, etc. And the worse part is > that you essentially reimplemented a supervisor in your Elli server process. > > Yesterday I was debugging an issue and found out file:consult/1 calls were > getting stuck (bug report incoming, still testing things). If I was using > Elli, I would have cursed your name many times because of all the time I'd > have lost trying to understand how to debug this thing. > > Following OTP principles is one of Cowboy's most important feature, and > this will soon be improved again by making even request processes special > processes that you can debug like you would a gen_server. At this point > everything will be a special process. This is especially important since > the Web continues moving toward long-running connections with Websocket, > SPDY and HTTP/2.0. > > [1] http://ferd.ca/poll-results-**erlang-maintenance.html > [2] http://ferd.ca/code-janitor-**nobody-s-dream-everyone-s-job-** > and-how-erlang-can-help.html > > > On 06/26/2013 11:23 AM, Knut Nesheim wrote: > >> I'm a bit curious to your comments about unsupervised processes in Elli >> and Elli not following the OTP principles. Why do you think unsupervised >> request processes is a bad thing? They are short-lived and you cannot >> restart them in any meaningful way. >> >> It is true that Elli does not hook any of the processes it creates into >> a supervisor. As a user of Elli, you include the Elli "server" inside >> your supervision tree. This server starts linked acceptor processes >> which accepts a new connection, then handles the request (and >> keep-alive) before they die and the Elli server starts a new acceptor. >> If any of these processes exits abnormally, Elli knows about it. If you >> wish to find these processes to debug something, you can use >> elli:get_acceptors(ElliPid) or process_info(ElliPid, links). Mochiweb >> and Yaws does more or less the same. >> >> Knut >> >> >> On Tue, Jun 25, 2013 at 10:06 PM, Lo?c Hoguin > > wrote: >> >> You'd also end up with tons of unsupervised processes, which isn't a >> good thing when you want to see what's wrong in your production >> system later on (something I last did today and do quite regularly >> when consulting). That's the biggest difference between the two, >> Cowboy follows OTP principles, elli doesn't. Unfortunately >> supervision does have a cost when you need to accept many >> short-lived processes quickly, but we have done a lot of work in >> Ranch to reduce that cost as much as possible. You could always >> replace Ranch with something that doesn't supervise if you really >> need to optimize to death, but so far even the handful of companies >> that I know that use it for ad bidding and other high frequency >> purposes haven't had the need to do that. >> >> In short: removing supervision only looks good on benchmarks. >> >> >> On 06/25/2013 09:52 PM, okeuday wrote: >> >> When I use their benchmark here: >> https://github.com/yesodweb/__**benchmarks >> >> >> > >> (as mentioned here: >> http://www.yesodweb.com/blog/_**_2011/03/preliminary-warp-__** >> cross-language-benchmarks >> > cross-language-benchmarks >> >) >> >> to test elli, here: >> https://github.com/knutin/elli >> >> I get 99000 req/s. The (httperf) test of CloudI's http_req erlang >> service using cowboy gives 13358 req/s. cowboy has more >> features, so >> that can explain the extra average latency which limits >> throughput. >> >> If you want to understand why their benchmark isn't decent, read >> this: >> http://www.mnot.net/blog/2011/**__05/18/http_benchmark_rules >> >> >> > >> >> So, if you want something faster in Erlang, you could use ellis, >> however, keep in mind their testing isn't long enough to be >> meaningful >> (due to garbage collection and other impacts on performance). >> >> >> On Tuesday, June 25, 2013 10:38:25 AM UTC-7, BM Kim wrote: >> >> Damian Dobroczy?ski gmail.com >> > writes: >> >> > >> > W dniu 25.06.2013 13 :51, BM Kim >> pisze: >> >> > > Hi folks, >> > > >> > > First of all, I want to apologise for my poor english >> skills, >> > > since english is not my first language, but I'll try >> my best >> > > to formulate my quesions as clear as possible. >> > > >> > > Second, I've just begun to learn erlang, so if I'm >> asking >> > > obvious "noob" questions I apologise for that too in >> advance... >> > > >> > > Anywho, now to my actual question: >> > > >> > > I am planning to write a high-performance server >> application in >> erlang, >> > > which will primarily handle HTTP requests. After some >> reseach with >> google, >> > > I narrowed down my choices to cowboy, misultin and >> mochiweb and >> decided >> > > to go with the cowboy library first... >> > > >> > > Looking at some tutorials, I've quickly built a small >> server >> capable of >> > > serving static files and was eager to see first >> benchmark-results... >> > > I've also built a small Haskell server using Warp >> library to >> compare it >> > > with erlang's cowboy... >> > > >> > > But my first impression was, that my cowboy server is >> much much >> slower >> than >> > > expected when serving static-files and after some >> research I >> found a >> presentation >> > > of the cowboy's author claiming that cowboy shouldn't >> be used for >> serving >> > > static-files. So I modified the server code, so that >> it replies >> to every >> > > request with in-memory 4Kb binary blob and compared it >> with my >> haskell >> warp >> > > server serving 4kb static file... >> > > >> > > this is my simple cowboy's http handler: >> > > >> > > >> >> ------------------------------**__----------------------------** >> --__---------- >> >> > > >> > > blob() -> >> > > [<<0:8>> || _ <- lists:seq(1,4096)]. >> > >> > First, try to replace blob/0 function with this: >> > >> > blob() -> <<0:(4096*8)>>. >> > >> > Then, restart the test and report ;) >> > >> >> >> Hi, >> >> Thank you very much for pointing out the obvious mistake... >> After correcting it, I got improvement from 5940 req/s to >> 8650 req/s... >> >> But still much slower than the haskell warp-server, which >> has throughput >> of 38000 req/s... >> >> But I have another question regarding blob/0. Is it going to >> be >> evaluated >> only once (like GHC would do) since it is a pure >> expression? I'm not >> so sure, since erlang is not pure and any function can have >> side-effects >> which you can't mark as with the IO monad in Haskell... >> >> ______________________________**___________________ >> erlang-questions mailing list >> erlang-q...@REDACTED >> >> http://erlang.org/mailman/__**listinfo/erlang-questions >> >> > >> >> >> >> >> >> >> >> >> ______________________________**___________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > >> http://erlang.org/mailman/__**listinfo/erlang-questions >> >> >> > >> >> >> >> -- >> Lo?c Hoguin >> Erlang Cowboy >> Nine Nines >> http://ninenines.eu >> ______________________________**___________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > >> http://erlang.org/mailman/__**listinfo/erlang-questions >> >> > >> >> >> > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Thu Jun 27 13:32:53 2013 From: essen@REDACTED (=?UTF-8?B?TG/Dr2MgSG9ndWlu?=) Date: Thu, 27 Jun 2013 13:32:53 +0200 Subject: [erlang-questions] Cowboy (Erlang) VS Haskell (Warp) In-Reply-To: References: <51C98427.6040402@gmail.com> <51C98774.3080009@gmail.com> <51C9F861.8080005@ninenines.eu> <51CAFF44.6080501@ninenines.eu> Message-ID: <51CC22E5.1070201@ninenines.eu> On 06/27/2013 11:17 AM, Knut Nesheim wrote: > I happily help anybody who comes to me with questions. Me too, but I'm not sure how that would help someone trying to debug an issue in production quickly to avoid losing money. You shouldn't make the product's usability dependent on the author's availability. (And yep, I am also in the process of learning that, except in my case it's with regards to documentation.) -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From mononcqc@REDACTED Thu Jun 27 15:01:59 2013 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 27 Jun 2013 09:01:59 -0400 Subject: [erlang-questions] Erlang blog platform In-Reply-To: References: Message-ID: <20130627130158.GC4510@ferdair.local> I wrote and use https://bitbucket.org/ferd/blogerl/ for my blog on http://ferd.ca It's a minimal blogging platform as it only deals with ErlyDTL templates and compiles them flat to disk to a small index, and generates a small RSS feed -- the strict minimum for everything -- and it can go on the cheapest hosting solution available. It has no comments, but I wasn't interested in them. Instead, I would probably add Disqus comments in there (http://disqus.com/admin/universalcode/) after disabling ads on them from the admin panel. Disqus bases itself on the URL to know where comments go, so it can work perfectly fine with a system like that. Looking at http://erlang-russian.org, it seems you have a few other prerequisites though: multi-author support, i18n, support for code highlight, etc. It makes me wonder if zotonic (http://zotonic.com) would be appropriate for you -- it seems to support most of these features and even comes with a blog skeleton to get started. Regards, Fred. On 06/27, Maxim Treskin wrote: > Hello > > Could you please recommend any Erlang blog platform? Years ago I wrote > Metalkia: https://github.com/metachord/metalkia_web (see rebar deps) using > Nitrogen and Riak but now I see it should be totally rewriten to use > PostgreSQL or MySQL. Now http://erlang-russian.org based on this platform. > > May be someone wrote best blog platform with commentaries in Erlang? > > -- > Max Treskin > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From dmercer@REDACTED Thu Jun 27 15:40:53 2013 From: dmercer@REDACTED (David Mercer) Date: Thu, 27 Jun 2013 08:40:53 -0500 Subject: [erlang-questions] Erlang and/or MPI In-Reply-To: References: <51CAF879.1080700@llaisdy.com> Message-ID: <009201ce733b$f2dd57e0$d89807a0$@gmail.com> On Wednesday, June 26, 2013, Richard A. O'Keefe wrote: > There's another idea which has often been applied in Haskell, and that's > writing a program in a high level language that _generates_ low level > code. Am I wrong in thinking LISP would be a good language for this (to write in, not to generate to)? LISP programs are data, after all. Cheers, DBM From ivan@REDACTED Thu Jun 27 16:05:33 2013 From: ivan@REDACTED (Ivan Uemlianin) Date: Thu, 27 Jun 2013 15:05:33 +0100 Subject: [erlang-questions] Erlang and/or MPI In-Reply-To: <009201ce733b$f2dd57e0$d89807a0$@gmail.com> References: <51CAF879.1080700@llaisdy.com> <009201ce733b$f2dd57e0$d89807a0$@gmail.com> Message-ID: <51CC46AD.6070703@llaisdy.com> Dear All Thank you for your very helpful comments. I certainly don't want to shoehorn erlang in where it's not appropriate. I think (especially after reading these comments and further ponderation) that my first step should be to understand the resource and how best to use it (eg before Tuesday I'd never heard of Infiniband). I had thought to use erlang as a kind of 'project manager' to orchestrate the ways tasks are shared between nodes/machines/cores. If MPI already does this, I should probably try both approaches, and compare for performance and ease of writing & reading. Writing an Infiniband driver for erlang sounds interesting but it would probably be a diversion. However, investigating languages good at numerical computations (esp. Haskell and Fortran) would probably not be a diversion. Fortran can talk MPI directly it seems. If Fortran is not too painful to write and read, that might be a monolingual solution. Best wishes Ivan -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From daniel.goertzen@REDACTED Thu Jun 27 18:23:35 2013 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Thu, 27 Jun 2013 11:23:35 -0500 Subject: [erlang-questions] ssl {active, once} {packet, 4} In-Reply-To: References: Message-ID: When using {packet, 4} the length does not include the length of the header. Dan. On Thu, Jun 27, 2013 at 12:39 AM, H.C. v. Stockhausen wrote: > Hi, > > this is my first post to this group. Hello everyone. > > I am writing a client that connects to a server via ssl:connect. Upon > connection the server returns a . Thereafter the client > sends commands. The protocol has a 4 byte header that specifies the > payload length followed by the xml payload. > > Here's an extract of my initial attempt that works: > > %%%%%%%%%%%%%%%%%%%%%%%% > ... > -behaviour(gen_server). > -define(HEADER_SIZE, 4). > ... > > init([]) -> > {ok, Socket} = ssl:connect( > , , [ > binary, > {active, false}, > {keepalive, true}, > {certfile, }, > {keyfile, } > ]), > {ok, #state{socket=Socket}, 0}. > > ... > > handle_info(timeout, State) -> > {ok, _Greeting} = recv(State#state.socket), > {noreply, State}. > > ... > % internal helpers > > send(Socket, Msg) -> > Length = string:len(Msg) + ?HEADER_SIZE, > Data = <>, > ok = ssl:send(Socket, Data), > ok. > > recv(Socket) -> > {ok, <>} = ssl:recv(Socket, ?HEADER_SIZE), > {ok, Data} = ssl:recv(Socket, Length-?HEADER_SIZE), > Response = binary_to_list(Data), > {ok, Response}. > > %%%%%%%%%%%%%%%%%%%%%%%% > > I have two questions. > > 1) As you can see init()/1 has a 0 timeout to trigger > handle_info(timeout, State) which then fetches and discard the > greeting message. > > Shouldn't I be able to just set {active, once} instead and handle the > initial server response in a matching handle_info clause? I tried but > it's never triggered. All I receive after a while is > {ssl_closed,{sslsocket,new_ssl,<0.65.0>}}. > > 2) I manually build and unwrap the header/payload but isn't that what > {packet, 4} is for? I also tried that but the server does not respond, > as incidentally was the case when I initially didn't set the header > properly. > > I realise that there are a lot of list_to_binary_to_list_to... > conversions but that's another issue, isn't it? > > Is this how {active, once} and {packet, 4} should work or am I missing > something, such as differences when it comes to ssl or client vs. > server behaviour? > > Best regards, > Hans Christian > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jon@REDACTED Thu Jun 27 18:27:48 2013 From: jon@REDACTED (Jonathan Schneider) Date: Thu, 27 Jun 2013 17:27:48 +0100 Subject: [erlang-questions] Erlang and/or MPI In-Reply-To: <51CC46AD.6070703@llaisdy.com> References: <51CAF879.1080700@llaisdy.com> <009201ce733b$f2dd57e0$d89807a0$@gmail.com> <51CC46AD.6070703@llaisdy.com> Message-ID: > I certainly don't want to shoehorn erlang in where it's not appropriate. I think (especially after reading these comments and further ponderation) that my first step should be to understand the resource and how best to use it (eg before Tuesday I'd never heard of Infiniband). If this is the same Infiniband that connects switches last time I fiddled with it was between (Marvell) switches. (Our kit supports the usual range of SFP+ connecte fibres and direct attach cables but their dev kit had some Infiniband in the box.) It no more needed specific drivers than say a CAT5 cable would. If you really do mean some sort of PC network card and want any kind of performance you'll want C. Jon From moxford@REDACTED Thu Jun 27 21:08:21 2013 From: moxford@REDACTED (Mike Oxford) Date: Thu, 27 Jun 2013 12:08:21 -0700 Subject: [erlang-questions] [HEADS UP] Emysql driver In-Reply-To: References: Message-ID: Can't look right now, as I'm about to run into a meeting, but this is good news. Did you happen to add an "as_dict" and/or "as_proplist" output settings in util? -mox On Fri, Jun 21, 2013 at 2:38 AM, Jesper Louis Andersen wrote: > data properly. On the other hand, this means that th From jesper.louis.andersen@REDACTED Thu Jun 27 21:45:38 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 27 Jun 2013 21:45:38 +0200 Subject: [erlang-questions] [HEADS UP] Emysql driver In-Reply-To: References: Message-ID: Hi Mike, There is an `as_json/1` in the works as a pull request. Feel free to provide the other convenience operations so we can add them to the util module. They could be rather nice to have since everyone needs something like that. On Thu, Jun 27, 2013 at 9:08 PM, Mike Oxford wrote: > Can't look right now, as I'm about to run into a meeting, but this is good > news. > > Did you happen to add an "as_dict" and/or "as_proplist" output settings in > util? > > -mox > > On Fri, Jun 21, 2013 at 2:38 AM, Jesper Louis Andersen > wrote: > > data properly. On the other hand, this means that th > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Thu Jun 27 21:59:02 2013 From: mjtruog@REDACTED (okeuday) Date: Thu, 27 Jun 2013 12:59:02 -0700 (PDT) Subject: [erlang-questions] Cowboy (Erlang) VS Haskell (Warp) In-Reply-To: <51CC22E5.1070201@ninenines.eu> References: <51C98427.6040402@gmail.com> <51C98774.3080009@gmail.com> <51C9F861.8080005@ninenines.eu> <51CAFF44.6080501@ninenines.eu> <51CC22E5.1070201@ninenines.eu> Message-ID: <1a6ace4c-ed09-46b5-8da0-22c3d0042292@googlegroups.com> To be a bit more accurate, when I hook-up elli to CloudI it shows that cowboy has better performance when doing the same http_req test with httperf. So, it is likely that the benchmark ( https://github.com/yesodweb/benchmarks) could be faster with cowboy (i.e., CloudI is limiting throughput, probably mainly due to erlang:now/0 usage). One of the main reasons cowboy is faster is that the connection rate is an order of magnitude larger: 124.1 conn/s instead of 11.6 conn/s. The throughput, for the curious was 12539.2 req/s cowboy and 11560.6 req/s elli. I feel compelled to say again how httperf is not an accurate way to judge performance (see http://www.mnot.net/blog/2011/05/18/http_benchmark_rules), but it does provide a quick number to guess with. There is more serious loadtesting of CloudI here https://github.com/okeuday/CloudI/tree/master/src/tests/http_req . On Thursday, June 27, 2013 4:32:53 AM UTC-7, Lo?c Hoguin wrote: > > On 06/27/2013 11:17 AM, Knut Nesheim wrote: > > I happily help anybody who comes to me with questions. > > Me too, but I'm not sure how that would help someone trying to debug an > issue in production quickly to avoid losing money. > > You shouldn't make the product's usability dependent on the author's > availability. (And yep, I am also in the process of learning that, > except in my case it's with regards to documentation.) > > -- > Lo?c Hoguin > Erlang Cowboy > Nine Nines > http://ninenines.eu > _______________________________________________ > erlang-questions mailing list > erlang-q...@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From paul@REDACTED Fri Jun 28 01:55:24 2013 From: paul@REDACTED (Paul Rubin) Date: Thu, 27 Jun 2013 16:55:24 -0700 Subject: [erlang-questions] observer process labels Message-ID: I've been using observer to watch what's happening in my program as processes come and go. It's great the way the little ovals on the applications tab show the supervision and worker structure, with the name of each registered process shown in the oval. It isn't quite as nice when the process is unregistered, as happens with a bunch of my processes since they're created dynamically based on external events. I do register some (maybe all) of those processes with gproc, but they can't each be on an atom since indeterminately many of them come and go during the program's operation. I'm wondering what people think of the idea of an observer patch that does one of the following: 1) if the process is registered with gproc, show that as the label 2) look at a special entry in the process dictionary, something like {observer, label, "some string"} (or arbitrary term instead of string) that gets displayed as the label. #2 is maybe more general, since it doesn't require gproc to be loaded, and it can be expanded to allow setting the ovals' colors in addition to the labels. On the other hand, using the process dictionary for this doesn't seem so nice. ETS doesn't seem like the right thing because of the need to clean it up as the processes exit. Anyone else like this idea? I could maybe try to write a patch, though I haven't looked at the observer code and don't really know what I'd be getting into. Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Fri Jun 28 03:16:20 2013 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 28 Jun 2013 13:16:20 +1200 Subject: [erlang-questions] Erlang and/or MPI In-Reply-To: <009201ce733b$f2dd57e0$d89807a0$@gmail.com> References: <51CAF879.1080700@llaisdy.com> <009201ce733b$f2dd57e0$d89807a0$@gmail.com> Message-ID: On 28/06/2013, at 1:40 AM, David Mercer wrote: > On Wednesday, June 26, 2013, Richard A. O'Keefe wrote: > >> There's another idea which has often been applied in Haskell, and that's >> writing a program in a high level language that _generates_ low level >> code. > > Am I wrong in thinking LISP would be a good language for this (to write in, not to generate to)? LISP programs are data, after all. Lisp is of course a wonderful language for writing programs that write programs. Erlang programs are _also_ data, though not as conveniently, which is one of the reasons why Lisp-Flavoured Erlang (LFE) exists. The Haskell people claim that an advantage of using Haskell for meta-programming is its type system. Debugging code written by hand is hard enough; debugging code that was written by a program is harder, *unless* something about the way it was generated gives you high confidence in it. For example, Atom (http://hackage.haskell.org/package/atom-1.0.12) had this text in the message announcing the 0.0.2 release (http://www.haskell.org/pipermail/haskell-cafe/2009-April/060602.html): Experiences with our Eaton project: - 5K lines of Haskell/atom replaced 120K lines of matlab, simulink, and visual basic. - 2 months to port simulink design to atom. - 3K lines of atom generates 22K lines of embedded C. - Design composed of 450 atomic state transition rules. - Rules with execution periods from 1ms to 10s all scheduled at compile time to a 1 ms main loop. - 3 minute compilation time from atom source to ECU. * - Atom design clears electronic/sw testing on first pass. * - Currently in vehicle testing with no major issues. where the asterisks are mine. For floating-point calculations, Repa (http://www.haskell.org/haskellwiki/Numeric_Haskell:_A_Repa_Tutorial) looks interesting. Quoting that tutorial, Repa is a Haskell library for high performance, regular, multi-dimensional parallel arrays. All numeric data is stored unboxed and functions written with the Repa combinators are automatically parallel (provided you supply "+RTS -N" on the command line when running the program). As for Fortran, if it is written in an antique style, it _can_ be painful to read. If it is written using the modern features of the language, it _can_ be quite easy to read. I suggest that the single most important issue is getting the code *right* in the first place. I've just spent a day debugging some code that manipulated quaternions. There were two bugs: -- my code for computing the absolute value while avoiding overflows and loss of precision was stupidly broken. My fault. -- the source I had consulted for the definition of the logarithm of a quaternion didn't bother mentioning that the formula they gave implied a division by zero for quaternions with a zero vector part and was outright wrong for (-w,0,0,0) quaternions. Their fault. I think my favourite example was a clustering algorithm by some highly respected researchers where the published pseudocode ran smack into an uninitialised variable on the first iteration. Array-crunching code can be *astonishingly* hard to get right. I believe that this is because an uncommonly high proportion of trivial mistakes result in syntactically legal (but wrong!) code. So I claim that the right language to *start* a project like this in is the language that will let you code and debug the algorithm most quickly. Heck, it could well be Matlab or R. See for example http://cran.r-project.org/web/views/HighPerformanceComputing.html From desired.mta@REDACTED Fri Jun 28 06:29:42 2013 From: desired.mta@REDACTED (=?UTF-8?Q?Motiejus_Jak=C5=A1tys?=) Date: Fri, 28 Jun 2013 07:29:42 +0300 Subject: [erlang-questions] observer process labels In-Reply-To: References: Message-ID: On Fri, Jun 28, 2013 at 2:55 AM, Paul Rubin wrote: > I've been using observer to watch what's happening in my program as > processes come and go. It's great the way the little ovals on the > applications tab show the supervision and worker structure, with the name > of each registered process shown in the oval. > > It isn't quite as nice when the process is unregistered, as happens with a > bunch of my processes since they're created dynamically based on external > events. I do register some (maybe all) of those processes with gproc, but > they can't each be on an atom since indeterminately many of them come and go > during the program's operation. > > I'm wondering what people think of the idea of an observer patch that does > one of the following: > > 2) look at a special entry in the process dictionary, something like > {observer, label, "some string"} (or arbitrary term instead of string) that > gets displayed as the label. I really like this idea. With easy way to extend observer (for instance, like you mentioned, adding colors to ovals. Can be also labels, shape of the figure, etc), this could be a great visualization tool. -- Motiejus Jak?tys From wbin00@REDACTED Fri Jun 28 11:08:16 2013 From: wbin00@REDACTED (Bin Wang) Date: Fri, 28 Jun 2013 17:08:16 +0800 Subject: [erlang-questions] How to use gpb? Message-ID: Hi, I'm using google protocol buffers. I found an erlang plugin named gpb(https://github.com/tomas-abrahamsson/gpb). But I don't know how to use it. When I add it as a dependency of my app using rebar, while compile, it always comes out this error: ERROR: Protobuffs library not present in code path! Which I searched in google and only found it in erlang_protobuffs code. I do use it as a dependency before, but dose it matters? From ivan@REDACTED Fri Jun 28 12:02:22 2013 From: ivan@REDACTED (Ivan Uemlianin) Date: Fri, 28 Jun 2013 11:02:22 +0100 Subject: [erlang-questions] Erlang and/or MPI In-Reply-To: References: <51CAF879.1080700@llaisdy.com> <009201ce733b$f2dd57e0$d89807a0$@gmail.com> Message-ID: <51CD5F2E.2000007@llaisdy.com> Dear Richard On 28/06/2013 02:16, Richard A. O'Keefe wrote: >... > So I claim that the right language to *start* a project like this > in is the language that will let you code and debug the algorithm > most quickly. Heck, it could well be Matlab or R. See for > example http://cran.r-project.org/web/views/HighPerformanceComputing.html Very good point about R. I'm using it on another project and HPC Wales have it installed on their system. Thanks Ivan -- ============================================================ Ivan A. Uemlianin PhD Llaisdy Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com github.com/llaisdy www.linkedin.com/in/ivanuemlianin festina lente ============================================================ From yvan.godin@REDACTED Fri Jun 28 13:13:17 2013 From: yvan.godin@REDACTED (Yvan Godin) Date: Fri, 28 Jun 2013 13:13:17 +0200 Subject: [erlang-questions] Erlang and/or MPI Message-ID: Hello, you could also give a try to Racket-lang Not as fast as C++ or Fortran but a lot faster than Erlang, they provide a good Math Lib with Typed-Racket mode and they use Parallelization on multicore. 80-bit floating-point numbers (Extflonums) are supported on some x86/x86_64 platforms ? including Windows, and including platforms where Racket is compiled to use SSE instructions for flonum arithmetic. For concurrency and MPI you have DistributedPlace just for fun, cheer Yvan -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Fri Jun 28 15:27:58 2013 From: g@REDACTED (Garrett Smith) Date: Fri, 28 Jun 2013 08:27:58 -0500 Subject: [erlang-questions] [ANN] Chicago Erlang Factory Lite - Oct 4, 2013 Message-ID: I've very excited to announce that Chicago is hosting an Erlang Factory Lite on October 4! Details here: http://www.erlang-factory.com/conference/Chicago2013 We're looking for speakers! If you can make it to Chicago and would like to participate, feel free to propose something here: http://goo.gl/in2pC Please spread the word - mid-America needs more Erlang!! :) From gumm@REDACTED Fri Jun 28 20:40:40 2013 From: gumm@REDACTED (Jesse Gumm) Date: Fri, 28 Jun 2013 13:40:40 -0500 Subject: [erlang-questions] [ANN] Chicago Erlang Factory Lite - Oct 4, 2013 In-Reply-To: References: Message-ID: Majorly excited about this! Glad to have some midwest love! On Fri, Jun 28, 2013 at 8:27 AM, Garrett Smith wrote: > I've very excited to announce that Chicago is hosting an Erlang > Factory Lite on October 4! > > Details here: > > http://www.erlang-factory.com/conference/Chicago2013 > > We're looking for speakers! If you can make it to Chicago and would > like to participate, feel free to propose something here: > > http://goo.gl/in2pC > > Please spread the word - mid-America needs more Erlang!! :) > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm -------------- next part -------------- An HTML attachment was scrubbed... URL: From tomas.abrahamsson@REDACTED Sat Jun 29 15:37:42 2013 From: tomas.abrahamsson@REDACTED (Tomas Abrahamsson) Date: Sat, 29 Jun 2013 15:37:42 +0200 Subject: [erlang-questions] How to use gpb? In-Reply-To: References: Message-ID: Hi, Rebar by default uses another protobuf compiler for any *.proto files it finds in the src subdirectory. One way to use gpb with rebar is to put your *.proto in another subdirectory, and use hooks to compile it. I got it working with the following stuff in rebar.config: %% -*- erlang -*- {pre_hooks, [{compile, "mkdir -p include"}, %% ensure the include dir exists {compile, "erl +B -noinput -pa /path/to/gpb/ebin " "-I`pwd`/proto -o-erl src -o-hrl include " "-s gpb_compile c `pwd`/proto/x.proto" }]}. {post_hooks, [{clean, "rm -f src/x.erl include/x.hrl"}]}. {erl_opts, [{i, "/path/to/gpb/include"}]}. and the following initial file/directory layout: prompt# find . . ./proto ./proto/x.proto ./src ./src/g_sup.erl ./src/g.app.src ./src/g_app.erl ./rebar.config BRs Tomas On Fri, Jun 28, 2013 at 11:08 AM, Bin Wang wrote: > Hi, > > I'm using google protocol buffers. I found an erlang plugin named > gpb(https://github.com/tomas-abrahamsson/gpb). But I don't know how > to use it. When I add it as a dependency of my app using rebar, while > compile, it always comes out this error: > > ERROR: Protobuffs library not present in code path! > > Which I searched in google and only found it in erlang_protobuffs > code. I do use it as a dependency before, but dose it matters? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yash_ganthe@REDACTED Sat Jun 29 15:42:22 2013 From: yash_ganthe@REDACTED (Yash Ganthe) Date: Sat, 29 Jun 2013 13:42:22 +0000 Subject: [erlang-questions] Waiting for a node to start using run_erl Message-ID: Hi, When staring an erl node using run_erl, if the -daemon switch is used, control returns to the terminal immediately without letting me know whether the node was really started. The switch is recommended in the documentation at: http://erlang.org/doc/man/run_erl.html It says: Without this option, run_erl must be started using several tricks in the shell to detach it completely from the terminal in use when starting it. Is there a way to use run_erl and ensure that the node is started. This would be especially useful when starting the node from a shell script using run_erl. Thanks, Yash DISCLAIMER ========== This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails. -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Sat Jun 29 15:46:11 2013 From: erlang@REDACTED (Tim) Date: Sat, 29 Jun 2013 23:16:11 +0930 Subject: [erlang-questions] ehtml Message-ID: <4A5D4E6E-0530-42A4-B355-DBEA33C67222@5869.us> Greetings, Just wondering if anyone can point me in the direction of an ehtml reference/guide/source code listing. Acronym overlap means most google searches return pages about the extended character set, and both of the links at http://cean.process-one.net/packages/index.yaws?action=detail&name=ehtml do not seem to work. Cheers! -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Sat Jun 29 16:13:01 2013 From: g@REDACTED (Garrett Smith) Date: Sat, 29 Jun 2013 09:13:01 -0500 Subject: [erlang-questions] Waiting for a node to start using run_erl In-Reply-To: References: Message-ID: What is "started" in this case? A node start is a system startup, much like an OS start. There should be a point that you system is "ready" to server/act/etc. Once you establish that point, test for for it. Just as an example of why this isn't always straight forward, consider a socket server (e.g. a web server). You might define "ready" as soon as the server is accepting connections on a port, which is easy to test for. But the server might in fact be ready several seconds later as database connections are opened, caches warmed, whatever. So a ready test might be a 200 response on a particular http request (which is able to test for all those conditions). On Sat, Jun 29, 2013 at 8:42 AM, Yash Ganthe wrote: > Hi, > > > > When staring an erl node using run_erl, if the -daemon switch is used, > control returns to the terminal immediately without letting me know whether > the node was really started. > > The switch is recommended in the documentation at: > http://erlang.org/doc/man/run_erl.html > > > > It says: Without this option, run_erl must be started using several tricks > in the shell to detach it completely from the terminal in use when starting > it. > > > > Is there a way to use run_erl and ensure that the node is started. This > would be especially useful when starting the node from a shell script using > run_erl. > > > > Thanks, > > Yash > > DISCLAIMER ========== This e-mail may contain privileged and confidential > information which is the property of Persistent Systems Ltd. It is intended > only for the use of the individual or entity to which it is addressed. If > you are not the intended recipient, you are not authorized to read, retain, > copy, print, distribute or use this message. If you have received this > communication in error, please notify the sender and delete all copies of > this message. Persistent Systems Ltd. does not accept any liability for > virus infected mails. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From max.lapshin@REDACTED Sat Jun 29 16:15:08 2013 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 29 Jun 2013 18:15:08 +0400 Subject: [erlang-questions] Waiting for a node to start using run_erl In-Reply-To: References: Message-ID: There is no way in erlang right now to write to stderr that ports are busy and web server cannot be started using run_erl. From a.zhuravlev@REDACTED Sat Jun 29 16:51:04 2013 From: a.zhuravlev@REDACTED (Alexander Zhuravlev) Date: Sat, 29 Jun 2013 18:51:04 +0400 Subject: [erlang-questions] ehtml In-Reply-To: <4A5D4E6E-0530-42A4-B355-DBEA33C67222@5869.us> References: <4A5D4E6E-0530-42A4-B355-DBEA33C67222@5869.us> Message-ID: <20130629145104.GA10409@zmac> On Sat, Jun 29, 2013 at 11:16:11PM +0930, Tim wrote: > Greetings, > > Just wondering if anyone can point me in the direction of an ehtml > reference/guide/source code listing. Acronym overlap means most > google searches return pages about the extended character set, and > both of the links at > http://cean.process-one.net/packages/index.yaws?action=detail&name=ehtml > do not seem to work. Are you asking about ehtml from yaws? If this is a case, you can find its description on page #12 of the Yaws Reference documentation: http://hyber.org/yaws.pdf -- Alexander Zhuravlev From yash_ganthe@REDACTED Sat Jun 29 16:48:30 2013 From: yash_ganthe@REDACTED (Yash Ganthe) Date: Sat, 29 Jun 2013 14:48:30 +0000 Subject: [erlang-questions] Waiting for a node to start using run_erl In-Reply-To: References: Message-ID: My definition of "started" is that I am able to PING the node and get a PONG response back. Thanks, Yash -----Original Message----- From: Garrett Smith [mailto:g@REDACTED] Sent: Saturday, June 29, 2013 7:43 PM To: Yash Ganthe Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] Waiting for a node to start using run_erl What is "started" in this case? A node start is a system startup, much like an OS start. There should be a point that you system is "ready" to server/act/etc. Once you establish that point, test for for it. Just as an example of why this isn't always straight forward, consider a socket server (e.g. a web server). You might define "ready" as soon as the server is accepting connections on a port, which is easy to test for. But the server might in fact be ready several seconds later as database connections are opened, caches warmed, whatever. So a ready test might be a 200 response on a particular http request (which is able to test for all those conditions). On Sat, Jun 29, 2013 at 8:42 AM, Yash Ganthe wrote: > Hi, > > > > When staring an erl node using run_erl, if the -daemon switch is used, > control returns to the terminal immediately without letting me know > whether the node was really started. > > The switch is recommended in the documentation at: > http://erlang.org/doc/man/run_erl.html > > > > It says: Without this option, run_erl must be started using several > tricks in the shell to detach it completely from the terminal in use > when starting it. > > > > Is there a way to use run_erl and ensure that the node is started. > This would be especially useful when starting the node from a shell > script using run_erl. > > > > Thanks, > > Yash > > DISCLAIMER ========== This e-mail may contain privileged and > confidential information which is the property of Persistent Systems > Ltd. It is intended only for the use of the individual or entity to > which it is addressed. If you are not the intended recipient, you are > not authorized to read, retain, copy, print, distribute or use this > message. If you have received this communication in error, please > notify the sender and delete all copies of this message. Persistent > Systems Ltd. does not accept any liability for virus infected mails. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > DISCLAIMER ========== This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails. From g@REDACTED Sat Jun 29 17:04:49 2013 From: g@REDACTED (Garrett Smith) Date: Sat, 29 Jun 2013 10:04:49 -0500 Subject: [erlang-questions] Waiting for a node to start using run_erl In-Reply-To: References: Message-ID: To measure that specifically then, the two approaches that occur to me are to either poll the target node from another node by issuing pings, timing out after a period -- or configuring your target node to ping back as soon as it starts. If you don't have another node to control this (which would be odd given your start definition) you could tail/scan a log file as a proxy for start state. Or create a sentinel file somewhere that signals the node is started. This problem is endemic to coordinating system activity -- it's not specific to run_erl (even if it blocked, you'd have the same problem unless it fortuitously exited when you wanted it to). On Sat, Jun 29, 2013 at 9:48 AM, Yash Ganthe wrote: > My definition of "started" is that I am able to PING the node and get a PONG response back. > > Thanks, > Yash > > -----Original Message----- > From: Garrett Smith [mailto:g@REDACTED] > Sent: Saturday, June 29, 2013 7:43 PM > To: Yash Ganthe > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Waiting for a node to start using run_erl > > What is "started" in this case? > > A node start is a system startup, much like an OS start. There should be a point that you system is "ready" to server/act/etc. Once you establish that point, test for for it. > > Just as an example of why this isn't always straight forward, consider a socket server (e.g. a web server). You might define "ready" as soon as the server is accepting connections on a port, which is easy to test for. But the server might in fact be ready several seconds later as database connections are opened, caches warmed, whatever. So a ready test might be a 200 response on a particular http request (which is able to test for all those conditions). > > On Sat, Jun 29, 2013 at 8:42 AM, Yash Ganthe wrote: >> Hi, >> >> >> >> When staring an erl node using run_erl, if the -daemon switch is used, >> control returns to the terminal immediately without letting me know >> whether the node was really started. >> >> The switch is recommended in the documentation at: >> http://erlang.org/doc/man/run_erl.html >> >> >> >> It says: Without this option, run_erl must be started using several >> tricks in the shell to detach it completely from the terminal in use >> when starting it. >> >> >> >> Is there a way to use run_erl and ensure that the node is started. >> This would be especially useful when starting the node from a shell >> script using run_erl. >> >> >> >> Thanks, >> >> Yash >> >> DISCLAIMER ========== This e-mail may contain privileged and >> confidential information which is the property of Persistent Systems >> Ltd. It is intended only for the use of the individual or entity to >> which it is addressed. If you are not the intended recipient, you are >> not authorized to read, retain, copy, print, distribute or use this >> message. If you have received this communication in error, please >> notify the sender and delete all copies of this message. Persistent >> Systems Ltd. does not accept any liability for virus infected mails. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > DISCLAIMER > ========== > This e-mail may contain privileged and confidential information which is the property of Persistent Systems Ltd. It is intended only for the use of the individual or entity to which it is addressed. If you are not the intended recipient, you are not authorized to read, retain, copy, print, distribute or use this message. If you have received this communication in error, please notify the sender and delete all copies of this message. Persistent Systems Ltd. does not accept any liability for virus infected mails. From essen@REDACTED Sat Jun 29 19:15:51 2013 From: essen@REDACTED (=?ISO-8859-1?Q?Lo=EFc_Hoguin?=) Date: Sat, 29 Jun 2013 19:15:51 +0200 Subject: [erlang-questions] Async ssl_accept, ssl:close, gen_tcp:send and ssl:send Message-ID: <51CF1647.6020001@ninenines.eu> Hello, I'd like to start a discussion on making additions to the gen_tcp and ssl API to allow it to handle more asynchronous operations. The reasons for allowing them async are as follow: * async ssl_accept: This call can take a long time because there's a few round trips between the two endpoints to perform the handshake. Therefore there's a lot of waiting going on, not just busy operations. It could be more interesting to not block a process for this because the process could for example initialize the state for the connection while it waits for the handshake to be completed, reducing the latency of applications. * async ssl:close: This call can also take a long time for similar reasons. I think the same can apply to gen_tcp? Anyway if you have a long running process that holds a connection at some point, and wants to close it, it might interrupt its normal workflow. A temporary solution would be to have a process dedicated to closing these sockets and pass it ownership, but if that can be avoided... * async send: Send also blocks, I suppose because it waits for the TCP ack before returning. Sometimes you don't have the luxury to wait, so it would be good to have an alternative documented solution. An undocumented solution can be found here: http://www.erlang-factory.com/conference/SFBay2013/speakers/GeoffCant I'm mostly interested in the first point, as this is something that doesn't really have solutions. I could help with implementing or testing it if there is interest. There could also be an async TCP accept I suppose, except I already looked into adding that and the current code doesn't allow it easily. And I don't need it anymore, but perhaps someone does. Thanks. -- Lo?c Hoguin Erlang Cowboy Nine Nines http://ninenines.eu From fly@REDACTED Sat Jun 29 22:02:23 2013 From: fly@REDACTED (Fred Youhanaie) Date: Sat, 29 Jun 2013 21:02:23 +0100 Subject: [erlang-questions] Waiting for a node to start using run_erl In-Reply-To: References: Message-ID: <51CF3D4F.3030600@anydata.co.uk> How about checking through epmd with a bash command like this? run_erl -daemon ... until epmd -names | grep 'name NODENAME at port'; do sleep 3; done; Of course, you can turn the bash one liner into a script with optional delay and retry counts :-) However, as Garrett said, you may be better off with a more sophisticated test that is specific to your application. Cheers Fred On 29/06/13 15:48, Yash Ganthe wrote: > My definition of "started" is that I am able to PING the node and get a PONG response back. > > Thanks, > Yash > > -----Original Message----- > From: Garrett Smith [mailto:g@REDACTED] > Sent: Saturday, June 29, 2013 7:43 PM > To: Yash Ganthe > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Waiting for a node to start using run_erl > > What is "started" in this case? > > A node start is a system startup, much like an OS start. There should be a point that you system is "ready" to server/act/etc. Once you establish that point, test for for it. > > Just as an example of why this isn't always straight forward, consider a socket server (e.g. a web server). You might define "ready" as soon as the server is accepting connections on a port, which is easy to test for. But the server might in fact be ready several seconds later as database connections are opened, caches warmed, whatever. So a ready test might be a 200 response on a particular http request (which is able to test for all those conditions). > > On Sat, Jun 29, 2013 at 8:42 AM, Yash Ganthe wrote: >> Hi, >> >> >> >> When staring an erl node using run_erl, if the -daemon switch is used, >> control returns to the terminal immediately without letting me know >> whether the node was really started. >> >> The switch is recommended in the documentation at: >> http://erlang.org/doc/man/run_erl.html >> >> >> >> It says: Without this option, run_erl must be started using several >> tricks in the shell to detach it completely from the terminal in use >> when starting it. >> >> >> >> Is there a way to use run_erl and ensure that the node is started. >> This would be especially useful when starting the node from a shell >> script using run_erl. >> >> >> >> Thanks, >> >> Yash >> From erlang@REDACTED Sun Jun 30 17:02:00 2013 From: erlang@REDACTED (Tim) Date: Mon, 1 Jul 2013 00:32:00 +0930 Subject: [erlang-questions] dets:info/2 Message-ID: <4446C22B-03B7-4E1E-8F61-9D443ED0B1B3@5869.us> Greetings, Can anyone provide some example code of dets:info/2 in action that actually works? I'm trying to get info on a dets table through a .yaws script and just can't get it to work. out(Arg) -> dets:open_file(ingredients, [{file,"ingredients.dets"},{type,set}]), MyInfo = dets:info(ingredients,no_objects), {html, MyInfo}. The error occurs on the {html, MyInfo} line - it appears to be unable to display the value. My reading of the docs suggests that the return value of sets:info/2 is a simple value, but I suspect that it is not and have not yet worked out a way of visualising what sort of structure is being returned by the function. Cheers. From norton@REDACTED Sun Jun 30 17:09:49 2013 From: norton@REDACTED (Joseph Wayne Norton) Date: Sun, 30 Jun 2013 10:09:49 -0500 Subject: [erlang-questions] dets:info/2 In-Reply-To: <4446C22B-03B7-4E1E-8F61-9D443ED0B1B3@5869.us> References: <4446C22B-03B7-4E1E-8F61-9D443ED0B1B3@5869.us> Message-ID: Tim - Have you tried these commands using the erlang shell? $ erl Erlang R16B01 (erts-5.10.2) [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] Eshell V5.10.2 (abort with ^G) 1> dets:open_file(ingredients, [{file,"ingredients.dets"},{type,set}]). {ok,ingredients} 2> MyInfo = dets:info(ingredients,no_objects). 0 3> On 2013/06/30, at 10:02, Tim wrote: > Greetings, > > Can anyone provide some example code of dets:info/2 in action that actually works? > > I'm trying to get info on a dets table through a .yaws script and just can't get it to work. > > > out(Arg) -> > dets:open_file(ingredients, [{file,"ingredients.dets"},{type,set}]), > MyInfo = dets:info(ingredients,no_objects), > {html, MyInfo}. > > > The error occurs on the {html, MyInfo} line - it appears to be unable to display the value. > > My reading of the docs suggests that the return value of sets:info/2 is a simple value, but I suspect that it is not and have not yet worked out a way of visualising what sort of structure is being returned by the function. > > Cheers. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From erlang@REDACTED Sun Jun 30 17:13:58 2013 From: erlang@REDACTED (Tim) Date: Mon, 1 Jul 2013 00:43:58 +0930 Subject: [erlang-questions] ehtml In-Reply-To: <20130629145104.GA10409@zmac> References: <4A5D4E6E-0530-42A4-B355-DBEA33C67222@5869.us> <20130629145104.GA10409@zmac> Message-ID: <21599A59-A52D-4BAD-9771-42E4F5080DA6@5869.us> On 30/06/2013, at 12:21 AM, Alexander Zhuravlev wrote: >> Just wondering if anyone can point me in the direction of an ehtml >> reference/guide/source code listing. Acronym overlap means most >> google searches return pages about the extended character set, and >> both of the links at >> http://cean.process-one.net/packages/index.yaws?action=detail&name=ehtml >> do not seem to work. > > Are you asking about ehtml from yaws? If this is a case, you can > find its description on page #12 of the Yaws Reference documentation: > http://hyber.org/yaws.pdf Yes, using ehtml to output structured content in .yaws pages. The manual reads thusly: The second way to generate output is by returning a tuple {ehtml, EHTML}. The term EHTML must adhere to the following structure: EHTML = [EHTML] | {TAG, Attrs, Body} | {TAG, Attrs} | {TAG} | {Module,Fun,[Args]} | fun/0 | binary() | character() TAG = atom() Attrs = [{HtmlAttribute,Value}] HtmlAttribute = atom() Value = string() | atom() | {Module,Fun,[Args]} | fun/0 Body = EHTML ? which suggests that it is pretty agnostic about what gets thrown in. I just need to be careful about reserved words like 'div'. Specifically, I want to know if it is possible to 'tune' the HTML generation so that it produces source that is neat and tidy. -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Sun Jun 30 17:37:19 2013 From: erlang@REDACTED (Tim) Date: Mon, 1 Jul 2013 01:07:19 +0930 Subject: [erlang-questions] dets:info/2 In-Reply-To: References: <4446C22B-03B7-4E1E-8F61-9D443ED0B1B3@5869.us> Message-ID: <7646A290-1EEC-4BDA-8A52-5E62B2A5E21E@5869.us> Joseph, Yes, I have entered the commands into the running shell, and all I get back is the simple value "3" on a line, by itself, with nothing else - no error. That behaviour is what was expected. When I simply try to display the variable that contains that value using {html, MyInfo}, it prints nothing - but does not generate an error either. If, however, I try to mix it in with some other HTML in this fashion: {html,"

" ++ MyInfo ++ "

\n"). then it crashes: ERROR erlang code threw an uncaught exception: File: /Users/tim/website/ingredients.yaws:51 Class: error Exception: badarg Req: {http_request,'GET',{abs_path,"/ingredients.yaws"},{1,1}} Stack: [{m41,out,1,[{file,"/Users/tim/.yaws/yaws/default/m41.erl"},{line,19}]}, {yaws_server,deliver_dyn_part,8, [{file,"yaws_server.erl"},{line,2750}]}, {yaws_server,aloop,4,[{file,"yaws_server.erl"},{line,1175}]}, {yaws_server,acceptor0,2,[{file,"yaws_server.erl"},{line,1016}]}, {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}] ?and, unfortunately, I can't make any sense of that error message. I don't understand this bizarre behaviour. On 01/07/2013, at 12:39 AM, Joseph Wayne Norton wrote: > Have you tried these commands using the erlang shell? > > $ erl > Erlang R16B01 (erts-5.10.2) [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V5.10.2 (abort with ^G) > 1> dets:open_file(ingredients, [{file,"ingredients.dets"},{type,set}]). > {ok,ingredients} > 2> MyInfo = dets:info(ingredients,no_objects). > 0 > 3> > > >> Can anyone provide some example code of dets:info/2 in action that actually works? >> >> I'm trying to get info on a dets table through a .yaws script and just can't get it to work. >> >> >> out(Arg) -> >> dets:open_file(ingredients, [{file,"ingredients.dets"},{type,set}]), >> MyInfo = dets:info(ingredients,no_objects), >> {html, MyInfo}. >> >> >> The error occurs on the {html, MyInfo} line - it appears to be unable to display the value. >> >> My reading of the docs suggests that the return value of sets:info/2 is a simple value, but I suspect that it is not and have not yet worked out a way of visualising what sort of structure is being returned by the function. From norton@REDACTED Sun Jun 30 17:51:14 2013 From: norton@REDACTED (Joseph Wayne Norton) Date: Sun, 30 Jun 2013 10:51:14 -0500 Subject: [erlang-questions] dets:info/2 In-Reply-To: <7646A290-1EEC-4BDA-8A52-5E62B2A5E21E@5869.us> References: <4446C22B-03B7-4E1E-8F61-9D443ED0B1B3@5869.us> <7646A290-1EEC-4BDA-8A52-5E62B2A5E21E@5869.us> Message-ID: Tim - MyInfo is an integer. You need to convert it to a string before using the "++" operator. A string is simply a list of integers in Erlang. $ erl Erlang R16B01 (erts-5.10.2) [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] Eshell V5.10.2 (abort with ^G) 1> dets:open_file(ingredients, [{file,"ingredients.dets"},{type,set}]). {ok,ingredients} 2> 2> MyInfo = dets:info(ingredients,no_objects). 0 3> integer_to_list(MyInfo). "0" On 2013/06/30, at 10:37, Tim wrote: > Joseph, > > Yes, I have entered the commands into the running shell, and all I get back is the simple value "3" on a line, by itself, with nothing else - no error. That behaviour is what was expected. > > When I simply try to display the variable that contains that value using {html, MyInfo}, it prints nothing - but does not generate an error either. > > If, however, I try to mix it in with some other HTML in this fashion: > > {html,"

" ++ MyInfo ++ "

\n"). > > then it crashes: > > ERROR erlang code threw an uncaught exception: > File: /Users/tim/website/ingredients.yaws:51 > Class: error > Exception: badarg > Req: {http_request,'GET',{abs_path,"/ingredients.yaws"},{1,1}} > Stack: [{m41,out,1,[{file,"/Users/tim/.yaws/yaws/default/m41.erl"},{line,19}]}, > {yaws_server,deliver_dyn_part,8, > [{file,"yaws_server.erl"},{line,2750}]}, > {yaws_server,aloop,4,[{file,"yaws_server.erl"},{line,1175}]}, > {yaws_server,acceptor0,2,[{file,"yaws_server.erl"},{line,1016}]}, > {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}] > > ?and, unfortunately, I can't make any sense of that error message. > > I don't understand this bizarre behaviour. > > > > On 01/07/2013, at 12:39 AM, Joseph Wayne Norton wrote: > >> Have you tried these commands using the erlang shell? >> >> $ erl >> Erlang R16B01 (erts-5.10.2) [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] >> >> Eshell V5.10.2 (abort with ^G) >> 1> dets:open_file(ingredients, [{file,"ingredients.dets"},{type,set}]). >> {ok,ingredients} >> 2> MyInfo = dets:info(ingredients,no_objects). >> 0 >> 3> >> >> >>> Can anyone provide some example code of dets:info/2 in action that actually works? >>> >>> I'm trying to get info on a dets table through a .yaws script and just can't get it to work. >>> >>> >>> out(Arg) -> >>> dets:open_file(ingredients, [{file,"ingredients.dets"},{type,set}]), >>> MyInfo = dets:info(ingredients,no_objects), >>> {html, MyInfo}. >>> >>> >>> The error occurs on the {html, MyInfo} line - it appears to be unable to display the value. >>> >>> My reading of the docs suggests that the return value of sets:info/2 is a simple value, but I suspect that it is not and have not yet worked out a way of visualising what sort of structure is being returned by the function. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From vinoski@REDACTED Sun Jun 30 18:22:47 2013 From: vinoski@REDACTED (Steve Vinoski) Date: Sun, 30 Jun 2013 12:22:47 -0400 Subject: [erlang-questions] ehtml In-Reply-To: <21599A59-A52D-4BAD-9771-42E4F5080DA6@5869.us> References: <4A5D4E6E-0530-42A4-B355-DBEA33C67222@5869.us> <20130629145104.GA10409@zmac> <21599A59-A52D-4BAD-9771-42E4F5080DA6@5869.us> Message-ID: On Sun, Jun 30, 2013 at 11:13 AM, Tim wrote: > > Specifically, I want to know if it is possible to 'tune' the HTML > generation so that it produces source that is neat and tidy. > What kind of "tuning" do you have in mind? If you want tidy HTML, why not use the tidy tool itself? See https://github.com/w3c/tidy-html5 or the original at http://tidy.sourceforge.net . Note also that Yaws has exhtml, for similar support for XHTML. Also, you might consider trying these questions on the Yaws mailing list: https://lists.sourceforge.net/lists/listinfo/erlyaws-list --steve -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Sun Jun 30 19:50:55 2013 From: erlang@REDACTED (Tim) Date: Mon, 1 Jul 2013 03:20:55 +0930 Subject: [erlang-questions] dets:info/2 In-Reply-To: References: <4446C22B-03B7-4E1E-8F61-9D443ED0B1B3@5869.us> <7646A290-1EEC-4BDA-8A52-5E62B2A5E21E@5869.us> Message-ID: <22A75B31-B3F0-4E03-8419-A79D7F188458@5869.us> Ooookaaaay. I see I'm going to have to unlearn a lot of assumptions to get my brain into the Erlang groove. :) Thanks for sorting that out - it does now work as expected. One parting question though: If integers and strings (lists of integers) are _not_ interchangeable, why doesn't this line generate an error? {html, MyInfo}. On 01/07/2013, at 1:21 AM, Joseph Wayne Norton wrote: > MyInfo is an integer. You need to convert it to a string before using the "++" operator. A string is simply a list of integers in Erlang. ... > 1> dets:open_file(ingredients, [{file,"ingredients.dets"},{type,set}]). > {ok,ingredients} > 2> > 2> MyInfo = dets:info(ingredients,no_objects). > 0 > 3> integer_to_list(MyInfo). > "0" From g@REDACTED Sun Jun 30 20:09:48 2013 From: g@REDACTED (Garrett Smith) Date: Sun, 30 Jun 2013 13:09:48 -0500 Subject: [erlang-questions] dets:info/2 In-Reply-To: <22A75B31-B3F0-4E03-8419-A79D7F188458@5869.us> References: <4446C22B-03B7-4E1E-8F61-9D443ED0B1B3@5869.us> <7646A290-1EEC-4BDA-8A52-5E62B2A5E21E@5869.us> <22A75B31-B3F0-4E03-8419-A79D7F188458@5869.us> Message-ID: Hi Tim, On Sun, Jun 30, 2013 at 12:50 PM, Tim wrote: > Ooookaaaay. I see I'm going to have to unlearn a lot of assumptions to get my brain into the Erlang groove. :) If you can avoid it, don't make assumptions when you're sorting this stuff out :) To reiterate Joseph's point, the Erlang shell is where you want to go first for info. It's usually pretty easy to experiment with APIs in the shell, which will give you a hands on experience that's sometimes hard to get from online docs and books. A lesser know pattern, related to your problem, is to avoid creating flattened strings when you're generating content for file-like IO (file system, sockets. etc.). Rather than this: Name = "Joe", "My name is " ++ Name Do this: Name = "Joe", ["My name is ", Name] This is an example of an "iolist", which is an arbitrarily nested list containing strings (lists of integers) and binaries. It's a powerful abstraction has it lets you efficiently build string/binary content without pointlessly flattening them. > Thanks for sorting that out - it does now work as expected. > > One parting question though: If integers and strings (lists of integers) are _not_ interchangeable, why doesn't this line generate an error? > > {html, MyInfo}. My guess is the server is treating the integer as a character and it's getting rolled into an iolist (see previous comment). Garrett From jesper.louis.andersen@REDACTED Sun Jun 30 21:21:24 2013 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 30 Jun 2013 21:21:24 +0200 Subject: [erlang-questions] Erlang and/or MPI In-Reply-To: References: <51CAF879.1080700@llaisdy.com> Message-ID: On Thu, Jun 27, 2013 at 3:26 AM, Richard A. O'Keefe wrote: > > Except for commands typed at the shell, Erlang is compiled, not > interpreted. > And there is a native code generator; c(File, [native]) does the trick if > native code generation was enabled at build time. > And the native code generator exploits -spec declarations (and some type > inference) to avoid boxing floats some of the time. > > It is bytecode interpreted to be precise. It is faster than blind AST-interpretation, but it is not really what I would call compiled in the sense of compiled-to-native-code. The native code generator can help for certain types of programs, but it does not yield enough speedup to make it interesting in my opinion. > So Erlang might well make good sense as a language for prototyping an > application, even a floating-point one, to the point where you can do > some benchmarking and find out where the time is really going. > For prototyping, Erlang is definitely an excellent vehicle. Haskell could be even better in this situation though. > > My experience has been that Fortran is *still* the King if you want > serious floating-point performance, and modern Fortran is a serious > modular programming language. GNU Fortran has _nearly_ caught up > with the 2003 standard, which saw some serious additions to the language. > Coarrays are part of the 2008 standard, and gfortran support for them is > still incomplete, which is a pity, because they look extremely useful > for this kind of problem. I haven't personally used the Intel Fortran > compiler, but I've used the Intel C compiler, and Intel's Parallel > Studio _looks_ impressive. I see the HPCWales systems have the Intel > compilers... > > Good point about Fortran. > What's often an issue in floating-point calculations is algorithm design, > and good facilities for managing abstractions may be more important than > raw floating-point speed. Haskell is pretty good here, and there are > some interesting libraries, like the beginnings of support for Geometric > Algebra. There's certainly an HMM library (version 0.04, so be warned). > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Sun Jun 30 21:25:29 2013 From: erlang@REDACTED (Tim) Date: Mon, 1 Jul 2013 04:55:29 +0930 Subject: [erlang-questions] dets:info/2 In-Reply-To: References: <4446C22B-03B7-4E1E-8F61-9D443ED0B1B3@5869.us> <7646A290-1EEC-4BDA-8A52-5E62B2A5E21E@5869.us> <22A75B31-B3F0-4E03-8419-A79D7F188458@5869.us> Message-ID: On 01/07/2013, at 3:39 AM, Garrett Smith wrote: > A lesser know pattern, related to your problem, is to avoid creating > flattened strings when you're generating content for file-like IO > (file system, sockets. etc.). > > Rather than this: > > Name = "Joe", > "My name is " ++ Name > > Do this: > > Name = "Joe", > ["My name is ", Name] > > This is an example of an "iolist", which is an arbitrarily nested list > containing strings (lists of integers) and binaries. It's a powerful > abstraction has it lets you efficiently build string/binary content > without pointlessly flattening them. Heh. I'd already slipped into using a lot of the ++ and the iolist approach will make my code much cleaner. Thanks for that!