From ok@REDACTED Fri Apr 1 01:28:05 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 1 Apr 2016 12:28:05 +1300 Subject: [erlang-questions] pre-load large data files when the application start In-Reply-To: References: <613f5b28-186b-9ba1-5a61-77052e2720c7@cs.otago.ac.nz> Message-ID: <8dee52ab-35e3-4c2e-df58-96498f1a005a@cs.otago.ac.nz> On 31/03/16 11:54 pm, Benoit Chesneau wrote: > > Do you mean having different functions in the same module using a map > each? Yes. One change at a time! It's the "column-oriented data base" idea. Instead of f(X) -> maps:get(X, {... K => {U,V,W} ...}, false) have u(X) -> maps:get(X, {...K => U...}, false). v(X) -> maps:get(X, {...K => V...}, false). w(X) -> maps:get(X, {...K=> W...}, false). One reason is that in each column the set of keys with non-default values is different, and another reason is that callers don't *want* all the information, so they have to spend time fishing what they do want out of the triple. From erlangsiri@REDACTED Fri Apr 1 12:43:59 2016 From: erlangsiri@REDACTED (Siri Hansen) Date: Fri, 1 Apr 2016 12:43:59 +0200 Subject: [erlang-questions] Should we deprecate or modernize log_mf_h and rb? Message-ID: Hello list! log_mf_h is an error_logger event handler which logs events to disk and does log rotation. rb (report browser) is the tool for reading and formatting the logs. The implementation of both modules is quite old and outdated. The fact that there is a size field for the events of only 16 bits, which we haven't got any complaints for, has made us believe that there might not be many users of these tools. I'm writing to the list to find out if this is correct... Are you using log_mf_h (e.g. by setting environment variables 'error_logger_mf_*' in sasl) and rb? If so, why - is it the log rotation or the binary logging (or both) that you are really after? Have you considered alternative tools? Kind Regards /siri@REDACTED -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbj@REDACTED Fri Apr 1 13:08:23 2016 From: mbj@REDACTED (Martin Bjorklund) Date: Fri, 01 Apr 2016 13:08:23 +0200 (CEST) Subject: [erlang-questions] Should we deprecate or modernize log_mf_h and rb? In-Reply-To: References: Message-ID: <20160401.130823.1006060931061872335.mbj@tail-f.com> Hi, We are not using log_mf_h, but instead we're using disk_log_h (see http://erlang.org/pipermail/erlang-patches/2012-January/002583.html) We use this handler to get log rotation, but we log all error logger messages in clear text (this should IMO be the default). If you're interested in adding disk_log_h to OTP (search for disk_log_h in disk_log.erl...), the latest version (updated as recently as 2006 :) can be found here: https://github.com/richcarl/otp/blob/disk_log_h/lib/kernel/src/disk_log_h.erl I support the deprecation initiative! /martin Siri Hansen wrote: > Hello list! > > log_mf_h is an error_logger event handler which logs events to disk and > does log rotation. rb (report browser) is the tool for reading and > formatting the logs. The implementation of both modules is quite old and > outdated. The fact that there is a size field for the events of only 16 > bits, which we haven't got any complaints for, has made us believe that > there might not be many users of these tools. I'm writing to the list to > find out if this is correct... > > Are you using log_mf_h (e.g. by setting environment variables > 'error_logger_mf_*' in sasl) and rb? If so, why - is it the log rotation or > the binary logging (or both) that you are really after? Have you considered > alternative tools? > > Kind Regards > /siri@REDACTED From carlsson.richard@REDACTED Fri Apr 1 13:34:13 2016 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Fri, 1 Apr 2016 13:34:13 +0200 Subject: [erlang-questions] Should we deprecate or modernize log_mf_h and rb? In-Reply-To: <20160401.130823.1006060931061872335.mbj@tail-f.com> References: <20160401.130823.1006060931061872335.mbj@tail-f.com> Message-ID: Thanks for pointing out that old mail, Martin! These days though, it's probably worthwhile to rethink/rewrite these handlers from scratch rather than import disk_log_h into OTP as it is. /Richard 2016-04-01 13:08 GMT+02:00 Martin Bjorklund : > Hi, > > We are not using log_mf_h, but instead we're using disk_log_h (see > http://erlang.org/pipermail/erlang-patches/2012-January/002583.html) > > We use this handler to get log rotation, but we log all error logger > messages in clear text (this should IMO be the default). > > If you're interested in adding disk_log_h to OTP (search for > disk_log_h in disk_log.erl...), the latest version > (updated as recently as 2006 :) can be found here: > > https://github.com/richcarl/otp/blob/disk_log_h/lib/kernel/src/disk_log_h.erl > > I support the deprecation initiative! > > > /martin > > > > Siri Hansen wrote: > > Hello list! > > > > log_mf_h is an error_logger event handler which logs events to disk and > > does log rotation. rb (report browser) is the tool for reading and > > formatting the logs. The implementation of both modules is quite old and > > outdated. The fact that there is a size field for the events of only 16 > > bits, which we haven't got any complaints for, has made us believe that > > there might not be many users of these tools. I'm writing to the list to > > find out if this is correct... > > > > Are you using log_mf_h (e.g. by setting environment variables > > 'error_logger_mf_*' in sasl) and rb? If so, why - is it the log rotation > or > > the binary logging (or both) that you are really after? Have you > considered > > alternative tools? > > > > Kind Regards > > /siri@REDACTED > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From christopher.meiklejohn@REDACTED Fri Apr 1 16:00:36 2016 From: christopher.meiklejohn@REDACTED (Christopher Meiklejohn) Date: Fri, 1 Apr 2016 15:00:36 +0100 Subject: [erlang-questions] [ANN] Curry On Rome! CFP (co-located with ECOOP 2016) Message-ID: We're looking for some Erlang talks, distributed systems talks, and more! Please submit! http://curry-on.org/2016/ http://curry-on.org/2016/call-for-presentations.html We ran a pretty special event last year in Prague, you can find all of the talks available online here: http://curry-on.org/2015/ - Christopher From steven@REDACTED Fri Apr 1 17:18:28 2016 From: steven@REDACTED (Steven Livingstone) Date: Fri, 1 Apr 2016 16:18:28 +0100 Subject: [erlang-questions] Fire and Forget Message-ID: Hi all, Erlang newbie here. So, hello all :-) Now, I wish to do a fire and forget http post in Erlang. I looked at the httpc module which seems to allow synchronous and asynchronous calls but in the latter case i do not expect a return value so don't need a request id .... and not sure whether that behaviour influences what I want ... does it hold open a connection or use up resources on a response that is never going to happen etc. Does anyone know the best approach? many thanks, steven From paulperegud@REDACTED Fri Apr 1 17:55:23 2016 From: paulperegud@REDACTED (Paul Peregud) Date: Fri, 1 Apr 2016 17:55:23 +0200 Subject: [erlang-questions] Fire and Forget In-Reply-To: References: Message-ID: Low complexity solution: just spawn a process which will send request and terminate. More complex solution: spawn process under supervisor. Restart strategy "temporary" is a good fit. Or "transient" if you want to restart failed attempts. As for "response never happens". Do you mean that server does not return anything ever? Not even HTTP 200 OK? Just holds up connection? On Fri, Apr 1, 2016 at 5:18 PM, Steven Livingstone wrote: > Hi all, Erlang newbie here. So, hello all :-) Now, > > I wish to do a fire and forget http post in Erlang. > > I looked at the httpc module which seems to allow synchronous and > asynchronous calls but in the latter case i do not expect a return > value so don't need a request id .... and not sure whether that > behaviour influences what I want ... does it hold open a connection or > use up resources on a response that is never going to happen etc. > > Does anyone know the best approach? > > many thanks, > steven > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Best regards, Paul Peregud +48602112091 From steven@REDACTED Fri Apr 1 18:12:58 2016 From: steven@REDACTED (Steven Livingstone) Date: Fri, 1 Apr 2016 17:12:58 +0100 Subject: [erlang-questions] Fire and Forget In-Reply-To: References: Message-ID: Thanks Paul. It was more that as i would be just logging lots of data, if some gets lost it is ok, hence it doesn't need to wait on a 200. So, I guess it was how to optimise under those conditions, but perhaps doing an async post and later getting a 200, even if it is just a "all is ok" even if i don't intend to do anything beyond that may be a sensible approach anyway. Restart temporary under supervisor sounds good - i mainly don't want to block anything. Unfortunately I come with the burden of years of procedural programming so my mindset is still adjusting to the fact spinning up many processes is a good thing in erlang. It may be I should be passing the data using something other than an HTTP POST and more erlang-like but one step at a time :-) regards, steven On Fri, Apr 1, 2016 at 4:55 PM, Paul Peregud wrote: > Low complexity solution: just spawn a process which will send request > and terminate. > More complex solution: spawn process under supervisor. Restart > strategy "temporary" is a good fit. Or "transient" if you want to > restart failed attempts. > > As for "response never happens". Do you mean that server does not > return anything ever? Not even HTTP 200 OK? Just holds up connection? > > On Fri, Apr 1, 2016 at 5:18 PM, Steven Livingstone wrote: >> Hi all, Erlang newbie here. So, hello all :-) Now, >> >> I wish to do a fire and forget http post in Erlang. >> >> I looked at the httpc module which seems to allow synchronous and >> asynchronous calls but in the latter case i do not expect a return >> value so don't need a request id .... and not sure whether that >> behaviour influences what I want ... does it hold open a connection or >> use up resources on a response that is never going to happen etc. >> >> Does anyone know the best approach? >> >> many thanks, >> steven >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > -- > Best regards, > Paul Peregud > +48602112091 From felixgallo@REDACTED Fri Apr 1 18:44:02 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Fri, 1 Apr 2016 09:44:02 -0700 Subject: [erlang-questions] Fire and Forget In-Reply-To: References: Message-ID: I accidentally once destroyed a 50 machine php http api cluster by pointing a single four core erlang instance at it and making requests as fast as I could go. You may wish to build rate limiting and error handling in from the start instead of ignoring the response. F. On Apr 1, 2016 9:14 AM, "Steven Livingstone" wrote: > Thanks Paul. It was more that as i would be just logging lots of data, > if some gets lost it is ok, hence it doesn't need to wait on a 200. > > So, I guess it was how to optimise under those conditions, but perhaps > doing an async post and later getting a 200, even if it is just a "all > is ok" even if i don't intend to do anything beyond that may be a > sensible approach anyway. > > Restart temporary under supervisor sounds good - i mainly don't want > to block anything. Unfortunately I come with the burden of years of > procedural programming so my mindset is still adjusting to the fact > spinning up many processes is a good thing in erlang. > > It may be I should be passing the data using something other than an > HTTP POST and more erlang-like but one step at a time :-) > > regards, > steven > > On Fri, Apr 1, 2016 at 4:55 PM, Paul Peregud > wrote: > > Low complexity solution: just spawn a process which will send request > > and terminate. > > More complex solution: spawn process under supervisor. Restart > > strategy "temporary" is a good fit. Or "transient" if you want to > > restart failed attempts. > > > > As for "response never happens". Do you mean that server does not > > return anything ever? Not even HTTP 200 OK? Just holds up connection? > > > > On Fri, Apr 1, 2016 at 5:18 PM, Steven Livingstone > wrote: > >> Hi all, Erlang newbie here. So, hello all :-) Now, > >> > >> I wish to do a fire and forget http post in Erlang. > >> > >> I looked at the httpc module which seems to allow synchronous and > >> asynchronous calls but in the latter case i do not expect a return > >> value so don't need a request id .... and not sure whether that > >> behaviour influences what I want ... does it hold open a connection or > >> use up resources on a response that is never going to happen etc. > >> > >> Does anyone know the best approach? > >> > >> many thanks, > >> steven > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > > > > -- > > Best regards, > > Paul Peregud > > +48602112091 > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Fri Apr 1 18:54:54 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Fri, 1 Apr 2016 09:54:54 -0700 Subject: [erlang-questions] Fire and Forget In-Reply-To: References: Message-ID: Also, as far as resources go - http clients by themselves are unlikely to tax a box at any rate you are likely to do. At high rates, you may encounter file handle or socket starvation on (at least) essentially all Linux boxes owing to pathologically poor defaults out of the box. You may need to tweak sysctl values on those systems if you need more than a few thousand http requests per second. F. On Apr 1, 2016 9:44 AM, "Felix Gallo" wrote: > I accidentally once destroyed a 50 machine php http api cluster by > pointing a single four core erlang instance at it and making requests as > fast as I could go. You may wish to build rate limiting and error handling > in from the start instead of ignoring the response. > > F. > On Apr 1, 2016 9:14 AM, "Steven Livingstone" wrote: > >> Thanks Paul. It was more that as i would be just logging lots of data, >> if some gets lost it is ok, hence it doesn't need to wait on a 200. >> >> So, I guess it was how to optimise under those conditions, but perhaps >> doing an async post and later getting a 200, even if it is just a "all >> is ok" even if i don't intend to do anything beyond that may be a >> sensible approach anyway. >> >> Restart temporary under supervisor sounds good - i mainly don't want >> to block anything. Unfortunately I come with the burden of years of >> procedural programming so my mindset is still adjusting to the fact >> spinning up many processes is a good thing in erlang. >> >> It may be I should be passing the data using something other than an >> HTTP POST and more erlang-like but one step at a time :-) >> >> regards, >> steven >> >> On Fri, Apr 1, 2016 at 4:55 PM, Paul Peregud >> wrote: >> > Low complexity solution: just spawn a process which will send request >> > and terminate. >> > More complex solution: spawn process under supervisor. Restart >> > strategy "temporary" is a good fit. Or "transient" if you want to >> > restart failed attempts. >> > >> > As for "response never happens". Do you mean that server does not >> > return anything ever? Not even HTTP 200 OK? Just holds up connection? >> > >> > On Fri, Apr 1, 2016 at 5:18 PM, Steven Livingstone >> wrote: >> >> Hi all, Erlang newbie here. So, hello all :-) Now, >> >> >> >> I wish to do a fire and forget http post in Erlang. >> >> >> >> I looked at the httpc module which seems to allow synchronous and >> >> asynchronous calls but in the latter case i do not expect a return >> >> value so don't need a request id .... and not sure whether that >> >> behaviour influences what I want ... does it hold open a connection or >> >> use up resources on a response that is never going to happen etc. >> >> >> >> Does anyone know the best approach? >> >> >> >> many thanks, >> >> steven >> >> _______________________________________________ >> >> erlang-questions mailing list >> >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions >> > >> > >> > >> > -- >> > Best regards, >> > Paul Peregud >> > +48602112091 >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven@REDACTED Fri Apr 1 19:05:48 2016 From: steven@REDACTED (Steven Livingstone) Date: Fri, 1 Apr 2016 18:05:48 +0100 Subject: [erlang-questions] Fire and Forget In-Reply-To: References: Message-ID: Interesting - thanks Felix. The 50 machine cluster destruction is quite an achievement :-) I'll take heed. I could be hooking some devices in and so there could be quite a lot of data as they scale. I am considering putting something like RabbitMQ in the middle if i need to scale but starting with the basics just now. I am new to Erlang but the entire approach sits well with me so i want to persist with it. The defaults are something i'll research - some endpoint will be on Docker instances and i know there could be additional things to figure out there too. Thanks again, steven On Fri, Apr 1, 2016 at 5:54 PM, Felix Gallo wrote: > Also, as far as resources go - http clients by themselves are unlikely to > tax a box at any rate you are likely to do. At high rates, you may encounter > file handle or socket starvation on (at least) essentially all Linux boxes > owing to pathologically poor defaults out of the box. You may need to tweak > sysctl values on those systems if you need more than a few thousand http > requests per second. > > F. > > On Apr 1, 2016 9:44 AM, "Felix Gallo" wrote: >> >> I accidentally once destroyed a 50 machine php http api cluster by >> pointing a single four core erlang instance at it and making requests as >> fast as I could go. You may wish to build rate limiting and error handling >> in from the start instead of ignoring the response. >> >> F. >> >> On Apr 1, 2016 9:14 AM, "Steven Livingstone" wrote: >>> >>> Thanks Paul. It was more that as i would be just logging lots of data, >>> if some gets lost it is ok, hence it doesn't need to wait on a 200. >>> >>> So, I guess it was how to optimise under those conditions, but perhaps >>> doing an async post and later getting a 200, even if it is just a "all >>> is ok" even if i don't intend to do anything beyond that may be a >>> sensible approach anyway. >>> >>> Restart temporary under supervisor sounds good - i mainly don't want >>> to block anything. Unfortunately I come with the burden of years of >>> procedural programming so my mindset is still adjusting to the fact >>> spinning up many processes is a good thing in erlang. >>> >>> It may be I should be passing the data using something other than an >>> HTTP POST and more erlang-like but one step at a time :-) >>> >>> regards, >>> steven >>> >>> On Fri, Apr 1, 2016 at 4:55 PM, Paul Peregud >>> wrote: >>> > Low complexity solution: just spawn a process which will send request >>> > and terminate. >>> > More complex solution: spawn process under supervisor. Restart >>> > strategy "temporary" is a good fit. Or "transient" if you want to >>> > restart failed attempts. >>> > >>> > As for "response never happens". Do you mean that server does not >>> > return anything ever? Not even HTTP 200 OK? Just holds up connection? >>> > >>> > On Fri, Apr 1, 2016 at 5:18 PM, Steven Livingstone >>> > wrote: >>> >> Hi all, Erlang newbie here. So, hello all :-) Now, >>> >> >>> >> I wish to do a fire and forget http post in Erlang. >>> >> >>> >> I looked at the httpc module which seems to allow synchronous and >>> >> asynchronous calls but in the latter case i do not expect a return >>> >> value so don't need a request id .... and not sure whether that >>> >> behaviour influences what I want ... does it hold open a connection or >>> >> use up resources on a response that is never going to happen etc. >>> >> >>> >> Does anyone know the best approach? >>> >> >>> >> many thanks, >>> >> steven >>> >> _______________________________________________ >>> >> erlang-questions mailing list >>> >> erlang-questions@REDACTED >>> >> http://erlang.org/mailman/listinfo/erlang-questions >>> > >>> > >>> > >>> > -- >>> > Best regards, >>> > Paul Peregud >>> > +48602112091 >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions From rtrlists@REDACTED Fri Apr 1 21:53:00 2016 From: rtrlists@REDACTED (Robert Raschke) Date: Fri, 1 Apr 2016 21:53:00 +0200 Subject: [erlang-questions] Should we deprecate or modernize log_mf_h and rb? In-Reply-To: References: Message-ID: I really like this "old" error logger, it's *easily* one of the fastest around actually, since it can leave formatting to read time! I've hacked it to truncate overlong entries. The rb application is also easily hackable to suit your needs. I have a version that looks up formatting functions based on a bit of metadata in the logged "messages". The major boon of being so blindingly fast, is that you do not have to set log levels at logging time. Instead you choose which log level you want to see when investigating! Unfortunately, straight logged binary data and formatting it at read time appears to be majorly out of fashion :-( Oh well. If you want to keep it, I think it would be worth adding the truncation bit, or make it accept any size log message. Cheers, Robby On Apr 1, 2016 12:44 PM, "Siri Hansen" wrote: > Hello list! > > log_mf_h is an error_logger event handler which logs events to disk and > does log rotation. rb (report browser) is the tool for reading and > formatting the logs. The implementation of both modules is quite old and > outdated. The fact that there is a size field for the events of only 16 > bits, which we haven't got any complaints for, has made us believe that > there might not be many users of these tools. I'm writing to the list to > find out if this is correct... > > Are you using log_mf_h (e.g. by setting environment variables > 'error_logger_mf_*' in sasl) and rb? If so, why - is it the log rotation or > the binary logging (or both) that you are really after? Have you considered > alternative tools? > > Kind Regards > /siri@REDACTED > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Apr 1 22:10:38 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Fri, 1 Apr 2016 22:10:38 +0200 Subject: [erlang-questions] Should we deprecate or modernize log_mf_h and rb? In-Reply-To: References: Message-ID: <56FED5BE.5040402@ninenines.eu> Hello, I didn't know we had that. It sounds like a hidden gem in OTP, and one I would say worth keeping/improving upon; but not necessarily as part of OTP. If all I had to do was add an application as dependency to benefit from this, I would most certainly do so. On 04/01/2016 12:43 PM, Siri Hansen wrote: > Hello list! > > log_mf_h is an error_logger event handler which logs events to disk and > does log rotation. rb (report browser) is the tool for reading and > formatting the logs. The implementation of both modules is quite old and > outdated. The fact that there is a size field for the events of only 16 > bits, which we haven't got any complaints for, has made us believe that > there might not be many users of these tools. I'm writing to the list to > find out if this is correct... > > Are you using log_mf_h (e.g. by setting environment variables > 'error_logger_mf_*' in sasl) and rb? If so, why - is it the log rotation > or the binary logging (or both) that you are really after? Have you > considered alternative tools? > > Kind Regards > /siri@REDACTED > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From eric.pailleau@REDACTED Fri Apr 1 22:46:30 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Fri, 01 Apr 2016 22:46:30 +0200 Subject: [erlang-questions] Should we deprecate or modernize log_mf_h and rb? In-Reply-To: References: Message-ID: Hi, I integrated rb log search in observer since several versions , and me too, like this fast error logger. I think rewriting it, or enhance it would be nice. The main issue was to write a pseudo io server to catch rb output. I would be very sad if it is deprecated and vanish. Dear Otp team, save rb and log_mf_h ! "Envoy? depuis mon mobile " Eric ---- Robert Raschke a ?crit ---- >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From mrallen1@REDACTED Sat Apr 2 00:58:41 2016 From: mrallen1@REDACTED (Mark Allen) Date: Fri, 1 Apr 2016 22:58:41 +0000 (UTC) Subject: [erlang-questions] Should we deprecate or modernize log_mf_h and rb? In-Reply-To: References: Message-ID: <655473439.1376794.1459551521101.JavaMail.yahoo@mail.yahoo.com> I support deprecation, for what its worth.? On Friday, April 1, 2016 5:44 AM, Siri Hansen wrote: Hello list! log_mf_h is an error_logger event handler which logs events to disk and does log rotation. rb (report browser) is the tool for reading and formatting the logs. The implementation of both modules is quite old and outdated. The fact that there is a size field for the events of only 16 bits, which we haven't got any complaints for, has made us believe that there might not be many users of these tools. I'm writing to the list to find out if this is correct...? Are you using log_mf_h (e.g. by setting environment variables 'error_logger_mf_*' in sasl) and rb? If so, why - is it the log rotation or the binary logging (or both) that you are really after? Have you considered alternative tools? Kind Regards/siri@REDACTED _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vances@REDACTED Sat Apr 2 09:05:32 2016 From: vances@REDACTED (Vance Shipley) Date: Sat, 2 Apr 2016 12:35:32 +0530 Subject: [erlang-questions] Should we deprecate or modernize log_mf_h and rb? In-Reply-To: References: Message-ID: On Apr 1, 2016 12:44 PM, "Siri Hansen" wrote: > Are you using log_mf_h (e.g. by setting environment variables 'error_logger_mf_*' in sasl) and rb? Yes, this is part of my default configuration for embedded systems. > If so, why - is it the log rotation or the binary logging (or both) that you are really after? Both. > Have you considered alternative tools? I've considered application specific versions of this scheme. On Sat, Apr 2, 2016 at 1:23 AM, Robert Raschke wrote: > Unfortunately, straight logged binary data and formatting it at read time appears to be majorly out of fashion :-( Stupidly so IMHO. I often see systems with a mission of moving massive amounts of message traffic logging data orders of magnitude larger than the size of the actual message. Doing so naively results in the servers spending much more time logging than doing the work they're intended to do. Logging is important and at this rate it costs real money to accomplish so it deserves to be engineered sensibly. My rational is that where a gen_fsm is spawned for each transaction it should simply do it's work and before terminating send it's State variable term, usually a record, directly to a log handler (e.g. disk_log). To have the worker for traffic doing string processing for an human to possibly read a long time later makes no sense. Log binary data to disk and format just the data requested, at request time, formatted for the requestor. -- -Vance From rvirding@REDACTED Sat Apr 2 18:34:58 2016 From: rvirding@REDACTED (Robert Virding) Date: Sat, 2 Apr 2016 18:34:58 +0200 Subject: [erlang-questions] Should we deprecate or modernize log_mf_h and rb? In-Reply-To: References: Message-ID: Would it be possible to split out the binary data so it could be used as an option with the rotating files? I don't know if this would be interesting as such but you colud maybe still use rb to view the logs. Another benefit of doing this would be to break out the formatting of the log messages outside of the erlang system. This would give all the speed benefits and still provide text logs for those who want it. It would also make it easier to provide different text formats. Robert On 2 April 2016 at 09:05, Vance Shipley wrote: > On Apr 1, 2016 12:44 PM, "Siri Hansen" wrote: > > Are you using log_mf_h (e.g. by setting environment variables > 'error_logger_mf_*' in sasl) and rb? > > Yes, this is part of my default configuration for embedded systems. > > > If so, why - is it the log rotation or the binary logging (or both) that > you are really after? > > Both. > > > Have you considered alternative tools? > > I've considered application specific versions of this scheme. > > On Sat, Apr 2, 2016 at 1:23 AM, Robert Raschke > wrote: > > Unfortunately, straight logged binary data and formatting it at read > time appears to be majorly out of fashion :-( > > Stupidly so IMHO. I often see systems with a mission of moving > massive amounts of message traffic logging data orders of magnitude > larger than the size of the actual message. Doing so naively results > in the servers spending much more time logging than doing the work > they're intended to do. Logging is important and at this rate it > costs real money to accomplish so it deserves to be engineered > sensibly. My rational is that where a gen_fsm is spawned for each > transaction it should simply do it's work and before terminating send > it's State variable term, usually a record, directly to a log handler > (e.g. disk_log). To have the worker for traffic doing string > processing for an human to possibly read a long time later makes no > sense. Log binary data to disk and format just the data requested, at > request time, formatted for the requestor. > > -- > -Vance > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sat Apr 2 22:35:23 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 2 Apr 2016 22:35:23 +0200 Subject: [erlang-questions] Should we deprecate or modernize log_mf_h and rb? In-Reply-To: References: Message-ID: I think this is an excellent idea, as it will revive a zombie thread from before 2010: http://erlang.2086793.n4.nabble.com/log-mf-h-and-rb-do-not-handle-terms-over-65k-td2220104.html even with Kenneth/OTP making an apperance :) The reason I like log_mf_h and rb is because I think it is a very good way of doing log information, especially in space constrained environments where a ring-buffer is good. I would have used this a lot were it not for the fact that it doesn't log anything above 64k in size, but then kills the whole log with that number afterwards. So I drifted toward lager instead. But had this been able to handle that situation gracefully, I'd definitely have used it in many situations. I would have fixed it long ago, were it not for the fact that it takes some effort to actually do correctly, and my 4-byte non-backwards-compatible hack worked back in the day. The linux world is moving to binary logging in journald, but I'm not entirely sure I like it since garbled logs in binary are harder to save. On the other hand, given a binary log and an index, searching through logs is *much* faster. There is also an old project of mine which uses machine learning to autoclassify errors into groups, which is based on log_mf_h: http://jlouisramblings.blogspot.dk/2011/01/errlxperimentation-clustering-of-erlang.html these kind of things are far harder to do on textual logs, where you have no erlang term structure readily available. The other question though: should this be part of OTP? I think it is an excellent library to have *outside* OTP because it should only rely on an error_logger inside OTP. That way, we can evolve the library in another development cycle than the OTP system, which is good. One problem with OTP currently is that adding functionality has a latency of roughly 0.75 years on average. You write the feature, but it is only available 0.75 years later. This is far too slow for most people. But were the things separate, it might be easier to release an intermediate version between major releases. On Fri, Apr 1, 2016 at 12:43 PM, Siri Hansen wrote: > Hello list! > > log_mf_h is an error_logger event handler which logs events to disk and > does log rotation. rb (report browser) is the tool for reading and > formatting the logs. The implementation of both modules is quite old and > outdated. The fact that there is a size field for the events of only 16 > bits, which we haven't got any complaints for, has made us believe that > there might not be many users of these tools. I'm writing to the list to > find out if this is correct... > > Are you using log_mf_h (e.g. by setting environment variables > 'error_logger_mf_*' in sasl) and rb? If so, why - is it the log rotation or > the binary logging (or both) that you are really after? Have you considered > alternative tools? > > Kind Regards > /siri@REDACTED > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mickael_trouve@REDACTED Sat Apr 2 22:32:32 2016 From: mickael_trouve@REDACTED (mickael trouve) Date: Sat, 2 Apr 2016 20:32:32 +0000 (UTC) Subject: [erlang-questions] unique_integer References: <1858403129.2605321.1459629152461.JavaMail.yahoo.ref@mail.yahoo.com> Message-ID: <1858403129.2605321.1459629152461.JavaMail.yahoo@mail.yahoo.com> Hi everyone, When trying to generate some unique integer I get some strange behavior.I'm currently running on a VM with only one core. Erlang/OTP 18 [erts-7.3] [source] [64-bit] [async-threads:10] [kernel-poll:false] Eshell V7.3? (abort with ^G)1> [erlang:unique_integer([positive]) || _ <- lists:seq(1, 15)]. [2,4,6,8,10,12,14,16,18,20,22,24,26,28,30] 2> Why does the step of the unique integer is 2 ? What about the odd values ? My second question is about the depth of this integer. I read on the documentation 2?? - 1.So i could parse those values in a 64 bits binary integer, right ? A = erlang:unique_integer([positive]).B = <>. Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From hq@REDACTED Sun Apr 3 00:21:24 2016 From: hq@REDACTED (Adam Rutkowski) Date: Sun, 03 Apr 2016 00:21:24 +0200 Subject: [erlang-questions] Should we deprecate or modernize log_mf_h and rb? In-Reply-To: References: Message-ID: <1459635684.2976813.566908242.64AA8A75@webmail.messagingengine.com> On Fri, Apr 1, 2016, at 12:43, Siri Hansen wrote: > > Are you using log_mf_h (e.g. by setting environment variables > 'error_logger_mf_*' in sasl) and rb? If so, why - is it the log > rotation or the binary logging (or both) that you are really after? > Have you considered alternative tools? > Hi, Report browser was one of the first "wow factors" for me, I was shipping for enterprise sysops people and they absolutely loved it. I doubt what I've worked on will be ever upgraded from R14 though. I support Loic's idea to keep it, but extract it from the OTP. /A. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Sun Apr 3 15:10:29 2016 From: ingela.andin@REDACTED (Ingela Andin) Date: Sun, 3 Apr 2016 15:10:29 +0200 Subject: [erlang-questions] SSL connections (in Common Test) intermittently fail with "unknown ca" In-Reply-To: References: Message-ID: Hi! 2016-03-31 17:33 GMT+02:00 Roger Lipscombe : > I've got a custom ranch protocol -- it's based on ranch_ssl, but it > adds a custom verify_fun, with configuration options. > > I'm attempting to test it in Common Test, and I'm seeing intermittent > "unknown ca" failures. I suspect, though I'm not sure, that it might > be due to the fact that each test starts a ranch listener with > different SSL options, in particular the 'cacertfile' option varies. > > Is there a race condition in the 'ssl' application which might get > confused by this? > Humm ... if there is such a race I think gen_statem will resolve it. (Planned for 19) There is not a "known" such race but I am crrently seeing some problems along these lines on some windows machines that have no good explenation and only occur for some windows builds and not others with same version of openssl and erlang but diffrent compilers for openssl. > > I've attempted to clean up by calling ssl:clear_pem_cache from > end_per_testcase, but it doesn't appear to make any difference. > > Clearing the pem cache will help under the circumstanses that you use the same pem-file for diffrent test cases but the pem-file contents on disk has changed between the test cases. Also putting such an "ensure clean start"-action in end_per_testcase might not do what you want as end_per_testcase will only run if the test does not fail, so it will proably fit better in init_per_testcase. Regards Ingela Erlang/OTP Team - Ericsson AB Help...? > > I can tidy up the test suite for public consumption if anyone thinks > that would be useful. > > -- > Roger. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g@REDACTED Mon Apr 4 01:18:09 2016 From: g@REDACTED (Guilherme Andrade) Date: Mon, 4 Apr 2016 00:18:09 +0100 Subject: [erlang-questions] unique_integer In-Reply-To: <1858403129.2605321.1459629152461.JavaMail.yahoo@mail.yahoo.com> References: <1858403129.2605321.1459629152461.JavaMail.yahoo.ref@mail.yahoo.com> <1858403129.2605321.1459629152461.JavaMail.yahoo@mail.yahoo.com> Message-ID: <5701A4B1.10002@gandrade.net> Hi, On 02/04/16 21:32, mickael trouve wrote: > > Why does the step of the unique integer is 2 ? What about the odd values ? By doing a quick test using 18.2.1, a na?ve analysis points to 1) the step being proportional to twice the number of online schedulers and 2) things getting fuzzier when trying to generate integers concurrently. I would speculate that maybe separate counters are kept for different schedulers in order to not have that as a bottleneck, and hence a step that's proportional to the number of counters (as well as adequate initial values) in order to avoid repetition. Or maybe it's something else entirely. -- Guilherme http://www.gandrade.net/ PGP: 0x602B2AD8 / B348 C976 CCE1 A02A 017E 4649 7A6E B621 602B 2AD8 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: From kennethlakin@REDACTED Mon Apr 4 07:57:35 2016 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Sun, 3 Apr 2016 22:57:35 -0700 Subject: [erlang-questions] unique_integer In-Reply-To: <1858403129.2605321.1459629152461.JavaMail.yahoo@mail.yahoo.com> References: <1858403129.2605321.1459629152461.JavaMail.yahoo.ref@mail.yahoo.com> <1858403129.2605321.1459629152461.JavaMail.yahoo@mail.yahoo.com> Message-ID: <5702024F.9040609@gmail.com> On 04/02/2016 01:32 PM, mickael trouve wrote: > My second question is about the depth of this integer. I read on the > documentation 2?? - 1. > So i could parse those values in a 64 bits binary integer, right ? The documentation indicates that that could only be true if you pass [monotonic]. If you don't, then unique_integer can produce (NoSchedulers + 1) * (2?? - 1) unique integers. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From henrik.x.nord@REDACTED Mon Apr 4 16:24:19 2016 From: henrik.x.nord@REDACTED (Henrik Nord X) Date: Mon, 4 Apr 2016 16:24:19 +0200 Subject: [erlang-questions] Patch package OTP 18.3.1 released Message-ID: <57027913.1060609@ericsson.com> Patch Package: OTP 18.3.1 Git Tag: OTP-18.3.1 Date: 2016-04-04 Trouble Report Id: OTP-13417, OTP-13418, OTP-13419, OTP-13420, OTP-13423, OTP-13424, OTP-13446, OTP-13452 Seq num: System: OTP Release: 18 Application: erts-7.3.1, inets-6.2.1, mnesia-4.13.4 Predecessor: OTP 18.3 Check out the git tag OTP-18.3.1, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- erts-7.3.1 ------------------------------------------------------ --------------------------------------------------------------------- The erts-7.3.1 application can be applied independently of other applications on a full OTP 18 installation. --- Fixed Bugs and Malfunctions --- OTP-13418 Application(s): erts process_info(Pid, last_calls) did not work for Pid /= self(). OTP-13419 Application(s): erts Make sure to create a crash dump when running out of memory. This was accidentally removed in the erts-7.3 release. OTP-13420 Application(s): erts Schedulers could be woken by a premature timeout on Linux. This premature wakeup was however harmless. OTP-13424 Application(s): erts Related Id(s): OTP-10336 A process communicating with a port via one of the erlang:port_* BIFs could potentially end up in an inconsistent state if the port terminated during the communication. When this occurred the process could later block in a receive even though it had messages matching in its message queue. This bug was introduced in erts version 5.10 (OTP R16A). OTP-13446 Application(s): erts The reference count of a process structure could under rare circumstances be erroneously managed. When this happened invalid memory accesses occurred. OTP-13452 Application(s): erts Fix race between process_flag(trap_exit,true) and a received exit signal. A process could terminate due to exit signal even though process_flag(trap_exit,true) had returned. A very specific timing between call to process_flag/2 and exit signal from another scheduler was required for this to happen. Full runtime dependencies of erts-7.3.1: kernel-4.0, sasl-2.4, stdlib-2.5 --------------------------------------------------------------------- --- inets-6.2.1 ----------------------------------------------------- --------------------------------------------------------------------- The inets-6.2.1 application can be applied independently of other applications on a full OTP 18 installation. --- Fixed Bugs and Malfunctions --- OTP-13417 Application(s): inets Mend ipv6_host_with_brackets option in httpc Full runtime dependencies of inets-6.2.1: erts-6.0, kernel-3.0, mnesia-4.12, runtime_tools-1.8.14, ssl-5.3.4, stdlib-2.0 --------------------------------------------------------------------- --- mnesia-4.13.4 --------------------------------------------------- --------------------------------------------------------------------- The mnesia-4.13.4 application can be applied independently of other applications on a full OTP 18 installation. --- Fixed Bugs and Malfunctions --- OTP-13423 Application(s): mnesia Mnesia transactions could hang while waiting on a response from a node who had stopped. Full runtime dependencies of mnesia-4.13.4: erts-7.0, kernel-3.0, stdlib-2.0 --------------------------------------------------------------------- --- Thanks to ------------------------------------------------------- --------------------------------------------------------------------- Andrey Mayorov, Lukas Larsson --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From erlang@REDACTED Mon Apr 4 21:59:28 2016 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 4 Apr 2016 21:59:28 +0200 Subject: [erlang-questions] repeatable builds Message-ID: Hello, I think I've asked this before but cannot find the answer: I want the beam file produced by $ erl file.erl to always have the same sha1 checksum - there was, if I remember correctly, a hidden flag that removed the time of compilation etc from the beam code. Any ideas how to do this? /Joe From jj@REDACTED Mon Apr 4 22:03:04 2016 From: jj@REDACTED (Giovanni Giorgi) Date: Mon, 4 Apr 2016 22:03:04 +0200 Subject: [erlang-questions] repeatable builds In-Reply-To: References: Message-ID: <2665CADE-6221-4736-8A18-327BA4EBA2B3@gioorgi.com> I have found http://erlang.org/pipermail/erlang-questions/2013-November/076137.html talking about beam_lib:strip/1 (just googling :) On 04/apr/2016, at 21:59, Joe Armstrong wrote: > Hello, > > I think I've asked this before but cannot find the answer: > > I want the beam file produced by > > $ erl file.erl > > to always have the same sha1 checksum - there was, if I remember > correctly, a hidden flag that removed the time of compilation etc from > the beam code. Any ideas how to do this? > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Giovanni Giorgi jj@REDACTED From carlsson.richard@REDACTED Mon Apr 4 22:37:54 2016 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Mon, 4 Apr 2016 22:37:54 +0200 Subject: [erlang-questions] repeatable builds In-Reply-To: References: Message-ID: Note that even if you strip the date, and any other things depending on the environment where you compiled the code, like file paths on the local machine, the resulting file will still depend on the version of the compiler. Changes in optimizations made (and of course any new beam operations) between releases of OTP will make the generated beam code different, even if it's just for some functions here and there. If you're aiming for some more universal property of the code, a hash of the parse tree (after preprocessor expansions, n.b.) would be more suitable - but then of course, it's hard to prove that the beam code corresponds to that parse tree. /Richard 2016-04-04 21:59 GMT+02:00 Joe Armstrong : > Hello, > > I think I've asked this before but cannot find the answer: > > I want the beam file produced by > > $ erl file.erl > > to always have the same sha1 checksum - there was, if I remember > correctly, a hidden flag that removed the time of compilation etc from > the beam code. Any ideas how to do this? > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthias@REDACTED Mon Apr 4 22:49:48 2016 From: matthias@REDACTED (Matthias Lang) Date: Mon, 4 Apr 2016 22:49:48 +0200 Subject: [erlang-questions] Should we deprecate or modernize log_mf_h and rb? In-Reply-To: References: Message-ID: <20160404204948.GA7278@corelatus.se> On 01. April 2016, Siri Hansen wrote: > Are you using log_mf_h In 2001, we used it and then abandoned it for two main reasons: 1. There was at least one bug in about R7 which made binary_to_term() crash the entire VM for some input. This made _reading_ logs after a power failure dangerous. 2. We had very limited space (less than 1 MB) for logging in earlier generations of our hardware. text logging + trunc_io (now in lager) + gzip gave us more log-per-MB than binary logging (without gzip; AFAIK it's not an option). The logging system we ended up using is completely home-made. Matthias From jordand@REDACTED Mon Apr 4 23:53:15 2016 From: jordand@REDACTED (Jordan Day) Date: Mon, 4 Apr 2016 16:53:15 -0500 Subject: [erlang-questions] Any guidance on tweaking allocator settings? Message-ID: <4EC0DE36-BFA6-4423-81AB-0BB5D6078CB9@companykitchen.com> I?m facing an issue with a program that is using hackney to download a number of large-ish (~8MB) JSON responses. OS-reported memory consumption is typically much larger than I?d expect (I?ve seen spikes of several GB), so I?ve spent quite a bit of time digging through the mailing list and blog posts trying to gain some insight into my problem. There have been several threads discussing the ?large binary? problem, which seems like it might be what I?m facing, but a lot of the suggested remedies (lowering fullsweep_after, manually running garbage_collect/1) haven?t seemed to have much effect. After watching Lukas Larsson?s talk from EUC (https://www.youtube.com/watch?v=YuPaX11vZyI), I?m wondering about tweaking allocator settings, but I?m really not sure where to start? While the problem I?m facing seems a lot like the ?large binary problem?, observer is reporting that most of my memory usage is tied up in eheap_alloc (https://imgur.com/OzPLvF0). I?ve taken some samples of the memory info over time using recon?s snapshot tool, which can be seen here: https://gist.github.com/jordan0day/3a127046514a5a9efb87665223e985c7 https://gist.github.com/jordan0day/679e5cc546cdfb4b8c7431f83305b86f https://gist.github.com/jordan0day/b2f6e69733732f35b612ab810a330ea5 These samples were taken just a minute or two apart from each other, spanning about 30 requests total, so I?d expect the growth to be around 240 MB or so, but you can see by the 3rd snapshot total memory usage is around 1.3 GB. Everything I?ve read so far has basically said ?don?t mess with the allocator settings unless you know what you?re doing!? ? and I definitely don?t know what I?m doing, but I?m really hoping to learn! I?d appreciate any guidance anyone can offer. The information contained in this message is confidential information and privileged. It is intended only for the use of the individual or individuals or entity identified above. You are hereby notified that any dissemination, distribution, use or copying of this message is strictly prohibited. If the receiver of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, use or copying of this message is strictly prohibited. If you have received this message in error, please immediately notify the sender by replying to his/her e-mail address noted above and delete the message. Thank You -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Tue Apr 5 09:31:53 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Tue, 5 Apr 2016 09:31:53 +0200 Subject: [erlang-questions] Any guidance on tweaking allocator settings? In-Reply-To: <4EC0DE36-BFA6-4423-81AB-0BB5D6078CB9@companykitchen.com> References: <4EC0DE36-BFA6-4423-81AB-0BB5D6078CB9@companykitchen.com> Message-ID: Hello, If you use recon_alloc:memory(usage) on the snapshots you have you can see you have a high usage percentage which means that you do not have any memory fragmentation issues that tweaking the erts allocators would fix. Your application is simply using a lot of memory and it seems to be process heap memory. I would recommend using observer to see which processes use a lot of memory and from there try to figure out why. Lukas On Mon, Apr 4, 2016 at 11:53 PM, Jordan Day wrote: > I?m facing an issue with a program that is using hackney to download a > number of large-ish (~8MB) JSON responses. OS-reported memory consumption > is typically much larger than I?d expect (I?ve seen spikes of several GB), > so I?ve spent quite a bit of time digging through the mailing list and blog > posts trying to gain some insight into my problem. There have been several > threads discussing the ?large binary? problem, which seems like it might be > what I?m facing, but a lot of the suggested remedies (lowering > fullsweep_after, manually running garbage_collect/1) haven?t seemed to have > much effect. After watching Lukas Larsson?s talk from EUC ( > https://www.youtube.com/watch?v=YuPaX11vZyI), I?m wondering about > tweaking allocator settings, but I?m really not sure where to start? While > the problem I?m facing seems a lot like the ?large binary problem?, > observer is reporting that most of my memory usage is tied up in > eheap_alloc (https://imgur.com/OzPLvF0). I?ve taken some samples of the > memory info over time using recon?s snapshot tool, which can be seen here: > https://gist.github.com/jordan0day/3a127046514a5a9efb87665223e985c7 > https://gist.github.com/jordan0day/679e5cc546cdfb4b8c7431f83305b86f > https://gist.github.com/jordan0day/b2f6e69733732f35b612ab810a330ea5 > > These samples were taken just a minute or two apart from each other, > spanning about 30 requests total, so I?d expect the growth to be around 240 > MB or so, but you can see by the 3rd snapshot total memory usage is around > 1.3 GB. > > Everything I?ve read so far has basically said ?don?t mess with the > allocator settings unless you know what you?re doing!? ? and I definitely > don?t know what I?m doing, but I?m really hoping to learn! I?d appreciate > any guidance anyone can offer. > > The information contained in this message is confidential information and > privileged. It is intended only for the use of the individual or > individuals or entity identified above. You are hereby notified that any > dissemination, distribution, use or copying of this message is strictly > prohibited. > > If the receiver of this message is not the intended recipient, you are > hereby notified that any dissemination, distribution, use or copying of > this message is strictly prohibited. If you have received this message in > error, please immediately notify the sender by replying to his/her e-mail > address noted above and delete the message. Thank You ?? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sg2342@REDACTED Tue Apr 5 10:49:53 2016 From: sg2342@REDACTED (Stefan Grundmann) Date: Tue, 5 Apr 2016 08:49:53 +0000 Subject: [erlang-questions] Should we deprecate or modernize log_mf_h and rb? In-Reply-To: References: Message-ID: <20160405084953.GA32163@seth> Hi log_mf_h is used in appliance systems we build at $work because it implements binary logging and log rotation and because it is part of OTP. We also have projects where lager is used but only because it was pulled as a dependency of $other_stuff and we did not yet spend the time to replace it. sg From adrian_lewis@REDACTED Tue Apr 5 11:33:52 2016 From: adrian_lewis@REDACTED (adrian_lewis@REDACTED) Date: Tue, 05 Apr 2016 10:33:52 +0100 Subject: [erlang-questions] Command-line application libraries? Message-ID: <20160405093353.16C83E069F@smtp.hushmail.com> Hi, I wondered if Erlang has any libraries to support building command-line applications? Many Thanks Aidy -------------- next part -------------- An HTML attachment was scrubbed... URL: From siraaj@REDACTED Tue Apr 5 13:34:03 2016 From: siraaj@REDACTED (Siraaj Khandkar) Date: Tue, 5 Apr 2016 07:34:03 -0400 Subject: [erlang-questions] Command-line application libraries? In-Reply-To: <20160405093353.16C83E069F@smtp.hushmail.com> References: <20160405093353.16C83E069F@smtp.hushmail.com> Message-ID: > On Apr 5, 2016, at 05:33, adrian_lewis@REDACTED wrote: > > Hi, > > I wondered if Erlang has any libraries to support building command-line applications? 1) https://github.com/jcomellas/getopt 2) rebar escriptize 3) profit :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Tue Apr 5 13:58:01 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Tue, 5 Apr 2016 07:58:01 -0400 Subject: [erlang-questions] Any guidance on tweaking allocator settings? In-Reply-To: References: <4EC0DE36-BFA6-4423-81AB-0BB5D6078CB9@companykitchen.com> Message-ID: <20160405115800.GA3319@ferdmbp.local> On 04/05, Lukas Larsson wrote: >Hello, > >If you use recon_alloc:memory(usage) on the snapshots you have you can see >you have a high usage percentage which means that you do not have any >memory fragmentation issues that tweaking the erts allocators would fix. >Your application is simply using a lot of memory and it seems to be process >heap memory. I would recommend using observer to see which processes use a >lot of memory and from there try to figure out why. > To add to this, I've got a full section on this stuff in Erlang in Anger (http://www.erlang-in-anger.com/). Look at Chapter 7 for memory issues, and 7.3 for a specific focus on allocators. That hopefully may prove helpful. Regards, Fred. From gguthrie@REDACTED Tue Apr 5 13:58:58 2016 From: gguthrie@REDACTED (Gordon Guthrie) Date: Tue, 5 Apr 2016 12:58:58 +0100 Subject: [erlang-questions] Command-line application libraries? In-Reply-To: <20160405093353.16C83E069F@smtp.hushmail.com> References: <20160405093353.16C83E069F@smtp.hushmail.com> Message-ID: <2FDCFDEC-23ED-440B-9981-CEBFD6F029E2@basho.com> Basho have a CLI library called Clique that you might want to look at https://github.com/basho/clique El Gordo! > On 5 Apr 2016, at 10:33, adrian_lewis@REDACTED wrote: > > Hi, > > I wondered if Erlang has any libraries to support building command-line applications? > > Many Thanks > > Aidy > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From andre@REDACTED Tue Apr 5 14:08:49 2016 From: andre@REDACTED (Andre Graf) Date: Tue, 5 Apr 2016 14:08:49 +0200 Subject: [erlang-questions] Command-line application libraries? In-Reply-To: References: <20160405093353.16C83E069F@smtp.hushmail.com> Message-ID: <5703AAD1.9090702@dergraf.org> Hi, We're using https://github.com/basho/clique a lot for writing cli administration tools for VerneMQ. Best, Andre On 05.04.2016 13:34, Siraaj Khandkar wrote: > On Apr 5, 2016, at 05:33, adrian_lewis@REDACTED > wrote: > >> Hi, >> >> I wondered if Erlang has any libraries to support building >> command-line applications? > > 1) https://github.com/jcomellas/getopt > 2) rebar escriptize > 3) profit :) > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From roberto@REDACTED Tue Apr 5 14:27:16 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 5 Apr 2016 14:27:16 +0200 Subject: [erlang-questions] Guards? Message-ID: All, I'm a little stumped and I feel I'm missing something sooooooo beginner here. 1> F = fun(X) when (is_integer(X) or is_float(X)) and X < 5 -> ok end. #Fun 2> F(1). ** exception error: no function clause matching erl_eval:'-inside-an-interpreted-fun-'(1) 3> F2 = fun(X) when (is_integer(X) or is_float(X)) and (X < 5) -> ok end. #Fun 4> F2(1). ok ...Can some kind soul clarify? Thank you, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Tue Apr 5 14:30:08 2016 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Tue, 5 Apr 2016 14:30:08 +0200 Subject: [erlang-questions] Guards? In-Reply-To: References: Message-ID: <5703AFD0.6030302@ericsson.com> For F, did you mean: ((is_integer(X) or is_float(X)) and X) < 5 On 04/05/2016 02:27 PM, Roberto Ostinelli wrote: > All, > I'm a little stumped and I feel I'm missing something sooooooo > beginner here. > > 1> F = fun(X) when (is_integer(X) or is_float(X)) and X < 5 -> ok end. > #Fun > 2> F(1). > ** exception error: no function clause matching > erl_eval:'-inside-an-interpreted-fun-'(1) > 3> F2 = fun(X) when (is_integer(X) or is_float(X)) and (X < 5) -> ok end. > #Fun > 4> F2(1). > ok > > ...Can some kind soul clarify? > > Thank you, > r. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Tue Apr 5 14:34:00 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 5 Apr 2016 14:34:00 +0200 Subject: [erlang-questions] Guards? In-Reply-To: <5703AFD0.6030302@ericsson.com> References: <5703AFD0.6030302@ericsson.com> Message-ID: On Tue, Apr 5, 2016 at 2:30 PM, Bengt Kleberg wrote: > For F, did you mean: > ((is_integer(X) or is_float(X)) and X) < 5 > Not really, no :) 1> F = fun(X) when ((is_integer(X) or is_float(X)) and X) < 5 -> ok end. #Fun 13> F(1). ** exception error: no function clause matching erl_eval:'-inside-an-interpreted-fun-'(1) -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Tue Apr 5 14:38:55 2016 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 5 Apr 2016 14:38:55 +0200 Subject: [erlang-questions] Guards? In-Reply-To: References: Message-ID: <5703B1DF.6080600@cs.ntua.gr> On 04/05/2016 02:27 PM, Roberto Ostinelli wrote: > I'm a little stumped and I feel I'm missing something sooooooo beginner > here. > > 1> F = fun(X) when (is_integer(X) or is_float(X)) and X < 5 -> ok end. > #Fun > 2> F(1). > ** exception error: no function clause matching > erl_eval:'-inside-an-interpreted-fun-'(1) > 3> F2 = fun(X) when (is_integer(X) or is_float(X)) and (X < 5) -> ok end. > #Fun > 4> F2(1). > ok > > ...Can some kind soul clarify? It's a long story. Its short version is that you will have a much better state of mind if you forget the presence of 'or' and 'and' in guards and use ';' and ',' when you can, and 'orelse' and 'andalso' when you cannot (as in your F fun). 1> F = fun(X) when (is_integer(X) orelse is_float(X)) andalso X < 5 -> ok enD. #Fun 2> F(1). ok Kostis From roger@REDACTED Tue Apr 5 14:50:55 2016 From: roger@REDACTED (Roger Lipscombe) Date: Tue, 5 Apr 2016 13:50:55 +0100 Subject: [erlang-questions] Command-line application libraries? In-Reply-To: <5703AAD1.9090702@dergraf.org> References: <20160405093353.16C83E069F@smtp.hushmail.com> <5703AAD1.9090702@dergraf.org> Message-ID: Slight threadjack: Clique talks about registering cuttlefish schemas; we're not using cuttlefish. Any concerns? On 5 April 2016 at 13:08, Andre Graf wrote: > Hi, > > We're using https://github.com/basho/clique a lot for writing cli > administration tools for VerneMQ. > > Best, > Andre > > > On 05.04.2016 13:34, Siraaj Khandkar wrote: >> On Apr 5, 2016, at 05:33, adrian_lewis@REDACTED >> wrote: >> >>> Hi, >>> >>> I wondered if Erlang has any libraries to support building >>> command-line applications? >> >> 1) https://github.com/jcomellas/getopt >> 2) rebar escriptize >> 3) profit :) >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From drohrer@REDACTED Tue Apr 5 14:56:33 2016 From: drohrer@REDACTED (Douglas Rohrer) Date: Tue, 5 Apr 2016 08:56:33 -0400 Subject: [erlang-questions] Guards? In-Reply-To: <5703B1DF.6080600@cs.ntua.gr> References: <5703B1DF.6080600@cs.ntua.gr> Message-ID: <1A6B929A-17D7-4768-865A-679CE4EDEF80@basho.com> What you're missing is operator precedence [1] - "and" takes precedence over "<", so your guard clause ends up looking, to the runtime, like what Bengt pointed out - you have the boolean expression `is_inteteger(X) or is_float(X)` followed by and, followed by X - which is evaluated before the < operator. The evaluation of your guard looks something like this: X = 1 (((is_integer(1) or is_float(1)) and 1) < 5 (((true) and 1) < 5) `true and 1` cannot be evaluated (throws an exception, as `and` doesn't take integer arguments). Now, your guard expression threw an exception - what happens then: "If an arithmetic expression, a Boolean expression, a short-circuit expression, or a call to a guard BIF fails (because of invalid arguments), the entire guard fails. If the guard was part of a guard sequence, the next guard in the sequence (that is, the guard following the next semicolon) is evaluated." [2] So, your guard ends up not matching - you have no other function clauses, so you have "no function clause matching." Make sense now? Doug [1] http://erlang.org/doc/reference_manual/expressions.html#id84275 [2] http://erlang.org/doc/reference_manual/expressions.html#id83710 > On Apr 5, 2016, at 8:38 AM, Kostis Sagonas wrote: > > On 04/05/2016 02:27 PM, Roberto Ostinelli wrote: >> I'm a little stumped and I feel I'm missing something sooooooo beginner >> here. >> >> 1> F = fun(X) when (is_integer(X) or is_float(X)) and X < 5 -> ok end. >> #Fun >> 2> F(1). >> ** exception error: no function clause matching >> erl_eval:'-inside-an-interpreted-fun-'(1) >> 3> F2 = fun(X) when (is_integer(X) or is_float(X)) and (X < 5) -> ok end. >> #Fun >> 4> F2(1). >> ok >> >> ...Can some kind soul clarify? > > It's a long story. Its short version is that you will have a much better state of mind if you forget the presence of 'or' and 'and' in guards and use ';' and ',' when you can, and 'orelse' and 'andalso' when you cannot (as in your F fun). > > 1> F = fun(X) when (is_integer(X) orelse is_float(X)) andalso X < 5 -> ok enD. > #Fun > 2> F(1). > ok > > > Kostis > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Tue Apr 5 14:56:59 2016 From: roger@REDACTED (Roger Lipscombe) Date: Tue, 5 Apr 2016 13:56:59 +0100 Subject: [erlang-questions] Guards? In-Reply-To: References: Message-ID: That gets parsed as: F = fun(X) when ((is_integer(X) or is_float(X)) and X) < 5 -> ok end. You can check by looking at the output of erlang:fun_info(F), which shows the parse tree for the guard condition. On 5 April 2016 at 13:27, Roberto Ostinelli wrote: > All, > I'm a little stumped and I feel I'm missing something sooooooo beginner > here. > > 1> F = fun(X) when (is_integer(X) or is_float(X)) and X < 5 -> ok end. > #Fun > 2> F(1). > ** exception error: no function clause matching > erl_eval:'-inside-an-interpreted-fun-'(1) > 3> F2 = fun(X) when (is_integer(X) or is_float(X)) and (X < 5) -> ok end. > #Fun > 4> F2(1). > ok > > ...Can some kind soul clarify? > > Thank you, > r. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From roberto@REDACTED Tue Apr 5 15:06:39 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 5 Apr 2016 15:06:39 +0200 Subject: [erlang-questions] Guards? In-Reply-To: <1A6B929A-17D7-4768-865A-679CE4EDEF80@basho.com> References: <5703B1DF.6080600@cs.ntua.gr> <1A6B929A-17D7-4768-865A-679CE4EDEF80@basho.com> Message-ID: > What you're missing is operator precedence [1] - "and" takes precedence > over "<", so your guard clause ends up looking, to the runtime, like what > Bengt pointed out - you have the boolean expression `is_inteteger(X) or > is_float(X)` followed by and, followed by X - which is evaluated before the > < operator. The evaluation of your guard looks something like this: > > X = 1 > (((is_integer(1) or is_float(1)) and 1) < 5 > > (((true) and 1) < 5) > > `true and 1` cannot be evaluated (throws an exception, as `and` doesn't > take integer arguments). Now, your guard expression threw an exception - > what happens then: > > "If an arithmetic expression, a Boolean expression, a short-circuit > expression, or a call to a guard BIF fails (because of invalid arguments), > the entire guard fails. If the guard was part of a guard sequence, the next > guard in the sequence (that is, the guard following the next semicolon) is > evaluated." [2] > > So, your guard ends up not matching - you have no other function clauses, > so you have "no function clause matching." > > Make sense now? > > Doug > Thank you Doug. Yes - it makes sense. However operator precedence, which I previously checked, do not specify and & or: http://erlang.org/doc/reference_manual/expressions.html#prec Therefore I assumed precedence was the same, and that is confusing. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Tue Apr 5 15:07:36 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Tue, 5 Apr 2016 15:07:36 +0200 Subject: [erlang-questions] Guards? In-Reply-To: References: <5703B1DF.6080600@cs.ntua.gr> <1A6B929A-17D7-4768-865A-679CE4EDEF80@basho.com> Message-ID: > > Thank you Doug. Yes - it makes sense. However operator precedence, which I > previously checked, do not specify and & or: > http://erlang.org/doc/reference_manual/expressions.html#prec > Scratch that. Going to get a coffee. ^^_ -------------- next part -------------- An HTML attachment was scrubbed... URL: From magnus@REDACTED Tue Apr 5 15:07:46 2016 From: magnus@REDACTED (Magnus Henoch) Date: Tue, 5 Apr 2016 14:07:46 +0100 Subject: [erlang-questions] repeatable builds In-Reply-To: References: Message-ID: Debian has included a patch that lets you use the environment variable SOURCE_DATE_EPOCH to fix the compile time, and thus obtain identical output (given the same compiler version and other things): https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=795834 This was briefly discussed on this mailing list: http://erlang.org/pipermail/erlang-questions/2015-January/082699.html Regards, Magnus On Mon, Apr 4, 2016 at 8:59 PM, Joe Armstrong wrote: > Hello, > > I think I've asked this before but cannot find the answer: > > I want the beam file produced by > > $ erl file.erl > > to always have the same sha1 checksum - there was, if I remember > correctly, a hidden flag that removed the time of compilation etc from > the beam code. Any ideas how to do this? > > /Joe > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From drohrer@REDACTED Tue Apr 5 15:12:36 2016 From: drohrer@REDACTED (Douglas Rohrer) Date: Tue, 05 Apr 2016 13:12:36 +0000 Subject: [erlang-questions] Guards? In-Reply-To: References: <5703B1DF.6080600@cs.ntua.gr> <1A6B929A-17D7-4768-865A-679CE4EDEF80@basho.com> Message-ID: I had the same reaction when I first scanned that table - and coffee is a good idea. Glad you got it worked out. Doug On Tue, Apr 5, 2016 at 9:07 AM Roberto Ostinelli wrote: > Thank you Doug. Yes - it makes sense. However operator precedence, which I >> previously checked, do not specify and & or: >> http://erlang.org/doc/reference_manual/expressions.html#prec >> > > Scratch that. Going to get a coffee. ^^_ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Wed Apr 6 01:58:47 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 6 Apr 2016 11:58:47 +1200 Subject: [erlang-questions] Guards? In-Reply-To: References: <5703B1DF.6080600@cs.ntua.gr> <1A6B929A-17D7-4768-865A-679CE4EDEF80@basho.com> Message-ID: <0b82f932-c426-315b-d672-b77facc9e367@cs.otago.ac.nz> Thank you Doug. Yes - it makes sense. However operator precedence, which I previously checked, do not specify and & or: > http://erlang.org/doc/reference_manual/expressions.html#prec Actually, that section of the reference manual DOES specify 'and' and 'or'. They are right there in the 4th and 5th rows of table 8.6. / * div rem band AND + - bor bxor bsl bsr OR xor This makes no sense to me. I mean, it REALLY makes no sense. 'and' and 'or' are for combining "boolean" values. The way to *get* boolean values is to do a comparison. What would have made *sense* would have been to put 'and' and 'andalso' at the same level (and forbid mixing them) and to put 'or' and 'orelse' at the same level (and forbid mixing them). If I were daft enough ever to use 'and' or 'or' in Erlang, this would *certainly* confuse the heck out of me. For the sake of my sanity, I make sure *never* to use them. For what it's worth, R has '&&' and '||' which operate lazily on single logical values, rather like C, and also has '&' and '|', which are strict and vectorised. In R, & and && have the same precedence, and | and || have the same precedence, and confusion would reign supreme if they did not. From schneider@REDACTED Wed Apr 6 08:38:33 2016 From: schneider@REDACTED (Schneider) Date: Wed, 6 Apr 2016 08:38:33 +0200 Subject: [erlang-questions] rebar3 and emacs C-c C-k Message-ID: <5704AEE9.8050501@xs4all.nl> Dear list, I am very much used to typing C-c C-k while editing to find and scroll through the syntax errors. With rebar3, all this doesn't work as well with the src directories getting messy. It's finding the syntax errors I am after. What do you use for this? Frans From andra.dinu@REDACTED Wed Apr 6 12:36:33 2016 From: andra.dinu@REDACTED (Andra Dinu) Date: Wed, 6 Apr 2016 11:36:33 +0100 Subject: [erlang-questions] [ANN] Erlang @ bet365: Past, Present and Future - London Erlang User Group 14 April Message-ID: Hi all, If you're in London on 14 April come to the Erlang User Group at 6.30 pm: Chandru Mullaparthi - Head of Software Architecture at bet365 will be talking about how Erlang got introduced at bet365, what problems it was used to solve and what to do when you hit the limits of traditional SQL databases and technologies. He will also be addressing the current open source initiatives at bet365, how he sees the use of Erlang evolving and what it needs to do stay competitive. RSVP on Meetup: http://www.meetup.com/erlangusergroup/events/229789989/ Thank you, Andra *Andra Dinu* Community & Social In London on 28 April? Come get your Startup tech pack: Elixir, Phoenix, Riak KV ! -------------- next part -------------- An HTML attachment was scrubbed... URL: From rob@REDACTED Wed Apr 6 13:25:00 2016 From: rob@REDACTED (Rob A'Court) Date: Wed, 6 Apr 2016 11:25:00 +0000 Subject: [erlang-questions] Issue using variable length fields with Erlang ODBC Message-ID: When querying variable length fields using ODBC in Erlang, the response returned seems to be gibberish. We can query tables in a MS SQL Server database but if the table contains a VarCharMax or NVarCharMax field (both variable length) then the result returned is not what we expect. For NVarCharMax a binary is returned which has part of the original query and other seemingly random data as if it?s the wrong area of memory. For VarCharMax an empty list is always returned. In our particular scenario we are trying to get a ShowPlanXML from MS SQL Server which comes back as a NVarCharMax and there is no way of converting it to a fixed length field type to work around the issue. The issue does not seem to be with the ODBC driver as trying the same thing in python works fine. Here is what we are trying to do in Elixir: :odbc.start {:ok, connection} = :odbc.connect('Driver={ODBC Driver 11 for SQL Server}; Server=TheServer;Uid=sa;Pwd=password;Database=TheDatabase',[]) IO.inspect :odbc.sql_query(connection, 'set showplan_xml on') IO.inspect :odbc.sql_query(connection, 'Select * from customers'), limit: 9000 Here is the equivalent in python that works fine: #!/usr/bin/env python import pyodbc conn = pyodbc.connect('Driver={ODBC Driver 11 for SQL Server}; Server=TheServer;Uid=sa;Pwd=password;Database=TheDatabase') cur = conn.cursor() cur.execute('set showplan_xml on') cur.execute('Select * from customers') for row in cur : print row Many thanks! Rob -------------- next part -------------- An HTML attachment was scrubbed... URL: From Catenacci@REDACTED Wed Apr 6 17:58:26 2016 From: Catenacci@REDACTED (Onorio Catenacci) Date: Wed, 6 Apr 2016 11:58:26 -0400 Subject: [erlang-questions] Patch package OTP 18.3.1 released Message-ID: > > > Message: 1 > Date: Mon, 4 Apr 2016 16:24:19 +0200 > From: Henrik Nord X > To: "Erlang (E-mail)" > Subject: [erlang-questions] Patch package OTP 18.3.1 released > Message-ID: <57027913.1060609@REDACTED> > Content-Type: text/plain; charset="utf-8"; format=flowed > > Patch Package: OTP 18.3.1 > Git Tag: OTP-18.3.1 > Date: 2016-04-04 > Trouble Report Id: OTP-13417, OTP-13418, OTP-13419, OTP-13420, > OTP-13423, OTP-13424, OTP-13446, OTP-13452 > Seq num: > System: OTP > Release: 18 > Application: erts-7.3.1, inets-6.2.1, mnesia-4.13.4 > Predecessor: OTP 18.3 > > Check out the git tag OTP-18.3.1, and build a full OTP system > including documentation. Apply one or more applications from this > build as patches to your installation using the 'otp_patch_apply' > tool. For information on install requirements, see descriptions for > each application version below. > Hi Henrik, Will 18.3.1 be available on the Erlang downloads page? http://www.erlang.org/downloads I'm asking since 18.2.1 was added so I was a little surprised that I didn't find 18.3.1 there as well. -- Onorio Catenacci http://onor.io http://www.google.com/+OnorioCatenacci -------------- next part -------------- An HTML attachment was scrubbed... URL: From tim@REDACTED Thu Apr 7 00:03:52 2016 From: tim@REDACTED (Tim Stewart) Date: Wed, 6 Apr 2016 18:03:52 -0400 Subject: [erlang-questions] Should we deprecate or modernize log_mf_h and rb? In-Reply-To: <20160405084953.GA32163@seth> References: <20160405084953.GA32163@seth> Message-ID: Hello, I use log_mf_h and rb every day, and I am quite fond of them. My systems are usually configured with both lager and log_mf_h. SASL + log_mf_h pick up all of the "core" OTP-based reports (info, warning, error, crash, supervisor, progress) and lager is used by our applications for application-level logging. Since lager also picks up reports and messages created by error_logger, our support team can monitor only a single *line-based* log file to find problems. Then, when we get actual crashes I turn to report browser for nicely formatted reports. In my opinion, rb's formatted reports are easier to read than lager's crash.log. I have error_logger configured to use only lager and log_mf_h handlers so that we are relatively protected from a high error_logger volume. And, as ?ric Pailleau mentioned, log_mf_h is the backend for Observer's log support. I also use this feature. I would be sad to see log_mf_h disappear. I would *love* to see more GUI support for reading and searching through these logs (additional Observer support, web UI, etc.) Anyway, I hope this helps. -TimS On Tue, Apr 5, 2016 at 4:49 AM, Stefan Grundmann wrote: > Hi > > log_mf_h is used in appliance systems we build at $work because it > implements binary logging and log rotation and because it is > part of OTP. > We also have projects where lager is used but only because it > was pulled as a dependency of $other_stuff and we did not yet spend the > time > to replace it. > > sg > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Tim Stewart tim@REDACTED +1 (404) 993-6492 -------------- next part -------------- An HTML attachment was scrubbed... URL: From liuzhongzheng2012@REDACTED Thu Apr 7 05:47:19 2016 From: liuzhongzheng2012@REDACTED (Zhongzheng Liu) Date: Thu, 7 Apr 2016 11:47:19 +0800 Subject: [erlang-questions] Connection of nodes with different cookie Message-ID: Hi mail list: I started two nodes in my pc: erl -name 'a@REDACTED' -setcookie aa erl -name 'b@REDACTED' -setcookie bb If I run erlang:set_cookie('b@REDACTED', bb) in node a, a and b is connectable with each other. But if i run both erlang:set_cookie('b@REDACTED', bb) in node a and erlang:set_cookie('a@REDACTED', aa) in node b before i connect a and b, the connection will fail. I found the code in dist_util.erl: %% %% Get the cookies for a node from auth %% get_cookies(Node) -> case auth:get_cookie(Node) of X when is_atom(X) -> {X,X} % {Y,Z} when is_atom(Y), is_atom(Z) -> % {Y,Z}; % _ -> % erlang:error("Corrupt cookie database") end. This function should return {MyCookie,HisCookie} as the other code written in this module, but currently it just simply make MyCookie and HisCookie the same one. I guess this design is for the convenience when you connecting a node having different cookie. In general you are not able to run erlang:set_cookie/2 before you connect it, you can only change your own cookie to connect other node. But I think connection should also success if cookie is set by erlang:set_cookie/2 in both side for each other. Do you think so? ------ Liu Zhongzheng -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Thu Apr 7 10:33:56 2016 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 7 Apr 2016 10:33:56 +0200 Subject: [erlang-questions] Erlang on VAX Message-ID: Hello, Can anybody remember the vital statistics of the original VAX that Erlang was developed on - I think it was a VAX 11/750. I'd like to know the basic vital statistics. Clock rate, RAM and disk volumes. OS version etc. Cheers /Joe From lukas@REDACTED Thu Apr 7 10:36:37 2016 From: lukas@REDACTED (Lukas Larsson) Date: Thu, 7 Apr 2016 10:36:37 +0200 Subject: [erlang-questions] Erlang tracing In-Reply-To: References: Message-ID: Hello again! On Mon, Sep 21, 2015 at 10:52 AM, Lukas Larsson wrote: > Hello everyone. > > As you may know, one of the OTP teams focus areas for the coming year is > make tracing better. At the moment we are gathering ideas and attempting to > put together a vision of what we would like to have, before deciding what > we can make. > > The first step of this work is now nearing it's completion. I've just opened up a new pull request on github ( https://github.com/erlang/otp/pull/1009) with part of the changes that will go into Erlang/OTP 19.0. I would urge you all (especially if you are a tracing tool maintainer) to check out the changes and come back with feedback to me. Either here, in the pull request or privately if you prefer. There is still quite a lot of work that can and will be done in the tracing area, but this, I believe, is a good first step. Thanks in advance, Lukas Erlang/OTP team -------------- next part -------------- An HTML attachment was scrubbed... URL: From henrik.x.nord@REDACTED Thu Apr 7 10:54:31 2016 From: henrik.x.nord@REDACTED (Henrik Nord X) Date: Thu, 7 Apr 2016 10:54:31 +0200 Subject: [erlang-questions] Patch package OTP 18.3.1 released In-Reply-To: References: Message-ID: <57062047.50606@ericsson.com> Hello! No it will not be available on erlang.org. We usually only upload the 3 planned patch packages and the new major releases there. However if there is something that warrants a new upload we will do that. 18.2.1 was a special case as none of the executables worked on windows due to a bug in path handling. And as few actually build Erlang from source on windows we decided to fix the bug and re upload that fix. 18.2.1 compared to 18.2 is only that bug fix for windows, and a small fix for HiPE on FreeBSD. So TLDR Nope 18.2.1 was a special case. You are however able to find all README files from all patch packages OTP-17 and onwards in http://erlang.org/download/ Regards Henrik Nord Erlang/OTP On 04/06/2016 05:58 PM, Onorio Catenacci wrote: > > > Message: 1 > Date: Mon, 4 Apr 2016 16:24:19 +0200 > From: Henrik Nord X > > To: "Erlang (E-mail)" > > Subject: [erlang-questions] Patch package OTP 18.3.1 released > Message-ID: <57027913.1060609@REDACTED > > > Content-Type: text/plain; charset="utf-8"; format=flowed > > Patch Package: OTP 18.3.1 > Git Tag: OTP-18.3.1 > Date: 2016-04-04 > Trouble Report Id: OTP-13417, OTP-13418, OTP-13419, OTP-13420, > OTP-13423, OTP-13424, OTP-13446, OTP-13452 > Seq num: > System: OTP > Release: 18 > Application: erts-7.3.1, inets-6.2.1, mnesia-4.13.4 > Predecessor: OTP 18.3 > > Check out the git tag OTP-18.3.1, and build a full OTP system > including documentation. Apply one or more applications from this > build as patches to your installation using the 'otp_patch_apply' > tool. For information on install requirements, see descriptions for > each application version below. > > > > Hi Henrik, > > Will 18.3.1 be available on the Erlang downloads page? > http://www.erlang.org/downloads > > I'm asking since 18.2.1 was added so I was a little surprised that I > didn't find 18.3.1 there as well. > > -- > Onorio Catenacci > > http://onor.io > http://www.google.com/+OnorioCatenacci > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From s.j.thompson@REDACTED Thu Apr 7 12:10:38 2016 From: s.j.thompson@REDACTED (Simon Thompson) Date: Thu, 7 Apr 2016 11:10:38 +0100 Subject: [erlang-questions] Erlang tracing In-Reply-To: References: Message-ID: Great stuff! Thanks Simon > On 7 Apr 2016, at 09:36, Lukas Larsson wrote: > > Hello again! > > On Mon, Sep 21, 2015 at 10:52 AM, Lukas Larsson > wrote: > Hello everyone. > > As you may know, one of the OTP teams focus areas for the coming year is make tracing better. At the moment we are gathering ideas and attempting to put together a vision of what we would like to have, before deciding what we can make. > > > The first step of this work is now nearing it's completion. I've just opened up a new pull request on github (https://github.com/erlang/otp/pull/1009 ) with part of the changes that will go into Erlang/OTP 19.0. > > I would urge you all (especially if you are a tracing tool maintainer) to check out the changes and come back with feedback to me. Either here, in the pull request or privately if you prefer. > > There is still quite a lot of work that can and will be done in the tracing area, but this, I believe, is a good first step. > > Thanks in advance, > Lukas > Erlang/OTP team > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions Simon Thompson | Professor of Logic and Computation School of Computing | University of Kent | Canterbury, CT2 7NF, UK s.j.thompson@REDACTED | M +44 7986 085754 | W www.cs.kent.ac.uk/~sjt -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjarne.dacker@REDACTED Thu Apr 7 13:04:26 2016 From: bjarne.dacker@REDACTED (=?utf-8?Q?Bjarne_D=C3=A4cker?=) Date: Thu, 7 Apr 2016 13:04:26 +0200 Subject: [erlang-questions] Erlang on VAX In-Reply-To: References: Message-ID: Sorry, I have no such documentation. Glad v?r Bjarne > Hello, > > Can anybody remember the vital statistics of the original VAX that > Erlang was developed on - I think it was a VAX 11/750. > > I'd like to know the basic vital statistics. Clock rate, RAM and disk > volumes. OS version etc. > > Cheers > > /Joe -------------- next part -------------- An HTML attachment was scrubbed... URL: From Catenacci@REDACTED Thu Apr 7 13:55:32 2016 From: Catenacci@REDACTED (Onorio Catenacci) Date: Thu, 7 Apr 2016 07:55:32 -0400 Subject: [erlang-questions] Patch package OTP 18.3.1 released Message-ID: > > > Hello! > > No it will not be available on erlang.org. > We usually only upload the 3 planned patch packages and the new major > releases there. > > However if there is something that warrants a new upload we will do that. > > 18.2.1 was a special case as none of the executables worked on windows > due to a bug in path handling. And as few actually build Erlang from > source on windows we decided to fix the bug and re upload that fix. > > 18.2.1 compared to 18.2 is only that bug fix for windows, and a small > fix for HiPE on FreeBSD. > > > > So TLDR > > Nope > 18.2.1 was a special case. > > You are however able to find all README files from all patch packages > OTP-17 and onwards in > http://erlang.org/download/ > > > Regards > Henrik Nord > Erlang/OTP > Thanks for the answer and the explanation Henrik! > -- Onorio Catenacci http://onor.io http://www.google.com/+OnorioCatenacci -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.james.morgan@REDACTED Thu Apr 7 16:14:31 2016 From: peter.james.morgan@REDACTED (Peter Morgan) Date: Thu, 7 Apr 2016 15:14:31 +0100 Subject: [erlang-questions] udp multicast: problem with OSX? Message-ID: Hello - I?m having some trouble getting UDP multicast working on OSX, whereas the same code on Linux works fine. On linux (fedora 23): [pmorgan@REDACTED ~]$ erl Erlang/OTP 18 [erts-7.3] [source] [64-bit] [async-threads:10] [hipe] [kernel-poll:false] Eshell V7.3 (abort with ^G) 1> Address = {224, 0, 0, 251}. {224,0,0,251} 2> Port = 5353. 5353 3> {ok, Socket} = gen_udp:open(Port, [binary, {ip, Address}, {add_membership, {Address, {0,0,0,0}}}, {reuseaddr, true}]). {ok,#Port<0.543>} 4> gen_udp:send(Socket, Address, Port, <<"hello world">>). ok Whereas on OSX: Office-iMac:mdns pmorgan$ sw_vers ProductName: Mac OS X ProductVersion: 10.11.4 BuildVersion: 15E61b Office-iMac:mdns pmorgan$ erl Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace] Eshell V7.3 (abort with ^G) 1> Address = {224, 0, 0, 251}. {224,0,0,251} 2> Port = 5353. 5353 3> {ok, Socket} = gen_udp:open(Port, [binary, {ip, Address}, {add_membership, {Address, {0,0,0,0}}}, {reuseaddr, true}]). {ok,#Port<0.553>} 4> gen_udp:send(Socket, Address, Port, <<"hello world">>). {error,eaddrnotavail} I?ve tried various other options in gen_udp:open/2, but not found a combination that works for OSX. Any ideas please? The above is a simplified test case of https://github.com/shortishly/mdns/blob/master/src/mdns_udp.erl#L41-L49 Thanks, Peter. From radek@REDACTED Thu Apr 7 16:39:26 2016 From: radek@REDACTED (Rad Gruchalski) Date: Thu, 7 Apr 2016 16:39:26 +0200 Subject: [erlang-questions] udp multicast: problem with OSX? In-Reply-To: References: Message-ID: Hi Peter, Multicast works absolutely fine on OSX. Here is how multicast is enabled: https://github.com/gossiperl/gossiperl/wiki/multicast-overlays#setting-up-multicast-on-os-x And then using the multicast options: https://github.com/gossiperl/gossiperl/blob/master/include/gossiperl.hrl#L107 Works perfect on Linux, OSX and RPi. Best regards, Radek Gruchalski radek@REDACTED (mailto:radek@REDACTED) (mailto:radek@REDACTED) de.linkedin.com/in/radgruchalski/ (http://de.linkedin.com/in/radgruchalski/) Confidentiality: This communication is intended for the above-named person and may be confidential and/or legally privileged. If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately. On Thursday, 7 April 2016 at 16:14, Peter Morgan wrote: > Hello - > > I?m having some trouble getting UDP multicast working on OSX, whereas the same code on Linux works fine. > > On linux (fedora 23): > > [pmorgan@REDACTED ~]$ erl > Erlang/OTP 18 [erts-7.3] [source] [64-bit] [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V7.3 (abort with ^G) > 1> Address = {224, 0, 0, 251}. > {224,0,0,251} > 2> Port = 5353. > 5353 > 3> {ok, Socket} = gen_udp:open(Port, [binary, {ip, Address}, {add_membership, {Address, {0,0,0,0}}}, {reuseaddr, true}]). > {ok,#Port<0.543>} > 4> gen_udp:send(Socket, Address, Port, <<"hello world">>). > ok > > > Whereas on OSX: > > Office-iMac:mdns pmorgan$ sw_vers > ProductName: Mac OS X > ProductVersion: 10.11.4 > BuildVersion: 15E61b > > Office-iMac:mdns pmorgan$ erl > Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace] > > Eshell V7.3 (abort with ^G) > 1> Address = {224, 0, 0, 251}. > {224,0,0,251} > 2> Port = 5353. > 5353 > 3> {ok, Socket} = gen_udp:open(Port, [binary, {ip, Address}, {add_membership, {Address, {0,0,0,0}}}, {reuseaddr, true}]). > {ok,#Port<0.553>} > 4> gen_udp:send(Socket, Address, Port, <<"hello world">>). > {error,eaddrnotavail} > > > I?ve tried various other options in gen_udp:open/2, but not found a combination that works for OSX. Any ideas please? The above is a simplified test case of https://github.com/shortishly/mdns/blob/master/src/mdns_udp.erl#L41-L49 > > Thanks, > Peter. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED (mailto:erlang-questions@REDACTED) > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Thu Apr 7 16:47:05 2016 From: comptekki@REDACTED (Wes James) Date: Thu, 7 Apr 2016 08:47:05 -0600 Subject: [erlang-questions] Patch package OTP 18.3.1 released In-Reply-To: References: Message-ID: On Thu, Apr 7, 2016 at 5:55 AM, Onorio Catenacci wrote: > >> Hello! >> >> No it will not be available on erlang.org. >> We usually only upload the 3 planned patch packages and the new major >> releases there > > > > > > Thanks for the answer and the explanation Henrik! > > >> -- > > Onorio Catenacci > Do you need a specific prebuilt package or can you download the source and compile it? If source works, you can follow these instructions: https://github.com/erlang/otp/wiki/Installation -------------- next part -------------- An HTML attachment was scrubbed... URL: From Catenacci@REDACTED Thu Apr 7 16:56:39 2016 From: Catenacci@REDACTED (Onorio Catenacci) Date: Thu, 7 Apr 2016 10:56:39 -0400 Subject: [erlang-questions] Patch package OTP 18.3.1 released Message-ID: Thanks Wes--it's actually neither in this case. I maintain the Chocolatey NuGet package for Erlang (CNG is like apt-get but for Windows) and I usually simply point at the Erlang.org site for the installer. -- Onorio On Thu, Apr 7, 2016 at 10:47 AM, Wes James wrote: > On Thu, Apr 7, 2016 at 5:55 AM, Onorio Catenacci > wrote: > >> >>> Hello! >>> >>> No it will not be available on erlang.org. >>> We usually only upload the 3 planned patch packages and the new major >>> releases there >> >> > > > > >> >> >> >> Thanks for the answer and the explanation Henrik! >> >> >>> -- >> >> Onorio Catenacci >> > > Do you need a specific prebuilt package or can you download the source and > compile it? > > If source works, you can follow these instructions: > > https://github.com/erlang/otp/wiki/Installation > -- Onorio Catenacci http://onor.io http://www.google.com/+OnorioCatenacci -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Thu Apr 7 17:02:14 2016 From: comptekki@REDACTED (Wes James) Date: Thu, 7 Apr 2016 09:02:14 -0600 Subject: [erlang-questions] Patch package OTP 18.3.1 released In-Reply-To: References: Message-ID: On Thu, Apr 7, 2016 at 8:56 AM, Onorio Catenacci wrote: > Thanks Wes--it's actually neither in this case. I maintain the Chocolatey > NuGet package for Erlang (CNG is like apt-get but for Windows) and I > usually simply point at the Erlang.org site for the installer. > > -- > Onorio > > On Thu, Apr 7, 2016 at 10:47 AM, Wes James wrote: > >> On Thu, Apr 7, 2016 at 5:55 AM, Onorio Catenacci >> wrote: >> >>> >>>> Hello! >>>> >>>> No it will not be available on erlang.org. >>>> We usually only upload the 3 planned patch packages and the new major >>>> releases there >>> >>> >> >> >> >> >>> >>> >>> >>> Thanks for the answer and the explanation Henrik! >>> >>> >>>> -- >>> >>> Onorio Catenacci >>> >> >> Do you need a specific prebuilt package or can you download the source >> and compile it? >> >> If source works, you can follow these instructions: >> >> https://github.com/erlang/otp/wiki/Installation >> > > > > Ah. Ok. Guess you'll need to wait. Good luck. -wes -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Thu Apr 7 17:06:51 2016 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 7 Apr 2016 17:06:51 +0200 Subject: [erlang-questions] Erlang on VAX In-Reply-To: References: Message-ID: On Thu, Apr 7, 2016 at 1:36 PM, Mike Williams wrote: > Probably 4.1 BSD UNIX, maybe 4.2. > > We bought the VAX with (I think) 4 MBytes of memory and upgraded to 16Mbytes > of memory. When was the upgrade bought - at the same time as the original or later? I'm trying to figure out the configuration when we made the first erlang version (so I can compare with a raspberry pi today :-) /Joe > > From the Wikipedia: > "The VAX-11/750, code-named "Comet", was a more compact, lower-performance > TTL gate array?based implementation of the VAX architecture introduced in > October 1980. The CPU had a 320 ns cycle time (3.125 MHz)" > > This fits with what I remember. We had 24 RS232 terminal connections, some > current loop and some for modems, and mainly VT100 terminals and moved on to > Facit Twists. > > We started with one RM03 67 Mbyte disk and later added two RP06 256 Mbyte > disks and even later added a Fujitsu Eagle 640 Mbyte disk. We also had a DEC > tape drive and moved on to a Fujitsu tape drive. There were some sort of > cartridge tape drives (small built in one and a larger one taking 3M Tapes) > , I can't remember what. > > For the initial JAM used for the Mobility Server (before ETS and > distribution added by Klacke) I wrote somewhere between 10K and 15K lines of > C. The Mobility Server had an Industrialised SUN computer initially 68030 > (68040??) but later Sparc from Force Computers (From Munchen / Munich) and 4 > Mytes of memory. > > /m > > > > > > On 7 April 2016 at 12:04, Bjarne D?cker > wrote: >> >> Sorry, I have no such documentation. >> >> Glad v?r >> >> Bjarne >> >> >> Hello, >> >> Can anybody remember the vital statistics of the original VAX that >> Erlang was developed on - I think it was a VAX 11/750. >> >> I'd like to know the basic vital statistics. Clock rate, RAM and disk >> volumes. OS version etc. >> >> Cheers >> >> /Joe >> >> >> >> > From gguthrie@REDACTED Thu Apr 7 17:19:09 2016 From: gguthrie@REDACTED (Gordon Guthrie) Date: Thu, 7 Apr 2016 16:19:09 +0100 Subject: [erlang-questions] Erlang on VAX In-Reply-To: References: Message-ID: <55D05A84-316A-4C8D-8542-3BBC6929C9D7@basho.com> That memory seems very large for 1980! When I was on the Cray-1X at ULU in ?86 it was *alleged* to be the only machine in the UK with a Mb of RAM? Mere programming peons like me didn?t actually see the computers in those days so anecdote is all I have :-( Gordon > On 7 Apr 2016, at 16:06, Joe Armstrong wrote: > > On Thu, Apr 7, 2016 at 1:36 PM, Mike Williams > wrote: >> Probably 4.1 BSD UNIX, maybe 4.2. >> >> We bought the VAX with (I think) 4 MBytes of memory and upgraded to 16Mbytes >> of memory. > > When was the upgrade bought - at the same time as the original or later? > > I'm trying to figure out the configuration when we made the first > erlang version (so I can compare with a raspberry pi today :-) > > /Joe > >> >> From the Wikipedia: >> "The VAX-11/750, code-named "Comet", was a more compact, lower-performance >> TTL gate array?based implementation of the VAX architecture introduced in >> October 1980. The CPU had a 320 ns cycle time (3.125 MHz)" >> >> This fits with what I remember. We had 24 RS232 terminal connections, some >> current loop and some for modems, and mainly VT100 terminals and moved on to >> Facit Twists. >> >> We started with one RM03 67 Mbyte disk and later added two RP06 256 Mbyte >> disks and even later added a Fujitsu Eagle 640 Mbyte disk. We also had a DEC >> tape drive and moved on to a Fujitsu tape drive. There were some sort of >> cartridge tape drives (small built in one and a larger one taking 3M Tapes) >> , I can't remember what. >> >> For the initial JAM used for the Mobility Server (before ETS and >> distribution added by Klacke) I wrote somewhere between 10K and 15K lines of >> C. The Mobility Server had an Industrialised SUN computer initially 68030 >> (68040??) but later Sparc from Force Computers (From Munchen / Munich) and 4 >> Mytes of memory. >> >> /m >> >> >> >> >> >> On 7 April 2016 at 12:04, Bjarne D?cker >> wrote: >>> >>> Sorry, I have no such documentation. >>> >>> Glad v?r >>> >>> Bjarne >>> >>> >>> Hello, >>> >>> Can anybody remember the vital statistics of the original VAX that >>> Erlang was developed on - I think it was a VAX 11/750. >>> >>> I'd like to know the basic vital statistics. Clock rate, RAM and disk >>> volumes. OS version etc. >>> >>> Cheers >>> >>> /Joe >>> >>> >>> >>> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From kunthar@REDACTED Thu Apr 7 21:02:47 2016 From: kunthar@REDACTED (Gokhan Boranalp) Date: Thu, 7 Apr 2016 22:02:47 +0300 Subject: [erlang-questions] udp multicast: problem with OSX? In-Reply-To: References: Message-ID: Maybe you could bump to El Capitan SIP http://www.macworld.com/article/2986118/security/how-to-modify-system-integrity-protection-in-el-capitan.html I don't know if it covers UDP too. On Thu, Apr 7, 2016 at 5:39 PM, Rad Gruchalski wrote: > Hi Peter, > > Multicast works absolutely fine on OSX. Here is how multicast is enabled: > > https://github.com/gossiperl/gossiperl/wiki/multicast-overlays#setting-up-multicast-on-os-x > > And then using the multicast options: > > https://github.com/gossiperl/gossiperl/blob/master/include/gossiperl.hrl#L107 > > Works perfect on Linux, OSX and RPi. > > Best regards, > Radek Gruchalski > radek@REDACTED > de.linkedin.com/in/radgruchalski/ > > > *Confidentiality:*This communication is intended for the above-named > person and may be confidential and/or legally privileged. > If it has come to you in error you must take no action based on it, nor > must you copy or show it to anyone; please delete/destroy and inform the > sender immediately. > > On Thursday, 7 April 2016 at 16:14, Peter Morgan wrote: > > Hello - > > I?m having some trouble getting UDP multicast working on OSX, whereas the > same code on Linux works fine. > > On linux (fedora 23): > > [pmorgan@REDACTED ~]$ erl > Erlang/OTP 18 [erts-7.3] [source] [64-bit] [async-threads:10] [hipe] > [kernel-poll:false] > > Eshell V7.3 (abort with ^G) > 1> Address = {224, 0, 0, 251}. > {224,0,0,251} > 2> Port = 5353. > 5353 > 3> {ok, Socket} = gen_udp:open(Port, [binary, {ip, Address}, > {add_membership, {Address, {0,0,0,0}}}, {reuseaddr, true}]). > {ok,#Port<0.543>} > 4> gen_udp:send(Socket, Address, Port, <<"hello world">>). > ok > > > Whereas on OSX: > > Office-iMac:mdns pmorgan$ sw_vers > ProductName: Mac OS X > ProductVersion: 10.11.4 > BuildVersion: 15E61b > > Office-iMac:mdns pmorgan$ erl > Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:4:4] [async-threads:10] > [hipe] [kernel-poll:false] [dtrace] > > Eshell V7.3 (abort with ^G) > 1> Address = {224, 0, 0, 251}. > {224,0,0,251} > 2> Port = 5353. > 5353 > 3> {ok, Socket} = gen_udp:open(Port, [binary, {ip, Address}, > {add_membership, {Address, {0,0,0,0}}}, {reuseaddr, true}]). > {ok,#Port<0.553>} > 4> gen_udp:send(Socket, Address, Port, <<"hello world">>). > {error,eaddrnotavail} > > > I?ve tried various other options in gen_udp:open/2, but not found a > combination that works for OSX. Any ideas please? The above is a simplified > test case of > https://github.com/shortishly/mdns/blob/master/src/mdns_udp.erl#L41-L49 > > Thanks, > Peter. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -- BR, \|/ Kunthar -------------- next part -------------- An HTML attachment was scrubbed... URL: From per@REDACTED Fri Apr 8 00:24:48 2016 From: per@REDACTED (Per Hedeland) Date: Fri, 8 Apr 2016 00:24:48 +0200 (CEST) Subject: [erlang-questions] Erlang on VAX In-Reply-To: Message-ID: <201604072224.u37MOmQF038758@pluto.hedeland.org> [I doubt that this is of any real interest to erlang-questions@, but I guess I'll just follow on...] Joe Armstrong wrote: > >On Thu, Apr 7, 2016 at 1:36 PM, Mike Williams > wrote: >> Probably 4.1 BSD UNIX, maybe 4.2. We started with 4.1, but pretty soon upgraded to 4.2 when it was released - that's the one that brought us TCP/IP, yeah! I'm pretty sure that we also went to 4.3, but not further. I know that we *had* the 4.4 release (on a cartridge mind you, not the good old 10.5-inch reels anymore), but I don't think we ever installed it. >> We bought the VAX with (I think) 4 MBytes of memory and upgraded to 16Mbytes >> of memory. My memory (sorry for the pun) is exactly half of those numbers, and I'm pretty sure about that too - specifically that we had to go from 2 to a whopping 8 MBytes when we had upgraded to 4.2, because it did much more agressive caching - with "too little" RAM, 4.2 was significantly slower than 4.1. >When was the upgrade bought - at the same time as the original or later? So the RAM upgrade should have been some time after the 4.2 release, probably 1984ish, maybe 1985. --Per >I'm trying to figure out the configuration when we made the first >erlang version (so I can compare with a raspberry pi today :-) > >/Joe > >> >> From the Wikipedia: >> "The VAX-11/750, code-named "Comet", was a more compact, lower-performance >> TTL gate array???based implementation of the VAX architecture introduced in >> October 1980. The CPU had a 320 ns cycle time (3.125 MHz)" >> >> This fits with what I remember. We had 24 RS232 terminal connections, some >> current loop and some for modems, and mainly VT100 terminals and moved on to >> Facit Twists. >> >> We started with one RM03 67 Mbyte disk and later added two RP06 256 Mbyte >> disks and even later added a Fujitsu Eagle 640 Mbyte disk. We also had a DEC >> tape drive and moved on to a Fujitsu tape drive. There were some sort of >> cartridge tape drives (small built in one and a larger one taking 3M Tapes) >> , I can't remember what. >> >> For the initial JAM used for the Mobility Server (before ETS and >> distribution added by Klacke) I wrote somewhere between 10K and 15K lines of >> C. The Mobility Server had an Industrialised SUN computer initially 68030 >> (68040??) but later Sparc from Force Computers (From Munchen / Munich) and 4 >> Mytes of memory. >> >> /m >> >> >> >> >> >> On 7 April 2016 at 12:04, Bjarne D??cker >> wrote: >>> >>> Sorry, I have no such documentation. >>> >>> Glad v??r >>> >>> Bjarne >>> >>> >>> Hello, >>> >>> Can anybody remember the vital statistics of the original VAX that >>> Erlang was developed on - I think it was a VAX 11/750. >>> >>> I'd like to know the basic vital statistics. Clock rate, RAM and disk >>> volumes. OS version etc. >>> >>> Cheers >>> >>> /Joe >>> >>> >>> >>> >> > From ok@REDACTED Fri Apr 8 01:47:13 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 8 Apr 2016 11:47:13 +1200 Subject: [erlang-questions] Erlang on VAX In-Reply-To: <55D05A84-316A-4C8D-8542-3BBC6929C9D7@basho.com> References: <55D05A84-316A-4C8D-8542-3BBC6929C9D7@basho.com> Message-ID: According to http://bitsavers.trailing-edge.com/pdf/dec/vax/750/EK-KA750-TD-002_VAX_750_CPU_Technical_Description_Mar81.pdf (note the date) early 750s came with "256 kB of ECC MOS memory" and the machine allowed (1.2.1.2) "up to ... (2 Mbytes of main memory)" and repeats (1.2.2.5) "the maximum system configuration of 2Mbytes". This would be the kind of thing DEC would have expanded, but even so, I expect the machine would have started at 2MB rather than 4MB. 16 MB would have been *enormous* for a 750; people needing that much memory would have done well to buy a faster machine. The 780 was by definition 1 Vax Unit of Processing and did about 500,000 instructions per second. The 750 was 0.65 VUPs, so about 300,000 instructions per second. For what it's worth, the Edinburgh Regional Computer Centre had a timeshared ICL 2980 in 1986. I don't know how much memory it *did* have, but it *could* have at most 8MB. That was the main computing service for Edinburgh University, and was also used by a couple of other places. When I was in Edinburgh in the early 80s they had a DEC-10 with at least a MB of memory. From rdm@REDACTED Fri Apr 8 05:17:08 2016 From: rdm@REDACTED (Rich Morin) Date: Thu, 7 Apr 2016 20:17:08 -0700 Subject: [erlang-questions] how to report errata in the Erlang User's Guide? Message-ID: Looking at http://erlang.org/doc/apps/odbc/databases.html, I ran into a typo ("..., oracle, cybase etc.") I don't see any link on the page for editing or even reporting errata. Wassup? -r -- http://www.cfcl.com/rdm Rich Morin rdm@REDACTED http://www.cfcl.com/rdm/resume San Bruno, CA, USA +1 650-873-7841 Software system design, development, and documentation From akat.metin@REDACTED Thu Apr 7 22:28:16 2016 From: akat.metin@REDACTED (Metin Akat) Date: Thu, 7 Apr 2016 23:28:16 +0300 Subject: [erlang-questions] Any guidance on tweaking allocator settings? In-Reply-To: <20160405115800.GA3319@ferdmbp.local> References: <4EC0DE36-BFA6-4423-81AB-0BB5D6078CB9@companykitchen.com> <20160405115800.GA3319@ferdmbp.local> Message-ID: Hello, I recently had the same problem in quite identical situation. In my case it was a load tester application that hammered onto a REST API. With several thousand bots when hitting a method that returns ~1M JSON, the VM started eating huge amounts of memory, until it crashed with out-of-memory exception. A very simple hack solved my problem (and that being a non-critical application I don't really care bout the implications): [garbage_collect(Pid) || Pid <- processes()]. I execute this every minute or so and my load tester never had the problem again. I imagine somehow it failed to trigger full-sweep GC ever and generational GC never cleared the old parsed JSON terms allocated in the processes' heap. But as it solved my problem, I never dug any deeper. Regards, Metin On Tue, Apr 5, 2016 at 2:58 PM, Fred Hebert wrote: > On 04/05, Lukas Larsson wrote: > >> Hello, >> >> If you use recon_alloc:memory(usage) on the snapshots you have you can see >> you have a high usage percentage which means that you do not have any >> memory fragmentation issues that tweaking the erts allocators would fix. >> Your application is simply using a lot of memory and it seems to be >> process >> heap memory. I would recommend using observer to see which processes use a >> lot of memory and from there try to figure out why. >> >> > To add to this, I've got a full section on this stuff in Erlang in Anger ( > http://www.erlang-in-anger.com/). Look at Chapter 7 for memory issues, > and 7.3 for a specific focus on allocators. > > That hopefully may prove helpful. > > Regards, > Fred. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kenneth@REDACTED Fri Apr 8 07:40:36 2016 From: kenneth@REDACTED (Kenneth Lundin) Date: Fri, 8 Apr 2016 07:40:36 +0200 Subject: [erlang-questions] how to report errata in the Erlang User's Guide? In-Reply-To: References: Message-ID: The documentation is part of the source code and resides in the git repository at Github. You can make a pull request with corrected doc there. Or you can report the doc errors as an issue via the issue tracker at bugs.erlang.org. /Kenneth, Erlang/OTP, Ericsson Den 8 apr 2016 5:18 fm skrev "Rich Morin" : > Looking at http://erlang.org/doc/apps/odbc/databases.html, I ran into > a typo ("..., oracle, cybase etc.") I don't see any link on the page > for editing or even reporting errata. Wassup? > > -r > > -- > http://www.cfcl.com/rdm Rich Morin rdm@REDACTED > http://www.cfcl.com/rdm/resume San Bruno, CA, USA +1 650-873-7841 > > Software system design, development, and documentation > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From byaruhaf@REDACTED Fri Apr 8 07:44:02 2016 From: byaruhaf@REDACTED (Franklin Byaruhanga) Date: Fri, 08 Apr 2016 05:44:02 +0000 Subject: [erlang-questions] how to report errata in the Erlang User's Guide? In-Reply-To: References: Message-ID: Hello, Rich you would need to edit the page below and create a pull request https://github.com/erlang/otp/blob/maint/lib/odbc/doc/src/databases.xml information on how to do that can be found on the link below. https://github.com/erlang/otp/wiki/submitting-patches -- Best Regards Franklin Byaruhanga -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas.larsson@REDACTED Fri Apr 8 11:29:54 2016 From: lukas.larsson@REDACTED (Lukas Larsson) Date: Fri, 8 Apr 2016 11:29:54 +0200 Subject: [erlang-questions] Description of Erlang Garbage Collection Message-ID: Hello everyone, I've long worked on a text that describes in detail exactly how the Erlang garbage collector works. During the last month I finally took the time to finish it and also included the changes that have been made in the current master branch to be released in Erlang/OTP 19.0. You can read the description here: https://www.erlang-solutions.com/blog/erlang-19-0-garbage-collector.html I'm personally am very excited about the off_heap message queue semantics that is coming in 19 and think that it will solve a lot of problems that systems have had with huge message queues running amok and also how refc binaries not been garbage collected fast enough. Also the way that literals are identified from 19 opens up a bunch of new optimization paths for us as it allows a much more fuzzy definition of what is the young heap. If you have any questions about anything, feel free to ask and I'll do my best to answere. Enjoy, Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From valentin@REDACTED Fri Apr 8 11:35:59 2016 From: valentin@REDACTED (Valentin Micic) Date: Fri, 8 Apr 2016 11:35:59 +0200 Subject: [erlang-questions] Description of Erlang Garbage Collection In-Reply-To: References: Message-ID: <7C6ED005-ECCA-4DAA-8D14-F53DC34B9216@pixie.co.za> Any chance of getting this in PDF format? Kind regards V/ On 08 Apr 2016, at 11:29 AM, Lukas Larsson wrote: > Hello everyone, > > I've long worked on a text that describes in detail exactly how the Erlang garbage collector works. During the last month I finally took the time to finish it and also included the changes that have been made in the current master branch to be released in Erlang/OTP 19.0. > > You can read the description here: https://www.erlang-solutions.com/blog/erlang-19-0-garbage-collector.html > > I'm personally am very excited about the off_heap message queue semantics that is coming in 19 and think that it will solve a lot of problems that systems have had with huge message queues running amok and also how refc binaries not been garbage collected fast enough. Also the way that literals are identified from 19 opens up a bunch of new optimization paths for us as it allows a much more fuzzy definition of what is the young heap. > > If you have any questions about anything, feel free to ask and I'll do my best to answere. > > Enjoy, > Lukas > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From pjotr.public12@REDACTED Fri Apr 8 08:59:34 2016 From: pjotr.public12@REDACTED (Pjotr Prins) Date: Fri, 8 Apr 2016 08:59:34 +0200 Subject: [erlang-questions] Reproducible and deterministic builds for GNU Guix (and Nix) In-Reply-To: References: Message-ID: <20160408065934.GA18715@thebird.nl> On Tue, Apr 05, 2016 at 02:07:46PM +0100, Magnus Henoch wrote: > Debian has included a patch that lets you use the environment variable > SOURCE_DATE_EPOCH to fix the compile time, and thus obtain identical > output (given the same compiler version and other things): > [1]https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=795834 > > This was briefly discussed on this mailing list: > [2]http://erlang.org/pipermail/erlang-questions/2015-January/082699.html You may have heard of GNU Guix, the modern (functional) package manager of the GNU project. We are trying to add Erlang and Elixir to Guix, but we are running into the problem that building the Erlang compiler is not deterministic and therefore not reproducible, i.e. the beam files contain time stamps. For normal software built by Erlang this can be overriden with SOURCE_DATE_EPOCH (as per mentioned Debian patch), but for the compiler itself we have not found how to do this. Do you have a suggestion how to bootstrap the compiler with SOURCE_DATE_EPOCH set or disable the time stamps? I am sure as a FP compiler designer you can appreciate determinism. Because GNU Guix is deterministic there is no need to keep track of time stamps. For hot reloading we can assume the start of EPOCH will do the trick, right? Pj. On Mon, Apr 04, 2016 at 01:49:44PM -0400, Leo Famulari wrote: > On Mon, Apr 04, 2016 at 12:50:12PM -0400, Leo Famulari wrote: > > On Mon, Apr 04, 2016 at 10:28:02AM +0200, Pjotr Prins wrote: > > > On Sun, Apr 03, 2016 at 11:39:24PM -0400, Leo Famulari wrote: > > > > Debian's package exhibits this problem. The timestamps are generated in > > > > the following places in the source code. I don't know how to approach > > > > this problem. > > > > > > > > lib/kernel/test/global_SUITE_data/global_trace.erl: io:format("The trace was generated at ~p~n", [EndTime]), > > > > lib/reltool/bin/reltool.escript: lists:flatten(io_lib:format("%% ~s generated at ~w ~w\n~p.\n\n", > > > > lib/reltool/src/reltool_server.erl: IoList = io_lib:format("%% config generated at ~w ~w\n~p.\n\n", > > > > lib/reltool/src/reltool_target.erl: RelIoList = io_lib:format("%% rel generated at ~w ~w\n~p.\n\n", > > > > lib/reltool/src/reltool_target.erl: ScriptIoList = io_lib:format("%% script generated at ~w ~w\n~p.\n\n", > > > > lib/reltool/src/reltool_target.erl: AppIoList = io_lib:format("%% app generated at ~w ~w\n~p.\n\n", > > > > lib/reltool/src/reltool_target.erl: AppIoList = io_lib:format("%% app generated at ~w ~w\n~p.\n\n", > > > > lib/runtime_tools/src/erts_alloc_config.erl: "generated at ~w-~2..0w-~2..0w ~2..0w:~2..0w.~2..0w by " > > > > lib/sasl/src/systools_make.erl: io:format(Fd, "%% script generated at ~w ~w\n~p.\n", > > > > lib/wx/src/gen/gl.erl:%% The program object's information log is updated and the program is generated at the time > > > > > > If there is no easy work around I suggest simply patching them. Fortunately > > > the Erlang compiler does not change much at this level. > > > > The ideal solution would be to use the value of the environment variable > > SOURCE_DATE_EPOCH if it is set, and else to behave as it does now. > > > > > We can also contact Joe Armstrong, the author of Erlang, to discuss > > > this point. He appears to be approachable. I am sure he is open to > > > the idea of deterministic builds in a deterministic build system ;) > > > > I could go to the Erlang IRC channel or forums (whatever they use) and > > ask for advice. Since you are actually using Erlang, I think you would > > be the better person to contact Joe Armstrong himself, if we decide to > > do that. > > I presented the situation on IRC and it was recommended that I start the > discussion on a mailing list. > > I think that the erlang-questions list [0] could be a good place to > start. > > Pjotr, would you like to start the conversation? I can do it if you are > too busy or something. > > [0] > http://www.erlang.org/community > -- > > Regards, > Magnus > > On Mon, Apr 4, 2016 at 8:59 PM, Joe Armstrong <[3]erlang@REDACTED> wrote: > > Hello, > > I think I've asked this before but cannot find the answer: > > I want the beam file produced by > > ? $ erl file.erl > > to always have the same sha1 checksum - there was, if I remember > correctly, a hidden flag that removed the time of compilation etc from > the beam code. Any ideas how to do this? > > /Joe > _______________________________________________ > erlang-questions mailing list > [4]erlang-questions@REDACTED > [5]http://erlang.org/mailman/listinfo/erlang-questions > > References > > Visible links > 1. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=795834 > 2. http://erlang.org/pipermail/erlang-questions/2015-January/082699.html > 3. mailto:erlang@REDACTED > 4. mailto:erlang-questions@REDACTED > 5. http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- From zxq9@REDACTED Fri Apr 8 11:39:23 2016 From: zxq9@REDACTED (zxq9) Date: Fri, 08 Apr 2016 18:39:23 +0900 Subject: [erlang-questions] Description of Erlang Garbage Collection In-Reply-To: References: Message-ID: <2124662.fOPiq5Nu38@changa> On 2016?4?8? ??? 11:29:54 Lukas Larsson wrote: > Hello everyone, > > I've long worked on a text that describes in detail exactly how the Erlang > garbage collector works. During the last month I finally took the time to > finish it and also included the changes that have been made in the current > master branch to be released in Erlang/OTP 19.0. > > You can read the description here: > https://www.erlang-solutions.com/blog/erlang-19-0-garbage-collector.html Nice, Lukas! Thanks for taking the time to write this up. That's right up there with the in-depth explanations of some of the map internals and the scheduler explanations that have gone round. Great stuff! -Craig From essen@REDACTED Fri Apr 8 12:03:53 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Fri, 8 Apr 2016 12:03:53 +0200 Subject: [erlang-questions] Description of Erlang Garbage Collection In-Reply-To: <7C6ED005-ECCA-4DAA-8D14-F53DC34B9216@pixie.co.za> References: <7C6ED005-ECCA-4DAA-8D14-F53DC34B9216@pixie.co.za> Message-ID: <57078209.3000106@ninenines.eu> Or included directly in the Erlang documentation? :-) On 04/08/2016 11:35 AM, Valentin Micic wrote: > Any chance of getting this in PDF format? > > Kind regards > > V/ > > On 08 Apr 2016, at 11:29 AM, Lukas Larsson wrote: > >> Hello everyone, >> >> I've long worked on a text that describes in detail exactly how the >> Erlang garbage collector works. During the last month I finally took >> the time to finish it and also included the changes that have been >> made in the current master branch to be released in Erlang/OTP 19.0. >> >> You can read the description here: >> https://www.erlang-solutions.com/blog/erlang-19-0-garbage-collector.html >> >> I'm personally am very excited about the off_heap message queue >> semantics that is coming in 19 and think that it will solve a lot of >> problems that systems have had with huge message queues running amok >> and also how refc binaries not been garbage collected fast enough. >> Also the way that literals are identified from 19 opens up a bunch of >> new optimization paths for us as it allows a much more fuzzy >> definition of what is the young heap. >> >> If you have any questions about anything, feel free to ask and I'll do >> my best to answere. >> >> Enjoy, >> Lukas >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From zxq9@REDACTED Fri Apr 8 12:04:56 2016 From: zxq9@REDACTED (zxq9) Date: Fri, 08 Apr 2016 19:04:56 +0900 Subject: [erlang-questions] Description of Erlang Garbage Collection In-Reply-To: <57078209.3000106@ninenines.eu> References: <7C6ED005-ECCA-4DAA-8D14-F53DC34B9216@pixie.co.za> <57078209.3000106@ninenines.eu> Message-ID: <3701972.3FnLhfVzEo@changa> On 2016?4?8? ??? 12:03:53 Lo?c Hoguin wrote: > Or included directly in the Erlang documentation? :-) +1 From bjorn@REDACTED Fri Apr 8 12:32:09 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Fri, 8 Apr 2016 12:32:09 +0200 Subject: [erlang-questions] Feedback wanted: Remove compilation times from BEAM files in OTP 19? Message-ID: As long as anyone can remember, the compilation time has been included in BEAM files (and can be retrieved using module_info()). As far as I know, the inclusion of the time has only caused problems. I can't remember that anyone has found any use for the compilation time. So if we decide to remove the compilation time by default in OTP 19... ... would it cause any problem for someone in some workflow? ... would we need an option to include the compilation time? /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From tony@REDACTED Fri Apr 8 12:58:29 2016 From: tony@REDACTED (Tony Rogvall) Date: Fri, 8 Apr 2016 12:58:29 +0200 Subject: [erlang-questions] Description of Erlang Garbage Collection In-Reply-To: References: Message-ID: <623A7446-43ED-4B40-B21C-51B16EB27F9C@rogvall.se> Fantastic!!! /Tony > On 8 apr 2016, at 11:29, Lukas Larsson wrote: > > Hello everyone, > > I've long worked on a text that describes in detail exactly how the Erlang garbage collector works. During the last month I finally took the time to finish it and also included the changes that have been made in the current master branch to be released in Erlang/OTP 19.0. > > You can read the description here: https://www.erlang-solutions.com/blog/erlang-19-0-garbage-collector.html > > I'm personally am very excited about the off_heap message queue semantics that is coming in 19 and think that it will solve a lot of problems that systems have had with huge message queues running amok and also how refc binaries not been garbage collected fast enough. Also the way that literals are identified from 19 opens up a bunch of new optimization paths for us as it allows a much more fuzzy definition of what is the young heap. > > If you have any questions about anything, feel free to ask and I'll do my best to answere. > > Enjoy, > Lukas > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From nick@REDACTED Fri Apr 8 13:01:06 2016 From: nick@REDACTED (Niclas Eklund) Date: Fri, 8 Apr 2016 13:01:06 +0200 Subject: [erlang-questions] Description of Erlang Garbage Collection In-Reply-To: <7C6ED005-ECCA-4DAA-8D14-F53DC34B9216@pixie.co.za> References: <7C6ED005-ECCA-4DAA-8D14-F53DC34B9216@pixie.co.za> Message-ID: <57078F72.5090208@tail-f.com> Hi! Use for example - http://www.web2pdfconvert.com/ Nice work Lukas! /Nick On 04/08/2016 11:35 AM, Valentin Micic wrote: > Any chance of getting this in PDF format? > > Kind regards > > V/ > > On 08 Apr 2016, at 11:29 AM, Lukas Larsson wrote: > >> Hello everyone, >> >> I've long worked on a text that describes in detail exactly how the >> Erlang garbage collector works. During the last month I finally took >> the time to finish it and also included the changes that have been >> made in the current master branch to be released in Erlang/OTP 19.0. >> >> You can read the description here: >> https://www.erlang-solutions.com/blog/erlang-19-0-garbage-collector.html >> >> I'm personally am very excited about the off_heap message queue >> semantics that is coming in 19 and think that it will solve a lot of >> problems that systems have had with huge message queues running amok >> and also how refc binaries not been garbage collected fast enough. >> Also the way that literals are identified from 19 opens up a bunch of >> new optimization paths for us as it allows a much more fuzzy >> definition of what is the young heap. >> >> If you have any questions about anything, feel free to ask and I'll >> do my best to answere. >> >> Enjoy, >> Lukas >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From cian@REDACTED Fri Apr 8 13:03:47 2016 From: cian@REDACTED (Cian Synnott) Date: Fri, 8 Apr 2016 12:03:47 +0100 Subject: [erlang-questions] Description of Erlang's distribution mechanisms. Message-ID: http://emauton.org/disterl/ I wrote this a year ago while digging around in Erlang's distribution mechanisms; I had intended to finish it (in particular, to provide some notes on performance characteristics, what blocks where, etc.) but it seems to have never quite surfaced to the top of my Copious Free Time list again. :o) Lukas' message about the GC description reminded me, and it's probably better to have something referenced on the list here (even unfinished) than for another engineer to do the same investigation. If anyone has any comments on accuracy or things that should be included, let me know. That said, I have a very good set of comments from Evan Vigil McClanahan that require a bit more work to integrate. One of these days. :o) Included below for competeness: ---- 8< ---- One thing I would love to have documented somewhere publicly are the performance/blocking semantics of sending messages over disterl (which is something that I haven't looked into myself, although it was on my long list of perf issues to look into before). What I mean here is: how is the message serialized when you send it? Does serialization block the sending pid or is it asynchronous? Are there differences between sending something over gen_tcp vs. disterl, etc. I have intuitions about all of these things, but it'd be nice to know. It might require some beam spelunking to figure those out though. Other things: - disterl does some work to make sure that messages arrive in order - it doesn't multiplex over multiple sockets, and is often subject to severe head-of-line blocking issues when you send large messages over it because of that. - please document the +zddbl argument and busy_dist_ports, which are important to understand and tune if you're ever sending lots of data over disterl pipes. - nodeup and nodedown messages ought to be covered - for fast sending there are some issues with the socket buffer resizing logic that can limit speed in the case of many small messages, setting the disterl socket buffers larger can help - often people want to dynamically or selectively startup the net_kernel from an escript or something, a quick recipe might help - a short discussion of net_ticktime and dead node detection might be helpful, as the default is quite high for some applications and migrating later can be quite painful ---- 8< ---- Cian From max.lapshin@REDACTED Fri Apr 8 13:05:24 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 8 Apr 2016 14:05:24 +0300 Subject: [erlang-questions] Feedback wanted: Remove compilation times from BEAM files in OTP 19? In-Reply-To: References: Message-ID: You want to achieve stable builds? -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn@REDACTED Fri Apr 8 13:16:04 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Fri, 8 Apr 2016 13:16:04 +0200 Subject: [erlang-questions] Feedback wanted: Remove compilation times from BEAM files in OTP 19? In-Reply-To: References: Message-ID: On Fri, Apr 8, 2016 at 1:05 PM, Max Lapshin wrote: > You want to achieve stable builds? Yes. See: http://erlang.org/pipermail/erlang-questions/2016-April/088717.html That is just the last time I heard of problems with the compilation times. /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From essen@REDACTED Fri Apr 8 13:16:06 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Fri, 8 Apr 2016 13:16:06 +0200 Subject: [erlang-questions] Feedback wanted: Remove compilation times from BEAM files in OTP 19? In-Reply-To: References: Message-ID: <570792F6.6080608@ninenines.eu> The build time is occasionally useful but the same can be achieved by looking at the beam file creation or modification time. Is it just the build time though? Wouldn't these infos also be a problem? * Auto generated vsn (it looks like repeated builds keep the same version number if the source doesn't change, but is that guaranteed?) [{vsn,[192941001008994618371492498907572068968]}, * Compiler version (if two different compiler versions produce the same result except for the compiler version, this is a waste; if compiler version of a build is desirable it should probably be kept separate) {version,"6.0.1"}, * Compile options with paths {outdir,"/build/erlang/src/otp/lib/ssl/src/../ebin"}, {i,"/build/erlang/src/otp/lib/kernel/src"}, * Source file name {source,"/build/erlang/src/otp/lib/ssl/src/ssl.erl"}]}, On 04/08/2016 12:32 PM, Bj?rn Gustavsson wrote: > As long as anyone can remember, the compilation time has been included > in BEAM files (and can be retrieved using module_info()). > > As far as I know, the inclusion of the time has only caused problems. > I can't remember that anyone has found any use for the compilation > time. > > So if we decide to remove the compilation time by default in OTP 19... > > ... would it cause any problem for someone in some workflow? > > ... would we need an option to include the compilation time? > > /Bjorn > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From ml-erlang@REDACTED Fri Apr 8 13:46:35 2016 From: ml-erlang@REDACTED (Matthias Rieber) Date: Fri, 8 Apr 2016 13:46:35 +0200 (CEST) Subject: [erlang-questions] Feedback wanted: Remove compilation times from BEAM files in OTP 19? In-Reply-To: References: Message-ID: On Fri, 8 Apr 2016, Bj?rn Gustavsson wrote: > As long as anyone can remember, the compilation time has been included > in BEAM files (and can be retrieved using module_info()). > > As far as I know, the inclusion of the time has only caused problems. > I can't remember that anyone has found any use for the compilation > time. > > So if we decide to remove the compilation time by default in OTP 19... > > ... would it cause any problem for someone in some workflow? I couldn't resist: https://xkcd.com/1172/ Matthias From eric.pailleau@REDACTED Fri Apr 8 14:11:05 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Fri, 08 Apr 2016 14:11:05 +0200 Subject: [erlang-questions] Feedback wanted: Remove compilation times from BEAM files in OTP 19? In-Reply-To: References: Message-ID: <7vjf66b2ps1bg1udr4apph0k.1460117465186@email.android.com> Hi, Beam files are separated in different chunks. All dynamic information should be placed in a separate chunck Mho. And sha1sum of code part, or checksum of its canonical abstract code maybe available in a chunk, existing or new one. This could allow to know that a beam, even without the same checksum on the file globally, run the same code than another Beam file, and so be able to do some Nix like distribution. Compiler version allow to estimate what Erlang version compiled the beam. This is something to keep I think. Regards "Envoy? depuis mon mobile " Eric ---- Matthias Rieber a ?crit ---- >On Fri, 8 Apr 2016, Bj?rn Gustavsson wrote: > >> As long as anyone can remember, the compilation time has been included >> in BEAM files (and can be retrieved using module_info()). >> >> As far as I know, the inclusion of the time has only caused problems. >> I can't remember that anyone has found any use for the compilation >> time. >> >> So if we decide to remove the compilation time by default in OTP 19... >> >> ... would it cause any problem for someone in some workflow? > >I couldn't resist: > >https://xkcd.com/1172/ > >Matthias > >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From serge@REDACTED Fri Apr 8 14:28:36 2016 From: serge@REDACTED (Serge Aleynikov) Date: Fri, 8 Apr 2016 08:28:36 -0400 Subject: [erlang-questions] Feedback wanted: Remove compilation times from BEAM files in OTP 19? In-Reply-To: References: Message-ID: I've been relying on module's compilation time for figuring out the changed modules (see below). In the absence of {time, CompileTime}, would there be a way to determine the modification time stamp of the file at the time it was loaded by code loader? Serge modified_modules() -> [M || {M, _} <- code:all_loaded(), module_modified(M) == true]. module_modified(Module) -> case code:is_loaded(Module) of {file, preloaded} -> false; {file, Path} -> CompileOpts = proplists:get_value(compile, Module:module_info()), CompileTime = proplists:get_value(time, CompileOpts), Src = proplists:get_value(source, CompileOpts), module_modified(Path, CompileTime, Src); _ -> false end. module_modified(Path, PrevCompileTime, PrevSrc) -> case find_module_file(Path) of false -> false; ModPath -> {ok, {_, [{_, CB}]}} = beam_lib:chunks(ModPath, ["CInf"]), CompileOpts = binary_to_term(CB), CompileTime = proplists:get_value(time, CompileOpts), Src = proplists:get_value(source, CompileOpts), not (CompileTime == PrevCompileTime) and (Src == PrevSrc) end. On Fri, Apr 8, 2016 at 6:32 AM, Bj?rn Gustavsson wrote: > As long as anyone can remember, the compilation time has been included > in BEAM files (and can be retrieved using module_info()). > > As far as I know, the inclusion of the time has only caused problems. > I can't remember that anyone has found any use for the compilation > time. > > So if we decide to remove the compilation time by default in OTP 19... > > ... would it cause any problem for someone in some workflow? > > ... would we need an option to include the compilation time? > > /Bjorn > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Apr 8 14:32:36 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Fri, 8 Apr 2016 14:32:36 +0200 Subject: [erlang-questions] Feedback wanted: Remove compilation times from BEAM files in OTP 19? In-Reply-To: <7vjf66b2ps1bg1udr4apph0k.1460117465186@email.android.com> References: <7vjf66b2ps1bg1udr4apph0k.1460117465186@email.android.com> Message-ID: <5707A4E4.7060903@ninenines.eu> If you don't have hash(file1) = hash(file2) that means the usual tooling doesn't work as well as it could: packages will do file changes that don't change the program, rsync will do similar with syncs, and so on. It's also a bit dubious to require Erlang to verify that the Erlang files (including the compiler itself) were compiled as expected. On 04/08/2016 02:11 PM, ?ric Pailleau wrote: > Hi, > Beam files are separated in different chunks. > All dynamic information should be placed in a separate chunck Mho. > And sha1sum of code part, or checksum of its canonical abstract code > maybe available in a chunk, existing or new one. > This could allow to know that a beam, even without the same checksum on > the file globally, run the same code than another Beam file, and so be > able to do some Nix like distribution. > > Compiler version allow to estimate what Erlang version compiled the > beam. This is something to keep I think. > > Regards > > "Envoy? depuis mon mobile " Eric > > > > ---- Matthias Rieber a ?crit ---- > > On Fri, 8 Apr 2016, Bj?rn Gustavsson wrote: > > > As long as anyone can remember, the compilation time has been included > > in BEAM files (and can be retrieved using module_info()). > > > > As far as I know, the inclusion of the time has only caused problems. > > I can't remember that anyone has found any use for the compilation > > time. > > > > So if we decide to remove the compilation time by default in OTP 19... > > > > ... would it cause any problem for someone in some workflow? > > I couldn't resist: > > https://xkcd.com/1172/ > > Matthias > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From erlangsiri@REDACTED Fri Apr 8 14:36:37 2016 From: erlangsiri@REDACTED (Siri Hansen) Date: Fri, 8 Apr 2016 14:36:37 +0200 Subject: [erlang-questions] Should we deprecate or modernize log_mf_h and rb? In-Reply-To: References: <20160405084953.GA32163@seth> Message-ID: Thanks for the feedback, everyone! We have now decided not to remove log_mf_h and rb. Instead there will be some work done to modernize these tools. Kind Regards /siri 2016-04-07 0:03 GMT+02:00 Tim Stewart : > Hello, > > I use log_mf_h and rb every day, and I am quite fond of them. > > My systems are usually configured with both lager and log_mf_h. SASL + > log_mf_h pick up all of the "core" OTP-based reports (info, warning, error, > crash, supervisor, progress) and lager is used by our applications for > application-level logging. Since lager also picks up reports and messages > created by error_logger, our support team can monitor only a single > *line-based* log file to find problems. > > Then, when we get actual crashes I turn to report browser for nicely > formatted reports. In my opinion, rb's formatted reports are easier to > read than lager's crash.log. I have error_logger configured to use only > lager and log_mf_h handlers so that we are relatively protected from a high > error_logger volume. > > And, as ?ric Pailleau mentioned, log_mf_h is the backend for Observer's > log support. I also use this feature. > > I would be sad to see log_mf_h disappear. I would *love* to see more GUI > support for reading and searching through these logs (additional Observer > support, web UI, etc.) > > Anyway, I hope this helps. > > -TimS > > > > On Tue, Apr 5, 2016 at 4:49 AM, Stefan Grundmann > wrote: > >> Hi >> >> log_mf_h is used in appliance systems we build at $work because it >> implements binary logging and log rotation and because it is >> part of OTP. >> We also have projects where lager is used but only because it >> was pulled as a dependency of $other_stuff and we did not yet spend the >> time >> to replace it. >> >> sg >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > > -- > Tim Stewart > tim@REDACTED > +1 (404) 993-6492 > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mbj@REDACTED Fri Apr 8 14:43:10 2016 From: mbj@REDACTED (Martin Bjorklund) Date: Fri, 08 Apr 2016 14:43:10 +0200 (CEST) Subject: [erlang-questions] Feedback wanted: Remove compilation times from BEAM files in OTP 19? In-Reply-To: References: Message-ID: <20160408.144310.592874016340419663.mbj@tail-f.com> Serge Aleynikov wrote: > I've been relying on module's compilation time for figuring out the changed > modules (see below). I also use this function all the time (I.e, in user_default: lm() -> [c:l(M) || M <- modified_modules()]. super convenient!) Anyway, if there's an alternative way to do the same thing that would be fine. > In the absence of {time, CompileTime}, would there be a way to determine > the modification time stamp of the file at the time it was loaded by code > loader? /martin > > Serge > > modified_modules() -> > [M || {M, _} <- code:all_loaded(), module_modified(M) == true]. > > module_modified(Module) -> > case code:is_loaded(Module) of > {file, preloaded} -> > false; > {file, Path} -> > CompileOpts = proplists:get_value(compile, Module:module_info()), > CompileTime = proplists:get_value(time, CompileOpts), > Src = proplists:get_value(source, CompileOpts), > module_modified(Path, CompileTime, Src); > _ -> > false > end. > > module_modified(Path, PrevCompileTime, PrevSrc) -> > case find_module_file(Path) of > false -> > false; > ModPath -> > {ok, {_, [{_, CB}]}} = beam_lib:chunks(ModPath, ["CInf"]), > CompileOpts = binary_to_term(CB), > CompileTime = proplists:get_value(time, CompileOpts), > Src = proplists:get_value(source, CompileOpts), > not (CompileTime == PrevCompileTime) and (Src == PrevSrc) > end. > > > On Fri, Apr 8, 2016 at 6:32 AM, Bj?rn Gustavsson wrote: > > > As long as anyone can remember, the compilation time has been included > > in BEAM files (and can be retrieved using module_info()). > > > > As far as I know, the inclusion of the time has only caused problems. > > I can't remember that anyone has found any use for the compilation > > time. > > > > So if we decide to remove the compilation time by default in OTP 19... > > > > ... would it cause any problem for someone in some workflow? > > > > ... would we need an option to include the compilation time? > > > > /Bjorn > > > > -- > > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > From bjorn@REDACTED Fri Apr 8 14:44:54 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Fri, 8 Apr 2016 14:44:54 +0200 Subject: [erlang-questions] Feedback wanted: Remove compilation times from BEAM files in OTP 19? In-Reply-To: <570792F6.6080608@ninenines.eu> References: <570792F6.6080608@ninenines.eu> Message-ID: On Fri, Apr 8, 2016 at 1:16 PM, Lo?c Hoguin wrote: > The build time is occasionally useful but the same can be achieved by > looking at the beam file creation or modification time. > > Is it just the build time though? > > Wouldn't these infos also be a problem? Yes, some of them are potential problems. > * Auto generated vsn (it looks like repeated builds keep the same version > number if the source doesn't change, but is that guaranteed?) > Yes, it is guaranteed as long as the compiler generates the code in the same way. That is, new optimizations could change it (but in that case, there would be other changes in the BEAM file as well). > [{vsn,[192941001008994618371492498907572068968]}, > > * Compiler version (if two different compiler versions produce the same > result except for the compiler version, this is a waste; if compiler version > of a build is desirable it should probably be kept separate) > > {version,"6.0.1"}, > > * Compile options with paths > > {outdir,"/build/erlang/src/otp/lib/ssl/src/../ebin"}, > {i,"/build/erlang/src/otp/lib/kernel/src"}, > > * Source file name > > {source,"/build/erlang/src/otp/lib/ssl/src/ssl.erl"}]}, > /Bjorn From lukas.larsson@REDACTED Fri Apr 8 14:47:35 2016 From: lukas.larsson@REDACTED (Lukas Larsson) Date: Fri, 8 Apr 2016 14:47:35 +0200 Subject: [erlang-questions] Description of Erlang Garbage Collection In-Reply-To: <57078209.3000106@ninenines.eu> References: <7C6ED005-ECCA-4DAA-8D14-F53DC34B9216@pixie.co.za> <57078209.3000106@ninenines.eu> Message-ID: On Fri, Apr 8, 2016 at 12:03 PM, Lo?c Hoguin wrote: > Or included directly in the Erlang documentation? :-) I'll see if can include it in the internal docs section of erts when 19 is released. As for pdf version, you can also use the print option from your web browser to save the page as a pdf. -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Fri Apr 8 14:51:04 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Fri, 08 Apr 2016 14:51:04 +0200 Subject: [erlang-questions] Feedback wanted: Remove compilation times from BEAM files in OTP 19? In-Reply-To: <5707A4E4.7060903@ninenines.eu> References: <7vjf66b2ps1bg1udr4apph0k.1460117465186@email.android.com> <5707A4E4.7060903@ninenines.eu> Message-ID: I agree Lo?c, but native code is in another chunk named with a mnemo of architecture. So a same code, compiled native or not will create Beam files with different checksum anyway. It is a hard problem to solve, with also the constraints to try to keep backward compatibility... Actually I m not sure it is possible to achieve this, or code part have to be in another independent file. This break compatibility... "Envoy? depuis mon mobile " Eric ---- Lo?c Hoguin a ?crit ---- >If you don't have hash(file1) = hash(file2) that means the usual tooling >doesn't work as well as it could: packages will do file changes that >don't change the program, rsync will do similar with syncs, and so on. > >It's also a bit dubious to require Erlang to verify that the Erlang >files (including the compiler itself) were compiled as expected. > >On 04/08/2016 02:11 PM, ?ric Pailleau wrote: >> Hi, >> Beam files are separated in different chunks. >> All dynamic information should be placed in a separate chunck Mho. >> And sha1sum of code part, or checksum of its canonical abstract code >> maybe available in a chunk, existing or new one. >> This could allow to know that a beam, even without the same checksum on >> the file globally, run the same code than another Beam file, and so be >> able to do some Nix like distribution. >> >> Compiler version allow to estimate what Erlang version compiled the >> beam. This is something to keep I think. >> >> Regards >> >> "Envoy? depuis mon mobile " Eric >> >> >> >> ---- Matthias Rieber a ?crit ---- >> >> On Fri, 8 Apr 2016, Bj?rn Gustavsson wrote: >> >> > As long as anyone can remember, the compilation time has been included >> > in BEAM files (and can be retrieved using module_info()). >> > >> > As far as I know, the inclusion of the time has only caused problems. >> > I can't remember that anyone has found any use for the compilation >> > time. >> > >> > So if we decide to remove the compilation time by default in OTP 19... >> > >> > ... would it cause any problem for someone in some workflow? >> >> I couldn't resist: >> >> https://xkcd.com/1172/ >> >> Matthias >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > >-- >Lo?c Hoguin >http://ninenines.eu >Author of The Erlanger Playbook, >A book about software development using Erlang -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn@REDACTED Fri Apr 8 14:54:26 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Fri, 8 Apr 2016 14:54:26 +0200 Subject: [erlang-questions] Feedback wanted: Remove compilation times from BEAM files in OTP 19? In-Reply-To: References: Message-ID: On Fri, Apr 8, 2016 at 2:28 PM, Serge Aleynikov wrote: > I've been relying on module's compilation time for figuring out the changed > modules (see below). > > In the absence of {time, CompileTime}, would there be a way to determine the > modification time stamp of the file at the time it was loaded by code > loader? Yes. Use Mod:module_info(md5) to calculate MD5 for the loaded module and compare it to beam_lib:md5(Mod). Example: 13> c:module_info(md5). <<79,26,188,243,168,60,58,45,34,69,19,222,138,190,214,118>> 14> beam_lib:md5(code:which(c)). {ok,{c,<<79,26,188,243,168,60,58,45,34,69,19,222,138, 190,214,118>>}} /Bjorn From serge@REDACTED Fri Apr 8 15:37:13 2016 From: serge@REDACTED (Serge Aleynikov) Date: Fri, 8 Apr 2016 09:37:13 -0400 Subject: [erlang-questions] Feedback wanted: Remove compilation times from BEAM files in OTP 19? In-Reply-To: References: Message-ID: ?Thank you! I updated the https://github.com/saleyn/util/blob/master/src/user_default.erl per your suggestion. On Fri, Apr 8, 2016 at 8:54 AM, Bj?rn Gustavsson wrote: > On Fri, Apr 8, 2016 at 2:28 PM, Serge Aleynikov > wrote: > > I've been relying on module's compilation time for figuring out the > changed > > modules (see below). > > > > In the absence of {time, CompileTime}, would there be a way to determine > the > > modification time stamp of the file at the time it was loaded by code > > loader? > > Yes. Use Mod:module_info(md5) to calculate MD5 for > the loaded module and compare it to beam_lib:md5(Mod). > > Example: > > 13> c:module_info(md5). > <<79,26,188,243,168,60,58,45,34,69,19,222,138,190,214,118>> > 14> beam_lib:md5(code:which(c)). > {ok,{c,<<79,26,188,243,168,60,58,45,34,69,19,222,138, > 190,214,118>>}} > > /Bjorn > -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.a.vinogradov@REDACTED Fri Apr 8 15:45:07 2016 From: g.a.vinogradov@REDACTED (Gleb Vinogradov) Date: Fri, 8 Apr 2016 19:45:07 +0600 Subject: [erlang-questions] Feedback wanted: Remove compilation times from BEAM files in OTP 19? In-Reply-To: References: Message-ID: May be we just need compiler option for creating stable builds? --stable-build , --pure-build or similar So no build time, and no paths will be included in *.beam file. It will not break workflow for anyone, and may be later, in OTP 21 non-stable builds will be deprecated. Gleb. 2016-04-08 16:32 GMT+06:00 Bj?rn Gustavsson : > As long as anyone can remember, the compilation time has been included > in BEAM files (and can be retrieved using module_info()). > > As far as I know, the inclusion of the time has only caused problems. > I can't remember that anyone has found any use for the compilation > time. > > So if we decide to remove the compilation time by default in OTP 19... > > ... would it cause any problem for someone in some workflow? > > ... would we need an option to include the compilation time? > > /Bjorn > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From drohrer@REDACTED Fri Apr 8 15:53:07 2016 From: drohrer@REDACTED (Douglas Rohrer) Date: Fri, 8 Apr 2016 09:53:07 -0400 Subject: [erlang-questions] Feedback wanted: Remove compilation times from BEAM files in OTP 19? In-Reply-To: References: Message-ID: <6768286D-AEF8-4014-9E8A-27CEAB658B0B@basho.com> I know it's not exactly the "principle of least surprise" but close enough - I'd agree that not changing the default, but providing a simple way to accomplish the goal, is probably the best way forward. Doug > On Apr 8, 2016, at 9:45 AM, Gleb Vinogradov wrote: > > May be we just need compiler option for creating stable builds? --stable-build , --pure-build or similar > So no build time, and no paths will be included in *.beam file. It will not break workflow for anyone, and may be later, in OTP 21 non-stable builds will be deprecated. > > Gleb. > > 2016-04-08 16:32 GMT+06:00 Bj?rn Gustavsson >: > As long as anyone can remember, the compilation time has been included > in BEAM files (and can be retrieved using module_info()). > > As far as I know, the inclusion of the time has only caused problems. > I can't remember that anyone has found any use for the compilation > time. > > So if we decide to remove the compilation time by default in OTP 19... > > ... would it cause any problem for someone in some workflow? > > ... would we need an option to include the compilation time? > > /Bjorn > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From nthauvin@REDACTED Fri Apr 8 16:13:23 2016 From: nthauvin@REDACTED (Nicolas Thauvin) Date: Fri, 08 Apr 2016 16:13:23 +0200 Subject: [erlang-questions] bug : ssl losing ciphers Message-ID: Hi, We've been trying to restrict SSL ciphers to a secure set in Yaws / OTP R18, but only a few of them were actually taken into account (leading to connection issues from old browsers). According to the documentation, one can list the availables ciphers with ssl:cipher_suites(). For example: [... {rsa,aes_256_gcm,null,sha384}, {rsa,aes_256_cbc,sha256}, ...] Note there are 3-tuples and 4-tuples in the result. Now, when the customised 'ciphers' SSL option is set, its content is processed by ssl:binary_cipher_suites/2 (Beam you up : https://github.com/erlang/otp/blob/maint-18/lib/ssl/src/ssl.erl#L1092) There comes the issue : this function expects all the entries to be the same tuple size (3 or 4) according to a matching on the first element, losing entries from the list when they don't match the tuple size. The patch for ssl:binary_cipher_suites/2 is trivial, but why does ssl_cipher:suite() still returns a mixed-size of tuples since 4-tuples seems to be considered as backward compatible according to the comments ? Cheers, -- Nicolas From erlang@REDACTED Fri Apr 8 17:06:49 2016 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 8 Apr 2016 17:06:49 +0200 Subject: [erlang-questions] Reproducible and deterministic builds for GNU Guix (and Nix) In-Reply-To: <20160408065934.GA18715@thebird.nl> References: <20160408065934.GA18715@thebird.nl> Message-ID: On Fri, Apr 8, 2016 at 8:59 AM, Pjotr Prins wrote: > On Tue, Apr 05, 2016 at 02:07:46PM +0100, Magnus Henoch wrote: >> Debian has included a patch that lets you use the environment variable >> SOURCE_DATE_EPOCH to fix the compile time, and thus obtain identical >> output (given the same compiler version and other things): This is very hacky - it might work by accident but you'd want stronger guarantees that if you compiled the same file many times by the same compiler that you'd always get the same object code file. As I see things this is crucial to making reproducible builds. Having the compilation time in the beam file is really bad (I don't remember if I did this, but if so I apologise) - it should not matter what time you compiled the file. If you want this information, stick it in a log file, or somewhere else. If the beam file is uniquely determined by the version of the compiler, the source and the macro definitons used when it was compiled then we can use the SHA1 checksum of the beam file as a key, and inject the code into a distributed hash table - this is the first step to making a global revision control system with strong guarantees on version consistency. I see absolutely no reason for the dozens of different version and revision control systems that pollute the planet when all that is needed is a DHT containing blogs identified by some checksum (like SHA1 or something). >> [1]https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=795834 >> >> This was briefly discussed on this mailing list: >> [2]http://erlang.org/pipermail/erlang-questions/2015-January/082699.html > > You may have heard of GNU Guix, the modern (functional) package > manager of the GNU project. We are trying to add Erlang and Elixir to > Guix, but we are running into the problem that building the Erlang > compiler is not deterministic and therefore not reproducible, i.e. the > beam files contain time stamps. Great - I've not looked at Guix but I've been following NiX - I've wanted GitTorrent (= Git + Bit Torrent) so both these seem like a step in the right direction. Re the time stamps - you can post-process the beam code to remove the time stamps, but I'd like stronger guarantees than that. What would happen if two different implementations of a module produced the same beam file (I think just rearranging the comments would achieve this, though I haven't tested this) should this be allowed? Personally, I think that the SHA of the source should be included in the beam file, which will identify the code used to create the beam file. For your purposes I can write a script to call erlc and strip out the parts that make compilation reproducible. In the long term we should discuss this, figure out what the correct thing to do is, and then do it. Cheers /Joe > > For normal software built by Erlang this can be overriden with > SOURCE_DATE_EPOCH (as per mentioned Debian patch), but for the > compiler itself we have not found how to do this. > > Do you have a suggestion how to bootstrap the compiler with > SOURCE_DATE_EPOCH set or disable the time stamps? I am sure as a FP > compiler designer you can appreciate determinism. Because GNU Guix is > deterministic there is no need to keep track of time stamps. For hot > reloading we can assume the start of EPOCH will do the trick, right? > > Pj. > > On Mon, Apr 04, 2016 at 01:49:44PM -0400, Leo Famulari wrote: >> On Mon, Apr 04, 2016 at 12:50:12PM -0400, Leo Famulari wrote: >> > On Mon, Apr 04, 2016 at 10:28:02AM +0200, Pjotr Prins wrote: >> > > On Sun, Apr 03, 2016 at 11:39:24PM -0400, Leo Famulari wrote: >> > > > Debian's package exhibits this problem. The timestamps are generated in >> > > > the following places in the source code. I don't know how to approach >> > > > this problem. >> > > > >> > > > lib/kernel/test/global_SUITE_data/global_trace.erl: io:format("The trace was generated at ~p~n", [EndTime]), >> > > > lib/reltool/bin/reltool.escript: lists:flatten(io_lib:format("%% ~s generated at ~w ~w\n~p.\n\n", >> > > > lib/reltool/src/reltool_server.erl: IoList = io_lib:format("%% config generated at ~w ~w\n~p.\n\n", >> > > > lib/reltool/src/reltool_target.erl: RelIoList = io_lib:format("%% rel generated at ~w ~w\n~p.\n\n", >> > > > lib/reltool/src/reltool_target.erl: ScriptIoList = io_lib:format("%% script generated at ~w ~w\n~p.\n\n", >> > > > lib/reltool/src/reltool_target.erl: AppIoList = io_lib:format("%% app generated at ~w ~w\n~p.\n\n", >> > > > lib/reltool/src/reltool_target.erl: AppIoList = io_lib:format("%% app generated at ~w ~w\n~p.\n\n", >> > > > lib/runtime_tools/src/erts_alloc_config.erl: "generated at ~w-~2..0w-~2..0w ~2..0w:~2..0w.~2..0w by " >> > > > lib/sasl/src/systools_make.erl: io:format(Fd, "%% script generated at ~w ~w\n~p.\n", >> > > > lib/wx/src/gen/gl.erl:%% The program object's information log is updated and the program is generated at the time >> > > >> > > If there is no easy work around I suggest simply patching them. Fortunately >> > > the Erlang compiler does not change much at this level. >> > >> > The ideal solution would be to use the value of the environment variable >> > SOURCE_DATE_EPOCH if it is set, and else to behave as it does now. >> > >> > > We can also contact Joe Armstrong, the author of Erlang, to discuss >> > > this point. He appears to be approachable. I am sure he is open to >> > > the idea of deterministic builds in a deterministic build system ;) >> > >> > I could go to the Erlang IRC channel or forums (whatever they use) and >> > ask for advice. Since you are actually using Erlang, I think you would >> > be the better person to contact Joe Armstrong himself, if we decide to >> > do that. >> >> I presented the situation on IRC and it was recommended that I start the >> discussion on a mailing list. >> >> I think that the erlang-questions list [0] could be a good place to >> start. >> >> Pjotr, would you like to start the conversation? I can do it if you are >> too busy or something. >> >> [0] >> http://www.erlang.org/community >> > > -- > >> >> Regards, >> Magnus >> >> On Mon, Apr 4, 2016 at 8:59 PM, Joe Armstrong <[3]erlang@REDACTED> wrote: >> >> Hello, >> >> I think I've asked this before but cannot find the answer: >> >> I want the beam file produced by >> >> $ erl file.erl >> >> to always have the same sha1 checksum - there was, if I remember >> correctly, a hidden flag that removed the time of compilation etc from >> the beam code. Any ideas how to do this? >> >> /Joe >> _______________________________________________ >> erlang-questions mailing list >> [4]erlang-questions@REDACTED >> [5]http://erlang.org/mailman/listinfo/erlang-questions >> >> References >> >> Visible links >> 1. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=795834 >> 2. http://erlang.org/pipermail/erlang-questions/2015-January/082699.html >> 3. mailto:erlang@REDACTED >> 4. mailto:erlang-questions@REDACTED >> 5. http://erlang.org/mailman/listinfo/erlang-questions > >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > -- From jordand@REDACTED Fri Apr 8 17:16:50 2016 From: jordand@REDACTED (Jordan Day) Date: Fri, 8 Apr 2016 10:16:50 -0500 Subject: [erlang-questions] Any guidance on tweaking allocator settings? Message-ID: Hi Metin, Fred, Lukas, Thanks for the suggestions. I had previously consulted Erlang in Anger about this issue (which lead me to Fred's great ?down the rabbit hole? blog post, and Lukas? EUC presentation), but, as Lukas mentions, it doesn?t seem to be an issue with fragmentation ? in addition, I?ve tried to manually run garbage collection on all the processes in the same manner as Metin suggests, but it hasn?t helped much. Based on some more digging, it seems the most likely culprit is probably just a combination of surprisingly large allocations made while parsing the JSON, and bad program design on my part. In a long-running process, I?m recursively calling into my function for downloading a bunch of these JSON documents in what I *think* is a tail-recursive manner, but I?m guessing I?m doing something that?s keeping the memory from being freed during the recursion. When I instead spawn a new process that fetches a single JSON document, parses it, and then dies, I see memory (in top) spike by 100-200 mb, but settle back down fairly quickly after the process terminates. (sorry for the delayed response, my mail server at work seems to be blocking me from sending replies?) Thanks, Jordan On Apr 7, 2016, at 3:28 PM, Metin Akat wrote: Hello, I recently had the same problem in quite identical situation. In my case it was a load tester application that hammered onto a REST API. With several thousand bots when hitting a method that returns ~1M JSON, the VM started eating huge amounts of memory, until it crashed with out-of-memory exception. A very simple hack solved my problem (and that being a non-critical application I don't really care bout the implications): [garbage_collect(Pid) || Pid <- processes()]. I execute this every minute or so and my load tester never had the problem again. I imagine somehow it failed to trigger full-sweep GC ever and generational GC never cleared the old parsed JSON terms allocated in the processes' heap. But as it solved my problem, I never dug any deeper. Regards, Metin -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Fri Apr 8 17:35:54 2016 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 8 Apr 2016 17:35:54 +0200 Subject: [erlang-questions] how to report errata in the Erlang User's Guide? In-Reply-To: References: Message-ID: Goodness - this strikes me as an enormously heavy way to report a typo. If I saw a typo and to fix it I had to download MBytes of code then trawl around in the code and *hope* that I'd found the place to fix it I probably wouldn't bother. The best way I've seen to report typos (and any other error) is in the on-line version of real-world haskell - I suggest you take a look at the following link - and pretend you want to report an error/typo http://book.realworldhaskell.org/read/io.html Here you can click on the comment link on any paragraph and report an error or typo - there in NO signup, no git push this and that. If the wikipedia required you to download the entire sources, find the error then send a pull request to somebody there would be no wikipedia. Fixing a typo is a small contribution that anybody can make - even if they know nothing about git. Small contributions should require minimal effort to make. This is not the case. The next time we improve the documentation system we should make sure that suggesting improvements is really really easy. The Real World Haskell book site is still in my opinion the best way to manage collaborative inputs - unlike the wikipedia, it's a few authors many eyes model, which I like. Many eyes find the bugs and suggest improvements, but the final responsibility is that of the authors, so it has a point of view with which I can agree or disagree. BTW if anybody knows of a better example that RWH then please let me know Cheers /Joe On Fri, Apr 8, 2016 at 7:44 AM, Franklin Byaruhanga wrote: > Hello, Rich > > you would need to edit the page below and create a pull request > https://github.com/erlang/otp/blob/maint/lib/odbc/doc/src/databases.xml > > information on how to do that can be found on the link below. > https://github.com/erlang/otp/wiki/submitting-patches > > -- > > > Best Regards > Franklin Byaruhanga > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From garret.smith@REDACTED Fri Apr 8 17:41:16 2016 From: garret.smith@REDACTED (Garret Smith) Date: Fri, 8 Apr 2016 08:41:16 -0700 Subject: [erlang-questions] Should we deprecate or modernize log_mf_h and rb? In-Reply-To: References: <20160405084953.GA32163@seth> Message-ID: On Fri, Apr 8, 2016 at 5:36 AM, Siri Hansen wrote: > Thanks for the feedback, everyone! > > We have now decided not to remove log_mf_h and rb. Instead there will be > some work done to modernize these tools. > +1 I stopped using it a while back, but would probably resume if it were modernized. Intriguing idea: log_mf_h as a lager sink? > Kind Regards > /siri > From rexxe98@REDACTED Fri Apr 8 18:25:16 2016 From: rexxe98@REDACTED (Andrew Berman) Date: Fri, 08 Apr 2016 16:25:16 +0000 Subject: [erlang-questions] Should we deprecate or modernize log_mf_h and rb? In-Reply-To: References: <20160405084953.GA32163@seth> Message-ID: +1 on modernizing and the lager sink On Fri, Apr 8, 2016, 8:41 AM Garret Smith wrote: > On Fri, Apr 8, 2016 at 5:36 AM, Siri Hansen wrote: > > Thanks for the feedback, everyone! > > > > We have now decided not to remove log_mf_h and rb. Instead there will be > > some work done to modernize these tools. > > > > +1 > > I stopped using it a while back, but would probably resume if it were > modernized. > Intriguing idea: log_mf_h as a lager sink? > > > Kind Regards > > /siri > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From community-manager@REDACTED Fri Apr 8 18:52:49 2016 From: community-manager@REDACTED (Bruce Yinhe) Date: Fri, 8 Apr 2016 18:52:49 +0200 Subject: [erlang-questions] [ANN] Join the Erlang Slack team Message-ID: Hello all You might have heard of the Erlang Slack team we launched last month. A total of *695 have registered* and *23 chat channels* have been created. Connect with other Erlangers and join the conversations at erlanger.slack.com. Request invitation: https://bit.ly/ErlangSlack Visit Erlang Slack team: https://erlanger.slack.com/ [image: Inline image 1] [image: Inline image 2] *Bruce Yinhe* Community Manager Industrial Erlang User Group +46 72 311 43 89 community-manager@REDACTED -- Visit our Erlang community site ErlangCentral.org | @ErlangCentral | Industrial Erlang User Group -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Screen Shot 2016-04-08 at 18.32.53.png Type: image/png Size: 47129 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Screen Shot 2016-04-08 at 18.39.07.png Type: image/png Size: 58063 bytes Desc: not available URL: From mononcqc@REDACTED Fri Apr 8 18:56:22 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Fri, 8 Apr 2016 12:56:22 -0400 Subject: [erlang-questions] Any guidance on tweaking allocator settings? In-Reply-To: References: Message-ID: <20160408165622.GB13113@fhebert-ltm2.internal.salesforce.com> On 04/08, Jordan Day wrote: >Hi Metin, Fred, Lukas, > > >In a long-running process, I?m recursively calling into my function >for downloading a bunch of these JSON documents in what I *think* is a >tail-recursive manner, but I?m guessing I?m doing something that?s keeping >the memory from being freed during the recursion. When I instead spawn a >new process that fetches a single JSON document, parses it, and then dies, >I see memory (in top) spike by 100-200 mb, but settle back down fairly >quickly after the process terminates. One thing here that may be at play is how large the original JSON is. If what you're doing is parsing, say, 10 characters out of a 20MB JSON, what you may get is a slice off a off-heap binary that keeps the references alive and keeps the large JSON from ever being deallocated. Pick the fields you want and call `binary:copy(Bin)` to get a smaller verson disjoint from the large one. This may be enough to let the VM free memory from that point on. Regards, Fred. From wallentin.dahlberg@REDACTED Fri Apr 8 20:31:37 2016 From: wallentin.dahlberg@REDACTED (=?UTF-8?Q?Bj=C3=B6rn=2DEgil_Dahlberg?=) Date: Fri, 8 Apr 2016 20:31:37 +0200 Subject: [erlang-questions] Wanted additions to the maps module? Message-ID: Hi there! I would like to know if you have any desires that we extend the current maps module with additional functionality? Have you repeated some code or built your own private lib to handle certain maps specific tasks and thought "why isn't this in the maps module?" I would like to know! There is still time before 19.0 to extend the current API. We have for example discussed adding {get, put, update, remove}_path type of functions where we handle nested maps. Would this be useful? Would it be used? What have we missed? // Bj?rn-Egil -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Fri Apr 8 20:48:07 2016 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Fri, 8 Apr 2016 20:48:07 +0200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: Message-ID: A pop (also called take in ets:take2) operation would be helpful. It would return and remove a value from the map in one operation: -spec pop(Key, Map1) -> {Value, Map2} | error I use it quite frequently and implementing it today requires two lookups. A BIF would be handy and be able to perform it in one step. *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Fri Apr 8 20:53:06 2016 From: ingela.andin@REDACTED (Ingela Andin) Date: Fri, 8 Apr 2016 20:53:06 +0200 Subject: [erlang-questions] bug : ssl losing ciphers In-Reply-To: References: Message-ID: Hi! 2016-04-08 16:13 GMT+02:00 Nicolas Thauvin : > Hi, > > We've been trying to restrict SSL ciphers to a secure set in Yaws / OTP > R18, but only a few of them were actually taken into account (leading to > connection issues from old browsers). > > According to the documentation, one can list the availables ciphers with > ssl:cipher_suites(). > For example: > > [... > {rsa,aes_256_gcm,null,sha384}, > {rsa,aes_256_cbc,sha256}, > ...] > > Note there are 3-tuples and 4-tuples in the result. > > Now, when the customised 'ciphers' SSL option is set, its content is > processed by ssl:binary_cipher_suites/2 > > (Beam you up : > https://github.com/erlang/otp/blob/maint-18/lib/ssl/src/ssl.erl#L1092) > > There comes the issue : this function expects all the entries to be the > same tuple size (3 or 4) according to a matching on the first element, > losing entries from the list when they don't match the tuple size. > > The patch for ssl:binary_cipher_suites/2 is trivial, but why does > ssl_cipher:suite() still returns a mixed-size of tuples since 4-tuples > seems to be considered as backward compatible according to the comments ? > > As of TLS-1.2 ciphersuites are a set of 4 algorithms. In earlier versions the set was three algorithms and the forth was implicitly hardcoded. So all cipher suites are represented as 4-tuples internaly but for backwards compatibility we need to be able to input old cipher suites as 3-tuples. However I thought the comment was a bit percuiler (it suggests its the other way around), and I looked into it and it turns out a very long time ago the cipher suites had a different forth element 'no_export', but then we did decide not to implement any export ciphers and the tuples became 3-tuples. And much later came TLS-1.2. So we need to fix that bug and remove that comment. Acctualy I reasently fixed the ssl:cipher_suites(), as it wrongly filtered the new 4-tuple ciphers so it returned always 3-tuples, and I do not think that will worked out greatly either. Regards Ingela Erlang/OTP team - Ericsson AB -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Fri Apr 8 21:01:03 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Fri, 8 Apr 2016 21:01:03 +0200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: Message-ID: <5707FFEF.6060203@ninenines.eu> Iterators and map comprehensions would be great. For example I receive JSON or equivalent coming in, convert to map and want to check the values (and sometimes convert types), and right now it goes through two conversions (to and from list). I know it's not functionality per se but it's the limitation I'm most often seeing. It's followed by the map key in function head issue (the one binaries also have) and deep updates is only third. I'll also +1 Jose's suggestion, that's a useful one to have. Cheers, On 04/08/2016 08:31 PM, Bj?rn-Egil Dahlberg wrote: > Hi there! > > I would like to know if you have any desires that we extend the current > maps module with additional functionality? > > Have you repeated some code or built your own private lib to handle > certain maps specific tasks and thought "why isn't this in the maps module?" > > I would like to know! > > There is still time before 19.0 to extend the current API. > > We have for example discussed adding {get, put, update, > remove}_path type of functions where we handle nested maps. Would this > be useful? Would it be used? > > What have we missed? > > // Bj?rn-Egil > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From felixgallo@REDACTED Fri Apr 8 21:25:08 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Fri, 8 Apr 2016 12:25:08 -0700 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <5707FFEF.6060203@ninenines.eu> References: <5707FFEF.6060203@ninenines.eu> Message-ID: Concurring with Lo?c and Jos?, all of their ideas would be outstanding. Perhaps slightly longer term/more generically, it would be nice if a maps Lens implementation made its way into core. ROK's lens.erl ( http://www.cs.otago.ac.nz/staffpriv/ok/lens.erl) is a pretty great start, but shipping a blessed Lens would be even better. F. On Fri, Apr 8, 2016 at 12:01 PM, Lo?c Hoguin wrote: > Iterators and map comprehensions would be great. > > For example I receive JSON or equivalent coming in, convert to map and > want to check the values (and sometimes convert types), and right now it > goes through two conversions (to and from list). > > I know it's not functionality per se but it's the limitation I'm most > often seeing. It's followed by the map key in function head issue (the one > binaries also have) and deep updates is only third. I'll also +1 Jose's > suggestion, that's a useful one to have. > > Cheers, > > > On 04/08/2016 08:31 PM, Bj?rn-Egil Dahlberg wrote: > >> Hi there! >> >> I would like to know if you have any desires that we extend the current >> maps module with additional functionality? >> >> Have you repeated some code or built your own private lib to handle >> certain maps specific tasks and thought "why isn't this in the maps >> module?" >> >> I would like to know! >> >> There is still time before 19.0 to extend the current API. >> >> We have for example discussed adding {get, put, update, >> remove}_path type of functions where we handle nested maps. Would this >> be useful? Would it be used? >> >> What have we missed? >> >> // Bj?rn-Egil >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -- > Lo?c Hoguin > http://ninenines.eu > Author of The Erlanger Playbook, > A book about software development using Erlang > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pjotr.public12@REDACTED Fri Apr 8 21:46:21 2016 From: pjotr.public12@REDACTED (Pjotr Prins) Date: Fri, 8 Apr 2016 21:46:21 +0200 Subject: [erlang-questions] Reproducible and deterministic builds for GNU Guix (and Nix) In-Reply-To: References: <20160408065934.GA18715@thebird.nl> Message-ID: <20160408194621.GA21757@thebird.nl> On Fri, Apr 08, 2016 at 05:06:49PM +0200, Joe Armstrong wrote: > For your purposes I can write a script to call erlc and strip out the > parts that make compilation reproducible. In the long term we should > discuss this, figure out what the correct thing to do is, and then do it. On the Erlang mailing list there is a discussion about removing time stamps and creating a compile time switch for that. That would do, or a post-processing script/hack would work for the time being - we can easily incorporate that. The sooner we can do reproducible builds, the sooner we can have Erlang and Elixir in Guix :) BTW, having a SHA value inside beam/object files may sound useful, but in the context of GNU Guix, and Nix, compiled files are immutable - i.e. we guarantee they can not be modified (well, there may be a way by overriding ro permissions, but that would mean a built in SHA can also be changed). I guess you need it for hot reloading, but can't you calculate the SHA on the fly for that? You'd need to open beam files anyway to get the value(s) and they will be loaded by the OS unless they are very large (> 8Mb). I'd go by file stamps and if different do a checksum. It only has to happen once for the original file. Pj. From pjotr.public12@REDACTED Fri Apr 8 21:52:56 2016 From: pjotr.public12@REDACTED (Pjotr Prins) Date: Fri, 8 Apr 2016 21:52:56 +0200 Subject: [erlang-questions] Reproducible and deterministic builds for GNU Guix (and Nix) In-Reply-To: <20160408194621.GA21757@thebird.nl> References: <20160408065934.GA18715@thebird.nl> <20160408194621.GA21757@thebird.nl> Message-ID: <20160408195256.GA21804@thebird.nl> > also be changed). I guess you need it for hot reloading, but can't you > calculate the SHA on the fly for that? You'd need to open beam files > anyway to get the value(s) and they will be loaded by the OS unless Aw, sorry Joe, my bad. The stored SHA value is calculated on the original source file, so that is not opened. Guix/Nix calculate their SHA values over the source and build environment and that path is stored in an immutable store. So, you already have a strong guarantee that the sands don't shift under you. But then people will use Erlang outside GNU Guix - and there a SHA value makes sense if you watn to have a guarantee that it has the same source. Much better than a time stamp. Pj. From carlsson.richard@REDACTED Fri Apr 8 22:42:07 2016 From: carlsson.richard@REDACTED (Richard Carlsson) Date: Fri, 8 Apr 2016 22:42:07 +0200 Subject: [erlang-questions] Feedback wanted: Remove compilation times from BEAM files in OTP 19? In-Reply-To: References: Message-ID: Hi Serge (and everyone else)! I've actually got an improved version of that module change detection (i.e., based on md5) that I was going to clean up and submit to OTP as a standard part of the shell. I think the original snippets have been drifting around on the interwebs for aeons, and I'm not sure who wrote them originally. We've been using them at Klarna forever. But we needed something more reliable (that works even if you have made a completely new build of every module, but not actually changing more than a few), so I started by pushing some patches to OTP that made the md5 of loaded modules easily available in module_info. Then based on that I wrote an improved shell function like you just did - we've now tried it out for a while and found it stable, so it's a good time to push this as well. I think my only hesitation is where to put the core "module changed" functionality if I'm to submit it as part of OTP: the code module? beam_lib? somewhere else? Any opinions out there? /Richard 2016-04-08 15:37 GMT+02:00 Serge Aleynikov : > ?Thank you! > > I updated the > https://github.com/saleyn/util/blob/master/src/user_default.erl per your > suggestion. > > On Fri, Apr 8, 2016 at 8:54 AM, Bj?rn Gustavsson wrote: > >> On Fri, Apr 8, 2016 at 2:28 PM, Serge Aleynikov >> wrote: >> > I've been relying on module's compilation time for figuring out the >> changed >> > modules (see below). >> > >> > In the absence of {time, CompileTime}, would there be a way to >> determine the >> > modification time stamp of the file at the time it was loaded by code >> > loader? >> >> Yes. Use Mod:module_info(md5) to calculate MD5 for >> the loaded module and compare it to beam_lib:md5(Mod). >> >> Example: >> >> 13> c:module_info(md5). >> <<79,26,188,243,168,60,58,45,34,69,19,222,138,190,214,118>> >> 14> beam_lib:md5(code:which(c)). >> {ok,{c,<<79,26,188,243,168,60,58,45,34,69,19,222,138, >> 190,214,118>>}} >> >> /Bjorn >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serge@REDACTED Fri Apr 8 23:56:30 2016 From: serge@REDACTED (Serge Aleynikov) Date: Fri, 8 Apr 2016 17:56:30 -0400 Subject: [erlang-questions] Feedback wanted: Remove compilation times from BEAM files in OTP 19? In-Reply-To: References: Message-ID: Hi Richard, It seems to me that code module would be the proper place for it, since that detection is directly related to code loading, which in turn could call the beam_lib's helper functions. Serge On Fri, Apr 8, 2016 at 4:42 PM, Richard Carlsson wrote: > Hi Serge (and everyone else)! I've actually got an improved version of > that module change detection (i.e., based on md5) that I was going to clean > up and submit to OTP as a standard part of the shell. I think the original > snippets have been drifting around on the interwebs for aeons, and I'm not > sure who wrote them originally. We've been using them at Klarna forever. > But we needed something more reliable (that works even if you have made a > completely new build of every module, but not actually changing more than a > few), so I started by pushing some patches to OTP that made the md5 of > loaded modules easily available in module_info. Then based on that I wrote > an improved shell function like you just did - we've now tried it out for a > while and found it stable, so it's a good time to push this as well. > > I think my only hesitation is where to put the core "module changed" > functionality if I'm to submit it as part of OTP: the code module? > beam_lib? somewhere else? Any opinions out there? > > /Richard > > 2016-04-08 15:37 GMT+02:00 Serge Aleynikov : > >> ?Thank you! >> >> I updated the >> https://github.com/saleyn/util/blob/master/src/user_default.erl per your >> suggestion. >> >> On Fri, Apr 8, 2016 at 8:54 AM, Bj?rn Gustavsson >> wrote: >> >>> On Fri, Apr 8, 2016 at 2:28 PM, Serge Aleynikov >>> wrote: >>> > I've been relying on module's compilation time for figuring out the >>> changed >>> > modules (see below). >>> > >>> > In the absence of {time, CompileTime}, would there be a way to >>> determine the >>> > modification time stamp of the file at the time it was loaded by code >>> > loader? >>> >>> Yes. Use Mod:module_info(md5) to calculate MD5 for >>> the loaded module and compare it to beam_lib:md5(Mod). >>> >>> Example: >>> >>> 13> c:module_info(md5). >>> <<79,26,188,243,168,60,58,45,34,69,19,222,138,190,214,118>> >>> 14> beam_lib:md5(code:which(c)). >>> {ok,{c,<<79,26,188,243,168,60,58,45,34,69,19,222,138, >>> 190,214,118>>}} >>> >>> /Bjorn >>> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn@REDACTED Sat Apr 9 08:48:59 2016 From: bjorn@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Sat, 9 Apr 2016 08:48:59 +0200 Subject: [erlang-questions] Feedback wanted: Remove compilation times from BEAM files in OTP 19? In-Reply-To: References: Message-ID: On Fri, Apr 8, 2016 at 10:42 PM, Richard Carlsson wrote: [...] > > I think my only hesitation is where to put the core "module changed" > functionality if I'm to submit it as part of OTP: the code module? beam_lib? > somewhere else? Any opinions out there? > I think the code module is the best place. /Bjorn -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From roger@REDACTED Sat Apr 9 11:12:05 2016 From: roger@REDACTED (Roger Lipscombe) Date: Sat, 9 Apr 2016 10:12:05 +0100 Subject: [erlang-questions] bug : ssl losing ciphers In-Reply-To: References: Message-ID: On 8 April 2016 at 15:13, Nicolas Thauvin wrote: > Now, when the customised 'ciphers' SSL option is set, its content is > processed by ssl:binary_cipher_suites/2 Rather than use the tuples, we use named suites. See, e.g., http://ezgr.net/increasing-security-erlang-ssl-cowboy/ From s.j.thompson@REDACTED Sat Apr 9 12:02:15 2016 From: s.j.thompson@REDACTED (Simon Thompson) Date: Sat, 9 Apr 2016 11:02:15 +0100 Subject: [erlang-questions] SBLP 2016 Deadline Extension References: <201604090338.u393ch7J005417@easychair.org> Message-ID: We'd like to let you know that, due to multiple requests, we have extended the deadline for both abstract and paper submissions to SBLP. The new dates are the following: Abstract submission (extended!): April 22nd 2016 Paper submission (extended!): April 29th 2016 Author notification: June 10th 2016 Camera ready deadline: June 24th 2016 Please distribute the attached call for papers (with the extended deadlines) wherever you may find appropriate. Do not hesitate to contact us if you have any questions. Kind regards, Fernando Castor Yu David Liu The Brazilian Symposium on Programming Languages is a well-established symposium which provides a venue for researchers and practitioners interested in the fundamental principles and innovations in the design and implementation of programming languages and systems. SBLP 2016 will be held in Maring?, in the Southern region of Brazil, and will be the 20th edition of the symposium. SBLP is part of the 7th edition of CBSoft, the Brazilian Congress on Software: Theory and Practice. More information is available at http://cbsoft.org/sblp2016/xx-brazilian-symposium-on-programming-languages. IMPORTANT DATES Abstract submission: April 22nd 2016 Paper submission: April 29th 2016 Author notification: June 10th 2016 Camera ready deadline: June 24th 2016 Symposium dates: September 22nd and 23rd Authors are invited to submit original research on any relevant topic which can be either in the form of regular or short papers. TOPICS Topics of interest include, but are not limited to: - Program generation and transformation, including domain-specific languages, and model-driven development in the context of programming languages. - Programming paradigms and styles, including functional, object-oriented, aspect-oriented, scripting languages, real-time, service-oriented, multithreaded, parallel, and distributed programming. - Formal semantics and theoretical foundations, including denotational, operational, algebraic, and categorical. - Program analysis and verification, including type systems, static analysis, and abstract interpretation. - Programming language design and implementation, including new programming models, programming language environments, compilation, and interpretation techniques. SUBMISSION AND PUBLICATION All submissions will be peer-reviewed and judged on the basis of its originality, contribution to the field, technical and presentation quality, and relevance to the symposium. Contributions should be written in Portuguese or English. Papers should fall into one of two different categories: regular papers, which can be up to 15 pages long in LNCS format, or short papers, with up to 6 pages in LNCS format. Short papers can discuss new ideas which are at an early stage of development and which have not yet been thoroughly evaluated. We encourage the submission of short papers reporting partial results of on-going master dissertations or doctoral theses. Accepted papers written in English will be published in a volume of Lecture Notes in Computer Science (LNCS), by Springer. Both regular and short papers must be prepared using the LNCS format, available at http://www.springer.com/computer/lncs?SGWID=0-164-6-793341-0. Papers must be submitted electronically (in PDF format) via the Easychair System: http://www.easychair.org/conferences/?conf=sblp2016. As in previous editions, after the conference, authors of selected regular papers will be invited to submit an extended version of their work to be considered for publication in a journal special issue. Since 2009, selected papers of each SBPL edition are being published in a special issue of Science of Computer Programming, by Elsevier. PROGRAM CHAIRS Fernando Castor, Federal University of Pernambuco Yu David Liu, State University of New York, Binghamton PROGRAM COMMITTEE Luis Barbosa, University of Minho Thiago Tonelli Bartolomei, LogicBlox Mariza Bigonha, Federal University of Minas Gerais Roberto Bigonha, Federal University of Minas Gerais Andre Rauber Du Bois, Federal University of Pelotas Christiano Braga, Fluminense Federal University Carlos Camar?o, Federal University of Minas Gerais Francisco Carvalho-Junior, Federal University of Ceara Fernando Castor, Federal University of Pernambuco Marcelo D'Amorim, Federal University of Pernambuco Jo?o Paulo Fernandes, University of Beira Interior Jo?o Ferreira, Teesside University Lucilia Figueiredo, Federal University of Ouro Preto Ismael Figueroa, Pontificia Universidad Cat?lica de Valparaiso Alex Garcia, IME Rodrigo Geraldo, Federal University of Ouro Preto Roberto Ierusalimschy, PUC-Rio Rafael Lins, Federal University of Pernambuco Yu David Liu, State University of New York at Binghamton Hans-Wolfgang Loidl, Heriot-Watt University Marcelo Maia, Federal University of Uberl?ndia Manuel-A. Martins, University of Aveiro Fabio Mascarenhas, Federal University of Rio de Janeiro S?rgio Medeiros, Federal University of Rio Grande do Norte Ana Milanova, Rensselaer Polytechnic Institute Alvaro Moreira, Federal University of Rio Grande do Sul Martin Musicante, Federal University of Rio Grande do Norte Bruno Oliveira, The University of Hong Kong Zachary Palmer, Swarthmore College Alberto Pardo, Universidad de la Rep?blica Fernando Pereira, Federal University of Minas Gerais Gustavo Pinto, Federal Institute of Science and Technology of Para Louis-Noel Pouchet, University of California Zongyan Qiu, Peking University Sandro Rigo, State University of Campinas Noemi Rodriguez, PUC-Rio Jo?o Saraiva, University of Minho Doaitse Swierstra, Utrecht University Leopoldo Teixeira, Federal University of Pernambuco Simon Thompson, University of Kent Varmo Vene, University of Tartu From erlang@REDACTED Sat Apr 9 12:45:24 2016 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 9 Apr 2016 12:45:24 +0200 Subject: [erlang-questions] Feedback wanted: Remove compilation times from BEAM files in OTP 19? In-Reply-To: References: Message-ID: I said this in another thread but I'll repeat it here: I think we need repeatable builds so that if we compile the same module several timeswith the same macros and library versions we should get a bit identical beam file (so we can hash the beam content and use the hash as the key in a version control system) What I don't know is how we should interpret the statement "the same module" - if I correct a typo in a comment is the result "the same module"? I would argue that it is is not. I think a hash (md5, sha1 etc) of the source and all the macros etc should be added to the beam file. That way we can do a post hoc analysis of the beam code and tie together exactly which version of the source was used. /Joe On Sat, Apr 9, 2016 at 8:48 AM, Bj?rn Gustavsson wrote: > On Fri, Apr 8, 2016 at 10:42 PM, Richard Carlsson > wrote: > [...] >> >> I think my only hesitation is where to put the core "module changed" >> functionality if I'm to submit it as part of OTP: the code module? beam_lib? >> somewhere else? Any opinions out there? >> > > I think the code module is the best place. > > /Bjorn > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ingela.andin@REDACTED Sat Apr 9 13:16:54 2016 From: ingela.andin@REDACTED (Ingela Andin) Date: Sat, 9 Apr 2016 13:16:54 +0200 Subject: [erlang-questions] bug : ssl losing ciphers In-Reply-To: References: Message-ID: Hi! Yes, the OpenSSL name strings are supported too. I would rather have the tuples be maps, and from 19 that will be ok. An other idea might be to put the macros for the binary representation of the cipher suites in an official include file. Then your list could look something like [?TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, ?TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384] which are the names from the TLS RFCs and similar to the openssl string names, but not the same. In this case we would need no conversion. We will see. Regards Ingela Erlang/OTP team - Ericsson AB 2016-04-09 11:12 GMT+02:00 Roger Lipscombe : > On 8 April 2016 at 15:13, Nicolas Thauvin wrote: > > Now, when the customised 'ciphers' SSL option is set, its content is > > processed by ssl:binary_cipher_suites/2 > > Rather than use the tuples, we use named suites. See, e.g., > http://ezgr.net/increasing-security-erlang-ssl-cowboy/ > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Sat Apr 9 14:35:49 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 9 Apr 2016 14:35:49 +0200 Subject: [erlang-questions] Feedback wanted: Remove compilation times from BEAM files in OTP 19? In-Reply-To: References: Message-ID: On Sat, Apr 9, 2016 at 12:45 PM, Joe Armstrong wrote: > I think we need repeatable builds so that if we compile the same > module several timeswith the same macros and library versions we > should get > a bit identical beam file (so we can hash the beam content and use the hash > as the key in a version control system) > Other valuable things you can do: * Catch compilers which have been built incorrectly or are executing on faulty hardware * Verify that two different compiler installations are the same. Sometimes this can lead to situations where you are hunting a bug only to learn later different compilers are used * Catch maliciously altered compilers The same model is what drives BitTorrent. A BitTorrent file is essentially a map: #{ info => ..., trackers => [{tracker, ...}, ...] } The "infohash" is the SHA1 checksum of the "info" block, but certain parts are left out of that block, notably the tracker list. It allows operators to alter the tracker list and add their own tracker, without changing the content of the torrent file. In other words, integrity is provided where it matter, but freedom is given to alter the parts around the integrity-protected parts if necessary. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Sat Apr 9 16:57:21 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Sat, 09 Apr 2016 14:57:21 +0000 Subject: [erlang-questions] dirty nifs vs custom pool of threads for a database Message-ID: Would dirty nifs be a good use case to handle call to an embedded database? Say for example if we want to replace the usage of the eleveldb threadpool in eleveldb by just marking async tasks as IO bound and make them synchronous? Is - benoit -------------- next part -------------- An HTML attachment was scrubbed... URL: From rickp@REDACTED Sun Apr 10 14:40:08 2016 From: rickp@REDACTED (Rick Payne) Date: Sun, 10 Apr 2016 08:40:08 -0400 Subject: [erlang-questions] Problems with percept and 18.3 Message-ID: Using the 18.3 standard build from Erlang Solutions on OSX 10.11.4, if I try and use percept (or percept2 for that matter) on my application, I?m running into a beam assert: (n1@REDACTED)1> percept:profile(?perf.dat", [procs]). Starting profiling. {ok,#Port<0.3421>} (n1@REDACTED)2> TYPE ASSERTION FAILED, file beam/erl_term.c, line 116: tag_val_def: 0x0 [os_mon] cpu supervisor port (cpu_sup): Erlang has closed [End] It doesn?t happen if I just start erlang and then percept, so something my application is doing is triggering the crash, but I?m struggling to know what it could be. So is anyone actually using percept on erlang 18.3? Or any suggestions as to what I?m hitting and should look at? Cheers, Rick From khitai.pang@REDACTED Sun Apr 10 23:04:29 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Mon, 11 Apr 2016 05:04:29 +0800 Subject: [erlang-questions] mnesia:dirty_write 'function not exported' Message-ID: Dear list, I met a strange problem that I can't figure out why: When I use ok = mnesia:drity_write(#my_record{key=Key, value=Value}) I get ** {'function not exported', [{mnesia,drity_write, [my_record, {my_record, ... but when I use Fun = fun() -> mnesia:write(#my_record{key=Key, value=Value}) end, {atomic, ok} = mnesia:transaction(Fun) Everything works fine. Any idea? Thanks Khitai From Oliver.Korpilla@REDACTED Sun Apr 10 23:18:43 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Sun, 10 Apr 2016 23:18:43 +0200 Subject: [erlang-questions] mnesia:dirty_write 'function not exported' In-Reply-To: References: Message-ID: Hello, Khitai. You misspelled "dirty" in your first version... Cheers, Oliver ? ? Gesendet:?Sonntag, 10. April 2016 um 23:04 Uhr Von:?"Khitai Pang" An:?"Erlang-Questions Questions" Betreff:?[erlang-questions] mnesia:dirty_write 'function not exported' Dear list, I met a strange problem that I can't figure out why: When I use ok = mnesia:drity_write(#my_record{key=Key, value=Value}) I get ** {'function not exported', [{mnesia,drity_write, [my_record, {my_record, ... but when I use Fun = fun() -> mnesia:write(#my_record{key=Key, value=Value}) end, {atomic, ok} = mnesia:transaction(Fun) Everything works fine. Any idea? Thanks Khitai _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions From khitai.pang@REDACTED Sun Apr 10 23:34:25 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Mon, 11 Apr 2016 05:34:25 +0800 Subject: [erlang-questions] ssl_closed not always received Message-ID: My erlang server process is a gen_server, it depends on ssl_closed, when it receives ssl_closed, it should terminate immediately. My erlang client process is also a gen_server running on a remote host, it has ssl:close(Socket) in terminate(). It connects to the server process and establishes a ssl connection. When I quit the entire erlang vm where my erlang client process runs, sometimes the server process receives ssl_closed, sometimes it doesn't. Why? How to make sure that the server process always get ssl_closed when the client process on a remote host quits? Thanks Khitai From sdl.web@REDACTED Mon Apr 11 05:43:48 2016 From: sdl.web@REDACTED (Leo Liu) Date: Mon, 11 Apr 2016 11:43:48 +0800 Subject: [erlang-questions] ...ANY KIND, either express or implied Message-ID: In many module headers in OTP there is this sentence: %% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. I wonder if `express' should be changed to `expressed'. Sorry for the nitpick. Leo From ok@REDACTED Mon Apr 11 06:24:04 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Mon, 11 Apr 2016 16:24:04 +1200 Subject: [erlang-questions] ...ANY KIND, either express or implied In-Reply-To: References: Message-ID: <4529c198-0fb1-cf3f-6496-253006f03272@cs.otago.ac.nz> On 11/04/16 3:43 pm, Leo Liu wrote: > In many module headers in OTP there is this sentence: > > %% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. > > I wonder if `express' should be changed to `expressed'. No. OED "express, adj" sense 3a: Of a meaning, purpose, stipulation, law, etc.: Expressed and not merely implied; definitely formulated; definite, explicit. Of language, statements, indications: Definite, unmistakable in import. Standard legal language. You find exactly the same words in the MIT "License", for example. From lee.sylvester@REDACTED Mon Apr 11 06:25:40 2016 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Mon, 11 Apr 2016 16:25:40 +1200 Subject: [erlang-questions] ...ANY KIND, either express or implied In-Reply-To: <4529c198-0fb1-cf3f-6496-253006f03272@cs.otago.ac.nz> References: <4529c198-0fb1-cf3f-6496-253006f03272@cs.otago.ac.nz> Message-ID: Legalese is a language unto itself. Could do with a compiler and interpreter if someones up for it ;-) On Mon, Apr 11, 2016 at 4:24 PM, Richard A. O'Keefe wrote: > > > On 11/04/16 3:43 pm, Leo Liu wrote: > >> In many module headers in OTP there is this sentence: >> >> %% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express >> or implied. >> >> I wonder if `express' should be changed to `expressed'. >> > No. > > OED "express, adj" sense 3a: > Of a meaning, purpose, stipulation, law, etc.: > Expressed and not merely implied; definitely formulated; > definite, explicit. Of language, statements, indications: > Definite, unmistakable in import. > > Standard legal language. You find exactly the same words in > the MIT "License", for example. > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sdl.web@REDACTED Mon Apr 11 06:32:29 2016 From: sdl.web@REDACTED (Leo Liu) Date: Mon, 11 Apr 2016 12:32:29 +0800 Subject: [erlang-questions] ...ANY KIND, either express or implied References: <4529c198-0fb1-cf3f-6496-253006f03272@cs.otago.ac.nz> Message-ID: On 2016-04-11 16:24 +1200, Richard A. O'Keefe wrote: > OED "express, adj" sense 3a: > Of a meaning, purpose, stipulation, law, etc.: > Expressed and not merely implied; definitely formulated; > definite, explicit. Of language, statements, indications: > Definite, unmistakable in import. > > Standard legal language. You find exactly the same words in > the MIT "License", for example. Thanks for the clarification. Leo From stefan.houtzager@REDACTED Mon Apr 11 09:22:11 2016 From: stefan.houtzager@REDACTED (Stefan Houtzager) Date: Mon, 11 Apr 2016 09:22:11 +0200 Subject: [erlang-questions] How to get wxerlang working in in Ubuntu 14.04.4 LTS Message-ID: Hi, I would appreciate an installscript for a recent erlang version on Ubuntu 14.04.4 LTS with wxerlang working. I am a starting elixir developer (and for that need erlang). I would like to get wxerlang working and created a dockerimage to make my installation easy portable. I use otp_src_18.3.tar.gz and install several wx packages and others, I'm not sure if I miss some or install some that are not needed. Anyway, after the install the output of which wx-config && wx-config --version-full is /usr/bin/wx-config 2.8.12.1 But wx:demo(). results in an error: Eshell V7.3 (abort with ^G) 1> wx:demo(). Error: Unable to initialize gtk, is DISPLAY set properly? ok 2> The most important installs in the dockerfile (just linux commands after the RUN statements, phusion/baseimage:0.9.18 is Ubuntu 14.04.4 LTS reworked for docker) I copied below, the full dockerfile and the logging of the output I arttached. FROM phusion/baseimage:0.9.18 MAINTAINER Stefan Houtzager # Important! Update this no-op ENV variable when this Dockerfile # is updated with the current date. It will force refresh of all # of the base images and things like `apt-get update` won't be using # old cached versions when the Dockerfile is built. ENV REFRESHED_AT 05-04-2016 # Set correct environment variables. # Setting ENV HOME does not seem to work currently. HOME is unset in Docker container. # See bug : https://github.com/phusion/baseimage-docker/issues/119 #ENV HOME /root # Workaround: RUN echo /root > /etc/container_environment/HOME # Regenerate SSH host keys. baseimage-docker does not contain any, so you # have to do that yourself. You may also comment out this instruction; the # init system will auto-generate one during boot. RUN /etc/my_init.d/00_regen_ssh_host_keys.sh # Use baseimage-docker's init system. CMD ["/sbin/my_init"] # ...put your own build instructions here... # Set the locale RUN locale-gen en_US.UTF-8 ENV LANG en_US.UTF-8 ENV LANGUAGE en_US:en ENV LC_ALL en_US.UTF-8 WORKDIR /tmp # See : https://github.com/phusion/baseimage-docker/issues/58 RUN echo 'debconf debconf/frontend select Noninteractive' | debconf-set-selections # update and install some software requirements RUN apt-get update && apt-get upgrade -y && apt-get install -y curl wget git unzip autoconf make g++ gcc fop libxml2-utils xsltproc WORKDIR / RUN add-apt-repository ppa:openjdk-r/ppa RUN apt-get update && apt-get install openjdk-8-jdk -y WORKDIR / # install nano (and pico) RUN apt-get update && apt-get install nano -y # Needed for terminal handling (libc-dev libncurses5 libtinfo-dev libtinfo5 ncurses-bin) RUN apt-get -y install libncurses5-dev openssl libssl-dev m4 libncurses-dev libgmp3-dev libglu-dev # For building with wxWidgets RUN apt-get -y install freeglut3-dev wx2.8-headers wx-common libwxbase2.8 libwxgtk2.8-dev libqt4-opengl-dev libgtk2.0-dev libgl1-mesa-dev libglu1-mesa-dev libpng3 # For building ssl (libssh-4 libssl-dev zlib1g-dev) RUN apt-get -y install libssh-dev # ODBC support (libltdl3-dev odbcinst1debian2 unixodbc) RUN apt-get -y install unixodbc-dev RUN apt-get -y install ncurses-dev RUN wget http://www.erlang.org/download/otp_src_18.3.tar.gz RUN tar -xvzf otp_src_18.3.tar.gz RUN chmod -R 777 otp_src_18.3 WORKDIR otp_src_18.3 RUN ./configure RUN make RUN make install RUN cd .. && rm -R otp_src_18.3/ RUN echo "ERLANG_HOME=/usr/local/lib/erlang" >> ~/.bashrc RUN echo "export PATH=$PATH:$ERLANG_HOME/bin" >> ~/.bashrc RUN echo "export DISPLAY=:100.0" >> ~/.bashrc RUN apt-get purge --auto-remove openjdk-8-jdk -y RUN rm -R /usr/lib/jvm -- Kind regards, Stefan Houtzager Houtzager ICT consultancy & development www.linkedin.com/in/stefanhoutzager -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Dockerfile Type: application/octet-stream Size: 5614 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: build.log Type: text/x-log Size: 1965844 bytes Desc: not available URL: From Oliver.Korpilla@REDACTED Mon Apr 11 09:51:12 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Mon, 11 Apr 2016 09:51:12 +0200 Subject: [erlang-questions] UDP Multicast - how does it work? Message-ID: Hello. I'm lost on the basics here. Sorry if this seems like a fishing expedition. What I want to do: Step 1) Reach several processes on the same machine with one send. Step 2) Reach several processes on possibly different machines with one send. I've tried the following: -module(test_multicast). -export([start/0, receiver/1]). -define(MULTICAST_ADDRESS, {127,0,10,1}). receiver(Id) -> io:format("Started multicast receiver ~p~n", [Id]), {ok, Socket} = gen_udp:open(3333, [binary, {active, true}, {reuseaddr, true}, {multicast_if, ?MULTICAST_ADDRESS}, {multicast_loop, true}]), receiver_loop(Id, Socket). receiver_loop(Id, Socket) -> receive {udp, _RemoteSocket, _RemoteHost, _RemotePort, Payload } -> io:format("Receiver ~p received message ~p~n", [Id, Payload]), receiver_loop(Id, Socket) end. send() -> io:format("Sending...~n"), {ok, Socket} = gen_udp:open(0), ok = gen_udp:send(Socket, ?MULTICAST_ADDRESS, 3333, << "Hello, World." >>). start() -> spawn_link(?MODULE, receiver, [1]), spawn_link(?MODULE, receiver, [2]), timer:sleep(100), send(). But the only one to receive the message is receiver #2. It's as if it takes the port away from receiver #1... I thought reuseaddr was meant to bind several sockets to the same port? Thank you for any advice, Oliver From boris.muehmer@REDACTED Mon Apr 11 10:00:22 2016 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Mon, 11 Apr 2016 10:00:22 +0200 Subject: [erlang-questions] How to get wxerlang working in in Ubuntu 14.04.4 LTS In-Reply-To: References: Message-ID: Hello Stefan, You need to install *libwxbase3.0-dev* and *libwxgtk3.0-dev*. I also believe You should install *libglu1-mesa-dev*. I have it in my erlang setup dependency list, but I can not remember why this was needed. It is possible that the gl demo did not work without it. But "my dependency list" started with 12.04, and some things were manuall changed/configured with 14.04. Maybe some things are automatically resolved now (like libwxgtk3.0-dev automatically includes libwxbase3.0-dev and libglu1-mesa-dev). Regards, Boris 2016-04-11 9:22 GMT+02:00 Stefan Houtzager : > Hi, > > I would appreciate an installscript for a recent erlang version on Ubuntu > 14.04.4 LTS with wxerlang working. > I am a starting elixir developer (and for that need erlang). I would > like to get wxerlang working and created a dockerimage to make my > installation easy portable. I use otp_src_18.3.tar.gz and install several > wx packages and others, I'm not sure if I miss some or install some that > are not needed. Anyway, after the install the output of which wx-config > && wx-config --version-full is > > /usr/bin/wx-config > 2.8.12.1 > > But wx:demo(). results in an error: > > Eshell V7.3 (abort with ^G) > 1> wx:demo(). > Error: Unable to initialize gtk, is DISPLAY set properly? > ok > 2> > > The most important installs in the dockerfile (just linux commands after > the RUN statements, phusion/baseimage:0.9.18 is Ubuntu 14.04.4 LTS reworked > for docker) I copied below, the full dockerfile and the logging of the > output I arttached. > > > FROM phusion/baseimage:0.9.18 > MAINTAINER Stefan Houtzager > > # Important! Update this no-op ENV variable when this Dockerfile > # is updated with the current date. It will force refresh of all > # of the base images and things like `apt-get update` won't be using > # old cached versions when the Dockerfile is built. > ENV REFRESHED_AT 05-04-2016 > > # Set correct environment variables. > > # Setting ENV HOME does not seem to work currently. HOME is unset in > Docker container. > # See bug : https://github.com/phusion/baseimage-docker/issues/119 > #ENV HOME /root > # Workaround: > RUN echo /root > /etc/container_environment/HOME > > # Regenerate SSH host keys. baseimage-docker does not contain any, so you > # have to do that yourself. You may also comment out this instruction; the > # init system will auto-generate one during boot. > RUN /etc/my_init.d/00_regen_ssh_host_keys.sh > > # Use baseimage-docker's init system. > CMD ["/sbin/my_init"] > > # ...put your own build instructions here... > > # Set the locale > RUN locale-gen en_US.UTF-8 > ENV LANG en_US.UTF-8 > ENV LANGUAGE en_US:en > ENV LC_ALL en_US.UTF-8 > > WORKDIR /tmp > > # See : https://github.com/phusion/baseimage-docker/issues/58 > RUN echo 'debconf debconf/frontend select Noninteractive' | > debconf-set-selections > > # update and install some software requirements > RUN apt-get update && apt-get upgrade -y && apt-get install -y curl wget > git unzip autoconf make g++ gcc fop libxml2-utils xsltproc > > WORKDIR / > > RUN add-apt-repository ppa:openjdk-r/ppa > RUN apt-get update && apt-get install openjdk-8-jdk -y > > WORKDIR / > > # install nano (and pico) > RUN apt-get update && apt-get install nano -y > > # Needed for terminal handling (libc-dev libncurses5 libtinfo-dev > libtinfo5 ncurses-bin) > RUN apt-get -y install libncurses5-dev openssl libssl-dev m4 > libncurses-dev libgmp3-dev libglu-dev > > # For building with wxWidgets > RUN apt-get -y install freeglut3-dev wx2.8-headers wx-common libwxbase2.8 > libwxgtk2.8-dev libqt4-opengl-dev libgtk2.0-dev libgl1-mesa-dev > libglu1-mesa-dev libpng3 > > # For building ssl (libssh-4 libssl-dev zlib1g-dev) > RUN apt-get -y install libssh-dev > > # ODBC support (libltdl3-dev odbcinst1debian2 unixodbc) > RUN apt-get -y install unixodbc-dev > RUN apt-get -y install ncurses-dev > RUN wget http://www.erlang.org/download/otp_src_18.3.tar.gz > RUN tar -xvzf otp_src_18.3.tar.gz > RUN chmod -R 777 otp_src_18.3 > WORKDIR otp_src_18.3 > RUN ./configure > RUN make > RUN make install > RUN cd .. && rm -R otp_src_18.3/ > > RUN echo "ERLANG_HOME=/usr/local/lib/erlang" >> ~/.bashrc > RUN echo "export PATH=$PATH:$ERLANG_HOME/bin" >> ~/.bashrc > RUN echo "export DISPLAY=:100.0" >> ~/.bashrc > > RUN apt-get purge --auto-remove openjdk-8-jdk -y > RUN rm -R /usr/lib/jvm > > > -- > Kind regards, > > Stefan Houtzager > > Houtzager ICT consultancy & development > > www.linkedin.com/in/stefanhoutzager > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Mon Apr 11 10:23:42 2016 From: roger@REDACTED (Roger Lipscombe) Date: Mon, 11 Apr 2016 09:23:42 +0100 Subject: [erlang-questions] ssl_closed not always received In-Reply-To: References: Message-ID: On 10 April 2016 at 22:34, Khitai Pang wrote: > How to make sure that the server process always get ssl_closed when the > client process on a remote host quits? In the general case, you *can't*. This is due to the vagaries of TCP. To close a socket, the client will send a FIN packet to the server. If the network connection is lost (consider simply unplugging the cable), then the FIN will *never* arrive. If you need to know when the client has gone away, either implement some kind of application-level keepalive or enable TCP keepalive. From erlang@REDACTED Mon Apr 11 10:45:29 2016 From: erlang@REDACTED (Stefan Marr) Date: Mon, 11 Apr 2016 10:45:29 +0200 Subject: [erlang-questions] Call for Papers: 11th ICOOOLPS Workshop at ECOOP. New Deadline April 22nd, 2016 Message-ID: <65831A03-20C4-4D8F-84E5-72124EBC031D@stefan-marr.de> Please note that the submission deadline is on the 22nd of April, 2016. Call for Papers: ICOOOLPS?16 ============================ 11th Workshop on Implementation, Compilation, Optimization of OO Languages, Programs and Systems Co-located with ECOOP July 18, 2016, Rome, Italy URL: http://2016.ecoop.org/track/ICOOOLPS-2016 Twitter: @ICOOOLPS The ICOOOLPS workshop series brings together researchers and practitioners working in the field of language implementation and optimization. The goal of the workshop is to discuss emerging problems and research directions as well as new solutions to classic performance challenges. The topics of interest for the workshop include techniques for the implementation and optimization of a wide range of languages including but not limited to object-oriented ones. Furthermore, meta-compilation techniques or language-agnostic approaches are welcome, too. A non-exclusive list of topics follows: - implementation and optimization of fundamental languages features (from automatic memory management to zero-overhead metaprogramming) - runtime systems technology (libraries, virtual machines) - static, adaptive, and speculative optimizations and compiler techniques - meta-compilation techniques and language-agnostic approaches for the efficient implementation of languages - compilers (intermediate representations, offline and online optimizations,...) - empirical studies on language usage, benchmark design, and benchmarking methodology - resource-sensitive systems (real-time, low power, mobile, cloud) - studies on design choices and tradeoffs (dynamic vs. static compilation, heuristics vs. programmer input,...) - tooling support, debuggability and observability of languages as well as their implementations ### Workshop Format and Submissions This workshop welcomes the presentation and discussion of new ideas and emerging problems that give a chance for interaction and exchange. More mature work is welcome as part of a mini-conference format, too. We aim to interleave interactive brainstorming and demonstration sessions between the formal presentations to foster an active exchange of ideas. The workshop papers will be published either in the ACM DL or in the Dagstuhl LIPIcs ECOOP Workshop proceedings. Until further notice, please use the ACM SIGPLAN template with a 10pt font size: http://www.sigplan.org/Resources/Author/. - position and work-in-progress paper: 1-4 pages - technical paper: max. 10 pages - demos and posters: 1-page abstract The page limits include references and appendixes. Note further that the upper page limit is a maximum and not a requirement. For the submission, please use the HotCRP system: http://ssw.jku.at/icooolps/ ### Important Dates - abstract submission: April 18, 2016 - paper submission: April 22, 2016 - notification: May 13, 2016 - all deadlines: Anywhere on Earth (AoE), i.e., GMT/UTC?12:00 hour - workshop: July 18th, 2016 ### Program Committee Edd Barrett, King?s College London, UK Clement Bera, Inria Lille, France Maxime Chevalier-Boisvert, Universit? de Montr?al, Canada Tim Felgentreff, Hasso Plattner Institute, Germany Roland Ducournau, LIRMM, Universit? de Montpellier, France Elisa Gonzalez Boix, Vrije Universiteit Brussel, Belgium David Gregg, Trinity College Dublin, Ireland Matthias Grimmer, Johannes Kepler University Linz, Austria Michael Haupt, Oracle, Germany Richard Jones, University of Kent, UK Tomas Kalibera, Northeastern University, USA Hidehiko Masuhara, Tokyo Institute of Technology, Japan Tiark Rompf, Purdue University, USA Jennifer B. Sartor, Ghent University, Belgium Sam Tobin-Hochstadt, Indiana University, USA ### Workshop Organizers Stefan Marr, Johannes Kepler University Linz, Austria Eric Jul, University of Oslo, Norway From valentin@REDACTED Mon Apr 11 10:58:43 2016 From: valentin@REDACTED (Valentin Micic) Date: Mon, 11 Apr 2016 10:58:43 +0200 Subject: [erlang-questions] UDP Multicast - how does it work? In-Reply-To: References: Message-ID: <4F9A99CD-D4E3-4687-8646-AC8E3A9E16B9@pixie.co.za> > Step 1) Reach several processes on the same machine with one send. UDP Multicast works as a part of the link layer ( as in MAC - Media Access Control*), and as such appears to a process on the machine as a "normal" UDP traffic. Although I've heard that some operating systems may have provide variants that allow multiple OS processes to use to the same UDP port (and hence receive the same multicast packet), I doubt t you'd be able to accomplish this in Erlang. > Step 2) Reach several processes on possibly different machines with one send. I think this would be possible if you rephrase your requirement to something like this: " Reach several machines with a single send". As for reaching several processes on the same machine, see comment for Step 1). Kind regards V/ (*) In a nutshell, the multicast address you specify is just used to set four (out of six) octets of the interface's MAC address to that value. In other words, it will add another link (MAC) address to your physical interface. When any given interface sends to a multicast group (so called class D address), instead of using ARP to obtain destination's link address, the driver will simply combine given multicast address with predefined destination (e.g. multicast) address. Conversely, when a multicast packet is received by the interface that is subscribed to the given multicast address, it will accept the packet and forwarded it to the UDP port (and hence OS process) referenced by the received packet. From boris.muehmer@REDACTED Mon Apr 11 11:04:59 2016 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Mon, 11 Apr 2016 11:04:59 +0200 Subject: [erlang-questions] How to get wxerlang working in in Ubuntu 14.04.4 LTS In-Reply-To: References: Message-ID: Maybe the problem is related to the docker environment. I only had a brief contact with docker so I am not sure if I remember everything correctly. How do You connect to Erlang in the docker environment? Do You use SSH? Do You use "-X" for the connection? Could You check if "DISPLAY" is set?! If DISPLAY is empty You will always get the above error. This is just a *pseudo example* using a ssh connection to localhost: LOCAL> ssh -X bsmr@REDACTED Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 3.13.0-85-generic x86_64) REMOTE> echo $DISPLAY localhost:10.0 But without a proper DISPLAY variable wx (or every other X11 application) will fail. Regards, Boris 2016-04-11 10:37 GMT+02:00 Stefan Houtzager : > Thanks Boris, > > When I install libwxbase3.0-dev, libwxgtk3.0-dev and libglu1-mesa-dev > instead of freeglut3-dev wx2.8-headers wx-common libwxbase2.8 > libwxgtk2.8-dev libqt4-opengl-dev libgtk2.0-dev libgl1-mesa-dev > libglu1-mesa-dev libpng3 > I get the same error on wx:demo(). > which wx-config && wx-config --version-full gives > /usr/bin/wx-config > 3.0.0.0 > Maybe I have to configure some files from within my script? > > > On Mon, Apr 11, 2016 at 10:00 AM, Boris M?hmer > wrote: > >> Hello Stefan, >> >> You need to install *libwxbase3.0-dev* and *libwxgtk3.0-dev*. >> >> I also believe You should install *libglu1-mesa-dev*. I have it in my >> erlang setup dependency list, but I can not remember why this was needed. >> It is possible that the gl demo did not work without it. >> >> But "my dependency list" started with 12.04, and some things were manuall >> changed/configured with 14.04. Maybe some things are automatically resolved >> now (like libwxgtk3.0-dev automatically includes libwxbase3.0-dev and >> libglu1-mesa-dev). >> >> >> Regards, >> Boris >> >> 2016-04-11 9:22 GMT+02:00 Stefan Houtzager : >> >>> Hi, >>> >>> I would appreciate an installscript for a recent erlang version >>> on Ubuntu 14.04.4 LTS with wxerlang working. >>> I am a starting elixir developer (and for that need erlang). I would >>> like to get wxerlang working and created a dockerimage to make my >>> installation easy portable. I use otp_src_18.3.tar.gz and install several >>> wx packages and others, I'm not sure if I miss some or install some that >>> are not needed. Anyway, after the install the output of which wx-config >>> && wx-config --version-full is >>> >>> /usr/bin/wx-config >>> 2.8.12.1 >>> >>> But wx:demo(). results in an error: >>> >>> Eshell V7.3 (abort with ^G) >>> 1> wx:demo(). >>> Error: Unable to initialize gtk, is DISPLAY set properly? >>> ok >>> 2> >>> >>> The most important installs in the dockerfile (just linux commands after >>> the RUN statements, phusion/baseimage:0.9.18 is Ubuntu 14.04.4 LTS reworked >>> for docker) I copied below, the full dockerfile and the logging of the >>> output I arttached. >>> >>> >>> FROM phusion/baseimage:0.9.18 >>> MAINTAINER Stefan Houtzager >>> >>> # Important! Update this no-op ENV variable when this Dockerfile >>> # is updated with the current date. It will force refresh of all >>> # of the base images and things like `apt-get update` won't be using >>> # old cached versions when the Dockerfile is built. >>> ENV REFRESHED_AT 05-04-2016 >>> >>> # Set correct environment variables. >>> >>> # Setting ENV HOME does not seem to work currently. HOME is unset in >>> Docker container. >>> # See bug : https://github.com/phusion/baseimage-docker/issues/119 >>> #ENV HOME /root >>> # Workaround: >>> RUN echo /root > /etc/container_environment/HOME >>> >>> # Regenerate SSH host keys. baseimage-docker does not contain any, so you >>> # have to do that yourself. You may also comment out this instruction; >>> the >>> # init system will auto-generate one during boot. >>> RUN /etc/my_init.d/00_regen_ssh_host_keys.sh >>> >>> # Use baseimage-docker's init system. >>> CMD ["/sbin/my_init"] >>> >>> # ...put your own build instructions here... >>> >>> # Set the locale >>> RUN locale-gen en_US.UTF-8 >>> ENV LANG en_US.UTF-8 >>> ENV LANGUAGE en_US:en >>> ENV LC_ALL en_US.UTF-8 >>> >>> WORKDIR /tmp >>> >>> # See : https://github.com/phusion/baseimage-docker/issues/58 >>> RUN echo 'debconf debconf/frontend select Noninteractive' | >>> debconf-set-selections >>> >>> # update and install some software requirements >>> RUN apt-get update && apt-get upgrade -y && apt-get install -y curl wget >>> git unzip autoconf make g++ gcc fop libxml2-utils xsltproc >>> >>> WORKDIR / >>> >>> RUN add-apt-repository ppa:openjdk-r/ppa >>> RUN apt-get update && apt-get install openjdk-8-jdk -y >>> >>> WORKDIR / >>> >>> # install nano (and pico) >>> RUN apt-get update && apt-get install nano -y >>> >>> # Needed for terminal handling (libc-dev libncurses5 libtinfo-dev >>> libtinfo5 ncurses-bin) >>> RUN apt-get -y install libncurses5-dev openssl libssl-dev m4 >>> libncurses-dev libgmp3-dev libglu-dev >>> >>> # For building with wxWidgets >>> RUN apt-get -y install freeglut3-dev wx2.8-headers wx-common >>> libwxbase2.8 libwxgtk2.8-dev libqt4-opengl-dev libgtk2.0-dev >>> libgl1-mesa-dev libglu1-mesa-dev libpng3 >>> >>> # For building ssl (libssh-4 libssl-dev zlib1g-dev) >>> RUN apt-get -y install libssh-dev >>> >>> # ODBC support (libltdl3-dev odbcinst1debian2 unixodbc) >>> RUN apt-get -y install unixodbc-dev >>> RUN apt-get -y install ncurses-dev >>> RUN wget http://www.erlang.org/download/otp_src_18.3.tar.gz >>> RUN tar -xvzf otp_src_18.3.tar.gz >>> RUN chmod -R 777 otp_src_18.3 >>> WORKDIR otp_src_18.3 >>> RUN ./configure >>> RUN make >>> RUN make install >>> RUN cd .. && rm -R otp_src_18.3/ >>> >>> RUN echo "ERLANG_HOME=/usr/local/lib/erlang" >> ~/.bashrc >>> RUN echo "export PATH=$PATH:$ERLANG_HOME/bin" >> ~/.bashrc >>> RUN echo "export DISPLAY=:100.0" >> ~/.bashrc >>> >>> RUN apt-get purge --auto-remove openjdk-8-jdk -y >>> RUN rm -R /usr/lib/jvm >>> >>> >>> -- >>> Kind regards, >>> >>> Stefan Houtzager >>> >>> Houtzager ICT consultancy & development >>> >>> www.linkedin.com/in/stefanhoutzager >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > > > -- > Kind regards, > > Stefan Houtzager > > Houtzager ICT consultancy & development > > www.linkedin.com/in/stefanhoutzager > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan.houtzager@REDACTED Mon Apr 11 10:37:30 2016 From: stefan.houtzager@REDACTED (Stefan Houtzager) Date: Mon, 11 Apr 2016 10:37:30 +0200 Subject: [erlang-questions] How to get wxerlang working in in Ubuntu 14.04.4 LTS In-Reply-To: References: Message-ID: Thanks Boris, When I install libwxbase3.0-dev, libwxgtk3.0-dev and libglu1-mesa-dev instead of freeglut3-dev wx2.8-headers wx-common libwxbase2.8 libwxgtk2.8-dev libqt4-opengl-dev libgtk2.0-dev libgl1-mesa-dev libglu1-mesa-dev libpng3 I get the same error on wx:demo(). which wx-config && wx-config --version-full gives /usr/bin/wx-config 3.0.0.0 Maybe I have to configure some files from within my script? On Mon, Apr 11, 2016 at 10:00 AM, Boris M?hmer wrote: > Hello Stefan, > > You need to install *libwxbase3.0-dev* and *libwxgtk3.0-dev*. > > I also believe You should install *libglu1-mesa-dev*. I have it in my > erlang setup dependency list, but I can not remember why this was needed. > It is possible that the gl demo did not work without it. > > But "my dependency list" started with 12.04, and some things were manuall > changed/configured with 14.04. Maybe some things are automatically resolved > now (like libwxgtk3.0-dev automatically includes libwxbase3.0-dev and > libglu1-mesa-dev). > > > Regards, > Boris > > 2016-04-11 9:22 GMT+02:00 Stefan Houtzager : > >> Hi, >> >> I would appreciate an installscript for a recent erlang version on Ubuntu >> 14.04.4 LTS with wxerlang working. >> I am a starting elixir developer (and for that need erlang). I would >> like to get wxerlang working and created a dockerimage to make my >> installation easy portable. I use otp_src_18.3.tar.gz and install several >> wx packages and others, I'm not sure if I miss some or install some that >> are not needed. Anyway, after the install the output of which wx-config >> && wx-config --version-full is >> >> /usr/bin/wx-config >> 2.8.12.1 >> >> But wx:demo(). results in an error: >> >> Eshell V7.3 (abort with ^G) >> 1> wx:demo(). >> Error: Unable to initialize gtk, is DISPLAY set properly? >> ok >> 2> >> >> The most important installs in the dockerfile (just linux commands after >> the RUN statements, phusion/baseimage:0.9.18 is Ubuntu 14.04.4 LTS reworked >> for docker) I copied below, the full dockerfile and the logging of the >> output I arttached. >> >> >> FROM phusion/baseimage:0.9.18 >> MAINTAINER Stefan Houtzager >> >> # Important! Update this no-op ENV variable when this Dockerfile >> # is updated with the current date. It will force refresh of all >> # of the base images and things like `apt-get update` won't be using >> # old cached versions when the Dockerfile is built. >> ENV REFRESHED_AT 05-04-2016 >> >> # Set correct environment variables. >> >> # Setting ENV HOME does not seem to work currently. HOME is unset in >> Docker container. >> # See bug : https://github.com/phusion/baseimage-docker/issues/119 >> #ENV HOME /root >> # Workaround: >> RUN echo /root > /etc/container_environment/HOME >> >> # Regenerate SSH host keys. baseimage-docker does not contain any, so you >> # have to do that yourself. You may also comment out this instruction; the >> # init system will auto-generate one during boot. >> RUN /etc/my_init.d/00_regen_ssh_host_keys.sh >> >> # Use baseimage-docker's init system. >> CMD ["/sbin/my_init"] >> >> # ...put your own build instructions here... >> >> # Set the locale >> RUN locale-gen en_US.UTF-8 >> ENV LANG en_US.UTF-8 >> ENV LANGUAGE en_US:en >> ENV LC_ALL en_US.UTF-8 >> >> WORKDIR /tmp >> >> # See : https://github.com/phusion/baseimage-docker/issues/58 >> RUN echo 'debconf debconf/frontend select Noninteractive' | >> debconf-set-selections >> >> # update and install some software requirements >> RUN apt-get update && apt-get upgrade -y && apt-get install -y curl wget >> git unzip autoconf make g++ gcc fop libxml2-utils xsltproc >> >> WORKDIR / >> >> RUN add-apt-repository ppa:openjdk-r/ppa >> RUN apt-get update && apt-get install openjdk-8-jdk -y >> >> WORKDIR / >> >> # install nano (and pico) >> RUN apt-get update && apt-get install nano -y >> >> # Needed for terminal handling (libc-dev libncurses5 libtinfo-dev >> libtinfo5 ncurses-bin) >> RUN apt-get -y install libncurses5-dev openssl libssl-dev m4 >> libncurses-dev libgmp3-dev libglu-dev >> >> # For building with wxWidgets >> RUN apt-get -y install freeglut3-dev wx2.8-headers wx-common libwxbase2.8 >> libwxgtk2.8-dev libqt4-opengl-dev libgtk2.0-dev libgl1-mesa-dev >> libglu1-mesa-dev libpng3 >> >> # For building ssl (libssh-4 libssl-dev zlib1g-dev) >> RUN apt-get -y install libssh-dev >> >> # ODBC support (libltdl3-dev odbcinst1debian2 unixodbc) >> RUN apt-get -y install unixodbc-dev >> RUN apt-get -y install ncurses-dev >> RUN wget http://www.erlang.org/download/otp_src_18.3.tar.gz >> RUN tar -xvzf otp_src_18.3.tar.gz >> RUN chmod -R 777 otp_src_18.3 >> WORKDIR otp_src_18.3 >> RUN ./configure >> RUN make >> RUN make install >> RUN cd .. && rm -R otp_src_18.3/ >> >> RUN echo "ERLANG_HOME=/usr/local/lib/erlang" >> ~/.bashrc >> RUN echo "export PATH=$PATH:$ERLANG_HOME/bin" >> ~/.bashrc >> RUN echo "export DISPLAY=:100.0" >> ~/.bashrc >> >> RUN apt-get purge --auto-remove openjdk-8-jdk -y >> RUN rm -R /usr/lib/jvm >> >> >> -- >> Kind regards, >> >> Stefan Houtzager >> >> Houtzager ICT consultancy & development >> >> www.linkedin.com/in/stefanhoutzager >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -- Kind regards, Stefan Houtzager Houtzager ICT consultancy & development www.linkedin.com/in/stefanhoutzager -------------- next part -------------- An HTML attachment was scrubbed... URL: From Oliver.Korpilla@REDACTED Mon Apr 11 11:19:00 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Mon, 11 Apr 2016 11:19:00 +0200 Subject: [erlang-questions] UDP Multicast - how does it work? In-Reply-To: <4F9A99CD-D4E3-4687-8646-AC8E3A9E16B9@pixie.co.za> References: , <4F9A99CD-D4E3-4687-8646-AC8E3A9E16B9@pixie.co.za> Message-ID: Hello, Valentin. That's the thing, though. I read through as many posts I can find and it seems like {reuseaddr, } is... behaving differently on Erlang than with other languages on same OS. I'm not fixed to UDP Multicast to solve my problem. Any solution would do. But it seems that the above construct really *is* a problem. I've seen references that multiple UDP sockets bound to the same port work in other languages on the same OS but somehow they might not in Erlang. I found no followup to this: http://erlang.org/pipermail/erlang-questions/2009-December/048067.html Maybe I'm misunderstanding something here. Things I could live with: * Addressing a range of ports or all ports on same machine in one send. * Several processes listening on the same port on the same machine. What I want to avoid is to send multiple messages, that's all. I want to address several clients in one machine with one send without previously knowing how many there are. Thanks and kind regards, Oliver? Gesendet:?Montag, 11. April 2016 um 10:58 Uhr Von:?"Valentin Micic" An:?"Oliver Korpilla" Cc:?erlang-questions@REDACTED Betreff:?Re: [erlang-questions] UDP Multicast - how does it work? > Step 1) Reach several processes on the same machine with one send. UDP Multicast works as a part of the link layer ( as in MAC - Media Access Control*), and as such appears to a process on the machine as a "normal" UDP traffic. Although I've heard that some operating systems may have provide variants that allow multiple OS processes to use to the same UDP port (and hence receive the same multicast packet), I doubt t you'd be able to accomplish this in Erlang. > Step 2) Reach several processes on possibly different machines with one send. I think this would be possible if you rephrase your requirement to something like this: " Reach several machines with a single send". As for reaching several processes on the same machine, see comment for Step 1). Kind regards V/ (*) In a nutshell, the multicast address you specify is just used to set four (out of six) octets of the interface's MAC address to that value. In other words, it will add another link (MAC) address to your physical interface. When any given interface sends to a multicast group (so called class D address), instead of using ARP to obtain destination's link address, the driver will simply combine given multicast address with predefined destination (e.g. multicast) address. Conversely, when a multicast packet is received by the interface that is subscribed to the given multicast address, it will accept the packet and forwarded it to the UDP port (and hence OS process) referenced by the received packet. ? From khitai.pang@REDACTED Mon Apr 11 11:20:22 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Mon, 11 Apr 2016 17:20:22 +0800 Subject: [erlang-questions] ssl_closed not always received In-Reply-To: References: Message-ID: On 2016/4/11 16:23, Roger Lipscombe wrote: > On 10 April 2016 at 22:34, Khitai Pang wrote: >> How to make sure that the server process always get ssl_closed when the >> client process on a remote host quits? > In the general case, you *can't*. This is due to the vagaries of TCP. > To close a socket, the client will send a FIN packet to the server. If > the network connection is lost (consider simply unplugging the cable), > then the FIN will *never* arrive. If you need to know when the client > has gone away, either implement some kind of application-level > keepalive some kind of heartbeat? > or enable TCP keepalive. I already have "ok = ssl:setopts(Socket, [{keepalive, true}, ..." on both server and client, but still ssl_closed is not guaranteed. Since we can never be sure about whether a socket is closed, should we use try ... catch for every ssl:send to catch {error, closed}? What is the best practice here? Thanks Khitai From essen@REDACTED Mon Apr 11 11:24:37 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Mon, 11 Apr 2016 11:24:37 +0200 Subject: [erlang-questions] ssl_closed not always received In-Reply-To: References: Message-ID: <570B6D55.7090604@ninenines.eu> On 04/11/2016 10:23 AM, Roger Lipscombe wrote: > On 10 April 2016 at 22:34, Khitai Pang wrote: >> How to make sure that the server process always get ssl_closed when the >> client process on a remote host quits? > > In the general case, you *can't*. This is due to the vagaries of TCP. > To close a socket, the client will send a FIN packet to the server. If > the network connection is lost (consider simply unplugging the cable), > then the FIN will *never* arrive. If you need to know when the client > has gone away, either implement some kind of application-level > keepalive or enable TCP keepalive. You need a bi-directional ping mechanism at the application level. TCP keepalive is not always enough. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From stefan.houtzager@REDACTED Mon Apr 11 11:30:39 2016 From: stefan.houtzager@REDACTED (Stefan Houtzager) Date: Mon, 11 Apr 2016 11:30:39 +0200 Subject: [erlang-questions] How to get wxerlang working in in Ubuntu 14.04.4 LTS In-Reply-To: References: Message-ID: That could be of course. How I connect to erlang? I just start a dockercontainer from a terminal session (pico): sudo docker run -it -h NEWCONTAINER --volumes-from stefancontainer stefan/phoenix /bin/bash and then type erl (in the same pico editor). echo $DISPLAY gives :100 as response (while in the dockercontainer of course), so RUN echo "export DISPLAY=:100.0" >> ~/.bashrc in the dockerfile seemed to work On Mon, Apr 11, 2016 at 11:04 AM, Boris M?hmer wrote: > Maybe the problem is related to the docker environment. I only had a brief > contact with docker so I am not sure if I remember everything correctly. > > How do You connect to Erlang in the docker environment? Do You use SSH? Do > You use "-X" for the connection? > Could You check if "DISPLAY" is set?! If DISPLAY is empty You will always > get the above error. > > This is just a *pseudo example* using a ssh connection to localhost: > > LOCAL> ssh -X bsmr@REDACTED > Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 3.13.0-85-generic x86_64) > REMOTE> echo $DISPLAY > localhost:10.0 > > > But without a proper DISPLAY variable wx (or every other X11 application) > will fail. > > > Regards, > Boris > > > > 2016-04-11 10:37 GMT+02:00 Stefan Houtzager : > >> Thanks Boris, >> >> When I install libwxbase3.0-dev, libwxgtk3.0-dev and libglu1-mesa-dev >> instead of freeglut3-dev wx2.8-headers wx-common libwxbase2.8 >> libwxgtk2.8-dev libqt4-opengl-dev libgtk2.0-dev libgl1-mesa-dev >> libglu1-mesa-dev libpng3 >> I get the same error on wx:demo(). >> which wx-config && wx-config --version-full gives >> /usr/bin/wx-config >> 3.0.0.0 >> Maybe I have to configure some files from within my script? >> >> >> On Mon, Apr 11, 2016 at 10:00 AM, Boris M?hmer >> wrote: >> >>> Hello Stefan, >>> >>> You need to install *libwxbase3.0-dev* and *libwxgtk3.0-dev*. >>> >>> I also believe You should install *libglu1-mesa-dev*. I have it in my >>> erlang setup dependency list, but I can not remember why this was needed. >>> It is possible that the gl demo did not work without it. >>> >>> But "my dependency list" started with 12.04, and some things were >>> manuall changed/configured with 14.04. Maybe some things are automatically >>> resolved now (like libwxgtk3.0-dev automatically includes libwxbase3.0-dev >>> and libglu1-mesa-dev). >>> >>> >>> Regards, >>> Boris >>> >>> 2016-04-11 9:22 GMT+02:00 Stefan Houtzager : >>> >>>> Hi, >>>> >>>> I would appreciate an installscript for a recent erlang version >>>> on Ubuntu 14.04.4 LTS with wxerlang working. >>>> I am a starting elixir developer (and for that need erlang). I would >>>> like to get wxerlang working and created a dockerimage to make my >>>> installation easy portable. I use otp_src_18.3.tar.gz and install several >>>> wx packages and others, I'm not sure if I miss some or install some that >>>> are not needed. Anyway, after the install the output of which >>>> wx-config && wx-config --version-full is >>>> >>>> /usr/bin/wx-config >>>> 2.8.12.1 >>>> >>>> But wx:demo(). results in an error: >>>> >>>> Eshell V7.3 (abort with ^G) >>>> 1> wx:demo(). >>>> Error: Unable to initialize gtk, is DISPLAY set properly? >>>> ok >>>> 2> >>>> >>>> The most important installs in the dockerfile (just linux commands >>>> after the RUN statements, phusion/baseimage:0.9.18 is Ubuntu 14.04.4 LTS >>>> reworked for docker) I copied below, the full dockerfile and the logging >>>> of the output I arttached. >>>> >>>> >>>> FROM phusion/baseimage:0.9.18 >>>> MAINTAINER Stefan Houtzager >>>> >>>> # Important! Update this no-op ENV variable when this Dockerfile >>>> # is updated with the current date. It will force refresh of all >>>> # of the base images and things like `apt-get update` won't be using >>>> # old cached versions when the Dockerfile is built. >>>> ENV REFRESHED_AT 05-04-2016 >>>> >>>> # Set correct environment variables. >>>> >>>> # Setting ENV HOME does not seem to work currently. HOME is unset in >>>> Docker container. >>>> # See bug : https://github.com/phusion/baseimage-docker/issues/119 >>>> #ENV HOME /root >>>> # Workaround: >>>> RUN echo /root > /etc/container_environment/HOME >>>> >>>> # Regenerate SSH host keys. baseimage-docker does not contain any, so >>>> you >>>> # have to do that yourself. You may also comment out this instruction; >>>> the >>>> # init system will auto-generate one during boot. >>>> RUN /etc/my_init.d/00_regen_ssh_host_keys.sh >>>> >>>> # Use baseimage-docker's init system. >>>> CMD ["/sbin/my_init"] >>>> >>>> # ...put your own build instructions here... >>>> >>>> # Set the locale >>>> RUN locale-gen en_US.UTF-8 >>>> ENV LANG en_US.UTF-8 >>>> ENV LANGUAGE en_US:en >>>> ENV LC_ALL en_US.UTF-8 >>>> >>>> WORKDIR /tmp >>>> >>>> # See : https://github.com/phusion/baseimage-docker/issues/58 >>>> RUN echo 'debconf debconf/frontend select Noninteractive' | >>>> debconf-set-selections >>>> >>>> # update and install some software requirements >>>> RUN apt-get update && apt-get upgrade -y && apt-get install -y curl >>>> wget git unzip autoconf make g++ gcc fop libxml2-utils xsltproc >>>> >>>> WORKDIR / >>>> >>>> RUN add-apt-repository ppa:openjdk-r/ppa >>>> RUN apt-get update && apt-get install openjdk-8-jdk -y >>>> >>>> WORKDIR / >>>> >>>> # install nano (and pico) >>>> RUN apt-get update && apt-get install nano -y >>>> >>>> # Needed for terminal handling (libc-dev libncurses5 libtinfo-dev >>>> libtinfo5 ncurses-bin) >>>> RUN apt-get -y install libncurses5-dev openssl libssl-dev m4 >>>> libncurses-dev libgmp3-dev libglu-dev >>>> >>>> # For building with wxWidgets >>>> RUN apt-get -y install freeglut3-dev wx2.8-headers wx-common >>>> libwxbase2.8 libwxgtk2.8-dev libqt4-opengl-dev libgtk2.0-dev >>>> libgl1-mesa-dev libglu1-mesa-dev libpng3 >>>> >>>> # For building ssl (libssh-4 libssl-dev zlib1g-dev) >>>> RUN apt-get -y install libssh-dev >>>> >>>> # ODBC support (libltdl3-dev odbcinst1debian2 unixodbc) >>>> RUN apt-get -y install unixodbc-dev >>>> RUN apt-get -y install ncurses-dev >>>> RUN wget http://www.erlang.org/download/otp_src_18.3.tar.gz >>>> RUN tar -xvzf otp_src_18.3.tar.gz >>>> RUN chmod -R 777 otp_src_18.3 >>>> WORKDIR otp_src_18.3 >>>> RUN ./configure >>>> RUN make >>>> RUN make install >>>> RUN cd .. && rm -R otp_src_18.3/ >>>> >>>> RUN echo "ERLANG_HOME=/usr/local/lib/erlang" >> ~/.bashrc >>>> RUN echo "export PATH=$PATH:$ERLANG_HOME/bin" >> ~/.bashrc >>>> RUN echo "export DISPLAY=:100.0" >> ~/.bashrc >>>> >>>> RUN apt-get purge --auto-remove openjdk-8-jdk -y >>>> RUN rm -R /usr/lib/jvm >>>> >>>> >>>> -- >>>> Kind regards, >>>> >>>> Stefan Houtzager >>>> >>>> Houtzager ICT consultancy & development >>>> >>>> www.linkedin.com/in/stefanhoutzager >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions >>>> >>>> >>> >> >> >> -- >> Kind regards, >> >> Stefan Houtzager >> >> Houtzager ICT consultancy & development >> >> www.linkedin.com/in/stefanhoutzager >> > > -- Kind regards, Stefan Houtzager Houtzager ICT consultancy & development www.linkedin.com/in/stefanhoutzager -------------- next part -------------- An HTML attachment was scrubbed... URL: From boris.muehmer@REDACTED Mon Apr 11 11:36:38 2016 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Mon, 11 Apr 2016 11:36:38 +0200 Subject: [erlang-questions] How to get wxerlang working in in Ubuntu 14.04.4 LTS In-Reply-To: References: Message-ID: I don't think it is a good idea to set the DISPLAY variable like You did. I just found this link: http://fabiorehm.com/blog/2014/09/11/running-gui-apps-with-docker/ Maybe this will give You some hints how to get it running. Regards, Boris 2016-04-11 11:30 GMT+02:00 Stefan Houtzager : > That could be of course. How I connect to erlang? I just start a > dockercontainer from a terminal session (pico): > > sudo docker run -it -h NEWCONTAINER --volumes-from stefancontainer > stefan/phoenix /bin/bash > > and then type erl (in the same pico editor). > > echo $DISPLAY gives :100 as response (while in the dockercontainer of > course), so > RUN echo "export DISPLAY=:100.0" >> ~/.bashrc > in the dockerfile seemed to work > > > On Mon, Apr 11, 2016 at 11:04 AM, Boris M?hmer > wrote: > >> Maybe the problem is related to the docker environment. I only had a >> brief contact with docker so I am not sure if I remember everything >> correctly. >> >> How do You connect to Erlang in the docker environment? Do You use SSH? >> Do You use "-X" for the connection? >> Could You check if "DISPLAY" is set?! If DISPLAY is empty You will always >> get the above error. >> >> This is just a *pseudo example* using a ssh connection to localhost: >> >> LOCAL> ssh -X bsmr@REDACTED >> Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 3.13.0-85-generic x86_64) >> REMOTE> echo $DISPLAY >> localhost:10.0 >> >> >> But without a proper DISPLAY variable wx (or every other X11 application) >> will fail. >> >> >> Regards, >> Boris >> >> >> >> 2016-04-11 10:37 GMT+02:00 Stefan Houtzager : >> >>> Thanks Boris, >>> >>> When I install libwxbase3.0-dev, libwxgtk3.0-dev and libglu1-mesa-dev >>> instead of freeglut3-dev wx2.8-headers wx-common libwxbase2.8 >>> libwxgtk2.8-dev libqt4-opengl-dev libgtk2.0-dev libgl1-mesa-dev >>> libglu1-mesa-dev libpng3 >>> I get the same error on wx:demo(). >>> which wx-config && wx-config --version-full gives >>> /usr/bin/wx-config >>> 3.0.0.0 >>> Maybe I have to configure some files from within my script? >>> >>> >>> On Mon, Apr 11, 2016 at 10:00 AM, Boris M?hmer >>> wrote: >>> >>>> Hello Stefan, >>>> >>>> You need to install *libwxbase3.0-dev* and *libwxgtk3.0-dev*. >>>> >>>> I also believe You should install *libglu1-mesa-dev*. I have it in my >>>> erlang setup dependency list, but I can not remember why this was needed. >>>> It is possible that the gl demo did not work without it. >>>> >>>> But "my dependency list" started with 12.04, and some things were >>>> manuall changed/configured with 14.04. Maybe some things are automatically >>>> resolved now (like libwxgtk3.0-dev automatically includes libwxbase3.0-dev >>>> and libglu1-mesa-dev). >>>> >>>> >>>> Regards, >>>> Boris >>>> >>>> 2016-04-11 9:22 GMT+02:00 Stefan Houtzager >>>> : >>>> >>>>> Hi, >>>>> >>>>> I would appreciate an installscript for a recent erlang version >>>>> on Ubuntu 14.04.4 LTS with wxerlang working. >>>>> I am a starting elixir developer (and for that need erlang). I would >>>>> like to get wxerlang working and created a dockerimage to make my >>>>> installation easy portable. I use otp_src_18.3.tar.gz and install several >>>>> wx packages and others, I'm not sure if I miss some or install some that >>>>> are not needed. Anyway, after the install the output of which >>>>> wx-config && wx-config --version-full is >>>>> >>>>> /usr/bin/wx-config >>>>> 2.8.12.1 >>>>> >>>>> But wx:demo(). results in an error: >>>>> >>>>> Eshell V7.3 (abort with ^G) >>>>> 1> wx:demo(). >>>>> Error: Unable to initialize gtk, is DISPLAY set properly? >>>>> ok >>>>> 2> >>>>> >>>>> The most important installs in the dockerfile (just linux commands >>>>> after the RUN statements, phusion/baseimage:0.9.18 is Ubuntu 14.04.4 LTS >>>>> reworked for docker) I copied below, the full dockerfile and the logging >>>>> of the output I arttached. >>>>> >>>>> >>>>> FROM phusion/baseimage:0.9.18 >>>>> MAINTAINER Stefan Houtzager >>>>> >>>>> # Important! Update this no-op ENV variable when this Dockerfile >>>>> # is updated with the current date. It will force refresh of all >>>>> # of the base images and things like `apt-get update` won't be using >>>>> # old cached versions when the Dockerfile is built. >>>>> ENV REFRESHED_AT 05-04-2016 >>>>> >>>>> # Set correct environment variables. >>>>> >>>>> # Setting ENV HOME does not seem to work currently. HOME is unset in >>>>> Docker container. >>>>> # See bug : https://github.com/phusion/baseimage-docker/issues/119 >>>>> #ENV HOME /root >>>>> # Workaround: >>>>> RUN echo /root > /etc/container_environment/HOME >>>>> >>>>> # Regenerate SSH host keys. baseimage-docker does not contain any, so >>>>> you >>>>> # have to do that yourself. You may also comment out this instruction; >>>>> the >>>>> # init system will auto-generate one during boot. >>>>> RUN /etc/my_init.d/00_regen_ssh_host_keys.sh >>>>> >>>>> # Use baseimage-docker's init system. >>>>> CMD ["/sbin/my_init"] >>>>> >>>>> # ...put your own build instructions here... >>>>> >>>>> # Set the locale >>>>> RUN locale-gen en_US.UTF-8 >>>>> ENV LANG en_US.UTF-8 >>>>> ENV LANGUAGE en_US:en >>>>> ENV LC_ALL en_US.UTF-8 >>>>> >>>>> WORKDIR /tmp >>>>> >>>>> # See : https://github.com/phusion/baseimage-docker/issues/58 >>>>> RUN echo 'debconf debconf/frontend select Noninteractive' | >>>>> debconf-set-selections >>>>> >>>>> # update and install some software requirements >>>>> RUN apt-get update && apt-get upgrade -y && apt-get install -y curl >>>>> wget git unzip autoconf make g++ gcc fop libxml2-utils xsltproc >>>>> >>>>> WORKDIR / >>>>> >>>>> RUN add-apt-repository ppa:openjdk-r/ppa >>>>> RUN apt-get update && apt-get install openjdk-8-jdk -y >>>>> >>>>> WORKDIR / >>>>> >>>>> # install nano (and pico) >>>>> RUN apt-get update && apt-get install nano -y >>>>> >>>>> # Needed for terminal handling (libc-dev libncurses5 libtinfo-dev >>>>> libtinfo5 ncurses-bin) >>>>> RUN apt-get -y install libncurses5-dev openssl libssl-dev m4 >>>>> libncurses-dev libgmp3-dev libglu-dev >>>>> >>>>> # For building with wxWidgets >>>>> RUN apt-get -y install freeglut3-dev wx2.8-headers wx-common >>>>> libwxbase2.8 libwxgtk2.8-dev libqt4-opengl-dev libgtk2.0-dev >>>>> libgl1-mesa-dev libglu1-mesa-dev libpng3 >>>>> >>>>> # For building ssl (libssh-4 libssl-dev zlib1g-dev) >>>>> RUN apt-get -y install libssh-dev >>>>> >>>>> # ODBC support (libltdl3-dev odbcinst1debian2 unixodbc) >>>>> RUN apt-get -y install unixodbc-dev >>>>> RUN apt-get -y install ncurses-dev >>>>> RUN wget http://www.erlang.org/download/otp_src_18.3.tar.gz >>>>> RUN tar -xvzf otp_src_18.3.tar.gz >>>>> RUN chmod -R 777 otp_src_18.3 >>>>> WORKDIR otp_src_18.3 >>>>> RUN ./configure >>>>> RUN make >>>>> RUN make install >>>>> RUN cd .. && rm -R otp_src_18.3/ >>>>> >>>>> RUN echo "ERLANG_HOME=/usr/local/lib/erlang" >> ~/.bashrc >>>>> RUN echo "export PATH=$PATH:$ERLANG_HOME/bin" >> ~/.bashrc >>>>> RUN echo "export DISPLAY=:100.0" >> ~/.bashrc >>>>> >>>>> RUN apt-get purge --auto-remove openjdk-8-jdk -y >>>>> RUN rm -R /usr/lib/jvm >>>>> >>>>> >>>>> -- >>>>> Kind regards, >>>>> >>>>> Stefan Houtzager >>>>> >>>>> Houtzager ICT consultancy & development >>>>> >>>>> www.linkedin.com/in/stefanhoutzager >>>>> >>>>> _______________________________________________ >>>>> erlang-questions mailing list >>>>> erlang-questions@REDACTED >>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>> >>>>> >>>> >>> >>> >>> -- >>> Kind regards, >>> >>> Stefan Houtzager >>> >>> Houtzager ICT consultancy & development >>> >>> www.linkedin.com/in/stefanhoutzager >>> >> >> > > > -- > Kind regards, > > Stefan Houtzager > > Houtzager ICT consultancy & development > > www.linkedin.com/in/stefanhoutzager > -------------- next part -------------- An HTML attachment was scrubbed... URL: From snar@REDACTED Mon Apr 11 11:38:58 2016 From: snar@REDACTED (Alexandre Snarskii) Date: Mon, 11 Apr 2016 12:38:58 +0300 Subject: [erlang-questions] UDP Multicast - how does it work? In-Reply-To: References: Message-ID: <20160411093858.GA44030@staff.retn.net> On Mon, Apr 11, 2016 at 09:51:12AM +0200, Oliver Korpilla wrote: > Hello. > > I'm lost on the basics here. Sorry if this seems like a fishing expedition. > > What I want to do: > > Step 1) Reach several processes on the same machine with one send. > Step 2) Reach several processes on possibly different machines with one send. > > I've tried the following: > > -module(test_multicast). > -export([start/0, receiver/1]). > -define(MULTICAST_ADDRESS, {127,0,10,1}). 127.0.10.1 is not a valid multicast address. Multicast addresses are in the range 224.0.0.0 - 239.255.255.255. More than that, rfc1122 section 3.2.1.3 clearly says that 127.x.x.x is an Internal host loopback address. Addresses of this form MUST NOT appear outside a host. and so no OS will accept packets addresses from/to 127.x.x.x on ethernet interfaces. > > receiver(Id) -> > io:format("Started multicast receiver ~p~n", [Id]), > {ok, Socket} = gen_udp:open(3333, [binary, {active, true}, {reuseaddr, true}, {multicast_if, ?MULTICAST_ADDRESS}, {multicast_loop, true}]), In order to receive multicast packets you must explicitly join this socket to a multicast group using {add_membership, {MultiAddress, InterfaceAddress}}: Join a multicast group. Example of working multicast between two servers in the same subnet: 1> {ok, Sock} = gen_udp:open(3333, [binary, {active, true}, {add_membership, {{239,0,10,1}, {0,0,0,0}}}]). {ok,#Port<0.481>} 2> gen_udp:send(Sock, {239,0,10,1}, 3333, <<"hello, world">>). ok 3> flush(). Shell got {udp,#Port<0.481>,{xx,xxx,xxx,2},3333,<<"hello, world">>} ok on another machine: 1> {ok, Sock} = gen_udp:open(3333, [binary, {active, true}, {add_membership, {{239,0,10,1}, {0,0,0,0}}}]). {ok,#Port<0.568>} 2> flush(). Shell got {udp,#Port<0.568>,{xx,xxx,xxx,2},3333,<<"hello, world">>} ok 3> > receiver_loop(Id, Socket). > > receiver_loop(Id, Socket) -> > receive > {udp, _RemoteSocket, _RemoteHost, _RemotePort, Payload } -> > io:format("Receiver ~p received message ~p~n", [Id, Payload]), > receiver_loop(Id, Socket) > end. > > send() -> > io:format("Sending...~n"), > {ok, Socket} = gen_udp:open(0), > ok = gen_udp:send(Socket, ?MULTICAST_ADDRESS, 3333, << "Hello, World." >>). > > start() -> > spawn_link(?MODULE, receiver, [1]), > spawn_link(?MODULE, receiver, [2]), > > timer:sleep(100), > send(). > > But the only one to receive the message is receiver #2. It's as if it takes the port away from receiver #1... I thought reuseaddr was meant to bind several sockets to the same port? > > Thank you for any advice, > Oliver > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From khitai.pang@REDACTED Mon Apr 11 11:43:33 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Mon, 11 Apr 2016 17:43:33 +0800 Subject: [erlang-questions] ssl_closed not always received In-Reply-To: <570B6D55.7090604@ninenines.eu> References: <570B6D55.7090604@ninenines.eu> Message-ID: On 2016/4/11 17:24, Lo?c Hoguin wrote: > On 04/11/2016 10:23 AM, Roger Lipscombe wrote: >> On 10 April 2016 at 22:34, Khitai Pang wrote: >>> How to make sure that the server process always get ssl_closed when the >>> client process on a remote host quits? >> >> In the general case, you *can't*. This is due to the vagaries of TCP. >> To close a socket, the client will send a FIN packet to the server. If >> the network connection is lost (consider simply unplugging the cable), >> then the FIN will *never* arrive. If you need to know when the client >> has gone away, either implement some kind of application-level >> keepalive or enable TCP keepalive. > > You need a bi-directional ping mechanism at the application level. TCP > keepalive is not always enough. How about this? The client sends a heartbeat message to the server every 2 minutes and the server replies with a heartbeat ack. The server deems the connection as lost if no heartbeat received in 2 minutes and the client deems the connection as lost if heartbeat ask is not received in 2 minutes. This sounds like uni-directional to me but would it suffice? Thanks Khitai From stefan.houtzager@REDACTED Mon Apr 11 11:50:45 2016 From: stefan.houtzager@REDACTED (Stefan Houtzager) Date: Mon, 11 Apr 2016 11:50:45 +0200 Subject: [erlang-questions] How to get wxerlang working in in Ubuntu 14.04.4 LTS In-Reply-To: References: Message-ID: Thanks Boris, I will try some things / search further with this info. On Mon, Apr 11, 2016 at 11:36 AM, Boris M?hmer wrote: > I don't think it is a good idea to set the DISPLAY variable like You did. > > I just found this link: > > http://fabiorehm.com/blog/2014/09/11/running-gui-apps-with-docker/ > > > Maybe this will give You some hints how to get it running. > > > Regards, > Boris > > > 2016-04-11 11:30 GMT+02:00 Stefan Houtzager : > >> That could be of course. How I connect to erlang? I just start a >> dockercontainer from a terminal session (pico): >> >> sudo docker run -it -h NEWCONTAINER --volumes-from stefancontainer >> stefan/phoenix /bin/bash >> >> and then type erl (in the same pico editor). >> >> echo $DISPLAY gives :100 as response (while in the dockercontainer of >> course), so >> RUN echo "export DISPLAY=:100.0" >> ~/.bashrc >> in the dockerfile seemed to work >> >> >> On Mon, Apr 11, 2016 at 11:04 AM, Boris M?hmer >> wrote: >> >>> Maybe the problem is related to the docker environment. I only had a >>> brief contact with docker so I am not sure if I remember everything >>> correctly. >>> >>> How do You connect to Erlang in the docker environment? Do You use SSH? >>> Do You use "-X" for the connection? >>> Could You check if "DISPLAY" is set?! If DISPLAY is empty You will >>> always get the above error. >>> >>> This is just a *pseudo example* using a ssh connection to localhost: >>> >>> LOCAL> ssh -X bsmr@REDACTED >>> Welcome to Ubuntu 14.04.4 LTS (GNU/Linux 3.13.0-85-generic x86_64) >>> REMOTE> echo $DISPLAY >>> localhost:10.0 >>> >>> >>> But without a proper DISPLAY variable wx (or every other X11 >>> application) will fail. >>> >>> >>> Regards, >>> Boris >>> >>> >>> >>> 2016-04-11 10:37 GMT+02:00 Stefan Houtzager >>> : >>> >>>> Thanks Boris, >>>> >>>> When I install libwxbase3.0-dev, libwxgtk3.0-dev and libglu1-mesa-dev >>>> instead of freeglut3-dev wx2.8-headers wx-common libwxbase2.8 >>>> libwxgtk2.8-dev libqt4-opengl-dev libgtk2.0-dev libgl1-mesa-dev >>>> libglu1-mesa-dev libpng3 >>>> I get the same error on wx:demo(). >>>> which wx-config && wx-config --version-full gives >>>> /usr/bin/wx-config >>>> 3.0.0.0 >>>> Maybe I have to configure some files from within my script? >>>> >>>> >>>> On Mon, Apr 11, 2016 at 10:00 AM, Boris M?hmer >>> > wrote: >>>> >>>>> Hello Stefan, >>>>> >>>>> You need to install *libwxbase3.0-dev* and *libwxgtk3.0-dev*. >>>>> >>>>> I also believe You should install *libglu1-mesa-dev*. I have it in my >>>>> erlang setup dependency list, but I can not remember why this was needed. >>>>> It is possible that the gl demo did not work without it. >>>>> >>>>> But "my dependency list" started with 12.04, and some things were >>>>> manuall changed/configured with 14.04. Maybe some things are automatically >>>>> resolved now (like libwxgtk3.0-dev automatically includes libwxbase3.0-dev >>>>> and libglu1-mesa-dev). >>>>> >>>>> >>>>> Regards, >>>>> Boris >>>>> >>>>> 2016-04-11 9:22 GMT+02:00 Stefan Houtzager >>>> >: >>>>> >>>>>> Hi, >>>>>> >>>>>> I would appreciate an installscript for a recent erlang version >>>>>> on Ubuntu 14.04.4 LTS with wxerlang working. >>>>>> I am a starting elixir developer (and for that need erlang). I >>>>>> would like to get wxerlang working and created a dockerimage to make my >>>>>> installation easy portable. I use otp_src_18.3.tar.gz and install several >>>>>> wx packages and others, I'm not sure if I miss some or install some that >>>>>> are not needed. Anyway, after the install the output of which >>>>>> wx-config && wx-config --version-full is >>>>>> >>>>>> /usr/bin/wx-config >>>>>> 2.8.12.1 >>>>>> >>>>>> But wx:demo(). results in an error: >>>>>> >>>>>> Eshell V7.3 (abort with ^G) >>>>>> 1> wx:demo(). >>>>>> Error: Unable to initialize gtk, is DISPLAY set properly? >>>>>> ok >>>>>> 2> >>>>>> >>>>>> The most important installs in the dockerfile (just linux commands >>>>>> after the RUN statements, phusion/baseimage:0.9.18 is Ubuntu 14.04.4 LTS >>>>>> reworked for docker) I copied below, the full dockerfile and the logging >>>>>> of the output I arttached. >>>>>> >>>>>> >>>>>> FROM phusion/baseimage:0.9.18 >>>>>> MAINTAINER Stefan Houtzager >>>>>> >>>>>> # Important! Update this no-op ENV variable when this Dockerfile >>>>>> # is updated with the current date. It will force refresh of all >>>>>> # of the base images and things like `apt-get update` won't be using >>>>>> # old cached versions when the Dockerfile is built. >>>>>> ENV REFRESHED_AT 05-04-2016 >>>>>> >>>>>> # Set correct environment variables. >>>>>> >>>>>> # Setting ENV HOME does not seem to work currently. HOME is unset in >>>>>> Docker container. >>>>>> # See bug : https://github.com/phusion/baseimage-docker/issues/119 >>>>>> #ENV HOME /root >>>>>> # Workaround: >>>>>> RUN echo /root > /etc/container_environment/HOME >>>>>> >>>>>> # Regenerate SSH host keys. baseimage-docker does not contain any, so >>>>>> you >>>>>> # have to do that yourself. You may also comment out this >>>>>> instruction; the >>>>>> # init system will auto-generate one during boot. >>>>>> RUN /etc/my_init.d/00_regen_ssh_host_keys.sh >>>>>> >>>>>> # Use baseimage-docker's init system. >>>>>> CMD ["/sbin/my_init"] >>>>>> >>>>>> # ...put your own build instructions here... >>>>>> >>>>>> # Set the locale >>>>>> RUN locale-gen en_US.UTF-8 >>>>>> ENV LANG en_US.UTF-8 >>>>>> ENV LANGUAGE en_US:en >>>>>> ENV LC_ALL en_US.UTF-8 >>>>>> >>>>>> WORKDIR /tmp >>>>>> >>>>>> # See : https://github.com/phusion/baseimage-docker/issues/58 >>>>>> RUN echo 'debconf debconf/frontend select Noninteractive' | >>>>>> debconf-set-selections >>>>>> >>>>>> # update and install some software requirements >>>>>> RUN apt-get update && apt-get upgrade -y && apt-get install -y curl >>>>>> wget git unzip autoconf make g++ gcc fop libxml2-utils xsltproc >>>>>> >>>>>> WORKDIR / >>>>>> >>>>>> RUN add-apt-repository ppa:openjdk-r/ppa >>>>>> RUN apt-get update && apt-get install openjdk-8-jdk -y >>>>>> >>>>>> WORKDIR / >>>>>> >>>>>> # install nano (and pico) >>>>>> RUN apt-get update && apt-get install nano -y >>>>>> >>>>>> # Needed for terminal handling (libc-dev libncurses5 libtinfo-dev >>>>>> libtinfo5 ncurses-bin) >>>>>> RUN apt-get -y install libncurses5-dev openssl libssl-dev m4 >>>>>> libncurses-dev libgmp3-dev libglu-dev >>>>>> >>>>>> # For building with wxWidgets >>>>>> RUN apt-get -y install freeglut3-dev wx2.8-headers wx-common >>>>>> libwxbase2.8 libwxgtk2.8-dev libqt4-opengl-dev libgtk2.0-dev >>>>>> libgl1-mesa-dev libglu1-mesa-dev libpng3 >>>>>> >>>>>> # For building ssl (libssh-4 libssl-dev zlib1g-dev) >>>>>> RUN apt-get -y install libssh-dev >>>>>> >>>>>> # ODBC support (libltdl3-dev odbcinst1debian2 unixodbc) >>>>>> RUN apt-get -y install unixodbc-dev >>>>>> RUN apt-get -y install ncurses-dev >>>>>> RUN wget http://www.erlang.org/download/otp_src_18.3.tar.gz >>>>>> RUN tar -xvzf otp_src_18.3.tar.gz >>>>>> RUN chmod -R 777 otp_src_18.3 >>>>>> WORKDIR otp_src_18.3 >>>>>> RUN ./configure >>>>>> RUN make >>>>>> RUN make install >>>>>> RUN cd .. && rm -R otp_src_18.3/ >>>>>> >>>>>> RUN echo "ERLANG_HOME=/usr/local/lib/erlang" >> ~/.bashrc >>>>>> RUN echo "export PATH=$PATH:$ERLANG_HOME/bin" >> ~/.bashrc >>>>>> RUN echo "export DISPLAY=:100.0" >> ~/.bashrc >>>>>> >>>>>> RUN apt-get purge --auto-remove openjdk-8-jdk -y >>>>>> RUN rm -R /usr/lib/jvm >>>>>> >>>>>> >>>>>> -- >>>>>> Kind regards, >>>>>> >>>>>> Stefan Houtzager >>>>>> >>>>>> Houtzager ICT consultancy & development >>>>>> >>>>>> www.linkedin.com/in/stefanhoutzager >>>>>> >>>>>> _______________________________________________ >>>>>> erlang-questions mailing list >>>>>> erlang-questions@REDACTED >>>>>> http://erlang.org/mailman/listinfo/erlang-questions >>>>>> >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Kind regards, >>>> >>>> Stefan Houtzager >>>> >>>> Houtzager ICT consultancy & development >>>> >>>> www.linkedin.com/in/stefanhoutzager >>>> >>> >>> >> >> >> -- >> Kind regards, >> >> Stefan Houtzager >> >> Houtzager ICT consultancy & development >> >> www.linkedin.com/in/stefanhoutzager >> > > -- Kind regards, Stefan Houtzager Houtzager ICT consultancy & development www.linkedin.com/in/stefanhoutzager -------------- next part -------------- An HTML attachment was scrubbed... URL: From essen@REDACTED Mon Apr 11 11:52:23 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Mon, 11 Apr 2016 11:52:23 +0200 Subject: [erlang-questions] ssl_closed not always received In-Reply-To: References: <570B6D55.7090604@ninenines.eu> Message-ID: <570B73D7.9080204@ninenines.eu> On 04/11/2016 11:43 AM, Khitai Pang wrote: > On 2016/4/11 17:24, Lo?c Hoguin wrote: >> On 04/11/2016 10:23 AM, Roger Lipscombe wrote: >>> On 10 April 2016 at 22:34, Khitai Pang wrote: >>>> How to make sure that the server process always get ssl_closed when the >>>> client process on a remote host quits? >>> >>> In the general case, you *can't*. This is due to the vagaries of TCP. >>> To close a socket, the client will send a FIN packet to the server. If >>> the network connection is lost (consider simply unplugging the cable), >>> then the FIN will *never* arrive. If you need to know when the client >>> has gone away, either implement some kind of application-level >>> keepalive or enable TCP keepalive. >> >> You need a bi-directional ping mechanism at the application level. TCP >> keepalive is not always enough. > > How about this? The client sends a heartbeat message to the server > every 2 minutes and the server replies with a heartbeat ack. The server > deems the connection as lost if no heartbeat received in 2 minutes and > the client deems the connection as lost if heartbeat ask is not received > in 2 minutes. This sounds like uni-directional to me but would it suffice? All that matters is that both endpoints send a ping message. It can be sent as a response. One important point is that the client and server agree on timeouts (this can be part of the protocol handshake or explicitly defined in the implementation, they just need to agree on it). Note that there's two timeouts: one for the client, and one for the server. They don't have to match. Using the client for sending pings (and the server just replies to pongs) scales better because the server only needs to keep track of one timeout (how long since last client message-- doesn't have to be an explicit ping) instead of two (the previous one + a timeout for sending pings). When you have one server for thousands of clients, it's usually better to offload processing to the clients. So tl;dr yes, you got it right. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From kostis@REDACTED Mon Apr 11 12:45:53 2016 From: kostis@REDACTED (Kostis Sagonas) Date: Mon, 11 Apr 2016 12:45:53 +0200 Subject: [erlang-questions] Status of map pair support in dialyzer In-Reply-To: <56D96A9D.6060502@ericsson.com> References: <56D96A9D.6060502@ericsson.com> Message-ID: <570B8061.4030009@cs.ntua.gr> On 03/04/2016 11:59 AM, Bj?rn-Egil Dahlberg XB wrote: > > I did the initial work for maps in Dialyzer but that application is not > really part of my work area. If you want to contribute to Dialyzer you > should probably talk to Hans Bolinder, Kostis Sagonas and/or Stavros > Aronis. Hans talked about increasing Dialyzers knowledge about map > associations the other day but I don't think it is on his agenda at the > moment. > > I don't think it will be supported in 19.0 either though I have no clue > if Kostis or Stavros, or anyone else for that matter, is working on it. Hijacking this thread to announce a proposal for a change in the type syntax related to maps. The relevant document is here: https://gist.github.com/kostis/eaf4a06e643cf49314ba We would appreciate feedback from the community at this point. The intention is for this change to be part of Erlang/OTP 19.0. Kostis PS. Most of the credit for this change goes to Magnus L?ng. PS2. For those that would like to experiment with the new type syntax and use the new version of dialyzer, or just fancy reading code diffs, there is also a related pull request that implements all these: https://github.com/erlang/otp/pull/1014 From jose.valim@REDACTED Mon Apr 11 13:25:00 2016 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Mon, 11 Apr 2016 13:25:00 +0200 Subject: [erlang-questions] Status of map pair support in dialyzer In-Reply-To: <570B8061.4030009@cs.ntua.gr> References: <56D96A9D.6060502@ericsson.com> <570B8061.4030009@cs.ntua.gr> Message-ID: That's excellent news Kostis! I have only one question/reservation: Another problem is related to the meaning of #{}. One would expect that this notation means "the empty map," but instead it means "any map". #{} in pattern matching also means "any map", so it may be confusing to have #{} in specs to mean the empty map, specially because both usages are often close to each other. For example: -spec increment(term(), #{}) -> #{} increment(Key, #{} = Map) -> Map#{Key := maps:get(Key, Map) + 1}. I also understand that [] and {} in typespecs implies emptiness and making #{} mean "the empty map" would make the typespecs more consistent. That said, maybe it would be better to not allow #{} altogether (deprecating it now and removing it later) and introduce an empty_map() construct? The empty_map() type is slightly more verbose but I don't expect it to be used frequently and, as such, the more explicit name is worth it. And if I understand the proposal correctly, both map() and #{_ => _} can still be used to represent any map, while #{_ := _} represents a non-empty map, so it feels all of our bases are covered. Thank you, *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D On Mon, Apr 11, 2016 at 12:45 PM, Kostis Sagonas wrote: > On 03/04/2016 11:59 AM, Bj?rn-Egil Dahlberg XB wrote: > >> >> I did the initial work for maps in Dialyzer but that application is not >> really part of my work area. If you want to contribute to Dialyzer you >> should probably talk to Hans Bolinder, Kostis Sagonas and/or Stavros >> Aronis. Hans talked about increasing Dialyzers knowledge about map >> associations the other day but I don't think it is on his agenda at the >> moment. >> >> I don't think it will be supported in 19.0 either though I have no clue >> if Kostis or Stavros, or anyone else for that matter, is working on it. >> > > > Hijacking this thread to announce a proposal for a change in the type > syntax related to maps. > > The relevant document is here: > > https://gist.github.com/kostis/eaf4a06e643cf49314ba > > We would appreciate feedback from the community at this point. > > The intention is for this change to be part of Erlang/OTP 19.0. > > > Kostis > > PS. Most of the credit for this change goes to Magnus L?ng. > > PS2. For those that would like to experiment with the new type syntax and > use the new version of dialyzer, or just fancy reading code diffs, there is > also a related pull request that implements all these: > > https://github.com/erlang/otp/pull/1014 > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From thoffmann@REDACTED Mon Apr 11 13:48:14 2016 From: thoffmann@REDACTED (Torben Hoffmann) Date: Mon, 11 Apr 2016 13:48:14 +0200 Subject: [erlang-questions] udp multicast: problem with OSX? In-Reply-To: References: Message-ID: Hi Peter, I cannot explain why it works on Fedora. But on the Mac I think it is because 224.0.0.251 is used for Multicast DNS. Furthermore, when you use the {ip, Addr} option the Addr should be one of the interfaces on your machine, not a multicast address. And you cannot use 127.0.0.1 - I have had it working with 0.0.0.0. Cannot remember how to change the address at the moment. Cheers, Torben Peter Morgan writes: > [ text/plain ] > Hello - > > I?m having some trouble getting UDP multicast working on OSX, whereas the same code on Linux works fine. > > On linux (fedora 23): > > [pmorgan@REDACTED ~]$ erl > Erlang/OTP 18 [erts-7.3] [source] [64-bit] [async-threads:10] [hipe] [kernel-poll:false] > > Eshell V7.3 (abort with ^G) > 1> Address = {224, 0, 0, 251}. > {224,0,0,251} > 2> Port = 5353. > 5353 > 3> {ok, Socket} = gen_udp:open(Port, [binary, {ip, Address}, {add_membership, {Address, {0,0,0,0}}}, {reuseaddr, true}]). > {ok,#Port<0.543>} > 4> gen_udp:send(Socket, Address, Port, <<"hello world">>). > ok > > > Whereas on OSX: > > Office-iMac:mdns pmorgan$ sw_vers > ProductName: Mac OS X > ProductVersion: 10.11.4 > BuildVersion: 15E61b > > Office-iMac:mdns pmorgan$ erl > Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace] > > Eshell V7.3 (abort with ^G) > 1> Address = {224, 0, 0, 251}. > {224,0,0,251} > 2> Port = 5353. > 5353 > 3> {ok, Socket} = gen_udp:open(Port, [binary, {ip, Address}, {add_membership, {Address, {0,0,0,0}}}, {reuseaddr, true}]). > {ok,#Port<0.553>} > 4> gen_udp:send(Socket, Address, Port, <<"hello world">>). > {error,eaddrnotavail} > > > I?ve tried various other options in gen_udp:open/2, but not found a combination that works for OSX. Any ideas please? The above is a simplified test case of https://github.com/shortishly/mdns/blob/master/src/mdns_udp.erl#L41-L49 > > Thanks, > Peter. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Torben Hoffmann Architect, basho.com M: +45 25 14 05 38 From jesper.louis.andersen@REDACTED Mon Apr 11 14:06:27 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 11 Apr 2016 14:06:27 +0200 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: Message-ID: Reviving this. There is now a PR for the proposal: https://github.com/erlang/otp/pull/1012 The name is hard. I'm on the verge of asking Robert Virding what it should be and then use him as the BDFL. I'm not going to ask Joe, since then we would have disagreement between him and Robert :) On Wed, Mar 2, 2016 at 3:47 PM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > Hi Erlangers, > > I'd really like to add two functions to the lists module from Haskell: > > intersperse(List, Seperator) produces a list where each element is > separated by separator, i.e. > > X = [1,2,3] > [1, x, 2, x, 3] = lists:intersperse(X, x), > > and it's cousin, intercalate(ListOfLists, Separator) is > append(intersperse(ListOfLists, Seperator)), i.e, > > Y = ["a", "b", "c"] > "a, b, c" = lists:intercalate(Y, ", "), > > The implementations are straightforward and easy to write tests for, even > property based tests if needed. > > The rationale for this proposal is that I find myself implementing this > function again and again in every project I write, and it is highly > generic. It belongs in a typical list module. OCaml libraries add it. > Haskell's Data.List has it. I believe Erlang, being a practical language, > should have it as well. > > Thoughts? > > -- > J. > -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rob@REDACTED Mon Apr 11 14:58:47 2016 From: rob@REDACTED (Rob A'Court) Date: Mon, 11 Apr 2016 12:58:47 +0000 Subject: [erlang-questions] Issue using variable length fields with Erlang ODBC Message-ID: When querying variable length fields using ODBC in Erlang, the response returned seems to be gibberish. We can query tables in a MS SQL Server database but if the table contains a VarCharMax or NVarCharMax field (both variable length) then the result returned is not what we expect. For NVarCharMax a binary is returned which has part of the original query and other seemingly random data as if it?s the wrong area of memory. For VarCharMax an empty list is always returned. In our particular scenario we are trying to get a ShowPlanXML from MS SQL Server which comes back as a NVarCharMax and there is no way of converting it to a fixed length field type to work around the issue. The issue does not seem to be with the ODBC driver as trying the same thing in python works fine. Here is what we are trying to do in Elixir: :odbc.start {:ok, connection} = :odbc.connect('Driver={ODBC Driver 11 for SQL Server}; Server=TheServer;Uid=sa;Pwd=password;Database=TheDatabase',[]) IO.inspect :odbc.sql_query(connection, 'set showplan_xml on') IO.inspect :odbc.sql_query(connection, 'Select * from customers'), limit: 9000 Here is the equivalent in python that works fine: #!/usr/bin/env python import pyodbc conn = pyodbc.connect('Driver={ODBC Driver 11 for SQL Server}; Server=TheServer;Uid=sa;Pwd=password;Database=TheDatabase') cur = conn.cursor() cur.execute('set showplan_xml on') cur.execute('Select * from customers') for row in cur : print row Many thanks! Rob -------------- next part -------------- An HTML attachment was scrubbed... URL: From siraaj@REDACTED Mon Apr 11 14:59:19 2016 From: siraaj@REDACTED (Siraaj Khandkar) Date: Mon, 11 Apr 2016 08:59:19 -0400 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: References: Message-ID: <570B9FA7.2050104@khandkar.net> I think my personal reservation about the name "intersperse" is addressed by the possibility of later extending to include interval specification, something like this: intersperse(A, [A], Interval) -> [A] when Interval :: random | {regular, pos_integer()} | ... . where the default is thought of as {regular, 1}. P.S. I'm crossing my fingers, hoping Robert will not like "join"! :-) On 4/11/16 8:06 AM, Jesper Louis Andersen wrote: > Reviving this. There is now a PR for the proposal: > > https://github.com/erlang/otp/pull/1012 > > The name is hard. I'm on the verge of asking Robert Virding what it should > be and then use him as the BDFL. I'm not going to ask Joe, since then we > would have disagreement between him and Robert :) > > > > On Wed, Mar 2, 2016 at 3:47 PM, Jesper Louis Andersen < > jesper.louis.andersen@REDACTED> wrote: > >> Hi Erlangers, >> >> I'd really like to add two functions to the lists module from Haskell: >> >> intersperse(List, Seperator) produces a list where each element is >> separated by separator, i.e. >> >> X = [1,2,3] >> [1, x, 2, x, 3] = lists:intersperse(X, x), >> >> and it's cousin, intercalate(ListOfLists, Separator) is >> append(intersperse(ListOfLists, Seperator)), i.e, >> >> Y = ["a", "b", "c"] >> "a, b, c" = lists:intercalate(Y, ", "), >> >> The implementations are straightforward and easy to write tests for, even >> property based tests if needed. >> >> The rationale for this proposal is that I find myself implementing this >> function again and again in every project I write, and it is highly >> generic. It belongs in a typical list module. OCaml libraries add it. >> Haskell's Data.List has it. I believe Erlang, being a practical language, >> should have it as well. >> >> Thoughts? From jose.valim@REDACTED Mon Apr 11 15:02:11 2016 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Mon, 11 Apr 2016 15:02:11 +0200 Subject: [erlang-questions] Status of map pair support in dialyzer In-Reply-To: <570B9738.3090408@gmail.com> References: <56D96A9D.6060502@ericsson.com> <570B8061.4030009@cs.ntua.gr> <570B9738.3090408@gmail.com> Message-ID: > > Now consider what happens when we change the type to #{x=>_}; the type is > made more restrictive. The key y is no longer allowed. But if we repeat > this process and change it to #{}, suddenly we have not made the type > more restrictive; we made it more allowing! > Thanks for the excellent explanation. That makes it clear to me changing #{} is indeed the best way to go. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Mon Apr 11 15:43:23 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 11 Apr 2016 09:43:23 -0400 Subject: [erlang-questions] ssl_closed not always received In-Reply-To: <570B73D7.9080204@ninenines.eu> References: <570B6D55.7090604@ninenines.eu> <570B73D7.9080204@ninenines.eu> Message-ID: <20160411134322.GE13113@fhebert-ltm2.internal.salesforce.com> On 04/11, Lo?c Hoguin wrote: >On 04/11/2016 11:43 AM, Khitai Pang wrote: >>On 2016/4/11 17:24, Lo?c Hoguin wrote: >>How about this? The client sends a heartbeat message to the server >>every 2 minutes and the server replies with a heartbeat ack. The server >>deems the connection as lost if no heartbeat received in 2 minutes and >>the client deems the connection as lost if heartbeat ask is not received >>in 2 minutes. This sounds like uni-directional to me but would it suffice? > > [...] > >So tl;dr yes, you got it right. > One small gotcha I'd like to add if you plan on supporting mobile users. A few months ago I read 'High Performance Browser Networking' by Ilya Grigorik, and an interesting section mentioned the usage of antennas and power consumption. An interesting thing described there is the cycle of powering antennas in your phones up and down as data transfers are required, involvement of remote servers, preservation of NAT caches, and so on. Basically, the antenna power for 2G/3G/4G/LTE stuff will tend to be the biggest power source of your phone, right after screen usage. WiFi's bad, but not nearly as bad for the average case. That's all okay, but the thing is that there is a thing the author calls the 'energy tail' that happens after each time a transmission is done: http://i.imgur.com/7KfixDQ.png The idea being that antenna power-up is time-based on a state machine, and transferring 2 kilobytes might be as significant for your battery as transfering 2 megabytes if they tend to mostly happen on the same high-energy cycle. A very significant section of the book is this one: > 46% of Battery Consumption to Transfer 0.2% of Total Bytes > > AT&T Labs Research published a great research paper ("Profiling > Resource Usage for Mobile Applications") in which it analyzed a > number of popular mobile applications for network and battery > efficiency. Among these applications, Pandora serves as a great case > study for the inefficiency of intermittent network transfers on > mobile networks. > > Whenever a Pandora user plays a song, the entire music file is > streamed by the application from the network in one shot, which is > the correct behavior: burst as much data as you can, then turn off > the radio for as long as possible. However, following the music > transfer, the application would conduct periodic audience > measurements by sending intermittent analytics pings every 60 > seconds. The net effect? The analytics beacons accounted for 0.2% of > the total transferred bytes and 46% of the total power consumption > of the application! > > The beacon transfers are small, but the energy tails induced by the > RRC state transitions were keeping the radio active for > significantly longer, unnecessarily wasting 46% of the battery. By > coalescing the analytics data into fewer requests, or by sending the > audience data when the radio is already active, we can eliminate the > unnecessary energy tails and almost double the power efficiency of > the application! (source: http://chimera.labs.oreilly.com/books/1230000000545/ch07.html#INEFFICIENCY_OF_PERIODIC_TRANSFERS) The fix? Raise the timeout to at least 5 minutes; it's the shortest value the book recommends as mobile carriers' NAT caches tend to hold for 5 mins: > Most mobile carriers set a 5?30 minute NAT connection timeout. Hence, > you may need a periodic (5 minute) keepalive to keep an idle > connection from being dropped. If you find yourself requiring more > frequent keepalives, check your own server, proxy, and load balancer > configuration first! (Source: http://chimera.labs.oreilly.com/books/1230000000545/ch08.html#ELIMINATE_POLLING) Regards, Fred. From jesper.louis.andersen@REDACTED Mon Apr 11 16:05:50 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 11 Apr 2016 16:05:50 +0200 Subject: [erlang-questions] Issue using variable length fields with Erlang ODBC In-Reply-To: References: Message-ID: On Mon, Apr 11, 2016 at 2:58 PM, Rob A'Court wrote: > When querying variable length fields using ODBC in Erlang, the response > returned seems to be gibberish. This appears to be a real bug. You can send it to bugs.erlang.org for tracking I think. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mallen@REDACTED Mon Apr 11 17:23:54 2016 From: mallen@REDACTED (Mark Allen) Date: Mon, 11 Apr 2016 10:23:54 -0500 Subject: [erlang-questions] [ANN] lager 3.2.0 and 2.2.2 Message-ID: Hello. Lager 3.2.0 and 2.2.2 were tagged last Friday. A brief changelog follows - please see the relevant PRs or Issues for more information. * Feature: Optional sink killer to shed load when mailbox size exceeds a configurable high water mark (#346) * Feature: Export `configure_sink/2` so users may dynamically configure previously setup and parse transformed sinks from their own code. (#342) * Feature: Re-enable Travis CI and update .travis.yml (#340) * Bugfix: Fix test race conditions for Travis CI (#344) * Bugfix: Add the atom 'none' to the log_level() type so downstream users won't get dialyzer failures if they use the 'none' log level. (#343) * Bugfix: Fix typo in documentation. (#341) * Bugfix: Fix OTP 18 test failures due to `warning_map/0` response change. (#337) * Bugfix: Make sure traces that use the file backend work correctly when specified in lager configuration. (#336) * Bugfix: Use `lager_app:get_env/3` for R15 compatibility. (#335) * Bugfix: Make sure lager uses `id` instead of `name` when reporting supervisor children failures. (The atom changed in OTP in 2014.) (#334) * Bugfix: Make lager handle improper iolists (#327) In the 2.x branch, we backported #337 and #334 -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Mon Apr 11 17:58:20 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Mon, 11 Apr 2016 18:58:20 +0300 Subject: [erlang-questions] udp multicast: problem with OSX? In-Reply-To: References: Message-ID: This is how it is working for me: {ok, Addr} = inet_parse:address(Host), Common = [binary,{reuseaddr,true},{recbuf,2*1024*1024},inet,{ip,Addr},{read_packets,100}], Options = case is_multicast(Addr) of true -> Multicast = [{multicast_ttl,4},{multicast_loop,true},{add_membership,{Addr,GwIP}}], Common ++ Multicast; false -> Common end, -------------- next part -------------- An HTML attachment was scrubbed... URL: From benmmurphy@REDACTED Mon Apr 11 18:10:46 2016 From: benmmurphy@REDACTED (Ben Murphy) Date: Mon, 11 Apr 2016 17:10:46 +0100 Subject: [erlang-questions] Issue using variable length fields with Erlang ODBC In-Reply-To: References: Message-ID: I think the problem is here: https://github.com/erlang/otp/blob/maint/lib/odbc/c_src/odbcserver.c#L1308 MAXCOLSIZE = 8001 I observed this appearing as truncation for large return results but I guess it is possible it allocates 8001 and tries to dump you back 9000. But ODBC driver has a lot of strange behaviour and I would avoid using it if possible. On Mon, Apr 11, 2016 at 1:58 PM, Rob A'Court wrote: > When querying variable length fields using ODBC in Erlang, the response > returned seems to be gibberish. > > > > We can query tables in a MS SQL Server database but if the table contains a > VarCharMax or NVarCharMax field (both variable length) then the result > returned is not what we expect. For NVarCharMax a binary is returned which > has part of the original query and other seemingly random data as if it?s > the wrong area of memory. For VarCharMax an empty list is always returned. > > > > In our particular scenario we are trying to get a ShowPlanXML from MS SQL > Server which comes back as a NVarCharMax and there is no way of converting > it to a fixed length field type to work around the issue. > > > > The issue does not seem to be with the ODBC driver as trying the same thing > in python works fine. > > > > Here is what we are trying to do in Elixir: > > > > :odbc.start > > {:ok, connection} = :odbc.connect('Driver={ODBC Driver 11 for SQL Server}; > Server=TheServer;Uid=sa;Pwd=password;Database=TheDatabase',[]) > > IO.inspect :odbc.sql_query(connection, 'set showplan_xml on') > > IO.inspect :odbc.sql_query(connection, 'Select * from customers'), limit: > 9000 > > > > > > Here is the equivalent in python that works fine: > > > > #!/usr/bin/env python > > import pyodbc > > conn = pyodbc.connect('Driver={ODBC Driver 11 for SQL Server}; > Server=TheServer;Uid=sa;Pwd=password;Database=TheDatabase') > > cur = conn.cursor() > > cur.execute('set showplan_xml on') > > cur.execute('Select * from customers') > > > > for row in cur : > > print row > > > > > > Many thanks! > > > > Rob > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From margnus1@REDACTED Mon Apr 11 14:23:20 2016 From: margnus1@REDACTED (=?UTF-8?Q?Magnus_L=c3=a5ng?=) Date: Mon, 11 Apr 2016 14:23:20 +0200 Subject: [erlang-questions] Status of map pair support in dialyzer In-Reply-To: References: <56D96A9D.6060502@ericsson.com> <570B8061.4030009@cs.ntua.gr> Message-ID: <570B9738.3090408@gmail.com> On 2016-04-11 13:25, Jos? Valim wrote: > That's excellent news Kostis! > > I have only one question/reservation: > > Another problem is related to the meaning of #{}. One would expect > that this notation means "the empty map," but instead it means > "any map". > > > #{} in pattern matching also means "any map", so it may be confusing > to have #{} in specs to mean the empty map, specially because both > usages are often close to each other. For example: > > -spec increment(term(), #{}) -> #{} > > increment(Key, #{} = Map) -> > > Map#{Key := maps:get(Key, Map) + 1}. > > I also understand that [] and {} in typespecs implies emptiness and > making #{} mean "the empty map" would make the typespecs more consistent. Actually, that is not the reason I wanted to make this change. Rather, it is inconsistent with the #{PairList} syntax. Consider the type #{x=>_, y=>_}. This type may only contain the keys x and y. So, the value #{x=>1, y=>2, z=>3} does not belong to this type. Now consider what happens when we change the type to #{x=>_}; the type is made more restrictive. The key y is no longer allowed. But if we repeat this process and change it to #{}, suddenly we have not made the type more restrictive; we made it more allowing! In fact, I have seen several people (myself included!) be fooled by the #{} base case into believing that the #{x=>_, y=>_}, means the set of values that match the pattern #{x=>_, y=>_}, which is not the case (written with the new syntax, that type would be #{x:=_, y:=_, ...}). So, I saw an opportunity to make the syntax self-consistent and more intuitive before it would become harder to change. Either by changing the semantics of everything but #{} in order to be consistent with pattern matches, or by just changing #{} to be consistent with the rest. Considering that the former would have broken the ability to write associative-array-maps as #{Key => Value} (or required even more special cases), the choice was clear. > That said, maybe it would be better to not allow #{} altogether > (deprecating it now and removing it later) and introduce an > empty_map() construct? The empty_map() type is slightly more verbose > but I don't expect it to be used frequently and, as such, the more > explicit name is worth it. And if I understand the proposal correctly, > both map() and #{_ => _} can still be used to represent any map, while > #{_ := _} represents a non-empty map, so it feels all of our bases are > covered. > > Thank you, > Although I am not against introducing an empty_map() alias, I don't think disallowing #{} would solve the problem. Indeed, your example can be trivially altered to commit the same mistake without using the #{} type; -spec increment(term(), #{count := C}) -> #{count := C}. increment(Key, #{count := C} = Map) when C >= 0 -> Map#{Key := maps:get(Key, Map) + 1}. If anything, I think the #{} case is the clearest example of type =/= pattern, and removing it would just obscure the problem. Thanks for your feedback // Magnus L?ng > *Jos? Valim* > www.plataformatec.com.br > Skype: jv.ptec > Founder and Director of R&D > > On Mon, Apr 11, 2016 at 12:45 PM, Kostis Sagonas > wrote: > > On 03/04/2016 11:59 AM, Bj?rn-Egil Dahlberg XB wrote: > > > I did the initial work for maps in Dialyzer but that > application is not > really part of my work area. If you want to contribute to > Dialyzer you > should probably talk to Hans Bolinder, Kostis Sagonas and/or > Stavros > Aronis. Hans talked about increasing Dialyzers knowledge about map > associations the other day but I don't think it is on his > agenda at the > moment. > > I don't think it will be supported in 19.0 either though I > have no clue > if Kostis or Stavros, or anyone else for that matter, is > working on it. > > > > Hijacking this thread to announce a proposal for a change in the > type syntax related to maps. > > The relevant document is here: > > https://gist.github.com/kostis/eaf4a06e643cf49314ba > > We would appreciate feedback from the community at this point. > > The intention is for this change to be part of Erlang/OTP 19.0. > > > Kostis > > PS. Most of the credit for this change goes to Magnus L?ng. > > PS2. For those that would like to experiment with the new type > syntax and use the new version of dialyzer, or just fancy reading > code diffs, there is also a related pull request that implements > all these: > > https://github.com/erlang/otp/pull/1014 > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From khitai.pang@REDACTED Mon Apr 11 18:43:01 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Tue, 12 Apr 2016 00:43:01 +0800 Subject: [erlang-questions] ssl_closed not always received In-Reply-To: <570B73D7.9080204@ninenines.eu> References: <570B6D55.7090604@ninenines.eu> <570B73D7.9080204@ninenines.eu> Message-ID: On 2016/4/11 17:52, Lo?c Hoguin wrote: > On 04/11/2016 11:43 AM, Khitai Pang wrote: >> On 2016/4/11 17:24, Lo?c Hoguin wrote: >>> On 04/11/2016 10:23 AM, Roger Lipscombe wrote: >>>> On 10 April 2016 at 22:34, Khitai Pang >>>> wrote: >>>>> How to make sure that the server process always get ssl_closed >>>>> when the >>>>> client process on a remote host quits? >>>> >>>> In the general case, you *can't*. This is due to the vagaries of TCP. >>>> To close a socket, the client will send a FIN packet to the server. If >>>> the network connection is lost (consider simply unplugging the cable), >>>> then the FIN will *never* arrive. If you need to know when the client >>>> has gone away, either implement some kind of application-level >>>> keepalive or enable TCP keepalive. >>> >>> You need a bi-directional ping mechanism at the application level. TCP >>> keepalive is not always enough. >> >> How about this? The client sends a heartbeat message to the server >> every 2 minutes and the server replies with a heartbeat ack. The server >> deems the connection as lost if no heartbeat received in 2 minutes and >> the client deems the connection as lost if heartbeat ask is not received >> in 2 minutes. This sounds like uni-directional to me but would it >> suffice? > > All that matters is that both endpoints send a ping message. It can be > sent as a response. > > One important point is that the client and server agree on timeouts > (this can be part of the protocol handshake or explicitly defined in > the implementation, they just need to agree on it). Note that there's > two timeouts: one for the client, and one for the server. They don't > have to match. By non-matching timeouts on client an server do you mean that we should give some grace time to the client? e.g. the client sends a message every 5 minutes but the server deems the connection as lost if no message is received in 6 minutes? Thanks Khitai > Using the client for sending pings (and the server just replies to > pongs) scales better because the server only needs to keep track of > one timeout (how long since last client message-- doesn't have to be > an explicit ping) instead of two (the previous one + a timeout for > sending pings). When you have one server for thousands of clients, > it's usually better to offload processing to the clients. > > So tl;dr yes, you got it right. From khitai.pang@REDACTED Mon Apr 11 18:48:28 2016 From: khitai.pang@REDACTED (Khitai Pang) Date: Tue, 12 Apr 2016 00:48:28 +0800 Subject: [erlang-questions] ssl_closed not always received In-Reply-To: <20160411134322.GE13113@fhebert-ltm2.internal.salesforce.com> References: <570B6D55.7090604@ninenines.eu> <570B73D7.9080204@ninenines.eu> <20160411134322.GE13113@fhebert-ltm2.internal.salesforce.com> Message-ID: On 2016/4/11 21:43, Fred Hebert wrote: > On 04/11, Lo?c Hoguin wrote: >> On 04/11/2016 11:43 AM, Khitai Pang wrote: >>> On 2016/4/11 17:24, Lo?c Hoguin wrote: >>> How about this? The client sends a heartbeat message to the server >>> every 2 minutes and the server replies with a heartbeat ack. The server >>> deems the connection as lost if no heartbeat received in 2 minutes and >>> the client deems the connection as lost if heartbeat ask is not >>> received >>> in 2 minutes. This sounds like uni-directional to me but would it >>> suffice? >> >> [...] >> >> So tl;dr yes, you got it right. >> > > One small gotcha I'd like to add if you plan on supporting mobile > users. A few months ago I read 'High Performance Browser Networking' > by Ilya Grigorik, and an interesting section mentioned the usage of > antennas and power consumption. > > An interesting thing described there is the cycle of powering antennas > in your phones up and down as data transfers are required, involvement > of remote servers, preservation of NAT caches, and so on. Basically, > the antenna power for 2G/3G/4G/LTE stuff will tend to be the biggest > power source of your phone, right after screen usage. WiFi's bad, but > not nearly as bad for the average case. > > That's all okay, but the thing is that there is a thing the author > calls the 'energy tail' that happens after each time a transmission is > done: > > http://i.imgur.com/7KfixDQ.png > > The idea being that antenna power-up is time-based on a state machine, > and transferring 2 kilobytes might be as significant for your battery > as transfering 2 megabytes if they tend to mostly happen on the same > high-energy cycle. > > A very significant section of the book is this one: > >> 46% of Battery Consumption to Transfer 0.2% of Total Bytes >> >> AT&T Labs Research published a great research paper ("Profiling >> Resource Usage for Mobile Applications") in which it analyzed a >> number of popular mobile applications for network and battery >> efficiency. Among these applications, Pandora serves as a great case >> study for the inefficiency of intermittent network transfers on >> mobile networks. >> >> Whenever a Pandora user plays a song, the entire music file is >> streamed by the application from the network in one shot, which is >> the correct behavior: burst as much data as you can, then turn off >> the radio for as long as possible. However, following the music >> transfer, the application would conduct periodic audience >> measurements by sending intermittent analytics pings every 60 >> seconds. The net effect? The analytics beacons accounted for 0.2% of >> the total transferred bytes and 46% of the total power consumption of >> the application! >> >> The beacon transfers are small, but the energy tails induced by the >> RRC state transitions were keeping the radio active for significantly >> longer, unnecessarily wasting 46% of the battery. By coalescing the >> analytics data into fewer requests, or by sending the audience data >> when the radio is already active, we can eliminate the unnecessary >> energy tails and almost double the power efficiency of the application! > > (source: > http://chimera.labs.oreilly.com/books/1230000000545/ch07.html#INEFFICIENCY_OF_PERIODIC_TRANSFERS) > > The fix? Raise the timeout to at least 5 minutes; it's the shortest > value the book recommends as mobile carriers' NAT caches tend to hold > for 5 mins: > >> Most mobile carriers set a 5?30 minute NAT connection timeout. >> Hence, you may need a periodic (5 minute) keepalive to keep an idle >> connection from being dropped. If you find yourself requiring more >> frequent keepalives, check your own server, proxy, and load balancer >> configuration first! > > (Source: > http://chimera.labs.oreilly.com/books/1230000000545/ch08.html#ELIMINATE_POLLING) > > Regards, > Fred. Great information! However, heartbeat messages seem to be necessary evil. I will try 5-min heartbeat. I also heard that when sending a message, keep the message as small as possible, because the bigger the message is, the higher power rate the antenna gets raised to and the longer time it keeps the power rate. Thanks Khitai From essen@REDACTED Mon Apr 11 18:57:56 2016 From: essen@REDACTED (=?UTF-8?Q?Lo=c3=afc_Hoguin?=) Date: Mon, 11 Apr 2016 18:57:56 +0200 Subject: [erlang-questions] ssl_closed not always received In-Reply-To: References: <570B6D55.7090604@ninenines.eu> <570B73D7.9080204@ninenines.eu> Message-ID: <570BD794.50203@ninenines.eu> On 04/11/2016 06:43 PM, Khitai Pang wrote: > On 2016/4/11 17:52, Lo?c Hoguin wrote: >> On 04/11/2016 11:43 AM, Khitai Pang wrote: >>> On 2016/4/11 17:24, Lo?c Hoguin wrote: >>>> On 04/11/2016 10:23 AM, Roger Lipscombe wrote: >>>>> On 10 April 2016 at 22:34, Khitai Pang >>>>> wrote: >>>>>> How to make sure that the server process always get ssl_closed >>>>>> when the >>>>>> client process on a remote host quits? >>>>> >>>>> In the general case, you *can't*. This is due to the vagaries of TCP. >>>>> To close a socket, the client will send a FIN packet to the server. If >>>>> the network connection is lost (consider simply unplugging the cable), >>>>> then the FIN will *never* arrive. If you need to know when the client >>>>> has gone away, either implement some kind of application-level >>>>> keepalive or enable TCP keepalive. >>>> >>>> You need a bi-directional ping mechanism at the application level. TCP >>>> keepalive is not always enough. >>> >>> How about this? The client sends a heartbeat message to the server >>> every 2 minutes and the server replies with a heartbeat ack. The server >>> deems the connection as lost if no heartbeat received in 2 minutes and >>> the client deems the connection as lost if heartbeat ask is not received >>> in 2 minutes. This sounds like uni-directional to me but would it >>> suffice? >> >> All that matters is that both endpoints send a ping message. It can be >> sent as a response. >> >> One important point is that the client and server agree on timeouts >> (this can be part of the protocol handshake or explicitly defined in >> the implementation, they just need to agree on it). Note that there's >> two timeouts: one for the client, and one for the server. They don't >> have to match. > > By non-matching timeouts on client an server do you mean that we should > give some grace time to the client? e.g. the client sends a message > every 5 minutes but the server deems the connection as lost if no > message is received in 6 minutes? I mean the client may consider the server disconnected if it hears nothing for X seconds, while the server may consider the client disconnected if it hears nothing for Y seconds. Then, you're right, there's also Z, which is the interval between client-initiated pings. But that one can be determined from X and Y. -- Lo?c Hoguin http://ninenines.eu Author of The Erlanger Playbook, A book about software development using Erlang From skribent_har@REDACTED Mon Apr 11 23:13:52 2016 From: skribent_har@REDACTED (Martin Hedberg) Date: Mon, 11 Apr 2016 21:13:52 +0000 Subject: [erlang-questions] Erlang application and web services Message-ID: Hi everyone I am more of a pointy hair boss crossed with Ratbert then a pure programmer so feel free to make a Dilbert joke if my question is stupid :-) . Have an idea for a desktop application that needs to connect to many external web services at the same time, handle both REST and SOAP and take some instructions from my own service in the cloud. This network oriented app made me think extra of Erlang. However it is said that premature optimization is the root to all evil in software development. My impression is that Erlang is much better at this type of work then Java. But there isn't (of what I know) any real all in one Erlang solution comparable to Red Hat JBoss. So I wonder: Would it be wise to include Erlang in an early phase (server side and app side) so I have a solid foundation or would it be premature optimization? I value tech support and if something goes south I would like that the support staff have an understanding of the whole system. I hope I make sense and I am very greatful for all help I can get. Thanks Martin -------------- next part -------------- An HTML attachment was scrubbed... URL: From Oliver.Korpilla@REDACTED Mon Apr 11 23:18:09 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Mon, 11 Apr 2016 23:18:09 +0200 Subject: [erlang-questions] UDP Multicast - how does it work? In-Reply-To: <20160411093858.GA44030@staff.retn.net> References: , <20160411093858.GA44030@staff.retn.net> Message-ID: Hello, Alexandre. Given your input we abandoned the approach we had before and will implement the application without broadcast or multicast. We'll just switch back to TCP/IP with a simple 1 server/multiple clients approach though that means foregoing something we wanted to showcase. Thank you for your answer. Kind regards, Oliver ? Gesendet:?Montag, 11. April 2016 um 11:38 Uhr Von:?"Alexandre Snarskii" An:?"Oliver Korpilla" Cc:?erlang-questions@REDACTED Betreff:?Re: [erlang-questions] UDP Multicast - how does it work? On Mon, Apr 11, 2016 at 09:51:12AM +0200, Oliver Korpilla wrote: > Hello. > > I'm lost on the basics here. Sorry if this seems like a fishing expedition. > > What I want to do: > > Step 1) Reach several processes on the same machine with one send. > Step 2) Reach several processes on possibly different machines with one send. > > I've tried the following: > > -module(test_multicast). > -export([start/0, receiver/1]). > -define(MULTICAST_ADDRESS, {127,0,10,1}). 127.0.10.1 is not a valid multicast address. Multicast addresses are in the range 224.0.0.0 - 239.255.255.255. More than that, rfc1122 section 3.2.1.3 clearly says that 127.x.x.x is an Internal host loopback address. Addresses of this form MUST NOT appear outside a host. and so no OS will accept packets addresses from/to 127.x.x.x on ethernet interfaces. > > receiver(Id) -> > io:format("Started multicast receiver ~p~n", [Id]), > {ok, Socket} = gen_udp:open(3333, [binary, {active, true}, {reuseaddr, true}, {multicast_if, ?MULTICAST_ADDRESS}, {multicast_loop, true}]), In order to receive multicast packets you must explicitly join this socket to a multicast group using {add_membership, {MultiAddress, InterfaceAddress}}: Join a multicast group. Example of working multicast between two servers in the same subnet: 1> {ok, Sock} = gen_udp:open(3333, [binary, {active, true}, {add_membership, {{239,0,10,1}, {0,0,0,0}}}]). {ok,#Port<0.481>} 2> gen_udp:send(Sock, {239,0,10,1}, 3333, <<"hello, world">>). ok 3> flush(). Shell got {udp,#Port<0.481>,{xx,xxx,xxx,2},3333,<<"hello, world">>} ok on another machine: 1> {ok, Sock} = gen_udp:open(3333, [binary, {active, true}, {add_membership, {{239,0,10,1}, {0,0,0,0}}}]). {ok,#Port<0.568>} 2> flush(). Shell got {udp,#Port<0.568>,{xx,xxx,xxx,2},3333,<<"hello, world">>} ok 3> > receiver_loop(Id, Socket). > > receiver_loop(Id, Socket) -> > receive > {udp, _RemoteSocket, _RemoteHost, _RemotePort, Payload } -> > io:format("Receiver ~p received message ~p~n", [Id, Payload]), > receiver_loop(Id, Socket) > end. > > send() -> > io:format("Sending...~n"), > {ok, Socket} = gen_udp:open(0), > ok = gen_udp:send(Socket, ?MULTICAST_ADDRESS, 3333, << "Hello, World." >>). > > start() -> > spawn_link(?MODULE, receiver, [1]), > spawn_link(?MODULE, receiver, [2]), > > timer:sleep(100), > send(). > > But the only one to receive the message is receiver #2. It's as if it takes the port away from receiver #1... I thought reuseaddr was meant to bind several sockets to the same port? > > Thank you for any advice, > Oliver > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From chandrashekhar.mullaparthi@REDACTED Tue Apr 12 00:47:08 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Mon, 11 Apr 2016 23:47:08 +0100 Subject: [erlang-questions] Erlang application and web services In-Reply-To: References: Message-ID: Hi Martin, On 11 April 2016 at 22:13, Martin Hedberg wrote: > Hi everyone > > I am more of a pointy hair boss crossed with Ratbert then a > pure programmer so feel free to make a Dilbert joke if my question is > stupid :-) . Have an idea for a desktop application that needs to connect > to many external web services at the same time, handle both REST and SOAP > and take some instructions from my own service in the cloud. This network > oriented app made me think extra of Erlang. > > However it is said that premature optimization is the root to all evil in > software development. > Yes, that is the Erlang way, but beware, it is a bit more nuanced than that [1]. But yes, for early stage development/prototyping when you're still trying to understand the problem, it is a valid statement. > My impression is that Erlang is much better at this type of work then > Java. > You will find a lot of agreement on this mailing list :-) > But there isn't (of what I know) any real all in one Erlang solution > comparable to Red Hat JBoss. > Not sure what you mean. The following apps will give you the foundation of what you have asked for. cowboy - for the web server side. https://github.com/ninenines/cowboy soap - for the SOAP encoding/decoding. https://github.com/bet365/soap/ > So I wonder: Would it be wise to include Erlang in an early phase (server > side and app side) so I have a solid foundation or would it be premature > optimization? > Erlang is perfect for prototyping as you can get working code quickly. Obviously this depends on having someone who is familiar with the language. So if you've got access to hardcore Java programmers, and you want something off the ground quickly, you're better off doing it in Java. > I value tech support and if something goes south I would like that the > support staff have an understanding of the whole system. > Enough understanding to fix the problem? In which case there is no shortcut to actually understanding the code. If not, then it very much depends on how much diagnostics are built into your running system. cheers, Chandru [1] http://joeduffyblog.com/2010/09/06/the-premature-optimization-is-evil-myth/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Tue Apr 12 00:48:15 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 12 Apr 2016 10:48:15 +1200 Subject: [erlang-questions] Status of map pair support in dialyzer In-Reply-To: References: <56D96A9D.6060502@ericsson.com> <570B8061.4030009@cs.ntua.gr> Message-ID: Couldn't the syntax for "any map" be something like #{_}? From bernard@REDACTED Tue Apr 12 02:13:58 2016 From: bernard@REDACTED (Bernard Duggan) Date: Tue, 12 Apr 2016 10:13:58 +1000 Subject: [erlang-questions] Status of map pair support in dialyzer In-Reply-To: <570B8061.4030009@cs.ntua.gr> References: <56D96A9D.6060502@ericsson.com> <570B8061.4030009@cs.ntua.gr> Message-ID: On Mon, Apr 11, 2016 at 8:45 PM, Kostis Sagonas wrote: > We would appreciate feedback from the community at this point. > Looks great. Am I right in my reading of this to think that I can describe a map containing, say, *only* the keys 'a' and 'b' like this: #{a := any(), b:= any(), any() => none()} ? Cheers, Bernard -------------- next part -------------- An HTML attachment was scrubbed... URL: From bernard@REDACTED Tue Apr 12 02:17:10 2016 From: bernard@REDACTED (Bernard Duggan) Date: Tue, 12 Apr 2016 10:17:10 +1000 Subject: [erlang-questions] Status of map pair support in dialyzer In-Reply-To: References: <56D96A9D.6060502@ericsson.com> <570B8061.4030009@cs.ntua.gr> Message-ID: Sorry, disregard this question - I just read some of the later posts and it's now clearer to me that the any() => none() bit is actually implied. B On Tue, Apr 12, 2016 at 10:13 AM, Bernard Duggan wrote: > On Mon, Apr 11, 2016 at 8:45 PM, Kostis Sagonas wrote: > >> We would appreciate feedback from the community at this point. >> > > Looks great. > > Am I right in my reading of this to think that I can describe a map > containing, say, *only* the keys 'a' and 'b' like this: > > #{a := any(), b:= any(), any() => none()} > > ? > > Cheers, > > Bernard > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Tue Apr 12 09:59:20 2016 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 12 Apr 2016 09:59:20 +0200 Subject: [erlang-questions] Status of map pair support in dialyzer In-Reply-To: References: <56D96A9D.6060502@ericsson.com> <570B8061.4030009@cs.ntua.gr> Message-ID: <570CAAD8.4030800@cs.ntua.gr> On 04/12/2016 12:48 AM, Richard A. O'Keefe wrote: > Couldn't the syntax for "any map" be something like #{_}? Disregarding for a moment that this most likely requires a change in the parser, this would buy us what exactly? Currently, we have the following two ways to write the any map: map() #{_=>_} Why introduce one more? Save typing three characters? (or just one, if you compare with the first) Besides, I think that #{_} is fundamentally wrong. The notation [_] exists because _ is the shorthand of the any() type and therefore [_] is shorthand for [any()]. Lists are parametric on just one type (variable), but maps, which are key-value associations are on two. That's why #{_=>_} makes more sense than #{_}. Another way of saying/seeing the above is that map() is a shorthand for map(_, _) in the same way that dict:dict() is a shorthand for dict:dict(_, _). There is no dict:dict(_). Kostis From bchesneau@REDACTED Tue Apr 12 10:40:44 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Tue, 12 Apr 2016 08:40:44 +0000 Subject: [erlang-questions] enif_schedule_nif return Message-ID: I'm not sure to understand what can I expect from the return of enif_schedule_nif. The doc says: Be aware that enif_schedule_nif, as its name implies, only schedules the NIF for future execution. The calling NIF does not block waiting for the scheduled NIF to execute and return, which means that the calling NIF can't expect to receive the scheduled NIF return value and use it for further operations. So should I handle to return the data asynchronously to the calling process using enf_send? In that case what is the return of the function? - benoit -------------- next part -------------- An HTML attachment was scrubbed... URL: From sverker.eriksson@REDACTED Tue Apr 12 11:55:52 2016 From: sverker.eriksson@REDACTED (Sverker Eriksson) Date: Tue, 12 Apr 2016 11:55:52 +0200 Subject: [erlang-questions] enif_schedule_nif return In-Reply-To: References: Message-ID: <570CC628.1070500@ericsson.com> On 04/12/2016 10:40 AM, Benoit Chesneau wrote: > I'm not sure to understand what can I expect from the return of > enif_schedule_nif. The doc says: > > Be aware that enif_schedule_nif, as its name implies, only > schedules the NIF for future execution. The calling NIF does not block > waiting for the scheduled NIF to execute and return, which means that > the calling NIF can't expect to receive the scheduled NIF return value > and use it for further operations. > > So should I handle to return the data asynchronously to the calling > process using enf_send? In that case what is the return of the function? > > No. The return value of your scheduled function will be the return value from your original Erlang nif call. /Sverker, Erlang/OTP -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.james.morgan@REDACTED Tue Apr 12 17:48:14 2016 From: peter.james.morgan@REDACTED (Peter Morgan) Date: Tue, 12 Apr 2016 16:48:14 +0100 Subject: [erlang-questions] udp multicast: problem with OSX? In-Reply-To: References: Message-ID: Hi Radek, Thanks - Yes, I have had multicast working previously on OSX and Linux way back somewhere in 2011 with https://github.com/shortishly/mdns. I had some trouble compiling gossiperl - I tried with R18 but the rebar.config needed R17, when I tried with R17 one of the dependencies required R18. Do you have another branch that I missed? My issue seems to be that I can receive multicast on OSX, but I cannot send? Thanks Peter. > On 7 Apr 2016, at 15:39, Rad Gruchalski wrote: > > Hi Peter, > > Multicast works absolutely fine on OSX. Here is how multicast is enabled: > https://github.com/gossiperl/gossiperl/wiki/multicast-overlays#setting-up-multicast-on-os-x > > And then using the multicast options: > https://github.com/gossiperl/gossiperl/blob/master/include/gossiperl.hrl#L107 > > Works perfect on Linux, OSX and RPi. > > > > Best regards, > > Radek Gruchalski > > radek@REDACTED > > de.linkedin.com/in/radgruchalski/ > > Confidentiality: > This communication is intended for the above-named person and may be confidential and/or legally privileged. > If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately. > On Thursday, 7 April 2016 at 16:14, Peter Morgan wrote: > >> Hello - >> >> I?m having some trouble getting UDP multicast working on OSX, whereas the same code on Linux works fine. >> >> On linux (fedora 23): >> >> [pmorgan@REDACTED ~]$ erl >> Erlang/OTP 18 [erts-7.3] [source] [64-bit] [async-threads:10] [hipe] [kernel-poll:false] >> >> Eshell V7.3 (abort with ^G) >> 1> Address = {224, 0, 0, 251}. >> {224,0,0,251} >> 2> Port = 5353. >> 5353 >> 3> {ok, Socket} = gen_udp:open(Port, [binary, {ip, Address}, {add_membership, {Address, {0,0,0,0}}}, {reuseaddr, true}]). >> {ok,#Port<0.543>} >> 4> gen_udp:send(Socket, Address, Port, <<"hello world">>). >> ok >> >> >> Whereas on OSX: >> >> Office-iMac:mdns pmorgan$ sw_vers >> ProductName: Mac OS X >> ProductVersion: 10.11.4 >> BuildVersion: 15E61b >> >> Office-iMac:mdns pmorgan$ erl >> Erlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace] >> >> Eshell V7.3 (abort with ^G) >> 1> Address = {224, 0, 0, 251}. >> {224,0,0,251} >> 2> Port = 5353. >> 5353 >> 3> {ok, Socket} = gen_udp:open(Port, [binary, {ip, Address}, {add_membership, {Address, {0,0,0,0}}}, {reuseaddr, true}]). >> {ok,#Port<0.553>} >> 4> gen_udp:send(Socket, Address, Port, <<"hello world">>). >> {error,eaddrnotavail} >> >> >> I?ve tried various other options in gen_udp:open/2, but not found a combination that works for OSX. Any ideas please? The above is a simplified test case of https://github.com/shortishly/mdns/blob/master/src/mdns_udp.erl#L41-L49 >> >> Thanks, >> Peter. >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From radek@REDACTED Tue Apr 12 18:04:40 2016 From: radek@REDACTED (Radoslaw Gruchalski) Date: Tue, 12 Apr 2016 16:04:40 +0000 (UTC) Subject: [erlang-questions] udp multicast: problem with OSX? In-Reply-To: References: Message-ID: Peter, I will have a look at the compilation problems before the end of this working week. Sent from Outlook Mobile On Tue, Apr 12, 2016 at 8:48 AM -0700, "Peter Morgan" wrote: Hi Radek, Thanks - Yes, I have had multicast working previously on OSX and Linux way back somewhere in 2011 with?https://github.com/shortishly/mdns. I had some trouble compiling gossiperl - I tried with R18 but the rebar.config needed R17, when I tried with R17 one of the dependencies required R18. Do you have another branch that I missed? My issue seems to be that I can receive multicast on OSX, but I cannot send? ThanksPeter. On 7 Apr 2016, at 15:39, Rad Gruchalski wrote: Hi Peter, Multicast works absolutely fine on OSX. Here is how multicast is enabled:https://github.com/gossiperl/gossiperl/wiki/multicast-overlays#setting-up-multicast-on-os-x And then using the multicast options:https://github.com/gossiperl/gossiperl/blob/master/include/gossiperl.hrl#L107 Works perfect on Linux, OSX and RPi. Best regards, Radek Gruchalski radek@REDACTED de.linkedin.com/in/radgruchalski/ Confidentiality: This communication is intended for the above-named person and may be confidential and/or legally privileged. If it has come to you in error you must take no action based on it, nor must you copy or show it to anyone; please delete/destroy and inform the sender immediately. On Thursday, 7 April 2016 at 16:14, Peter Morgan wrote: Hello - I?m having some trouble getting UDP multicast working on OSX, whereas the same code on Linux works fine. On linux (fedora 23): [pmorgan@REDACTED ~]$ erlErlang/OTP 18 [erts-7.3] [source] [64-bit] [async-threads:10] [hipe] [kernel-poll:false] Eshell V7.3 (abort with ^G)1> Address = {224, 0, 0, 251}.{224,0,0,251}2> Port = 5353.53533> {ok, Socket} = gen_udp:open(Port, [binary, {ip, Address}, {add_membership, {Address, {0,0,0,0}}}, {reuseaddr, true}]).{ok,#Port<0.543>}4> gen_udp:send(Socket, Address, Port, <<"hello world">>).ok Whereas on OSX: Office-iMac:mdns pmorgan$ sw_vers ProductName: Mac OS XProductVersion: 10.11.4BuildVersion: 15E61b Office-iMac:mdns pmorgan$ erlErlang/OTP 18 [erts-7.3] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace] Eshell V7.3 (abort with ^G)1> Address = {224, 0, 0, 251}.{224,0,0,251}2> Port = 5353.53533> {ok, Socket} = gen_udp:open(Port, [binary, {ip, Address}, {add_membership, {Address, {0,0,0,0}}}, {reuseaddr, true}]).{ok,#Port<0.553>}4> gen_udp:send(Socket, Address, Port, <<"hello world">>).{error,eaddrnotavail} I?ve tried various other options in gen_udp:open/2, but not found a combination that works for OSX. Any ideas please? The above is a simplified test case of https://github.com/shortishly/mdns/blob/master/src/mdns_udp.erl#L41-L49 Thanks,Peter. _______________________________________________erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.james.morgan@REDACTED Tue Apr 12 18:06:36 2016 From: peter.james.morgan@REDACTED (Peter Morgan) Date: Tue, 12 Apr 2016 17:06:36 +0100 Subject: [erlang-questions] udp multicast: problem with OSX? In-Reply-To: References: Message-ID: Hi Torben, Max, On OSX I can receive multicast traffic using the following options (the same as Max?s pretty much): [binary, inet, {reuseaddr, true}, {ip, MulticastAddress}, {multicast_ttl, 4}, {multicast_loop, true}, {add_membership, {MulticastAddress, {0, 0, 0, 0}}}, {active, once}]. Where MulticastAddress is {224, 0, 0, 251}. I?m using that address because it is one used by OSX, Linux/everything for multicast DNS :) On OSX I can happily receive multicast traffic on a gen_udp socket that is opened with the above parameters. However, on OSX I cannot send to the same socket using: gen_udp:send(Socket, MulticastAddress, Port, SomeMessage). On Linux (Fedora 23) the above send does work. On OSX I get {error, eaddrnotavail} on send. I?m sure I have had this working on OSX and Linux together, so I?m probably doing some rather stupid? OSX is El Capitan (10.11.4 beta). Erlang is 18.3 (brew). I?ve tried replacing the {0, 0, 0, 0} in {add_membership, {MulticastAddress, {0, 0, 0, 0}}} with IP address of the interface, but it didn?t make any difference. Thanks, Peter. > On 11 Apr 2016, at 16:58, Max Lapshin wrote: > > This is how it is working for me: > > {ok, Addr} = inet_parse:address(Host), > Common = [binary,{reuseaddr,true},{recbuf,2*1024*1024},inet,{ip,Addr},{read_packets,100}], > Options = case is_multicast(Addr) of > true -> > Multicast = [{multicast_ttl,4},{multicast_loop,true},{add_membership,{Addr,GwIP}}], > Common ++ Multicast; > false -> > Common > end, > From peter.james.morgan@REDACTED Tue Apr 12 20:40:48 2016 From: peter.james.morgan@REDACTED (Peter Morgan) Date: Tue, 12 Apr 2016 19:40:48 +0100 Subject: [erlang-questions] udp multicast: problem with OSX? [solved] In-Reply-To: References: Message-ID: <29B26EB0-01EA-4DA7-8C42-E14E1A2BABBD@gmail.com> Hello - It appears that OSX doesn?t like to reuse a port on gen_udp:open/2 when sending, whereas Linux appears to be quite happy doing so: I had for both sender and receiver: gen_udp:open(5353, [{mode, binary}, {reuseaddr, true}, {ip, {224, 0, 0, 251}}, {multicast_ttl, 4}, {multicast_loop, true}, {broadcast, true}, {add_membership, {{224, 0, 0, 251}, {0, 0, 0, 0}}}, {active, once}]). Where 5353 is the port for mDNS - already being used by mDNS daemons on both Linux and OSX. The above works fine for Linux (to send and receive), whereas on OSX it only appears to work for receiving. To make it work on OSX, I need to call gen_udp:open/2 slightly differently depending on whether I want to call gen_udp:send/4 on the returned socket: {ok, Sender} = gen_udp:open(0, [binary]). gen_udp:send(Sender, {224, 0, 0, 251}, 5353, <>). Note the ?0? as the Port parameter to gen_udp:open/2 above. Whereas the receiver must be (as originally): {ok, Sender} = gen_udp:open(5353, [{mode, binary}, {reuseaddr, true}, {ip, {224, 0, 0, 251}}, {multicast_ttl, 4}, {multicast_loop, true}, {broadcast, true}, {add_membership, {{224, 0, 0, 251}, {0, 0, 0, 0}}}, {active, once}]). On Linux, it seems quite happy for the sender and receiver to use the {reuseaddr, true} form immediately above and port 5353 as the parameter to gen_udp:open/2. OSX is OK until gen_udp:send/4 is called it responds with: {error, eaddrnotavail}. The updated code is https://github.com/shortishly/mdns/blob/develop/src/mdns_udp.erl and now works for both OSX and Linux. Thanks, Peter. > On 12 Apr 2016, at 17:06, Peter Morgan wrote: > > Hi Torben, Max, > > On OSX I can receive multicast traffic using the following options (the same as Max?s pretty much): > > [binary, > inet, > {reuseaddr, true}, > {ip, MulticastAddress}, > {multicast_ttl, 4}, > {multicast_loop, true}, > {add_membership, {MulticastAddress, {0, 0, 0, 0}}}, > {active, once}]. > > Where MulticastAddress is {224, 0, 0, 251}. I?m using that address because it is one used by OSX, Linux/everything for multicast DNS :) > > On OSX I can happily receive multicast traffic on a gen_udp socket that is opened with the above parameters. > > However, on OSX I cannot send to the same socket using: > > gen_udp:send(Socket, MulticastAddress, Port, SomeMessage). > > On Linux (Fedora 23) the above send does work. On OSX I get {error, eaddrnotavail} on send. > > I?m sure I have had this working on OSX and Linux together, so I?m probably doing some rather stupid? > > OSX is El Capitan (10.11.4 beta). Erlang is 18.3 (brew). > > I?ve tried replacing the {0, 0, 0, 0} in {add_membership, {MulticastAddress, {0, 0, 0, 0}}} with IP address of the interface, but it didn?t make any difference. > > Thanks, > Peter. > > > > >> On 11 Apr 2016, at 16:58, Max Lapshin wrote: >> >> This is how it is working for me: >> >> {ok, Addr} = inet_parse:address(Host), >> Common = [binary,{reuseaddr,true},{recbuf,2*1024*1024},inet,{ip,Addr},{read_packets,100}], >> Options = case is_multicast(Addr) of >> true -> >> Multicast = [{multicast_ttl,4},{multicast_loop,true},{add_membership,{Addr,GwIP}}], >> Common ++ Multicast; >> false -> >> Common >> end, >> > From steve@REDACTED Tue Apr 12 20:43:00 2016 From: steve@REDACTED (steve@REDACTED) Date: Tue, 12 Apr 2016 19:43:00 +0100 Subject: [erlang-questions] udp multicast: problem with OSX? [solved] In-Reply-To: <29B26EB0-01EA-4DA7-8C42-E14E1A2BABBD@gmail.com> References: <29B26EB0-01EA-4DA7-8C42-E14E1A2BABBD@gmail.com> Message-ID: Hi Peter, On OSX, try adding: { raw, 16#ffff, 16#0200, <<1:32/native>> } into the options list on the gen_udp open? Cheers, Steve --? Sent with Airmail On 12 April 2016 at 19:41:00, Peter Morgan (peter.james.morgan@REDACTED) wrote: Hello - It appears that OSX doesn?t like to reuse a port on gen_udp:open/2 when sending, whereas Linux appears to be quite happy doing so: I had for both sender and receiver: gen_udp:open(5353, [{mode, binary}, {reuseaddr, true}, {ip, {224, 0, 0, 251}}, {multicast_ttl, 4}, {multicast_loop, true}, {broadcast, true}, {add_membership, {{224, 0, 0, 251}, {0, 0, 0, 0}}}, {active, once}]). Where 5353 is the port for mDNS - already being used by mDNS daemons on both Linux and OSX. The above works fine for Linux (to send and receive), whereas on OSX it only appears to work for receiving. To make it work on OSX, I need to call gen_udp:open/2 slightly differently depending on whether I want to call gen_udp:send/4 on the returned socket: {ok, Sender} = gen_udp:open(0, [binary]). gen_udp:send(Sender, {224, 0, 0, 251}, 5353, <>). Note the ?0? as the Port parameter to gen_udp:open/2 above. Whereas the receiver must be (as originally): {ok, Sender} = gen_udp:open(5353, [{mode, binary}, {reuseaddr, true}, {ip, {224, 0, 0, 251}}, {multicast_ttl, 4}, {multicast_loop, true}, {broadcast, true}, {add_membership, {{224, 0, 0, 251}, {0, 0, 0, 0}}}, {active, once}]). On Linux, it seems quite happy for the sender and receiver to use the {reuseaddr, true} form immediately above and port 5353 as the parameter to gen_udp:open/2. OSX is OK until gen_udp:send/4 is called it responds with: {error, eaddrnotavail}. The updated code is https://github.com/shortishly/mdns/blob/develop/src/mdns_udp.erl and now works for both OSX and Linux. Thanks, Peter. > On 12 Apr 2016, at 17:06, Peter Morgan wrote: > > Hi Torben, Max, > > On OSX I can receive multicast traffic using the following options (the same as Max?s pretty much): > > [binary, > inet, > {reuseaddr, true}, > {ip, MulticastAddress}, > {multicast_ttl, 4}, > {multicast_loop, true}, > {add_membership, {MulticastAddress, {0, 0, 0, 0}}}, > {active, once}]. > > Where MulticastAddress is {224, 0, 0, 251}. I?m using that address because it is one used by OSX, Linux/everything for multicast DNS :) > > On OSX I can happily receive multicast traffic on a gen_udp socket that is opened with the above parameters. > > However, on OSX I cannot send to the same socket using: > > gen_udp:send(Socket, MulticastAddress, Port, SomeMessage). > > On Linux (Fedora 23) the above send does work. On OSX I get {error, eaddrnotavail} on send. > > I?m sure I have had this working on OSX and Linux together, so I?m probably doing some rather stupid? > > OSX is El Capitan (10.11.4 beta). Erlang is 18.3 (brew). > > I?ve tried replacing the {0, 0, 0, 0} in {add_membership, {MulticastAddress, {0, 0, 0, 0}}} with IP address of the interface, but it didn?t make any difference. > > Thanks, > Peter. > > > > >> On 11 Apr 2016, at 16:58, Max Lapshin wrote: >> >> This is how it is working for me: >> >> {ok, Addr} = inet_parse:address(Host), >> Common = [binary,{reuseaddr,true},{recbuf,2*1024*1024},inet,{ip,Addr},{read_packets,100}], >> Options = case is_multicast(Addr) of >> true -> >> Multicast = [{multicast_ttl,4},{multicast_loop,true},{add_membership,{Addr,GwIP}}], >> Common ++ Multicast; >> false -> >> Common >> end, >> > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.james.morgan@REDACTED Tue Apr 12 20:54:25 2016 From: peter.james.morgan@REDACTED (Peter Morgan) Date: Tue, 12 Apr 2016 19:54:25 +0100 Subject: [erlang-questions] udp multicast: problem with OSX? [solved] In-Reply-To: References: <29B26EB0-01EA-4DA7-8C42-E14E1A2BABBD@gmail.com> Message-ID: <84F4B3EF-9CC1-4E29-9580-0E014287656E@gmail.com> Hi Steve - Whoa! What magic is that doing? Thanks Peter > On 12 Apr 2016, at 19:43, steve@REDACTED wrote: > > Hi Peter, > > On OSX, try adding: > > { raw, 16#ffff, 16#0200, <<1:32/native>> } > > into the options list on the gen_udp open? > > Cheers, > > Steve > > -- > > Sent with Airmail > >> On 12 April 2016 at 19:41:00, Peter Morgan (peter.james.morgan@REDACTED) wrote: >> >> Hello - >> >> It appears that OSX doesn?t like to reuse a port on gen_udp:open/2 when sending, whereas Linux appears to be quite happy doing so: >> >> I had for both sender and receiver: >> >> gen_udp:open(5353, [{mode, binary}, >> {reuseaddr, true}, >> {ip, {224, 0, 0, 251}}, >> {multicast_ttl, 4}, >> {multicast_loop, true}, >> {broadcast, true}, >> {add_membership, {{224, 0, 0, 251}, {0, 0, 0, 0}}}, >> {active, once}]). >> >> Where 5353 is the port for mDNS - already being used by mDNS daemons on both Linux and OSX. The above works fine for Linux (to send and receive), whereas on OSX it only appears to work for receiving. >> >> To make it work on OSX, I need to call gen_udp:open/2 slightly differently depending on whether I want to call gen_udp:send/4 on the returned socket: >> >> {ok, Sender} = gen_udp:open(0, [binary]). >> gen_udp:send(Sender, {224, 0, 0, 251}, 5353, <>). >> >> Note the ?0? as the Port parameter to gen_udp:open/2 above. >> >> Whereas the receiver must be (as originally): >> >> {ok, Sender} = gen_udp:open(5353, [{mode, binary}, >> {reuseaddr, true}, >> {ip, {224, 0, 0, 251}}, >> {multicast_ttl, 4}, >> {multicast_loop, true}, >> {broadcast, true}, >> {add_membership, {{224, 0, 0, 251}, {0, 0, 0, 0}}}, >> {active, once}]). >> >> On Linux, it seems quite happy for the sender and receiver to use the {reuseaddr, true} form immediately above and port 5353 as the parameter to gen_udp:open/2. OSX is OK until gen_udp:send/4 is called it responds with: {error, eaddrnotavail}. >> >> The updated code is https://github.com/shortishly/mdns/blob/develop/src/mdns_udp.erl and now works for both OSX and Linux. >> >> >> Thanks, >> Peter. >> >> >> >> >> >> >> > On 12 Apr 2016, at 17:06, Peter Morgan wrote: >> > >> > Hi Torben, Max, >> > >> > On OSX I can receive multicast traffic using the following options (the same as Max?s pretty much): >> > >> > [binary, >> > inet, >> > {reuseaddr, true}, >> > {ip, MulticastAddress}, >> > {multicast_ttl, 4}, >> > {multicast_loop, true}, >> > {add_membership, {MulticastAddress, {0, 0, 0, 0}}}, >> > {active, once}]. >> > >> > Where MulticastAddress is {224, 0, 0, 251}. I?m using that address because it is one used by OSX, Linux/everything for multicast DNS :) >> > >> > On OSX I can happily receive multicast traffic on a gen_udp socket that is opened with the above parameters. >> > >> > However, on OSX I cannot send to the same socket using: >> > >> > gen_udp:send(Socket, MulticastAddress, Port, SomeMessage). >> > >> > On Linux (Fedora 23) the above send does work. On OSX I get {error, eaddrnotavail} on send. >> > >> > I?m sure I have had this working on OSX and Linux together, so I?m probably doing some rather stupid? >> > >> > OSX is El Capitan (10.11.4 beta). Erlang is 18.3 (brew). >> > >> > I?ve tried replacing the {0, 0, 0, 0} in {add_membership, {MulticastAddress, {0, 0, 0, 0}}} with IP address of the interface, but it didn?t make any difference. >> > >> > Thanks, >> > Peter. >> > >> > >> > >> > >> >> On 11 Apr 2016, at 16:58, Max Lapshin wrote: >> >> >> >> This is how it is working for me: >> >> >> >> {ok, Addr} = inet_parse:address(Host), >> >> Common = [binary,{reuseaddr,true},{recbuf,2*1024*1024},inet,{ip,Addr},{read_packets,100}], >> >> Options = case is_multicast(Addr) of >> >> true -> >> >> Multicast = [{multicast_ttl,4},{multicast_loop,true},{add_membership,{Addr,GwIP}}], >> >> Common ++ Multicast; >> >> false -> >> >> Common >> >> end, >> >> >> > >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve@REDACTED Tue Apr 12 21:00:25 2016 From: steve@REDACTED (steve@REDACTED) Date: Tue, 12 Apr 2016 20:00:25 +0100 Subject: [erlang-questions] udp multicast: problem with OSX? [solved] In-Reply-To: <84F4B3EF-9CC1-4E29-9580-0E014287656E@gmail.com> References: <29B26EB0-01EA-4DA7-8C42-E14E1A2BABBD@gmail.com> <84F4B3EF-9CC1-4E29-9580-0E014287656E@gmail.com> Message-ID: It?s setting the SO_REUSEPORT option -?https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man2/setsockopt.2.html 16#ffff == SOL_SOCKET 16#0200 == SO_REUSEPORT 1:32 == true? Some -defines would probably have made it more readable :) --? Sent with Airmail On 12 April 2016 at 19:54:27, Peter Morgan (peter.james.morgan@REDACTED) wrote: Hi Steve - Whoa! What magic is that doing? Thanks Peter On 12 Apr 2016, at 19:43, steve@REDACTED wrote: Hi Peter, On OSX, try adding: { raw, 16#ffff, 16#0200, <<1:32/native>> } into the options list on the gen_udp open? Cheers, Steve --? Sent with Airmail On 12 April 2016 at 19:41:00, Peter Morgan (peter.james.morgan@REDACTED) wrote: Hello - It appears that OSX doesn?t like to reuse a port on gen_udp:open/2 when sending, whereas Linux appears to be quite happy doing so: I had for both sender and receiver: gen_udp:open(5353, [{mode, binary}, {reuseaddr, true}, {ip, {224, 0, 0, 251}}, {multicast_ttl, 4}, {multicast_loop, true}, {broadcast, true}, {add_membership, {{224, 0, 0, 251}, {0, 0, 0, 0}}}, {active, once}]). Where 5353 is the port for mDNS - already being used by mDNS daemons on both Linux and OSX. The above works fine for Linux (to send and receive), whereas on OSX it only appears to work for receiving. To make it work on OSX, I need to call gen_udp:open/2 slightly differently depending on whether I want to call gen_udp:send/4 on the returned socket: {ok, Sender} = gen_udp:open(0, [binary]). gen_udp:send(Sender, {224, 0, 0, 251}, 5353, <>). Note the ?0? as the Port parameter to gen_udp:open/2 above. Whereas the receiver must be (as originally): {ok, Sender} = gen_udp:open(5353, [{mode, binary}, {reuseaddr, true}, {ip, {224, 0, 0, 251}}, {multicast_ttl, 4}, {multicast_loop, true}, {broadcast, true}, {add_membership, {{224, 0, 0, 251}, {0, 0, 0, 0}}}, {active, once}]). On Linux, it seems quite happy for the sender and receiver to use the {reuseaddr, true} form immediately above and port 5353 as the parameter to gen_udp:open/2. OSX is OK until gen_udp:send/4 is called it responds with: {error, eaddrnotavail}. The updated code is https://github.com/shortishly/mdns/blob/develop/src/mdns_udp.erl and now works for both OSX and Linux. Thanks, Peter. > On 12 Apr 2016, at 17:06, Peter Morgan wrote: > > Hi Torben, Max, > > On OSX I can receive multicast traffic using the following options (the same as Max?s pretty much): > > [binary, > inet, > {reuseaddr, true}, > {ip, MulticastAddress}, > {multicast_ttl, 4}, > {multicast_loop, true}, > {add_membership, {MulticastAddress, {0, 0, 0, 0}}}, > {active, once}]. > > Where MulticastAddress is {224, 0, 0, 251}. I?m using that address because it is one used by OSX, Linux/everything for multicast DNS :) > > On OSX I can happily receive multicast traffic on a gen_udp socket that is opened with the above parameters. > > However, on OSX I cannot send to the same socket using: > > gen_udp:send(Socket, MulticastAddress, Port, SomeMessage). > > On Linux (Fedora 23) the above send does work. On OSX I get {error, eaddrnotavail} on send. > > I?m sure I have had this working on OSX and Linux together, so I?m probably doing some rather stupid? > > OSX is El Capitan (10.11.4 beta). Erlang is 18.3 (brew). > > I?ve tried replacing the {0, 0, 0, 0} in {add_membership, {MulticastAddress, {0, 0, 0, 0}}} with IP address of the interface, but it didn?t make any difference. > > Thanks, > Peter. > > > > >> On 11 Apr 2016, at 16:58, Max Lapshin wrote: >> >> This is how it is working for me: >> >> {ok, Addr} = inet_parse:address(Host), >> Common = [binary,{reuseaddr,true},{recbuf,2*1024*1024},inet,{ip,Addr},{read_packets,100}], >> Options = case is_multicast(Addr) of >> true -> >> Multicast = [{multicast_ttl,4},{multicast_loop,true},{add_membership,{Addr,GwIP}}], >> Common ++ Multicast; >> false -> >> Common >> end, >> > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter.james.morgan@REDACTED Tue Apr 12 21:19:41 2016 From: peter.james.morgan@REDACTED (Peter Morgan) Date: Tue, 12 Apr 2016 20:19:41 +0100 Subject: [erlang-questions] udp multicast: problem with OSX? [solved] In-Reply-To: References: <29B26EB0-01EA-4DA7-8C42-E14E1A2BABBD@gmail.com> <84F4B3EF-9CC1-4E29-9580-0E014287656E@gmail.com> Message-ID: <169B72BD-9B26-4996-B2AF-6308ACCA824A@gmail.com> Thanks! So does Linux do SO_REUSEPORT by default? > On 12 Apr 2016, at 20:00, steve@REDACTED wrote: > > It?s setting the SO_REUSEPORT option - https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man2/setsockopt.2.html > > 16#ffff == SOL_SOCKET > 16#0200 == SO_REUSEPORT > 1:32 == true > > Some -defines would probably have made it more readable :) > -- > > Sent with Airmail > >> On 12 April 2016 at 19:54:27, Peter Morgan (peter.james.morgan@REDACTED) wrote: >> >> Hi Steve - >> >> Whoa! What magic is that doing? >> >> Thanks >> Peter >> >> On 12 Apr 2016, at 19:43, steve@REDACTED wrote: >> >>> Hi Peter, >>> >>> On OSX, try adding: >>> >>> { raw, 16#ffff, 16#0200, <<1:32/native>> } >>> >>> into the options list on the gen_udp open? >>> >>> Cheers, >>> >>> Steve >>> >>> -- >>> >>> Sent with Airmail >>> >>>> On 12 April 2016 at 19:41:00, Peter Morgan (peter.james.morgan@REDACTED) wrote: >>>> >>>> Hello - >>>> >>>> It appears that OSX doesn?t like to reuse a port on gen_udp:open/2 when sending, whereas Linux appears to be quite happy doing so: >>>> >>>> I had for both sender and receiver: >>>> >>>> gen_udp:open(5353, [{mode, binary}, >>>> {reuseaddr, true}, >>>> {ip, {224, 0, 0, 251}}, >>>> {multicast_ttl, 4}, >>>> {multicast_loop, true}, >>>> {broadcast, true}, >>>> {add_membership, {{224, 0, 0, 251}, {0, 0, 0, 0}}}, >>>> {active, once}]). >>>> >>>> Where 5353 is the port for mDNS - already being used by mDNS daemons on both Linux and OSX. The above works fine for Linux (to send and receive), whereas on OSX it only appears to work for receiving. >>>> >>>> To make it work on OSX, I need to call gen_udp:open/2 slightly differently depending on whether I want to call gen_udp:send/4 on the returned socket: >>>> >>>> {ok, Sender} = gen_udp:open(0, [binary]). >>>> gen_udp:send(Sender, {224, 0, 0, 251}, 5353, <>). >>>> >>>> Note the ?0? as the Port parameter to gen_udp:open/2 above. >>>> >>>> Whereas the receiver must be (as originally): >>>> >>>> {ok, Sender} = gen_udp:open(5353, [{mode, binary}, >>>> {reuseaddr, true}, >>>> {ip, {224, 0, 0, 251}}, >>>> {multicast_ttl, 4}, >>>> {multicast_loop, true}, >>>> {broadcast, true}, >>>> {add_membership, {{224, 0, 0, 251}, {0, 0, 0, 0}}}, >>>> {active, once}]). >>>> >>>> On Linux, it seems quite happy for the sender and receiver to use the {reuseaddr, true} form immediately above and port 5353 as the parameter to gen_udp:open/2. OSX is OK until gen_udp:send/4 is called it responds with: {error, eaddrnotavail}. >>>> >>>> The updated code is https://github.com/shortishly/mdns/blob/develop/src/mdns_udp.erl and now works for both OSX and Linux. >>>> >>>> >>>> Thanks, >>>> Peter. >>>> >>>> >>>> >>>> >>>> >>>> >>>> > On 12 Apr 2016, at 17:06, Peter Morgan wrote: >>>> > >>>> > Hi Torben, Max, >>>> > >>>> > On OSX I can receive multicast traffic using the following options (the same as Max?s pretty much): >>>> > >>>> > [binary, >>>> > inet, >>>> > {reuseaddr, true}, >>>> > {ip, MulticastAddress}, >>>> > {multicast_ttl, 4}, >>>> > {multicast_loop, true}, >>>> > {add_membership, {MulticastAddress, {0, 0, 0, 0}}}, >>>> > {active, once}]. >>>> > >>>> > Where MulticastAddress is {224, 0, 0, 251}. I?m using that address because it is one used by OSX, Linux/everything for multicast DNS :) >>>> > >>>> > On OSX I can happily receive multicast traffic on a gen_udp socket that is opened with the above parameters. >>>> > >>>> > However, on OSX I cannot send to the same socket using: >>>> > >>>> > gen_udp:send(Socket, MulticastAddress, Port, SomeMessage). >>>> > >>>> > On Linux (Fedora 23) the above send does work. On OSX I get {error, eaddrnotavail} on send. >>>> > >>>> > I?m sure I have had this working on OSX and Linux together, so I?m probably doing some rather stupid? >>>> > >>>> > OSX is El Capitan (10.11.4 beta). Erlang is 18.3 (brew). >>>> > >>>> > I?ve tried replacing the {0, 0, 0, 0} in {add_membership, {MulticastAddress, {0, 0, 0, 0}}} with IP address of the interface, but it didn?t make any difference. >>>> > >>>> > Thanks, >>>> > Peter. >>>> > >>>> > >>>> > >>>> > >>>> >> On 11 Apr 2016, at 16:58, Max Lapshin wrote: >>>> >> >>>> >> This is how it is working for me: >>>> >> >>>> >> {ok, Addr} = inet_parse:address(Host), >>>> >> Common = [binary,{reuseaddr,true},{recbuf,2*1024*1024},inet,{ip,Addr},{read_packets,100}], >>>> >> Options = case is_multicast(Addr) of >>>> >> true -> >>>> >> Multicast = [{multicast_ttl,4},{multicast_loop,true},{add_membership,{Addr,GwIP}}], >>>> >> Common ++ Multicast; >>>> >> false -> >>>> >> Common >>>> >> end, >>>> >> >>>> > >>>> >>>> _______________________________________________ >>>> erlang-questions mailing list >>>> erlang-questions@REDACTED >>>> http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve@REDACTED Tue Apr 12 21:25:43 2016 From: steve@REDACTED (steve@REDACTED) Date: Tue, 12 Apr 2016 20:25:43 +0100 Subject: [erlang-questions] udp multicast: problem with OSX? [solved] In-Reply-To: <169B72BD-9B26-4996-B2AF-6308ACCA824A@gmail.com> References: <29B26EB0-01EA-4DA7-8C42-E14E1A2BABBD@gmail.com> <84F4B3EF-9CC1-4E29-9580-0E014287656E@gmail.com> <169B72BD-9B26-4996-B2AF-6308ACCA824A@gmail.com> Message-ID: I believe it?s a BSD thing rather than Linux - but we?ve had that in our low level UDP code for such a long time that tbh I?ve forgotten the details! --? Sent with Airmail On 12 April 2016 at 20:19:43, Peter Morgan (peter.james.morgan@REDACTED) wrote: Thanks! So does Linux do SO_REUSEPORT by default? On 12 Apr 2016, at 20:00, steve@REDACTED wrote: It?s setting the SO_REUSEPORT option -?https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man2/setsockopt.2.html 16#ffff == SOL_SOCKET 16#0200 == SO_REUSEPORT 1:32 == true? Some -defines would probably have made it more readable :) --? Sent with Airmail On 12 April 2016 at 19:54:27, Peter Morgan (peter.james.morgan@REDACTED) wrote: Hi Steve - Whoa! What magic is that doing? Thanks Peter On 12 Apr 2016, at 19:43, steve@REDACTED wrote: Hi Peter, On OSX, try adding: { raw, 16#ffff, 16#0200, <<1:32/native>> } into the options list on the gen_udp open? Cheers, Steve --? Sent with Airmail On 12 April 2016 at 19:41:00, Peter Morgan (peter.james.morgan@REDACTED) wrote: Hello - It appears that OSX doesn?t like to reuse a port on gen_udp:open/2 when sending, whereas Linux appears to be quite happy doing so: I had for both sender and receiver: gen_udp:open(5353, [{mode, binary}, {reuseaddr, true}, {ip, {224, 0, 0, 251}}, {multicast_ttl, 4}, {multicast_loop, true}, {broadcast, true}, {add_membership, {{224, 0, 0, 251}, {0, 0, 0, 0}}}, {active, once}]). Where 5353 is the port for mDNS - already being used by mDNS daemons on both Linux and OSX. The above works fine for Linux (to send and receive), whereas on OSX it only appears to work for receiving. To make it work on OSX, I need to call gen_udp:open/2 slightly differently depending on whether I want to call gen_udp:send/4 on the returned socket: {ok, Sender} = gen_udp:open(0, [binary]). gen_udp:send(Sender, {224, 0, 0, 251}, 5353, <>). Note the ?0? as the Port parameter to gen_udp:open/2 above. Whereas the receiver must be (as originally): {ok, Sender} = gen_udp:open(5353, [{mode, binary}, {reuseaddr, true}, {ip, {224, 0, 0, 251}}, {multicast_ttl, 4}, {multicast_loop, true}, {broadcast, true}, {add_membership, {{224, 0, 0, 251}, {0, 0, 0, 0}}}, {active, once}]). On Linux, it seems quite happy for the sender and receiver to use the {reuseaddr, true} form immediately above and port 5353 as the parameter to gen_udp:open/2. OSX is OK until gen_udp:send/4 is called it responds with: {error, eaddrnotavail}. The updated code is https://github.com/shortishly/mdns/blob/develop/src/mdns_udp.erl and now works for both OSX and Linux. Thanks, Peter. > On 12 Apr 2016, at 17:06, Peter Morgan wrote: > > Hi Torben, Max, > > On OSX I can receive multicast traffic using the following options (the same as Max?s pretty much): > > [binary, > inet, > {reuseaddr, true}, > {ip, MulticastAddress}, > {multicast_ttl, 4}, > {multicast_loop, true}, > {add_membership, {MulticastAddress, {0, 0, 0, 0}}}, > {active, once}]. > > Where MulticastAddress is {224, 0, 0, 251}. I?m using that address because it is one used by OSX, Linux/everything for multicast DNS :) > > On OSX I can happily receive multicast traffic on a gen_udp socket that is opened with the above parameters. > > However, on OSX I cannot send to the same socket using: > > gen_udp:send(Socket, MulticastAddress, Port, SomeMessage). > > On Linux (Fedora 23) the above send does work. On OSX I get {error, eaddrnotavail} on send. > > I?m sure I have had this working on OSX and Linux together, so I?m probably doing some rather stupid? > > OSX is El Capitan (10.11.4 beta). Erlang is 18.3 (brew). > > I?ve tried replacing the {0, 0, 0, 0} in {add_membership, {MulticastAddress, {0, 0, 0, 0}}} with IP address of the interface, but it didn?t make any difference. > > Thanks, > Peter. > > > > >> On 11 Apr 2016, at 16:58, Max Lapshin wrote: >> >> This is how it is working for me: >> >> {ok, Addr} = inet_parse:address(Host), >> Common = [binary,{reuseaddr,true},{recbuf,2*1024*1024},inet,{ip,Addr},{read_packets,100}], >> Options = case is_multicast(Addr) of >> true -> >> Multicast = [{multicast_ttl,4},{multicast_loop,true},{add_membership,{Addr,GwIP}}], >> Common ++ Multicast; >> false -> >> Common >> end, >> > _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From srinivas.talluru@REDACTED Wed Apr 13 08:16:08 2016 From: srinivas.talluru@REDACTED (srinivas talluru) Date: Wed, 13 Apr 2016 11:46:08 +0530 Subject: [erlang-questions] Tsung1.6 installation is failing against erlang R14B Message-ID: As per Tsung doc, We are installing R14B(erlang), and then installing Tsung 1.6 R14B-erlang [configure/compilation and installation was successful] tsung1.6- compilation successful, But while installing we see below error. --------------------------------------------------------------------------------------------------------------- find: cycle detected for /usr/lib/locale/en_US.UTF-8/LC_CTYPE/32/ install: ts_websocket.hrl was not found anywhere! gmake: *** [install_recorder] Error 2 --------------------------------------------------------------------------------------------------------------- Did any once faced similar kind of issue? Note: Because with Latest Erlang version (OTP18) we are facing compilation issues with erlang in Solaris x86 Thanks, Srinivas -------------- next part -------------- An HTML attachment was scrubbed... URL: From srinivas.talluru@REDACTED Wed Apr 13 08:13:32 2016 From: srinivas.talluru@REDACTED (srinivas talluru) Date: Wed, 13 Apr 2016 11:43:32 +0530 Subject: [erlang-questions] erlang[OTP 18.0] compilation failing in Solaris x86 Message-ID: Hi, We are trying to compile erlang (OTP18.0) in Solaris x86 [tried on both S10 and S11] erlang configure - success used below switch, [./configure --enable-m32-build --disable-hipe] compilation is failing with errors. ----------------------------------------------------------------------------------------------------------- bash-3.2# gmake .. . . CC obj/i386-pc-solaris2.11/opt/smp/erl_bif_guard.o gmake[3]: *** No rule to make target `i386-pc-solaris2.11/opt/smp/erl_compile_flags.h', needed by `obj/i386-pc-solaris2.11/opt/smp/erl_bif_info.o'. Stop. gmake[3]: Leaving directory `/space2/tsung1.6/otp_src_18.0/erts/emulator' gmake[2]: *** [opt] Error 2 gmake[2]: Leaving directory `/space2/tsung1.6/otp_src_18.0/erts/emulator' gmake[1]: *** [smp] Error 2 gmake[1]: Leaving directory `/space2/tsung1.6/otp_src_18.0/erts' gmake: *** [emulator] Error 2 -------------------------------------------------------------------------------------------------------------- Did any once faced the same issue? Thanks, Srinivas -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Wed Apr 13 13:09:53 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 13 Apr 2016 13:09:53 +0200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: Message-ID: On Fri, Apr 8, 2016 at 8:31 PM, Bj?rn-Egil Dahlberg < wallentin.dahlberg@REDACTED> wrote: > Have you repeated some code or built your own private lib to handle > certain maps specific tasks and thought "why isn't this in the maps module?" One function I like to have is "apply update via function", and I implement it somewhat often: maps_apply(F, K, Map) -> V = maps:get(K, Map), maps:put(K, F(V), Map). But it can be made much faster in a direct implementation since I don't have to first pick up the value: I can apply F when I sit with the value at hand and thus avoid the "put" lookup path. It might, however, be nasty to implement because F is in Erlang-context, whereas the maps operations are in BIF-context. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn-egil.xb.dahlberg@REDACTED Wed Apr 13 15:40:57 2016 From: bjorn-egil.xb.dahlberg@REDACTED (=?UTF-8?Q?Bj=c3=b6rn-Egil_Dahlberg_XB?=) Date: Wed, 13 Apr 2016 15:40:57 +0200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: Message-ID: <570E4C69.90004@ericsson.com> There is currently no (good) way to get out of C-context to execute some erlang snippet and then get back into c-context. Traps does this but only in a controlled manner. I think it is doable though. Your proposal of maps_apply/3 is the maps equivalent of dict:update/3 and sadly that name is already occupied in maps (it's the equivalent of gb_trees). I think we should add it to the API without thinking about the performance too much. Performance can always be improved later on. Is maps:apply/3 the best name we can think of? I don't really trust you and naming things. =) update/3,4 would probably have been best but c'est la vie. // Bj?rn-Egil On 2016-04-13 13:09, Jesper Louis Andersen wrote: > > On Fri, Apr 8, 2016 at 8:31 PM, Bj?rn-Egil Dahlberg > > > wrote: > > Have you repeated some code or built your own private lib to > handle certain maps specific tasks and thought "why isn't this in > the maps module?" > > > One function I like to have is "apply update via function", and I > implement it somewhat often: > > maps_apply(F, K, Map) -> > V = maps:get(K, Map), > maps:put(K, F(V), Map). > > But it can be made much faster in a direct implementation since I > don't have to first pick up the value: I can apply F when I sit with > the value at hand and thus avoid the "put" lookup path. It might, > however, be nasty to implement because F is in Erlang-context, whereas > the maps operations are in BIF-context. > > > -- > J. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierrefenoll@REDACTED Wed Apr 13 15:50:29 2016 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Wed, 13 Apr 2016 15:50:29 +0200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <570E4C69.90004@ericsson.com> References: <570E4C69.90004@ericsson.com> Message-ID: Maps iterator, maps_apply? How about support for map comprehensions (generator and all)? I remember there was already syntax for it in erl_parse.yrl along the lines of: #{ K => V || K := V <- Map} Cheers, -- Pierre Fenoll On 13 April 2016 at 15:40, Bj?rn-Egil Dahlberg XB < bjorn-egil.xb.dahlberg@REDACTED> wrote: > There is currently no (good) way to get out of C-context to execute some > erlang snippet and then get back into c-context. Traps does this but only > in a controlled manner. I think it is doable though. > > Your proposal of maps_apply/3 is the maps equivalent of dict:update/3 and > sadly that name is already occupied in maps (it's the equivalent of > gb_trees). > > I think we should add it to the API without thinking about the performance > too much. Performance can always be improved later on. > > Is maps:apply/3 the best name we can think of? I don't really trust you > and naming things. =) update/3,4 would probably have been best but c'est la > vie. > > // Bj?rn-Egil > > > On 2016-04-13 13:09, Jesper Louis Andersen wrote: > > > On Fri, Apr 8, 2016 at 8:31 PM, Bj?rn-Egil Dahlberg < > wallentin.dahlberg@REDACTED> wrote: > >> Have you repeated some code or built your own private lib to handle >> certain maps specific tasks and thought "why isn't this in the maps module?" > > > One function I like to have is "apply update via function", and I > implement it somewhat often: > > maps_apply(F, K, Map) -> > V = maps:get(K, Map), > maps:put(K, F(V), Map). > > But it can be made much faster in a direct implementation since I don't > have to first pick up the value: I can apply F when I sit with the value at > hand and thus avoid the "put" lookup path. It might, however, be nasty to > implement because F is in Erlang-context, whereas the maps operations are > in BIF-context. > > > -- > J. > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sdl.web@REDACTED Wed Apr 13 16:28:55 2016 From: sdl.web@REDACTED (Leo Liu) Date: Wed, 13 Apr 2016 22:28:55 +0800 Subject: [erlang-questions] Wanted additions to the maps module? References: <570E4C69.90004@ericsson.com> Message-ID: On 2016-04-13 15:40 +0200, Bj?rn-Egil Dahlberg XB wrote: > Is maps:apply/3 the best name we can think of? For consistency how about a name containing `update', for example, update_with/3, update_by/3? Leo From serge@REDACTED Wed Apr 13 17:43:07 2016 From: serge@REDACTED (Serge Aleynikov) Date: Wed, 13 Apr 2016 11:43:07 -0400 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <570E4C69.90004@ericsson.com> References: <570E4C69.90004@ericsson.com> Message-ID: I'd like to note that if such a function is added, there should be also a variant that doesn't throw if the key is not present in the map. It's a pretty common use case to "apply update" or "insert", e.g.: maps_apply(Key, Fun, Default, Map) -> case maps:find(Key, Map) of {ok, Value} -> maps:put(Key, Fun(Value), Map); _ -> maps:put(Key, Default, Map) end. Also possibly have functions similar to the ones in ets dealing with increments of counters whose values are maintained in the map: increment(Key, Inc, Default, Map) -> case maps:find(Key, Map) of {ok, Value} -> maps:put(Key, Value+Inc, Map); error -> maps:put(Key, Default, Map) end. increment(Key, Pos, Inc, InitValueFun, Map) when is_function(InitValueFun, 1) -> case maps:find(Key, Map) of {ok, Value} when is_tuple(Value), tuple_size(Value) >= Pos -> ok; error -> Value = InitValueFun(Key) end, V = setelement(Pos, Value, element(Pos, Value)+Inc), maps:put(Key, V, Map). On Wed, Apr 13, 2016 at 9:40 AM, Bj?rn-Egil Dahlberg XB < bjorn-egil.xb.dahlberg@REDACTED> wrote: > There is currently no (good) way to get out of C-context to execute some > erlang snippet and then get back into c-context. Traps does this but only > in a controlled manner. I think it is doable though. > > Your proposal of maps_apply/3 is the maps equivalent of dict:update/3 and > sadly that name is already occupied in maps (it's the equivalent of > gb_trees). > > I think we should add it to the API without thinking about the performance > too much. Performance can always be improved later on. > > Is maps:apply/3 the best name we can think of? I don't really trust you > and naming things. =) update/3,4 would probably have been best but c'est la > vie. > > // Bj?rn-Egil > > On 2016-04-13 13:09, Jesper Louis Andersen wrote: > > > On Fri, Apr 8, 2016 at 8:31 PM, Bj?rn-Egil Dahlberg < > wallentin.dahlberg@REDACTED> wrote: > >> Have you repeated some code or built your own private lib to handle >> certain maps specific tasks and thought "why isn't this in the maps module?" > > > One function I like to have is "apply update via function", and I > implement it somewhat often: > > maps_apply(F, K, Map) -> > V = maps:get(K, Map), > maps:put(K, F(V), Map). > > But it can be made much faster in a direct implementation since I don't > have to first pick up the value: I can apply F when I sit with the value at > hand and thus avoid the "put" lookup path. It might, however, be nasty to > implement because F is in Erlang-context, whereas the maps operations are > in BIF-context. > > > -- > J. > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lloyd@REDACTED Thu Apr 14 00:50:11 2016 From: lloyd@REDACTED (lloyd@REDACTED) Date: Wed, 13 Apr 2016 18:50:11 -0400 (EDT) Subject: [erlang-questions] =?utf-8?q?While_we=27re_talking_about_adding_f?= =?utf-8?q?unctions=2E=2E=2E_how_about_file=3Aunconsult/1_or_some_such=3F?= Message-ID: <1460587811.848827746@apps.rackspace.com> Hello, Nice discussion at the moment on this list re: adding map functions. In a parallel vein, I'd like to strongly second zxq9's request for an inverse function for file:consult/1. https://zxq9.com/archives/1021 I've run up against the absence of such a function several times and, each time, reached into Joe Armstrong's book for a solution. As you'd expect, by the time I find the right page and copy out the code, my mind is well off topic at hand; takes minutes to get back into the swing--- unless I happen to get distracted by Hacker News or some other time waster, in which case it might be hours before I remember what I needed the function for in the first place. Chalk it up to distractibility of the aging brain. Ran into the problem again moments ago and thought I'd checked into what others on the list might have had to say about the issue--- which brought me to zxq9's eloquent request. So, can say, "Pretty please?" Many thanks, LRP ********************************************* My books: THE GOSPEL OF ASHES http://thegospelofashes.com Strength is not enough. Do they have the courage and the cunning? Can they survive long enough to save the lives of millions? FREEIN' PANCHO http://freeinpancho.com A community of misfits help a troubled boy find his way AYA TAKEO http://ayatakeo.com Star-crossed love, war and power in an alternative universe Available through Amazon or by request from your favorite bookstore ********************************************** From dcy.dengcaiyun@REDACTED Thu Apr 14 15:53:32 2016 From: dcy.dengcaiyun@REDACTED (Caiyun Deng) Date: Thu, 14 Apr 2016 21:53:32 +0800 Subject: [erlang-questions] Is ejabberd/MongooseIM not a good choise for mobile IM? Message-ID: Hi! I want to develop a mobile IM app, i searched something about. Because xmpp is heavyweight, is ejabberd/MongooseIM not a good choise for mobile IM? If you develop a mobile IM today, which tool will you choose? -------------- next part -------------- An HTML attachment was scrubbed... URL: From goran.bage@REDACTED Thu Apr 14 16:54:56 2016 From: goran.bage@REDACTED (=?UTF-8?B?R8O2cmFuIELDpWdl?=) Date: Thu, 14 Apr 2016 16:54:56 +0200 Subject: [erlang-questions] Description of Erlang Garbage Collection In-Reply-To: <623A7446-43ED-4B40-B21C-51B16EB27F9C@rogvall.se> References: <623A7446-43ED-4B40-B21C-51B16EB27F9C@rogvall.se> Message-ID: <570FAF40.5000707@mobilearts.com> I agree, very good description. --G?ran Tony Rogvall wrote: > Fantastic!!! > > /Tony > >> On 8 apr 2016, at 11:29, Lukas Larsson > wrote: >> >> Hello everyone, >> >> I've long worked on a text that describes in detail exactly how the Erlang garbage collector works. During the last month I finally took the time to finish it and also included the changes that have >> been made in the current master branch to be released in Erlang/OTP 19.0. >> >> You can read the description here: https://www.erlang-solutions.com/blog/erlang-19-0-garbage-collector.html >> >> I'm personally am very excited about the off_heap message queue semantics that is coming in 19 and think that it will solve a lot of problems that systems have had with huge message queues running >> amok and also how refc binaries not been garbage collected fast enough. Also the way that literals are identified from 19 opens up a bunch of new optimization paths for us as it allows a much more >> fuzzy definition of what is the young heap. >> >> If you have any questions about anything, feel free to ask and I'll do my best to answere. >> >> Enjoy, >> Lukas >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -- -- Goran ------------------------- May the Snow be with you ---- Goran Bage, Mobile Arts, www.mobilearts.com Tjarhovsgatan 56, SE-116 28 Stockholm, Sweden email:goran.bage@REDACTED, phone: +46 733 358405 From fnanninga@REDACTED Thu Apr 14 18:00:55 2016 From: fnanninga@REDACTED (Feiko Nanninga) Date: Thu, 14 Apr 2016 18:00:55 +0200 Subject: [erlang-questions] Disable Distribution Message-ID: <570FBEB7.8060207@fnanninga.de> Hello, I'd like to deploy a non-distributed application with a sane security configuration (preferably using a relx release). How can I entirely disable other nodes from connecting? Is there an option to pass to erl (to add in vm.args)?. It seems using a release requires me give the node a name and set a cookie. Now I can hope nobody guesses the cookie or I can keep other users on the same system from reading files which contain the cookie; but this is not a clean solution. Would not setting -sname or -name achieve this goal? Best regards, Feiko PS: If you don't provide vm.args yourself, relx generates one for you with a predictable cookie. This is a BAD default. From bombadil@REDACTED Thu Apr 14 18:27:24 2016 From: bombadil@REDACTED (Manuel A. Rubio "Bombadil") Date: Thu, 14 Apr 2016 18:27:24 +0200 Subject: [erlang-questions] Is ejabberd/MongooseIM not a good choise for mobile IM? In-Reply-To: References: Message-ID: <7242389aae247ac0a5f9ff9e2b86c966@bosqueviejo.net> Hi Caiyun, El 2016-04-14 15:53, Caiyun Deng escribi?: > Hi! I want to develop a mobile IM app, i searched something about. Because xmpp is heavyweight, is ejabberd/MongooseIM not a good choise for mobile IM? > If you develop a mobile IM today, which tool will you choose? Well, XMPP was not created for mobile, that's right, the last improvements in MongooseIM and ejabberd let you use XMPP for mobile, but you maybe will need to do customizations on that. Another approach a lot of people is using now is MQTT instead of XMPP. It's more flexible because the specifications is shorter and it doesn't define how to do the messaging between users and other things. You have to do it for yourself. In this approach you can use VerneMQ or RabbitMQ (with the MQTT module). Anyway there are not a turn-key solution, there are only SaaS you can hire. Regards. Manuel Rubio. -------------- next part -------------- An HTML attachment was scrubbed... URL: From fnanninga@REDACTED Thu Apr 14 18:29:38 2016 From: fnanninga@REDACTED (Feiko Nanninga) Date: Thu, 14 Apr 2016 18:29:38 +0200 Subject: [erlang-questions] Disable Distribution In-Reply-To: <570FBEB7.8060207@fnanninga.de> References: <570FBEB7.8060207@fnanninga.de> Message-ID: <570FC572.5080908@fnanninga.de> As it turns out, I can leave -sname/-name and -setcookie out in vm.args as long as I do not use relx's extended starting script. But will that be enough to keep other nodes from connecting? On 14.04.2016 18:00, Feiko Nanninga wrote: > Hello, > > I'd like to deploy a non-distributed application with a sane > security configuration (preferably using a relx release). > > How can I entirely disable other nodes from connecting? Is there an > option to pass to erl (to add in vm.args)?. It seems using a release > requires me give the node a name and set a cookie. Now I can hope nobody > guesses the cookie or I can keep other users on the same system from > reading files which contain the cookie; but this is not a clean solution. > > Would not setting -sname or -name achieve this goal? > > Best regards, > Feiko > > > PS: If you don't provide vm.args yourself, relx generates one for you > with a predictable cookie. This is a BAD default. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From magnus@REDACTED Thu Apr 14 19:03:13 2016 From: magnus@REDACTED (Magnus Henoch) Date: Thu, 14 Apr 2016 18:03:13 +0100 Subject: [erlang-questions] Disable Distribution In-Reply-To: <570FC572.5080908@fnanninga.de> References: <570FBEB7.8060207@fnanninga.de> <570FC572.5080908@fnanninga.de> Message-ID: Yes, leaving out the -sname/-name option is enough. You can verify this by looking for open ports for "beam" processes with netstat: $ netstat -ltpn | grep beam With -sname, it shows one open port: tcp 0 0 0.0.0.0:36551 0.0.0.0:* LISTEN 29262/beam.smp Without -sname, it shows nothing. Regards, Magnus On Thu, Apr 14, 2016 at 5:29 PM, Feiko Nanninga wrote: > As it turns out, I can leave -sname/-name and -setcookie out in vm.args > as long as I do not use relx's extended starting script. > > But will that be enough to keep other nodes from connecting? > > > On 14.04.2016 18:00, Feiko Nanninga wrote: > > Hello, > > > > I'd like to deploy a non-distributed application with a sane > > security configuration (preferably using a relx release). > > > > How can I entirely disable other nodes from connecting? Is there an > > option to pass to erl (to add in vm.args)?. It seems using a release > > requires me give the node a name and set a cookie. Now I can hope nobody > > guesses the cookie or I can keep other users on the same system from > > reading files which contain the cookie; but this is not a clean solution. > > > > Would not setting -sname or -name achieve this goal? > > > > Best regards, > > Feiko > > > > > > PS: If you don't provide vm.args yourself, relx generates one for you > > with a predictable cookie. This is a BAD default. > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Thu Apr 14 19:10:28 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Thu, 14 Apr 2016 19:10:28 +0200 Subject: [erlang-questions] Disable Distribution In-Reply-To: <570FBEB7.8060207@fnanninga.de> References: <570FBEB7.8060207@fnanninga.de> Message-ID: Hi, You can set a random cookie at start in your code so that it is unlikely that another node can connect. Even if cookie file is readable by someone. "Envoy? depuis mon mobile " Eric ---- Feiko Nanninga a ?crit ---- >Hello, > >I'd like to deploy a non-distributed application with a sane >security configuration (preferably using a relx release). > >How can I entirely disable other nodes from connecting? Is there an >option to pass to erl (to add in vm.args)?. It seems using a release >requires me give the node a name and set a cookie. Now I can hope nobody >guesses the cookie or I can keep other users on the same system from >reading files which contain the cookie; but this is not a clean solution. > >Would not setting -sname or -name achieve this goal? > >Best regards, >Feiko > > >PS: If you don't provide vm.args yourself, relx generates one for you >with a predictable cookie. This is a BAD default. >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From Oliver.Korpilla@REDACTED Fri Apr 15 00:23:36 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Fri, 15 Apr 2016 00:23:36 +0200 Subject: [erlang-questions] gen_server etc: blocking init, blocking cast Message-ID: Hello. I was trying to write a gen_server for handling a TCP connection. So, there would be an instance of gen_server for each open connection. (Each instance would have a copy of the socket returned by gen_tcp:listen to start accepting a connection from.) One thing I noticed is that gen_tcp does not seem to play very well with OTP's assumptions since accept is a blocking call, at least I understood it (and the principles) that way. So, on the net I saw examples dealing with accept in init/1. So, this will delay init/1 from finishing until there is either a timeout or there is a connection. I read somewhere else that it is a good practice to keep init from blocking... I saw an example (I think it was in "Learn You Some Erlang Good") that delayed the accept call by calling gen_server:cast and sending your own mailbox a trigger that was immediately followed by the blocking accept call. While this certainly was keeping init short, I wonder what was exactly gained here? Both init and handle_cast get executed in the new process started by gen_server:start_link. If either of them blocks, the process cannot react to anything else, right? So, how would a cast blocking the server be better than init? The process is still not responsive to events. I might be missing something here, but this seems to be the case with all blocking calls - they do not seem to play so well with OTP principles. I mean, gen_tcp could actually also allow the socket returned by gen_tcp:listen to act like an active socket and generate {tcp_accept, ...} or something. Did I miss something here? Or is there an entirely different pattern for writing workers with blocking calls? Thanks for reading, Oliver From ok@REDACTED Fri Apr 15 08:27:17 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 15 Apr 2016 18:27:17 +1200 Subject: [erlang-questions] Disable Distribution In-Reply-To: References: <570FBEB7.8060207@fnanninga.de> <570FC572.5080908@fnanninga.de> Message-ID: <4c0f6e9b-9569-86c6-46b8-6fc487eed2b1@cs.otago.ac.nz> On 15/04/16 5:03 am, Magnus Henoch wrote: > Yes, leaving out the -sname/-name option is enough. You can verify > this by looking for open ports for "beam" processes with netstat: > > $ netstat -ltpn | grep beam > > With -sname, it shows one open port: > > tcp 0 0 0.0.0.0:36551 > 0.0.0.0:* LISTEN 29262/beam.smp For what it's worth, MacOS X 10.9 doesn't accept those command line options. % netstat -a shows these tcp4 0 0 localhost.epmd localhost.61991 ESTABLISHED tcp4 0 0 localhost.61991 localhost.epmd ESTABLISHED tcp4 0 0 *.61990 *.* LISTEN tcp4 0 0 *.epmd *.* LISTEN when running with -sname; no lines contain 'beam'. I would like to know what the right command line options are for Mac OS X, if this is wrong. From bchesneau@REDACTED Fri Apr 15 09:12:55 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Fri, 15 Apr 2016 07:12:55 +0000 Subject: [erlang-questions] Proposal: add lists:intersperse/2 and lists:intercalate/2 In-Reply-To: <570B9FA7.2050104@khandkar.net> References: <570B9FA7.2050104@khandkar.net> Message-ID: this would be. an interresting addition... On Mon, 11 Apr 2016 at 14:59, Siraaj Khandkar wrote: > I think my personal reservation about the name "intersperse" is > addressed by the possibility of later extending to include interval > specification, something like this: > > intersperse(A, [A], Interval) -> [A] > when Interval :: random | {regular, pos_integer()} | ... . > > where the default is thought of as {regular, 1}. > > > P.S. I'm crossing my fingers, hoping Robert will not like "join"! :-) > > > On 4/11/16 8:06 AM, Jesper Louis Andersen wrote: > > Reviving this. There is now a PR for the proposal: > > > > https://github.com/erlang/otp/pull/1012 > > > > The name is hard. I'm on the verge of asking Robert Virding what it > should > > be and then use him as the BDFL. I'm not going to ask Joe, since then we > > would have disagreement between him and Robert :) > > > > > > > > On Wed, Mar 2, 2016 at 3:47 PM, Jesper Louis Andersen < > > jesper.louis.andersen@REDACTED> wrote: > > > >> Hi Erlangers, > >> > >> I'd really like to add two functions to the lists module from Haskell: > >> > >> intersperse(List, Seperator) produces a list where each element is > >> separated by separator, i.e. > >> > >> X = [1,2,3] > >> [1, x, 2, x, 3] = lists:intersperse(X, x), > >> > >> and it's cousin, intercalate(ListOfLists, Separator) is > >> append(intersperse(ListOfLists, Seperator)), i.e, > >> > >> Y = ["a", "b", "c"] > >> "a, b, c" = lists:intercalate(Y, ", "), > >> > >> The implementations are straightforward and easy to write tests for, > even > >> property based tests if needed. > >> > >> The rationale for this proposal is that I find myself implementing this > >> function again and again in every project I write, and it is highly > >> generic. It belongs in a typical list module. OCaml libraries add it. > >> Haskell's Data.List has it. I believe Erlang, being a practical > language, > >> should have it as well. > >> > >> Thoughts? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marco.molteni@REDACTED Fri Apr 15 10:49:25 2016 From: marco.molteni@REDACTED (Marco Molteni) Date: Fri, 15 Apr 2016 10:49:25 +0200 Subject: [erlang-questions] Disable Distribution In-Reply-To: <4c0f6e9b-9569-86c6-46b8-6fc487eed2b1@cs.otago.ac.nz> References: <570FBEB7.8060207@fnanninga.de> <570FC572.5080908@fnanninga.de> <4c0f6e9b-9569-86c6-46b8-6fc487eed2b1@cs.otago.ac.nz> Message-ID: <8C6EB14D-70FB-470A-AF3C-6EC98E728338@laposte.net> On 15 Apr 2016, at 08:27, Richard A. O'Keefe wrote: > On 15/04/16 5:03 am, Magnus Henoch wrote: >> Yes, leaving out the -sname/-name option is enough. You can verify this by looking for open ports for "beam" processes with netstat: >> >> $ netstat -ltpn | grep beam >> >> With -sname, it shows one open port: >> >> tcp 0 0 0.0.0.0:36551 0.0.0.0:* LISTEN 29262/beam.smp > > For what it's worth, MacOS X 10.9 doesn't accept those command line options. > % netstat -a > shows these > tcp4 0 0 localhost.epmd localhost.61991 ESTABLISHED > tcp4 0 0 localhost.61991 localhost.epmd ESTABLISHED > tcp4 0 0 *.61990 *.* LISTEN > tcp4 0 0 *.epmd *.* LISTEN > when running with -sname; no lines contain 'beam'. > > I would like to know what the right command line options are for Mac OS X, > if this is wrong. You can use `lsof`, available on Linux, *BSD and Mac: terminal 1: $ erl -sname pizza terminal 2: $ lsof | grep TCP | grep beam beam.smp 5732 mmolteni 24u IPv4 0xa52d786c22d20475 0t0 TCP *:51822 (LISTEN) beam.smp 5732 mmolteni 25u IPv4 0xa52d786c27d8519d 0t0 TCP localhost:51823->localhost:epmd (ESTABLISHED) From spawn.think@REDACTED Fri Apr 15 11:02:58 2016 From: spawn.think@REDACTED (Ahmed Omar) Date: Fri, 15 Apr 2016 11:02:58 +0200 Subject: [erlang-questions] Disable Distribution In-Reply-To: <4c0f6e9b-9569-86c6-46b8-6fc487eed2b1@cs.otago.ac.nz> References: <570FBEB7.8060207@fnanninga.de> <570FC572.5080908@fnanninga.de> <4c0f6e9b-9569-86c6-46b8-6fc487eed2b1@cs.otago.ac.nz> Message-ID: On Mac OS X netstat doesn't support the same options. Instead you could use lsof -n -iTCP | grep LISTEN | grep beam Best Regards, - Ahmed Omar http://about.me/spawn.think/ On Fri, Apr 15, 2016 at 8:27 AM, Richard A. O'Keefe wrote: > > > On 15/04/16 5:03 am, Magnus Henoch wrote: > >> Yes, leaving out the -sname/-name option is enough. You can verify this >> by looking for open ports for "beam" processes with netstat: >> >> $ netstat -ltpn | grep beam >> >> With -sname, it shows one open port: >> >> tcp 0 0 0.0.0.0:36551 0.0.0.0:* >> LISTEN 29262/beam.smp >> > > For what it's worth, MacOS X 10.9 doesn't accept those command line > options. > % netstat -a > shows these > tcp4 0 0 localhost.epmd localhost.61991 ESTABLISHED > tcp4 0 0 localhost.61991 localhost.epmd ESTABLISHED > tcp4 0 0 *.61990 *.* LISTEN > tcp4 0 0 *.epmd *.* LISTEN > when running with -sname; no lines contain 'beam'. > > I would like to know what the right command line options are for Mac OS X, > if this is wrong. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Fri Apr 15 11:25:35 2016 From: tony@REDACTED (Tony Rogvall) Date: Fri, 15 Apr 2016 11:25:35 +0200 Subject: [erlang-questions] Disable Distribution In-Reply-To: References: <570FBEB7.8060207@fnanninga.de> <570FC572.5080908@fnanninga.de> <4c0f6e9b-9569-86c6-46b8-6fc487eed2b1@cs.otago.ac.nz> Message-ID: <2491EBEB-02A8-442B-A79B-FB1CD2285D9F@rogvall.se> You can use inet:i(), if you trust the biased view of the emulator. This will work on all platforms, even windows. /Tony > On 15 apr 2016, at 11:02, Ahmed Omar wrote: > > On Mac OS X netstat doesn't support the same options. > > Instead you could use > > lsof -n -iTCP | grep LISTEN | grep beam > > Best Regards, > - Ahmed Omar > http://about.me/spawn.think/ > On Fri, Apr 15, 2016 at 8:27 AM, Richard A. O'Keefe > wrote: > > > On 15/04/16 5:03 am, Magnus Henoch wrote: > Yes, leaving out the -sname/-name option is enough. You can verify this by looking for open ports for "beam" processes with netstat: > > $ netstat -ltpn | grep beam > > With -sname, it shows one open port: > > tcp 0 0 0.0.0.0:36551 > 0.0.0.0:* LISTEN 29262/beam.smp > > For what it's worth, MacOS X 10.9 doesn't accept those command line options. > % netstat -a > shows these > tcp4 0 0 localhost.epmd localhost.61991 ESTABLISHED > tcp4 0 0 localhost.61991 localhost.epmd ESTABLISHED > tcp4 0 0 *.61990 *.* LISTEN > tcp4 0 0 *.epmd *.* LISTEN > when running with -sname; no lines contain 'beam'. > > I would like to know what the right command line options are for Mac OS X, > if this is wrong. > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From jelena@REDACTED Fri Apr 15 12:00:08 2016 From: jelena@REDACTED (jelena@REDACTED) Date: Fri, 15 Apr 2016 10:00:08 GMT Subject: [erlang-questions] erlang-questions Digest, Vol 265, Issue 9 Message-ID: <99B23F847F694930A3E721219190A8C4.MAI@server2.totohost.hr> From nmarino@REDACTED Fri Apr 15 17:57:44 2016 From: nmarino@REDACTED (Nick Marino) Date: Fri, 15 Apr 2016 10:57:44 -0500 Subject: [erlang-questions] Command-line application libraries? In-Reply-To: References: <20160405093353.16C83E069F@smtp.hushmail.com> <5703AAD1.9090702@dergraf.org> Message-ID: Hi Roger, not sure if this reply is too late to be helpful, but I was just catching up on mailing list posts and thought I'd weigh in: Clique is definitely designed to be used with Cuttlefish. Cuttlefish is listed as a dependency of Clique's in rebar.config, and Clique has several builtin commands that integrate with Cuttlefish for dealing with app config (specifically, "set", "show", and "describe"). That said, I can't see any reason why you couldn't use Clique without registering any Cuttlefish schemas, and just ignore the Cuttlefish integration features. I don't think there's currently a way to completely disable set/show/describe, so those commands would still be present, and just wouldn't be useful for much. Not the prettiest solution, but maybe not so bad in practice. I think it would be nice if there were a way to more fully decouple Clique from Cuttlefish for cases like yours. It will probably never be a priority for us at Basho since we have no need to use Clique without Cuttlefish, but if someone were to submit a well-written pull request from the community, I'd be happy to consider it! Nick On Tue, Apr 5, 2016 at 7:50 AM, Roger Lipscombe wrote: > Slight threadjack: Clique talks about registering cuttlefish schemas; > we're not using cuttlefish. Any concerns? > > On 5 April 2016 at 13:08, Andre Graf wrote: > > Hi, > > > > We're using https://github.com/basho/clique a lot for writing cli > > administration tools for VerneMQ. > > > > Best, > > Andre > > > > > > On 05.04.2016 13:34, Siraaj Khandkar wrote: > >> On Apr 5, 2016, at 05:33, adrian_lewis@REDACTED > >> wrote: > >> > >>> Hi, > >>> > >>> I wondered if Erlang has any libraries to support building > >>> command-line applications? > >> > >> 1) https://github.com/jcomellas/getopt > >> 2) rebar escriptize > >> 3) profit :) > >> > >> > >> > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > >> > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chandrashekhar.mullaparthi@REDACTED Sat Apr 16 07:25:19 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Sat, 16 Apr 2016 06:25:19 +0100 Subject: [erlang-questions] gen_server etc: blocking init, blocking cast In-Reply-To: References: Message-ID: Hi Oliver, On 14 April 2016 at 23:23, Oliver Korpilla wrote: > Hello. > > I was trying to write a gen_server for handling a TCP connection. So, > there would be an instance of gen_server for each open connection. (Each > instance would have a copy of the socket returned by gen_tcp:listen to > start accepting a connection from.) > > One thing I noticed is that gen_tcp does not seem to play very well with > OTP's assumptions since accept is a blocking call, at least I understood it > (and the principles) that way. > > So, on the net I saw examples dealing with accept in init/1. So, this will > delay init/1 from finishing until there is either a timeout or there is a > connection. I read somewhere else that it is a good practice to keep init > from blocking... > Yes, that is correct, because until your callback_module:init returns, your supervisor:init will not return which means your application startup hasn't finished, which in turn means it will block any applications after yours from starting. > > I saw an example (I think it was in "Learn You Some Erlang Good") that > delayed the accept call by calling gen_server:cast and sending your own > mailbox a trigger that was immediately followed by the blocking accept > call. While this certainly was keeping init short, I wonder what was > exactly gained here? > > Both init and handle_cast get executed in the new process started by > gen_server:start_link. If either of them blocks, the process cannot react > to anything else, right? So, how would a cast blocking the server be better > than init? The process is still not responsive to events. > init is executing in the context of the spawned gen_server process. It sends itself a message (which is non-blocking) and the init function returns, which means the supervisor:init function finishes executing, and your application startup is complete. Now your gen_server callback handle_cast is immediately called which then calls the gen_tcp:accept blocking call. It is okay for this to be blocked, because that is the main purpose of this process - block until a connection is available. The blocking nature is a problem only in two situations. 1. Code upgrade tends to be a problem. If you update the module invoking the blocking gen_tcp:accept twice, with no intervening socket connections being accepted, the server process will be killed, because your process hasn't handled the code upgrade message. 2. The gen_server process handles any other application specific messages - which as you've surmised will be blocked. I typically use the following pattern for accepting TCP connections. https://gist.github.com/cmullaparthi/18ba2219befbf7a3c44c28cab004456f This frees up your process which owns the listen socket (tcp_server.erl) to handle other messages as well. The process which is blocking (tcp_socket.erl) is not doing anything useful until it gets a TCP connection. I hope this helps. cheers, Chandru -------------- next part -------------- An HTML attachment was scrubbed... URL: From Oliver.Korpilla@REDACTED Sat Apr 16 08:36:59 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Sat, 16 Apr 2016 08:36:59 +0200 Subject: [erlang-questions] gen_server etc: blocking init, blocking cast In-Reply-To: References: , Message-ID: Hello, Chandru. Thank you very much. That clears it up for me. :) Cheers, Oliver ? Gesendet:?Samstag, 16. April 2016 um 07:25 Uhr Von:?Chandru An:?"Oliver Korpilla" Cc:?"Erlang-Questions Questions" Betreff:?Re: [erlang-questions] gen_server etc: blocking init, blocking cast Hi Oliver, ? init is executing in the context of the spawned gen_server process. It sends itself a message (which is non-blocking) and the init function returns, which means the supervisor:init function finishes executing, and your application startup is complete. Now your gen_server callback handle_cast is immediately called which then calls the gen_tcp:accept blocking call. It is okay for this to be blocked, because that is the main purpose of this process - block until a connection is available. The blocking nature is a problem only in two situations. ? 1. Code upgrade tends to be a problem. If you update the module invoking the blocking gen_tcp:accept twice, with no intervening socket connections being accepted, the server process will be killed, because your process hasn't handled the code upgrade message. ? 2. The gen_server process handles any other application specific messages - which as you've surmised will be blocked. ? I typically use the following pattern for accepting TCP connections. ? https://gist.github.com/cmullaparthi/18ba2219befbf7a3c44c28cab004456f[https://gist.github.com/cmullaparthi/18ba2219befbf7a3c44c28cab004456f]? ? This frees up your process which owns the listen socket (tcp_server.erl) to handle other messages as well. The process which is blocking (tcp_socket.erl) is not doing anything useful until it gets a TCP connection. ? I hope this helps. ? cheers, Chandru ? From mikpelinux@REDACTED Sat Apr 16 11:33:07 2016 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Sat, 16 Apr 2016 11:33:07 +0200 Subject: [erlang-questions] where is OTP-17.5.6.9? Message-ID: <22290.1747.922557.275472@gargle.gargle.HOWL> The download page has an OTP-17.5.6.9.README which mentions an OTP-17.5.6.9 tag, and the maint branch has a commit mentioning merging OTP-17.5.6.9, but: - a fresh git clone of otp.git from github doesn't know about that tag (it lists OTP-17 tags up to and including OTP-17.5.6.8 but not .9) - likewise the tags tab at github's web i/f doesn't list the .9 tag - browsing the commits on the maint-17 branch shows nothing newer than 17.5.6.8 Did someone forget to push to github? From Trond.Endrestol@REDACTED Mon Apr 18 08:33:49 2016 From: Trond.Endrestol@REDACTED (=?UTF-8?Q?Trond_Endrest=C3=B8l?=) Date: Mon, 18 Apr 2016 08:33:49 +0200 (CEST) Subject: [erlang-questions] Detecting high CPU utilisation in Erlang Message-ID: Hi, Is there a way for the Erlang RTS to detect high CPU utilisation and act accordingly? Could such events be handled by OTP supervisors? Some more background information: Telenor in Norway got som strange SS7 signalling from Priority One Security, as it was later revealed, late in February 2016. The Telenor HLR got stuck in an infinite loop, consuming resources, grinding to a halt, more or less. I'm probably oversimplifying. Source, in Norwegian: http://www.digi.no/tele-kommunikasjon/2016/04/15/roper-hvem-som-forarsaket-telenors-mobil-havari I'm just curious as to how Erlang (RTS) would handle such events. -- ---------------------------------------------------------------------- Trond Endrest?l | Trond.Endrestol@REDACTED ACM, NAS, NUUG, SAGE, USENIX | FreeBSD 10.3-S & Alpine 2.20 From cian@REDACTED Mon Apr 18 12:00:54 2016 From: cian@REDACTED (Cian Synnott) Date: Mon, 18 Apr 2016 11:00:54 +0100 Subject: [erlang-questions] Detecting high CPU utilisation in Erlang In-Reply-To: References: Message-ID: Hi Trond, On Mon, Apr 18, 2016 at 7:33 AM, Trond Endrest?l wrote: > Is there a way for the Erlang RTS to detect high CPU utilisation and > act accordingly? Could such events be handled by OTP supervisors? > Fred H?bert's "Stuff Goes Bad: Erlang in Anger" is the best resource on this topic: http://www.erlang-in-anger.com/ See especially chapters 5 (runtime metrics) and 8 (CPU and scheduler hogs) for the available instrumentation. Chapter 3 discusses the general problem of overload and strategies for degrading service gracefully. There are various applications that can help implement some of these strategies, e.g. circuit breakers (https://github.com/jlouis/fuse, https://github.com/klarna/circuit_breaker) or load regulation & capacity management (https://github.com/esl/jobs, https://github.com/basho/sidejob). The short answer is that there's no magic bullet in Erlang for this type of problem - designing systems that are resilient to overload is always difficult and application-dependent - but the instrumentation, tooling and community support are in place to help. Cian From henrik.x.nord@REDACTED Mon Apr 18 12:46:22 2016 From: henrik.x.nord@REDACTED (Henrik Nord X) Date: Mon, 18 Apr 2016 12:46:22 +0200 Subject: [erlang-questions] where is OTP-17.5.6.9? In-Reply-To: <22290.1747.922557.275472@gargle.gargle.HOWL> References: <22290.1747.922557.275472@gargle.gargle.HOWL> Message-ID: <5714BAFE.80006@ericsson.com> Yes It looks like it was not pushed to github for some reason. It should be there now. Thanks for pointing this out. /Henrik On 04/16/2016 11:33 AM, Mikael Pettersson wrote: > The download page has an OTP-17.5.6.9.README which mentions an OTP-17.5.6.9 tag, > and the maint branch has a commit mentioning merging OTP-17.5.6.9, but: > > - a fresh git clone of otp.git from github doesn't know about that tag > (it lists OTP-17 tags up to and including OTP-17.5.6.8 but not .9) > - likewise the tags tab at github's web i/f doesn't list the .9 tag > - browsing the commits on the maint-17 branch shows nothing newer than 17.5.6.8 > > Did someone forget to push to github? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From galina.lezina@REDACTED Mon Apr 18 13:11:08 2016 From: galina.lezina@REDACTED (Galina Lezina) Date: Mon, 18 Apr 2016 16:11:08 +0500 Subject: [erlang-questions] Erlang init_per_group terminates gen_server beforehand Message-ID: Common test init_per_group/2 terminates gen_server beforehand (right in the end of init_per_group/2 call) when it is started with gen_server:start_link. But it is ok to start server with gen_server:start. gen_server can be started with any of these methods (start and start_link) in init_per_suite/1 and init_per_testcase/2 and it will work fine. Why it is not possible to start gen_server in init_per_group/2 with gen_server:start_link? -------------- next part -------------- An HTML attachment was scrubbed... URL: From jesper.louis.andersen@REDACTED Mon Apr 18 13:44:41 2016 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 18 Apr 2016 13:44:41 +0200 Subject: [erlang-questions] Detecting high CPU utilisation in Erlang In-Reply-To: References: Message-ID: On Mon, Apr 18, 2016 at 8:33 AM, Trond Endrest?l < Trond.Endrestol@REDACTED> wrote: > Is there a way for the Erlang RTS to detect high CPU utilisation and > act accordingly? Could such events be handled by OTP supervisors? > One thing is detecting high CPU, which the os_mon application will do, and then raise an alarm if it is consistenly high. The other thing is doing something about it. Some systems are operating nominally when their CPU load hits 100%, whereas this is a grave outlier in others. One system which can be used is uwiger/jobs, which allow you to plug into the high-CPU scenario and apply dampening to your queueing systems. Coupled with some kind of queue priority, it can be used to avoid processing excess amounts of data. In general, the rule of the border applies: detect overload early and reject service at the border of the system boundary. Once a job is accepted, process it to the end. -- J. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Mon Apr 18 15:38:36 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Mon, 18 Apr 2016 09:38:36 -0400 Subject: [erlang-questions] Erlang init_per_group terminates gen_server beforehand In-Reply-To: References: Message-ID: <20160418133836.GA2013@fhebert-ltm2.internal.salesforce.com> On 04/18, Galina Lezina wrote: >Common test init_per_group/2 terminates gen_server beforehand (right in the >end of init_per_group/2 call) when it is started with gen_server:start_link. >But it is ok to start server with gen_server:start. > >gen_server can be started with any of these methods (start and start_link) >in init_per_suite/1 and init_per_testcase/2 and it will work fine. > >Why it is not possible to start gen_server in init_per_group/2 with >gen_server:start_link? The distinction is that the running of group initialization and shutdown appears to take place in a short-lived transient process. Because of that, anything that is linked to the process running init_per_group will die when the initialization phase for the group is done. I do not know why this is, but this has been a stable implementation detail for years now. Regards, Fred. From moonsolo@REDACTED Tue Apr 19 08:50:33 2016 From: moonsolo@REDACTED (PhayTsukiming) Date: Tue, 19 Apr 2016 06:50:33 +0000 Subject: [erlang-questions] Dialyzer and List Type Message-ID: Hi list, I am new to erlang and dialyzer. I have encounter a problem when I tried to fiddle with type specifications. Say, I have two functions: -module(t). -export([foo/1, bar/1]). -spec foo([integer()])->[integer()]. foo(Arg)-> [I+1||I<-Arg]. -spec bar(atom())->[integer()]. bar(_Arg)-> foo(_Arg). Then apply dialyzer to this module and dialyzer warns: t.erl:8: Invalid type specification for function t:bar/1. The success typing is ([integer()]) -> [integer()] But if I change the specification of bar to: -spec bar([atom()])->[integer()]. dialyzer will show no warnings. My intention is to make sure the argument of bar is the same type of list as the argument of foo. Is this normal? Do I miss or misunderstand something about success typing or dialyzer? -------------- next part -------------- An HTML attachment was scrubbed... URL: From aronisstav@REDACTED Tue Apr 19 10:03:46 2016 From: aronisstav@REDACTED (Stavros Aronis) Date: Tue, 19 Apr 2016 08:03:46 +0000 Subject: [erlang-questions] Dialyzer and List Type In-Reply-To: References: Message-ID: Hi! The most general type that foo can have is [number()] -> [number()]. The spec restricts both number() instances to just integer(). Then, since bar's only argument is given as is to foo, it should also be [integer()]. If you specify bar's argument as atom() then there is no way to make a successful call to foo and Dialyzer complains. If however bar's argument is a list of atoms, then there is a way, though a bit trivial, for a call to succeed: an empty list is both a valid list of atoms and a valid list of integers. Dialyzer can see this, and does not complain. Perhaps a patch is needed, for a warning of type "only the empty list satisfies the given spec"... Cheers, Stavros On Tue, 19 Apr 2016, 08:50 PhayTsukiming, wrote: > Hi list, I am new to erlang and dialyzer. I have encounter a problem when > I tried to fiddle with type specifications. > > Say, I have two functions: > > -module(t). > -export([foo/1, bar/1]). > > -spec foo([integer()])->[integer()]. > foo(Arg)-> > [I+1||I<-Arg]. > > -spec bar(atom())->[integer()]. > bar(_Arg)-> > foo(_Arg). > > Then apply dialyzer to this module and dialyzer warns: > > t.erl:8: Invalid type specification for function t:bar/1. The success > typing is ([integer()]) -> [integer()] > > But if I change the specification of bar to: > > -spec bar([atom()])->[integer()]. > > dialyzer will show no warnings. My intention is to make sure the argument > of bar is the same type of list as the argument of foo. > > Is this normal? Do I miss or misunderstand something about success typing > or dialyzer? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kostis@REDACTED Tue Apr 19 10:06:21 2016 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 19 Apr 2016 10:06:21 +0200 Subject: [erlang-questions] Dialyzer and List Type In-Reply-To: References: Message-ID: <5715E6FD.3060700@cs.ntua.gr> On 04/19/2016 08:50 AM, PhayTsukiming wrote: > > Then apply dialyzer to this module and dialyzer warns: > > t.erl:8: Invalid type specification for function t:bar/1. The success > typing is ([integer()]) -> [integer()] > > But if I change the specification of bar to: > > -spec bar([atom()])->[integer()]. > > dialyzer will show no warnings. My intention is to make sure the > argument of bar is the same type of list as the argument of foo. OK, but then why don't you put the same spec there? Is there any problem in this case? > Is this normal? Do I miss or misunderstand something about success > typing or dialyzer? Yes, you do. All that dialyzer ever promised is that when it spits a warning then this warning is correct. (This is the "Dialyzer is never wrong" slogan.) On the other hand, it never promised to produce warnings in all cases you may expect it to do so. Kostis From moonsolo@REDACTED Tue Apr 19 10:27:25 2016 From: moonsolo@REDACTED (PhayTsukiming) Date: Tue, 19 Apr 2016 08:27:25 +0000 Subject: [erlang-questions] Dialyzer and List Type In-Reply-To: <5715E6FD.3060700@cs.ntua.gr> References: <5715E6FD.3060700@cs.ntua.gr> Message-ID: On Tue, Apr 19, 2016 at 4:08 PM Kostis Sagonas wrote: > On 04/19/2016 08:50 AM, PhayTsukiming wrote: > > > > Then apply dialyzer to this module and dialyzer warns: > > > > t.erl:8: Invalid type specification for function t:bar/1. The success > > typing is ([integer()]) -> [integer()] > > > > But if I change the specification of bar to: > > > > -spec bar([atom()])->[integer()]. > > > > dialyzer will show no warnings. My intention is to make sure the > > argument of bar is the same type of list as the argument of foo. > > OK, but then why don't you put the same spec there? Is there any > problem in this case? > Sorry, I forgot to mention that I was experimenting with my understanding of dialyzer. In the beginning, I thought type specification in erlang works as just the same as in other static type language. So I used -spec bar(atom())->[integer()] then dialyzer complained. This worked as expected. But when I used -spec bar([atom()])->[integer()], then dialyzer didn't say a word and in static type languages, similar specifications would cause the compiler to generate errors. > > > Is this normal? Do I miss or misunderstand something about success > > typing or dialyzer? > > Yes, you do. All that dialyzer ever promised is that when it spits a > warning then this warning is correct. (This is the "Dialyzer is never > wrong" slogan.) > > On the other hand, it never promised to produce warnings in all cases > you may expect it to do so. > > Kostis > -------------- next part -------------- An HTML attachment was scrubbed... URL: From moonsolo@REDACTED Tue Apr 19 10:32:09 2016 From: moonsolo@REDACTED (PhayTsukiming) Date: Tue, 19 Apr 2016 08:32:09 +0000 Subject: [erlang-questions] Dialyzer and List Type In-Reply-To: References: Message-ID: Thanks for you explanation. One more question: should I use non empty list specification to make dialyzer complain? On Tue, Apr 19, 2016 at 4:03 PM Stavros Aronis wrote: > Hi! > > The most general type that foo can have is [number()] -> [number()]. The > spec restricts both number() instances to just integer(). Then, since bar's > only argument is given as is to foo, it should also be [integer()]. > > If you specify bar's argument as atom() then there is no way to make a > successful call to foo and Dialyzer complains. If however bar's argument is > a list of atoms, then there is a way, though a bit trivial, for a call to > succeed: an empty list is both a valid list of atoms and a valid list of > integers. > > Dialyzer can see this, and does not complain. Perhaps a patch is needed, > for a warning of type "only the empty list satisfies the given spec"... > > Cheers, > > Stavros > > On Tue, 19 Apr 2016, 08:50 PhayTsukiming, wrote: > >> Hi list, I am new to erlang and dialyzer. I have encounter a problem >> when I tried to fiddle with type specifications. >> >> Say, I have two functions: >> >> -module(t). >> -export([foo/1, bar/1]). >> >> -spec foo([integer()])->[integer()]. >> foo(Arg)-> >> [I+1||I<-Arg]. >> >> -spec bar(atom())->[integer()]. >> bar(_Arg)-> >> foo(_Arg). >> >> Then apply dialyzer to this module and dialyzer warns: >> >> t.erl:8: Invalid type specification for function t:bar/1. The success >> typing is ([integer()]) -> [integer()] >> >> But if I change the specification of bar to: >> >> -spec bar([atom()])->[integer()]. >> >> dialyzer will show no warnings. My intention is to make sure the argument >> of bar is the same type of list as the argument of foo. >> >> Is this normal? Do I miss or misunderstand something about success typing >> or dialyzer? >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From aronisstav@REDACTED Tue Apr 19 11:10:18 2016 From: aronisstav@REDACTED (Stavros Aronis) Date: Tue, 19 Apr 2016 09:10:18 +0000 Subject: [erlang-questions] Dialyzer and List Type In-Reply-To: References: Message-ID: I would not phrase it like that... you are hopefully not writing specifications just for the sake of testing or being shouted at by Dialyzer! :-) If your function should work for the empty list, then your specification should include it. It is true, however, that with an "[atom(),...]" spec (non-empty list of atoms), you will get an error, as this type has no values in common with the "[integer()]" (possibly empty list of integers). Stavros On Tue, Apr 19, 2016 at 10:32 AM PhayTsukiming wrote: > Thanks for you explanation. One more question: should I use non empty list > specification to make dialyzer complain? > > > On Tue, Apr 19, 2016 at 4:03 PM Stavros Aronis > wrote: > >> Hi! >> >> The most general type that foo can have is [number()] -> [number()]. The >> spec restricts both number() instances to just integer(). Then, since bar's >> only argument is given as is to foo, it should also be [integer()]. >> >> If you specify bar's argument as atom() then there is no way to make a >> successful call to foo and Dialyzer complains. If however bar's argument is >> a list of atoms, then there is a way, though a bit trivial, for a call to >> succeed: an empty list is both a valid list of atoms and a valid list of >> integers. >> >> Dialyzer can see this, and does not complain. Perhaps a patch is needed, >> for a warning of type "only the empty list satisfies the given spec"... >> >> Cheers, >> >> Stavros >> >> On Tue, 19 Apr 2016, 08:50 PhayTsukiming, wrote: >> >>> Hi list, I am new to erlang and dialyzer. I have encounter a problem >>> when I tried to fiddle with type specifications. >>> >>> Say, I have two functions: >>> >>> -module(t). >>> -export([foo/1, bar/1]). >>> >>> -spec foo([integer()])->[integer()]. >>> foo(Arg)-> >>> [I+1||I<-Arg]. >>> >>> -spec bar(atom())->[integer()]. >>> bar(_Arg)-> >>> foo(_Arg). >>> >>> Then apply dialyzer to this module and dialyzer warns: >>> >>> t.erl:8: Invalid type specification for function t:bar/1. The success >>> typing is ([integer()]) -> [integer()] >>> >>> But if I change the specification of bar to: >>> >>> -spec bar([atom()])->[integer()]. >>> >>> dialyzer will show no warnings. My intention is to make sure the >>> argument of bar is the same type of list as the argument of foo. >>> >>> Is this normal? Do I miss or misunderstand something about success >>> typing or dialyzer? >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From rvirding@REDACTED Tue Apr 19 22:29:39 2016 From: rvirding@REDACTED (Robert Virding) Date: Tue, 19 Apr 2016 22:29:39 +0200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <570E4C69.90004@ericsson.com> References: <570E4C69.90004@ericsson.com> Message-ID: I missed this thread, but Bj?rn-Egil you know what I want. :-) maps:first(Map) -> {Key,Value} | error. maps:next(Key, Map) -> {Key,Value} | error. Robert On 13 April 2016 at 15:40, Bj?rn-Egil Dahlberg XB < bjorn-egil.xb.dahlberg@REDACTED> wrote: > There is currently no (good) way to get out of C-context to execute some > erlang snippet and then get back into c-context. Traps does this but only > in a controlled manner. I think it is doable though. > > Your proposal of maps_apply/3 is the maps equivalent of dict:update/3 and > sadly that name is already occupied in maps (it's the equivalent of > gb_trees). > > I think we should add it to the API without thinking about the performance > too much. Performance can always be improved later on. > > Is maps:apply/3 the best name we can think of? I don't really trust you > and naming things. =) update/3,4 would probably have been best but c'est la > vie. > > // Bj?rn-Egil > > > On 2016-04-13 13:09, Jesper Louis Andersen wrote: > > > On Fri, Apr 8, 2016 at 8:31 PM, Bj?rn-Egil Dahlberg < > wallentin.dahlberg@REDACTED> wrote: > >> Have you repeated some code or built your own private lib to handle >> certain maps specific tasks and thought "why isn't this in the maps module?" > > > One function I like to have is "apply update via function", and I > implement it somewhat often: > > maps_apply(F, K, Map) -> > V = maps:get(K, Map), > maps:put(K, F(V), Map). > > But it can be made much faster in a direct implementation since I don't > have to first pick up the value: I can apply F when I sit with the value at > hand and thus avoid the "put" lookup path. It might, however, be nasty to > implement because F is in Erlang-context, whereas the maps operations are > in BIF-context. > > > -- > J. > > > _______________________________________________ > erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From chauhan.kuldeep.sinh@REDACTED Wed Apr 20 09:16:19 2016 From: chauhan.kuldeep.sinh@REDACTED (KuldeepSinh Chauhan) Date: Wed, 20 Apr 2016 00:16:19 -0700 Subject: [erlang-questions] Wrangler Issues - Renaming of a function fails with an error. Message-ID: Renaming of a function fails, with following error. The Wrangler application is not started, or is not working properly, please restart the refactorer! This is my first time experience with installing and using Wrangler. The steps I have followed, are described below. 1. I have Erlang 18.3 Installed on my Ubuntu 15.04. 2. I have built and installed Wrangler, with the latest code downloaded from Wrangler repository on GitHub. 3. Thus the installed version on my machine is Wrangler 1.2.0 4. I have configured, it as mentioned in the INSTALL document, available in the same repository 5. Initially, Wrangler menu did not appear in my Emacs, on pressing Cc Cr. Though, pressing Cc Cr started Wrangler - and I could see following message : (Successfully uploaded backend modules into node). 6. In order to see Wrangler menu on Emacs, I ran following command : Mx wrangler-menu-init - and I was able to see wrangler menu. 7. Following it, I customized Wragler and provided my source directory to .erl files. 8. After following above steps to configure Wrangler, when I tried to rename a function with Wrangler, I got the error mentioned at the top. Please let me know, if something I am missing. Thank you, Kuldeep. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jozsef.berces@REDACTED Wed Apr 20 16:15:59 2016 From: jozsef.berces@REDACTED (=?iso-8859-1?Q?J=F3zsef_B=E9rces?=) Date: Wed, 20 Apr 2016 14:15:59 +0000 Subject: [erlang-questions] Erlang editor in Erlang Message-ID: <7460EBDDCF52084A849D0F271CE059B818470427@ESESSMB101.ericsson.se> This might be a bit odd question knowing that every popular editor has Erlang mode but I ask it. Do we have any Erlang source code Editor (with syntax highlight) written in Erlang? Thanks, Jozsef -------------- next part -------------- An HTML attachment was scrubbed... URL: From federico.carrone@REDACTED Wed Apr 20 16:38:00 2016 From: federico.carrone@REDACTED (Federico Carrone) Date: Wed, 20 Apr 2016 14:38:00 +0000 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <570E4C69.90004@ericsson.com> Message-ID: It would be awesome to have a a map iterator/map comprehension. On Tue, Apr 19, 2016 at 5:30 PM Robert Virding wrote: > I missed this thread, but Bj?rn-Egil you know what I want. :-) > > maps:first(Map) -> {Key,Value} | error. > maps:next(Key, Map) -> {Key,Value} | error. > > Robert > > > On 13 April 2016 at 15:40, Bj?rn-Egil Dahlberg XB < > bjorn-egil.xb.dahlberg@REDACTED> wrote: > >> There is currently no (good) way to get out of C-context to execute some >> erlang snippet and then get back into c-context. Traps does this but only >> in a controlled manner. I think it is doable though. >> >> Your proposal of maps_apply/3 is the maps equivalent of dict:update/3 and >> sadly that name is already occupied in maps (it's the equivalent of >> gb_trees). >> >> I think we should add it to the API without thinking about the >> performance too much. Performance can always be improved later on. >> >> Is maps:apply/3 the best name we can think of? I don't really trust you >> and naming things. =) update/3,4 would probably have been best but c'est la >> vie. >> >> // Bj?rn-Egil >> >> >> On 2016-04-13 13:09, Jesper Louis Andersen wrote: >> >> >> On Fri, Apr 8, 2016 at 8:31 PM, Bj?rn-Egil Dahlberg < >> wallentin.dahlberg@REDACTED> wrote: >> >>> Have you repeated some code or built your own private lib to handle >>> certain maps specific tasks and thought "why isn't this in the maps module?" >> >> >> One function I like to have is "apply update via function", and I >> implement it somewhat often: >> >> maps_apply(F, K, Map) -> >> V = maps:get(K, Map), >> maps:put(K, F(V), Map). >> >> But it can be made much faster in a direct implementation since I don't >> have to first pick up the value: I can apply F when I sit with the value at >> hand and thus avoid the "put" lookup path. It might, however, be nasty to >> implement because F is in Erlang-context, whereas the maps operations are >> in BIF-context. >> >> >> -- >> J. >> >> >> _______________________________________________ >> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From federico.carrone@REDACTED Wed Apr 20 16:44:37 2016 From: federico.carrone@REDACTED (Federico Carrone) Date: Wed, 20 Apr 2016 14:44:37 +0000 Subject: [erlang-questions] Is ejabberd/MongooseIM not a good choise for mobile IM? In-Reply-To: References: Message-ID: I think they are good tools, however xmpp is pretty heavy. You could implement a chat on top of vernemq (https://github.com/erlio/vernemq) - an Erlang MQTT message broker - or Cowboy with the websocket module or REST handler + SSE for pushing events. I have been using Cowboy with websockets or REST/HTTP +SSE with great success. On Thu, Apr 14, 2016 at 10:53 AM Caiyun Deng wrote: > Hi! I want to develop a mobile IM app, i searched something about. > Because xmpp is heavyweight, is ejabberd/MongooseIM not a good choise for > mobile IM? > If you develop a mobile IM today, which tool will you choose? > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bjorn-egil.xb.dahlberg@REDACTED Wed Apr 20 16:44:51 2016 From: bjorn-egil.xb.dahlberg@REDACTED (=?UTF-8?Q?Bj=c3=b6rn-Egil_Dahlberg_XB?=) Date: Wed, 20 Apr 2016 16:44:51 +0200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: <570E4C69.90004@ericsson.com> Message-ID: <571795E3.20700@ericsson.com> On 2016-04-20 16:38, Federico Carrone wrote: > It would be awesome to have a a map iterator/map comprehension. Many are of the same opinion. However iterators are not in this scope. Only API additions to the maps module are. Efficient iterators are necessary to implement efficient generators for maps. Efficient maps comprehension is whole other ballgame. I will post a PR of maps:take/2 and maps:apply/3,4 later this week. I'm not sure about apply .. or it's name. > > On Tue, Apr 19, 2016 at 5:30 PM Robert Virding > wrote: > > I missed this thread, but Bj?rn-Egil you know what I want. :-) > > maps:first(Map) -> {Key,Value} | error. > maps:next(Key, Map) -> {Key,Value} | error. > > Robert > > > On 13 April 2016 at 15:40, Bj?rn-Egil Dahlberg XB > > wrote: > > There is currently no (good) way to get out of C-context to > execute some erlang snippet and then get back into c-context. > Traps does this but only in a controlled manner. I think it is > doable though. > > Your proposal of maps_apply/3 is the maps equivalent of > dict:update/3 and sadly that name is already occupied in maps > (it's the equivalent of gb_trees). > > I think we should add it to the API without thinking about the > performance too much. Performance can always be improved later on. > > Is maps:apply/3 the best name we can think of? I don't really > trust you and naming things. =) update/3,4 would probably have > been best but c'est la vie. > > // Bj?rn-Egil > > > On 2016-04-13 13:09, Jesper Louis Andersen wrote: >> >> On Fri, Apr 8, 2016 at 8:31 PM, Bj?rn-Egil Dahlberg >> > > wrote: >> >> Have you repeated some code or built your own private lib >> to handle certain maps specific tasks and thought "why >> isn't this in the maps module?" >> >> >> One function I like to have is "apply update via function", >> and I implement it somewhat often: >> >> maps_apply(F, K, Map) -> >> V = maps:get(K, Map), >> maps:put(K, F(V), Map). >> >> But it can be made much faster in a direct implementation >> since I don't have to first pick up the value: I can apply F >> when I sit with the value at hand and thus avoid the "put" >> lookup path. It might, however, be nasty to implement because >> F is in Erlang-context, whereas the maps operations are in >> BIF-context. >> >> >> -- >> J. >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From zxq9@REDACTED Wed Apr 20 16:49:48 2016 From: zxq9@REDACTED (zxq9) Date: Wed, 20 Apr 2016 23:49:48 +0900 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: References: Message-ID: <2594613.LD87LRxh9p@changa> On 2016?4?8? ??? 20:31:37 Bj?rn-Egil Dahlberg wrote: > Hi there! > > I would like to know if you have any desires that we extend the current > maps module with additional functionality? Not sure whether this has been mentioned yet, but a maps:partition/2 would be wonderful. -Craig From maxim@REDACTED Wed Apr 20 17:52:52 2016 From: maxim@REDACTED (Maxim Sokhatsky) Date: Wed, 20 Apr 2016 18:52:52 +0300 Subject: [erlang-questions] Erlang editor in Erlang In-Reply-To: <7460EBDDCF52084A849D0F271CE059B818470427@ESESSMB101.ericsson.se> References: <7460EBDDCF52084A849D0F271CE059B818470427@ESESSMB101.ericsson.se> Message-ID: We have Pie editor:?https://github.com/5HT/pie Originally written by?Torbjorn Tornkvist, once maintained by?Luke Gorrie and now is refreshed by 5HT. Namdak Tonpa > On Apr 20, 2016, at 5:15 PM, J?zsef B?rces wrote: > > > This might be a bit odd question knowing that every popular editor has Erlang mode but I ask it. > > > > ? > > Do we have any Erlang source code Editor (with syntax highlight) written in Erlang? > > > > ? > > Thanks, > > > Jozsef > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sdl.web@REDACTED Thu Apr 21 06:52:51 2016 From: sdl.web@REDACTED (Leo Liu) Date: Thu, 21 Apr 2016 12:52:51 +0800 Subject: [erlang-questions] Wanted additions to the maps module? References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> Message-ID: On 2016-04-20 16:44 +0200, Bj?rn-Egil Dahlberg XB wrote: > I will post a PR of maps:take/2 and maps:apply/3,4 later this week. > I'm not sure about apply .. or it's name. Is either of the suggested update_with/3 or update_by/3 better? Sorry I use lisp every day and `apply' just feels wrong to me. Leo From rvirding@REDACTED Fri Apr 22 02:09:10 2016 From: rvirding@REDACTED (Robert Virding) Date: Fri, 22 Apr 2016 02:09:10 +0200 Subject: [erlang-questions] Wanted additions to the maps module? In-Reply-To: <571795E3.20700@ericsson.com> References: <570E4C69.90004@ericsson.com> <571795E3.20700@ericsson.com> Message-ID: My maps:first/1 and maps:next/2will be coming when? :-) Robert On 20 April 2016 at 16:44, Bj?rn-Egil Dahlberg XB < bjorn-egil.xb.dahlberg@REDACTED> wrote: > On 2016-04-20 16:38, Federico Carrone wrote: > > It would be awesome to have a a map iterator/map comprehension. > > > Many are of the same opinion. However iterators are not in this scope. > Only API additions to the maps module are. > Efficient iterators are necessary to implement efficient generators for > maps. Efficient maps comprehension is whole other ballgame. > > I will post a PR of maps:take/2 and maps:apply/3,4 later this week. I'm > not sure about apply .. or it's name. > > > > > On Tue, Apr 19, 2016 at 5:30 PM Robert Virding wrote: > >> I missed this thread, but Bj?rn-Egil you know what I want. :-) >> >> maps:first(Map) -> {Key,Value} | error. >> maps:next(Key, Map) -> {Key,Value} | error. >> >> Robert >> >> >> On 13 April 2016 at 15:40, Bj?rn-Egil Dahlberg XB < >> bjorn-egil.xb.dahlberg@REDACTED> >> wrote: >> >>> There is currently no (good) way to get out of C-context to execute some >>> erlang snippet and then get back into c-context. Traps does this but only >>> in a controlled manner. I think it is doable though. >>> >>> Your proposal of maps_apply/3 is the maps equivalent of dict:update/3 >>> and sadly that name is already occupied in maps (it's the equivalent of >>> gb_trees). >>> >>> I think we should add it to the API without thinking about the >>> performance too much. Performance can always be improved later on. >>> >>> Is maps:apply/3 the best name we can think of? I don't really trust you >>> and naming things. =) update/3,4 would probably have been best but c'est la >>> vie. >>> >>> // Bj?rn-Egil >>> >>> >>> On 2016-04-13 13:09, Jesper Louis Andersen wrote: >>> >>> >>> On Fri, Apr 8, 2016 at 8:31 PM, Bj?rn-Egil Dahlberg < >>> wallentin.dahlberg@REDACTED> wrote: >>> >>>> Have you repeated some code or built your own private lib to handle >>>> certain maps specific tasks and thought "why isn't this in the maps module?" >>> >>> >>> One function I like to have is "apply update via function", and I >>> implement it somewhat often: >>> >>> maps_apply(F, K, Map) -> >>> V = maps:get(K, Map), >>> maps:put(K, F(V), Map). >>> >>> But it can be made much faster in a direct implementation since I don't >>> have to first pick up the value: I can apply F when I sit with the value at >>> hand and thus avoid the "put" lookup path. It might, however, be nasty to >>> implement because F is in Erlang-context, whereas the maps operations are >>> in BIF-context. >>> >>> >>> -- >>> J. >>> >>> >>> _______________________________________________ >>> erlang-questions mailing listerlang-questions@REDACTED://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Fri Apr 22 11:39:19 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 22 Apr 2016 11:39:19 +0200 Subject: [erlang-questions] Reconnect to nodes Message-ID: Dear list, A simple question: am I correct that, when a node is removed because of a net split, you need to have your own application logic to reconnect to it, and nothing in the VM will try doing that for you? Let me show you an example. I have two nodes: 1@REDACTED and 2@REDACTED that are connected to each other: (1@REDACTED)1> nodes(). ['2@REDACTED'] On node 2 I listen for nodedown events of node 1: (2@REDACTED)1> monitor_node('1@REDACTED', true). true On node 1, I simulate a net splits with the best option I've found until now, i.e suspending the net_kernel process: (1@REDACTED)2> sys:suspend(net_kernel). ok After ~60 seconds on node 2 I get: =ERROR REPORT==== 22-Apr-2016::11:28:21 === ** Node '1@REDACTED' not responding ** ** Removing (timedout) connection ** (2@REDACTED)2> flush(). Shell got {nodedown,'1@REDACTED'} Now the two nodes are disconnected: (1@REDACTED)3> nodes(). [] (2@REDACTED)3> nodes(). [] Even when I resume the net_kernel process: (1@REDACTED)4> sys:resume(net_kernel). ok The nodes do not reconnect: (1@REDACTED)5> nodes(). [] I'm ok with this, though I would like to confirm that my understanding is correct. If so, does everyone just implement some standard connection manager that does only reconnections? Thank you, r. -------------- next part -------------- An HTML attachment was scrubbed... URL: From andras.boroska@REDACTED Fri Apr 22 12:31:02 2016 From: andras.boroska@REDACTED (=?UTF-8?Q?Boroska_Andr=C3=A1s?=) Date: Fri, 22 Apr 2016 11:31:02 +0100 Subject: [erlang-questions] Reconnect to nodes In-Reply-To: References: Message-ID: Hi Roberto, Your assumptions are correct. Note, however, that rpc calls via the rpc module will reconnect the nodes automatically which is a nice convenience. Br, Andras -------------- next part -------------- An HTML attachment was scrubbed... URL: From serge@REDACTED Fri Apr 22 12:35:56 2016 From: serge@REDACTED (Serge Aleynikov) Date: Fri, 22 Apr 2016 06:35:56 -0400 Subject: [erlang-questions] Reconnect to nodes In-Reply-To: References: Message-ID: Roberto, This is the expected behavior. Note that the nodes will automatically reconnect by default when either node has a process that sends a message to a process on the remote node. This reconnection behavior can be modified by setting the kernel's 'dist_auto_connect' option. Other applications (such as mnesia) may require custom recovery from a network split, which is one of the reasons why automatic reconnection may not be desirable. Regards, Serge On Fri, Apr 22, 2016 at 5:39 AM, Roberto Ostinelli wrote: > Dear list, > A simple question: am I correct that, when a node is removed because of a > net split, you need to have your own application logic to reconnect to it, > and nothing in the VM will try doing that for you? > > Let me show you an example. I have two nodes: 1@REDACTED and 2@REDACTED > that are connected to each other: > > (1@REDACTED)1> nodes(). > ['2@REDACTED'] > > On node 2 I listen for nodedown events of node 1: > > (2@REDACTED)1> monitor_node('1@REDACTED', true). > true > > On node 1, I simulate a net splits with the best option I've found until > now, i.e suspending the net_kernel process: > > (1@REDACTED)2> sys:suspend(net_kernel). > ok > > After ~60 seconds on node 2 I get: > > =ERROR REPORT==== 22-Apr-2016::11:28:21 === > ** Node '1@REDACTED' not responding ** > ** Removing (timedout) connection ** > (2@REDACTED)2> flush(). > Shell got {nodedown,'1@REDACTED'} > > Now the two nodes are disconnected: > > (1@REDACTED)3> nodes(). > [] > > (2@REDACTED)3> nodes(). > [] > > Even when I resume the net_kernel process: > > (1@REDACTED)4> sys:resume(net_kernel). > ok > > The nodes do not reconnect: > > (1@REDACTED)5> nodes(). > [] > > I'm ok with this, though I would like to confirm that my understanding is > correct. > If so, does everyone just implement some standard connection manager that > does only reconnections? > > Thank you, > r. > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Fri Apr 22 13:42:50 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 22 Apr 2016 13:42:50 +0200 Subject: [erlang-questions] Reconnect to nodes In-Reply-To: References: Message-ID: Thank you both for confirming. Hence I guess some kind of a basic reconnection manager might be helpful here. Thanks! r. On Fri, Apr 22, 2016 at 12:35 PM, Serge Aleynikov wrote: > Roberto, > > This is the expected behavior. Note that the nodes will automatically > reconnect by default when either node has a process that sends a message to > a process on the remote node. This reconnection behavior can be modified > by setting the kernel's 'dist_auto_connect' option. > > Other applications (such as mnesia) may require custom recovery from a > network split, which is one of the reasons why automatic reconnection may > not be desirable. > > Regards, > > Serge > > On Fri, Apr 22, 2016 at 5:39 AM, Roberto Ostinelli > wrote: > >> Dear list, >> A simple question: am I correct that, when a node is removed because of a >> net split, you need to have your own application logic to reconnect to it, >> and nothing in the VM will try doing that for you? >> >> Let me show you an example. I have two nodes: 1@REDACTED and 2@REDACTED >> that are connected to each other: >> >> (1@REDACTED)1> nodes(). >> ['2@REDACTED'] >> >> On node 2 I listen for nodedown events of node 1: >> >> (2@REDACTED)1> monitor_node('1@REDACTED', true). >> true >> >> On node 1, I simulate a net splits with the best option I've found until >> now, i.e suspending the net_kernel process: >> >> (1@REDACTED)2> sys:suspend(net_kernel). >> ok >> >> After ~60 seconds on node 2 I get: >> >> =ERROR REPORT==== 22-Apr-2016::11:28:21 === >> ** Node '1@REDACTED' not responding ** >> ** Removing (timedout) connection ** >> (2@REDACTED)2> flush(). >> Shell got {nodedown,'1@REDACTED'} >> >> Now the two nodes are disconnected: >> >> (1@REDACTED)3> nodes(). >> [] >> >> (2@REDACTED)3> nodes(). >> [] >> >> Even when I resume the net_kernel process: >> >> (1@REDACTED)4> sys:resume(net_kernel). >> ok >> >> The nodes do not reconnect: >> >> (1@REDACTED)5> nodes(). >> [] >> >> I'm ok with this, though I would like to confirm that my understanding is >> correct. >> If so, does everyone just implement some standard connection manager that >> does only reconnections? >> >> Thank you, >> r. >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From serge@REDACTED Fri Apr 22 14:59:26 2016 From: serge@REDACTED (Serge Aleynikov) Date: Fri, 22 Apr 2016 08:59:26 -0400 Subject: [erlang-questions] Reconnect to nodes In-Reply-To: References: Message-ID: You can take a look at this app that handles detection of network splits and auto-reconnecting: https://github.com/saleyn/netmon Serge On Fri, Apr 22, 2016 at 7:42 AM, Roberto Ostinelli wrote: > Thank you both for confirming. > Hence I guess some kind of a basic reconnection manager might be helpful > here. > > Thanks! > > r. > > On Fri, Apr 22, 2016 at 12:35 PM, Serge Aleynikov > wrote: > >> Roberto, >> >> This is the expected behavior. Note that the nodes will automatically >> reconnect by default when either node has a process that sends a message to >> a process on the remote node. This reconnection behavior can be modified >> by setting the kernel's 'dist_auto_connect' option. >> >> Other applications (such as mnesia) may require custom recovery from a >> network split, which is one of the reasons why automatic reconnection may >> not be desirable. >> >> Regards, >> >> Serge >> >> On Fri, Apr 22, 2016 at 5:39 AM, Roberto Ostinelli >> wrote: >> >>> Dear list, >>> A simple question: am I correct that, when a node is removed because of >>> a net split, you need to have your own application logic to reconnect to >>> it, and nothing in the VM will try doing that for you? >>> >>> Let me show you an example. I have two nodes: 1@REDACTED and >>> 2@REDACTED that are connected to each other: >>> >>> (1@REDACTED)1> nodes(). >>> ['2@REDACTED'] >>> >>> On node 2 I listen for nodedown events of node 1: >>> >>> (2@REDACTED)1> monitor_node('1@REDACTED', true). >>> true >>> >>> On node 1, I simulate a net splits with the best option I've found until >>> now, i.e suspending the net_kernel process: >>> >>> (1@REDACTED)2> sys:suspend(net_kernel). >>> ok >>> >>> After ~60 seconds on node 2 I get: >>> >>> =ERROR REPORT==== 22-Apr-2016::11:28:21 === >>> ** Node '1@REDACTED' not responding ** >>> ** Removing (timedout) connection ** >>> (2@REDACTED)2> flush(). >>> Shell got {nodedown,'1@REDACTED'} >>> >>> Now the two nodes are disconnected: >>> >>> (1@REDACTED)3> nodes(). >>> [] >>> >>> (2@REDACTED)3> nodes(). >>> [] >>> >>> Even when I resume the net_kernel process: >>> >>> (1@REDACTED)4> sys:resume(net_kernel). >>> ok >>> >>> The nodes do not reconnect: >>> >>> (1@REDACTED)5> nodes(). >>> [] >>> >>> I'm ok with this, though I would like to confirm that my understanding >>> is correct. >>> If so, does everyone just implement some standard connection manager >>> that does only reconnections? >>> >>> Thank you, >>> r. >>> >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto@REDACTED Fri Apr 22 15:25:21 2016 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 22 Apr 2016 15:25:21 +0200 Subject: [erlang-questions] Reconnect to nodes In-Reply-To: References: Message-ID: Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjtruog@REDACTED Fri Apr 22 20:25:08 2016 From: mjtruog@REDACTED (Michael Truog) Date: Fri, 22 Apr 2016 11:25:08 -0700 Subject: [erlang-questions] Reconnect to nodes In-Reply-To: References: Message-ID: <571A6C84.80407@gmail.com> This is in CloudI (in cloudi_core) as shown at http://cloudi.org/api.html#2_nodes_set with reconnect_start and reconnect_delay determining the interval for checking the node connections. If you need help using cloudi_core, that is at https://github.com/CloudI/CloudI/tree/master/examples/hello_world5 . On 04/22/2016 04:42 AM, Roberto Ostinelli wrote: > Thank you both for confirming. > Hence I guess some kind of a basic reconnection manager might be helpful here. > > Thanks! > > r. > > On Fri, Apr 22, 2016 at 12:35 PM, Serge Aleynikov > wrote: > > Roberto, > > This is the expected behavior. Note that the nodes will automatically reconnect by default when either node has a process that sends a message to a process on the remote node. This reconnection behavior can be modified by setting the kernel's 'dist_auto_connect' option. > > Other applications (such as mnesia) may require custom recovery from a network split, which is one of the reasons why automatic reconnection may not be desirable. > > Regards, > > Serge > > On Fri, Apr 22, 2016 at 5:39 AM, Roberto Ostinelli > wrote: > > Dear list, > A simple question: am I correct that, when a node is removed because of a net split, you need to have your own application logic to reconnect to it, and nothing in the VM will try doing that for you? > > Let me show you an example. I have two nodes: 1@REDACTED and 2@REDACTED that are connected to each other: > > (1@REDACTED )1> nodes(). > ['2@REDACTED '] > > On node 2 I listen for nodedown events of node 1: > > (2@REDACTED )1> monitor_node('1@REDACTED ', true). > true > > On node 1, I simulate a net splits with the best option I've found until now, i.e suspending the net_kernel process: > > (1@REDACTED )2> sys:suspend(net_kernel). > ok > > After ~60 seconds on node 2 I get: > > =ERROR REPORT==== 22-Apr-2016::11:28:21 === > ** Node '1@REDACTED ' not responding ** > ** Removing (timedout) connection ** > (2@REDACTED )2> flush(). > Shell got {nodedown,'1@REDACTED '} > > Now the two nodes are disconnected: > > (1@REDACTED )3> nodes(). > [] > > (2@REDACTED )3> nodes(). > [] > > Even when I resume the net_kernel process: > > (1@REDACTED )4> sys:resume(net_kernel). > ok > > The nodes do not reconnect: > > (1@REDACTED )5> nodes(). > [] > > I'm ok with this, though I would like to confirm that my understanding is correct. > If so, does everyone just implement some standard connection manager that does only reconnections? > > Thank you, > r. > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Fri Apr 22 20:30:01 2016 From: rexxe98@REDACTED (Andrew Berman) Date: Fri, 22 Apr 2016 18:30:01 +0000 Subject: [erlang-questions] Database abstraction libraries Message-ID: Hey list, Are there any well supported database abstraction libraries out there? I found a couple on Github (sqerl and sumo_db), but I wanted to see if there was one that is popular that people are using in production under heavy load. I'm not looking for an ORM (I had enough bad experience with it using Hibernate in Java). Any experiences? Thanks, Andrew -------------- next part -------------- An HTML attachment was scrubbed... URL: From gumm@REDACTED Fri Apr 22 22:49:29 2016 From: gumm@REDACTED (Jesse Gumm) Date: Fri, 22 Apr 2016 15:49:29 -0500 Subject: [erlang-questions] Database abstraction libraries In-Reply-To: References: Message-ID: Hi Andrew, I'm working on an abstraction library for sql databases. It currently supports mysql (with both mysql-otp and emysql) and postgresql (with epgsql). I have it running in production, but it still has things I'm working on before I would make an official announcement post, but it's at: https://github.com/choptastic/sql_bridge Of the things does that is useful to me: 1) dynamically adding pools for databases using a lookup function. 2) replacement tokens by configuration (for example, if you want to use $1 as a replacement token for queries instead of ? In mysql, just change the configuration. 3) very minimal orm-type features (like converting a map or proplist to an insert statement). 4) retrieving query results in just about any format you want: lists, tuples, maps, proplist, dicts. Early feedback and/or pull requests would be great if this is what you're looking for. -Jesse -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm On Apr 22, 2016 1:30 PM, "Andrew Berman" wrote: > Hey list, > > Are there any well supported database abstraction libraries out there? I > found a couple on Github (sqerl and sumo_db), but I wanted to see if there > was one that is popular that people are using in production under heavy > load. I'm not looking for an ORM (I had enough bad experience with it > using Hibernate in Java). > > Any experiences? > > Thanks, > > Andrew > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gumm@REDACTED Fri Apr 22 22:53:26 2016 From: gumm@REDACTED (Jesse Gumm) Date: Fri, 22 Apr 2016 15:53:26 -0500 Subject: [erlang-questions] Database abstraction libraries In-Reply-To: References: Message-ID: Oh, also, while i it's running in production, my systems are not exactly heavy load, so it's not terribly battle tested in the heavy load department. My layer over the underlying drivers is pretty thin however, so there's not a *ton* of overhead. -- Jesse Gumm Owner, Sigma Star Systems 414.940.4866 || sigma-star.com || @jessegumm On Apr 22, 2016 3:49 PM, "Jesse Gumm" wrote: > Hi Andrew, > > I'm working on an abstraction library for sql databases. It currently > supports mysql (with both mysql-otp and emysql) and postgresql (with > epgsql). > > I have it running in production, but it still has things I'm working on > before I would make an official announcement post, but it's at: > > https://github.com/choptastic/sql_bridge > > Of the things does that is useful to me: > > 1) dynamically adding pools for databases using a lookup function. > 2) replacement tokens by configuration (for example, if you want to use $1 > as a replacement token for queries instead of ? In mysql, just change the > configuration. > 3) very minimal orm-type features (like converting a map or proplist to an > insert statement). > 4) retrieving query results in just about any format you want: lists, > tuples, maps, proplist, dicts. > > Early feedback and/or pull requests would be great if this is what you're > looking for. > > -Jesse > > -- > Jesse Gumm > Owner, Sigma Star Systems > 414.940.4866 || sigma-star.com || @jessegumm > On Apr 22, 2016 1:30 PM, "Andrew Berman" wrote: > >> Hey list, >> >> Are there any well supported database abstraction libraries out there? I >> found a couple on Github (sqerl and sumo_db), but I wanted to see if there >> was one that is popular that people are using in production under heavy >> load. I'm not looking for an ORM (I had enough bad experience with it >> using Hibernate in Java). >> >> Any experiences? >> >> Thanks, >> >> Andrew >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From fernando.benavides@REDACTED Fri Apr 22 22:53:36 2016 From: fernando.benavides@REDACTED (Brujo Benavides) Date: Fri, 22 Apr 2016 17:53:36 -0300 Subject: [erlang-questions] Database abstraction libraries In-Reply-To: References: Message-ID: Andrew, SumoDB latest version (with the different drivers in each individual repo) was never tested in a production system yet, but previous versions have been used in at least 10 systems under heavy load already. It?s basically what we use in every Erlang system we develop for every client we have. It?s one of our most battle-tested libraries so far. Hope this helps :) Cheers! > On Apr 22, 2016, at 17:49, Jesse Gumm wrote: > > Hi Andrew, > > I'm working on an abstraction library for sql databases. It currently supports mysql (with both mysql-otp and emysql) and postgresql (with epgsql). > > I have it running in production, but it still has things I'm working on before I would make an official announcement post, but it's at: > > https://github.com/choptastic/sql_bridge > Of the things does that is useful to me: > > 1) dynamically adding pools for databases using a lookup function. > 2) replacement tokens by configuration (for example, if you want to use $1 as a replacement token for queries instead of ? In mysql, just change the configuration. > 3) very minimal orm-type features (like converting a map or proplist to an insert statement). > 4) retrieving query results in just about any format you want: lists, tuples, maps, proplist, dicts. > > Early feedback and/or pull requests would be great if this is what you're looking for. > > -Jesse > > -- > Jesse Gumm > Owner, Sigma Star Systems > 414.940.4866 || sigma-star.com || @jessegumm > > On Apr 22, 2016 1:30 PM, "Andrew Berman" > wrote: > Hey list, > > Are there any well supported database abstraction libraries out there? I found a couple on Github (sqerl and sumo_db), but I wanted to see if there was one that is popular that people are using in production under heavy load. I'm not looking for an ORM (I had enough bad experience with it using Hibernate in Java). > > Any experiences? > > Thanks, > > Andrew > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Fri Apr 22 22:58:03 2016 From: rexxe98@REDACTED (Andrew Berman) Date: Fri, 22 Apr 2016 20:58:03 +0000 Subject: [erlang-questions] Database abstraction libraries In-Reply-To: References: Message-ID: Cool, I'll check it out. Thanks, Jesse! On Fri, Apr 22, 2016 at 1:53 PM Jesse Gumm wrote: > Oh, also, while i it's running in production, my systems are not exactly > heavy load, so it's not terribly battle tested in the heavy load department. > > My layer over the underlying drivers is pretty thin however, so there's > not a *ton* of overhead. > > -- > Jesse Gumm > Owner, Sigma Star Systems > 414.940.4866 || sigma-star.com || @jessegumm > On Apr 22, 2016 3:49 PM, "Jesse Gumm" wrote: > >> Hi Andrew, >> >> I'm working on an abstraction library for sql databases. It currently >> supports mysql (with both mysql-otp and emysql) and postgresql (with >> epgsql). >> >> I have it running in production, but it still has things I'm working on >> before I would make an official announcement post, but it's at: >> >> https://github.com/choptastic/sql_bridge >> >> Of the things does that is useful to me: >> >> 1) dynamically adding pools for databases using a lookup function. >> 2) replacement tokens by configuration (for example, if you want to use >> $1 as a replacement token for queries instead of ? In mysql, just change >> the configuration. >> 3) very minimal orm-type features (like converting a map or proplist to >> an insert statement). >> 4) retrieving query results in just about any format you want: lists, >> tuples, maps, proplist, dicts. >> >> Early feedback and/or pull requests would be great if this is what you're >> looking for. >> >> -Jesse >> >> -- >> Jesse Gumm >> Owner, Sigma Star Systems >> 414.940.4866 || sigma-star.com || @jessegumm >> On Apr 22, 2016 1:30 PM, "Andrew Berman" wrote: >> >>> Hey list, >>> >>> Are there any well supported database abstraction libraries out there? >>> I found a couple on Github (sqerl and sumo_db), but I wanted to see if >>> there was one that is popular that people are using in production under >>> heavy load. I'm not looking for an ORM (I had enough bad experience with >>> it using Hibernate in Java). >>> >>> Any experiences? >>> >>> Thanks, >>> >>> Andrew >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Fri Apr 22 22:59:05 2016 From: rexxe98@REDACTED (Andrew Berman) Date: Fri, 22 Apr 2016 20:59:05 +0000 Subject: [erlang-questions] Database abstraction libraries In-Reply-To: References: Message-ID: That's good to know, Brujo. That was the lib I was looking at the most since it's pretty much what I'm looking for. Thanks, Andrew On Fri, Apr 22, 2016 at 1:53 PM Brujo Benavides < fernando.benavides@REDACTED> wrote: > Andrew, > > SumoDB latest version (with the different drivers in each individual repo) > was never tested in a production system yet, but previous versions have > been used in at least 10 systems under heavy load already. > It?s basically what we use in every Erlang system we develop for every > client we have. > It?s one of our most battle-tested libraries so far. > Hope this helps :) > Cheers! > > On Apr 22, 2016, at 17:49, Jesse Gumm wrote: > > Hi Andrew, > > I'm working on an abstraction library for sql databases. It currently > supports mysql (with both mysql-otp and emysql) and postgresql (with > epgsql). > > I have it running in production, but it still has things I'm working on > before I would make an official announcement post, but it's at: > > https://github.com/choptastic/sql_bridge > > Of the things does that is useful to me: > > 1) dynamically adding pools for databases using a lookup function. > 2) replacement tokens by configuration (for example, if you want to use $1 > as a replacement token for queries instead of ? In mysql, just change the > configuration. > 3) very minimal orm-type features (like converting a map or proplist to an > insert statement). > 4) retrieving query results in just about any format you want: lists, > tuples, maps, proplist, dicts. > > Early feedback and/or pull requests would be great if this is what you're > looking for. > > -Jesse > > -- > Jesse Gumm > Owner, Sigma Star Systems > 414.940.4866 || sigma-star.com || @jessegumm > On Apr 22, 2016 1:30 PM, "Andrew Berman" wrote: > >> Hey list, >> >> Are there any well supported database abstraction libraries out there? I >> found a couple on Github (sqerl and sumo_db), but I wanted to see if there >> was one that is popular that people are using in production under heavy >> load. I'm not looking for an ORM (I had enough bad experience with it >> using Hibernate in Java). >> >> Any experiences? >> >> Thanks, >> >> Andrew >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Fri Apr 22 23:31:03 2016 From: rexxe98@REDACTED (Andrew Berman) Date: Fri, 22 Apr 2016 21:31:03 +0000 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: <22249.32674.198390.541898@gargle.gargle.HOWL> References: <56D18081.7030307@ninenines.eu> <22237.18270.703914.235613@gargle.gargle.HOWL> <56e82cf8.94a2190a.46e1a.6ecd@mx.google.com> <22249.32674.198390.541898@gargle.gargle.HOWL> Message-ID: Reviving this thread I started. I did decide to stick with Erlang. While I think Elixir has a lot going for it and I'm happy that people are migrating to it, I started to get a bit put off when I saw repos like this: https://github.com/johnotander/is_ur l. That is actually published to Hex, by the way. While Hex might have solved the problem npm had a little while ago, it looks like Elixir is breeding the same mentality of relying on tiny libraries. Also, the Erlang motto of just letting it fail seems to have gotten lost somewhere. Based on the dev's code (and I could be very wrong on this), it doesn't look like Elixir's URI.parse actually throws any error when it cannot parse a valid URI. Even found a stackoverflow on it: http://stackoverflow.com/questions/30696761/check-if-a-url-is-valid-in-elixir. One of my favorite things about Erlang is just letting things fail since it saves me tons of keystrokes! On Wed, Mar 16, 2016 at 8:45 AM Mikael Pettersson wrote: > Torben Hoffmann writes: > > > > > > Mikael Pettersson writes: > > > > > [ text/plain ] > > > Zachary Kessin writes: > > > > I for one would love to see an ML-family language on the beam! > > > > > > I'm working on a Standard ML (SML'97) compiler targeting the > > > Erlang/OTP VM. I hope to have it finished enough to make it > > > public sometime this year. > > > > I have been thinking about this a bit, so I'll throw in some questions. > > > > What kind of type system are you using? > > The SML'97 one, with minimal extensions for interfacing with Erlang > (see below), and possibly fixes from the Successor-ML effort. > > > I have been told that a type-and-effect system would be the way to > > capture the effects of statements on the mailboxes. > > > > Have you given any thought to upgrades? > > > > The problem is that an upgrade can change the types, so that leads to > > some thinking about versioning the types. Tricky business. > > Without some attention to this it will be hard to evolve a distributed > > system. > > Yes. Basically, we'd need to redo type checking before allowing calls > to or from a dynamically upgraded module. Unfortunately the VM doesn't > give us a hook for that. I'm still thinking about the best way to > address this. Initially I'll ignore the problem. > > In fact, there isn't even a cast-in-stone mapping between SML and Erlang > modules. While I could compile each functor or top-level structure to > a module (Erlang-like), I could also compile entire applications to > single modules (at least two SML implementations already do that). > > > And then there is interfacing to the external world. > > My instinct tells me that it would make sense to attach some sort of > > adapter code that will have the type signature > > val adapt: binary -> some_type > > My plan is to support something like typecase on values of type dynamic, > as I've seen other functional languages do before. That's enough to > enable safe import of data from the dynamically typed world. > > > Which leads me to... how will you describe binary pattern matching? > > Binary matching is already statically typed, so I don't see any > problem there. binary_to_term would have result type dynamic. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Oliver.Korpilla@REDACTED Fri Apr 22 23:48:02 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Fri, 22 Apr 2016 23:48:02 +0200 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: <56D18081.7030307@ninenines.eu> <22237.18270.703914.235613@gargle.gargle.HOWL> <56e82cf8.94a2190a.46e1a.6ecd@mx.google.com> <22249.32674.198390.541898@gargle.gargle.HOWL>, Message-ID: Hi. I'm currently pursueing Elixir at work, and find that the elegance and power of its language core is something which I do appreciate a lot. Elixir's community seems to be still small but growing. The core libraries are sound, and Erlang OTP is always just one call away when needed which is arguably one of its biggest advantages and definitely one any language could wish for. I find the idea that libraries that have been published so far outside the language core are somehow a defining feature of Elixir a bit contrived. So, is there something to be criticized with the language itself, its design philosophy, or with its libraries? So far, I could only wish for a bigger core library and more documentation and examples of its trove of features. I do agree that packages out there can be hard to get support for. I would say that this actually poses an argument for expanding the community, bringing people with experience in that might have another mindset. As far as I see it the community is still in flux and it is too early to see where it will go. Oliver ? Gesendet:?Freitag, 22. April 2016 um 23:31 Uhr Von:?"Andrew Berman" An:?"Mikael Pettersson" , "Torben Hoffmann" Cc:?erlang-questions Betreff:?Re: [erlang-questions] Any Erlang Devs Contemplating Elixir? Reviving this thread I started.? I did decide to stick with Erlang.? While I think Elixir has a lot going for it and I'm happy that people are migrating to it, I started to get a bit put off when I saw repos like this:?https://github.com/johnotander/is_ur[https://github.com/johnotander/is_url]l.? That is actually published to Hex, by the way. ?While Hex might have solved the problem npm had a little while ago, it looks like Elixir is breeding the same mentality of relying on tiny libraries.? Also, the Erlang motto of just letting it fail seems to have gotten lost somewhere.? Based on the dev's code (and I could be very wrong on this), it doesn't look like Elixir's URI.parse actually throws any error when it cannot parse a valid URI.? Even found a stackoverflow on it:?http://stackoverflow.com/questions/30696761/check-if-a-url-is-valid-in-elixir[http://stackoverflow.com/questions/30696761/check-if-a-url-is-valid-in-elixir].? One of my favorite things about Erlang is just letting things fail since it saves me tons of keystrokes! ? ?? On Wed, Mar 16, 2016 at 8:45 AM Mikael Pettersson wrote:Torben Hoffmann writes: ?> ?> ?> Mikael Pettersson writes: ?> ?> > [ text/plain ] ?> > Zachary Kessin writes: ?> >? > I for one would love to see an ML-family language on the beam! ?> > ?> > I'm working on a Standard ML (SML'97) compiler targeting the ?> > Erlang/OTP VM.? I hope to have it finished enough to make it ?> > public sometime this year. ?> ?> I have been thinking about this a bit, so I'll throw in some questions. ?> ?> What kind of type system are you using? The SML'97 one, with minimal extensions for interfacing with Erlang (see below), and possibly fixes from the Successor-ML effort. ?> I have been told that a type-and-effect system would be the way to ?> capture the effects of statements on the mailboxes. ?> ?> Have you given any thought to upgrades? ?> ?> The problem is that an upgrade can change the types, so that leads to ?> some thinking about versioning the types. Tricky business. ?> Without some attention to this it will be hard to evolve a distributed ?> system. Yes.? Basically, we'd need to redo type checking before allowing calls to or from a dynamically upgraded module.? Unfortunately the VM doesn't give us a hook for that.? I'm still thinking about the best way to address this.? Initially I'll ignore the problem. In fact, there isn't even a cast-in-stone mapping between SML and Erlang modules.? While I could compile each functor or top-level structure to a module (Erlang-like), I could also compile entire applications to single modules (at least two SML implementations already do that). ?> And then there is interfacing to the external world. ?> My instinct tells me that it would make sense to attach some sort of ?> adapter code that will have the type signature ?> val adapt: binary -> some_type My plan is to support something like typecase on values of type dynamic, as I've seen other functional languages do before.? That's enough to enable safe import of data from the dynamically typed world. ?> Which leads me to... how will you describe binary pattern matching? Binary matching is already statically typed, so I don't see any problem there.? binary_to_term would have result type dynamic. _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED[erlang-questions@REDACTED] http://erlang.org/mailman/listinfo/erlang-questions_______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] From michal@REDACTED Sat Apr 23 00:15:10 2016 From: michal@REDACTED (=?UTF-8?B?TWljaGHFgiBNdXNrYcWCYQ==?=) Date: Sat, 23 Apr 2016 00:15:10 +0200 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: <56D18081.7030307@ninenines.eu> <22237.18270.703914.235613@gargle.gargle.HOWL> <56e82cf8.94a2190a.46e1a.6ecd@mx.google.com> <22249.32674.198390.541898@gargle.gargle.HOWL> Message-ID: Andrew, while there are many factors in play, I find the idea of deciding on a merits of whole language based on a package that over 6 months was downloaded 22 times appalling. I'm not sure you can go anywhere near stating that such a library is representative of any ecosystem. Micha?. 2016-04-22 23:31 GMT+02:00 Andrew Berman : > Reviving this thread I started. I did decide to stick with Erlang. While I > think Elixir has a lot going for it and I'm happy that people are migrating > to it, I started to get a bit put off when I saw repos like this: > https://github.com/johnotander/is_url. That is actually published to Hex, > by the way. While Hex might have solved the problem npm had a little while > ago, it looks like Elixir is breeding the same mentality of relying on tiny > libraries. Also, the Erlang motto of just letting it fail seems to have > gotten lost somewhere. Based on the dev's code (and I could be very wrong > on this), it doesn't look like Elixir's URI.parse actually throws any error > when it cannot parse a valid URI. Even found a stackoverflow on it: > http://stackoverflow.com/questions/30696761/check-if-a-url-is-valid-in-elixir. > One of my favorite things about Erlang is just letting things fail since it > saves me tons of keystrokes! > > > > On Wed, Mar 16, 2016 at 8:45 AM Mikael Pettersson > wrote: >> >> Torben Hoffmann writes: >> > >> > >> > Mikael Pettersson writes: >> > >> > > [ text/plain ] >> > > Zachary Kessin writes: >> > > > I for one would love to see an ML-family language on the beam! >> > > >> > > I'm working on a Standard ML (SML'97) compiler targeting the >> > > Erlang/OTP VM. I hope to have it finished enough to make it >> > > public sometime this year. >> > >> > I have been thinking about this a bit, so I'll throw in some questions. >> > >> > What kind of type system are you using? >> >> The SML'97 one, with minimal extensions for interfacing with Erlang >> (see below), and possibly fixes from the Successor-ML effort. >> >> > I have been told that a type-and-effect system would be the way to >> > capture the effects of statements on the mailboxes. >> > >> > Have you given any thought to upgrades? >> > >> > The problem is that an upgrade can change the types, so that leads to >> > some thinking about versioning the types. Tricky business. >> > Without some attention to this it will be hard to evolve a distributed >> > system. >> >> Yes. Basically, we'd need to redo type checking before allowing calls >> to or from a dynamically upgraded module. Unfortunately the VM doesn't >> give us a hook for that. I'm still thinking about the best way to >> address this. Initially I'll ignore the problem. >> >> In fact, there isn't even a cast-in-stone mapping between SML and Erlang >> modules. While I could compile each functor or top-level structure to >> a module (Erlang-like), I could also compile entire applications to >> single modules (at least two SML implementations already do that). >> >> > And then there is interfacing to the external world. >> > My instinct tells me that it would make sense to attach some sort of >> > adapter code that will have the type signature >> > val adapt: binary -> some_type >> >> My plan is to support something like typecase on values of type dynamic, >> as I've seen other functional languages do before. That's enough to >> enable safe import of data from the dynamically typed world. >> >> > Which leads me to... how will you describe binary pattern matching? >> >> Binary matching is already statically typed, so I don't see any >> problem there. binary_to_term would have result type dynamic. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From kennethlakin@REDACTED Sat Apr 23 00:27:11 2016 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Fri, 22 Apr 2016 15:27:11 -0700 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: <56D18081.7030307@ninenines.eu> <22237.18270.703914.235613@gargle.gargle.HOWL> <56e82cf8.94a2190a.46e1a.6ecd@mx.google.com> <22249.32674.198390.541898@gargle.gargle.HOWL> Message-ID: <571AA53F.2090106@gmail.com> On 04/22/2016 02:31 PM, Andrew Berman wrote: > Also, the Erlang motto of just letting it > fail seems to have gotten lost somewhere. In regards to is_url, Erlang's is_integer doesn't throw when passed a non-integer. It would be kind of disastrous if it did. If you want is_url to throw when passed something that's not a URL, then I expect that you can do true=IsUrl.validate(url); > ... it doesn't look like Elixir's URI.parse > actually throws any error when it cannot parse a valid URI. Elixir's URI.parse _appears_ to behave in exactly the same way that Python3's urllib.parse.urlparse does. The problem here (if there is one) is that it kind of seems like both libs use RFC1808's grammar for parsing URLs rather than one of the more strict (and non-obsolete!) RFCs. Note that section 2.2 of RFC 1808 defines a URL as "( absoluteURL | relativeURL ) [ "#" fragment ]", so the only possible *failure* condition that I see is passing a thing that's not a string. URI.parse throws when passed something that's not a string, so that seems to be in order. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From rexxe98@REDACTED Sat Apr 23 00:41:53 2016 From: rexxe98@REDACTED (Andrew Berman) Date: Fri, 22 Apr 2016 22:41:53 +0000 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: <56D18081.7030307@ninenines.eu> <22237.18270.703914.235613@gargle.gargle.HOWL> <56e82cf8.94a2190a.46e1a.6ecd@mx.google.com> <22249.32674.198390.541898@gargle.gargle.HOWL> Message-ID: Totally agree. Very happy that it exists and it's growing. On Fri, Apr 22, 2016 at 2:52 PM Shane Utt wrote: > I have tried Elixir but have not felt the need to use it for anything over > Erlang, because all of my current needs are met. > > The two main compelling reasons I have found for using Elixir are however: > > * People I've shown it to (who don't work on Erlang projects) seem to be > more receptive towards the Elixir syntax > * Phoenix and Ecto > > I personally think its a good thing and a sign of health for N languages > to be sharing Beam, and so I welcome Elixir and I follow its progress > despite not needing it at the moment. > > If I was to use Elixir for future projects, I expect it would be because > of a desire to implement a new application using Pheonix. > > > On Fri, Apr 22, 2016 at 5:31 PM, Andrew Berman wrote: > >> Reviving this thread I started. I did decide to stick with Erlang. >> While I think Elixir has a lot going for it and I'm happy that people are >> migrating to it, I started to get a bit put off when I saw repos like this: >> https://github.com/johnotander/is_ur >> l. That is actually published to >> Hex, by the way. While Hex might have solved the problem npm had a >> little while ago, it looks like Elixir is breeding the same mentality of >> relying on tiny libraries. Also, the Erlang motto of just letting it fail >> seems to have gotten lost somewhere. Based on the dev's code (and I could >> be very wrong on this), it doesn't look like Elixir's URI.parse actually >> throws any error when it cannot parse a valid URI. Even found a >> stackoverflow on it: >> http://stackoverflow.com/questions/30696761/check-if-a-url-is-valid-in-elixir. >> One of my favorite things about Erlang is just letting things fail since it >> saves me tons of keystrokes! >> >> >> >> On Wed, Mar 16, 2016 at 8:45 AM Mikael Pettersson >> wrote: >> >>> Torben Hoffmann writes: >>> > >>> > >>> > Mikael Pettersson writes: >>> > >>> > > [ text/plain ] >>> > > Zachary Kessin writes: >>> > > > I for one would love to see an ML-family language on the beam! >>> > > >>> > > I'm working on a Standard ML (SML'97) compiler targeting the >>> > > Erlang/OTP VM. I hope to have it finished enough to make it >>> > > public sometime this year. >>> > >>> > I have been thinking about this a bit, so I'll throw in some >>> questions. >>> > >>> > What kind of type system are you using? >>> >>> The SML'97 one, with minimal extensions for interfacing with Erlang >>> (see below), and possibly fixes from the Successor-ML effort. >>> >>> > I have been told that a type-and-effect system would be the way to >>> > capture the effects of statements on the mailboxes. >>> > >>> > Have you given any thought to upgrades? >>> > >>> > The problem is that an upgrade can change the types, so that leads to >>> > some thinking about versioning the types. Tricky business. >>> > Without some attention to this it will be hard to evolve a distributed >>> > system. >>> >>> Yes. Basically, we'd need to redo type checking before allowing calls >>> to or from a dynamically upgraded module. Unfortunately the VM doesn't >>> give us a hook for that. I'm still thinking about the best way to >>> address this. Initially I'll ignore the problem. >>> >>> In fact, there isn't even a cast-in-stone mapping between SML and Erlang >>> modules. While I could compile each functor or top-level structure to >>> a module (Erlang-like), I could also compile entire applications to >>> single modules (at least two SML implementations already do that). >>> >>> > And then there is interfacing to the external world. >>> > My instinct tells me that it would make sense to attach some sort of >>> > adapter code that will have the type signature >>> > val adapt: binary -> some_type >>> >>> My plan is to support something like typecase on values of type dynamic, >>> as I've seen other functional languages do before. That's enough to >>> enable safe import of data from the dynamically typed world. >>> >>> > Which leads me to... how will you describe binary pattern matching? >>> >>> Binary matching is already statically typed, so I don't see any >>> problem there. binary_to_term would have result type dynamic. >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Sat Apr 23 00:56:11 2016 From: rexxe98@REDACTED (Andrew Berman) Date: Fri, 22 Apr 2016 22:56:11 +0000 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: <571AA53F.2090106@gmail.com> References: <56D18081.7030307@ninenines.eu> <22237.18270.703914.235613@gargle.gargle.HOWL> <56e82cf8.94a2190a.46e1a.6ecd@mx.google.com> <22249.32674.198390.541898@gargle.gargle.HOWL> <571AA53F.2090106@gmail.com> Message-ID: No, I wasn't saying that is_uri should throw an error, it most certainly shouldn't. Anytime I see is_foo, I expect some sort of boolean to answer the question of is it a foo. In Elixir when I do URI.parse(""), I get no error and everything returned is nil. Why? It's clearly not a valid URI, and the parse function is happy consuming it. Just seems like an odd choice to me. At least the docs for parse do say it expects a valid URI and doesn't validate. On Fri, Apr 22, 2016 at 3:27 PM Kenneth Lakin wrote: > On 04/22/2016 02:31 PM, Andrew Berman wrote: > > Also, the Erlang motto of just letting it > > fail seems to have gotten lost somewhere. > > In regards to is_url, Erlang's is_integer doesn't throw when passed a > non-integer. It would be kind of disastrous if it did. If you want > is_url to throw when passed something that's not a URL, then I expect > that you can do > > true=IsUrl.validate(url); > > > ... it doesn't look like Elixir's URI.parse > > actually throws any error when it cannot parse a valid URI. > > Elixir's URI.parse _appears_ to behave in exactly the same way that > Python3's urllib.parse.urlparse does. The problem here (if there is one) > is that it kind of seems like both libs use RFC1808's grammar for > parsing URLs rather than one of the more strict (and non-obsolete!) RFCs. > > Note that section 2.2 of RFC 1808 defines a URL as "( absoluteURL | > relativeURL ) [ "#" fragment ]", so the only possible *failure* > condition that I see is passing a thing that's not a string. URI.parse > throws when passed something that's not a string, so that seems to be in > order. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rexxe98@REDACTED Sat Apr 23 01:02:04 2016 From: rexxe98@REDACTED (Andrew Berman) Date: Fri, 22 Apr 2016 23:02:04 +0000 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: <56D18081.7030307@ninenines.eu> <22237.18270.703914.235613@gargle.gargle.HOWL> <56e82cf8.94a2190a.46e1a.6ecd@mx.google.com> <22249.32674.198390.541898@gargle.gargle.HOWL> Message-ID: Ha, no, I didn't make the decision based on that repo, that was merely a quick example. I find it so funny when people get defensive about a programming language. Believe me, Erlang has tons of issues too! On Fri, Apr 22, 2016 at 3:15 PM Micha? Muska?a wrote: > Andrew, > > while there are many factors in play, I find the idea of deciding on a > merits of whole language based on a package that over 6 months was > downloaded 22 times appalling. > I'm not sure you can go anywhere near stating that such a library is > representative of any ecosystem. > > Micha?. > > 2016-04-22 23:31 GMT+02:00 Andrew Berman : > > Reviving this thread I started. I did decide to stick with Erlang. > While I > > think Elixir has a lot going for it and I'm happy that people are > migrating > > to it, I started to get a bit put off when I saw repos like this: > > https://github.com/johnotander/is_url. That is actually published to > Hex, > > by the way. While Hex might have solved the problem npm had a little > while > > ago, it looks like Elixir is breeding the same mentality of relying on > tiny > > libraries. Also, the Erlang motto of just letting it fail seems to have > > gotten lost somewhere. Based on the dev's code (and I could be very > wrong > > on this), it doesn't look like Elixir's URI.parse actually throws any > error > > when it cannot parse a valid URI. Even found a stackoverflow on it: > > > http://stackoverflow.com/questions/30696761/check-if-a-url-is-valid-in-elixir > . > > One of my favorite things about Erlang is just letting things fail since > it > > saves me tons of keystrokes! > > > > > > > > On Wed, Mar 16, 2016 at 8:45 AM Mikael Pettersson > > wrote: > >> > >> Torben Hoffmann writes: > >> > > >> > > >> > Mikael Pettersson writes: > >> > > >> > > [ text/plain ] > >> > > Zachary Kessin writes: > >> > > > I for one would love to see an ML-family language on the beam! > >> > > > >> > > I'm working on a Standard ML (SML'97) compiler targeting the > >> > > Erlang/OTP VM. I hope to have it finished enough to make it > >> > > public sometime this year. > >> > > >> > I have been thinking about this a bit, so I'll throw in some > questions. > >> > > >> > What kind of type system are you using? > >> > >> The SML'97 one, with minimal extensions for interfacing with Erlang > >> (see below), and possibly fixes from the Successor-ML effort. > >> > >> > I have been told that a type-and-effect system would be the way to > >> > capture the effects of statements on the mailboxes. > >> > > >> > Have you given any thought to upgrades? > >> > > >> > The problem is that an upgrade can change the types, so that leads to > >> > some thinking about versioning the types. Tricky business. > >> > Without some attention to this it will be hard to evolve a > distributed > >> > system. > >> > >> Yes. Basically, we'd need to redo type checking before allowing calls > >> to or from a dynamically upgraded module. Unfortunately the VM doesn't > >> give us a hook for that. I'm still thinking about the best way to > >> address this. Initially I'll ignore the problem. > >> > >> In fact, there isn't even a cast-in-stone mapping between SML and Erlang > >> modules. While I could compile each functor or top-level structure to > >> a module (Erlang-like), I could also compile entire applications to > >> single modules (at least two SML implementations already do that). > >> > >> > And then there is interfacing to the external world. > >> > My instinct tells me that it would make sense to attach some sort of > >> > adapter code that will have the type signature > >> > val adapt: binary -> some_type > >> > >> My plan is to support something like typecase on values of type dynamic, > >> as I've seen other functional languages do before. That's enough to > >> enable safe import of data from the dynamically typed world. > >> > >> > Which leads me to... how will you describe binary pattern matching? > >> > >> Binary matching is already statically typed, so I don't see any > >> problem there. binary_to_term would have result type dynamic. > >> _______________________________________________ > >> erlang-questions mailing list > >> erlang-questions@REDACTED > >> http://erlang.org/mailman/listinfo/erlang-questions > > > > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethlakin@REDACTED Sat Apr 23 01:04:01 2016 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Fri, 22 Apr 2016 16:04:01 -0700 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: <56D18081.7030307@ninenines.eu> <22237.18270.703914.235613@gargle.gargle.HOWL> <56e82cf8.94a2190a.46e1a.6ecd@mx.google.com> <22249.32674.198390.541898@gargle.gargle.HOWL> <571AA53F.2090106@gmail.com> Message-ID: <571AADE1.7040908@gmail.com> On 04/22/2016 03:56 PM, Andrew Berman wrote: > In Elixir when I do URI.parse(""), > I get no error and everything returned is nil. Why? It's clearly not a > valid URI... I don't agree. As I mentioned in my mail, it looks like the URL parser of both Python and Elixir are kinda using RFC 1808's rules for URL validity. My interpretation of section 2.2 indicates that the empty string is a valid relative URL. (A URL can be a relativeURL, which can be a rel_path, which -I think- can be zero characters.) Moreover, if Elixir is doing it wrong, then so is Python. Note that both URL parsers in question are in the standard libraries of their respective languages. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From rexxe98@REDACTED Sat Apr 23 01:05:21 2016 From: rexxe98@REDACTED (Andrew Berman) Date: Fri, 22 Apr 2016 23:05:21 +0000 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: <571AADE1.7040908@gmail.com> References: <56D18081.7030307@ninenines.eu> <22237.18270.703914.235613@gargle.gargle.HOWL> <56e82cf8.94a2190a.46e1a.6ecd@mx.google.com> <22249.32674.198390.541898@gargle.gargle.HOWL> <571AA53F.2090106@gmail.com> <571AADE1.7040908@gmail.com> Message-ID: Valid point. Then why test on is_uri? Every string should come back as true. On Fri, Apr 22, 2016 at 4:04 PM Kenneth Lakin wrote: > On 04/22/2016 03:56 PM, Andrew Berman wrote: > > In Elixir when I do URI.parse(""), > > I get no error and everything returned is nil. Why? It's clearly not a > > valid URI... > > I don't agree. As I mentioned in my mail, it looks like the URL parser > of both Python and Elixir are kinda using RFC 1808's rules for URL > validity. My interpretation of section 2.2 indicates that the empty > string is a valid relative URL. (A URL can be a relativeURL, which can > be a rel_path, which -I think- can be zero characters.) > > Moreover, if Elixir is doing it wrong, then so is Python. Note that both > URL parsers in question are in the standard libraries of their > respective languages. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kennethlakin@REDACTED Sat Apr 23 01:22:10 2016 From: kennethlakin@REDACTED (Kenneth Lakin) Date: Fri, 22 Apr 2016 16:22:10 -0700 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: <56D18081.7030307@ninenines.eu> <22237.18270.703914.235613@gargle.gargle.HOWL> <56e82cf8.94a2190a.46e1a.6ecd@mx.google.com> <22249.32674.198390.541898@gargle.gargle.HOWL> <571AA53F.2090106@gmail.com> <571AADE1.7040908@gmail.com> Message-ID: <571AB222.1080506@gmail.com> On 04/22/2016 04:05 PM, Andrew Berman wrote: > Valid point. Then why test on is_uri? Every string should come back as > true. Perhaps I'm confused, but a glance at the output from the two libraries you mentioned (one in Elixir's standard library and one out of tree and maintained by a third party) indicates that they're using different standards to determine the validity of a URI. (is_url rejects "foo.bar", but Elixir's URI.parse does not.) Regardless, discussion of which RFC to use when determining the validity of a URI seems to me to be pretty off-topic for this list. You're welcome to continue *that* discussion with me off-list. :) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From eric.meadows.jonsson@REDACTED Sat Apr 23 02:54:02 2016 From: eric.meadows.jonsson@REDACTED (=?UTF-8?Q?Eric_Meadows=2DJ=C3=B6nsson?=) Date: Sat, 23 Apr 2016 02:54:02 +0200 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: <56D18081.7030307@ninenines.eu> <22237.18270.703914.235613@gargle.gargle.HOWL> <56e82cf8.94a2190a.46e1a.6ecd@mx.google.com> <22249.32674.198390.541898@gargle.gargle.HOWL> <571AA53F.2090106@gmail.com> <571AADE1.7040908@gmail.com> Message-ID: On Sat, Apr 23, 2016 at 1:05 AM, Andrew Berman wrote: > Valid point. Then why test on is_uri? Every string should come back as > true. > I'd like to reiterate what Micha? Muska?a said. Why are you hung up on what a single, practically unused library is doing and judging an entire language on it? It's strongly recommended to not create packages like is_url, we suggest to only publish packages that provide some minimum value and it's definitely against the spirit of what kind of packages we would like to be pushed to hex.pm. It's not against Hex's code of conduct to create libraries like is_url because there is no practical way to enforce it because the limit you draw would be somewhat arbitrary - how do you determine what's a good or minimum viable package? There are nearly 2000 packages on hex.pm, you are bound to found some that are tiny or of low quality if look hard enough. To me it seems more like someone was trying out how to publish packages and published this one as a test or for fun. It also seems like Kenneth Lakin has a very good explanation of why URI.parse works like it does. Let it crash is just as much a core philosophy in Elixir as it is in Erlang and if you'd look at the rest of the standard library that would be obvious. -- Eric Meadows-J?nsson -------------- next part -------------- An HTML attachment was scrubbed... URL: From shaneutt@REDACTED Fri Apr 22 23:52:45 2016 From: shaneutt@REDACTED (Shane Utt) Date: Fri, 22 Apr 2016 17:52:45 -0400 Subject: [erlang-questions] Any Erlang Devs Contemplating Elixir? In-Reply-To: References: <56D18081.7030307@ninenines.eu> <22237.18270.703914.235613@gargle.gargle.HOWL> <56e82cf8.94a2190a.46e1a.6ecd@mx.google.com> <22249.32674.198390.541898@gargle.gargle.HOWL> Message-ID: I have tried Elixir but have not felt the need to use it for anything over Erlang, because all of my current needs are met. The two main compelling reasons I have found for using Elixir are however: * People I've shown it to (who don't work on Erlang projects) seem to be more receptive towards the Elixir syntax * Phoenix and Ecto I personally think its a good thing and a sign of health for N languages to be sharing Beam, and so I welcome Elixir and I follow its progress despite not needing it at the moment. If I was to use Elixir for future projects, I expect it would be because of a desire to implement a new application using Pheonix. On Fri, Apr 22, 2016 at 5:31 PM, Andrew Berman wrote: > Reviving this thread I started. I did decide to stick with Erlang. While > I think Elixir has a lot going for it and I'm happy that people are > migrating to it, I started to get a bit put off when I saw repos like this: > https://github.com/johnotander/is_ur > l. That is actually published to > Hex, by the way. While Hex might have solved the problem npm had a > little while ago, it looks like Elixir is breeding the same mentality of > relying on tiny libraries. Also, the Erlang motto of just letting it fail > seems to have gotten lost somewhere. Based on the dev's code (and I could > be very wrong on this), it doesn't look like Elixir's URI.parse actually > throws any error when it cannot parse a valid URI. Even found a > stackoverflow on it: > http://stackoverflow.com/questions/30696761/check-if-a-url-is-valid-in-elixir. > One of my favorite things about Erlang is just letting things fail since it > saves me tons of keystrokes! > > > > On Wed, Mar 16, 2016 at 8:45 AM Mikael Pettersson > wrote: > >> Torben Hoffmann writes: >> > >> > >> > Mikael Pettersson writes: >> > >> > > [ text/plain ] >> > > Zachary Kessin writes: >> > > > I for one would love to see an ML-family language on the beam! >> > > >> > > I'm working on a Standard ML (SML'97) compiler targeting the >> > > Erlang/OTP VM. I hope to have it finished enough to make it >> > > public sometime this year. >> > >> > I have been thinking about this a bit, so I'll throw in some questions. >> > >> > What kind of type system are you using? >> >> The SML'97 one, with minimal extensions for interfacing with Erlang >> (see below), and possibly fixes from the Successor-ML effort. >> >> > I have been told that a type-and-effect system would be the way to >> > capture the effects of statements on the mailboxes. >> > >> > Have you given any thought to upgrades? >> > >> > The problem is that an upgrade can change the types, so that leads to >> > some thinking about versioning the types. Tricky business. >> > Without some attention to this it will be hard to evolve a distributed >> > system. >> >> Yes. Basically, we'd need to redo type checking before allowing calls >> to or from a dynamically upgraded module. Unfortunately the VM doesn't >> give us a hook for that. I'm still thinking about the best way to >> address this. Initially I'll ignore the problem. >> >> In fact, there isn't even a cast-in-stone mapping between SML and Erlang >> modules. While I could compile each functor or top-level structure to >> a module (Erlang-like), I could also compile entire applications to >> single modules (at least two SML implementations already do that). >> >> > And then there is interfacing to the external world. >> > My instinct tells me that it would make sense to attach some sort of >> > adapter code that will have the type signature >> > val adapt: binary -> some_type >> >> My plan is to support something like typecase on values of type dynamic, >> as I've seen other functional languages do before. That's enough to >> enable safe import of data from the dynamically typed world. >> >> > Which leads me to... how will you describe binary pattern matching? >> >> Binary matching is already statically typed, so I don't see any >> problem there. binary_to_term would have result type dynamic. >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From eshikafe@REDACTED Sat Apr 23 14:35:46 2016 From: eshikafe@REDACTED (austin aigbe) Date: Sat, 23 Apr 2016 13:35:46 +0100 Subject: [erlang-questions] Is ejabberd/MongooseIM not a good choise for mobile IM? In-Reply-To: References: Message-ID: Hello, Checkout this recent webinar from Erlang Solutions on Building a mobile chat app https://www.youtube.com/watch?v=iRhjzJ5g8H8 Regards, Austin On Wed, Apr 20, 2016 at 3:44 PM, Federico Carrone < federico.carrone@REDACTED> wrote: > I think they are good tools, however xmpp is pretty heavy. You could > implement a chat on top of vernemq (https://github.com/erlio/vernemq) - > an Erlang MQTT message broker - or Cowboy with the websocket module or REST > handler + SSE for pushing events. I have been using Cowboy with websockets > or REST/HTTP +SSE with great success. > > On Thu, Apr 14, 2016 at 10:53 AM Caiyun Deng > wrote: > >> Hi! I want to develop a mobile IM app, i searched something about. >> Because xmpp is heavyweight, is ejabberd/MongooseIM not a good choise >> for mobile IM? >> If you develop a mobile IM today, which tool will you choose? >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Catenacci@REDACTED Sat Apr 23 22:39:27 2016 From: Catenacci@REDACTED (Onorio Catenacci) Date: Sat, 23 Apr 2016 16:39:27 -0400 Subject: [erlang-questions] Elixir vs. Erlang In-Reply-To: References: Message-ID: Hi Andrew, I'm not sure why you bother to announce that you're picking Erlang over Elixir? I don't know why you feel the need to make an announcement but it sounds as if you're looking for others to confirm you've made the right choice. Only you can know what the right choice is in your situation. I don't see Elixir vs Erlang as a zero-sum game. I think it's a shame some developers think of it as a either/or choice. -- Onorio -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Sat Apr 23 22:42:11 2016 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Sun, 24 Apr 2016 08:42:11 +1200 Subject: [erlang-questions] Elixir vs. Erlang In-Reply-To: References: Message-ID: I love Elixir, but I love Erlang, too. In my opinion, there's little enough to justify the change in language, and certainly little enough in the capabilities to even justify giving it thought. The only reason I choose Elixir over Erlang for much of my work is that I prefer mix to Rebar and I like the macro capabilities of Elixir. My 5 cents. Lee On Sun, Apr 24, 2016 at 8:39 AM, Onorio Catenacci wrote: > Hi Andrew, > > I'm not sure why you bother to announce that you're picking Erlang over > Elixir? I don't know why you feel the need to make an announcement but it > sounds as if you're looking for others to confirm you've made the right > choice. Only you can know what the right choice is in your situation. > > I don't see Elixir vs Erlang as a zero-sum game. I think it's a shame > some developers think of it as a either/or choice. > > -- > Onorio > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From akat.metin@REDACTED Sun Apr 24 07:32:31 2016 From: akat.metin@REDACTED (Metin Akat) Date: Sun, 24 Apr 2016 08:32:31 +0300 Subject: [erlang-questions] Elixir vs. Erlang In-Reply-To: References: Message-ID: I've been writing Erlang exclusively for more than 5 years now so my opinion is certainly biased, but here are my 2 cents. Advantages over Erlang (sorted by importance to me): * Ability to define modules with code, instead of with new files * The meta-programming capabilities Disadvantages compared to Erlang (again, sorted by importance to me): * Elixir strongly discourages the use of records. I know records are controversial, but they are "the right choice" for me in lots of situations. They are faster; they get verified during compile time and as a result editors can display errors while you are typing the code. * Elixir gives you enough rope to hang yourself with its modules, imports and aliasing capabilities. They are good ideas per se, but because of these things I find it more difficult to read Elixir code than it is to read Erlang code. On the other hand, Erlang is harder to type, but much easier to read and understand. I tend to prefer the Erlang way. * Elixir will always lag behind with new functionality when new Erlang versions get released. This is normal and usually not a big deal, but certainly a thing to be aware of. Metin On Sat, Apr 23, 2016 at 11:42 PM, Lee Sylvester wrote: > I love Elixir, but I love Erlang, too. In my opinion, there's little > enough to justify the change in language, and certainly little enough in > the capabilities to even justify giving it thought. The only reason I > choose Elixir over Erlang for much of my work is that I prefer mix to Rebar > and I like the macro capabilities of Elixir. > > My 5 cents. > > Lee > > > On Sun, Apr 24, 2016 at 8:39 AM, Onorio Catenacci > wrote: > >> Hi Andrew, >> >> I'm not sure why you bother to announce that you're picking Erlang over >> Elixir? I don't know why you feel the need to make an announcement but it >> sounds as if you're looking for others to confirm you've made the right >> choice. Only you can know what the right choice is in your situation. >> >> I don't see Elixir vs Erlang as a zero-sum game. I think it's a shame >> some developers think of it as a either/or choice. >> >> -- >> Onorio >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Sun Apr 24 07:47:37 2016 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Sun, 24 Apr 2016 17:47:37 +1200 Subject: [erlang-questions] Elixir vs. Erlang In-Reply-To: References: Message-ID: These are all good points. Erlang is a "better" syntax, imho, but then I didn't come from Ruby. I love records and find your point in that regard valid. The rope hanging issue is developer dependent, though, I feel. I will say, that having Ruby developers in the office, I'm more likely to get BEAM used in a project thanks to Elixir. On Sun, Apr 24, 2016 at 5:32 PM, Metin Akat wrote: > I've been writing Erlang exclusively for more than 5 years now so my > opinion is certainly biased, but here are my 2 cents. > > Advantages over Erlang (sorted by importance to me): > > * Ability to define modules with code, instead of with new files > * The meta-programming capabilities > > Disadvantages compared to Erlang (again, sorted by importance to me): > > * Elixir strongly discourages the use of records. I know records are > controversial, but they are "the right choice" for me in lots of > situations. They are faster; they get verified during compile time and as a > result editors can display errors while you are typing the code. > * Elixir gives you enough rope to hang yourself with its modules, imports > and aliasing capabilities. They are good ideas per se, but because of these > things I find it more difficult to read Elixir code than it is to read > Erlang code. On the other hand, Erlang is harder to type, but much easier > to read and understand. I tend to prefer the Erlang way. > * Elixir will always lag behind with new functionality when new Erlang > versions get released. This is normal and usually not a big deal, but > certainly a thing to be aware of. > > > Metin > > > > On Sat, Apr 23, 2016 at 11:42 PM, Lee Sylvester > wrote: > >> I love Elixir, but I love Erlang, too. In my opinion, there's little >> enough to justify the change in language, and certainly little enough in >> the capabilities to even justify giving it thought. The only reason I >> choose Elixir over Erlang for much of my work is that I prefer mix to Rebar >> and I like the macro capabilities of Elixir. >> >> My 5 cents. >> >> Lee >> >> >> On Sun, Apr 24, 2016 at 8:39 AM, Onorio Catenacci >> wrote: >> >>> Hi Andrew, >>> >>> I'm not sure why you bother to announce that you're picking Erlang over >>> Elixir? I don't know why you feel the need to make an announcement but it >>> sounds as if you're looking for others to confirm you've made the right >>> choice. Only you can know what the right choice is in your situation. >>> >>> I don't see Elixir vs Erlang as a zero-sum game. I think it's a shame >>> some developers think of it as a either/or choice. >>> >>> -- >>> Onorio >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sun Apr 24 09:12:02 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 24 Apr 2016 10:12:02 +0300 Subject: [erlang-questions] Support for polling HDD in linux 4.4 Message-ID: There is a news on LWN: http://lwn.net/Articles/663879/ and Ubuntu 16.04 with kernel that seems to support it. We in erlyvideo have to maintain our own "thread pool" inside erlang in top of erlang thread pool because it is very important to have different pools for different devices. All this is very complicated and could be more elegant if disk had async API. Have anyone looked at this polling API? Maybe it is time to implement it in erlang? -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric.pailleau@REDACTED Sun Apr 24 10:29:22 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Sun, 24 Apr 2016 10:29:22 +0200 Subject: [erlang-questions] Elixir vs. Erlang In-Reply-To: References: Message-ID: <7vtjvs3tcs1890o3k326eimb.1461486562199@email.android.com> Hi, I m tired of such versus. Looks like Elixir guys wants to justify their choice and try to convince Erlang guys they're right. To me, Elixir is a potion to let non functional programmers get into Erlang ecosystem. I do not use it because I started Erlang first and love it. No offence. I would say instead : "to Beam or not to Beam... That is the question." Regards "Envoy? depuis mon mobile " Eric ---- Lee Sylvester a ?crit ---- >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From garazdawi@REDACTED Sun Apr 24 10:58:58 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Sun, 24 Apr 2016 10:58:58 +0200 Subject: [erlang-questions] Support for polling HDD in linux 4.4 In-Reply-To: References: Message-ID: Hello, On Sun, Apr 24, 2016 at 9:12 AM, Max Lapshin wrote: > There is a news on LWN: http://lwn.net/Articles/663879/ > and Ubuntu 16.04 with kernel that seems to support it. > > We in erlyvideo have to maintain our own "thread pool" inside erlang in > top of erlang thread pool because it is very important to have different > pools for different devices. > May I ask why you need this? Is it because you don't want work on different devices that accidentally are hashed to the same thread to slow down each other? > All this is very complicated and could be more elegant if disk had async > API. > One of the main reasons that the async file threads exists is because the VM does not know if it is reading from an NFS or not. NFSs are notoriously unreliable and setting fds to non-blocking when writing/reading to it is not a guarantee that the write/read will be non-blocking, at least it wasn't when I last checked a couple of years ago. > Have anyone looked at this polling API? Maybe it is time to implement it > in erlang? > Our current idea is to move the entire file driver to use dirty I/O schedulers. This will get rid of the hashing problem of the current thread pool as the dirty schedulers are load balanced. This is still quite a way off, as we first have to work out some of the oddities of dirty schedulers and also add support for ports to run on them. We may also change our minds and do it some other way if that seems better at that time, for instance the file polling you mention. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From bchesneau@REDACTED Sun Apr 24 11:10:24 2016 From: bchesneau@REDACTED (Benoit Chesneau) Date: Sun, 24 Apr 2016 09:10:24 +0000 Subject: [erlang-questions] Support for polling HDD in linux 4.4 In-Reply-To: References: Message-ID: Related but not totally I was wondering if it couldn't be possible to have custom dirty schedulers so we could start a vm with a custom thread pools to separate usages. For example we could have custom dirty schedulers to handle different loads and still using the dirty nif API. Is there any paper somewhere describing the internals of the dirty nif schedulers? - beno?t On Sun, Apr 24, 2016 at 10:59 AM Lukas Larsson wrote: > Hello, > > On Sun, Apr 24, 2016 at 9:12 AM, Max Lapshin > wrote: > >> There is a news on LWN: http://lwn.net/Articles/663879/ >> and Ubuntu 16.04 with kernel that seems to support it. >> >> We in erlyvideo have to maintain our own "thread pool" inside erlang in >> top of erlang thread pool because it is very important to have different >> pools for different devices. >> > > May I ask why you need this? Is it because you don't want work on > different devices that accidentally are hashed to the same thread to slow > down each other? > > >> All this is very complicated and could be more elegant if disk had async >> API. >> > > One of the main reasons that the async file threads exists is because the > VM does not know if it is reading from an NFS or not. NFSs are notoriously > unreliable and setting fds to non-blocking when writing/reading to it is > not a guarantee that the write/read will be non-blocking, at least it > wasn't when I last checked a couple of years ago. > > >> Have anyone looked at this polling API? Maybe it is time to implement it >> in erlang? >> > > Our current idea is to move the entire file driver to use dirty I/O > schedulers. This will get rid of the hashing problem of the current thread > pool as the dirty schedulers are load balanced. This is still quite a way > off, as we first have to work out some of the oddities of dirty schedulers > and also add support for ports to run on them. > > We may also change our minds and do it some other way if that seems better > at that time, for instance the file polling you mention. > > Lukas > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jose.valim@REDACTED Sun Apr 24 11:25:23 2016 From: jose.valim@REDACTED (=?UTF-8?Q?Jos=C3=A9_Valim?=) Date: Sun, 24 Apr 2016 11:25:23 +0200 Subject: [erlang-questions] Elixir vs. Erlang In-Reply-To: <7vtjvs3tcs1890o3k326eimb.1461486562199@email.android.com> References: <7vtjvs3tcs1890o3k326eimb.1461486562199@email.android.com> Message-ID: > I m tired of such versus. > Me too. However, for better or worse, I think we will still see a lot of those. Which is why your last quote is great, in my opinion: > "to Beam or not to Beam... That is the question." > -- *Jos? Valim* www.plataformatec.com.br Skype: jv.ptec Founder and Director of R&D -------------- next part -------------- An HTML attachment was scrubbed... URL: From michal@REDACTED Sun Apr 24 11:54:10 2016 From: michal@REDACTED (=?UTF-8?B?TWljaGHFgiBNdXNrYcWCYQ==?=) Date: Sun, 24 Apr 2016 11:54:10 +0200 Subject: [erlang-questions] Elixir vs. Erlang In-Reply-To: <7vtjvs3tcs1890o3k326eimb.1461486562199@email.android.com> References: <7vtjvs3tcs1890o3k326eimb.1461486562199@email.android.com> Message-ID: > I m tired of such versus. I fully agree. I don't think the real question is Erlang or Elixir. The real question, as ?ric said is "to Beam or not to Beam". Both Erlang and Elixir have merits and both have faults, but neither is going away any time soon. Every single person using Erlang, Elixir or LFE benefits all of us, we should strive to expand the communities together instead of antagonizing them. I really hope we'll be able to find a way to make it easy to use Elixir libraries from rebar3, so the cross-use of libraries is even simpler. Micha?. From eric.pailleau@REDACTED Sun Apr 24 14:58:06 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Sun, 24 Apr 2016 14:58:06 +0200 Subject: [erlang-questions] Elixir vs. Erlang In-Reply-To: References: <7vtjvs3tcs1890o3k326eimb.1461486562199@email.android.com> Message-ID: Hi, For the quote, you can reuse it, no royalties ;-) The original author won't claim for royalties after 4 hundreds years in a grave... About rebar3, I wanted to precise it is not the alpha and omega. I prefere to use erlang.mk, but try to propose also rebar plugins for my projects. Having a single builder, as well having a single language on Beam is not a good thing. From several points of view come sane emulation, and different ways to solve problems. From one language, from one builder tool come sclerosis. "Envoy? depuis mon mobile " Eric ---- Micha? Muska?a a ?crit ---- >> I m tired of such versus. > >I fully agree. >I don't think the real question is Erlang or Elixir. The real >question, as ?ric said is "to Beam or not to Beam". Both Erlang and >Elixir have merits and both have faults, but neither is going away any >time soon. > >Every single person using Erlang, Elixir or LFE benefits all of us, we >should strive to expand the communities together instead of >antagonizing them. >I really hope we'll be able to find a way to make it easy to use >Elixir libraries from rebar3, so the cross-use of libraries is even >simpler. > >Micha?. -------------- next part -------------- An HTML attachment was scrubbed... URL: From max.lapshin@REDACTED Sun Apr 24 19:07:37 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Sun, 24 Apr 2016 20:07:37 +0300 Subject: [erlang-questions] Support for polling HDD in linux 4.4 In-Reply-To: References: Message-ID: Lukas, we can meet situation, when even SSD is too slow because it is plugged via slow SATA cable that can do only 6 Gbit/s but we want 10 Gbit/s or even 20 More often we have to handle load when there are 10 or 20 HDD (JBOD, so no raid, just 20 mounted devices) and we need to serve data from them. If too many users want data from one disk, this disk blocks. Without any explicit actions all disk requests to this disk are taking all threads from async pool. If we try to use file:read_file at this moment to try to read from SSD, we cannot read it, because all threads from thread pool are blocked on slow HDD. This is why we maintain our own internal queue with timeout per each device and try to send 10-40 simultaneous requests to each device. It is almost impossible to do with LVM or RAID and completely impossible to do with hardware RAID (but hardware raid is not about speed), but such approach is possible with separate HDD. -------------- next part -------------- An HTML attachment was scrubbed... URL: From t6sn7gt@REDACTED Sun Apr 24 17:48:59 2016 From: t6sn7gt@REDACTED (Donald Steven) Date: Sun, 24 Apr 2016 11:48:59 -0400 Subject: [erlang-questions] How to keep adding items to a data structure Message-ID: <571CEAEB.6040202@aim.com> Hi, I'm very new to erlang (though quite comfortable with C), trying to write a program to do some AI composition. Can anyone please suggest a solution to a very basic problem I've been struggling with for days, looking at lists, arrays, and records to no avail: === How do I code the following in erlang (problem example is written in a pseudo-C style): start with an empty music-event list-or-array-or-record for i = 1 to 1000 append new musical-event to the end of the music-event list-or-array-or-record === Thanks so much. Don From kouzan@REDACTED Sun Apr 24 21:13:55 2016 From: kouzan@REDACTED (Antonios Kouzoupis) Date: Sun, 24 Apr 2016 21:13:55 +0200 Subject: [erlang-questions] How to keep adding items to a data structure In-Reply-To: <571CEAEB.6040202@aim.com> References: <571CEAEB.6040202@aim.com> Message-ID: <571D1AF3.3080107@riseup.net> Hi Don, The way you iterate in Erlang and I guess in most functional programming languages is by recursive call. So if you want to add/append some numbers to a list, one way to go is the following: populate(Num) -> populate(Num, []). populate(0, Acc) -> Acc; populate(Num, Acc) -> populate(Num - 1, [Num | Acc]). Now if you call populate(100), you'll get the list [1,...,100] BR, -- ??????? ????????? Antonios Kouzoupis -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: From kouzan@REDACTED Sun Apr 24 22:23:29 2016 From: kouzan@REDACTED (Antonios Kouzoupis) Date: Sun, 24 Apr 2016 22:23:29 +0200 Subject: [erlang-questions] How to keep adding items to a data structure In-Reply-To: <571D270F.3050805@aim.com> References: <571CEAEB.6040202@aim.com> <571D1AF3.3080107@riseup.net> <571D270F.3050805@aim.com> Message-ID: <571D2B41.2090807@riseup.net> On 04/24/2016 10:05 PM, Donald Steven wrote: > Hi Antonios, > > You are kind and generous to help. I'll study this carefully. I will > be wanting to add a list or a record, not just numbers. Is this > possible? > Well I don't know exactly what you want to achieve, but the logic is the same. In [Num | Acc], instead of adding numbers you can add your record/tuple/whatever. You might also need to reverse the list in the edge case, when your counter reach zero. -- ??????? ????????? Antonios Kouzoupis -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: From t6sn7gt@REDACTED Sun Apr 24 22:05:35 2016 From: t6sn7gt@REDACTED (Donald Steven) Date: Sun, 24 Apr 2016 16:05:35 -0400 Subject: [erlang-questions] How to keep adding items to a data structure In-Reply-To: <571D1AF3.3080107@riseup.net> References: <571CEAEB.6040202@aim.com> <571D1AF3.3080107@riseup.net> Message-ID: <571D270F.3050805@aim.com> Hi Antonios, You are kind and generous to help. I'll study this carefully. I will be wanting to add a list or a record, not just numbers. Is this possible? Don On 04/24/2016 03:13 PM, Antonios Kouzoupis wrote: > Hi Don, > > The way you iterate in Erlang and I guess in most functional programming > languages is by recursive call. So if you want to add/append some > numbers to a list, one way to go is the following: > > populate(Num) -> > populate(Num, []). > > populate(0, Acc) -> > Acc; > populate(Num, Acc) -> > populate(Num - 1, [Num | Acc]). > > > Now if you call populate(100), you'll get the list [1,...,100] > > BR, > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From Oliver.Korpilla@REDACTED Sun Apr 24 23:05:39 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Sun, 24 Apr 2016 23:05:39 +0200 Subject: [erlang-questions] How to keep adding items to a data structure In-Reply-To: <571D270F.3050805@aim.com> References: <571CEAEB.6040202@aim.com> <571D1AF3.3080107@riseup.net>, <571D270F.3050805@aim.com> Message-ID: Hello, Donald. Lists are not fixed to one type like arrays in C are. ? [[1,2,3],"hello",5,world] is a valid Erlang expression, even though the list contains another list, a string, a number, and an atom. You might want to read a book like "Programming Erlang" to get into the language. I highly recommend it, it really helped laying down the basics and understand the design principles of the OTP shipped with Erlang. Cheers, Oliver ? Gesendet:?Sonntag, 24. April 2016 um 22:05 Uhr Von:?"Donald Steven" An:?"Antonios Kouzoupis" , erlang-questions@REDACTED Betreff:?Re: [erlang-questions] How to keep adding items to a data structure Hi Antonios, You are kind and generous to help. I'll study this carefully. I will be wanting to add a list or a record, not just numbers. Is this possible? Don On 04/24/2016 03:13 PM, Antonios Kouzoupis wrote: > Hi Don, > > The way you iterate in Erlang and I guess in most functional programming > languages is by recursive call. So if you want to add/append some > numbers to a list, one way to go is the following: > > populate(Num) -> > populate(Num, []). > > populate(0, Acc) -> > Acc; > populate(Num, Acc) -> > populate(Num - 1, [Num | Acc]). > > > Now if you call populate(100), you'll get the list [1,...,100] > > BR, > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions _______________________________________________ erlang-questions mailing list erlang-questions@REDACTED http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] From Oliver.Korpilla@REDACTED Mon Apr 25 00:06:26 2016 From: Oliver.Korpilla@REDACTED (Oliver Korpilla) Date: Mon, 25 Apr 2016 00:06:26 +0200 Subject: [erlang-questions] How to keep adding items to a data structure In-Reply-To: <571D3975.8030205@aim.com> References: <571CEAEB.6040202@aim.com> <571D1AF3.3080107@riseup.net> <571D270F.3050805@aim.com> , <571D3975.8030205@aim.com> Message-ID: Hello, Donald. Like Lisp Erlang is not very performant for joining elements to the end of a list. Adding things to the front is simple, cheap, and has its own syntax: List2 = [ Item | List1]. If you accumulate your list in the wrong order, a highly optimized list:reverse/1 function is available: http://erlang.org/doc/man/lists.html#reverse-1 Also, most semantics you might have in mind would also work for lists with newest entry in front. What you describe below is a problem in any language as long as you don't use some sort of looping construct. A simple process that just waits for new events to learn about might look like this: -module(example). -export([start/0]). start() -> spawn(fun() -> receive_events_loop([]) end). receive_events_loop(EventList) -> receive {query, From} -> From ! {reply, lists:reverse(EventList) }; {add, Entry} -> ? receive_events_loop([ Entry | EventList ]) end. If you start this process, it will loop forever, waiting quietly for events. You would work with it like this: Pid = example:start(). Pid ! {add, hello}. Pid ! {add, world}. Pid ! {query, self()}. You would see that this process does accumulate a list internally while running through its receive loop repeatedly. Nowhere do you need variables of the kind List1, List2, List3, ... You run a loop, but this loop is implemented through recursion. Understanding recursion, list accumulation, and especially tail recursion for implementing loops is key to programming Erlang. (I did not compile my example. I just wrote it for the email. Excuse any mistakes I made.) Hope this helped, Oliver ? Gesendet:?Sonntag, 24. April 2016 um 23:24 Uhr Von:?"Donald Steven" An:?"Oliver Korpilla" Cc:?"Antonios Kouzoupis" , erlang-questions@REDACTED Betreff:?Re: Aw: Re: [erlang-questions] How to keep adding items to a data structure Thanks Oliver, I appreciate your note. I understand that a list can contain different elements, as you've indicated. That's wonderful! And I have that book on order. What I'm so frustrated by is, apparently, in order to extend a list by adding new musical events, that I have to keep creating new lists, like: List1 = [musical-event], List2 = List1 ++ [new-musical-event], List3 = List2 ++ [new-musical-event], List4 = List3 ++ [new-musical-event], etc. all the way to perhaps a few thousand. This is primitive and absurd. I need a better solution, whether it's a list or an array or a record. Don On 04/24/2016 05:05 PM, Oliver Korpilla wrote: > Hello, Donald. > > Lists are not fixed to one type like arrays in C are. > > [[1,2,3],"hello",5,world] is a valid Erlang expression, even though the list contains another list, a string, a number, and an atom. > > You might want to read a book like "Programming Erlang" to get into the language. I highly recommend it, it really helped laying down the basics and understand the design principles of the OTP shipped with Erlang. > > Cheers, > Oliver > > > Gesendet: Sonntag, 24. April 2016 um 22:05 Uhr > Von: "Donald Steven" > An: "Antonios Kouzoupis" , erlang-questions@REDACTED > Betreff: Re: [erlang-questions] How to keep adding items to a data structure > Hi Antonios, > > You are kind and generous to help. I'll study this carefully. I will > be wanting to add a list or a record, not just numbers. Is this possible? > > Don > > On 04/24/2016 03:13 PM, Antonios Kouzoupis wrote: >> Hi Don, >> >> The way you iterate in Erlang and I guess in most functional programming >> languages is by recursive call. So if you want to add/append some >> numbers to a list, one way to go is the following: >> >> populate(Num) -> >> populate(Num, []). >> >> populate(0, Acc) -> >> Acc; >> populate(Num, Acc) -> >> populate(Num - 1, [Num | Acc]). >> >> >> Now if you call populate(100), you'll get the list [1,...,100] >> >> BR, >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions][http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions]] ? From t6sn7gt@REDACTED Sun Apr 24 23:24:05 2016 From: t6sn7gt@REDACTED (Donald Steven) Date: Sun, 24 Apr 2016 17:24:05 -0400 Subject: [erlang-questions] How to keep adding items to a data structure In-Reply-To: References: <571CEAEB.6040202@aim.com> <571D1AF3.3080107@riseup.net> <571D270F.3050805@aim.com> Message-ID: <571D3975.8030205@aim.com> Thanks Oliver, I appreciate your note. I understand that a list can contain different elements, as you've indicated. That's wonderful! And I have that book on order. What I'm so frustrated by is, apparently, in order to extend a list by adding new musical events, that I have to keep creating new lists, like: List1 = [musical-event], List2 = List1 ++ [new-musical-event], List3 = List2 ++ [new-musical-event], List4 = List3 ++ [new-musical-event], etc. all the way to perhaps a few thousand. This is primitive and absurd. I need a better solution, whether it's a list or an array or a record. Don On 04/24/2016 05:05 PM, Oliver Korpilla wrote: > Hello, Donald. > > Lists are not fixed to one type like arrays in C are. > > [[1,2,3],"hello",5,world] is a valid Erlang expression, even though the list contains another list, a string, a number, and an atom. > > You might want to read a book like "Programming Erlang" to get into the language. I highly recommend it, it really helped laying down the basics and understand the design principles of the OTP shipped with Erlang. > > Cheers, > Oliver > > > Gesendet: Sonntag, 24. April 2016 um 22:05 Uhr > Von: "Donald Steven" > An: "Antonios Kouzoupis" , erlang-questions@REDACTED > Betreff: Re: [erlang-questions] How to keep adding items to a data structure > Hi Antonios, > > You are kind and generous to help. I'll study this carefully. I will > be wanting to add a list or a record, not just numbers. Is this possible? > > Don > > On 04/24/2016 03:13 PM, Antonios Kouzoupis wrote: >> Hi Don, >> >> The way you iterate in Erlang and I guess in most functional programming >> languages is by recursive call. So if you want to add/append some >> numbers to a list, one way to go is the following: >> >> populate(Num) -> >> populate(Num, []). >> >> populate(0, Acc) -> >> Acc; >> populate(Num, Acc) -> >> populate(Num - 1, [Num | Acc]). >> >> >> Now if you call populate(100), you'll get the list [1,...,100] >> >> BR, >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] From t6sn7gt@REDACTED Mon Apr 25 00:36:39 2016 From: t6sn7gt@REDACTED (Donald Steven) Date: Sun, 24 Apr 2016 18:36:39 -0400 Subject: [erlang-questions] Counters Message-ID: <571D4A77.7060403@aim.com> Hi all, With the archived help provided by Alpar Juttner and Witold Baryluk, I thought I would share (primarily for other newbees like me), one way to implement counters. Best, Don =================================================================================== %%%-------------------------------------------------------------------------- %%% File : counter.erl, version 1.00 %%% Author : Donald Steven , adapted from code %%% suggested by Alpar Juttner and Witold Baryluk of the Erlang %%% community %%% Purpose : To demonstrate multiple independent counters %%% Created : 24 Apr 2016 %%% Invocation : erlc counter.erl %%% erl -noshell -s counter main -s init stop %%%-------------------------------------------------------------------------- -module(counter). -export([main/0,inc/1,dec/1,current/1,start/1,loop/1]). %%%-------------------------------------------------------------------------- %%% Fun : inc(X), dec(X), current(X) %%% Purpose : Increment, decrement the counter X, and provide its current %%% state %%% Created : 24 Apr 2016 %%%-------------------------------------------------------------------------- inc(X) -> list_to_atom("counter_" ++ [X]) ! inc_counter, ok. dec(X) -> list_to_atom("counter_" ++ [X]) ! dec_counter, ok. current(X) -> list_to_atom("counter_" ++ [X]) ! {get_counter, self()}, receive { counter_value, Cnt } -> Cnt end. %%%-------------------------------------------------------------------------- %%% Fun : start(X) and loop(Counter) %%% Purpose : Start the counter X %%% Created : 24 Apr 2016 %%%-------------------------------------------------------------------------- start(X) -> Pid = spawn(counter,loop,[0]), Name = list_to_atom("counter_" ++ [X]), register(Name, Pid), ok. loop(Counter) -> receive inc_counter -> NewCounter = Counter+1; dec_counter -> NewCounter = Counter-1; {get_counter, Pid} -> Pid ! { counter_value, Counter }, NewCounter = Counter end, loop(NewCounter). %%%-------------------------------------------------------------------------- %%% Fun : main() %%% Purpose : Demonstrate usage %%% Created : 24 Apr 2016 %%%-------------------------------------------------------------------------- main() -> I = 1, % IMPORTANT: these are counter numbers, not values J = 2, % They will be transliterated into "counter_1" and % "counter_2" internal to the various funs start(I), % start counters I and J start(J), % all counters start at 0 inc(I), inc(I), % increment and decrement 'out of order' inc(J), inc(J), inc(J), inc(J), % to verify counter independence inc(I), % results should be counter I = 2 and dec(J), % counter J = 3 dec(I), io:format("Current counter I: ~p~n", [current(I)]), io:format("Current counter J: ~p~n", [current(J)]). From vimal7370@REDACTED Mon Apr 25 07:17:51 2016 From: vimal7370@REDACTED (Vimal Kumar) Date: Mon, 25 Apr 2016 10:47:51 +0530 Subject: [erlang-questions] How to keep adding items to a data structure In-Reply-To: <571D3975.8030205@aim.com> References: <571CEAEB.6040202@aim.com> <571D1AF3.3080107@riseup.net> <571D270F.3050805@aim.com> <571D3975.8030205@aim.com> Message-ID: In Erlang, you add elements to existing list, and finally reverse it. E.g: List1 = [orange], List2 = [mango|List1], List3 = [apple|List2], FinalList = lists:reverse(List3). Result: [orange, mango, apple] If you have thousand items to add, you can either create thousand different NEW variables, OR use a recursive function or whatever depending on your exact requirement. It isn't absurd or primitive. It can look that way when you start out in Erlang, but read the book you ordered and it will all make sense. On Apr 25, 2016 9:25 AM, "Donald Steven" wrote: > Thanks Oliver, I appreciate your note. I understand that a list can > contain different elements, as you've indicated. That's wonderful! And I > have that book on order. What I'm so frustrated by is, apparently, in > order to extend a list by adding new musical events, that I have to keep > creating new lists, like: > > List1 = [musical-event], > List2 = List1 ++ [new-musical-event], > List3 = List2 ++ [new-musical-event], > List4 = List3 ++ [new-musical-event], > etc. > > all the way to perhaps a few thousand. This is primitive and absurd. I > need a better solution, whether it's a list or an array or a record. > > Don > > On 04/24/2016 05:05 PM, Oliver Korpilla wrote: > >> Hello, Donald. >> >> Lists are not fixed to one type like arrays in C are. >> [[1,2,3],"hello",5,world] is a valid Erlang expression, even though the >> list contains another list, a string, a number, and an atom. >> >> You might want to read a book like "Programming Erlang" to get into the >> language. I highly recommend it, it really helped laying down the basics >> and understand the design principles of the OTP shipped with Erlang. >> >> Cheers, >> Oliver >> >> Gesendet: Sonntag, 24. April 2016 um 22:05 Uhr >> Von: "Donald Steven" >> An: "Antonios Kouzoupis" , erlang-questions@REDACTED >> Betreff: Re: [erlang-questions] How to keep adding items to a data >> structure >> Hi Antonios, >> >> You are kind and generous to help. I'll study this carefully. I will >> be wanting to add a list or a record, not just numbers. Is this possible? >> >> Don >> >> On 04/24/2016 03:13 PM, Antonios Kouzoupis wrote: >> >>> Hi Don, >>> >>> The way you iterate in Erlang and I guess in most functional programming >>> languages is by recursive call. So if you want to add/append some >>> numbers to a list, one way to go is the following: >>> >>> populate(Num) -> >>> populate(Num, []). >>> >>> populate(0, Acc) -> >>> Acc; >>> populate(Num, Acc) -> >>> populate(Num - 1, [Num | Acc]). >>> >>> >>> Now if you call populate(100), you'll get the list [1,...,100] >>> >>> BR, >>> >>> >>> >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> >> http://erlang.org/mailman/listinfo/erlang-questions[http://erlang.org/mailman/listinfo/erlang-questions] >> > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bengt.kleberg@REDACTED Mon Apr 25 07:34:10 2016 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 25 Apr 2016 07:34:10 +0200 Subject: [erlang-questions] Counters In-Reply-To: <571D4A77.7060403@aim.com> References: <571D4A77.7060403@aim.com> Message-ID: <571DAC52.7060900@ericsson.com> Greetings, Two things, that I think should be mention, about the code below: The comment They will be transliterated into "counter_1" is wrong. The code suffixes "counter_" with the integer 1, which is not "1", but SOH. Also, it is good to comment each usage of erlang:list_to_atom/1 with an acknowledgment that it has the potential to exhaust memory. Not when use in moderation and definitely not in this case, but as a reminder, lest the problem be forgotten. bengt On 04/25/2016 12:36 AM, Donald Steven wrote: > Hi all, > > With the archived help provided by Alpar Juttner and Witold Baryluk, I > thought I would share (primarily for other newbees like me), one way > to implement counters. > > Best, > > Don > > =================================================================================== > > > %%%-------------------------------------------------------------------------- > > %%% File : counter.erl, version 1.00 > %%% Author : Donald Steven , adapted from > code > %%% suggested by Alpar Juttner and Witold Baryluk of the > Erlang > %%% community > %%% Purpose : To demonstrate multiple independent counters > %%% Created : 24 Apr 2016 > %%% Invocation : erlc counter.erl > %%% erl -noshell -s counter main -s init stop > %%%-------------------------------------------------------------------------- > > > -module(counter). > -export([main/0,inc/1,dec/1,current/1,start/1,loop/1]). > > %%%-------------------------------------------------------------------------- > > %%% Fun : inc(X), dec(X), current(X) > %%% Purpose : Increment, decrement the counter X, and provide its current > %%% state > %%% Created : 24 Apr 2016 > %%%-------------------------------------------------------------------------- > > > inc(X) -> > list_to_atom("counter_" ++ [X]) ! inc_counter, > ok. > > dec(X) -> > list_to_atom("counter_" ++ [X]) ! dec_counter, > ok. > > current(X) -> > list_to_atom("counter_" ++ [X]) ! {get_counter, self()}, > receive > { counter_value, Cnt } -> > Cnt > end. > > %%%-------------------------------------------------------------------------- > > %%% Fun : start(X) and loop(Counter) > %%% Purpose : Start the counter X > %%% Created : 24 Apr 2016 > %%%-------------------------------------------------------------------------- > > > start(X) -> > > Pid = spawn(counter,loop,[0]), > Name = list_to_atom("counter_" ++ [X]), > register(Name, Pid), > ok. > > loop(Counter) -> > receive > inc_counter -> > NewCounter = Counter+1; > dec_counter -> > NewCounter = Counter-1; > {get_counter, Pid} -> > Pid ! { counter_value, Counter }, > NewCounter = Counter > end, > loop(NewCounter). > > %%%-------------------------------------------------------------------------- > > %%% Fun : main() > %%% Purpose : Demonstrate usage > %%% Created : 24 Apr 2016 > %%%-------------------------------------------------------------------------- > > > main() -> > > I = 1, % IMPORTANT: these are counter numbers, not values > J = 2, % They will be transliterated into "counter_1" and > % "counter_2" internal to the various funs > > start(I), % start counters I and J > start(J), % all counters start at 0 > > inc(I), inc(I), % increment and decrement 'out of > order' > inc(J), inc(J), inc(J), inc(J), % to verify counter independence > inc(I), % results should be counter I = 2 and > dec(J), % counter J = 3 > dec(I), > > io:format("Current counter I: ~p~n", [current(I)]), > io:format("Current counter J: ~p~n", [current(J)]). > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From raimo+erlang-questions@REDACTED Mon Apr 25 12:32:36 2016 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Mon, 25 Apr 2016 12:32:36 +0200 Subject: [erlang-questions] How to keep adding items to a data structure In-Reply-To: <571D3975.8030205@aim.com> References: <571CEAEB.6040202@aim.com> <571D1AF3.3080107@riseup.net> <571D270F.3050805@aim.com> <571D3975.8030205@aim.com> Message-ID: <20160425103236.GA8451@erix.ericsson.se> On Sun, Apr 24, 2016 at 05:24:05PM -0400, Donald Steven wrote: > Thanks Oliver, I appreciate your note. I understand that a list can > contain different elements, as you've indicated. That's wonderful! And > I have that book on order. What I'm so frustrated by is, apparently, in > order to extend a list by adding new musical events, that I have to keep > creating new lists, like: > > List1 = [musical-event], > List2 = List1 ++ [new-musical-event], > List3 = List2 ++ [new-musical-event], > List4 = List3 ++ [new-musical-event], > etc. > > all the way to perhaps a few thousand. This is primitive and absurd. I > need a better solution, whether it's a list or an array or a record. Well, if you do it that way you do it the primitive and absurd way. Lists are single linked and that is a feature. You should add to the head of the list since that is the end that is accesible in constant time: List1 = [musical_event], List2 = [new_musical_event|List1], List3 = ... Regarding the names List1, List2, List3, ... that is a consequence of having single assignment variables, which also is a feature. It stems from normal mathematical notation where this is gibberish: I = I + 1 You really mean I_1 = I_0 + 1 where _N often is written as a subscript to indicate that these are different instances of a variable. We programmers then after learning imperative programming such as C get used to '=' meaning a destructive update of a memory location, which is just the kind of low level concept that functional programming wants to avoid talking about. The compiler will figure out that List1 is not used after List 2 = ... and hence not keep that value around so you really just prepend to the same list and end up with just the last list List4. But normally you just prepend one item to the head and then iterate through recursion so you do not need to name that many instances of a variable within one function clause. I hope that helped somewhat. -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From vladdu55@REDACTED Mon Apr 25 13:44:11 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 25 Apr 2016 13:44:11 +0200 Subject: [erlang-questions] EEP site not working Message-ID: Hi, I'm not sure if it's just me, but when I check the EEP web pages, like https://www.erlang.org/erlang-enhancement-proposals/home/eep-0003.html, I get an error because there is a redirection loop between www.erlang.org and erlang.org best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From t6sn7gt@REDACTED Mon Apr 25 12:17:53 2016 From: t6sn7gt@REDACTED (Donald Steven) Date: Mon, 25 Apr 2016 06:17:53 -0400 Subject: [erlang-questions] Counters In-Reply-To: <571DAC52.7060900@ericsson.com> References: <571D4A77.7060403@aim.com> <571DAC52.7060900@ericsson.com> Message-ID: <571DEED1.9090602@aim.com> Thanks Bengt. On 04/25/2016 01:34 AM, Bengt Kleberg wrote: > Greetings, > > Two things, that I think should be mention, about the code below: > The comment > They will be transliterated into "counter_1" > is wrong. The code suffixes "counter_" with the integer 1, which is > not "1", but SOH. > Also, it is good to comment each usage of erlang:list_to_atom/1 with > an acknowledgment that it has the potential to exhaust memory. Not > when use in moderation and definitely not in this case, but as a > reminder, lest the problem be forgotten. > > > bengt > > On 04/25/2016 12:36 AM, Donald Steven wrote: >> Hi all, >> >> With the archived help provided by Alpar Juttner and Witold Baryluk, >> I thought I would share (primarily for other newbees like me), one >> way to implement counters. >> >> Best, >> >> Don >> >> =================================================================================== >> >> >> %%%-------------------------------------------------------------------------- >> >> %%% File : counter.erl, version 1.00 >> %%% Author : Donald Steven , adapted from >> code >> %%% suggested by Alpar Juttner and Witold Baryluk of the >> Erlang >> %%% community >> %%% Purpose : To demonstrate multiple independent counters >> %%% Created : 24 Apr 2016 >> %%% Invocation : erlc counter.erl >> %%% erl -noshell -s counter main -s init stop >> %%%-------------------------------------------------------------------------- >> >> >> -module(counter). >> -export([main/0,inc/1,dec/1,current/1,start/1,loop/1]). >> >> %%%-------------------------------------------------------------------------- >> >> %%% Fun : inc(X), dec(X), current(X) >> %%% Purpose : Increment, decrement the counter X, and provide its >> current >> %%% state >> %%% Created : 24 Apr 2016 >> %%%-------------------------------------------------------------------------- >> >> >> inc(X) -> >> list_to_atom("counter_" ++ [X]) ! inc_counter, >> ok. >> >> dec(X) -> >> list_to_atom("counter_" ++ [X]) ! dec_counter, >> ok. >> >> current(X) -> >> list_to_atom("counter_" ++ [X]) ! {get_counter, self()}, >> receive >> { counter_value, Cnt } -> >> Cnt >> end. >> >> %%%-------------------------------------------------------------------------- >> >> %%% Fun : start(X) and loop(Counter) >> %%% Purpose : Start the counter X >> %%% Created : 24 Apr 2016 >> %%%-------------------------------------------------------------------------- >> >> >> start(X) -> >> >> Pid = spawn(counter,loop,[0]), >> Name = list_to_atom("counter_" ++ [X]), >> register(Name, Pid), >> ok. >> >> loop(Counter) -> >> receive >> inc_counter -> >> NewCounter = Counter+1; >> dec_counter -> >> NewCounter = Counter-1; >> {get_counter, Pid} -> >> Pid ! { counter_value, Counter }, >> NewCounter = Counter >> end, >> loop(NewCounter). >> >> %%%-------------------------------------------------------------------------- >> >> %%% Fun : main() >> %%% Purpose : Demonstrate usage >> %%% Created : 24 Apr 2016 >> %%%-------------------------------------------------------------------------- >> >> >> main() -> >> >> I = 1, % IMPORTANT: these are counter numbers, not values >> J = 2, % They will be transliterated into "counter_1" and >> % "counter_2" internal to the various funs >> >> start(I), % start counters I and J >> start(J), % all counters start at 0 >> >> inc(I), inc(I), % increment and decrement 'out of >> order' >> inc(J), inc(J), inc(J), inc(J), % to verify counter independence >> inc(I), % results should be counter I = 2 >> and >> dec(J), % counter J = 3 >> dec(I), >> >> io:format("Current counter I: ~p~n", [current(I)]), >> io:format("Current counter J: ~p~n", [current(J)]). >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From t6sn7gt@REDACTED Mon Apr 25 12:51:28 2016 From: t6sn7gt@REDACTED (Donald Steven) Date: Mon, 25 Apr 2016 06:51:28 -0400 Subject: [erlang-questions] How to keep adding items to a data structure In-Reply-To: <20160425103236.GA8451@erix.ericsson.se> References: <571CEAEB.6040202@aim.com> <571D1AF3.3080107@riseup.net> <571D270F.3050805@aim.com> <571D3975.8030205@aim.com> <20160425103236.GA8451@erix.ericsson.se> Message-ID: <571DF6B0.4060304@aim.com> Thanks Raimo, that's very helpful. I hadn't realized that the compiler would free up resources. I had this vision of neverending piling up. Best, Don On 04/25/2016 06:32 AM, Raimo Niskanen wrote: > On Sun, Apr 24, 2016 at 05:24:05PM -0400, Donald Steven wrote: >> Thanks Oliver, I appreciate your note. I understand that a list can >> contain different elements, as you've indicated. That's wonderful! And >> I have that book on order. What I'm so frustrated by is, apparently, in >> order to extend a list by adding new musical events, that I have to keep >> creating new lists, like: >> >> List1 = [musical-event], >> List2 = List1 ++ [new-musical-event], >> List3 = List2 ++ [new-musical-event], >> List4 = List3 ++ [new-musical-event], >> etc. >> >> all the way to perhaps a few thousand. This is primitive and absurd. I >> need a better solution, whether it's a list or an array or a record. > Well, if you do it that way you do it the primitive and absurd way. > > Lists are single linked and that is a feature. You should add to the head > of the list since that is the end that is accesible in constant time: > List1 = [musical_event], > List2 = [new_musical_event|List1], > List3 = ... > > Regarding the names List1, List2, List3, ... that is a consequence of > having single assignment variables, which also is a feature. It stems from > normal mathematical notation where this is gibberish: > I = I + 1 > You really mean > I_1 = I_0 + 1 > where _N often is written as a subscript to indicate that these are > different instances of a variable. > > We programmers then after learning imperative programming such as C get > used to '=' meaning a destructive update of a memory location, which is > just the kind of low level concept that functional programming wants to > avoid talking about. > > The compiler will figure out that List1 is not used after List 2 = ... > and hence not keep that value around so you really just prepend to the same > list and end up with just the last list List4. > > But normally you just prepend one item to the head and then iterate through > recursion so you do not need to name that many instances of a variable > within one function clause. > > I hope that helped somewhat. From vicbaz@REDACTED Mon Apr 25 14:35:28 2016 From: vicbaz@REDACTED (vicbaz@REDACTED) Date: Mon, 25 Apr 2016 15:35:28 +0300 Subject: [erlang-questions] Memory consumption in ssl application (OTP-18.3.1) Message-ID: <571E0F10.80507@yandex.ru> Hello, When I run https://github.com/ninenines/cowboy/tree/1.0.4/examples/ssl_hello_world under load (using https://github.com/JoeDog/siege) I found a big difference in memory consumption between OTP-17.5 and OTP-18.3. I know the ssl application has many changes in new versions. Is this memory usage normal now or can be reduced using some ssl application settings? I use OTP-17.5.6.9 and OTP-18.3.1. ================================================================================ Erlang/OTP 17 [erts-6.4.1.6] [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] 1> application:which_applications(). [{ssl_hello_world,"Cowboy Hello World example with SSL.", "1"}, {cowboy,"Small, fast, modular HTTP server.","1.0.4"}, {ranch,"Socket acceptor pool for TCP protocols.","1.2.1"}, {cowlib,"Support library for manipulating Web protocols.", "1.0.2"}, {ssl,"Erlang/OTP SSL application","6.0.1.2"}, {public_key,"Public key infrastructure","0.23"}, {crypto,"CRYPTO","3.5"}, {asn1,"The Erlang ASN1 compiler version 3.0.4","3.0.4"}, {stdlib,"ERTS CXC 138 10","2.4"}, {kernel,"ERTS CXC 138 10","3.2.0.1"}] 2> erlang:memory(). [{total,43913840}, {processes,8036312}, {processes_used,8030744}, {system,35877528}, {atom,470537}, {atom_used,455904}, {binary,1498800}, {code,11074476}, {ets,14516424}] 3> recon_alloc:memory(used). 45519464 4> recon_alloc:memory(usage). 0.7212344918526264 5> recon:proc_window(memory, 10, 500). [{<0.25910.2>,68120, [{current_function,{gen_fsm,loop,7}}, {initial_call,{proc_lib,init_p,5}}]}, {<0.25908.2>,68120, [{current_function,{gen_fsm,loop,7}}, {initial_call,{proc_lib,init_p,5}}]}, {<0.284.0>,15808, [ssl_manager, {current_function,{gen_server,loop,6}}, {initial_call,{proc_lib,init_p,5}}]}, {<0.285.0>,12712, [tls_connection_sup, {current_function,{gen_server,loop,6}}, {initial_call,{proc_lib,init_p,5}}]}, {<0.25909.2>,5888, [{current_function,{gen,do_call,4}}, {initial_call,{cowboy_protocol,init,4}}]}, {<0.324.0>,4888, [{current_function,{prim_inet,accept0,2}}, {initial_call,{ranch_acceptor,loop,3}}]}, {<0.302.0>,4808, [{current_function,{ranch_conns_sup,loop,4}}, {initial_call,{proc_lib,init_p,5}}]}, {<0.330.0>,3016, [{current_function,{prim_inet,accept0,2}}, {initial_call,{ranch_acceptor,loop,3}}]}, {<0.25911.2>,2848, [{current_function,{gen,do_call,4}}, {initial_call,{cowboy_protocol,init,4}}]}, {<0.340.0>,1872, [{current_function,{prim_inet,accept0,2}}, {initial_call,{ranch_acceptor,loop,3}}]}] 6> recon:proc_window(reductions, 10, 500). [{<0.2634.3>,11161, [{current_function,{gen_fsm,loop,7}}, {initial_call,{proc_lib,init_p,5}}]}, {<0.2633.3>,9747, [{current_function,{crypto,int_to_bin_pos,2}}, {initial_call,{proc_lib,init_p,5}}]}, {<0.284.0>,4097, [ssl_manager, {current_function,{gen_server,loop,6}}, {initial_call,{proc_lib,init_p,5}}]}, {<0.285.0>,3314, [tls_connection_sup, {current_function,{gen_server,loop,6}}, {initial_call,{proc_lib,init_p,5}}]}, {<0.302.0>,1003, [{current_function,{ranch_conns_sup,loop,4}}, {initial_call,{proc_lib,init_p,5}}]}, {<0.378.0>,426, [{current_function,{prim_inet,accept0,2}}, {initial_call,{ranch_acceptor,loop,3}}]}, {<0.372.0>,425, [{current_function,{prim_inet,accept0,2}}, {initial_call,{ranch_acceptor,loop,3}}]}, {<0.379.0>,425, [{current_function,{prim_inet,accept0,2}}, {initial_call,{ranch_acceptor,loop,3}}]}, {<0.377.0>,425, [{current_function,{prim_inet,accept0,2}}, {initial_call,{ranch_acceptor,loop,3}}]}, {<0.343.0>,425, [{current_function,{prim_inet,accept0,2}}, {initial_call,{ranch_acceptor,loop,3}}]}] RES in htop ~50-60M ================================================================================ Erlang/OTP 18 [erts-7.3.1] [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] 1> application:which_applications(). [{ssl_hello_world,"Cowboy Hello World example with SSL.", "1"}, {cowboy,"Small, fast, modular HTTP server.","1.0.4"}, {ranch,"Socket acceptor pool for TCP protocols.","1.2.1"}, {cowlib,"Support library for manipulating Web protocols.", "1.0.2"}, {ssl,"Erlang/OTP SSL application","7.3"}, {public_key,"Public key infrastructure","1.1.1"}, {crypto,"CRYPTO","3.6.3"}, {asn1,"The Erlang ASN1 compiler version 4.0.2","4.0.2"}, {recon,"Diagnostic tools for production use","2.3.1"}, {stdlib,"ERTS CXC 138 10","2.8"}, {kernel,"ERTS CXC 138 10","4.2"}] 2> erlang:memory(). [{total,949153560}, {processes,902969888}, {processes_used,902558312}, {system,46183672}, {atom,437761}, {atom_used,432875}, {binary,48320}, {code,11407283}, {ets,883640}] 3> recon_alloc:memory(used). 863194560 4> recon_alloc:memory(usage). 0.8005009562139471 5> recon:proc_window(memory, 10, 500). [{<0.290.0>,94360, [ssl_manager, {current_function,{ssl_manager,handle_cast,2}}, {initial_call,{proc_lib,init_p,5}}]}, {<0.13589.3>,24640, [{current_function,{ssl_manager,validate_session,3}}, {initial_call,{ssl_manager,init_session_validator,1}}]}, {<0.291.0>,12832, [tls_connection_sup, {current_function,{gen_server,loop,6}}, {initial_call,{proc_lib,init_p,5}}]}, {<0.13590.3>,10960, [{current_function,{gen,do_call,4}}, {initial_call,{proc_lib,init_p,5}}]}, {<0.318.0>,7904, [{current_function,{prim_inet,accept0,2}}, {initial_call,{ranch_acceptor,loop,3}}]}, {<0.308.0>,3056, [{current_function,{ranch_conns_sup,loop,4}}, {initial_call,{proc_lib,init_p,5}}]}, {<0.385.0>,1872, [{current_function,{prim_inet,accept0,2}}, {initial_call,{ranch_acceptor,loop,3}}]}, {<0.254.0>,0, [code_server, {current_function,{code_server,loop,1}}, {initial_call,{erlang,apply,2}}]}, {<0.325.0>,0, [{current_function,{prim_inet,accept0,2}}, {initial_call,{ranch_acceptor,loop,3}}]}, {<0.357.0>,0, [{current_function,{prim_inet,accept0,2}}, {initial_call,{ranch_acceptor,loop,3}}]}] 6> recon:proc_window(reductions, 10, 500). [{<0.290.0>,232358, [ssl_manager, {current_function,{gen_server,loop,6}}, {initial_call,{proc_lib,init_p,5}}]}, {<0.291.0>,3058, [tls_connection_sup, {current_function,{gen_server,loop,6}}, {initial_call,{proc_lib,init_p,5}}]}, {<0.308.0>,1129, [{current_function,{ranch_conns_sup,loop,4}}, {initial_call,{proc_lib,init_p,5}}]}, {<0.339.0>,435, [{current_function,{prim_inet,accept0,2}}, {initial_call,{ranch_acceptor,loop,3}}]}, {<0.340.0>,430, [{current_function,{prim_inet,accept0,2}}, {initial_call,{ranch_acceptor,loop,3}}]}, {<0.355.0>,430, [{current_function,{prim_inet,accept0,2}}, {initial_call,{ranch_acceptor,loop,3}}]}, {<0.353.0>,430, [{current_function,{prim_inet,accept0,2}}, {initial_call,{ranch_acceptor,loop,3}}]}, {<0.334.0>,430, [{current_function,{prim_inet,accept0,2}}, {initial_call,{ranch_acceptor,loop,3}}]}, {<0.366.0>,430, [{current_function,{prim_inet,accept0,2}}, {initial_call,{ranch_acceptor,loop,3}}]}, {<0.360.0>,430, [{current_function,{prim_inet,accept0,2}}, {initial_call,{ranch_acceptor,loop,3}}]}] RES in htop ~900-1000M From g@REDACTED Mon Apr 25 15:27:16 2016 From: g@REDACTED (Garrett Smith) Date: Mon, 25 Apr 2016 08:27:16 -0500 Subject: [erlang-questions] Counters In-Reply-To: <571DAC52.7060900@ericsson.com> References: <571D4A77.7060403@aim.com> <571DAC52.7060900@ericsson.com> Message-ID: That's an interesting exercise. I personally wouldn't run that code outside of tinkering/experimentation, primarily because it's not easily used within an OTP supervisory hierarchy. That comes pretty close to a strict rule in Erlang production systems - and in this case in particular you'd want to formally account for your incr processes. It's also unorthodox to use atoms that way, particularly when they can grow unbounded by design (see Begnt's comment about memory). I'm also trying to recollect when I ever needed a general purpose counter. Typically any incrementing values are part of some other process/service state. In that case, it's trivial to maintain a counter internally. There's also ets/dets update_counter, in the event you're keeping track of a large number of counters, or other stats as well and it's convenient to use term storage. On Mon, Apr 25, 2016 at 12:34 AM, Bengt Kleberg wrote: > Greetings, > > Two things, that I think should be mention, about the code below: > The comment > They will be transliterated into "counter_1" > is wrong. The code suffixes "counter_" with the integer 1, which is not "1", > but SOH. > Also, it is good to comment each usage of erlang:list_to_atom/1 with an > acknowledgment that it has the potential to exhaust memory. Not when use in > moderation and definitely not in this case, but as a reminder, lest the > problem be forgotten. > > > bengt > > On 04/25/2016 12:36 AM, Donald Steven wrote: >> >> Hi all, >> >> With the archived help provided by Alpar Juttner and Witold Baryluk, I >> thought I would share (primarily for other newbees like me), one way to >> implement counters. >> >> Best, >> >> Don >> >> >> =================================================================================== >> >> >> %%%-------------------------------------------------------------------------- >> %%% File : counter.erl, version 1.00 >> %%% Author : Donald Steven , adapted from code >> %%% suggested by Alpar Juttner and Witold Baryluk of the >> Erlang >> %%% community >> %%% Purpose : To demonstrate multiple independent counters >> %%% Created : 24 Apr 2016 >> %%% Invocation : erlc counter.erl >> %%% erl -noshell -s counter main -s init stop >> >> %%%-------------------------------------------------------------------------- >> >> -module(counter). >> -export([main/0,inc/1,dec/1,current/1,start/1,loop/1]). >> >> >> %%%-------------------------------------------------------------------------- >> %%% Fun : inc(X), dec(X), current(X) >> %%% Purpose : Increment, decrement the counter X, and provide its current >> %%% state >> %%% Created : 24 Apr 2016 >> >> %%%-------------------------------------------------------------------------- >> >> inc(X) -> >> list_to_atom("counter_" ++ [X]) ! inc_counter, >> ok. >> >> dec(X) -> >> list_to_atom("counter_" ++ [X]) ! dec_counter, >> ok. >> >> current(X) -> >> list_to_atom("counter_" ++ [X]) ! {get_counter, self()}, >> receive >> { counter_value, Cnt } -> >> Cnt >> end. >> >> >> %%%-------------------------------------------------------------------------- >> %%% Fun : start(X) and loop(Counter) >> %%% Purpose : Start the counter X >> %%% Created : 24 Apr 2016 >> >> %%%-------------------------------------------------------------------------- >> >> start(X) -> >> >> Pid = spawn(counter,loop,[0]), >> Name = list_to_atom("counter_" ++ [X]), >> register(Name, Pid), >> ok. >> >> loop(Counter) -> >> receive >> inc_counter -> >> NewCounter = Counter+1; >> dec_counter -> >> NewCounter = Counter-1; >> {get_counter, Pid} -> >> Pid ! { counter_value, Counter }, >> NewCounter = Counter >> end, >> loop(NewCounter). >> >> >> %%%-------------------------------------------------------------------------- >> %%% Fun : main() >> %%% Purpose : Demonstrate usage >> %%% Created : 24 Apr 2016 >> >> %%%-------------------------------------------------------------------------- >> >> main() -> >> >> I = 1, % IMPORTANT: these are counter numbers, not values >> J = 2, % They will be transliterated into "counter_1" and >> % "counter_2" internal to the various funs >> >> start(I), % start counters I and J >> start(J), % all counters start at 0 >> >> inc(I), inc(I), % increment and decrement 'out of >> order' >> inc(J), inc(J), inc(J), inc(J), % to verify counter independence >> inc(I), % results should be counter I = 2 and >> dec(J), % counter J = 3 >> dec(I), >> >> io:format("Current counter I: ~p~n", [current(I)]), >> io:format("Current counter J: ~p~n", [current(J)]). >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mikpelinux@REDACTED Mon Apr 25 15:48:08 2016 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Mon, 25 Apr 2016 15:48:08 +0200 Subject: [erlang-questions] Support for polling HDD in linux 4.4 In-Reply-To: References: Message-ID: <22302.8216.563036.922923@gargle.gargle.HOWL> Max Lapshin writes: > There is a news on LWN: http://lwn.net/Articles/663879/ > and Ubuntu 16.04 with kernel that seems to support it. > > We in erlyvideo have to maintain our own "thread pool" inside erlang in top > of erlang thread pool because it is very important to have different pools > for different devices. > > All this is very complicated and could be more elegant if disk had async > API. > > > Have anyone looked at this polling API? Maybe it is time to implement it > in erlang? I didn't see much of an "API" there, only a sysfs file to toggle, and that's not something we should (and often cannot) do from ERTS. From t6sn7gt@REDACTED Mon Apr 25 21:15:56 2016 From: t6sn7gt@REDACTED (Donald Steven) Date: Mon, 25 Apr 2016 15:15:56 -0400 Subject: [erlang-questions] Clueless am I Message-ID: <571E6CEC.5070305@aim.com> Dear friends, Please forgive me for being clueless. (I'm sure it's because I come to Erlang steeped in C.) Below are 2 very simple programs. Both run: the first one works (but requires all these variables which are meaningless for my purposes) but the second one does not set the array values and I don't understand why not or how to do this. Thank for any help you can provide. Don %% THIS WORKS -module(arraytest). -export([main/0]). main() -> A1 = array:set(0, nextevent(), array:new(20)), A2 = array:set(1, nextevent(), A1), A3 = array:set(2, nextevent(), A2), A4 = array:set(3, nextevent(), A3), A5 = array:set(4, nextevent(), A4), io:format("~nArray: ~p~n", [A5]), io:format("Array size: ~p~n~n", [array:size(A5)]). nextevent() -> [rand:uniform()]. ============================================ %% THIS DOES NOT WORK -module(arraytest). -export([main/0]). main() -> A1 = array:set(0, nextevent(), array:new(20)), array:set(1, nextevent(), A1), array:set(2, nextevent(), A1), array:set(3, nextevent(), A1), array:set(4, nextevent(), A1), io:format("~nArray: ~p~n", [A1]), io:format("Array size: ~p~n~n", [array:size(A1)]). nextevent() -> [rand:uniform()]. From s@REDACTED Mon Apr 25 21:29:46 2016 From: s@REDACTED (Stefan Schmiedl) Date: Mon, 25 Apr 2016 21:29:46 +0200 Subject: [erlang-questions] Clueless am I In-Reply-To: <571E6CEC.5070305@aim.com> References: <571E6CEC.5070305@aim.com> Message-ID: <008101d19f28$d3891660$7a9b4320$@xss.de> Hi Don, these are not the arrays you're looking for. http://erlang.org/doc/man/array.html starts with "Functional, extendible arrays". A bit further down you can find "The representation is not documented and is subject to change without notice." So the array datatype is not the contiguous block of memory you are used to, but a functional data structure. This means that an array, is basically read-only. Mathematical functions are reproducible: You pass in the same arguments, you get the same result. So the array A1 in your non-working example references the original datastructure and your code is equivalent to > A1 = array:set(0, nextevent(), array:new(20)), > A2 = array:set(1, nextevent(), A1), > A3 = array:set(2, nextevent(), A1), > A4 = array:set(3, nextevent(), A1), > A5 = array:set(4, nextevent(), A1), s. > -----Urspr?ngliche Nachricht----- > Von: erlang-questions-bounces@REDACTED [mailto:erlang-questions- > bounces@REDACTED] Im Auftrag von Donald Steven > Gesendet: Montag, 25. April 2016 21:16 > An: erlang-questions@REDACTED > Betreff: [erlang-questions] Clueless am I > > Dear friends, > > Please forgive me for being clueless. (I'm sure it's because I come to Erlang > steeped in C.) Yes. SCNR. Below are 2 very simple programs. Both run: the first one > works (but requires all these variables which are meaningless for my > purposes) but the second one does not set the array values and I don't > understand why not or how to do this. > > Thank for any help you can provide. > > Don > > %% THIS WORKS > > -module(arraytest). > -export([main/0]). > > main() -> > > A1 = array:set(0, nextevent(), array:new(20)), > A2 = array:set(1, nextevent(), A1), > A3 = array:set(2, nextevent(), A2), > A4 = array:set(3, nextevent(), A3), > A5 = array:set(4, nextevent(), A4), > > io:format("~nArray: ~p~n", [A5]), > io:format("Array size: ~p~n~n", [array:size(A5)]). > > nextevent() -> > > [rand:uniform()]. > > ============================================ > > %% THIS DOES NOT WORK > > -module(arraytest). > -export([main/0]). > > main() -> > > A1 = array:set(0, nextevent(), array:new(20)), > array:set(1, nextevent(), A1), > array:set(2, nextevent(), A1), > array:set(3, nextevent(), A1), > array:set(4, nextevent(), A1), > > io:format("~nArray: ~p~n", [A1]), > io:format("Array size: ~p~n~n", [array:size(A1)]). > > nextevent() -> > > [rand:uniform()]. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From hugo@REDACTED Mon Apr 25 21:29:50 2016 From: hugo@REDACTED (Hugo Mills) Date: Mon, 25 Apr 2016 19:29:50 +0000 Subject: [erlang-questions] Clueless am I In-Reply-To: <571E6CEC.5070305@aim.com> References: <571E6CEC.5070305@aim.com> Message-ID: <20160425192950.GH7228@carfax.org.uk> On Mon, Apr 25, 2016 at 03:15:56PM -0400, Donald Steven wrote: > Dear friends, > > Please forgive me for being clueless. (I'm sure it's because I come > to Erlang steeped in C.) Below are 2 very simple programs. Both > run: the first one works (but requires all these variables which are > meaningless for my purposes) but the second one does not set the > array values and I don't understand why not or how to do this. > > Thank for any help you can provide. The thing to remember is that in general Erlang's variables are immutable -- so once you've set the value of a variable, it can't be changed. So functions that modify things don't do so in-place, but instead return a new thing which has the requested modification. So what you're doing in the second example is creating a new array, then creating a new structure which is a modification of that array (with element 0 set), and setting A1 to the result. Then you're making a new thing based on A1 (with element 1 set) and throwing it away. Then you're making a new thing based on A1 (with element 2 set) and throwing it away... At the end of all this, you're using the original A1, which only has element 0 set. The first example works because you're *not* throwing away the modified copy each time, but instead setting it to a new variable, and using that for the next call. So yes, the variables *are* useful, and no, they *aren't* meaningless to you -- they're the sequence of modifications you've made to the array. Hugo. > Don > > %% THIS WORKS > > -module(arraytest). > -export([main/0]). > > main() -> > > A1 = array:set(0, nextevent(), array:new(20)), > A2 = array:set(1, nextevent(), A1), > A3 = array:set(2, nextevent(), A2), > A4 = array:set(3, nextevent(), A3), > A5 = array:set(4, nextevent(), A4), > > io:format("~nArray: ~p~n", [A5]), > io:format("Array size: ~p~n~n", [array:size(A5)]). > > nextevent() -> > > [rand:uniform()]. > > ============================================ > > %% THIS DOES NOT WORK > > -module(arraytest). > -export([main/0]). > > main() -> > > A1 = array:set(0, nextevent(), array:new(20)), > array:set(1, nextevent(), A1), > array:set(2, nextevent(), A1), > array:set(3, nextevent(), A1), > array:set(4, nextevent(), A1), > > io:format("~nArray: ~p~n", [A1]), > io:format("Array size: ~p~n~n", [array:size(A1)]). > > nextevent() -> > > [rand:uniform()]. > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- Hugo Mills | I think that everything darkling says is actually a hugo@REDACTED carfax.org.uk | joke. It's just that we haven't worked out most of http://carfax.org.uk/ | them yet. PGP: E2AB1DE4 | Vashka -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From t6sn7gt@REDACTED Mon Apr 25 21:39:39 2016 From: t6sn7gt@REDACTED (Donald Steven) Date: Mon, 25 Apr 2016 15:39:39 -0400 Subject: [erlang-questions] Clueless am I In-Reply-To: <008101d19f28$d3891660$7a9b4320$@xss.de> References: <571E6CEC.5070305@aim.com> <008101d19f28$d3891660$7a9b4320$@xss.de> Message-ID: <571E727B.30309@aim.com> Stefan and Hugo, Is there any data structure in Erlang that will accept modification? This is quite beyond me as to how to add elements to a data structure. I've hit the same dead end with lists. If one has, for example, a database, and one wants to add a customer, do you really need to have a new (second) database? Don On 04/25/2016 03:29 PM, Stefan Schmiedl wrote: > Hi Don, > > these are not the arrays you're looking for. > > http://erlang.org/doc/man/array.html starts with "Functional, extendible > arrays". > A bit further down you can find "The representation is not documented and is > subject to change without notice." > > So the array datatype is not the contiguous block of memory you are used to, > but a functional data structure. This means that an array, is basically > read-only. > > Mathematical functions are reproducible: You pass in the same arguments, you > get > the same result. So the array A1 in your non-working example references the > original > datastructure and your code is equivalent to > >> A1 = array:set(0, nextevent(), array:new(20)), >> A2 = array:set(1, nextevent(), A1), >> A3 = array:set(2, nextevent(), A1), >> A4 = array:set(3, nextevent(), A1), >> A5 = array:set(4, nextevent(), A1), > s. > >> -----Urspr?ngliche Nachricht----- >> Von: erlang-questions-bounces@REDACTED [mailto:erlang-questions- >> bounces@REDACTED] Im Auftrag von Donald Steven >> Gesendet: Montag, 25. April 2016 21:16 >> An: erlang-questions@REDACTED >> Betreff: [erlang-questions] Clueless am I >> >> Dear friends, >> >> Please forgive me for being clueless. (I'm sure it's because I come to > Erlang >> steeped in C.) > Yes. SCNR. > > Below are 2 very simple programs. Both run: the first one >> works (but requires all these variables which are meaningless for my >> purposes) but the second one does not set the array values and I don't >> understand why not or how to do this. >> >> Thank for any help you can provide. >> >> Don >> >> %% THIS WORKS >> >> -module(arraytest). >> -export([main/0]). >> >> main() -> >> >> A1 = array:set(0, nextevent(), array:new(20)), >> A2 = array:set(1, nextevent(), A1), >> A3 = array:set(2, nextevent(), A2), >> A4 = array:set(3, nextevent(), A3), >> A5 = array:set(4, nextevent(), A4), >> >> io:format("~nArray: ~p~n", [A5]), >> io:format("Array size: ~p~n~n", [array:size(A5)]). >> >> nextevent() -> >> >> [rand:uniform()]. >> >> ============================================ >> >> %% THIS DOES NOT WORK >> >> -module(arraytest). >> -export([main/0]). >> >> main() -> >> >> A1 = array:set(0, nextevent(), array:new(20)), >> array:set(1, nextevent(), A1), >> array:set(2, nextevent(), A1), >> array:set(3, nextevent(), A1), >> array:set(4, nextevent(), A1), >> >> io:format("~nArray: ~p~n", [A1]), >> io:format("Array size: ~p~n~n", [array:size(A1)]). >> >> nextevent() -> >> >> [rand:uniform()]. >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From hugo@REDACTED Mon Apr 25 21:51:39 2016 From: hugo@REDACTED (Hugo Mills) Date: Mon, 25 Apr 2016 19:51:39 +0000 Subject: [erlang-questions] Clueless am I In-Reply-To: <571E727B.30309@aim.com> References: <571E6CEC.5070305@aim.com> <008101d19f28$d3891660$7a9b4320$@xss.de> <571E727B.30309@aim.com> Message-ID: <20160425195139.GI7228@carfax.org.uk> On Mon, Apr 25, 2016 at 03:39:39PM -0400, Donald Steven wrote: > Stefan and Hugo, > > Is there any data structure in Erlang that will accept modification? > This is quite beyond me as to how to add elements to a data > structure. I've hit the same dead end with lists. If one has, for > example, a database, and one wants to add a customer, do you really > need to have a new (second) database? Yes, you do exactly that. :) *However*, most of the main data structures are designed quite carefully to be efficient about modifications. So, for example, adding something to the start of a list is a trivial operation. The result is a new list, but the system doesn't copy the whole of the old list -- lists in Erlang are singly-linked, so adding the new item is just creating a small data structure to hold the new item and pointing its "next" pointer to the original list. So, typically, modifying a large data structure involves making a small change to part of it, and leaving the rest of it unchanged. Data structures are automatically copy-on-write things. You can do this precisely *because* the data structures are immutable: the system can reuse the existing unchanged parts of the data structure because they won't be modified once they're set. This is a feature, not a restriction. It's kind of hard to get used to it at first if you're used to everything being mutable, but it starts making sense once you've used it for a while. You have a variety of different basic data structures to work with: tuples, lists, maps, dicts. These are all efficiently updatable, so creating "a new (second) database" is an operation that is going to be reasonably competitive with modify-in-place both in time and in space used. Hugo. > Don > > On 04/25/2016 03:29 PM, Stefan Schmiedl wrote: > >Hi Don, > > > >these are not the arrays you're looking for. > > > >http://erlang.org/doc/man/array.html starts with "Functional, extendible > >arrays". > >A bit further down you can find "The representation is not documented and is > >subject to change without notice." > > > >So the array datatype is not the contiguous block of memory you are used to, > >but a functional data structure. This means that an array, is basically > >read-only. > > > >Mathematical functions are reproducible: You pass in the same arguments, you > >get > >the same result. So the array A1 in your non-working example references the > >original > >datastructure and your code is equivalent to > > > >> A1 = array:set(0, nextevent(), array:new(20)), > >> A2 = array:set(1, nextevent(), A1), > >> A3 = array:set(2, nextevent(), A1), > >> A4 = array:set(3, nextevent(), A1), > >> A5 = array:set(4, nextevent(), A1), > >s. > > > >>-----Urspr?ngliche Nachricht----- > >>Von: erlang-questions-bounces@REDACTED [mailto:erlang-questions- > >>bounces@REDACTED] Im Auftrag von Donald Steven > >>Gesendet: Montag, 25. April 2016 21:16 > >>An: erlang-questions@REDACTED > >>Betreff: [erlang-questions] Clueless am I > >> > >>Dear friends, > >> > >>Please forgive me for being clueless. (I'm sure it's because I come to > >Erlang > >>steeped in C.) > >Yes. SCNR. > > > > Below are 2 very simple programs. Both run: the first one > >>works (but requires all these variables which are meaningless for my > >>purposes) but the second one does not set the array values and I don't > >>understand why not or how to do this. > >> > >>Thank for any help you can provide. > >> > >>Don > >> > >>%% THIS WORKS > >> > >>-module(arraytest). > >>-export([main/0]). > >> > >>main() -> > >> > >> A1 = array:set(0, nextevent(), array:new(20)), > >> A2 = array:set(1, nextevent(), A1), > >> A3 = array:set(2, nextevent(), A2), > >> A4 = array:set(3, nextevent(), A3), > >> A5 = array:set(4, nextevent(), A4), > >> > >> io:format("~nArray: ~p~n", [A5]), > >> io:format("Array size: ~p~n~n", [array:size(A5)]). > >> > >>nextevent() -> > >> > >> [rand:uniform()]. > >> > >>============================================ > >> > >>%% THIS DOES NOT WORK > >> > >>-module(arraytest). > >>-export([main/0]). > >> > >>main() -> > >> > >> A1 = array:set(0, nextevent(), array:new(20)), > >> array:set(1, nextevent(), A1), > >> array:set(2, nextevent(), A1), > >> array:set(3, nextevent(), A1), > >> array:set(4, nextevent(), A1), > >> > >> io:format("~nArray: ~p~n", [A1]), > >> io:format("Array size: ~p~n~n", [array:size(A1)]). > >> > >>nextevent() -> > >> > >> [rand:uniform()]. > >> > >>_______________________________________________ > >>erlang-questions mailing list > >>erlang-questions@REDACTED > >>http://erlang.org/mailman/listinfo/erlang-questions > -- Hugo Mills | I think that everything darkling says is actually a hugo@REDACTED carfax.org.uk | joke. It's just that we haven't worked out most of http://carfax.org.uk/ | them yet. PGP: E2AB1DE4 | Vashka -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: From s@REDACTED Mon Apr 25 22:11:56 2016 From: s@REDACTED (Stefan Schmiedl) Date: Mon, 25 Apr 2016 22:11:56 +0200 Subject: [erlang-questions] Clueless am I In-Reply-To: <571E727B.30309@aim.com> References: <571E6CEC.5070305@aim.com> <008101d19f28$d3891660$7a9b4320$@xss.de> <571E727B.30309@aim.com> Message-ID: <00ab01d19f2e$b78f7840$26ae68c0$@xss.de> > -----Urspr?ngliche Nachricht----- > Von: Donald Steven [mailto:t6sn7gt@REDACTED] > Gesendet: Montag, 25. April 2016 21:40 > An: Stefan Schmiedl ; Hugo Mills ; erlang- > questions@REDACTED > Betreff: Re: AW: [erlang-questions] Clueless am I > > Stefan and Hugo, > > Is there any data structure in Erlang that will accept modification? don't know, never needed one :-) > This is quite beyond me as to how to add elements to a data structure. > I've hit the same dead end with lists. If one has, for example, a database, > and one wants to add a customer, do you really need to have a new (second) > database? yes ... and no. yes: you get a reference to the new database no: much of the "old" data structure can be reused Lists are actually a good example for this type of thinking. They are made of "cells", which hold the value and a reference to the rest of the list. You keep track of a list by a firm grasp on its first cell. Adding a value to a given list comes down to wrapping the value into a new cell, which references the "old first cell" as its rest. So you're getting a new list on each "change", but all of the "old" data structure is reused. BTW: Suffering a few headaches while getting used to these concepts is not unusual :-) s. From james@REDACTED Tue Apr 26 01:30:17 2016 From: james@REDACTED (James Aimonetti) Date: Mon, 25 Apr 2016 16:30:17 -0700 Subject: [erlang-questions] ETS delete_trap Message-ID: <571EA889.1010400@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 We have a process, basically a gen_server with some AMQP stuff added in (a behaviour we call gen_listener) that is getting a backed up message queue, which obviously kills the VM after a short time. The curious bit that I can't really figure out is the {current_function, {ets,delete_trap,1}} bit from the process_info/1 call (output below). As far as I can tell, that is buried in the VM's C code. The process itself wraps an ETS table and provides a LRU-style cache with AMQP bindings to programmatically flush entries based on AMQP events elsewhere in the system. Any thoughts on what delete_trap is doing and why it appears to dead-lock the process? Or perhaps where to look to mitigate this? > process_info(Pid). [{registered_name,my_process_name}, {current_function,{ets,delete_trap,1}}, {initial_call,{proc_lib,init_p,5}}, {status,runnable}, {message_queue_len,1586057}, {message_queue, [...huge...]}, {links,[<0.3239.0>]}, {dictionary, [{'$ancestors',[whistle_couch_sup,<0.3227.0>]}, {callid,<<"3fc98d435ea14ab7">>}, {'$initial_call',{gen_listener,init,1}}]}, {trap_exit,true}, {error_handler,error_handler}, {priority,normal}, {group_leader,<0.3226.0>}, {total_heap_size,462590525}, {heap_size,145962050}, {stack_size,24}, {reductions,71467274818}, {garbage_collection, [{min_bin_vheap_size,46368}, {min_heap_size,233}, {fullsweep_after,65535}, {minor_gcs,1}]}, {suspending,[]}] - -- James Aimonetti Lead Systems Architect / Impressionable Scallywag "If Dialyzer doesn't care, I don't care" 2600HzPDX | http://2600hz.com sip:james@REDACTED tel:415.886.7905 irc:mc_ @ freenode -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCAAGBQJXHqiJAAoJENTKa+JPXCVglwcH/igh3Jtw0aChB1H/NLcD96fZ pKqs3IN64tUb1pAQd7ia9DZqpIVooYhDn9gXsxDcw1XsOoerQj8ef6/7hp9V13U0 CJpXztiNegLhmqNFosod5vXwj9DkWD+Z0AyEmdq6/kA2msEDWnjw9EPgLUn2e8aI q56DN+5mhorR9f2SW8JpW8KFRsEft4lzAoZwauUhgln0onqEv01SCLU1OY9E7pAd dfhlIGY7bymmWJnieEtM6++nd3f/3zfaRQ2wQ0WfvnWaQ+U8OI9iwGwykS/7AgyK 0PQsK/NOgfcUyRP1pkMalXpf64/GcFQn7EUaWtqPC7JQ+K0+/HxNnhwPa1Qpj00= =/Z74 -----END PGP SIGNATURE----- From fernando.benavides@REDACTED Tue Apr 26 03:23:05 2016 From: fernando.benavides@REDACTED (Brujo Benavides) Date: Mon, 25 Apr 2016 22:23:05 -0300 Subject: [erlang-questions] ETS delete_trap In-Reply-To: <571EA889.1010400@2600hz.com> References: <571EA889.1010400@2600hz.com> Message-ID: <346C283B-7052-44DC-AF66-B62128B15746@inakanetworks.com> Not sure if this will be much help, but just out of curiosity I grepped otp codebase to see if I could find that function, but the only place it pops up is in erl_db.c[1] and I don?t really understand that code. Maybe some devs with c-trained eyes will be more suited to help you, James. [1] https://github.com/erlang/otp/blob/e1489c448b7486cdcfec6a89fea238d88e6ce2f3/erts/emulator/beam/erl_db.c#L3038 > On Apr 25, 2016, at 20:30, James Aimonetti wrote: > > Signed PGP part > We have a process, basically a gen_server with some AMQP stuff added > in (a behaviour we call gen_listener) that is getting a backed up > message queue, which obviously kills the VM after a short time. > > The curious bit that I can't really figure out is the > {current_function, {ets,delete_trap,1}} bit from the process_info/1 > call (output below). As far as I can tell, that is buried in the VM's > C code. > > The process itself wraps an ETS table and provides a LRU-style cache > with AMQP bindings to programmatically flush entries based on AMQP > events elsewhere in the system. > > Any thoughts on what delete_trap is doing and why it appears to > dead-lock the process? Or perhaps where to look to mitigate this? > > > process_info(Pid). > [{registered_name,my_process_name}, > {current_function,{ets,delete_trap,1}}, > {initial_call,{proc_lib,init_p,5}}, > {status,runnable}, > {message_queue_len,1586057}, > {message_queue, [...huge...]}, > {links,[<0.3239.0>]}, > {dictionary, > [{'$ancestors',[whistle_couch_sup,<0.3227.0>]}, > {callid,<<"3fc98d435ea14ab7">>}, > {'$initial_call',{gen_listener,init,1}}]}, > {trap_exit,true}, > {error_handler,error_handler}, > {priority,normal}, > {group_leader,<0.3226.0>}, > {total_heap_size,462590525}, > {heap_size,145962050}, > {stack_size,24}, > {reductions,71467274818}, > {garbage_collection, > [{min_bin_vheap_size,46368}, > {min_heap_size,233}, > {fullsweep_after,65535}, > {minor_gcs,1}]}, > {suspending,[]}] > > -- > James Aimonetti > Lead Systems Architect / Impressionable Scallywag > "If Dialyzer doesn't care, I don't care" > > 2600HzPDX | http://2600hz.com > sip:james@REDACTED > tel:415.886.7905 > irc:mc_ @ freenode > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From fernando.benavides@REDACTED Tue Apr 26 03:24:03 2016 From: fernando.benavides@REDACTED (Brujo Benavides) Date: Mon, 25 Apr 2016 22:24:03 -0300 Subject: [erlang-questions] ETS delete_trap In-Reply-To: <346C283B-7052-44DC-AF66-B62128B15746@inakanetworks.com> References: <571EA889.1010400@2600hz.com> <346C283B-7052-44DC-AF66-B62128B15746@inakanetworks.com> Message-ID: Oh, sorry? now I reread your email? you said that already. My bad. > On Apr 25, 2016, at 22:23, Brujo Benavides wrote: > > Not sure if this will be much help, but just out of curiosity I grepped otp codebase to see if I could find that function, but the only place it pops up is in erl_db.c[1] and I don?t really understand that code. > Maybe some devs with c-trained eyes will be more suited to help you, James. > > [1] https://github.com/erlang/otp/blob/e1489c448b7486cdcfec6a89fea238d88e6ce2f3/erts/emulator/beam/erl_db.c#L3038 > >> On Apr 25, 2016, at 20:30, James Aimonetti wrote: >> >> Signed PGP part >> We have a process, basically a gen_server with some AMQP stuff added >> in (a behaviour we call gen_listener) that is getting a backed up >> message queue, which obviously kills the VM after a short time. >> >> The curious bit that I can't really figure out is the >> {current_function, {ets,delete_trap,1}} bit from the process_info/1 >> call (output below). As far as I can tell, that is buried in the VM's >> C code. >> >> The process itself wraps an ETS table and provides a LRU-style cache >> with AMQP bindings to programmatically flush entries based on AMQP >> events elsewhere in the system. >> >> Any thoughts on what delete_trap is doing and why it appears to >> dead-lock the process? Or perhaps where to look to mitigate this? >> >>> process_info(Pid). >> [{registered_name,my_process_name}, >> {current_function,{ets,delete_trap,1}}, >> {initial_call,{proc_lib,init_p,5}}, >> {status,runnable}, >> {message_queue_len,1586057}, >> {message_queue, [...huge...]}, >> {links,[<0.3239.0>]}, >> {dictionary, >> [{'$ancestors',[whistle_couch_sup,<0.3227.0>]}, >> {callid,<<"3fc98d435ea14ab7">>}, >> {'$initial_call',{gen_listener,init,1}}]}, >> {trap_exit,true}, >> {error_handler,error_handler}, >> {priority,normal}, >> {group_leader,<0.3226.0>}, >> {total_heap_size,462590525}, >> {heap_size,145962050}, >> {stack_size,24}, >> {reductions,71467274818}, >> {garbage_collection, >> [{min_bin_vheap_size,46368}, >> {min_heap_size,233}, >> {fullsweep_after,65535}, >> {minor_gcs,1}]}, >> {suspending,[]}] >> >> -- >> James Aimonetti >> Lead Systems Architect / Impressionable Scallywag >> "If Dialyzer doesn't care, I don't care" >> >> 2600HzPDX | http://2600hz.com >> sip:james@REDACTED >> tel:415.886.7905 >> irc:mc_ @ freenode >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > From ok@REDACTED Tue Apr 26 04:59:11 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 26 Apr 2016 14:59:11 +1200 Subject: [erlang-questions] Elixir vs. Erlang In-Reply-To: References: <7vtjvs3tcs1890o3k326eimb.1461486562199@email.android.com> Message-ID: <6ac7a763-37b4-2702-74ae-9e5448d6989e@cs.otago.ac.nz> On 24/04/16 9:54 PM, Micha? Muska?a wrote: >> I m tired of such versus. We seem to be in danger of forgetting the context. The original poster asked for advice a while back on whether to use Erlang or Elixir for a project. Now, we get thanks and a report on the decision. There was actually nothing to discuss. Nobody was ever suggesting that Elixir should not exist or that Erlang should no longer exist or anything like that. Just today I was looking at an R package I hadn't met before. Some of it is written in Fortran, and a fair bit of support code in R is written in C. As a user of the package, my only reason to care is knowing what compilers I need to have installed. Is there *one* web page somewhere listing recent compilers for BEAM-hosted languages? (Yes, I have done a web search.) From ok@REDACTED Tue Apr 26 05:13:43 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 26 Apr 2016 15:13:43 +1200 Subject: [erlang-questions] How to keep adding items to a data structure In-Reply-To: <571CEAEB.6040202@aim.com> References: <571CEAEB.6040202@aim.com> Message-ID: <4a4603dd-d4e6-ac55-c175-90c7f1e09d6e@cs.otago.ac.nz> On 25/04/16 3:48 AM, Donald Steven wrote: > I'm very new to erlang (though quite comfortable with C), trying to > write a program to do some AI composition. Can anyone please suggest > a solution to a very basic problem I've been struggling with for > days, looking at lists, arrays, and records to no avail: > > === > > How do I code the following in erlang (problem example is written in a > pseudo-C style): > > start with an empty music-event list-or-array-or-record > for i = 1 to 1000 > append new musical-event to the end of the music-event > list-or-array-or-record Records are not sequences, so appending to them does not make sense. Lists and tuples are sequences. Is what you described *really* what you want, or is what you want My_Event_Sequence = [new_musical_event(I) || I <- lists:seq(1, 1000)] ? SML version: List.tabulate 1000 new_musical_event Haskell version: [new_musical_event i | i <- [1..1000]] From ok@REDACTED Tue Apr 26 05:34:55 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 26 Apr 2016 15:34:55 +1200 Subject: [erlang-questions] How to keep adding items to a data structure In-Reply-To: <571D270F.3050805@aim.com> References: <571CEAEB.6040202@aim.com> <571D1AF3.3080107@riseup.net> <571D270F.3050805@aim.com> Message-ID: On 25/04/16 8:05 AM, Donald Steven wrote: > Hi Antonios, > > You are kind and generous to help. I'll study this carefully. I will > be wanting to add a list or a record, not just numbers. Is this > possible? There is no restriction on what can be in a list, unless you add such a restriction yourself. Good list processing in functional languages is like good array processing in Fortran, APL, or R: *don't* think element-at-a-time, think whole-collection. In Fortran, you can add two arrays two ways: do I = 1, N A(I) = B(I) + C(I) end do element-at-a-time or A = B + C whole-collection. In Erlang, you *can* write a recursive function to generate the elements of a list, but most of the time it's less work and more readable to use a higher order function from the lists module. For example, to add corresponding elements of two lists, you could write add([X|Xs], [Y|Ys]) -> [X+Y | add(Xs, Ys)]; add([], [] ) -> []. ... A = add(B, C) ... but it's less work to write ... A = lists:zipwith(fun (X,Y) -> X+Y end, B, C) ... where you focus on saying *what* is to be done with corresponding elements instead of *how to find them and construct a result*. EXAGGERATION WARNING: the next sentence is an exaggeration. If you have two loops with the same structure in a functional language, you are doing it wrong, because you should put name the shared control structure and write it just once and then use it twice. EXAGGERATION WARNING: the previous sentence was an exaggeration. But not *much* of an exaggeration. Two more pieces of advice: (1) Look up "Learn You Some Erlang for Great Good!" http://learnyousomeerlang.com When you have read it, buy a copy of the book to thank Fred H?bert for his help, and read it again a couple of times. (2) Erlang/OTP 18.2 came with 4057 *.erl files in lib/. You are never going to read all of them. But there are some that you are going to use over and over again. You should at least read their documentation. If there is a module mentioned in Learn You Some Erlang, look it up at http://erlang.org/erldoc You should find out what is there in the 'erlang' module and what modules are in 'stdlib' especially the 'lists' module. You won't stand a chance of remembering every- thing you see, but do get a feel for what kind of stuff is there and where you might look. Did I mentioned Learn You Some Erlang? There are other great books about Erlang, but that's on-line. > Don > > On 04/24/2016 03:13 PM, Antonios Kouzoupis wrote: >> Hi Don, >> >> The way you iterate in Erlang and I guess in most functional programming >> languages is by recursive call. So if you want to add/append some >> numbers to a list, one way to go is the following: >> >> populate(Num) -> >> populate(Num, []). >> >> populate(0, Acc) -> >> Acc; >> populate(Num, Acc) -> >> populate(Num - 1, [Num | Acc]). >> >> >> Now if you call populate(100), you'll get the list [1,...,100] >> >> BR, >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Tue Apr 26 05:44:08 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 26 Apr 2016 15:44:08 +1200 Subject: [erlang-questions] How to keep adding items to a data structure In-Reply-To: <571D3975.8030205@aim.com> References: <571CEAEB.6040202@aim.com> <571D1AF3.3080107@riseup.net> <571D270F.3050805@aim.com> <571D3975.8030205@aim.com> Message-ID: On 25/04/16 9:24 AM, Donald Steven wrote: > Thanks Oliver, I appreciate your note. I understand that a list can > contain different elements, as you've indicated. That's wonderful! > And I have that book on order. What I'm so frustrated by is, > apparently, in order to extend a list by adding new musical events, > that I have to keep creating new lists, like: > > List1 = [musical-event], > List2 = List1 ++ [new-musical-event], > List3 = List2 ++ [new-musical-event], > List4 = List3 ++ [new-musical-event], > etc. > > all the way to perhaps a few thousand. This is primitive and absurd That *IS* an absurd way to do it, but it's not primitive. The *primitive* way to do it is List0 = [], List1 = [new_musical_event() | List0], ... List1000 = [new_musical_event() | List999], which gives you the list in 1000...1 order. That is as efficient a way to build a list as you could possibly hope for: each step is O(1) in time and space. If you really really need the list in the opposite order, you might be able to generate the elements in the reverse order, or you might simply use lists:reverse/1 to reverse the result, which takes O(n) time and space. This is the same for Lisp, Scheme, Erlang, SML, CAML, F#, Clean, and Haskell: *don't* append things one at a time at the "slow" end of a list. The preferred method is something like lists:map(fun (I) -> compute element I end, lists:seq(1, 1000)) or [compute element I || I <- lists:seq(1, 1000)] An alternative would be to use the 'array' module which gives you extensible arrays implemented as some sort of tree, where fetching, storing, and extending take logarithmic time. But generating a list in the "easy" direction is space- and time- efficient. From ok@REDACTED Tue Apr 26 05:53:38 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 26 Apr 2016 15:53:38 +1200 Subject: [erlang-questions] Counters In-Reply-To: <571D4A77.7060403@aim.com> References: <571D4A77.7060403@aim.com> Message-ID: <1435766f-4ef9-79d5-8324-94be9fef26d8@cs.otago.ac.nz> On 25/04/16 10:36 AM, Donald Steven wrote: > With the archived help provided by Alpar Juttner and Witold Baryluk, I > thought I would share (primarily for other newbees like me), one way > to implement counters. > > Best, > > Don > > =================================================================================== > > > %%%-------------------------------------------------------------------------- > > %%% File : counter.erl, version 1.00 > %%% Author : Donald Steven , adapted from > code > %%% suggested by Alpar Juttner and Witold Baryluk of the > Erlang > %%% community > %%% Purpose : To demonstrate multiple independent counters > %%% Created : 24 Apr 2016 > %%% Invocation : erlc counter.erl > %%% erl -noshell -s counter main -s init stop > %%%-------------------------------------------------------------------------- > > > -module(counter). > -export([main/0,inc/1,dec/1,current/1,start/1,loop/1]). > > %%%-------------------------------------------------------------------------- > > %%% Fun : inc(X), dec(X), current(X) > %%% Purpose : Increment, decrement the counter X, and provide its current > %%% state > %%% Created : 24 Apr 2016 > %%%-------------------------------------------------------------------------- > > > inc(X) -> > list_to_atom("counter_" ++ [X]) ! inc_counter, > ok. NO! DANGER, WILL ROBINSON! Atoms are held in a fixed size table and are not garbage collected. Constructing new atoms at run time is generally a bad idea. That was the first thing I thought of when I saw this, but it's actually not the worst problem. The only values of X that make list_to_atom("counter_" ++ [X]) legal are the integers 0 to 255 inclusive. That is, a counter name X *must* be an integer 0 to 255 AND YOU HAVE NOT DOCUMENTED THIS. Your module should NOT be registering counters. There should simply be functions new_counter() -> new_counter(0). new_counter(N) -> spawn(fun () -> loop(N) end). and the *CALLER* can register a counter if there is a reason to. It's not just that doing it that way removes an arbitrary limitation, it's that as things stand, a malicious program can easily destroy all the counters. *And* it avoids the problem where two processes both try to start counter 5. I have used two programming languages (Fortran and IMP) in which I/O channels were referred to by number and it was seriously painful trying to combine libraries that assumed they had sole control of channel N. By the way, I didn't see anything to *terminate* a counter. From ok@REDACTED Tue Apr 26 07:25:57 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 26 Apr 2016 17:25:57 +1200 Subject: [erlang-questions] Clueless am I In-Reply-To: <571E6CEC.5070305@aim.com> References: <571E6CEC.5070305@aim.com> Message-ID: On 26/04/16 7:15 AM, Donald Steven wrote: > Dear friends, > > Please forgive me for being clueless. (I'm sure it's because I come > to Erlang steeped in C.) > %% THIS DOES NOT WORK > > -module(arraytest). > -export([main/0]). > > main() -> > > A1 = array:set(0, nextevent(), array:new(20)), > array:set(1, nextevent(), A1), > array:set(2, nextevent(), A1), > array:set(3, nextevent(), A1), > array:set(4, nextevent(), A1), > > io:format("~nArray: ~p~n", [A1]), > io:format("Array size: ~p~n~n", [array:size(A1)]). Erlang has no mutable data structures. Once you have given A1 a value, that value can never change. When you call array:set(1, nextevent(), A1), you pass three immutable values. array:set/3 responds by creating and returning a NEW value, which typically reuses bits of its inputs (without changing them in any way). So main() -> A0 = array:new(20), A1 = array:set(0, nextevent(), A0), A2 = array:set(1, nextevent(), A1), A3 = array:set(2, nextevent(), A2), A4 = array:set(3, nextevent(), A3), A5 = array:set(4, nextevent(), A4), io:format("Array: ~p~n", [A5]), io:format("Array size: ~p~n", [array:size(A5)]). This is absolutely basic to Erlang, and is one of the reasons why Erlang is one of the very few languages in which exception handling is worth anything: no matter what went wrong, your data structures CANNOT have been corrupted. Of course you don't want to write a whole list of updates like that, which is why you would write A5 = lists:foldl(fun (N, A) -> array:set(N, nextevent(), A end, array:new(20), lists:seq(0, 4)) producing the output {array,20,0,undefined, {{0,1,4,9,16,undefined,undefined,undefined,undefined, undefined}, 10,10,10,10,10,10,10,10,10,10}} From ok@REDACTED Tue Apr 26 07:34:12 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Tue, 26 Apr 2016 17:34:12 +1200 Subject: [erlang-questions] Clueless am I In-Reply-To: <571E727B.30309@aim.com> References: <571E6CEC.5070305@aim.com> <008101d19f28$d3891660$7a9b4320$@xss.de> <571E727B.30309@aim.com> Message-ID: On 26/04/16 7:39 AM, Donald Steven wrote: > Stefan and Hugo, > > Is there any data structure in Erlang that will accept modification? No. This is a major safety feature of Erlang. > This is quite beyond me as to how to add elements to a data structure. You can't, and you don't want to. You create a *NEW* data structure containing the additional information PLUS the information in the old data structure, sharing much of the original data structure. There is by now a substantial literature on "persistent data structures", meaning ones where you create new versions of data structures and the old versions can still hang around. This is fantastic for "undo", for example. > I've hit the same dead end with lists. If one has, for example, a > database, and one wants to add a customer, do you really need to have > a new (second) database? Data *bases* are not your average data *structure*. Amongst other things, Erlang data *structures* are private to a process, whereas the point of data *bases* is to be shared between processes. Erlang has mutable data *bases* (ETS, DETS, Mnesia, an assortment of interfaces to 3rd party data bases). Horses for courses. (And mutations to data bases are indeed nasty to debug in any programming language.) Put it this way: if you have a variable X holding the number 5 and you compute X+1, you have "added 1 to X", but 5 has not been changed into 6. (Unless you have a very old Fortran compiler where that bug was not unknown.) Data structures in Erlang act like numbers. From zxq9@REDACTED Tue Apr 26 09:16:14 2016 From: zxq9@REDACTED (zxq9) Date: Tue, 26 Apr 2016 16:16:14 +0900 Subject: [erlang-questions] Clueless am I In-Reply-To: References: <571E6CEC.5070305@aim.com> Message-ID: <2148774.XZ0pOLKKN2@changa> On 2016?4?26? ??? 17:25:57 Richard A. O'Keefe wrote: > > On 26/04/16 7:15 AM, Donald Steven wrote: > > Dear friends, > > > > Please forgive me for being clueless. (I'm sure it's because I come > > to Erlang steeped in C.) > > %% THIS DOES NOT WORK > > > > -module(arraytest). > > -export([main/0]). > > > > main() -> > > > > A1 = array:set(0, nextevent(), array:new(20)), > > array:set(1, nextevent(), A1), > > array:set(2, nextevent(), A1), > > array:set(3, nextevent(), A1), > > array:set(4, nextevent(), A1), > > > > io:format("~nArray: ~p~n", [A1]), > > io:format("Array size: ~p~n~n", [array:size(A1)]). > > > Erlang has no mutable data structures. > Once you have given A1 a value, that value can never change. > When you call array:set(1, nextevent(), A1), > you pass three immutable values. > array:set/3 responds by creating and returning a NEW value, > which typically reuses bits of its inputs (without changing them > in any way). So > > main() -> > A0 = array:new(20), > A1 = array:set(0, nextevent(), A0), > A2 = array:set(1, nextevent(), A1), > A3 = array:set(2, nextevent(), A2), > A4 = array:set(3, nextevent(), A3), > A5 = array:set(4, nextevent(), A4), > io:format("Array: ~p~n", [A5]), > io:format("Array size: ~p~n", [array:size(A5)]). > > This is absolutely basic to Erlang, and is one of the reasons why Erlang > is one of the very few languages in which exception handling is worth > anything: no matter what went wrong, your data structures CANNOT have > been corrupted. > > Of course you don't want to write a whole list of updates like that, > which is why you would write > > A5 = lists:foldl(fun (N, A) -> array:set(N, nextevent(), A end, > array:new(20), lists:seq(0, 4)) > > producing the output > > {array,20,0,undefined, > {{0,1,4,9,16,undefined,undefined,undefined,undefined, > undefined}, > 10,10,10,10,10,10,10,10,10,10}} Going a step further, into the nature of the task itself... (This is a long post, anyone uninterested in nitpicking beginner stuff should skip it...) In Erlang an "event" can mean a few things, but generally speaking we are trying to represent external events as messages, which means that the process that will *react* to a given event is actually reacting to a *message* of a specific form that represents a given type of event. In your example, you instantiate an array. This is a mismatch that your C background is telling you fits the problem, but very likely does not. A more basic functional data structure is a list. DO NOT WORRY ABOUT WHAT HAPPENS IN THE RUNTIME TO MAKE THIS WORK. [That is rule #1 of using a higher-level language. Or rather, it is a strong rule at the outset of a project (letting you think mostly about the essence of the problem rather than the way that manifests in machine instructions), and then becomes a guideline later once you have a working system and can prove you actually have a performance problem of some sort.] So let's start with a list. We will need to initialize our process somehow. So let's make a `start_link/0` function that spawns a function and gives us back its PID so that we can send it messages, allowing us to directly simulate receipt of those messages from the shell. We also need some way of letting the process report back its current state without getting too wizardly with the runtime (otherwise that process is a black-hole for data, which probably isn't your intention). So we wind up with a module, saved in a file "eventest.erl": -module(eventest). -export([start_link/0]). start_link() -> spawn_link(fun init/0). init() -> State = [], loop(State). loop(State) -> receive {event, Data} -> NewState = [Data | State], loop(NewState); {report, Asker} -> Asker ! State, loop(State); shutdown -> ok = io:format("~p: shutting down. Bye!~n", [self()]), exit(normal); Unexpected -> ok = io:format("~p: Received unexpected messages: ~tp~n", [self(), Unexpected]), loop(State) end. This is pretty much the most elementary way of doing things in a process-based way imaginable in Erlang -- and in this case the *process* spawned when calling `start_link/0` above is on its own timeline, independent of anything else in the world, and within that process it has a list-type data structure that represents its state, and we will be appending to any time a message of the form `{event, Data}` is received: This is what playing with it in the shell looks like: ceverett@REDACTED:~/Code/erlang$ erl Erlang/OTP 18 [erts-7.2] [source] [64-bit] [smp:2:2] [async-threads:10] [kernel-poll:false] Eshell V7.2 (abort with ^G) 1> c(eventest). {ok,eventest} 2> Tester = eventest:start_link(). <0.40.0> 3> Tester ! {report, self()}. {report,<0.33.0>} 4> flush(). Shell got [] ok 5> Tester ! {event, "foo"}. {event,"foo"} 6> Tester ! {report, self()}. {report,<0.33.0>} 7> flush(). Shell got ["foo"] ok 8> EventList = ["bar", "baz", "balls"]. ["bar","baz","balls"] 9> SendToTester = fun(Data) -> Tester ! {event, Data} end. #Fun 10> lists:foreach(SendToTester, EventList). ok 11> Tester ! {report, self()}. {report,<0.33.0>} 12> flush(). Shell got ["balls","baz","bar","foo"] ok 13> Tester ! "A random message it is not expecting.". <0.40.0>: Received unexpected messages: "A random message it is not expecting." "A random message it is not expecting." 14> Tester ! shutdown. <0.40.0>: shutting down. Bye! shutdown And that's it. So looking at your original case and the super simple model I wrote above, maybe we want to be able to handle a list of events, perform some transform over them (or execute some side-effecty operation), and then store them to the list of events received so far. Maybe we also want to be able to abstract the details of sending messages behind some function calls as well (so I can just call `eventest:report(Tester)` instead of always writing out `PID ! {report, self()}`: -module(eventest). -export([start_link/0, report/1, single/2, multi/2, shutdown/1]). %% Public interface start_link() -> spawn_link(fun init/0). report(PID) -> PID ! {report, self()}. single(PID, Event) -> PID ! {single, Event}. multi(PID, Events) -> PID ! {multi, Events}. shutdown(PID) -> PID ! shutdown. %% Private functions init() -> State = [], loop(State). loop(State) -> receive {single, Event} -> Data = do_processing(Event), NewState = [Data | State], loop(NewState); {multi, Events} -> DataList = lists:map(fun do_processing/1, Events), NewState = DataList ++ State, loop(NewState); {report, Asker} -> Asker ! State, loop(State); shutdown -> ok = io:format("~p: shutting down. Bye!~n", [self()]), exit(normal); Unexpected -> ok = io:format("~p: Received unexpected messages: ~tp~n", [self(), Unexpected]), loop(State) end. do_processing(Event) -> Size = length(Event), ok = io:format("~p: Received event of size: ~p~n", [self(), Size]), {Size, Event}. Obviously, the `do_processing/1` function is arbitrary, but you can imagine a process maintaining its own state that is mutable *in time* but uses symbols which are *immutable within their scope* to represent the actual calculations involved in whatever transforms are being performed. This is a *lot* closer to the way things are often done in "pure" Erlang, meaning, this is how things are often done in Erlang when using processes to maintain state, modules to represent a process's definition and hide implementation from public functions, and messages to represent events within a system. There are more details, but I think you're just getting hung up on some of the very early differences -- and that is totally normal. As far as functional idioms go... there is a very basic core of operations one expects to find in pretty much any functional language, and they tend to be centered around list operations. The Erlang `lists` standard lib module documentation provides a terse, but pretty solid representation of them (but also includes a lot of stuff specific to Erlang's own lib). What you are looking for is list operations like folds, maps and filters, the basics of recursion (which can seem at first more weird than explicit looping, but winds up becoming easier than explicit looping over time), and some of the basic concepts underlying Erlang's style of managing concurrency. I wrote a few bits about these basics -- forgive the links, I just don't feel like re-writing it all here: An explanation of fold (which Richard was using as an example above): http://stackoverflow.com/questions/26854586/explanation-of-listsfold-function/26855055#26855055 Erlang processes VS Java threads: (tl;dr: The difference between Java and Erlang is much deeper than the essentially cosmetic differences between OOP and FP.) http://stackoverflow.com/questions/32294367/erlang-process-vs-java-thread/32296577#32296577 Clearing up some misunderstanding about recursion: http://stackoverflow.com/questions/27234898/erlang-recursive-end-loop/27236236#27236236 A word on "function chaining", touching on error code: http://stackoverflow.com/questions/34622869/function-chaining-in-erlang/34626317#34626317 Blah blah. Start messing around with the module above, since it runs. Morph it into something that does whatever you want. Make two that talk to each other. Every little experiment will introduce you to new things at first, and give you better insights into what is being said if you start to read a book like "Learn You Some Erlang" (free online http://learnyousomeerlang.com/ ), "Erlang and OTP In Action", "Programming Erlang", etc. (all strong recommendations). As for getting a feel for what exists beyond C and asm, and letting your mind relax and come to terms with abstractions based on primitives, I still can't think of a better text than "Structure and Interpretation of Computer Programs" (also free online: https://mitpress.mit.edu/sicp/full-text/book/book.html ). -Craig From garazdawi@REDACTED Tue Apr 26 09:37:58 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Tue, 26 Apr 2016 09:37:58 +0200 Subject: [erlang-questions] ETS delete_trap In-Reply-To: <571EA889.1010400@2600hz.com> References: <571EA889.1010400@2600hz.com> Message-ID: On Tue, Apr 26, 2016 at 1:30 AM, James Aimonetti wrote: > > The curious bit that I can't really figure out is the > {current_function, {ets,delete_trap,1}} bit from the process_info/1 > call (output below). As far as I can tell, that is buried in the VM's > C code. > > The delete_trap function is used when you are calling either ets:select_delete/2 or ets:delete/1 on a table with many many elements. When working with ets tables that have many elements, erts takes smaller chunks of the table at a time and reschedules the process doing the processing. So for ets:delete it would first delete the first X elements (calling delete), then reschedule, then delete the next X (calling delete_trap), and so on deleting X records per iteration. Most ets functions work this way in order to not hog the schedulers for too a long time when working with large tables. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From kost@REDACTED Tue Apr 26 09:59:04 2016 From: kost@REDACTED (Konstantin Vsochanov) Date: Tue, 26 Apr 2016 10:59:04 +0300 Subject: [erlang-questions] Clueless am I In-Reply-To: References: <571E6CEC.5070305@aim.com> <008101d19f28$d3891660$7a9b4320$@xss.de> <571E727B.30309@aim.com> Message-ID: <571F1FC8.5000205@rol.ru> On 4/26/16 08:34 , Richard A. O'Keefe wrote: > > On 26/04/16 7:39 AM, Donald Steven wrote: >> Stefan and Hugo, >> >> Is there any data structure in Erlang that will accept modification? > No. This is a major safety feature of Erlang. > Actually, there is one - process dictionary But you have to use it with care. From max.lapshin@REDACTED Tue Apr 26 11:17:47 2016 From: max.lapshin@REDACTED (Max Lapshin) Date: Tue, 26 Apr 2016 12:17:47 +0300 Subject: [erlang-questions] Support for polling HDD in linux 4.4 In-Reply-To: <22302.8216.563036.922923@gargle.gargle.HOWL> References: <22302.8216.563036.922923@gargle.gargle.HOWL> Message-ID: Yes, I also haven't found any working pieces of code, this is why I told that I cannot decrypt this article. Seems that there is nothing to discuss yet =( -------------- next part -------------- An HTML attachment was scrubbed... URL: From t6sn7gt@REDACTED Tue Apr 26 12:27:53 2016 From: t6sn7gt@REDACTED (Donald Steven) Date: Tue, 26 Apr 2016 06:27:53 -0400 Subject: [erlang-questions] Counters In-Reply-To: <1435766f-4ef9-79d5-8324-94be9fef26d8@cs.otago.ac.nz> References: <571D4A77.7060403@aim.com> <1435766f-4ef9-79d5-8324-94be9fef26d8@cs.otago.ac.nz> Message-ID: <571F42A9.2080502@aim.com> Thanks Richard and Garret for your comments and patience with a newbie. Don On 04/25/2016 11:53 PM, Richard A. O'Keefe wrote: > > > On 25/04/16 10:36 AM, Donald Steven wrote: >> With the archived help provided by Alpar Juttner and Witold Baryluk, >> I thought I would share (primarily for other newbees like me), one >> way to implement counters. >> >> Best, >> >> Don >> >> =================================================================================== >> >> >> %%%-------------------------------------------------------------------------- >> >> %%% File : counter.erl, version 1.00 >> %%% Author : Donald Steven , adapted from >> code >> %%% suggested by Alpar Juttner and Witold Baryluk of the >> Erlang >> %%% community >> %%% Purpose : To demonstrate multiple independent counters >> %%% Created : 24 Apr 2016 >> %%% Invocation : erlc counter.erl >> %%% erl -noshell -s counter main -s init stop >> %%%-------------------------------------------------------------------------- >> >> >> -module(counter). >> -export([main/0,inc/1,dec/1,current/1,start/1,loop/1]). >> >> %%%-------------------------------------------------------------------------- >> >> %%% Fun : inc(X), dec(X), current(X) >> %%% Purpose : Increment, decrement the counter X, and provide its >> current >> %%% state >> %%% Created : 24 Apr 2016 >> %%%-------------------------------------------------------------------------- >> >> >> inc(X) -> >> list_to_atom("counter_" ++ [X]) ! inc_counter, >> ok. > > NO! DANGER, WILL ROBINSON! > > Atoms are held in a fixed size table and are not garbage collected. > Constructing new atoms at run time is generally a bad idea. > > That was the first thing I thought of when I saw this, but it's > actually not the worst problem. The only values of X that make > list_to_atom("counter_" ++ [X]) legal are the integers 0 to 255 > inclusive. That is, a counter name X *must* be an integer 0 to 255 > AND YOU HAVE NOT DOCUMENTED THIS. > > Your module should NOT be registering counters. > > There should simply be functions > > new_counter() -> > new_counter(0). > > new_counter(N) -> > spawn(fun () -> loop(N) end). > > and the *CALLER* can register a counter if there is a reason to. > > It's not just that doing it that way removes an arbitrary limitation, > it's that as things stand, a malicious program can easily destroy all > the counters. *And* it avoids the problem where two processes > both try to start counter 5. I have used two programming languages > (Fortran and IMP) in which I/O channels were referred to by number > and it was seriously painful trying to combine libraries that assumed > they had sole control of channel N. > > By the way, I didn't see anything to *terminate* a counter. > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From t6sn7gt@REDACTED Tue Apr 26 12:33:42 2016 From: t6sn7gt@REDACTED (Donald Steven) Date: Tue, 26 Apr 2016 06:33:42 -0400 Subject: [erlang-questions] Clueless am I In-Reply-To: <571F1FC8.5000205@rol.ru> References: <571E6CEC.5070305@aim.com> <008101d19f28$d3891660$7a9b4320$@xss.de> <571E727B.30309@aim.com> <571F1FC8.5000205@rol.ru> Message-ID: <571F4406.2060808@aim.com> Thanks everyone. I've got a lot of reading to do. You are all very gerneous to help someone get started. Don On 04/26/2016 03:59 AM, Konstantin Vsochanov wrote: > On 4/26/16 08:34 , Richard A. O'Keefe wrote: >> On 26/04/16 7:39 AM, Donald Steven wrote: >>> Stefan and Hugo, >>> >>> Is there any data structure in Erlang that will accept modification? >> No. This is a major safety feature of Erlang. >> > Actually, there is one - process dictionary > But you have to use it with care. > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From mremond@REDACTED Tue Apr 26 12:44:48 2016 From: mremond@REDACTED (=?UTF-8?B?TWlja2HDq2wgUsOpbW9uZA==?=) Date: Tue, 26 Apr 2016 10:44:48 +0000 Subject: [erlang-questions] Is ejabberd/MongooseIM not a good choise for mobile IM? In-Reply-To: References: Message-ID: Hello, On Thu, Apr 14, 2016 at 3:53 PM Caiyun Deng wrote: > Hi! I want to develop a mobile IM app, i searched something about. > Because xmpp is heavyweight, is ejabberd/MongooseIM not a good choise for > mobile IM? > If you develop a mobile IM today, which tool will you choose? > >From experience building infrastructure for mobile applications, ejabberd is a very good choice. A good way to understand how it can help is to watch XMPP Academy video series: https://www.youtube.com/playlist?list=PLXeQZzENE-sJsivVxEvfPP-4nBfDhDqH- Many of the questions address are around mobile support. I hope this helps, -- Micka?l R?mond -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlangworkshop@REDACTED Tue Apr 26 11:43:33 2016 From: erlangworkshop@REDACTED (Erlang Workshop) Date: Tue, 26 Apr 2016 11:43:33 +0200 Subject: [erlang-questions] [ANN] Erlang Workshop 2016 - CFP Message-ID: <571F3845.2020105@gmail.com> Apologies for any duplicates you may receive. CALL FOR PAPERS =============== Fifteenth ACM SIGPLAN Erlang Workshop ------------------------------ ----------------------------- Nara, Japan, September 23, 2016 Satellite event of the 21st ACM SIGPLAN International Conference on Functional Programming (ICFP 2016) September 18-24, 2016 The Erlang Workshop aims to bring together the open source, academic, and industrial communities of Erlang, to discuss technologies and languages related to Erlang. The Erlang model of concurrent programming has been widely emulated, for example by Akka in Scala, and even new programming languages were designed atop of the Erlang VM, such as Elixir. Therefore we would like to broaden the scope of the workshop to include systems like those mentioned above. The workshop will enable participants to familiarize themselves with recent developments on new techniques and tools, novel applications, draw lessons from users' experiences and identify research problems and common areas relevant to the practice of Erlang, Erlang-like languages, functional programming, distribution, concurrency etc. We invite three types of submissions. 1. Technical papers describing interesting contributions either in theoretical work or real world applications. Submission related to Erlang, Elixir, Akka, CloudHaskell, Occam, and functional programming are welcome and encouraged. Topics of interest include (but are not limited to): - virtual machine extensions and compilation techniques - implementations and interfaces of Erlang in/with other languages - new tools (profilers, tracers, debuggers, testing frameworks etc.) - language extensions - formal semantics, correctness and verification - testing Erlang programs - program analysis and transformation - Erlang-like languages and technologies - functional languages and multi-processing - concurrency in functional languages - functional languages and distributed computing - parallel programming - pattern based programming - Erlang in education The maximum length for technical papers is restricted to 12 pages. 2. Experience reports describing uses of Erlang in the "real-world", Erlang libraries for specific tasks, experiences from using Erlang in specific application domains, reusable programming idioms and elegant new ways of using Erlang to approach or solve a particular problem. The maximum length for the experience report is restricted to 2 pages. 3. Poster presentations describing topics related to the workshop goals. Each includes a maximum of 2 pages of the abstract and summary. Presentations in this category will be given an hour of shared simultaneous demonstration time. Workshop Co-Chairs ------------------ Melinda T?th, E?tv?s Lor?nd University, Hungary Scott Lystig Fritchie, Basho Japan KK Program Committee ----------------------------- (Note: the Workshop Co-Chairs are also committee members) Jamie Allen, Typesafe Laura M. Castro, University of A Coru?a, Spain Natalia Chechina, University of Glasgow Viktoria F?rd?s, Erlang Solutions Yosuke Hara, Rakuten, Inc. Kenji Rikitake, KRPEO Bruce Tate, iCanMakeItBetter Simon Thompson, University of Kent, UK Important Dates ----------------------- Submissions due: Friday, 3 June, 2016 Author notification: Friday, 8 July, 2016 Final copy due: Sunday, 31 July, 2016 Workshop date: September 23, 2016 Instructions to authors -------------------------------- Papers must be submitted online via EasyChair (via the "Erlang2016" event). The submission page is https://www.easychair.org/conferences/?conf=erlang2016 Submitted papers should be in portable document format (PDF), formatted using the ACM SIGPLAN style guidelines. Each submission must adhere to SIGPLAN's republication policy. Violation risks summary rejection of the offending submission. Accepted papers will be published by the ACM and will appear in the ACM Digital Library. Paper submissions will be considered for poster submission in the case they are not accepted as full papers. Venue & Registration Details ------------------------------------------ For registration, please see the ICFP 2016 web site at: http://conf.researchr.org/home/icfp-2016 Related Links -------------------- CFP: http://conf.researchr.org/track/icfp-2016/erlang-2016-papers ICFP 2016 web site: http://conf.researchr.org/home/icfp-2016 Past ACM SIGPLAN Erlang workshops: http://www.erlang.org/workshop/ Open Source Erlang: http://www.erlang.org/ EasyChair submission site: https://www.easychair.org/conferences/?conf=erlang2016 Author Information for SIGPLAN Conferences: http://www.sigplan.org/authorInformation.htm Atendee Information for SIGPLAN Events: http://www.sigplan.org/Resources/Policies/Anti-harassment -------------- next part -------------- An HTML attachment was scrubbed... URL: From james@REDACTED Tue Apr 26 16:39:44 2016 From: james@REDACTED (James Aimonetti) Date: Tue, 26 Apr 2016 07:39:44 -0700 Subject: [erlang-questions] ETS delete_trap In-Reply-To: References: <571EA889.1010400@2600hz.com> Message-ID: <571F7DB0.9030600@2600hz.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On 04/26/2016 12:37 AM, Lukas Larsson wrote: > On Tue, Apr 26, 2016 at 1:30 AM, James Aimonetti > wrote: > >> >> The curious bit that I can't really figure out is the >> {current_function, {ets,delete_trap,1}} bit from the >> process_info/1 call (output below). As far as I can tell, that is >> buried in the VM's C code. >> >> > The delete_trap function is used when you are calling either > ets:select_delete/2 or ets:delete/1 on a table with many many > elements. > > When working with ets tables that have many elements, erts takes > smaller chunks of the table at a time and reschedules the process > doing the processing. > > So for ets:delete it would first delete the first X elements > (calling delete), then reschedule, then delete the next X (calling > delete_trap), and so on deleting X records per iteration. Most ets > functions work this way in order to not hog the schedulers for too > a long time when working with large tables. > > Lukas > This makes sense to a degree. We read the ETS table (protected, set) from the worker process to see if there are any entries that should be erased and cast into the gen_listener process to erase the individual entries by key. The table is large-ish, maybe 1.5GB, but we are using ets:delete(Tab, Key) for each Key found in the worker process (there are associated callbacks so client code can be notified if the cached entry is flushed/erased) which I assume should be fast. The only other delete operation is to flush the whole table with ets:delete_all_objects/1. We haven't tried enabling the read/write concurrency flags yet; would write_concurrency make a difference when calling ets:delete/2? This is 18.2 too, by the way. Sorry for not mentioning that first. Any tips on profiling ETS usage and finding efficiencies? Thanks for the information so far. - -- James Aimonetti Lead Systems Architect / Impressionable Scallywag "If Dialyzer doesn't care, I don't care" 2600HzPDX | http://2600hz.com sip:james@REDACTED tel:415.886.7905 irc:mc_ @ freenode -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBCAAGBQJXH32wAAoJENTKa+JPXCVgEcEIAIqug2b14Sq604f+Kkwd+SF1 oNhFC9ZN9pllzaEMzmEQ+k2gFVWdMnpUg4s2pPLnHiRPP24XcpAQzGXmU+R8hlTb XmVrpQ6TFrTbVOi8lJlN3afMlK0XOgyjfd+QzbIMWIRLPyKnPq36bBbmD5Ybbrt1 /wNHVTxFJ/vJ7CxfaAXNNvli2DvRjiFYxzUn6t1b/C18ad0a+pFm1SZ1NdTOsHh9 Bcminbe8y3J/QJU/O62A8l2N40MvqD10O5O9RoHo9Wp3Uyz6NftDP/caDYKSbCLw BYY0ziI7Wovk5IWZfaYAEY/5uJCH+DmCq6xdL+BKesKY/AMnlIWDH9O8jWq5RHI= =z88N -----END PGP SIGNATURE----- From garazdawi@REDACTED Tue Apr 26 16:51:36 2016 From: garazdawi@REDACTED (Lukas Larsson) Date: Tue, 26 Apr 2016 16:51:36 +0200 Subject: [erlang-questions] ETS delete_trap In-Reply-To: <571F7DB0.9030600@2600hz.com> References: <571EA889.1010400@2600hz.com> <571F7DB0.9030600@2600hz.com> Message-ID: On Tue, Apr 26, 2016 at 4:39 PM, James Aimonetti wrote: > we are using ets:delete(Tab, Key) for each Key found in the worker process > > The only other delete operation is to flush the whole table > with ets:delete_all_objects/1. > >From what I can tell in the vm source neither ets:delete/2 or ets:delete_all_objects/1 should end up in a yield to ets:delete_trap/1. Only ets:select_delete/2 or ets:delete/1 take that code path. > > We haven't tried enabling the read/write concurrency flags yet; would > write_concurrency make a difference when calling ets:delete/2? If you have multiple other processes doing reads/writes into the table at the same time as you are doing deletion in it, then yes it may help speed things up. > Any tips on profiling ETS usage and finding efficiencies? > I would use tracing, either call_time or just plain ordinary call+return tracing together with timestamps to see if any suspect ets function take a long time to complete and then from there try to figure out why. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From duncan@REDACTED Tue Apr 26 19:45:20 2016 From: duncan@REDACTED (duncan@REDACTED) Date: Tue, 26 Apr 2016 10:45:20 -0700 Subject: [erlang-questions] Clueless am I Message-ID: <20160426104520.7e43b23f706d1a78218bd3e1c66e57ee.1da16ce41a.wbe@email23.secureserver.net> An HTML attachment was scrubbed... URL: From dan@REDACTED Tue Apr 26 21:30:37 2016 From: dan@REDACTED (Daniel Dormont) Date: Tue, 26 Apr 2016 15:30:37 -0400 Subject: [erlang-questions] Mnesia: inconsistent views without netsplit? Message-ID: Hi all, I have a three node Mnesia cluster (hosting a somewhat outdated version of ejabberd, but I'm not sure that matters). I have a table that is stored as ram_copies on all three nodes. Yet, this table has differing numbers of records among the three. The table info from one of them is pasted below. Running the same query on one of my other nodes, I get more or less the same result, but the "size" is very different: 553 vs 867. And indeed, there are individual records that turn up in a mnesia:read/2 or mnesia:dirty_read/2 on one node and not the other. Yet, nothing in my log indicates that there was ever a netsplit or disconnection. So I have two questions: 1) What might cause this? and 2) Is there any way, especially given I know which records are affected, to force some kind of replication on this table without completely restarting one of the nodes? thanks, Dan Dormont [{access_mode,read_write}, {active_replicas,['ejabberd@REDACTED', 'ejabberd@REDACTED', 'ejabberd@REDACTED']}, {all_nodes,['ejabberd@REDACTED', 'ejabberd@REDACTED', 'ejabberd@REDACTED']}, {arity,3}, {attributes,[name_host,pid]}, {checkpoints,[]}, {commit_work,[]}, {cookie,{{1341,344810,207763},'ejabberd@REDACTED'}}, {cstruct,{cstruct,muc_online_room,set, ['ejabberd@REDACTED', 'ejabberd@REDACTED', 'ejabberd@REDACTED'], [],[],0,read_write,false,[],[],false,muc_online_room, [name_host,pid], [],[],[],{...},...}}, {disc_copies,[]}, {disc_only_copies,[]}, {frag_properties,[]}, {index,[]}, {load_by_force,false}, {load_node,'ejabberd@REDACTED'}, {load_order,0}, {load_reason,{active_remote,'ejabberd@REDACTED'}}, {local_content,false}, {majority,false}, {master_nodes,[]}, {memory,73643}, {ram_copies,['ejabberd@REDACTED', 'ejabberd@REDACTED', 'ejabberd@REDACTED']}, {record_name,muc_online_room}, {record_validation,{muc_online_room,3,set}}, {type,set}, {size,867}, {snmp,[]}, {storage_properties,...}, {...}|...] From erlang@REDACTED Tue Apr 26 22:53:37 2016 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 26 Apr 2016 22:53:37 +0200 Subject: [erlang-questions] Clueless am I In-Reply-To: <20160426104520.7e43b23f706d1a78218bd3e1c66e57ee.1da16ce41a.wbe@email23.secureserver.net> References: <20160426104520.7e43b23f706d1a78218bd3e1c66e57ee.1da16ce41a.wbe@email23.secureserver.net> Message-ID: On Tue, Apr 26, 2016 at 7:45 PM, wrote: > One thing that helped me as a beginner wrt the 'immutable variable', and I > think will help with understanding this whole thread, is that the C mindset > contains not only the 'mutable variable' but also the 'loop' construct. It's > important to note there are no loops in erlang. Even though 'loop' is used > in the post below, it's not a c-like loop - it's a function call. > > Erlang uses recursion where C uses loops. And the new instance of the > recursed function has it's own name space. So it sort of looks like a given > 'variable' is mutable (ie the variable X is 5 in one instance of the > function and 6 in the next instance) - but it really is a different variable > because it's in a different namespace. To a newbie like me, this is like > 'updating the variable' in the initial post. > > C has loop over x, erlang has recurse over x. > > Ie the answer to the original question 'how to add elements to a data > structure' is 'with recursive function calls'. And that avoids the A1 to A5 > in the original program (ie you only need the variable A). > > Hopefully this helps more than it confuses. It makes sense in my head but > never comes out right on paper. You're quite right. The other thing to point out is that when I talk about code like foo(N, A) -> .. do something ... foo(N-1, B). I'll uses the words "loop" - You might think of the final call foo(N-1,B) as a function call that returns to this place in the code. I see it as a GOTO that hops to the function entry point. In code like foo(A) -> B = x(A), C = y(B), foo(C). There is a significant difference between the calls to x, y, and foo. When we call x(A) we return to the caller (same for y(B)). When we call foo(C) we don't actually call foo(C) we replace the current stack frame containing the local variable A with the variable C and jump to the top of the code. So it's a loop and executes in finite space without creating an additional stack frame (and the garbage collector can reclaim the space used by B since nobody can ever access this) - this is how we don't run out of space. The equivalent code in C (python etc) will create a stack frame and will crash in an infinite recursion. We *have* to do it this way to create infinite process loops to store data like: store(Data) -> receive Msg -> Data1 = update(Msg, Data), store(Data1) end. If the call to store(Data1) was subroutine call that created a new stack frame we'd eventually run out of space. Old timers (like me) get so used to this idea that we just call code like this "loops" - so to me tail recursive code (to give it a name) is just the FP equivalent of a loop in C - the same is true for all FPLs Cheers /Joe > > Duncan Sparrell > s-Fractal Consulting LLC > > > -------- Original Message -------- > Subject: Re: [erlang-questions] Clueless am I > From: zxq9 > Date: Tue, April 26, 2016 3:16 am > To: erlang-questions@REDACTED > > On 2016?4?26? ??? 17:25:57 Richard A. O'Keefe wrote: >> >> On 26/04/16 7:15 AM, Donald Steven wrote: >> > Dear friends, >> > >> > Please forgive me for being clueless. (I'm sure it's because I come >> > to Erlang steeped in C.) >> > %% THIS DOES NOT WORK >> > >> > -module(arraytest). >> > -export([main/0]). >> > >> > main() -> >> > >> > A1 = array:set(0, nextevent(), array:new(20)), >> > array:set(1, nextevent(), A1), >> > array:set(2, nextevent(), A1), >> > array:set(3, nextevent(), A1), >> > array:set(4, nextevent(), A1), >> > >> > io:format("~nArray: ~p~n", [A1]), >> > io:format("Array size: ~p~n~n", [array:size(A1)]). >> >> >> Erlang has no mutable data structures. >> Once you have given A1 a value, that value can never change. >> When you call array:set(1, nextevent(), A1), >> you pass three immutable values. >> array:set/3 responds by creating and returning a NEW value, >> which typically reuses bits of its inputs (without changing them >> in any way). So >> >> main() -> >> A0 = array:new(20), >> A1 = array:set(0, nextevent(), A0), >> A2 = array:set(1, nextevent(), A1), >> A3 = array:set(2, nextevent(), A2), >> A4 = array:set(3, nextevent(), A3), >> A5 = array:set(4, nextevent(), A4), >> io:format("Array: ~p~n", [A5]), >> io:format("Array size: ~p~n", [array:size(A5)]). >> >> This is absolutely basic to Erlang, and is one of the reasons why Erlang >> is one of the very few languages in which exception handling is worth >> anything: no matter what went wrong, your data structures CANNOT have >> been corrupted. >> >> Of course you don't want to write a whole list of updates like that, >> which is why you would write >> >> A5 = lists:foldl(fun (N, A) -> array:set(N, nextevent(), A end, >> array:new(20), lists:seq(0, 4)) >> >> producing the output >> >> {array,20,0,undefined, >> {{0,1,4,9,16,undefined,undefined,undefined,undefined, >> undefined}, >> 10,10,10,10,10,10,10,10,10,10}} > > Going a step further, into the nature of the task itself... > > (This is a long post, anyone uninterested in nitpicking beginner stuff > should skip it...) > > In Erlang an "event" can mean a few things, but generally speaking > we are trying to represent external events as messages, which means > that the process that will *react* to a given event is actually > reacting to a *message* of a specific form that represents a given > type of event. > > In your example, you instantiate an array. This is a mismatch that > your C background is telling you fits the problem, but very likely > does not. A more basic functional data structure is a list. > > DO NOT WORRY ABOUT WHAT HAPPENS IN THE RUNTIME TO MAKE THIS WORK. > > [That is rule #1 of using a higher-level language. Or rather, it is > a strong rule at the outset of a project (letting you think mostly > about the essence of the problem rather than the way that manifests > in machine instructions), and then becomes a guideline later once > you have a working system and can prove you actually have a performance > problem of some sort.] > > So let's start with a list. We will need to initialize our process > somehow. So let's make a `start_link/0` function that spawns a > function and gives us back its PID so that we can send it messages, > allowing us to directly simulate receipt of those messages from the > shell. > > We also need some way of letting the process report back its current > state without getting too wizardly with the runtime (otherwise that > process is a black-hole for data, which probably isn't your intention). > > So we wind up with a module, saved in a file "eventest.erl": > > -module(eventest). > -export([start_link/0]). > > start_link() -> > spawn_link(fun init/0). > > init() -> > State = [], > loop(State). > > loop(State) -> > receive > {event, Data} -> > NewState = [Data | State], > loop(NewState); > {report, Asker} -> > Asker ! State, > loop(State); > shutdown -> > ok = io:format("~p: shutting down. Bye!~n", [self()]), > exit(normal); > Unexpected -> > ok = io:format("~p: Received unexpected messages: ~tp~n", > [self(), Unexpected]), > loop(State) > end. > > This is pretty much the most elementary way of doing things in a > process-based way imaginable in Erlang -- and in this case the *process* > spawned when calling `start_link/0` above is on its own timeline, > independent of anything else in the world, and within that process it > has a list-type data structure that represents its state, and we will > be appending to any time a message of the form `{event, Data}` is received: > > This is what playing with it in the shell looks like: > > > ceverett@REDACTED:~/Code/erlang$ erl > Erlang/OTP 18 [erts-7.2] [source] [64-bit] [smp:2:2] [async-threads:10] > [kernel-poll:false] > > Eshell V7.2 (abort with ^G) > 1> c(eventest). > {ok,eventest} > 2> Tester = eventest:start_link(). > <0.40.0> > 3> Tester ! {report, self()}. > {report,<0.33.0>} > 4> flush(). > Shell got [] > ok > 5> Tester ! {event, "foo"}. > {event,"foo"} > 6> Tester ! {report, self()}. > {report,<0.33.0>} > 7> flush(). > Shell got ["foo"] > ok > 8> EventList = ["bar", "baz", "balls"]. > ["bar","baz","balls"] > 9> SendToTester = fun(Data) -> Tester ! {event, Data} end. > #Fun > 10> lists:foreach(SendToTester, EventList). > ok > 11> Tester ! {report, self()}. > {report,<0.33.0>} > 12> flush(). > Shell got ["balls","baz","bar","foo"] > ok > 13> Tester ! "A random message it is not expecting.". > <0.40.0>: Received unexpected messages: "A random message it is not > expecting." > "A random message it is not expecting." > 14> Tester ! shutdown. > <0.40.0>: shutting down. Bye! > shutdown > > > And that's it. > > So looking at your original case and the super simple model I wrote above, > maybe we want to be able to handle a list of events, perform some transform > over them (or execute some side-effecty operation), and then store them to > the list of events received so far. Maybe we also want to be able to > abstract > the details of sending messages behind some function calls as well (so I can > just call `eventest:report(Tester)` instead of always writing out > `PID ! {report, self()}`: > > > -module(eventest). > -export([start_link/0, report/1, single/2, multi/2, shutdown/1]). > > > %% Public interface > > start_link() -> > spawn_link(fun init/0). > > report(PID) -> > PID ! {report, self()}. > > single(PID, Event) -> > PID ! {single, Event}. > > multi(PID, Events) -> > PID ! {multi, Events}. > > shutdown(PID) -> > PID ! shutdown. > > > %% Private functions > > init() -> > State = [], > loop(State). > > loop(State) -> > receive > {single, Event} -> > Data = do_processing(Event), > NewState = [Data | State], > loop(NewState); > {multi, Events} -> > DataList = lists:map(fun do_processing/1, Events), > NewState = DataList ++ State, > loop(NewState); > {report, Asker} -> > Asker ! State, > loop(State); > shutdown -> > ok = io:format("~p: shutting down. Bye!~n", [self()]), > exit(normal); > Unexpected -> > ok = io:format("~p: Received unexpected messages: ~tp~n", > [self(), Unexpected]), > loop(State) > end. > > do_processing(Event) -> > Size = length(Event), > ok = io:format("~p: Received event of size: ~p~n", [self(), Size]), > {Size, Event}. > > > Obviously, the `do_processing/1` function is arbitrary, but you can imagine > a process maintaining its own state that is mutable *in time* but uses > symbols which are *immutable within their scope* to represent the > actual calculations involved in whatever transforms are being performed. > > This is a *lot* closer to the way things are often done in "pure" Erlang, > meaning, this is how things are often done in Erlang when using processes > to maintain state, modules to represent a process's definition and hide > implementation from public functions, and messages to represent events > within a system. > > There are more details, but I think you're just getting hung up on some > of the very early differences -- and that is totally normal. > > As far as functional idioms go... there is a very basic core of operations > one expects to find in pretty much any functional language, and they tend > to be centered around list operations. The Erlang `lists` standard lib > module documentation provides a terse, but pretty solid representation of > them (but also includes a lot of stuff specific to Erlang's own lib). > > What you are looking for is list operations like folds, maps and filters, > the basics of recursion (which can seem at first more weird than explicit > looping, but winds up becoming easier than explicit looping over time), and > some of the basic concepts underlying Erlang's style of managing > concurrency. > > I wrote a few bits about these basics -- forgive the links, I just don't > feel like re-writing it all here: > > An explanation of fold (which Richard was using as an example above): > http://stackoverflow.com/questions/26854586/explanation-of-listsfold-function/26855055#26855055 > > Erlang processes VS Java threads: > (tl;dr: The difference between Java and Erlang is much deeper than the > essentially cosmetic differences between OOP and FP.) > http://stackoverflow.com/questions/32294367/erlang-process-vs-java-thread/32296577#32296577 > > Clearing up some misunderstanding about recursion: > http://stackoverflow.com/questions/27234898/erlang-recursive-end-loop/27236236#27236236 > > A word on "function chaining", touching on error code: > http://stackoverflow.com/questions/34622869/function-chaining-in-erlang/34626317#34626317 > > Blah blah. > > Start messing around with the module above, since it runs. Morph it into > something that does whatever you want. Make two that talk to each other. > Every little experiment will introduce you to new things at first, and > give you better insights into what is being said if you start to read a > book like "Learn You Some Erlang" (free online > http://learnyousomeerlang.com/ ), > "Erlang and OTP In Action", "Programming Erlang", etc. (all strong > recommendations). > > As for getting a feel for what exists beyond C and asm, and letting your > mind relax and come to terms with abstractions based on primitives, I still > can't think of a better text than "Structure and Interpretation of Computer > Programs" (also free online: > https://mitpress.mit.edu/sicp/full-text/book/book.html ). > > -Craig > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From ok@REDACTED Wed Apr 27 02:08:13 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 27 Apr 2016 12:08:13 +1200 Subject: [erlang-questions] Clueless am I In-Reply-To: <571F1FC8.5000205@rol.ru> References: <571E6CEC.5070305@aim.com> <008101d19f28$d3891660$7a9b4320$@xss.de> <571E727B.30309@aim.com> <571F1FC8.5000205@rol.ru> Message-ID: <39b2bdc2-5eff-471f-ceb1-75feb2839c40@cs.otago.ac.nz> On 26/04/16 7:59 PM, Konstantin Vsochanov wrote: > Actually, there is one - process dictionary > But you have to use it with care. The process dictionary facility is not strictly speaking a DATA STRUCTURE. (You could call it a data OBJECT.) In any process, there is only one process dictionary. You cannot create new ones. You cannot pass them as arguments. You cannot return them from function. You cannot bind a variable to a process dictionary. The keys and values stored in a process's dictionary are themselves immutable. Process dictionaries, except for their scope, are much more like ETS and DETS tables than they are like lists or tuples or maps. (Come to think of it, there *is* a C analogue for the process dictionary, and that's the unique hash table manipulated by the traditional hsearch() function. #include didn't give you any way to create additional hash tables. No data type, only a single data object.) From ok@REDACTED Wed Apr 27 02:45:19 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 27 Apr 2016 12:45:19 +1200 Subject: [erlang-questions] Clueless am I In-Reply-To: <20160426104520.7e43b23f706d1a78218bd3e1c66e57ee.1da16ce41a.wbe@email23.secureserver.net> References: <20160426104520.7e43b23f706d1a78218bd3e1c66e57ee.1da16ce41a.wbe@email23.secureserver.net> Message-ID: On 27/04/16 5:45 AM, duncan@REDACTED wrote: > One thing that helped me as a beginner wrt the 'immutable variable', > and I think will help with understanding this whole thread, is that > the C mindset contains not only the 'mutable variable' but also the > 'loop' construct. It's important to note there are no loops in erlang. > Even though 'loop' is used in the post below, it's not a c-like loop - > it's a function call. Pop-2 had two "if"-like constructs: if E1 then S1 {elseif E2 then S2}... [else S0] close loopif E1 then S1 {elseif E2 then S2}... [else S0] close Then conditions E1... are tested in the order written and when Ei turns out to be true, the corresponding Si is executed. loopif is the same as if except that if any Si is executed, control returns to the beginning. (If there's an 'else' in there, this means you had better have a 'return' somewhere to stop the loop.) Dijkstra's classic "A Discipline of Programming" had two "if"-like constructs: if E1 -> S1 {[] E2 -> S2}... fi do E1 -> S1 {[] E2 -> S2}... od The conditions E1... may be tested in order; "[] E2 -> S2" does *not* get to assume that E1 is false. If no Ei is true, "if" reports an error and "do" stops. Otherwise, "do" is just like "if" except that each Si will branch back to the beginning. Now Erlang doesn't really have an equivalent of "if". Erlang's basic control structure is "case": case E of P1 when G1 -> S1 ; ... ; Pn when Gn -> Sn end If we wrap that in a function, trichoplusia(Parameters) -> case E of P1 when G1 -> S1, trichoplusia(Parameters') ; ... ; Pn when Gn -> Sn, trichoplusia(Parameters') ; _ -> Result end. (trichoplusia? The cabbage looper.) what we get is *precisely* a Dijkstra/Pop-2 multibranch loop. Let's take a simple C-like program for summing the elements of an array. n = size of the array; i = 0; s = 0; while (i < n) s += array[i], i += 1; return s; Now let's turn that into Erlang, allowing for the fact that Erlang tuples are indexed from 1. sum(tuple_size(Tuple), 1, 0, Tuple) where sum(N, I, S, T) -> if N < I -> S ; true -> sum(N, I+1, S+element(I,S), T) end. Thanks to last call optimisation, the recursive call to sum/4 is just "assign updated values to parameters and JUMP". From where I stand, this *IS* a while loop. (More precisely, a while loop is a not-very-interesting special case of a multiway loop.) C has loop over x, erlang has recurse over x. I suppose the battle is lost, but the verb that "recursion" is derived from is "recurrere" from which we get "recur". It's "recur" + "tion". English "Recurse" is an ugly and pointless back-formation. Recurring function calls are the *primitive* mechanism that functional programming languages use to build traversals of recursively defined data structures. It is useful to write them out when learning a language, to show yourself there's no magic. In practice, you want to package common patterns up as higher order functions, and write as little recursive code of your own as you practically can. For example, a better way to handle summing the elements of a tuple is to separate that into "folding" over a tuple and an application of that: tuple_foldl(Fun, Acc, Tuple) -> tuple_foldl_aux(Fun, Acc, Tuple, 1, tuple_size(Tuple)). tuple_foldl_aux(Fun, Acc, Tuple, I, N) -> if N < I -> Acc ; true -> tuple_foldl_aux(Fun, Fun(element(I, Tuple), Acc), Tuple, I+1, N) end. sum(Tuple) -> tuple_foldl(fun (E, S) -> S+E end, 0, Tuple). Why is this better? Because tuple_foldl/3 and its helper function can go in a library module and their details forgotten. From then on, when you want to combine the elements of a tuple, all you have to do is figure out what the result should be for no elements, and how to combine one more element into a running totl, and you're done. Your code becomes "index-free". You don't make any indexing mistakes. From ok@REDACTED Wed Apr 27 02:50:39 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Wed, 27 Apr 2016 12:50:39 +1200 Subject: [erlang-questions] Clueless am I In-Reply-To: References: <20160426104520.7e43b23f706d1a78218bd3e1c66e57ee.1da16ce41a.wbe@email23.secureserver.net> Message-ID: <0564bf00-6edc-5d86-c223-1b10ab58f5ce@cs.otago.ac.nz> On 27/04/16 12:45 PM, Richard A. O'Keefe wrote: > > Dijkstra's classic "A Discipline of Programming" had two "if"-like > constructs: > The conditions E1... may be tested in order; "in order" should have been "in ANY order". Sorry about that. From dangud@REDACTED Wed Apr 27 09:03:10 2016 From: dangud@REDACTED (Dan Gudmundsson) Date: Wed, 27 Apr 2016 07:03:10 +0000 Subject: [erlang-questions] Mnesia: inconsistent views without netsplit? In-Reply-To: References: Message-ID: 1) No clue. But would be interested if you have an idea what have gone wrong. 2) mnesia:del_table_copy(...) followed by mnesia:add_table_copy(..) should re-copy the table from the other nodes. On Tue, Apr 26, 2016 at 9:30 PM Daniel Dormont wrote: > Hi all, > > I have a three node Mnesia cluster (hosting a somewhat outdated > version of ejabberd, but I'm not sure that matters). I have a table > that is stored as ram_copies on all three nodes. Yet, this table has > differing numbers of records among the three. > > The table info from one of them is pasted below. Running the same > query on one of my other nodes, I get more or less the same result, > but the "size" is very different: 553 vs 867. And indeed, there are > individual records that turn up in a mnesia:read/2 or > mnesia:dirty_read/2 on one node and not the other. > > Yet, nothing in my log indicates that there was ever a netsplit or > disconnection. So I have two questions: > > 1) What might cause this? and > 2) Is there any way, especially given I know which records are > affected, to force some kind of replication on this table without > completely restarting one of the nodes? > > thanks, > Dan Dormont > > > [{access_mode,read_write}, > {active_replicas,['ejabberd@REDACTED', > 'ejabberd@REDACTED', > 'ejabberd@REDACTED']}, > {all_nodes,['ejabberd@REDACTED', > 'ejabberd@REDACTED', > 'ejabberd@REDACTED']}, > {arity,3}, > {attributes,[name_host,pid]}, > {checkpoints,[]}, > {commit_work,[]}, > {cookie,{{1341,344810,207763},'ejabberd@REDACTED'}}, > {cstruct,{cstruct,muc_online_room,set, > ['ejabberd@REDACTED', > 'ejabberd@REDACTED', > 'ejabberd@REDACTED'], > [],[],0,read_write,false,[],[],false,muc_online_room, > [name_host,pid], > [],[],[],{...},...}}, > {disc_copies,[]}, > {disc_only_copies,[]}, > {frag_properties,[]}, > {index,[]}, > {load_by_force,false}, > {load_node,'ejabberd@REDACTED'}, > {load_order,0}, > {load_reason,{active_remote,'ejabberd@REDACTED'}}, > {local_content,false}, > {majority,false}, > {master_nodes,[]}, > {memory,73643}, > {ram_copies,['ejabberd@REDACTED', > 'ejabberd@REDACTED', > 'ejabberd@REDACTED']}, > {record_name,muc_online_room}, > {record_validation,{muc_online_room,3,set}}, > {type,set}, > {size,867}, > {snmp,[]}, > {storage_properties,...}, > {...}|...] > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Wed Apr 27 09:12:33 2016 From: lukas@REDACTED (Lukas Larsson) Date: Wed, 27 Apr 2016 09:12:33 +0200 Subject: [erlang-questions] Max heap size Message-ID: Hello everyone, I recently opened up a new pull request on Github that adds a new process_flag that can be used to limit the heap size of a process. https://github.com/erlang/otp/pull/1032 I'd appreciate feedback on the api and semantics of the functionality. A simple example of the new functionality looks like this: 1> spawn_opt(fun() -> lists:seq(1,10000) end, [{max_heap_size, #{size => 1024, kill => true, error_logger => true}}]). <0.66.0> =ERROR REPORT==== 27-Apr-2016::09:08:26 === Process: <0.66.0> on node test@REDACTED Context: maximum heap size reached Max heap size: 1024 Total heap size: 1219 Kill: true Error Logger: true GC Info: [{old_heap_block_size,376}, {heap_block_size,843}, {mbuf_size,0}, {recent_size,176}, {stack_size,0}, {old_heap_size,288}, {heap_size,232}, {bin_vheap_size,0}, {bin_vheap_block_size,46422}, {bin_old_vheap_size,0}, {bin_old_vheap_block_size,46422}] Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From tony@REDACTED Wed Apr 27 10:10:20 2016 From: tony@REDACTED (Tony Rogvall) Date: Wed, 27 Apr 2016 10:10:20 +0200 Subject: [erlang-questions] Max heap size In-Reply-To: References: Message-ID: <90655E9B-9D6F-4ADE-8E1B-CCAD081AD7E2@rogvall.se> Wonderful! This will be useful. Thanks /Tony > On 27 apr 2016, at 09:12, Lukas Larsson wrote: > > Hello everyone, > > I recently opened up a new pull request on Github that adds a new process_flag that can be used to limit the heap size of a process. > > https://github.com/erlang/otp/pull/1032 > > I'd appreciate feedback on the api and semantics of the functionality. > > A simple example of the new functionality looks like this: > > > 1> spawn_opt(fun() -> lists:seq(1,10000) end, [{max_heap_size, #{size => 1024, kill => true, error_logger => true}}]). > <0.66.0> > > =ERROR REPORT==== 27-Apr-2016::09:08:26 === > Process: <0.66.0> on node test@REDACTED > Context: maximum heap size reached > Max heap size: 1024 > Total heap size: 1219 > Kill: true > Error Logger: true > GC Info: [{old_heap_block_size,376}, > {heap_block_size,843}, > {mbuf_size,0}, > {recent_size,176}, > {stack_size,0}, > {old_heap_size,288}, > {heap_size,232}, > {bin_vheap_size,0}, > {bin_vheap_block_size,46422}, > {bin_old_vheap_size,0}, > {bin_old_vheap_block_size,46422}] > > Lukas > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From henrik.x.nord@REDACTED Wed Apr 27 10:34:43 2016 From: henrik.x.nord@REDACTED (Henrik Nord X) Date: Wed, 27 Apr 2016 10:34:43 +0200 Subject: [erlang-questions] Patch package OTP 18.3.2 released Message-ID: <572079A3.2000607@ericsson.com> Patch Package: OTP 18.3.2 Git Tag: OTP-18.3.2 Date: 2016-04-27 Trouble Report Id: OTP-13261, OTP-13510, OTP-13511 Seq num: System: OTP Release: 18 Application: inets-6.2.2, ssl-7.3.1 Predecessor: OTP 18.3.1 Check out the git tag OTP-18.3.2, and build a full OTP system including documentation. Apply one or more applications from this build as patches to your installation using the 'otp_patch_apply' tool. For information on install requirements, see descriptions for each application version below. --------------------------------------------------------------------- --- inets-6.2.2 ----------------------------------------------------- --------------------------------------------------------------------- The inets-6.2.2 application can be applied independently of other applications on a full OTP 18 installation. --- Improvements and New Features --- OTP-13510 Application(s): inets Add environment information item peer_cert to mod_esi Full runtime dependencies of inets-6.2.2: erts-6.0, kernel-3.0, mnesia-4.12, runtime_tools-1.8.14, ssl-5.3.4, stdlib-2.0 --------------------------------------------------------------------- --- ssl-7.3.1 ------------------------------------------------------- --------------------------------------------------------------------- The ssl-7.3.1 application can be applied independently of other applications on a full OTP 18 installation. --- Fixed Bugs and Malfunctions --- OTP-13511 Application(s): ssl Corrections to cipher suite handling using the 3 and 4 tuple format in addition to commit 89d7e21cf4ae988c57c8ef047bfe85127875c70c --- Improvements and New Features --- OTP-13261 Application(s): ssl Make values for the TLS-1.2 signature_algorithms extension configurable Full runtime dependencies of ssl-7.3.1: crypto-3.3, erts-7.0, inets-5.10.7, kernel-3.0, public_key-1.0, stdlib-2.0 --------------------------------------------------------------------- --------------------------------------------------------------------- --------------------------------------------------------------------- From ingela.andin@REDACTED Wed Apr 27 12:29:23 2016 From: ingela.andin@REDACTED (Ingela Andin) Date: Wed, 27 Apr 2016 12:29:23 +0200 Subject: [erlang-questions] Memory consumption in ssl application (OTP-18.3.1) In-Reply-To: <571E0F10.80507@yandex.ru> References: <571E0F10.80507@yandex.ru> Message-ID: Hi! You could try using the observer application to find out more information about your system state. There is no apparent reason why ssl-7.3 should consume a lot more memory than ssl-6.0.1.2. But a lot has append between the versions and heavy load may find corner cases that where missed. Regards Ingela Erlang/OTP Team - Ericsson AB 2016-04-25 14:35 GMT+02:00 : > Hello, > > When I run > https://github.com/ninenines/cowboy/tree/1.0.4/examples/ssl_hello_world > under load (using https://github.com/JoeDog/siege) I found a big > difference in memory consumption between OTP-17.5 and OTP-18.3. I know > the ssl application has many changes in new versions. Is this memory > usage normal now or can be reduced using some ssl application settings? > > I use OTP-17.5.6.9 and OTP-18.3.1. > > > ================================================================================ > Erlang/OTP 17 [erts-6.4.1.6] [source] [64-bit] [smp:2:2] > [async-threads:10] [hipe] [kernel-poll:false] > > 1> application:which_applications(). > [{ssl_hello_world,"Cowboy Hello World example with SSL.", > "1"}, > {cowboy,"Small, fast, modular HTTP server.","1.0.4"}, > {ranch,"Socket acceptor pool for TCP protocols.","1.2.1"}, > {cowlib,"Support library for manipulating Web protocols.", > "1.0.2"}, > {ssl,"Erlang/OTP SSL application","6.0.1.2"}, > {public_key,"Public key infrastructure","0.23"}, > {crypto,"CRYPTO","3.5"}, > {asn1,"The Erlang ASN1 compiler version 3.0.4","3.0.4"}, > {stdlib,"ERTS CXC 138 10","2.4"}, > {kernel,"ERTS CXC 138 10","3.2.0.1"}] > > 2> erlang:memory(). > [{total,43913840}, > {processes,8036312}, > {processes_used,8030744}, > {system,35877528}, > {atom,470537}, > {atom_used,455904}, > {binary,1498800}, > {code,11074476}, > {ets,14516424}] > > 3> recon_alloc:memory(used). > 45519464 > > 4> recon_alloc:memory(usage). > 0.7212344918526264 > > 5> recon:proc_window(memory, 10, 500). > [{<0.25910.2>,68120, > [{current_function,{gen_fsm,loop,7}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.25908.2>,68120, > [{current_function,{gen_fsm,loop,7}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.284.0>,15808, > [ssl_manager, > {current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.285.0>,12712, > [tls_connection_sup, > {current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.25909.2>,5888, > [{current_function,{gen,do_call,4}}, > {initial_call,{cowboy_protocol,init,4}}]}, > {<0.324.0>,4888, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.302.0>,4808, > [{current_function,{ranch_conns_sup,loop,4}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.330.0>,3016, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.25911.2>,2848, > [{current_function,{gen,do_call,4}}, > {initial_call,{cowboy_protocol,init,4}}]}, > {<0.340.0>,1872, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}] > > 6> recon:proc_window(reductions, 10, 500). > [{<0.2634.3>,11161, > [{current_function,{gen_fsm,loop,7}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.2633.3>,9747, > [{current_function,{crypto,int_to_bin_pos,2}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.284.0>,4097, > [ssl_manager, > {current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.285.0>,3314, > [tls_connection_sup, > {current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.302.0>,1003, > [{current_function,{ranch_conns_sup,loop,4}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.378.0>,426, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.372.0>,425, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.379.0>,425, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.377.0>,425, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.343.0>,425, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}] > > RES in htop ~50-60M > > > ================================================================================ > > Erlang/OTP 18 [erts-7.3.1] [source] [64-bit] [smp:2:2] [async-threads:10] > [hipe] [kernel-poll:false] > > 1> application:which_applications(). > [{ssl_hello_world,"Cowboy Hello World example with SSL.", > "1"}, > {cowboy,"Small, fast, modular HTTP server.","1.0.4"}, > {ranch,"Socket acceptor pool for TCP protocols.","1.2.1"}, > {cowlib,"Support library for manipulating Web protocols.", > "1.0.2"}, > {ssl,"Erlang/OTP SSL application","7.3"}, > {public_key,"Public key infrastructure","1.1.1"}, > {crypto,"CRYPTO","3.6.3"}, > {asn1,"The Erlang ASN1 compiler version 4.0.2","4.0.2"}, > {recon,"Diagnostic tools for production use","2.3.1"}, > {stdlib,"ERTS CXC 138 10","2.8"}, > {kernel,"ERTS CXC 138 10","4.2"}] > > 2> erlang:memory(). > [{total,949153560}, > {processes,902969888}, > {processes_used,902558312}, > {system,46183672}, > {atom,437761}, > {atom_used,432875}, > {binary,48320}, > {code,11407283}, > {ets,883640}] > > 3> recon_alloc:memory(used). > 863194560 > > 4> recon_alloc:memory(usage). > 0.8005009562139471 > > 5> recon:proc_window(memory, 10, 500). > [{<0.290.0>,94360, > [ssl_manager, > {current_function,{ssl_manager,handle_cast,2}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.13589.3>,24640, > [{current_function,{ssl_manager,validate_session,3}}, > {initial_call,{ssl_manager,init_session_validator,1}}]}, > {<0.291.0>,12832, > [tls_connection_sup, > {current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.13590.3>,10960, > [{current_function,{gen,do_call,4}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.318.0>,7904, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.308.0>,3056, > [{current_function,{ranch_conns_sup,loop,4}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.385.0>,1872, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.254.0>,0, > [code_server, > {current_function,{code_server,loop,1}}, > {initial_call,{erlang,apply,2}}]}, > {<0.325.0>,0, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.357.0>,0, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}] > > 6> recon:proc_window(reductions, 10, 500). > [{<0.290.0>,232358, > [ssl_manager, > {current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.291.0>,3058, > [tls_connection_sup, > {current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.308.0>,1129, > [{current_function,{ranch_conns_sup,loop,4}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.339.0>,435, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.340.0>,430, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.355.0>,430, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.353.0>,430, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.334.0>,430, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.366.0>,430, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.360.0>,430, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}] > > RES in htop ~900-1000M > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From v@REDACTED Wed Apr 27 14:23:21 2016 From: v@REDACTED (Valentin Micic) Date: Wed, 27 Apr 2016 14:23:21 +0200 Subject: [erlang-questions] Max heap size In-Reply-To: References: Message-ID: Hi, Are you saying that a process will be terminated regardless of how much actual memory we have within a system as a whole? Well, how useful would it be to kill a mission critical process just because it is using slightly more memory that we have originally envisaged? Wouldn't it be more suitable to have a process flag that would instruct the VM to send a specific message to the process once particular heap size have been reached and let process decide what to do about it? Kind reagards V/ On 27 Apr 2016, at 9:12 AM, Lukas Larsson wrote: > Hello everyone, > > I recently opened up a new pull request on Github that adds a new process_flag that can be used to limit the heap size of a process. > > https://github.com/erlang/otp/pull/1032 > > I'd appreciate feedback on the api and semantics of the functionality. > > A simple example of the new functionality looks like this: > > > 1> spawn_opt(fun() -> lists:seq(1,10000) end, [{max_heap_size, #{size => 1024, kill => true, error_logger => true}}]). > <0.66.0> > > =ERROR REPORT==== 27-Apr-2016::09:08:26 === > Process: <0.66.0> on node test@REDACTED > Context: maximum heap size reached > Max heap size: 1024 > Total heap size: 1219 > Kill: true > Error Logger: true > GC Info: [{old_heap_block_size,376}, > {heap_block_size,843}, > {mbuf_size,0}, > {recent_size,176}, > {stack_size,0}, > {old_heap_size,288}, > {heap_size,232}, > {bin_vheap_size,0}, > {bin_vheap_block_size,46422}, > {bin_old_vheap_size,0}, > {bin_old_vheap_block_size,46422}] > > Lukas > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas@REDACTED Wed Apr 27 15:12:39 2016 From: lukas@REDACTED (Lukas Larsson) Date: Wed, 27 Apr 2016 15:12:39 +0200 Subject: [erlang-questions] Max heap size In-Reply-To: References: Message-ID: On Wed, Apr 27, 2016 at 2:23 PM, Valentin Micic wrote: > > Are you saying that a process will be terminated regardless of how much > actual memory we have within a system as a whole? > yes > Well, how useful would it be to kill a mission critical process just > because it is using slightly more memory that we have originally envisaged? > That would depend on the application. If you have a process that cannot die, then you should not put a max_heap_size on it. The limit should be a guard against extreme memory usage growth, not be a something you put very close to your normal maximum. If you want to use this limit I suggest first running it with kill => false and see what heap sizes gets reported to the error_logger and then set the max_heap_size to an appropriately high value. Wouldn't it be more suitable to have a process flag that would instruct the > VM to send a specific message to the process once particular heap size have > been reached and let process decide what to do about it? > You can build this using erlang:system_monitor(self(), [{large_heap, Sz}]) if you want to have it, or you could hook into the error_logger and react to the messages sent by max_heap_size. Part of the point of the max_heap_size is that the process should be killed without having to do the potentially expensive garbage collection that is about to come, which is only achievable through an untrappable exit signal. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From nathaniel@REDACTED Wed Apr 27 15:39:43 2016 From: nathaniel@REDACTED (Nathaniel Waisbrot) Date: Wed, 27 Apr 2016 09:39:43 -0400 Subject: [erlang-questions] Max heap size In-Reply-To: References: Message-ID: <62C98FFC-973D-4B24-BFF7-A137935549DC@waisbrot.net> Is there a way to globally set the spawn options? The typical case where I'd imagine using this is that I have an unforeseen bug in my code that exhausts memory and then the OOM-killer screws up the system and I don't get a clean trace of the error. So what I'd really like is to always set all of my processes to some max size (something close to the host's limit) without having to think about it. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: Message signed with OpenPGP using GPGMail URL: From vicbaz@REDACTED Wed Apr 27 15:44:15 2016 From: vicbaz@REDACTED (vicbaz@REDACTED) Date: Wed, 27 Apr 2016 16:44:15 +0300 Subject: [erlang-questions] Memory consumption in ssl application (OTP-18.3.1) In-Reply-To: References: <571E0F10.80507@yandex.ru> Message-ID: <5720C22F.3000404@yandex.ru> Hello, The reason was session_cache_server_max (defaults to 1000). The ssl_manager quickly get filled by {'$gen_cast',{invalidate_session,8443,{session,<<...>>,...}}} {delayed_clean_session,{8443,<<14,50,229,...>>},40986} messages. 27/04/16 13:29, Ingela Andin ?????: > Hi! > > You could try using the observer application to find out more > information about your system state. > There is no apparent reason why ssl-7.3 should consume a lot more memory > than ssl-6.0.1.2. > But a lot has append between the versions and heavy load may find corner > cases that where missed. > > Regards Ingela Erlang/OTP Team - Ericsson AB > > > 2016-04-25 14:35 GMT+02:00 >: > > Hello, > > When I run > https://github.com/ninenines/cowboy/tree/1.0.4/examples/ssl_hello_world > under load (using https://github.com/JoeDog/siege) I found a big > difference in memory consumption between OTP-17.5 and OTP-18.3. I know > the ssl application has many changes in new versions. Is this memory > usage normal now or can be reduced using some ssl application settings? > > I use OTP-17.5.6.9 and OTP-18.3.1. > > ================================================================================ > Erlang/OTP 17 [erts-6.4.1.6] [source] [64-bit] [smp:2:2] > [async-threads:10] [hipe] [kernel-poll:false] > > 1> application:which_applications(). > [{ssl_hello_world,"Cowboy Hello World example with SSL.", > "1"}, > {cowboy,"Small, fast, modular HTTP server.","1.0.4"}, > {ranch,"Socket acceptor pool for TCP protocols.","1.2.1"}, > {cowlib,"Support library for manipulating Web protocols.", > "1.0.2"}, > {ssl,"Erlang/OTP SSL application","6.0.1.2"}, > {public_key,"Public key infrastructure","0.23"}, > {crypto,"CRYPTO","3.5"}, > {asn1,"The Erlang ASN1 compiler version 3.0.4","3.0.4"}, > {stdlib,"ERTS CXC 138 10","2.4"}, > {kernel,"ERTS CXC 138 10","3.2.0.1"}] > > 2> erlang:memory(). > [{total,43913840}, > {processes,8036312}, > {processes_used,8030744}, > {system,35877528}, > {atom,470537}, > {atom_used,455904}, > {binary,1498800}, > {code,11074476}, > {ets,14516424}] > > 3> recon_alloc:memory(used). > 45519464 > > 4> recon_alloc:memory(usage). > 0.7212344918526264 > > 5> recon:proc_window(memory, 10, 500). > [{<0.25910.2>,68120, > [{current_function,{gen_fsm,loop,7}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.25908.2>,68120, > [{current_function,{gen_fsm,loop,7}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.284.0>,15808, > [ssl_manager, > {current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.285.0>,12712, > [tls_connection_sup, > {current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.25909.2>,5888, > [{current_function,{gen,do_call,4}}, > {initial_call,{cowboy_protocol,init,4}}]}, > {<0.324.0>,4888, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.302.0>,4808, > [{current_function,{ranch_conns_sup,loop,4}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.330.0>,3016, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.25911.2>,2848, > [{current_function,{gen,do_call,4}}, > {initial_call,{cowboy_protocol,init,4}}]}, > {<0.340.0>,1872, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}] > > 6> recon:proc_window(reductions, 10, 500). > [{<0.2634.3>,11161, > [{current_function,{gen_fsm,loop,7}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.2633.3>,9747, > [{current_function,{crypto,int_to_bin_pos,2}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.284.0>,4097, > [ssl_manager, > {current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.285.0>,3314, > [tls_connection_sup, > {current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.302.0>,1003, > [{current_function,{ranch_conns_sup,loop,4}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.378.0>,426, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.372.0>,425, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.379.0>,425, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.377.0>,425, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.343.0>,425, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}] > > RES in htop ~50-60M > > ================================================================================ > > Erlang/OTP 18 [erts-7.3.1] [source] [64-bit] [smp:2:2] > [async-threads:10] [hipe] [kernel-poll:false] > > 1> application:which_applications(). > [{ssl_hello_world,"Cowboy Hello World example with SSL.", > "1"}, > {cowboy,"Small, fast, modular HTTP server.","1.0.4"}, > {ranch,"Socket acceptor pool for TCP protocols.","1.2.1"}, > {cowlib,"Support library for manipulating Web protocols.", > "1.0.2"}, > {ssl,"Erlang/OTP SSL application","7.3"}, > {public_key,"Public key infrastructure","1.1.1"}, > {crypto,"CRYPTO","3.6.3"}, > {asn1,"The Erlang ASN1 compiler version 4.0.2","4.0.2"}, > {recon,"Diagnostic tools for production use","2.3.1"}, > {stdlib,"ERTS CXC 138 10","2.8"}, > {kernel,"ERTS CXC 138 10","4.2"}] > > 2> erlang:memory(). > [{total,949153560}, > {processes,902969888}, > {processes_used,902558312}, > {system,46183672}, > {atom,437761}, > {atom_used,432875}, > {binary,48320}, > {code,11407283}, > {ets,883640}] > > 3> recon_alloc:memory(used). > 863194560 > > 4> recon_alloc:memory(usage). > 0.8005009562139471 > > 5> recon:proc_window(memory, 10, 500). > [{<0.290.0>,94360, > [ssl_manager, > {current_function,{ssl_manager,handle_cast,2}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.13589.3>,24640, > [{current_function,{ssl_manager,validate_session,3}}, > {initial_call,{ssl_manager,init_session_validator,1}}]}, > {<0.291.0>,12832, > [tls_connection_sup, > {current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.13590.3>,10960, > [{current_function,{gen,do_call,4}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.318.0>,7904, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.308.0>,3056, > [{current_function,{ranch_conns_sup,loop,4}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.385.0>,1872, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.254.0>,0, > [code_server, > {current_function,{code_server,loop,1}}, > {initial_call,{erlang,apply,2}}]}, > {<0.325.0>,0, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.357.0>,0, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}] > > 6> recon:proc_window(reductions, 10, 500). > [{<0.290.0>,232358, > [ssl_manager, > {current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.291.0>,3058, > [tls_connection_sup, > {current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.308.0>,1129, > [{current_function,{ranch_conns_sup,loop,4}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.339.0>,435, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.340.0>,430, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.355.0>,430, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.353.0>,430, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.334.0>,430, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.366.0>,430, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.360.0>,430, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}] > > RES in htop ~900-1000M > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > From lukas@REDACTED Wed Apr 27 16:06:12 2016 From: lukas@REDACTED (Lukas Larsson) Date: Wed, 27 Apr 2016 16:06:12 +0200 Subject: [erlang-questions] Max heap size In-Reply-To: <62C98FFC-973D-4B24-BFF7-A137935549DC@waisbrot.net> References: <62C98FFC-973D-4B24-BFF7-A137935549DC@waisbrot.net> Message-ID: On Wed, Apr 27, 2016 at 3:39 PM, Nathaniel Waisbrot wrote: > Is there a way to globally set the spawn options? The typical case where > I'd imagine using this is that I have an unforeseen bug in my code that > exhausts memory and then the OOM-killer screws up the system and I don't > get a clean trace of the error. So what I'd really like is to always set > all of my processes to some max size (something close to the host's limit) > without having to think about it. > Yes, you can either set a default limit using "erl +hmax Limit", or in runtime using "erlang:system_flag(max_heap_size, Limit)". However if it is a global maximum you want, then you might want to consider to allocate a super carrier and then disallow any other memory creation. i.e. "erl +MMsco true +Musac false +MMscs 256". This will limit the data that the emulator will allocate to 256 MB, when reached a crash dump will be created. If you want to protect yourself further from the OOM killer you may also want to use +MMscrpm true, but that will make the memory Erlang uses unavailable to other processes on the system while Erlang is running. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Wed Apr 27 16:43:57 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 27 Apr 2016 10:43:57 -0400 Subject: [erlang-questions] Patch package OTP 18.3.2 released In-Reply-To: <572079A3.2000607@ericsson.com> References: <572079A3.2000607@ericsson.com> Message-ID: <20160427144355.GD2013@fhebert-ltm2.internal.salesforce.com> On 04/27, Henrik Nord X wrote: > --- Fixed Bugs and Malfunctions --- > > OTP-13511 Application(s): ssl > > Corrections to cipher suite handling using the 3 and 4 > tuple format in addition to commit > 89d7e21cf4ae988c57c8ef047bfe85127875c70c Well that is tricky now since it appears there is no longer any fix that allows to support all versions at once with a single configuration. I could get a config going in pre 18.3 by submitting 3-tuples. I could get 18.3 to work by using all tuples, looking for what was supported, and submitting the 3-tuple version no matter what. In 18.3.2, I *must* submit the 4-tuple version in this case, which can work, but I still get some odd failures, specifically around {rsa,aes_256_gcm,null,sha384} which still does not work with the new format: 37> lists:member({rsa,aes_256_gcm,null,sha384},ssl:cipher_suites()). true 38> ssl_cipher:suite({rsa,aes_256_gcm,null,sha384}). ** exception error: no function clause matching ssl_cipher:suite({rsa,aes_256_gcm,null,sha384}) (ssl_cipher.erl, line 754) 39> ssl_cipher:suite({rsa,aes_256_gcm,null}). <<0,157>> So this suite is supported, but cannot be checked there in a specific manner and requires a special case because it still only works with the 3-tuple as input -- and there may be more of these. It seems like the patch is not properly covering all cases? From ingela.andin@REDACTED Wed Apr 27 17:45:06 2016 From: ingela.andin@REDACTED (Ingela Andin) Date: Wed, 27 Apr 2016 17:45:06 +0200 Subject: [erlang-questions] Patch package OTP 18.3.2 released In-Reply-To: <20160427144355.GD2013@fhebert-ltm2.internal.salesforce.com> References: <572079A3.2000607@ericsson.com> <20160427144355.GD2013@fhebert-ltm2.internal.salesforce.com> Message-ID: Hi! We promise to be backwards compatible, not bug compatible. As a workaround you may use the openSSL string names for cipher suites this should work for all versions. I think that the tuple format has turned out to be a nuisance for more than one reason, and we may deprecate it in favour of for instance maps in the future. Regards Ingela Erlang/OTP Team - Ericsson AB 2016-04-27 16:43 GMT+02:00 Fred Hebert : > On 04/27, Henrik Nord X wrote: > >> --- Fixed Bugs and Malfunctions --- >> >> OTP-13511 Application(s): ssl >> >> Corrections to cipher suite handling using the 3 and 4 >> tuple format in addition to commit >> 89d7e21cf4ae988c57c8ef047bfe85127875c70c >> > > Well that is tricky now since it appears there is no longer any fix that > allows to support all versions at once with a single configuration. > > I could get a config going in pre 18.3 by submitting 3-tuples. I could get > 18.3 to work by using all tuples, looking for what was supported, and > submitting the 3-tuple version no matter what. > > In 18.3.2, I *must* submit the 4-tuple version in this case, which can > work, but I still get some odd failures, specifically around > {rsa,aes_256_gcm,null,sha384} which still does not work with the new format: > > 37> lists:member({rsa,aes_256_gcm,null,sha384},ssl:cipher_suites()). > true > 38> ssl_cipher:suite({rsa,aes_256_gcm,null,sha384}). > ** exception error: no function clause matching > ssl_cipher:suite({rsa,aes_256_gcm,null,sha384}) (ssl_cipher.erl, line > 754) > 39> ssl_cipher:suite({rsa,aes_256_gcm,null}). > <<0,157>> > > So this suite is supported, but cannot be checked there in a specific > manner and requires a special case because it still only works with the > 3-tuple as input -- and there may be more of these. > > It seems like the patch is not properly covering all cases? > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Wed Apr 27 18:33:39 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 27 Apr 2016 12:33:39 -0400 Subject: [erlang-questions] Patch package OTP 18.3.2 released In-Reply-To: References: <572079A3.2000607@ericsson.com> <20160427144355.GD2013@fhebert-ltm2.internal.salesforce.com> Message-ID: <20160427163338.GE2013@fhebert-ltm2.internal.salesforce.com> On 04/27, Ingela Andin wrote: >Hi! > >We promise to be backwards compatible, not bug compatible. As a workaround >you may use the openSSL string names for cipher suites this should work for >all versions. >I think that the tuple format has turned out to be a nuisance for more than >one reason, and we may deprecate it in favour of for instance maps in the >future. Given the documentation for SSL mentions: > Returns a list of supported cipher suites. cipher_suites() is > equivalent to cipher_suites(erlang). Type openssl is provided for > backwards compatibility with the old SSL, which used OpenSSL. I'm a bit hesitant about using the openssl format here, since it does not sound like the modern approach that should be taken. I've opened up a pull request with the required changes for what I believe would make things work: https://github.com/erlang/otp/pull/1033 Regards, Fred. From mononcqc@REDACTED Wed Apr 27 19:10:03 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Wed, 27 Apr 2016 13:10:03 -0400 Subject: [erlang-questions] Patch package OTP 18.3.2 released In-Reply-To: References: <572079A3.2000607@ericsson.com> <20160427144355.GD2013@fhebert-ltm2.internal.salesforce.com> Message-ID: <20160427171002.GF2013@fhebert-ltm2.internal.salesforce.com> On 04/27, Ingela Andin wrote: >We promise to be backwards compatible, not bug compatible. I feel I should expand on that. Right now I have a list of specified SSL ciphers of the form: {ecdhe_ecdsa,aes_256_gcm,null,sha384} {ecdhe_ecdsa,aes_256_cbc,sha384,sha384} {ecdhe_ecdsa,aes_256_cbc,sha384}, ... {rsa,aes_256_gcm,null,sha384} {rsa,aes_256_gcm,null} {rsa,aes_256_cbc,sha256} You'll note that this list contains some ciphers that may or may not contain the GCM component in them. The thing I'm doing then is that because GCM support is dependent upon platform and OpenSSL version (i.e. OSX by default does not ship with support for ecliptic curve + GCM mode, but does so on other versions), I will do either of the following things: 1. Filter out ciphersuites not supported by the VM 2. Mandate some ciphersuites to be supported for the app to boot. Prior to 18.3, the easiest way to do that was to compare the values given to what was being returned by `ssl:cipher_suites()': if the value was in, it was supported by the VM. Starting with 18.3, this started breaking since I had to check for 4-tuple support in there to make sure that a 3-tuple wasn't being passed where a 4-tuple was required and the opposite. THat's okay, I can work with that. The problem is that starting with 18.3.2, `ssl:cipher_suites()' returns a 4-tuple whether this format is actually being supported or not. This means that there isn't any way for me to configure things in any dynamic manner whatsoever, since I cannot validate which ciphersuite configurations are supported by using the functionality of the SSL library itself. See the following example in 18.3: 1> ssl:listen(0, [{ciphers,ssl:cipher_suites()}]). {ok,{sslsocket,...}} And see the same result in 18.3.2: 1> ssl:listen(0, [{ciphers,ssl:cipher_suites()}]). {error,{options,{ciphers,[{ecdhe_ecdsa,aes_256_gcm,null, sha384}, ... {ecdh_rsa,...}, {...}|...]}}} Maybe there's something I'm not getting, but it seems to me that the bug is in 18.3.2, not in the other versions. From vicbaz@REDACTED Wed Apr 27 20:52:54 2016 From: vicbaz@REDACTED (vicbaz@REDACTED) Date: Wed, 27 Apr 2016 21:52:54 +0300 Subject: [erlang-questions] Patch package OTP 18.3.2 released In-Reply-To: <572079A3.2000607@ericsson.com> References: <572079A3.2000607@ericsson.com> Message-ID: <57210A86.4040508@yandex.ru> Hello, When I run https://github.com/ninenines/cowboy/tree/1.0.4/examples/ssl_hello_world under OTP-18.3.2 I got 'unknown POSIX error' (see below). Could someone explain to me what happened? It works without problem under OTP-18.3.1. $ ./_rel/ssl_hello_world_example/bin/ssl_hello_world_example console Exec: /home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/erts-7.3.1/bin/erlexec -boot /home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/releases/1/ssl_hello_world_example -mode embedded -boot_var ERTS_LIB_DIR /home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/erts-7.3.1/../lib -config /home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/releases/1/sys.config -args_file /home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/releases/1/vm.args -- console Root: /home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example /home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example Erlang/OTP 18 [erts-7.3.1] [source] [64-bit] [smp:2:2] [async-threads:10] [hipe] [kernel-poll:false] =ERROR REPORT==== 27-Apr-2016::21:43:11 === Failed to start Ranch listener https in ranch_ssl:listen([{port,8443}, {cacertfile, "/home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/lib/ssl_hello_world-1/priv/ssl/cowboy-ca.crt"}, {certfile, "/home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/lib/ssl_hello_world-1/priv/ssl/server.crt"}, {keyfile, "/home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/lib/ssl_hello_world-1/priv/ssl/server.key"}]) for reason {options, {ciphers, [{ecdhe_ecdsa, aes_256_gcm, null, sha384}, {ecdhe_rsa, aes_256_gcm, null, sha384}, {ecdhe_ecdsa, aes_256_cbc, sha384, sha384}, {ecdhe_rsa, aes_256_cbc, sha384, sha384}, {ecdh_ecdsa, aes_256_gcm, null, sha384}, {ecdh_rsa, aes_256_gcm, null, sha384}, {ecdh_ecdsa, aes_256_cbc, sha384, sha384}, {ecdh_rsa, aes_256_cbc, sha384, sha384}, {dhe_rsa, aes_256_gcm, null, sha384}, {dhe_dss, aes_256_gcm, null, sha384}, {dhe_rsa, aes_256_cbc, sha256}, {dhe_dss, aes_256_cbc, sha256}, {rsa, aes_256_gcm, null, sha384}, {rsa, aes_256_cbc, sha256}, {ecdhe_ecdsa, aes_128_gcm, null, sha256}, {ecdhe_rsa, aes_128_gcm, null, sha256}, {ecdhe_ecdsa, aes_128_cbc, sha256, sha256}, {ecdhe_rsa, aes_128_cbc, sha256, sha256}, {ecdh_ecdsa, aes_128_gcm, null, sha256}, {ecdh_rsa, aes_128_gcm, null, sha256}, {ecdh_ecdsa, aes_128_cbc, sha256, sha256}, {ecdh_rsa, aes_128_cbc, sha256, sha256}, {dhe_rsa, aes_128_gcm, null, sha256}, {dhe_dss, aes_128_gcm, null, sha256}, {dhe_rsa, aes_128_cbc, sha256}, {dhe_dss, aes_128_cbc, sha256}, {rsa, aes_128_gcm, null, sha256}, {rsa, aes_128_cbc, sha256}, {ecdhe_ecdsa, aes_256_cbc, sha}, {ecdhe_rsa, aes_256_cbc, sha}, {dhe_rsa, aes_256_cbc, sha}, {dhe_dss, aes_256_cbc, sha}, {ecdh_ecdsa, aes_256_cbc, sha}, {ecdh_rsa, aes_256_cbc, sha}, {rsa, aes_256_cbc, sha}, {ecdhe_ecdsa, '3des_ede_cbc', sha}, {ecdhe_rsa, '3des_ede_cbc', sha}, {dhe_rsa, '3des_ede_cbc', sha}, {dhe_dss, '3des_ede_cbc', sha}, {ecdh_ecdsa, '3des_ede_cbc', sha}, {ecdh_rsa, '3des_ede_cbc', sha}, {rsa, '3des_ede_cbc', sha}, {ecdhe_ecdsa, aes_128_cbc, sha}, {ecdhe_rsa, aes_128_cbc, sha}, {dhe_rsa, aes_128_cbc, sha}, {dhe_dss, aes_128_cbc, sha}, {ecdh_ecdsa, aes_128_cbc, sha}, {ecdh_rsa, aes_128_cbc, sha}, {rsa, aes_128_cbc, sha}, {dhe_rsa, des_cbc, sha}, {rsa, des_cbc, sha}]}} (unknown POSIX error) Eshell V7.3.1 (abort with ^G) (ssl_hello_world_example@REDACTED)1> =INFO REPORT==== 27-Apr-2016::21:43:11 === application: ssl_hello_world exited: {bad_return, {{ssl_hello_world_app,start,[normal,[]]}, {'EXIT', {{badmatch, {error, {{shutdown, {failed_to_start_child,ranch_acceptors_sup, {listen_error,https, {options, {ciphers, [{ecdhe_ecdsa,aes_256_gcm,null,sha384}, {ecdhe_rsa,aes_256_gcm,null,sha384}, {ecdhe_ecdsa,aes_256_cbc,sha384,sha384}, {ecdhe_rsa,aes_256_cbc,sha384,sha384}, {ecdh_ecdsa,aes_256_gcm,null,sha384}, {ecdh_rsa,aes_256_gcm,null,sha384}, {ecdh_ecdsa,aes_256_cbc,sha384,sha384}, {ecdh_rsa,aes_256_cbc,sha384,sha384}, {dhe_rsa,aes_256_gcm,null,sha384}, {dhe_dss,aes_256_gcm,null,sha384}, {dhe_rsa,aes_256_cbc,sha256}, {dhe_dss,aes_256_cbc,sha256}, {rsa,aes_256_gcm,null,sha384}, {rsa,aes_256_cbc,sha256}, {ecdhe_ecdsa,aes_128_gcm,null,sha256}, {ecdhe_rsa,aes_128_gcm,null,sha256}, {ecdhe_ecdsa,aes_128_cbc,sha256,sha256}, {ecdhe_rsa,aes_128_cbc,sha256,sha256}, {ecdh_ecdsa,aes_128_gcm,null,sha256}, {ecdh_rsa,aes_128_gcm,null,sha256}, {ecdh_ecdsa,aes_128_cbc,sha256,sha256}, {ecdh_rsa,aes_128_cbc,sha256,sha256}, {dhe_rsa,aes_128_gcm,null,sha256}, {dhe_dss,aes_128_gcm,null,sha256}, {dhe_rsa,aes_128_cbc,sha256}, {dhe_dss,aes_128_cbc,sha256}, {rsa,aes_128_gcm,null,sha256}, {rsa,aes_128_cbc,sha256}, {ecdhe_ecdsa,aes_256_cbc,sha}, {ecdhe_rsa,aes_256_cbc,sha}, {dhe_rsa,aes_256_cbc,sha}, {dhe_dss,aes_256_cbc,sha}, {ecdh_ecdsa,aes_256_cbc,sha}, {ecdh_rsa,aes_256_cbc,sha}, {rsa,aes_256_cbc,sha}, {ecdhe_ecdsa,'3des_ede_cbc',sha}, {ecdhe_rsa,'3des_ede_cbc',sha}, {dhe_rsa,'3des_ede_cbc',sha}, {dhe_dss,'3des_ede_cbc',sha}, {ecdh_ecdsa,'3des_ede_cbc',sha}, {ecdh_rsa,'3des_ede_cbc',sha}, {rsa,'3des_ede_cbc',sha}, {ecdhe_ecdsa,aes_128_cbc,sha}, {ecdhe_rsa,aes_128_cbc,sha}, {dhe_rsa,aes_128_cbc,sha}, {dhe_dss,aes_128_cbc,sha}, {ecdh_ecdsa,aes_128_cbc,sha}, {ecdh_rsa,aes_128_cbc,sha}, {rsa,aes_128_cbc,sha}, {dhe_rsa,des_cbc,sha}, {rsa,des_cbc,sha}]}}}}}, {child,undefined, {ranch_listener_sup,https}, {ranch_listener_sup,start_link, [https,100,ranch_ssl, [{port,8443}, {cacertfile, "/home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/lib/ssl_hello_world-1/priv/ssl/cowboy-ca.crt"}, {certfile, "/home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/lib/ssl_hello_world-1/priv/ssl/server.crt"}, {keyfile, "/home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/lib/ssl_hello_world-1/priv/ssl/server.key"}], cowboy_protocol, [{env, [{dispatch, [{'_',[],[{[],[],toppage_handler,[]}]}]}]}]]}, permanent,infinity,supervisor, [ranch_listener_sup]}}}}, [{ssl_hello_world_app,start,2, [{file,"src/ssl_hello_world_app.erl"},{line,20}]}, {application_master,start_it_old,4, [{file,"application_master.erl"},{line,273}]}]}}}} type: permanent {"Kernel pid terminated",application_controller,"{application_start_failure,ssl_hello_world,{bad_return,{{ssl_hello_world_app,start,[normal,[]]},{'EXIT',{{badmatch,{error,{{shutdown,{failed_to_start_child,ranch_acceptors_sup,{listen_error,https,{options,{ciphers,[{ecdhe_ecdsa,aes_256_gcm,null,sha384},{ecdhe_rsa,aes_256_gcm,null,sha384},{ecdhe_ecdsa,aes_256_cbc,sha384,sha384},{ecdhe_rsa,aes_256_cbc,sha384,sha384},{ecdh_ecdsa,aes_256_gcm,null,sha384},{ecdh_rsa,aes_256_gcm,null,sha384},{ecdh_ecdsa,aes_256_cbc,sha384,sha384},{ecdh_rsa,aes_256_cbc,sha384,sha384},{dhe_rsa,aes_256_gcm,null,sha384},{dhe_dss,aes_256_gcm,null,sha384},{dhe_rsa,aes_256_cbc,sha256},{dhe_dss,aes_256_cbc,sha256},{rsa,aes_256_gcm,null,sha384},{rsa,aes_256_cbc,sha256},{ecdhe_ecdsa,aes_128_gcm,null,sha256},{ecdhe_rsa,aes_128_gcm,null,sha256},{ecdhe_ecdsa,aes_128_cbc,sha256,sha256},{ecdhe_rsa,aes_128_cbc,sha256,sha256},{ecdh_ecdsa,aes_128_gcm,null,sha256},{ecdh_rsa,aes_128_gcm,null,sha256},{ecdh_ecdsa,aes_128_cbc,sha256,sha256},{ec d h_rsa,aes_128_cbc,sha256,sha256},{dhe_rsa,aes_128_gcm,null,sha256},{dhe_dss,aes_128_gcm,null,sha256},{dhe_rsa,aes_128_cbc,sha256},{dhe_dss,aes_128_cbc,sha256},{rsa,aes_128_gcm,null,sha256},{rsa,aes_128_cbc,sha256},{ecdhe_ecdsa,aes_256_cbc,sha},{ecdhe_rsa,aes_256_cbc,sha},{dhe_rsa,aes_256_cbc,sha},{dhe_dss,aes_256_cbc,sha},{ecdh_ecdsa,aes_256_cbc,sha},{ecdh_rsa,aes_256_cbc,sha},{rsa,aes_256_cbc,sha},{ecdhe_ecdsa,'3des_ede_cbc',sha},{ecdhe_rsa,'3des_ede_cbc',sha},{dhe_rsa,'3des_ede_cbc',sha},{dhe_dss,'3des_ede_cbc',sha},{ecdh_ecdsa,'3des_ede_cbc',sha},{ecdh_rsa,'3des_ede_cbc',sha},{rsa,'3des_ede_cbc',sha},{ecdhe_ecdsa,aes_128_cbc,sha},{ecdhe_rsa,aes_128_cbc,sha},{dhe_rsa,aes_128_cbc,sha},{dhe_dss,aes_128_cbc,sha},{ecdh_ecdsa,aes_128_cbc,sha},{ecdh_rsa,aes_128_cbc,sha},{rsa,aes_128_cbc,sha},{dhe_rsa,des_cbc,sha},{rsa,des_cbc,sha}]}}}}},{child,undefined,{ranch_listener_sup,https},{ranch_listener_sup,start_link,[https,100,ranch_ssl,[{port,8443},{cacertfile,\"/home/victor/src/cowboy/exampl e s/ssl_hello_world/_rel/ssl_hello_world_example/lib/ssl_hello_world-1/priv/ssl/cowboy-ca.crt\"},{certfile,\"/home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/lib/ssl_hello_world-1/priv/ssl/server.crt\"},{keyfile,\"/home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/lib/ssl_hello_world-1/priv/ssl/server.key\"}],cowboy_protocol,[{env,[{dispatch,[{'_',[],[{[],[],toppage_handler,[]}]}]}]}]]},permanent,infinity,supervisor,[ranch_listener_sup]}}}},[{ssl_hello_world_app,start,2,[{file,\"src/ssl_hello_world_app.erl\"},{line,20}]},{application_master,start_it_old,4,[{file,\"application_master.erl\"},{line,273}]}]}}}}}"} Crash dump is being written to: erl_crash.dump...done On 27/04/16 11:34, Henrik Nord X wrote: > Patch Package: OTP 18.3.2 > Git Tag: OTP-18.3.2 > Date: 2016-04-27 > Trouble Report Id: OTP-13261, OTP-13510, OTP-13511 > Seq num: > System: OTP > Release: 18 > Application: inets-6.2.2, ssl-7.3.1 > Predecessor: OTP 18.3.1 > > Check out the git tag OTP-18.3.2, and build a full OTP system > including documentation. Apply one or more applications from this > build as patches to your installation using the 'otp_patch_apply' > tool. For information on install requirements, see descriptions for > each application version below. > > --------------------------------------------------------------------- > --- inets-6.2.2 ----------------------------------------------------- > --------------------------------------------------------------------- > > The inets-6.2.2 application can be applied independently of other > applications on a full OTP 18 installation. > > --- Improvements and New Features --- > > OTP-13510 Application(s): inets > > Add environment information item peer_cert to mod_esi > > > Full runtime dependencies of inets-6.2.2: erts-6.0, kernel-3.0, > mnesia-4.12, runtime_tools-1.8.14, ssl-5.3.4, stdlib-2.0 > > > --------------------------------------------------------------------- > --- ssl-7.3.1 ------------------------------------------------------- > --------------------------------------------------------------------- > > The ssl-7.3.1 application can be applied independently of other > applications on a full OTP 18 installation. > > --- Fixed Bugs and Malfunctions --- > > OTP-13511 Application(s): ssl > > Corrections to cipher suite handling using the 3 and 4 > tuple format in addition to commit > 89d7e21cf4ae988c57c8ef047bfe85127875c70c > > > --- Improvements and New Features --- > > OTP-13261 Application(s): ssl > > Make values for the TLS-1.2 signature_algorithms > extension configurable > > > Full runtime dependencies of ssl-7.3.1: crypto-3.3, erts-7.0, > inets-5.10.7, kernel-3.0, public_key-1.0, stdlib-2.0 > > > --------------------------------------------------------------------- > --------------------------------------------------------------------- > --------------------------------------------------------------------- > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From dan@REDACTED Wed Apr 27 21:32:23 2016 From: dan@REDACTED (Daniel Dormont) Date: Wed, 27 Apr 2016 15:32:23 -0400 Subject: [erlang-questions] Mnesia: inconsistent views without netsplit? In-Reply-To: References: Message-ID: On Wed, Apr 27, 2016 at 3:03 AM, Dan Gudmundsson wrote: > 1) No clue. But would be interested if you have an idea what have gone > wrong. > > 2) mnesia:del_table_copy(...) followed by mnesia:add_table_copy(..) should > re-copy the table from the other nodes. Thanks. I'll give that a try. Along the same lines I was wondering: is there a setting I can use to adjust the sensitivity of the system's detection of node disconnects, either generically or specifically within Mnesia? My production environment appears to have occasional momentary network hiccups (it's Amazon EC2 instances spanning zones within a region, for anyone curious). I'd like to make it less likely for those hiccups to cause Mnesia to enter an inconsistent state, even if it means real failures take a little longer to detect. thanks, Dan > > On Tue, Apr 26, 2016 at 9:30 PM Daniel Dormont > wrote: >> >> Hi all, >> >> I have a three node Mnesia cluster (hosting a somewhat outdated >> version of ejabberd, but I'm not sure that matters). I have a table >> that is stored as ram_copies on all three nodes. Yet, this table has >> differing numbers of records among the three. >> >> The table info from one of them is pasted below. Running the same >> query on one of my other nodes, I get more or less the same result, >> but the "size" is very different: 553 vs 867. And indeed, there are >> individual records that turn up in a mnesia:read/2 or >> mnesia:dirty_read/2 on one node and not the other. >> >> Yet, nothing in my log indicates that there was ever a netsplit or >> disconnection. So I have two questions: >> >> 1) What might cause this? and >> 2) Is there any way, especially given I know which records are >> affected, to force some kind of replication on this table without >> completely restarting one of the nodes? >> >> thanks, >> Dan Dormont >> >> >> [{access_mode,read_write}, >> {active_replicas,['ejabberd@REDACTED', >> 'ejabberd@REDACTED', >> 'ejabberd@REDACTED']}, >> {all_nodes,['ejabberd@REDACTED', >> 'ejabberd@REDACTED', >> 'ejabberd@REDACTED']}, >> {arity,3}, >> {attributes,[name_host,pid]}, >> {checkpoints,[]}, >> {commit_work,[]}, >> {cookie,{{1341,344810,207763},'ejabberd@REDACTED'}}, >> {cstruct,{cstruct,muc_online_room,set, >> ['ejabberd@REDACTED', >> 'ejabberd@REDACTED', >> 'ejabberd@REDACTED'], >> [],[],0,read_write,false,[],[],false,muc_online_room, >> [name_host,pid], >> [],[],[],{...},...}}, >> {disc_copies,[]}, >> {disc_only_copies,[]}, >> {frag_properties,[]}, >> {index,[]}, >> {load_by_force,false}, >> {load_node,'ejabberd@REDACTED'}, >> {load_order,0}, >> {load_reason,{active_remote,'ejabberd@REDACTED'}}, >> {local_content,false}, >> {majority,false}, >> {master_nodes,[]}, >> {memory,73643}, >> {ram_copies,['ejabberd@REDACTED', >> 'ejabberd@REDACTED', >> 'ejabberd@REDACTED']}, >> {record_name,muc_online_room}, >> {record_validation,{muc_online_room,3,set}}, >> {type,set}, >> {size,867}, >> {snmp,[]}, >> {storage_properties,...}, >> {...}|...] >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions From mkbucc@REDACTED Thu Apr 28 00:37:39 2016 From: mkbucc@REDACTED (Mark Bucciarelli) Date: Wed, 27 Apr 2016 18:37:39 -0400 Subject: [erlang-questions] erlfmt? Message-ID: Hi, I'm wondering if anyone here uses or knows of a utility like gofmt for Erlang. If you're not familiar with gofmt, it simply reads source code on stdin and writes formatted source code to stdout. I looked at erl_tidy but cannot see how to make it read stdin. Thanks, Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: From t@REDACTED Thu Apr 28 00:46:48 2016 From: t@REDACTED (Tristan Sloughter) Date: Wed, 27 Apr 2016 17:46:48 -0500 Subject: [erlang-questions] erlfmt? In-Reply-To: References: Message-ID: <1461797208.3148043.591715937.1FAE1C3E@webmail.messagingengine.com> Short answer: No. Long answer: No, but I'd love for someone with the time to clean up https://github.com/tsloughter/erl_tidy which is a rebar3 plugin named `fmt` around erl_tidy. the erl_tidy code has issues with type specs and other newer syntax and will currently just dump out the AST for those. These fixes would then, of course, need to be submitted upstream to OTP, but this plugin provides a testing ground and hopefully motivation to someone out there :) -- Tristan Sloughter t@REDACTED On Wed, Apr 27, 2016, at 05:37 PM, Mark Bucciarelli wrote: > Hi, > > I'm wondering if anyone here uses or knows of a utility like gofmt for > Erlang.? If you're not familiar with gofmt, it simply reads source > code on stdin and writes formatted source code to stdout. > > I looked at erl_tidy but cannot see how to make it read stdin. > > Thanks, > > Mark > _________________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Apr 28 01:27:37 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 28 Apr 2016 11:27:37 +1200 Subject: [erlang-questions] Max heap size In-Reply-To: References: Message-ID: <9b834742-3078-8fd9-e4e7-d1909a954c9d@cs.otago.ac.nz> On 27/04/16 7:12 PM, Lukas Larsson wrote: > > https://github.com/erlang/otp/pull/1032 Where is the documentation? - what are the units in which 'size' is measured? It would be *very* nasty if a program that worked fine in a 32-bit environment stopped working in a 64-bit environment because the size was in bytes. - are size, kill, and error_logger the only suboptions? - what is the performance cost of this? (Presumably it gets checked only after a garbage collection, prior to increasing the size of a process.) - I get twitchy when a parameter that can result in the death of a process is defined as the sum of something that *is* under the process's control and something that is *not*, and further, how much of that uncontrolled stuff is counted is determined by yet another flag that wasn't there in 18.2. As it is, the effect is that a process can be killed because *another* process is doing something bad. What, if anything, can be done to prevent that needs to be explained. - Guidelines about how to choose sensible sizes would be valuable. No, wait. They are *indispensable*. Of *course* this is useful, but it's starting to smell like pthread_attr_setstacksize(), where there is *no* way, given even complete knowledge of the intended behaviour of the code and sensible bounds on the amount of data to be processed, that you can determine a *safe* stack size by anything other than experiment. You are entirely at the mercy of the implementation, and the C implementation has no mercy. I'd personally be happier with something like ulimit(), where there is a hard limit (the one where you kill the process) and a soft limit (where you raise() an exception in the process to let it know there's going to be a problem soon). The garbage collector explanation we were recently told about is a good beginning. Memory management measures and options are now so complex that we need a complete and exceedingly clear chapter in the official Erlang documentation to explain them. From ok@REDACTED Thu Apr 28 01:32:41 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 28 Apr 2016 11:32:41 +1200 Subject: [erlang-questions] Max heap size In-Reply-To: References: Message-ID: On 28/04/16 1:12 AM, Lukas Larsson wrote: > The limit should be a guard against extreme memory usage growth, > not be a something you put very close to your normal maximum. > If you want to use this limit I suggest first running it with kill => false > and see what heap sizes gets reported to the error_logger and then > set the max_heap_size to an appropriately high value. This must go into the documentation for the new flag. Wouldn't it be more suitable to have a process flag that would instruct the VM to send a specific message to the process once particular heap size have been reached and let process decide what to do about it? > You can build this using erlang:system_monitor(self(), [{large_heap, Sz}]) > if you want to have it, or you could hook into the error_logger and react > to the messages sent by max_heap_size. This also must go into the documentation. > Part of the point of the max_heap_size is that the process should be killed > without having to do the potentially expensive garbage collection that is > about to come, which is only achievable through an untrappable exit signal. I don't understand this. How can you determine how much heap a process really needs without doing a garbage collection? From ok@REDACTED Thu Apr 28 05:52:28 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 28 Apr 2016 15:52:28 +1200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? Message-ID: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> I've been thinking for some time of writing a paper with the title "Why can't I see the structure?" based on the idea that modules in every programming language I know look like blobs. I'm aware of visual notations like UML, BON, SDL, and what was it, Visual Erlang? But for me, those are just spaghetti with meatballs; once you get beyond just a handful of boxes in your diagram, all diagrams look much the same. In any case, I'm interested in the medium scale. Why can't I see the structure in a 3000-line module, or even a 1000-line module? (I am not asserting that Erlang is particularly bad here. It isn't.) The kind of structure I'm interested in can, I think, be described as *rhetorical* structure, like relationships between paragraphs. My *belief* is that if this structure were made explicit, perhaps by textual structure, perhaps by annotations, perhaps by some graphical form (but probably derived from annotations), it would be easier to understand medium-sized wodges of code. I'm aware of annotation support in languages like Java and C# and for that matter, Smalltalk, but with the exception of Smalltalk, nobody seems to be using annotations in this way (and that exception is me). I'd be very interested in hearing from anyone else who has been thinking in this area. From tangxiaosheng@REDACTED Thu Apr 28 04:16:20 2016 From: tangxiaosheng@REDACTED (tangxiaosheng) Date: Thu, 28 Apr 2016 10:16:20 +0800 Subject: [erlang-questions] Does odbc support stored process Message-ID: <57217274.80201@21cn.com> Question for odbc stored process: I read odbc document, found that sql_query return: ResultTuple | [ResultTuple] |{error, Reason} result_tuple() = {updated, n_rows()} | {selected, col_names(), rows()} But for transaction: the return should be like: {executed, 1, [{19}]} General format maybe : {executed, n_rows(), [{Out1, Out2 ...] I read odbc test files, found executed, but that is only text, not real test. Jack Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From raould@REDACTED Thu Apr 28 06:36:17 2016 From: raould@REDACTED (Raoul Duke) Date: Wed, 27 Apr 2016 21:36:17 -0700 Subject: [erlang-questions] [ppig-discuss] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> Message-ID: Hear, hear! (I don't know what the solution is.) -------------- next part -------------- An HTML attachment was scrubbed... URL: From ok@REDACTED Thu Apr 28 08:08:52 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 28 Apr 2016 18:08:52 +1200 Subject: [erlang-questions] [ppig-discuss] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> Message-ID: <83aea38a-ae17-d6dd-49db-4269c51c6957@cs.otago.ac.nz> On 28/04/16 5:04 PM, Gergely Buday wrote: > I am not sure if it matters for you but how about Standard ML modules? Suffice it to say that I've been well aware of ML for a long time and have the latest release of SML/NJ installed on my desktop and laptop machines for a reason, think that SML's structures and functors are the bee's knees, and no, they *don't* help with *this* issue one little bit. Don't believe me? Try finding your way around /usr/local/smlnj/ckit/src/ast/build-ast.sml (all 3105 lines of it) without a native guide. OK, so build-ast.sml is something of an extreme case: nearly half of the lines of code in ckit/src/ast/ are in that one file. And I'm sure there's good reason for that. There are other rather large modules in the SML/NJ distribution. Even fairly small modules can have the same problem, where you're just presented with a list of blobs (although blobs with types are somewhat easier to make sense of) all looking pretty similar. When I say that "modules in EVERY programming language I know look like blobs", that includes languages I *like* and *esteem*, and it certainly includes languages with nested modules. People are getting sort of OK at commenting single functions. They've got the idea about providing comments for an entire module. But with the single exception of Smalltalk, nobody seems to do anything about meso-scale structures. And Smalltalk just classifies the methods of a class into groups. For example, in Pharo the methods for SequenceableCollection are categorised as - accessing - comparing - converting - copying - enumerating (iteration, to you) - private - removing - shuffling - sorting - splitjoin - testing and then a bunch of * extensions for package so-and-so and the categories are partly well defined and partly ad hoc. But the *relations* between the methods are not stated. (For examples, methods in the private category exist to support methods that are public. My Smalltalk annotations already let me express that, but I suspect that "support" isn't a simple concept.) From ok@REDACTED Thu Apr 28 08:16:40 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Thu, 28 Apr 2016 18:16:40 +1200 Subject: [erlang-questions] [ppig-discuss] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> Message-ID: <6268e968-7448-e100-c298-3e7f3ea6b781@cs.otago.ac.nz> On 28/04/16 5:09 PM, Flavius Aspra wrote: > > Do you have any real-life example where complex stuff do not look like > blobs? > Literate programs. I'm thinking of the book about LCC, where I never felt lost at all. > > I'd start with this. > Unfortunately, while literate programs help a lot, they do so by throwing a *lot* of text at the problem, and still don't make relationships other than "mentions" really explicit. The major problem with literate programming is that it's not happening. At *best* people are writing stuff like JavaDoc, and while that isn't too bad at describing interfaces that are just piles of procedures, it's useless for someone trying to maintain a Java class, because it encourages you not to write about hidden methods and there's no way to describe the relationships between the methods, so people are in effect being trained NOT to write the stuff I need to read. I'm hoping that comparatively lightweight annotations that could be exploited by an IDE (at least in principle) are something programmers might be willing to use. Even me! From pierrefenoll@REDACTED Thu Apr 28 08:24:33 2016 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Thu, 28 Apr 2016 08:24:33 +0200 Subject: [erlang-questions] erlfmt? In-Reply-To: <1461797208.3148043.591715937.1FAE1C3E@webmail.messagingengine.com> References: <1461797208.3148043.591715937.1FAE1C3E@webmail.messagingengine.com> Message-ID: I made https://github.com/fenollp/erlang-formatter though it depends on OTP's Emacs erlang-mode. erlang-mode has been the de facto authority in terms of linting. However it has some issues, like when indenting typed record definitions. Dropping the dependency on Emacs would be great but again, which dev/CI machine doesn't have Emacs? > On 28 Apr 2016, at 00:46, Tristan Sloughter wrote: > > Short answer: No. > > Long answer: No, but I'd love for someone with the time to clean up https://github.com/tsloughter/erl_tidy which is a rebar3 plugin named `fmt` around erl_tidy. the erl_tidy code has issues with type specs and other newer syntax and will currently just dump out the AST for those. These fixes would then, of course, need to be submitted upstream to OTP, but this plugin provides a testing ground and hopefully motivation to someone out there :) > > -- > Tristan Sloughter > t@REDACTED > > > >> On Wed, Apr 27, 2016, at 05:37 PM, Mark Bucciarelli wrote: >> Hi, >> >> I'm wondering if anyone here uses or knows of a utility like gofmt for Erlang. If you're not familiar with gofmt, it simply reads source code on stdin and writes formatted source code to stdout. >> >> I looked at erl_tidy but cannot see how to make it read stdin. >> >> Thanks, >> >> Mark >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From vladdu55@REDACTED Thu Apr 28 08:57:35 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 28 Apr 2016 08:57:35 +0200 Subject: [erlang-questions] [ppig-discuss] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: <6268e968-7448-e100-c298-3e7f3ea6b781@cs.otago.ac.nz> References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <6268e968-7448-e100-c298-3e7f3ea6b781@cs.otago.ac.nz> Message-ID: Hi! On Thu, Apr 28, 2016 at 8:16 AM, Richard A. O'Keefe wrote: > I'm hoping that comparatively lightweight annotations that could be > exploited by an IDE (at least in principle) are something programmers > might be willing to use. Even me! > What kind of annotations do you have in mind? For me, there are two main kinds of code complexity that need explanations: static and dynamic. The static one refers to the way functions call each other and can be mitigated by using small functions that do just one thing and by naming them in a meaningful way. Then a cross-referencing tool can connect the dots and produce a useful description of the structure. Annotations could help even further by providing a bit of "why"-knowledge. My feeling is that the same reason that makes naming functions difficult might make writing useful annotations just as difficult. The dynamic complexity is harder to tame, though. It's about run-time interactions that are best described by message diagrams, protocol descriptions and the like. (Often, the interactions are inter-modular too, but maybe this is not the meso-level you want to document?) I can't see what kind of annotations in the code would help here, except in toy-level examples. For me, this is the part that is usually most difficult to understand when reading code. best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From roger@REDACTED Thu Apr 28 09:09:20 2016 From: roger@REDACTED (Roger Lipscombe) Date: Thu, 28 Apr 2016 08:09:20 +0100 Subject: [erlang-questions] erlfmt? In-Reply-To: References: <1461797208.3148043.591715937.1FAE1C3E@webmail.messagingengine.com> Message-ID: I use vimerl (https://github.com/jimenezrick/vimerl), which has an escript included: https://github.com/jimenezrick/vimerl/blob/master/indent/erlang_indent.erl I *believe* that it's equivalent to Emacs erlang-mode, but I don't have Emacs installed :-P On 28 April 2016 at 07:24, Pierre Fenoll wrote: > I made https://github.com/fenollp/erlang-formatter though it depends on > OTP's Emacs erlang-mode. > > erlang-mode has been the de facto authority in terms of linting. > However it has some issues, like when indenting typed record definitions. > > Dropping the dependency on Emacs would be great but again, which dev/CI > machine doesn't have Emacs? > > On 28 Apr 2016, at 00:46, Tristan Sloughter wrote: > > Short answer: No. > > Long answer: No, but I'd love for someone with the time to clean up > https://github.com/tsloughter/erl_tidy which is a rebar3 plugin named `fmt` > around erl_tidy. the erl_tidy code has issues with type specs and other > newer syntax and will currently just dump out the AST for those. These fixes > would then, of course, need to be submitted upstream to OTP, but this plugin > provides a testing ground and hopefully motivation to someone out there :) > > -- > Tristan Sloughter > t@REDACTED > > > > On Wed, Apr 27, 2016, at 05:37 PM, Mark Bucciarelli wrote: > > Hi, > > I'm wondering if anyone here uses or knows of a utility like gofmt for > Erlang. If you're not familiar with gofmt, it simply reads source code on > stdin and writes formatted source code to stdout. > > I looked at erl_tidy but cannot see how to make it read stdin. > > Thanks, > > Mark > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From aschultz@REDACTED Thu Apr 28 09:57:10 2016 From: aschultz@REDACTED (Andreas Schultz) Date: Thu, 28 Apr 2016 09:57:10 +0200 (CEST) Subject: [erlang-questions] TLS 1.2 hash and signature selection Message-ID: <1537025893.11074.1461830230491.JavaMail.zimbra@tpip.net> Hi, I'm trying to understand how this code in tls_handshake.erl is supposed to work: available_signature_algs(undefined, SupportedHashSigns, _, {Major, Minor}) when (Major < 3) andalso (Minor < 3) -> SupportedHashSigns; available_signature_algs(#hash_sign_algos{hash_sign_algos = ClientHashSigns}, SupportedHashSigns, _, {Major, Minor}) when (Major < 3) andalso (Minor < 3) -> ordsets:intersection(ClientHashSigns, SupportedHashSigns); available_signature_algs(_, _, _, _) -> undefined. The signature extension was introduce in TLS 1.2, but the above code seems to perform signature algorithm filtering only when the version is lower than TLS 1.2. Or do I miss something? Regards Andreas From lukas.larsson@REDACTED Thu Apr 28 10:31:45 2016 From: lukas.larsson@REDACTED (Lukas Larsson) Date: Thu, 28 Apr 2016 10:31:45 +0200 Subject: [erlang-questions] Max heap size In-Reply-To: <9b834742-3078-8fd9-e4e7-d1909a954c9d@cs.otago.ac.nz> References: <9b834742-3078-8fd9-e4e7-d1909a954c9d@cs.otago.ac.nz> Message-ID: On Thu, Apr 28, 2016 at 1:27 AM, Richard A. O'Keefe wrote: > > > On 27/04/16 7:12 PM, Lukas Larsson wrote: > >> >> https://github.com/erlang/otp/pull/1032 >> > > Where is the documentation? > The documentation is part of the pull request, the options and behavior are described here: https://github.com/erlang/otp/pull/1032/commits/629c6c0a4aea094bea43a74ca1c1664ec1041e43#diff-0fc9fb0d3e12721dd1574a543916e8c6R4349 > - what are the units in which 'size' is measured? > It would be *very* nasty if a program that worked fine in a 32-bit > environment stopped working in a 64-bit environment because the > size was in bytes. > > It is the same as min_heap_size and all other options that we that effect the process heap, the internal word size, erlang:system_info({wordsize,internal}). > - are size, kill, and error_logger the only suboptions? > > at the moment yes. > - what is the performance cost of this? > If not enabled, it costs one branch per gc. If enabled, most of the calculations have to be done anyways in order to allocate the new to space for the collector, so not much extra calculations needed there either. It also costs one machine word of memory per process. > (Presumably it gets checked only after a garbage collection, > prior to increasing the size of a process.) > It gets checked after what may be called the initialization phase of the GC. The first thing the GC does it to calculate how large a to space is needed for the GC to do it's job. After this calculation is done, the new code checks to see if the total heap size during collection will exceed the max heap size, if it does the appropriate action is taken before the collector starts. If that action is that the process should be killed, the collection does not start at all. > - I get twitchy when a parameter that can result in the death of > a process is defined as the sum of something that *is* under > the process's control and something that is *not*, and further, > how much of that uncontrolled stuff is counted is determined > by yet another flag that wasn't there in 18.2. As it is, the > effect is that a process can be killed because *another* process > is doing something bad. What, if anything, can be done to > prevent that needs to be explained. > > Yes indeed. This is one of the reasons that I'm skeptical about the usefulness of the option. It can be used to effectively protect against heap growth caused by bad code in the process, for instance doing binary_to_term on something unexpectedly large that someone on the internet sent you. It may catch some cases when the message queue grows huge, but if you have processes that may grow to have huge message queues you probably want to use the new `off_heap` message queue data option anyways which means that the messages in the queue are guaranteed to not be part of the heap which in turn means that they will not be counted towards the max_heap_size. > - Guidelines about how to choose sensible sizes would be valuable. > No, wait. They are *indispensable*. > > Of *course* this is useful, but it's starting to smell like > pthread_attr_setstacksize(), where there is *no* way, given even > complete knowledge of the intended behaviour of the code and > sensible bounds on the amount of data to be processed, that > you can determine a *safe* stack size by anything other than > experiment. You are entirely at the mercy of the implementation, > and the C implementation has no mercy. > > The main reason that setstacksize is so hard to do is that you don't want to put the limit too high as you would then waste that memory. So you want to put it as close as possible to you actual max stack size, but have to make very sure that you don't give too little. The analogy to ulimit seems more appropriate, as you can put the limit well above (one or two orders of magnitude) what you expect the process to use and still catch it before the VM is brought down due to out of memory. > I'd personally be happier with something like ulimit(), where there > is a hard limit (the one where you kill the process) and a soft > limit (where you raise() an exception in the process to let it know > there's going to be a problem soon). > > We talked quite a lot about having the option of raising an exception when the max heap size is reached but decided against it as it would mean that all code that you write and libraries you use has to expect the exception. So any old code that has a catch all would catch the max_heap_size exception and possibly hide that the error happened. The semantics also become very convoluted once we started looking at the details of how such an exception might work. I'm unsure how useful having a softlimit that sends a message would be. It is however something that we may add in the future if a good use-case for it is presented. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Thu Apr 28 10:35:54 2016 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 28 Apr 2016 10:35:54 +0200 Subject: [erlang-questions] EEP site not working In-Reply-To: References: Message-ID: <20160428083554.GA44314@erix.ericsson.se> On Mon, Apr 25, 2016 at 01:44:11PM +0200, Vlad Dumitrescu wrote: > Hi, > > I'm not sure if it's just me, but when I check the EEP web pages, like > https://www.erlang.org/erlang-enhancement-proposals/home/eep-0003.html, I > get an error because there is a redirection loop between www.erlang.org and > erlang.org Now Erlang Solutions have fixed the problem. > > best regards, > Vlad > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From vladdu55@REDACTED Thu Apr 28 10:38:55 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 28 Apr 2016 10:38:55 +0200 Subject: [erlang-questions] EEP site not working In-Reply-To: <20160428083554.GA44314@erix.ericsson.se> References: <20160428083554.GA44314@erix.ericsson.se> Message-ID: Hi, Now I get 404 errors for these pages. regards, Vlad On Thu, Apr 28, 2016 at 10:35 AM, Raimo Niskanen < raimo+erlang-questions@REDACTED> wrote: > On Mon, Apr 25, 2016 at 01:44:11PM +0200, Vlad Dumitrescu wrote: > > Hi, > > > > I'm not sure if it's just me, but when I check the EEP web pages, like > > https://www.erlang.org/erlang-enhancement-proposals/home/eep-0003.html, > I > > get an error because there is a redirection loop between www.erlang.org > and > > erlang.org > > Now Erlang Solutions have fixed the problem. > > > > > best regards, > > Vlad > > > _______________________________________________ > > erlang-questions mailing list > > erlang-questions@REDACTED > > http://erlang.org/mailman/listinfo/erlang-questions > > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lukas.larsson@REDACTED Thu Apr 28 10:45:28 2016 From: lukas.larsson@REDACTED (Lukas Larsson) Date: Thu, 28 Apr 2016 10:45:28 +0200 Subject: [erlang-questions] Max heap size In-Reply-To: References: Message-ID: On Thu, Apr 28, 2016 at 1:32 AM, Richard A. O'Keefe wrote: > > Wouldn't it be more suitable to have a process flag that would > instruct the VM to send a specific message to the process once > particular heap size have been reached and let process decide what > to do about it? > > We could. I've yet to see a use-case for it though. The main problem is that if you involve another process you introduce latency in between the max_heap_size being triggered and the other process reacting to the problem, and by then it may be too late. By instructing the process to kill itself the user can know that the limit put on the process was not breached. > > > Part of the point of the max_heap_size is that the process should be > killed > > without having to do the potentially expensive garbage collection that is > > about to come, which is only achievable through an untrappable exit > signal. > > I don't understand this. How can you determine how much heap a process > really needs without doing a garbage collection? > > Feel free to ask for more details about how this is done if my answer in the other mail regarding the GC initialization phase was not clear enough. Lukas -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Thu Apr 28 11:17:54 2016 From: ingela.andin@REDACTED (Ingela Andin) Date: Thu, 28 Apr 2016 11:17:54 +0200 Subject: [erlang-questions] Patch package OTP 18.3.2 released In-Reply-To: <20160427171002.GF2013@fhebert-ltm2.internal.salesforce.com> References: <572079A3.2000607@ericsson.com> <20160427144355.GD2013@fhebert-ltm2.internal.salesforce.com> <20160427171002.GF2013@fhebert-ltm2.internal.salesforce.com> Message-ID: Hi! 2016-04-27 19:10 GMT+02:00 Fred Hebert : > On 04/27, Ingela Andin wrote: > >> We promise to be backwards compatible, not bug compatible. >> > > I feel I should expand on that. > > Right now I have a list of specified SSL ciphers of the form: > > {ecdhe_ecdsa,aes_256_gcm,null,sha384} > {ecdhe_ecdsa,aes_256_cbc,sha384,sha384} > {ecdhe_ecdsa,aes_256_cbc,sha384}, > ... > {rsa,aes_256_gcm,null,sha384} > {rsa,aes_256_gcm,null} > {rsa,aes_256_cbc,sha256} > > You'll note that this list contains some ciphers that may or may not > contain the GCM component in them. The thing I'm doing then is that because > GCM support is dependent upon platform and OpenSSL version (i.e. OSX by > default does not ship with support for ecliptic curve + GCM mode, but does > so on other versions), I will do either of the following things: > > 1. Filter out ciphersuites not supported by the VM > There is an internal function in ssl that does that ssl_cipher:filter_suites/1. It is used when the cipher option is not set to filter out suites from the default value that may not have support by crypto. I am thinking maybe it should be applied to the users option too and return an error if it becomes empty, or maybe make a new API function?! > 2. Mandate some ciphersuites to be supported for the app to boot. > > Prior to 18.3, the easiest way to do that was to compare the values > given to what was being returned by `ssl:cipher_suites()': if the value > was in, it was supported by the VM. > > Starting with 18.3, this started breaking since I had to check for 4-tuple > support in there to make sure that a 3-tuple wasn't being passed where a > 4-tuple was required and the opposite. THat's okay, I can work with that. > > The problem is that starting with 18.3.2, `ssl:cipher_suites()' returns a > 4-tuple whether this format is actually being supported or not. > > This means that there isn't any way for me to configure things in any > dynamic manner whatsoever, since I cannot validate which ciphersuite > configurations are supported by using the functionality of the SSL library > itself. > > See the following example in 18.3: > > 1> ssl:listen(0, [{ciphers,ssl:cipher_suites()}]). > {ok,{sslsocket,...}} > > And see the same result in 18.3.2: > > 1> ssl:listen(0, [{ciphers,ssl:cipher_suites()}]). > {error,{options,{ciphers,[{ecdhe_ecdsa,aes_256_gcm,null, > sha384}, > ... > {ecdh_rsa,...}, > {...}|...]}}} > > Maybe there's something I'm not getting, but it seems to me that the bug > is in 18.3.2, not in the other versions. > Humm there seems to be something missing from this patch that is present in master. I will look into if your PR if it is the correct fix or what would be. Of course we will fix this. Regards Ingela Erlang/OTP team - Ericsson AB -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Thu Apr 28 11:21:01 2016 From: ingela.andin@REDACTED (Ingela Andin) Date: Thu, 28 Apr 2016 11:21:01 +0200 Subject: [erlang-questions] Memory consumption in ssl application (OTP-18.3.1) In-Reply-To: <5720C22F.3000404@yandex.ru> References: <571E0F10.80507@yandex.ru> <5720C22F.3000404@yandex.ru> Message-ID: Hi! 2016-04-27 15:44 GMT+02:00 : > Hello, > > The reason was session_cache_server_max (defaults to 1000). > The ssl_manager quickly get filled by > > {'$gen_cast',{invalidate_session,8443,{session,<<...>>,...}}} > {delayed_clean_session,{8443,<<14,50,229,...>>},40986} > > messages. > > I see, we will looking into mitigating this effect. Regards Ingela Erlang/OTP team - Ericsson AB > 27/04/16 13:29, Ingela Andin ?????: > >> Hi! >> >> You could try using the observer application to find out more >> information about your system state. >> There is no apparent reason why ssl-7.3 should consume a lot more memory >> than ssl-6.0.1.2. >> But a lot has append between the versions and heavy load may find corner >> cases that where missed. >> >> Regards Ingela Erlang/OTP Team - Ericsson AB >> >> >> 2016-04-25 14:35 GMT+02:00 >: >> >> >> Hello, >> >> When I run >> >> https://github.com/ninenines/cowboy/tree/1.0.4/examples/ssl_hello_world >> under load (using https://github.com/JoeDog/siege) I found a big >> difference in memory consumption between OTP-17.5 and OTP-18.3. I know >> the ssl application has many changes in new versions. Is this memory >> usage normal now or can be reduced using some ssl application >> settings? >> >> I use OTP-17.5.6.9 and OTP-18.3.1. >> >> >> ================================================================================ >> Erlang/OTP 17 [erts-6.4.1.6] [source] [64-bit] [smp:2:2] >> [async-threads:10] [hipe] [kernel-poll:false] >> >> 1> application:which_applications(). >> [{ssl_hello_world,"Cowboy Hello World example with SSL.", >> "1"}, >> {cowboy,"Small, fast, modular HTTP server.","1.0.4"}, >> {ranch,"Socket acceptor pool for TCP protocols.","1.2.1"}, >> {cowlib,"Support library for manipulating Web protocols.", >> "1.0.2"}, >> {ssl,"Erlang/OTP SSL application","6.0.1.2"}, >> {public_key,"Public key infrastructure","0.23"}, >> {crypto,"CRYPTO","3.5"}, >> {asn1,"The Erlang ASN1 compiler version 3.0.4","3.0.4"}, >> {stdlib,"ERTS CXC 138 10","2.4"}, >> {kernel,"ERTS CXC 138 10","3.2.0.1"}] >> >> 2> erlang:memory(). >> [{total,43913840}, >> {processes,8036312}, >> {processes_used,8030744}, >> {system,35877528}, >> {atom,470537}, >> {atom_used,455904}, >> {binary,1498800}, >> {code,11074476}, >> {ets,14516424}] >> >> 3> recon_alloc:memory(used). >> 45519464 >> >> 4> recon_alloc:memory(usage). >> 0.7212344918526264 >> >> 5> recon:proc_window(memory, 10, 500). >> [{<0.25910.2>,68120, >> [{current_function,{gen_fsm,loop,7}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.25908.2>,68120, >> [{current_function,{gen_fsm,loop,7}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.284.0>,15808, >> [ssl_manager, >> {current_function,{gen_server,loop,6}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.285.0>,12712, >> [tls_connection_sup, >> {current_function,{gen_server,loop,6}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.25909.2>,5888, >> [{current_function,{gen,do_call,4}}, >> {initial_call,{cowboy_protocol,init,4}}]}, >> {<0.324.0>,4888, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.302.0>,4808, >> [{current_function,{ranch_conns_sup,loop,4}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.330.0>,3016, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.25911.2>,2848, >> [{current_function,{gen,do_call,4}}, >> {initial_call,{cowboy_protocol,init,4}}]}, >> {<0.340.0>,1872, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}] >> >> 6> recon:proc_window(reductions, 10, 500). >> [{<0.2634.3>,11161, >> [{current_function,{gen_fsm,loop,7}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.2633.3>,9747, >> [{current_function,{crypto,int_to_bin_pos,2}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.284.0>,4097, >> [ssl_manager, >> {current_function,{gen_server,loop,6}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.285.0>,3314, >> [tls_connection_sup, >> {current_function,{gen_server,loop,6}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.302.0>,1003, >> [{current_function,{ranch_conns_sup,loop,4}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.378.0>,426, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.372.0>,425, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.379.0>,425, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.377.0>,425, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.343.0>,425, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}] >> >> RES in htop ~50-60M >> >> >> ================================================================================ >> >> Erlang/OTP 18 [erts-7.3.1] [source] [64-bit] [smp:2:2] >> [async-threads:10] [hipe] [kernel-poll:false] >> >> 1> application:which_applications(). >> [{ssl_hello_world,"Cowboy Hello World example with SSL.", >> "1"}, >> {cowboy,"Small, fast, modular HTTP server.","1.0.4"}, >> {ranch,"Socket acceptor pool for TCP protocols.","1.2.1"}, >> {cowlib,"Support library for manipulating Web protocols.", >> "1.0.2"}, >> {ssl,"Erlang/OTP SSL application","7.3"}, >> {public_key,"Public key infrastructure","1.1.1"}, >> {crypto,"CRYPTO","3.6.3"}, >> {asn1,"The Erlang ASN1 compiler version 4.0.2","4.0.2"}, >> {recon,"Diagnostic tools for production use","2.3.1"}, >> {stdlib,"ERTS CXC 138 10","2.8"}, >> {kernel,"ERTS CXC 138 10","4.2"}] >> >> 2> erlang:memory(). >> [{total,949153560}, >> {processes,902969888}, >> {processes_used,902558312}, >> {system,46183672}, >> {atom,437761}, >> {atom_used,432875}, >> {binary,48320}, >> {code,11407283}, >> {ets,883640}] >> >> 3> recon_alloc:memory(used). >> 863194560 >> >> 4> recon_alloc:memory(usage). >> 0.8005009562139471 >> >> 5> recon:proc_window(memory, 10, 500). >> [{<0.290.0>,94360, >> [ssl_manager, >> {current_function,{ssl_manager,handle_cast,2}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.13589.3>,24640, >> [{current_function,{ssl_manager,validate_session,3}}, >> {initial_call,{ssl_manager,init_session_validator,1}}]}, >> {<0.291.0>,12832, >> [tls_connection_sup, >> {current_function,{gen_server,loop,6}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.13590.3>,10960, >> [{current_function,{gen,do_call,4}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.318.0>,7904, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.308.0>,3056, >> [{current_function,{ranch_conns_sup,loop,4}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.385.0>,1872, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.254.0>,0, >> [code_server, >> {current_function,{code_server,loop,1}}, >> {initial_call,{erlang,apply,2}}]}, >> {<0.325.0>,0, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.357.0>,0, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}] >> >> 6> recon:proc_window(reductions, 10, 500). >> [{<0.290.0>,232358, >> [ssl_manager, >> {current_function,{gen_server,loop,6}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.291.0>,3058, >> [tls_connection_sup, >> {current_function,{gen_server,loop,6}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.308.0>,1129, >> [{current_function,{ranch_conns_sup,loop,4}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.339.0>,435, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.340.0>,430, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.355.0>,430, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.353.0>,430, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.334.0>,430, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.366.0>,430, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.360.0>,430, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}] >> >> RES in htop ~900-1000M >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Thu Apr 28 11:51:34 2016 From: ingela.andin@REDACTED (Ingela Andin) Date: Thu, 28 Apr 2016 11:51:34 +0200 Subject: [erlang-questions] TLS 1.2 hash and signature selection In-Reply-To: <1537025893.11074.1461830230491.JavaMail.zimbra@tpip.net> References: <1537025893.11074.1461830230491.JavaMail.zimbra@tpip.net> Message-ID: Hi! No I think your understanding is correct. It ought to be (Major >= 3) andalso (Minor >= 3) Alas it seems the positive test case will succeeded in spite of this, embarrassing :( Good we caught it before 19 :), and 18.3.2 needs to be patched anyway ;) Regards Ingela OTP/Team - Ericsson AB 2016-04-28 9:57 GMT+02:00 Andreas Schultz : > Hi, > > I'm trying to understand how this code in tls_handshake.erl is > supposed to work: > > available_signature_algs(undefined, SupportedHashSigns, _, {Major, Minor}) > when (Major < 3) andalso (Minor < 3) -> > SupportedHashSigns; > available_signature_algs(#hash_sign_algos{hash_sign_algos = > ClientHashSigns}, SupportedHashSigns, > _, {Major, Minor}) when (Major < 3) andalso (Minor < > 3) -> > ordsets:intersection(ClientHashSigns, SupportedHashSigns); > available_signature_algs(_, _, _, _) -> > undefined. > > The signature extension was introduce in TLS 1.2, but the > above code seems to perform signature algorithm filtering > only when the version is lower than TLS 1.2. > > Or do I miss something? > > Regards > Andreas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From v@REDACTED Thu Apr 28 11:56:34 2016 From: v@REDACTED (Valentin Micic) Date: Thu, 28 Apr 2016 11:56:34 +0200 Subject: [erlang-questions] Max heap size In-Reply-To: <658F03F5-4F82-492F-BEA5-80388FA81AA9@gmail.com> References: <658F03F5-4F82-492F-BEA5-80388FA81AA9@gmail.com> Message-ID: <4421A0C3-0D9D-430E-B6E6-EA280B64D31D@pharos-avantgard.com> On 27 Apr 2016, at 3:14 PM, Ulf Wiger wrote: > >> On 27 Apr 2016, at 14:23, Valentin Micic wrote: >> >> Are you saying that a process will be terminated regardless of how much actual memory we have within a system as a whole? >> >> Well, how useful would it be to kill a mission critical process just because it is using slightly more memory that we have originally envisaged? >> > > Note that you are in full control of this - per process! > > If you have a mission-critical process that shouldn?t be killed, you simply don?t add this process flag. > > A great future addition would be allowing multiple tracers, so that you could e.g. put a GC tracer on processes that may need to be allowed to exceed a max size and not just be brutally killed for it. > > BR, > Ulf W A while back I've been testing a behavior of the Erlang's queue module "under stress conditions": I would enqueue, say, 500,000 elements; then remove them all, and subsequently enqueue another 100 elements. My prediction was that the memory will grow up to the point (as required by 500,000 entries), and then stop growing; that is to say, upon dequeueing all 500,000 entries, the subsequent enqueueing of 100 elements would not cause new memory to be allocated. This turned up to be a naive expectation as the memory just kept growing (and, in some way, that made sense, but let me stick to the topic). (*) For a solution to this problem I ended up doing something similar to what Lukas suggested in his reply to my earlier comment on this thread ( You can build this using erlang:system_monitor(self(), [{large_heap, Sz}]) ) As I knew (more or less) what the size of the individual element was, I was able to track memory consumption as a function of number of elements in the queue, and thus establish anomalies where I had extraordinary amount of memory relative to the size of the queue, and induce GC in order to free the excess memory. * * * I had two reasons for writing the above introduction. First, I wanted to show a specific situation where "trigger-happy-kill-it" approach would not do us any justice (and, yeah, yeah... I know. I don't have to use it). Second, I would really like to learn about specific situation where killing the process for exceeding the memory quota would actually make sense. In my current view: - if you can afford to kill a process, maybe you shouldn't be running it in a first place; - Nobody learned much from fixing problems by switching things on and off; - if you don't know what you're doing, maybe you should learn how to do it before doing it. My suggestion to let the process know and decide what to do next was motivated by these views. Ulf, for what is worth, your suggestion to add additional tracers resonates quite well with me. Using error_logger for that purpose is actually presuming that reaching a memory quota is an actual error and it doesn't require any leap of faith from there to conclude it would be logical to terminate the "offender". In my view, this situation should be rather classified as an "operating condition", and if we were to spend any system resources in tracking that condition, maybe we should consider a scenario which has a chance to provide the most bang for the CPU buck. Kind regards V/ (*) To be fair, I did this testing about 8 years ago and I may have missed something out then, and there is strong possibility that the current VM handles memory management in a different way. -------------- next part -------------- An HTML attachment was scrubbed... URL: From raimo+erlang-questions@REDACTED Thu Apr 28 12:21:39 2016 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Thu, 28 Apr 2016 12:21:39 +0200 Subject: [erlang-questions] EEP site not working In-Reply-To: References: <20160428083554.GA44314@erix.ericsson.se> Message-ID: <20160428102139.GA79617@erix.ericsson.se> On Thu, Apr 28, 2016 at 10:38:55AM +0200, Vlad Dumitrescu wrote: > Hi, > > Now I get 404 errors for these pages. Does this one work for you? It works for me: http://www.erlang.org/erlang-enhancement-proposals/eep-0003.html > > regards, > Vlad > > > On Thu, Apr 28, 2016 at 10:35 AM, Raimo Niskanen < > raimo+erlang-questions@REDACTED> wrote: > > > On Mon, Apr 25, 2016 at 01:44:11PM +0200, Vlad Dumitrescu wrote: > > > Hi, > > > > > > I'm not sure if it's just me, but when I check the EEP web pages, like > > > https://www.erlang.org/erlang-enhancement-proposals/home/eep-0003.html, > > I > > > get an error because there is a redirection loop between www.erlang.org > > and > > > erlang.org > > > > Now Erlang Solutions have fixed the problem. > > > > > > > > best regards, > > > Vlad -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From chandrashekhar.mullaparthi@REDACTED Thu Apr 28 12:34:02 2016 From: chandrashekhar.mullaparthi@REDACTED (Chandru) Date: Thu, 28 Apr 2016 11:34:02 +0100 Subject: [erlang-questions] Max heap size In-Reply-To: <4421A0C3-0D9D-430E-B6E6-EA280B64D31D@pharos-avantgard.com> References: <658F03F5-4F82-492F-BEA5-80388FA81AA9@gmail.com> <4421A0C3-0D9D-430E-B6E6-EA280B64D31D@pharos-avantgard.com> Message-ID: One use I can see for this option is during stress testing where if one was left scratching one's head about where a problem lay, adding this option into some of the suspect processes might give some clues. Or, adding this option to some long lived processes might highlight hotspots in the system where one wasn't expecting a hotspot. I must admit I'm finding it difficult to imagine using this option sensibly. Partly because in a language where a user is not expected to do any memory management, it is hard to really understand how the heap size is growing. And as memory management in beam becomes more and more complex (quite rightly), it will be harder to keep track of this. Admittedly, for high performance systems there is no shying away from understanding all this but still this option seems a bit 'artificial'. In the ticket on Github I raised the issue of bounded message queues (again). I think for the average Erlang programmer, that is a more intuitive thing to understand. Too many messages in a message queue means that my process is a bottleneck and I need to redesign. Off-heap message queues which are coming in R19 don't really help with this - regardless of how efficient we make the behaviour of a process with large message queues, it is not helping us fix the root cause of the problem which is lack of flow control for message passing. Again, I understand that there is no easy way to deliver this feature, but in my view that would be more useful. regards, Chandru On 28 April 2016 at 10:56, Valentin Micic wrote: > > On 27 Apr 2016, at 3:14 PM, Ulf Wiger wrote: > > > On 27 Apr 2016, at 14:23, Valentin Micic wrote: > > Are you saying that a process will be terminated regardless of how much > actual memory we have within a system as a whole? > > Well, how useful would it be to kill a mission critical process just > because it is using slightly more memory that we have originally envisaged? > > > Note that you are in full control of this - per process! > > If you have a mission-critical process that shouldn?t be killed, you > simply don?t add this process flag. > > A great future addition would be allowing multiple tracers, so that you > could e.g. put a GC tracer on processes that may need to be allowed to > exceed a max size and not just be brutally killed for it. > > BR, > Ulf W > > > A while back I've been testing a behavior of the Erlang's queue module > "under stress conditions": I would enqueue, say, 500,000 elements; then > remove them all, and subsequently enqueue another 100 elements. > My prediction was that the memory will grow up to the point (as required > by 500,000 entries), and then stop growing; that is to say, upon dequeueing > all 500,000 entries, the subsequent enqueueing of 100 elements would not > cause new memory to be allocated. > > This turned up to be a naive expectation as the memory just kept growing > (and, in some way, that made sense, but let me stick to the topic). (*) > > For a solution to this problem I ended up doing something similar to what > Lukas suggested in his reply to my earlier comment on this thread ( *You > can build this using erlang:system_monitor(self(), [{large_heap, Sz}]) *) > > As I knew (more or less) what the size of the individual element was, I > was able to track memory consumption as a function of number of elements in > the queue, and thus establish anomalies where I had extraordinary amount of > memory relative to the size of the queue, and induce GC in order to free > the excess memory. > > * * * > > I had two reasons for writing the above introduction. > > First, I wanted to show a specific situation where "trigger-happy-kill-it" > approach would not do us any justice (and, yeah, yeah... I know. I don't > have to use it). > > Second, I would really like to learn about specific situation where > killing the process for exceeding the memory quota would actually make > sense. > In my current view: > > - if you can afford to kill a process, maybe you shouldn't be running > it in a first place; > - Nobody learned much from fixing problems by switching things on and > off; > - if you don't know what you're doing, maybe you should learn how to do > it before doing it. > > My suggestion to let the process know and decide what to do next was > motivated by these views. > > Ulf, for what is worth, your suggestion to add additional tracers > resonates quite well with me. > Using error_logger for that purpose is actually presuming that reaching a > memory quota is an actual error and it doesn't require any leap of faith > from there to conclude it would be logical to terminate the "offender". > In my view, this situation should be rather classified as an "operating > condition", and if we were to spend any system resources in tracking that > condition, maybe we should consider a scenario which has a chance to > provide the most bang for the CPU buck. > > Kind regards > > V/ > > > (*) To be fair, I did this testing about 8 years ago and I may have missed > something out then, and there is strong possibility that the current VM > handles memory management in a different way. > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vicbaz@REDACTED Thu Apr 28 12:36:44 2016 From: vicbaz@REDACTED (vicbaz@REDACTED) Date: Thu, 28 Apr 2016 13:36:44 +0300 Subject: [erlang-questions] Memory consumption in ssl application (OTP-18.3.1) In-Reply-To: References: <571E0F10.80507@yandex.ru> <5720C22F.3000404@yandex.ru> Message-ID: <5721E7BC.9050605@yandex.ru> Hello, Could you please clarify one moment. Is the ssl session cache used only when reuse_sessions is set to true? Is there a way to turn off the cache completely? I'm trying to figure out what settings are needed for thousands of short-lived connections. 28/04/16 12:21, Ingela Andin ?????: > Hi! > > > 2016-04-27 15:44 GMT+02:00 >: > > Hello, > > The reason was session_cache_server_max (defaults to 1000). > The ssl_manager quickly get filled by > > {'$gen_cast',{invalidate_session,8443,{session,<<...>>,...}}} > {delayed_clean_session,{8443,<<14,50,229,...>>},40986} > > messages. > > I see, we will looking into mitigating this effect. > > Regards Ingela Erlang/OTP team - Ericsson AB > > > > 27/04/16 13:29, Ingela Andin ?????: > > Hi! > > You could try using the observer application to find out more > information about your system state. > There is no apparent reason why ssl-7.3 should consume a lot > more memory > than ssl-6.0.1.2. > But a lot has append between the versions and heavy load may > find corner > cases that where missed. > > Regards Ingela Erlang/OTP Team - Ericsson AB > > > 2016-04-25 14:35 GMT+02:00 >>: > > > Hello, > > When I run > https://github.com/ninenines/cowboy/tree/1.0.4/examples/ssl_hello_world > under load (using https://github.com/JoeDog/siege) I found > a big > difference in memory consumption between OTP-17.5 and > OTP-18.3. I know > the ssl application has many changes in new versions. Is > this memory > usage normal now or can be reduced using some ssl > application settings? > > I use OTP-17.5.6.9 and OTP-18.3.1. > > > ================================================================================ > Erlang/OTP 17 [erts-6.4.1.6] [source] [64-bit] [smp:2:2] > [async-threads:10] [hipe] [kernel-poll:false] > > 1> application:which_applications(). > [{ssl_hello_world,"Cowboy Hello World example with SSL.", > "1"}, > {cowboy,"Small, fast, modular HTTP server.","1.0.4"}, > {ranch,"Socket acceptor pool for TCP protocols.","1.2.1"}, > {cowlib,"Support library for manipulating Web protocols.", > "1.0.2"}, > {ssl,"Erlang/OTP SSL application","6.0.1.2"}, > {public_key,"Public key infrastructure","0.23"}, > {crypto,"CRYPTO","3.5"}, > {asn1,"The Erlang ASN1 compiler version 3.0.4","3.0.4"}, > {stdlib,"ERTS CXC 138 10","2.4"}, > {kernel,"ERTS CXC 138 10","3.2.0.1"}] > > 2> erlang:memory(). > [{total,43913840}, > {processes,8036312}, > {processes_used,8030744}, > {system,35877528}, > {atom,470537}, > {atom_used,455904}, > {binary,1498800}, > {code,11074476}, > {ets,14516424}] > > 3> recon_alloc:memory(used). > 45519464 > > 4> recon_alloc:memory(usage). > 0.7212344918526264 > > 5> recon:proc_window(memory, 10, 500). > [{<0.25910.2>,68120, > [{current_function,{gen_fsm,loop,7}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.25908.2>,68120, > [{current_function,{gen_fsm,loop,7}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.284.0>,15808, > [ssl_manager, > {current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.285.0>,12712, > [tls_connection_sup, > {current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.25909.2>,5888, > [{current_function,{gen,do_call,4}}, > {initial_call,{cowboy_protocol,init,4}}]}, > {<0.324.0>,4888, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.302.0>,4808, > [{current_function,{ranch_conns_sup,loop,4}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.330.0>,3016, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.25911.2>,2848, > [{current_function,{gen,do_call,4}}, > {initial_call,{cowboy_protocol,init,4}}]}, > {<0.340.0>,1872, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}] > > 6> recon:proc_window(reductions, 10, 500). > [{<0.2634.3>,11161, > [{current_function,{gen_fsm,loop,7}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.2633.3>,9747, > [{current_function,{crypto,int_to_bin_pos,2}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.284.0>,4097, > [ssl_manager, > {current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.285.0>,3314, > [tls_connection_sup, > {current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.302.0>,1003, > [{current_function,{ranch_conns_sup,loop,4}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.378.0>,426, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.372.0>,425, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.379.0>,425, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.377.0>,425, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.343.0>,425, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}] > > RES in htop ~50-60M > > > ================================================================================ > > Erlang/OTP 18 [erts-7.3.1] [source] [64-bit] [smp:2:2] > [async-threads:10] [hipe] [kernel-poll:false] > > 1> application:which_applications(). > [{ssl_hello_world,"Cowboy Hello World example with SSL.", > "1"}, > {cowboy,"Small, fast, modular HTTP server.","1.0.4"}, > {ranch,"Socket acceptor pool for TCP protocols.","1.2.1"}, > {cowlib,"Support library for manipulating Web protocols.", > "1.0.2"}, > {ssl,"Erlang/OTP SSL application","7.3"}, > {public_key,"Public key infrastructure","1.1.1"}, > {crypto,"CRYPTO","3.6.3"}, > {asn1,"The Erlang ASN1 compiler version 4.0.2","4.0.2"}, > {recon,"Diagnostic tools for production use","2.3.1"}, > {stdlib,"ERTS CXC 138 10","2.8"}, > {kernel,"ERTS CXC 138 10","4.2"}] > > 2> erlang:memory(). > [{total,949153560}, > {processes,902969888}, > {processes_used,902558312}, > {system,46183672}, > {atom,437761}, > {atom_used,432875}, > {binary,48320}, > {code,11407283}, > {ets,883640}] > > 3> recon_alloc:memory(used). > 863194560 > > 4> recon_alloc:memory(usage). > 0.8005009562139471 > > 5> recon:proc_window(memory, 10, 500). > [{<0.290.0>,94360, > [ssl_manager, > {current_function,{ssl_manager,handle_cast,2}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.13589.3>,24640, > [{current_function,{ssl_manager,validate_session,3}}, > {initial_call,{ssl_manager,init_session_validator,1}}]}, > {<0.291.0>,12832, > [tls_connection_sup, > {current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.13590.3>,10960, > [{current_function,{gen,do_call,4}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.318.0>,7904, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.308.0>,3056, > [{current_function,{ranch_conns_sup,loop,4}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.385.0>,1872, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.254.0>,0, > [code_server, > {current_function,{code_server,loop,1}}, > {initial_call,{erlang,apply,2}}]}, > {<0.325.0>,0, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.357.0>,0, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}] > > 6> recon:proc_window(reductions, 10, 500). > [{<0.290.0>,232358, > [ssl_manager, > {current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.291.0>,3058, > [tls_connection_sup, > {current_function,{gen_server,loop,6}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.308.0>,1129, > [{current_function,{ranch_conns_sup,loop,4}}, > {initial_call,{proc_lib,init_p,5}}]}, > {<0.339.0>,435, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.340.0>,430, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.355.0>,430, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.353.0>,430, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.334.0>,430, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.366.0>,430, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}, > {<0.360.0>,430, > [{current_function,{prim_inet,accept0,2}}, > {initial_call,{ranch_acceptor,loop,3}}]}] > > RES in htop ~900-1000M > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > > > http://erlang.org/mailman/listinfo/erlang-questions > > > From vladdu55@REDACTED Thu Apr 28 12:40:40 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 28 Apr 2016 12:40:40 +0200 Subject: [erlang-questions] EEP site not working In-Reply-To: <20160428102139.GA79617@erix.ericsson.se> References: <20160428083554.GA44314@erix.ericsson.se> <20160428102139.GA79617@erix.ericsson.se> Message-ID: It works now. Thanks! /Vlad On Thu, Apr 28, 2016 at 12:21 PM, Raimo Niskanen < raimo+erlang-questions@REDACTED> wrote: > On Thu, Apr 28, 2016 at 10:38:55AM +0200, Vlad Dumitrescu wrote: > > Hi, > > > > Now I get 404 errors for these pages. > > Does this one work for you? It works for me: > http://www.erlang.org/erlang-enhancement-proposals/eep-0003.html > > > > > regards, > > Vlad > > > > > > On Thu, Apr 28, 2016 at 10:35 AM, Raimo Niskanen < > > raimo+erlang-questions@REDACTED> wrote: > > > > > On Mon, Apr 25, 2016 at 01:44:11PM +0200, Vlad Dumitrescu wrote: > > > > Hi, > > > > > > > > I'm not sure if it's just me, but when I check the EEP web pages, > like > > > > > https://www.erlang.org/erlang-enhancement-proposals/home/eep-0003.html, > > > I > > > > get an error because there is a redirection loop between > www.erlang.org > > > and > > > > erlang.org > > > > > > Now Erlang Solutions have fixed the problem. > > > > > > > > > > > best regards, > > > > Vlad > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From touch.sereysethy@REDACTED Thu Apr 28 16:53:07 2016 From: touch.sereysethy@REDACTED (Sereysethy TOUCH) Date: Thu, 28 Apr 2016 21:53:07 +0700 Subject: [erlang-questions] Memory consumption in ssl application (OTP-18.3.1) In-Reply-To: <5721E7BC.9050605@yandex.ru> References: <571E0F10.80507@yandex.ru> <5720C22F.3000404@yandex.ru> <5721E7BC.9050605@yandex.ru> Message-ID: Hi, I used to face this kind of problem when I updated erlang from OTP 17.x.x to 18 but I think the problem was fixed already in 18.2 Here is what I found out: My problem was that `ets` table keeps growing and it is caused by `server_ssl_otp_session_cache` I traced it back to the source code of Erlang 17 & Erlang 18 release branch, in ssl module/ssl_manager.erl. There was a mistake in calculation of 24H_in_msec in Erlang 17. `Erlang OTP 17` -define('24H_in_msec', 8640000). -define('24H_in_sec', 8640). -define(GEN_UNIQUE_ID_MAX_TRIES, 10). -define(SESSION_VALIDATION_INTERVAL, 60000). -define(CLEAR_PEM_CACHE, 120000). -define(CLEAN_SESSION_DB, 60000). -define(CLEAN_CERT_DB, 500). -define(NOT_TO_BIG, 10). `Erlang OTP 18` -define('24H_in_msec', 86400000). -define('24H_in_sec', 86400). -define(GEN_UNIQUE_ID_MAX_TRIES, 10). -define(SESSION_VALIDATION_INTERVAL, 60000). -define(CLEAR_PEM_CACHE, 120000). -define(CLEAN_SESSION_DB, 60000). -define(CLEAN_CERT_DB, 500). -define(NOT_TO_BIG, 10). So you can try to pass `session_lifetime` to 30mn (1800 seconds) to `ssl` application (module). Sethy On Thu, Apr 28, 2016 at 5:36 PM, wrote: > Hello, > > Could you please clarify one moment. Is the ssl session cache used only > when reuse_sessions is set to true? Is there a way to turn off the cache > completely? I'm trying to figure out what settings are needed for > thousands of short-lived connections. > > 28/04/16 12:21, Ingela Andin ?????: > >> Hi! >> >> >> 2016-04-27 15:44 GMT+02:00 >: >> >> Hello, >> >> The reason was session_cache_server_max (defaults to 1000). >> The ssl_manager quickly get filled by >> >> {'$gen_cast',{invalidate_session,8443,{session,<<...>>,...}}} >> {delayed_clean_session,{8443,<<14,50,229,...>>},40986} >> >> messages. >> >> I see, we will looking into mitigating this effect. >> >> Regards Ingela Erlang/OTP team - Ericsson AB >> >> >> >> 27/04/16 13:29, Ingela Andin ?????: >> >> Hi! >> >> You could try using the observer application to find out more >> information about your system state. >> There is no apparent reason why ssl-7.3 should consume a lot >> more memory >> than ssl-6.0.1.2. >> But a lot has append between the versions and heavy load may >> find corner >> cases that where missed. >> >> Regards Ingela Erlang/OTP Team - Ericsson AB >> >> >> 2016-04-25 14:35 GMT+02:00 > > >> >>: >> >> >> Hello, >> >> When I run >> >> https://github.com/ninenines/cowboy/tree/1.0.4/examples/ssl_hello_world >> under load (using https://github.com/JoeDog/siege) I found >> a big >> difference in memory consumption between OTP-17.5 and >> OTP-18.3. I know >> the ssl application has many changes in new versions. Is >> this memory >> usage normal now or can be reduced using some ssl >> application settings? >> >> I use OTP-17.5.6.9 and OTP-18.3.1. >> >> >> >> ================================================================================ >> Erlang/OTP 17 [erts-6.4.1.6] [source] [64-bit] [smp:2:2] >> [async-threads:10] [hipe] [kernel-poll:false] >> >> 1> application:which_applications(). >> [{ssl_hello_world,"Cowboy Hello World example with SSL.", >> "1"}, >> {cowboy,"Small, fast, modular HTTP server.","1.0.4"}, >> {ranch,"Socket acceptor pool for TCP protocols.","1.2.1"}, >> {cowlib,"Support library for manipulating Web protocols.", >> "1.0.2"}, >> {ssl,"Erlang/OTP SSL application","6.0.1.2"}, >> {public_key,"Public key infrastructure","0.23"}, >> {crypto,"CRYPTO","3.5"}, >> {asn1,"The Erlang ASN1 compiler version 3.0.4","3.0.4"}, >> {stdlib,"ERTS CXC 138 10","2.4"}, >> {kernel,"ERTS CXC 138 10","3.2.0.1"}] >> >> 2> erlang:memory(). >> [{total,43913840}, >> {processes,8036312}, >> {processes_used,8030744}, >> {system,35877528}, >> {atom,470537}, >> {atom_used,455904}, >> {binary,1498800}, >> {code,11074476}, >> {ets,14516424}] >> >> 3> recon_alloc:memory(used). >> 45519464 >> >> 4> recon_alloc:memory(usage). >> 0.7212344918526264 >> >> 5> recon:proc_window(memory, 10, 500). >> [{<0.25910.2>,68120, >> [{current_function,{gen_fsm,loop,7}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.25908.2>,68120, >> [{current_function,{gen_fsm,loop,7}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.284.0>,15808, >> [ssl_manager, >> {current_function,{gen_server,loop,6}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.285.0>,12712, >> [tls_connection_sup, >> {current_function,{gen_server,loop,6}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.25909.2>,5888, >> [{current_function,{gen,do_call,4}}, >> {initial_call,{cowboy_protocol,init,4}}]}, >> {<0.324.0>,4888, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.302.0>,4808, >> [{current_function,{ranch_conns_sup,loop,4}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.330.0>,3016, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.25911.2>,2848, >> [{current_function,{gen,do_call,4}}, >> {initial_call,{cowboy_protocol,init,4}}]}, >> {<0.340.0>,1872, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}] >> >> 6> recon:proc_window(reductions, 10, 500). >> [{<0.2634.3>,11161, >> [{current_function,{gen_fsm,loop,7}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.2633.3>,9747, >> [{current_function,{crypto,int_to_bin_pos,2}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.284.0>,4097, >> [ssl_manager, >> {current_function,{gen_server,loop,6}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.285.0>,3314, >> [tls_connection_sup, >> {current_function,{gen_server,loop,6}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.302.0>,1003, >> [{current_function,{ranch_conns_sup,loop,4}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.378.0>,426, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.372.0>,425, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.379.0>,425, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.377.0>,425, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.343.0>,425, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}] >> >> RES in htop ~50-60M >> >> >> >> ================================================================================ >> >> Erlang/OTP 18 [erts-7.3.1] [source] [64-bit] [smp:2:2] >> [async-threads:10] [hipe] [kernel-poll:false] >> >> 1> application:which_applications(). >> [{ssl_hello_world,"Cowboy Hello World example with SSL.", >> "1"}, >> {cowboy,"Small, fast, modular HTTP server.","1.0.4"}, >> {ranch,"Socket acceptor pool for TCP protocols.","1.2.1"}, >> {cowlib,"Support library for manipulating Web protocols.", >> "1.0.2"}, >> {ssl,"Erlang/OTP SSL application","7.3"}, >> {public_key,"Public key infrastructure","1.1.1"}, >> {crypto,"CRYPTO","3.6.3"}, >> {asn1,"The Erlang ASN1 compiler version 4.0.2","4.0.2"}, >> {recon,"Diagnostic tools for production use","2.3.1"}, >> {stdlib,"ERTS CXC 138 10","2.8"}, >> {kernel,"ERTS CXC 138 10","4.2"}] >> >> 2> erlang:memory(). >> [{total,949153560}, >> {processes,902969888}, >> {processes_used,902558312}, >> {system,46183672}, >> {atom,437761}, >> {atom_used,432875}, >> {binary,48320}, >> {code,11407283}, >> {ets,883640}] >> >> 3> recon_alloc:memory(used). >> 863194560 >> >> 4> recon_alloc:memory(usage). >> 0.8005009562139471 >> >> 5> recon:proc_window(memory, 10, 500). >> [{<0.290.0>,94360, >> [ssl_manager, >> {current_function,{ssl_manager,handle_cast,2}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.13589.3>,24640, >> [{current_function,{ssl_manager,validate_session,3}}, >> {initial_call,{ssl_manager,init_session_validator,1}}]}, >> {<0.291.0>,12832, >> [tls_connection_sup, >> {current_function,{gen_server,loop,6}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.13590.3>,10960, >> [{current_function,{gen,do_call,4}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.318.0>,7904, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.308.0>,3056, >> [{current_function,{ranch_conns_sup,loop,4}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.385.0>,1872, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.254.0>,0, >> [code_server, >> {current_function,{code_server,loop,1}}, >> {initial_call,{erlang,apply,2}}]}, >> {<0.325.0>,0, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.357.0>,0, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}] >> >> 6> recon:proc_window(reductions, 10, 500). >> [{<0.290.0>,232358, >> [ssl_manager, >> {current_function,{gen_server,loop,6}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.291.0>,3058, >> [tls_connection_sup, >> {current_function,{gen_server,loop,6}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.308.0>,1129, >> [{current_function,{ranch_conns_sup,loop,4}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.339.0>,435, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.340.0>,430, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.355.0>,430, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.353.0>,430, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.334.0>,430, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.366.0>,430, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.360.0>,430, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}] >> >> RES in htop ~900-1000M >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > > >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> >> _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mononcqc@REDACTED Thu Apr 28 17:02:12 2016 From: mononcqc@REDACTED (Fred Hebert) Date: Thu, 28 Apr 2016 11:02:12 -0400 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> Message-ID: <20160428150211.GG2013@fhebert-ltm2.internal.salesforce.com> On 04/28, Richard A. O'Keefe wrote: >I've been thinking for some time of writing a paper with the >title "Why can't I see the structure?" based on the idea that >modules in every programming language I know look like blobs. >I'm aware of visual notations like UML, BON, SDL, and what >was it, Visual Erlang? But for me, those are just spaghetti >with meatballs; once you get beyond just a handful of boxes >in your diagram, all diagrams look much the same. In any >case, I'm interested in the medium scale. > >Why can't I see the structure in a 3000-line module, or even >a 1000-line module? (I am not asserting that Erlang is >particularly bad here. It isn't.) > >The kind of structure I'm interested in can, I think, be >described as *rhetorical* structure, like relationships >between paragraphs. > For me, the conceptual leap seems to be related to the same kind of challenges that exist when trying to explain technical material. The structure of dependencies to understand a piece of content is based on a graph, where any item may require understanding of N dependencies at once to make sense out of it. However, what we can display, explain, or absorb, is generally done linearly: reading a text a paragraph at a time, giving lessons on concepts one by one, and so on. In the case of code, every function or method or dependency I hit is a fractal of new information to explore. In every single module, I can at most display code sequentially, with a few variations: - bottom-up - top-to-bottom - divisions into API or Public / Protected / Private - divisions into proper module hierarchies (where a 'util' module eventually develops and eats up all organisation that once were) There's use of ctags and other equivalents to be able to jump around code with ease, and then stuff that happened like code bubbles[1], which attempted to break up modules and classes into independent units that could be displayed on their own. There's probably a lot more equivalent that you know more than I do. Mostly I've been sticking to vim for the last X years of my career and the only sensical steps I have found to traverse code have been to pick either a level-order, depth-first, or bottom-up traversal and stick with it. In the worst cases, I've had to just whip out a piece of paper and draw bubbles and arrows of everything with at least a second dimension (I can use both vertical and horizontal space! to convey things!) Erlang has made a few things very interesting by forcing an additional structure in supervision trees and well-defined behaviour, which lets me scan things at a glance in terms of project structure and infer what it does without actually needing to understand code. But when I get to the code, then there's the well-defined API, then behaviour callbacks (which tie the API and private functions to do modifications together) and then the rest of modules as libraries that get to be used there. >My *belief* is that if this structure were made explicit, >perhaps by textual structure, perhaps by annotations, perhaps >by some graphical form (but probably derived from annotations), >it would be easier to understand medium-sized wodges of code. > >I'm aware of annotation support in languages like Java and C# >and for that matter, Smalltalk, but with the exception of >Smalltalk, nobody seems to be using annotations in this way >(and that exception is me). > >I'd be very interested in hearing from anyone else who has been >thinking in this area. > I've been trying to think about it for some time, as it also has an impact when writing documentation and tutorials: how much context is needed for this to make sense? Can the context be derived? Another one for me has been trying to figure out how to enforce system-wide constraints within local pieces of code: this ID is assumed to be opaque and not something you can sort on, even if for now it is an integer. Breaking such an assumption can ruin systems over time, but there's no good way to communicate it outside of assiduous inline comment discipline. One other interesting thing is that from past informal polls[2], the more experimented a developer is, the least they tend to want or desire comments, but newer developers think the opposite. [1]: http://cs.brown.edu/~spr/codebubbles/ [2]: http://ferd.ca/poll-results-erlang-maintenance.html From felixgallo@REDACTED Thu Apr 28 17:23:14 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Thu, 28 Apr 2016 08:23:14 -0700 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: <20160428150211.GG2013@fhebert-ltm2.internal.salesforce.com> References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <20160428150211.GG2013@fhebert-ltm2.internal.salesforce.com> Message-ID: I agree with Fred's point that there are such a profusion of cross-cutting concerns that it seems unlikely that there's a single unifying view paradigm that works. For example, I want a Brendan Gregg(tm) Flamegraph to pop up when I run my code in debug, and I want the functions of my code to start glowing with their relative heat; and I want 'receive' statements to have a tiny little number spinning above each branch showing how many messages of each type were received; and I want lines of code that cause a crash to have a little stop sign in their gutter; and so on. But that 'operational code view' would be nonsensical in early coding, where you might want code annotated with quality/linter marks and documentation notes. On the topic of finding a way to constrain developers, though, I think that's resolvable with powerful type systems. If a particular variable is of opaque type 'identifier' which itself isn't a member of 'enumerable', then the compiler can decline to permit variable.sort(). Although much smarter minds than mine (e.g. Wadler and Kostis) seem to have concluded that strong algebraic data typing is problematic in erlang, the folks over at ponylang are making some pretty incredible and novel inroads in an erlangesque scenario. Would that I were a young computer science researcher. F. On Thu, Apr 28, 2016 at 8:02 AM, Fred Hebert wrote: > On 04/28, Richard A. O'Keefe wrote: > >> I've been thinking for some time of writing a paper with the >> title "Why can't I see the structure?" based on the idea that >> modules in every programming language I know look like blobs. >> I'm aware of visual notations like UML, BON, SDL, and what >> was it, Visual Erlang? But for me, those are just spaghetti >> with meatballs; once you get beyond just a handful of boxes >> in your diagram, all diagrams look much the same. In any >> case, I'm interested in the medium scale. >> >> Why can't I see the structure in a 3000-line module, or even >> a 1000-line module? (I am not asserting that Erlang is >> particularly bad here. It isn't.) >> >> The kind of structure I'm interested in can, I think, be >> described as *rhetorical* structure, like relationships >> between paragraphs. >> >> > For me, the conceptual leap seems to be related to the same kind of > challenges that exist when trying to explain technical material. > > The structure of dependencies to understand a piece of content is based on > a graph, where any item may require understanding of N dependencies at once > to make sense out of it. > > However, what we can display, explain, or absorb, is generally done > linearly: reading a text a paragraph at a time, giving lessons on concepts > one by one, and so on. > > In the case of code, every function or method or dependency I hit is a > fractal of new information to explore. In every single module, I can at > most display code sequentially, with a few variations: > > - bottom-up > - top-to-bottom > - divisions into API or Public / Protected / Private > - divisions into proper module hierarchies (where a 'util' module > eventually develops and eats up all organisation that once were) > > There's use of ctags and other equivalents to be able to jump around code > with ease, and then stuff that happened like code bubbles[1], which > attempted to break up modules and classes into independent units that could > be displayed on their own. There's probably a lot more equivalent that you > know more than I do. > > Mostly I've been sticking to vim for the last X years of my career and the > only sensical steps I have found to traverse code have been to pick either > a level-order, depth-first, or bottom-up traversal and stick with it. > > In the worst cases, I've had to just whip out a piece of paper and draw > bubbles and arrows of everything with at least a second dimension (I can > use both vertical and horizontal space! to convey things!) > > Erlang has made a few things very interesting by forcing an additional > structure in supervision trees and well-defined behaviour, which lets me > scan things at a glance in terms of project structure and infer what it > does without actually needing to understand code. > > But when I get to the code, then there's the well-defined API, then > behaviour callbacks (which tie the API and private functions to do > modifications together) and then the rest of modules as libraries that get > to be used there. > > My *belief* is that if this structure were made explicit, >> perhaps by textual structure, perhaps by annotations, perhaps >> by some graphical form (but probably derived from annotations), >> it would be easier to understand medium-sized wodges of code. >> >> I'm aware of annotation support in languages like Java and C# >> and for that matter, Smalltalk, but with the exception of >> Smalltalk, nobody seems to be using annotations in this way >> (and that exception is me). >> >> I'd be very interested in hearing from anyone else who has been >> thinking in this area. >> >> > I've been trying to think about it for some time, as it also has an impact > when writing documentation and tutorials: how much context is needed for > this to make sense? Can the context be derived? > > Another one for me has been trying to figure out how to enforce > system-wide constraints within local pieces of code: this ID is assumed to > be opaque and not something you can sort on, even if for now it is an > integer. Breaking such an assumption can ruin systems over time, but > there's no good way to communicate it outside of assiduous inline comment > discipline. > > One other interesting thing is that from past informal polls[2], the more > experimented a developer is, the least they tend to want or desire > comments, but newer developers think the opposite. > > [1]: http://cs.brown.edu/~spr/codebubbles/ > [2]: http://ferd.ca/poll-results-erlang-maintenance.html > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From raould@REDACTED Thu Apr 28 17:28:13 2016 From: raould@REDACTED (Raoul Duke) Date: Thu, 28 Apr 2016 08:28:13 -0700 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <20160428150211.GG2013@fhebert-ltm2.internal.salesforce.com> Message-ID: Right on. Code often becomes so complicated that we really need multiple ways to tackle, visualize, sort, query, etc. it. The fact that we have such a paucity of tools and toolkits/frameworks for the tooling is I feel a long standing indictment of the whole industry. I guess most people don't grok UX, and there seems to be sometimes an extra special antipathy or cluelessness or something in the software engineering world about it, in many regards. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vicbaz@REDACTED Thu Apr 28 17:57:27 2016 From: vicbaz@REDACTED (vicbaz@REDACTED) Date: Thu, 28 Apr 2016 18:57:27 +0300 Subject: [erlang-questions] Memory consumption in ssl application (OTP-18.3.1) In-Reply-To: <5721E7BC.9050605@yandex.ru> References: <571E0F10.80507@yandex.ru> <5720C22F.3000404@yandex.ru> <5721E7BC.9050605@yandex.ru> Message-ID: <572232E7.3050903@yandex.ru> It seems that my assumption for the first question is correct. https://github.com/erlang/otp/blob/OTP-18.3.2/lib/ssl/src/ssl_session.erl#L94 https://github.com/erlang/otp/blob/OTP-18.3.2/lib/ssl/src/ssl_session.erl#L114 The session_cb could be set to an "empty" implementation to completely disable the cache. https://github.com/erlang/otp/blob/OTP-18.3.2/lib/ssl/src/ssl_manager.erl#L241 28/04/16 13:36, vicbaz@REDACTED ?????: > Hello, > > Could you please clarify one moment. Is the ssl session cache used only > when reuse_sessions is set to true? Is there a way to turn off the cache > completely? I'm trying to figure out what settings are needed for > thousands of short-lived connections. > > 28/04/16 12:21, Ingela Andin ?????: >> Hi! >> >> >> 2016-04-27 15:44 GMT+02:00 >: >> >> Hello, >> >> The reason was session_cache_server_max (defaults to 1000). >> The ssl_manager quickly get filled by >> >> {'$gen_cast',{invalidate_session,8443,{session,<<...>>,...}}} >> {delayed_clean_session,{8443,<<14,50,229,...>>},40986} >> >> messages. >> >> I see, we will looking into mitigating this effect. >> >> Regards Ingela Erlang/OTP team - Ericsson AB >> >> >> >> 27/04/16 13:29, Ingela Andin ?????: >> >> Hi! >> >> You could try using the observer application to find out more >> information about your system state. >> There is no apparent reason why ssl-7.3 should consume a lot >> more memory >> than ssl-6.0.1.2. >> But a lot has append between the versions and heavy load may >> find corner >> cases that where missed. >> >> Regards Ingela Erlang/OTP Team - Ericsson AB >> >> >> 2016-04-25 14:35 GMT+02:00 > > >>: >> >> >> Hello, >> >> When I run >> >> https://github.com/ninenines/cowboy/tree/1.0.4/examples/ssl_hello_world >> under load (using https://github.com/JoeDog/siege) I found >> a big >> difference in memory consumption between OTP-17.5 and >> OTP-18.3. I know >> the ssl application has many changes in new versions. Is >> this memory >> usage normal now or can be reduced using some ssl >> application settings? >> >> I use OTP-17.5.6.9 and OTP-18.3.1. >> >> >> >> ================================================================================ >> >> Erlang/OTP 17 [erts-6.4.1.6] [source] [64-bit] [smp:2:2] >> [async-threads:10] [hipe] [kernel-poll:false] >> >> 1> application:which_applications(). >> [{ssl_hello_world,"Cowboy Hello World example with SSL.", >> "1"}, >> {cowboy,"Small, fast, modular HTTP server.","1.0.4"}, >> {ranch,"Socket acceptor pool for TCP protocols.","1.2.1"}, >> {cowlib,"Support library for manipulating Web protocols.", >> "1.0.2"}, >> {ssl,"Erlang/OTP SSL application","6.0.1.2"}, >> {public_key,"Public key infrastructure","0.23"}, >> {crypto,"CRYPTO","3.5"}, >> {asn1,"The Erlang ASN1 compiler version 3.0.4","3.0.4"}, >> {stdlib,"ERTS CXC 138 10","2.4"}, >> {kernel,"ERTS CXC 138 10","3.2.0.1"}] >> >> 2> erlang:memory(). >> [{total,43913840}, >> {processes,8036312}, >> {processes_used,8030744}, >> {system,35877528}, >> {atom,470537}, >> {atom_used,455904}, >> {binary,1498800}, >> {code,11074476}, >> {ets,14516424}] >> >> 3> recon_alloc:memory(used). >> 45519464 >> >> 4> recon_alloc:memory(usage). >> 0.7212344918526264 >> >> 5> recon:proc_window(memory, 10, 500). >> [{<0.25910.2>,68120, >> [{current_function,{gen_fsm,loop,7}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.25908.2>,68120, >> [{current_function,{gen_fsm,loop,7}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.284.0>,15808, >> [ssl_manager, >> {current_function,{gen_server,loop,6}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.285.0>,12712, >> [tls_connection_sup, >> {current_function,{gen_server,loop,6}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.25909.2>,5888, >> [{current_function,{gen,do_call,4}}, >> {initial_call,{cowboy_protocol,init,4}}]}, >> {<0.324.0>,4888, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.302.0>,4808, >> [{current_function,{ranch_conns_sup,loop,4}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.330.0>,3016, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.25911.2>,2848, >> [{current_function,{gen,do_call,4}}, >> {initial_call,{cowboy_protocol,init,4}}]}, >> {<0.340.0>,1872, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}] >> >> 6> recon:proc_window(reductions, 10, 500). >> [{<0.2634.3>,11161, >> [{current_function,{gen_fsm,loop,7}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.2633.3>,9747, >> [{current_function,{crypto,int_to_bin_pos,2}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.284.0>,4097, >> [ssl_manager, >> {current_function,{gen_server,loop,6}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.285.0>,3314, >> [tls_connection_sup, >> {current_function,{gen_server,loop,6}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.302.0>,1003, >> [{current_function,{ranch_conns_sup,loop,4}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.378.0>,426, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.372.0>,425, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.379.0>,425, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.377.0>,425, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.343.0>,425, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}] >> >> RES in htop ~50-60M >> >> >> >> ================================================================================ >> >> >> Erlang/OTP 18 [erts-7.3.1] [source] [64-bit] [smp:2:2] >> [async-threads:10] [hipe] [kernel-poll:false] >> >> 1> application:which_applications(). >> [{ssl_hello_world,"Cowboy Hello World example with SSL.", >> "1"}, >> {cowboy,"Small, fast, modular HTTP server.","1.0.4"}, >> {ranch,"Socket acceptor pool for TCP protocols.","1.2.1"}, >> {cowlib,"Support library for manipulating Web protocols.", >> "1.0.2"}, >> {ssl,"Erlang/OTP SSL application","7.3"}, >> {public_key,"Public key infrastructure","1.1.1"}, >> {crypto,"CRYPTO","3.6.3"}, >> {asn1,"The Erlang ASN1 compiler version 4.0.2","4.0.2"}, >> {recon,"Diagnostic tools for production use","2.3.1"}, >> {stdlib,"ERTS CXC 138 10","2.8"}, >> {kernel,"ERTS CXC 138 10","4.2"}] >> >> 2> erlang:memory(). >> [{total,949153560}, >> {processes,902969888}, >> {processes_used,902558312}, >> {system,46183672}, >> {atom,437761}, >> {atom_used,432875}, >> {binary,48320}, >> {code,11407283}, >> {ets,883640}] >> >> 3> recon_alloc:memory(used). >> 863194560 >> >> 4> recon_alloc:memory(usage). >> 0.8005009562139471 >> >> 5> recon:proc_window(memory, 10, 500). >> [{<0.290.0>,94360, >> [ssl_manager, >> {current_function,{ssl_manager,handle_cast,2}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.13589.3>,24640, >> [{current_function,{ssl_manager,validate_session,3}}, >> {initial_call,{ssl_manager,init_session_validator,1}}]}, >> {<0.291.0>,12832, >> [tls_connection_sup, >> {current_function,{gen_server,loop,6}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.13590.3>,10960, >> [{current_function,{gen,do_call,4}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.318.0>,7904, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.308.0>,3056, >> [{current_function,{ranch_conns_sup,loop,4}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.385.0>,1872, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.254.0>,0, >> [code_server, >> {current_function,{code_server,loop,1}}, >> {initial_call,{erlang,apply,2}}]}, >> {<0.325.0>,0, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.357.0>,0, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}] >> >> 6> recon:proc_window(reductions, 10, 500). >> [{<0.290.0>,232358, >> [ssl_manager, >> {current_function,{gen_server,loop,6}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.291.0>,3058, >> [tls_connection_sup, >> {current_function,{gen_server,loop,6}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.308.0>,1129, >> [{current_function,{ranch_conns_sup,loop,4}}, >> {initial_call,{proc_lib,init_p,5}}]}, >> {<0.339.0>,435, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.340.0>,430, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.355.0>,430, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.353.0>,430, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.334.0>,430, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.366.0>,430, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}, >> {<0.360.0>,430, >> [{current_function,{prim_inet,accept0,2}}, >> {initial_call,{ranch_acceptor,loop,3}}]}] >> >> RES in htop ~900-1000M >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> > > >> http://erlang.org/mailman/listinfo/erlang-questions >> >> >> From binarin@REDACTED Thu Apr 28 18:50:25 2016 From: binarin@REDACTED (Alexey Lebedeff) Date: Thu, 28 Apr 2016 19:50:25 +0300 Subject: [erlang-questions] Improvements in TLS error reporting Message-ID: Hi, I have some ideas about making SSL error reporting more friendlier for operators, because currently troubleshooting can be very problematic. If the solution outlined at the end of the message is acceptable, I'll make proper PR on github. There are several places where SSL/TLS handshake errors are being converted to SSL alerts. During this process some details about errors are being discarded, so they will fit SSL alert protocol. But the problem is that the same simplified errors are then used for event logging - and sometimes they are completely useless. So several times I've had to resort to adding debug print statements to ssl app just to find what I've done wrong in my SSL config. Some examples I've observed: - Certificate was in wrong format, but detailed error was discarded at [1] And 'internal error' message wasn't helpful at all. - Certificate had wrong ext-key-usage field (there was attempt to use server certificate as a client one) This info was happily discarded at [2]. And again, the error was logged as unhelful 'handshake failure'. This situation can be improved. I have a suggestion to add additional field to SSL alert record at [3] which will contain detailed error description which is currently being throwed out. And then use this description in ssl_alert:alert_txt/1 at [4]. [1] https://github.com/erlang/otp/blob/523e048754f5086a6cc4fd9a250e1b495fc5b9b8/lib/ssl/src/ssl_handshake.erl#L169 [2] https://github.com/erlang/otp/blob/523e048754f5086a6cc4fd9a250e1b495fc5b9b8/lib/ssl/src/ssl_handshake.erl#L1491 [3] https://github.com/erlang/otp/blob/523e048754f5086a6cc4fd9a250e1b495fc5b9b8/lib/ssl/src/ssl_alert.hrl#L116 [4] https://github.com/erlang/otp/blob/523e048754f5086a6cc4fd9a250e1b495fc5b9b8/lib/ssl/src/ssl_alert.erl#L77 Best, Alexey -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Thu Apr 28 22:06:46 2016 From: ingela.andin@REDACTED (Ingela Andin) Date: Thu, 28 Apr 2016 22:06:46 +0200 Subject: [erlang-questions] Patch package OTP 18.3.2 released In-Reply-To: <57210A86.4040508@yandex.ru> References: <572079A3.2000607@ericsson.com> <57210A86.4040508@yandex.ru> Message-ID: *Hi!* I am not sure exactly where in the git handling this went wrong, but here is the patch solves the problem. The solution was already merged to master and it seems it was an old incorrect version of the branch that was used for the patch :( *diff --git a/lib/ssl/src/ssl_cipher.erl b/lib/ssl/src/ssl_ciphe**r.erl* *index 6ea1bdb..af53d4a 100644* *--- a/lib/ssl/src/ssl_cipher.erl* *+++ b/lib/ssl/src/ssl_cipher.erl* @@ -979,21 +979,21 @@ suite({ecdh_rsa, aes_256_cbc, sha384, sha384}) -> %% RFC 5288 AES-GCM Cipher Suites suite({rsa, aes_128_gcm, null, sha256}) -> ?TLS_RSA_WITH_AES_128_GCM_SHA256; -suite({rsa, aes_256_gcm, null}) -> +suite({rsa, aes_256_gcm, null, sha384}) -> ?TLS_RSA_WITH_AES_256_GCM_SHA384; -suite({dhe_rsa, aes_128_gcm, null, sha384}) -> +suite({dhe_rsa, aes_128_gcm, null, sha256}) -> ?TLS_DHE_RSA_WITH_AES_128_GCM_SHA256; -suite({dhe_rsa, aes_256_gcm, null, sha256}) -> +suite({dhe_rsa, aes_256_gcm, null, sha384}) -> ?TLS_DHE_RSA_WITH_AES_256_GCM_SHA384; -suite({dh_rsa, aes_128_gcm, null, sha384}) -> +suite({dh_rsa, aes_128_gcm, null, sha256}) -> ?TLS_DH_RSA_WITH_AES_128_GCM_SHA256; -suite({dh_rsa, aes_256_gcm, null, sha256}) -> +suite({dh_rsa, aes_256_gcm, null, sha384}) -> ?TLS_DH_RSA_WITH_AES_256_GCM_SHA384; -suite({dhe_dss, aes_128_gcm, null, sha384}) -> +suite({dhe_dss, aes_128_gcm, null, sha256}) -> ?TLS_DHE_DSS_WITH_AES_128_GCM_SHA256; -suite({dhe_dss, aes_256_gcm, null, sha256}) -> +suite({dhe_dss, aes_256_gcm, null, sha384}) -> ?TLS_DHE_DSS_WITH_AES_256_GCM_SHA384; -suite({dh_dss, aes_128_gcm, null, sha384}) -> +suite({dh_dss, aes_128_gcm, null, sha256}) -> ?TLS_DH_DSS_WITH_AES_128_GCM_SHA256; suite({dh_dss, aes_256_gcm, null, sha384}) -> ?TLS_DH_DSS_WITH_AES_256_GCM_SHA384; 2016-04-27 20:52 GMT+02:00 : > Hello, > > When I run > > https://github.com/ninenines/cowboy/tree/1.0.4/examples/ssl_hello_world > > under OTP-18.3.2 I got 'unknown POSIX error' (see below). > > Could someone explain to me what happened? > It works without problem under OTP-18.3.1. > > $ ./_rel/ssl_hello_world_example/bin/ssl_hello_world_example console > Exec: > /home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/erts-7.3.1/bin/erlexec > -boot > /home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/releases/1/ssl_hello_world_example > -mode embedded -boot_var ERTS_LIB_DIR > /home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/erts-7.3.1/../lib > -config > /home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/releases/1/sys.config > -args_file > /home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/releases/1/vm.args > -- console > Root: > /home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example > > /home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example > Erlang/OTP 18 [erts-7.3.1] [source] [64-bit] [smp:2:2] [async-threads:10] > [hipe] [kernel-poll:false] > > > =ERROR REPORT==== 27-Apr-2016::21:43:11 === > Failed to start Ranch listener https in ranch_ssl:listen([{port,8443}, > {cacertfile, > > > "/home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/lib/ssl_hello_world-1/priv/ssl/cowboy-ca.crt"}, > {certfile, > > > "/home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/lib/ssl_hello_world-1/priv/ssl/server.crt"}, > {keyfile, > > "/home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/lib/ssl_hello_world-1/priv/ssl/server.key"}]) > for reason {options, > > > {ciphers, > > > [{ecdhe_ecdsa, > > > aes_256_gcm, > > > null, > > > sha384}, > > > {ecdhe_rsa, > > > aes_256_gcm, > > > null, > > > sha384}, > > > {ecdhe_ecdsa, > > > aes_256_cbc, > > > sha384, > > > sha384}, > > > {ecdhe_rsa, > > > aes_256_cbc, > > > sha384, > > > sha384}, > > > {ecdh_ecdsa, > > > aes_256_gcm, > > > null, > > > sha384}, > > > {ecdh_rsa, > > > aes_256_gcm, > > > null, > > > sha384}, > > > {ecdh_ecdsa, > > > aes_256_cbc, > > > sha384, > > > sha384}, > > > {ecdh_rsa, > > > aes_256_cbc, > > > sha384, > > > sha384}, > > > {dhe_rsa, > > > aes_256_gcm, > > > null, > > > sha384}, > > > {dhe_dss, > > > aes_256_gcm, > > > null, > > > sha384}, > > > {dhe_rsa, > > > aes_256_cbc, > > > sha256}, > > > {dhe_dss, > > > aes_256_cbc, > > > sha256}, > > > {rsa, > > > aes_256_gcm, > > > null, > > > sha384}, > > > {rsa, > > > aes_256_cbc, > > > sha256}, > > > {ecdhe_ecdsa, > > > aes_128_gcm, > > > null, > > > sha256}, > > > {ecdhe_rsa, > > > aes_128_gcm, > > > null, > > > sha256}, > > > {ecdhe_ecdsa, > > > aes_128_cbc, > > > sha256, > > > sha256}, > > > {ecdhe_rsa, > > > aes_128_cbc, > > > sha256, > > > sha256}, > > > {ecdh_ecdsa, > > > aes_128_gcm, > > > null, > > > sha256}, > > > {ecdh_rsa, > > > aes_128_gcm, > > > null, > > > sha256}, > > > {ecdh_ecdsa, > > > aes_128_cbc, > > > sha256, > > > sha256}, > > > {ecdh_rsa, > > > aes_128_cbc, > > > sha256, > > > sha256}, > > > {dhe_rsa, > > > aes_128_gcm, > > > null, > > > sha256}, > > > {dhe_dss, > > > aes_128_gcm, > > > null, > > > sha256}, > > > {dhe_rsa, > > > aes_128_cbc, > > > sha256}, > > > {dhe_dss, > > > aes_128_cbc, > > > sha256}, > > > {rsa, > > > aes_128_gcm, > > > null, > > > sha256}, > > > {rsa, > > > aes_128_cbc, > > > sha256}, > > > {ecdhe_ecdsa, > > > aes_256_cbc, > > > sha}, > > > {ecdhe_rsa, > > > aes_256_cbc, > > > sha}, > > > {dhe_rsa, > > > aes_256_cbc, > > > sha}, > > > {dhe_dss, > > > aes_256_cbc, > > > sha}, > > > {ecdh_ecdsa, > > > aes_256_cbc, > > > sha}, > > > {ecdh_rsa, > > > aes_256_cbc, > > > sha}, > > > {rsa, > > > aes_256_cbc, > > > sha}, > > > {ecdhe_ecdsa, > > > '3des_ede_cbc', > > > sha}, > > > {ecdhe_rsa, > > > '3des_ede_cbc', > > > sha}, > > > {dhe_rsa, > > > '3des_ede_cbc', > > > sha}, > > > {dhe_dss, > > > '3des_ede_cbc', > > > sha}, > > > {ecdh_ecdsa, > > > '3des_ede_cbc', > > > sha}, > > > {ecdh_rsa, > > > '3des_ede_cbc', > > > sha}, > > > {rsa, > > > '3des_ede_cbc', > > > sha}, > > > {ecdhe_ecdsa, > > > aes_128_cbc, > > > sha}, > > > {ecdhe_rsa, > > > aes_128_cbc, > > > sha}, > > > {dhe_rsa, > > > aes_128_cbc, > > > sha}, > > > {dhe_dss, > > > aes_128_cbc, > > > sha}, > > > {ecdh_ecdsa, > > > aes_128_cbc, > > > sha}, > > > {ecdh_rsa, > > > aes_128_cbc, > > > sha}, > > > {rsa, > > > aes_128_cbc, > > > sha}, > > > {dhe_rsa, > > > des_cbc, > > > sha}, > > > {rsa, > > > des_cbc, > > > sha}]}} (unknown > POSIX error) > Eshell V7.3.1 (abort with ^G) > (ssl_hello_world_example@REDACTED)1> > =INFO REPORT==== 27-Apr-2016::21:43:11 === > application: ssl_hello_world > exited: {bad_return, > {{ssl_hello_world_app,start,[normal,[]]}, > {'EXIT', > {{badmatch, > {error, > {{shutdown, > {failed_to_start_child,ranch_acceptors_sup, > {listen_error,https, > {options, > {ciphers, > [{ecdhe_ecdsa,aes_256_gcm,null,sha384}, > {ecdhe_rsa,aes_256_gcm,null,sha384}, > {ecdhe_ecdsa,aes_256_cbc,sha384,sha384}, > {ecdhe_rsa,aes_256_cbc,sha384,sha384}, > {ecdh_ecdsa,aes_256_gcm,null,sha384}, > {ecdh_rsa,aes_256_gcm,null,sha384}, > {ecdh_ecdsa,aes_256_cbc,sha384,sha384}, > {ecdh_rsa,aes_256_cbc,sha384,sha384}, > {dhe_rsa,aes_256_gcm,null,sha384}, > {dhe_dss,aes_256_gcm,null,sha384}, > {dhe_rsa,aes_256_cbc,sha256}, > {dhe_dss,aes_256_cbc,sha256}, > {rsa,aes_256_gcm,null,sha384}, > {rsa,aes_256_cbc,sha256}, > {ecdhe_ecdsa,aes_128_gcm,null,sha256}, > {ecdhe_rsa,aes_128_gcm,null,sha256}, > {ecdhe_ecdsa,aes_128_cbc,sha256,sha256}, > {ecdhe_rsa,aes_128_cbc,sha256,sha256}, > {ecdh_ecdsa,aes_128_gcm,null,sha256}, > {ecdh_rsa,aes_128_gcm,null,sha256}, > {ecdh_ecdsa,aes_128_cbc,sha256,sha256}, > {ecdh_rsa,aes_128_cbc,sha256,sha256}, > {dhe_rsa,aes_128_gcm,null,sha256}, > {dhe_dss,aes_128_gcm,null,sha256}, > {dhe_rsa,aes_128_cbc,sha256}, > {dhe_dss,aes_128_cbc,sha256}, > {rsa,aes_128_gcm,null,sha256}, > {rsa,aes_128_cbc,sha256}, > {ecdhe_ecdsa,aes_256_cbc,sha}, > {ecdhe_rsa,aes_256_cbc,sha}, > {dhe_rsa,aes_256_cbc,sha}, > {dhe_dss,aes_256_cbc,sha}, > {ecdh_ecdsa,aes_256_cbc,sha}, > {ecdh_rsa,aes_256_cbc,sha}, > {rsa,aes_256_cbc,sha}, > {ecdhe_ecdsa,'3des_ede_cbc',sha}, > {ecdhe_rsa,'3des_ede_cbc',sha}, > {dhe_rsa,'3des_ede_cbc',sha}, > {dhe_dss,'3des_ede_cbc',sha}, > {ecdh_ecdsa,'3des_ede_cbc',sha}, > {ecdh_rsa,'3des_ede_cbc',sha}, > {rsa,'3des_ede_cbc',sha}, > {ecdhe_ecdsa,aes_128_cbc,sha}, > {ecdhe_rsa,aes_128_cbc,sha}, > {dhe_rsa,aes_128_cbc,sha}, > {dhe_dss,aes_128_cbc,sha}, > {ecdh_ecdsa,aes_128_cbc,sha}, > {ecdh_rsa,aes_128_cbc,sha}, > {rsa,aes_128_cbc,sha}, > {dhe_rsa,des_cbc,sha}, > {rsa,des_cbc,sha}]}}}}}, > {child,undefined, > {ranch_listener_sup,https}, > {ranch_listener_sup,start_link, > [https,100,ranch_ssl, > [{port,8443}, > {cacertfile, > > > "/home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/lib/ssl_hello_world-1/priv/ssl/cowboy-ca.crt"}, > {certfile, > > > "/home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/lib/ssl_hello_world-1/priv/ssl/server.crt"}, > {keyfile, > > > "/home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/lib/ssl_hello_world-1/priv/ssl/server.key"}], > cowboy_protocol, > [{env, > [{dispatch, > [{'_',[],[{[],[],toppage_handler,[]}]}]}]}]]}, > permanent,infinity,supervisor, > [ranch_listener_sup]}}}}, > [{ssl_hello_world_app,start,2, > [{file,"src/ssl_hello_world_app.erl"},{line,20}]}, > {application_master,start_it_old,4, > [{file,"application_master.erl"},{line,273}]}]}}}} > type: permanent > {"Kernel pid > terminated",application_controller,"{application_start_failure,ssl_hello_world,{bad_return,{{ssl_hello_world_app,start,[normal,[]]},{'EXIT',{{badmatch,{error,{{shutdown,{failed_to_start_child,ranch_acceptors_sup,{listen_error,https,{options,{ciphers,[{ecdhe_ecdsa,aes_256_gcm,null,sha384},{ecdhe_rsa,aes_256_gcm,null,sha384},{ecdhe_ecdsa,aes_256_cbc,sha384,sha384},{ecdhe_rsa,aes_256_cbc,sha384,sha384},{ecdh_ecdsa,aes_256_gcm,null,sha384},{ecdh_rsa,aes_256_gcm,null,sha384},{ecdh_ecdsa,aes_256_cbc,sha384,sha384},{ecdh_rsa,aes_256_cbc,sha384,sha384},{dhe_rsa,aes_256_gcm,null,sha384},{dhe_dss,aes_256_gcm,null,sha384},{dhe_rsa,aes_256_cbc,sha256},{dhe_dss,aes_256_cbc,sha256},{rsa,aes_256_gcm,null,sha384},{rsa,aes_256_cbc,sha256},{ecdhe_ecdsa,aes_128_gcm,null,sha256},{ecdhe_rsa,aes_128_gcm,null,sha256},{ecdhe_ecdsa,aes_128_cbc,sha256,sha256},{ecdhe_rsa,aes_128_cbc,sha256,sha256},{ecdh_ecdsa,aes_128_gcm,null,sha256},{ecdh_rsa,aes_128_gcm,null,sha256},{ecdh_ecdsa,aes_128_cbc,sha256,sha > 256},{ec > d > > h_rsa,aes_128_cbc,sha256,sha256},{dhe_rsa,aes_128_gcm,null,sha256},{dhe_dss,aes_128_gcm,null,sha256},{dhe_rsa,aes_128_cbc,sha256},{dhe_dss,aes_128_cbc,sha256},{rsa,aes_128_gcm,null,sha256},{rsa,aes_128_cbc,sha256},{ecdhe_ecdsa,aes_256_cbc,sha},{ecdhe_rsa,aes_256_cbc,sha},{dhe_rsa,aes_256_cbc,sha},{dhe_dss,aes_256_cbc,sha},{ecdh_ecdsa,aes_256_cbc,sha},{ecdh_rsa,aes_256_cbc,sha},{rsa,aes_256_cbc,sha},{ecdhe_ecdsa,'3des_ede_cbc',sha},{ecdhe_rsa,'3des_ede_cbc',sha},{dhe_rsa,'3des_ede_cbc',sha},{dhe_dss,'3des_ede_cbc',sha},{ecdh_ecdsa,'3des_ede_cbc',sha},{ecdh_rsa,'3des_ede_cbc',sha},{rsa,'3des_ede_cbc',sha},{ecdhe_ecdsa,aes_128_cbc,sha},{ecdhe_rsa,aes_128_cbc,sha},{dhe_rsa,aes_128_cbc,sha},{dhe_dss,aes_128_cbc,sha},{ecdh_ecdsa,aes_128_cbc,sha},{ecdh_rsa,aes_128_cbc,sha},{rsa,aes_128_cbc,sha},{dhe_rsa,des_cbc,sha},{rsa,des_cbc,sha}]}}}}},{child,undefined,{ranch_listener_sup,https},{ranch_listener_sup,start_link,[https,100,ranch_ssl,[{port,8443},{cacertfile,\"/home/victor/src/cowbo > y/exampl > e > > s/ssl_hello_world/_rel/ssl_hello_world_example/lib/ssl_hello_world-1/priv/ssl/cowboy-ca.crt\"},{certfile,\"/home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/lib/ssl_hello_world-1/priv/ssl/server.crt\"},{keyfile,\"/home/victor/src/cowboy/examples/ssl_hello_world/_rel/ssl_hello_world_example/lib/ssl_hello_world-1/priv/ssl/server.key\"}],cowboy_protocol,[{env,[{dispatch,[{'_',[],[{[],[],toppage_handler,[]}]}]}]}]]},permanent,infinity,supervisor,[ranch_listener_sup]}}}},[{ssl_hello_world_app,start,2,[{file,\"src/ssl_hello_world_app.erl\"},{line,20}]},{application_master,start_it_old,4,[{file,\"application_master.erl\"},{line,273}]}]}}}}}"} > > Crash dump is being written to: erl_crash.dump...done > > > On 27/04/16 11:34, Henrik Nord X wrote: > >> Patch Package: OTP 18.3.2 >> Git Tag: OTP-18.3.2 >> Date: 2016-04-27 >> Trouble Report Id: OTP-13261, OTP-13510, OTP-13511 >> Seq num: >> System: OTP >> Release: 18 >> Application: inets-6.2.2, ssl-7.3.1 >> Predecessor: OTP 18.3.1 >> >> Check out the git tag OTP-18.3.2, and build a full OTP system >> including documentation. Apply one or more applications from this >> build as patches to your installation using the 'otp_patch_apply' >> tool. For information on install requirements, see descriptions for >> each application version below. >> >> --------------------------------------------------------------------- >> --- inets-6.2.2 ----------------------------------------------------- >> --------------------------------------------------------------------- >> >> The inets-6.2.2 application can be applied independently of other >> applications on a full OTP 18 installation. >> >> --- Improvements and New Features --- >> >> OTP-13510 Application(s): inets >> >> Add environment information item peer_cert to mod_esi >> >> >> Full runtime dependencies of inets-6.2.2: erts-6.0, kernel-3.0, >> mnesia-4.12, runtime_tools-1.8.14, ssl-5.3.4, stdlib-2.0 >> >> >> --------------------------------------------------------------------- >> --- ssl-7.3.1 ------------------------------------------------------- >> --------------------------------------------------------------------- >> >> The ssl-7.3.1 application can be applied independently of other >> applications on a full OTP 18 installation. >> >> --- Fixed Bugs and Malfunctions --- >> >> OTP-13511 Application(s): ssl >> >> Corrections to cipher suite handling using the 3 and 4 >> tuple format in addition to commit >> 89d7e21cf4ae988c57c8ef047bfe85127875c70c >> >> >> --- Improvements and New Features --- >> >> OTP-13261 Application(s): ssl >> >> Make values for the TLS-1.2 signature_algorithms >> extension configurable >> >> >> Full runtime dependencies of ssl-7.3.1: crypto-3.3, erts-7.0, >> inets-5.10.7, kernel-3.0, public_key-1.0, stdlib-2.0 >> >> >> --------------------------------------------------------------------- >> --------------------------------------------------------------------- >> --------------------------------------------------------------------- >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions >> > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Thu Apr 28 22:21:24 2016 From: ingela.andin@REDACTED (Ingela Andin) Date: Thu, 28 Apr 2016 22:21:24 +0200 Subject: [erlang-questions] Memory consumption in ssl application (OTP-18.3.1) In-Reply-To: <572232E7.3050903@yandex.ru> References: <571E0F10.80507@yandex.ru> <5720C22F.3000404@yandex.ru> <5721E7BC.9050605@yandex.ru> <572232E7.3050903@yandex.ru> Message-ID: Hi! 2016-04-28 17:57 GMT+02:00 : > It seems that my assumption for the first question is correct. > > > https://github.com/erlang/otp/blob/OTP-18.3.2/lib/ssl/src/ssl_session.erl#L94 > > https://github.com/erlang/otp/blob/OTP-18.3.2/lib/ssl/src/ssl_session.erl#L114 > > The session_cb could be set to an "empty" implementation to completely > disable the cache. > Yes that is possible. Regards Ingela Erlang/OTP team - Ericsson AB > > > https://github.com/erlang/otp/blob/OTP-18.3.2/lib/ssl/src/ssl_manager.erl#L241 > > 28/04/16 13:36, vicbaz@REDACTED ?????: > > Hello, >> >> Could you please clarify one moment. Is the ssl session cache used only >> when reuse_sessions is set to true? Is there a way to turn off the cache >> completely? I'm trying to figure out what settings are needed for >> thousands of short-lived connections. >> >> 28/04/16 12:21, Ingela Andin ?????: >> >>> Hi! >>> >>> >>> 2016-04-27 15:44 GMT+02:00 >: >>> >>> Hello, >>> >>> The reason was session_cache_server_max (defaults to 1000). >>> The ssl_manager quickly get filled by >>> >>> {'$gen_cast',{invalidate_session,8443,{session,<<...>>,...}}} >>> {delayed_clean_session,{8443,<<14,50,229,...>>},40986} >>> >>> messages. >>> >>> I see, we will looking into mitigating this effect. >>> >>> Regards Ingela Erlang/OTP team - Ericsson AB >>> >>> >>> >>> 27/04/16 13:29, Ingela Andin ?????: >>> >>> Hi! >>> >>> You could try using the observer application to find out more >>> information about your system state. >>> There is no apparent reason why ssl-7.3 should consume a lot >>> more memory >>> than ssl-6.0.1.2. >>> But a lot has append between the versions and heavy load may >>> find corner >>> cases that where missed. >>> >>> Regards Ingela Erlang/OTP Team - Ericsson AB >>> >>> >>> 2016-04-25 14:35 GMT+02:00 >> >> >>: >>> >>> >>> Hello, >>> >>> When I run >>> >>> https://github.com/ninenines/cowboy/tree/1.0.4/examples/ssl_hello_world >>> under load (using https://github.com/JoeDog/siege) I found >>> a big >>> difference in memory consumption between OTP-17.5 and >>> OTP-18.3. I know >>> the ssl application has many changes in new versions. Is >>> this memory >>> usage normal now or can be reduced using some ssl >>> application settings? >>> >>> I use OTP-17.5.6.9 and OTP-18.3.1. >>> >>> >>> >>> >>> ================================================================================ >>> >>> Erlang/OTP 17 [erts-6.4.1.6] [source] [64-bit] [smp:2:2] >>> [async-threads:10] [hipe] [kernel-poll:false] >>> >>> 1> application:which_applications(). >>> [{ssl_hello_world,"Cowboy Hello World example with SSL.", >>> "1"}, >>> {cowboy,"Small, fast, modular HTTP server.","1.0.4"}, >>> {ranch,"Socket acceptor pool for TCP protocols.","1.2.1"}, >>> {cowlib,"Support library for manipulating Web protocols.", >>> "1.0.2"}, >>> {ssl,"Erlang/OTP SSL application","6.0.1.2"}, >>> {public_key,"Public key infrastructure","0.23"}, >>> {crypto,"CRYPTO","3.5"}, >>> {asn1,"The Erlang ASN1 compiler version 3.0.4","3.0.4"}, >>> {stdlib,"ERTS CXC 138 10","2.4"}, >>> {kernel,"ERTS CXC 138 10","3.2.0.1"}] >>> >>> 2> erlang:memory(). >>> [{total,43913840}, >>> {processes,8036312}, >>> {processes_used,8030744}, >>> {system,35877528}, >>> {atom,470537}, >>> {atom_used,455904}, >>> {binary,1498800}, >>> {code,11074476}, >>> {ets,14516424}] >>> >>> 3> recon_alloc:memory(used). >>> 45519464 >>> >>> 4> recon_alloc:memory(usage). >>> 0.7212344918526264 >>> >>> 5> recon:proc_window(memory, 10, 500). >>> [{<0.25910.2>,68120, >>> [{current_function,{gen_fsm,loop,7}}, >>> {initial_call,{proc_lib,init_p,5}}]}, >>> {<0.25908.2>,68120, >>> [{current_function,{gen_fsm,loop,7}}, >>> {initial_call,{proc_lib,init_p,5}}]}, >>> {<0.284.0>,15808, >>> [ssl_manager, >>> {current_function,{gen_server,loop,6}}, >>> {initial_call,{proc_lib,init_p,5}}]}, >>> {<0.285.0>,12712, >>> [tls_connection_sup, >>> {current_function,{gen_server,loop,6}}, >>> {initial_call,{proc_lib,init_p,5}}]}, >>> {<0.25909.2>,5888, >>> [{current_function,{gen,do_call,4}}, >>> {initial_call,{cowboy_protocol,init,4}}]}, >>> {<0.324.0>,4888, >>> [{current_function,{prim_inet,accept0,2}}, >>> {initial_call,{ranch_acceptor,loop,3}}]}, >>> {<0.302.0>,4808, >>> [{current_function,{ranch_conns_sup,loop,4}}, >>> {initial_call,{proc_lib,init_p,5}}]}, >>> {<0.330.0>,3016, >>> [{current_function,{prim_inet,accept0,2}}, >>> {initial_call,{ranch_acceptor,loop,3}}]}, >>> {<0.25911.2>,2848, >>> [{current_function,{gen,do_call,4}}, >>> {initial_call,{cowboy_protocol,init,4}}]}, >>> {<0.340.0>,1872, >>> [{current_function,{prim_inet,accept0,2}}, >>> {initial_call,{ranch_acceptor,loop,3}}]}] >>> >>> 6> recon:proc_window(reductions, 10, 500). >>> [{<0.2634.3>,11161, >>> [{current_function,{gen_fsm,loop,7}}, >>> {initial_call,{proc_lib,init_p,5}}]}, >>> {<0.2633.3>,9747, >>> [{current_function,{crypto,int_to_bin_pos,2}}, >>> {initial_call,{proc_lib,init_p,5}}]}, >>> {<0.284.0>,4097, >>> [ssl_manager, >>> {current_function,{gen_server,loop,6}}, >>> {initial_call,{proc_lib,init_p,5}}]}, >>> {<0.285.0>,3314, >>> [tls_connection_sup, >>> {current_function,{gen_server,loop,6}}, >>> {initial_call,{proc_lib,init_p,5}}]}, >>> {<0.302.0>,1003, >>> [{current_function,{ranch_conns_sup,loop,4}}, >>> {initial_call,{proc_lib,init_p,5}}]}, >>> {<0.378.0>,426, >>> [{current_function,{prim_inet,accept0,2}}, >>> {initial_call,{ranch_acceptor,loop,3}}]}, >>> {<0.372.0>,425, >>> [{current_function,{prim_inet,accept0,2}}, >>> {initial_call,{ranch_acceptor,loop,3}}]}, >>> {<0.379.0>,425, >>> [{current_function,{prim_inet,accept0,2}}, >>> {initial_call,{ranch_acceptor,loop,3}}]}, >>> {<0.377.0>,425, >>> [{current_function,{prim_inet,accept0,2}}, >>> {initial_call,{ranch_acceptor,loop,3}}]}, >>> {<0.343.0>,425, >>> [{current_function,{prim_inet,accept0,2}}, >>> {initial_call,{ranch_acceptor,loop,3}}]}] >>> >>> RES in htop ~50-60M >>> >>> >>> >>> >>> ================================================================================ >>> >>> >>> Erlang/OTP 18 [erts-7.3.1] [source] [64-bit] [smp:2:2] >>> [async-threads:10] [hipe] [kernel-poll:false] >>> >>> 1> application:which_applications(). >>> [{ssl_hello_world,"Cowboy Hello World example with SSL.", >>> "1"}, >>> {cowboy,"Small, fast, modular HTTP server.","1.0.4"}, >>> {ranch,"Socket acceptor pool for TCP protocols.","1.2.1"}, >>> {cowlib,"Support library for manipulating Web protocols.", >>> "1.0.2"}, >>> {ssl,"Erlang/OTP SSL application","7.3"}, >>> {public_key,"Public key infrastructure","1.1.1"}, >>> {crypto,"CRYPTO","3.6.3"}, >>> {asn1,"The Erlang ASN1 compiler version 4.0.2","4.0.2"}, >>> {recon,"Diagnostic tools for production use","2.3.1"}, >>> {stdlib,"ERTS CXC 138 10","2.8"}, >>> {kernel,"ERTS CXC 138 10","4.2"}] >>> >>> 2> erlang:memory(). >>> [{total,949153560}, >>> {processes,902969888}, >>> {processes_used,902558312}, >>> {system,46183672}, >>> {atom,437761}, >>> {atom_used,432875}, >>> {binary,48320}, >>> {code,11407283}, >>> {ets,883640}] >>> >>> 3> recon_alloc:memory(used). >>> 863194560 >>> >>> 4> recon_alloc:memory(usage). >>> 0.8005009562139471 >>> >>> 5> recon:proc_window(memory, 10, 500). >>> [{<0.290.0>,94360, >>> [ssl_manager, >>> {current_function,{ssl_manager,handle_cast,2}}, >>> {initial_call,{proc_lib,init_p,5}}]}, >>> {<0.13589.3>,24640, >>> [{current_function,{ssl_manager,validate_session,3}}, >>> {initial_call,{ssl_manager,init_session_validator,1}}]}, >>> {<0.291.0>,12832, >>> [tls_connection_sup, >>> {current_function,{gen_server,loop,6}}, >>> {initial_call,{proc_lib,init_p,5}}]}, >>> {<0.13590.3>,10960, >>> [{current_function,{gen,do_call,4}}, >>> {initial_call,{proc_lib,init_p,5}}]}, >>> {<0.318.0>,7904, >>> [{current_function,{prim_inet,accept0,2}}, >>> {initial_call,{ranch_acceptor,loop,3}}]}, >>> {<0.308.0>,3056, >>> [{current_function,{ranch_conns_sup,loop,4}}, >>> {initial_call,{proc_lib,init_p,5}}]}, >>> {<0.385.0>,1872, >>> [{current_function,{prim_inet,accept0,2}}, >>> {initial_call,{ranch_acceptor,loop,3}}]}, >>> {<0.254.0>,0, >>> [code_server, >>> {current_function,{code_server,loop,1}}, >>> {initial_call,{erlang,apply,2}}]}, >>> {<0.325.0>,0, >>> [{current_function,{prim_inet,accept0,2}}, >>> {initial_call,{ranch_acceptor,loop,3}}]}, >>> {<0.357.0>,0, >>> [{current_function,{prim_inet,accept0,2}}, >>> {initial_call,{ranch_acceptor,loop,3}}]}] >>> >>> 6> recon:proc_window(reductions, 10, 500). >>> [{<0.290.0>,232358, >>> [ssl_manager, >>> {current_function,{gen_server,loop,6}}, >>> {initial_call,{proc_lib,init_p,5}}]}, >>> {<0.291.0>,3058, >>> [tls_connection_sup, >>> {current_function,{gen_server,loop,6}}, >>> {initial_call,{proc_lib,init_p,5}}]}, >>> {<0.308.0>,1129, >>> [{current_function,{ranch_conns_sup,loop,4}}, >>> {initial_call,{proc_lib,init_p,5}}]}, >>> {<0.339.0>,435, >>> [{current_function,{prim_inet,accept0,2}}, >>> {initial_call,{ranch_acceptor,loop,3}}]}, >>> {<0.340.0>,430, >>> [{current_function,{prim_inet,accept0,2}}, >>> {initial_call,{ranch_acceptor,loop,3}}]}, >>> {<0.355.0>,430, >>> [{current_function,{prim_inet,accept0,2}}, >>> {initial_call,{ranch_acceptor,loop,3}}]}, >>> {<0.353.0>,430, >>> [{current_function,{prim_inet,accept0,2}}, >>> {initial_call,{ranch_acceptor,loop,3}}]}, >>> {<0.334.0>,430, >>> [{current_function,{prim_inet,accept0,2}}, >>> {initial_call,{ranch_acceptor,loop,3}}]}, >>> {<0.366.0>,430, >>> [{current_function,{prim_inet,accept0,2}}, >>> {initial_call,{ranch_acceptor,loop,3}}]}, >>> {<0.360.0>,430, >>> [{current_function,{prim_inet,accept0,2}}, >>> {initial_call,{ranch_acceptor,loop,3}}]}] >>> >>> RES in htop ~900-1000M >>> _______________________________________________ >>> erlang-questions mailing list >>> erlang-questions@REDACTED >>> >> > >>> http://erlang.org/mailman/listinfo/erlang-questions >>> >>> >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Thu Apr 28 22:47:45 2016 From: ingela.andin@REDACTED (Ingela Andin) Date: Thu, 28 Apr 2016 22:47:45 +0200 Subject: [erlang-questions] Improvements in TLS error reporting In-Reply-To: References: Message-ID: Hi! Sounds interesting, but remember that not all information is desirable to have in the logs. Sometimes the protocol will hide certain errors on purpose. Also too extensive logging can be bad. Although as a developer I appreciate as informative error messages as possible. Regards Ingela Erlang/OTP team - Ericsson AB 2016-04-28 18:50 GMT+02:00 Alexey Lebedeff : > Hi, > > I have some ideas about making SSL error reporting more friendlier for > operators, because currently troubleshooting can be very > problematic. If the solution outlined at the end of the message is > acceptable, I'll make proper PR on github. > > There are several places where SSL/TLS handshake errors are being > converted to SSL alerts. During this process some details about errors > are being discarded, so they will fit SSL alert protocol. But the > problem is that the same simplified errors are then used for event > logging - and sometimes they are completely useless. > > So several times I've had to resort to adding debug print statements > to ssl app just to find what I've done wrong in my SSL config. > > Some examples I've observed: > - Certificate was in wrong format, but detailed error was discarded at [1] > And 'internal error' message wasn't helpful at all. > - Certificate had wrong ext-key-usage field (there was attempt to use > server certificate as a client one) This info was happily discarded > at [2]. And again, the error was logged as unhelful 'handshake > failure'. > > This situation can be improved. I have a suggestion to add additional > field to SSL alert record at [3] which will contain detailed error > description which is currently being throwed out. And then use this > description in ssl_alert:alert_txt/1 at [4]. > > [1] > https://github.com/erlang/otp/blob/523e048754f5086a6cc4fd9a250e1b495fc5b9b8/lib/ssl/src/ssl_handshake.erl#L169 > [2] > https://github.com/erlang/otp/blob/523e048754f5086a6cc4fd9a250e1b495fc5b9b8/lib/ssl/src/ssl_handshake.erl#L1491 > [3] > https://github.com/erlang/otp/blob/523e048754f5086a6cc4fd9a250e1b495fc5b9b8/lib/ssl/src/ssl_alert.hrl#L116 > [4] > https://github.com/erlang/otp/blob/523e048754f5086a6cc4fd9a250e1b495fc5b9b8/lib/ssl/src/ssl_alert.erl#L77 > > Best, > Alexey > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erlang@REDACTED Thu Apr 28 22:53:12 2016 From: erlang@REDACTED (Joe Armstrong) Date: Thu, 28 Apr 2016 22:53:12 +0200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> Message-ID: On Thu, Apr 28, 2016 at 5:52 AM, Richard A. O'Keefe wrote: > I've been thinking for some time of writing a paper with the > title "Why can't I see the structure?" based on the idea that > modules in every programming language I know look like blobs. > I'm aware of visual notations like UML, BON, SDL, and what > was it, Visual Erlang? But for me, those are just spaghetti > with meatballs; once you get beyond just a handful of boxes > in your diagram, all diagrams look much the same. In any > case, I'm interested in the medium scale. > > Why can't I see the structure in a 3000-line module, or even > a 1000-line module? (I am not asserting that Erlang is > particularly bad here. It isn't.) [long rambling possibly off topic reply follows] [you have been warned] Well I would hope that you don't have to see the structure of such a module. To me systems should be made of black boxes that communicate through well-specified protocols - so the top level description would tell me how the boxes were connected together and describe the protocols at a high level of abstraction. To me the structure is in state S receive {X, Pattern1} -> ... loads of code I don't need to understand ... Y ! Value1 All I should need to know about the module is that it implements the protocol - How it works should be irrelevant. This in fact how Erlang was used - in the early days we'd start with a CCITT spec in SDL - this had the top level - which black boxes there were, how they were connected and what protocols they obeyed. Unfortunately the structure (which process starts another process) which protocol is obeyed is implicit in code and not explicit. We can only talk about things if they have names and most often we don't name the protocols used between processes. Saying the process P1 and modules M1, M2, M3 etc terminate the XYZ protocol is an extremely precise way of describing what a system does. Tracing the protocol and comparing to what you expect gives an immediate way to discover errors. If I'm debugging code I don't place print statements randomly, but at the points where messages enter and leave the application - unfortunately we can't distinguish "important" messages from background chatter. Erlang was designed to be more-or-less isomorphic to SDL (see https://en.wikipedia.org/wiki/Specification_and_Description_Language) Where Erlang was the bottom of three levels of design. Unfortunately SDL, which was great, got incorporated into UML which was a *!@#$$*$$$ ing mess. Erlang is all about sending messages to things, so message sequence charts (https://en.wikipedia.org/wiki/Message_sequence_chart) are brilliant for describing how Erlang programs work. When I was at Ericsson we drew MSC diagrams ( the systems guys did this) then the programmers took these and implemented them - the problem here is that the MSC diagrams, from which the code was derived, are not included in the code. A big problem with code is that we throw away the design documents necessary to write the code - and don't say what specifications the code fulfills. Code that starts with a comment %% This code implements the protocol described in [Ref] Is great > > The kind of structure I'm interested in can, I think, be > described as *rhetorical* structure, like relationships > between paragraphs. > > My *belief* is that if this structure were made explicit, > perhaps by textual structure, perhaps by annotations, perhaps > by some graphical form (but probably derived from annotations), > it would be easier to understand medium-sized wodges of code. I'm beginning to wonder if the problem is with editors - I use emacs erlang mode which has no help for structuring and organising files. For writing text and lectures I use emacs org-mode with wiki like links. Hierarchical structure (in org mode) is really easy to see and change and with wiki links is pretty powerful. I'm beginning to think that a wiki style of programming might work. In a wiki if you click on a link and no page exists you are prompted to write some text - we could make a similar thing to develop programs. > > I'm aware of annotation support in languages like Java and C# > and for that matter, Smalltalk, but with the exception of > Smalltalk, nobody seems to be using annotations in this way > (and that exception is me). > > I'd be very interested in hearing from anyone else who has been > thinking in this area. Me too :-) Another idea I had was "monkey patching done right" The problem is this: What I'm writing a module "a" I think "I wish module z exported foo, and I wish file.erl exported boo Now I don't want to have to stop what I'm doing in a and add something to z (since then I'll get distracted and mess around with z) - and goodness changing file.erl is really difficult - I'll have to argue blue in the face and write an eep - just think about adding something to lists.erl In a.erl why can't I say -module(a). -patch(lists) my_super_func(...) -> -end. which adds my_super_func to lists (only it doesn't really) only for me and not for everybody else. I think controlling the name space is the way to solve a load of problems but hierarchical names spaces are a mess, and we can't all agree. I think we need constructs like " what joe call x anne calls y" I can imaging blocks of code just being named by their MD5 checksums and having aliases alias factorial = "0xad45..c13" (a md5 checksum) and naming and discovery mechanism so we could say "md5-1" computes the same thing as "md5-2" The bigger problem of understanding "my" code or "this" code is " has somebody else already solved this problem". My feeling is that most problems have already been solved - but we write code because it's quicker to write code than discover it. When I get really stuck I post a message to stack overflow and post a tweet. This is a *fantastic* way to solve problems - but I'm privileged to have several followers. Understanding code seems to be related to the idea of making a minimal kernel that we can understand - so one thing I can't read from code is "this is the bit you have to understand". In code reviews I would ask "how does this work" or "what's the most important stuff" ... Robert Virding (Blessed be his name) once wrote a very long module for pattern matching with exactly one comment: %% And now for the tricky bit It was nice to know ... [I warned you] Cheers /Joe > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions From ok@REDACTED Fri Apr 29 02:48:16 2016 From: ok@REDACTED (Richard A. O'Keefe) Date: Fri, 29 Apr 2016 12:48:16 +1200 Subject: [erlang-questions] Max heap size In-Reply-To: References: <9b834742-3078-8fd9-e4e7-d1909a954c9d@cs.otago.ac.nz> Message-ID: <8add20af-fdee-35b3-1577-dbeccb14ae17@cs.otago.ac.nz> On 28/04/16 8:31 PM, Lukas Larsson wrote: > > The documentation is part of the pull request, the options and > behavior are described here: > https://github.com/erlang/otp/pull/1032/commits/629c6c0a4aea094bea43a74ca1c1664ec1041e43#diff-0fc9fb0d3e12721dd1574a543916e8c6R4349 Good stuff. "The heap size of a process is quite hard to predict, especially the + amount of memory that is used during the garbage collection." You can say that again. Advice about what to DO about that fact would be welcome at this point. "When contemplating to use this option," s/to use/using/. > > - what are the units in which 'size' is measured? > > > It is the same as min_heap_size and all other options that we that > effect the process heap, the internal word size, Worth saying *explicitly*. I'm used to C, Java, and C# specifying thread sizes in bytes, which the current Java documentation admits is a problem. does something sane > > - what is the performance cost of this? > > > If not enabled, it costs one branch per gc Brag of it. All things considered, I now see that this is thoroughly thought out, and unlikely to be needed very often, but valued greatly on the few occasions when it is needed. From vladdu55@REDACTED Fri Apr 29 09:47:49 2016 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Fri, 29 Apr 2016 09:47:49 +0200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> Message-ID: Hi Joe and all, On Thu, Apr 28, 2016 at 10:53 PM, Joe Armstrong wrote: > To me systems should be made of black boxes that communicate through > well-specified protocols - so the top level description would tell me > how the boxes were connected together and describe the protocols > at a high level of abstraction. > That's just half the story, right? Somewhere, the system has to do something with those messages and use "normal" sequential programming. So structuring that code is just as important. > A big problem with code is that we throw away the design documents > necessary to write the code - and don't say what specifications the code > fulfills. > There are tools that can process MSCs in textual form (that can be stored alongside the code and referred from it) and the documentation generators can produce graphical representations (Doxygen has support for this, for example). So the tools are there (or can be written), it is mostly a matter of using them. > My *belief* is that if this structure were made explicit, > > perhaps by textual structure, perhaps by annotations, perhaps > > by some graphical form (but probably derived from annotations), > > it would be easier to understand medium-sized wodges of code. > > I'm beginning to wonder if the problem is with editors - I use > emacs erlang mode which has no help for structuring and organising > files. > > For writing text and lectures I use emacs org-mode with wiki like links. > > Hierarchical structure (in org mode) is really easy to see and change > and with wiki links is pretty powerful. > > I'm beginning to think that a wiki style of programming might work. > > In a wiki if you click on a link and no page exists you are prompted > to write some text - we could make a similar thing to develop > programs. > ... > Another idea I had was "monkey patching done right" > > The problem is this: > > What I'm writing a module "a" I think > "I wish module z exported foo, and I wish file.erl exported boo > > Now I don't want to have to stop what I'm doing in a and add something > to z (since then I'll get distracted and mess around with z) - and > goodness > changing file.erl is really difficult - I'll have to argue blue in the > face and write > an eep - just think about adding something to lists.erl > I have many (more or less wild) similar ideas about how to better support development, but most of them require an IDE and many people want an all-text-and-command-line experience. My belief is that would change once they get a better feeling of the IDE experience, but unfortunately I am a bit stuck with more mundane implementation issues... For example, the code editor could be editing a "virtual file" where functions names in the definitions are qualified with a module name. These functions get written and compiled into the respective modules, but are shown together. As I write code, for example calling a function, the referred code is pulled in the editor and if it didn't existed it is created. So we get a dynamic view of the code, relevant to what I'm currently doing. There are already tools experimenting with such ways of structuring code. For example, Eclipse has Mylyn which can provide a filtered view of the code (https://en.wikipedia.org/wiki/Task-focused_interface), hiding aspects that are currently irrelevant. CodeBubbles ( http://cs.brown.edu/~spr/codebubbles/) tries a non-linear editing experience where you can see dependencies explicitly. best regards, Vlad -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierrefenoll@REDACTED Fri Apr 29 09:52:22 2016 From: pierrefenoll@REDACTED (Pierre Fenoll) Date: Fri, 29 Apr 2016 09:52:22 +0200 Subject: [erlang-questions] Rhetorical structure of code: Anyone interested in collaborating? In-Reply-To: <20160428150211.GG2013@fhebert-ltm2.internal.salesforce.com> References: <9db646a4-2965-f922-b5c9-fc346399c907@cs.otago.ac.nz> <20160428150211.GG2013@fhebert-ltm2.internal.salesforce.com> Message-ID: On 28 April 2016 at 17:02, Fred Hebert wrote: > Erlang has made a few things very interesting by forcing an additional > structure in supervision trees and well-defined behaviour, which lets me > scan things at a glance in terms of project structure and infer what it > does without actually needing to understand code. > I think a graphical tool that shows the OTP structure of an Erlang project would be useful: master application then supervision tree then worker modules along with their registered names, restart strategy & supervisor : #workers|type mapping. Some issues could be identified visually = quickly. I believe this could be done without the need to pre-run the actual application, just looking for start_child and such. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sgolovan@REDACTED Fri Apr 29 10:36:42 2016 From: sgolovan@REDACTED (Sergei Golovan) Date: Fri, 29 Apr 2016 11:36:42 +0300 Subject: [erlang-questions] inets_regexp is missing in 18.3 Message-ID: Hi! Appears that the inets_regexp module disappeared in Erlang 18.3. I can't find any justification for that in the Readme, so I have to ask here: was this removal intentional? There's software which uses it (e.g. cowboy). Cheers! -- Sergei Golovan From eric.pailleau@REDACTED Fri Apr 29 11:04:49 2016 From: eric.pailleau@REDACTED (=?ISO-8859-1?Q?=C9ric_Pailleau?=) Date: Fri, 29 Apr 2016 11:04:49 +0200 Subject: [erlang-questions] inets_regexp is missing in 18.3 In-Reply-To: References: Message-ID: Hi, Yes true, you can check addition and deletion below in incremental release difference of geas_devel https://github.com/crownedgrouse/geas_devel/blob/master/doc/reldiffs/yaml/18.2~18.3 This however does not gives you the reason. Regards "Envoy? depuis mon mobile " Eric ---- Sergei Golovan a ?crit ---- >Hi! > >Appears that the inets_regexp module disappeared in Erlang 18.3. I >can't find any justification for that in the Readme, so I have to ask >here: was this removal intentional? There's software which uses it >(e.g. cowboy). > >Cheers! >-- >Sergei Golovan >_______________________________________________ >erlang-questions mailing list >erlang-questions@REDACTED >http://erlang.org/mailman/listinfo/erlang-questions -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Fri Apr 29 15:28:21 2016 From: ingela.andin@REDACTED (Ingela Andin) Date: Fri, 29 Apr 2016 15:28:21 +0200 Subject: [erlang-questions] inets_regexp is missing in 18.3 In-Reply-To: References: Message-ID: Hi! 2016-04-29 10:36 GMT+02:00 Sergei Golovan : > Hi! > > Appears that the inets_regexp module disappeared in Erlang 18.3. I > can't find any justification for that in the Readme, so I have to ask > here: was this removal intentional? There's software which uses it > (e.g. cowboy). > > Cheers! > inets_regexp has never been a supported API module, use the re module instead. Regards Ingela Erlang/OTP Team - Ericsson AB > -- > Sergei Golovan > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From community-manager@REDACTED Fri Apr 29 15:33:00 2016 From: community-manager@REDACTED (Bruce Yinhe) Date: Fri, 29 Apr 2016 15:33:00 +0200 Subject: [erlang-questions] New Erlang job openings Message-ID: Hi There are 34 Erlang-related job openings on https://erlangcentral.org/jobs. Subscribe to weekly Erlang job updates: https://erlangcentral.org/login/?action=register Publish an Erlang job opening: https://erlangcentral.org/jobs/add *Europe, the Middle East and Africa:* Back-end Developer - NGTI - Rotterdam, Netherlands https://erlangcentral.org/back-end-developer-ngti-nl/ Apply before: 2016-04-29 Senior Engineer with DevOps Focus - Campanja - Stockholm, Sweden https://erlangcentral.org/senior-engineer-with-devops-focus-campanja/ Apply before: 2016-04-29 Senior Erlang Developer - Theta Partners - London, UK https://erlangcentral.org/senior-erlang-developer-theta/ Apply before: 2016-04-29 Software Developer with Erlang - SumUp - Berlin, Germany https://erlangcentral.org/software-developer-with-erlang-sumup/ Apply before: 2016-04-29 Erlang Engineer - Expend.io - London, UK http://erlangcentral.org/erlang-engineer-expend/ Apply before: 2016-04-29 Senior Engineer with DevOps Focus - Campanja - Stockholm, Sweden http://erlangcentral.org/senior-engineer-with-devops-focus-campanja/ Apply before: 2016-04-29 Backend developeur - Kbrw Ad-Venture - Paris, France http://erlangcentral.org/backend-developeur-kbrw-ad-venture/ Apply before: 2016-04-29 Software Engineer - Systems Networking - LinuxRecruit - London, UK http://erlangcentral.org/software-engineer-systems-networking-mesosphere/ Apply before: 2016-04-29 Backend Engineer - Erlang - LinuxRecruit - Central London, UK http://erlangcentral.org/backend-engineer-erlang-linuxrecruit/ Apply before: 2016-05-21 Software Engineer in Erlang - Erlang Solutions - London, UK http://erlangcentral.org/software-engineer-erlang-solutions-london/ Apply before: 2016-05-30 Software Engineer - Erlang/Functional Programming - Klarna - Stockholm, Sweden http://erlangcentral.org/software-engineer-erlang-functional-programming-klarna/ Apply before: 2016-05-30 Elixir Developer - Contract - Darwin Recruitment - Billericay, UK http://erlangcentral.org/elixir-developer-contract-darwin-recruitment/ Apply before: 2016-05-30 Erlang/Elixir Developer - Darwin Recruitment - London, UK http://erlangcentral.org/erlang-elixir-developer-darwin/ Apply before: 2016-05-30 Erlang and Elixir interns - Erlang Solutions - Krak?w, Poland http://erlangcentral.org/erlang-and-elixir-interns-erlang-solutions/ Apply before: 2016-05-30 Senior Software Developer in Erlang - DEK Technologies Sweden AB - Stockholm, Sweden http://erlangcentral.org/senior-software-developer-in-erlang-dek-technologies/ Apply before: 2016-06-29 Erlang Developer - Contract - Darwin Recruitment - Billericay, UK http://erlangcentral.org/erlang-developer-contract-darwin-recruitment/ Apply before: 2016-06-29 Erlang Developer - Vocalink - London, UK http://erlangcentral.org/erlang-developer-vocalink/ Apply before: 2016-06-29 Erlang & Java Developer - Darwin Recruitment - London, UK http://erlangcentral.org/erlang-java-developer-darwin-recruitment/ Apply before: 2016-07-28 Backend software engineer - Electic Imp - Any (remote); London, UK preferred, UK http://erlangcentral.org/backend-software-engineer-electric-imp/ Apply before: 2016-07-30 Erlang Developer - Krubera Group - Hertfordshire, UK https://erlangcentral.org/erlang-developer-krubera-group/ Apply before: 2016-11-04 Erlang Software Engineer - F Technologies - Dubai, UAE http://erlangcentral.org/erlang-software-engineer-f-technologies/ Apply before: 2016-12-30 *Americas:* DevOps Engineer - CyberCoders - Millbrae, CA, USA http://erlangcentral.org/millbrae-devops-engineer-cybercoders/ Apply before: 2016-04-29 Software Engineer - Systems Networking - Mesosphere - San Francisco, CA, USA http://erlangcentral.org/software-engineer-systems-networking-mesosphere/ Apply before: 2016-04-29 Erlang Backend Engineer - Tigertext - Santa Monica, CA, USA http://erlangcentral.org/erlang-backend-engineer-tigertext/ Apply before: 2016-05-30 Lead Erlang Engineer - Relo Offered - SoCal Startup - Los Angeles, CA; Santa Monica, CA, USA https://erlangcentral.org/lead-erlang-engineer-relo-offered-socal-startup/ Apply before: 2016-05-30 Software Developer - LiveHelpNow - Willow Grove/Bethlehem, PA, USA http://erlangcentral.org/software-developer-livehelpnow/ Apply before: 2016-05-30 Sr Application Security Engineer, Erlang - Machine Zone - Palo Alto, CA, USA http://erlangcentral.org/sr-application-security-engineer-erlang-machine-zone/ Apply before: 2016-05-30 Erlang Developer - Inaka - Buenos Aires, Argentina https://erlangcentral.org/erlang-developer-inaka/ Apply before: 2016-05-31 Erlang Developer - Sqor Sports - San Francisco, CA, USA https://erlangcentral.org/erlang-developer-sqor/ Apply before: 2016-06-29 Senior Erlang Engineer - Go Factory - San Francisco, CA, USA http://erlangcentral.org/senior-erlang-engineer-go-factory/ Apply before: 2016-06-29 Lead Infrastructure and Erlang Engineer - Go Factory - San Francisco, CA, USA https://erlangcentral.org/lead-infrastructure-and-erlang-engineer-gofactory/ Apply before: 2016-06-29 Senior Erlang Engineer - Grindr - Hollywood, CA, USA http://erlangcentral.org/senior-erlang-engineer-grindr/ Apply before: 2016-06-29 Platform and Data Engineer - Handy's culture deck - New York City, USA http://erlangcentral.org/platform-and-data-engineer-handy/ Apply before: 2016-07-30 Back-end Engineer - 2600hz - San Francisco, CA, USA http://erlangcentral.org/back-end-engineer-2600hz/ Apply before: 2017-08-30 *Bruce Yinhe* Community Manager Industrial Erlang User Group +46 72 311 43 89 community-manager@REDACTED -- Visit our Erlang community site ErlangCentral.org | @ErlangCentral | Industrial Erlang User Group -------------- next part -------------- An HTML attachment was scrubbed... URL: From comptekki@REDACTED Fri Apr 29 17:55:28 2016 From: comptekki@REDACTED (Wes James) Date: Fri, 29 Apr 2016 09:55:28 -0600 Subject: [erlang-questions] erlsrv on windows 10 problem Message-ID: I use erlsrv to install an erlang application on windows. Windows 7 works fine, but on windows 10, I have to go in and change the service from automatic to Automatic (delayed start), otherwise the app never runs on system start up. Any idea how to fix this? I just found this windows command: sc config SVCNAME start= delayed-auto I can use that in the install script. I'd still like it to start with just automatic. With the delayed option, it takes several extra minutes for the service to start running. Thanks, -wes -------------- next part -------------- An HTML attachment was scrubbed... URL: From ingela.andin@REDACTED Fri Apr 29 20:46:18 2016 From: ingela.andin@REDACTED (Ingela Andin) Date: Fri, 29 Apr 2016 20:46:18 +0200 Subject: [erlang-questions] gen_statem - real example Message-ID: Hello! Now there is a real example of code using gen_statem. Check it out. https://github.com/erlang/otp/pull/1037 Regards Ingela Erlang/OTP team - Ericsson AB -------------- next part -------------- An HTML attachment was scrubbed... URL: From h_sowwan@REDACTED Sat Apr 30 00:09:16 2016 From: h_sowwan@REDACTED (Hassan Sowwan) Date: Fri, 29 Apr 2016 22:09:16 +0000 Subject: [erlang-questions] Erlang for Payment Systems Message-ID: Hello, I am trying to implement payment messaging middleware and would like to explore the option of using Erlang/OTP. The application will be used in banking industry to interface with EFT payment switch/networks and core banking system to process card transactions. It will be responsible to perform following tasks: Communicate with external interfaces using ISO 8583 messaging format ( thru TCP/IP)Receive huge amount of data over the socket ( HEX, BINARY, EBCIDIC), which represents financial transactions.Parse/decode the received data.Perform some checking in database for validationInterface with host security module to validate customer PIN and other security checks.Sends the request to core banking system via XML or web services callRespond back to external interfaces by formulating the response message in ISO 8583 format Obviously, such applications have to be concurrent and fast enough to process transactions within few seconds. Now my question here, is Erlang a good choice for implementing this type of applications ? Can Erlang handle string processing efficiently without impacting the system performance? As stated before, there will be a lot of string manipulation to decode data received over the network, so I am not sure whether erlang fits perfectly or not. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lee.sylvester@REDACTED Sat Apr 30 01:07:14 2016 From: lee.sylvester@REDACTED (Lee Sylvester) Date: Sat, 30 Apr 2016 11:07:14 +1200 Subject: [erlang-questions] Erlang for Payment Systems In-Reply-To: References: Message-ID: Hi Hassan, String manipulation is potentially Erlang's main weakness. However, there are ways and means. Firstly, I would create a simple prototype that performs the kind of string manipulations you need and testing its speed. If it is not fast enough, you can always implement the string specific functionality in a C library and hook into it via Erlang, so that Erlang handles all your processes and load balancing, but having the string specific work handled natively. I hope this makes sense? Thanks, Lee On Sat, Apr 30, 2016 at 10:09 AM, Hassan Sowwan wrote: > Hello, > > I am trying to implement payment messaging middleware and would like to > explore the option of using Erlang/OTP. > > The application will be used in banking industry to interface with EFT > payment switch/networks and core banking system to process card > transactions. > > It will be responsible to perform following tasks: > > > - Communicate with external interfaces using ISO 8583 messaging format > ( thru TCP/IP) > - Receive huge amount of data over the socket ( HEX, BINARY, EBCIDIC), > which represents financial transactions. > - Parse/decode the received data. > - Perform some checking in database for validation > - Interface with host security module to validate customer PIN and > other security checks. > - Sends the request to core banking system via XML or web services call > - Respond back to external interfaces by formulating the response > message in ISO 8583 format > > > Obviously, such applications have to be concurrent and fast enough to > process transactions within few seconds. > > Now my question here, is Erlang a good choice for implementing this type > of applications ? > Can Erlang handle string processing efficiently without impacting the > system performance? > As stated before, there will be a lot of string manipulation to decode > data received over the network, so I am not sure whether erlang fits > perfectly or not. > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From felixgallo@REDACTED Sat Apr 30 01:09:17 2016 From: felixgallo@REDACTED (Felix Gallo) Date: Fri, 29 Apr 2016 16:09:17 -0700 Subject: [erlang-questions] Erlang for Payment Systems In-Reply-To: References: Message-ID: You left off some salient details. Does your system need a very high transaction rate? At the high end, what is the aggregate bandwidth flowing through your system? Does your system require intense mathematical calculation or is it primarily, as described, performing a routing function? Erlang is used in many financial contexts by companies like Goldman Sachs, various betting firms, etc. So it can definitely do well in that space. Erlang is excellent, perhaps even unparalleled, at packet structuring/destructuring logic. I suspect you have used strings for that in the past, but Erlang does it a little differently, using binaries, bins, and pattern matching. In my experience in financial services, string manipulation in protocol handling is super rare, but perhaps your experience and need is different. F. On Fri, Apr 29, 2016 at 3:09 PM, Hassan Sowwan wrote: > Hello, > > I am trying to implement payment messaging middleware and would like to > explore the option of using Erlang/OTP. > > The application will be used in banking industry to interface with EFT > payment switch/networks and core banking system to process card > transactions. > > It will be responsible to perform following tasks: > > > - Communicate with external interfaces using ISO 8583 messaging format > ( thru TCP/IP) > - Receive huge amount of data over the socket ( HEX, BINARY, EBCIDIC), > which represents financial transactions. > - Parse/decode the received data. > - Perform some checking in database for validation > - Interface with host security module to validate customer PIN and > other security checks. > - Sends the request to core banking system via XML or web services call > - Respond back to external interfaces by formulating the response > message in ISO 8583 format > > > Obviously, such applications have to be concurrent and fast enough to > process transactions within few seconds. > > Now my question here, is Erlang a good choice for implementing this type > of applications ? > Can Erlang handle string processing efficiently without impacting the > system performance? > As stated before, there will be a lot of string manipulation to decode > data received over the network, so I am not sure whether erlang fits > perfectly or not. > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dgud@REDACTED Sat Apr 30 07:59:14 2016 From: dgud@REDACTED (Dan Gudmundsson) Date: Sat, 30 Apr 2016 05:59:14 +0000 Subject: [erlang-questions] erlsrv on windows 10 problem In-Reply-To: References: Message-ID: Well you got it run at all, that is something, I have disabled our testing of services on Windows 10 for now, master branch, since I could not get it run at all. >From what I googled and understood you will need start the services from an elevated shell, an administer user is not enough. But you might already do that or I got it wrong. Anyway, our Windows knowledge, and priority, is not at the highest level. So any help would be appreciated. /Dan On Fri, Apr 29, 2016 at 5:55 PM Wes James wrote: > I use erlsrv to install an erlang application on windows. Windows 7 works > fine, but on windows 10, I have to go in and change the service from > automatic to Automatic (delayed start), otherwise the app never runs on > system start up. Any idea how to fix this? > > I just found this windows command: > > sc config SVCNAME start= delayed-auto > > I can use that in the install script. > > I'd still like it to start with just automatic. With the delayed option, > it takes several extra minutes for the service to start running. > > Thanks, > > -wes > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikpelinux@REDACTED Sat Apr 30 10:40:24 2016 From: mikpelinux@REDACTED (Mikael Pettersson) Date: Sat, 30 Apr 2016 10:40:24 +0200 Subject: [erlang-questions] Erlang for Payment Systems In-Reply-To: References: Message-ID: <22308.28536.28941.52968@gargle.gargle.HOWL> Hassan Sowwan writes: > Hello, > > I am trying to implement payment messaging middleware and would like to explore the option of using Erlang/OTP. > > The application will be used in banking industry to interface with EFT payment switch/networks and core banking system to process card transactions. > > It will be responsible to perform following tasks: > > Communicate with external interfaces using ISO 8583 messaging format ( thru TCP/IP)Receive huge amount of data over the socket ( HEX, BINARY, EBCIDIC), which represents financial transactions.Parse/decode the received data.Perform some checking in database for validationInterface with host security module to validate customer PIN and other security checks.Sends the request to core banking system via XML or web services callRespond back to external interfaces by formulating the response message in ISO 8583 format > Obviously, such applications have to be concurrent and fast enough to process transactions within few seconds. > > Now my question here, is Erlang a good choice for implementing this type of applications ? Erlang works fine in the fintech domain, as long as your system architecture is sound. > Can Erlang handle string processing efficiently without impacting the system performance? With approproate design and implementation, yes. > As stated before, there will be a lot of string manipulation to decode data received over the network, so I am not sure whether erlang fits perfectly or not. For low-level protocol decoding use binaries and binary matching. From erlang@REDACTED Sat Apr 30 10:43:49 2016 From: erlang@REDACTED (Joe Armstrong) Date: Sat, 30 Apr 2016 10:43:49 +0200 Subject: [erlang-questions] Erlang for Payment Systems In-Reply-To: References: Message-ID: On Sat, Apr 30, 2016 at 12:09 AM, Hassan Sowwan wrote: > Hello, > > I am trying to implement payment messaging middleware and would like to > explore the option of using Erlang/OTP. > > The application will be used in banking industry to interface with EFT > payment switch/networks and core banking system to process card > transactions. > > It will be responsible to perform following tasks: > > > Communicate with external interfaces using ISO 8583 messaging format ( thru > TCP/IP) > Receive huge amount of data over the socket ( HEX, BINARY, EBCIDIC), which > represents financial transactions. > Parse/decode the received data. > Perform some checking in database for validation > Interface with host security module to validate customer PIN and other > security checks. > Sends the request to core banking system via XML or web services call > Respond back to external interfaces by formulating the response message in > ISO 8583 format > > > Obviously, such applications have to be concurrent and fast enough to > process transactions within few seconds. > > Now my question here, is Erlang a good choice for implementing this type of > applications ? > Can Erlang handle string processing efficiently without impacting the system > performance? > As stated before, there will be a lot of string manipulation to decode data > received over the network, so I am not sure whether erlang fits perfectly or > not. > I think you've missed the most important criteria for choosing a technique. Fault tolerance - If I was building a financial system #1 of my requirements would be fault tolerance - people will be very cross if money just mysteriously vanishes from the system. #2 would be correctness So my questions would be how can we make a system fault tolerant and how can we ensure correctness. Making things fast is *easy* - making things fault-tolerant is extremely difficult. String manipulation in Erlang is blindingly fast (it's admittedly a lot slower than C) but your program should not be manipulation strings - but trees representing financial objects. /Joe > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From g@REDACTED Sat Apr 30 17:52:00 2016 From: g@REDACTED (Garrett Smith) Date: Sat, 30 Apr 2016 08:52:00 -0700 Subject: [erlang-questions] Erlang for Payment Systems In-Reply-To: References: Message-ID: Decoding data received on the network does not sound like string manipulation - but data decoding. This is one of Erlang's strengths. Once decoded, transforming data and generating string like output (e.g. encoding JSON or XML) can be both very fast and memory efficient - it depends on what you're doing as programmer. There's nothing about Erlang that's particularly weak in these areas - certainly not to ward you off without specifics. I'd say that Erlang's glaring weakness wrt other language options is lack of static typing. However, this comes with the benefit of hugely simplifying distribution. That's a deliberate trade off that's been made in this language. There are ways to mitigate the language's lack of static typing (e.g. see TypEr and Dialyzer). Also consider that there are classes of bugs that cannot be detected by static type checks. And even if you code is bug free (good luck) your code has nothing to say about hardware failure. You need a system that reaches outside your program, which means distribution of some sort. What you're describing is a critical, long running, unattended system. Those are really hard to build and you should put a premium on tools and languages that help you build them successfully (i.e. they actually do what you want at acceptable service levels) at low cost. You'll be hard pressed to find a better language environment along these lines than Erlang. On Fri, Apr 29, 2016 at 3:09 PM, Hassan Sowwan wrote: > Hello, > > I am trying to implement payment messaging middleware and would like to > explore the option of using Erlang/OTP. > > The application will be used in banking industry to interface with EFT > payment switch/networks and core banking system to process card > transactions. > > It will be responsible to perform following tasks: > > > Communicate with external interfaces using ISO 8583 messaging format ( thru > TCP/IP) > Receive huge amount of data over the socket ( HEX, BINARY, EBCIDIC), which > represents financial transactions. > Parse/decode the received data. > Perform some checking in database for validation > Interface with host security module to validate customer PIN and other > security checks. > Sends the request to core banking system via XML or web services call > Respond back to external interfaces by formulating the response message in > ISO 8583 format > > > Obviously, such applications have to be concurrent and fast enough to > process transactions within few seconds. > > Now my question here, is Erlang a good choice for implementing this type of > applications ? > Can Erlang handle string processing efficiently without impacting the system > performance? > As stated before, there will be a lot of string manipulation to decode data > received over the network, so I am not sure whether erlang fits perfectly or > not. > > > > > > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions > From drew.varner@REDACTED Sat Apr 30 20:32:23 2016 From: drew.varner@REDACTED (Drew Varner) Date: Sat, 30 Apr 2016 14:32:23 -0400 Subject: [erlang-questions] Erlang for Payment Systems In-Reply-To: References: Message-ID: <3E7E1782-0203-4F03-BD1E-6F42949E7446@redops.org> Joe's #2 goal of correctness and Garrett's use of success typing (Dialyzer and type specs) naturally point to property-based testing. Erlang QuickCheck (EQC) and other property-based testing frameworks provide additional confidence in the code. Specifically, generators and shrinking strategies work hand-in-hand with Erlang type specs. Basic property-based testing provides a more thoughtful way to test code with solid coverage and the ability to try many edge cases that a developer may have missed in traditional unit tests. Advanced property-based testing with stateful models offers the opportunity to uncover elusive concurrency errors that typically show up in obscure bug reports. If I were playing with money in an Erlang program, I'd leverage these tools. > On Apr 30, 2016, at 11:52 AM, Garrett Smith wrote: > > Decoding data received on the network does not sound like string > manipulation - but data decoding. This is one of Erlang's strengths. > > Once decoded, transforming data and generating string like output > (e.g. encoding JSON or XML) can be both very fast and memory efficient > - it depends on what you're doing as programmer. > > There's nothing about Erlang that's particularly weak in these areas - > certainly not to ward you off without specifics. > > I'd say that Erlang's glaring weakness wrt other language options is > lack of static typing. However, this comes with the benefit of hugely > simplifying distribution. That's a deliberate trade off that's been > made in this language. There are ways to mitigate the language's lack > of static typing (e.g. see TypEr and Dialyzer). > > Also consider that there are classes of bugs that cannot be detected > by static type checks. And even if you code is bug free (good luck) > your code has nothing to say about hardware failure. You need a system > that reaches outside your program, which means distribution of some > sort. > > What you're describing is a critical, long running, unattended system. > Those are really hard to build and you should put a premium on tools > and languages that help you build them successfully (i.e. they > actually do what you want at acceptable service levels) at low cost. > You'll be hard pressed to find a better language environment along > these lines than Erlang. > >> On Fri, Apr 29, 2016 at 3:09 PM, Hassan Sowwan wrote: >> Hello, >> >> I am trying to implement payment messaging middleware and would like to >> explore the option of using Erlang/OTP. >> >> The application will be used in banking industry to interface with EFT >> payment switch/networks and core banking system to process card >> transactions. >> >> It will be responsible to perform following tasks: >> >> >> Communicate with external interfaces using ISO 8583 messaging format ( thru >> TCP/IP) >> Receive huge amount of data over the socket ( HEX, BINARY, EBCIDIC), which >> represents financial transactions. >> Parse/decode the received data. >> Perform some checking in database for validation >> Interface with host security module to validate customer PIN and other >> security checks. >> Sends the request to core banking system via XML or web services call >> Respond back to external interfaces by formulating the response message in >> ISO 8583 format >> >> >> Obviously, such applications have to be concurrent and fast enough to >> process transactions within few seconds. >> >> Now my question here, is Erlang a good choice for implementing this type of >> applications ? >> Can Erlang handle string processing efficiently without impacting the system >> performance? >> As stated before, there will be a lot of string manipulation to decode data >> received over the network, so I am not sure whether erlang fits perfectly or >> not. >> >> >> >> >> >> _______________________________________________ >> erlang-questions mailing list >> erlang-questions@REDACTED >> http://erlang.org/mailman/listinfo/erlang-questions > _______________________________________________ > erlang-questions mailing list > erlang-questions@REDACTED > http://erlang.org/mailman/listinfo/erlang-questions