From co7eb@REDACTED Sat Jan 1 17:14:23 2011 From: co7eb@REDACTED (Gilberto Carmenate =?iso-8859-1?Q?Garc=EDa?=) Date: Sat, 01 Jan 2011 11:14:23 -0500 Subject: About to languages Message-ID: Hi Edmond, well trust me, I know Spanish very well ?I was born speaking Spanish? and compared to English is true that it?s more expressive but for me it?s just a lot of words trying to express what cannot be expressed. English is more magic and we have to use the imagination to believe whatever we want to believe in our own language in our own mind, that?s what makes the magic in a language, that?s why Erlang is so magic too, because we can do almost anything just with a bit of words. I?ve been never talk in English with anybody because I?ve been never had the chance before, But I know someday I will, oh trust me you really don?t what you miss, talking to a girl, expressing your feelings to that girl in that so magic Language, I think she will not resist me lol. Well I have so much to learn just yet but I think I could write a novel some day lol. ======================================================================= Este mensaje ha sido enviado mediante el servicio de correo electr?nico que ofrece la Federaci?n de Radioaficionados de Cuba a sus miembros para respaldar el cumplimiento de los objetivos de la organizaci?n y su pol?tica informativa. La persona que env?a este correo asume el compromiso de usar el servicio a tales fines y cumplir con las regulaciones establecidas. From B.Candler@REDACTED Sat Jan 1 17:34:02 2011 From: B.Candler@REDACTED (Brian Candler) Date: Sat, 1 Jan 2011 16:34:02 +0000 Subject: mnesia memory usage on 32 and 64 bits machines In-Reply-To: <4D1DC27A.80507@tiscali.it> References: <4D1DC27A.80507@tiscali.it> Message-ID: <20110101163402.GA4251@talktalkplc.com> On Fri, Dec 31, 2010 at 12:46:02PM +0100, franz wrote: > I would like to know your opinions about a load test I'm doing, > whose result quite shocked me. > Briefly, I created a mnesia ets table and inserted 50000 records into it. > I tried this bulk load on two machines: the first, say it A, running > a 64bits kernel on a 64 bits machine, the second, say it B, running > a 32bits kernel on a 64 bits machine. > On A the records occupy 202707450 words of mem, and misuring the RAM > consumption before and after the insertion I know that they occupy > 1560Mb. > On B the records occupy 202707473 words of mem, with a RAM > consumption of 795Mb. > On one hand this makes sense, because as far as I know a word of mem > on a 32bits machine is 4 bytes, 8 on a 64bits machine. But does this > mean that having a 64 bits kernel doubles the memory usage?!? When you boot a 32-bit kernel, your processor is running in a backwards-compatibility 32-bit mode, as an i386 processor. The x86_64 mode is essentially a different processor, with different instruction set and register model. And if you boot into DOS, you'll be running 16-bit "real mode" :-) But in summary: booting a 32-bit OS turns your machine into a 32-bit PC. Regards, Brian. From ebegumisa@REDACTED Sat Jan 1 17:41:54 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Sun, 02 Jan 2011 03:41:54 +1100 Subject: [erlang-questions] About to languages In-Reply-To: References: Message-ID: On Sun, 02 Jan 2011 03:14:23 +1100, Gilberto Carmenate Garc?a wrote: > Erlang is so magic too, > because we can do almost anything just with a bit of words Erlang being my first functional language (I've always used imperative languages), I'm beginning to see this. What Paul Graham and others have tried to describe -- the expressiveness and terseness of functional languages. Paul once made a case for learning Lisp (though he kind of discarded it's usefulness in daily programming) by making an analogy with English and Latin... "..This is the same argument you tend to hear for learning Latin... improve your mind, and make you a better writer in languages you do want to use, like English..." http://www.paulgraham.com/avg.html I think Erlang has the same effect, especially improving how one thinks about concurrency and scalable design regardless of whether you are writing C++ or JavaScript. After using Erlang for a little while, you start to think about these things differently. - Edmond - -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From co7eb@REDACTED Sat Jan 1 18:40:12 2011 From: co7eb@REDACTED (Gilberto Carmenate =?iso-8859-1?Q?Garc=EDa?=) Date: Sat, 01 Jan 2011 12:40:12 -0500 Subject: Free web host site??? Message-ID: Hi all, it?s is possible in this world that somebody knows if there are any free web host out there to publish my things?. I cannot do that from here so, I need a little host out there. Regards, Ivan. ======================================================================= Este mensaje ha sido enviado mediante el servicio de correo electr?nico que ofrece la Federaci?n de Radioaficionados de Cuba a sus miembros para respaldar el cumplimiento de los objetivos de la organizaci?n y su pol?tica informativa. La persona que env?a este correo asume el compromiso de usar el servicio a tales fines y cumplir con las regulaciones establecidas. From alain.odea@REDACTED Sat Jan 1 22:35:08 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Sat, 1 Jan 2011 18:05:08 -0330 Subject: [erlang-questions] Free web host site??? In-Reply-To: References: Message-ID: Would http://pages.github.com/ or http://sites.google.com/ meet your needs? On 2011-01-01, at 14:10, "Gilberto Carmenate Garc?a" wrote: > Hi all, it?s is possible in this world that somebody knows if there are > any free web host out there to publish my things?. I cannot do that from > here so, I need a little host out there. > > > Regards, > > > Ivan. > > > ======================================================================= > Este mensaje ha sido enviado mediante el servicio de correo electr?nico que ofrece la Federaci?n de Radioaficionados de Cuba a sus miembros para respaldar el cumplimiento de los objetivos de la organizaci?n y su pol?tica informativa. La persona que env?a este correo asume el compromiso de usar el servicio a tales fines y cumplir con las regulaciones establecidas. > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From ulf.wiger@REDACTED Sun Jan 2 14:08:33 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 2 Jan 2011 14:08:33 +0100 Subject: [erlang-questions] mnesia memory usage on 32 and 64 bits machines In-Reply-To: <4D1DC27A.80507@tiscali.it> References: <4D1DC27A.80507@tiscali.it> Message-ID: <7549C4D5-1AED-4C77-A656-3D26CA4C6048@erlang-solutions.com> On 31 Dec 2010, at 12:46, franz wrote: > But does this mean that having a 64 bits kernel doubles the memory usage?!? As a general rule, running with 64-bit Erlang doubles the memory footprint. The exception is binaries, which use only marginally more space. The OTP team is working on a "half-word" emulator, which uses a 32-bit memory space for the process heaps, but keeps other data (e.g. ETS and binaries) in 64-bit space. This VM is still experimental, but can be built with configure --enable-halfword-emulator BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From koenigjm@REDACTED Sun Jan 2 20:29:21 2011 From: koenigjm@REDACTED (John Koenig) Date: Sun, 2 Jan 2011 13:29:21 -0600 Subject: Decoding Erlang Messages Sent Over TCP With EI Message-ID: I am currently porting a server process (written in C) to ei, which used to interact with an erlang process via the older erl_interface. I am able to get the the two sides talking over a localhost TCP socket, but I am having trouble decoding the tuples sent to the C based server. Tuples are being sent from an interactive erlang session as follows: {any, mynode@REDACTED} ! {tuple1, typle2, []} When the message is received in the C based server via ei_receive, however, ei_decode_tuple_header returns non-zero and I am unable to proceed further. Calling ei_get_type on the buffer results in 'p' which, according to ei.h, corresponds to ERL_NEW_FUN_EXT. Why is this failing and why is this tuple being reported as an ERL_NEW_FUN_EXT? The only example I have seen thus far (http://www.trapexit.org/How_to_use_ei_to_marshal_binary_terms_in_port_programs) uses ei stdin/stdout, can ei be used over a TCP socket at all? Thanks in advance, John From vinoski@REDACTED Sun Jan 2 20:48:41 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Sun, 2 Jan 2011 14:48:41 -0500 Subject: [erlang-questions] Decoding Erlang Messages Sent Over TCP With EI In-Reply-To: References: Message-ID: On Sun, Jan 2, 2011 at 2:29 PM, John Koenig wrote: > I am currently porting a server process (written in C) to ei, which used to interact with an > erlang process via the older erl_interface. ?I am able to get the the two sides talking over a > localhost TCP socket, but I am having trouble decoding the tuples sent to the C based server. > > Tuples are being sent from an interactive erlang session as follows: > > {any, mynode@REDACTED} ! {tuple1, typle2, []} > > When the message is received in the C based server via ei_receive, ?however, > ei_decode_tuple_header returns non-zero and I am unable to proceed further. ?Calling > ei_get_type on the buffer results in 'p' which, according to ei.h, corresponds to > ERL_NEW_FUN_EXT. ?Why is this failing and why is this tuple being reported as an > ERL_NEW_FUN_EXT? > > The only example I have seen thus far > (http://www.trapexit.org/How_to_use_ei_to_marshal_binary_terms_in_port_programs) uses ei > stdin/stdout, can ei be used over a TCP socket at all? Ei most definitely works over TCP. Can you post more of your decoding code? Hard to tell what's going on from what you've told us so far, but p is value 112 and there are a couple instances of that value in the encoded form of your tuple above, so you're likely just decoding at the wrong offset. --steve From koenigjm@REDACTED Sun Jan 2 21:05:42 2011 From: koenigjm@REDACTED (John Koenig) Date: Sun, 2 Jan 2011 14:05:42 -0600 Subject: [erlang-questions] Decoding Erlang Messages Sent Over TCP With EI In-Reply-To: References: Message-ID: <8AD61E29-5A9F-40B7-AC87-DD20A8C7F19A@acalledshot.org> Code is attached. What you have said so far has brought me close of identifying the problem. Up until now I have assumed that ei_decode_tuple_header would scan me ahead to the proper location, but I starting to think I was wrong to do so. Especially after looking at the received data more closely in gdb. How does one know the proper offset from which to start decoding the actual message? John -------------- next part -------------- On Jan 2, 2011, at 1:48 PM, Steve Vinoski wrote: > On Sun, Jan 2, 2011 at 2:29 PM, John Koenig wrote: >> I am currently porting a server process (written in C) to ei, which used to interact with an >> erlang process via the older erl_interface. I am able to get the the two sides talking over a >> localhost TCP socket, but I am having trouble decoding the tuples sent to the C based server. >> >> Tuples are being sent from an interactive erlang session as follows: >> >> {any, mynode@REDACTED} ! {tuple1, typle2, []} >> >> When the message is received in the C based server via ei_receive, however, >> ei_decode_tuple_header returns non-zero and I am unable to proceed further. Calling >> ei_get_type on the buffer results in 'p' which, according to ei.h, corresponds to >> ERL_NEW_FUN_EXT. Why is this failing and why is this tuple being reported as an >> ERL_NEW_FUN_EXT? >> >> The only example I have seen thus far >> (http://www.trapexit.org/How_to_use_ei_to_marshal_binary_terms_in_port_programs) uses ei >> stdin/stdout, can ei be used over a TCP socket at all? > > Ei most definitely works over TCP. Can you post more of your decoding > code? Hard to tell what's going on from what you've told us so far, > but p is value 112 and there are a couple instances of that value in > the encoded form of your tuple above, so you're likely just decoding > at the wrong offset. > > --steve From alexey.v.romanov@REDACTED Sun Jan 2 21:45:14 2011 From: alexey.v.romanov@REDACTED (Alexey Romanov) Date: Sun, 2 Jan 2011 23:45:14 +0300 Subject: [erlang-questions] Decoding Erlang Messages Sent Over TCP With EI In-Reply-To: <8AD61E29-5A9F-40B7-AC87-DD20A8C7F19A@acalledshot.org> References: <8AD61E29-5A9F-40B7-AC87-DD20A8C7F19A@acalledshot.org> Message-ID: Use ei_decode_version first (I ran into the same problem first time I used EI). Yours, Alexey Romanov On Sun, Jan 2, 2011 at 11:05 PM, John Koenig wrote: > Code is attached. > > What you have said so far has brought me close of identifying the problem. ?Up until now I have assumed that ei_decode_tuple_header would scan me ahead to the proper location, but I starting to think I was wrong to do so. ?Especially after looking at the received data more closely in gdb. > > How does one know the proper offset from which to start decoding the actual message? > > John > > > > > > On Jan 2, 2011, at 1:48 PM, Steve Vinoski wrote: > >> On Sun, Jan 2, 2011 at 2:29 PM, John Koenig wrote: >>> I am currently porting a server process (written in C) to ei, which used to interact with an >>> erlang process via the older erl_interface. ?I am able to get the the two sides talking over a >>> localhost TCP socket, but I am having trouble decoding the tuples sent to the C based server. >>> >>> Tuples are being sent from an interactive erlang session as follows: >>> >>> {any, mynode@REDACTED} ! {tuple1, typle2, []} >>> >>> When the message is received in the C based server via ei_receive, ?however, >>> ei_decode_tuple_header returns non-zero and I am unable to proceed further. ?Calling >>> ei_get_type on the buffer results in 'p' which, according to ei.h, corresponds to >>> ERL_NEW_FUN_EXT. ?Why is this failing and why is this tuple being reported as an >>> ERL_NEW_FUN_EXT? >>> >>> The only example I have seen thus far >>> (http://www.trapexit.org/How_to_use_ei_to_marshal_binary_terms_in_port_programs) uses ei >>> stdin/stdout, can ei be used over a TCP socket at all? >> >> Ei most definitely works over TCP. Can you post more of your decoding >> code? Hard to tell what's going on from what you've told us so far, >> but p is value 112 and there are a couple instances of that value in >> the encoded form of your tuple above, so you're likely just decoding >> at the wrong offset. >> >> --steve > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From vinoski@REDACTED Sun Jan 2 21:46:51 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Sun, 2 Jan 2011 15:46:51 -0500 Subject: [erlang-questions] Decoding Erlang Messages Sent Over TCP With EI In-Reply-To: <8AD61E29-5A9F-40B7-AC87-DD20A8C7F19A@acalledshot.org> References: <8AD61E29-5A9F-40B7-AC87-DD20A8C7F19A@acalledshot.org> Message-ID: On Sun, Jan 2, 2011 at 3:05 PM, John Koenig wrote: > Code is attached. > > What you have said so far has brought me close of identifying the problem. ?Up until now I > have assumed that ei_decode_tuple_header would scan me ahead to the proper location, but I > starting to think I was wrong to do so. ?Especially after looking at the received data more > closely in gdb. > > How does one know the proper offset from which to start decoding the actual message? You start at offset 0 and the decode functions update the "index" variable you pass to them. Are you getting a version number on your message? You probably have to call ei_decode_version first to take the version number off the message. After that, I think the rest of your code will work. --steve From koenigjm@REDACTED Sun Jan 2 22:04:57 2011 From: koenigjm@REDACTED (John Koenig) Date: Sun, 2 Jan 2011 15:04:57 -0600 Subject: [erlang-questions] Decoding Erlang Messages Sent Over TCP With EI In-Reply-To: References: <8AD61E29-5A9F-40B7-AC87-DD20A8C7F19A@acalledshot.org> Message-ID: <5FA802A7-337F-4657-95F4-A9DE72EB7D07@acalledshot.org> I have just retried calling ei_decode_version before decode_tuple_header, and it returns -1. Below is the output from gdb when I display the contents of the message buffer: (gdb) p/x *msg@REDACTED $3 = {0x70, 0x83, 0x68, 0x4, 0x61, 0x6, 0x67, 0x64, 0x0, 0xb, 0x65, 0x72, 0x6c, 0x79, 0x40, 0x73, 0x68, 0x61, 0x64, 0x6f, 0x77, 0x0, 0x0, 0x0, 0x25, 0x0, 0x0, 0x0, 0x0, 0x3, 0x64, 0x0, 0x0, 0x64, 0x0, 0x3, 0x61, 0x6e, 0x79, 0x83, 0x68, 0x3, 0x64, 0x0, 0x6, 0x74, 0x75, 0x70, 0x6c, 0x65, 0x30, 0x64, 0x0, 0x6, 0x74, 0x75, 0x70, 0x6c, 0x65, 0x31, 0x6a} (gdb) p *msg@REDACTED $4 = "p?h\004a\006gd\000\verly@REDACTED\000\000\000%\000\000\000\000\003d\000\000d\000\003any?h\003d\000\006tuple0d\000\006tuple1j" I can see portions of the original message in there...so at least I have that going for me... :S John On Jan 2, 2011, at 2:46 PM, Steve Vinoski wrote: > On Sun, Jan 2, 2011 at 3:05 PM, John Koenig wrote: >> Code is attached. >> >> What you have said so far has brought me close of identifying the problem. Up until now I >> have assumed that ei_decode_tuple_header would scan me ahead to the proper location, but I >> starting to think I was wrong to do so. Especially after looking at the received data more >> closely in gdb. >> >> How does one know the proper offset from which to start decoding the actual message? > > You start at offset 0 and the decode functions update the "index" > variable you pass to them. > > Are you getting a version number on your message? You probably have to > call ei_decode_version first to take the version number off the > message. After that, I think the rest of your code will work. > > --steve From michael.santos@REDACTED Sun Jan 2 22:20:39 2011 From: michael.santos@REDACTED (Michael Santos) Date: Sun, 2 Jan 2011 16:20:39 -0500 Subject: question about NIF resources Message-ID: <20110102212039.GA30469@ecn.lan> The documentation is quite explicit about the behaviour of enif_make_resource(): "The term returned by enif_make_resource is totally opaque in nature. It can be stored and passed between processses on the same node, but the only real end usage is to pass it back as argument to a NIF. <...> Other operations such as matching or term_to_binary will have unpredictable (but harmless) results." That's clear enough but I wonder if unpredictable behaviour is really harmless or if I'm confused about how it works. For example, with an NIF wrapping malloc() [1] such as: static int load(ErlNifEnv *env, void **priv_data, ERL_NIF_TERM load_info) { atom_ok = enif_make_atom(env, "ok"); atom_error = enif_make_atom(env, "error"); if ( (MEM_RESOURCE = enif_open_resource_type(env, NULL, "mem_resource", cleanup, ERL_NIF_RT_CREATE, NULL)) == NULL) return -1; return 0; } static ERL_NIF_TERM nif_malloc(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) { size_t size = 0; void **p = NULL; ERL_NIF_TERM res = {0}; if (!enif_get_ulong(env, argv[0], (ulong *)&size)) return enif_make_badarg(env); p = enif_alloc_resource(MEM_RESOURCE, sizeof(char *)); if (p == NULL) return atom_error; *p = (char *)malloc(size); if (*p == NULL) { enif_release_resource(p); return atom_error; } (void)fprintf(stderr, "alloc: p=%p/%p\n", *p, p); res = enif_make_resource(env, p); enif_release_resource(p); return enif_make_tuple(env, 2, atom_ok, res); } static ERL_NIF_TERM nif_free(ErlNifEnv *env, int argc, const ERL_NIF_TERM argv[]) { void **p = NULL; if (!enif_get_resource(env, argv[0], MEM_RESOURCE, (void **)&p)) return enif_make_badarg(env); (void)fprintf(stderr, "free: p=%p/%p\n", *p, p); free(*p); *p = NULL; return atom_ok; } Running it: Erlang R14B02 (erts-5.8.3) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.3 (abort with ^G) 1> {ok,P} = mem:alloc(10). alloc: p=0x92caa30/0x925ed58 {ok,<<>>} 2> {ok,P1} = mem:alloc(100). alloc: p=0x92cb008/0x925ecd0 {ok,<<>>} 3> P = P1. <<>> 4> {ok, P} = mem:alloc(20). alloc: p=0x92caa40/0x9263560 {ok,<<>>} I would expect P /= P1 and a second invocation of mem:alloc/1 with a bound variable to fail. As it is, the NIF will leak resources: 5> mem:free(P). % P really is bound during the first operation free: p=0x92caa30/0x925ed58 ok 6> mem:free(P). free: p=(nil)/0x925ed58 Presumably the memory will be freed when the process dies and the dtor runs. But there is pattern matching behaviour, the caller just never knows what to expect since the behaviour is undefined. At least at this time erl_make_resource returns something that looks like an empty binary: 1> spawn(fun() -> {ok, foo} = mem:alloc(10) end). alloc: p=0xb53055a0/0xb4cc7270 cleanup: *p=0xb53055a0/0xb4cc7270 =ERROR REPORT==== 2-Jan-2011::15:29:56 === Error in process <0.38.0> with exit value: {{badmatch,{ok,<<0 bytes>>}},[{erl_eval,expr,3}]} <0.38.0> I worked around this behaviour by returning a tagged tuple containing the resource and a reference. But would it make sense for erl_make_resource() to return a defined type with some unique value (even if it's thrown away)? [1] https://gist.github.com/762795 https://gist.github.com/762796 From mjtruog@REDACTED Sun Jan 2 22:31:56 2011 From: mjtruog@REDACTED (Michael Truog) Date: Sun, 2 Jan 2011 13:31:56 -0800 Subject: [erlang-questions] Trie Data Structure In-Reply-To: References: Message-ID: Actually, use the benchmark code here: https://github.com/okeuday/erlbench Blog Post: http://okeuday.livejournal.com/16707.html The trie fetch performance is roughly 15% faster than a dict for 10000 entries, but might get better as the number of entries increases (though the performance depends on the distribution of the key size, i.e., string length). On Thu, Dec 30, 2010 at 6:54 AM, Michael Truog wrote: > The trie:test_speed/0 function is what I used for determining how fast the > trie was for lookups, when compared to a dict. The wordlist used comes from > the Ubuntu 10.04 installation and is located at /usr/share/dict/words (98569 > words with an average of 9.5 characters per word). You can also specify > your own wordlist with test_speed/1. The comparison stores all the words > and does lookups on all the words (using is_key, fetch, and find) while > tracking the time taken to do all the lookups. > > > On Wed, Dec 29, 2010 at 9:04 PM, Ryan Zezeski wrote: > >> On Wed, Dec 29, 2010 at 9:21 PM, Michael Truog wrote: >> >>> Hi, >>> >>> I implemented a trie data structure that has performed 20% faster lookups >>> than the dict implementation in R14B01. I have tested the code but it >>> can >>> always benefit from more testing. The trie module provides the same >>> interface as the dict module, but requires a list of integers for the >>> key. >>> Please tell me if you find issues in the code. >>> >>> Code: >>> >>> https://github.com/okeuday/CloudI/blob/master/src/lib/cloud_stdlib/src/trie.erl >>> >>> Blog posts: >>> http://okeuday.livejournal.com/16454.html >>> http://okeuday.livejournal.com/16182.html (dict as the fastest general >>> tree >>> data structure) >>> >>> I know you can use ETS for similar problems, but avoiding shared data >>> helps >>> utilize Erlang's scalability advantages. >>> >>> Thanks, >>> Michael Truog >>> >> >> Do you have the benchmark and dataset available as well? I wrote a trie >> in Erlang a few weeks ago and I'm just curious how it stacks up. >> >> -- >> *.........01010...00..00...0100...00..00. >> .........10..11...0100...11..00..101.01. >> .111010..01100.....10....101110..01.101. >> .........10..10....11....01..10..10..01. >> .........00..00....00....00..00..00..00. >> ........................................* >> > > From vinoski@REDACTED Mon Jan 3 01:27:04 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Sun, 2 Jan 2011 19:27:04 -0500 Subject: [erlang-questions] Decoding Erlang Messages Sent Over TCP With EI In-Reply-To: <5FA802A7-337F-4657-95F4-A9DE72EB7D07@acalledshot.org> References: <8AD61E29-5A9F-40B7-AC87-DD20A8C7F19A@acalledshot.org> <5FA802A7-337F-4657-95F4-A9DE72EB7D07@acalledshot.org> Message-ID: On Sun, Jan 2, 2011 at 4:04 PM, John Koenig wrote: > I have just retried calling ei_decode_version before decode_tuple_header, and it returns -1. > > Below is the output from gdb when I display the contents of the message buffer: > > (gdb) p/x *msg@REDACTED > $3 = {0x70, 0x83, 0x68, 0x4, 0x61, 0x6, 0x67, 0x64, 0x0, 0xb, 0x65, 0x72, 0x6c, 0x79, 0x40, > 0x73, 0x68, 0x61, 0x64, 0x6f, 0x77, 0x0, 0x0, 0x0, 0x25, 0x0, 0x0, 0x0, 0x0, 0x3, 0x64, 0x0, > 0x0, 0x64, 0x0, 0x3, 0x61, 0x6e, 0x79, 0x83, 0x68, 0x3, 0x64, 0x0, 0x6, 0x74, 0x75, 0x70, > 0x6c, 0x65, 0x30, 0x64, 0x0, 0x6, 0x74, 0x75, 0x70, 0x6c, 0x65, 0x31, 0x6a} I believe the 'p' (value 0x70) you see here is the ERL_PASS_THROUGH indicator, which I think indicates that the message contains both addressing information metadata for the recipient along with the message being sent to that recipient. This is what we see in your data above: 0x70: ERL_PASS_THROUGH 0x83: ERL_VERSION_MAGIC (version number) tuple header showing a 4-tuple the value 6, indicating an ERL_REG_SEND message the pid of the sender an atom for the cookie (empty here) the atom 'any' 0x83 ERL_VERSION_MAGIC (version number) tuple header showing a 3-tuple the atom 'tuple0' the atom 'tuple1' 0x6A: the nil indicator (for the empty list) Instead of having to decode all this yourself, just use ei_receive_msg or ei_xreceive_msg to retrieve your messages, as these functions will first decode the addressing info and use it to populate an erlang_msg struct, and then you can use your existing code to decode the message itself. The ei_x_buff struct you pass to these functions contains a pointer to the encoded buffer and an index member that the functions will fill in to tell you where to start decoding. --steve From ebegumisa@REDACTED Mon Jan 3 02:36:27 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Mon, 03 Jan 2011 12:36:27 +1100 Subject: Topic Relevance Message-ID: Hello all, Is there a particular wep-page, previous discussion on what topics/posts are relevant/irrelevant for this list? Gmane lists the group as "Erlang functional programming language and its environment general." This is nice and wide. But what can someone use to *politely* point to when a thread strays far too off topic (if there is such a thing) while trying to bring it back on topic? - Edmond - -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From ebegumisa@REDACTED Mon Jan 3 02:38:38 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Mon, 03 Jan 2011 12:38:38 +1100 Subject: Processes & Fault Tolerance Message-ID: Hello all, I've been trying to wrap my Erlang's fault tolerant features particularly in relation to processes. I've heard/read repeatedly that the primary reason why Erlang's designers opted for a share-nothing policy is not rooted in concurrency but rather in fault-tolerance. When nothing is shared, everything is copied. When everything is copied processes can take over from one another when things fail. I follow this reasoning but I don't follow how to apply it. I fully understand and appreciate how supervision trees are used to restart processes if they fail. What I don't get is what to do when you don't want to restart but want to take over, say on another node. I know that at a higher-level, OTP has some take-over/fail-over schematics (at the application level.) I'm trying to understand things at the processes level - why Erlang is the way it is so I can better use it to make my currently fault-intolerant program fault tolerant. Specifically, how can one process take over from another if it fails? It appears to may that the only way to do this would be to somehow retrieve not only the state of the process (say, gen_server's state) but also the messages in its mailbox. Where does the design decision to share-nothing for the sake of fault-tolerance come into play for processes? Please help me "get" this! Thanks in advance. - Edmond - -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From robert.virding@REDACTED Mon Jan 3 02:58:16 2011 From: robert.virding@REDACTED (Robert Virding) Date: Mon, 3 Jan 2011 01:58:16 +0000 (GMT) Subject: [erlang-questions] Trie Data Structure In-Reply-To: <383354494.238061294019859717.JavaMail.root@zimbra> Message-ID: <1051035310.238081294019896519.JavaMail.root@zimbra> It is a little unfair comparison you have done between a trie and dict/gb_trees/... as the trie is designed for a specific type of key, a list of integers/characters, while the others can have keys which can be any valid term. So the trie *should* be more efficient for this case. That being said this definitely shows the need of having more good implementations of different data structures like the trie. Improving our toolbox is never wrong. Robert ----- "Michael Truog" wrote: > Actually, use the benchmark code here: > https://github.com/okeuday/erlbench > > Blog Post: > http://okeuday.livejournal.com/16707.html > > The trie fetch performance is roughly 15% faster than a dict for > 10000 > entries, but might get better as the number of entries increases > (though the > performance depends on the distribution of the key size, i.e., string > length). > > On Thu, Dec 30, 2010 at 6:54 AM, Michael Truog > wrote: > > > The trie:test_speed/0 function is what I used for determining how > fast the > > trie was for lookups, when compared to a dict. The wordlist used > comes from > > the Ubuntu 10.04 installation and is located at > /usr/share/dict/words (98569 > > words with an average of 9.5 characters per word). You can also > specify > > your own wordlist with test_speed/1. The comparison stores all the > words > > and does lookups on all the words (using is_key, fetch, and find) > while > > tracking the time taken to do all the lookups. > > > > > > On Wed, Dec 29, 2010 at 9:04 PM, Ryan Zezeski > wrote: > > > >> On Wed, Dec 29, 2010 at 9:21 PM, Michael Truog > wrote: > >> > >>> Hi, > >>> > >>> I implemented a trie data structure that has performed 20% faster > lookups > >>> than the dict implementation in R14B01. I have tested the code > but it > >>> can > >>> always benefit from more testing. The trie module provides the > same > >>> interface as the dict module, but requires a list of integers for > the > >>> key. > >>> Please tell me if you find issues in the code. > >>> > >>> Code: > >>> > >>> > https://github.com/okeuday/CloudI/blob/master/src/lib/cloud_stdlib/src/trie.erl > >>> > >>> Blog posts: > >>> http://okeuday.livejournal.com/16454.html > >>> http://okeuday.livejournal.com/16182.html (dict as the fastest > general > >>> tree > >>> data structure) > >>> > >>> I know you can use ETS for similar problems, but avoiding shared > data > >>> helps > >>> utilize Erlang's scalability advantages. > >>> > >>> Thanks, > >>> Michael Truog > >>> > >> > >> Do you have the benchmark and dataset available as well? I wrote a > trie > >> in Erlang a few weeks ago and I'm just curious how it stacks up. > >> > >> -- > >> *.........01010...00..00...0100...00..00. > >> .........10..11...0100...11..00..101.01. > >> .111010..01100.....10....101110..01.101. > >> .........10..10....11....01..10..10..01. > >> .........00..00....00....00..00..00..00. > >> ........................................* > >> > > > > From ebegumisa@REDACTED Mon Jan 3 03:06:47 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Mon, 03 Jan 2011 13:06:47 +1100 Subject: [erlang-questions] Processes & Fault Tolerance In-Reply-To: References: Message-ID: Slight correction... On Mon, 03 Jan 2011 12:38:38 +1100, Edmond Begumisa wrote: > Hello all, > > I've been trying to wrap my Erlang's fault tolerant features > particularly in relation to processes. > Should be: I've been trying to wrap my head around Erlang's fault tolerant features particularly in relation to processes. Sorry. > I've heard/read repeatedly that the primary reason why Erlang's > designers opted for a share-nothing policy is not rooted in concurrency > but rather in fault-tolerance. When nothing is shared, everything is > copied. When everything is copied processes can take over from one > another when things fail. I follow this reasoning but I don't follow how > to apply it. > > I fully understand and appreciate how supervision trees are used to > restart processes if they fail. What I don't get is what to do when you > don't want to restart but want to take over, say on another node. I know > that at a higher-level, OTP has some take-over/fail-over schematics (at > the application level.) I'm trying to understand things at the processes > level - why Erlang is the way it is so I can better use it to make my > currently fault-intolerant program fault tolerant. > > Specifically, how can one process take over from another if it fails? It > appears to may that the only way to do this would be to somehow retrieve > not only the state of the process (say, gen_server's state) but also the > messages in its mailbox. Where does the design decision to share-nothing > for the sake of fault-tolerance come into play for processes? Please > help me "get" this! > > Thanks in advance. > > - Edmond - > > > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From alain.odea@REDACTED Mon Jan 3 04:56:51 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Mon, 3 Jan 2011 00:26:51 -0330 Subject: [erlang-questions] Processes & Fault Tolerance In-Reply-To: References: Message-ID: On 2011-01-02, at 22:36, "Edmond Begumisa" wrote: > Slight correction... > > On Mon, 03 Jan 2011 12:38:38 +1100, Edmond Begumisa wrote: > >> Hello all, >> >> I've been trying to wrap my Erlang's fault tolerant features particularly in relation to processes. >> > > Should be: I've been trying to wrap my head around Erlang's fault tolerant features particularly in relation to processes. > > Sorry. > >> I've heard/read repeatedly that the primary reason why Erlang's designers opted for a share-nothing policy is not rooted in concurrency but rather in fault-tolerance. When nothing is shared, everything is copied. When everything is copied processes can take over from one another when things fail. I follow this reasoning but I don't follow how to apply it. >> >> I fully understand and appreciate how supervision trees are used to restart processes if they fail. What I don't get is what to do when you don't want to restart but want to take over, say on another node. I know that at a higher-level, OTP has some take-over/fail-over schematics (at the application level.) I'm trying to understand things at the processes level - why Erlang is the way it is so I can better use it to make my currently fault-intolerant program fault tolerant. >> >> Specifically, how can one process take over from another if it fails? It appears to may that the only way to do this would be to somehow retrieve not only the state of the process (say, gen_server's state) but also the messages in its mailbox. Where does the design decision to share-nothing for the sake of fault-tolerance come into play for processes? Please help me "get" this! >> >> Thanks in advance. >> >> - Edmond - >> >> Hi Edmond: Share-nothing helps with concurrent fault-tolerance by preventing one process from corrupting the state of another. Receive is a process' choice and it corrupts its own state if it receives bad data and lets it in. AFAIK OTP fault-tolerance doesn't mean no requests will fail, it means the system/sub-system will recover if a single request causes a process to crash. It's kind of like proper try/catch recovery applied to concurrent code. How you recover from the crash depends on the supervision strategy chosen. In some cases the supervisor can pass the state to the replacement process. In others this isn't necessary or even desirable since the state itself may involve resources lost in the crash or corrupted state that led to the crash. I am straying outside my knowledge here so this paragraph is guesswork. The message queue for a gen_server need not necessarily be lost when the callback module crashes. In theory OTP could (and might already) simply delegate the messages to the replacement process following a crash. Someone who knows OTP better than me would need to weigh in here though. I found http://manning.com/logan very informative in understanding OTP and its supervisor hierarchies. Cheers, Alain From ebegumisa@REDACTED Mon Jan 3 06:21:07 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Mon, 03 Jan 2011 16:21:07 +1100 Subject: [erlang-questions] Processes & Fault Tolerance In-Reply-To: References: Message-ID: Thanks for your response. Firstly, let me make my question a little clearer... To rephrase: For processes, "share nothing for the sake of concurrency" - I get, both in concept and application. "Share nothing for the sake of fault-tolerance" - I get in concept but not in application. Yet as I understand it, it is for the latter reason Erlang shares nothing* and not the former. Interpretation: I must be completely missing the point in regards to Erlang processes and sharing nothing. This is what I want to understand in application. In addition to the "side" effect of sane concurrency (which coming from a chaotic multi-threading shared-memory world I fully appreciate and practically make use of everyday), how can I also make use of the "real" reason Erlang processes share nothing -- fault tolerance? Practically/illustratively speaking? *ETS being the obvious exception. Secondly, here's a mantra from Joe Armstrong... @ minute 17:26 http://www.se-radio.net/2008/03/episode-89-joe-armstrong-on-erlang/ "[message passing concurrency]... the original reasons have to do with fault tolerance... you have to copy all the data you need from computer 1 to computer 2... if computer 1 crashes you take over on computer 2... you can't have dangling pointers... that's the reason for copying everything... it's got nothing to do with concurrency, it's got a lot to do with fault-tolerance... if they don't crash you could just have a dangling pointer and copy less data but it won't work in the presence of errors..." I interpret this to mean that share-nothing between processes is more about replicating valid state than isolating corrupted state as you described. Indeed, Joe created an example on his blog... http://armstrongonsoftware.blogspot.com/2007/07/scalable-fault-tolerant-upgradable.html It's algorithm 3 there I'm struggling with. Particularly where he says... "... In practise we would send an asynchronous stream of messages from N to N+1 containing enough information to recover if things go wrong." Unfortunately, I couldn't find part II to that post (I don't think there is one.) And I'm too green and inexperienced in the field of fault-tolerant systems to figure it out on my own. I'm having trouble visualising the practical here from the conceptual -- I need to be shown how :( Also, I seem to be under the impression that the Erlang language has some sort of schematics to do this built-in (i.e. deal with one process taking over from another if the first fails) and this is the reason processes share nothing. This seems to me to be something different from supervision trees, which use exit-trapping to re-spawn if a process fails with the active 'job' disappearing and any errors logged (like restarting a daemon). My interpretation of the fault-tolerance Erlang is supposed to enable (for those in the know) is seamless take-over. The 'job' lives on but elsewhere. Using telecoms as an example: a phone call wouldn't be cut-off when a fault occurs, another node would seamlessly take over. This is how I interpreted Joe's post and other descriptions of Erlang's fault-tolerant features and I understand the key is in the share nothing policy for processes. I'm sure I've mis-understood something or everything :) - Edmond - PS: I've read the Manning draft. Great book. I don't know if the answer lies in OTP (I searched and didn't find it). I suspect it's lower -- probalby how you organise your processes. Some distributed-programming black-magic only Erlanger's know about :) On Mon, 03 Jan 2011 14:56:51 +1100, Alain O'Dea wrote: > On 2011-01-02, at 22:36, "Edmond Begumisa" > wrote: > >> Slight correction... >> >> On Mon, 03 Jan 2011 12:38:38 +1100, Edmond Begumisa >> wrote: >> >>> Hello all, >>> >>> I've been trying to wrap my Erlang's fault tolerant features >>> particularly in relation to processes. >>> >> >> Should be: I've been trying to wrap my head around Erlang's fault >> tolerant features particularly in relation to processes. >> >> Sorry. >> >>> I've heard/read repeatedly that the primary reason why Erlang's >>> designers opted for a share-nothing policy is not rooted in >>> concurrency but rather in fault-tolerance. When nothing is shared, >>> everything is copied. When everything is copied processes can take >>> over from one another when things fail. I follow this reasoning but I >>> don't follow how to apply it. >>> >>> I fully understand and appreciate how supervision trees are used to >>> restart processes if they fail. What I don't get is what to do when >>> you don't want to restart but want to take over, say on another node. >>> I know that at a higher-level, OTP has some take-over/fail-over >>> schematics (at the application level.) I'm trying to understand things >>> at the processes level - why Erlang is the way it is so I can better >>> use it to make my currently fault-intolerant program fault tolerant. >>> >>> Specifically, how can one process take over from another if it fails? >>> It appears to may that the only way to do this would be to somehow >>> retrieve not only the state of the process (say, gen_server's state) >>> but also the messages in its mailbox. Where does the design decision >>> to share-nothing for the sake of fault-tolerance come into play for >>> processes? Please help me "get" this! >>> >>> Thanks in advance. >>> >>> - Edmond - >>> >>> > > Hi Edmond: > > Share-nothing helps with concurrent fault-tolerance by preventing one > process from corrupting the state of another. Receive is a process' > choice and it corrupts its own state if it receives bad data and lets it > in. > > AFAIK OTP fault-tolerance doesn't mean no requests will fail, it means > the system/sub-system will recover if a single request causes a process > to crash. It's kind of like proper try/catch recovery applied to > concurrent code. How you recover from the crash depends on the > supervision strategy chosen. In some cases the supervisor can pass the > state to the replacement process. In others this isn't necessary or even > desirable since the state itself may involve resources lost in the crash > or corrupted state that led to the crash. > > I am straying outside my knowledge here so this paragraph is guesswork. > The message queue for a gen_server need not necessarily be lost when the > callback module crashes. In theory OTP could (and might already) simply > delegate the messages to the replacement process following a crash. > Someone who knows OTP better than me would need to weigh in here though. > > I found http://manning.com/logan very informative in understanding OTP > and its supervisor hierarchies. > > Cheers, > Alain > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From ulf.wiger@REDACTED Mon Jan 3 09:35:25 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 3 Jan 2011 09:35:25 +0100 Subject: [erlang-questions] Processes & Fault Tolerance In-Reply-To: References: Message-ID: On 3 Jan 2011, at 02:38, Edmond Begumisa wrote: > I fully understand and appreciate how supervision trees are used to restart processes if they fail. What I don't get is what to do when you don't want to restart but want to take over, say on another node. I know that at a higher-level, OTP has some take-over/fail-over schematics (at the application level.) I'm trying to understand things at the processes level - why Erlang is the way it is so I can better use it to make my currently fault-intolerant program fault tolerant. > > Specifically, how can one process take over from another if it fails? It appears to may that the only way to do this would be to somehow retrieve not only the state of the process (say, gen_server's state) but also the messages in its mailbox. Where does the design decision to share-nothing for the sake of fault-tolerance come into play for processes? Please help me "get" this! To start with the simpler form of redundancy - "cold standby" - it can be achieved by grouping your code into OTP applications, and configuring the Distributed Application Controller (dist_ac), so that it will run an application on one node (e.g. A), if available, and otherwise start it on B. The application started on B will be given a hint in the first argument of the start/2 function: Type = {failover, A}, indicating the reason why it is being started. The application can then recover state in the initialization function. (I made a simple example of this in http://www.trapexit.org/OTP_Release_Handling_Tutorial I actually developed this example further for a tutorial in Israel, but my Subversion repository became corrupt and I lost most of it before I could put it on line. It involved adding mnesia replication, and I had a more-or-less working riak alternative too. :) The "recover state" part requires some forethought, as, obviously, the original instance of the application is no more. For "less hot" versions of standby, one can use some form of "stable-state replication", e.g. by storing data in a replicated mnesia table, in riak, memBase, etc. There is no single answer for how to do this. It depends on the robustness requirements and dynamics of your application. You have to decide what types of errors you need to handle transparently to the user, and what type of recovery action it is reasonable to expect of the user itself. To tie back to telecoms, the basic service that Erlang was designed for is the telephone call. In this case, we (used to) rely on the phone service being extremely reliable - it practically never happened that we picked up the phone and didn't get dial tone; when it did, usually, if we tried again, we *would* get dial tone. Also, if an ongoing call suddenly disconnected, we would try again, and usually succeed. As long as this didn't occur too often, we wouldn't think twice about it. With mobile telephony, it is a much more common event (in fairness, mobile telephony is *much* more complex), but we accept it, since the service gives us much greater freedom. From a programming point of view then, recovering a phone call would involve resetting it to its most stable state (on hook) and preparing for the event that the user may try again. In the AXD 301, we would take it a bit further and replicate the "connected" state to a standby node. This was done using erlang message passing, with an intermediary that aggregated lots of call states together in order to reduce cost. You could view this as a form of "best-effort hot standby", where it wasn't really necessary to know if each state made it to the standby; we might lose some in the transition. We used a 'standby' application on the standby node, which collected the call states and prepared the data structures (exactly how this was done changed over time). Then we used the failover mechanism above to fire up the call handling application on the other side. Since it knew it was a failover, it also knew to handshake with the standby app to get the data. It then had a few seconds to get everything in order and respond appropriately to the status enquiry messages from the clients, which were built into the protocol. The actual media stream was handled by different hardware, so it was necessary to audit all known calls with that hardware layer. All calls that were not known to the Erlang side were simply reset. We had some true hot-standby problems as well. In some cases, Erlang was involved, but depending on the problem, we had to resort to hardware-based solutions for some, and in at least one case, I remember that putting a cheap, passive (i.e. reliable) splitter in front of the system was the simplest way to "replicate" the signal, making hot standby possible. The hottest of them all was that we could fail over from one ATM switching plane to another, losing at most 6 ATM cells. No Erlang there... To wrap this up, Erlang provides a number of different ways to detect that it's time to take over. I find that the OTP application concept offers a very clean way - with the start({failover,FromNode}, Args) function - allowing you to contain the logic in one place. For some requirements, this may not be sufficiently "hot", in which case you may need to roll your own mechanism. The trick, as you noted, is to get hold of the data needed to take over. Erlang doesn't solve this automatically; you have to plan your data storage, state replication, etc. to make it possible. In my experience, you should really think about what you can get away with. Users normally accept outages, if they are sufficiently rare and short, and the user doesn't lose money or other precious assets. Don't knock the simple solutions because it's sexy to be "infinitely robust" - most likely, the added complexity will make your system *less* robust - not more so. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From ulf.wiger@REDACTED Mon Jan 3 09:38:53 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 3 Jan 2011 09:38:53 +0100 Subject: [erlang-questions] Trie Data Structure In-Reply-To: <1051035310.238081294019896519.JavaMail.root@zimbra> References: <1051035310.238081294019896519.JavaMail.root@zimbra> Message-ID: Still, this is an important point. It is easy to always go for the ultimately type-agnostic solution with Erlang, but there is of course a cost involved. If performance is sufficiently important, it is good to have alternatives that limit your options in exchange for speed. BR, Ulf W On 3 Jan 2011, at 02:58, Robert Virding wrote: > It is a little unfair comparison you have done between a trie and dict/gb_trees/... as the trie is designed for a specific type of key, a list of integers/characters, while the others can have keys which can be any valid term. So the trie *should* be more efficient for this case. That being said this definitely shows the need of having more good implementations of different data structures like the trie. Improving our toolbox is never wrong. > > Robert > > ----- "Michael Truog" wrote: > >> Actually, use the benchmark code here: >> https://github.com/okeuday/erlbench >> >> Blog Post: >> http://okeuday.livejournal.com/16707.html >> >> The trie fetch performance is roughly 15% faster than a dict for >> 10000 >> entries, but might get better as the number of entries increases >> (though the >> performance depends on the distribution of the key size, i.e., string >> length). >> >> On Thu, Dec 30, 2010 at 6:54 AM, Michael Truog >> wrote: >> >>> The trie:test_speed/0 function is what I used for determining how >> fast the >>> trie was for lookups, when compared to a dict. The wordlist used >> comes from >>> the Ubuntu 10.04 installation and is located at >> /usr/share/dict/words (98569 >>> words with an average of 9.5 characters per word). You can also >> specify >>> your own wordlist with test_speed/1. The comparison stores all the >> words >>> and does lookups on all the words (using is_key, fetch, and find) >> while >>> tracking the time taken to do all the lookups. >>> >>> >>> On Wed, Dec 29, 2010 at 9:04 PM, Ryan Zezeski >> wrote: >>> >>>> On Wed, Dec 29, 2010 at 9:21 PM, Michael Truog >> wrote: >>>> >>>>> Hi, >>>>> >>>>> I implemented a trie data structure that has performed 20% faster >> lookups >>>>> than the dict implementation in R14B01. I have tested the code >> but it >>>>> can >>>>> always benefit from more testing. The trie module provides the >> same >>>>> interface as the dict module, but requires a list of integers for >> the >>>>> key. >>>>> Please tell me if you find issues in the code. >>>>> >>>>> Code: >>>>> >>>>> >> https://github.com/okeuday/CloudI/blob/master/src/lib/cloud_stdlib/src/trie.erl >>>>> >>>>> Blog posts: >>>>> http://okeuday.livejournal.com/16454.html >>>>> http://okeuday.livejournal.com/16182.html (dict as the fastest >> general >>>>> tree >>>>> data structure) >>>>> >>>>> I know you can use ETS for similar problems, but avoiding shared >> data >>>>> helps >>>>> utilize Erlang's scalability advantages. >>>>> >>>>> Thanks, >>>>> Michael Truog >>>>> >>>> >>>> Do you have the benchmark and dataset available as well? I wrote a >> trie >>>> in Erlang a few weeks ago and I'm just curious how it stacks up. >>>> >>>> -- >>>> *.........01010...00..00...0100...00..00. >>>> .........10..11...0100...11..00..101.01. >>>> .111010..01100.....10....101110..01.101. >>>> .........10..10....11....01..10..10..01. >>>> .........00..00....00....00..00..00..00. >>>> ........................................* >>>> >>> >>> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From ulf.wiger@REDACTED Mon Jan 3 10:08:11 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 3 Jan 2011 10:08:11 +0100 Subject: common setup utility and call for convention Message-ID: <65FA67E0-D8E4-486D-82BA-4CACD403F57E@erlang-solutions.com> Since some people were asking on twitter, I put up some code that you could say is an evolution of my old 'builder' project (found in Jungerl, and used e.g. by CEAN), and also of the concepts used in AXD 301 for creating a modular and extensible install script. http://github.com/esl/setup For quick perusal: https://github.com/esl/setup/blob/master/README.md https://github.com/esl/setup/blob/master/doc/README.md The app contains only two modules - one (setup_gen) for building .rel file and OTP boot script from a simple config, and one (setup) for implementing a pluggable installation utility, for initializing databases, creating tables, etc. I think they're pretty useful, even at this stage (setup.erl is ca 200 LOC, setup_gen.erl is ca 400 LOC). I will freely admit that setup_gen is a quick-and-dirty reaction to not having the time to figure out how to make reltool do something really simple (well, rebar *almost* does that for you, but for me, it was easier to write setup_gen, since I've done it so many times before). The setup module proposes a convention for specifying install hooks, and I think it would be very helpful if we could start using a common method. In short, if you don't want to follow the above links, you'd add an environment variable to your app: {'$setup_hooks', [ {PhaseNumber, [{M,F,A}]} ]} and the 'setup' utility will collect all hooks, sort them and call them sequentially. PhaseNumber should be an integer, and my initial suggestion is to use 100 for creating your database, 200 for modifying its schema and 300 for filling it with data. There may be other ubiquitous phases, but it's probably a good idea to keep that list short. I'd really appreciate comments. Perhaps this has broader utility than my esoteric experiments extending mnesia? :) BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From mazen.harake@REDACTED Mon Jan 3 11:26:23 2011 From: mazen.harake@REDACTED (Mazen Harake) Date: Mon, 03 Jan 2011 11:26:23 +0100 Subject: [erlang-questions] Processes & Fault Tolerance In-Reply-To: References: Message-ID: <4D21A44F.8010509@erlang-solutions.com> Don't think of it as "taking over" but more about "recovering from" and "minimizing affect area". If process A dies you are not expected to "save" the state of A and _transfer_ it to B and thus continue as if B was A. This doesn't make any sense. The idea is rather that if A crashes then you spawn a new process B which a) has unaffected any other process (say C and D) because there is no data corruption and b) when it starts it might read in a persistent state (say from an ETS table) which it uses as a base to continue an operation. This doesn't mean that you transferred the state of A to B, you recovered from A by starting B. There is a difference between types of state as well, E.g. a gen_server state is not necessarily important to store in a resilient matter but perhaps the id for the active session is because it can be used to recover and repopulate the gen_server state. Some examples: * If you have an ongoing call with someone and there is a bug that disconnects your call then someone else who is having a call should not be affected. The fault tolerance part here is not that your call will continue, it is that you are able to pick up the dial again and call the user back I.e. the system is still alive even though it suffered a minor glitch in a particular process. (There are ways to keep the call alive but I'm not sure they do that). * If you have say an IM session running (ejabberd) then you might have a process per request/message/whatever. In this case perhaps your id, session id and some other key data would be stored and shared on several mnesia nodes but it isn't shared in the sense that it is used by many processes, it is just persistent. This means that if a process which is in the middle of something that has to do with that state crashes then another (newly spawned process) can read the various keys and ids and repopulate the state and continue. Share nothing in both cases mean that where ever the data is, it is only used by one process at a time. The first example might not have persistent data that it keeps but it doesn't affect any other part of the system if the call goes down. The second example has persistent data but it doesn't mean it is shared among processes it just means it handles node disruptions so that new ones can continue. Now if you move on to Node level then these two scenarios still apply but with small differences. The first example will cut off all calls routed through that node but as soon as some would try to call again it would simply go through another node. The second example would distribute the session state so that if a node goes down a new process on another node can handle the continuation of the session. In the later case it is important to realize that the shared data (the state between the nodes) is an obvious bottle neck but it is another type of shared data because it is, through abstraction, only manipulated by 1 process. Makes sense? %-) On 03/01/2011 06:21, Edmond Begumisa wrote: > Thanks for your response. > > Firstly, let me make my question a little clearer... > > To rephrase: For processes, "share nothing for the sake of > concurrency" - I get, both in concept and application. "Share nothing > for the sake of fault-tolerance" - I get in concept but not in > application. > > Yet as I understand it, it is for the latter reason Erlang shares > nothing* and not the former. Interpretation: I must be completely > missing the point in regards to Erlang processes and sharing nothing. > This is what I want to understand in application. In addition to the > "side" effect of sane concurrency (which coming from a chaotic > multi-threading shared-memory world I fully appreciate and practically > make use of everyday), how can I also make use of the "real" reason > Erlang processes share nothing -- fault tolerance? > Practically/illustratively speaking? > > *ETS being the obvious exception. > > Secondly, here's a mantra from Joe Armstrong... > > @ minute 17:26 > http://www.se-radio.net/2008/03/episode-89-joe-armstrong-on-erlang/ > > "[message passing concurrency]... the original reasons have to do with > fault tolerance... you have to copy all the data you need from > computer 1 to computer 2... if computer 1 crashes you take over on > computer 2... you can't have dangling pointers... that's the reason > for copying everything... it's got nothing to do with concurrency, > it's got a lot to do with fault-tolerance... if they don't crash you > could just have a dangling pointer and copy less data but it won't > work in the presence of errors..." > > I interpret this to mean that share-nothing between processes is more > about replicating valid state than isolating corrupted state as you > described. > > Indeed, Joe created an example on his blog... > > http://armstrongonsoftware.blogspot.com/2007/07/scalable-fault-tolerant-upgradable.html > > > It's algorithm 3 there I'm struggling with. Particularly where he says... > > "... In practise we would send an asynchronous stream of messages from > N to N+1 containing enough information to recover if things go wrong." > > Unfortunately, I couldn't find part II to that post (I don't think > there is one.) And I'm too green and inexperienced in the field of > fault-tolerant systems to figure it out on my own. I'm having trouble > visualising the practical here from the conceptual -- I need to be > shown how :( > > Also, I seem to be under the impression that the Erlang language has > some sort of schematics to do this built-in (i.e. deal with one > process taking over from another if the first fails) and this is the > reason processes share nothing. This seems to me to be something > different from supervision trees, which use exit-trapping to re-spawn > if a process fails with the active 'job' disappearing and any errors > logged (like restarting a daemon). My interpretation of the > fault-tolerance Erlang is supposed to enable (for those in the know) > is seamless take-over. The 'job' lives on but elsewhere. > > Using telecoms as an example: a phone call wouldn't be cut-off when a > fault occurs, another node would seamlessly take over. This is how I > interpreted Joe's post and other descriptions of Erlang's > fault-tolerant features and I understand the key is in the share > nothing policy for processes. I'm sure I've mis-understood something > or everything :) > > - Edmond - > > PS: I've read the Manning draft. Great book. I don't know if the > answer lies in OTP (I searched and didn't find it). I suspect it's > lower -- probalby how you organise your processes. Some > distributed-programming black-magic only Erlanger's know about :) > > > On Mon, 03 Jan 2011 14:56:51 +1100, Alain O'Dea > wrote: > >> On 2011-01-02, at 22:36, "Edmond Begumisa" >> wrote: >> >>> Slight correction... >>> >>> On Mon, 03 Jan 2011 12:38:38 +1100, Edmond Begumisa >>> wrote: >>> >>>> Hello all, >>>> >>>> I've been trying to wrap my Erlang's fault tolerant features >>>> particularly in relation to processes. >>>> >>> >>> Should be: I've been trying to wrap my head around Erlang's fault >>> tolerant features particularly in relation to processes. >>> >>> Sorry. >>> >>>> I've heard/read repeatedly that the primary reason why Erlang's >>>> designers opted for a share-nothing policy is not rooted in >>>> concurrency but rather in fault-tolerance. When nothing is shared, >>>> everything is copied. When everything is copied processes can take >>>> over from one another when things fail. I follow this reasoning but >>>> I don't follow how to apply it. >>>> >>>> I fully understand and appreciate how supervision trees are used to >>>> restart processes if they fail. What I don't get is what to do when >>>> you don't want to restart but want to take over, say on another >>>> node. I know that at a higher-level, OTP has some >>>> take-over/fail-over schematics (at the application level.) I'm >>>> trying to understand things at the processes level - why Erlang is >>>> the way it is so I can better use it to make my currently >>>> fault-intolerant program fault tolerant. >>>> >>>> Specifically, how can one process take over from another if it >>>> fails? It appears to may that the only way to do this would be to >>>> somehow retrieve not only the state of the process (say, >>>> gen_server's state) but also the messages in its mailbox. Where >>>> does the design decision to share-nothing for the sake of >>>> fault-tolerance come into play for processes? Please help me "get" >>>> this! >>>> >>>> Thanks in advance. >>>> >>>> - Edmond - >>>> >>>> >> >> Hi Edmond: >> >> Share-nothing helps with concurrent fault-tolerance by preventing one >> process from corrupting the state of another. Receive is a process' >> choice and it corrupts its own state if it receives bad data and lets >> it in. >> >> AFAIK OTP fault-tolerance doesn't mean no requests will fail, it >> means the system/sub-system will recover if a single request causes a >> process to crash. It's kind of like proper try/catch recovery >> applied to concurrent code. How you recover from the crash depends >> on the supervision strategy chosen. In some cases the supervisor can >> pass the state to the replacement process. In others this isn't >> necessary or even desirable since the state itself may involve >> resources lost in the crash or corrupted state that led to the crash. >> >> I am straying outside my knowledge here so this paragraph is >> guesswork. The message queue for a gen_server need not necessarily >> be lost when the callback module crashes. In theory OTP could (and >> might already) simply delegate the messages to the replacement >> process following a crash. Someone who knows OTP better than me >> would need to weigh in here though. >> >> I found http://manning.com/logan very informative in understanding >> OTP and its supervisor hierarchies. >> >> Cheers, >> Alain >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > From info@REDACTED Mon Jan 3 12:14:59 2011 From: info@REDACTED (=?utf-8?B?aW5mbw==?=) Date: Mon, 3 Jan 2011 12:14:59 +0100 Subject: =?utf-8?B?UmU6IFJlOiBbZXJsYW5nLXF1ZXN0aW9uc10gRXJsYW5nICsgRW1hY3MgKyBEaXN0ZWwgb24gd2luZG93cw==?= References: <201012281349493298586@its3.ch> Message-ID: <201101031214586449427@its3.ch> Hello, I already updated my .emacs file for distel. I defined in c:\ documents and settings\administrator the .erlang_cookie file. *** Is it the correct directory for the erlang cookie ? *** I opened an erl module in emacs. I opened a shell window, I checked node(). = emacs@REDACTED, I checked erlang:get_cookie(). I compiled the erl module by c("mymodule",[debug_info]). I connected to the node emacs by c-c c-d n. *** error in process filter: Opening input file: no such file or directory, c:/.erlang.cookie *** I moved the .erlang.cookie file in c: I connected to the node emacs by c-c c-d n. *** no error message *** I reloaded the module by c-c c-d L *** Distel Warning: node emacs@REDACTED can't seem to load the `distel' module. This means that most Distel commands won't function correctly, because the supporting library is not available. Please check your node's code path, and make sure that Distel's "ebin" directory is included. The most likely cause of this problem is either: a) Your ~/.erlang file doesn't add Distel to your load path (the Distel "make config_install" target can set this up for you.) b) Your system's boot script doesn't consult your ~/.erlang file to read your code path setting. To disable this warning in future, set `distel-inhibit-backend-check' to t. *** .erlang file doesn't exist ! I created it with .emacs because windows doesn't accept a file beginning with a dot. I insert this line: code:add_pathz("C:/distel-4.03/ebin"). make config_install seems to be a linux command ? *** How can I progress ? *** My .beam files are not in /distel-4.03/ebin I am not the first one to use distel and erlang on windows !!! J-Ph. Constantin ITS3 Gen?ve www.its3.ch From sverker@REDACTED Mon Jan 3 12:36:49 2011 From: sverker@REDACTED (Sverker Eriksson) Date: Mon, 3 Jan 2011 12:36:49 +0100 Subject: [erlang-questions] question about NIF resources In-Reply-To: <20110102212039.GA30469@ecn.lan> References: <20110102212039.GA30469@ecn.lan> Message-ID: <4D21B4D1.3090008@erix.ericsson.se> Michael Santos wrote: > The documentation is quite explicit about the behaviour of > enif_make_resource(): > > "The term returned by enif_make_resource is totally opaque in nature. It > can be stored and passed between processses on the same node, but the > only real end usage is to pass it back as argument to a NIF. > <...> > Other operations such as matching or term_to_binary will have > unpredictable (but harmless) results." > > That's clear enough but I wonder if unpredictable behaviour is really > harmless or if I'm confused about how it works. For example, with an > NIF wrapping malloc() [1] such as: > > "Harmless" only in the sense that it will not crash your VM. You will get an Erlangish result, but unpredictable. In other words: Dont do it! In current implementation, resource terms look and behave like empty binaries. Matching is thus pointless. It's on the ToDo list to implement matching. But even if the resource terms look alike, they are unique and there is no bug leaking NIF resources (that I know of). A resource is released (and destructor called) when the last reference is garbage collected. The shell can fool you however, as it keeps a command history that can retain terms even though you think the variables are forgotten. Test NIF resource cleanup by running a test module and call erlang:garbage_collect to force destructors to be called. /Sverker, Erlang/OTP From erlang@REDACTED Mon Jan 3 14:36:30 2011 From: erlang@REDACTED (Joe Armstrong) Date: Mon, 3 Jan 2011 14:36:30 +0100 Subject: [erlang-questions] Processes & Fault Tolerance In-Reply-To: References: Message-ID: I think you are getting confused between OTP supervisors etc. and the general notion of "take-over" Let's start with how we do error recovery. Imagine two linked processes A and B on separate machines. A is the master process. B is a passive processes that will take over if A fails. A sends a stream of state update messages S1, S2, S3,... to B. The state messages contain enough information for B to do whatever A was doing should A fail. If A fails B will receive an EXIT signal. If B does receive an exit signal it knows A has failed, so it carries on doing whatever A was doing using the information it last received in a state update message. That's it and all about it - nothing to do with supervisors etc, To make the above possible we need a couple of primitives in Erlang, namely process_flag trap_exits, and spawn_link. That's all you need. The basic system properties you need to make a fault-tolerant systems are the abilities to detect remote errors and send and receive messages. What the OTP supervisors etc. do are build layers of abstraction on top of spawn-link and trap_exits in order to simplify programming common use-cases. Fault detection and recovery is not arbitrary or magic in any sense, It has to be part of the system architecture like everything else. One common fault in making systems is to specify and design for the normal behavior but not specify and design how fault-detection and correction work. This is probably due to a legacy of programming in languages that have very limited ability to detect errors and recover from them. A C programmer with a single thread of control will take extreme measures to avoid crashing their program they have to, there is only one thread of control. Given multiple threads of control and the ability to remotely detect errors your entire way of thinking changes and the emphasis changes from thinking "how to I avoid a crash" to thinking "given that a crash has occurred and been detected, how to I fix things up and carry on" The Erlang philosophy is that "things will crash", so we have to detect the crashes, fix the bit that broke and carry on - we do not make excessive efforts to avoid crashes, but we do try very hard to repair things after they have gone wrong. Fixing things after they have broken is often easier than preventing them from breaking. Preventing them from breaking is not theoretically possible - we can't even prove simple things, like that a program will terminate (the famous halting problem) - but we can easily detect failures and put the system to rights by applying certain invariants. OTP supervisors are just trees of processes with certain restart rules that experience has shown us to be useful. The main value of supervisors etc. (and of all the OTP behaviors) is in building large systems with many programmers involved. Rather than inventing their own patterns, the programmers reuse a common set of patterns, which makes the projects easier to manage. /Joe On Mon, Jan 3, 2011 at 6:21 AM, Edmond Begumisa wrote: > Thanks for your response. > > Firstly, let me make my question a little clearer... > > To rephrase: For processes, "share nothing for the sake of concurrency" - I > get, both in concept and application. "Share nothing for the sake of > fault-tolerance" - I get in concept but not in application. > > Yet as I understand it, it is for the latter reason Erlang shares nothing* > and not the former. Interpretation: I must be completely missing the point > in regards to Erlang processes and sharing nothing. This is what I want to > understand in application. In addition to the "side" effect of sane > concurrency (which coming from a chaotic multi-threading shared-memory world > I fully appreciate and practically make use of everyday), how can I also > make use of the "real" reason Erlang processes share nothing -- fault > tolerance? Practically/illustratively speaking? > > *ETS being the obvious exception. > > Secondly, here's a mantra from Joe Armstrong... > > @ minute 17:26 > http://www.se-radio.net/2008/03/episode-89-joe-armstrong-on-erlang/ > > "[message passing concurrency]... the original reasons have to do with fault > tolerance... you have to copy all the data you need from computer 1 to > computer 2... if computer 1 crashes you take over on computer 2... you can't > have dangling pointers... that's the reason for copying everything... it's > got nothing to do with concurrency, it's got a lot to do with > fault-tolerance... if they don't crash you could just have a dangling > pointer and copy less data but it won't work in the presence of errors..." > > I interpret this to mean that share-nothing between processes is more about > replicating valid state than isolating corrupted state as you described. > > Indeed, Joe created an example on his blog... > > http://armstrongonsoftware.blogspot.com/2007/07/scalable-fault-tolerant-upgradable.html > > It's algorithm 3 there I'm struggling with. Particularly where he says... > > "... In practise we would send an asynchronous stream of messages from N to > N+1 containing enough information to recover if things go wrong." > > Unfortunately, I couldn't find part II to that post (I don't think there is > one.) And I'm too green and inexperienced in the field of fault-tolerant > systems to figure it out on my own. I'm having trouble visualising the > practical here from the conceptual -- I need to be shown how :( > > Also, I seem to be under the impression that the Erlang language has some > sort of schematics to do this built-in (i.e. deal with one process taking > over from another if the first fails) and this is the reason processes share > nothing. This seems to me to be something different from supervision trees, > which use exit-trapping to re-spawn if a process fails with the active 'job' > disappearing and any errors logged (like restarting a daemon). My > interpretation of the fault-tolerance Erlang is supposed to enable (for > those in the know) is seamless take-over. The 'job' lives on but elsewhere. > > Using telecoms as an example: a phone call wouldn't be cut-off when a fault > occurs, another node would seamlessly take over. This is how I interpreted > Joe's post and other descriptions of Erlang's fault-tolerant features and I > understand the key is in the share nothing policy for processes. I'm sure > I've mis-understood something or everything :) > > - Edmond - > > PS: I've read the Manning draft. Great book. I don't know if the answer lies > in OTP (I searched and didn't find it). I suspect it's lower -- probalby how > you organise your processes. Some distributed-programming black-magic only > Erlanger's know about :) > > > On Mon, 03 Jan 2011 14:56:51 +1100, Alain O'Dea > wrote: > >> On 2011-01-02, at 22:36, "Edmond Begumisa" >> wrote: >> >>> Slight correction... >>> >>> On Mon, 03 Jan 2011 12:38:38 +1100, Edmond Begumisa >>> wrote: >>> >>>> Hello all, >>>> >>>> I've been trying to wrap my Erlang's fault tolerant features >>>> particularly in relation to processes. >>>> >>> >>> Should be: I've been trying to wrap my head around Erlang's fault >>> tolerant features particularly in relation to processes. >>> >>> Sorry. >>> >>>> I've heard/read repeatedly that the primary reason why Erlang's >>>> designers opted for a share-nothing policy is not rooted in concurrency but >>>> rather in fault-tolerance. When nothing is shared, everything is copied. >>>> When everything is copied processes can take over from one another when >>>> things fail. I follow this reasoning but I don't follow how to apply it. >>>> >>>> I fully understand and appreciate how supervision trees are used to >>>> restart processes if they fail. What I don't get is what to do when you >>>> don't want to restart but want to take over, say on another node. I know >>>> that at a higher-level, OTP has some take-over/fail-over schematics (at the >>>> application level.) I'm trying to understand things at the processes level - >>>> why Erlang is the way it is so I can better use it to make my currently >>>> fault-intolerant program fault tolerant. >>>> >>>> Specifically, how can one process take over from another if it fails? It >>>> appears to may that the only way to do this would be to somehow retrieve not >>>> only the state of the process (say, gen_server's state) but also the >>>> messages in its mailbox. Where does the design decision to share-nothing for >>>> the sake of fault-tolerance come into play for processes? Please help me >>>> "get" this! >>>> >>>> Thanks in advance. >>>> >>>> - Edmond - >>>> >>>> >> >> Hi Edmond: >> >> Share-nothing helps with concurrent fault-tolerance by preventing one >> process from corrupting the state of another. Receive is a process' choice >> and it corrupts its own state if it receives bad data and lets it in. >> >> AFAIK OTP fault-tolerance doesn't mean no requests will fail, it means the >> system/sub-system will recover if a single request causes a process to >> crash. ?It's kind of like proper try/catch recovery applied to concurrent >> code. ?How you recover from the crash depends on the supervision strategy >> chosen. ?In some cases the supervisor can pass the state to the replacement >> process. In others this isn't necessary or even desirable since the state >> itself may involve resources lost in the crash or corrupted state that led >> to the crash. >> >> I am straying outside my knowledge here so this paragraph is guesswork. >> ?The message queue for a gen_server need not necessarily be lost when the >> callback module crashes. ?In theory OTP could (and might already) simply >> delegate the messages to the replacement process following a crash. ?Someone >> who knows OTP better than me would need to weigh in here though. >> >> I found http://manning.com/logan very informative in understanding OTP and >> its supervisor hierarchies. >> >> Cheers, >> Alain >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > -- > Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From michael.santos@REDACTED Mon Jan 3 14:45:08 2011 From: michael.santos@REDACTED (Michael Santos) Date: Mon, 3 Jan 2011 08:45:08 -0500 Subject: [erlang-questions] question about NIF resources In-Reply-To: <4D21B4D1.3090008@erix.ericsson.se> References: <20110102212039.GA30469@ecn.lan> <4D21B4D1.3090008@erix.ericsson.se> Message-ID: <20110103134508.GA6097@ecn.lan> On Mon, Jan 03, 2011 at 12:36:49PM +0100, Sverker Eriksson wrote: > Michael Santos wrote: >> The documentation is quite explicit about the behaviour of >> enif_make_resource(): >> >> "The term returned by enif_make_resource is totally opaque in nature. It >> can be stored and passed between processses on the same node, but the >> only real end usage is to pass it back as argument to a NIF. >> <...> >> Other operations such as matching or term_to_binary will have >> unpredictable (but harmless) results." >> >> That's clear enough but I wonder if unpredictable behaviour is really >> harmless or if I'm confused about how it works. For example, with an >> NIF wrapping malloc() [1] such as: >> >> > "Harmless" only in the sense that it will not crash your VM. You will > get an Erlangish result, but unpredictable. In other words: Dont do it! > > In current implementation, resource terms look and behave like empty > binaries. Matching is thus pointless. It's on the ToDo list to implement > matching. Great! > But even if the resource terms look alike, they are unique and there is > no bug leaking NIF resources (that I know of). A resource is released > (and destructor called) when the last reference is garbage collected. > The shell can fool you however, as it keeps a command history that can > retain terms even though you think the variables are forgotten. Test NIF > resource cleanup by running a test module and call > erlang:garbage_collect to force destructors to be called. Works exactly as you said: t() -> {ok,P} = mem:alloc(10), {ok,P} = mem:alloc(10), {ok,P} = mem:alloc(10), {ok,P} = mem:alloc(10), erlang:garbage_collect(), io:format("sleeping now ...~n"), timer:sleep(120000). 1> tt:t(). alloc: p=0x91eca30/0x918f630 alloc: p=0x91eca40/0x918f738 alloc: p=0x91eca50/0x918fd90 alloc: p=0x91ed008/0x918fdb8 cleanup: *p=0x91ed008/0x918fdb8 cleanup: *p=0x91eca50/0x918fd90 cleanup: *p=0x91eca40/0x918f738 cleanup: *p=0x91eca30/0x918f630 sleeping now ... Thank you for the explanation! From ponton@REDACTED Mon Jan 3 15:34:08 2011 From: ponton@REDACTED (Tomasz Maciejewski) Date: Mon, 3 Jan 2011 15:34:08 +0100 Subject: Restricting the connection between two nodes Message-ID: Is there a way to restrict the connection between two nodes, allowing executing only specific functions (eg. only from specific "exported" module)? Suppose we have two nodes running one application, respectively: private_node (eg. Mnesia with sensitive data, or a port owner) and public_node (eg. HTTP/FTP daemon). The two nodes are connected together and the public_node needs some data from private_node, but private_node has some sensitive functions. Now, if someone gains control over public_node, he can call any function from private_node by simply rpc:call, as well as sending arbitrary messages to processes on private_node. How can I protect against breaking in to Erlang by public internet service? Is it even possible to gain access to a remote shell (assuming cookie wasn't stolen) by some remote hole in Erlang/Inets/My application? With the solution of "restricring remote function calls only to functions from specific module", I could implement public API in private_server and be protected against gaining control over private_server. I will really appreciate any comments on security in Erlang. -- Tomasz Maciejewski From martin.reinsprecht@REDACTED Mon Jan 3 15:50:43 2011 From: martin.reinsprecht@REDACTED (Martin Reinsprecht) Date: Mon, 03 Jan 2011 15:50:43 +0100 Subject: Netbeans 6.9.1 Plugin Message-ID: <4D21E243.4070202@activenet.at> Is there a compatible plugin available? Thanks in advance From dcaoyuan@REDACTED Mon Jan 3 15:54:31 2011 From: dcaoyuan@REDACTED (Caoyuan) Date: Mon, 3 Jan 2011 22:54:31 +0800 Subject: [erlang-questions] Netbeans 6.9.1 Plugin In-Reply-To: <4D21E243.4070202@activenet.at> References: <4D21E243.4070202@activenet.at> Message-ID: Not yet. Not sure when I have time to upgrade it. -Caoyuan On Mon, Jan 3, 2011 at 10:50 PM, Martin Reinsprecht wrote: > Is there a compatible ?plugin available? > > Thanks in advance > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From ebegumisa@REDACTED Mon Jan 3 17:21:19 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Tue, 04 Jan 2011 03:21:19 +1100 Subject: [erlang-questions] Processes & Fault Tolerance In-Reply-To: References: Message-ID: Thank you Ulf, now I'm beginning to understand things better at the higher OTP level. As usual, your AXD stories from the wild are priceless (I always tag them in my e-mail client). I think you should write a book or a series of tutorials on designing large systems with Erlang -- it would be a great follow up to the existing 3 main texts (Armstrong, Cesarini + Thomson, Manning.) However, there's something going on at the lower level I still don't get -- why Erlang copies everything between processes and why this is said to lead to fault tolerance. Say I'm writing a fault-tolerant distributed database. I have code in place to replicate the actual writes on disk on different nodes. Now for reads. A request from a client comes in. Process A accesses the disk on Node 1 but the disk fails, so the database needs to a mechanism to have Process B on Node 2 to handle the very same request (it should log the problem but not burden the client with it). There is no natural way I can have a redundant Process B waiting for a disk_fail_error (or a predetermined list of errors) from Process A and handle any pending requests? I'd have to centrally code for the database to spawn a new process and resend any messages the were sent to Process A to a new Process B? - Edmond - On Mon, 03 Jan 2011 19:35:25 +1100, Ulf Wiger wrote: > > On 3 Jan 2011, at 02:38, Edmond Begumisa wrote: > >> I fully understand and appreciate how supervision trees are used to >> restart processes if they fail. What I don't get is what to do when you >> don't want to restart but want to take over, say on another node. I >> know that at a higher-level, OTP has some take-over/fail-over >> schematics (at the application level.) I'm trying to understand things >> at the processes level - why Erlang is the way it is so I can better >> use it to make my currently fault-intolerant program fault tolerant. >> >> Specifically, how can one process take over from another if it fails? >> It appears to may that the only way to do this would be to somehow >> retrieve not only the state of the process (say, gen_server's state) >> but also the messages in its mailbox. Where does the design decision to >> share-nothing for the sake of fault-tolerance come into play for >> processes? Please help me "get" this! > > > To start with the simpler form of redundancy - "cold standby" - it can be > achieved by grouping your code into OTP applications, and configuring > the Distributed Application Controller (dist_ac), so that it will run an > application on one node (e.g. A), if available, and otherwise start it > on B. > > The application started on B will be given a hint in the first argument > of > the start/2 function: Type = {failover, A}, indicating the reason why it > is > being started. The application can then recover state in the > initialization > function. > > (I made a simple example of this in > http://www.trapexit.org/OTP_Release_Handling_Tutorial > I actually developed this example further for a tutorial in Israel, > but my Subversion repository became corrupt and I lost most of it > before I could put it on line. It involved adding mnesia replication, > and I had a more-or-less working riak alternative too. :) > > The "recover state" part requires some forethought, as, obviously, the > original instance of the application is no more. For "less hot" versions > of > standby, one can use some form of "stable-state replication", e.g. by > storing data in a replicated mnesia table, in riak, memBase, etc. > > There is no single answer for how to do this. It depends on the > robustness > requirements and dynamics of your application. You have to decide what > types of errors you need to handle transparently to the user, and what > type > of recovery action it is reasonable to expect of the user itself. > > To tie back to telecoms, the basic service that Erlang was designed for > is > the telephone call. In this case, we (used to) rely on the phone service > being > extremely reliable - it practically never happened that we picked up the > phone and didn't get dial tone; when it did, usually, if we tried again, > we > *would* get dial tone. Also, if an ongoing call suddenly disconnected, we > would try again, and usually succeed. As long as this didn't occur too > often, > we wouldn't think twice about it. With mobile telephony, it is a much > more > common event (in fairness, mobile telephony is *much* more complex), > but we accept it, since the service gives us much greater freedom. > > From a programming point of view then, recovering a phone call would > involve resetting it to its most stable state (on hook) and preparing for > the event that the user may try again. In the AXD 301, we would take it > a bit further and replicate the "connected" state to a standby node. This > was done using erlang message passing, with an intermediary that > aggregated lots of call states together in order to reduce cost. You > could > view this as a form of "best-effort hot standby", where it wasn't really > necessary to know if each state made it to the standby; we might lose > some > in the transition. We used a 'standby' application on the standby node, > which > collected the call states and prepared the data structures (exactly how > this > was done changed over time). Then we used the failover mechanism above > to fire up the call handling application on the other side. Since it > knew it > was a failover, it also knew to handshake with the standby app to get the > data. It then had a few seconds to get everything in order and respond > appropriately to the status enquiry messages from the clients, which > were built into the protocol. The actual media stream was handled by > different hardware, so it was necessary to audit all known calls with > that > hardware layer. All calls that were not known to the Erlang side were > simply reset. > > We had some true hot-standby problems as well. In some cases, > Erlang was involved, but depending on the problem, we had to > resort to hardware-based solutions for some, and in at least one case, > I remember that putting a cheap, passive (i.e. reliable) splitter in > front of the system was the simplest way to "replicate" the signal, > making hot standby possible. The hottest of them all was that we > could fail over from one ATM switching plane to another, losing at > most 6 ATM cells. No Erlang there... > > To wrap this up, Erlang provides a number of different ways to detect > that it's time to take over. I find that the OTP application concept > offers > a very clean way - with the start({failover,FromNode}, Args) function - > allowing you to contain the logic in one place. For some requirements, > this may not be sufficiently "hot", in which case you may need to roll > your own mechanism. The trick, as you noted, is to get hold of the data > needed to take over. Erlang doesn't solve this automatically; you have > to plan your data storage, state replication, etc. to make it possible. > > In my experience, you should really think about what you can get away > with. Users normally accept outages, if they are sufficiently rare and > short, and the user doesn't lose money or other precious assets. > Don't knock the simple solutions because it's sexy to be "infinitely > robust" - most likely, the added complexity will make your system *less* > robust - not more so. > > BR, > Ulf W > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From ebegumisa@REDACTED Mon Jan 3 17:21:24 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Tue, 04 Jan 2011 03:21:24 +1100 Subject: [erlang-questions] Processes & Fault Tolerance In-Reply-To: <4D21A44F.8010509@erlang-solutions.com> References: <4D21A44F.8010509@erlang-solutions.com> Message-ID: Thanks. I see what you're saying... so share-nothing is more about fault-isolation than fault-tolerance. Isolate the fault to affect as few processes as possible rather than deal with the fault then continue. So there's no natural I can arrange a group of active redundant processes on different nodes sharing replicated state, ready to take over if one fails. Like what distributed fault-tolerant databases do with disks, but instead of nodes with replicated disks, I have nodes with replicated processes. - Edmond - On Mon, 03 Jan 2011 21:26:23 +1100, Mazen Harake wrote: > Don't think of it as "taking over" but more about "recovering from" and > "minimizing affect area". > > If process A dies you are not expected to "save" the state of A and > _transfer_ it to B and thus continue as if B was A. This doesn't make > any sense. The idea is rather that if A crashes then you spawn a new > process B which a) has unaffected any other process (say C and D) > because there is no data corruption and b) when it starts it might read > in a persistent state (say from an ETS table) which it uses as a base to > continue an operation. This doesn't mean that you transferred the state > of A to B, you recovered from A by starting B. There is a difference > between types of state as well, E.g. a gen_server state is not > necessarily important to store in a resilient matter but perhaps the id > for the active session is because it can be used to recover and > repopulate the gen_server state. > > Some examples: > > * If you have an ongoing call with someone and there is a bug that > disconnects your call then someone else who is having a call should not > be affected. The fault tolerance part here is not that your call will > continue, it is that you are able to pick up the dial again and call the > user back I.e. the system is still alive even though it suffered a minor > glitch in a particular process. (There are ways to keep the call alive > but I'm not sure they do that). > > * If you have say an IM session running (ejabberd) then you might have a > process per request/message/whatever. In this case perhaps your id, > session id and some other key data would be stored and shared on several > mnesia nodes but it isn't shared in the sense that it is used by many > processes, it is just persistent. This means that if a process which is > in the middle of something that has to do with that state crashes then > another (newly spawned process) can read the various keys and ids and > repopulate the state and continue. > > Share nothing in both cases mean that where ever the data is, it is only > used by one process at a time. The first example might not have > persistent data that it keeps but it doesn't affect any other part of > the system if the call goes down. The second example has persistent data > but it doesn't mean it is shared among processes it just means it > handles node disruptions so that new ones can continue. > > Now if you move on to Node level then these two scenarios still apply > but with small differences. The first example will cut off all calls > routed through that node but as soon as some would try to call again it > would simply go through another node. The second example would > distribute the session state so that if a node goes down a new process > on another node can handle the continuation of the session. In the later > case it is important to realize that the shared data (the state between > the nodes) is an obvious bottle neck but it is another type of shared > data because it is, through abstraction, only manipulated by 1 process. > > Makes sense? %-) > > On 03/01/2011 06:21, Edmond Begumisa wrote: >> Thanks for your response. >> >> Firstly, let me make my question a little clearer... >> >> To rephrase: For processes, "share nothing for the sake of concurrency" >> - I get, both in concept and application. "Share nothing for the sake >> of fault-tolerance" - I get in concept but not in application. >> >> Yet as I understand it, it is for the latter reason Erlang shares >> nothing* and not the former. Interpretation: I must be completely >> missing the point in regards to Erlang processes and sharing nothing. >> This is what I want to understand in application. In addition to the >> "side" effect of sane concurrency (which coming from a chaotic >> multi-threading shared-memory world I fully appreciate and practically >> make use of everyday), how can I also make use of the "real" reason >> Erlang processes share nothing -- fault tolerance? >> Practically/illustratively speaking? >> >> *ETS being the obvious exception. >> >> Secondly, here's a mantra from Joe Armstrong... >> >> @ minute 17:26 >> http://www.se-radio.net/2008/03/episode-89-joe-armstrong-on-erlang/ >> >> "[message passing concurrency]... the original reasons have to do with >> fault tolerance... you have to copy all the data you need from computer >> 1 to computer 2... if computer 1 crashes you take over on computer 2... >> you can't have dangling pointers... that's the reason for copying >> everything... it's got nothing to do with concurrency, it's got a lot >> to do with fault-tolerance... if they don't crash you could just have a >> dangling pointer and copy less data but it won't work in the presence >> of errors..." >> >> I interpret this to mean that share-nothing between processes is more >> about replicating valid state than isolating corrupted state as you >> described. >> >> Indeed, Joe created an example on his blog... >> >> http://armstrongonsoftware.blogspot.com/2007/07/scalable-fault-tolerant-upgradable.html >> It's algorithm 3 there I'm struggling with. Particularly where he >> says... >> >> "... In practise we would send an asynchronous stream of messages from >> N to N+1 containing enough information to recover if things go wrong." >> >> Unfortunately, I couldn't find part II to that post (I don't think >> there is one.) And I'm too green and inexperienced in the field of >> fault-tolerant systems to figure it out on my own. I'm having trouble >> visualising the practical here from the conceptual -- I need to be >> shown how :( >> >> Also, I seem to be under the impression that the Erlang language has >> some sort of schematics to do this built-in (i.e. deal with one process >> taking over from another if the first fails) and this is the reason >> processes share nothing. This seems to me to be something different >> from supervision trees, which use exit-trapping to re-spawn if a >> process fails with the active 'job' disappearing and any errors logged >> (like restarting a daemon). My interpretation of the fault-tolerance >> Erlang is supposed to enable (for those in the know) is seamless >> take-over. The 'job' lives on but elsewhere. >> >> Using telecoms as an example: a phone call wouldn't be cut-off when a >> fault occurs, another node would seamlessly take over. This is how I >> interpreted Joe's post and other descriptions of Erlang's >> fault-tolerant features and I understand the key is in the share >> nothing policy for processes. I'm sure I've mis-understood something or >> everything :) >> >> - Edmond - >> >> PS: I've read the Manning draft. Great book. I don't know if the answer >> lies in OTP (I searched and didn't find it). I suspect it's lower -- >> probalby how you organise your processes. Some distributed-programming >> black-magic only Erlanger's know about :) >> >> >> On Mon, 03 Jan 2011 14:56:51 +1100, Alain O'Dea >> wrote: >> >>> On 2011-01-02, at 22:36, "Edmond Begumisa" >>> wrote: >>> >>>> Slight correction... >>>> >>>> On Mon, 03 Jan 2011 12:38:38 +1100, Edmond Begumisa >>>> wrote: >>>> >>>>> Hello all, >>>>> >>>>> I've been trying to wrap my Erlang's fault tolerant features >>>>> particularly in relation to processes. >>>>> >>>> >>>> Should be: I've been trying to wrap my head around Erlang's fault >>>> tolerant features particularly in relation to processes. >>>> >>>> Sorry. >>>> >>>>> I've heard/read repeatedly that the primary reason why Erlang's >>>>> designers opted for a share-nothing policy is not rooted in >>>>> concurrency but rather in fault-tolerance. When nothing is shared, >>>>> everything is copied. When everything is copied processes can take >>>>> over from one another when things fail. I follow this reasoning but >>>>> I don't follow how to apply it. >>>>> >>>>> I fully understand and appreciate how supervision trees are used to >>>>> restart processes if they fail. What I don't get is what to do when >>>>> you don't want to restart but want to take over, say on another >>>>> node. I know that at a higher-level, OTP has some >>>>> take-over/fail-over schematics (at the application level.) I'm >>>>> trying to understand things at the processes level - why Erlang is >>>>> the way it is so I can better use it to make my currently >>>>> fault-intolerant program fault tolerant. >>>>> >>>>> Specifically, how can one process take over from another if it >>>>> fails? It appears to may that the only way to do this would be to >>>>> somehow retrieve not only the state of the process (say, >>>>> gen_server's state) but also the messages in its mailbox. Where does >>>>> the design decision to share-nothing for the sake of fault-tolerance >>>>> come into play for processes? Please help me "get" this! >>>>> >>>>> Thanks in advance. >>>>> >>>>> - Edmond - >>>>> >>>>> >>> >>> Hi Edmond: >>> >>> Share-nothing helps with concurrent fault-tolerance by preventing one >>> process from corrupting the state of another. Receive is a process' >>> choice and it corrupts its own state if it receives bad data and lets >>> it in. >>> >>> AFAIK OTP fault-tolerance doesn't mean no requests will fail, it means >>> the system/sub-system will recover if a single request causes a >>> process to crash. It's kind of like proper try/catch recovery applied >>> to concurrent code. How you recover from the crash depends on the >>> supervision strategy chosen. In some cases the supervisor can pass >>> the state to the replacement process. In others this isn't necessary >>> or even desirable since the state itself may involve resources lost in >>> the crash or corrupted state that led to the crash. >>> >>> I am straying outside my knowledge here so this paragraph is >>> guesswork. The message queue for a gen_server need not necessarily be >>> lost when the callback module crashes. In theory OTP could (and might >>> already) simply delegate the messages to the replacement process >>> following a crash. Someone who knows OTP better than me would need to >>> weigh in here though. >>> >>> I found http://manning.com/logan very informative in understanding OTP >>> and its supervisor hierarchies. >>> >>> Cheers, >>> Alain >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From ebegumisa@REDACTED Mon Jan 3 17:30:03 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Tue, 04 Jan 2011 03:30:03 +1100 Subject: [erlang-questions] Processes & Fault Tolerance In-Reply-To: References: Message-ID: > Let's start with how we do error recovery. > > Imagine two linked processes A and B on separate machines. A is the > master process. > B is a passive processes that will take over if A fails. > > A sends a stream of state update messages S1, S2, S3,... to B. The > state messages contain enough information > for B to do whatever A was doing should A fail. If A fails B will > receive an EXIT signal. > > If B does receive an exit signal it knows A has failed, so it carries > on doing whatever A was doing > using the information it last received in a state update message. > > That's it and all about it - nothing to do with supervisors etc, Ooooohh! THANK YOU, THANK YOU, THANK YOU. *Now* I get it. This is precisely what I was trying to understand. The missing link was sending redundant messages. It all makes sense now -- it's so simple. All I have to do to get fault tolerance at the process level is have a group of N redundant processes waiting for exit-signals and forward state changes to N-1 of them. I could even send to N/2 and have some decent tolerance at the expense of consistency of the nodes. To preemptively answer the question I sent Ulf and Mazen... for that distributed database, Process B would be set up to also receive read requests forwarded from Process A but not respond to them unless it gets an exit signal it can understand like disk_fail_error. Any changes in state would also be forwarded to existing redundant processes. Thanks everyone for your patience in helping me understand fault-tolerance. I've already started thinking about some cool things I can do with this in my program. - Edmond - On Tue, 04 Jan 2011 00:36:30 +1100, Joe Armstrong wrote: > I think you are getting confused between OTP supervisors etc. and the > general notion of "take-over" > > Let's start with how we do error recovery. > > Imagine two linked processes A and B on separate machines. A is the > master process. > B is a passive processes that will take over if A fails. > > A sends a stream of state update messages S1, S2, S3,... to B. The > state messages contain enough information > for B to do whatever A was doing should A fail. If A fails B will > receive an EXIT signal. > > If B does receive an exit signal it knows A has failed, so it carries > on doing whatever A was doing > using the information it last received in a state update message. > > That's it and all about it - nothing to do with supervisors etc, > > > To make the above possible we need a couple of primitives in Erlang, > namely process_flag trap_exits, and spawn_link. That's all you need. > > The basic system properties you need to make a fault-tolerant systems > are the abilities to detect remote errors and send and receive messages. > > What the OTP supervisors etc. do are build layers of abstraction on > top of spawn-link and trap_exits > in order to simplify programming common use-cases. > > Fault detection and recovery is not arbitrary or magic in any sense, > It has to be part of the system architecture like everything else. One > common fault in making systems is to specify and design for the normal > behavior > but not specify and design how fault-detection and correction work. > This is probably due to a legacy of > programming in languages that have very limited ability to detect > errors and recover from them. > > A C programmer with a single thread of control will take extreme > measures to avoid crashing their program > they have to, there is only one thread of control. Given multiple > threads of control and the ability to remotely > detect errors your entire way of thinking changes and the emphasis > changes from thinking "how to I avoid a crash" > to thinking "given that a crash has occurred and been detected, how to > I fix things up and carry on" > > The Erlang philosophy is that "things will crash", so we have to > detect the crashes, fix the bit that broke and > carry on - we do not make excessive efforts to avoid crashes, but we > do try very hard to repair things after they have gone wrong. > > Fixing things after they have broken is often easier than preventing > them from breaking. Preventing them > from breaking is not theoretically possible - we can't even prove > simple things, like that a program will terminate > (the famous halting problem) - but we can easily detect failures and > put the system to rights by applying > certain invariants. > > OTP supervisors are just trees of processes with certain restart rules > that experience has shown us > to be useful. > > The main value of supervisors etc. (and of all the OTP behaviors) is > in building large systems with many > programmers involved. Rather than inventing their own patterns, the > programmers reuse a common set of > patterns, which makes the projects easier to manage. > > /Joe > > > > > On Mon, Jan 3, 2011 at 6:21 AM, Edmond Begumisa > wrote: >> Thanks for your response. >> >> Firstly, let me make my question a little clearer... >> >> To rephrase: For processes, "share nothing for the sake of concurrency" >> - I >> get, both in concept and application. "Share nothing for the sake of >> fault-tolerance" - I get in concept but not in application. >> >> Yet as I understand it, it is for the latter reason Erlang shares >> nothing* >> and not the former. Interpretation: I must be completely missing the >> point >> in regards to Erlang processes and sharing nothing. This is what I want >> to >> understand in application. In addition to the "side" effect of sane >> concurrency (which coming from a chaotic multi-threading shared-memory >> world >> I fully appreciate and practically make use of everyday), how can I also >> make use of the "real" reason Erlang processes share nothing -- fault >> tolerance? Practically/illustratively speaking? >> >> *ETS being the obvious exception. >> >> Secondly, here's a mantra from Joe Armstrong... >> >> @ minute 17:26 >> http://www.se-radio.net/2008/03/episode-89-joe-armstrong-on-erlang/ >> >> "[message passing concurrency]... the original reasons have to do with >> fault >> tolerance... you have to copy all the data you need from computer 1 to >> computer 2... if computer 1 crashes you take over on computer 2... you >> can't >> have dangling pointers... that's the reason for copying everything... >> it's >> got nothing to do with concurrency, it's got a lot to do with >> fault-tolerance... if they don't crash you could just have a dangling >> pointer and copy less data but it won't work in the presence of >> errors..." >> >> I interpret this to mean that share-nothing between processes is more >> about >> replicating valid state than isolating corrupted state as you described. >> >> Indeed, Joe created an example on his blog... >> >> http://armstrongonsoftware.blogspot.com/2007/07/scalable-fault-tolerant-upgradable.html >> >> It's algorithm 3 there I'm struggling with. Particularly where he >> says... >> >> "... In practise we would send an asynchronous stream of messages from >> N to >> N+1 containing enough information to recover if things go wrong." >> >> Unfortunately, I couldn't find part II to that post (I don't think >> there is >> one.) And I'm too green and inexperienced in the field of fault-tolerant >> systems to figure it out on my own. I'm having trouble visualising the >> practical here from the conceptual -- I need to be shown how :( >> >> Also, I seem to be under the impression that the Erlang language has >> some >> sort of schematics to do this built-in (i.e. deal with one process >> taking >> over from another if the first fails) and this is the reason processes >> share >> nothing. This seems to me to be something different from supervision >> trees, >> which use exit-trapping to re-spawn if a process fails with the active >> 'job' >> disappearing and any errors logged (like restarting a daemon). My >> interpretation of the fault-tolerance Erlang is supposed to enable (for >> those in the know) is seamless take-over. The 'job' lives on but >> elsewhere. >> >> Using telecoms as an example: a phone call wouldn't be cut-off when a >> fault >> occurs, another node would seamlessly take over. This is how I >> interpreted >> Joe's post and other descriptions of Erlang's fault-tolerant features >> and I >> understand the key is in the share nothing policy for processes. I'm >> sure >> I've mis-understood something or everything :) >> >> - Edmond - >> >> PS: I've read the Manning draft. Great book. I don't know if the answer >> lies >> in OTP (I searched and didn't find it). I suspect it's lower -- >> probalby how >> you organise your processes. Some distributed-programming black-magic >> only >> Erlanger's know about :) >> >> >> On Mon, 03 Jan 2011 14:56:51 +1100, Alain O'Dea >> wrote: >> >>> On 2011-01-02, at 22:36, "Edmond Begumisa" >>> >>> wrote: >>> >>>> Slight correction... >>>> >>>> On Mon, 03 Jan 2011 12:38:38 +1100, Edmond Begumisa >>>> wrote: >>>> >>>>> Hello all, >>>>> >>>>> I've been trying to wrap my Erlang's fault tolerant features >>>>> particularly in relation to processes. >>>>> >>>> >>>> Should be: I've been trying to wrap my head around Erlang's fault >>>> tolerant features particularly in relation to processes. >>>> >>>> Sorry. >>>> >>>>> I've heard/read repeatedly that the primary reason why Erlang's >>>>> designers opted for a share-nothing policy is not rooted in >>>>> concurrency but >>>>> rather in fault-tolerance. When nothing is shared, everything is >>>>> copied. >>>>> When everything is copied processes can take over from one another >>>>> when >>>>> things fail. I follow this reasoning but I don't follow how to apply >>>>> it. >>>>> >>>>> I fully understand and appreciate how supervision trees are used to >>>>> restart processes if they fail. What I don't get is what to do when >>>>> you >>>>> don't want to restart but want to take over, say on another node. I >>>>> know >>>>> that at a higher-level, OTP has some take-over/fail-over schematics >>>>> (at the >>>>> application level.) I'm trying to understand things at the processes >>>>> level - >>>>> why Erlang is the way it is so I can better use it to make my >>>>> currently >>>>> fault-intolerant program fault tolerant. >>>>> >>>>> Specifically, how can one process take over from another if it >>>>> fails? It >>>>> appears to may that the only way to do this would be to somehow >>>>> retrieve not >>>>> only the state of the process (say, gen_server's state) but also the >>>>> messages in its mailbox. Where does the design decision to >>>>> share-nothing for >>>>> the sake of fault-tolerance come into play for processes? Please >>>>> help me >>>>> "get" this! >>>>> >>>>> Thanks in advance. >>>>> >>>>> - Edmond - >>>>> >>>>> >>> >>> Hi Edmond: >>> >>> Share-nothing helps with concurrent fault-tolerance by preventing one >>> process from corrupting the state of another. Receive is a process' >>> choice >>> and it corrupts its own state if it receives bad data and lets it in. >>> >>> AFAIK OTP fault-tolerance doesn't mean no requests will fail, it means >>> the >>> system/sub-system will recover if a single request causes a process to >>> crash. It's kind of like proper try/catch recovery applied to >>> concurrent >>> code. How you recover from the crash depends on the supervision >>> strategy >>> chosen. In some cases the supervisor can pass the state to the >>> replacement >>> process. In others this isn't necessary or even desirable since the >>> state >>> itself may involve resources lost in the crash or corrupted state that >>> led >>> to the crash. >>> >>> I am straying outside my knowledge here so this paragraph is guesswork. >>> The message queue for a gen_server need not necessarily be lost when >>> the >>> callback module crashes. In theory OTP could (and might already) >>> simply >>> delegate the messages to the replacement process following a crash. >>> Someone >>> who knows OTP better than me would need to weigh in here though. >>> >>> I found http://manning.com/logan very informative in understanding OTP >>> and >>> its supervisor hierarchies. >>> >>> Cheers, >>> Alain >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> >> -- >> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From co7eb@REDACTED Mon Jan 3 19:12:38 2011 From: co7eb@REDACTED (=?iso-8859-1?Q?Gilberio_Carmenates_Garc=EDa?=) Date: Mon, 3 Jan 2011 13:12:38 -0500 Subject: [erlang-questions] Free web host site??? In-Reply-To: References: Message-ID: <000f01cbab71$cf476d50$6dd647f0$@co.cu> Hi all!!!, Well let me check! But what I want is to have a web-page somewhere where I could publish and upload files, for example the EVO (ExtendedVisualOtp) Release 1.5 module for Erlang and other works or projects I could make. Regards, Ivan. ======================================================================= Este mensaje ha sido enviado mediante el servicio de correo electr?nico que ofrece la Federaci?n de Radioaficionados de Cuba a sus miembros para respaldar el cumplimiento de los objetivos de la organizaci?n y su pol?tica informativa. La persona que env?a este correo asume el compromiso de usar el servicio a tales fines y cumplir con las regulaciones establecidas. From ebegumisa@REDACTED Mon Jan 3 19:47:35 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Tue, 04 Jan 2011 05:47:35 +1100 Subject: [erlang-questions] Free web host site??? In-Reply-To: <000f01cbab71$cf476d50$6dd647f0$@co.cu> References: <000f01cbab71$cf476d50$6dd647f0$@co.cu> Message-ID: Hi Ivan, You want github (free for open-source code). Publish and share your code files with others and source/version control too. You won't have to keep sending EVO attachments on the mailing list. http://en.wikipedia.org/wiki/GitHub https://github.com/plans If you don't like github (or git), explore... http://en.wikipedia.org/wiki/Comparison_of_open_source_software_hosting_facilities - Edmond - On Tue, 04 Jan 2011 05:12:38 +1100, Gilberio Carmenates Garc?a wrote: > Hi all!!!, > > Well let me check! But what I want is to have a web-page somewhere where > I > could publish and upload files, for example the EVO (ExtendedVisualOtp) > Release 1.5 module for Erlang and other works or projects I could make. > > > Regards, > > Ivan. > > > ======================================================================= > Este mensaje ha sido enviado mediante el servicio de correo electr?nico > que ofrece la Federaci?n de Radioaficionados de Cuba a sus miembros para > respaldar el cumplimiento de los objetivos de la organizaci?n y su > pol?tica informativa. La persona que env?a este correo asume el > compromiso de usar el servicio a tales fines y cumplir con las > regulaciones establecidas. > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From jetkoten@REDACTED Tue Jan 4 02:45:47 2011 From: jetkoten@REDACTED (JETkoten) Date: Mon, 03 Jan 2011 20:45:47 -0500 Subject: [erlang-questions] Beginner Screen Scaping, and Auto-login/input on 3rd party web app? In-Reply-To: References: <4D1B5E89.50503@gmail.com> <4d1b60e7.8b02e30a.3541.ffff8c69@mx.google.com> Message-ID: <4D227BCB.6080002@gmail.com> Thanks Jared, Jeroen and Jesse... I've been looking into the options you've sent and trying to wrap my head around the concepts. Thanks again, Jack On 12/29/10 11:29 AM, Jeroen Koops wrote: > Yes, mochiweb has the mochiweb_html module, which parses HTML. > > On Wed, Dec 29, 2010 at 5:24 PM, Jesse Gumm wrote: >> Hi there, >> >> I'd recommend looking into httpc module, which is an http client. >> >> As for parsing the data, you could check out the 're' module for regular expressions, and it could work mostly, but html is not technically regex parseable. I don't know of an erlang module for parsing an html dom tree, perhaps mochiweb provides something. >> >> -Jesse >> >> >> -- >> Jesse Gumm >> Sigma Star Systems >> 414.940.4866 >> On Dec 29, 2010 10:19 AM, JETkoten<jetkoten@REDACTED> wrote: >> >> On 12/24/10 12:05 PM, Alain O'Dea wrote: >> >> > On 2010-12-24, at 11:08, JETkoten<jetkoten@REDACTED> wrote: >> >> > >> >> >> [...] Hi Everyone, >> >> >> >> >> >> I'm (very) new to Erlang, and hoping to get some basic experience with it. >> >> >> >> >> >> I really learn best by doing something I'm interested in. I have a "pet project" that I would like to implement now in Erlang. >> >> >> >> >> >> Here it is: >> >> >> >> >> >> I have a large personal library of books and find that I don't need many of them anymore. I'd like to create a program that will help me manage my online sales on a marketplace site, by automatically checking competing sellers' prices at a set time interval an then logging into their website and adjusting my prices according to a formula I'd set based on the other prices. >> >> >> >> >> >> I did a Google search on Erlang "screen scraping" and saw some options: >> >> >> >> >> >> www_tools, Yaws parser, xmerl, mochiweb >> >> >> >> >> >> However, none of the posts that suggest those are less than 2 years old... which is the best/easiest way, and/or are there newer, better options now? >> >> >> >> >> >> Any ideas? >> >> >> >> >> >> Thanks in advance, >> >> >> Jack >> >> > Hi Jack: >> >> > >> >> > [...] Gradually it probably makes sense to switch to native Erlang utilities if you find them to perform or integrate better. >> >> > >> >> > [...] >> >> > >> >> > Eventually it makes sense to use OTP and supervisors to consistently handle agent crashes. If you find yourself writing a lot of try/catch logic, then stop and refactor to OTP. Erlang and OTP in Action http://manning.com/logan is the best book for this. >> >> > >> >> > Cheers and Merry Christmas, >> >> > Alain >> >> Hi Alain, >> >> >> >> Thanks very much for your reply. >> >> >> >> So, I do want to try and implement this with the native Erlang >> >> utilities, and am not sure where to begin. I started writing a module >> >> and then tried to think of what kind of functions I could use to perform >> >> these tasks, but I don't know how to access a website to screen scrape >> >> with Erlang or how/where to efficiently store and retreve the >> >> price/title data that I would scrape. >> >> >> >> Would Mnesia or something like Riak be good for the storage part? >> >> >> >> I also don't know how to get my program to log in to the site after >> >> calculating the new price from the scraped data and then changing it on >> >> the marketplace site... >> >> >> >> In the OTP version, would I be looking at gen_server or maybe gen_fsm to >> >> complete the tasks? I looked through the Erlang and OTP in Action book >> >> in a bookstore, but it seems too advanced for me at this point to get >> >> much benefit from. >> >> >> >> I'm truly a beginner here, so any concrete steps/tools anyone can offer >> >> would be a huge help! I've been looking through the online tutorials and >> >> books, but can't seem to find much about Erlang and WWW related tasks >> >> like these. >> >> >> >> Thanks again, >> >> Jack >> >> >> >> ________________________________________________________________ >> >> erlang-questions (at) erlang.org mailing list. >> >> See http://www.erlang.org/faq.html >> >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> >> >> >> From jetkoten@REDACTED Tue Jan 4 02:52:53 2011 From: jetkoten@REDACTED (JETkoten) Date: Mon, 03 Jan 2011 20:52:53 -0500 Subject: Erlang and the learning curve Message-ID: <4D227D75.1060108@gmail.com> Hi All, I've been reading/watching everything I can find (off and online) for learning Erlang for the past two or three weeks, but am just not getting it, in terms of being able to do anything with Erlang in the real world. I've watched some videos about functional programming where the hosts joked about people looking into functional programming deciding it's too hard and then never touching it again... but seriously, how do people bridge the gap between understanding the basics and implementing real world projects with Erlang? I know the three major books are available, but none of them are really using Erlang for what I'd like to do... How have any of you who have come to know big-picture how great Erlang is been able to get enough knowledge of it to actually do what you'd like to do? Any and all replies are most welcome. Thanks, Jack From bob@REDACTED Tue Jan 4 03:00:22 2011 From: bob@REDACTED (Bob Ippolito) Date: Tue, 4 Jan 2011 10:00:22 +0800 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D227D75.1060108@gmail.com> References: <4D227D75.1060108@gmail.com> Message-ID: Do you know what it is you'd like to do with Erlang? In my case I knew what I wanted to do before I started (build an ad server), and I knew how to do it without Erlang, so taking the basics and applying it to a real world problem was pretty straightforward for me. The thing that finally made me "get" Erlang was Joe Armstrong's Ph.d thesis [1], but at the time there were no books in print on Erlang so there are probably better resources these days :) [1] http://www.erlang.org/download/armstrong_thesis_2003.pdf On Tue, Jan 4, 2011 at 9:52 AM, JETkoten wrote: > Hi All, > > I've been reading/watching everything I can find (off and online) for > learning Erlang for the past two or three weeks, but am just not getting it, > in terms of being able to do anything with Erlang in the real world. > > I've watched some videos about functional programming where the hosts joked > about people looking into functional programming deciding it's too hard and > then never touching it again... but seriously, how do people bridge the gap > between understanding the basics and implementing real world projects with > Erlang? I know the three major books are available, but none of them are > really using Erlang for what I'd like to do... > > How have any of you who have come to know big-picture how great Erlang is > been able to get enough knowledge of it to actually do what you'd like to > do? > > Any and all replies are most welcome. > > Thanks, > Jack > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From serge@REDACTED Tue Jan 4 03:06:12 2011 From: serge@REDACTED (Serge Aleynikov) Date: Mon, 03 Jan 2011 21:06:12 -0500 Subject: [ANN] eixx - C++ interface to Erlang Message-ID: <4D228094.5030805@aleynikov.org> http://github.com/saleyn/eixx eixx - Erlang C++ Interface Library =================================== This library provides a set of classes for convenient marshaling of Erlang terms between processes as well as connecting to other distributed Erlang nodes from a C++ application. The marshaling classes are built on top of ei library included in {@link http://www.erlang.org/doc/apps/erl_interface erl_interface}. It is largely inspired by the {@link http://code.google.com/p/epi epi} project, but is a complete rewrite with many new features and optimization enhancements. This library adds on the following features: - Use of reference-counted smart pointers for complex terms and by-value copied simple terms (e.i. integers, doubles, bool, atoms). - Ability to provide custom memory allocators. - Encoding/decoding of nested terms using a single function call (eterm::encode() and eterm::eterm() constructor). - Global atom table for fast manipulation of atoms. The library consists of two separate parts: - Term marshaling (included by marshal.hpp or eixx.hpp) - Distributed node connectivity (included by connect.hpp or eixx.hpp) The connectivity library implements a richer set of features than what's available in erl_interface - it fully supports process linking and monitoring. The library is fully asynchronous and allows handling many connections and mailboxes in one OS thread. Ths library is dependend on {@link http://www.boost.org BOOST} project and erl_interface, which is a part of the {@link www.erlang.org Erlang} distribution. Regards, Serge From g9414002.pccu.edu.tw@REDACTED Tue Jan 4 03:06:32 2011 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Tue, 4 Jan 2011 10:06:32 +0800 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D227D75.1060108@gmail.com> References: <4D227D75.1060108@gmail.com> Message-ID: On Tue, Jan 4, 2011 at 9:52 AM, JETkoten wrote: > Hi All, > > I've been reading/watching everything I can find (off and online) for > learning Erlang for the past two or three weeks, but am just not getting it, > in terms of being able to do anything with Erlang in the real world. > > I've watched some videos about functional programming where the hosts joked > about people looking into functional programming deciding it's too hard and > then never touching it again... but seriously, how do people bridge the gap > between understanding the basics and implementing real world projects with > Erlang? I know the three major books are available, but none of them are > really using Erlang for what I'd like to do... > > Joe's book showed how to write instant messaging clients, but you think it not so real? > How have any of you who have come to know big-picture how great Erlang is > been able to get enough knowledge of it to actually do what you'd like to > do? > > Any and all replies are most welcome. > > Thanks, > Jack > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > I think most of us treat it so serious that "if it is very easy, it ought to be now that we have many such and such applications make by Erlang." But you won't take it so seriously when you learn to write a segment of C code. Good applications always go from simple examples. -- Best Regards. --- Y-H. H. From jeffm@REDACTED Tue Jan 4 03:06:07 2011 From: jeffm@REDACTED (jm) Date: Tue, 04 Jan 2011 13:06:07 +1100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D227D75.1060108@gmail.com> References: <4D227D75.1060108@gmail.com> Message-ID: <4D22808F.6030202@ghostgun.com> Would you like to post a small snippet of code in Ruby and someone of the list could propose an equivalent piece of Erlang? Jeff. On 4/01/11 12:52 PM, JETkoten wrote: > Hi All, > > I've been reading/watching everything I can find (off and online) for > learning Erlang for the past two or three weeks, but am just not > getting it, in terms of being able to do anything with Erlang in the > real world. From jetkoten@REDACTED Tue Jan 4 03:12:35 2011 From: jetkoten@REDACTED (JETkoten) Date: Mon, 03 Jan 2011 21:12:35 -0500 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: References: <4D227D75.1060108@gmail.com> Message-ID: <4D228213.8050502@gmail.com> Thanks for your reply. Yes, I've got my pet project in mind to auto-monitor and update an online book sales marketplace. I know the tools to use to make it happen with Ruby and had a prototype with some of the core functionality started (with some help I got from a local user group member). Overall though I do lack experience there too, and don't quite know how to do it without Erlang per se. I did a CS degree about 7 years ago, but didn't ever really learn to code there. I focused more on OO analysis and design and software requirements engineering... along with distributed digital multimedia. Now I'm trying to get some coding skills, and happened across Erlang. I'm fascinated but finding the learning curve much more unforgiving than Ruby, for instance... Joe's Ph. d thesis has been the most useful for me too so far in getting a big picture idea about how things work in Erlang, but it's figuring out how to go from basics to real world that is absolutely boggling my mind at the moment. :) On 1/3/11 9:00 PM, Bob Ippolito wrote: > Do you know what it is you'd like to do with Erlang? > > In my case I knew what I wanted to do before I started (build an ad > server), and I knew how to do it without Erlang, so taking the basics > and applying it to a real world problem was pretty straightforward for > me. > > The thing that finally made me "get" Erlang was Joe Armstrong's Ph.d > thesis [1], but at the time there were no books in print on Erlang so > there are probably better resources these days :) > > [1] http://www.erlang.org/download/armstrong_thesis_2003.pdf > > On Tue, Jan 4, 2011 at 9:52 AM, JETkoten wrote: >> Hi All, >> >> I've been reading/watching everything I can find (off and online) for >> learning Erlang for the past two or three weeks, but am just not getting it, >> in terms of being able to do anything with Erlang in the real world. >> >> I've watched some videos about functional programming where the hosts joked >> about people looking into functional programming deciding it's too hard and >> then never touching it again... but seriously, how do people bridge the gap >> between understanding the basics and implementing real world projects with >> Erlang? I know the three major books are available, but none of them are >> really using Erlang for what I'd like to do... >> >> How have any of you who have come to know big-picture how great Erlang is >> been able to get enough knowledge of it to actually do what you'd like to >> do? >> >> Any and all replies are most welcome. >> >> Thanks, >> Jack >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> From g9414002.pccu.edu.tw@REDACTED Tue Jan 4 03:22:37 2011 From: g9414002.pccu.edu.tw@REDACTED (=?UTF-8?B?6buD6ICA6LOiIChZYXUtSHNpZW4gSHVhbmcp?=) Date: Tue, 4 Jan 2011 10:22:37 +0800 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D22808F.6030202@ghostgun.com> References: <4D227D75.1060108@gmail.com> <4D22808F.6030202@ghostgun.com> Message-ID: Hi, all. Do you think Erlang is not well wrapped and promoted? For examples for well introduced gadgets: PHP and Ruby on Rail is a Web development framework. Java is a programming language with OO concepts support and VM environment. ColdFusion is a Client/Server web development framework with a preporcessing language. On Erlang, we know it's generic, and it's functional. But it's so hard for us to say: "Boss, I decide to use Erlang in my next plan." Why? How to introduce YAWS/Erlang in a office using Windows/IIS? On Tue, Jan 4, 2011 at 10:06 AM, jm wrote: > Would you like to post a small snippet of code in Ruby and someone of the > list could propose an equivalent piece of Erlang? > > > Jeff. > > > On 4/01/11 12:52 PM, JETkoten wrote: > >> Hi All, >> >> I've been reading/watching everything I can find (off and online) for >> learning Erlang for the past two or three weeks, but am just not getting it, >> in terms of being able to do anything with Erlang in the real world. >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Best Regards. --- Y-H. H. From bob@REDACTED Tue Jan 4 03:37:27 2011 From: bob@REDACTED (Bob Ippolito) Date: Tue, 4 Jan 2011 10:37:27 +0800 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D228213.8050502@gmail.com> References: <4D227D75.1060108@gmail.com> <4D228213.8050502@gmail.com> Message-ID: Well it sounds like the scraping you're trying to do isn't something well supported by the open source libraries currently available for Erlang. This means you'll either have to use some other tools (e.g. in a port process, or spawned in some other way) or you'll have to write these libraries yourself in Erlang. Something like Ruby might be a better choice to actually do the scraping, just because you won't have to code everything from scratch. On Tue, Jan 4, 2011 at 10:12 AM, JETkoten wrote: > Thanks for your reply. Yes, I've got my pet project in mind to auto-monitor > and update an online book sales marketplace. > > I know the tools to use to make it happen with Ruby and had a prototype with > some of the core functionality started (with some help I got from a local > user group member). ?Overall though I do lack experience there too, and > don't quite know how to do it without Erlang per se. > > I did a CS degree about 7 years ago, but didn't ever really learn to code > there. I focused more on OO analysis and design and software requirements > engineering... along with distributed digital multimedia. Now I'm trying to > get some coding skills, and happened across Erlang. I'm fascinated but > finding the learning curve much more unforgiving than Ruby, for instance... > > Joe's Ph. d thesis has been the most useful for me too so far in getting a > big picture idea about how things work in Erlang, but it's figuring out how > to go from basics to real world that is absolutely boggling my mind at the > moment. :) > > On 1/3/11 9:00 PM, Bob Ippolito wrote: >> >> Do you know what it is you'd like to do with Erlang? >> >> In my case I knew what I wanted to do before I started (build an ad >> server), and I knew how to do it without Erlang, so taking the basics >> and applying it to a real world problem was pretty straightforward for >> me. >> >> The thing that finally made me "get" Erlang was Joe Armstrong's Ph.d >> thesis [1], but at the time there were no books in print on Erlang so >> there are probably better resources these days :) >> >> [1] http://www.erlang.org/download/armstrong_thesis_2003.pdf >> >> On Tue, Jan 4, 2011 at 9:52 AM, JETkoten ?wrote: >>> >>> Hi All, >>> >>> I've been reading/watching everything I can find (off and online) for >>> learning Erlang for the past two or three weeks, but am just not getting >>> it, >>> in terms of being able to do anything with Erlang in the real world. >>> >>> I've watched some videos about functional programming where the hosts >>> joked >>> about people looking into functional programming deciding it's too hard >>> and >>> then never touching it again... but seriously, how do people bridge the >>> gap >>> between understanding the basics and implementing real world projects >>> with >>> Erlang? I know the three major books are available, but none of them are >>> really using Erlang for what I'd like to do... >>> >>> How have any of you who have come to know big-picture how great Erlang is >>> been able to get enough knowledge of it to actually do what you'd like to >>> do? >>> >>> Any and all replies are most welcome. >>> >>> Thanks, >>> Jack >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> > From xiaopong.tran@REDACTED Tue Jan 4 03:45:07 2011 From: xiaopong.tran@REDACTED (Xiaopong Tran) Date: Tue, 04 Jan 2011 10:45:07 +0800 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D227D75.1060108@gmail.com> References: <4D227D75.1060108@gmail.com> Message-ID: <1294109107.19912.12.camel@shanghai> I'd say it would come with practice. For whatever language you learn, be it programming language or natural language, you'll never really "get" it until you do a lot of practice. That means, writing codes. It's like standing on the side line and watching people play football (soccer for the Americans), you'll never get the fun until you go into the field, run after the ball, kick the ball, practice and get your ankle hurt badly (yeah, like having a system core dump in front of you). Watching on the side line will never let you get it why people are having fun running after a $10 ball and get themselves cover with mud and get hurt too (I'm not talking about professional football, they probably play for very different reasons, but I don't really watch them). On Mon, 2011-01-03 at 20:52 -0500, JETkoten wrote: > Hi All, > > I've been reading/watching everything I can find (off and online) for > learning Erlang for the past two or three weeks, but am just not getting > it, in terms of being able to do anything with Erlang in the real world. > > I've watched some videos about functional programming where the hosts > joked about people looking into functional programming deciding it's too > hard and then never touching it again... but seriously, how do people > bridge the gap between understanding the basics and implementing real > world projects with Erlang? I know the three major books are available, > but none of them are really using Erlang for what I'd like to do... > > How have any of you who have come to know big-picture how great Erlang > is been able to get enough knowledge of it to actually do what you'd > like to do? > > Any and all replies are most welcome. > > Thanks, > Jack > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From therevoltingx@REDACTED Tue Jan 4 03:53:22 2011 From: therevoltingx@REDACTED (Miguel Morales) Date: Mon, 3 Jan 2011 18:53:22 -0800 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D228213.8050502@gmail.com> References: <4D227D75.1060108@gmail.com> <4D228213.8050502@gmail.com> Message-ID: I think Erlang, functional programming, and multi-process programming are complicated enough to take more than a few weeks to learn. It's great that you have your pet project in mind, that's how I got started (I built a 2D MMO). Just dive into the code. Start to setup your environment, makefiles, etc. All of this will make you familiar with the Erlang environment. Then just start with baby steps, and start building your application step by step. Perhaps start by being able to reach your url and pulling some JSON or perhaps start a process that scrapes the info and saves it somewhere. Your first version will probably contain some errors and performance mistakes, these will be fun learning experiences later. I've been using it for over a year now and I've just barely begun to use behaviours and other more 'advanced' Erlang features. However I was able to build my pet project as a learning experience and implement whatever more advanced features as I learn them. On Mon, Jan 3, 2011 at 6:12 PM, JETkoten wrote: > Thanks for your reply. Yes, I've got my pet project in mind to auto-monitor > and update an online book sales marketplace. > > I know the tools to use to make it happen with Ruby and had a prototype with > some of the core functionality started (with some help I got from a local > user group member). ?Overall though I do lack experience there too, and > don't quite know how to do it without Erlang per se. > > I did a CS degree about 7 years ago, but didn't ever really learn to code > there. I focused more on OO analysis and design and software requirements > engineering... along with distributed digital multimedia. Now I'm trying to > get some coding skills, and happened across Erlang. I'm fascinated but > finding the learning curve much more unforgiving than Ruby, for instance... > > Joe's Ph. d thesis has been the most useful for me too so far in getting a > big picture idea about how things work in Erlang, but it's figuring out how > to go from basics to real world that is absolutely boggling my mind at the > moment. :) > > On 1/3/11 9:00 PM, Bob Ippolito wrote: >> >> Do you know what it is you'd like to do with Erlang? >> >> In my case I knew what I wanted to do before I started (build an ad >> server), and I knew how to do it without Erlang, so taking the basics >> and applying it to a real world problem was pretty straightforward for >> me. >> >> The thing that finally made me "get" Erlang was Joe Armstrong's Ph.d >> thesis [1], but at the time there were no books in print on Erlang so >> there are probably better resources these days :) >> >> [1] http://www.erlang.org/download/armstrong_thesis_2003.pdf >> >> On Tue, Jan 4, 2011 at 9:52 AM, JETkoten ?wrote: >>> >>> Hi All, >>> >>> I've been reading/watching everything I can find (off and online) for >>> learning Erlang for the past two or three weeks, but am just not getting >>> it, >>> in terms of being able to do anything with Erlang in the real world. >>> >>> I've watched some videos about functional programming where the hosts >>> joked >>> about people looking into functional programming deciding it's too hard >>> and >>> then never touching it again... but seriously, how do people bridge the >>> gap >>> between understanding the basics and implementing real world projects >>> with >>> Erlang? I know the three major books are available, but none of them are >>> really using Erlang for what I'd like to do... >>> >>> How have any of you who have come to know big-picture how great Erlang is >>> been able to get enough knowledge of it to actually do what you'd like to >>> do? >>> >>> Any and all replies are most welcome. >>> >>> Thanks, >>> Jack >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- ~ Jeremiah:9:23-24 Android 2D MMORPG: http://developingthedream.blogspot.com/, http://www.youtube.com/user/revoltingx From gnoblin@REDACTED Tue Jan 4 04:27:50 2011 From: gnoblin@REDACTED (Slav Pankratov) Date: Tue, 4 Jan 2011 05:27:50 +0200 Subject: game development & erlang Message-ID: Hello! Is anyone on the list involved in game development with Erlang? :) (I've seen a mention of 2d Android online game in a recent discussion and decided to ask) thanks, Slav From therevoltingx@REDACTED Tue Jan 4 05:18:13 2011 From: therevoltingx@REDACTED (Miguel Morales) Date: Mon, 3 Jan 2011 20:18:13 -0800 Subject: [erlang-questions] game development & erlang In-Reply-To: References: Message-ID: Hi, I am the developer of that game. Although right now you need an Android device to play it, this year I want to add a web client. I remember an old thread in this list that you might want to read over: http://www.trapexit.org/forum/viewtopic.php?p=56870 On Mon, Jan 3, 2011 at 7:27 PM, Slav Pankratov wrote: > Hello! > > Is anyone on the list involved in game development with Erlang? :) > (I've seen a mention of 2d Android online game in a recent discussion and > decided to ask) > > thanks, > Slav > -- ~ Jeremiah:9:23-24 Android 2D MMORPG: http://developingthedream.blogspot.com/, http://www.youtube.com/user/revoltingx From serge@REDACTED Tue Jan 4 06:31:24 2011 From: serge@REDACTED (Serge Aleynikov) Date: Tue, 04 Jan 2011 00:31:24 -0500 Subject: [erlang-questions] [ANN] eixx - C++ interface to Erlang In-Reply-To: References: <4D228094.5030805@aleynikov.org> Message-ID: <4D22B0AC.2020301@aleynikov.org> Hi Steve, Yes it does. In this project I am using boost's ASIO for asynchronous I/O (build around epoll/select/poll depending on the platform), which has full support for non-blocking I/O operations. The combination of that with deadline timers allowed implementing support for asynchronous receipt of messages in multiple mailboxes in the context of a single thread. Unfortunately ei is not very friendly on establishing distributed node connections in non-blocking way, so I had to copy/paste some code to make it work property. Initially I thought of patching ei, but at some point gave up on the idea because it involved to many API changes, and just reused the handshaking protocol-related functions. The project currently only supports client-initiated connections (i.e. C++ node connecting to an Erlang node). If you are coding in C++ rather than C, you'll find it quite a time saver - the use of polymorphic template-based Erlang terms along with reference counting, and pluggable memory allocators gives a rather efficient way of manipulating terms. E.g.: eterm term = tuple::make(atom("ok"), 1, 10.0, "string"); std::cout << term << std::endl; // Prints: {ok, 1, 10.0, "string"} std::cout << term.encode(/* header size */ 0) << std::endl; // Prints: // <<131,104,4,100,0,2,111,107,97,1,70,64,36,0,0,0,0,0,0,107, // 0,6,115,116,114,105,110,103>> Regards, Serge On 1/3/2011 10:01 PM, Steve Vinoski wrote: > Hi Serge, sounds interesting. Does it have a proper event loop, unlike ei? > > --steve > > On Mon, Jan 3, 2011 at 9:06 PM, Serge Aleynikov wrote: >> http://github.com/saleyn/eixx >> >> eixx - Erlang C++ Interface Library >> =================================== >> >> This library provides a set of classes for convenient marshaling >> of Erlang terms between processes as well as connecting to other >> distributed Erlang nodes from a C++ application. >> >> The marshaling classes are built on top of ei library included in >> {@link http://www.erlang.org/doc/apps/erl_interface erl_interface}. >> It is largely inspired by the {@link http://code.google.com/p/epi epi} >> project, but is a complete rewrite with many new features and >> optimization enhancements. >> >> This library adds on the following features: >> - Use of reference-counted smart pointers for complex terms and >> by-value copied simple terms (e.i. integers, doubles, bool, atoms). >> - Ability to provide custom memory allocators. >> - Encoding/decoding of nested terms using a single function call >> (eterm::encode() and eterm::eterm() constructor). >> - Global atom table for fast manipulation of atoms. >> >> The library consists of two separate parts: >> - Term marshaling (included by marshal.hpp or eixx.hpp) >> - Distributed node connectivity (included by connect.hpp or eixx.hpp) >> >> The connectivity library implements a richer set of features than >> what's available in erl_interface - it fully supports process >> linking and monitoring. The library is fully asynchronous and allows >> handling many connections and mailboxes in one OS thread. >> >> Ths library is dependend on {@link http://www.boost.org BOOST} >> project and erl_interface, which is a part of the >> {@link www.erlang.org Erlang} distribution. >> >> Regards, >> >> Serge >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> From jesper.louis.andersen@REDACTED Tue Jan 4 08:50:04 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Tue, 4 Jan 2011 08:50:04 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D228213.8050502@gmail.com> References: <4D227D75.1060108@gmail.com> <4D228213.8050502@gmail.com> Message-ID: On Tue, Jan 4, 2011 at 03:12, JETkoten wrote: > I did a CS degree about 7 years ago, but didn't ever really learn to code > there. I focused more on OO analysis and design and software requirements > engineering... along with distributed digital multimedia. Now I'm trying to > get some coding skills, and happened across Erlang. I'm fascinated but > finding the learning curve much more unforgiving than Ruby, for instance... You are trying to do these things all at once: * Get a good idea of what the functional programming paradigm is. * Get a good idea of what the concurrency primitives of Erlang looks like. * Understanding how to do web development with Erlang. * Perhaps understand what OTP is. * Having a specific project you want to implement on top of all of the above. I am with Bob on having a project as a driver for the study. But you must accept that the above things, taken all at the same time, means it will take longer in the beginning. I'd suggest that you don't necessarily build to succeed in the first place. Build to learn. And then build to succeed. If you have no prior exposure at all to functional programming, that is where your focus should be in the beginning. The learning curve feels steep because there are so much new stuff to take in. However, here is the secret: Erlang is a very simple language. So as soon as you have worked a bit on the curve, you are almost over it and from there on out, the climb will be much less steep. Believe me, it is the first hurdle that is the hardest one. -- J. From rapsey@REDACTED Tue Jan 4 08:58:23 2011 From: rapsey@REDACTED (Rapsey) Date: Tue, 4 Jan 2011 08:58:23 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D227D75.1060108@gmail.com> References: <4D227D75.1060108@gmail.com> Message-ID: You should really read one of the books about Erlang, not rely on online tutorials and guides. You will learn much faster. I had a pretty good grasp on how to write erlang programs within the first week of starting to read Programming Erlang. Are you familiar with http://learnyousomeerlang.com/ ? Sergej On Tue, Jan 4, 2011 at 2:52 AM, JETkoten wrote: > Hi All, > > I've been reading/watching everything I can find (off and online) for > learning Erlang for the past two or three weeks, but am just not getting it, > in terms of being able to do anything with Erlang in the real world. > > I've watched some videos about functional programming where the hosts joked > about people looking into functional programming deciding it's too hard and > then never touching it again... but seriously, how do people bridge the gap > between understanding the basics and implementing real world projects with > Erlang? I know the three major books are available, but none of them are > really using Erlang for what I'd like to do... > > How have any of you who have come to know big-picture how great Erlang is > been able to get enough knowledge of it to actually do what you'd like to > do? > > Any and all replies are most welcome. > > Thanks, > Jack > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From robert.virding@REDACTED Tue Jan 4 09:20:03 2011 From: robert.virding@REDACTED (Robert Virding) Date: Tue, 4 Jan 2011 08:20:03 +0000 (GMT) Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: Message-ID: <309876917.239711294129202978.JavaMail.root@zimbra> Sergej, When I read your first sentence I thought that I must suggest "Learn You Some Erlang", but the I saw that you did that as well. But then LYSE is neither a tutorial nor a guide but a book, and definitely worth reading and an alternative to buying a book. And it keeps growing all the time as Fred adds more chapters. Robert ----- "Rapsey" wrote: > You should really read one of the books about Erlang, not rely on > online > tutorials and guides. You will learn much faster. I had a pretty good > grasp > on how to write erlang programs within the first week of starting to > read > Programming Erlang. > Are you familiar with http://learnyousomeerlang.com/ ? > > > Sergej > > > On Tue, Jan 4, 2011 at 2:52 AM, JETkoten wrote: > > > Hi All, > > > > I've been reading/watching everything I can find (off and online) > for > > learning Erlang for the past two or three weeks, but am just not > getting it, > > in terms of being able to do anything with Erlang in the real > world. > > > > I've watched some videos about functional programming where the > hosts joked > > about people looking into functional programming deciding it's too > hard and > > then never touching it again... but seriously, how do people bridge > the gap > > between understanding the basics and implementing real world > projects with > > Erlang? I know the three major books are available, but none of them > are > > really using Erlang for what I'd like to do... > > > > How have any of you who have come to know big-picture how great > Erlang is > > been able to get enough knowledge of it to actually do what you'd > like to > > do? > > > > Any and all replies are most welcome. > > > > Thanks, > > Jack > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > From erlang@REDACTED Tue Jan 4 10:17:22 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 4 Jan 2011 10:17:22 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D227D75.1060108@gmail.com> References: <4D227D75.1060108@gmail.com> Message-ID: On Tue, Jan 4, 2011 at 2:52 AM, JETkoten wrote: > Hi All, > > I've been reading/watching everything I can find (off and online) for > learning Erlang for the past two or three weeks, but am just not getting it, > in terms of being able to do anything with Erlang in the real world. > > I've watched some videos about functional programming where the hosts joked > about people looking into functional programming deciding it's too hard and > then never touching it again... No no no no no ..... Somehow you've gotten on the wrong foot here. Functional programming is *much* easier to understand than imperative programming. There is a sense in which *all* programming languages are difficult to understand, this is the phase where you are not familiar with the syntax and what the various construct mean. Take "C" for example - this is the Lingua Franca of all programming languages - we all know C. So let's do an experiment. Stop a C programmer in the corridor and ask them the difference between: int *p[10]; and int (*p)[10]; One is "a pointer to (an array of 10 integers)" the other is "an array of (ten pointers to integers)" Then ask them what char * const *(*next)(); means (next is a pointer to a function returning a pointer to a constant pointer to a char") A C program is actually extrememly difficult to understand - the above problem is purely syntactic, then the problem of understanding a C program is much deeper - you have to mentally execute the program in your head to understand what it means - this is due to side effects. Functional programs are really really easy to understand, since given that you understand what the inputs to a function are the fucntion will always, always always, forever and ever and ever return the same value. This make the code extremely reusable. if you are not familiar with the erlang syntax your eyes will not see what my eyes see. When I see code I mentally scan for the use of variables, I look for the first occurrence and subsequent occurrences. Imagine some function foo, that uses a variable X, when I look for the uses of X in foo my brain "sees" this: foo(....) -> ... X = .... <--- 1 ... ... bar(...X...) ... <--- 2 {...,...,X,....} = ..... <--- 3 I now know that X is defined at point 1 in the program. That it is used in point 2 and that in point 3 that whatever is on the right hand side of the equals in line 3 must return a tuple whose 3'rd argument must be X The fact that X never changes once it is set make the program extremely easy to understand, and debug. If a value is incorrect, it can only set in one place. You just peer at the program, you don't need a debugger. (in C for example, a value once set, might be changed a trillion times, and one one occasion when it is changed it gets the wrong value, but which of these times was it? - this is *extremely* difficult to understand, and a debugger might help) I really think that FPLs are an order of magnitude easier to understand than conventional PLs - (don't get me started on languages like java - is Java easy compared to erlang? - try writing the simplest of callback functions in java and tell me java is easy to understand - what's the visitor pattern like in java? read http://en.wikipedia.org/wiki/Visitor_pattern then read the comment about the "Visitor Pattern in a Dynamic programming language" as the wikipedia says: "The visitor pattern disappears completely. No extra classes and methods are needed" ) > but seriously, how do people bridge the gap > between understanding the basics and implementing real world projects with > Erlang? I know the three major books are available, but none of them are > really using Erlang for what I'd like to do... This is what you do - buy one of the books - modestly prevents me from telling you which is best - then type in the programs staring at the simplest. It's very important that you type in the programs *yourself* - don't download the code and run it. You have to get the syntax into your spine not your brain. When you type in the programs, you'll make tiny errors, missing commas etc. Fix these error - this is the tricky bit. Every programming language under the sun produces totally bizarre error message when you get the punctuation wrong. (aside Image you're new to C and write this int add(int a; int b){ return a+b; } gcc says test3.c:1: error: parameter ?a? has just a forward declaration test3.c: In function ?add?: test3.c:2: error: ?a? undeclared (first use in this function) Which is it's helpful way of saying - "sorry Joe it should be a comma in the function arguments not a semicolon" ) all error messages are difficult to understand. The purpose of a book is to present you with small completely understandable self-contained examples that you can type in and learn from. It takes a few months to become fluent in any programming language - reckon a couple of weeks for the syntax - then a few months before you know your way around the libraries /Joe > > How have any of you who have come to know big-picture how great Erlang is > been able to get enough knowledge of it to actually do what you'd like to > do? > > Any and all replies are most welcome. > > Thanks, > Jack > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From connorsml@REDACTED Tue Jan 4 10:45:29 2011 From: connorsml@REDACTED (Michael Connors) Date: Tue, 4 Jan 2011 10:45:29 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D228213.8050502@gmail.com> References: <4D227D75.1060108@gmail.com> <4D228213.8050502@gmail.com> Message-ID: On 4 January 2011 03:12, JETkoten wrote: > Thanks for your reply. Yes, I've got my pet project in mind to auto-monitor > and update an online book sales marketplace. > > I considered learning Erlang for a while, and went through Joe Armstrong's book and learned how to write some basic Erlang examples, but as for writing production code or starting a project from scratch; I would have been lost. There are a lot of new things to learn beyond the syntax. You need to learn how to structure a project, learn how to think as a functional programmer, learn new data storage mechanisms and become aware of what has been written for you, and what you will need to write for yourself. I wanted to do web-development initially, like yourself (I think). I choose a web-framework, in my case Zotonic http://zotonic.com and became familiar with it's codebase and started to write simple modules to extend it. This allowed me to get started quickly and do useful things straight away. You can find a comparison of web-frameworks here: http://www.chicagoboss.org/compare.html -- Michael Connors Alencon, France http://www.michaelconnors.net From erlang@REDACTED Tue Jan 4 10:52:50 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 4 Jan 2011 10:52:50 +0100 Subject: [erlang-questions] Processes & Fault Tolerance In-Reply-To: References: Message-ID: On Mon, Jan 3, 2011 at 5:30 PM, Edmond Begumisa wrote: >> Let's start with how we do error recovery. >> >> Imagine two linked processes A and B on separate machines. A is the >> master process. >> B is a passive processes that will take over if A fails. >> >> A sends a stream of state update messages S1, S2, S3,... to B. The >> state messages contain enough information >> for B to do whatever A was doing should A fail. If A fails B will >> receive an EXIT signal. >> >> If B does receive an exit signal it knows A has failed, so it carries >> on doing whatever A was doing >> using the information it last received in a state update message. >> >> That's it and all about it - nothing to do with supervisors etc, > > Ooooohh! THANK YOU, THANK YOU, THANK YOU. Thanks. Another problem in understand is that we assume the meaning of words. Once upon a time I was talking to a guy and used the word "fault tolerance" we talked for hours and thought we understood what this word meant. Then he said something that implied he was building a fault-tolerant system on one computer. I said it was impossible. He said, "you catch and handle all the exceptions .." I said, "The entire computer might crash" He said, "Oh" There was a long silence. I told him about Erlang ... /Joe > > *Now* I get it. This is precisely what I was trying to understand. The > missing link was sending redundant messages. It all makes sense now -- it's > so simple. All I have to do to get fault tolerance at the process level is > have a group of N redundant processes waiting for exit-signals and forward > state changes to N-1 of them. I could even send to N/2 and have some decent > tolerance at the expense of consistency of the nodes. > > To preemptively answer the question I sent Ulf and Mazen... for that > distributed database, Process B would be set up to also receive read > requests forwarded from Process A but not respond to them unless it gets an > exit signal it can understand like disk_fail_error. Any changes in state > would also be forwarded to existing redundant processes. > > Thanks everyone for your patience in helping me understand fault-tolerance. > I've already started thinking about some cool things I can do with this in > my program. > > - Edmond - > > > > On Tue, 04 Jan 2011 00:36:30 +1100, Joe Armstrong wrote: > >> I think you are getting confused between OTP supervisors etc. and the >> general notion of "take-over" >> >> Let's start with how we do error recovery. >> >> Imagine two linked processes A and B on separate machines. A is the >> master process. >> B is a passive processes that will take over if A fails. >> >> A sends a stream of state update messages S1, S2, S3,... to B. The >> state messages contain enough information >> for B to do whatever A was doing should A fail. If A fails B will >> receive an EXIT signal. >> >> If B does receive an exit signal it knows A has failed, so it carries >> on doing whatever A was doing >> using the information it last received in a state update message. >> >> That's it and all about it - nothing to do with supervisors etc, >> >> >> To make the above possible we need a couple of primitives in Erlang, >> namely process_flag trap_exits, and spawn_link. That's all you need. >> >> The basic system properties you need to make a fault-tolerant systems >> are the abilities to detect remote errors and send and receive messages. >> >> What the OTP supervisors etc. do are build layers of abstraction on >> top of spawn-link and trap_exits >> in order to simplify programming common use-cases. >> >> Fault detection and recovery is not arbitrary or magic in any sense, >> It has to be part of the system architecture like everything else. One >> common fault in making systems is to specify and design for the normal >> behavior >> but not specify and design how fault-detection and correction work. >> This is probably due to a legacy of >> programming in languages that have very limited ability to detect >> errors and recover from them. >> >> A C programmer with a single thread of control will take extreme >> measures to avoid crashing their program >> they have to, ?there is only one thread of control. Given multiple >> threads of control and the ability to remotely >> detect errors your entire way of thinking changes and the emphasis >> changes from thinking "how to I avoid a crash" >> to thinking "given that a crash has occurred and been detected, how to >> I fix things up and carry on" >> >> The Erlang philosophy is that "things will crash", so we have to >> detect the crashes, fix the bit that broke and >> carry on - we do not make excessive efforts to avoid crashes, but we >> do try very hard to repair things after they have gone wrong. >> >> Fixing things after they have broken is often easier than preventing >> them from breaking. Preventing them >> from breaking is not theoretically possible - we can't even prove >> simple things, like that a program will terminate >> (the famous halting problem) - but we can easily detect failures and >> put the system to rights by applying >> certain invariants. >> >> OTP supervisors are just trees of processes with certain restart rules >> that experience has shown us >> to be useful. >> >> The main value of supervisors etc. (and of all the OTP behaviors) is >> in building large systems with many >> programmers involved. Rather than inventing their own patterns, the >> programmers reuse a common set of >> patterns, which makes the projects easier to manage. >> >> /Joe >> >> >> >> >> On Mon, Jan 3, 2011 at 6:21 AM, Edmond Begumisa >> wrote: >>> >>> Thanks for your response. >>> >>> Firstly, let me make my question a little clearer... >>> >>> To rephrase: For processes, "share nothing for the sake of concurrency" - >>> I >>> get, both in concept and application. "Share nothing for the sake of >>> fault-tolerance" - I get in concept but not in application. >>> >>> Yet as I understand it, it is for the latter reason Erlang shares >>> nothing* >>> and not the former. Interpretation: I must be completely missing the >>> point >>> in regards to Erlang processes and sharing nothing. This is what I want >>> to >>> understand in application. In addition to the "side" effect of sane >>> concurrency (which coming from a chaotic multi-threading shared-memory >>> world >>> I fully appreciate and practically make use of everyday), how can I also >>> make use of the "real" reason Erlang processes share nothing -- fault >>> tolerance? Practically/illustratively speaking? >>> >>> *ETS being the obvious exception. >>> >>> Secondly, here's a mantra from Joe Armstrong... >>> >>> @ minute 17:26 >>> http://www.se-radio.net/2008/03/episode-89-joe-armstrong-on-erlang/ >>> >>> "[message passing concurrency]... the original reasons have to do with >>> fault >>> tolerance... you have to copy all the data you need from computer 1 to >>> computer 2... if computer 1 crashes you take over on computer 2... you >>> can't >>> have dangling pointers... that's the reason for copying everything... >>> it's >>> got nothing to do with concurrency, it's got a lot to do with >>> fault-tolerance... if they don't crash you could just have a dangling >>> pointer and copy less data but it won't work in the presence of >>> errors..." >>> >>> I interpret this to mean that share-nothing between processes is more >>> about >>> replicating valid state than isolating corrupted state as you described. >>> >>> Indeed, Joe created an example on his blog... >>> >>> >>> http://armstrongonsoftware.blogspot.com/2007/07/scalable-fault-tolerant-upgradable.html >>> >>> It's algorithm 3 there I'm struggling with. Particularly where he says... >>> >>> "... In practise we would send an asynchronous stream of messages from N >>> to >>> N+1 containing enough information to recover if things go wrong." >>> >>> Unfortunately, I couldn't find part II to that post (I don't think there >>> is >>> one.) And I'm too green and inexperienced in the field of fault-tolerant >>> systems to figure it out on my own. I'm having trouble visualising the >>> practical here from the conceptual -- I need to be shown how :( >>> >>> Also, I seem to be under the impression that the Erlang language has some >>> sort of schematics to do this built-in (i.e. deal with one process taking >>> over from another if the first fails) and this is the reason processes >>> share >>> nothing. This seems to me to be something different from supervision >>> trees, >>> which use exit-trapping to re-spawn if a process fails with the active >>> 'job' >>> disappearing and any errors logged (like restarting a daemon). My >>> interpretation of the fault-tolerance Erlang is supposed to enable (for >>> those in the know) is seamless take-over. The 'job' lives on but >>> elsewhere. >>> >>> Using telecoms as an example: a phone call wouldn't be cut-off when a >>> fault >>> occurs, another node would seamlessly take over. This is how I >>> interpreted >>> Joe's post and other descriptions of Erlang's fault-tolerant features and >>> I >>> understand the key is in the share nothing policy for processes. I'm sure >>> I've mis-understood something or everything :) >>> >>> - Edmond - >>> >>> PS: I've read the Manning draft. Great book. I don't know if the answer >>> lies >>> in OTP (I searched and didn't find it). I suspect it's lower -- probalby >>> how >>> you organise your processes. Some distributed-programming black-magic >>> only >>> Erlanger's know about :) >>> >>> >>> On Mon, 03 Jan 2011 14:56:51 +1100, Alain O'Dea >>> wrote: >>> >>>> On 2011-01-02, at 22:36, "Edmond Begumisa" >>>> wrote: >>>> >>>>> Slight correction... >>>>> >>>>> On Mon, 03 Jan 2011 12:38:38 +1100, Edmond Begumisa >>>>> wrote: >>>>> >>>>>> Hello all, >>>>>> >>>>>> I've been trying to wrap my Erlang's fault tolerant features >>>>>> particularly in relation to processes. >>>>>> >>>>> >>>>> Should be: I've been trying to wrap my head around Erlang's fault >>>>> tolerant features particularly in relation to processes. >>>>> >>>>> Sorry. >>>>> >>>>>> I've heard/read repeatedly that the primary reason why Erlang's >>>>>> designers opted for a share-nothing policy is not rooted in >>>>>> concurrency but >>>>>> rather in fault-tolerance. When nothing is shared, everything is >>>>>> copied. >>>>>> When everything is copied processes can take over from one another >>>>>> when >>>>>> things fail. I follow this reasoning but I don't follow how to apply >>>>>> it. >>>>>> >>>>>> I fully understand and appreciate how supervision trees are used to >>>>>> restart processes if they fail. What I don't get is what to do when >>>>>> you >>>>>> don't want to restart but want to take over, say on another node. I >>>>>> know >>>>>> that at a higher-level, OTP has some take-over/fail-over schematics >>>>>> (at the >>>>>> application level.) I'm trying to understand things at the processes >>>>>> level - >>>>>> why Erlang is the way it is so I can better use it to make my >>>>>> currently >>>>>> fault-intolerant program fault tolerant. >>>>>> >>>>>> Specifically, how can one process take over from another if it fails? >>>>>> It >>>>>> appears to may that the only way to do this would be to somehow >>>>>> retrieve not >>>>>> only the state of the process (say, gen_server's state) but also the >>>>>> messages in its mailbox. Where does the design decision to >>>>>> share-nothing for >>>>>> the sake of fault-tolerance come into play for processes? Please help >>>>>> me >>>>>> "get" this! >>>>>> >>>>>> Thanks in advance. >>>>>> >>>>>> - Edmond - >>>>>> >>>>>> >>>> >>>> Hi Edmond: >>>> >>>> Share-nothing helps with concurrent fault-tolerance by preventing one >>>> process from corrupting the state of another. Receive is a process' >>>> choice >>>> and it corrupts its own state if it receives bad data and lets it in. >>>> >>>> AFAIK OTP fault-tolerance doesn't mean no requests will fail, it means >>>> the >>>> system/sub-system will recover if a single request causes a process to >>>> crash. ?It's kind of like proper try/catch recovery applied to >>>> concurrent >>>> code. ?How you recover from the crash depends on the supervision >>>> strategy >>>> chosen. ?In some cases the supervisor can pass the state to the >>>> replacement >>>> process. In others this isn't necessary or even desirable since the >>>> state >>>> itself may involve resources lost in the crash or corrupted state that >>>> led >>>> to the crash. >>>> >>>> I am straying outside my knowledge here so this paragraph is guesswork. >>>> ?The message queue for a gen_server need not necessarily be lost when >>>> the >>>> callback module crashes. ?In theory OTP could (and might already) simply >>>> delegate the messages to the replacement process following a crash. >>>> ?Someone >>>> who knows OTP better than me would need to weigh in here though. >>>> >>>> I found http://manning.com/logan very informative in understanding OTP >>>> and >>>> its supervisor hierarchies. >>>> >>>> Cheers, >>>> Alain >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>> >>> >>> -- >>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > -- > Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > From hynek@REDACTED Tue Jan 4 11:21:21 2011 From: hynek@REDACTED (Hynek Vychodil) Date: Tue, 4 Jan 2011 11:21:21 +0100 Subject: [erlang-questions] SSL memory consumption In-Reply-To: References: Message-ID: On Thu, Dec 30, 2010 at 6:32 AM, Kaiduan Xie wrote: > After tried two different ways to call ssl:transport_accept and > ssl:ssl_accept, the following observation was found, > > 1. Calls ssl:transport_accept and ssl:ssl_accept in a server loop, and > for each SSL connection, starts a gen_fsm to receive data. The code is > listed as below, > > loop3(LSock) -> > ? ?{ok, Socket} = ssl:transport_accept(LSock), > ? ?ok = ssl:ssl_accept(Socket), > ? ?{ok, {PeerAddr, PeerPort}} = ssl:peername(Socket), > ? ?io:format("Accepted peer:~p:~p:~p:~p~n", [self(),PeerAddr, PeerPort, I]), > ? ?{ok, Pid} = gen_fsm:start(erlsip_transport_tls_connection, > {Socket, PeerAddr, PeerPort}, []), > ? ?ssl:controlling_process(Socket, Pid), > ? ?loop3(LSock). > > 2. Calls ssl:transport_accept, and ssl:ssl_accept in a gen_fsm. Upon a > SSL connection is accepted, starts another gen_fsm to accept new SSL > connection. The code is listed as below. > > init({server, LSocket, Parent}) -> > ? ?{ok, wait_for_connection, #state{lsocket = LSocket, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? parent = Parent}, 0}; > wait_for_connection(timeout, State) -> > ? ?io:format("Waiting for TLS connection ... ~p/~p~n", [self(), State#state.par > ent]), > ? ?{ok, Socket} = ssl:transport_accept(State#state.lsocket), > ? ?ok = ssl:ssl_accept(Socket), > ? ?gen_fsm:start(erlsip_transport_tls_connection, > ? ? ? ? ? ? ? ? {server, State#state.lsocket, State#state.parent}, > ? ? ? ? ? ? ? ? ?[]), > ? ?{ok, {PeerAddr, PeerPort}} = ssl:peername(Socket), > ? ?io:format("Accepted peer:~p/~p:~p~n", [self(), PeerAddr, PeerPort]), > ? ?{next_state, wait_for_crlf_crlf, State#state{socket = Socket, > ? ? ? ? ? ? ? ? ? peer_address = PeerAddr, peer_port = PeerPort}}. > > Memory consumption per connection in the second way is about 120 K > with R14B01 while a connection takes about 30 K in first way. > > I am very happy with the result, but can someone explain the difference? > > Thanks, > > /Kaiduan I'm just guessing but it can be GC issue. In first case you do all those calls inside one process so all used structures inside loop3 can be GCed. In second case you do this sequence always in new process and when there are not future reductions (there is not any communication - fsm doesn't work) this memory is not freed. Try add explicit GC to your init/1 in second case and measure again. > On Thu, Dec 23, 2010 at 9:31 PM, Kaiduan Xie wrote: >> I just replaced R14A with R14B01, each SSL connection consumes about >> 120 K memory, 50K less than that of R14A. >> >> Thank Tony, Jesper for the help, can we reduce more? >> >> /Kaiduan >> >> On Thu, Dec 23, 2010 at 8:38 PM, Jesper Louis Andersen >> wrote: >>> On Fri, Dec 24, 2010 at 02:33, Kaiduan Xie wrote: >>> >>>> What do you mean the old ssl and new ssl, Tony? >>> >>> SSL was replaced recently with a new implementation and you have the >>> 'new' version here if you are on R14b >>> >>> -- >>> J. >>> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From mk@REDACTED Tue Jan 4 12:31:52 2011 From: mk@REDACTED (Morten Krogh) Date: Tue, 04 Jan 2011 12:31:52 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: References: <4D227D75.1060108@gmail.com> Message-ID: <4D230528.1000700@amberbio.com> On 1/4/11 10:17 AM, Joe Armstrong wrote: > No no no no no ..... Somehow you've gotten on the wrong foot here. > Functional programming is *much* easier to understand than imperative > programming. > Definitely not much easier. It depends on the problem, and whether you include performance requirements. As soon as you want to control memory and performance, functional languages can actually be much much harder. And I mean much harder. Why are one fourth, or whatever, of the posts on this list about nif or c nodes or ports? Even forgetting completely about performance, there are lots of problems that are more easily solved and written using imperative techniques. > Functional programs are really really easy to understand, since given > that you understand what the inputs to a function are the fucntion > will always, always always, forever and ever and ever return the same > value. > This make the code extremely reusable. > You have pure functions in all languages, even if the type system doesn't force it on you. I don't think it is such a big problem in C. You almost always understand whether a function has side effects or not. Library functions rarely do, except for possibly changing pointer arguments, which is documented in the api. But is Erlang so different here. I could write a sin function, that changed a file, say, during execution. The issue is the same in C and Erlang. Library writers should not do it and mostly don't. Do you feel this is a real problem in C? What are the functions that are reused in Erlang but not in C? > (in C for example, a value once set, might be changed a trillion > times, and one one occasion when > it is changed it gets the wrong value, but which of these times was > it? - this is *extremely* difficult > to understand, and a debugger might help) > Are you talking about a loop? Otherwise, a variable doesn't change that many times. In Erlang, you would have recursive function invocations. The problem would then just be to figure out in which invocation the error happened. The debugging of this problem would be similar in C and Erlang, given that you use side effects. A print statement in the loop or function is a side effect, which is very useful for this purpose. Imagine a language (Haskell?), where you can't insert a print statement anywhere you want to for debugging purposes, because the language is religious about avoiding side effects. Even mutating a local variable can be valuable for debugging purposes. > I really think that FPLs are an order of magnitude easier to > understand than conventional PLs - > What? An order of magnitude? Programming languages are actually still so similar, that the main flow of the program would be the same no matter what language you use. Getting the data structures and data flow right is usually the same problem. The syntax would then be somewhat shorter in Erlang than C or Java. But what about perl or python or ruby? And most lines of code in C are easy to understand: sum = 0; for (i = 0; i < 1000000; i++) { sum += i; } It is longer than a one liner in a FPL, but it is effort less to write. I think it is more correct to say that all languages have domains where they are strong and others where they are weak, but if memory and performance really matters, you actually need languages that are closer to the hardware, and they are all imperative at the moment. Cheers, Morten. From hd2010@REDACTED Tue Jan 4 12:54:04 2011 From: hd2010@REDACTED (Henning Diedrich) Date: Tue, 04 Jan 2011 12:54:04 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: References: <4D227D75.1060108@gmail.com> Message-ID: <4D230A5C.9080504@eonblast.com> Hi Jack, On Tue, Jan 4, 2011 at 9:52 AM, JETkoten wrote: > ... just not getting it, in terms of being able to do anything with Erlang in the real world. > > [snip] > > ... the gap between understanding the basics and implementing real world projects There's probably a difference between the big picture and learning to think and program functional. And then, learning to code in the first place. Erlang's strengths would dictate were you want to use it for in the real world. They may come from Erlang being functional. But at least some strengths are unrelated, built into OTP. Learning Erlang probably means "getting" different stuff then "getting" what Erlang/OTP is good for. And the learning isn't really going to be easier if you don't have a strong coding background already, because most literature will address you as someone coming from imperative languages and needing to be re-educated. But still building on that in a way. E.g. it's surprising how often you will run into Java on the way. And, of course, C. But you will probably *still* have to also unlearn some stuff about coding, no matter how little exposure you had to imperative languages before. if you are serious about coding, it will be a most valuable experience I think. > How For me it was going for a project, reading, hacking, re-reading, re-hacking. Reading samples and trying to adapt them on my way. Spending some time, as Tran wrote. The speed is way different from, say, using Ruby: some simple things take way longer because you have to learn how it works, language wise. but as Jesper said, it's a simple language. There are some things that you may try that you find are not how it works in Erlang at all. And then there are a lot of things that are already ready made and you don't have to do anything except learning to use them. So you may find you crawl where it's a breeze with Ruby, and other things are a breeze in Erlang/OTP where it's a pain in other ecosystems. I read a lot of language comparisons to get an idea. And literally let them lie and later read them again after some hacking. I started out with the free first chapter of Armstrong's book. I particularly liked http://learnyousomeerlang.com/, and also re- read that a while later to fill gaps. I bought Armstrong's book relatively late, only after having hacked a little, and I read and re-read ets and dets, mnesia and supervisor stuff a couple of times. I tried putting forth a benchmark that was interesting for me, too. At the same time I started right into what I wanted to program, and started from scratch again two times after understanding more, based on samples from the web. I tried to define simple modules and get them done, maybe not in perfect style, but working. Plus reading the list, including archives on what looked strange, and sometimes daring to get on somebodies nerves, helped a lot. There are a handful of things that at a certain point "you get" and then it 'starts to make sense.' I would dare say that if you keep writing stuff and have some urge to produce nice code, you will understand those things on your own. While you refactor you realize why what makes sense and most likely it will take a dozen iterations to get from your intuitively imperative approach to something that looks functional. But if you do that a couple of times, you'll do it right the first time after a while. I think it is difficult to rush. It's also the mind bending fun of it. In a way the things you can't do or will look ugly, guide you. The stuff that looks pretty wrong, even though it works, if you try to make it look right you will often see that it unravels your whole approach and if you follow that through, and allow that to happen you end up rewriting it all until your source looks right (i.e. does not do stuff that is more imperative than appropriate, not repetitive, hard to follow etc.) and on the way you learn what functional element replaces what imperative one. Like in OO polymorphic late binding replacing cases over types. Then tackle OTP, that's really a lot of vocabulary to learn and procedure. The least attractive part for me and still essential for productivity [1]. OTP is really inseparable from Erlang in a way. You can't do certain things the Erlang way and skip that. But that's quite separate from understanding functional programming. Since Erlang/OTP is not only functional, it's also enormously practical, plus other things that don't have too much to do with functionality really. Maybe they all come from that, in a way, but there is also a very strong desire to get things done visible in Erlang's DNA, and more so, in OTP. I'd look at Yaws [2], Erlyweb [3] and MochiWeb [4] for your stuff and start from there. I DONT'T think it's necessarily NOT a good fit for Erlang, see the time sheet service BeeBole[5]. Take [6] as inspiration ... and Chicago Boss [7] is something that I don't know about but saw mentioned on the list - and at least they claim to be the future of the web, so --- maybe the package is still small enough you could learn tons looking at their code. Have fun, Henning [1] http://erlware.blogspot.com/2010/02/create-erlang-mochiweb-web-application.html [2] http://yaws.hyber.org/ [3] http://erlyweb.org/ [4] https://github.com/mochi/mochiweb [5] http://beebole.com/en/blog/erlang/why-erlang/ [6] http://www.metabrew.com/article/a-million-user-comet-application-with-mochiweb-part-1/ [7] http://www.chicagoboss.org/index.html From rtrlists@REDACTED Tue Jan 4 13:38:17 2011 From: rtrlists@REDACTED (Robert Raschke) Date: Tue, 4 Jan 2011 12:38:17 +0000 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D230528.1000700@amberbio.com> References: <4D227D75.1060108@gmail.com> <4D230528.1000700@amberbio.com> Message-ID: On Tue, Jan 4, 2011 at 11:31 AM, Morten Krogh wrote: > On 1/4/11 10:17 AM, Joe Armstrong wrote: > >> No no no no no ..... Somehow you've gotten on the wrong foot here. >> Functional programming is *much* easier to understand than imperative >> programming. >> >> Definitely not much easier. It depends on the problem, and whether you > include > performance requirements. As soon as you want to control memory and > performance, > functional languages can actually be much much harder. And I mean much > harder. > > Why are one fourth, or whatever, of the posts on this list about nif or c > nodes or ports? > While I understand the sentiment, it is worth pointing out that premature optimisation is a bad thing. Most (but not all) of the desire to control memory and performance is completely unwarranted and is a waste of time and leads to maintenance issues in the future and usually also negates any performance work done at the OS or language level. > You have pure functions in all languages, even if the type system doesn't > force it on you. > I don't think it is such a big problem in C. You almost always understand > whether a function has side effects or not. > Library functions rarely do, except for possibly changing pointer > arguments, which is documented in the api. > You usually do not have nice high-level data structures, pattern matching and functions as first-class citizens. This leads to loads of code for unpacking/accessing, checking and repacking of data. Combine this with premature optimisation and you frightfully easily fall into the trap of using global variables. > But is Erlang so different here. I could write a sin function, that changed > a file, say, during execution. > The issue is the same in C and Erlang. Library writers should not do it and > mostly don't. > Do you feel this is a real problem in C? What are the functions that are reused in Erlang but not in C? > > If you write in a functional style in C (or Perl or Python or even C++) then things are pretty much as you say. But Java, for example, makes this unbelievably hard. > I really think that FPLs are an order of magnitude easier to >> understand than conventional PLs - >> >> What? An order of magnitude? > Programming languages are actually still so similar, that the main flow of > the program would > be the same no matter what language you use. Getting the data structures > and data flow right is usually the same problem. > I recently attempted to understand the flow in a (real-life) Java program of calculating one value (an SLA time calculation, this involves summing up time spent on a task taking agreed time windows and holidays into account). I waded through a total of 20 files to find bits of that calculation spread across ten of them, the other ten were taking data apart or accessing things like the holiday plan. And this even though it was possible to extract the necessary numbers up front and then do the actually pretty simple calculation based solely on those numbers. My point is that while it is probably possible to write a similarly poorly structured piece of code in a functional language, it would be much much harder to do. This is because you do not have the luxury of simply stashing some random value in the "global" space (I count inheritance as part of global!) because you're going to need it in a completely different part of your code later on. A functional approach to programming changes the way you think about data structures, data flow, and the operation you want/need to carry out on your data. And this in turn makes it easier to come up with a simpler main program flow. The programming language you use does influence the way you think about programming. Or maybe more accurately, the programming paradigms supported by the language you use .... Tuppence, etc. Robby From hd2010@REDACTED Tue Jan 4 14:12:02 2011 From: hd2010@REDACTED (Henning Diedrich) Date: Tue, 04 Jan 2011 14:12:02 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: References: <4D227D75.1060108@gmail.com> Message-ID: <4D231CA2.8040904@eonblast.com> Hi Joe, On 1/4/11 10:17 AM, Joe Armstrong wrote: > Image you're new to C and write this > > int add(int a; int b){ > return a+b; > } > > gcc says > > test3.c:1: error: parameter ?a? has just a forward declaration > test3.c: In function ?add?: > test3.c:2: error: ?a? undeclared (first use in this function) Pretty convincing. In the literal sense, before I read this I may have insisted Erlang error messages are particularly confusing. This example demonstrates that I was probably really just comparing beginner error's error messages with 'advanced error' error messages, including how you try to interprete them. Likely some of the notoriety of Erlang error messages come from that. That said 'no match of right hand side' could STILL be formulated in a more newbie friendly way. It should be possible and would be a good service to mankind to make the extra effort for the feeble. It's really only a handful of error messages that could be changed to make a big difference. C did not always have "first use in this function" hints, that was a helpful later addition. Henning From Stephane.Pamelard@REDACTED Tue Jan 4 14:26:37 2011 From: Stephane.Pamelard@REDACTED (Stephane Pamelard) Date: Tue, 4 Jan 2011 14:26:37 +0100 Subject: ASN1 compilation error in OTP R14 Message-ID: <4D23200D.80704@myriadgroup.com> Hi all, Does someone have any idea about this asn1 problem ? Previously, we compiled our code with otp_R11B (that was ok). Now, we try to compile with otp_R14 and we have an issue with ASN compiler. You can find below: - outputs with R14 compilator (with error) - outputs with R11B compilator (no error) - ASN file ------------------ Result with otp_R14 --------------------------- make -k /usr/erlang/otp_R14B/lib/erlang/bin/erlc +noobj -bber_bin CAP.asn asn1error:23:'CAP':'ExtensionField' {internal_error, {function_clause, [{asn1ct_check,componentrelation_leadingattr, [{state, {module,1,'CAP',[],'IMPLICIT',{exports,all},{imports,[]},undefined,[]}, 'CAP', {typedef,false,23,'ExtensionField', {type,[], {'SEQUENCE',false,false,undefined, [{'ComponentType',24,type, {type,[], {'ObjectClassFieldType', {'Externaltypereference',24,'CAP','EXTENSION'}, {'Externaltypereference',24,'CAP','EXTENSION'}, [{valuefieldreference,id}], undefined}, [{simpletable, {type,[], {'Externaltypereference',24,'CAP','SupportedExtensions'}, [],[],no}}], [],no}, mandatory,undefined,undefined}, {'ComponentType',25,criticality,.................. ------------------ Result with otp_R11B --------------------------- otp_R11 make -k /usr/erlang/otp_R11B-5/lib/erlang/bin/erlc +noobj -bber_bin CAP.asn Erlang ASN.1 version "1.4.5" compiling "/home/CAP.asn" Compiler Options: [ber_bin, report_errors, {cwd,"/home/src"}, {outdir,"/home/src"}, noobj, {i,"."}, {i,"/home/src"}] --{generated,"/home/src/CAP.asn1db"}-- --{generated,"/home/src/CAP.hrl"}-- --{generated,"/home/src/CAP.erl"}-- ok ------------------ ASN CODE ------------------------- CAP {ccitt(0) identified-organization(4) etsi(0) mobileDomain(0) umts-network(1) modules(3) cap-datatypes(52) version3(2)} DEFINITIONS IMPLICIT TAGS ::= BEGIN EXTENSION ::= CLASS { &ExtensionType, &criticality CriticalityType DEFAULT ignore, &id Code } WITH SYNTAX { EXTENSION-SYNTAX &ExtensionType CRITICALITY &criticality IDENTIFIED BY &id } ExtensionField ::= SEQUENCE { type EXTENSION.&id ({SupportedExtensions }), criticality CriticalityType DEFAULT ignore, value [1] EXTENSION.&ExtensionType ({SupportedExtensions }{@type}), ...} SupportedExtensions EXTENSION ::= {firstExtension, ...} firstExtension EXTENSION ::= { EXTENSION-SYNTAX NULL CRITICALITY ignore IDENTIFIED BY global : {itu-t(0) identified-organization(4) organisation(0) gsm(1) capextension(2)}} CriticalityType ::= ENUMERATED { ignore (0), abort (1) } Code ::= CHOICE {local INTEGER, global OBJECT IDENTIFIER} END Thanks for your help. St?phane This message, including attachments, is intended solely for the addressee indicated in this message and is strictly confidential or otherwise privileged. If you are not the intended recipient (or responsible for delivery of the message to such person) : - (1) please immediately (i) notify the sender by reply email and (ii) delete this message and attachments, - (2) any use, copy or dissemination of this transmission is strictly prohibited. If you or your employer does not consent to Internet email messages of this kind, please advise Myriad Group AG by reply e-mail immediately. Opinions, conclusions and other information expressed in this message are not given or endorsed by Myriad Group AG unless otherwise indicated by an authorized representative independent of this message. From erlang@REDACTED Tue Jan 4 15:00:28 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 4 Jan 2011 15:00:28 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D231CA2.8040904@eonblast.com> References: <4D227D75.1060108@gmail.com> <4D231CA2.8040904@eonblast.com> Message-ID: On Tue, Jan 4, 2011 at 2:12 PM, Henning Diedrich wrote: > Hi Joe, > > On 1/4/11 10:17 AM, Joe Armstrong wrote: > > Image you're new to C and write this > > int add(int a; int b){ > return a+b; > } > > gcc says > > test3.c:1: error: parameter ?a? has just a forward declaration > test3.c: In function ?add?: > test3.c:2: error: ?a? undeclared (first use in this function) > > Pretty convincing. In the literal sense, before I read this I may have > insisted Erlang error messages are particularly confusing. I use the following rule - find the line number of the first error message totally ignore the text that is printed - say to yourself "there is something wrong with this line" mess with the line until the error goes away. This method works for all know languages and environments (with the exception of eclipse, where I gave up after days of trying to find out what some error message meant) /Joe > This example demonstrates that I was probably really just comparing > beginner error's error messages with 'advanced error' error messages, > including how you try to interprete them. Likely some of the > notoriety of Erlang error messages come from that. > > That said 'no match of right hand side' could STILL be formulated in > a more newbie friendly way. It should be possible and would be a > good service to mankind to make the extra effort for the feeble. > It's really only a handful of error messages that could be changed > to make a big difference. > > C did not always have "first use in this function" hints, that was > a helpful later addition. > > Henning > From attila.r.nohl@REDACTED Tue Jan 4 15:11:26 2011 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Tue, 4 Jan 2011 15:11:26 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: References: <4D227D75.1060108@gmail.com> <4D230528.1000700@amberbio.com> Message-ID: 2011/1/4, Robert Raschke : [...] > You usually do not have nice high-level data structures, pattern matching > and functions as first-class citizens. This leads to loads of code for > unpacking/accessing, checking and repacking of data. Unpacking/accessing is complicated in every language, if the data structure is deep enough. Pattern matching helps here, but you can't do pattern matching on e.g. a dict object, have to use wrapper functions - just like you'd do in Java too. Not to mention that the pattern matching on record fields is seriously confusing at the first (thousand) glance. [...] > My point is that while it is probably possible to write a similarly poorly > structured piece of code in a functional language, it would be much much > harder to do. This is because you do not have the luxury of simply stashing > some random value in the "global" space (I count inheritance as part of > global!) because you're going to need it in a completely different part of > your code later on. In my experience it's the contrary. A call of a method of a class in e.g. Java is a single piece of code, but in Erlang the classes are usually mapped to gen_servers and usually there are three functions for every method: - an interface method: exported, executing in the calling process, usually only doing a gen_server:call - a handle_call clause usually only calling an implementation function - and the actual implementing function, executing in the gen_server process. These functions are usually in the same file, so when I'm looking at the code, I have to keep in mind that which function is executing in which process, to know that which values can be available from the state. From jeff.zellner@REDACTED Tue Jan 4 15:13:01 2011 From: jeff.zellner@REDACTED (Jeff Zellner) Date: Tue, 4 Jan 2011 09:13:01 -0500 Subject: [erlang-questions] game development & erlang In-Reply-To: References: Message-ID: I don't know if this counts, but my pet personal project is a Rogue-like in Erlang. I don't know if it counts since it's not very fun right now! :) http://github.com/jzellner/rogueunlike Probably the only OTP rogue ever made... ;) On Mon, Jan 3, 2011 at 10:27 PM, Slav Pankratov wrote: > Hello! > > Is anyone on the list involved in game development with Erlang? :) > (I've seen a mention of 2d Android online game in a recent discussion and > decided to ask) > > thanks, > Slav > From hd2010@REDACTED Tue Jan 4 16:00:06 2011 From: hd2010@REDACTED (Henning Diedrich) Date: Tue, 04 Jan 2011 16:00:06 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: References: <4D227D75.1060108@gmail.com> <4D231CA2.8040904@eonblast.com> Message-ID: <4D2335F6.7060708@eonblast.com> On 1/4/11 3:00 PM, Joe Armstrong wrote: > ... find the line number of the first error message totally ignore > the text that is printed - say to yourself "there is something wrong > with this line" mess with the line until the error goes away. > > This method works for all know languages and environments (with the > exception of eclipse, where I gave up after days of trying to find > out what some error message meant) How does a noob know she is not confronted, in Erlang, with something like you experienced with eclipse, and that she could in fact rely on at least, and exactly, the line number of an error message making sense - she can't. From experience, having seen some strange error messages in strange places, coming to Erlang, I did not trust the line numbers to indicate that the error was in fact were the line said. How could I? In fact, I think you can fool almost any parser to get it wrong, where the error actually originates. You have to be prepared, especially as a noob, to look elsewhere for your fault. Which is then almost a guarantee that you look for 'no match' elsewhere in the first few days, and assume that e.g. a=1 can't be it. But maybe the line after that holding a 'case' etc. Even though you read all the explanations about patterns and for-good binding and atoms being lower case. Of course you soon get it, why that is the 'right' message. But why not be more forthcoming to those folks who probably never meant for the line to be a match in the first place, but an assignment and haven't gotten it yet. There is enough stuff to learn, errors get mixed up. If I remember right, I would have found a hint helpful in the incident. I did stuff exactly like that above and the problem was not even that I re-assigned a value but camelCase creep. There are not many places I'd opt for a modest change, but this one, certainly. I did feel more left alone with Erlang than with other languages concerning error messages. If that was for the wrong reasons, that's still not an argument against that they could be made more newbie-friendly in a calm moment. Except of course, you want to have a threshold everybody can fondly brag about, later. That must certainly be accepted as overriding point. Henning From kaiduanx@REDACTED Tue Jan 4 16:11:41 2011 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Tue, 4 Jan 2011 10:11:41 -0500 Subject: [erlang-questions] SSL memory consumption In-Reply-To: References: Message-ID: Thank you Hynek. I had figured out the problem. The root cause is that every time a gen_fsm is created in way 2, it inherits the '$ancestors' from its parent, and its parent is also added to the '$ancestors' of the process's dictionary. '$ancestors' of 1st gen_fsm: [p0], '$ancestors' of 2nd gen_fsm: [p0, p1], '$ancestors' of 3rd gen_fsm: [p0, p1, p2], .... '$ancestors' of nth gen_fsm: [p0, p1, p2, ...., pn-1] After resetting the '$ancestors' before creating gen_fsm by calling put('$ancestors', Parent), the memory footprint is reduced dramatically. What is the usage of '$ancestors'? Best regards, /Kaiduan On Tue, Jan 4, 2011 at 5:21 AM, Hynek Vychodil wrote: > On Thu, Dec 30, 2010 at 6:32 AM, Kaiduan Xie wrote: >> After tried two different ways to call ssl:transport_accept and >> ssl:ssl_accept, the following observation was found, >> >> 1. Calls ssl:transport_accept and ssl:ssl_accept in a server loop, and >> for each SSL connection, starts a gen_fsm to receive data. The code is >> listed as below, >> >> loop3(LSock) -> >> ? ?{ok, Socket} = ssl:transport_accept(LSock), >> ? ?ok = ssl:ssl_accept(Socket), >> ? ?{ok, {PeerAddr, PeerPort}} = ssl:peername(Socket), >> ? ?io:format("Accepted peer:~p:~p:~p:~p~n", [self(),PeerAddr, PeerPort, I]), >> ? ?{ok, Pid} = gen_fsm:start(erlsip_transport_tls_connection, >> {Socket, PeerAddr, PeerPort}, []), >> ? ?ssl:controlling_process(Socket, Pid), >> ? ?loop3(LSock). >> >> 2. Calls ssl:transport_accept, and ssl:ssl_accept in a gen_fsm. Upon a >> SSL connection is accepted, starts another gen_fsm to accept new SSL >> connection. The code is listed as below. >> >> init({server, LSocket, Parent}) -> >> ? ?{ok, wait_for_connection, #state{lsocket = LSocket, >> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? parent = Parent}, 0}; >> wait_for_connection(timeout, State) -> >> ? ?io:format("Waiting for TLS connection ... ~p/~p~n", [self(), State#state.par >> ent]), >> ? ?{ok, Socket} = ssl:transport_accept(State#state.lsocket), >> ? ?ok = ssl:ssl_accept(Socket), >> ? ?gen_fsm:start(erlsip_transport_tls_connection, >> ? ? ? ? ? ? ? ? {server, State#state.lsocket, State#state.parent}, >> ? ? ? ? ? ? ? ? ?[]), >> ? ?{ok, {PeerAddr, PeerPort}} = ssl:peername(Socket), >> ? ?io:format("Accepted peer:~p/~p:~p~n", [self(), PeerAddr, PeerPort]), >> ? ?{next_state, wait_for_crlf_crlf, State#state{socket = Socket, >> ? ? ? ? ? ? ? ? ? peer_address = PeerAddr, peer_port = PeerPort}}. >> >> Memory consumption per connection in the second way is about 120 K >> with R14B01 while a connection takes about 30 K in first way. >> >> I am very happy with the result, but can someone explain the difference? >> >> Thanks, >> >> /Kaiduan > > I'm just guessing but it can be GC issue. In first case you do all > those calls inside one process so all used structures inside loop3 can > be GCed. In second case you do this sequence always in new process and > when there are not future reductions (there is not any communication - > fsm doesn't work) this memory is not freed. Try add explicit GC to > your init/1 in second case and measure again. > >> On Thu, Dec 23, 2010 at 9:31 PM, Kaiduan Xie wrote: >>> I just replaced R14A with R14B01, each SSL connection consumes about >>> 120 K memory, 50K less than that of R14A. >>> >>> Thank Tony, Jesper for the help, can we reduce more? >>> >>> /Kaiduan >>> >>> On Thu, Dec 23, 2010 at 8:38 PM, Jesper Louis Andersen >>> wrote: >>>> On Fri, Dec 24, 2010 at 02:33, Kaiduan Xie wrote: >>>> >>>>> What do you mean the old ssl and new ssl, Tony? >>>> >>>> SSL was replaced recently with a new implementation and you have the >>>> 'new' version here if you are on R14b >>>> >>>> -- >>>> J. >>>> >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > > > -- > --Hynek (Pichi) Vychodil > > Analyze your data in minutes. Share your insights instantly. Thrill > your boss.? Be a data hero! > Try GoodData now for free: www.gooddata.com > From attila.r.nohl@REDACTED Tue Jan 4 16:20:28 2011 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Tue, 4 Jan 2011 16:20:28 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D2335F6.7060708@eonblast.com> References: <4D227D75.1060108@gmail.com> <4D231CA2.8040904@eonblast.com> <4D2335F6.7060708@eonblast.com> Message-ID: 2011/1/4, Henning Diedrich : [...] > If I remember right, I would have found a hint helpful in the incident. > I did stuff exactly like that above and the problem was not even that > I re-assigned a value but camelCase creep. This is where syntax highlight helps in the editor. From vinoski@REDACTED Tue Jan 4 16:37:04 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Tue, 4 Jan 2011 10:37:04 -0500 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D230528.1000700@amberbio.com> References: <4D227D75.1060108@gmail.com> <4D230528.1000700@amberbio.com> Message-ID: On Tue, Jan 4, 2011 at 6:31 AM, Morten Krogh wrote: > > Why are one fourth, or whatever, of the posts on this list about nif or c > nodes or ports? Even if we assume your percentage is accurate (which I don't), simply assuming these features are used only for performance reasons is a case of premature optimization. ;-) While some uses of these features involve performance concerns, C nodes, ports, drivers, and NIFs all serve as points for integrating Erlang with other environments. NIFs have already proven themselves particularly useful in this regard. Because of the continued growth in the popularity and application of Erlang, the number of developers interested in using these features to integrate with non-Erlang code is also growing. These are areas of Erlang involving languages other than Erlang, such as C, so it could be they're harder to understand than other parts of the language. It also could be that even with the aforementioned growing popularity of the language, the number of Erlang developers who've ever written C nodes and drivers used in production remains relatively small, and so finding good sources of detailed information about how best to write them is challenging. But still, I'd argue there really aren't that many questions posted here about these features. --steve From hd2010@REDACTED Tue Jan 4 16:44:11 2011 From: hd2010@REDACTED (Henning Diedrich) Date: Tue, 04 Jan 2011 16:44:11 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: References: <4D227D75.1060108@gmail.com> <4D231CA2.8040904@eonblast.com> <4D2335F6.7060708@eonblast.com> Message-ID: <4D23404B.1060407@eonblast.com> On 1/4/11 4:20 PM, Attila Rajmund Nohl wrote: > 2011/1/4, Henning Diedrich: > [...] >> If I remember right, I would have found a hint helpful in the incident. >> I did stuff exactly like that above and the problem was not even that >> I re-assigned a value but camelCase creep. > This is where syntax highlight helps in the editor. I am not sure that's an answer. I could also have been more intelligent as an alternate solution. Or we all stop making errors. Is the learning curve smooth enough? Maybe. Could it be better, probably. Henning From mevans@REDACTED Tue Jan 4 17:10:03 2011 From: mevans@REDACTED (Evans, Matthew) Date: Tue, 4 Jan 2011 11:10:03 -0500 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: References: <4D227D75.1060108@gmail.com> Message-ID: I agree Joe. Interesting story. When I was at university studying computer science there was an introductory course in Prolog, which was also offered to non computer science students as an optional course. The funny thing is that the non computer science folks found it easier to follow than the computer science folks. It seems that we comp sci people had had our brains polluted by the imperative programming method. -----Original Message----- From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Joe Armstrong Sent: Tuesday, January 04, 2011 4:17 AM To: JETkoten Cc: Erlang Subject: Re: [erlang-questions] Erlang and the learning curve On Tue, Jan 4, 2011 at 2:52 AM, JETkoten wrote: > Hi All, > > I've been reading/watching everything I can find (off and online) for > learning Erlang for the past two or three weeks, but am just not getting it, > in terms of being able to do anything with Erlang in the real world. > > I've watched some videos about functional programming where the hosts joked > about people looking into functional programming deciding it's too hard and > then never touching it again... No no no no no ..... Somehow you've gotten on the wrong foot here. Functional programming is *much* easier to understand than imperative programming. There is a sense in which *all* programming languages are difficult to understand, this is the phase where you are not familiar with the syntax and what the various construct mean. Take "C" for example - this is the Lingua Franca of all programming languages - we all know C. So let's do an experiment. Stop a C programmer in the corridor and ask them the difference between: int *p[10]; and int (*p)[10]; One is "a pointer to (an array of 10 integers)" the other is "an array of (ten pointers to integers)" Then ask them what char * const *(*next)(); means (next is a pointer to a function returning a pointer to a constant pointer to a char") A C program is actually extrememly difficult to understand - the above problem is purely syntactic, then the problem of understanding a C program is much deeper - you have to mentally execute the program in your head to understand what it means - this is due to side effects. Functional programs are really really easy to understand, since given that you understand what the inputs to a function are the fucntion will always, always always, forever and ever and ever return the same value. This make the code extremely reusable. if you are not familiar with the erlang syntax your eyes will not see what my eyes see. When I see code I mentally scan for the use of variables, I look for the first occurrence and subsequent occurrences. Imagine some function foo, that uses a variable X, when I look for the uses of X in foo my brain "sees" this: foo(....) -> ... X = .... <--- 1 ... ... bar(...X...) ... <--- 2 {...,...,X,....} = ..... <--- 3 I now know that X is defined at point 1 in the program. That it is used in point 2 and that in point 3 that whatever is on the right hand side of the equals in line 3 must return a tuple whose 3'rd argument must be X The fact that X never changes once it is set make the program extremely easy to understand, and debug. If a value is incorrect, it can only set in one place. You just peer at the program, you don't need a debugger. (in C for example, a value once set, might be changed a trillion times, and one one occasion when it is changed it gets the wrong value, but which of these times was it? - this is *extremely* difficult to understand, and a debugger might help) I really think that FPLs are an order of magnitude easier to understand than conventional PLs - (don't get me started on languages like java - is Java easy compared to erlang? - try writing the simplest of callback functions in java and tell me java is easy to understand - what's the visitor pattern like in java? read http://en.wikipedia.org/wiki/Visitor_pattern then read the comment about the "Visitor Pattern in a Dynamic programming language" as the wikipedia says: "The visitor pattern disappears completely. No extra classes and methods are needed" ) > but seriously, how do people bridge the gap > between understanding the basics and implementing real world projects with > Erlang? I know the three major books are available, but none of them are > really using Erlang for what I'd like to do... This is what you do - buy one of the books - modestly prevents me from telling you which is best - then type in the programs staring at the simplest. It's very important that you type in the programs *yourself* - don't download the code and run it. You have to get the syntax into your spine not your brain. When you type in the programs, you'll make tiny errors, missing commas etc. Fix these error - this is the tricky bit. Every programming language under the sun produces totally bizarre error message when you get the punctuation wrong. (aside Image you're new to C and write this int add(int a; int b){ return a+b; } gcc says test3.c:1: error: parameter 'a' has just a forward declaration test3.c: In function 'add': test3.c:2: error: 'a' undeclared (first use in this function) Which is it's helpful way of saying - "sorry Joe it should be a comma in the function arguments not a semicolon" ) all error messages are difficult to understand. The purpose of a book is to present you with small completely understandable self-contained examples that you can type in and learn from. It takes a few months to become fluent in any programming language - reckon a couple of weeks for the syntax - then a few months before you know your way around the libraries /Joe > > How have any of you who have come to know big-picture how great Erlang is > been able to get enough knowledge of it to actually do what you'd like to > do? > > Any and all replies are most welcome. > > Thanks, > Jack > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From brobinson@REDACTED Tue Jan 4 17:20:36 2011 From: brobinson@REDACTED (Bryan Robinson [ICG-MKTS]) Date: Tue, 04 Jan 2011 11:20:36 -0500 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: References: <4D227D75.1060108@gmail.com> Message-ID: <4D2348D4.80802@atdesk.com> Jack (JETkoten), I cannot stress what Joe wrote enough: On 01/04/11 04:17, Joe Armstrong wrote: >> but seriously, how do people bridge the gap >> > between understanding the basics and implementing real world projects with >> > Erlang? I know the three major books are available, but none of them are >> > really using Erlang for what I'd like to do... > This is what you do - buy one of the books - modestly prevents me from > telling you which is best - > then type in the programs staring at the simplest. > > It's very important that you type in the programs *yourself* - don't > download the code > and run it. > > You have to get the syntax into your spine not your brain. > > When you type in the programs, you'll make tiny errors, missing commas etc. > Fix these error - this is the tricky bit. A book is a great investment for saving time in learning Erlang. All of the Erlang books available are *very readable* and accessible. You can often buy used copies through Amazon Marketplace or something similar for even lower prices. As for your project, just start writing a piece of it in Erlang, just a small piece. It helps if you've read a few chapters from an Erlang book on sequential and concurrent programming in Erlang. You don't need to overload yourself with processes, patterns, OTP and more advanced features--learn them as you need them. As everyone is saying, Erlang really is a simple language. Once you have this small piece working to your satisfaction, you'll also find that you have more confidence with Erlang, and most likely you'll be itching to continue. The first serious piece of Erlang code I wrote was for parsing an XML file using Erlang's SAX library. I understood the ideas behind XML and SAX, so it was a matter of referencing the docs for the Erlang library and then trying out things...and failing a lot. It took longer than I would have liked, but I learned an a lot about Erlang too. During the development of this small code, I gained a lot of first hand experience with lists, error handling, records, compiler error messages, and most significantly, how to handle state in a functional environment. When I went back to make the code more readable, macros then came into the picture. When I was stuck, I kept going back to simple examples in the book I purchased for ideas on how to do things, and even looking at the Erlang docs and the simple examples in there as well. From hd2010@REDACTED Tue Jan 4 17:33:34 2011 From: hd2010@REDACTED (Henning Diedrich) Date: Tue, 04 Jan 2011 17:33:34 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D230528.1000700@amberbio.com> References: <4D227D75.1060108@gmail.com> <4D230528.1000700@amberbio.com> Message-ID: <4D234BDE.3040209@eonblast.com> On 1/4/11 12:31 PM, Morten Krogh wrote: > As soon as you want to control memory and performance, > functional languages can actually be much much harder. And I mean much > harder. Yes but it may matter way fewer times than you seem to imply. As has been pointed out that holds only in cases where you plan to do the optimization yourself down to a deep level. But inbetween, e.g. for the right list representation, you have four flavors in Erlang to optimze for performance. That's more than in, say, JS or Lua where you have basically one. Erlang is not religious Haskell in that regard and there are many more cases where Erlang opted for the practical solution. Also, there's at least one imperative language, Python*, where the mantra is that your time is expensive, not the hardware that will provide for faster execution. In other words, development time should be optimized, not execution time. So yours seems not to be an argument for imperative languages per se. Some reject that view voluntarily. It simply depends on what you want to do. Finally, you get stuff for free in Erlang that you have to code or use libraries in other languages, as results from share-nothing memory and micro- processes. Think scaling on multi-core. That is really not trivial for other languages and where performance-gains are coming from today. I am not sure if you are aware to what degree e.g. Python, even Stackless Python simply forfeit that whole playing field wholesale, last time I checked. Of course you can mimic functional or share-nothing in, e.g. C. But have one programmer mess up and he can ruin everybody elses discipline with horrendous errors. In Erlang you simply can't. (Or, let's say, it's way harder). Best, Henning From martin.reinsprecht@REDACTED Tue Jan 4 18:22:08 2011 From: martin.reinsprecht@REDACTED (Martin Reinsprecht) Date: Tue, 04 Jan 2011 18:22:08 +0100 Subject: Looking for a code and config sample for inets 5.5 HTTP Server Message-ID: <4D235740.4040803@activenet.at> Could someone provide me with a code an configuration sample for http://www.erlang.org/doc/apps/inets/http_server.html thanks in advance Martin Reinsprecht From mk@REDACTED Tue Jan 4 19:26:06 2011 From: mk@REDACTED (Morten Krogh) Date: Tue, 04 Jan 2011 19:26:06 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: References: <4D227D75.1060108@gmail.com> <4D230528.1000700@amberbio.com> Message-ID: <4D23663E.6040909@amberbio.com> On 1/4/11 1:38 PM, Robert Raschke wrote: > > While I understand the sentiment, it is worth pointing out that premature > optimisation is a bad thing. Most (but not all) of the desire to control > memory and performance is completely unwarranted and is a waste of time and > leads to maintenance issues in the future and usually also negates any > performance work done at the OS or language level. > What work at the OS level is being negated? > If you write in a functional style in C (or Perl or Python or even C++) then > things are pretty much as you say. But Java, for example, makes this > unbelievably hard. > I will repeat my question. What are the functions that can be reused in Erlang but not in imperative languages? Feel free to use Java instead of C in your examples. It seems to me that it is completely equivalent. Every time you have an Erlang function, there is a corresponding Java function. Functional languages should have larger libraries then, if these claims were true. But that doesn't seem to be the case. Morten. From mk@REDACTED Tue Jan 4 19:35:32 2011 From: mk@REDACTED (Morten Krogh) Date: Tue, 04 Jan 2011 19:35:32 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: References: <4D227D75.1060108@gmail.com> <4D230528.1000700@amberbio.com> Message-ID: <4D236874.7070602@amberbio.com> On 1/4/11 4:37 PM, Steve Vinoski wrote: > On Tue, Jan 4, 2011 at 6:31 AM, Morten Krogh wrote: >> Why are one fourth, or whatever, of the posts on this list about nif or c >> nodes or ports? > Even if we assume your percentage is accurate (which I don't), simply > assuming these features are used only for performance reasons is a > case of premature optimization. ;-) > > While some uses of these features involve performance concerns, C > nodes, ports, drivers, and NIFs all serve as points for integrating > Erlang with other environments. NIFs have already proven themselves > particularly useful in this regard. Because of the continued growth in > the popularity and application of Erlang, the number of developers > interested in using these features to integrate with non-Erlang code > is also growing. > > These are areas of Erlang involving languages other than Erlang, such > as C, so it could be they're harder to understand than other parts of > the language. It also could be that even with the aforementioned > growing popularity of the language, the number of Erlang developers > who've ever written C nodes and drivers used in production remains > relatively small, and so finding good sources of detailed information > about how best to write them is challenging. > > But still, I'd argue there really aren't that many questions posted > here about these features. > > --steve My percentage is probably wrong. I didn't count. I just wanted to stress that I see quite a lot of performance optimization in Erlang. Why were NIFs introduced at all? Why new BIFs all the time? All of that could be done in Erlang. Steve, I have never seen a NIF that was not performance motivated. But you have seen many more NIFs than me. Do you have any examples? Morten. From raould@REDACTED Tue Jan 4 19:41:07 2011 From: raould@REDACTED (Raoul Duke) Date: Tue, 4 Jan 2011 10:41:07 -0800 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D236874.7070602@amberbio.com> References: <4D227D75.1060108@gmail.com> <4D230528.1000700@amberbio.com> <4D236874.7070602@amberbio.com> Message-ID: On Tue, Jan 4, 2011 at 10:35 AM, Morten Krogh wrote: > Why were NIFs introduced at all? Why new BIFs all the time? All of that > could be done in Erlang. yeah, i'm curious, too :) From co7eb@REDACTED Tue Jan 4 19:49:47 2011 From: co7eb@REDACTED (=?iso-8859-1?Q?Gilberio_Carmenates_Garc=EDa?=) Date: Tue, 4 Jan 2011 13:49:47 -0500 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D227D75.1060108@gmail.com> References: <4D227D75.1060108@gmail.com> Message-ID: <002401cbac40$2ac3a3a0$804aeae0$@co.cu> Hi Jack, When I started with Erlang, me and a friend starting doing some project that we called LanSearch, was a server to a LanSearch's client that scanned the network looking for a particularly stuff (videos, music, source codes; cs, c, java, etc.) and we started to build a server to communicate all those clients, chat functionality and search share (upload/download) and many others things. So the interesting here is that we gain many clients with that software and we actually did that software without any knowledge about Erlang, just with a simple knowledge about mnesia and rpc calls, using the JInterface for .Net c#. So now I have some others ideas like a MMORG Games and stuffs like that, that seems pretty easy to build using Erlang for me right now; with a little more knowledge about server and supervisor behaviors and mnesia, all you need to know it's there, you just must want to do it and the other comes along, I wish I could have a graphical programmer, then I would send you an small example that you can do with Erlang and other language like C#, think about Erlang as if you just need it for build very ultrafast servers and the other visual languages to build clients, for me EVO (ExtendedVisualOtp) is the best for communications Erlang/C#, EVO is a little framework, that I made, to build client/server applications using .Net and Erlang, Some day I will translate the .Net part to java. Best, Ivan. -----Mensaje original----- De: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] En nombre de JETkoten Enviado el: Lunes, 03 de Enero de 2011 20:53 Para: erlang-questions@REDACTED Asunto: [erlang-questions] Erlang and the learning curve Hi All, I've been reading/watching everything I can find (off and online) for learning Erlang for the past two or three weeks, but am just not getting it, in terms of being able to do anything with Erlang in the real world. I've watched some videos about functional programming where the hosts joked about people looking into functional programming deciding it's too hard and then never touching it again... but seriously, how do people bridge the gap between understanding the basics and implementing real world projects with Erlang? I know the three major books are available, but none of them are really using Erlang for what I'd like to do... How have any of you who have come to know big-picture how great Erlang is been able to get enough knowledge of it to actually do what you'd like to do? Any and all replies are most welcome. Thanks, Jack ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED ======================================================================= Este mensaje ha sido enviado mediante el servicio de correo electr?nico que ofrece la Federaci?n de Radioaficionados de Cuba a sus miembros para respaldar el cumplimiento de los objetivos de la organizaci?n y su pol?tica informativa. La persona que env?a este correo asume el compromiso de usar el servicio a tales fines y cumplir con las regulaciones establecidas. From mk@REDACTED Tue Jan 4 20:25:47 2011 From: mk@REDACTED (Morten Krogh) Date: Tue, 04 Jan 2011 20:25:47 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D234BDE.3040209@eonblast.com> References: <4D227D75.1060108@gmail.com> <4D230528.1000700@amberbio.com> <4D234BDE.3040209@eonblast.com> Message-ID: <4D23743B.4010805@amberbio.com> On 1/4/11 5:33 PM, Henning Diedrich wrote: > On 1/4/11 12:31 PM, Morten Krogh wrote: >> As soon as you want to control memory and performance, >> functional languages can actually be much much harder. And I mean >> much harder. > > Yes but it may matter way fewer times than you seem to imply. As has > been pointed out that > holds only in cases where you plan to do the optimization yourself > down to a deep level. > > But inbetween, e.g. for the right list representation, you have four > flavors in Erlang to optimze > for performance. That's more than in, say, JS or Lua where you have > basically one. Erlang is not > religious Haskell in that regard and there are many more cases where > Erlang opted for the practical > solution. > What are those four flavors? > Also, there's at least one imperative language, Python*, where the > mantra is that your time > is expensive, not the hardware that will provide for faster execution. > In other words, development > time should be optimized, not execution time. So yours seems not to be > an argument for imperative > languages per se. Some reject that view voluntarily. > Of course, you can always build a slow language, both functionally and imperatively. The question is if it is possible to build a fast functional language, especially without making BIFs and NIFS for everything. And then the functional language is just a glorified C library. > It simply depends on what you want to do. Finally, you get stuff for > free in Erlang that you have > to code or use libraries in other languages, as results from > share-nothing memory and micro- > processes. Think scaling on multi-core. That is really not trivial for > other languages and where > performance-gains are coming from today. I am not sure if you are > aware to what degree e.g. Python, even Stackless Python simply forfeit > that whole playing field wholesale, last time I checked. This multicore stuff is somewhat off topic. Multicore scaling is pure performance oriented, which you and many others claim is not so important. Why bother about multicore when single core performance is considered premature optimization? But I am curious about what you mean. You think Erlang will scale better than the fast imperative languages like C? What makes you say that? Is the claim that the otp team will write a better user space threading (process) library than anyone else? Even in that case other people could copy their code. I assume the scheduling and migration code is in C not Erlang or what? The kernel people, working at a lower level and in C, has an advantage. They can integrate user threads with kernel threads. For instance, how will Erlang solve the problem that the Erlang thread on a given core could be preempted by the kernel right after an Erlang process has been migrated to that core? I think it is fair to say that the jury is still out here. I think Erlang will do very fine. It is just that I am not as pessimistic about the other languages. Especially not C. Maybe Java's garbage collector will get stuck when we get to thousands of cores. Possibly. But not imperative languages in general. Morten. > Of course you can mimic functional or share-nothing in, e.g. C. But > have one programmer mess > up and he can ruin everybody elses discipline with horrendous errors. > In Erlang you simply can't. > (Or, let's say, it's way harder). > > Best, > Henning > From vinoski@REDACTED Tue Jan 4 20:59:40 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Tue, 4 Jan 2011 14:59:40 -0500 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D236874.7070602@amberbio.com> References: <4D227D75.1060108@gmail.com> <4D230528.1000700@amberbio.com> <4D236874.7070602@amberbio.com> Message-ID: On Tue, Jan 4, 2011 at 1:35 PM, Morten Krogh wrote: > On 1/4/11 4:37 PM, Steve Vinoski wrote: >> >> On Tue, Jan 4, 2011 at 6:31 AM, Morten Krogh ?wrote: >>> >>> Why are one fourth, or whatever, of the posts on this list about nif or c >>> nodes or ports? >> >> Even if we assume your percentage is accurate (which I don't), simply >> assuming these features are used only for performance reasons is a >> case of premature optimization. ;-) >> >> While some uses of these features involve performance concerns, C >> nodes, ports, drivers, and NIFs all serve as points for integrating >> Erlang with other environments. NIFs have already proven themselves >> particularly useful in this regard. Because of the continued growth in >> the popularity and application of Erlang, the number of developers >> interested in using these features to integrate with non-Erlang code >> is also growing. >> >> These are areas of Erlang involving languages other than Erlang, such >> as C, so it could be they're harder to understand than other parts of >> the language. It also could be that even with the aforementioned >> growing popularity of the language, the number of Erlang developers >> who've ever written C nodes and drivers used in production remains >> relatively small, and so finding good sources of detailed information >> about how best to write them is challenging. >> >> But still, I'd argue there really aren't that many questions posted >> here about these features. >> >> --steve > > My percentage is probably wrong. I didn't count. I just wanted to stress > that I see quite a lot of performance optimization in Erlang. As do I, but not by jumping out of Erlang. Using Erlang's profilers and other tools is very straightforward and can significantly help improve performance. > Why were NIFs introduced at all? Why new BIFs all the time? All of that > could be done in Erlang. Only the OTP team can say for sure why they introduced NIFs, but my understanding is that it allows for easier integration with and access to C than drivers. I personally don't find drivers difficult, but I know they can be overkill for simple integration needs. Are new BIFs really being introduced "all the time" strictly for performance reasons? If so can you cite some examples? > Steve, I have never seen a NIF that was not performance motivated. > But you have seen many more NIFs than me. > Do you have any examples? I have two NIFs in the system I work on, for example, neither of which is performance-oriented: * One NIF allows simple access the Linux UUID library so that my Erlang code can generate UUIDs the same as the C++ components with which it interoperates over the internal chassis network. This has nothing to do with performance and everything to do with consistent UUID generation across all the components in the chassis. * The other NIF enables control plane calls to some specialized offload sockets on our video delivery hardware. The system components providing these sockets are written in C, so the NIF imports their header files to pick up all their #defines, constants, and function prototypes, and then presents that functionality in Erlang in a way that's natural and simple for the Erlang programmer to use. This NIF has nothing to do with performance and everything to do with code coherency, integration with the C code, and minimizing duplication between Erlang and C. --steve From mk@REDACTED Tue Jan 4 22:07:58 2011 From: mk@REDACTED (Morten Krogh) Date: Tue, 04 Jan 2011 22:07:58 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: References: <4D227D75.1060108@gmail.com> <4D230528.1000700@amberbio.com> <4D236874.7070602@amberbio.com> Message-ID: <4D238C2E.8010900@amberbio.com> On 1/4/11 8:59 PM, Steve Vinoski wrote: > On Tue, Jan 4, 2011 at 1:35 PM, Morten Krogh wrote: >> On 1/4/11 4:37 PM, Steve Vinoski wrote: >>> On Tue, Jan 4, 2011 at 6:31 AM, Morten Krogh wrote: >>>> Why are one fourth, or whatever, of the posts on this list about nif or c >>>> nodes or ports? >>> Even if we assume your percentage is accurate (which I don't), simply >>> assuming these features are used only for performance reasons is a >>> case of premature optimization. ;-) >>> >>> While some uses of these features involve performance concerns, C >>> nodes, ports, drivers, and NIFs all serve as points for integrating >>> Erlang with other environments. NIFs have already proven themselves >>> particularly useful in this regard. Because of the continued growth in >>> the popularity and application of Erlang, the number of developers >>> interested in using these features to integrate with non-Erlang code >>> is also growing. >>> >>> These are areas of Erlang involving languages other than Erlang, such >>> as C, so it could be they're harder to understand than other parts of >>> the language. It also could be that even with the aforementioned >>> growing popularity of the language, the number of Erlang developers >>> who've ever written C nodes and drivers used in production remains >>> relatively small, and so finding good sources of detailed information >>> about how best to write them is challenging. >>> >>> But still, I'd argue there really aren't that many questions posted >>> here about these features. >>> >>> --steve >> My percentage is probably wrong. I didn't count. I just wanted to stress >> that I see quite a lot of performance optimization in Erlang. > As do I, but not by jumping out of Erlang. Using Erlang's profilers > and other tools is very straightforward and can significantly help > improve performance. > Yes, and that is also my point. Because people have think more about performance in Erlang than in C or C++ say, profilers and other tools are needed earlier and should be included in a discussion of the difficulty of various languages. Erlang with profilers and tools is a harder "language" than Erlang without. >> Why were NIFs introduced at all? Why new BIFs all the time? All of that >> could be done in Erlang. > Only the OTP team can say for sure why they introduced NIFs, but my > understanding is that it allows for easier integration with and access > to C than drivers. I personally don't find drivers difficult, but I > know they can be overkill for simple integration needs. > Maybe, but why C to start with. Performance must be a big issue here, but anyway. > Are new BIFs really being introduced "all the time" strictly for > performance reasons? If so can you cite some examples? > I was thinking about the Binary module for example. Isn't that whole module just for performance reasons more or less. I don't have a list of all new BIFs. It was just my understanding that every new release would have some new BIFS, that could be implemented in pure Erlang. What about min and max? Aren't they BIFs? Anyway, my point in this thread is just that performance matters, and when you compare languages you need to include the whole package including that you some language needs more work on performance than others. I was citing NIFs and BIFs etc to claim that other people than me cares about performance in Erlang. >> Steve, I have never seen a NIF that was not performance motivated. >> But you have seen many more NIFs than me. >> Do you have any examples? > I have two NIFs in the system I work on, for example, neither of which > is performance-oriented: > > * One NIF allows simple access the Linux UUID library so that my > Erlang code can generate UUIDs the same as the C++ components with > which it interoperates over the internal chassis network. This has > nothing to do with performance and everything to do with consistent > UUID generation across all the components in the chassis. > > * The other NIF enables control plane calls to some specialized > offload sockets on our video delivery hardware. The system components > providing these sockets are written in C, so the NIF imports their > header files to pick up all their #defines, constants, and function > prototypes, and then presents that functionality in Erlang in a way > that's natural and simple for the Erlang programmer to use. This NIF > has nothing to do with performance and everything to do with code > coherency, integration with the C code, and minimizing duplication > between Erlang and C. > Fine, so performance is not your reason to use more than pure Erlang. I will still claim that it is for many others. But it leads to the same conclusion, namely that functional languages are not so much easier. With such interfaces in your code, you are not supporting the notion of functional languages being easy, quite on the contrary. Morten. From hd2010@REDACTED Tue Jan 4 23:31:28 2011 From: hd2010@REDACTED (Henning Diedrich) Date: Tue, 04 Jan 2011 23:31:28 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D23743B.4010805@amberbio.com> References: <4D227D75.1060108@gmail.com> <4D230528.1000700@amberbio.com> <4D234BDE.3040209@eonblast.com> <4D23743B.4010805@amberbio.com> Message-ID: <4D239FC0.4050706@eonblast.com> Hi Morten, to get that straight up front: I am not arguing against imperative languages or against the virtues of C. This thread is about approaching an FP language, Erlang and I am addressing your argument that performance is an argument against any FP language. Or /can/ be as you correctly put it. And the keyword in all this was 'ease'. That 'can' seems to introduce a troubled logic and to induce some talking past each other. With 'can' you qualified correctly that it is /not/ always the case that FP /cannot/ be optimized, or by extension, could even come optimal right out of the box. So I am arguing two things: performance optimization is not always important, it is sometimes not even limited in FP. At the least. I would even say it is rarely important, and it is mostly not limited. We need not not agree on that, it will be hard to measure. Finally, it's no argument against the ease of FP languages, specifically Erlang, but to the contrary, it performance can in the future more and more be on the side of FP, in a perfectly batteries included way: you don't have to bother at all, which is the ultimate ease obviously. Also note, I absolutely love C and speed. And I do know the limitations of Erlang where raw calculation power is concerned. And I would still not use FORTRAN. Which is a good parallel, by the way, because you could do anything you do in FORTRAN in C, right, and probably optimize it further. Then why would FORTRAN /still/ exist? For the ease of the speed. Because it gives a better ratio of ease to speed. On 1/4/11 8:25 PM, Morten Krogh wrote: > On 1/4/11 5:33 PM, Henning Diedrich wrote: > > On 1/4/11 12:31 PM, Morten Krogh wrote: > > > > But inbetween, e.g. for the right list representation, you have > > four flavors in Erlang to optimze for performance. That's more than > > in, say, JS or Lua where you have basically one. Erlang is not > > religious Haskell in that regard and there are many more cases > > where Erlang opted for the practical solution. > > > What are those four flavors? The ets: set, ordered_set, bag and duplicate_bag. They are integral to Erlang and yes, that's not so FP, it's pure side effects. Erlang can be very practical where it has to. A good thing. > > Also, there's at least one imperative language, Python*, where the > > mantra is that your time is expensive, not the hardware that will > > provide for faster execution. In other words, development time > > should be optimized, not execution time. So yours seems not to be > > an argument for imperative languages per se. Some reject that view > > voluntarily. > > > Of course, you can always build a slow language, both functionally > and imperatively. But it does illustrate how explicitly unimportant performance can be, no matter if FP or imperative. And that imperative does not mean fast, or even optimisable. > The question is if it is possible to build a fast functional > language, especially without making BIFs and NIFS for everything. Yes that is possible and Erlang is. But that was not /actually/ the question, that was 'ease', and there Erlang mostly beats C by a long margin, C + MPI even longer. This is not just touchy feely estimation, it's a result of the implications of share-nothing processes. And that is no co-incidence, that distinction comes from Erlang's origins. It was made to be fool proof to a much higher degree than C. It has defense 'built in' as opposed to C, and way more sensibly than most other languages. It was made for parallel environments, as opposed to C. That was not as important a while back as it has become now with the advent of multi-core CPUs [7] AND of clusters as the new normal. That's why CouchDB [4] or Scalaris/x [5] exist and are written in Erlang. But the OP's question was on learning the language, getting the picture and you weighed in that FP /can/ be hard to optimize for speed, which I said is true but neither always important nor true without the qualifying 'can'. And if we need to wind to there, that was your qualified answer to the issue of 'ease of learning'. > > On 1/4/11 10:17 AM, Joe Armstrong wrote: Functional programming is > > *much* easier to understand than imperative programming. > > > Definitely not much easier. It depends on the problem, and whether you include > performance requirements. As soon as you want to control memory and performance, > functional languages can actually be much much harder. And I mean much harder. As illustrated by imperative languages that reject performance to be their purpose out of hand, your point won't count as a general point against an FP language that matters in all cases. I think we agree there. Yes, there are things for C as BIFS and NIFS illustrate and that's just fine. You can always go there. It's not the only spot where Erlang likes to be sequentiality. > And then the functional language is just a glorified C library. No, where do you take that from? Do you know what happens if you try to program continuation- passing style in an imperative language? You can do that but it has nothing whatsoever to do with ease any more. And single value variable binding becomes the natural thing to do on the way [8]. Yes they're all Turing complete and you can use MPI, and you might in the end use libevent to get the most out of a certain task. But again that doesn't take the "can" qualifier out of the argument, it can be true, but often it doesn't matter and for many cases it is not even true in the first place. > > to code or use libraries in other languages, as results from > > share-nothing memory and micro- processes. Think scaling on > > multi-core. That is really not trivial for other languages and > > where performance-gains are coming from today. I am not sure if you > > are aware to what degree e.g. Python, even Stackless Python simply > > forfeit that whole playing field wholesale, last time I checked. It > > simply depends on what you want to do. Finally, you get stuff for > > free in Erlang that you have > > This multicore stuff is somewhat off topic. it is hardly off topic if you talk about performance? Python can't really use it so well, C needs quite a bit overhead and offers a plethora of ways to kill yourself, Java looses it's niceness and presents programmers with dead lock opportunities who were quite glad to leave 0 pointer induced core dumps behind. > Multicore scaling is pure performance oriented, which you and many > others claim is not so important. Then what are we talking about? We are talking about how and why it may not be important in all cases, and even for the cases that it is, the qualifying "can" applies as to optimization limits. > Why bother about multicore when single core performance is considered > premature optimization? Something is missing in that sentence I guess. But again, for one it is not a killer argument against FP, and above that Erlang gives you stuff free that can cost you tripping up your favored C parallel library. Sure with endless time you can do everything in assembler and be faster then anything else, including C. Coming back to 'ease' ... > > But I am curious about what you mean. > > You think Erlang will scale better than the fast imperative languages > like C? What makes you say that? Because what is good scaling for your? A thousand node MySQL cluster? It is being done and billions are earned from it. But not because it actually works that well. More likely because it was to late to change at some point. Good scaling in the sense of less painful, with Erlang you don't even having to think about a parallelization step (well, almost not) that's a lightyear from bringing MPI in to C once the logic works. And that is because Erlang was made for this and does prevent certain things that become painful when you try to distribute stuff and there, Erlang processes, in their share-nothingness, are in fact an advantage. > > Is the claim that the otp team will write a better user space threading (process) library than anyone else? No, it's that the Erlang makers did it for you already in a way intimately woven into the fabric of the language so it's 'transparent' compared to the pains of getting familiar with, say C and on top of it, MPI. > Even in that case other people could copy their code. I assume the > scheduling and migration code is in C not Erlang or what? You mean it's all really C, and there is no end to optimizing? C might be about the only language you could use to emulate Erlang stuff and be faster, at the usual price. But unless you use Erlang as a library, maybe to write a domain language that might look like Erlang proper and then tweak a bit, maybe using an interface like NIFs ... it won't be anything but easy or at a good ease to speed ration. The gains from FP /are/ intrinsic in the language features. You don't have them when using straight imperative style. But sure Erlang "is" C underneath. So are most other languages, which cannot really be a point when discussing languages, or no other languages should exist in the first place, see FORTRAN. > > The kernel people, working at a lower level and in C, has an > advantage. They can integrate user threads with kernel threads. Where are you going here? That you *can* be faster than an Erlang program? That was my very first word: yes. And it's not the point. That there are ways to be faster than ANY Erlang program? Scientifically, 100% yes. But hardly the point either, as it applies to all languages, including C, if you dig deep enough. > [snip of convenience] > > Maybe Java's > garbage collector will get stuck when we get to thousands of cores. > Possibly. It gets stuck as soon as you go beyond 2GB. I mean, not officcialy, but in reality. And that is about global memory. [3] Best, Henning [1] http://www.erlang.org/doc/man/ets.html [2] https://computing.llnl.gov/tutorials/mpi/ [3] http://blogs.forrester.com/john_r_rymer/10-06-21-why_im_worried_about_javas_future [4] http://couchdb.apache.org/ [5] http://www.onscale.de/scalarix.html [6] http://stackoverflow.com/questions/641326/what-is-the-difference-between-multicore-programming-in-erlang-and-other-language [7] http://www.gotw.ca/publications/concurrency-ddj.htm [8] http://en.wikipedia.org/wiki/Static_single_assignment_form From co7eb@REDACTED Wed Jan 5 03:38:31 2011 From: co7eb@REDACTED (=?iso-8859-1?Q?Gilberio_Carmenates_Garc=EDa?=) Date: Tue, 4 Jan 2011 21:38:31 -0500 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D2378AC.20507@eonblast.com> References: <4D227D75.1060108@gmail.com> <002401cbac40$2ac3a3a0$804aeae0$@co.cu> <4D2378AC.20507@eonblast.com> Message-ID: <000501cbac81$a8138000$f83a8000$@co.cu> Hi, well because nobody likes C# because it's not all free and well Java is a big community language, so what other language you think I could translate it? The thing is that the EVO framework has a part in Erlang for the server development, and the other part is in .Net C#, you can use this other part in whichever of .Net languages except Silver Light because I need to recompile the dlls for it. But talking about .Net is not well received in the free soft community, so I?m in trouble because I?m in the middle of both tendencies, the free one and the not free like Microsoft, so, I know that C# itself is free if we use mono, but Who likes mono? I think the mono?s developers and its fans only. So that the real problem here, if I could find a community witch likes C# and Erlang and the same time I think my life shall be realized at all. Best, Ivan. De: Henning Diedrich [mailto:hd2010@REDACTED] Enviado el: Martes, 04 de Enero de 2011 14:45 Para: Gilberio Carmenates Garc?a Asunto: Re: [erlang-questions] Erlang and the learning curve Hi Gilberio, > Some day I will translate the .Net part to java. Why to Java? Cheers, Henning ======================================================================= Este mensaje ha sido enviado mediante el servicio de correo electr?nico que ofrece la Federaci?n de Radioaficionados de Cuba a sus miembros para respaldar el cumplimiento de los objetivos de la organizaci?n y su pol?tica informativa. La persona que env?a este correo asume el compromiso de usar el servicio a tales fines y cumplir con las regulaciones establecidas. From vinoski@REDACTED Wed Jan 5 05:03:37 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Tue, 4 Jan 2011 23:03:37 -0500 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D238C2E.8010900@amberbio.com> References: <4D227D75.1060108@gmail.com> <4D230528.1000700@amberbio.com> <4D236874.7070602@amberbio.com> <4D238C2E.8010900@amberbio.com> Message-ID: On Tue, Jan 4, 2011 at 4:07 PM, Morten Krogh wrote: > On 1/4/11 8:59 PM, Steve Vinoski wrote: >> >> On Tue, Jan 4, 2011 at 1:35 PM, Morten Krogh ?wrote: >>> >>> On 1/4/11 4:37 PM, Steve Vinoski wrote: >>>> >>>> On Tue, Jan 4, 2011 at 6:31 AM, Morten Krogh ? ?wrote: >>>>> >>>>> Why are one fourth, or whatever, of the posts on this list about nif or >>>>> c >>>>> nodes or ports? >>>> >>>> Even if we assume your percentage is accurate (which I don't), simply >>>> assuming these features are used only for performance reasons is a >>>> case of premature optimization. ;-) >>>> >>>> While some uses of these features involve performance concerns, C >>>> nodes, ports, drivers, and NIFs all serve as points for integrating >>>> Erlang with other environments. NIFs have already proven themselves >>>> particularly useful in this regard. Because of the continued growth in >>>> the popularity and application of Erlang, the number of developers >>>> interested in using these features to integrate with non-Erlang code >>>> is also growing. >>>> >>>> These are areas of Erlang involving languages other than Erlang, such >>>> as C, so it could be they're harder to understand than other parts of >>>> the language. It also could be that even with the aforementioned >>>> growing popularity of the language, the number of Erlang developers >>>> who've ever written C nodes and drivers used in production remains >>>> relatively small, and so finding good sources of detailed information >>>> about how best to write them is challenging. >>>> >>>> But still, I'd argue there really aren't that many questions posted >>>> here about these features. >>>> >>>> --steve >>> >>> My percentage is probably wrong. I didn't count. I just wanted to stress >>> that I see quite a lot of performance optimization in Erlang. >> >> As do I, but not by jumping out of Erlang. Using Erlang's profilers >> and other tools is very straightforward and can significantly help >> improve performance. >> > Yes, and that is also my point. Because people have think more about > performance in Erlang than in C or C++ say, profilers and other tools are > needed > earlier and should be included in a discussion of the difficulty of various > languages. ?Erlang with profilers and tools is a harder "language" than > Erlang without. So what you're claiming is the fact that Erlang provides profiling tools proves that it performs poorly? No offense, Morten, but that doesn't make sense. Name me a system that doesn't offer profiling tools and I'll name you a system that isn't used for real-world development. Erlang provides these tools so that programmers can analyze their programs at the level of abstraction of the language itself. I've never tried the following, but maybe you could instead use a tool like callgrind at the Erlang VM level to analyze your Erlang programs, but I think it would probably be pretty challenging to match your findings back to your Erlang source code. As far as I've seen, I use profiling tools (regardless of language) a lot more than most programmers, yet the amount of Erlang code I've worked on that I've felt the need to actually profile and measure is likely in the 2-3% range. I'm sure I'm not alone on this mailing list in my opinion that idiomatic Erlang code generally performs very well and needs no specific tuning or measurement. Also, consider this: some of the first hard Erlang performance issues I ever faced was when I tried tackling Tim Bray's original WideFinder problem several years ago. I was still relatively new to Erlang then and compared to what I know now, I barely knew anything, and yet the Erlang solution that I submitted was one of the fastest solutions among all languages for awhile. I'm sure if I looked at it now I'd find that the code I submitted was actually fairly terrible, and I knew nothing about Erlang's profilers then (I was just using the timer module to measure how long various functions took). Eventually my solution fell by the wayside in the contest because it had a bug somewhere that I was too inexperienced too resolve, but IIRC correctly the winner ended up being an OCaml program (and yes, it was faster than any C or C++ that was submitted). The point of all this, of course, is that all of this information runs contrary to your claims that Erlang requires deep, continuous focus on performance. In my experience, it doesn't. >>> Why were NIFs introduced at all? Why new BIFs all the time? All of that >>> could be done in Erlang. >> >> Only the OTP team can say for sure why they introduced NIFs, but my >> understanding is that it allows for easier integration with and access >> to C than drivers. I personally don't find drivers difficult, but I >> know they can be overkill for simple integration needs. >> > Maybe, but why C to start with. Performance must be a big issue here, but > anyway. The Erlang runtime system is written in C, so NIFs and drivers are C. What else would you write it in? C is a universal portable language, so it's a logical choice, especially given the timeframe in which Erlang development started. I don't know if Kresten's Erjang, implemented in Java, supports NIFs yet, but they'd be written in Java just as Erjang's drivers are written in Java. >> Are new BIFs really being introduced "all the time" strictly for >> performance reasons? If so can you cite some examples? > > I was thinking about the Binary module for example. Isn't that whole module > just for performance reasons more or less. I thought it provided a richer way of working with binaries. Are binaries present in the language only for performance reasons? Klacke can tell us for sure, but I don't think so; they're there to provide a simple yet extremely powerful abstraction for manipulating bytes and bits. When you're dealing with network protocols, like you do in the telecoms world for which Erlang was originally created, the need to do that easily and in a way that makes for readable and maintainable code is critically important. The fact that binaries and bitstrings also turn out to be useful in other app domains should come as no surprise. > I don't have a list of all new BIFs. It was just my understanding that every > new release would have some new BIFS, that could be > implemented in pure Erlang. What about min and max? Aren't they BIFs? I don't know off the top of my head, but I actually don't care because to me BIFs look and act like regular Erlang. > Anyway, my point in this thread is just that performance matters, and when > you compare languages you need to include the whole package including > that you some language needs more work on performance than others. I was > citing NIFs and BIFs etc to claim that other people than me cares about > performance > in Erlang. Sure, performance matters, but only in areas that are proven to be deficient via actual measurements of actual running programs. Still, performance is only one of a number of engineering trade-offs that have to be properly balanced in real-world systems. I invite you to read an article I wrote about this topic a few years ago, specifically about the misguided focus on middleware performance that seemed to be rampant in my world back then: http://steve.vinoski.net/pdf/IEEE-The_Performance_Presumption.pdf >>> Steve, I have never seen a NIF that was not performance motivated. >>> But you have seen many more NIFs than me. >>> Do you have any examples? >> >> I have two NIFs in the system I work on, for example, neither of which >> is performance-oriented: >> >> * One NIF allows simple access the Linux UUID library so that my >> Erlang code can generate UUIDs the same as the C++ components with >> which it interoperates over the internal chassis network. This has >> nothing to do with performance and everything to do with consistent >> UUID generation across all the components in the chassis. >> >> * The other NIF enables control plane calls to some specialized >> offload sockets on our video delivery hardware. The system components >> providing these sockets are written in C, so the NIF imports their >> header files to pick up all their #defines, constants, and function >> prototypes, and then presents that functionality in Erlang in a way >> that's natural and simple for the Erlang programmer to use. This NIF >> has nothing to do with performance and everything to do with code >> coherency, integration with the C code, and minimizing duplication >> between Erlang and C. >> > Fine, so performance is not your reason to use more than pure Erlang. I will > still claim that it is for many others. > But it leads to the same conclusion, namely that functional languages are > not so much easier. > With such interfaces in your code, you are not supporting the notion of > functional languages being easy, quite on the contrary. I have to call BS on you there, since you can't logically reach your conclusions from what's been discussed here. I wrote a relatively small amount of C in NIF form to integrate Erlang with the rest of our system, which means my fellow Erlang colleagues are able to use these facilities from their regular Erlang code without having to think about C. They're able to stay in the functional programming world and still get our specialized hardware to do what they need it to do -- it's the best of both worlds. I don't know how much actual real-world experience you have with functional programming, but everything I've ever done with FP tells me that yes, things actually *are* easier with FP -- the code is easier to write, easier to read, smaller, more maintainable, easier to debug, easier to extend, and it performs and scales reasonably well, and all of this can be done in less time than with most other languages. I've watched half a dozen of my colleagues learn Erlang from the start and they were all, without exception, able to become very proficient in a very short time, shorter than with any other language I've ever seen. If you have different experiences, then fine -- all that means is that you have different experiences with FP than many of us on this list. It certainly doesn't mean the IMO faulty broad conclusions you've been drawing trump our individual experiences. I'm bowing out of this discussion now, too much work to do. --steve From ulf.wiger@REDACTED Wed Jan 5 07:47:09 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 5 Jan 2011 07:47:09 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: References: <4D227D75.1060108@gmail.com> <4D230528.1000700@amberbio.com> <4D236874.7070602@amberbio.com> <4D238C2E.8010900@amberbio.com> Message-ID: On 5 Jan 2011, at 05:03, Steve Vinoski wrote: > I thought it provided a richer way of working with binaries. Are > binaries present in the language only for performance reasons? Klacke > can tell us for sure, but I don't think so; they're there to provide a > simple yet extremely powerful abstraction for manipulating bytes and > bits. When you're dealing with network protocols, like you do in the > telecoms world for which Erlang was originally created, the need to do > that easily and in a way that makes for readable and maintainable code > is critically important. The fact that binaries and bitstrings also > turn out to be useful in other app domains should come as no surprise. Steve, Just to straighten out the ancient history part, binaries were indeed introduced into the language for performance reasons - or rather, in order to provide a more semantically suitable datatype for handling network packets. This was back in the Good Old Days, when network protocols tended to be byte-coded rather than bulky text, so the most common ways to handle the packets were: - decode them, or extract some component part, based on offsets and byte codes - pass them along as payload Binaries are much more appropriate for this than linked lists of bytes, and protocol handling is so commonplace in Erlang that it makes perfect sense to provide the most suitable datatypes for the task. Note that originally, we had binaries without the bit syntax. You manipulated binaries using split_binary/2, binary_to_list/1, list_to_binary/1, and decoded them (after converting to lists) mainly with bsl, bsr, band and bor, which was quite horrible, especially when the data was bit-oriented rather than byte-oriented. The bit syntax was added to make the important task of encoding and decoding network packets more tractable. It was added for performance, and indeed, for several years, a hand-coded encode/decode would outperform the bit syntax. Still, practically everyone started using the bit syntax. The binary module was largely created because with the bit syntax, binaries turned out to be more suitable for many text processing tasks than lists (especially after providing a fast regexp library). Given that text representations are now ubiquitous also in protocol handling, and in other areas where Erlang has started becoming popular, it feels pretty natural to have a module that provides efficient text searching and manipulation operations - the kind that practically every mainstream language supports. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From rmelton@REDACTED Wed Jan 5 07:55:36 2011 From: rmelton@REDACTED (Robert Melton) Date: Wed, 5 Jan 2011 01:55:36 -0500 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D227D75.1060108@gmail.com> References: <4D227D75.1060108@gmail.com> Message-ID: Jack-- DON'T PANIC! You started a very lively thread, with some incredible feedback from some very bright people too modest to toot their own horns, but feel free to Google around for some of the people who replied, a ton of them have heavyweight experience, great production products you may well soon depend on, and at least one incredible book. This is a fantastic group. > I've been reading/watching everything I can find (off and online) for > learning Erlang for the past two or three weeks, but am just not getting it, > in terms of being able to do anything with Erlang in the real world. I am also a Erlang neophyte, learning as I go, by no means an expert. Here is some of my advice: #1. Don't be terribly discouraged if you can't find a library to do exactly what you want. One of my first little play projects after reading the "big three" books was to do minor (tens of thousands) html scraping. In the past I have used TokeParser (perl) and Hpricot (ruby) to do tasks like this, which are extremely userfriendly and easy to get started with... I was discouraged, but decided to jump in anyway, and was shocked by how easily the task broke down into components... and yes, it was very fast comparatively to those, both in development time and runtime. #2. Look at the great projects (Basho is a nice place to start IMHO), and really look at them, not just like you would a library in Perl, Ruby or Python -- open them up and follow along, understanding and integrating bits of Erlang out of major projects is very different than what you might be used too... you find something you need, just snipe it out and put it in your code, the joys of limited side effects. #3. Continue to use the community, I am overjoyed by how incredible the community is in general... specifically #erlang on freenode has been a wonderful place for me to idle and just get to listen to some of the great discussions. This community has a lot of people experienced in a plethora of languages, with gobs of real world experience (both in Erlang and other languages). > I've watched some videos about functional programming where the hosts joked > about people looking into functional programming deciding it's too hard and > then never touching it again... Functional programming in general is "different" (not harder) when you first bump into it, but once you get over the "difference", I wouldn't say it is any harder. As Joe pointed out, in many ways it is simpler, more straightforward and easier for a non-expert to understand (once you get past those first few rough days). > but seriously, how do people bridge the gap > between understanding the basics and implementing real world projects with > Erlang? I know the three major books are available, but none of them are > really using Erlang for what I'd like to do... The gap was non-existent for me, I have been doing development professional on some mission critical type stuff (real time trading / credit reporting, military command centers, live image tracking, etc) for over a decade. As I read up on Erlang, I was excited by the way it would have simplified my prior work, as I had poorly implemented parts of OTP multiple times over the years. I started replaying those projects in my head, and thought of the hundreds of hours I would have saved building them in Erlang. Most of them are in C++, but I did one in C#, and even an odd Java/Perl/PHP Hybrid slipped in. I would argue that the more real world stuff you have built, the more you understand and appreciate Erlang and its trade-offs. To me, the entire language and OTP seem to be mercilessly focused on one thing -- getting real world production systems up and running and keeping them maintained. It is NOT a high and mighty ivory tower language, it is a workhorse to its very core... beautiful at times, ugly at others... compromised by real world requirements. All these things together make me appreciate the language so much more. As for being unable to find your exact problem in a book, well how boring would that be? Anyway, if a book is written about your specific problem, maybe it is well solved. :) > How have any of you who have come to know big-picture how great Erlang is > been able to get enough knowledge of it to actually do what you'd like to > do? Well, the books give you an exceptional "core", you follow them and use OTP to build out your core, and maybe a few noteworthy other projects (in my case: webmachine, gproc, mochiweb, and various bits pulled out of other places). After you have all that in place, you should be sitting squarely on your problem domain. You problem domain is what you are writing the code to do -- so get started and have fun. If there were projects that covered your problem domain, what would we need you for? Also, you might have a little bit more success if you work your way along until you are "stuck" and then try to contact the community for help, because I am not sure where specifically you got stuck, so I can't reply in more than vague generalities, and for that I am sorry. Don't give up, keep plugging along, we will be here to help when you get stuck. -- Robert Melton | Contact Information @ http://robertmelton.com/contact From erlang@REDACTED Wed Jan 5 10:19:56 2011 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 5 Jan 2011 10:19:56 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: References: <4D227D75.1060108@gmail.com> <4D230528.1000700@amberbio.com> <4D236874.7070602@amberbio.com> <4D238C2E.8010900@amberbio.com> Message-ID: Re: Performance my take What do I think about performance C, NIFs etc. Erlang came from an industrial (Ericsson) lab - so our ideas about performance backward compatibility etc. are highly influenced by the environment we work in. What follows are my take on efficiency in our particular (industrial) environment. It's not a general position statement, but rather a reflection of how I view efficiency etc. in our environment. #1 - Performance is only has meaning with respect to a specified target performance. The spec says: we have to do this in 10 ms. If you do it in 9 ms it's ok - 11 ms is not. It's either good enough - or it's not. If it's not good enough we can tweak the algorithms (ie stay in the same language) or drop to a lower level language (C). Tweaking the algorithm is preferred since it's easier to stay in one language than use two languages. Also a bloody sight easier to maintain. #2 - The cost of optimization must be weighted against the cost of a) buying more hardware b) maintenance c) time to market c) quality d) delivery time it is by no means clear that optimization is good. In small runs - "throw more hardware at it" is often far cheaper than spending programmer hours on the problem. (Small run means we ship one product) If you ship a volume product - tens of thousands of units then you can spend enormous amounts of money on optimizations - since the *total* costs are minimized. #3 - NIFs are a convenient way to call C functions - you often want to call C to gain access to libraries that you do not want to recode in Erlang, or hit parts of the system that Erlang cannot reach. #4 - Optimizations in Erlang (think binaries) ) were made when there was a *generic* benefit from the optimization. ie we though all users would benefit - other optimizations (like reverse is a native function) were made because they were easy to do and have generic benefit. Doing reverse in C rather than Erlang is a purely pragmatic thing - there is no philosophy involved here. #5 - Erlang is not "just a layer on top of C" as some postings in this thread imply. It would be more accurate to think of the underlying "C" as composed of two parts. a) a run-time b) and interface to assembler. The run-time a) includes things like the garbage collector and process-scheduler (actually the gc etc. should have been written in a low level dialect of erlang and cross compiled to C (or assembler), but that's a duifferent story). The generated beam code is an example of b) - beam started as a set of C macros, but the macros themselves looked like instructions in some weird assembler language. For pragmatic reasons it's better to interpret rather than compile the C macro (don't ask ...). In this sence C is used as a very thin layer over assembler. I guess one day in the not too distance future we'll JIT the beam code and bypass C. When this happens Erlang will in no way represent a layer over C. #6 - efficient linking of foreign code assumes that caller and callee both obey the same ABI (Application Binary Interface) - ABIs reflect either a) the real machine b) a VM Examples of a) are the ABI 32 bit code on an intel X86, or the ADM64 conventions for 64 bit systems. All ABIs reflecting real processes have an incredibly anal approach to data. An integer passed as an argument in a register or on the stack must have *exactly* 32 or 64 bits. Saying that integers are (say) always exactly 32 bits and that "it is the programmers responsibility to avoid integer overflows" results in systems that are highly efficient, but the code is error prone, difficult to prove correct and (very) difficult to extent if you need to extend the domain. examples: In 32 bits you can address 4GBytes of memory - addressing more memory will cause a major rewrite of your code. If 32 bits precision doesn't cut it and you need 64 bits - then a major rewrite is needed. b) VM's suffer no such problems. If you want 87 bit registers and bignum arithmetic then this is easy (if you design your own VM) - if you choose a "well known" VM like .NET or the JVM then you have to live with all the design constraints imposed by the VM - these can be just as bothersome as living with an ABI. You can JIT the VM to a real machine code, but JIT'ing will never be eccicient if there is not a 1:1 relationship between the VM and the underlying machine. If the VM defined registers of 33 bits then JIT'ing to a 32 bit processor would never be efficient. #7 - Erlang was designed for building fault-tolerant systems A lot of the design decisions in erlang were made to reduce the number of programmer errors. bignums are there to eliminate precision problems in inter arithmetic. We *don't* use a binary ABI to include foreign code since this would make the system unsafe. These design decisions make the system inherently less efficient - it is *designed* to be safe not efficient. The amazing thing is that we can even compete in terms of efficiency with system that were designed to be efficient but not safe. Safety costs - if you need the extra performance do it *outside* erlang in C - that's the way we designed it to be. This is not a bug or a feature it was designed this way. NIFS BTW make it *somewhat* easier to include foreign code and use a ABI interface at the expense of making the system unsafe. There is a trade off between safety and perfomance here. if you want safety - don't use NIFs- If you want performance AND ACCEPT THE RISKS INVOLVED use NIFs - this is the same trade off as offered by external port programs and linked in drivers. I'd almost always go with external port programs - they are pretty easy to write and you don't get into ABI nightmares (I tried making a NIF with a mixtures of 64 bit and 32 shared libraries on a mac - mixing 32 and 64 bit native compiled code is a complete and total mess, unless you know exactly what you are doing (which I don't)) #8 - real efficiency is done by moving from low level languages into hardware For example, parsing XML is about 3000 times faster on an FPGA than the fastest C++ parser Decisions to go to hardware involve massive escalations in cost and time to market. #9 - finally The old masters said it all 'first make it right, then make it fast' This is what we do: (We = ///) 1) hack it in erlang 2) fast enough - ship it 3) tweak the erlang 4) fast enough? - ship it 5) hack it in C? 6) fast enough - ship it 7) tweak the C 8) fast enough? - ship it 9) make an FPGA 10) fast enough - ship it 11) make an ASIC 12) fast enough - ship it As you go down this list things cost more and more - step 11 costs 1M$/try - to pass step 9 you need to have a high volume product (10's to hundreds of thousands of units) If you ship one unit you'll hover around 1) + 3) and throw more hardware at the problem. Decisions to optimize or not have long reaching consequences for cost of maintenance, quality, etc. In mail groups like this most discussion will hover round the 1) - 7) - what happens beyond this 8) - 12) is cloaked in secrecy - yeah we make chips - what do they do? how do they work? what are they programmed in? - that's all secret - you'll have to guess. Now I have to do some real work - somewhere in the 8) - 12) region - can't tell you what :-) /Joe On Wed, Jan 5, 2011 at 7:47 AM, Ulf Wiger wrote: > > On 5 Jan 2011, at 05:03, Steve Vinoski wrote: > >> I thought it provided a richer way of working with binaries. Are >> binaries present in the language only for performance reasons? Klacke >> can tell us for sure, but I don't think so; they're there to provide a >> simple yet extremely powerful abstraction for manipulating bytes and >> bits. When you're dealing with network protocols, like you do in the >> telecoms world for which Erlang was originally created, the need to do >> that easily and in a way that makes for readable and maintainable code >> is critically important. The fact that binaries and bitstrings also >> turn out to be useful in other app domains should come as no surprise. > > > Steve, > > Just to straighten out the ancient history part, binaries were indeed > introduced into the language for performance reasons - or rather, in > order to provide a more semantically suitable datatype for handling > network packets. This was back in the Good Old Days, when network > protocols tended to be byte-coded rather than bulky text, so the most > common ways to handle the packets were: > > - decode them, or extract some component part, based on offsets and > ?byte codes > - pass them along as payload > > Binaries are much more appropriate for this than linked lists of bytes, > and protocol handling is so commonplace in Erlang that it makes perfect > sense to provide the most suitable datatypes for the task. > > Note that originally, we had binaries without the bit syntax. You manipulated > binaries using split_binary/2, binary_to_list/1, list_to_binary/1, and decoded > them (after converting to lists) mainly with bsl, bsr, band and bor, which was > quite horrible, especially when the data was bit-oriented rather than byte-oriented. > > The bit syntax was added to make the important task of encoding and decoding > network packets more tractable. It was added for performance, and indeed, > for several years, a hand-coded encode/decode would outperform the bit > syntax. Still, practically everyone started using the bit syntax. > > The binary module was largely created because with the bit syntax, binaries > turned out to be more suitable for many text processing tasks than lists > (especially after providing a fast regexp library). Given that text representations > are now ubiquitous also in protocol handling, and in other areas where > Erlang has started becoming popular, it feels pretty natural to have a module > that provides efficient text searching and manipulation operations - the kind > that practically every mainstream language supports. > > BR, > Ulf W > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From alavrik@REDACTED Wed Jan 5 10:30:23 2011 From: alavrik@REDACTED (Anton Lavrik) Date: Wed, 5 Jan 2011 03:30:23 -0600 Subject: Record-like syntax for dictionaries Message-ID: Hi all, While playing with Erlang parse transformations, I came up with the idea of using a record-like syntax for basic operations with dictionaries from stdlib's "dict" module. Dictionary construction and accessing elements by keys are conceptually similar to record construction and accessing record fields, however standard dictionary interface is rather cumbersome compared to Erlang record syntax. I created a parse transformation to test this idea. This is a usage example: -include_lib("erl_aliases.hrl"). -dict_alias(''). dict_alias_test() -> % creating an empty dictionary _ = #{}, % associating foo with 1 D = #{foo = 1}, 1 = D.foo, % setting foo to foo + 1 and baz to 100 D1 = D#{foo = D.foo + 1}#{baz = 100}, 2 = D1.foo, 100 = D1.baz, % accessing undefined field {'EXIT', {badarg, _}} = (catch D.bar), % several elements D2 = D#{bar = 1, fum = 10, obj = D}, 1 = D2.bar, 10 = D2.fum, % accessing field of a dict included in another dict 1 = D2.obj.foo, ok. This is the result of the parse transformation: dict_alias_test() -> _ = dict:new(), D = dict:store(foo, 1, dict:new()), 1 = dict:fetch(foo, D), D1 = dict:store(baz, 100, dict:store(foo, dict:fetch(foo, D) + 1, D)), 2 = dict:fetch(foo, D1), 100 = dict:fetch(baz, D1), {'EXIT',{badarg,_}} = (catch dict:fetch(bar, D)), D2 = dict:store(obj, D, dict:store(fum, 10, dict:store(bar, 1, D))), 1 = dict:fetch(bar, D2), 10 = dict:fetch(fum, D2), 1 = dict:fetch(foo, dict:fetch(obj, D2)), ok. Note that in the beginning I defined a dict alias for an empty atom: -dict_alias(''). If any other non-empty atom is used, then dictionary construction and element access look exactly as correspondent record operations. For example, with -dict_alias(dict), #dict{} will be transformed to dict:new(), X#dict.a -- to dict:fetch(a, X) and so on. This was the original idea, but then I discovered that it is possible to get rid of record names, i.e. use #{} instead of #dict{} and X.foo instead of X#dict.foo, etc. This, however, required definition of three extra Yecc rules in erl_parse.yrl to handle expressions like #{...}. Obviously, such parse transformations work only with expressions -- they are not applied to guards and patterns. The code is on GitHub (https://github.com/alavrik/erl_aliases), but this functionality is not documented yet. Overall, this looks pretty neat to me. Beyond simple use, I'm thinking, for example, about converting arbitrary JSON objects to dictionaries and then accessing their contents using this method. Parsed JSON object keys can be converted to atoms using binary_to_existing_atom. What do you think? Is there anything I'm missing here that makes such parse transformation potentially unreliable? Anton From rtrlists@REDACTED Wed Jan 5 11:45:35 2011 From: rtrlists@REDACTED (Robert Raschke) Date: Wed, 5 Jan 2011 10:45:35 +0000 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D23743B.4010805@amberbio.com> References: <4D227D75.1060108@gmail.com> <4D230528.1000700@amberbio.com> <4D234BDE.3040209@eonblast.com> <4D23743B.4010805@amberbio.com> Message-ID: On Tue, Jan 4, 2011 at 7:25 PM, Morten Krogh wrote: > And then the functional language is just a glorified C library. C is just a glorified Assembler library. From nick@REDACTED Wed Jan 5 12:02:53 2011 From: nick@REDACTED (Niclas Eklund) Date: Wed, 5 Jan 2011 12:02:53 +0100 Subject: [erlang-questions] ASN1 compilation error in OTP R14 In-Reply-To: <4D23200D.80704@myriadgroup.com> References: <4D23200D.80704@myriadgroup.com> Message-ID: Hello! Thank you for reporting this! Seems like it was introduced in version 1.6.13.1 (R13B). We'll look into this. Best Regards, Niclas E @ Erlang/OTP On Tue, 4 Jan 2011, Stephane Pamelard wrote: > Hi all, > Does someone have any idea about this asn1 problem ? > > Previously, we compiled our code with otp_R11B (that was ok). > Now, we try to compile with otp_R14 and we have an issue with ASN compiler. > > You can find below: > - outputs with R14 compilator (with error) > - outputs with R11B compilator (no error) > - ASN file > > ------------------ Result with otp_R14 --------------------------- > make -k > /usr/erlang/otp_R14B/lib/erlang/bin/erlc +noobj -bber_bin CAP.asn > asn1error:23:'CAP':'ExtensionField' > {internal_error, > {function_clause, > [{asn1ct_check,componentrelation_leadingattr, > [{state, > {module,1,'CAP',[],'IMPLICIT',{exports,all},{imports,[]},undefined,[]}, > 'CAP', > {typedef,false,23,'ExtensionField', > {type,[], > {'SEQUENCE',false,false,undefined, > [{'ComponentType',24,type, > {type,[], > {'ObjectClassFieldType', > {'Externaltypereference',24,'CAP','EXTENSION'}, > {'Externaltypereference',24,'CAP','EXTENSION'}, > [{valuefieldreference,id}], > undefined}, > [{simpletable, > {type,[], > {'Externaltypereference',24,'CAP','SupportedExtensions'}, > [],[],no}}], > [],no}, > mandatory,undefined,undefined}, > {'ComponentType',25,criticality,.................. > > ------------------ Result with otp_R11B --------------------------- > > otp_R11 > make -k > /usr/erlang/otp_R11B-5/lib/erlang/bin/erlc +noobj -bber_bin CAP.asn > Erlang ASN.1 version "1.4.5" compiling "/home/CAP.asn" > Compiler Options: [ber_bin, > report_errors, > {cwd,"/home/src"}, > {outdir,"/home/src"}, > noobj, > {i,"."}, > {i,"/home/src"}] > --{generated,"/home/src/CAP.asn1db"}-- > --{generated,"/home/src/CAP.hrl"}-- > --{generated,"/home/src/CAP.erl"}-- > ok > > ------------------ ASN CODE ------------------------- > CAP {ccitt(0) identified-organization(4) etsi(0) mobileDomain(0) > umts-network(1) modules(3) cap-datatypes(52) version3(2)} > > DEFINITIONS IMPLICIT TAGS ::= > > BEGIN > > EXTENSION ::= CLASS { > &ExtensionType, > &criticality CriticalityType DEFAULT ignore, > &id Code > } > WITH SYNTAX { > EXTENSION-SYNTAX &ExtensionType > CRITICALITY &criticality > IDENTIFIED BY &id > } > > ExtensionField ::= SEQUENCE { > type EXTENSION.&id ({SupportedExtensions }), > criticality CriticalityType DEFAULT ignore, > value [1] EXTENSION.&ExtensionType ({SupportedExtensions > }{@type}), > ...} > > SupportedExtensions EXTENSION ::= {firstExtension, ...} > > firstExtension EXTENSION ::= { > EXTENSION-SYNTAX NULL > CRITICALITY ignore > IDENTIFIED BY global : {itu-t(0) identified-organization(4) > organisation(0) gsm(1) > capextension(2)}} > > CriticalityType ::= ENUMERATED { > ignore (0), > abort (1) > } > > Code ::= CHOICE {local INTEGER, > global OBJECT IDENTIFIER} > > > END > > Thanks for your help. > > St?phane > > > > > This message, including attachments, is intended solely for the addressee > indicated in this message and is strictly confidential or otherwise > privileged. If you are not the intended recipient (or responsible for > delivery of the message to such person) : - (1) please immediately (i) notify > the sender by reply email and (ii) delete this message and attachments, - (2) > any use, copy or dissemination of this transmission is strictly prohibited. > If you or your employer does not consent to Internet email messages of this > kind, please advise Myriad Group AG by reply e-mail immediately. Opinions, > conclusions and other information expressed in this message are not given or > endorsed by Myriad Group AG unless otherwise indicated by an authorized > representative independent of this message. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From michael.eugene.turner@REDACTED Wed Jan 5 13:00:54 2011 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Wed, 5 Jan 2011 21:00:54 +0900 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: References: <4D227D75.1060108@gmail.com> <4D230528.1000700@amberbio.com> <4D236874.7070602@amberbio.com> <4D238C2E.8010900@amberbio.com> Message-ID: "I'm sure I'm not alone on this mailing list in my opinion that idiomatic Erlang code generally performs very well and needs no specific tuning or measurement." And those few who would beg to differ ought to go look at the sobering Great Shootout benchmarks and compare Erlang with PHP. PHP isn't just slow. It's *gob-smackingly* slow. Some of the PHP benchmarks fail because they time out after an *hour* (where the fastest implementations, in languages like C++ and Ada, finish in seconds, and Erlang versions are hardly more than an order of magnitude slower than the fastest.) A great deal of the world's server-room fan-noise must be generated by the need to carefully scale PHP/MySQL apps. I wouldn't be surprised if to learn that there are many niches where an Erlang/NoSQL solution would stay uniprocessor and reach market saturation for its services, where the equivalent PHP/MySQL implementation would require considerable babysitting and diaper changes to scale up sensibly. -michael turner On Wed, Jan 5, 2011 at 1:03 PM, Steve Vinoski wrote: > On Tue, Jan 4, 2011 at 4:07 PM, Morten Krogh wrote: > > On 1/4/11 8:59 PM, Steve Vinoski wrote: > >> > >> On Tue, Jan 4, 2011 at 1:35 PM, Morten Krogh wrote: > >>> > >>> On 1/4/11 4:37 PM, Steve Vinoski wrote: > >>>> > >>>> On Tue, Jan 4, 2011 at 6:31 AM, Morten Krogh > wrote: > >>>>> > >>>>> Why are one fourth, or whatever, of the posts on this list about nif > or > >>>>> c > >>>>> nodes or ports? > >>>> > >>>> Even if we assume your percentage is accurate (which I don't), simply > >>>> assuming these features are used only for performance reasons is a > >>>> case of premature optimization. ;-) > >>>> > >>>> While some uses of these features involve performance concerns, C > >>>> nodes, ports, drivers, and NIFs all serve as points for integrating > >>>> Erlang with other environments. NIFs have already proven themselves > >>>> particularly useful in this regard. Because of the continued growth in > >>>> the popularity and application of Erlang, the number of developers > >>>> interested in using these features to integrate with non-Erlang code > >>>> is also growing. > >>>> > >>>> These are areas of Erlang involving languages other than Erlang, such > >>>> as C, so it could be they're harder to understand than other parts of > >>>> the language. It also could be that even with the aforementioned > >>>> growing popularity of the language, the number of Erlang developers > >>>> who've ever written C nodes and drivers used in production remains > >>>> relatively small, and so finding good sources of detailed information > >>>> about how best to write them is challenging. > >>>> > >>>> But still, I'd argue there really aren't that many questions posted > >>>> here about these features. > >>>> > >>>> --steve > >>> > >>> My percentage is probably wrong. I didn't count. I just wanted to > stress > >>> that I see quite a lot of performance optimization in Erlang. > >> > >> As do I, but not by jumping out of Erlang. Using Erlang's profilers > >> and other tools is very straightforward and can significantly help > >> improve performance. > >> > > Yes, and that is also my point. Because people have think more about > > performance in Erlang than in C or C++ say, profilers and other tools are > > needed > > earlier and should be included in a discussion of the difficulty of > various > > languages. Erlang with profilers and tools is a harder "language" than > > Erlang without. > > So what you're claiming is the fact that Erlang provides profiling > tools proves that it performs poorly? No offense, Morten, but that > doesn't make sense. Name me a system that doesn't offer profiling > tools and I'll name you a system that isn't used for real-world > development. > > Erlang provides these tools so that programmers can analyze their > programs at the level of abstraction of the language itself. I've > never tried the following, but maybe you could instead use a tool like > callgrind at the Erlang VM level to analyze your Erlang programs, but > I think it would probably be pretty challenging to match your findings > back to your Erlang source code. > > As far as I've seen, I use profiling tools (regardless of language) a > lot more than most programmers, yet the amount of Erlang code I've > worked on that I've felt the need to actually profile and measure is > likely in the 2-3% range. I'm sure I'm not alone on this mailing list > in my opinion that idiomatic Erlang code generally performs very well > and needs no specific tuning or measurement. > > Also, consider this: some of the first hard Erlang performance issues > I ever faced was when I tried tackling Tim Bray's original WideFinder > problem several years ago. I was still relatively new to Erlang then > and compared to what I know now, I barely knew anything, and yet the > Erlang solution that I submitted was one of the fastest solutions > among all languages for awhile. I'm sure if I looked at it now I'd > find that the code I submitted was actually fairly terrible, and I > knew nothing about Erlang's profilers then (I was just using the timer > module to measure how long various functions took). Eventually my > solution fell by the wayside in the contest because it had a bug > somewhere that I was too inexperienced too resolve, but IIRC correctly > the winner ended up being an OCaml program (and yes, it was faster > than any C or C++ that was submitted). > > The point of all this, of course, is that all of this information runs > contrary to your claims that Erlang requires deep, continuous focus on > performance. In my experience, it doesn't. > > >>> Why were NIFs introduced at all? Why new BIFs all the time? All of that > >>> could be done in Erlang. > >> > >> Only the OTP team can say for sure why they introduced NIFs, but my > >> understanding is that it allows for easier integration with and access > >> to C than drivers. I personally don't find drivers difficult, but I > >> know they can be overkill for simple integration needs. > >> > > Maybe, but why C to start with. Performance must be a big issue here, but > > anyway. > > The Erlang runtime system is written in C, so NIFs and drivers are C. > What else would you write it in? C is a universal portable language, > so it's a logical choice, especially given the timeframe in which > Erlang development started. I don't know if Kresten's Erjang, > implemented in Java, supports NIFs yet, but they'd be written in Java > just as Erjang's drivers are written in Java. > > >> Are new BIFs really being introduced "all the time" strictly for > >> performance reasons? If so can you cite some examples? > > > > I was thinking about the Binary module for example. Isn't that whole > module > > just for performance reasons more or less. > > I thought it provided a richer way of working with binaries. Are > binaries present in the language only for performance reasons? Klacke > can tell us for sure, but I don't think so; they're there to provide a > simple yet extremely powerful abstraction for manipulating bytes and > bits. When you're dealing with network protocols, like you do in the > telecoms world for which Erlang was originally created, the need to do > that easily and in a way that makes for readable and maintainable code > is critically important. The fact that binaries and bitstrings also > turn out to be useful in other app domains should come as no surprise. > > > I don't have a list of all new BIFs. It was just my understanding that > every > > new release would have some new BIFS, that could be > > implemented in pure Erlang. What about min and max? Aren't they BIFs? > > I don't know off the top of my head, but I actually don't care because > to me BIFs look and act like regular Erlang. > > > Anyway, my point in this thread is just that performance matters, and > when > > you compare languages you need to include the whole package including > > that you some language needs more work on performance than others. I was > > citing NIFs and BIFs etc to claim that other people than me cares about > > performance > > in Erlang. > > Sure, performance matters, but only in areas that are proven to be > deficient via actual measurements of actual running programs. Still, > performance is only one of a number of engineering trade-offs that > have to be properly balanced in real-world systems. I invite you to > read an article I wrote about this topic a few years ago, specifically > about the misguided focus on middleware performance that seemed to be > rampant in my world back then: > > http://steve.vinoski.net/pdf/IEEE-The_Performance_Presumption.pdf > > >>> Steve, I have never seen a NIF that was not performance motivated. > >>> But you have seen many more NIFs than me. > >>> Do you have any examples? > >> > >> I have two NIFs in the system I work on, for example, neither of which > >> is performance-oriented: > >> > >> * One NIF allows simple access the Linux UUID library so that my > >> Erlang code can generate UUIDs the same as the C++ components with > >> which it interoperates over the internal chassis network. This has > >> nothing to do with performance and everything to do with consistent > >> UUID generation across all the components in the chassis. > >> > >> * The other NIF enables control plane calls to some specialized > >> offload sockets on our video delivery hardware. The system components > >> providing these sockets are written in C, so the NIF imports their > >> header files to pick up all their #defines, constants, and function > >> prototypes, and then presents that functionality in Erlang in a way > >> that's natural and simple for the Erlang programmer to use. This NIF > >> has nothing to do with performance and everything to do with code > >> coherency, integration with the C code, and minimizing duplication > >> between Erlang and C. > >> > > Fine, so performance is not your reason to use more than pure Erlang. I > will > > still claim that it is for many others. > > But it leads to the same conclusion, namely that functional languages are > > not so much easier. > > With such interfaces in your code, you are not supporting the notion of > > functional languages being easy, quite on the contrary. > > I have to call BS on you there, since you can't logically reach your > conclusions from what's been discussed here. I wrote a relatively > small amount of C in NIF form to integrate Erlang with the rest of our > system, which means my fellow Erlang colleagues are able to use these > facilities from their regular Erlang code without having to think > about C. They're able to stay in the functional programming world and > still get our specialized hardware to do what they need it to do -- > it's the best of both worlds. > > I don't know how much actual real-world experience you have with > functional programming, but everything I've ever done with FP tells me > that yes, things actually *are* easier with FP -- the code is easier > to write, easier to read, smaller, more maintainable, easier to debug, > easier to extend, and it performs and scales reasonably well, and all > of this can be done in less time than with most other languages. I've > watched half a dozen of my colleagues learn Erlang from the start and > they were all, without exception, able to become very proficient in a > very short time, shorter than with any other language I've ever seen. > If you have different experiences, then fine -- all that means is that > you have different experiences with FP than many of us on this list. > It certainly doesn't mean the IMO faulty broad conclusions you've been > drawing trump our individual experiences. > > I'm bowing out of this discussion now, too much work to do. > > --steve > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From mk@REDACTED Wed Jan 5 13:22:42 2011 From: mk@REDACTED (Morten Krogh) Date: Wed, 05 Jan 2011 13:22:42 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: References: <4D227D75.1060108@gmail.com> <4D230528.1000700@amberbio.com> <4D234BDE.3040209@eonblast.com> <4D23743B.4010805@amberbio.com> Message-ID: <4D246292.1000605@amberbio.com> It is fine with me to claim that often performance doesn't matter, but then the imperative languages of relevance are perl, python, ruby, js etc. Functional languages are not easier or more productive than these. In some sense functional languages just constrain themselves by giving up loops and other constructs that are useful. But I still think that a lot of what I see in the Erlang world, is performance motivated. Why not interface with other languages through the file system then. It is very robust and easily debuggable. Interfacing with some legacy code could be done by running the legacy code in a separate OS process, make it write the result to a file and have Erlang open the file. erlang could do with sockets and file access. Why aren't many of these libraries written in functional languages to start with? Why hasn't the higher productivity in functional languages led to all those libraries being available in functional languages. Why is it more important for Erlang to interface with C, C++, Java etc than with Haskell, Lisp, Ocaml. Same question for them. Does Haskell care about interfacing with Erlang to get libraries? Functional languages are old, there have been many functional programmers, functional programmers are often very very skilled. Where are all those libraries that the imperative world doesn't have? Henning, do you care to make an exact falsifiable claim about the multicore scalability of Erlang? What will happen, say in 2020? What kind of application is it that Erlang will have but not C or any other imperative language? How exactly will we recognize Erlang's victory over all imperative languages in multicore programming? Personally, I don't see why C can not have threading libraries with work scheduling and migration in a way that is approachable for many application programmers. Why is the Erlang environment even considered functional? There is a runtime in C, there is ets, the process dictionary, nifs, lots of bifs. The important parts of Erlang don't seem to be the functional syntax but more many of the other parts. On 1/5/11 11:45 AM, Robert Raschke wrote: > On Tue, Jan 4, 2011 at 7:25 PM, Morten Krogh wrote: > >> And then the functional language is just a glorified C library. > > C is just a glorified Assembler library. > I don't thinks so. Let me explain where I see the difference. It is a quantitative difference, not a qualitative one. C is much more productive than assembler, and the speed loss in C is not that big. All higher level languages after C, are not that much more productive than C, or they lose a lot of performance. C hit a sweet spot. Some day, there will be a higher level language that displaces C, sure, but when? You can see the difference in how new algorithms are implemented. Say someone invents a new type of data structure, an abc tree, say. Then C programmers would write abc tree in ansi C and publish it as a library. Erlang would almost certainly not write it in Erlang. The Erlang module abc_tree would just be a wrapper to the C functions. Erlang people might even write their own C program instead of using the C people's library and wrap it. I would make similar statements for other languages. Python would just write a wrapper to C etc. Erlang could have done something completely different. Made a small core 20 years ago, and then stayed within pure Erlang. The abc_tree would be written in pure Erlang. The thought of writing a wrapper in C could have been silly. Someone invents a new cryptographic algottihm. C people would implement it in C. Assembler would not be worth it. It is too difficult, unportable, and the performance gain is minor. Erlang, python, java?, perl, ruby, haskell etc would write a wrapper to a C function. The situation is not symmetric. Some arbitrary numbers: C is 20 time more productive and portable than assembler, C is 2-3 times slower. Erlang is 0.5 - 3 times more productive than C, and say 10 or more times slower. (Erlang speed is higher the more you use the wrappers to C of course, but that is forbidden). I am not picking on Erlang. The same is true for all other higher level languages, and is, in my opinion, the reason that old C is still so popular. In the productivity performance trade-off C has been extraordinarily successful. No later language in my opinion has done that. Probably almost no one will agree with me, but anyway. Cheers, Morten. so most code in written in C, and it is very rare to move into assembler in order to speed up C. There can be other unrelated assembler programs, but that is a different story. From hynek@REDACTED Wed Jan 5 14:50:33 2011 From: hynek@REDACTED (Hynek Vychodil) Date: Wed, 5 Jan 2011 14:50:33 +0100 Subject: [erlang-questions] SSL memory consumption In-Reply-To: References: Message-ID: On Tue, Jan 4, 2011 at 4:11 PM, Kaiduan Xie wrote: > Thank you Hynek. I had figured out the problem. The root cause is that > every time a gen_fsm is created in way 2, it inherits the '$ancestors' > from its parent, and its parent is also added to the '$ancestors' of > the process's dictionary. > > '$ancestors' of 1st gen_fsm: [p0], > '$ancestors' of 2nd gen_fsm: [p0, p1], > '$ancestors' of 3rd gen_fsm: [p0, p1, p2], > > > .... > '$ancestors' of nth gen_fsm: [p0, p1, p2, ...., pn-1] > > After resetting the '$ancestors' before creating gen_fsm by calling > put('$ancestors', Parent), the memory footprint is reduced > dramatically. What is the usage of '$ancestors'? It is OTP related stuff (proc_lib mainly). > > Best regards, > > /Kaiduan > > On Tue, Jan 4, 2011 at 5:21 AM, Hynek Vychodil wrote: >> On Thu, Dec 30, 2010 at 6:32 AM, Kaiduan Xie wrote: >>> After tried two different ways to call ssl:transport_accept and >>> ssl:ssl_accept, the following observation was found, >>> >>> 1. Calls ssl:transport_accept and ssl:ssl_accept in a server loop, and >>> for each SSL connection, starts a gen_fsm to receive data. The code is >>> listed as below, >>> >>> loop3(LSock) -> >>> ? ?{ok, Socket} = ssl:transport_accept(LSock), >>> ? ?ok = ssl:ssl_accept(Socket), >>> ? ?{ok, {PeerAddr, PeerPort}} = ssl:peername(Socket), >>> ? ?io:format("Accepted peer:~p:~p:~p:~p~n", [self(),PeerAddr, PeerPort, I]), >>> ? ?{ok, Pid} = gen_fsm:start(erlsip_transport_tls_connection, >>> {Socket, PeerAddr, PeerPort}, []), >>> ? ?ssl:controlling_process(Socket, Pid), >>> ? ?loop3(LSock). >>> >>> 2. Calls ssl:transport_accept, and ssl:ssl_accept in a gen_fsm. Upon a >>> SSL connection is accepted, starts another gen_fsm to accept new SSL >>> connection. The code is listed as below. >>> >>> init({server, LSocket, Parent}) -> >>> ? ?{ok, wait_for_connection, #state{lsocket = LSocket, >>> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? parent = Parent}, 0}; >>> wait_for_connection(timeout, State) -> >>> ? ?io:format("Waiting for TLS connection ... ~p/~p~n", [self(), State#state.par >>> ent]), >>> ? ?{ok, Socket} = ssl:transport_accept(State#state.lsocket), >>> ? ?ok = ssl:ssl_accept(Socket), >>> ? ?gen_fsm:start(erlsip_transport_tls_connection, >>> ? ? ? ? ? ? ? ? {server, State#state.lsocket, State#state.parent}, >>> ? ? ? ? ? ? ? ? ?[]), >>> ? ?{ok, {PeerAddr, PeerPort}} = ssl:peername(Socket), >>> ? ?io:format("Accepted peer:~p/~p:~p~n", [self(), PeerAddr, PeerPort]), >>> ? ?{next_state, wait_for_crlf_crlf, State#state{socket = Socket, >>> ? ? ? ? ? ? ? ? ? peer_address = PeerAddr, peer_port = PeerPort}}. >>> >>> Memory consumption per connection in the second way is about 120 K >>> with R14B01 while a connection takes about 30 K in first way. >>> >>> I am very happy with the result, but can someone explain the difference? >>> >>> Thanks, >>> >>> /Kaiduan >> >> I'm just guessing but it can be GC issue. In first case you do all >> those calls inside one process so all used structures inside loop3 can >> be GCed. In second case you do this sequence always in new process and >> when there are not future reductions (there is not any communication - >> fsm doesn't work) this memory is not freed. Try add explicit GC to >> your init/1 in second case and measure again. >> >>> On Thu, Dec 23, 2010 at 9:31 PM, Kaiduan Xie wrote: >>>> I just replaced R14A with R14B01, each SSL connection consumes about >>>> 120 K memory, 50K less than that of R14A. >>>> >>>> Thank Tony, Jesper for the help, can we reduce more? >>>> >>>> /Kaiduan >>>> >>>> On Thu, Dec 23, 2010 at 8:38 PM, Jesper Louis Andersen >>>> wrote: >>>>> On Fri, Dec 24, 2010 at 02:33, Kaiduan Xie wrote: >>>>> >>>>>> What do you mean the old ssl and new ssl, Tony? >>>>> >>>>> SSL was replaced recently with a new implementation and you have the >>>>> 'new' version here if you are on R14b >>>>> >>>>> -- >>>>> J. >>>>> >>>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >> >> >> >> -- >> --Hynek (Pichi) Vychodil >> >> Analyze your data in minutes. Share your insights instantly. Thrill >> your boss.? Be a data hero! >> Try GoodData now for free: www.gooddata.com >> > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From andy.kriger@REDACTED Wed Jan 5 15:56:02 2011 From: andy.kriger@REDACTED (Andy Kriger) Date: Wed, 5 Jan 2011 09:56:02 -0500 Subject: [erlang-questions] Record-like syntax for dictionaries In-Reply-To: References: Message-ID: That looks great. Is there a way that fetching a non-existent field can return undefined instead of requiring a catch? On Wed, Jan 5, 2011 at 4:30 AM, Anton Lavrik wrote: > Hi all, > > While playing with Erlang parse transformations, I came up with the > idea of using a record-like syntax for basic operations with > dictionaries from stdlib's "dict" module. Dictionary construction and > accessing elements by keys are conceptually similar to record > construction and accessing record fields, however standard dictionary > interface is rather cumbersome compared to Erlang record syntax. > > I created a parse transformation to test this idea. This is a usage > example: > > -include_lib("erl_aliases.hrl"). > -dict_alias(''). > dict_alias_test() -> > % creating an empty dictionary > _ = #{}, > > % associating foo with 1 > D = #{foo = 1}, > 1 = D.foo, > > % setting foo to foo + 1 and baz to 100 > D1 = D#{foo = D.foo + 1}#{baz = 100}, > 2 = D1.foo, > 100 = D1.baz, > > % accessing undefined field > {'EXIT', {badarg, _}} = (catch D.bar), > > % several elements > D2 = D#{bar = 1, fum = 10, obj = D}, > 1 = D2.bar, > 10 = D2.fum, > > % accessing field of a dict included in another dict > 1 = D2.obj.foo, > ok. > > This is the result of the parse transformation: > > dict_alias_test() -> > _ = dict:new(), > D = dict:store(foo, 1, dict:new()), > 1 = dict:fetch(foo, D), > D1 = > dict:store(baz, 100, dict:store(foo, dict:fetch(foo, D) + 1, D)), > 2 = dict:fetch(foo, D1), > 100 = dict:fetch(baz, D1), > {'EXIT',{badarg,_}} = (catch dict:fetch(bar, D)), > D2 = dict:store(obj, D, dict:store(fum, 10, dict:store(bar, 1, D))), > 1 = dict:fetch(bar, D2), > 10 = dict:fetch(fum, D2), > 1 = dict:fetch(foo, dict:fetch(obj, D2)), > ok. > > Note that in the beginning I defined a dict alias for an empty atom: > -dict_alias(''). If any other non-empty atom is used, then dictionary > construction and element access look exactly as correspondent record > operations. For example, with -dict_alias(dict), #dict{} will be > transformed to dict:new(), X#dict.a -- to dict:fetch(a, X) and so on. > This was the original idea, but then I discovered that it is possible > to get rid of record names, i.e. use #{} instead of #dict{} and X.foo > instead of X#dict.foo, etc. This, however, required definition of > three extra Yecc rules in erl_parse.yrl to handle expressions like > #{...}. > > Obviously, such parse transformations work only with expressions -- > they are not applied to guards and patterns. > > The code is on GitHub (https://github.com/alavrik/erl_aliases), but > this functionality is not documented yet. > > Overall, this looks pretty neat to me. Beyond simple use, I'm > thinking, for example, about converting arbitrary JSON objects to > dictionaries and then accessing their contents using this method. > Parsed JSON object keys can be converted to atoms using > binary_to_existing_atom. > > What do you think? Is there anything I'm missing here that makes such > parse transformation potentially unreliable? > > Anton > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Be well, andy Welcome to http://householder-yogi.net On family, NYC, and practicing yoga. From attila.r.nohl@REDACTED Wed Jan 5 15:59:56 2011 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Wed, 5 Jan 2011 15:59:56 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D246292.1000605@amberbio.com> References: <4D227D75.1060108@gmail.com> <4D230528.1000700@amberbio.com> <4D234BDE.3040209@eonblast.com> <4D23743B.4010805@amberbio.com> <4D246292.1000605@amberbio.com> Message-ID: 2011/1/5, Morten Krogh : [...] > Why aren't many of these libraries written in functional languages to > start with? Why hasn't the higher productivity in > functional languages led to all those libraries being available in > functional languages. Why is it more important for Erlang to interface > with C, C++, Java etc than with Haskell, Lisp, Ocaml. Same question for > them. Does Haskell care about interfacing with Erlang to get libraries? > Functional languages are old, there have been many functional > programmers, functional programmers are often very very skilled. Where > are all those libraries > that the imperative world doesn't have? Writing a library is much harder than writing an application. Also less rewarding. The software developers are very rarely tasked to "create a library", they are paid to create an application that can be sold. Because the "time to market" is a very important factor, it's understandable that the fast "create a wrapper over existing code" is chosen instead of "reimplement all that stuff from scratch", when some functionality is necessary. Also don't forget that some of the existing C libraries are out there for 10-20-30 years, which means 10-20-30 years of testing - it will take a good while for the new Erlang library to reach the same level of maturity, especially that there are much fewer users out there. Also there's the fact that C is the lingua franca of today's computing. Virtually every platform has a C compiler, so if you want to create a library that can be used on many platforms, C is the way to go. For this reason every other language has bindings for C to make these libraries available. [...] > C is much more productive than assembler, and the speed loss in C is not > that big. > All higher level languages after C, are not that much more productive > than C, or they lose a lot of performance. C hit a sweet spot. > Some day, there will be a higher level language that displaces C, sure, > but when? For the applications I think it already happened a decade ago. The whole web-based stuff is Java/PHP/JavaScript/Perl/Erlang/Ruby/etc. On Linux the whole KDE is written in C++, the iPhone applications are written in Objective-C, the Android applications in a Java-dialect, etc. The C holdouts are low-level stuff (e.g. systemd for Linux - it needs to run without a VM) or code that needs good portability between UNIXes. From ebegumisa@REDACTED Wed Jan 5 16:17:39 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Thu, 06 Jan 2011 02:17:39 +1100 Subject: [erlang-questions] Processes & Fault Tolerance In-Reply-To: References: Message-ID: On Tue, 04 Jan 2011 20:52:50 +1100, Joe Armstrong wrote: > On Mon, Jan 3, 2011 at 5:30 PM, Edmond Begumisa > wrote: >>> Let's start with how we do error recovery. >>> >>> Imagine two linked processes A and B on separate machines. A is the >>> master process. >>> B is a passive processes that will take over if A fails. >>> >>> A sends a stream of state update messages S1, S2, S3,... to B. The >>> state messages contain enough information >>> for B to do whatever A was doing should A fail. If A fails B will >>> receive an EXIT signal. >>> >>> If B does receive an exit signal it knows A has failed, so it carries >>> on doing whatever A was doing >>> using the information it last received in a state update message. >>> >>> That's it and all about it - nothing to do with supervisors etc, >> >> Ooooohh! THANK YOU, THANK YOU, THANK YOU. > > Thanks. > > Another problem in understand is that we assume the meaning of words. > Yes. There was some confusion about whether we were all talking about replicating valid state or isolating invalid state. Which I guess "fault-tolerance" could be interpreted to mean either. But I'm no expert on the subject so I shouldn't comment on terminology -- I just knew the feature I wanted and was pretty sure it could be done at the process level, I just had no-idea how! From now on, I'll stick to the term "redundancy" or Ulf's "hot take-over" so it's clearer what I mean. - Edmond - > Once upon a time I was talking to a guy and used the word "fault > tolerance" > we talked for hours and thought we understood what this word meant. Then > he said something that implied he was building a fault-tolerant system > on one computer. > I said it was impossible. He said, "you catch and handle all the > exceptions .." > > I said, "The entire computer might crash" > > He said, "Oh" > > There was a long silence. > > I told him about Erlang ... > > > /Joe > >> >> *Now* I get it. This is precisely what I was trying to understand. The >> missing link was sending redundant messages. It all makes sense now -- >> it's >> so simple. All I have to do to get fault tolerance at the process level >> is >> have a group of N redundant processes waiting for exit-signals and >> forward >> state changes to N-1 of them. I could even send to N/2 and have some >> decent >> tolerance at the expense of consistency of the nodes. >> >> To preemptively answer the question I sent Ulf and Mazen... for that >> distributed database, Process B would be set up to also receive read >> requests forwarded from Process A but not respond to them unless it >> gets an >> exit signal it can understand like disk_fail_error. Any changes in state >> would also be forwarded to existing redundant processes. >> >> Thanks everyone for your patience in helping me understand >> fault-tolerance. >> I've already started thinking about some cool things I can do with this >> in >> my program. >> >> - Edmond - >> >> >> >> On Tue, 04 Jan 2011 00:36:30 +1100, Joe Armstrong >> wrote: >> >>> I think you are getting confused between OTP supervisors etc. and the >>> general notion of "take-over" >>> >>> Let's start with how we do error recovery. >>> >>> Imagine two linked processes A and B on separate machines. A is the >>> master process. >>> B is a passive processes that will take over if A fails. >>> >>> A sends a stream of state update messages S1, S2, S3,... to B. The >>> state messages contain enough information >>> for B to do whatever A was doing should A fail. If A fails B will >>> receive an EXIT signal. >>> >>> If B does receive an exit signal it knows A has failed, so it carries >>> on doing whatever A was doing >>> using the information it last received in a state update message. >>> >>> That's it and all about it - nothing to do with supervisors etc, >>> >>> >>> To make the above possible we need a couple of primitives in Erlang, >>> namely process_flag trap_exits, and spawn_link. That's all you need. >>> >>> The basic system properties you need to make a fault-tolerant systems >>> are the abilities to detect remote errors and send and receive >>> messages. >>> >>> What the OTP supervisors etc. do are build layers of abstraction on >>> top of spawn-link and trap_exits >>> in order to simplify programming common use-cases. >>> >>> Fault detection and recovery is not arbitrary or magic in any sense, >>> It has to be part of the system architecture like everything else. One >>> common fault in making systems is to specify and design for the normal >>> behavior >>> but not specify and design how fault-detection and correction work. >>> This is probably due to a legacy of >>> programming in languages that have very limited ability to detect >>> errors and recover from them. >>> >>> A C programmer with a single thread of control will take extreme >>> measures to avoid crashing their program >>> they have to, there is only one thread of control. Given multiple >>> threads of control and the ability to remotely >>> detect errors your entire way of thinking changes and the emphasis >>> changes from thinking "how to I avoid a crash" >>> to thinking "given that a crash has occurred and been detected, how to >>> I fix things up and carry on" >>> >>> The Erlang philosophy is that "things will crash", so we have to >>> detect the crashes, fix the bit that broke and >>> carry on - we do not make excessive efforts to avoid crashes, but we >>> do try very hard to repair things after they have gone wrong. >>> >>> Fixing things after they have broken is often easier than preventing >>> them from breaking. Preventing them >>> from breaking is not theoretically possible - we can't even prove >>> simple things, like that a program will terminate >>> (the famous halting problem) - but we can easily detect failures and >>> put the system to rights by applying >>> certain invariants. >>> >>> OTP supervisors are just trees of processes with certain restart rules >>> that experience has shown us >>> to be useful. >>> >>> The main value of supervisors etc. (and of all the OTP behaviors) is >>> in building large systems with many >>> programmers involved. Rather than inventing their own patterns, the >>> programmers reuse a common set of >>> patterns, which makes the projects easier to manage. >>> >>> /Joe >>> >>> >>> >>> >>> On Mon, Jan 3, 2011 at 6:21 AM, Edmond Begumisa >>> wrote: >>>> >>>> Thanks for your response. >>>> >>>> Firstly, let me make my question a little clearer... >>>> >>>> To rephrase: For processes, "share nothing for the sake of >>>> concurrency" - >>>> I >>>> get, both in concept and application. "Share nothing for the sake of >>>> fault-tolerance" - I get in concept but not in application. >>>> >>>> Yet as I understand it, it is for the latter reason Erlang shares >>>> nothing* >>>> and not the former. Interpretation: I must be completely missing the >>>> point >>>> in regards to Erlang processes and sharing nothing. This is what I >>>> want >>>> to >>>> understand in application. In addition to the "side" effect of sane >>>> concurrency (which coming from a chaotic multi-threading shared-memory >>>> world >>>> I fully appreciate and practically make use of everyday), how can I >>>> also >>>> make use of the "real" reason Erlang processes share nothing -- fault >>>> tolerance? Practically/illustratively speaking? >>>> >>>> *ETS being the obvious exception. >>>> >>>> Secondly, here's a mantra from Joe Armstrong... >>>> >>>> @ minute 17:26 >>>> http://www.se-radio.net/2008/03/episode-89-joe-armstrong-on-erlang/ >>>> >>>> "[message passing concurrency]... the original reasons have to do with >>>> fault >>>> tolerance... you have to copy all the data you need from computer 1 to >>>> computer 2... if computer 1 crashes you take over on computer 2... you >>>> can't >>>> have dangling pointers... that's the reason for copying everything... >>>> it's >>>> got nothing to do with concurrency, it's got a lot to do with >>>> fault-tolerance... if they don't crash you could just have a dangling >>>> pointer and copy less data but it won't work in the presence of >>>> errors..." >>>> >>>> I interpret this to mean that share-nothing between processes is more >>>> about >>>> replicating valid state than isolating corrupted state as you >>>> described. >>>> >>>> Indeed, Joe created an example on his blog... >>>> >>>> >>>> http://armstrongonsoftware.blogspot.com/2007/07/scalable-fault-tolerant-upgradable.html >>>> >>>> It's algorithm 3 there I'm struggling with. Particularly where he >>>> says... >>>> >>>> "... In practise we would send an asynchronous stream of messages >>>> from N >>>> to >>>> N+1 containing enough information to recover if things go wrong." >>>> >>>> Unfortunately, I couldn't find part II to that post (I don't think >>>> there >>>> is >>>> one.) And I'm too green and inexperienced in the field of >>>> fault-tolerant >>>> systems to figure it out on my own. I'm having trouble visualising the >>>> practical here from the conceptual -- I need to be shown how :( >>>> >>>> Also, I seem to be under the impression that the Erlang language has >>>> some >>>> sort of schematics to do this built-in (i.e. deal with one process >>>> taking >>>> over from another if the first fails) and this is the reason processes >>>> share >>>> nothing. This seems to me to be something different from supervision >>>> trees, >>>> which use exit-trapping to re-spawn if a process fails with the active >>>> 'job' >>>> disappearing and any errors logged (like restarting a daemon). My >>>> interpretation of the fault-tolerance Erlang is supposed to enable >>>> (for >>>> those in the know) is seamless take-over. The 'job' lives on but >>>> elsewhere. >>>> >>>> Using telecoms as an example: a phone call wouldn't be cut-off when a >>>> fault >>>> occurs, another node would seamlessly take over. This is how I >>>> interpreted >>>> Joe's post and other descriptions of Erlang's fault-tolerant features >>>> and >>>> I >>>> understand the key is in the share nothing policy for processes. I'm >>>> sure >>>> I've mis-understood something or everything :) >>>> >>>> - Edmond - >>>> >>>> PS: I've read the Manning draft. Great book. I don't know if the >>>> answer >>>> lies >>>> in OTP (I searched and didn't find it). I suspect it's lower -- >>>> probalby >>>> how >>>> you organise your processes. Some distributed-programming black-magic >>>> only >>>> Erlanger's know about :) >>>> >>>> >>>> On Mon, 03 Jan 2011 14:56:51 +1100, Alain O'Dea >>>> wrote: >>>> >>>>> On 2011-01-02, at 22:36, "Edmond Begumisa" >>>>> >>>>> wrote: >>>>> >>>>>> Slight correction... >>>>>> >>>>>> On Mon, 03 Jan 2011 12:38:38 +1100, Edmond Begumisa >>>>>> wrote: >>>>>> >>>>>>> Hello all, >>>>>>> >>>>>>> I've been trying to wrap my Erlang's fault tolerant features >>>>>>> particularly in relation to processes. >>>>>>> >>>>>> >>>>>> Should be: I've been trying to wrap my head around Erlang's fault >>>>>> tolerant features particularly in relation to processes. >>>>>> >>>>>> Sorry. >>>>>> >>>>>>> I've heard/read repeatedly that the primary reason why Erlang's >>>>>>> designers opted for a share-nothing policy is not rooted in >>>>>>> concurrency but >>>>>>> rather in fault-tolerance. When nothing is shared, everything is >>>>>>> copied. >>>>>>> When everything is copied processes can take over from one another >>>>>>> when >>>>>>> things fail. I follow this reasoning but I don't follow how to >>>>>>> apply >>>>>>> it. >>>>>>> >>>>>>> I fully understand and appreciate how supervision trees are used to >>>>>>> restart processes if they fail. What I don't get is what to do when >>>>>>> you >>>>>>> don't want to restart but want to take over, say on another node. I >>>>>>> know >>>>>>> that at a higher-level, OTP has some take-over/fail-over schematics >>>>>>> (at the >>>>>>> application level.) I'm trying to understand things at the >>>>>>> processes >>>>>>> level - >>>>>>> why Erlang is the way it is so I can better use it to make my >>>>>>> currently >>>>>>> fault-intolerant program fault tolerant. >>>>>>> >>>>>>> Specifically, how can one process take over from another if it >>>>>>> fails? >>>>>>> It >>>>>>> appears to may that the only way to do this would be to somehow >>>>>>> retrieve not >>>>>>> only the state of the process (say, gen_server's state) but also >>>>>>> the >>>>>>> messages in its mailbox. Where does the design decision to >>>>>>> share-nothing for >>>>>>> the sake of fault-tolerance come into play for processes? Please >>>>>>> help >>>>>>> me >>>>>>> "get" this! >>>>>>> >>>>>>> Thanks in advance. >>>>>>> >>>>>>> - Edmond - >>>>>>> >>>>>>> >>>>> >>>>> Hi Edmond: >>>>> >>>>> Share-nothing helps with concurrent fault-tolerance by preventing one >>>>> process from corrupting the state of another. Receive is a process' >>>>> choice >>>>> and it corrupts its own state if it receives bad data and lets it in. >>>>> >>>>> AFAIK OTP fault-tolerance doesn't mean no requests will fail, it >>>>> means >>>>> the >>>>> system/sub-system will recover if a single request causes a process >>>>> to >>>>> crash. It's kind of like proper try/catch recovery applied to >>>>> concurrent >>>>> code. How you recover from the crash depends on the supervision >>>>> strategy >>>>> chosen. In some cases the supervisor can pass the state to the >>>>> replacement >>>>> process. In others this isn't necessary or even desirable since the >>>>> state >>>>> itself may involve resources lost in the crash or corrupted state >>>>> that >>>>> led >>>>> to the crash. >>>>> >>>>> I am straying outside my knowledge here so this paragraph is >>>>> guesswork. >>>>> The message queue for a gen_server need not necessarily be lost when >>>>> the >>>>> callback module crashes. In theory OTP could (and might already) >>>>> simply >>>>> delegate the messages to the replacement process following a crash. >>>>> Someone >>>>> who knows OTP better than me would need to weigh in here though. >>>>> >>>>> I found http://manning.com/logan very informative in understanding >>>>> OTP >>>>> and >>>>> its supervisor hierarchies. >>>>> >>>>> Cheers, >>>>> Alain >>>>> ________________________________________________________________ >>>>> erlang-questions (at) erlang.org mailing list. >>>>> See http://www.erlang.org/faq.html >>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>>> >>>> >>>> >>>> -- >>>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> >> -- >> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From alavrik@REDACTED Wed Jan 5 17:18:53 2011 From: alavrik@REDACTED (Anton Lavrik) Date: Wed, 5 Jan 2011 10:18:53 -0600 Subject: [erlang-questions] Record-like syntax for dictionaries In-Reply-To: References: Message-ID: On Wed, Jan 5, 2011 at 8:56 AM, Andy Kriger wrote: > That looks great. Is there a way that fetching a non-existent field can > return undefined instead of requiring a catch? No, but it is easy to add support for such behavior. However, I wouldn't make it the default and I'm not sure what would be the best way to specify which of the two behaviors you want. In general, I'm reluctant to introducing complicated features that are based on parse transformations. This is why, for instance, it is allowed to define -dict_alais() only once per module. Adding configurable behavior would make things less explicit -- looking at the code you wouldn't be able to tell which behavior is applied without looking at the configuration. Anton > > On Wed, Jan 5, 2011 at 4:30 AM, Anton Lavrik wrote: > >> Hi all, >> >> While playing with Erlang parse transformations, I came up with the >> idea of using a record-like syntax for basic operations with >> dictionaries from stdlib's "dict" module. Dictionary construction and >> accessing elements by keys are conceptually similar to record >> construction and accessing record fields, however standard dictionary >> interface is rather cumbersome compared to Erlang record syntax. >> >> I created a parse transformation to test this idea. This is a usage >> example: >> >> -include_lib("erl_aliases.hrl"). >> -dict_alias(''). >> dict_alias_test() -> >> ? ?% creating an empty dictionary >> ? ?_ = #{}, >> >> ? ?% associating foo with 1 >> ? ?D = #{foo = 1}, >> ? ?1 = D.foo, >> >> ? ?% setting foo to foo + 1 and baz to 100 >> ? ?D1 = D#{foo = D.foo + 1}#{baz = 100}, >> ? ?2 = D1.foo, >> ? ?100 = D1.baz, >> >> ? ?% accessing undefined field >> ? ?{'EXIT', {badarg, _}} = (catch D.bar), >> >> ? ?% several elements >> ? ?D2 = D#{bar = 1, fum = 10, obj = D}, >> ? ?1 = D2.bar, >> ? ?10 = D2.fum, >> >> ? ?% accessing field of a dict included in another dict >> ? ?1 = D2.obj.foo, >> ? ?ok. >> >> This is the result of the parse transformation: >> >> dict_alias_test() -> >> ? ?_ = dict:new(), >> ? ?D = dict:store(foo, 1, dict:new()), >> ? ?1 = dict:fetch(foo, D), >> ? ?D1 = >> ? ? ? ?dict:store(baz, 100, dict:store(foo, dict:fetch(foo, D) + 1, D)), >> ? ?2 = dict:fetch(foo, D1), >> ? ?100 = dict:fetch(baz, D1), >> ? ?{'EXIT',{badarg,_}} = (catch dict:fetch(bar, D)), >> ? ?D2 = dict:store(obj, D, dict:store(fum, 10, dict:store(bar, 1, D))), >> ? ?1 = dict:fetch(bar, D2), >> ? ?10 = dict:fetch(fum, D2), >> ? ?1 = dict:fetch(foo, dict:fetch(obj, D2)), >> ? ?ok. >> >> Note that in the beginning I defined a dict alias for an empty atom: >> -dict_alias(''). If any other non-empty atom is used, then dictionary >> construction and element access look exactly as correspondent record >> operations. For example, with -dict_alias(dict), #dict{} will be >> transformed to dict:new(), X#dict.a -- to dict:fetch(a, X) and so on. >> This was the original idea, but then I discovered that it is possible >> to get rid of record names, i.e. use #{} instead of #dict{} and X.foo >> instead of X#dict.foo, etc. This, however, required definition of >> three extra Yecc rules in erl_parse.yrl to handle expressions like >> #{...}. >> >> Obviously, such parse transformations work only with expressions -- >> they are not applied to guards and patterns. >> >> The code is on GitHub (https://github.com/alavrik/erl_aliases), but >> this functionality is not documented yet. >> >> Overall, this looks pretty neat to me. Beyond simple use, I'm >> thinking, for example, about converting arbitrary JSON objects to >> dictionaries and then accessing their contents using this method. >> Parsed JSON object keys can be converted to atoms using >> binary_to_existing_atom. >> >> What do you think? Is there anything I'm missing here that makes such >> parse transformation potentially unreliable? >> >> Anton >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > > -- > Be well, > andy > > Welcome to http://householder-yogi.net > On family, NYC, and practicing yoga. > From jesper.louis.andersen@REDACTED Wed Jan 5 18:40:46 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 5 Jan 2011 18:40:46 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D230528.1000700@amberbio.com> References: <4D227D75.1060108@gmail.com> <4D230528.1000700@amberbio.com> Message-ID: On Tue, Jan 4, 2011 at 12:31, Morten Krogh wrote: I am late to the ball, but: > Definitely not much easier. It depends on the problem, and whether you > include > performance requirements. As soon as you want to control memory and > performance, > functional languages can actually be much much harder. And I mean much > harder. I disagree totally. First of all, performance and control of memory are orthogonal entities. You don't need memory control to get performance. In an imperative setting, it is clear that you can explicitly manipulate memory to avoid allocation, go for cache optimizations and so on. In functional languages, the trick is to prefer algorithmic optimizations. Take operations research for instance: The algorithm is everything. Even if you get the problem to run 300 times faster, it is only a little bit more than 8 variables (math:pow(2,8)) you can add to the problem. An algorithmic optimization, like a cutting-plane algorithm or a clique cut generator easily lets you add 5000-10000 variables. The advantage of a higher level language in general is that you can write code faster. So you can choose the optimal algorithm everywhere for a fraction extra development time. For large projects where there is no singular hot-spot intensive kernel in the code that takes all the cycles, C is dwarfed. A video decoder is a prime example of an application where Assembly language or C is beneficial. There are certain operations which benefit from running as fast as possible - and hence they are written in assembly. Functional languages naturally have their blessings and curses. But I think the blessings far outweigh the curses. Functional languages tend to be simpler than imperative counterparts - especially in the semantics department. And by default, there are virtually no errors pertaining to state manipulation, good locality for the reader and predictable execution rules. Of course the price is to be had in raw speed - something the functional programmer will try to counterbalance by algorithmic optimization. Performance optimizations are not free. They cost resources, most notably time. The point is: How do you prioritize your resources to maximize their potential. My claim is that you can get a much better benefit from using a functional language because: * There are far fewer errors in the resulting programs to weed out. * They are way easier to maintain - everything is essentially straightforward data flow questions. * It is way easier to abstract code (Higher order functions and anonymous functions is a blast here). * Symbolic manipulation is much easier in functional programming. and hence, more time can be spent on getting clever algorithms into your designs. And if the etorrent application I write is any measurement point, we have around the same performance as the C BitTorrent clients now. This is with no HiPE by the way :) Why? Because much of the performance is handled by the VM and because the key algorithms are chosen to be fast. And I am not even counting the algorithmic optimizations a co-author is doing on etorrent right now. We expect it to be even faster. > Why are one fourth, or whatever, of the posts on this list about nif or c > nodes or ports? It has already been covered: Often it is way easier to reuse a library written in another language than it is to rewrite it yourself. If you can make the use seamless by the use of a port or a NIF, then why shouldn't we? If somebody spent 3-5 years meticulously optimizing a library in another language, we can expect to have to use some time on getting there. Even if we utilize that we are the second person to try it so we don't have to carry out all the mistakes, and even if we hypothesize we can write it faster in pure Erlang, it is simply not worth the resource cost. Nay, it is a better solution to steal and reuse. > Even forgetting completely about performance, there are lots of problems > that are more easily > solved and written using imperative techniques. Hardly. The last formal definition I made embedded an imperative language inside a functional one because it is that easy. It is easy to transform an imperative program into SSA-form and SSA is functional programming, see: http://www.cs.princeton.edu/~appel/papers/ssafun.ps which of course begs the question "Is imperative and functional programming really that different?" At least it shows what we already knew: Any imperative program is easily transformed into a functional one. But the problem you are hinting at is that the resulting program might not be idiomatic to functional principles. This is the reason it is "more easily" solved in imperative programs: The exposition is imperative in the paper from which you take the algorithm. But if you think about it, you will realize that many of the techniques have a "dual" counterpart in FP so you can get an elegant solution anyway. > You have pure functions in all languages, even if the type system doesn't > force it on you. I fail to see how the type system matter. It is more a question of semantics. > I don't think it is such a big problem in C. You almost always understand > whether a function has side effects or not. > Library functions rarely do, except for possibly changing pointer arguments, > which is documented in the api. Interestingly, library subroutines may be re-entrant or not. So even if the API interface is nicely pure, the internals of the library call may pose to be problematic if you have thread or co-routine access to the library. You can of course document this behavior as well - but in a functional setting this is way easier to ensure. > But is Erlang so different here. I could write a sin function, that changed > a file, say, during execution. > The issue is the same in C and Erlang. Library writers should not do it and > mostly don't. > Do you feel this is a real problem in C? Erlang is not a purely functional language (Its message communication is deeply imperative). Neither is Ocaml (it has an imperative subset). Writing code in functional style is a matter of responsibility on the programmer. Yet I claim that the primitives for doing this in Erlang or Ocaml are by far better than those of C for achieving this style. > What are the functions that are reused in Erlang but not in C? C doesn't have closures. So anything with closure has to be explicitly closure-converted by the programmer. In effect, you don't see code that pass function pointers that much. Contrast with Erlang, where you can do it easily. Since hoisting pieces of code and abstracting function control flow desperately needs closures, I think I've made my point. >> I really think that FPLs are an order of magnitude easier to >> understand than conventional PLs - >> > What? An order of magnitude? This is probably true. The key difference is the execution model. In C it is -> W' where Stmt is a statement to be executed, W and W' are *worlds* which is the current state of the machine. In FP the execution model is closer to: Exp -> Exp' Where Exp, Exp' are *expressions* and execution happen by reducing expressions. The key difference is that state in FP is explicitly mentioned in the expression, whereas in C the state is partially hidden in W. To understand the program, you must understand what values W can take as well. Erlang is of course more complicated because there is a reduction possibility Exp -> Exp for any process that is alive at the moment - and the execution is non-deterministic to boot. Yet, in my experience, it is still easier to grasp what the program is doing. > And most lines of code in C are easy to understand: > > sum = 0; > for (i = 0; i < 1000000; i++) { > ? ?sum += i; > } > > It is longer than a one liner in a FPL, but it is effort less to write. It is not effortless to read though. > I think it is more correct to say that all languages have domains where they > are strong and others where they are weak, but if > memory and performance really matters, you actually need languages that are > closer to the hardware, and they are all imperative at the moment. It is more specific than that. It takes a specific kind of performance requirements before it is worth getting closer to the hardware. For a lot of programs, C is a much better choice than assembly language. Because the compiler is able to do optimizations automatically which are good enough to dominate the effort it would take a human being to do the same. The very same argument is true with Erlang: It is simpler to express the program and get algorithmic optimizations into it - especially if the program is concurrent in Erlangs case. The elephant in the room currently that might completely change the status quo is of course the multi-core question. It doesn't pertain that much to Erlang as it does to FP in general: If you have 1024 cores or more, the raw execution speed of each core matter less in the long run compared to how many of those you can get to do useful work. And it may also be that the shared-memory model has to go - completely altering the state of what is the "best thing" to do. -- J. From jetkoten@REDACTED Wed Jan 5 23:52:01 2011 From: jetkoten@REDACTED (JETkoten) Date: Wed, 05 Jan 2011 17:52:01 -0500 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: References: <4D227D75.1060108@gmail.com> <4D230528.1000700@amberbio.com> Message-ID: <4D24F611.7040907@gmail.com> Hi All, Thanks for the variety of perspectives and resources that you've shared regarding my learning curve with Erlang. I'm still making my way through them all, and am very grateful for such an overwhelming response. Two pieces that have really resonated with me so far are Henning's point (as I took it) that each programming language basically has its own core usage domain and built-in strengths that don't necessarily overlap with other languages. Therefore, my expectation that I can come into Erlang and accomplish this kind of screen scraping auto-login stuff that I got partly prototyped in just a couple of hours using the Nokogiri and Mechanize Ruby gems (being more or less new to programming and new to Ruby) is pretty much just as unreasonable as expecting that I could come into Ruby and write a fault-tolerant, distributed, concurrent application framework like OTP in just a couple of hours. Good point. Next was Joe's admonition to put Erlang into my spine instead of just putting into my brain. In all this time I've been trying to wrap my head around Erlang, Now I can see that I should have been spending more time wrapping my spine around it. :) For my books project, I was formerly going to try and approach it from start to finish implementing the flow of the functionality starting with the http connection (that's not coming easy for me right now) and moving on step by step... but since reading some of the posts I've decided to see if I can just build whatever parts look like they make sense to me now (like the data structure for a book) and go from there, while working on Erlang in general with the books and online tutorials everyone has shared. Lastly, thanks for your several offers of help with specifics... I have a number of other things going on right now and don't have as much time to work on Erlang as I might like at the moment, but as I run into specific implementation issues I will gladly post back again and ask for the help that you have kindly offered! Thanks *very* much, Jack From ebegumisa@REDACTED Thu Jan 6 07:32:01 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Thu, 06 Jan 2011 17:32:01 +1100 Subject: [erlang-questions] Processes & Fault Tolerance In-Reply-To: References: Message-ID: UPDATE: I'm excited! I've just added hot take-over to my server program (for specific jobs). I've watched one node take-over the handling of a request from another as it's network connection died. The client didn't notice. Very eerie to witness, and quite impressive how easy it was to add that functionality if like me, you've come from other languages and imagined the pain involved in doing this. Using Joe's illustration, I was able to do this without the functionality of OTP supervisors, application take-over/fail-over, etc (even though my application is an OTP application.) Just using redundant gen_server worker processes watching each-other and updating each-other on state changes + new requests. To summerise the important lessons I've learned from all who helped: In regards to processes, fault-tolerance and the share-nothing principle... * We can say: Share nothing because we want to replicate valid state which leads to one form of fault tolerance (a failed job can be done elsewhere). * Or we can say: Share nothing because we want to isolate invalid state which leads to another form fault tolerance (a failed job won't affect the rest of the system). * For job hot take-over + redundancy, one should think in terms of the former (you need two or more nodes for this). * For worker supervision + restarts, one should think in terms of the latter (you can use one or more nodes for this). Supervisors monitor workers. Workers do jobs. In-case of faults: workers can be restarted but also **jobs can be taken over by one redundant worker from another**. This is powerful, Erlang is pretty cool :) I'm gonna go marvel as I plug and unplug my network cables now :P Thanks again to all. - Edmond - On Thu, 06 Jan 2011 02:17:39 +1100, Edmond Begumisa wrote: > On Tue, 04 Jan 2011 20:52:50 +1100, Joe Armstrong > wrote: > >> On Mon, Jan 3, 2011 at 5:30 PM, Edmond Begumisa >> wrote: >>>> Let's start with how we do error recovery. >>>> >>>> Imagine two linked processes A and B on separate machines. A is the >>>> master process. >>>> B is a passive processes that will take over if A fails. >>>> >>>> A sends a stream of state update messages S1, S2, S3,... to B. The >>>> state messages contain enough information >>>> for B to do whatever A was doing should A fail. If A fails B will >>>> receive an EXIT signal. >>>> >>>> If B does receive an exit signal it knows A has failed, so it carries >>>> on doing whatever A was doing >>>> using the information it last received in a state update message. >>>> >>>> That's it and all about it - nothing to do with supervisors etc, >>> >>> Ooooohh! THANK YOU, THANK YOU, THANK YOU. >> >> Thanks. >> >> Another problem in understand is that we assume the meaning of words. >> > > Yes. There was some confusion about whether we were all talking about > replicating valid state or isolating invalid state. Which I guess > "fault-tolerance" could be interpreted to mean either. But I'm no expert > on the subject so I shouldn't comment on terminology -- I just knew the > feature I wanted and was pretty sure it could be done at the process > level, I just had no-idea how! From now on, I'll stick to the term > "redundancy" or Ulf's "hot take-over" so it's clearer what I mean. > > - Edmond - > > >> Once upon a time I was talking to a guy and used the word "fault >> tolerance" >> we talked for hours and thought we understood what this word meant. Then >> he said something that implied he was building a fault-tolerant system >> on one computer. >> I said it was impossible. He said, "you catch and handle all the >> exceptions .." >> >> I said, "The entire computer might crash" >> >> He said, "Oh" >> >> There was a long silence. >> >> I told him about Erlang ... >> >> >> /Joe >> >>> >>> *Now* I get it. This is precisely what I was trying to understand. The >>> missing link was sending redundant messages. It all makes sense now -- >>> it's >>> so simple. All I have to do to get fault tolerance at the process >>> level is >>> have a group of N redundant processes waiting for exit-signals and >>> forward >>> state changes to N-1 of them. I could even send to N/2 and have some >>> decent >>> tolerance at the expense of consistency of the nodes. >>> >>> To preemptively answer the question I sent Ulf and Mazen... for that >>> distributed database, Process B would be set up to also receive read >>> requests forwarded from Process A but not respond to them unless it >>> gets an >>> exit signal it can understand like disk_fail_error. Any changes in >>> state >>> would also be forwarded to existing redundant processes. >>> >>> Thanks everyone for your patience in helping me understand >>> fault-tolerance. >>> I've already started thinking about some cool things I can do with >>> this in >>> my program. >>> >>> - Edmond - >>> >>> >>> >>> On Tue, 04 Jan 2011 00:36:30 +1100, Joe Armstrong >>> wrote: >>> >>>> I think you are getting confused between OTP supervisors etc. and the >>>> general notion of "take-over" >>>> >>>> Let's start with how we do error recovery. >>>> >>>> Imagine two linked processes A and B on separate machines. A is the >>>> master process. >>>> B is a passive processes that will take over if A fails. >>>> >>>> A sends a stream of state update messages S1, S2, S3,... to B. The >>>> state messages contain enough information >>>> for B to do whatever A was doing should A fail. If A fails B will >>>> receive an EXIT signal. >>>> >>>> If B does receive an exit signal it knows A has failed, so it carries >>>> on doing whatever A was doing >>>> using the information it last received in a state update message. >>>> >>>> That's it and all about it - nothing to do with supervisors etc, >>>> >>>> >>>> To make the above possible we need a couple of primitives in Erlang, >>>> namely process_flag trap_exits, and spawn_link. That's all you need. >>>> >>>> The basic system properties you need to make a fault-tolerant systems >>>> are the abilities to detect remote errors and send and receive >>>> messages. >>>> >>>> What the OTP supervisors etc. do are build layers of abstraction on >>>> top of spawn-link and trap_exits >>>> in order to simplify programming common use-cases. >>>> >>>> Fault detection and recovery is not arbitrary or magic in any sense, >>>> It has to be part of the system architecture like everything else. One >>>> common fault in making systems is to specify and design for the normal >>>> behavior >>>> but not specify and design how fault-detection and correction work. >>>> This is probably due to a legacy of >>>> programming in languages that have very limited ability to detect >>>> errors and recover from them. >>>> >>>> A C programmer with a single thread of control will take extreme >>>> measures to avoid crashing their program >>>> they have to, there is only one thread of control. Given multiple >>>> threads of control and the ability to remotely >>>> detect errors your entire way of thinking changes and the emphasis >>>> changes from thinking "how to I avoid a crash" >>>> to thinking "given that a crash has occurred and been detected, how to >>>> I fix things up and carry on" >>>> >>>> The Erlang philosophy is that "things will crash", so we have to >>>> detect the crashes, fix the bit that broke and >>>> carry on - we do not make excessive efforts to avoid crashes, but we >>>> do try very hard to repair things after they have gone wrong. >>>> >>>> Fixing things after they have broken is often easier than preventing >>>> them from breaking. Preventing them >>>> from breaking is not theoretically possible - we can't even prove >>>> simple things, like that a program will terminate >>>> (the famous halting problem) - but we can easily detect failures and >>>> put the system to rights by applying >>>> certain invariants. >>>> >>>> OTP supervisors are just trees of processes with certain restart rules >>>> that experience has shown us >>>> to be useful. >>>> >>>> The main value of supervisors etc. (and of all the OTP behaviors) is >>>> in building large systems with many >>>> programmers involved. Rather than inventing their own patterns, the >>>> programmers reuse a common set of >>>> patterns, which makes the projects easier to manage. >>>> >>>> /Joe >>>> >>>> >>>> >>>> >>>> On Mon, Jan 3, 2011 at 6:21 AM, Edmond Begumisa >>>> wrote: >>>>> >>>>> Thanks for your response. >>>>> >>>>> Firstly, let me make my question a little clearer... >>>>> >>>>> To rephrase: For processes, "share nothing for the sake of >>>>> concurrency" - >>>>> I >>>>> get, both in concept and application. "Share nothing for the sake of >>>>> fault-tolerance" - I get in concept but not in application. >>>>> >>>>> Yet as I understand it, it is for the latter reason Erlang shares >>>>> nothing* >>>>> and not the former. Interpretation: I must be completely missing the >>>>> point >>>>> in regards to Erlang processes and sharing nothing. This is what I >>>>> want >>>>> to >>>>> understand in application. In addition to the "side" effect of sane >>>>> concurrency (which coming from a chaotic multi-threading >>>>> shared-memory >>>>> world >>>>> I fully appreciate and practically make use of everyday), how can I >>>>> also >>>>> make use of the "real" reason Erlang processes share nothing -- fault >>>>> tolerance? Practically/illustratively speaking? >>>>> >>>>> *ETS being the obvious exception. >>>>> >>>>> Secondly, here's a mantra from Joe Armstrong... >>>>> >>>>> @ minute 17:26 >>>>> http://www.se-radio.net/2008/03/episode-89-joe-armstrong-on-erlang/ >>>>> >>>>> "[message passing concurrency]... the original reasons have to do >>>>> with >>>>> fault >>>>> tolerance... you have to copy all the data you need from computer 1 >>>>> to >>>>> computer 2... if computer 1 crashes you take over on computer 2... >>>>> you >>>>> can't >>>>> have dangling pointers... that's the reason for copying everything... >>>>> it's >>>>> got nothing to do with concurrency, it's got a lot to do with >>>>> fault-tolerance... if they don't crash you could just have a dangling >>>>> pointer and copy less data but it won't work in the presence of >>>>> errors..." >>>>> >>>>> I interpret this to mean that share-nothing between processes is more >>>>> about >>>>> replicating valid state than isolating corrupted state as you >>>>> described. >>>>> >>>>> Indeed, Joe created an example on his blog... >>>>> >>>>> >>>>> http://armstrongonsoftware.blogspot.com/2007/07/scalable-fault-tolerant-upgradable.html >>>>> >>>>> It's algorithm 3 there I'm struggling with. Particularly where he >>>>> says... >>>>> >>>>> "... In practise we would send an asynchronous stream of messages >>>>> from N >>>>> to >>>>> N+1 containing enough information to recover if things go wrong." >>>>> >>>>> Unfortunately, I couldn't find part II to that post (I don't think >>>>> there >>>>> is >>>>> one.) And I'm too green and inexperienced in the field of >>>>> fault-tolerant >>>>> systems to figure it out on my own. I'm having trouble visualising >>>>> the >>>>> practical here from the conceptual -- I need to be shown how :( >>>>> >>>>> Also, I seem to be under the impression that the Erlang language has >>>>> some >>>>> sort of schematics to do this built-in (i.e. deal with one process >>>>> taking >>>>> over from another if the first fails) and this is the reason >>>>> processes >>>>> share >>>>> nothing. This seems to me to be something different from supervision >>>>> trees, >>>>> which use exit-trapping to re-spawn if a process fails with the >>>>> active >>>>> 'job' >>>>> disappearing and any errors logged (like restarting a daemon). My >>>>> interpretation of the fault-tolerance Erlang is supposed to enable >>>>> (for >>>>> those in the know) is seamless take-over. The 'job' lives on but >>>>> elsewhere. >>>>> >>>>> Using telecoms as an example: a phone call wouldn't be cut-off when a >>>>> fault >>>>> occurs, another node would seamlessly take over. This is how I >>>>> interpreted >>>>> Joe's post and other descriptions of Erlang's fault-tolerant >>>>> features and >>>>> I >>>>> understand the key is in the share nothing policy for processes. I'm >>>>> sure >>>>> I've mis-understood something or everything :) >>>>> >>>>> - Edmond - >>>>> >>>>> PS: I've read the Manning draft. Great book. I don't know if the >>>>> answer >>>>> lies >>>>> in OTP (I searched and didn't find it). I suspect it's lower -- >>>>> probalby >>>>> how >>>>> you organise your processes. Some distributed-programming black-magic >>>>> only >>>>> Erlanger's know about :) >>>>> >>>>> >>>>> On Mon, 03 Jan 2011 14:56:51 +1100, Alain O'Dea >>>>> >>>>> wrote: >>>>> >>>>>> On 2011-01-02, at 22:36, "Edmond Begumisa" >>>>>> >>>>>> wrote: >>>>>> >>>>>>> Slight correction... >>>>>>> >>>>>>> On Mon, 03 Jan 2011 12:38:38 +1100, Edmond Begumisa >>>>>>> wrote: >>>>>>> >>>>>>>> Hello all, >>>>>>>> >>>>>>>> I've been trying to wrap my Erlang's fault tolerant features >>>>>>>> particularly in relation to processes. >>>>>>>> >>>>>>> >>>>>>> Should be: I've been trying to wrap my head around Erlang's fault >>>>>>> tolerant features particularly in relation to processes. >>>>>>> >>>>>>> Sorry. >>>>>>> >>>>>>>> I've heard/read repeatedly that the primary reason why Erlang's >>>>>>>> designers opted for a share-nothing policy is not rooted in >>>>>>>> concurrency but >>>>>>>> rather in fault-tolerance. When nothing is shared, everything is >>>>>>>> copied. >>>>>>>> When everything is copied processes can take over from one another >>>>>>>> when >>>>>>>> things fail. I follow this reasoning but I don't follow how to >>>>>>>> apply >>>>>>>> it. >>>>>>>> >>>>>>>> I fully understand and appreciate how supervision trees are used >>>>>>>> to >>>>>>>> restart processes if they fail. What I don't get is what to do >>>>>>>> when >>>>>>>> you >>>>>>>> don't want to restart but want to take over, say on another node. >>>>>>>> I >>>>>>>> know >>>>>>>> that at a higher-level, OTP has some take-over/fail-over >>>>>>>> schematics >>>>>>>> (at the >>>>>>>> application level.) I'm trying to understand things at the >>>>>>>> processes >>>>>>>> level - >>>>>>>> why Erlang is the way it is so I can better use it to make my >>>>>>>> currently >>>>>>>> fault-intolerant program fault tolerant. >>>>>>>> >>>>>>>> Specifically, how can one process take over from another if it >>>>>>>> fails? >>>>>>>> It >>>>>>>> appears to may that the only way to do this would be to somehow >>>>>>>> retrieve not >>>>>>>> only the state of the process (say, gen_server's state) but also >>>>>>>> the >>>>>>>> messages in its mailbox. Where does the design decision to >>>>>>>> share-nothing for >>>>>>>> the sake of fault-tolerance come into play for processes? Please >>>>>>>> help >>>>>>>> me >>>>>>>> "get" this! >>>>>>>> >>>>>>>> Thanks in advance. >>>>>>>> >>>>>>>> - Edmond - >>>>>>>> >>>>>>>> >>>>>> >>>>>> Hi Edmond: >>>>>> >>>>>> Share-nothing helps with concurrent fault-tolerance by preventing >>>>>> one >>>>>> process from corrupting the state of another. Receive is a process' >>>>>> choice >>>>>> and it corrupts its own state if it receives bad data and lets it >>>>>> in. >>>>>> >>>>>> AFAIK OTP fault-tolerance doesn't mean no requests will fail, it >>>>>> means >>>>>> the >>>>>> system/sub-system will recover if a single request causes a process >>>>>> to >>>>>> crash. It's kind of like proper try/catch recovery applied to >>>>>> concurrent >>>>>> code. How you recover from the crash depends on the supervision >>>>>> strategy >>>>>> chosen. In some cases the supervisor can pass the state to the >>>>>> replacement >>>>>> process. In others this isn't necessary or even desirable since the >>>>>> state >>>>>> itself may involve resources lost in the crash or corrupted state >>>>>> that >>>>>> led >>>>>> to the crash. >>>>>> >>>>>> I am straying outside my knowledge here so this paragraph is >>>>>> guesswork. >>>>>> The message queue for a gen_server need not necessarily be lost >>>>>> when >>>>>> the >>>>>> callback module crashes. In theory OTP could (and might already) >>>>>> simply >>>>>> delegate the messages to the replacement process following a crash. >>>>>> Someone >>>>>> who knows OTP better than me would need to weigh in here though. >>>>>> >>>>>> I found http://manning.com/logan very informative in understanding >>>>>> OTP >>>>>> and >>>>>> its supervisor hierarchies. >>>>>> >>>>>> Cheers, >>>>>> Alain >>>>>> ________________________________________________________________ >>>>>> erlang-questions (at) erlang.org mailing list. >>>>>> See http://www.erlang.org/faq.html >>>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>>>> >>>>> >>>>> >>>>> -- >>>>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >>>>> >>>>> ________________________________________________________________ >>>>> erlang-questions (at) erlang.org mailing list. >>>>> See http://www.erlang.org/faq.html >>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>>> >>>>> >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>> >>> >>> -- >>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From xapwing@REDACTED Thu Jan 6 10:08:44 2011 From: xapwing@REDACTED (Arie van Wingerden) Date: Thu, 6 Jan 2011 10:08:44 +0100 Subject: 1000 cores on a chip Message-ID: Hi, maybe off topic, but interesting still! A 1000 core chip: http://www.kurzweilai.net/scientists-squeeze-more-than-1000-cores-onto-computer-chip Regards, Arie From paolo.negri@REDACTED Thu Jan 6 10:25:07 2011 From: paolo.negri@REDACTED (Paolo Negri) Date: Thu, 6 Jan 2011 10:25:07 +0100 Subject: string in functions and memory allocation Message-ID: Dear list I have a newbie question regarding the following 2 (dummy) functions dummy(Proplist) Value = proplist:get_value(<<"booo">>), Value . dummy_string(Proplist) Value = proplist:get_value("booo"), Value . What I would like to know it's how calling these functions an elevate number of times would affect memory allocation and garbage collection, I'm asking because in other languages something like dummy_string would allocate a new string at every call that would then need to be garbage collected. Thanks in advance for your help. Paolo -- Engineering http://www.wooga.com | phone +49-30-8962 5058? | fax +49-30-8964 9064 wooga GmbH | Saarbruecker Str. 38 | 10405 Berlin | Germany Sitz der Gesellschaft: Berlin; HRB 117846 B Registergericht Berlin-Charlottenburg Geschaeftsfuehrung: Jens Begemann, Philipp Moeser From paolo.negri@REDACTED Thu Jan 6 10:30:31 2011 From: paolo.negri@REDACTED (Paolo Negri) Date: Thu, 6 Jan 2011 10:30:31 +0100 Subject: string in functions and memory allocation In-Reply-To: References: Message-ID: Sorry, the 2 dummy functions were wrong here's a less broken version dummy(Proplist) Value = proplist:get_value(<<"booo">>, Proplist), Value . dummy(Proplist) ? ?Value = proplist:get_value(<<"booo">>, Proplist), ? ?Value . -- Engineering http://www.wooga.com | phone +49-30-8962 5058? | fax +49-30-8964 9064 wooga GmbH | Saarbruecker Str. 38 | 10405 Berlin | Germany Sitz der Gesellschaft: Berlin; HRB 117846 B Registergericht Berlin-Charlottenburg Geschaeftsfuehrung: Jens Begemann, Philipp Moeser From hynek@REDACTED Thu Jan 6 11:12:50 2011 From: hynek@REDACTED (Hynek Vychodil) Date: Thu, 6 Jan 2011 11:12:50 +0100 Subject: [erlang-questions] string in functions and memory allocation In-Reply-To: References: Message-ID: Constants (literals) are allocated in constant pool since Erlang R12B-0 so it will not be constructed in each function call. On Thu, Jan 6, 2011 at 10:25 AM, Paolo Negri wrote: > Dear list > > I have a newbie question regarding the following 2 (dummy) functions > > dummy(Proplist) > ? ?Value = proplist:get_value(<<"booo">>), > ? ?Value > . > > dummy_string(Proplist) > ? ?Value = proplist:get_value("booo"), > ? ?Value > . > > What I would like to know it's how calling these functions an elevate > number of times would affect memory allocation and garbage collection, > I'm asking because in other languages something like dummy_string > would allocate a new string at every call that would then need to be > garbage collected. > > Thanks in advance for your help. > > Paolo > > -- > Engineering > http://www.wooga.com | phone +49-30-8962 5058? | fax +49-30-8964 9064 > > wooga GmbH | Saarbruecker Str. 38 | 10405 Berlin | Germany > Sitz der Gesellschaft: Berlin; HRB 117846 B > Registergericht Berlin-Charlottenburg > Geschaeftsfuehrung: Jens Begemann, Philipp Moeser > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From jesper.louis.andersen@REDACTED Thu Jan 6 13:47:18 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Thu, 6 Jan 2011 13:47:18 +0100 Subject: [erlang-questions] string in functions and memory allocation In-Reply-To: References: Message-ID: On Thu, Jan 6, 2011 at 11:12, Hynek Vychodil wrote: > Constants (literals) are allocated in constant pool since Erlang > R12B-0 so it will not be constructed in each function call. And to add, neither is 'Value' in the examples. It will at most be a smallish reference to a large value. And is no copy of 'PropList' either but for the reference to it. Technically we are staring at a function that will allocate nothing. -- J. From jodie.burch@REDACTED Thu Jan 6 18:12:39 2011 From: jodie.burch@REDACTED (Jodie Burch) Date: Thu, 06 Jan 2011 17:12:39 +0000 Subject: Erlang Factory Lite, Munich, Germany - Jan 31st Message-ID: Hi Everyone Just to let you know, there will be an Erlang Factory Lite in Munich on the 31st January 2011. The event will be hosted by TNG Technology Consulting GmbH and organised by author Pavlo Baron and Erlang Solutions. The event will start in the afternoon at 1pm at TNG Technology. Information can be found here: http://www.erlang-factory.com/conference/ErlangFactoryLiteMunich Registration for this event is free but required for venue security purposes. You can register here: https://www.erlang-factory.com/conference/ErlangFactoryLiteMunich/register If you want to present a talk please feel free to submit a proposal here: http://www.erlang-factory.com/conference/ErlangFactoryLiteMunich/submit_talk The event is for you so we would love for you to present. If you have any questions please feel free to email me and we look forward to seeing you there! Thanks Jodie From info@REDACTED Thu Jan 6 18:46:01 2011 From: info@REDACTED (=?utf-8?B?aW5mbw==?=) Date: Thu, 6 Jan 2011 18:46:01 +0100 Subject: =?utf-8?B?UmU6IFJlOiBbZXJsYW5nLXF1ZXN0aW9uc10gRXJsYW5nICsgRW1hY3MgKyBEaXN0ZWwgb24gd2luZG93cw==?= References: <201012281349493298586@its3.ch> Message-ID: <201101061846011934516@its3.ch> Hello, I already updated my .emacs file for distel. I defined in c:\ documents and settings\administrator the .erlang_cookie file. *** Is it the correct directory for the erlang cookie ? *** I opened an erl module in emacs. I opened a shell window, I checked node(). = emacs@REDACTED, I checked erlang:get_cookie(). I compiled the erl module by c("mymodule",[debug_info]). I connected to the node emacs by c-c c-d n. *** error in process filter: Opening input file: no such file or directory, c:/.erlang.cookie *** I moved the .erlang.cookie file in c: I connected to the node emacs by c-c c-d n. *** no error message *** I reloaded the module by c-c c-d L *** Distel Warning: node emacs@REDACTED can't seem to load the `distel' module. This means that most Distel commands won't function correctly, because the supporting library is not available. Please check your node's code path, and make sure that Distel's "ebin" directory is included. The most likely cause of this problem is either: a) Your ~/.erlang file doesn't add Distel to your load path (the Distel "make config_install" target can set this up for you.) b) Your system's boot script doesn't consult your ~/.erlang file to read your code path setting. To disable this warning in future, set `distel-inhibit-backend-check' to t. *** .erlang file doesn't exist ! I created it with .emacs because windows doesn't accept a file beginning with a dot. I insert this line: code:add_pathz("C:/distel-4.03/ebin"). make config_install seems to be a linux command ? *** How can I progress ? *** My .beam files are not in /distel-4.03/ebin I am not the first one to use distel and erlang on windows !!! J-Ph. Constantin ITS3 Gen?ve www.its3.ch From steven.charles.davis@REDACTED Fri Jan 7 03:41:14 2011 From: steven.charles.davis@REDACTED (Steve Davis) Date: Thu, 06 Jan 2011 20:41:14 -0600 Subject: FPGA coming around the corner Message-ID: <4D267D4A.6060101@gmail.com> A very interesting article... and all the more interesting because it lines up with Joe's predictions. http://www.kurzweilai.net/scientists-squeeze-more-than-1000-cores-onto-computer-chip > ?FPGAs are not used within standard computers because they are fairly > difficult to program, but their processing power is huge while their > energy consumption is very small because they are so much quicker, so > they are also a greener option,? said researcher Dr. Wim > Vanderbauwhede. > While most computers sold today now contain more than one processing > core, which allows them to carry out different processes > simultaneously, traditional multi-core processors must share access > to one memory source, which slows the system down. The scientists in > this research were able to make the processor faster by giving each > core a certain amount of dedicated memory. If the time is to come, then surely this is the *right time* for the Erlang/OTP 'product' to step up and meet the challenges and mindset that bind us daily owing to the chosen OO style forced onto us by C++/Java. /s From banibrata.dutta@REDACTED Fri Jan 7 04:30:07 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Fri, 7 Jan 2011 09:00:07 +0530 Subject: [erlang-questions] 1000 cores on a chip In-Reply-To: References: Message-ID: On Thu, Jan 6, 2011 at 2:38 PM, Arie van Wingerden wrote: > Hi, > > maybe off topic, but interesting still! > A 1000 core chip: > > http://www.kurzweilai.net/scientists-squeeze-more-than-1000-cores-onto-computer-chip > > > Regards, > Arie > Interesting. However, those many cores in GPU world isn't new. CUDA has shown that to us for a while. People are making use of GPU for general computation increasingly, and a while back had come accross a startup that was using the extreme parallelism (in a box) by CUDA for video analytics and video encoding/decoding. Not sure how well FPGA's really fare w.r.t. power consumption in such setups, but the CUDA GPU units do consume a good bit of power.. and generate good bit of heat. Since this post was made on Erlang forum, wondering if someone here can shed some light on how well such developments might aid/impact Erlang based projects... especially from the extreme GPU parallelism standpoint and leveraging those for generic algorithmic computation ? Is someone aware of some such projects already ? -- regards, Banibrata http://www.linkedin.com/in/bdutta http://twitter.com/edgeliving From ebegumisa@REDACTED Fri Jan 7 05:40:39 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Fri, 07 Jan 2011 15:40:39 +1100 Subject: [erlang-questions] FPGA coming around the corner In-Reply-To: <4D267D4A.6060101@gmail.com> References: <4D267D4A.6060101@gmail.com> Message-ID: > If the time is to come, then surely this is the *right time* for the > Erlang/OTP 'product' to step up and meet the challenges and mindset that > bind us daily owing to the chosen OO style forced onto us by C++/Java. On a related note: I'm curious -- why is it that many seem unaware of Erlang's advantages with multi-core? Even vocal MP experts that are advocating for the change in mindset you describe and the need for programmers to be aware of the disconnect and quickly adapt -- these advocates don't sound like they're aware of Erlang (at least they don't mention Erlang in their talks). A blog post recently alerted me to a talk Maurice Herlihy gave back in 2007 entitled "Taking Concurrency Seriously: New Directions in Multiprocessor Synchronization." http://www.stanford.edu/class/ee380/Abstracts/070502.html http://ee380.stanford.edu/cgi-bin/videologger.php?target=070509-ee380-300.asx To quote him "... Computer architecture is about to undergo, if not another revolution, then a vigorous shaking-up. The major chip manufacturers have, for the time being, simply given up trying to make processors run faster... As a result, system designers and software engineers can no longer rely on increasing clock speed to hide software bloat. Instead, they must somehow learn to make effective use of increasing parallelism. This adaptation will not be easy. Conventional synchronization techniques based on locks and conditions are unlikely to be effective in such a demanding environment..." He then goes on the show some pretty fancy Java code illustrating concurrent data structures and a concept he calls "transactional boosting" in an attempt to deal with the disconnect without burdening the programmer "too much." There are a number of more recent talks on that Standford page with the same tone... http://www.stanford.edu/class/ee380/previous.html They all sound panic-stricken and suggest there is currently no complete viable solution out-there yet to programming for the multi-core age that don't burden the programmer. These are some very smart people. But when I saw these videos recently I thought "Have they not heard of Erlang? If not why not?" - Edmond - On Fri, 07 Jan 2011 13:41:14 +1100, Steve Davis wrote: > > A very interesting article... and all the more interesting because it > lines up with Joe's predictions. > > http://www.kurzweilai.net/scientists-squeeze-more-than-1000-cores-onto-computer-chip > > > ?FPGAs are not used within standard computers because they are fairly > > difficult to program, but their processing power is huge while their > > energy consumption is very small because they are so much quicker, so > > they are also a greener option,? said researcher Dr. Wim > > Vanderbauwhede. > > > While most computers sold today now contain more than one processing > > core, which allows them to carry out different processes > > simultaneously, traditional multi-core processors must share access > > to one memory source, which slows the system down. The scientists in > > this research were able to make the processor faster by giving each > > core a certain amount of dedicated memory. > > If the time is to come, then surely this is the *right time* for the > Erlang/OTP 'product' to step up and meet the challenges and mindset that > bind us daily owing to the chosen OO style forced onto us by C++/Java. > > /s > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From kenji.rikitake@REDACTED Fri Jan 7 06:04:16 2011 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Fri, 7 Jan 2011 14:04:16 +0900 Subject: [erlang-questions] Restricting the connection between two nodes In-Reply-To: References: Message-ID: <20110107050416.GA1896@k2r.org> In the message dated Mon, Jan 03, 2011 at 03:33:44PM +0100, Tomasz Maciejewski writes: > Is there a way to restrict the connection between two nodes, allowing > executing only specific functions (eg. only from specific "exported" > module)? > > Suppose we have two nodes running one application, respectively: > private_node (eg. Mnesia with sensitive data, or a port owner) and > public_node (eg. HTTP/FTP daemon). The two nodes are connected > together and the public_node needs some data from private_node, but > private_node has some sensitive functions. Now, if someone gains > control over > public_node, he can call any function from private_node by simply > rpc:call, as well as sending arbitrary messages to processes on > private_node. In the case above, once the public_node is taken over, you can't do much to protect the private node. So don't let the public_node get taken. > How can I protect against breaking in to Erlang by public internet > service? Is it even possible to gain access to a remote shell > (assuming cookie wasn't stolen) by some remote hole in Erlang/Inets/My > application? As default, the Distributed Erlang RPC packets are unprotected (i.e., not encrypted) at all; they are built on the plain TCP connections. What you might need is secure-and-private connection between the two nodes with encryption. The secure connection can be on many different levels: application software or on the IP (IPsec). About a year ago I tried to build an example of Erlang RPC over the SSH module [1]. Dave "Dizzyd" Smith, the rebar guru, also once published a paper of his idea on securely building inter-node RPCs on Erlang [2]. Related links: [1] https://github.com/jj1bdx/sshrpc If you want to see the video on my presentation at Erlang Factory SF Bay 2010: http://www.erlang-factory.com/conference/SFBay2010/speakers/kenjirikitake And the slides: http://www.erlang-factory.com/upload/presentations/214/ErlangFactorySFBay2010-KenjiRikitake.pdf [2] http://dizzyd.com/sdist.pdf > With the solution of "restricring remote function calls only to > functions from specific module", I could implement public API in > private_server and be protected against gaining control over > private_server. Currently there's no native mechanism to restrict function calls in Erlang, so far I've learned. If you really want this level of protection, Distributed Erlang RPC itself is not suitable. > I will really appreciate any comments on security in Erlang. > > -- > Tomasz Maciejewski Kenji Rikitake From michael.eugene.turner@REDACTED Fri Jan 7 06:04:23 2011 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Fri, 7 Jan 2011 14:04:23 +0900 Subject: [erlang-questions] FPGA coming around the corner In-Reply-To: References: <4D267D4A.6060101@gmail.com> Message-ID: "... They all sound panic-stricken and suggest there is currently no complete viable solution out-there yet to programming for the multi-core age that don't burden the programmer. These are some very smart people. But when I saw these videos recently I thought "Have they not heard of Erlang? If not why not?" If you're overridingly concerned with fault-tolerance and ease of scaling on multi-core systems, and starting with a clean slate, Erlang makes a lot of sense. But that's not the focus of the "panicked" people mentioned here. Their interest is in raw performance, and perhaps also very much raw performance for legacy systems written in more mainstream languages that people would like to see parallelized. Erlang's not bad with respect to speed, but it's not great either: distributing an Erlang program over 5-10 cores might give you the same speedup you could get on a *single* core in C/C++/Java simply by optimizing your code savagely. And requiring programmers to rewrite legacy apps in a new language with unfamiliar paradigms -- well, it might be good in the long run, but in the long run, we're all dead, and there's an awful lot of working, useful code out there. As pointed out in a recent thread touching on the raison d'etre for NIFs in Erlang, NIFs are less about making Erlang code fast and more about interfacing with all the useful stuff people have been writing and debugging for decades. And why not? OTP is large, a great toolbox, but outside its core strengths, it only scratches the surface of what people might want to do in software. Most of these "panic-stricken" people have probably heard of Erlang -- it just doesn't necessarily solve their problems. Maybe nothing can. But there's been great progress with tools that help model and debug concurrency in legacy apps, and that's probably going to continue. They aren't out of rope yet. Face it: successful software technologies take a long time to die. Whenever you log onto your bank's website, it's virtually guaranteed that the part of the delay involves cranking up some virtual machine environment to run some crufty old accounting package written 30 years ago in COBOL. -michael turner On Fri, Jan 7, 2011 at 1:40 PM, Edmond Begumisa wrote: > If the time is to come, then surely this is the *right time* for the >> Erlang/OTP 'product' to step up and meet the challenges and mindset that >> bind us daily owing to the chosen OO style forced onto us by C++/Java. >> > > On a related note: I'm curious -- why is it that many seem unaware of > Erlang's advantages with multi-core? Even vocal MP experts that are > advocating for the change in mindset you describe and the need for > programmers to be aware of the disconnect and quickly adapt -- these > advocates don't sound like they're aware of Erlang (at least they don't > mention Erlang in their talks). > > A blog post recently alerted me to a talk Maurice Herlihy gave back in 2007 > entitled "Taking Concurrency Seriously: New Directions in Multiprocessor > Synchronization." > > http://www.stanford.edu/class/ee380/Abstracts/070502.html > > http://ee380.stanford.edu/cgi-bin/videologger.php?target=070509-ee380-300.asx > > To quote him "... Computer architecture is about to undergo, if not another > revolution, then a vigorous shaking-up. The major chip manufacturers have, > for the time being, simply given up trying to make processors run faster... > As a result, system designers and software engineers can no longer rely on > increasing clock speed to hide software bloat. Instead, they must somehow > learn to make effective use of increasing parallelism. This adaptation will > not be easy. Conventional synchronization techniques based on locks and > conditions are unlikely to be effective in such a demanding environment..." > > He then goes on the show some pretty fancy Java code illustrating > concurrent data structures and a concept he calls "transactional boosting" > in an attempt to deal with the disconnect without burdening the programmer > "too much." > > There are a number of more recent talks on that Standford page with the > same tone... > > http://www.stanford.edu/class/ee380/previous.html > > They all sound panic-stricken and suggest there is currently no complete > viable solution out-there yet to programming for the multi-core age that > don't burden the programmer. These are some very smart people. But when I > saw these videos recently I thought "Have they not heard of Erlang? If not > why not?" > > - Edmond - > > > On Fri, 07 Jan 2011 13:41:14 +1100, Steve Davis < > steven.charles.davis@REDACTED> wrote: > > >> A very interesting article... and all the more interesting because it >> lines up with Joe's predictions. >> >> >> http://www.kurzweilai.net/scientists-squeeze-more-than-1000-cores-onto-computer-chip >> >> > ?FPGAs are not used within standard computers because they are fairly >> > difficult to program, but their processing power is huge while their >> > energy consumption is very small because they are so much quicker, so >> > they are also a greener option,? said researcher Dr. Wim >> > Vanderbauwhede. >> >> > While most computers sold today now contain more than one processing >> > core, which allows them to carry out different processes >> > simultaneously, traditional multi-core processors must share access >> > to one memory source, which slows the system down. The scientists in >> > this research were able to make the processor faster by giving each >> > core a certain amount of dedicated memory. >> >> If the time is to come, then surely this is the *right time* for the >> Erlang/OTP 'product' to step up and meet the challenges and mindset that >> bind us daily owing to the chosen OO style forced onto us by C++/Java. >> >> /s >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > -- > Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From jeffm@REDACTED Fri Jan 7 06:02:02 2011 From: jeffm@REDACTED (jm) Date: Fri, 07 Jan 2011 16:02:02 +1100 Subject: [erlang-questions] FPGA coming around the corner In-Reply-To: References: <4D267D4A.6060101@gmail.com> Message-ID: <4D269E4A.9030902@ghostgun.com> My simplistic guess is that they are are either * Unaware of Erlang as a viable option. * Are looking for a C/C++/Java solution. * Are looking for a general purpose *high performance* language, eg, what Fortress is aiming to become. * Have vested interests, eg are using panic a marketing method to raise funding. Jeff. On 7/01/11 3:40 PM, Edmond Begumisa wrote: >> If the time is to come, then surely this is the *right time* for the >> Erlang/OTP 'product' to step up and meet the challenges and mindset >> that bind us daily owing to the chosen OO style forced onto us by >> C++/Java. > > On a related note: I'm curious -- why is it that many seem unaware of > Erlang's advantages with multi-core? Even vocal MP experts that are > advocating for the change in mindset you describe and the need for > programmers to be aware of the disconnect and quickly adapt -- these > advocates don't sound like they're aware of Erlang (at least they > don't mention Erlang in their talks). From kenji.rikitake@REDACTED Fri Jan 7 06:34:57 2011 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Fri, 7 Jan 2011 14:34:57 +0900 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: References: <4D227D75.1060108@gmail.com> <4D228213.8050502@gmail.com> Message-ID: <20110107053457.GB1896@k2r.org> Learning Erlang/OTP is much harder than I thought; the world of Erlang/OTP is more profound and complex than I first viewed. The language itself is relatively simple, but the idea and philosophy expressed in the OTP are not something you can grasp in a few days. In the message dated Tue, Jan 04, 2011 at 08:49:40AM +0100, Jesper Louis Andersen writes: > * Get a good idea of what the functional programming paradigm is. I need to confess that I really had to take 20 (1988 to 2008) or even more years to find out why the functional programming is useful in the *real world programming*. So it could be very tough. Erlang/OTP is the first functional language systems I really wanted to learn. > * Get a good idea of what the concurrency primitives of Erlang looks like. Reading the library source code of Erlang/OTP will help. I still need to absorb the hidden and important ideas of the primitives. > * Understanding how to do web development with Erlang. Writing code with sticking to the OTP gen_server framework will help speeding up the learning process. I think Mochiweb is another very good set of code to begin with. > * Perhaps understand what OTP is. "Erlang and OTP in Action" from Manning Books will help, as well as thoroughly and carefully reading OTP User's Guide many times. > * Having a specific project you want to implement on top of all of the above. I understand Erlang/OTP is a language system for building computing infrastructures rather than user-interface application software. So I think writing server or P2P agent software looks the fastest way. (Of course you can write almost anything for computers with Erlang/OTP and nobody will prohibit you from doing so (I was amazed by the Othello/Reversi code in Erlang/OTP games :)), but every language system has its own advantages and disadvantages.) YMMV, but it's fun to learn Erlang/OTP. Kenji Rikitake From kenji.rikitake@REDACTED Fri Jan 7 07:12:38 2011 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Fri, 7 Jan 2011 15:12:38 +0900 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: References: <4D227D75.1060108@gmail.com> Message-ID: <20110107061238.GC1896@k2r.org> In the message dated Tue, Jan 04, 2011 at 11:09:39AM -0500, "Evans, Matthew" writes: > The funny thing is that the non computer science folks found it easier to follow than the computer science folks. It seems that we comp sci people had had our brains polluted by the imperative programming method. +1. Or even +infinity. :) Disclaimer: nothing personal to Prolog people here. I just recall my personal experience. My brain cells have been contaminated with the imperative language and destructive assignments (e.g., A=A+1 in FORTRAN) so early as I started learning English and FORTRAN (in 1974 when I was nine). I've been in the continuous detoxication process since 2008. It's not easy. I failed to learn Prolog during my CS grad student times in 1988. Literally I got an D (which means "failed") at University of Tokyo. The idea in Prolog which alienated me is backtracking. Erlang does not have it :) Now *after* discovering Erlang/OTP and learning the history (especially about the language systax of Erlang), my rejection against Prolog is much cured, though I still don't feel like using it, or even relearning it. I accept I'm an irrational being on computer languages. :) Kenji Rikitake From kenji.rikitake@REDACTED Fri Jan 7 07:40:48 2011 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Fri, 7 Jan 2011 15:40:48 +0900 Subject: Erlang/OTP and HPC Re: [erlang-questions] FPGA coming around the corner In-Reply-To: References: <4D267D4A.6060101@gmail.com> Message-ID: <20110107064048.GD1896@k2r.org> My understanding is that the most of high-performance computing people are tackling different problems from the one Erlang/OTP addresses. And existing code pieces there are mostly in C++ or FORTRAN. Quite a few HPC supercomputers have already explored GPUs and their own CPU chipsets (e.g., IBM Blue Gene). I suggest you to take a look at an explanation of one of the highest performance of running computer systems, TSUBAME 2.0, in Tokyo Institute of Technology, to find out what sort of computer systems are fitting the requirement of the HPC people. http://www.gsic.titech.ac.jp/sites/default/files/TSUBAME-ESJ_No1_2.pdf I'd like to hear about what kind of contribution Erlang/OTP can make to the HPC community too. I'm not an HPC specialist at all, but dealing with the multicore environment Erlang/OTP might surely has a role to work well with the other HPC programming frameworks (OpenMP, MPI, etc.)_ In fact, quite a few supercomputers now accept general programming languages; I use my employer Kyoto University's Supercomputer Thin Cluster as a 64bit Linux test environment for my own code. Kenji Rikitake From ebegumisa@REDACTED Fri Jan 7 07:44:59 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Fri, 07 Jan 2011 17:44:59 +1100 Subject: [erlang-questions] FPGA coming around the corner In-Reply-To: References: <4D267D4A.6060101@gmail.com> Message-ID: On Fri, 07 Jan 2011 16:04:23 +1100, Michael Turner wrote: > "... They all sound panic-stricken and suggest there is currently no > complete viable solution out-there yet to programming for the multi-core > age > that don't burden the programmer. These are some very smart people. But > when > I saw these videos recently I thought "Have they not heard of Erlang? If > not > why not?" > > If you're overridingly concerned with fault-tolerance and ease of > scaling on > multi-core systems, and starting with a clean slate, Erlang makes a lot > of > sense. But that's not the focus of the "panicked" people mentioned here. > Their interest is in raw performance, and perhaps also very much raw > performance for legacy systems written in more mainstream languages that > people would like to see parallelized. > > Erlang's not bad with respect to speed, but it's not great either: > distributing an Erlang program over 5-10 cores might give you the same > speedup you could get on a *single* core in C/C++/Java simply by > optimizing > your code savagely. (Assuming this is true) It's kind of the point! The fact that you don't have to do much to get the speed up with the parallelised Erlang program! That using your scale, the same speed up is obtained by... a) Running the parallelised Erlang program on multi-core with little change & no savage optimisation, versus b) Savagely optimising the single core C/C++/Java, OR c) beating both by parallelising the C/C++/Java code on multi-core and loosing your mind in the process (again, assuming the code in question runs sequentially faster in these languages) > And requiring programmers to rewrite legacy apps in a > new language with unfamiliar paradigms -- well, it might be good in the > long run, but in the long run, we're all dead, and there's an awful lot > of working, useful code out there. But that's precisely what these advocates are warning their audiences about! That the *legacy apps and code will have to be re-written* in order to be parallelised for multi-core. That's what the panic is about. That these programs and libraries will need to be carefully re-coded using lots of threads, concurrent data structures and unfamiliar paradigms like "transactional boosting." The Herlihy's are effectively saying: "You will need to re-write you're apps and libs in order for your users not to complain that things are slower, and here's a bunch of tools you can use that we *hope* will make it a little easier. But it will still be pretty hard." My issue is that these tools really suck! May as well just re-write the damn thing in Erlang :) > As pointed out in a recent thread touching > on the raison d'etre for NIFs in Erlang, NIFs are less about making > Erlang > code fast and more about interfacing with all the useful stuff people > have > been writing and debugging for decades. And why not? OTP is large, a > great > toolbox, but outside its core strengths, it only scratches the surface of > what people might want to do in software. > > Most of these "panic-stricken" people have probably heard of Erlang -- it > just doesn't necessarily solve their problems. Maybe nothing can. But > there's been great progress with tools that help model and debug > concurrency > in legacy apps, and that's probably going to continue. They aren't out > of > rope yet. Face it: successful software technologies take a long time to > die. Whenever you log onto your bank's website, it's virtually > guaranteed > that the part of the delay involves cranking up some virtual machine > environment to run some crufty old accounting package written 30 years > ago > in COBOL. > > -michael turner > > > > On Fri, Jan 7, 2011 at 1:40 PM, Edmond Begumisa > > wrote: > >> If the time is to come, then surely this is the *right time* for the >>> Erlang/OTP 'product' to step up and meet the challenges and mindset >>> that >>> bind us daily owing to the chosen OO style forced onto us by C++/Java. >>> >> >> On a related note: I'm curious -- why is it that many seem unaware of >> Erlang's advantages with multi-core? Even vocal MP experts that are >> advocating for the change in mindset you describe and the need for >> programmers to be aware of the disconnect and quickly adapt -- these >> advocates don't sound like they're aware of Erlang (at least they don't >> mention Erlang in their talks). >> >> A blog post recently alerted me to a talk Maurice Herlihy gave back in >> 2007 >> entitled "Taking Concurrency Seriously: New Directions in Multiprocessor >> Synchronization." >> >> http://www.stanford.edu/class/ee380/Abstracts/070502.html >> >> http://ee380.stanford.edu/cgi-bin/videologger.php?target=070509-ee380-300.asx >> >> To quote him "... Computer architecture is about to undergo, if not >> another >> revolution, then a vigorous shaking-up. The major chip manufacturers >> have, >> for the time being, simply given up trying to make processors run >> faster... >> As a result, system designers and software engineers can no longer rely >> on >> increasing clock speed to hide software bloat. Instead, they must >> somehow >> learn to make effective use of increasing parallelism. This adaptation >> will >> not be easy. Conventional synchronization techniques based on locks and >> conditions are unlikely to be effective in such a demanding >> environment..." >> >> He then goes on the show some pretty fancy Java code illustrating >> concurrent data structures and a concept he calls "transactional >> boosting" >> in an attempt to deal with the disconnect without burdening the >> programmer >> "too much." >> >> There are a number of more recent talks on that Standford page with the >> same tone... >> >> http://www.stanford.edu/class/ee380/previous.html >> >> They all sound panic-stricken and suggest there is currently no complete >> viable solution out-there yet to programming for the multi-core age that >> don't burden the programmer. These are some very smart people. But when >> I >> saw these videos recently I thought "Have they not heard of Erlang? If >> not >> why not?" >> >> - Edmond - >> >> >> On Fri, 07 Jan 2011 13:41:14 +1100, Steve Davis < >> steven.charles.davis@REDACTED> wrote: >> >> >>> A very interesting article... and all the more interesting because it >>> lines up with Joe's predictions. >>> >>> >>> http://www.kurzweilai.net/scientists-squeeze-more-than-1000-cores-onto-computer-chip >>> >>> > ?FPGAs are not used within standard computers because they are >>> fairly >>> > difficult to program, but their processing power is huge while their >>> > energy consumption is very small because they are so much quicker, >>> so >>> > they are also a greener option,? said researcher Dr. Wim >>> > Vanderbauwhede. >>> >>> > While most computers sold today now contain more than one processing >>> > core, which allows them to carry out different processes >>> > simultaneously, traditional multi-core processors must share access >>> > to one memory source, which slows the system down. The scientists in >>> > this research were able to make the processor faster by giving each >>> > core a certain amount of dedicated memory. >>> >>> If the time is to come, then surely this is the *right time* for the >>> Erlang/OTP 'product' to step up and meet the challenges and mindset >>> that >>> bind us daily owing to the chosen OO style forced onto us by C++/Java. >>> >>> /s >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >> >> -- >> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From erlang@REDACTED Fri Jan 7 10:45:13 2011 From: erlang@REDACTED (Joe Armstrong) Date: Fri, 7 Jan 2011 10:45:13 +0100 Subject: [erlang-questions] FPGA coming around the corner In-Reply-To: References: <4D267D4A.6060101@gmail.com> Message-ID: On Fri, Jan 7, 2011 at 5:40 AM, Edmond Begumisa wrote: >> If the time is to come, then surely this is the *right time* for the >> Erlang/OTP 'product' to step up and meet the challenges and mindset that >> bind us daily owing to the chosen OO style forced onto us by C++/Java. > > On a related note: I'm curious -- why is it that many seem unaware of > Erlang's advantages with multi-core? Even vocal MP experts that are > advocating for the change in mindset you describe and the need for > programmers to be aware of the disconnect and quickly adapt -- these > advocates don't sound like they're aware of Erlang (at least they don't > mention Erlang in their talks). Reminds me of a weird meeting at work a few weeks ago: They: But why doesn't it go 60 times faster (on a tile64) - it's only 40 times faster? Me: Well are you happy about the times 40 or sad about the fact it's not 60? They: Well happy, but it would be nice if it were 60 Me: We're working on it Our domain of discourse, is "why we don't get a factor N speedup on N processors" - my feeling is that in many places the domain of discourse is "how can we go faster than times one on two processors and not slower" > > A blog post recently alerted me to a talk Maurice Herlihy gave back in 2007 > entitled "Taking Concurrency Seriously: New Directions in Multiprocessor > Synchronization." > > http://www.stanford.edu/class/ee380/Abstracts/070502.html > http://ee380.stanford.edu/cgi-bin/videologger.php?target=070509-ee380-300.asx > > To quote him "... Computer architecture is about to undergo, if not another > revolution, then a vigorous shaking-up. The major chip manufacturers have, > for the time being, simply given up trying to make processors run faster... > As a result, system designers and software engineers can no longer rely on > increasing clock speed to hide software bloat. Instead, they must somehow > learn to make effective use of increasing parallelism. This adaptation will > not be easy. Conventional synchronization techniques based on locks and > conditions are unlikely to be effective in such a demanding environment..." > > He then goes on the show some pretty fancy Java code illustrating concurrent > data structures and a concept he calls "transactional boosting" in an > attempt to deal with the disconnect without burdening the programmer "too > much." > > There are a number of more recent talks on that Standford page with the same > tone... > > http://www.stanford.edu/class/ee380/previous.html > > They all sound panic-stricken and suggest there is currently no complete > viable solution out-there yet to programming for the multi-core age that > don't burden the programmer. These are some very smart people. But when I > saw these videos recently I thought "Have they not heard of Erlang? If not > why not?" This question is asked to the the wrong group, I guess most people in this group know that Erlang is (according to the wikipedia) "a Chinese God with a third truth-seeing eye in the middle of his forehead." /Joe > > - Edmond - > > > On Fri, 07 Jan 2011 13:41:14 +1100, Steve Davis > wrote: > >> >> A very interesting article... and all the more interesting because it >> lines up with Joe's predictions. >> >> >> http://www.kurzweilai.net/scientists-squeeze-more-than-1000-cores-onto-computer-chip >> >> ?> ?FPGAs are not used within standard computers because they are fairly >> ?> difficult to program, but their processing power is huge while their >> ?> energy consumption is very small because they are so much quicker, so >> ?> they are also a greener option,? said researcher Dr. Wim >> ?> Vanderbauwhede. >> >> ?> While most computers sold today now contain more than one processing >> ?> core, which allows them to carry out different processes >> ?> simultaneously, traditional multi-core processors must share access >> ?> to one memory source, which slows the system down. The scientists in >> ?> this research were able to make the processor faster by giving each >> ?> core a certain amount of dedicated memory. >> >> If the time is to come, then surely this is the *right time* for the >> Erlang/OTP 'product' to step up and meet the challenges and mindset that >> bind us daily owing to the chosen OO style forced onto us by C++/Java. >> >> /s >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > -- > Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From attila.r.nohl@REDACTED Fri Jan 7 12:58:50 2011 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Fri, 7 Jan 2011 12:58:50 +0100 Subject: [erlang-questions] FPGA coming around the corner In-Reply-To: References: <4D267D4A.6060101@gmail.com> Message-ID: 2011/1/7, Edmond Begumisa : >> If the time is to come, then surely this is the *right time* for the >> Erlang/OTP 'product' to step up and meet the challenges and mindset that >> bind us daily owing to the chosen OO style forced onto us by C++/Java. > > On a related note: I'm curious -- why is it that many seem unaware of > Erlang's advantages with multi-core? Even vocal MP experts that are > advocating for the change in mindset you describe and the need for > programmers to be aware of the disconnect and quickly adapt -- these > advocates don't sound like they're aware of Erlang (at least they don't > mention Erlang in their talks). Erlang is not a "silver bullet for multicore". The root cause of many scalability problems is shared data (e.g. which users are logged in to the system). You have to protect the shared data (using e.g. mnesia transactions or the state of a single gen_server) in Erlang too. If the algorithm doesn't scale, it doesn't matter what language is used. In Erlang it is harder to accidentally share data, but probably this is the only strong point of the language in this regard. From michael.eugene.turner@REDACTED Fri Jan 7 13:56:50 2011 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Fri, 7 Jan 2011 21:56:50 +0900 Subject: [erlang-questions] FPGA coming around the corner In-Reply-To: References: <4D267D4A.6060101@gmail.com> Message-ID: "... precisely what these advocates are warning their audiences about! That the *legacy apps and code will have to be re-written* in order to be parallelised for multi-core. That's what the panic is about." If that's the case (I haven't delved much into the sources you're citing), then the real problem here is that these people are panic-mongers. Legacy apps are typically written for, and running on, hardware that's 5-10 years old or older. Even with Moore's Law running out of steam, the users of these legacy apps have been getting free speedups every time they retired old hardware and moved their system to something newer. And if it was fast enough 5-10 years ago, it's much more than fast enough now, and on cheaper hardware to boot. If it hadn't been fast enough then, or had required prohibitively expensive hardware, it would have been rejected by its users back then, and it wouldn't be today's legacy app. The assumption seems to be that everything must be parallelized for multi-core. I fail to see why. > My issue is that these tools really suck! May as well just re-write the damn thing in Erlang At any given time, almost every piece of software sucks and seems it ought to be rewritten, preferably in a more appropriate language. (Even I daydream of how certain C++ features, used judiciously, would go a long way toward cleaning up the BEAM VM code.) Dwell on this perennial truth enough, and you will go insane, guaranteed. Oh well, as long as this is all really about impotent complaining, let me jump right in with my own impotent complaint: it drives me crazy that I have to type a $ in front of every variable name in PHP. I was doing that in BASIC in 1971, back when there was some excuse for parsers being stupid. What's the excuse now?! In fact, it drives me crazy that I have to deal with PHP at all. But, you know, if I want to hack on skins for MediaWiki (I do, at least this week, perhaps sanity will return next week), that's what I have to live with. The weight of history is great, people and production software systems and computer languages change only rather slowly, and half the people out there who are hectoring others about the need for dramatic change are in fact only trying to sell you something you don't necessarily need. What else is new? -michael turner On Fri, Jan 7, 2011 at 3:44 PM, Edmond Begumisa wrote: > On Fri, 07 Jan 2011 16:04:23 +1100, Michael Turner < > michael.eugene.turner@REDACTED> wrote: > > "... They all sound panic-stricken and suggest there is currently no >> complete viable solution out-there yet to programming for the multi-core >> age >> that don't burden the programmer. These are some very smart people. But >> when >> I saw these videos recently I thought "Have they not heard of Erlang? If >> not >> why not?" >> >> If you're overridingly concerned with fault-tolerance and ease of scaling >> on >> multi-core systems, and starting with a clean slate, Erlang makes a lot of >> sense. But that's not the focus of the "panicked" people mentioned here. >> Their interest is in raw performance, and perhaps also very much raw >> performance for legacy systems written in more mainstream languages that >> people would like to see parallelized. >> >> Erlang's not bad with respect to speed, but it's not great either: >> distributing an Erlang program over 5-10 cores might give you the same >> speedup you could get on a *single* core in C/C++/Java simply by >> optimizing >> your code savagely. >> > > (Assuming this is true) It's kind of the point! The fact that you don't > have to do much to get the speed up with the parallelised Erlang program! > That using your scale, the same speed up is obtained by... > > a) Running the parallelised Erlang program on multi-core with little change > & no savage optimisation, versus > b) Savagely optimising the single core C/C++/Java, OR > c) beating both by parallelising the C/C++/Java code on multi-core and > loosing your mind in the process (again, assuming the code in question runs > sequentially faster in these languages) > > And requiring programmers to rewrite legacy apps in a >> new language with unfamiliar paradigms -- well, it might be good in the >> long run, but in the long run, we're all dead, and there's an awful lotof >> working, useful code out there. >> > > But that's precisely what these advocates are warning their audiences > about! That the *legacy apps and code will have to be re-written* in order > to be parallelised for multi-core. That's what the panic is about. That > these programs and libraries will need to be carefully re-coded using lots > of threads, concurrent data structures and unfamiliar paradigms like > "transactional boosting." > > The Herlihy's are effectively saying: "You will need to re-write you're > apps and libs in order for your users not to complain that things are > slower, and here's a bunch of tools you can use that we *hope* will make it > a little easier. But it will still be pretty hard." > > My issue is that these tools really suck! May as well just re-write the > damn thing in Erlang :) > > > As pointed out in a recent thread touching >> on the raison d'etre for NIFs in Erlang, NIFs are less about making Erlang >> code fast and more about interfacing with all the useful stuff people have >> been writing and debugging for decades. And why not? OTP is large, a >> great >> toolbox, but outside its core strengths, it only scratches the surface of >> what people might want to do in software. >> >> Most of these "panic-stricken" people have probably heard of Erlang -- it >> just doesn't necessarily solve their problems. Maybe nothing can. But >> there's been great progress with tools that help model and debug >> concurrency >> in legacy apps, and that's probably going to continue. They aren't out >> of >> rope yet. Face it: successful software technologies take a long time to >> die. Whenever you log onto your bank's website, it's virtually guaranteed >> that the part of the delay involves cranking up some virtual machine >> environment to run some crufty old accounting package written 30 years ago >> in COBOL. >> >> -michael turner >> >> >> >> On Fri, Jan 7, 2011 at 1:40 PM, Edmond Begumisa < >> ebegumisa@REDACTED >> >>> wrote: >>> >> >> If the time is to come, then surely this is the *right time* for the >>> >>>> Erlang/OTP 'product' to step up and meet the challenges and mindset that >>>> bind us daily owing to the chosen OO style forced onto us by C++/Java. >>>> >>>> >>> On a related note: I'm curious -- why is it that many seem unaware of >>> Erlang's advantages with multi-core? Even vocal MP experts that are >>> advocating for the change in mindset you describe and the need for >>> programmers to be aware of the disconnect and quickly adapt -- these >>> advocates don't sound like they're aware of Erlang (at least they don't >>> mention Erlang in their talks). >>> >>> A blog post recently alerted me to a talk Maurice Herlihy gave back in >>> 2007 >>> entitled "Taking Concurrency Seriously: New Directions in Multiprocessor >>> Synchronization." >>> >>> http://www.stanford.edu/class/ee380/Abstracts/070502.html >>> >>> >>> http://ee380.stanford.edu/cgi-bin/videologger.php?target=070509-ee380-300.asx >>> >>> To quote him "... Computer architecture is about to undergo, if not >>> another >>> revolution, then a vigorous shaking-up. The major chip manufacturers >>> have, >>> for the time being, simply given up trying to make processors run >>> faster... >>> As a result, system designers and software engineers can no longer rely >>> on >>> increasing clock speed to hide software bloat. Instead, they must somehow >>> learn to make effective use of increasing parallelism. This adaptation >>> will >>> not be easy. Conventional synchronization techniques based on locks and >>> conditions are unlikely to be effective in such a demanding >>> environment..." >>> >>> He then goes on the show some pretty fancy Java code illustrating >>> concurrent data structures and a concept he calls "transactional >>> boosting" >>> in an attempt to deal with the disconnect without burdening the >>> programmer >>> "too much." >>> >>> There are a number of more recent talks on that Standford page with the >>> same tone... >>> >>> http://www.stanford.edu/class/ee380/previous.html >>> >>> They all sound panic-stricken and suggest there is currently no complete >>> viable solution out-there yet to programming for the multi-core age that >>> don't burden the programmer. These are some very smart people. But when I >>> saw these videos recently I thought "Have they not heard of Erlang? If >>> not >>> why not?" >>> >>> - Edmond - >>> >>> >>> On Fri, 07 Jan 2011 13:41:14 +1100, Steve Davis < >>> steven.charles.davis@REDACTED> wrote: >>> >>> >>> A very interesting article... and all the more interesting because it >>>> lines up with Joe's predictions. >>>> >>>> >>>> >>>> http://www.kurzweilai.net/scientists-squeeze-more-than-1000-cores-onto-computer-chip >>>> >>>> > ?FPGAs are not used within standard computers because they are fairly >>>> > difficult to program, but their processing power is huge while their >>>> > energy consumption is very small because they are so much quicker, so >>>> > they are also a greener option,? said researcher Dr. Wim >>>> > Vanderbauwhede. >>>> >>>> > While most computers sold today now contain more than one processing >>>> > core, which allows them to carry out different processes >>>> > simultaneously, traditional multi-core processors must share access >>>> > to one memory source, which slows the system down. The scientists in >>>> > this research were able to make the processor faster by giving each >>>> > core a certain amount of dedicated memory. >>>> >>>> >>>> If the time is to come, then surely this is the *right time* for the >>>> Erlang/OTP 'product' to step up and meet the challenges and mindset that >>>> bind us daily owing to the chosen OO style forced onto us by C++/Java. >>>> >>>> /s >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>>> >>>> >>> -- >>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >>> > > -- > Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > From info@REDACTED Fri Jan 7 14:04:46 2011 From: info@REDACTED (info) Date: Fri, 7 Jan 2011 14:04:46 +0100 Subject: [erlang-questions] Erlang + Emacs + Distel on windows References: <201012281349493298586@its3.ch>, <201101061846011934516@its3.ch>, Message-ID: <201101071404462225892@its3.ch> Hello Mohamed, 1) Install is a text file with explanation 2) makefile is for Linux, not for Windows ! 3) I downloaded distel from this address: http://distel.googlecode.com/files/distel-4.03.tgz J-Ph. Constantin ITS3 Gen?ve www.its3.ch From ebegumisa@REDACTED Fri Jan 7 14:29:38 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Sat, 08 Jan 2011 00:29:38 +1100 Subject: [erlang-questions] FPGA coming around the corner In-Reply-To: References: <4D267D4A.6060101@gmail.com> Message-ID: On Fri, 07 Jan 2011 20:45:13 +1100, Joe Armstrong wrote: > On Fri, Jan 7, 2011 at 5:40 AM, Edmond Begumisa > wrote: >>> If the time is to come, then surely this is the *right time* for the >>> Erlang/OTP 'product' to step up and meet the challenges and mindset >>> that >>> bind us daily owing to the chosen OO style forced onto us by C++/Java. >> >> On a related note: I'm curious -- why is it that many seem unaware of >> Erlang's advantages with multi-core? Even vocal MP experts that are >> advocating for the change in mindset you describe and the need for >> programmers to be aware of the disconnect and quickly adapt -- these >> advocates don't sound like they're aware of Erlang (at least they don't >> mention Erlang in their talks). > > Reminds me of a weird meeting at work a few weeks ago: > > They: But why doesn't it go 60 times faster (on a tile64) - it's > only 40 times faster? > Me: Well are you happy about the times 40 or sad about the fact it's > not 60? > They: Well happy, but it would be nice if it were 60 > Me: We're working on it > > Our domain of discourse, is "why we don't get a factor N speedup on > N processors" - my feeling is that in many places the domain of > discourse is "how can we go faster than times one on two processors > and not slower" > > >> >> A blog post recently alerted me to a talk Maurice Herlihy gave back in >> 2007 >> entitled "Taking Concurrency Seriously: New Directions in Multiprocessor >> Synchronization." >> >> http://www.stanford.edu/class/ee380/Abstracts/070502.html >> http://ee380.stanford.edu/cgi-bin/videologger.php?target=070509-ee380-300.asx >> >> To quote him "... Computer architecture is about to undergo, if not >> another >> revolution, then a vigorous shaking-up. The major chip manufacturers >> have, >> for the time being, simply given up trying to make processors run >> faster... >> As a result, system designers and software engineers can no longer rely >> on >> increasing clock speed to hide software bloat. Instead, they must >> somehow >> learn to make effective use of increasing parallelism. This adaptation >> will >> not be easy. Conventional synchronization techniques based on locks and >> conditions are unlikely to be effective in such a demanding >> environment..." >> >> He then goes on the show some pretty fancy Java code illustrating >> concurrent >> data structures and a concept he calls "transactional boosting" in an >> attempt to deal with the disconnect without burdening the programmer >> "too >> much." >> >> There are a number of more recent talks on that Standford page with the >> same >> tone... >> >> http://www.stanford.edu/class/ee380/previous.html >> >> They all sound panic-stricken and suggest there is currently no complete >> viable solution out-there yet to programming for the multi-core age that >> don't burden the programmer. These are some very smart people. But when >> I >> saw these videos recently I thought "Have they not heard of Erlang? If >> not >> why not?" > > This question is asked to the the wrong group. Very nice answer! Maybe you should send them that chapter from your book :) - Edmond - > I guess most people in > this group know that Erlang is (according to the wikipedia) > > "a Chinese God with a third truth-seeing eye in the middle of his > forehead." > > /Joe > > > >> >> - Edmond - >> >> >> On Fri, 07 Jan 2011 13:41:14 +1100, Steve Davis >> wrote: >> >>> >>> A very interesting article... and all the more interesting because it >>> lines up with Joe's predictions. >>> >>> >>> http://www.kurzweilai.net/scientists-squeeze-more-than-1000-cores-onto-computer-chip >>> >>> > ?FPGAs are not used within standard computers because they are >>> fairly >>> > difficult to program, but their processing power is huge while their >>> > energy consumption is very small because they are so much quicker, >>> so >>> > they are also a greener option,? said researcher Dr. Wim >>> > Vanderbauwhede. >>> >>> > While most computers sold today now contain more than one processing >>> > core, which allows them to carry out different processes >>> > simultaneously, traditional multi-core processors must share access >>> > to one memory source, which slows the system down. The scientists in >>> > this research were able to make the processor faster by giving each >>> > core a certain amount of dedicated memory. >>> >>> If the time is to come, then surely this is the *right time* for the >>> Erlang/OTP 'product' to step up and meet the challenges and mindset >>> that >>> bind us daily owing to the chosen OO style forced onto us by C++/Java. >>> >>> /s >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> >> -- >> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From michael.eugene.turner@REDACTED Fri Jan 7 14:42:54 2011 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Fri, 7 Jan 2011 22:42:54 +0900 Subject: [erlang-questions] Erlang Intro/Evangelism Presentation -- Starbucks! In-Reply-To: References: Message-ID: "... I started to think about how Starbucks processes drink orders. Starbucks, like most other businesses is primarily interested in maximizing throughput of orders. More orders equals more revenue. As a result they use asynchronous processing ..." Starbucks might actually be an interesting metaphor here, instructive for thinking about how to write Erlang apps, but I beg to differ about what's being optimized. Starbucks is NOT optimizing throughput. It's optimizing replicability. I must explain: I come from Berkeley, a town that lives on espresso. At certain cafes, I could routinely see two people behind the counter delivering twice as much espresso per minute as Starbucks does with five or six people behind the counter. What Starbucks is optimizing is how to rapidly deploy relatively slow, lower-skill employees to provide a consistent experience of a relatively even flow of order processing, a sense (for the customer) that inexorable progress is being made. So long as this continues to attract a flow of customers who've never known better, it will work. And I have to admit, I've gotten used to finally getting to the Starbucks counter, ordering a capuccino, and having two other employees echo my order, even though it's rarely under three minutes before I actually get the drink. But at first I couldn't stand it. In some of the cafes I got used to in Berkeley, someone at the counter would often be taking my order before I even got to the cash register, and putting the drink in front of me by the time I had my wallet out to pay the cashier. THAT was some real throughput. Not to mention that watching these people furiously take orders, crank out drinks and collect money was like being at a martial arts performance. Sometimes the way to speed something up is to parallelize it. Sometimes the way to speed it up is with a fast uni-processor running highly optimized procedures. But sometimes what you're trying to optimize isn't speed of operations, really, but growth. Starbucks was in fact doing that last. It's optimizing for blanketing cityscapes with its storefronts, as it has in Tokyo. Optimizing throughput -- which requires optimizing employees -- would only have slowed that growth process. Those cafes in Berkeley I worshipped for their ferociously pirouetting baristas were wonderful, but they would never scale. Starbucks scaled. A good process structure in Erlang also scales. That's the real lesson in any such analogy, I think. -michael turner On Fri, Dec 17, 2010 at 9:48 PM, Edmond Begumisa < ebegumisa@REDACTED> wrote: > MESSAGE-PASSING @ STARBUCKS > > "... I started to think about how Starbucks processes drink orders. > Starbucks, like most other businesses is primarily interested in maximizing > throughput of orders. More orders equals more revenue. As a result they use > asynchronous processing ..." > > "... we can see that the real world is often asynchronous. Our daily lives > consists of many coordinated, but asynchronous interactions... asynchronous > messaging architecture can often be a natural way to model these types of > interactions... also means that often we can look at daily life to help > design successful messaging solutions ..."* > > http://www.eaipatterns.com/ramblings/18_starbucks.html > > I hope I'm not too late with this, but I really liked this old article by > Google's Gregor Hohpe on asynchronous message passing. He wasn't talking > explicitly about Erlang but the content should resonate instantaneously with > Erlang programmers while at the same time being insightful for > non-Erlangers. You might want to reference a few of his well-worded > analogies for evangelising some of the core ideas built into Erlang. > > *Joe Armstrong made a similar compelling argument at the beginning of his > book. As a relatively new Erlang programmer myself, _THIS_ is the main line > of argument that convinced me to try out the Erlang environment. > Particularly the implications on concurrency which I had struggled with in > other environments (in retrospect, it was because those environments didn't > have these ideas built-in or actively encouraged -- I've found Erlang does a > lot of hand-holding to make sure you do the sane thing in order to avoid > painting yourself into a corner -- something you could emphasise in your > presentation.) > > - Edmond - > > > On Sat, 04 Dec 2010 05:43:34 +1100, Ryan Zezeski > wrote: > > I've been tasked with giving an introductory level presentation on Erlang >> at >> work. The focus is on why you would use Erlang and what does it look like >> on the surface level. We have a lot of your standard Java/C#/C developers >> and this will act as a potential launching pad to introducing Erlang to >> the >> company. There is the potential for a large number of people to video >> conference into this possibly spanning several countries so I really want >> to >> knock this out of the park. >> >> I'm writing as an inquiry for links to any prior art that I may use for >> inspiration or even steal and use as my own. Anything that you think >> might >> be helpful is appreciated, even input on possible approaches. I'd really >> like to hear from people who have done this sort of thing before, i.e. >> pubic >> speaking on Erlang. >> >> Thanks, >> -Ryan >> > > > -- > Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From ebegumisa@REDACTED Fri Jan 7 14:59:50 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Sat, 08 Jan 2011 00:59:50 +1100 Subject: [erlang-questions] FPGA coming around the corner In-Reply-To: References: <4D267D4A.6060101@gmail.com> Message-ID: On Fri, 07 Jan 2011 22:58:50 +1100, Attila Rajmund Nohl wrote: > 2011/1/7, Edmond Begumisa : >>> If the time is to come, then surely this is the *right time* for the >>> Erlang/OTP 'product' to step up and meet the challenges and mindset >>> that >>> bind us daily owing to the chosen OO style forced onto us by C++/Java. >> >> On a related note: I'm curious -- why is it that many seem unaware of >> Erlang's advantages with multi-core? Even vocal MP experts that are >> advocating for the change in mindset you describe and the need for >> programmers to be aware of the disconnect and quickly adapt -- these >> advocates don't sound like they're aware of Erlang (at least they don't >> mention Erlang in their talks). > > Erlang is not a "silver bullet for multicore". I don't think it is either! My point was parallelising parallel-able code is much much easier in Erlang than some of the options being offered up for the same purpose in other programming environments. It's as though the Erlang team has done most of the heavy lifting regarding SMP for you so you don't have to go looking at "transactional boosting." > The root cause of many > scalability problems is shared data (e.g. which users are logged in to > the system). You have to protect the shared data (using e.g. mnesia > transactions or the state of a single gen_server) in Erlang too. If > the algorithm doesn't scale, it doesn't matter what language is used. > In Erlang it is harder to accidentally share data, but probably this > is the only strong point of the language in this regard. > I don't know about it being the only strong point. I think it's more than that. What floored me about Erlang in this area is the ease with which you can scan through your code looking for stuff you can parallelise and actually parallelise it without much fuss (whether or not you'll get the expected speed-ups is another question). Garcia's recent Pythagorus thread comes to mind... http://groups.google.com/group/erlang-programming/browse_thread/thread/bbe170bb331ea995?tvc=2 But as Joe pointed out, the concern is programs suddenly starting to run slower and what you can do about it. The "what you can do about it" part seems to me to be a lot saner in Erlang than the pain say Java programmers have to go have to go through (at least when following that video.) - Edmond - > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From ebegumisa@REDACTED Fri Jan 7 15:54:47 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Sat, 08 Jan 2011 01:54:47 +1100 Subject: [erlang-questions] Erlang Intro/Evangelism Presentation -- Starbucks! In-Reply-To: References: Message-ID: On Sat, 08 Jan 2011 00:42:54 +1100, Michael Turner wrote: > "... I started to think about how Starbucks processes drink orders. > Starbucks, like most other businesses is primarily interested in > maximizing > throughput of orders. More orders equals more revenue. As a result they > use > asynchronous processing ..." > > Starbucks might actually be an interesting metaphor here, instructive for > thinking about how to write Erlang apps, but I beg to differ about what's > being optimized. > > Starbucks is NOT optimizing throughput. It's optimizing replicability. > > I must explain: I come from Berkeley, a town that lives on espresso. At > certain cafes, I could routinely see two people behind the counter > delivering twice as much espresso per minute as Starbucks does with five > or > six people behind the counter. > > What Starbucks is optimizing is how to rapidly deploy relatively slow, > lower-skill employees to provide a consistent experience of a relatively > even flow of order processing, a sense (for the customer) that inexorable > progress is being made. So long as this continues to attract a flow of > customers who've never known better, it will work. And I have to admit, > I've gotten used to finally getting to the Starbucks counter, ordering a > capuccino, and having two other employees echo my order, even though it's > rarely under three minutes before I actually get the drink. But at > first I > couldn't stand it. In some of the cafes I got used to in Berkeley, > someone > at the counter would often be taking my order before I even got to the > cash > register, and putting the drink in front of me by the time I had my > wallet > out to pay the cashier. THAT was some real throughput. Not to mention > that > watching these people furiously take orders, crank out drinks and collect > money was like being at a martial arts performance. > > Sometimes the way to speed something up is to parallelize it. Sometimes > the > way to speed it up is with a fast uni-processor running highly optimized > procedures. Food for thought: What do you do when your highly optimised procedures have to run on slower sequential processors? (i.e when your program has to run on newer hardware) If your procedures are already optimised to death, your only option is to _try_ and parellise. I think that's what's getting peoples knickers in knots regarding multi-core. > But sometimes what you're trying to optimize isn't speed of > operations, really, but growth. Starbucks was in fact doing that last. > It's optimizing for blanketing cityscapes with its storefronts, as it > has > in Tokyo. Optimizing throughput -- which requires optimizing employees > -- > would only have slowed that growth process. Those cafes in Berkeley I > worshipped for their ferociously pirouetting baristas were wonderful, but > they would never scale. Starbucks scaled. A good process structure in > Erlang also scales. That's the real lesson in any such analogy, I think. > Interesting observation -- I hadn't thought of the replication angle or process structure. In terms of "Optimizing throughput": I think the point of the article though was to illustrate an alternative to two-phase commit. Specifically, the use of async message-passing techniques to avoid blocking transactions improving throughput in the process. I think async message-passing + non-blocking operations and the impact this has on responsiveness is an important lesson for Erlang newcomers and/or can be used as a "pitch" to sell the environment to those thinking about dipping their feet and needing a reason to jump in. Well, at least it was for me :) - Edmond - > -michael turner > > > > On Fri, Dec 17, 2010 at 9:48 PM, Edmond Begumisa < > ebegumisa@REDACTED> wrote: > >> MESSAGE-PASSING @ STARBUCKS >> >> "... I started to think about how Starbucks processes drink orders. >> Starbucks, like most other businesses is primarily interested in >> maximizing >> throughput of orders. More orders equals more revenue. As a result they >> use >> asynchronous processing ..." >> >> "... we can see that the real world is often asynchronous. Our daily >> lives >> consists of many coordinated, but asynchronous interactions... >> asynchronous >> messaging architecture can often be a natural way to model these types >> of >> interactions... also means that often we can look at daily life to help >> design successful messaging solutions ..."* >> >> http://www.eaipatterns.com/ramblings/18_starbucks.html >> >> I hope I'm not too late with this, but I really liked this old article >> by >> Google's Gregor Hohpe on asynchronous message passing. He wasn't talking >> explicitly about Erlang but the content should resonate instantaneously >> with >> Erlang programmers while at the same time being insightful for >> non-Erlangers. You might want to reference a few of his well-worded >> analogies for evangelising some of the core ideas built into Erlang. >> >> *Joe Armstrong made a similar compelling argument at the beginning of >> his >> book. As a relatively new Erlang programmer myself, _THIS_ is the main >> line >> of argument that convinced me to try out the Erlang environment. >> Particularly the implications on concurrency which I had struggled with >> in >> other environments (in retrospect, it was because those environments >> didn't >> have these ideas built-in or actively encouraged -- I've found Erlang >> does a >> lot of hand-holding to make sure you do the sane thing in order to avoid >> painting yourself into a corner -- something you could emphasise in your >> presentation.) >> >> - Edmond - >> >> >> On Sat, 04 Dec 2010 05:43:34 +1100, Ryan Zezeski >> wrote: >> >> I've been tasked with giving an introductory level presentation on >> Erlang >>> at >>> work. The focus is on why you would use Erlang and what does it look >>> like >>> on the surface level. We have a lot of your standard Java/C#/C >>> developers >>> and this will act as a potential launching pad to introducing Erlang to >>> the >>> company. There is the potential for a large number of people to video >>> conference into this possibly spanning several countries so I really >>> want >>> to >>> knock this out of the park. >>> >>> I'm writing as an inquiry for links to any prior art that I may use for >>> inspiration or even steal and use as my own. Anything that you think >>> might >>> be helpful is appreciated, even input on possible approaches. I'd >>> really >>> like to hear from people who have done this sort of thing before, i.e. >>> pubic >>> speaking on Erlang. >>> >>> Thanks, >>> -Ryan >>> >> >> >> -- >> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From joe@REDACTED Fri Jan 7 17:04:30 2011 From: joe@REDACTED (Joe Williams) Date: Fri, 7 Jan 2011 08:04:30 -0800 Subject: [erlang-questions] FPGA coming around the corner In-Reply-To: References: <4D267D4A.6060101@gmail.com> Message-ID: <8534E548-1A7D-4B10-A3D8-A733C9205761@joetify.com> On Jan 7, 2011, at 3:58 AM, Attila Rajmund Nohl wrote: > 2011/1/7, Edmond Begumisa : >>> If the time is to come, then surely this is the *right time* for the >>> Erlang/OTP 'product' to step up and meet the challenges and mindset that >>> bind us daily owing to the chosen OO style forced onto us by C++/Java. >> >> On a related note: I'm curious -- why is it that many seem unaware of >> Erlang's advantages with multi-core? Even vocal MP experts that are >> advocating for the change in mindset you describe and the need for >> programmers to be aware of the disconnect and quickly adapt -- these >> advocates don't sound like they're aware of Erlang (at least they don't >> mention Erlang in their talks). > > Erlang is not a "silver bullet for multicore". The root cause of many > scalability problems is shared data (e.g. which users are logged in to > the system). You have to protect the shared data (using e.g. mnesia > transactions or the state of a single gen_server) in Erlang too. If > the algorithm doesn't scale, it doesn't matter what language is used. > In Erlang it is harder to accidentally share data, but probably this > is the only strong point of the language in this regard. Agreed, Erlang nor any other language will ever be a silver bullet for problems that can't be paralellized. Thanks to Amdahl's Law (https://secure.wikimedia.org/wikipedia/en/wiki/Amdahl's_law). What it really comes down to is that in HPC they have been doing shared memory calculations in highly coupled systems for years in Fortran, C and friends and changing not only the code but the mindset is a huge barrier. I'm sure in many cases shared memory is the only way to do what they want without a prohibitive amount of pain. Not to mention industry has built up around this, SGI, IBM and etc make lots of money building ever larger systems to do this work on. So at the end of the day I think it's lot more than "just use Erlang". -Joe Name: Joseph A. Williams Email: joe@REDACTED Blog: http://www.joeandmotorboat.com/ Twitter: http://twitter.com/williamsjoe From gordon@REDACTED Fri Jan 7 17:41:52 2011 From: gordon@REDACTED (Gordon Guthrie) Date: Fri, 7 Jan 2011 16:41:52 +0000 Subject: Erlounge Edinburgh, Bonnie Scotland (Short Notice) Message-ID: Folks There is a functional meetup in Edinburgh on Monday (10th Jan) in which I will be doing a talk entitled: "Building A Commercial Functional Programming Language In A Commercial Functional Programming Language" I figured we might was well start the year with an impromptu Erlounge Details here: http://bloop.co/events/wz/ If anyone wants to come along and say the magic words "nine-nines" I will buy them a drink :) Gordon -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669 From attila.r.nohl@REDACTED Fri Jan 7 19:27:07 2011 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Fri, 7 Jan 2011 19:27:07 +0100 Subject: [erlang-questions] FPGA coming around the corner In-Reply-To: References: <4D267D4A.6060101@gmail.com> Message-ID: 2011/1/7, Edmond Begumisa : > On Fri, 07 Jan 2011 22:58:50 +1100, Attila Rajmund Nohl [...] >> Erlang is not a "silver bullet for multicore". > > I don't think it is either! My point was parallelising parallel-able code > is much much easier in Erlang than some of the options being offered up > for the same purpose in other programming environments. It's as though the > Erlang team has done most of the heavy lifting regarding SMP for you so > you don't have to go looking at "transactional boosting." In a previous project I was working on, we had a C++ server implementing some CORBA functions. Some stub code had to be generated from the IDLs, then our code had to inherit from these generated classes. The orber run all requests in a separate thread - essentially we got parallelism for free (i.e. no extra programming necessary). We also had a Thread class, so when something had to run in the background, the code was put in a separate class that inherited from this Thread class and we just had to call it's start function. This is not unlike what I had to do in Erlang when I write a new gen_server... In my experience parallelism does not differentiate Erlang from other languages. At first glance the message passing seems to be radically different to function calling, but I very very very rarely use the ! operator, 99.99% of the cases it is hidden behind a gen_server or gen_fsm:call. What makes Erlang different from other languages is its VM. Hot code loading makes compile&test cycles blazingly fast - you don't even have to restart the application to test the changes! Not to mention that the live system can be patched (useful for the many 9s reliability). I think the real productivity gains from Erlang come from this fact, it doesn't have to do anything with the language. The ability to connect to live systems and poke around their internals helps debugging greatly. I've just spent half an hour investigating a deadlock situation (which turned out to be not a deadlock after all, just a slow TCP stack) using erlang:process_info, sys:get_status and friends to get the state of the system. It is much harder to do on a C++ software with e.g. gdb (especially when there's no gdb at the live system for security reasons). I don't know the scriptability of gdb, but in the Erlang shell you can write - well, Erlang programs, which is dead useful if you need to find 1 process out of 100s. From paf@REDACTED Fri Jan 7 22:25:06 2011 From: paf@REDACTED (Paulo Alexandre Ferreira) Date: Fri, 7 Jan 2011 21:25:06 +0000 Subject: [erlang-questions] FPGA coming around the corner Message-ID: <20110107212506.M74181@dei.isep.ipp.pt> First, let's get the meaning of "core" straight. For the software people core means one PROCESSOR core. For the hardware guys core mean one "block of hardware". You can have one USB interface core, one PCI interface core, and so on. On the work of Dr. Vanderbauwhede the cores are more like "processing elements" than traditional processor cores. One can see that on one of his papers: http://eprints.gla.ac.uk/40011/ The work is remarkable but is not about 1000 "traditional processors" on a chip. It is about "coarse grained" reconfigurable computing. But Erlang is a very interesting model for FPGAs for several reasons. One is the message passing model. As long as the messages are delivered with the right content, the system works ok. So one could grab a slow part of an Erlang system, and implement them in hardware. For instance, could have an FPGA board in a computer implementing the crypto modules. Another interesting thing is heterogeneous computing. In "traditional" multicore processors, all the processors are equal. But, in an Erlang system some processes are more complex, and other processes are simple. If we want all the processes running at the same time, why give all the processes the same kind of processor? With FPGAs one can build the "right" processor for each process, if we have the resources to give each process an "hardware block". Some interesting (and hard to do) things are dynamic processes but I like the topic and in "my spare time" I managed to write some things about it: FPL2010 http://www.dei.isep.ipp.pt/~paf/papers/FPL2010-final.pdf DCIS2010 http://www.dei.isep.ipp.pt/~paf/papers/DCIS2010-final.pdf Hope some of you like it, and that some of you tell me I am wrong and why. ;-) ;-) ;-) And once again, many thanks for Erlang. Paulo Ferreira -- DEI-ISEP (http://www.dei.isep.ipp.pt) From jesper.louis.andersen@REDACTED Sat Jan 8 02:42:54 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sat, 8 Jan 2011 02:42:54 +0100 Subject: [erlang-questions] 1000 cores on a chip In-Reply-To: References: Message-ID: On Fri, Jan 7, 2011 at 04:30, Banibrata Dutta wrote: > However, those many cores in GPU world isn't new. CUDA has shown that to us > for a while. People are making use of GPU for general computation > increasingly, and a while back had come accross a startup that was using the > extreme parallelism (in a box) by CUDA for video analytics and video > encoding/decoding. Not sure how well FPGA's really fare w.r.t. power > consumption in such setups, but the CUDA GPU units do consume a good bit of > power.. and generate good bit of heat. The most important difference is that GPUs tend to be SIMD architectures with almost no branching instructions. This 1000-core beast is much closer to a thousand cores each running their own individual program - making them a better fit for Erlang as it is now. Suppose you have case Foo of true -> do_x(); false -> do_y() end In (most) GPUs this is compiled into: case Foo of true -> do_x(); false -> nop end, case Foo of true -> nop; false -> do_y() end So essentially the code runs twice: Once for the true branch and once for the false branch. Why? Because in a SIMD architecture all cores are locked to execute the same instruction, so you mask out the operation with a 'nop' for one branch at a time. The more complicated your branching tree is, the more expensive it is to do this. Many of the problems where you see GPUs smite the competition is where you have relatively few branches and can process data in a streamed fashion, more or less. Video decoding is an excellent example. Modern GPUs can load more than one program so only part of the GPU has to run locked to each other, but it is by no means easy to carry out and even harder to make automatic by compilation. Furthermore modern GPUs will to a greater and greater extent accept computational error. If the output is to a screen a single pixel error for 1/60th of a second isn't noticeable. It has been speculated that you can still use the GPU for more exact computation in the future though: if you can bound the error by some measurement, you can run the computation and then fix up smaller errors on the CPU afterwards. I've seen this trick used in another setting: Approximate integer calculations on FP hardware (SSE3) and then fixup the cases where computational error might have crept in by using the integer unit for this. The key difference between Erlang and SIMD is that while SIMD leans itself to parallel computation, it does not lean itself to concurrency that well. In Erlangs case we would like parallelism and concurrency at the same time which to some extent is easier pulled off on MIMD architectures. In other words: SIMD is very cool (and blazingly fast!) for some problems. Yet, it probably does not fit into the forte of Erlang that much and it would take considerable machinery to add it while it is not clear any benefit would be provided by the addition. -- J. From ulf.wiger@REDACTED Sat Jan 8 11:00:23 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 8 Jan 2011 11:00:23 +0100 Subject: [erlang-questions] FPGA coming around the corner In-Reply-To: References: <4D267D4A.6060101@gmail.com> Message-ID: On 7 Jan 2011, at 05:40, Edmond Begumisa wrote: > > They all sound panic-stricken and suggest there is currently no complete viable solution out-there yet to programming for the multi-core age that don't burden the programmer. These are some very smart people. But when I saw these videos recently I thought "Have they not heard of Erlang? If not why not?" Legacy software, sticking to their familiar paradigms, ? I also am reminded of Todd Proebsting's talk on "Disruptive Language Technologies" at the MIT LL2 workshop 2002 (same workshop where Joe first used the "nine nines" figure). http://ll2.ai.mit.edu/talks/proebsting.ppt He actually used a slightly different set of slides in his talk, and proclaimed Erlang as THE answer for disruptive concurrent language. An interesting strategy might then be to focus more on an emerging market rather than trying to beat the giants on their own game. I've been watching the evolution of a new breed of mobile device hardware, waiting for them to become suitable for running Erlang on. Today, they are. My Samsung Galaxy S is a significantly more powerful computer than the UltraSPARCs we first used to run Erlang on the AXD 301. The latest ARM design (http://www.theregister.co.uk/2010/09/09/new_arm/) is a system-on- a-chip design with up to 16 cores running at up to 2.5GHz. Programming that with a low-level shared memory approach is going to be a significant challenge? The challenge in the mobile device space is (at least) twofold: - Must minimize power consumption - Support increasingly complex demands on software A problem is that there is a fairly clear answer to the speed/Watt question: lower the clock frequency and replicate the cores. The problem is that this makes it harder to program the device using traditional techniques, which counteracts the second issue? unless you use something like Erlang, which does well on both counts. In this context, NIFs also become highly relevant, as a nice way to interface to libraries and custom chip features (GPUs, DSP functionality, etc, which are another key factor in getting performance on low-power chips.) Erlang has always had a conceptually nice approach to interfacing and controlling external hardware. I think there is huge potential for a language that loves multicore and is strong on coordination and fault tolerance, on the new generation of mobile device hardware. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From michael.eugene.turner@REDACTED Sat Jan 8 12:19:22 2011 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sat, 8 Jan 2011 20:19:22 +0900 Subject: [erlang-questions] Erlang Intro/Evangelism Presentation -- Starbucks! In-Reply-To: References: Message-ID: "What do you do when your highly optimised procedures have to run on slower sequential processors? " How often is that *truly* a requirement? In those cases where it is, these people who sell tools to parallelize legacy code are offering something Erlang can't offer, except perhaps through NIF interfaces to those highly optimized procedures. Either way, you're re-architecting the app. And requiring that the app be re-architected in a way that necessitates learning a new language will lose out pretty often to doing it in a way that preserves existing skill-sets. Companies like to get human skill-sets built for free, ideally by their competitors. Up-front investments in training -- even for conventional technologies -- are one of the hardest things to sell to management, in corporate life. People keep saying Erlang is a simple language, implying that getting over the learning curve must therefore be a simple matter. No. Erlang is a *small* language, but one that incorporates some powerful ideas and idiosyncratic approaches, requiring some time to digest. I'd had exposure to Prolog (helpful in understanding that "=" is pattern-match, not assignment, and in getting used to the odd convention for symbols where capitalization means "variable"). I had some experience in Lisp (helpful in understanding how Erlang handles lists). I'd also gotten some experience in parallel programming in more conventional terms (albeit in C- and C++-coded apps). And yet, Erlang was not, and is still not, easy for me. No amount of repetition of Erlang's selling points is going to make much dent in that sense of difficulty. Only long hard practice will get me to the point where I find Erlang programs as readable -- or more readable -- than their more conventional equivalents. And businesses don't pay employees for long, hard practice. They don't pay them for learning. That's just a cost, a necessary evil to be minimized. They pay employees for results that some customer will pay for, ideally tomorrow, though all too often the deadline was yesterday. -michael turner On Fri, Jan 7, 2011 at 11:54 PM, Edmond Begumisa < ebegumisa@REDACTED> wrote: > On Sat, 08 Jan 2011 00:42:54 +1100, Michael Turner < > michael.eugene.turner@REDACTED> wrote: > > "... I started to think about how Starbucks processes drink orders. >> Starbucks, like most other businesses is primarily interested in >> maximizing >> throughput of orders. More orders equals more revenue. As a result they >> use >> asynchronous processing ..." >> >> Starbucks might actually be an interesting metaphor here, instructive for >> thinking about how to write Erlang apps, but I beg to differ about what's >> being optimized. >> >> Starbucks is NOT optimizing throughput. It's optimizing replicability. >> >> I must explain: I come from Berkeley, a town that lives on espresso. At >> certain cafes, I could routinely see two people behind the counter >> delivering twice as much espresso per minute as Starbucks does with five >> or >> six people behind the counter. >> >> What Starbucks is optimizing is how to rapidly deploy relatively slow, >> lower-skill employees to provide a consistent experience of a relatively >> even flow of order processing, a sense (for the customer) that inexorable >> progress is being made. So long as this continues to attract a flow of >> customers who've never known better, it will work. And I have to admit, >> I've gotten used to finally getting to the Starbucks counter, ordering a >> capuccino, and having two other employees echo my order, even though it's >> rarely under three minutes before I actually get the drink. But at first >> I >> couldn't stand it. In some of the cafes I got used to in Berkeley, someone >> at the counter would often be taking my order before I even got to the >> cash >> register, and putting the drink in front of me by the time I had my wallet >> out to pay the cashier. THAT was some real throughput. Not to mention >> that >> watching these people furiously take orders, crank out drinks and collect >> money was like being at a martial arts performance. >> >> Sometimes the way to speed something up is to parallelize it. Sometimes >> the >> way to speed it up is with a fast uni-processor running highly optimized >> procedures. >> > > Food for thought: What do you do when your highly optimised procedures have > to run on slower sequential processors? (i.e when your program has to run on > newer hardware) > > If your procedures are already optimised to death, your only option is to > _try_ and parellise. I think that's what's getting peoples knickers in knots > regarding multi-core. > > > But sometimes what you're trying to optimize isn't speed of >> operations, really, but growth. Starbucks was in fact doing that last. >> It's optimizing for blanketing cityscapes with its storefronts, as it has >> in Tokyo. Optimizing throughput -- which requires optimizing employees -- >> would only have slowed that growth process. Those cafes in Berkeley I >> worshipped for their ferociously pirouetting baristas were wonderful, but >> they would never scale. Starbucks scaled. A good process structure in >> Erlang also scales. That's the real lesson in any such analogy, I think. >> >> > Interesting observation -- I hadn't thought of the replication angle or > process structure. > > In terms of "Optimizing throughput": I think the point of the article > though was to illustrate an alternative to two-phase commit. Specifically, > the use of async message-passing techniques to avoid blocking transactions > improving throughput in the process. I think async message-passing + > non-blocking operations and the impact this has on responsiveness is an > important lesson for Erlang newcomers and/or can be used as a "pitch" to > sell the environment to those thinking about dipping their feet and needing > a reason to jump in. Well, at least it was for me :) > > - Edmond - > > > -michael turner >> >> >> >> On Fri, Dec 17, 2010 at 9:48 PM, Edmond Begumisa < >> ebegumisa@REDACTED> wrote: >> >> MESSAGE-PASSING @ STARBUCKS >>> >>> "... I started to think about how Starbucks processes drink orders. >>> Starbucks, like most other businesses is primarily interested in >>> maximizing >>> throughput of orders. More orders equals more revenue. As a result they >>> use >>> asynchronous processing ..." >>> >>> "... we can see that the real world is often asynchronous. Our daily >>> lives >>> consists of many coordinated, but asynchronous interactions... >>> asynchronous >>> messaging architecture can often be a natural way to model these types of >>> interactions... also means that often we can look at daily life to help >>> design successful messaging solutions ..."* >>> >>> http://www.eaipatterns.com/ramblings/18_starbucks.html >>> >>> I hope I'm not too late with this, but I really liked this old article by >>> Google's Gregor Hohpe on asynchronous message passing. He wasn't talking >>> explicitly about Erlang but the content should resonate instantaneously >>> with >>> Erlang programmers while at the same time being insightful for >>> non-Erlangers. You might want to reference a few of his well-worded >>> analogies for evangelising some of the core ideas built into Erlang. >>> >>> *Joe Armstrong made a similar compelling argument at the beginning of his >>> book. As a relatively new Erlang programmer myself, _THIS_ is the main >>> line >>> of argument that convinced me to try out the Erlang environment. >>> Particularly the implications on concurrency which I had struggled with >>> in >>> other environments (in retrospect, it was because those environments >>> didn't >>> have these ideas built-in or actively encouraged -- I've found Erlang >>> does a >>> lot of hand-holding to make sure you do the sane thing in order to avoid >>> painting yourself into a corner -- something you could emphasise in your >>> presentation.) >>> >>> - Edmond - >>> >>> >>> On Sat, 04 Dec 2010 05:43:34 +1100, Ryan Zezeski >>> wrote: >>> >>> I've been tasked with giving an introductory level presentation on >>> Erlang >>> >>>> at >>>> work. The focus is on why you would use Erlang and what does it look >>>> like >>>> on the surface level. We have a lot of your standard Java/C#/C >>>> developers >>>> and this will act as a potential launching pad to introducing Erlang to >>>> the >>>> company. There is the potential for a large number of people to video >>>> conference into this possibly spanning several countries so I really >>>> want >>>> to >>>> knock this out of the park. >>>> >>>> I'm writing as an inquiry for links to any prior art that I may use for >>>> inspiration or even steal and use as my own. Anything that you think >>>> might >>>> be helpful is appreciated, even input on possible approaches. I'd >>>> really >>>> like to hear from people who have done this sort of thing before, i.e. >>>> pubic >>>> speaking on Erlang. >>>> >>>> Thanks, >>>> -Ryan >>>> >>>> >>> >>> -- >>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >>> > > -- > Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > From dirk.scharff@REDACTED Sat Jan 8 15:32:36 2011 From: dirk.scharff@REDACTED (Dirk Scharff) Date: Sat, 8 Jan 2011 15:32:36 +0100 Subject: split not splitting at pipe symbol Message-ID: Hi all! I'm experiencing a problem with re:split/2 which doesn't seem to be able to split at a pipe symbol "|". Is there a reason for this or is this a bug? Reproduce as follows: 1> re:split("123|456","|"). [<<"1">>,<<"2">>,<<"3">>,<<"|">>,<<"4">>,<<"5">>,<<"6">>, <<>>] 2> re:split("123|456",[124]). [<<"1">>,<<"2">>,<<"3">>,<<"|">>,<<"4">>,<<"5">>,<<"6">>, <<>>] 3> regexp:split("123|456","|"). {ok,["123","456"]} Regards, Dirk. From alain.odea@REDACTED Sat Jan 8 15:58:21 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Sat, 8 Jan 2011 11:28:21 -0330 Subject: [erlang-questions] re:split not splitting at pipe symbol In-Reply-To: References: Message-ID: <849E31A4-D4A8-48D8-8D4E-45CA7F1E95C3@gmail.com> On 2011-01-08, at 11:02, Dirk Scharff wrote: > Hi all! > > I'm experiencing a problem with re:split/2 which doesn't seem to be able to split at a pipe symbol "|". Is there a reason for this or is this a bug? > > Reproduce as follows: > 1> re:split("123|456","|"). Pipe is a meaningful symbol in regular expressions. It means 'or'. You need to escape the pipe: re:split("123|456","\\|"). > [<<"1">>,<<"2">>,<<"3">>,<<"|">>,<<"4">>,<<"5">>,<<"6">>, > <<>>] > 2> re:split("123|456",[124]). > [<<"1">>,<<"2">>,<<"3">>,<<"|">>,<<"4">>,<<"5">>,<<"6">>, > <<>>] > 3> regexp:split("123|456","|"). > {ok,["123","456"]} > > Regards, > Dirk. > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From ebegumisa@REDACTED Sat Jan 8 16:01:00 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Sun, 09 Jan 2011 02:01:00 +1100 Subject: [erlang-questions] FPGA coming around the corner In-Reply-To: References: <4D267D4A.6060101@gmail.com> Message-ID: On Sat, 08 Jan 2011 05:27:07 +1100, Attila Rajmund Nohl wrote: > 2011/1/7, Edmond Begumisa : >> On Fri, 07 Jan 2011 22:58:50 +1100, Attila Rajmund Nohl > [...] >>> Erlang is not a "silver bullet for multicore". >> >> I don't think it is either! My point was parallelising parallel-able >> code >> is much much easier in Erlang than some of the options being offered up >> for the same purpose in other programming environments. It's as though >> the >> Erlang team has done most of the heavy lifting regarding SMP for you so >> you don't have to go looking at "transactional boosting." > > In a previous project I was working on, we had a C++ server > implementing some CORBA functions. Some stub code had to be generated > from the IDLs, then our code had to inherit from these generated > classes. The orber run all requests in a separate thread - essentially > we got parallelism for free (i.e. no extra programming necessary). > [NOTE: I've never used CORBA but I have done plenty of MS (D)COM and Mozilla XPCOM, so maybe the following is due to mis-interpretation of your statement] Firstly, Multi OS Threads =/= (SMP) Parallelism. There is a big difference between apparent concurrency (what your code looks like) and true concurrency (what the machine is doing.) This is largely what the SMP advocacy is about. After you've got your code running on separate OS threads in C++/Java/etc, the next step -- ensuring the threads can *actually* do significant work in parallel on SMP -- is *usually* non-trivial. What Herlihy and hardware people are saying is that a lot of programmers are not aware that this next step is non-trivial or the subtitles involved in getting it right and their research is about making that next step easier. Secondly, the advantage I was describing here with Erlang is the ability to take some sequential chunk of code, like using the previous example of calculating a 300 Pythagorean triples, and then parallelise it (in code) by calculating each triple on a different process, then letting Erlang figure out how to actually run it in parallel on SMP. And the fact that when doing so, the code hardly changes. This is powerful. I haven't found another environment that offers me that as cleanly as Erlang. In contrast, from my experience, converting ST code to MT in C++ normally involves a lot of re-writing and careful planning. Compared to Erlang, it's harder in many languages to look at a piece of code and say: "let me modify this to do each little bit of the work on separate threads then combine the result." You can attempt to do things concurrently in Erlang that you probably wouldn't attempt in C++ therefore greatly increasing the opportunities of parallelising (I think it's partly to do with Erlang's functional nature, partly the quantity and speed at which processes can be created, and partly to do with share-nothing concurrency.) And it's even harder in C++ to say: "now let me try and get better use of my cores on SMP," IMO, Erlang is much better at that. > We also had a Thread class, so when something had to run in the > background, the code was put in a separate class that inherited from > this Thread class and we just had to call it's start function. This is > not unlike what I had to do in Erlang when I write a new gen_server... > In your thread class, what would you do when you wanted the objects to interact the way gen_servers interact? The *traditional* approach to this is shared state and inevitably thread synchronisation (you don't have to do this, but sharing state being the point of using threads rather than processes, it's the typical way.) This is where things get wonky with OS threads and where Erlang processes are very different. Having no shared state for your gen_server means you're *not likely* to have to do much to get them to do a good chunk of their work in true parallel on an SMP machine (the VM will take care of where things are scheduled and any locking that needs to be done if any -- it's none of your business). Many OS threads sharing state makes taking advantage of multiple cores significantly harder -- things you probably haven't thought about become your business. You're likely to have to re-write your code replacing standard data structures with concurrent versions so that you don't have to lock them and some other adaptations to get things to interleave better on SMP. Erlang doesn't place such heavy burdens on you. > In my experience parallelism does not differentiate Erlang from other > languages. That's interesting... my experience is the very opposite, at least in terms of "common mindsets". A lot of the paradigms and mindsets popular in other languages (e.g. threads accessing the same data structures with mutexes/rwlocks) don't lend themselves to parallelism. As Herlihy says, if these mindsets don't change then programmers who think in these terms won't be able to take advantage of increasing parallelism and *might* even suffer from the decreasing sequential clock speeds (though there's excessive panic about that last bit IMO). In contrast, the mindsets popular in Erlang lend themselves nicely to parallelism (like share nothing and message-passing.) This is probably because it's a concurrent language. Concurrency is a first-class concept in Erlang. This is a HUGE differentiation for me. I always found working with threads in C/C++ to be contrived. It felt like a tacked-on after-thought... http://weblogs.mozillazine.org/roadmap/archives/2007/02/threads_suck.html Concurrency and parallelism is the *reason* I chose to learn Erlang. I was working on version 3 of a distributed lock manager and a MT Server then one day thought: "No, no, no. This just won't do. I'm too old for maintaining this insanity. My brain can't operate at that clock-rate anymore. There must be a better way." Everything else was secondary. For me, the hot-code swapping, tracing, and other goodies you describe below were more of a pleasant surprise. I was attracted precisely to the differentiated concurrency and parallelism Erlang provides. I find it interesting that also coming from MT C++, you were not. Maybe there's more to CORBA than I've always thought :) - Edmond - > At first glance the message passing seems to be radically > different to function calling, but I very very very rarely use the ! > operator, 99.99% of the cases it is hidden behind a gen_server or > gen_fsm:call. > > What makes Erlang different from other languages is its VM. Hot code > loading makes compile&test cycles blazingly fast - you don't even have > to restart the application to test the changes! Not to mention that > the live system can be patched (useful for the many 9s reliability). I > think the real productivity gains from Erlang come from this fact, it > doesn't have to do anything with the language. > > The ability to connect to live systems and poke around their internals > helps debugging greatly. I've just spent half an hour investigating a > deadlock situation (which turned out to be not a deadlock after all, > just a slow TCP stack) using erlang:process_info, sys:get_status and > friends to get the state of the system. It is much harder to do on a > C++ software with e.g. gdb (especially when there's no gdb at the live > system for security reasons). I don't know the scriptability of gdb, > but in the Erlang shell you can write - well, Erlang programs, which > is dead useful if you need to find 1 process out of 100s. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From ebegumisa@REDACTED Sat Jan 8 17:35:48 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Sun, 09 Jan 2011 03:35:48 +1100 Subject: [erlang-questions] FPGA coming around the corner In-Reply-To: References: <4D267D4A.6060101@gmail.com> Message-ID: On Sat, 08 Jan 2011 21:00:23 +1100, Ulf Wiger wrote: > > On 7 Jan 2011, at 05:40, Edmond Begumisa wrote: >> >> They all sound panic-stricken and suggest there is currently no >> complete viable solution out-there yet to programming for the >> multi-core age that don't burden the programmer. These are some very >> smart people. But when I saw these videos recently I thought "Have they >> not heard of Erlang? If not why not?" > > > Legacy software, sticking to their familiar paradigms, ? > > I also am reminded of Todd Proebsting's talk on "Disruptive Language > Technologies" at the MIT LL2 workshop 2002 (same workshop where Joe > first used the "nine nines" figure). > > http://ll2.ai.mit.edu/talks/proebsting.ppt > > He actually used a slightly different set of slides in his talk, and > proclaimed > Erlang as THE answer for disruptive concurrent language. > Two statements stood out for me here... 1. "?Established [languages] concentrate on primary differentiator - SPEED?" This probably explains what the panic is about in other language camps. The worry that the speed differentiator is being threatened. That programs might run slower unless big adaptations are made to programming for SMP. Hmmmm... maybe this is a sign! 2. "?My criteria: [destructive] technology must - Have disadvantages - Be mostly ignored by recent PLD and POPL conferences* - Alleviate real problems?" * Answers my original question! > An interesting strategy might then be to focus more on an emerging market > rather than trying to beat the giants on their own game. > > I've been watching the evolution of a new breed of mobile device > hardware, > waiting for them to become suitable for running Erlang on. Today, they > are. > My Samsung Galaxy S is a significantly more powerful computer than the > UltraSPARCs we first used to run Erlang on the AXD 301. The latest ARM > design (http://www.theregister.co.uk/2010/09/09/new_arm/) is a system-on- > a-chip design with up to 16 cores running at up to 2.5GHz. Programming > that with a low-level shared memory approach is going to be a significant > challenge? > I'm very interested in Mobile too! For the current projects I'm working on, being able to deploy on mobiles in addition to boxes would give us huge advantages in countries where we intend to ship (across the developing world.) I was excited when I saw some determined folks had managed to get Erlang running on a Maemo Nokia N900 and an Android HTC Desire... http://thomas.apestaart.org/log/?p=1086 http://www.h-online.com/open/news/item/Couchio-announce-CouchDB-SDK-for-Android-1053555.html SMP Erlang on mobile sounds really cool :) Too bad about the annoying iPhone issues though :( > The challenge in the mobile device space is (at least) twofold: > - Must minimize power consumptio > - Support increasingly complex demands on software > > A problem is that there is a fairly clear answer to the speed/Watt > question: > lower the clock frequency and replicate the cores. The problem is that > this > makes it harder to program the device using traditional techniques, which > counteracts the second issue? unless you use something like Erlang, > which does well on both counts. > > In this context, NIFs also become highly relevant, as a nice way to > interface > to libraries and custom chip features (GPUs, DSP functionality, etc, > which > are another key factor in getting performance on low-power chips.) > Erlang has always had a conceptually nice approach to interfacing and > controlling external hardware. > > I think there is huge potential for a language that loves multicore and > is > strong on coordination and fault tolerance, on the new generation of > mobile device hardware. > I agree 110. Maybe Erickson can ship some official packaged mobile releases even for the current generation. It's still early days, not too late to attract developers starting afresh in the mobile sector. A painless-to-install language environment might be the difference in getting people to write their first mobile application in Erlang and not some other tool. - Edmond - > BR, > Ulf W > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From ebegumisa@REDACTED Sat Jan 8 17:53:20 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Sun, 09 Jan 2011 03:53:20 +1100 Subject: [erlang-questions] Erlang Intro/Evangelism Presentation -- Starbucks! In-Reply-To: References: Message-ID: On Sat, 08 Jan 2011 22:19:22 +1100, Michael Turner wrote: > "What do you do when your highly optimised procedures have to run on > slower > sequential processors? " > > How often is that *truly* a requirement? It's not now. But the worry (which I don't necessarily agree with) is that it will be in the near future. > In those cases where it is, these > people who sell tools to parallelize legacy code are offering something > Erlang can't offer, except perhaps through NIF interfaces to those highly > optimized procedures. Either way, you're re-architecting the app. And > requiring that the app be re-architected in a way that necessitates > learning > a new language will lose out pretty often to doing it in a way that > preserves existing skill-sets. Companies like to get human skill-sets > built > for free, ideally by their competitors. Up-front investments in > training -- > even for conventional technologies -- are one of the hardest things to > sell > to management, in corporate life. > Probably true. But I am reminded of something very interesting Ulf has just pointed me to... "The Innovator's Dilemma (C. Christensen) as quoted by Todd A. Proebsting ... "? companies [languages] that did everything right---were in tune with their competition, listened to their customers, and invested aggressively in new technologies---*still* lost their market leadership when confronted with disruptive changes in technology?" http://ll2.ai.mit.edu/talks/proebsting.ppt - Edmond - > People keep saying Erlang is a simple language, implying that getting > over > the learning curve must therefore be a simple matter. No. Erlang is a > *small* language, but one that incorporates some powerful ideas and > idiosyncratic approaches, requiring some time to digest. I'd had > exposure > to Prolog (helpful in understanding that "=" is pattern-match, not > assignment, and in getting used to the odd convention for symbols where > capitalization means "variable"). I had some experience in Lisp > (helpful in > understanding how Erlang handles lists). I'd also gotten some > experience in > parallel programming in more conventional terms (albeit in C- and > C++-coded > apps). And yet, Erlang was not, and is still not, easy for me. No > amount > of repetition of Erlang's selling points is going to make much dent in > that > sense of difficulty. Only long hard practice will get me to the point > where > I find Erlang programs as readable -- or more readable -- than their more > conventional equivalents. And businesses don't pay employees for long, > hard > practice. They don't pay them for learning. That's just a cost, a > necessary evil to be minimized. They pay employees for results that some > customer will pay for, ideally tomorrow, though all too often the > deadline > was yesterday. > > -michael turner > > On Fri, Jan 7, 2011 at 11:54 PM, Edmond Begumisa < > ebegumisa@REDACTED> wrote: > >> On Sat, 08 Jan 2011 00:42:54 +1100, Michael Turner < >> michael.eugene.turner@REDACTED> wrote: >> >> "... I started to think about how Starbucks processes drink orders. >>> Starbucks, like most other businesses is primarily interested in >>> maximizing >>> throughput of orders. More orders equals more revenue. As a result they >>> use >>> asynchronous processing ..." >>> >>> Starbucks might actually be an interesting metaphor here, instructive >>> for >>> thinking about how to write Erlang apps, but I beg to differ about >>> what's >>> being optimized. >>> >>> Starbucks is NOT optimizing throughput. It's optimizing replicability. >>> >>> I must explain: I come from Berkeley, a town that lives on espresso. >>> At >>> certain cafes, I could routinely see two people behind the counter >>> delivering twice as much espresso per minute as Starbucks does with >>> five >>> or >>> six people behind the counter. >>> >>> What Starbucks is optimizing is how to rapidly deploy relatively slow, >>> lower-skill employees to provide a consistent experience of a >>> relatively >>> even flow of order processing, a sense (for the customer) that >>> inexorable >>> progress is being made. So long as this continues to attract a flow of >>> customers who've never known better, it will work. And I have to >>> admit, >>> I've gotten used to finally getting to the Starbucks counter, ordering >>> a >>> capuccino, and having two other employees echo my order, even though >>> it's >>> rarely under three minutes before I actually get the drink. But at >>> first >>> I >>> couldn't stand it. In some of the cafes I got used to in Berkeley, >>> someone >>> at the counter would often be taking my order before I even got to the >>> cash >>> register, and putting the drink in front of me by the time I had my >>> wallet >>> out to pay the cashier. THAT was some real throughput. Not to mention >>> that >>> watching these people furiously take orders, crank out drinks and >>> collect >>> money was like being at a martial arts performance. >>> >>> Sometimes the way to speed something up is to parallelize it. >>> Sometimes >>> the >>> way to speed it up is with a fast uni-processor running highly >>> optimized >>> procedures. >>> >> >> Food for thought: What do you do when your highly optimised procedures >> have >> to run on slower sequential processors? (i.e when your program has to >> run on >> newer hardware) >> >> If your procedures are already optimised to death, your only option is >> to >> _try_ and parellise. I think that's what's getting peoples knickers in >> knots >> regarding multi-core. >> >> >> But sometimes what you're trying to optimize isn't speed of >>> operations, really, but growth. Starbucks was in fact doing that last. >>> It's optimizing for blanketing cityscapes with its storefronts, as it >>> has >>> in Tokyo. Optimizing throughput -- which requires optimizing >>> employees -- >>> would only have slowed that growth process. Those cafes in Berkeley I >>> worshipped for their ferociously pirouetting baristas were wonderful, >>> but >>> they would never scale. Starbucks scaled. A good process structure in >>> Erlang also scales. That's the real lesson in any such analogy, I >>> think. >>> >>> >> Interesting observation -- I hadn't thought of the replication angle or >> process structure. >> >> In terms of "Optimizing throughput": I think the point of the article >> though was to illustrate an alternative to two-phase commit. >> Specifically, >> the use of async message-passing techniques to avoid blocking >> transactions >> improving throughput in the process. I think async message-passing + >> non-blocking operations and the impact this has on responsiveness is an >> important lesson for Erlang newcomers and/or can be used as a "pitch" to >> sell the environment to those thinking about dipping their feet and >> needing >> a reason to jump in. Well, at least it was for me :) >> >> - Edmond - >> >> >> -michael turner >>> >>> >>> >>> On Fri, Dec 17, 2010 at 9:48 PM, Edmond Begumisa < >>> ebegumisa@REDACTED> wrote: >>> >>> MESSAGE-PASSING @ STARBUCKS >>>> >>>> "... I started to think about how Starbucks processes drink orders. >>>> Starbucks, like most other businesses is primarily interested in >>>> maximizing >>>> throughput of orders. More orders equals more revenue. As a result >>>> they >>>> use >>>> asynchronous processing ..." >>>> >>>> "... we can see that the real world is often asynchronous. Our daily >>>> lives >>>> consists of many coordinated, but asynchronous interactions... >>>> asynchronous >>>> messaging architecture can often be a natural way to model these >>>> types of >>>> interactions... also means that often we can look at daily life to >>>> help >>>> design successful messaging solutions ..."* >>>> >>>> http://www.eaipatterns.com/ramblings/18_starbucks.html >>>> >>>> I hope I'm not too late with this, but I really liked this old >>>> article by >>>> Google's Gregor Hohpe on asynchronous message passing. He wasn't >>>> talking >>>> explicitly about Erlang but the content should resonate >>>> instantaneously >>>> with >>>> Erlang programmers while at the same time being insightful for >>>> non-Erlangers. You might want to reference a few of his well-worded >>>> analogies for evangelising some of the core ideas built into Erlang. >>>> >>>> *Joe Armstrong made a similar compelling argument at the beginning of >>>> his >>>> book. As a relatively new Erlang programmer myself, _THIS_ is the main >>>> line >>>> of argument that convinced me to try out the Erlang environment. >>>> Particularly the implications on concurrency which I had struggled >>>> with >>>> in >>>> other environments (in retrospect, it was because those environments >>>> didn't >>>> have these ideas built-in or actively encouraged -- I've found Erlang >>>> does a >>>> lot of hand-holding to make sure you do the sane thing in order to >>>> avoid >>>> painting yourself into a corner -- something you could emphasise in >>>> your >>>> presentation.) >>>> >>>> - Edmond - >>>> >>>> >>>> On Sat, 04 Dec 2010 05:43:34 +1100, Ryan Zezeski >>>> wrote: >>>> >>>> I've been tasked with giving an introductory level presentation on >>>> Erlang >>>> >>>>> at >>>>> work. The focus is on why you would use Erlang and what does it look >>>>> like >>>>> on the surface level. We have a lot of your standard Java/C#/C >>>>> developers >>>>> and this will act as a potential launching pad to introducing Erlang >>>>> to >>>>> the >>>>> company. There is the potential for a large number of people to >>>>> video >>>>> conference into this possibly spanning several countries so I really >>>>> want >>>>> to >>>>> knock this out of the park. >>>>> >>>>> I'm writing as an inquiry for links to any prior art that I may use >>>>> for >>>>> inspiration or even steal and use as my own. Anything that you think >>>>> might >>>>> be helpful is appreciated, even input on possible approaches. I'd >>>>> really >>>>> like to hear from people who have done this sort of thing before, >>>>> i.e. >>>>> pubic >>>>> speaking on Erlang. >>>>> >>>>> Thanks, >>>>> -Ryan >>>>> >>>>> >>>> >>>> -- >>>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>>> >>>> >> >> -- >> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >> -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From andrey.pampukha@REDACTED Sat Jan 8 19:59:11 2011 From: andrey.pampukha@REDACTED (Andrey Pampukha) Date: Sat, 8 Jan 2011 21:59:11 +0300 Subject: [erlang-questions] re:split not splitting at pipe symbol In-Reply-To: <849E31A4-D4A8-48D8-8D4E-45CA7F1E95C3@gmail.com> References: <849E31A4-D4A8-48D8-8D4E-45CA7F1E95C3@gmail.com> Message-ID: I thought, string:tokens/2 is enough for such purpose :) (main@REDACTED)1> string:tokens("123|456","|"). ["123","456"] Andrey. 2011/1/8, Alain O'Dea : > On 2011-01-08, at 11:02, Dirk Scharff wrote: > >> Hi all! >> >> I'm experiencing a problem with re:split/2 which doesn't seem to be able >> to split at a pipe symbol "|". Is there a reason for this or is this a >> bug? >> >> Reproduce as follows: >> 1> re:split("123|456","|"). > > Pipe is a meaningful symbol in regular expressions. It means 'or'. You need > to escape the pipe: > > re:split("123|456","\\|"). > >> [<<"1">>,<<"2">>,<<"3">>,<<"|">>,<<"4">>,<<"5">>,<<"6">>, >> <<>>] >> 2> re:split("123|456",[124]). >> [<<"1">>,<<"2">>,<<"3">>,<<"|">>,<<"4">>,<<"5">>,<<"6">>, >> <<>>] >> 3> regexp:split("123|456","|"). >> {ok,["123","456"]} >> >> Regards, >> Dirk. >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From alain.odea@REDACTED Sat Jan 8 21:37:26 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Sat, 8 Jan 2011 17:07:26 -0330 Subject: [erlang-questions] re:split not splitting at pipe symbol In-Reply-To: References: <849E31A4-D4A8-48D8-8D4E-45CA7F1E95C3@gmail.com> Message-ID: It can be better to use string:tokens/2 over re:split/2 if you don't need to know the position of a token. The problem is that string:tokens/2 drops empty tokens silently. If you have a record like: GivenName|MiddleName|Surname And some records don't have middle names like: Alain||O'Dea Then string:tokens/2 will give you ["Alain","O'Dea"] This may be okay, but in my experience it complicates code that needs the tokenization since it is hard to identify positional field of a delimited record. On 2011-01-08, at 15:29, Andrey Pampukha wrote: > I thought, string:tokens/2 is enough for such purpose :) > (main@REDACTED)1> string:tokens("123|456","|"). > ["123","456"] > > Andrey. > > 2011/1/8, Alain O'Dea : >> On 2011-01-08, at 11:02, Dirk Scharff wrote: >> >>> Hi all! >>> >>> I'm experiencing a problem with re:split/2 which doesn't seem to be able >>> to split at a pipe symbol "|". Is there a reason for this or is this a >>> bug? >>> >>> Reproduce as follows: >>> 1> re:split("123|456","|"). >> >> Pipe is a meaningful symbol in regular expressions. It means 'or'. You need >> to escape the pipe: >> >> re:split("123|456","\\|"). >> >>> [<<"1">>,<<"2">>,<<"3">>,<<"|">>,<<"4">>,<<"5">>,<<"6">>, >>> <<>>] >>> 2> re:split("123|456",[124]). >>> [<<"1">>,<<"2">>,<<"3">>,<<"|">>,<<"4">>,<<"5">>,<<"6">>, >>> <<>>] >>> 3> regexp:split("123|456","|"). >>> {ok,["123","456"]} >>> >>> Regards, >>> Dirk. >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> From robert.virding@REDACTED Sat Jan 8 22:20:19 2011 From: robert.virding@REDACTED (Robert Virding) Date: Sat, 8 Jan 2011 21:20:19 +0000 (GMT) Subject: [erlang-questions] Erlang Intro/Evangelism Presentation -- Starbucks! In-Reply-To: <511794668.258121294521466549.JavaMail.root@zimbra> Message-ID: <179103885.258141294521619929.JavaMail.root@zimbra> ----- "Edmond Begumisa" wrote: > Probably true. But I am reminded of something very interesting Ulf has > > just pointed me to... > > "The Innovator's Dilemma (C. Christensen) as quoted by Todd A. > Proebsting > ... > > "? companies [languages] that did everything right---were in tune with > > their competition, listened to their customers, and invested > aggressively > in new technologies---*still* lost their market leadership when > confronted > with disruptive changes in technology?" > > http://ll2.ai.mit.edu/talks/proebsting.ppt This is definitely a very interesting book to read (the Innovator's Dilemma). Also interesting is Paul Graham's essay "Beating the Averages", http://www.paulgraham.com/avg.html , which is also about disruptive technologies. Robert -- Robert Virding, Erlang Solutions Ltd. From robert.virding@REDACTED Sat Jan 8 22:24:59 2011 From: robert.virding@REDACTED (Robert Virding) Date: Sat, 8 Jan 2011 21:24:59 +0000 (GMT) Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <992935859.258191294521884974.JavaMail.root@zimbra> Message-ID: <1693153562.258211294521899230.JavaMail.root@zimbra> This whole dscussion reminds a little of the Paul Graham essay "Beating the Averages", http://www.paulgraham.com/avg.html , where he discusses the Blub paradox. Robert ----- "Morten Krogh" wrote: > It is fine with me to claim that often performance doesn't matter, but > > then the imperative languages > of relevance are perl, python, ruby, js etc. Functional languages are > > not easier or more productive than these. > In some sense functional languages just constrain themselves by giving > > up loops and other constructs that are useful. > > But I still think that a lot of what I see in the Erlang world, is > performance motivated. > Why not interface with other languages through the file system then. > It > is very robust and easily debuggable. > Interfacing with some legacy code could be done by running the legacy > > code in a separate OS process, make it write the > result to a file and have Erlang open the file. erlang could do with > sockets and file access. > > Why aren't many of these libraries written in functional languages to > > start with? Why hasn't the higher productivity in > functional languages led to all those libraries being available in > functional languages. Why is it more important for Erlang to > interface > with C, C++, Java etc than with Haskell, Lisp, Ocaml. Same question > for > them. Does Haskell care about interfacing with Erlang to get > libraries? > Functional languages are old, there have been many functional > programmers, functional programmers are often very very skilled. Where > > are all those libraries > that the imperative world doesn't have? > > Henning, do you care to make an exact falsifiable claim about the > multicore scalability of Erlang? What will happen, say in 2020? What > kind of application is it that > Erlang will have but not C or any other imperative language? How > exactly > will we recognize Erlang's victory over all imperative languages in > multicore programming? > Personally, I don't see why C can not have threading libraries with > work > scheduling and migration in a way that is approachable for many > application programmers. > > > Why is the Erlang environment even considered functional? There is a > runtime in C, there is ets, the process dictionary, nifs, lots of > bifs. > The important parts of Erlang don't seem to be the functional syntax > but > more many of the other parts. > > > On 1/5/11 11:45 AM, Robert Raschke wrote: > > On Tue, Jan 4, 2011 at 7:25 PM, Morten Krogh > wrote: > > > >> And then the functional language is just a glorified C library. > > > > C is just a glorified Assembler library. > > > I don't thinks so. Let me explain where I see the difference. It is a > > quantitative difference, not a qualitative one. > > C is much more productive than assembler, and the speed loss in C is > not > that big. > All higher level languages after C, are not that much more productive > > than C, or they lose a lot of performance. C hit a sweet spot. > Some day, there will be a higher level language that displaces C, > sure, > but when? > > You can see the difference in how new algorithms are implemented. Say > > someone invents a new type of data structure, an abc tree, say. > Then C programmers would write abc tree in ansi C and publish it as a > > library. > > Erlang would almost certainly not write it in Erlang. The Erlang > module > abc_tree would just be a wrapper to the C functions. Erlang people > might > even write their > own C program instead of using the C people's library and wrap it. > I would make similar statements for other languages. Python would > just > write a wrapper to C etc. > > Erlang could have done something completely different. Made a small > core > 20 years ago, and then stayed within pure Erlang. The abc_tree would > be > written in pure Erlang. > The thought of writing a wrapper in C could have been silly. > > Someone invents a new cryptographic algottihm. C people would > implement > it in C. Assembler would not be worth it. It is too difficult, > unportable, and the performance gain > is minor. Erlang, python, java?, perl, ruby, haskell etc would write a > > wrapper to a C function. > > The situation is not symmetric. > > Some arbitrary numbers: > > C is 20 time more productive and portable than assembler, C is 2-3 > times > slower. > Erlang is 0.5 - 3 times more productive than C, and say 10 or more > times > slower. > (Erlang speed is higher the more you use the wrappers to C of course, > > but that is forbidden). > > I am not picking on Erlang. The same is true for all other higher > level > languages, and is, in my opinion, the reason that old C is still so > popular. > In the productivity performance trade-off C has been extraordinarily > successful. No later language in my opinion has done that. > > Probably almost no one will agree with me, but anyway. > > Cheers, > > Morten. > > > > > > > > > > > > > > so most code in written in C, and it is very rare to move into > assembler in order to > speed up C. There can be other unrelated assembler programs, but that > is > a different story. > > > > > > > > > > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From jesper.louis.andersen@REDACTED Sun Jan 9 04:23:32 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 9 Jan 2011 04:23:32 +0100 Subject: [erlang-questions] FPGA coming around the corner In-Reply-To: References: <4D267D4A.6060101@gmail.com> Message-ID: On Sat, Jan 8, 2011 at 17:35, Edmond Begumisa wrote: > Two statements stood out for me here... > > 1. "?Established [languages] concentrate on primary differentiator - SPEED?" > > This probably explains what the panic is about in other language camps. The > worry that the speed differentiator is being threatened. That programs might > run slower unless big adaptations are made to programming for SMP. Hmmmm... > maybe this is a sign! I have this notion that I call the computational gaps. Suppose a sequential C program is 10 times faster than a sequential Erlang program. Now, assume near-perfect speedup is possible for the program (it is embarrassingly parallel). The gap is then 10, as if we had 10 cores, we could run as fast as the C program. If we had 100 cores, we would be 10 times faster. If we had 1000 cores, 100 times faster and so on. The CPUs today are much faster than the memory. Even for the L1 cache hits, there is an overhead. Essentially, there is not always a reason to make the CPU run faster - as the extra cycles is used by doing nothing but waiting on main memory. This gives you a credit of essentially free instructions before the next memory load can be satisfied. This is another example of a computational gap. A third one can be had by waiting on disk or network. While we are waiting and have nothing else to do, we could use the cycles for something else. Another gap. The speed argument is disrupted by gaps. In the SMP case, the disruption happens because of parallelism. In the second and third cases, the gap occurs and disrupts due to concurrency as well: If we can schedule another operation while waiting, the final throughput of our work will be a lot better. Some of the gaps are driven mostly by hardware (e.g. SMP) and some are driven mostly by the complexity needs of software (the concurrency need). > 2. "?My criteria: [destructive] technology must > ? - Have disadvantages > ? - Be mostly ignored by recent PLD and POPL conferences* > ? - Alleviate real problems?" It looks like POPL has mostly ignored the problem for some years, so we are set for that one. We are also set for the disadvantages: Erlang is not known for its blazing numerical computational speed when a brute force method is the fastest. Interestingly it often does not matter. Partially due to the gaps I presume. -- J. From jesper.louis.andersen@REDACTED Sun Jan 9 04:29:09 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 9 Jan 2011 04:29:09 +0100 Subject: [erlang-questions] re:split not splitting at pipe symbol In-Reply-To: References: <849E31A4-D4A8-48D8-8D4E-45CA7F1E95C3@gmail.com> Message-ID: On Sat, Jan 8, 2011 at 21:37, Alain O'Dea wrote: > It can be better to use string:tokens/2 over re:split/2 if you don't need to know the position of a token. ?The problem is that string:tokens/2 drops empty tokens silently. > > If you have a record like: > GivenName|MiddleName|Surname > > And some records don't have middle names like: > Alain||O'Dea > > Then string:tokens/2 will give you ["Alain","O'Dea"] Perhaps it would be worth looking at the standard ml basis library in this respect: - String.tokens (fn c => c = #"|") "Alain||O'Dea"; val it = ["Alain","O'Dea"] : string list - String.fields (fn c => c = #"|") "Alain||O'Dea"; val it = ["Alain","","O'Dea"] : string list It is worth having both variants I think. The patch shouldn't be hard to provide against the stdlib. And I am a firm believer that a regular expression is almost always the wrong thing to do. They are nice for lexers, but keep them there :) -- J. From michael.eugene.turner@REDACTED Sun Jan 9 05:34:16 2011 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sun, 9 Jan 2011 13:34:16 +0900 Subject: [erlang-questions] Erlang Intro/Evangelism Presentation -- Starbucks! In-Reply-To: <179103885.258141294521619929.JavaMail.root@zimbra> References: <511794668.258121294521466549.JavaMail.root@zimbra> <179103885.258141294521619929.JavaMail.root@zimbra> Message-ID: Paul Graham's Beating the Averages can always bear another re-reading (notwithstanding it's various exaggerations -- I'm sorry, Paul, but Cobol IS closer to Python than to assembly language.) You could probably substitute "Erlang" for "Lisp" everywhere in the essay, without doing violence to an excellent point. The problem is, Graham is addressing a special case: the startup context, when there is (almost by definition, from the VC point of view) a market window for a new class of application. The overwhelming bulk of production code in the world does not match that description. Faced with the problem of maintaining Yahoo! Stores as production code, Yahoo! rewrote it in C++ and Perl. Paul Graham opines that they did this only out of Lispophobia, but I really wonder: if he'd been golden-handcuffed to Yahoo! and put in a position to hire underlings, he'd be in an awkward position, wouldn't he? "Gee, I have to hire quite a few Lisp hackers who are both (a) more or less in my league, and (b) all hot to work at Yahoo! on something like Viaweb, rather than, say, some interesting problem in computational linguistics. And, uh-oh, in the meantime, I'm getting a million resumes from people who have written a few hundred lines of bad Autolisp." I think the pickings would be slim. In an interesting thread of commentary on Paul Graham's essay, none other than Dan Weinreb, one of the preeminent Lisp hackers of our time (or any time) writes: "... it would take a lot, or require special circumstances, to persuade me to choose Lisp for a major software project, if I were in charge of making the choice." http://discuss.fogcreek.com/joelonsoftware/default.asp?cmd=show&ixPost=31402 -michael turner On Sun, Jan 9, 2011 at 6:20 AM, Robert Virding < robert.virding@REDACTED> wrote: > ----- "Edmond Begumisa" wrote: > > > Probably true. But I am reminded of something very interesting Ulf has > > > > just pointed me to... > > > > "The Innovator's Dilemma (C. Christensen) as quoted by Todd A. > > Proebsting > > ... > > > > "? companies [languages] that did everything right---were in tune with > > > > their competition, listened to their customers, and invested > > aggressively > > in new technologies---*still* lost their market leadership when > > confronted > > with disruptive changes in technology?" > > > > http://ll2.ai.mit.edu/talks/proebsting.ppt > > This is definitely a very interesting book to read (the Innovator's > Dilemma). Also interesting is Paul Graham's essay "Beating the Averages", > http://www.paulgraham.com/avg.html , which is also about disruptive > technologies. > > Robert > > -- > Robert Virding, Erlang Solutions Ltd. > From alessandro.sivieri@REDACTED Sun Jan 9 08:56:09 2011 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Sun, 9 Jan 2011 08:56:09 +0100 Subject: UML or other modeling tools Message-ID: Hi all, has anyone of you ever used any modeling tool for describing a system written in Erlang (or any other functional language)? I have to describe the architecture of an application, in terms of processes structure/tree and/or modules, and I was wondering if there is some UML diagram that could be used, or any "standard" format for doing it; I have already created several sequence diagrams, and usually in my other projects I have used (more or less detailed) diagram classes, but for functional languages I don't know precisely what tools there exist... -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From dirk.scharff@REDACTED Sun Jan 9 10:00:39 2011 From: dirk.scharff@REDACTED (Dirk Scharff) Date: Sun, 9 Jan 2011 10:00:39 +0100 Subject: [erlang-questions] UML or other modeling tools In-Reply-To: References: Message-ID: Hi Alessandro, I haven't tried to do so, but I think you could use UML component diagrams ( http://en.wikipedia.org/wiki/Component_diagram) to describe the modules of your system. If you have to describe an algorithm in detail you could use a flow chart. Depending on the situation an UML state machine diagram (if you use finite state machines in your system) or activity diagram could also be useful. Regards, Dirk. On Sun, Jan 9, 2011 at 8:56 AM, Alessandro Sivieri < alessandro.sivieri@REDACTED> wrote: > Hi all, > > has anyone of you ever used any modeling tool for describing a system > written in Erlang (or any other functional language)? > I have to describe the architecture of an application, in terms of > processes > structure/tree and/or modules, and I was wondering if there is some UML > diagram that could be used, or any "standard" format for doing it; I have > already created several sequence diagrams, and usually in my other projects > I have used (more or less detailed) diagram classes, but for functional > languages I don't know precisely what tools there exist... > > -- > Sivieri Alessandro > alessandro.sivieri@REDACTED > http://www.chimera-bellerofonte.eu/ > http://www.poul.org/ > From ulf.wiger@REDACTED Sun Jan 9 14:21:53 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 9 Jan 2011 14:21:53 +0100 Subject: [erlang-questions] FPGA coming around the corner In-Reply-To: References: <4D267D4A.6060101@gmail.com> Message-ID: <7460E1E3-89B5-40A4-9508-CEDFF72FE32D@erlang-solutions.com> On 9 Jan 2011, at 04:23, Jesper Louis Andersen wrote: > I have this notion that I call the computational gaps. Suppose a > sequential C program is 10 times faster than a sequential Erlang > program. Now, assume near-perfect speedup is possible for the program > (it is embarrassingly parallel). The gap is then 10, as if we had 10 > cores, we could run as fast as the C program. If we had 100 cores, we > would be 10 times faster. If we had 1000 cores, 100 times faster and > so on. Of course, if the problem is embarrassingly parallel, it will not be particularly hard to use something like OpenMP to get good speedups for the C program as well, ;-), but I understand that that wasn't your point. The thing that often gets lost in these discussions is that for sufficiently "interesting" applications, the things that matter most for performance are often the things we must do to achieve stability over time, as requirements evolve; to achieve robustness and debuggability; have a fighting chance to stay on top of memory management and synchronization issues, etc. I realize YMMV (greatly). The trick is to recognize what your main challenges will be and pick a technology and strategy that will serve you all the way (I was going to say "to the finish line", but "through the product lifecycle" is probably more accurate). BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From ulf.wiger@REDACTED Sun Jan 9 15:06:19 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 9 Jan 2011 15:06:19 +0100 Subject: [erlang-questions] UML or other modeling tools In-Reply-To: References: Message-ID: <2E275C30-C1EE-453B-8CE2-1C934F17A50F@erlang-solutions.com> I am quite sceptical towards using UML for modeling Erlang programs, for a number of reasons. - I mainly subscribe to Fred Brooks' "The Flow Chart Curse" in "The Mythical Man-Month": visual modeling provides very little extra information when used to illustrate a very high-level programming language. In the case of UML and Erlang, Erlang is at roughly the same abstraction level as the various versions of Executable UML. - The things most interesting to model in an Erlang program are the state machines and the interaction between processes, but UML state machines have very complex - and most importantly, *different* - semantics, making it quite probable that the design becomes subject to compromises forced by the modeling tool, rather than reflecting the power of the programming language. - The main purpose of a standard notation is that everyone should be able to agree on what it means. I assert that very few people can claim to understand the subtleties of UML's state machine semantics, and even if they do, so much the worse, since Erlang's semantics are different. Better to choose a simple notation and explain what it means in the relation to Erlang's building blocks. Taking a ubiquitous symbol and defining it to mean something different is counter-productive. - The FMC (http://fmc-modeling.org/) notation seems to complement Erlang fairly well, describing roughly the things that are most interesting to describe visually in an Erlang program. BR, Ulf W On 9 Jan 2011, at 08:56, Alessandro Sivieri wrote: > Hi all, > > has anyone of you ever used any modeling tool for describing a system > written in Erlang (or any other functional language)? > I have to describe the architecture of an application, in terms of processes > structure/tree and/or modules, and I was wondering if there is some UML > diagram that could be used, or any "standard" format for doing it; I have > already created several sequence diagrams, and usually in my other projects > I have used (more or less detailed) diagram classes, but for functional > languages I don't know precisely what tools there exist... > > -- > Sivieri Alessandro > alessandro.sivieri@REDACTED > http://www.chimera-bellerofonte.eu/ > http://www.poul.org/ Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From ulf.wiger@REDACTED Sun Jan 9 15:25:16 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sun, 9 Jan 2011 15:25:16 +0100 Subject: [erlang-questions] FPGA coming around the corner In-Reply-To: References: <4D267D4A.6060101@gmail.com> Message-ID: <12118B45-1D13-4E58-87ED-3211D60A8D04@erlang-solutions.com> On 7 Jan 2011, at 07:44, Edmond Begumisa wrote: > But that's precisely what these advocates are warning their audiences about! That the *legacy apps and code will have to be re-written* in order to be parallelised for multi-core. That's what the panic is about. That these programs and libraries will need to be carefully re-coded using lots of threads, concurrent data structures and unfamiliar paradigms like "transactional boosting." > > The Herlihy's are effectively saying: "You will need to re-write you're apps and libs in order for your users not to complain that things are slower, and here's a bunch of tools you can use that we *hope* will make it a little easier. But it will still be pretty hard." Actually achieving scalability on multicore is only one of the tough challenges facing legacy software developers; *debugging* shared- memory imperative applications on multicore is essentially a research area. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From jameschurchman@REDACTED Sun Jan 9 21:44:39 2011 From: jameschurchman@REDACTED (James Churchman) Date: Sun, 9 Jan 2011 20:44:39 +0000 Subject: [erlang-questions] FPGA coming around the corner In-Reply-To: <12118B45-1D13-4E58-87ED-3211D60A8D04@erlang-solutions.com> References: <4D267D4A.6060101@gmail.com> <12118B45-1D13-4E58-87ED-3211D60A8D04@erlang-solutions.com> Message-ID: How much of the average erlang code spends its cpu time siting in BIF's and how much is the erlang code its self? (and does this change significantly with HiPE??) Im assuming its not a significant amount of time, but if it is should be posible to produce a set of some of the bifs for an FPGA Been a few years now but had to do a moderate amount of VHDL for FPGA's at uni, tho i hear good things about Bluespec, its bassed on haskell and is also a hardware description language james On 9 January 2011 14:25, Ulf Wiger wrote: > > On 7 Jan 2011, at 07:44, Edmond Begumisa wrote: > > > But that's precisely what these advocates are warning their audiences > about! That the *legacy apps and code will have to be re-written* in order > to be parallelised for multi-core. That's what the panic is about. That > these programs and libraries will need to be carefully re-coded using lots > of threads, concurrent data structures and unfamiliar paradigms like > "transactional boosting." > > > > The Herlihy's are effectively saying: "You will need to re-write you're > apps and libs in order for your users not to complain that things are > slower, and here's a bunch of tools you can use that we *hope* will make it > a little easier. But it will still be pretty hard." > > > Actually achieving scalability on multicore is only one of the tough > challenges facing legacy software developers; *debugging* shared- > memory imperative applications on multicore is essentially a research > area. > > BR, > Ulf W > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From alain.odea@REDACTED Sun Jan 9 22:30:29 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Sun, 9 Jan 2011 18:00:29 -0330 Subject: [erlang-questions] UML or other modeling tools In-Reply-To: <2E275C30-C1EE-453B-8CE2-1C934F17A50F@erlang-solutions.com> References: <2E275C30-C1EE-453B-8CE2-1C934F17A50F@erlang-solutions.com> Message-ID: <8483A9E6-5AB9-4A1E-9920-A07B8A305460@gmail.com> On 2011-01-09, at 10:36, Ulf Wiger wrote: > > I am quite sceptical towards using UML for modeling Erlang > programs, for a number of reasons. > > - I mainly subscribe to Fred Brooks' "The Flow Chart Curse" in "The Mythical > Man-Month": visual modeling provides very little extra information when > used to illustrate a very high-level programming language. In the case of > UML and Erlang, Erlang is at roughly the same abstraction level as the > various versions of Executable UML. > > - The things most interesting to model in an Erlang program are the > state machines and the interaction between processes, but UML > state machines have very complex - and most importantly, *different* > - semantics, making it quite probable that the design becomes subject > to compromises forced by the modeling tool, rather than reflecting the > power of the programming language. > > - The main purpose of a standard notation is that everyone should be > able to agree on what it means. I assert that very few people can claim > to understand the subtleties of UML's state machine semantics, and > even if they do, so much the worse, since Erlang's semantics are > different. Better to choose a simple notation and explain what it means > in the relation to Erlang's building blocks. Taking a ubiquitous symbol > and defining it to mean something different is counter-productive. > > - The FMC (http://fmc-modeling.org/) notation seems to complement > Erlang fairly well, describing roughly the things that are most interesting > to describe visually in an Erlang program. Hi Ulf: What success stories can you share about using FMC? I have been looking for a lightweight and comprehensive form of design documentation. I like the idea of using CRC/MRC (Class/Module Responsibilities and Collaborators) cards, but they have limitations. I imagine CRC/MRC cards effectively describe the parts of the system, but cannot effectively describe how the entire system of parts moves together. Is FMC worth serious investigation? Thank you, Alain > > BR, > Ulf W > > On 9 Jan 2011, at 08:56, Alessandro Sivieri wrote: > >> Hi all, >> >> has anyone of you ever used any modeling tool for describing a system >> written in Erlang (or any other functional language)? >> I have to describe the architecture of an application, in terms of processes >> structure/tree and/or modules, and I was wondering if there is some UML >> diagram that could be used, or any "standard" format for doing it; I have >> already created several sequence diagrams, and usually in my other projects >> I have used (more or less detailed) diagram classes, but for functional >> languages I don't know precisely what tools there exist... >> >> -- >> Sivieri Alessandro >> alessandro.sivieri@REDACTED >> http://www.chimera-bellerofonte.eu/ >> http://www.poul.org/ > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From banibrata.dutta@REDACTED Mon Jan 10 05:38:11 2011 From: banibrata.dutta@REDACTED (Banibrata Dutta) Date: Mon, 10 Jan 2011 10:08:11 +0530 Subject: [erlang-questions] 1000 cores on a chip In-Reply-To: References: Message-ID: Thanks, Jesper. That was very informative and enlightening. Given your description, it clearly looks like Erlang (on general purpose multi-core CPUs) + specific algos running on CUDA(/-like) architecture can create a very powerful solution mix. BTW, an OT info as many on this list might have noted is that Amazon has started offering CUDA GPU processing power as part of it's AWS offering. On Sat, Jan 8, 2011 at 7:12 AM, Jesper Louis Andersen < jesper.louis.andersen@REDACTED> wrote: > On Fri, Jan 7, 2011 at 04:30, Banibrata Dutta > wrote: > > > However, those many cores in GPU world isn't new. CUDA has shown that to > us > > for a while. People are making use of GPU for general computation > > increasingly, and a while back had come accross a startup that was using > the > > extreme parallelism (in a box) by CUDA for video analytics and video > > encoding/decoding. Not sure how well FPGA's really fare w.r.t. power > > consumption in such setups, but the CUDA GPU units do consume a good bit > of > > power.. and generate good bit of heat. > > The most important difference is that GPUs tend to be SIMD > architectures with almost no branching instructions. This 1000-core > beast is much closer to a thousand cores each running their own > individual program - making them a better fit for Erlang as it is now. > Suppose you have > > case Foo of > true -> do_x(); > false -> do_y() > end > > In (most) GPUs this is compiled into: > > case Foo of > true -> do_x(); > false -> nop > end, > case Foo of > true -> nop; > false -> do_y() > end > > So essentially the code runs twice: Once for the true branch and once > for the false branch. Why? Because in a SIMD architecture all cores > are locked to execute the same instruction, so you mask out the > operation with a 'nop' for one branch at a time. The more complicated > your branching tree is, the more expensive it is to do this. Many of > the problems where you see GPUs smite the competition is where you > have relatively few branches and can process data in a streamed > fashion, more or less. Video decoding is an excellent example. > > Modern GPUs can load more than one program so only part of the GPU has > to run locked to each other, but it is by no means easy to carry out > and even harder to make automatic by compilation. Furthermore modern > GPUs will to a greater and greater extent accept computational error. > If the output is to a screen a single pixel error for 1/60th of a > second isn't noticeable. It has been speculated that you can still use > the GPU for more exact computation in the future though: if you can > bound the error by some measurement, you can run the computation and > then fix up smaller errors on the CPU afterwards. I've seen this trick > used in another setting: Approximate integer calculations on FP > hardware (SSE3) and then fixup the cases where computational error > might have crept in by using the integer unit for this. > > > The key difference between Erlang and SIMD is that while SIMD leans > itself to parallel computation, it does not lean itself to concurrency > that well. In Erlangs case we would like parallelism and concurrency > at the same time which to some extent is easier pulled off on MIMD > architectures. > > In other words: SIMD is very cool (and blazingly fast!) for some > problems. Yet, it probably does not fit into the forte of Erlang that > much and it would take considerable machinery to add it while it is > not clear any benefit would be provided by the addition. > > > > -- > J. > -- regards, Banibrata http://www.linkedin.com/in/bdutta http://twitter.com/edgeliving From jesper.louis.andersen@REDACTED Mon Jan 10 05:48:51 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 10 Jan 2011 05:48:51 +0100 Subject: [erlang-questions] 1000 cores on a chip In-Reply-To: References: Message-ID: On Mon, Jan 10, 2011 at 05:38, Banibrata Dutta wrote: > Thanks, Jesper. That was very informative and enlightening. > > Given your description, it clearly looks like Erlang (on general purpose > multi-core CPUs) + specific algos running on CUDA(/-like) architecture can > create a very powerful solution mix. I think so too. If you want to use GPUs *now*, going for CUDA through a port or a NIF is the right way. My vision is that we can create a programming language (essentially domain-specific) for computations on GPUs. Yet, the GPUs are currently too quickly changing so it is not yet a stable enough platform. -- J. From dangud@REDACTED Mon Jan 10 08:14:26 2011 From: dangud@REDACTED (Dan Gudmundsson) Date: Mon, 10 Jan 2011 08:14:26 +0100 Subject: [erlang-questions] 1000 cores on a chip In-Reply-To: References: Message-ID: Why CUDA? Tony's OpenCL wrapper have worked nice for me? OpenCL is a standard that works on Intel CPU's and ATI cards (as well as Nvidia cards). /Dan PS: https://github.com/tonyrog/cl On Mon, Jan 10, 2011 at 5:48 AM, Jesper Louis Andersen wrote: > On Mon, Jan 10, 2011 at 05:38, Banibrata Dutta > wrote: >> Thanks, Jesper. That was very informative and enlightening. >> >> Given your description, it clearly looks like Erlang (on general purpose >> multi-core CPUs) + specific algos running on CUDA(/-like) architecture can >> create a very powerful solution mix. > > I think so too. If you want to use GPUs *now*, going for CUDA through > a port or a NIF is the right way. My vision is that we can create a > programming language (essentially domain-specific) for computations on > GPUs. Yet, the GPUs are currently too quickly changing so it is not > yet a stable enough platform. > > -- > J. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From ulf.wiger@REDACTED Mon Jan 10 10:05:19 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Mon, 10 Jan 2011 10:05:19 +0100 Subject: [erlang-questions] UML or other modeling tools In-Reply-To: <8483A9E6-5AB9-4A1E-9920-A07B8A305460@gmail.com> References: <2E275C30-C1EE-453B-8CE2-1C934F17A50F@erlang-solutions.com> <8483A9E6-5AB9-4A1E-9920-A07B8A305460@gmail.com> Message-ID: On 9 Jan 2011, at 22:30, Alain O'Dea wrote: > On 2011-01-09, at 10:36, Ulf Wiger wrote: > >> - The FMC (http://fmc-modeling.org/) notation seems to complement >> Erlang fairly well, describing roughly the things that are most interesting >> to describe visually in an Erlang program. > > Hi Ulf: > > What success stories can you share about using FMC? None, actually. It would be interesting to hear from someone who's actually tried it. :) The most ambitious story around FMC is The Apache Modelling Project http://www.fmc-modeling.org/download/projects/apache/the_apache_modelling_project.pdf as far as I know. Last time I looked at it, the state of FMC-capable editors was a bit troubling. MS Visio supposedly supports FMC notation, as well as OpenOffice. Oryx is an Open Source project and an online editor, but it failed to convert me, at least. http://bpt.hpi.uni-potsdam.de/Oryx/FMC BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From rtrlists@REDACTED Mon Jan 10 11:41:07 2011 From: rtrlists@REDACTED (Robert Raschke) Date: Mon, 10 Jan 2011 10:41:07 +0000 Subject: [erlang-questions] UML or other modeling tools In-Reply-To: References: Message-ID: Hi Allessandro, On Sun, Jan 9, 2011 at 7:56 AM, Alessandro Sivieri < alessandro.sivieri@REDACTED> wrote: > has anyone of you ever used any modeling tool for describing a system > written in Erlang (or any other functional language)? > I have to describe the architecture of an application, in terms of > processes > structure/tree and/or modules, and I was wondering if there is some UML > diagram that could be used, or any "standard" format for doing it; I have > already created several sequence diagrams, and usually in my other projects > I have used (more or less detailed) diagram classes, but for functional > languages I don't know precisely what tools there exist... > > I write business level applications that bridge between an ITSM (IT Service Management) tool and other ITSM tools and system monitoring tools. For this we use an Erlang/RabbitMQ framework that runs Lua business logic snippets based on the state of the linkage between the two systems. So, for a bridged "thing" (which exists in two different guises on either side of the bridge), we manage its state and what kind of interactions/messages in each direction are allowed, how those messages get transformed (i.e., what effect they have on the other system) and what state changes they trigger. For this, I personally, use UML state charts. And that works very well so far. The trick is to keep the level of detail at an acceptable level. I am not interested in implementing the logic in a state diagram. All I want is to capture the salient points and manage the overall complexity of the logic. While I ship these diagrams as part of the ongoing (requirements) documentation, I don't think they've added enormously to a customers understanding. But they do help me rather a lot when discussing details with them. I find it is important to realise that modelling tools are there to model your proposed solution, not implement it. They are very valuable during design of a complex system. They make mistakes in your design a bit cheaper to handle, because you may not have gone and implemented all of it yet. This makes changing your design a little bit less risky. Robby From paolo.negri@REDACTED Mon Jan 10 11:56:27 2011 From: paolo.negri@REDACTED (Paolo Negri) Date: Mon, 10 Jan 2011 11:56:27 +0100 Subject: message copying overhead atoms vs binaries Message-ID: Dear list, I'm trying to understand what approach of building values contained in messages to pass across processes would be more efficient (in terms of memory and cpu) and if the difference is significant. I could build my message values in two ways a) [{color, red}, {shape, circle}] b) [{color, <<"red">>}, {shape, <<"circle">>}] The set of values is strictly limited (in the order of hundreds and not over 1000) and these values are constantly used by the system so the fact that atoms would be permanently allocated is not a problem in this case. The actual lists are longer and can vary in length between 5 to 10 elements. The processes will exchange thousands of these messages each second and this is the reason why I'm asking this question. I'd like to know which form is cheaper in term of copy operation in both cases internally on a single node or across nodes hosted on different physical machines. Thanks for your help, Paolo -- Engineering http://www.wooga.com | phone +49-30-8962 5058? | fax +49-30-8964 9064 wooga GmbH | Saarbruecker Str. 38 | 10405 Berlin | Germany Sitz der Gesellschaft: Berlin; HRB 117846 B Registergericht Berlin-Charlottenburg Geschaeftsfuehrung: Jens Begemann, Philipp Moeser From bob@REDACTED Mon Jan 10 12:12:01 2011 From: bob@REDACTED (Bob Ippolito) Date: Mon, 10 Jan 2011 19:12:01 +0800 Subject: [erlang-questions] message copying overhead atoms vs binaries In-Reply-To: References: Message-ID: On Mon, Jan 10, 2011 at 6:56 PM, Paolo Negri wrote: > Dear list, > > I'm trying to understand what approach of building values contained in > messages to pass across processes would be more efficient (in terms of > memory and cpu) and if the difference is significant. > I could build my message values in two ways > > a) [{color, red}, {shape, circle}] > > b) [{color, <<"red">>}, {shape, <<"circle">>}] > > The set of values is strictly limited (in the order of hundreds and > not over 1000) and these values are constantly used by the system so > the fact that atoms would be permanently allocated is not a problem in > this case. > > The actual lists are longer and can vary in length between 5 to 10 elements. > The processes will exchange thousands of these messages each second > and this is the reason why I'm asking this question. > > I'd like to know which form is cheaper in term of copy operation in > both cases internally on a single node or across nodes hosted on > different physical machines. Atoms are cheaper. They are always 1 word on the heap and the rest shared in the atom table. Even when binaries are shared, they are larger than this [1]. In your case the binaries would not be shared anyway, they are smaller than 64 bytes so they are considered heap binaries [2]. The external term format also has optimizations that allow for an atom table to compactly represent them by reference instead of value, where other types to not have this kind of optimization [3]. [1] http://www.erlang.org/doc/efficiency_guide/advanced.html [2] http://www.erlang.org/doc/efficiency_guide/binaryhandling.html [3] http://www.erlang.org/doc/apps/erts/erl_ext_dist.html -bob From paolo.negri@REDACTED Mon Jan 10 12:31:33 2011 From: paolo.negri@REDACTED (Paolo Negri) Date: Mon, 10 Jan 2011 12:31:33 +0100 Subject: [erlang-questions] message copying overhead atoms vs binaries In-Reply-To: References: Message-ID: Hi Bob, Thanks for taking the time to answer and pointing to the resources. I would also like to to say thanks more in general to the whole list since as a newcomer being part of this list so far has been very helpful and interesting. Thanks, Paolo On Mon, Jan 10, 2011 at 12:12 PM, Bob Ippolito wrote: > On Mon, Jan 10, 2011 at 6:56 PM, Paolo Negri wrote: >> Dear list, >> >> I'm trying to understand what approach of building values contained in >> messages to pass across processes would be more efficient (in terms of >> memory and cpu) and if the difference is significant. >> I could build my message values in two ways >> >> a) [{color, red}, {shape, circle}] >> >> b) [{color, <<"red">>}, {shape, <<"circle">>}] >> >> The set of values is strictly limited (in the order of hundreds and >> not over 1000) and these values are constantly used by the system so >> the fact that atoms would be permanently allocated is not a problem in >> this case. >> >> The actual lists are longer and can vary in length between 5 to 10 elements. >> The processes will exchange thousands of these messages each second >> and this is the reason why I'm asking this question. >> >> I'd like to know which form is cheaper in term of copy operation in >> both cases internally on a single node or across nodes hosted on >> different physical machines. > > Atoms are cheaper. They are always 1 word on the heap and the rest > shared in the atom table. Even when binaries are shared, they are > larger than this [1]. In your case the binaries would not be shared > anyway, they are smaller than 64 bytes so they are considered heap > binaries [2]. The external term format also has optimizations that > allow for an atom table to compactly represent them by reference > instead of value, where other types to not have this kind of > optimization [3]. > > [1] http://www.erlang.org/doc/efficiency_guide/advanced.html > [2] http://www.erlang.org/doc/efficiency_guide/binaryhandling.html > [3] http://www.erlang.org/doc/apps/erts/erl_ext_dist.html > > -bob > -- Engineering http://www.wooga.com | phone +49-30-8962 5058? | fax +49-30-8964 9064 wooga GmbH | Saarbruecker Str. 38 | 10405 Berlin | Germany Sitz der Gesellschaft: Berlin; HRB 117846 B Registergericht Berlin-Charlottenburg Geschaeftsfuehrung: Jens Begemann, Philipp Moeser From alain.odea@REDACTED Mon Jan 10 15:11:25 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Mon, 10 Jan 2011 10:41:25 -0330 Subject: [erlang-questions] UML or other modeling tools In-Reply-To: References: <2E275C30-C1EE-453B-8CE2-1C934F17A50F@erlang-solutions.com> <8483A9E6-5AB9-4A1E-9920-A07B8A305460@gmail.com> Message-ID: On 2011-01-10, at 5:35, Ulf Wiger wrote: > > On 9 Jan 2011, at 22:30, Alain O'Dea wrote: > >> On 2011-01-09, at 10:36, Ulf Wiger wrote: >> >>> - The FMC (http://fmc-modeling.org/) notation seems to complement >>> Erlang fairly well, describing roughly the things that are most interesting >>> to describe visually in an Erlang program. >> >> Hi Ulf: >> >> What success stories can you share about using FMC? > > None, actually. It would be interesting to hear from someone who's > actually tried it. :) > > The most ambitious story around FMC is The Apache Modelling Project > > http://www.fmc-modeling.org/download/projects/apache/the_apache_modelling_project.pdf > > as far as I know. > > Last time I looked at it, the state of FMC-capable editors was a bit > troubling. MS Visio supposedly supports FMC notation, as well as > OpenOffice. Oryx is an Open Source project and an online editor, > but it failed to convert me, at least. > > http://bpt.hpi.uni-potsdam.de/Oryx/FMC > > BR, > Ulf W > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com FMC looks easy enough to draw. One key for a modeling system for me is the ease with which it can be drawn on a flip-chart or whiteboard. Tools that understand the semantics are less important to me. I imagine that OmniGraffle/Visio would work reasonably since they support the magnets and connectors elements needed to make the editing comfortable. A more purposed editor would be interesting, but in my experience they limit the expression and annotation too much to be generally useful. From alessandro.sivieri@REDACTED Mon Jan 10 15:10:08 2011 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Mon, 10 Jan 2011 15:10:08 +0100 Subject: [erlang-questions] UML or other modeling tools In-Reply-To: References: <2E275C30-C1EE-453B-8CE2-1C934F17A50F@erlang-solutions.com> <8483A9E6-5AB9-4A1E-9920-A07B8A305460@gmail.com> Message-ID: In the end I'm using FMC with the OOo template, it looks good for drawing a description of my server application. -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From paolo.negri@REDACTED Mon Jan 10 16:09:35 2011 From: paolo.negri@REDACTED (Paolo Negri) Date: Mon, 10 Jan 2011 16:09:35 +0100 Subject: [erlang-questions] message copying overhead atoms vs binaries In-Reply-To: References: Message-ID: On Mon, Jan 10, 2011 at 12:12 PM, Bob Ippolito wrote: > On Mon, Jan 10, 2011 at 6:56 PM, Paolo Negri wrote: >> Dear list, >> >> I'm trying to understand what approach of building values contained in >> messages to pass across processes would be more efficient (in terms of >> memory and cpu) and if the difference is significant. >> I could build my message values in two ways >> >> a) [{color, red}, {shape, circle}] >> >> b) [{color, <<"red">>}, {shape, <<"circle">>}] >> >> The set of values is strictly limited (in the order of hundreds and >> not over 1000) and these values are constantly used by the system so >> the fact that atoms would be permanently allocated is not a problem in >> this case. >> >> The actual lists are longer and can vary in length between 5 to 10 elements. >> The processes will exchange thousands of these messages each second >> and this is the reason why I'm asking this question. >> >> I'd like to know which form is cheaper in term of copy operation in >> both cases internally on a single node or across nodes hosted on >> different physical machines. > > Atoms are cheaper. They are always 1 word on the heap and the rest > shared in the atom table. Even when binaries are shared, they are > larger than this [1]. In your case the binaries would not be shared > anyway, they are smaller than 64 bytes so they are considered heap > binaries [2]. The external term format also has optimizations that > allow for an atom table to compactly represent them by reference > instead of value, where other types to not have this kind of > optimization [3]. There's a sentence in [2] that I can't fully understand. "Heap binaries are small binaries, up to 64 bytes, that are stored directly on the process heap. They will be copied when the process is garbage collected and when they are sent as a message. They don't require any special handling by the garbage collector." Specifically I'm confused about why (and where) heap binaries will be copied when the process is garbage collected. Thanks, Paolo > > [1] http://www.erlang.org/doc/efficiency_guide/advanced.html > [2] http://www.erlang.org/doc/efficiency_guide/binaryhandling.html > [3] http://www.erlang.org/doc/apps/erts/erl_ext_dist.html > > -bob > From bgustavsson@REDACTED Mon Jan 10 16:37:35 2011 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Mon, 10 Jan 2011 16:37:35 +0100 Subject: [erlang-questions] "Old inliner"? In-Reply-To: <000901cba062$11c42ee0$354c8ca0$@com> References: <000901cba062$11c42ee0$354c8ca0$@com> Message-ID: On Mon, Dec 20, 2010 at 5:22 PM, David Mercer wrote: > > I am explicitly inlining call/3, but a web search on what the "old inliner" > is only seems to find readme's with information about it being fixed. What > is the "old inliner"? Are there other inliners (like a new one)? > The old inliner is now only used for doing explicit inlining. The old inliner only does inlining without any optimizations. The new inliner does implicit inling and also does optimizations while inlining. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From dsc@REDACTED Mon Jan 10 17:02:19 2011 From: dsc@REDACTED (Dave Challis) Date: Mon, 10 Jan 2011 16:02:19 +0000 Subject: Running a regular expression on each line of a file References: <4D2B2D8B.5040907@ecs.soton.ac.uk> Message-ID: I've got a file containing lines of the format " ", which I'm trying to pipe to an erlang script, and pull apart with a regular expression. The script is based on http://www.erlang.org/faq/how_do_i.html#id53404 . It works fine for small input files, but is crashing for very large input files (e.g. one containing 17 million lines of text). The crash dump file that is generated indicates that something is running away somewhere: =memory total: 125063616 processes: 8918232 processes_used: 8902304 system: 116145384 I'm fairly new to erlang, so may well have structured the code for this incorrectly. Here's the full module which is causing the problems: -module(test_parse). -export([parse/0]). parse() -> {ok, Re} = re:compile("<([^>]+)> <([^>]+)> <([^>]+)>"), parse(Re). parse(Re) -> case io:get_chars('', 8192) of eof -> init:stop(); Text -> Result = re:run(Text, Re, [{capture, all_but_first, list}]), case Result of {match, Captured} -> io:format("~p ~p ~p~n", Captured) end end, parse(Re). The script is then run (on the command line, ubuntu linux) using: cat bigfile.txt | erl -noshell -s test_parse parse Any pointers on what I'm doing wrong would be appreciated! Thanks, -- Dave Challis dsc@REDACTED From toby@REDACTED Mon Jan 10 18:15:07 2011 From: toby@REDACTED (Toby Thain) Date: Mon, 10 Jan 2011 14:15:07 -0300 Subject: [erlang-questions] FPGA coming around the corner In-Reply-To: References: <4D267D4A.6060101@gmail.com> <12118B45-1D13-4E58-87ED-3211D60A8D04@erlang-solutions.com> Message-ID: <4D2B3E9B.6010208@telegraphics.com.au> On 11-01-09 5:44 PM, James Churchman wrote: > ... > Been a few years now but had to do a moderate amount of VHDL for FPGA's at > uni, tho i hear good things about Bluespec, its bassed on haskell and is > also a hardware description language There is also Lava, Haskell-based: http://www.raintown.org/lava/ --Toby > > > james > From jesper.louis.andersen@REDACTED Mon Jan 10 18:42:02 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Mon, 10 Jan 2011 18:42:02 +0100 Subject: [erlang-questions] Running a regular expression on each line of a file In-Reply-To: References: <4D2B2D8B.5040907@ecs.soton.ac.uk> Message-ID: On Mon, Jan 10, 2011 at 17:02, Dave Challis wrote: > parse(Re) -> > ? ?case io:get_chars('', 8192) of Since your data is line-oriented, try io:get_line/1 here. You are not going to get a line of input at a time with your approach I think but rather 8K. So how simple are your small inputs? -- J. From dmercer@REDACTED Mon Jan 10 19:31:20 2011 From: dmercer@REDACTED (David Mercer) Date: Mon, 10 Jan 2011 12:31:20 -0600 Subject: [erlang-questions] "Old inliner"? In-Reply-To: References: <000901cba062$11c42ee0$354c8ca0$@com> Message-ID: <002801cbb0f4$954de510$bfe9af30$@com> On Monday, January 10, 2011, Bj?rn Gustavsson wrote: > The old inliner is now only used for doing explicit inlining. The old > inliner only does inlining without any optimizations. > > The new inliner does implicit inling and also does optimizations > while inlining. If I also have the -compile(inline) directive in my source file, will it perform these optimizations -- whatever they are -- on the inlined call/3, or will it skip them on account of the explicit inlining? When I add the -compile(inline) directive, I still get the old verbose output about the old inliner being used, and also an additional line: Inlining: inline_size=24 inline_effort=150 Thanks for your reply. Cheers, DBM From dmercer@REDACTED Mon Jan 10 19:39:25 2011 From: dmercer@REDACTED (David Mercer) Date: Mon, 10 Jan 2011 12:39:25 -0600 Subject: [erlang-questions] message copying overhead atoms vs binaries In-Reply-To: References: Message-ID: <002901cbb0f5$b593a7a0$20baf6e0$@com> On Monday, January 10, 2011, Paolo Negri wrote: > There's a sentence in [2] that I can't fully understand. > > "Heap binaries are small binaries, up to 64 bytes, that are stored > directly on the process heap. They will be copied when the process is > garbage collected and when they are sent as a message. They don't > require any special handling by the garbage collector." > > Specifically I'm confused about why (and where) heap binaries will be > copied when the process is garbage collected. [I'm not the expert, but since no-one jumped up to answer, I'll give you what I think is the answer, and if no-one more qualified answers, mine can be considered the default correct answer.] Presumably, the Erlang runtime uses a copying garbage collector when it garbage collects a process. At a high level: it creates a new heap area and copies the "active" portions of the old heap to the new, and then returns the old heap area in its entirety to the system for reuse. There are umpteen variations on this, but at a high level, I'm guessing that is what is meant by copying during garbage collection. Cheers, DBM From karol.skocik@REDACTED Mon Jan 10 19:41:45 2011 From: karol.skocik@REDACTED (karol skocik) Date: Mon, 10 Jan 2011 19:41:45 +0100 Subject: hacked gmail account -> spam Message-ID: Dear Erlang users, my gmail account was hijacked and somebody sent a spam from my address. I would like to apologize for inconvenience. Thank you for understanding. Karol From rzezeski@REDACTED Mon Jan 10 20:20:18 2011 From: rzezeski@REDACTED (Ryan Zezeski) Date: Mon, 10 Jan 2011 14:20:18 -0500 Subject: error: Removing (timedout) connection Message-ID: Hi guys/gals, Recently I've been converting my non-distributed Erlang app into a distributed one and I ran into some troubles. If you want to skip straight to the question it's at the end, but I try to give some insight into what I'm doing below. First off, I attached a PDF (sorry, PDF was not my choice) which contains a diagram I drew of the current setup. I apologize for my utter failure as an artist. In this diagram you'll see 3 vertical partitions representing 3 different machines and a horizontal one representing the fact that each machine has 2 Erland nodes on it. 3 of the Erlang nodes form a riak cluster. The other 3 are the application (or should I say release) I wrote, and to distribute my app I utilized riak's underlying technology, riak_core (I use it as an easy way to persist cluster membership and use the ring metadata to store some data). These six nodes are fully connected, i.e. each node has connection to the other. Occasionally, I've noticed the following message on any one of the six nodes: =ERROR REPORT==== ... ** Node not responding ** ** Removing (timedout) connection ** Furthermore, using net_kernel:monitor_nodes(true, [nodedown_reason]) I've noticed messages like the following: {nodedown, , [{nodedown_reason, connection_closed}]} You'll notice there is a system process running on machine A, and it makes a gen_server:cast to three processes to do some work, and these processes each call link (L). Each of these three (gen_server) processes makes a call (at roughly the same time) to the riak cluster performing the _same exact_ map/reduce job. Sometimes I'll see errors where this map/reduce job times out on one of the nodes. So at lunch, I wondered, is it because there is just too much communication going on between the nodes that the kernel ticks are getting lost or delayed? I wondered if each node was using the same TCP connection to talk to every other node. That could explain my symptoms, right? A few netcats later and I realized that it's a dedicated conn for each node, so that theory was blown. However, I still think that many msgs being passed back and forth could be the cause of the problem, and I wondered if it blocks the VM in some way so that the kernel tick can't get through? Q: Can a chatty cluster cause the kernel ticks to be lost/delayed thus causing nodes to disconnect from each other? Thanks, -Ryan -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.virding@REDACTED Mon Jan 10 21:36:30 2011 From: robert.virding@REDACTED (Robert Virding) Date: Mon, 10 Jan 2011 20:36:30 +0000 (GMT) Subject: [erlang-questions] message copying overhead atoms vs binaries In-Reply-To: <907354010.265311294691369650.JavaMail.root@zimbra> Message-ID: <631496444.265331294691790270.JavaMail.root@zimbra> ----- "David Mercer" wrote: > On Monday, January 10, 2011, Paolo Negri wrote: > > > There's a sentence in [2] that I can't fully understand. > > > > "Heap binaries are small binaries, up to 64 bytes, that are stored > > directly on the process heap. They will be copied when the process > is > > garbage collected and when they are sent as a message. They don't > > require any special handling by the garbage collector." > > > > Specifically I'm confused about why (and where) heap binaries will > be > > copied when the process is garbage collected. > > [I'm not the expert, but since no-one jumped up to answer, I'll give > you > what I think is the answer, and if no-one more qualified answers, mine > can > be considered the default correct answer.] > > Presumably, the Erlang runtime uses a copying garbage collector when > it > garbage collects a process. At a high level: it creates a new heap > area and > copies the "active" portions of the old heap to the new, and then > returns > the old heap area in its entirety to the system for reuse. There are > umpteen variations on this, but at a high level, I'm guessing that is > what > is meant by copying during garbage collection. Yes, that is what is meant by a copying collector, which is what the BEAM uses to gc process heaps. It might seem inefficient to copy data instead of leaving it be and just marking the free areas but if the amount of live data is small, which it usually is, then it becomes more efficient. You also get compaction for "free" which is a big win. The algorithm is also simpler. Messages are copied from the sender's heap to the receiver's heap. Small binaries, less then 64 bytes, are stored in the process heaps so are gc'ed and copied as other data. As binaries can easily become VERY large this would become very inefficient. To get around this large binaries, more than 64 bytes, are stored outside the process heaps. They are not copied in either gc or message passing which makes it quite efficient to send them in messages. The problem is that this complicates the collector as it has to be able to handle that many processes may be referencing the binary so it can take a longer time for the collector to detect when they are free. Robert -- Robert Virding, Erlang Solutions Ltd. From robert.virding@REDACTED Mon Jan 10 21:39:12 2011 From: robert.virding@REDACTED (Robert Virding) Date: Mon, 10 Jan 2011 20:39:12 +0000 (GMT) Subject: [erlang-questions] Erlounge Edinburgh, Bonnie Scotland (Short Notice) In-Reply-To: Message-ID: <1640879593.265361294691952535.JavaMail.root@zimbra> Is there a link to the actual presentation? Would have loved to come. Robert ----- "Gordon Guthrie" wrote: > Folks > > There is a functional meetup in Edinburgh on Monday (10th Jan) in > which I > will be doing a talk entitled: > > "Building A Commercial Functional Programming Language In A > Commercial > Functional Programming Language" > > I figured we might was well start the year with an impromptu Erlounge > > Details here: > http://bloop.co/events/wz/ > > If anyone wants to come along and say the > magic > words "nine-nines" I will buy them a drink :) > > Gordon > > -- > Gordon Guthrie > CEO hypernumbers > > http://hypernumbers.com > t: hypernumbers > +44 7776 251669 -- Robert Virding, Erlang Solutions Ltd. From ok@REDACTED Mon Jan 10 23:52:47 2011 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 11 Jan 2011 11:52:47 +1300 Subject: [erlang-questions] 1000 cores on a chip In-Reply-To: References: Message-ID: On 10/01/2011, at 5:48 PM, Jesper Louis Andersen wrote: > I think so too. If you want to use GPUs *now*, going for CUDA through > a port or a NIF is the right way. My vision is that we can create a > programming language (essentially domain-specific) for computations on > GPUs. Yet, the GPUs are currently too quickly changing so it is not > yet a stable enough platform. Quite apart from things like OpenCL which are specifically designed for GPUs, there are languages like APL and ZPL which should lend themselves to adaptation to GPUs. For that matter, I'd have thought Fortran 95 a better starting point for GPU programming than C. Languages like Matlab (Octave) and S (R) encourage whole-array thinking, which is good news for GPUs, but GPUs aren't obviously a good host for dynamically typed languages. There's classic work on APL runtime compilation that suggests that forming expressions in a dynamic language, checking/ scheduling/compiling them on the host, and executing them on a GPU could be an effective approach. Come to think of it, reinterpreting Erlang numbers as the rank 0 members of an APL-like family of homogenous arrays needn't do much violence to the Erlang _language_. This has been done at least twice for Lisp (the Interactive Data-Analysis Language at Xerox in Interlisp, and XLisp for XLispStat). From ok@REDACTED Tue Jan 11 00:13:03 2011 From: ok@REDACTED (Richard O'Keefe) Date: Tue, 11 Jan 2011 12:13:03 +1300 Subject: [erlang-questions] message copying overhead atoms vs binaries In-Reply-To: References: Message-ID: <87367EE7-0B07-4C33-BAD4-4E38715CF449@cs.otago.ac.nz> On 10/01/2011, at 11:56 PM, Paolo Negri wrote: > Dear list, > > I'm trying to understand what approach of building values contained in > messages to pass across processes would be more efficient (in terms of > memory and cpu) and if the difference is significant. > I could build my message values in two ways > > a) [{color, red}, {shape, circle}] > > b) [{color, <<"red">>}, {shape, <<"circle">>}] Atoms do not need to be copied. Large binaries are not copied, but small ones are. From the Efficiency Guide: "The run-time system can represent binaries up to 64 bytes as heap binaries. They will always be copied when sent in a messages," > The actual lists are longer and can vary in length between 5 to 10 elements. > The processes will exchange thousands of these messages each second > and this is the reason why I'm asking this question. Does the order of the pairs in a list matter? I'm guessing that it doesn't. If so, the information in [{colour,red}, {shape,circle}] could equally well be held as {{colour,shape}, {red,circle}} and then one wonders how many different properties there are and how many combinations of properties and whether there might be some way to exploit commonality between groups of property names. But build it first and get it working and then measure it before doing anything tricky. From dsc@REDACTED Tue Jan 11 11:28:32 2011 From: dsc@REDACTED (Dave Challis) Date: Tue, 11 Jan 2011 10:28:32 +0000 Subject: [erlang-questions] Running a regular expression on each line of a file In-Reply-To: References: <4D2B2D8B.5040907@ecs.soton.ac.uk> <4D2C30D0.7000305@ecs.soton.ac.uk> Message-ID: Ah, thanks, io:get_line/1 makes more sense here. I solved the problem in the end by adding a another clause to the case statement: ... Result = re:run(Text, Re, [{capture, all_but_first, list}]), case Result of {match, Captured} -> io:format("~p ~p ~p~n", Captured); _False -> false end, ... It's still pretty slow to run though (~10 minutes to parse ~1 million lines). Interestingly enough, I tried removing the regex and just passing the input out unchanged, and the whole thing still takes ~9m30s to run, the bottleneck wasn't in the regex as I"d assumed. I'm guessing that io:get_line is being pretty slow. Is there a preferred method for faster I/O? On 10/01/11 17:42, Jesper Louis Andersen wrote: > On Mon, Jan 10, 2011 at 17:02, Dave Challis wrote: > >> parse(Re) -> >> case io:get_chars('', 8192) of > > Since your data is line-oriented, try io:get_line/1 here. You are not > going to get a line of input at a time with your approach I think but > rather 8K. So how simple are your small inputs? > -- Dave Challis dsc@REDACTED From hynek@REDACTED Tue Jan 11 13:14:45 2011 From: hynek@REDACTED (Hynek Vychodil) Date: Tue, 11 Jan 2011 13:14:45 +0100 Subject: [erlang-questions] Running a regular expression on each line of a file In-Reply-To: References: <4D2B2D8B.5040907@ecs.soton.ac.uk> <4D2C30D0.7000305@ecs.soton.ac.uk> Message-ID: If you are looking for fastest line oriented IO from STDIN you can find fastest (known to me) in http://shootout.alioth.debian.org/u32q/program.php?test=regexdna&lang=hipe&id=6 If you can read from file, your hand crafted block IO using file:open/2 in raw, binary mode should be better. Use binary module for splitting by lines and don't forget glue lines on block boundaries. You can look for wide finder project for some inspiration. And of course in both cases don't transform to lists but keep it in binaries. On Tue, Jan 11, 2011 at 11:28 AM, Dave Challis wrote: > Ah, thanks, io:get_line/1 makes more sense here. > > I solved the problem in the end by adding a another clause to the case > statement: > > ... > Result = re:run(Text, Re, [{capture, all_but_first, list}]), > case Result of > ? ?{match, Captured} -> > ? ? ? ?io:format("~p ~p ~p~n", Captured); > ? ?_False -> > ? ? ? ?false > end, > ... > > It's still pretty slow to run though (~10 minutes to parse ~1 million > lines). ?Interestingly enough, I tried removing the regex and just passing > the input out unchanged, and the whole thing still takes ~9m30s to run, the > bottleneck wasn't in the regex as I"d assumed. > > I'm guessing that io:get_line is being pretty slow. ?Is there a preferred > method for faster I/O? > > > > On 10/01/11 17:42, Jesper Louis Andersen wrote: >> >> On Mon, Jan 10, 2011 at 17:02, Dave Challis ?wrote: >> >>> parse(Re) -> >>> ? ?case io:get_chars('', 8192) of >> >> Since your data is line-oriented, try io:get_line/1 here. You are not >> going to get a line of input at a time with your approach I think but >> rather 8K. So how simple are your small inputs? >> > > > -- > Dave Challis > dsc@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From jameschurchman@REDACTED Tue Jan 11 14:41:58 2011 From: jameschurchman@REDACTED (James Churchman) Date: Tue, 11 Jan 2011 13:41:58 +0000 Subject: odd one out Message-ID: <97EC79D9-56F8-43A0-A815-5B003A6995C2@gmail.com> can anyone spot the (very) odd one out ! http://qconlondon.com/london-2011/tracks/show_track.jsp?trackOID=424 From vinoski@REDACTED Tue Jan 11 14:48:17 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Tue, 11 Jan 2011 08:48:17 -0500 Subject: [erlang-questions] odd one out In-Reply-To: <97EC79D9-56F8-43A0-A815-5B003A6995C2@gmail.com> References: <97EC79D9-56F8-43A0-A815-5B003A6995C2@gmail.com> Message-ID: On Tue, Jan 11, 2011 at 8:41 AM, James Churchman wrote: > can anyone spot the (very) odd one out ! > > http://qconlondon.com/london-2011/tracks/show_track.jsp?trackOID=424 Hmm, let's see...the track host? ;-) Not sure what you're getting at, but are you hinting that node.js isn't written in a functional language? If so, you should know that JavaScript supports functional programming. Ever used jQuery for example? --steve From nox@REDACTED Tue Jan 11 14:54:58 2011 From: nox@REDACTED (nox) Date: Tue, 11 Jan 2011 14:54:58 +0100 Subject: [erlang-questions] odd one out In-Reply-To: References: <97EC79D9-56F8-43A0-A815-5B003A6995C2@gmail.com> Message-ID: <28B3F1D9-BB56-462D-8212-C4D8B7E3BA3E@dev-extend.eu> Le 11 janv. 2011 ? 14:48, Steve Vinoski a ?crit : > On Tue, Jan 11, 2011 at 8:41 AM, James Churchman > wrote: >> can anyone spot the (very) odd one out ! >> >> http://qconlondon.com/london-2011/tracks/show_track.jsp?trackOID=424 > > Hmm, let's see...the track host? ;-) > > Not sure what you're getting at, but are you hinting that node.js > isn't written in a functional language? If so, you should know that > JavaScript supports functional programming. Ever used jQuery for > example? > > --steve I wouldn't say ECMAScript doesn't support functional programming, but it sure is not its main paradigm. And since when jQuery was called "functional"? For me, functional programming is very tied to the concept of pure functions and jQuery is pretty much a big object encapsulating in-place modifications of the DOM. Regards, -- Anthony Ramine Dev:Extend http://dev-extend.eu From jameschurchman@REDACTED Tue Jan 11 15:04:45 2011 From: jameschurchman@REDACTED (James Churchman) Date: Tue, 11 Jan 2011 14:04:45 +0000 Subject: [erlang-questions] odd one out In-Reply-To: <28B3F1D9-BB56-462D-8212-C4D8B7E3BA3E@dev-extend.eu> References: <97EC79D9-56F8-43A0-A815-5B003A6995C2@gmail.com> <28B3F1D9-BB56-462D-8212-C4D8B7E3BA3E@dev-extend.eu> Message-ID: hah well im sure it will be a great talk!!, but you can do "functional style" in just about anything + lots of languages have first class functions these days... PHP 5.3 , D 2.0 etc.. by that definition just about every language would be functional i think that the entire track would be beter named "things that are not java, php or rails for web" :-) On 11 January 2011 13:54, nox wrote: > Le 11 janv. 2011 ? 14:48, Steve Vinoski a ?crit : > > > On Tue, Jan 11, 2011 at 8:41 AM, James Churchman > > wrote: > >> can anyone spot the (very) odd one out ! > >> > >> http://qconlondon.com/london-2011/tracks/show_track.jsp?trackOID=424 > > > > Hmm, let's see...the track host? ;-) > > > > Not sure what you're getting at, but are you hinting that node.js > > isn't written in a functional language? If so, you should know that > > JavaScript supports functional programming. Ever used jQuery for > > example? > > > > --steve > > I wouldn't say ECMAScript doesn't support functional programming, but it > sure is not its main paradigm. And since when jQuery was called > "functional"? For me, functional programming is very tied to the concept of > pure functions and jQuery is pretty much a big object encapsulating in-place > modifications of the DOM. > > Regards, > > -- > Anthony Ramine > Dev:Extend > http://dev-extend.eu > > > > > From bgustavsson@REDACTED Tue Jan 11 15:29:31 2011 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Tue, 11 Jan 2011 15:29:31 +0100 Subject: [erlang-questions] "Old inliner"? In-Reply-To: <002801cbb0f4$954de510$bfe9af30$@com> References: <000901cba062$11c42ee0$354c8ca0$@com> <002801cbb0f4$954de510$bfe9af30$@com> Message-ID: 2011/1/10 David Mercer : > On Monday, January 10, 2011, Bj?rn Gustavsson wrote: > >> The old inliner is now only used for doing explicit inlining. The old >> inliner only does inlining without any optimizations. >> >> The new inliner does implicit inling and also does optimizations >> while inlining. > > If I also have the -compile(inline) directive in my source file, will it perform these optimizations -- whatever they are -- on the inlined call/3, or will it skip them on account of the explicit inlining? When I add the -compile(inline) directive, I still get the old verbose output about the old inliner being used, and also an additional line: > > Inlining: inline_size=24 inline_effort=150 > That means that both inliners are run, meaning that the optimizations in the new inliner will be performed. -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From vinoski@REDACTED Tue Jan 11 15:49:06 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Tue, 11 Jan 2011 09:49:06 -0500 Subject: [erlang-questions] odd one out In-Reply-To: <28B3F1D9-BB56-462D-8212-C4D8B7E3BA3E@dev-extend.eu> References: <97EC79D9-56F8-43A0-A815-5B003A6995C2@gmail.com> <28B3F1D9-BB56-462D-8212-C4D8B7E3BA3E@dev-extend.eu> Message-ID: On Tue, Jan 11, 2011 at 8:54 AM, nox wrote: > Le 11 janv. 2011 ? 14:48, Steve Vinoski a ?crit : > >> On Tue, Jan 11, 2011 at 8:41 AM, James Churchman >> wrote: >>> can anyone spot the (very) odd one out ! >>> >>> http://qconlondon.com/london-2011/tracks/show_track.jsp?trackOID=424 >> >> Hmm, let's see...the track host? ;-) >> >> Not sure what you're getting at, but are you hinting that node.js >> isn't written in a functional language? If so, you should know that >> JavaScript supports functional programming. Ever used jQuery for >> example? >> >> --steve > > I wouldn't say ECMAScript doesn't support functional programming, but it sure is not its main > paradigm. And since when jQuery was called "functional"? For me, functional programming is > very tied to the concept of pure functions and jQuery is pretty much a big object > encapsulating in-place modifications of the DOM. I disagree, having used it extensively I know jQuery is FP-oriented in style and approach. The fact that you have to manipulate the DOM in place isn't jQuery's fault. Heck, if you want to get really picky, we shouldn't include Erlang in the conference track either, given how rampantly side-effects occur within normal Erlang apps (as Erik Meijer is so fond of reminding me whenever he sees me). --steve From toby@REDACTED Tue Jan 11 16:01:25 2011 From: toby@REDACTED (Toby Thain) Date: Tue, 11 Jan 2011 12:01:25 -0300 Subject: [erlang-questions] odd one out In-Reply-To: <28B3F1D9-BB56-462D-8212-C4D8B7E3BA3E@dev-extend.eu> References: <97EC79D9-56F8-43A0-A815-5B003A6995C2@gmail.com> <28B3F1D9-BB56-462D-8212-C4D8B7E3BA3E@dev-extend.eu> Message-ID: <4D2C70C5.7000708@telegraphics.com.au> On 11-01-11 10:54 AM, nox wrote: > Le 11 janv. 2011 ? 14:48, Steve Vinoski a ?crit : > >> On Tue, Jan 11, 2011 at 8:41 AM, James Churchman >> wrote: >>> can anyone spot the (very) odd one out ! >>> >>> http://qconlondon.com/london-2011/tracks/show_track.jsp?trackOID=424 >> >> Hmm, let's see...the track host? ;-) >> >> Not sure what you're getting at, but are you hinting that node.js >> isn't written in a functional language? If so, you should know that >> JavaScript supports functional programming. Ever used jQuery for >> example? >> >> --steve > > I wouldn't say ECMAScript doesn't support functional programming, but it sure is not its main paradigm. And since when jQuery was called "functional"? For me, functional programming is very tied to the concept of pure functions and jQuery is pretty much a big object encapsulating in-place modifications of the DOM. > Wouldn't one have to see how *the code cited* is structured? Something makes me doubt it's what a Lisper would call 'functional'. --Toby > Regards, > > -- > Anthony Ramine > Dev:Extend > http://dev-extend.eu > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From vinoski@REDACTED Tue Jan 11 16:02:59 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Tue, 11 Jan 2011 10:02:59 -0500 Subject: [erlang-questions] odd one out In-Reply-To: References: <97EC79D9-56F8-43A0-A815-5B003A6995C2@gmail.com> <28B3F1D9-BB56-462D-8212-C4D8B7E3BA3E@dev-extend.eu> Message-ID: On Tue, Jan 11, 2011 at 9:04 AM, James Churchman wrote: > hah well im sure it will be a great talk!!, but you can do "functional > style"?in just about anything + lots of languages have first class functions > these days... PHP 5.3 , D 2.0 etc.. by that definition just about every > language would be functional > i think that the entire track would be beter named "things that are not > java, php or rails for web" :-) Yes, a number of languages, including PHP and C++, have been adopting FP features lately. It's not such a bad thing. The conference track is an offshoot of my "Functional Web" magazine column that I write for the IEEE Internet Computing magazine. In the column I try to write about (or get guest authors to write about) how and where FP techniques and approaches can best be applied to web development. It's not about FP purity, but rather about practical application of such techniques for industry engineers like me. A number of the speakers in my track have guest-authored or co-authored in the column. All the columns are online here: http://steve.vinoski.net/blog/internet-computing-columns/ The brand new column on Haskell Snap just came out a couple days ago, so I'll try to post the PDF there later today. If you attend QCon, which I highly recommend, I hope you'll attend the track. --steve From raimo+erlang-questions@REDACTED Tue Jan 11 16:26:28 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 11 Jan 2011 16:26:28 +0100 Subject: [erlang-questions] hacked gmail account -> spam In-Reply-To: References: Message-ID: <20110111152628.GA15773@erix.ericsson.se> On Mon, Jan 10, 2011 at 07:41:45PM +0100, karol skocik wrote: > Dear Erlang users, > my gmail account was hijacked and somebody sent a spam from my address. > I would like to apologize for inconvenience. > Thank you for understanding. > Karol Thanks for your explanation. I was just about to ban you. Perhaps time to change password on my gmail account... > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From karol.skocik@REDACTED Tue Jan 11 16:33:31 2011 From: karol.skocik@REDACTED (karol skocik) Date: Tue, 11 Jan 2011 16:33:31 +0100 Subject: [erlang-questions] hacked gmail account -> spam In-Reply-To: <20110111152628.GA15773@erix.ericsson.se> References: <20110111152628.GA15773@erix.ericsson.se> Message-ID: Hi Raimo, On Tue, Jan 11, 2011 at 4:26 PM, Raimo Niskanen wrote: > On Mon, Jan 10, 2011 at 07:41:45PM +0100, karol skocik wrote: >> Dear Erlang users, >> ? my gmail account was hijacked and somebody sent a spam from my address. >> I would like to apologize for inconvenience. >> Thank you for understanding. >> Karol > > Thanks for your explanation. I was just about to ban you. > Thanks for not doing that so swiftly :) > Perhaps time to change password on my gmail account... > And possibly also time to switch to Chromium, to take advantage of tab sandboxing, as suggested by Tuncer. >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > From jameschurchman@REDACTED Tue Jan 11 16:53:42 2011 From: jameschurchman@REDACTED (James Churchman) Date: Tue, 11 Jan 2011 15:53:42 +0000 Subject: [erlang-questions] odd one out In-Reply-To: <4D2C70C5.7000708@telegraphics.com.au> References: <97EC79D9-56F8-43A0-A815-5B003A6995C2@gmail.com> <28B3F1D9-BB56-462D-8212-C4D8B7E3BA3E@dev-extend.eu> <4D2C70C5.7000708@telegraphics.com.au> Message-ID: i would agree that adopting a functional style is ideal for javascript, but javascript is so far from being side effect free its complete madness to regard it a functional language. i dont know if node differs from traditional JS but i could write a book (10 in fact) on side effects in javascript, excluding the dom , and im sure there are many i have yet to encounter, i think that it gives a misleading impression to all other functional languages to label js one, and even attempting functional programming in the browser in JS is still uniquely tricky. i would personaly say that most language can be made to be like a functional language, except javascript :-) On 11 January 2011 15:01, Toby Thain wrote: > On 11-01-11 10:54 AM, nox wrote: > > Le 11 janv. 2011 ? 14:48, Steve Vinoski a ?crit : > > > >> On Tue, Jan 11, 2011 at 8:41 AM, James Churchman > >> wrote: > >>> can anyone spot the (very) odd one out ! > >>> > >>> http://qconlondon.com/london-2011/tracks/show_track.jsp?trackOID=424 > >> > >> Hmm, let's see...the track host? ;-) > >> > >> Not sure what you're getting at, but are you hinting that node.js > >> isn't written in a functional language? If so, you should know that > >> JavaScript supports functional programming. Ever used jQuery for > >> example? > >> > >> --steve > > > > I wouldn't say ECMAScript doesn't support functional programming, but it > sure is not its main paradigm. And since when jQuery was called > "functional"? For me, functional programming is very tied to the concept of > pure functions and jQuery is pretty much a big object encapsulating in-place > modifications of the DOM. > > > > Wouldn't one have to see how *the code cited* is structured? Something > makes me doubt it's what a Lisper would call 'functional'. > > --Toby > > > > Regards, > > > > -- > > Anthony Ramine > > Dev:Extend > > http://dev-extend.eu > > > > > > > > > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From robert.virding@REDACTED Tue Jan 11 18:32:43 2011 From: robert.virding@REDACTED (Robert Virding) Date: Tue, 11 Jan 2011 17:32:43 +0000 (GMT) Subject: We need a better marketing division :-) Message-ID: <24250519.271261294767163150.JavaMail.root@zimbra> http://lambda-the-ultimate.org/node/4178 From raould@REDACTED Tue Jan 11 18:55:46 2011 From: raould@REDACTED (Raoul Duke) Date: Tue, 11 Jan 2011 09:55:46 -0800 Subject: [erlang-questions] We need a better marketing division :-) In-Reply-To: <24250519.271261294767163150.JavaMail.root@zimbra> References: <24250519.271261294767163150.JavaMail.root@zimbra> Message-ID: On Tue, Jan 11, 2011 at 9:32 AM, Robert Virding wrote: > http://lambda-the-ultimate.org/node/4178 wait, is your name really Tim Sweeney? :-) From alain.odea@REDACTED Tue Jan 11 19:21:16 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Tue, 11 Jan 2011 14:51:16 -0330 Subject: [erlang-questions] hacked gmail account -> spam In-Reply-To: References: <20110111152628.GA15773@erix.ericsson.se> Message-ID: <79F3C0A8-9170-4EC8-8796-38564FAFBD8C@gmail.com> I recently updated all my service passwords to independent 14-20 character long passwords with mixed case, numbers and symbols, etc. which I got from http://www.pctools.com/guides/password/ I generated them in batches of 3-5 and saved them in a keychain secure note for reference before changing the service passwords. On 2011-01-11, at 12:03, karol skocik wrote: > Hi Raimo, > > On Tue, Jan 11, 2011 at 4:26 PM, Raimo Niskanen > wrote: >> On Mon, Jan 10, 2011 at 07:41:45PM +0100, karol skocik wrote: >>> Dear Erlang users, >>> my gmail account was hijacked and somebody sent a spam from my address. >>> I would like to apologize for inconvenience. >>> Thank you for understanding. >>> Karol >> >> Thanks for your explanation. I was just about to ban you. >> > > Thanks for not doing that so swiftly :) > >> Perhaps time to change password on my gmail account... >> > > And possibly also time to switch to Chromium, to take advantage of tab > sandboxing, as suggested by Tuncer. > >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> -- >> >> / Raimo Niskanen, Erlang/OTP, Ericsson AB >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From daniel.goertzen@REDACTED Tue Jan 11 21:01:30 2011 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Tue, 11 Jan 2011 14:01:30 -0600 Subject: no sasl output in erlide Message-ID: I'm giving erlide a test drive (very nice BTW), and I really miss the sasl output that I normally see when using erl.exe/werl.exe. How do I get it back? erlide console session... Eshell V5.8.2 (just_a_console_node@REDACTED)1> application:loaded_applications(). [{ssh,"SSH-2 for Erlang/OTP","2.0.3"}, {mnesia,"MNESIA CXC 138 12","4.4.16"}, {kernel,"ERTS CXC 138 10","2.14.2"}, {crypto,"CRYPTO version 2","2.0.2"}, {sasl,"SASL CXC 138 11","2.1.9.2"}, {stdlib,"ERTS CXC 138 10","1.17.2"}] (just_a_console_node@REDACTED)2> application:get_all_env(sasl). [{errlog_type,all}, {sasl_error_logger,tty}, {included_applications,[]}] (just_a_console_node@REDACTED)3> Thanks, Dan. From vladdu55@REDACTED Tue Jan 11 22:05:20 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 11 Jan 2011 22:05:20 +0100 Subject: [erlang-questions] no sasl output in erlide In-Reply-To: References: Message-ID: Hi! On Tue, Jan 11, 2011 at 21:01, Daniel Goertzen wrote: > I'm giving erlide a test drive (very nice BTW), and I really miss the sasl > output that I normally see when using erl.exe/werl.exe. How do I get it > back? > > This is a known limitation for the current implementation. The technical reason is that the default error_logger handler sends the printouts directly to the tty, while the shell you see in the erlide console is just a remote shell, not the stdout. I am currently working on a related issue and maybe I will be able to do something about it without having to rewrite everything. There are a few workarounds, for the time being: * configure your node to send sasl messages to a file and use a separate console to "tail -f" that file. For this, enter something like -boot start_sasl -sasl sasl_error_logger '{file,"/home/me/mylog.log"}' in the "extra arguments" field on the "runtimes" tab of the launch configuration dialog (that's bunch of Eclipse jargon, sorry about that) * start your erlang node manually in a console and configure the launch to use the right node name and cookie. It will then work as usual, the sasl output will of course appear in the console. * in your eclipse.ini file (in the eclipse directory), add at the end on a separate line -Derlide.console.stdout=true This will show the stdout consoles for all nodes used by erlide and will probably look quite confusing, but if you are willing to live with it and can find the right console, you can use it directly (with very limited editing capabilities and ugly colours, though). regards, Vlad From robert.virding@REDACTED Tue Jan 11 23:09:41 2011 From: robert.virding@REDACTED (Robert Virding) Date: Tue, 11 Jan 2011 22:09:41 +0000 (GMT) Subject: [erlang-questions] We need a better marketing division :-) In-Reply-To: Message-ID: <1232899433.272271294783781218.JavaMail.root@zimbra> ----- "Raoul Duke" wrote: > On Tue, Jan 11, 2011 at 9:32 AM, Robert Virding > wrote: > > http://lambda-the-ultimate.org/node/4178 > > wait, is your name really Tim Sweeney? :-) The real question is of course whether Tim Sweeney is Robert Virding? :-) He does raise some good points. Robert -- Robert Virding, Erlang Solutions Ltd. From mjtruog@REDACTED Wed Jan 12 04:59:45 2011 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 11 Jan 2011 19:59:45 -0800 Subject: [erlang-questions] We need a better marketing division :-) In-Reply-To: <1232899433.272271294783781218.JavaMail.root@zimbra> References: <1232899433.272271294783781218.JavaMail.root@zimbra> Message-ID: <4D2D2731.3020408@gmail.com> >>> http://lambda-the-ultimate.org/node/4178 >>> I am wondering if anyone has gotten to do comparisons between Erlang and Scala when Scala is using Akka (which uses jetlang). The comparison would be interesting because both setups are using lightweight processes as actors. The messaging with both setups might be similar performance-wise, if you ignore the garbage collection differences. A few links are: http://akka.io/ http://code.google.com/p/jetlang/ From rzezeski@REDACTED Wed Jan 12 05:03:12 2011 From: rzezeski@REDACTED (Ryan Zezeski) Date: Tue, 11 Jan 2011 23:03:12 -0500 Subject: [erlang-questions] Processes & Fault Tolerance In-Reply-To: References: Message-ID: This thread is pure GOLD. I know that all this stuff is covered in various places but seeing it compressed in this thread was really profound...for me anyways. I'm to keep this in my pocket for the presentation I plan to give later in the year. -Ryan On Thu, Jan 6, 2011 at 1:32 AM, Edmond Begumisa wrote: > UPDATE: > > I'm excited! I've just added hot take-over to my server program (for > specific jobs). I've watched one node take-over the handling of a request > from another as it's network connection died. The client didn't notice. Very > eerie to witness, and quite impressive how easy it was to add that > functionality if like me, you've come from other languages and imagined the > pain involved in doing this. > > Using Joe's illustration, I was able to do this without the functionality > of OTP supervisors, application take-over/fail-over, etc (even though my > application is an OTP application.) Just using redundant gen_server worker > processes watching each-other and updating each-other on state changes + new > requests. > > To summerise the important lessons I've learned from all who helped: In > regards to processes, fault-tolerance and the share-nothing principle... > > * We can say: Share nothing because we want to replicate valid state which > leads to one form of fault tolerance (a failed job can be done elsewhere). > * Or we can say: Share nothing because we want to isolate invalid state > which leads to another form fault tolerance (a failed job won't affect the > rest of the system). > * For job hot take-over + redundancy, one should think in terms of the > former (you need two or more nodes for this). > * For worker supervision + restarts, one should think in terms of the > latter (you can use one or more nodes for this). > > Supervisors monitor workers. Workers do jobs. In-case of faults: workers > can be restarted but also **jobs can be taken over by one redundant worker > from another**. This is powerful, Erlang is pretty cool :) > > I'm gonna go marvel as I plug and unplug my network cables now :P > > Thanks again to all. > > - Edmond - > > > > On Thu, 06 Jan 2011 02:17:39 +1100, Edmond Begumisa < > ebegumisa@REDACTED> wrote: > > On Tue, 04 Jan 2011 20:52:50 +1100, Joe Armstrong >> wrote: >> >> On Mon, Jan 3, 2011 at 5:30 PM, Edmond Begumisa >>> wrote: >>> >>>> Let's start with how we do error recovery. >>>>> >>>>> Imagine two linked processes A and B on separate machines. A is the >>>>> master process. >>>>> B is a passive processes that will take over if A fails. >>>>> >>>>> A sends a stream of state update messages S1, S2, S3,... to B. The >>>>> state messages contain enough information >>>>> for B to do whatever A was doing should A fail. If A fails B will >>>>> receive an EXIT signal. >>>>> >>>>> If B does receive an exit signal it knows A has failed, so it carries >>>>> on doing whatever A was doing >>>>> using the information it last received in a state update message. >>>>> >>>>> That's it and all about it - nothing to do with supervisors etc, >>>>> >>>> >>>> Ooooohh! THANK YOU, THANK YOU, THANK YOU. >>>> >>> >>> Thanks. >>> >>> Another problem in understand is that we assume the meaning of words. >>> >>> >> Yes. There was some confusion about whether we were all talking about >> replicating valid state or isolating invalid state. Which I guess >> "fault-tolerance" could be interpreted to mean either. But I'm no expert on >> the subject so I shouldn't comment on terminology -- I just knew the feature >> I wanted and was pretty sure it could be done at the process level, I just >> had no-idea how! From now on, I'll stick to the term "redundancy" or Ulf's >> "hot take-over" so it's clearer what I mean. >> >> - Edmond - >> >> >> Once upon a time I was talking to a guy and used the word "fault >>> tolerance" >>> we talked for hours and thought we understood what this word meant. Then >>> he said something that implied he was building a fault-tolerant system >>> on one computer. >>> I said it was impossible. He said, "you catch and handle all the >>> exceptions .." >>> >>> I said, "The entire computer might crash" >>> >>> He said, "Oh" >>> >>> There was a long silence. >>> >>> I told him about Erlang ... >>> >>> >>> /Joe >>> >>> >>>> *Now* I get it. This is precisely what I was trying to understand. The >>>> missing link was sending redundant messages. It all makes sense now -- >>>> it's >>>> so simple. All I have to do to get fault tolerance at the process level >>>> is >>>> have a group of N redundant processes waiting for exit-signals and >>>> forward >>>> state changes to N-1 of them. I could even send to N/2 and have some >>>> decent >>>> tolerance at the expense of consistency of the nodes. >>>> >>>> To preemptively answer the question I sent Ulf and Mazen... for that >>>> distributed database, Process B would be set up to also receive read >>>> requests forwarded from Process A but not respond to them unless it gets >>>> an >>>> exit signal it can understand like disk_fail_error. Any changes in state >>>> would also be forwarded to existing redundant processes. >>>> >>>> Thanks everyone for your patience in helping me understand >>>> fault-tolerance. >>>> I've already started thinking about some cool things I can do with this >>>> in >>>> my program. >>>> >>>> - Edmond - >>>> >>>> >>>> >>>> On Tue, 04 Jan 2011 00:36:30 +1100, Joe Armstrong >>>> wrote: >>>> >>>> I think you are getting confused between OTP supervisors etc. and the >>>>> general notion of "take-over" >>>>> >>>>> Let's start with how we do error recovery. >>>>> >>>>> Imagine two linked processes A and B on separate machines. A is the >>>>> master process. >>>>> B is a passive processes that will take over if A fails. >>>>> >>>>> A sends a stream of state update messages S1, S2, S3,... to B. The >>>>> state messages contain enough information >>>>> for B to do whatever A was doing should A fail. If A fails B will >>>>> receive an EXIT signal. >>>>> >>>>> If B does receive an exit signal it knows A has failed, so it carries >>>>> on doing whatever A was doing >>>>> using the information it last received in a state update message. >>>>> >>>>> That's it and all about it - nothing to do with supervisors etc, >>>>> >>>>> >>>>> To make the above possible we need a couple of primitives in Erlang, >>>>> namely process_flag trap_exits, and spawn_link. That's all you need. >>>>> >>>>> The basic system properties you need to make a fault-tolerant systems >>>>> are the abilities to detect remote errors and send and receive >>>>> messages. >>>>> >>>>> What the OTP supervisors etc. do are build layers of abstraction on >>>>> top of spawn-link and trap_exits >>>>> in order to simplify programming common use-cases. >>>>> >>>>> Fault detection and recovery is not arbitrary or magic in any sense, >>>>> It has to be part of the system architecture like everything else. One >>>>> common fault in making systems is to specify and design for the normal >>>>> behavior >>>>> but not specify and design how fault-detection and correction work. >>>>> This is probably due to a legacy of >>>>> programming in languages that have very limited ability to detect >>>>> errors and recover from them. >>>>> >>>>> A C programmer with a single thread of control will take extreme >>>>> measures to avoid crashing their program >>>>> they have to, there is only one thread of control. Given multiple >>>>> threads of control and the ability to remotely >>>>> detect errors your entire way of thinking changes and the emphasis >>>>> changes from thinking "how to I avoid a crash" >>>>> to thinking "given that a crash has occurred and been detected, how to >>>>> I fix things up and carry on" >>>>> >>>>> The Erlang philosophy is that "things will crash", so we have to >>>>> detect the crashes, fix the bit that broke and >>>>> carry on - we do not make excessive efforts to avoid crashes, but we >>>>> do try very hard to repair things after they have gone wrong. >>>>> >>>>> Fixing things after they have broken is often easier than preventing >>>>> them from breaking. Preventing them >>>>> from breaking is not theoretically possible - we can't even prove >>>>> simple things, like that a program will terminate >>>>> (the famous halting problem) - but we can easily detect failures and >>>>> put the system to rights by applying >>>>> certain invariants. >>>>> >>>>> OTP supervisors are just trees of processes with certain restart rules >>>>> that experience has shown us >>>>> to be useful. >>>>> >>>>> The main value of supervisors etc. (and of all the OTP behaviors) is >>>>> in building large systems with many >>>>> programmers involved. Rather than inventing their own patterns, the >>>>> programmers reuse a common set of >>>>> patterns, which makes the projects easier to manage. >>>>> >>>>> /Joe >>>>> >>>>> >>>>> >>>>> >>>>> On Mon, Jan 3, 2011 at 6:21 AM, Edmond Begumisa >>>>> wrote: >>>>> >>>>>> >>>>>> Thanks for your response. >>>>>> >>>>>> Firstly, let me make my question a little clearer... >>>>>> >>>>>> To rephrase: For processes, "share nothing for the sake of >>>>>> concurrency" - >>>>>> I >>>>>> get, both in concept and application. "Share nothing for the sake of >>>>>> fault-tolerance" - I get in concept but not in application. >>>>>> >>>>>> Yet as I understand it, it is for the latter reason Erlang shares >>>>>> nothing* >>>>>> and not the former. Interpretation: I must be completely missing the >>>>>> point >>>>>> in regards to Erlang processes and sharing nothing. This is what I >>>>>> want >>>>>> to >>>>>> understand in application. In addition to the "side" effect of sane >>>>>> concurrency (which coming from a chaotic multi-threading shared-memory >>>>>> world >>>>>> I fully appreciate and practically make use of everyday), how can I >>>>>> also >>>>>> make use of the "real" reason Erlang processes share nothing -- fault >>>>>> tolerance? Practically/illustratively speaking? >>>>>> >>>>>> *ETS being the obvious exception. >>>>>> >>>>>> Secondly, here's a mantra from Joe Armstrong... >>>>>> >>>>>> @ minute 17:26 >>>>>> http://www.se-radio.net/2008/03/episode-89-joe-armstrong-on-erlang/ >>>>>> >>>>>> "[message passing concurrency]... the original reasons have to do with >>>>>> fault >>>>>> tolerance... you have to copy all the data you need from computer 1 to >>>>>> computer 2... if computer 1 crashes you take over on computer 2... you >>>>>> can't >>>>>> have dangling pointers... that's the reason for copying everything... >>>>>> it's >>>>>> got nothing to do with concurrency, it's got a lot to do with >>>>>> fault-tolerance... if they don't crash you could just have a dangling >>>>>> pointer and copy less data but it won't work in the presence of >>>>>> errors..." >>>>>> >>>>>> I interpret this to mean that share-nothing between processes is more >>>>>> about >>>>>> replicating valid state than isolating corrupted state as you >>>>>> described. >>>>>> >>>>>> Indeed, Joe created an example on his blog... >>>>>> >>>>>> >>>>>> >>>>>> http://armstrongonsoftware.blogspot.com/2007/07/scalable-fault-tolerant-upgradable.html >>>>>> >>>>>> It's algorithm 3 there I'm struggling with. Particularly where he >>>>>> says... >>>>>> >>>>>> "... In practise we would send an asynchronous stream of messages from >>>>>> N >>>>>> to >>>>>> N+1 containing enough information to recover if things go wrong." >>>>>> >>>>>> Unfortunately, I couldn't find part II to that post (I don't think >>>>>> there >>>>>> is >>>>>> one.) And I'm too green and inexperienced in the field of >>>>>> fault-tolerant >>>>>> systems to figure it out on my own. I'm having trouble visualising the >>>>>> practical here from the conceptual -- I need to be shown how :( >>>>>> >>>>>> Also, I seem to be under the impression that the Erlang language has >>>>>> some >>>>>> sort of schematics to do this built-in (i.e. deal with one process >>>>>> taking >>>>>> over from another if the first fails) and this is the reason processes >>>>>> share >>>>>> nothing. This seems to me to be something different from supervision >>>>>> trees, >>>>>> which use exit-trapping to re-spawn if a process fails with the active >>>>>> 'job' >>>>>> disappearing and any errors logged (like restarting a daemon). My >>>>>> interpretation of the fault-tolerance Erlang is supposed to enable >>>>>> (for >>>>>> those in the know) is seamless take-over. The 'job' lives on but >>>>>> elsewhere. >>>>>> >>>>>> Using telecoms as an example: a phone call wouldn't be cut-off when a >>>>>> fault >>>>>> occurs, another node would seamlessly take over. This is how I >>>>>> interpreted >>>>>> Joe's post and other descriptions of Erlang's fault-tolerant features >>>>>> and >>>>>> I >>>>>> understand the key is in the share nothing policy for processes. I'm >>>>>> sure >>>>>> I've mis-understood something or everything :) >>>>>> >>>>>> - Edmond - >>>>>> >>>>>> PS: I've read the Manning draft. Great book. I don't know if the >>>>>> answer >>>>>> lies >>>>>> in OTP (I searched and didn't find it). I suspect it's lower -- >>>>>> probalby >>>>>> how >>>>>> you organise your processes. Some distributed-programming black-magic >>>>>> only >>>>>> Erlanger's know about :) >>>>>> >>>>>> >>>>>> On Mon, 03 Jan 2011 14:56:51 +1100, Alain O'Dea >>>>> > >>>>>> wrote: >>>>>> >>>>>> On 2011-01-02, at 22:36, "Edmond Begumisa" < >>>>>>> ebegumisa@REDACTED> >>>>>>> wrote: >>>>>>> >>>>>>> Slight correction... >>>>>>>> >>>>>>>> On Mon, 03 Jan 2011 12:38:38 +1100, Edmond Begumisa >>>>>>>> wrote: >>>>>>>> >>>>>>>> Hello all, >>>>>>>>> >>>>>>>>> I've been trying to wrap my Erlang's fault tolerant features >>>>>>>>> particularly in relation to processes. >>>>>>>>> >>>>>>>>> >>>>>>>> Should be: I've been trying to wrap my head around Erlang's fault >>>>>>>> tolerant features particularly in relation to processes. >>>>>>>> >>>>>>>> Sorry. >>>>>>>> >>>>>>>> I've heard/read repeatedly that the primary reason why Erlang's >>>>>>>>> designers opted for a share-nothing policy is not rooted in >>>>>>>>> concurrency but >>>>>>>>> rather in fault-tolerance. When nothing is shared, everything is >>>>>>>>> copied. >>>>>>>>> When everything is copied processes can take over from one another >>>>>>>>> when >>>>>>>>> things fail. I follow this reasoning but I don't follow how to >>>>>>>>> apply >>>>>>>>> it. >>>>>>>>> >>>>>>>>> I fully understand and appreciate how supervision trees are used to >>>>>>>>> restart processes if they fail. What I don't get is what to do when >>>>>>>>> you >>>>>>>>> don't want to restart but want to take over, say on another node. I >>>>>>>>> know >>>>>>>>> that at a higher-level, OTP has some take-over/fail-over schematics >>>>>>>>> (at the >>>>>>>>> application level.) I'm trying to understand things at the >>>>>>>>> processes >>>>>>>>> level - >>>>>>>>> why Erlang is the way it is so I can better use it to make my >>>>>>>>> currently >>>>>>>>> fault-intolerant program fault tolerant. >>>>>>>>> >>>>>>>>> Specifically, how can one process take over from another if it >>>>>>>>> fails? >>>>>>>>> It >>>>>>>>> appears to may that the only way to do this would be to somehow >>>>>>>>> retrieve not >>>>>>>>> only the state of the process (say, gen_server's state) but also >>>>>>>>> the >>>>>>>>> messages in its mailbox. Where does the design decision to >>>>>>>>> share-nothing for >>>>>>>>> the sake of fault-tolerance come into play for processes? Please >>>>>>>>> help >>>>>>>>> me >>>>>>>>> "get" this! >>>>>>>>> >>>>>>>>> Thanks in advance. >>>>>>>>> >>>>>>>>> - Edmond - >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>> Hi Edmond: >>>>>>> >>>>>>> Share-nothing helps with concurrent fault-tolerance by preventing one >>>>>>> process from corrupting the state of another. Receive is a process' >>>>>>> choice >>>>>>> and it corrupts its own state if it receives bad data and lets it in. >>>>>>> >>>>>>> AFAIK OTP fault-tolerance doesn't mean no requests will fail, it >>>>>>> means >>>>>>> the >>>>>>> system/sub-system will recover if a single request causes a process >>>>>>> to >>>>>>> crash. It's kind of like proper try/catch recovery applied to >>>>>>> concurrent >>>>>>> code. How you recover from the crash depends on the supervision >>>>>>> strategy >>>>>>> chosen. In some cases the supervisor can pass the state to the >>>>>>> replacement >>>>>>> process. In others this isn't necessary or even desirable since the >>>>>>> state >>>>>>> itself may involve resources lost in the crash or corrupted state >>>>>>> that >>>>>>> led >>>>>>> to the crash. >>>>>>> >>>>>>> I am straying outside my knowledge here so this paragraph is >>>>>>> guesswork. >>>>>>> The message queue for a gen_server need not necessarily be lost when >>>>>>> the >>>>>>> callback module crashes. In theory OTP could (and might already) >>>>>>> simply >>>>>>> delegate the messages to the replacement process following a crash. >>>>>>> Someone >>>>>>> who knows OTP better than me would need to weigh in here though. >>>>>>> >>>>>>> I found http://manning.com/logan very informative in understanding >>>>>>> OTP >>>>>>> and >>>>>>> its supervisor hierarchies. >>>>>>> >>>>>>> Cheers, >>>>>>> Alain >>>>>>> ________________________________________________________________ >>>>>>> erlang-questions (at) erlang.org mailing list. >>>>>>> See http://www.erlang.org/faq.html >>>>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >>>>>> >>>>>> ________________________________________________________________ >>>>>> erlang-questions (at) erlang.org mailing list. >>>>>> See http://www.erlang.org/faq.html >>>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>>>> >>>>>> >>>>>> >>>>> ________________________________________________________________ >>>>> erlang-questions (at) erlang.org mailing list. >>>>> See http://www.erlang.org/faq.html >>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>>> >>>>> >>>> >>>> -- >>>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ >>>> >>>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> >> >> > > -- > Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From serge@REDACTED Wed Jan 12 06:44:08 2011 From: serge@REDACTED (Serge Aleynikov) Date: Wed, 12 Jan 2011 00:44:08 -0500 Subject: [patch] new float_to_list/2 Message-ID: <4D2D3FA8.1000009@aleynikov.org> Attached please find a patch that adds a new float_to_list/2 BIF. The patch was created off of the master branch of https://github.com/erlang/otp. This BIF solves a problem of float_to_list/1 that doesn't allow specifying the number of digits after the decimal point when formatting floats. float_to_list(Float, Options) -> string() Float = float() Options = [Option] Option = {precision, Precision::integer()} | compact Text representation of a float formatted using given options Returns a string which corresponds to the text representation of Float using fixed decimal point formatting. When precision option is specified the returned value will contain at most Precision number of digits past the decimal point. When compact option is provided the trailing zeros at the end of the list are truncated. 1> float_to_list(7.12, [{precision, 4}]). "7.1200" 2> float_to_list(7.12, [{precision, 4}, compact]). "7.12" Documentation and test cases are updated to reflect the changes. Though I included identical support for this new bif in unix/win/vxworks I only tested it on Linux. Regards, Serge -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: float_to_list.patch URL: From boris.muehmer@REDACTED Wed Jan 12 07:00:07 2011 From: boris.muehmer@REDACTED (Boris =?ISO-8859-1?Q?M=FChmer?=) Date: Wed, 12 Jan 2011 07:00:07 +0100 Subject: [erlang-questions] hacked gmail account -> spam In-Reply-To: <79F3C0A8-9170-4EC8-8796-38564FAFBD8C@gmail.com> References: <20110111152628.GA15773@erix.ericsson.se> <79F3C0A8-9170-4EC8-8796-38564FAFBD8C@gmail.com> Message-ID: <1294812007.12584.9.camel@yang> Am Dienstag, den 11.01.2011, 14:51 -0330 schrieb Alain O'Dea: > I recently updated all my service passwords to independent 14-20 character long passwords with mixed case, numbers and symbols, etc. which I got from http://www.pctools.com/guides/password/ > > I generated them in batches of 3-5 and saved them in a keychain secure note for reference before changing the service passwords. At work, at home and all of my family use KeePass (1.x, http://keepass.info/) or its ports to Linux or Mac. The main feature of KeePass is, that the database file can be used on almost all the devices (there are even clients for some mobile phones). - boris From rzezeski@REDACTED Wed Jan 12 08:15:03 2011 From: rzezeski@REDACTED (Ryan Zezeski) Date: Wed, 12 Jan 2011 02:15:03 -0500 Subject: error: Removing (timedout) connection In-Reply-To: References: Message-ID: Update: I know that a big part of my problem was running that map-reduce query on every node at the same time. That's an expensive operation, especially since I'm using the filesystem to back my luwak cluster. With that in mind, I went ahead and migrated to riak-search and wrote a custom extractor/schema to index my luwak files. The system, at least for the last 4+ hours, has shown much more stability than what it had previously. In fact, I've only seen 4 of the timed-out connection errors thus far. Furthermore, I believe I've pinpointed the cause of these errors. All 4 of them occurred in a particularly nasty piece of code where I call ets:tab2file, then read the file into a binary, then zlib:gzip it and then finally send the compressed binary to luwak via the native Erlang client. I'm wondering about that last part, sending it via Erlang external format as a binary. Even compressed, these binaries can be as large as 120M! Would this be a potential problem, possibly delaying the net kernel ticks between nodes and causing my timeouts? I imagine using the riak protocol buffers interface might be a better choice? Thanks, -Ryan On Mon, Jan 10, 2011 at 2:20 PM, Ryan Zezeski wrote: > Hi guys/gals, > > Recently I've been converting my non-distributed Erlang app into a > distributed one and I ran into some troubles. If you want to skip straight > to the question it's at the end, but I try to give some insight into what > I'm doing below. > > First off, I attached a PDF (sorry, PDF was not my choice) which contains a > diagram I drew of the current setup. I apologize for my utter failure as an > artist. In this diagram you'll see 3 vertical partitions representing 3 > different machines and a horizontal one representing the fact that each > machine has 2 Erland nodes on it. 3 of the Erlang nodes form a riak > cluster. The other 3 are the application (or should I say release) I wrote, > and to distribute my app I utilized riak's underlying technology, riak_core > (I use it as an easy way to persist cluster membership and use the ring > metadata to store some data). These six nodes are fully connected, i.e. > each node has connection to the other. > > Occasionally, I've noticed the following message on any one of the six > nodes: > > =ERROR REPORT==== ... > ** Node not responding ** > ** Removing (timedout) connection ** > > Furthermore, using net_kernel:monitor_nodes(true, [nodedown_reason]) I've > noticed messages like the following: > > {nodedown, , [{nodedown_reason, connection_closed}]} > > > You'll notice there is a system process running on machine A, and it makes > a gen_server:cast to three processes to do some work, and these processes > each call link (L). Each of these three (gen_server) processes makes a call > (at roughly the same time) to the riak cluster performing the _same exact_ > map/reduce job. Sometimes I'll see errors where this map/reduce job times > out on one of the nodes. So at lunch, I wondered, is it because there is > just too much communication going on between the nodes that the kernel ticks > are getting lost or delayed? I wondered if each node was using the same TCP > connection to talk to every other node. That could explain my symptoms, > right? A few netcats later and I realized that it's a dedicated conn for > each node, so that theory was blown. However, I still think that many msgs > being passed back and forth could be the cause of the problem, and I > wondered if it blocks the VM in some way so that the kernel tick can't get > through? > > > Q: Can a chatty cluster cause the kernel ticks to be lost/delayed thus > causing nodes to disconnect from each other? > > Thanks, > > -Ryan > From muharem@REDACTED Wed Jan 12 08:22:33 2011 From: muharem@REDACTED (Muharem Hrnjadovic) Date: Wed, 12 Jan 2011 08:22:33 +0100 Subject: [erlang-questions] We need a better marketing division :-) In-Reply-To: <24250519.271261294767163150.JavaMail.root@zimbra> References: <24250519.271261294767163150.JavaMail.root@zimbra> Message-ID: <4D2D56B9.90103@lbox.cc> On 01/11/2011 06:32 PM, Robert Virding wrote: > http://lambda-the-ultimate.org/node/4178 > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED Erlang definitely needs more and better marketing. The research grant the scala crowd won [1] is supposed to tackle the "Popular Parallel Programming" challenge. The first question in my mind after reading [1] was: "Have these good folks never heard of erlang?" Apparently not, or not enough. In a sense erlang is reminiscent of the debian linux distribution: it's rock solid, proven and works well for almost all work loads. However, all the excitement and the hype is on the cool new(?) kids that fork off debian (e.g. ubuntu). [1] http://www.scala-lang.org/node/8579 Best regards/Mit freundlichen Gr??en -- Muharem Hrnjadovic Public key id : B2BBFCFC Key fingerprint : A5A3 CC67 2B87 D641 103F 5602 219F 6B60 B2BB FCFC -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 900 bytes Desc: OpenPGP digital signature URL: From muharem@REDACTED Wed Jan 12 08:29:18 2011 From: muharem@REDACTED (Muharem Hrnjadovic) Date: Wed, 12 Jan 2011 08:29:18 +0100 Subject: [erlang-questions] We need a better marketing division :-) In-Reply-To: <4D2D56B9.90103@lbox.cc> References: <24250519.271261294767163150.JavaMail.root@zimbra> <4D2D56B9.90103@lbox.cc> Message-ID: <4D2D584E.60202@lbox.cc> On 01/12/2011 08:22 AM, Muharem Hrnjadovic wrote: > On 01/11/2011 06:32 PM, Robert Virding wrote: >> http://lambda-the-ultimate.org/node/4178 >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > Erlang definitely needs more and better marketing. The research grant > the scala crowd won [1] is supposed to tackle the "Popular Parallel > Programming" challenge. > The first question in my mind after reading [1] was: "Have these good > folks never heard of erlang?" Apparently not, or not enough. > > In a sense erlang is reminiscent of the debian linux distribution: it's > rock solid, proven and works well for almost all work loads. However, > all the excitement and the hype is on the cool new(?) kids that fork off > debian (e.g. ubuntu). > > [1] http://www.scala-lang.org/node/8579 Oh, and I forgot to mention: I was playing with scala for a month or so and it is *grotesquely* complex IMHO. Best regards/Mit freundlichen Gr??en -- Muharem Hrnjadovic Public key id : B2BBFCFC Key fingerprint : A5A3 CC67 2B87 D641 103F 5602 219F 6B60 B2BB FCFC -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 900 bytes Desc: OpenPGP digital signature URL: From mk@REDACTED Wed Jan 12 08:56:07 2011 From: mk@REDACTED (Morten Krogh) Date: Wed, 12 Jan 2011 08:56:07 +0100 Subject: [erlang-questions] Re: error: Removing (timedout) connection In-Reply-To: References: Message-ID: <4D2D5E97.1000007@amberbio.com> Ryan, see the thread "node to node message passing" from September last year. Morten. On 1/12/11 8:15 AM, Ryan Zezeski wrote: > Update: > > I know that a big part of my problem was running that map-reduce query on > every node at the same time. That's an expensive operation, especially > since I'm using the filesystem to back my luwak cluster. With that in mind, > I went ahead and migrated to riak-search and wrote a custom extractor/schema > to index my luwak files. The system, at least for the last 4+ hours, has > shown much more stability than what it had previously. In fact, I've only > seen 4 of the timed-out connection errors thus far. Furthermore, I believe > I've pinpointed the cause of these errors. All 4 of them occurred in a > particularly nasty piece of code where I call ets:tab2file, then read the > file into a binary, then zlib:gzip it and then finally send the compressed > binary to luwak via the native Erlang client. > > I'm wondering about that last part, sending it via Erlang external format as > a binary. Even compressed, these binaries can be as large as 120M! Would > this be a potential problem, possibly delaying the net kernel ticks between > nodes and causing my timeouts? I imagine using the riak protocol buffers > interface might be a better choice? > > Thanks, > > -Ryan > > On Mon, Jan 10, 2011 at 2:20 PM, Ryan Zezeski wrote: > >> Hi guys/gals, >> >> Recently I've been converting my non-distributed Erlang app into a >> distributed one and I ran into some troubles. If you want to skip straight >> to the question it's at the end, but I try to give some insight into what >> I'm doing below. >> >> First off, I attached a PDF (sorry, PDF was not my choice) which contains a >> diagram I drew of the current setup. I apologize for my utter failure as an >> artist. In this diagram you'll see 3 vertical partitions representing 3 >> different machines and a horizontal one representing the fact that each >> machine has 2 Erland nodes on it. 3 of the Erlang nodes form a riak >> cluster. The other 3 are the application (or should I say release) I wrote, >> and to distribute my app I utilized riak's underlying technology, riak_core >> (I use it as an easy way to persist cluster membership and use the ring >> metadata to store some data). These six nodes are fully connected, i.e. >> each node has connection to the other. >> >> Occasionally, I've noticed the following message on any one of the six >> nodes: >> >> =ERROR REPORT==== ... >> ** Node not responding ** >> ** Removing (timedout) connection ** >> >> Furthermore, using net_kernel:monitor_nodes(true, [nodedown_reason]) I've >> noticed messages like the following: >> >> {nodedown,, [{nodedown_reason, connection_closed}]} >> >> >> You'll notice there is a system process running on machine A, and it makes >> a gen_server:cast to three processes to do some work, and these processes >> each call link (L). Each of these three (gen_server) processes makes a call >> (at roughly the same time) to the riak cluster performing the _same exact_ >> map/reduce job. Sometimes I'll see errors where this map/reduce job times >> out on one of the nodes. So at lunch, I wondered, is it because there is >> just too much communication going on between the nodes that the kernel ticks >> are getting lost or delayed? I wondered if each node was using the same TCP >> connection to talk to every other node. That could explain my symptoms, >> right? A few netcats later and I realized that it's a dedicated conn for >> each node, so that theory was blown. However, I still think that many msgs >> being passed back and forth could be the cause of the problem, and I >> wondered if it blocks the VM in some way so that the kernel tick can't get >> through? >> >> >> Q: Can a chatty cluster cause the kernel ticks to be lost/delayed thus >> causing nodes to disconnect from each other? >> >> Thanks, >> >> -Ryan >> From karol.skocik@REDACTED Wed Jan 12 09:43:38 2011 From: karol.skocik@REDACTED (karol skocik) Date: Wed, 12 Jan 2011 09:43:38 +0100 Subject: [erlang-questions] hacked gmail account -> spam In-Reply-To: <1294812007.12584.9.camel@yang> References: <20110111152628.GA15773@erix.ericsson.se> <79F3C0A8-9170-4EC8-8796-38564FAFBD8C@gmail.com> <1294812007.12584.9.camel@yang> Message-ID: Thank you both for suggestions. Karol On Wed, Jan 12, 2011 at 7:00 AM, Boris M?hmer wrote: > Am Dienstag, den 11.01.2011, 14:51 -0330 schrieb Alain O'Dea: >> I recently updated all my service passwords to independent 14-20 character long passwords with mixed case, numbers and symbols, etc. which I got from http://www.pctools.com/guides/password/ >> >> I generated them in batches of 3-5 and saved them in a keychain secure note for reference before changing the service passwords. > > At work, at home and all of my family use KeePass (1.x, > http://keepass.info/) or its ports to Linux or Mac. > > The main feature of KeePass is, that the database file can be used on > almost all the devices (there are even clients for some mobile phones). > > > ?- boris > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From arnaud74130@REDACTED Wed Jan 12 10:08:55 2011 From: arnaud74130@REDACTED (Arnaud GARCIA) Date: Wed, 12 Jan 2011 10:08:55 +0100 Subject: List comprehensions challenge ;-) Message-ID: Hello, Just playing with list comprehensions... I was asking if it possible to do it without lists:sum, by using an anonymous function and add it directly in the list comprehensions ? 26> [A+B || A <-[1,2,3],B <- [7,8,9,10,11]]. [8,9,10,11,12,9,10,11,12,13,10,11,12,13,14] 30> lists:sum([A+B || A <-[1,2,3],B <- [7,8,9,10,11]]). 165 Any idea ? Thanks Arnaud From demeshchuk@REDACTED Wed Jan 12 10:15:19 2011 From: demeshchuk@REDACTED (Dmitry Demeshchuk) Date: Wed, 12 Jan 2011 12:15:19 +0300 Subject: [erlang-questions] List comprehensions challenge ;-) In-Reply-To: References: Message-ID: List comprehensions are a combination of lists:map/2 and lists:filter/2. But lists:sum/1 is a lists:foldl/3 type, this cannot be handled with list comprehensions. On Wed, Jan 12, 2011 at 12:08 PM, Arnaud GARCIA wrote: > Hello, > > Just playing with list comprehensions... > > I was asking if it possible to do it without lists:sum, by using an > anonymous function and add it directly in the list comprehensions ? > > 26> [A+B || A <-[1,2,3],B <- [7,8,9,10,11]]. > [8,9,10,11,12,9,10,11,12,13,10,11,12,13,14] > 30> lists:sum([A+B || A <-[1,2,3],B <- [7,8,9,10,11]]). > 165 > > Any idea ? > > Thanks > > Arnaud > -- Best regards, Dmitry Demeshchuk From b.ghose@REDACTED Wed Jan 12 10:20:04 2011 From: b.ghose@REDACTED (Baishampayan Ghose) Date: Wed, 12 Jan 2011 14:50:04 +0530 Subject: [erlang-questions] List comprehensions challenge ;-) In-Reply-To: References: Message-ID: > Just playing with list comprehensions... > > I was asking if it possible to do it without lists:sum, by using an > anonymous function and add it directly in the list comprehensions ? > > 26> [A+B || A <-[1,2,3],B <- [7,8,9,10,11]]. > [8,9,10,11,12,9,10,11,12,13,10,11,12,13,14] > 30> lists:sum([A+B || A <-[1,2,3],B <- [7,8,9,10,11]]). > 165 I don't think it's possible since list comprehensions will always return a list of items. The lists:sum approach is correct, but you can also try out lists:foldl/3 for more fun (pun intended). 42> lists:foldl(fun (X, Y) -> X + Y end, 0, [A+B || A <-[1,2,3],B <- [7,8,9,10,11]]). 165 Regards, BG -- Baishampayan Ghose b.ghose at gmail.com From olopierpa@REDACTED Wed Jan 12 10:26:38 2011 From: olopierpa@REDACTED (Pierpaolo Bernardi) Date: Wed, 12 Jan 2011 10:26:38 +0100 Subject: [erlang-patches] [patch] new float_to_list/2 In-Reply-To: <4D2D3FA8.1000009@aleynikov.org> References: <4D2D3FA8.1000009@aleynikov.org> Message-ID: On Wed, Jan 12, 2011 at 06:44, Serge Aleynikov wrote: > Attached please find a patch that adds a new float_to_list/2 BIF. ?The patch > was created off of the master branch of https://github.com/erlang/otp. > > This BIF solves a problem of float_to_list/1 that doesn't allow specifying > the number of digits after the decimal point when formatting floats. > > ? ? ? ?float_to_list(Float, Options) -> string() > > ? ? ? ?Float = float() > ? ? ? ?Options = [Option] > ? ? ? ?Option = {precision, Precision::integer()} | compact > > ? ? ? ?Text representation of a float formatted using given options > > ? ? ? ?Returns a string which corresponds to the text > ? ? ? ?representation of Float using fixed decimal point formatting. > ? ? ? ?When precision option is specified > ? ? ? ?the returned value will contain at most Precision number of > ? ? ? ?digits past the decimal point. ?When compact option is provided > ? ? ? ?the trailing zeros at the end of the list are truncated. I think the option is misnamed. In the usual terminology, 'precision' is the total number of significative digits, not only the ones past the decimal point. Cheers P. From ulf.wiger@REDACTED Wed Jan 12 10:49:59 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Wed, 12 Jan 2011 10:49:59 +0100 Subject: [erlang-questions] We need a better marketing division :-) In-Reply-To: <4D2D56B9.90103@lbox.cc> References: <24250519.271261294767163150.JavaMail.root@zimbra> <4D2D56B9.90103@lbox.cc> Message-ID: On 12 Jan 2011, at 08:22, Muharem Hrnjadovic wrote: > > Erlang definitely needs more and better marketing. The research grant > the scala crowd won [1] is supposed to tackle the "Popular Parallel > Programming" challenge. > The first question in my mind after reading [1] was: "Have these good > folks never heard of erlang?" Apparently not, or not enough. ?or Haskell, perhaps. :) The proposal in question means to design a DSL for large-scale "embarrassingly parallel" computing. This is not really a domain where Erlang is the obvious choice (although it wouldn't necessarily be a very bad one). Haskell, OTOH, has some really great facilities for creating DSLs and also some pretty advanced research on data parallelism. Then again, there is definitely room for more than one initiative, and there are also exciting developments on the Erlang front in addressing the scalability challenge. Some, I am hoping to be able to talk about in the future; others are beginning to become public. One thing that comes to mind is work done by the USAF Cognitive Modelling Group in Scottsdale, AZ on using Erlang for very-large- scale cognitive modeling. Part of the work has been published http://iccm2010.cs.drexel.edu/proceedings/papers/Douglass.pdf This work has turned some heads in their community, not least parts of the work that draw on Erlang's FSM power, for simulating adaptive and learning processes, essentially re-programming state machines on the fly (which is fairly trivial in Erlang, but not necessarily so in other languages). One thing that caught my eye in the Extended Synopsis for the Scala project was this: "The challenge is hard to meet because concurrent and parallel programming are fundamentally difficult. The conventional view is that one is left with two choices. The first possibility is to have programs manage their degree of concurrency explicitly through threads or processes. This results in a state-space explosion which tends to overwhelm the capability to understand software?s behavior, even if standard locks are replaced by some higher-level synchronization mechanism such as trans- actions or messages." Perhaps I'm reading too much into it, but as many of you know, I have been crusading a bit around the "state-space explosion" problem, arguing that it's not messaging per-se, but the event-handling semantics (lack of selective event handling) that explodes the state space. I'd say that we have plenty of empirical evidence that Erlang is great for keeping the state space from growing out of control. The interesting thing is that Scala *does* support erlang-style concurrency, to a greater extent than most languages. Why, then, would they say this? I am reminded of the first draft of the Scala manual that I ever came across. It illustrated how to do Erlang-style concurrency in the Introduction, but when I turned to the Concurrency chapter, it was all about Java-style concurrency. Not a word about Erlang-style concurrency or selective message reception. Perhaps this boils down to choosing your paradigm and sticking to it? While you can whip up a demo and proclaim "we can also do Erlang-style concurrency", there is really much more to it than that. It would be hard to imagine a synopsis for an erlang-oriented research project claiming that concurrent programming is fundamentally difficult and message passing leads to state-space explosion - about as hard as imagining an erlang-oriented proposal aiming at advancing the state of the art in OO modeling (even though you *can* do OO in Erlang too). The intersection between language, libraries, prominent applications and community somehow define the core set of concepts. For Erlang, lightweight concurrency, fault-tolerance and powerful message passing are all right smack in the core, and thus permeate everything we do, for better and for worse. BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From jxm@REDACTED Wed Jan 12 10:55:38 2011 From: jxm@REDACTED (Jerome Martin) Date: Wed, 12 Jan 2011 10:55:38 +0100 Subject: [erlang-questions] We need a better marketing division :-) In-Reply-To: <4D2D56B9.90103@lbox.cc> References: <24250519.271261294767163150.JavaMail.root@zimbra> <4D2D56B9.90103@lbox.cc> Message-ID: Hi everyone! I am really new to the Erlang world, just finished reading the Programming Erlang Book and Joe Armstrong thesis, probably still having done less than 500 LOCs in Erlang. So as coming straight from the "outside world" I figured I must be your average sample victim of "languages marketing". So here is what : 1) I am working on an application aiming to provide high availability (nodes failover mostly), multinode provisioning facilities and a unified administrative view of a cluster of nodes. I already have a reference implementation, and tried to leverage as many OSS tools already written (do not reinvent the wheel), but came to the conclusion (skipping the details) that for my needs rewriting a membership and communications layer was required. 2) After the first reference implementation, including both HA logic and distributed algorithms (think distributed min. weight spanning tree, etc.) went on the internet looking for how other people approach the various questions/uncertainties about critical design points: - IPC communications (currently using synchronous RPC-ish for my real time, user-interactive nodes configuration needs, but GHS class algorithms actually fare better with messages queues). - Error handling (currently encapsulate remote exceptions in a special exception class that allows RPC callers to access it) - Scalability (This is a hard one) - Concurrent data access (started in a nothing-shared fashion, converted for efficiency and perfs to locked and shared structures, now seing that this is a dead end). So bottom-line, I spent the time to experiment end "re-discover" possible mistakes and implementation issues. Then looked on the internet how others handle that, using the keywords matching the topics above, reading articles, watching presentations and skimming blogs and found either: - Claims of silver bullet (makes me run away as fast as I can), usually nicely marketed. - Very interesting but lab-only theory, usually nicely marketed too. - Flawed approaches, claiming many things but apparently ignoring decades of CS research (are people actually reading papers in CS anymore ?), but using "industry standard" companion techs as a justification (as in bad OO, etc.) The only approach that was pragmatic, seemed to works and was coherent (despite raising questions and obviously not being a silver bullet) that I've found is Erlang. I would be very interested in understanding what kind of approach leads one to pick an other solution (apart from writing one's own), methodology or set of tools for a down-to-earth project. So yes, marketing is _not_ Erlang strength. But maybe this is what makes people like me (BTW, a long-time debian-only user for that matter ;-) ) not run away at first glance :-) And TBH, I think that people having done enough research on their own to understand what Erlang solves do not need marketing, and that the other are probably just not worth worrying about, *let them use caffeinated-paradigmed languages and platforms in a pool of quicksand-filled industry standards and hope that natural selection will provide us with a better species of CS/IT "professionals"*, one that at least caught up with the problems solved 2 or 3 decades ago :-) On Wed, Jan 12, 2011 at 8:22 AM, Muharem Hrnjadovic wrote: > On 01/11/2011 06:32 PM, Robert Virding wrote: > > http://lambda-the-ultimate.org/node/4178 > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > Erlang definitely needs more and better marketing. The research grant > the scala crowd won [1] is supposed to tackle the "Popular Parallel > Programming" challenge. > The first question in my mind after reading [1] was: "Have these good > folks never heard of erlang?" Apparently not, or not enough. > > In a sense erlang is reminiscent of the debian linux distribution: it's > rock solid, proven and works well for almost all work loads. However, > all the excitement and the hype is on the cool new(?) kids that fork off > debian (e.g. ubuntu). > > [1] http://www.scala-lang.org/node/8579 > > Best regards/Mit freundlichen Gr??en > > -- > Muharem Hrnjadovic > Public key id : B2BBFCFC > Key fingerprint : A5A3 CC67 2B87 D641 103F 5602 219F 6B60 B2BB FCFC > > -- J?r?me Martin From clist@REDACTED Wed Jan 12 10:58:26 2011 From: clist@REDACTED (Angel Alvarez) Date: Wed, 12 Jan 2011 10:58:26 +0100 Subject: [erlang-questions] Processes & Fault Tolerance In-Reply-To: References: Message-ID: <201101121058.26612.clist@uah.es> The entire list is Gold!! While not having too much time for erlang issues i keep reading the list everyday. You usually learn a lot of great things here... I recommend this list even for non erlangers... OTOH this is ciclic topic, the OTP picture gets reviewed in the list form time to time and multicore always leads to some philosophic and interesting threads... Regards, Angel El Mi?rcoles, 12 de Enero de 2011 05:03:12 Ryan Zezeski escribi?: > This thread is pure GOLD. I know that all this stuff is covered > in various places but seeing it compressed in this thread was really > profound...for me anyways. I'm to keep this in my pocket for the > presentation I plan to give later in the year. > > -Ryan > > On Thu, Jan 6, 2011 at 1:32 AM, Edmond Begumisa > wrote: > > > UPDATE: > > > > I'm excited! I've just added hot take-over to my server program (for > > specific jobs). I've watched one node take-over the handling of a request > > from another as it's network connection died. The client didn't notice. Very > > eerie to witness, and quite impressive how easy it was to add that > > functionality if like me, you've come from other languages and imagined the > > pain involved in doing this. > > > > Using Joe's illustration, I was able to do this without the functionality > > of OTP supervisors, application take-over/fail-over, etc (even though my > > application is an OTP application.) Just using redundant gen_server worker > > processes watching each-other and updating each-other on state changes + new > > requests. > > > > To summerise the important lessons I've learned from all who helped: In > > regards to processes, fault-tolerance and the share-nothing principle... > > > > * We can say: Share nothing because we want to replicate valid state which > > leads to one form of fault tolerance (a failed job can be done elsewhere). > > * Or we can say: Share nothing because we want to isolate invalid state > > which leads to another form fault tolerance (a failed job won't affect the > > rest of the system). > > * For job hot take-over + redundancy, one should think in terms of the > > former (you need two or more nodes for this). > > * For worker supervision + restarts, one should think in terms of the > > latter (you can use one or more nodes for this). > > > > Supervisors monitor workers. Workers do jobs. In-case of faults: workers > > can be restarted but also **jobs can be taken over by one redundant worker > > from another**. This is powerful, Erlang is pretty cool :) > > > > I'm gonna go marvel as I plug and unplug my network cables now :P > > > > Thanks again to all. > > > > - Edmond - > > > > > > > > On Thu, 06 Jan 2011 02:17:39 +1100, Edmond Begumisa < > > ebegumisa@REDACTED> wrote: > > > > On Tue, 04 Jan 2011 20:52:50 +1100, Joe Armstrong > >> wrote: > >> > >> On Mon, Jan 3, 2011 at 5:30 PM, Edmond Begumisa > >>> wrote: > >>> > >>>> Let's start with how we do error recovery. > >>>>> > >>>>> Imagine two linked processes A and B on separate machines. A is the > >>>>> master process. > >>>>> B is a passive processes that will take over if A fails. > >>>>> > >>>>> A sends a stream of state update messages S1, S2, S3,... to B. The > >>>>> state messages contain enough information > >>>>> for B to do whatever A was doing should A fail. If A fails B will > >>>>> receive an EXIT signal. > >>>>> > >>>>> If B does receive an exit signal it knows A has failed, so it carries > >>>>> on doing whatever A was doing > >>>>> using the information it last received in a state update message. > >>>>> > >>>>> That's it and all about it - nothing to do with supervisors etc, > >>>>> > >>>> > >>>> Ooooohh! THANK YOU, THANK YOU, THANK YOU. > >>>> > >>> > >>> Thanks. > >>> > >>> Another problem in understand is that we assume the meaning of words. > >>> > >>> > >> Yes. There was some confusion about whether we were all talking about > >> replicating valid state or isolating invalid state. Which I guess > >> "fault-tolerance" could be interpreted to mean either. But I'm no expert on > >> the subject so I shouldn't comment on terminology -- I just knew the feature > >> I wanted and was pretty sure it could be done at the process level, I just > >> had no-idea how! From now on, I'll stick to the term "redundancy" or Ulf's > >> "hot take-over" so it's clearer what I mean. > >> > >> - Edmond - > >> > >> > >> Once upon a time I was talking to a guy and used the word "fault > >>> tolerance" > >>> we talked for hours and thought we understood what this word meant. Then > >>> he said something that implied he was building a fault-tolerant system > >>> on one computer. > >>> I said it was impossible. He said, "you catch and handle all the > >>> exceptions .." > >>> > >>> I said, "The entire computer might crash" > >>> > >>> He said, "Oh" > >>> > >>> There was a long silence. > >>> > >>> I told him about Erlang ... > >>> > >>> > >>> /Joe > >>> > >>> > >>>> *Now* I get it. This is precisely what I was trying to understand. The > >>>> missing link was sending redundant messages. It all makes sense now -- > >>>> it's > >>>> so simple. All I have to do to get fault tolerance at the process level > >>>> is > >>>> have a group of N redundant processes waiting for exit-signals and > >>>> forward > >>>> state changes to N-1 of them. I could even send to N/2 and have some > >>>> decent > >>>> tolerance at the expense of consistency of the nodes. > >>>> > >>>> To preemptively answer the question I sent Ulf and Mazen... for that > >>>> distributed database, Process B would be set up to also receive read > >>>> requests forwarded from Process A but not respond to them unless it gets > >>>> an > >>>> exit signal it can understand like disk_fail_error. Any changes in state > >>>> would also be forwarded to existing redundant processes. > >>>> > >>>> Thanks everyone for your patience in helping me understand > >>>> fault-tolerance. > >>>> I've already started thinking about some cool things I can do with this > >>>> in > >>>> my program. > >>>> > >>>> - Edmond - > >>>> > >>>> > >>>> > >>>> On Tue, 04 Jan 2011 00:36:30 +1100, Joe Armstrong > >>>> wrote: > >>>> > >>>> I think you are getting confused between OTP supervisors etc. and the > >>>>> general notion of "take-over" > >>>>> > >>>>> Let's start with how we do error recovery. > >>>>> > >>>>> Imagine two linked processes A and B on separate machines. A is the > >>>>> master process. > >>>>> B is a passive processes that will take over if A fails. > >>>>> > >>>>> A sends a stream of state update messages S1, S2, S3,... to B. The > >>>>> state messages contain enough information > >>>>> for B to do whatever A was doing should A fail. If A fails B will > >>>>> receive an EXIT signal. > >>>>> > >>>>> If B does receive an exit signal it knows A has failed, so it carries > >>>>> on doing whatever A was doing > >>>>> using the information it last received in a state update message. > >>>>> > >>>>> That's it and all about it - nothing to do with supervisors etc, > >>>>> > >>>>> > >>>>> To make the above possible we need a couple of primitives in Erlang, > >>>>> namely process_flag trap_exits, and spawn_link. That's all you need. > >>>>> > >>>>> The basic system properties you need to make a fault-tolerant systems > >>>>> are the abilities to detect remote errors and send and receive > >>>>> messages. > >>>>> > >>>>> What the OTP supervisors etc. do are build layers of abstraction on > >>>>> top of spawn-link and trap_exits > >>>>> in order to simplify programming common use-cases. > >>>>> > >>>>> Fault detection and recovery is not arbitrary or magic in any sense, > >>>>> It has to be part of the system architecture like everything else. One > >>>>> common fault in making systems is to specify and design for the normal > >>>>> behavior > >>>>> but not specify and design how fault-detection and correction work. > >>>>> This is probably due to a legacy of > >>>>> programming in languages that have very limited ability to detect > >>>>> errors and recover from them. > >>>>> > >>>>> A C programmer with a single thread of control will take extreme > >>>>> measures to avoid crashing their program > >>>>> they have to, there is only one thread of control. Given multiple > >>>>> threads of control and the ability to remotely > >>>>> detect errors your entire way of thinking changes and the emphasis > >>>>> changes from thinking "how to I avoid a crash" > >>>>> to thinking "given that a crash has occurred and been detected, how to > >>>>> I fix things up and carry on" > >>>>> > >>>>> The Erlang philosophy is that "things will crash", so we have to > >>>>> detect the crashes, fix the bit that broke and > >>>>> carry on - we do not make excessive efforts to avoid crashes, but we > >>>>> do try very hard to repair things after they have gone wrong. > >>>>> > >>>>> Fixing things after they have broken is often easier than preventing > >>>>> them from breaking. Preventing them > >>>>> from breaking is not theoretically possible - we can't even prove > >>>>> simple things, like that a program will terminate > >>>>> (the famous halting problem) - but we can easily detect failures and > >>>>> put the system to rights by applying > >>>>> certain invariants. > >>>>> > >>>>> OTP supervisors are just trees of processes with certain restart rules > >>>>> that experience has shown us > >>>>> to be useful. > >>>>> > >>>>> The main value of supervisors etc. (and of all the OTP behaviors) is > >>>>> in building large systems with many > >>>>> programmers involved. Rather than inventing their own patterns, the > >>>>> programmers reuse a common set of > >>>>> patterns, which makes the projects easier to manage. > >>>>> > >>>>> /Joe > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> On Mon, Jan 3, 2011 at 6:21 AM, Edmond Begumisa > >>>>> wrote: > >>>>> > >>>>>> > >>>>>> Thanks for your response. > >>>>>> > >>>>>> Firstly, let me make my question a little clearer... > >>>>>> > >>>>>> To rephrase: For processes, "share nothing for the sake of > >>>>>> concurrency" - > >>>>>> I > >>>>>> get, both in concept and application. "Share nothing for the sake of > >>>>>> fault-tolerance" - I get in concept but not in application. > >>>>>> > >>>>>> Yet as I understand it, it is for the latter reason Erlang shares > >>>>>> nothing* > >>>>>> and not the former. Interpretation: I must be completely missing the > >>>>>> point > >>>>>> in regards to Erlang processes and sharing nothing. This is what I > >>>>>> want > >>>>>> to > >>>>>> understand in application. In addition to the "side" effect of sane > >>>>>> concurrency (which coming from a chaotic multi-threading shared-memory > >>>>>> world > >>>>>> I fully appreciate and practically make use of everyday), how can I > >>>>>> also > >>>>>> make use of the "real" reason Erlang processes share nothing -- fault > >>>>>> tolerance? Practically/illustratively speaking? > >>>>>> > >>>>>> *ETS being the obvious exception. > >>>>>> > >>>>>> Secondly, here's a mantra from Joe Armstrong... > >>>>>> > >>>>>> @ minute 17:26 > >>>>>> http://www.se-radio.net/2008/03/episode-89-joe-armstrong-on-erlang/ > >>>>>> > >>>>>> "[message passing concurrency]... the original reasons have to do with > >>>>>> fault > >>>>>> tolerance... you have to copy all the data you need from computer 1 to > >>>>>> computer 2... if computer 1 crashes you take over on computer 2... you > >>>>>> can't > >>>>>> have dangling pointers... that's the reason for copying everything... > >>>>>> it's > >>>>>> got nothing to do with concurrency, it's got a lot to do with > >>>>>> fault-tolerance... if they don't crash you could just have a dangling > >>>>>> pointer and copy less data but it won't work in the presence of > >>>>>> errors..." > >>>>>> > >>>>>> I interpret this to mean that share-nothing between processes is more > >>>>>> about > >>>>>> replicating valid state than isolating corrupted state as you > >>>>>> described. > >>>>>> > >>>>>> Indeed, Joe created an example on his blog... > >>>>>> > >>>>>> > >>>>>> > >>>>>> http://armstrongonsoftware.blogspot.com/2007/07/scalable-fault-tolerant-upgradable.html > >>>>>> > >>>>>> It's algorithm 3 there I'm struggling with. Particularly where he > >>>>>> says... > >>>>>> > >>>>>> "... In practise we would send an asynchronous stream of messages from > >>>>>> N > >>>>>> to > >>>>>> N+1 containing enough information to recover if things go wrong." > >>>>>> > >>>>>> Unfortunately, I couldn't find part II to that post (I don't think > >>>>>> there > >>>>>> is > >>>>>> one.) And I'm too green and inexperienced in the field of > >>>>>> fault-tolerant > >>>>>> systems to figure it out on my own. I'm having trouble visualising the > >>>>>> practical here from the conceptual -- I need to be shown how :( > >>>>>> > >>>>>> Also, I seem to be under the impression that the Erlang language has > >>>>>> some > >>>>>> sort of schematics to do this built-in (i.e. deal with one process > >>>>>> taking > >>>>>> over from another if the first fails) and this is the reason processes > >>>>>> share > >>>>>> nothing. This seems to me to be something different from supervision > >>>>>> trees, > >>>>>> which use exit-trapping to re-spawn if a process fails with the active > >>>>>> 'job' > >>>>>> disappearing and any errors logged (like restarting a daemon). My > >>>>>> interpretation of the fault-tolerance Erlang is supposed to enable > >>>>>> (for > >>>>>> those in the know) is seamless take-over. The 'job' lives on but > >>>>>> elsewhere. > >>>>>> > >>>>>> Using telecoms as an example: a phone call wouldn't be cut-off when a > >>>>>> fault > >>>>>> occurs, another node would seamlessly take over. This is how I > >>>>>> interpreted > >>>>>> Joe's post and other descriptions of Erlang's fault-tolerant features > >>>>>> and > >>>>>> I > >>>>>> understand the key is in the share nothing policy for processes. I'm > >>>>>> sure > >>>>>> I've mis-understood something or everything :) > >>>>>> > >>>>>> - Edmond - > >>>>>> > >>>>>> PS: I've read the Manning draft. Great book. I don't know if the > >>>>>> answer > >>>>>> lies > >>>>>> in OTP (I searched and didn't find it). I suspect it's lower -- > >>>>>> probalby > >>>>>> how > >>>>>> you organise your processes. Some distributed-programming black-magic > >>>>>> only > >>>>>> Erlanger's know about :) > >>>>>> > >>>>>> > >>>>>> On Mon, 03 Jan 2011 14:56:51 +1100, Alain O'Dea >>>>>> > > >>>>>> wrote: > >>>>>> > >>>>>> On 2011-01-02, at 22:36, "Edmond Begumisa" < > >>>>>>> ebegumisa@REDACTED> > >>>>>>> wrote: > >>>>>>> > >>>>>>> Slight correction... > >>>>>>>> > >>>>>>>> On Mon, 03 Jan 2011 12:38:38 +1100, Edmond Begumisa > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>> Hello all, > >>>>>>>>> > >>>>>>>>> I've been trying to wrap my Erlang's fault tolerant features > >>>>>>>>> particularly in relation to processes. > >>>>>>>>> > >>>>>>>>> > >>>>>>>> Should be: I've been trying to wrap my head around Erlang's fault > >>>>>>>> tolerant features particularly in relation to processes. > >>>>>>>> > >>>>>>>> Sorry. > >>>>>>>> > >>>>>>>> I've heard/read repeatedly that the primary reason why Erlang's > >>>>>>>>> designers opted for a share-nothing policy is not rooted in > >>>>>>>>> concurrency but > >>>>>>>>> rather in fault-tolerance. When nothing is shared, everything is > >>>>>>>>> copied. > >>>>>>>>> When everything is copied processes can take over from one another > >>>>>>>>> when > >>>>>>>>> things fail. I follow this reasoning but I don't follow how to > >>>>>>>>> apply > >>>>>>>>> it. > >>>>>>>>> > >>>>>>>>> I fully understand and appreciate how supervision trees are used to > >>>>>>>>> restart processes if they fail. What I don't get is what to do when > >>>>>>>>> you > >>>>>>>>> don't want to restart but want to take over, say on another node. I > >>>>>>>>> know > >>>>>>>>> that at a higher-level, OTP has some take-over/fail-over schematics > >>>>>>>>> (at the > >>>>>>>>> application level.) I'm trying to understand things at the > >>>>>>>>> processes > >>>>>>>>> level - > >>>>>>>>> why Erlang is the way it is so I can better use it to make my > >>>>>>>>> currently > >>>>>>>>> fault-intolerant program fault tolerant. > >>>>>>>>> > >>>>>>>>> Specifically, how can one process take over from another if it > >>>>>>>>> fails? > >>>>>>>>> It > >>>>>>>>> appears to may that the only way to do this would be to somehow > >>>>>>>>> retrieve not > >>>>>>>>> only the state of the process (say, gen_server's state) but also > >>>>>>>>> the > >>>>>>>>> messages in its mailbox. Where does the design decision to > >>>>>>>>> share-nothing for > >>>>>>>>> the sake of fault-tolerance come into play for processes? Please > >>>>>>>>> help > >>>>>>>>> me > >>>>>>>>> "get" this! > >>>>>>>>> > >>>>>>>>> Thanks in advance. > >>>>>>>>> > >>>>>>>>> - Edmond - > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > >>>>>>> Hi Edmond: > >>>>>>> > >>>>>>> Share-nothing helps with concurrent fault-tolerance by preventing one > >>>>>>> process from corrupting the state of another. Receive is a process' > >>>>>>> choice > >>>>>>> and it corrupts its own state if it receives bad data and lets it in. > >>>>>>> > >>>>>>> AFAIK OTP fault-tolerance doesn't mean no requests will fail, it > >>>>>>> means > >>>>>>> the > >>>>>>> system/sub-system will recover if a single request causes a process > >>>>>>> to > >>>>>>> crash. It's kind of like proper try/catch recovery applied to > >>>>>>> concurrent > >>>>>>> code. How you recover from the crash depends on the supervision > >>>>>>> strategy > >>>>>>> chosen. In some cases the supervisor can pass the state to the > >>>>>>> replacement > >>>>>>> process. In others this isn't necessary or even desirable since the > >>>>>>> state > >>>>>>> itself may involve resources lost in the crash or corrupted state > >>>>>>> that > >>>>>>> led > >>>>>>> to the crash. > >>>>>>> > >>>>>>> I am straying outside my knowledge here so this paragraph is > >>>>>>> guesswork. > >>>>>>> The message queue for a gen_server need not necessarily be lost when > >>>>>>> the > >>>>>>> callback module crashes. In theory OTP could (and might already) > >>>>>>> simply > >>>>>>> delegate the messages to the replacement process following a crash. > >>>>>>> Someone > >>>>>>> who knows OTP better than me would need to weigh in here though. > >>>>>>> > >>>>>>> I found http://manning.com/logan very informative in understanding > >>>>>>> OTP > >>>>>>> and > >>>>>>> its supervisor hierarchies. > >>>>>>> > >>>>>>> Cheers, > >>>>>>> Alain > >>>>>>> ________________________________________________________________ > >>>>>>> erlang-questions (at) erlang.org mailing list. > >>>>>>> See http://www.erlang.org/faq.html > >>>>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >>>>>>> > >>>>>>> > >>>>>> > >>>>>> -- > >>>>>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > >>>>>> > >>>>>> ________________________________________________________________ > >>>>>> erlang-questions (at) erlang.org mailing list. > >>>>>> See http://www.erlang.org/faq.html > >>>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >>>>>> > >>>>>> > >>>>>> > >>>>> ________________________________________________________________ > >>>>> erlang-questions (at) erlang.org mailing list. > >>>>> See http://www.erlang.org/faq.html > >>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >>>>> > >>>>> > >>>> > >>>> -- > >>>> Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > >>>> > >>>> > >>> ________________________________________________________________ > >>> erlang-questions (at) erlang.org mailing list. > >>> See http://www.erlang.org/faq.html > >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >>> > >>> > >> > >> > > > > -- > > Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > -- No imprima este correo si no es necesario. El medio ambiente est? en nuestras manos. __________________________________________ Clist UAH a.k.a Angel AT uah DOT es Angel Alvarez , Servicios Inform?ticos Edificio Torre de Control, Campus Externo UAH Alcal? de Henares 28806, Madrid ** ESPA?A ** __________________________________________ MySQL4: Bien, Usa transacciones solo si no necesitas velocidad. From n.oxyde@REDACTED Wed Jan 12 10:58:10 2011 From: n.oxyde@REDACTED (nox) Date: Wed, 12 Jan 2011 10:58:10 +0100 Subject: [erlang-questions] Re: [erlang-patches] [patch] new float_to_list/2 In-Reply-To: References: <4D2D3FA8.1000009@aleynikov.org> Message-ID: Il should be called "scale", shouldn't it? Le 12 janv. 2011 ? 10:26, Pierpaolo Bernardi a ?crit : > On Wed, Jan 12, 2011 at 06:44, Serge Aleynikov wrote: >> Attached please find a patch that adds a new float_to_list/2 BIF. The patch >> was created off of the master branch of https://github.com/erlang/otp. >> >> This BIF solves a problem of float_to_list/1 that doesn't allow specifying >> the number of digits after the decimal point when formatting floats. >> >> float_to_list(Float, Options) -> string() >> >> Float = float() >> Options = [Option] >> Option = {precision, Precision::integer()} | compact >> >> Text representation of a float formatted using given options >> >> Returns a string which corresponds to the text >> representation of Float using fixed decimal point formatting. >> When precision option is specified >> the returned value will contain at most Precision number of >> digits past the decimal point. When compact option is provided >> the trailing zeros at the end of the list are truncated. > > I think the option is misnamed. > > In the usual terminology, 'precision' is the total number of > significative digits, not only the ones past the decimal point. > > Cheers > P. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From kimabrandt@REDACTED Wed Jan 12 13:43:43 2011 From: kimabrandt@REDACTED (Kim A. Brandt) Date: Wed, 12 Jan 2011 13:43:43 +0100 Subject: [ASN.1] REAL-type problem!? Message-ID: <4D2DA1FF.8020009@gmx.de> Hello list, (first-time mailer here) Erlang is awesome! I am looking at ASN.1 and got a problem with the REAL-type. A sample (MyTest.asn) ASN.1-specification: MyTest DEFINITIONS AUTOMATIC TAGS ::= BEGIN MyComponent ::= SEQUENCE { enum ENUMERATED { a, b }, real REAL } END And this is what I get: ~$ erl Erlang R13B03 (erts-5.7.4) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.4 (abort with ^G) 1> asn1ct:compile("MyTest",[uper_bin]). Erlang ASN.1 version "1.6.12" compiling "MyTest.asn" Compiler Options: [uper_bin,{i,"."}] --{generated,"MyTest.asn1db"}-- --{generated,"MyTest.hrl"}-- --{generated,"MyTest.erl"}-- ok 2> rr("MyTest.hrl"). ['MyComponent'] 3> {ok,Bytes}=asn1rt:encode('MyTest','MyComponent',#'MyComponent'{enum=a,real="123.456"}). {ok,<<5,129,152,153,25,154,26,155,23,34,150,153,128>>} 4> asn1rt:decode('MyTest','MyComponent',Bytes). {error,{asn1,{{badmatch,{"123456.E-3",<<>>,11}}, [{asn1rt_uper_bin,decode_real,1}, {'MyTest',dec_MyComponent,2}, {'MyTest',decode,2}, {asn1rt,decode,3}, {erl_eval,do_apply,5}, {shell,exprs,6}, {shell,eval_exprs,6}, {shell,eval_loop,3}]}}} I tried with the latest OTP-release (R14B01), which gives the same error!? My instinct tells me there is a problem with the REAL-type in conjunction with the ENUMERATED-type, D'oh! But, what can I do? And could this be a bug or did I forget or do something wrong? Kind regards, Kim From tony@REDACTED Wed Jan 12 13:02:00 2011 From: tony@REDACTED (Tony Rogvall) Date: Wed, 12 Jan 2011 13:02:00 +0100 Subject: [erlang-questions] List comprehensions challenge ;-) In-Reply-To: References: Message-ID: On 12 jan 2011, at 10.08, Arnaud GARCIA wrote: > Hello, > > Just playing with list comprehensions... > > I was asking if it possible to do it without lists:sum, by using an > anonymous function and add it directly in the list comprehensions ? > > 26> [A+B || A <-[1,2,3],B <- [7,8,9,10,11]]. > [8,9,10,11,12,9,10,11,12,13,10,11,12,13,14] > 30> lists:sum([A+B || A <-[1,2,3],B <- [7,8,9,10,11]]). > 165 > > Any idea ? > You need to fold it. Out of topic, this problem have a solution that has better complexity. > length(As)*lists:sum(Bs)+length(Bs)*lists:sum(As) 165 /Tony > Thanks > > Arnaud "Have run Make so many times I dunno what's installed anymore" From jesper.louis.andersen@REDACTED Wed Jan 12 14:05:55 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Wed, 12 Jan 2011 14:05:55 +0100 Subject: [erlang-questions] We need a better marketing division :-) In-Reply-To: References: <24250519.271261294767163150.JavaMail.root@zimbra> <4D2D56B9.90103@lbox.cc> Message-ID: On Wed, Jan 12, 2011 at 10:49, Ulf Wiger wrote: > ?or Haskell, perhaps. :) > > The proposal in question means to design a DSL for large-scale > "embarrassingly parallel" computing. This is not really a domain where > Erlang is the obvious choice (although it wouldn't necessarily be a very > bad one). Haskell, OTOH, has some really great facilities for creating DSLs > and also some pretty advanced research on data parallelism. The proposal looks to be significantly less ambitious than Haskell. If the goal is to support the *heterogenous* environment of a CPU/GPU fusion merger, then the only problems that will run fast are those which are * Parallelizable: This includes problems which are embarrassingly parallel, but also those for which it is possible to hide the latency of communication by doing something else while waiting for information to transfer between execution units. Traditionally, this is not that well supported by GPUs. * LinAlgibrazible: In the sense you need to transform the problem into Linear Algebra, essentially speaking. To a certain extent this is true for all programming, but it is much easier to do with, e.g., video decoding or raytracing than symbolic manipulation in a compiler. * Sizable: Every block of data operated on must be of rougly equal size. Otherwise throughput is wasted. * BasicBlockable: GPUs operate as an old army infantry unit. If the sergeant yells "FORWARD!", then the whole unit does so, even though it consist of 480 people/cores. Same with "STOP!" or "FIRE!". 480 muskets firing is of course quite dangerous, but it has ramifications to computation because it means that GPUs prefer long Basic blocks of code with no jumps. Haskells approach does not per se expect a heterogenous environment and they are researching into Nested Data Level Parallelism in order to beat the 'Sizable' problem above. It makes their parallelism approach far more general. Yet, since Haskell is pure, they have a very clean slate to work on. In other words: If they can't solve the problem they face, then there is little hope anybody else will in the foreseeable future. And Ulf is right: Traditionally, this is not the domain of Erlang at all. Erlang excels at harnessing concurrency first and foremost. None of those problems will be solved with the Scala/Haskell approach, neither GPUs. If history is a teacher of worth, the GPU-idea will die again but for supercomputing. The Commodore Amiga of the 80'es had a "GPU' as a 2d blitter which made it superior to the PCs at the time. But it lost because the CPUs in the PCs became so fast they could compete. We've had numerous vector-computer incarnations which were fast at vector operations, yet still did not break through. Intel clearly bets on a large amount of CPU-cores at the moment, perhaps with small SSE-like instruction sets tied to them. And Multicore ARM looks like the same traditional design. Considering the size of the heat sinks on GPUs, we can't claim that they are not currently power-efficient, though that might in principle change. Yet, Erlang has another problem it is solving at the moment: If we have a concurrent description of a program, which is essentially what we have in Erlang, then we can try to map this program to multiple cores. We probably won't achieve perfect speedup, but that hardly matter. We just got a much higher throughput for a highly concurrent program. These programs/problems are not addressed by my points above - so it seems we have a nice niche there, together with perhaps Google Go :) > Perhaps I'm reading too much into it, but as many of you know, I have > been crusading a bit around the "state-space explosion" problem, arguing > that it's not messaging per-se, but the event-handling semantics > (lack of selective event handling) that explodes the state space. I'd say > that we have plenty of empirical evidence that Erlang is great for keeping > the state space from growing out of control. I just think they are viewing the state space incorrectly. If you line it up as a list: [S1, S2, ..., Sn] where each Si is a state of a process then of course, it gets nasty. The interleaving of instructions goes through the roof here as any state can be advanced if that process is chosen to run. Correctness arguments now need to take the whole list into account meaning much more work and an exploding state space. But Erlang programs are not like this in reality, even though you could describe its semantics like it. What happens in Erlang programs are that the state space is given by {{S11, S12, ...}, {...}, ...} in other words as a tree of states. Why? Because of *isolation* of internal process state, we can take a group of processes with state, smack a label on one of the processes and call it an "API". The rest of the system can then view the whole group as a single state with a few well-designed invariants. In some cases the group is *stateless* to the outside, so it acts like a functional program. Essentially it ties in with the ability of selective receive and it dampens the state-space-explosion. States in a group gets interlocked to each other and they can only advance as a group on the larger scale. Internally, there might be interleaving which is problematic, but it has closure and is limited to the group. Take this view, and the state-space-explosion problem can be tackled. > The interesting thing is that Scala *does* support erlang-style concurrency, > to a greater extent than most languages. Why, then, would they say this? It is a grant proposal, so it is important it breeds the fear of impending doom of computation upon us so the grant is given :) > The intersection between language, libraries, prominent applications and > community somehow define the core set of concepts. For Erlang, lightweight > concurrency, fault-tolerance and powerful message passing are all right > smack in the core, and thus permeate everything we do, for better and for > worse. Indeed. The approach to heterogenous computation is right now at the point where you define a small program called a *kernel* which is then automatically run on each core in the infantry unit. Each soldier gets its own target (as in: its own piece of the vector-array to process). The CPU-sarge then yells "FIRE!" and counts the dead amongst the enemy. The language of the kernel can be chosen freely although for familiarity a C- or FORTRAN-like syntax and semantics tend to be chosen. The rest is simply plumbing to get it to run correctly and marshal data around in the system. You could add this if you wanted to Erlang - but we already have NIFs that gives you a practical solution here-n-now and for the core value of Erlang, it wouldn't do that much. I'd much rather gamble on a world with multicore CPUs for Erlang. These already exist and they are needed very much for servers. Currently, imagining running a web-server off of your GPU is distant and not even in the horizon. So even while your desktop computer might go the CPU/GPU heterogenous way, I don't think the servers will - for anything but supercomputing needs. -- J. From serge@REDACTED Wed Jan 12 15:10:36 2011 From: serge@REDACTED (Serge Aleynikov) Date: Wed, 12 Jan 2011 09:10:36 -0500 Subject: [erlang-questions] [patch] new float_to_list/2 In-Reply-To: References: <4D2D3FA8.1000009@aleynikov.org> Message-ID: <4D2DB65C.1090106@aleynikov.org> The reason I called it precision was to be consistent with the naming convention of the printf function. Below is the extract from "man 3 printf", which refers to the digits after the decimal point as "precision": f, F The double argument is rounded and converted to decimal notation in the style [-]ddd.ddd, where the number of digits after the decimal-point character is equal to the precision specification. If the precision is missing, it is taken as 6; if the precision is explicitly zero, no decimal-point character appears. If a decimal point appears, at least one digit appears before it. I don't have a very strong preference for calling it precision or scale, but I do have a strong preference for including this patch in the distribution, because the default behavior of float_to_list/1 hard-coded in C is deficient. On 1/12/2011 4:58 AM, nox wrote: > Il should be called "scale", shouldn't it? > > Le 12 janv. 2011 ? 10:26, Pierpaolo Bernardi a ?crit : > >> On Wed, Jan 12, 2011 at 06:44, Serge Aleynikov wrote: >>> Attached please find a patch that adds a new float_to_list/2 BIF. The patch >>> was created off of the master branch of https://github.com/erlang/otp. >>> >>> This BIF solves a problem of float_to_list/1 that doesn't allow specifying >>> the number of digits after the decimal point when formatting floats. >>> >>> float_to_list(Float, Options) -> string() >>> >>> Float = float() >>> Options = [Option] >>> Option = {precision, Precision::integer()} | compact >>> >>> Text representation of a float formatted using given options >>> >>> Returns a string which corresponds to the text >>> representation of Float using fixed decimal point formatting. >>> When precision option is specified >>> the returned value will contain at most Precision number of >>> digits past the decimal point. When compact option is provided >>> the trailing zeros at the end of the list are truncated. >> >> I think the option is misnamed. >> >> In the usual terminology, 'precision' is the total number of >> significative digits, not only the ones past the decimal point. >> >> Cheers >> P. >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> From gleber.p@REDACTED Wed Jan 12 15:23:29 2011 From: gleber.p@REDACTED (Gleb Peregud) Date: Wed, 12 Jan 2011 15:23:29 +0100 Subject: [erlang-questions] [patch] new float_to_list/2 In-Reply-To: <4D2DB65C.1090106@aleynikov.org> References: <4D2D3FA8.1000009@aleynikov.org> <4D2DB65C.1090106@aleynikov.org> Message-ID: Some time ago I've submitted similar but simpler patch. It was a float_to_list/2 with a second parameter being an integer specifying precision. For me it was important to generate A LOT of floats as strings as fast as possible with specified precision. Serge's version has an overhead of inspecting proplist of the second parameter. So I was wondering about introducing two versions of this function: with a proplist as a second parameter and with a number as a second parameter. Alternatively proplist version can be factored out into float_to_list_opts/2. Just my 0.2 cents On Wed, Jan 12, 2011 at 15:10, Serge Aleynikov wrote: > The reason I called it precision was to be consistent with the naming > convention of the printf function. ?Below is the extract from "man 3 > printf", which refers to the digits after the decimal point as "precision": > > ? ? ? f, F ? The ?double ?argument is rounded and converted to decimal > ? ? ? ? ? ? ?notation in the style [-]ddd.ddd, where the number of > ? ? ? ? ? ? ?digits after the decimal-point character is equal to the > ? ? ? ? ? ? ?precision specification. ?If the precision is missing, > ? ? ? ? ? ? ?it ?is taken as 6; if the precision is explicitly zero, > ? ? ? ? ? ? ?no decimal-point character appears. ?If a decimal point > ? ? ? ? ? ? ?appears, at least one digit appears before it. > > I don't have a very strong preference for calling it precision or scale, but > I do have a strong preference for including this patch in the distribution, > because the default behavior of float_to_list/1 hard-coded in C is > deficient. > > > On 1/12/2011 4:58 AM, nox wrote: >> >> Il should be called "scale", shouldn't it? >> >> Le 12 janv. 2011 ? 10:26, Pierpaolo Bernardi ?a ?crit >> : >> >>> On Wed, Jan 12, 2011 at 06:44, Serge Aleynikov >>> ?wrote: >>>> >>>> Attached please find a patch that adds a new float_to_list/2 BIF. ?The >>>> patch >>>> was created off of the master branch of https://github.com/erlang/otp. >>>> >>>> This BIF solves a problem of float_to_list/1 that doesn't allow >>>> specifying >>>> the number of digits after the decimal point when formatting floats. >>>> >>>> ? ? ? ?float_to_list(Float, Options) -> ?string() >>>> >>>> ? ? ? ?Float = float() >>>> ? ? ? ?Options = [Option] >>>> ? ? ? ?Option = {precision, Precision::integer()} | compact >>>> >>>> ? ? ? ?Text representation of a float formatted using given options >>>> >>>> ? ? ? ?Returns a string which corresponds to the text >>>> ? ? ? ?representation of Float using fixed decimal point formatting. >>>> ? ? ? ?When precision option is specified >>>> ? ? ? ?the returned value will contain at most Precision number of >>>> ? ? ? ?digits past the decimal point. ?When compact option is provided >>>> ? ? ? ?the trailing zeros at the end of the list are truncated. >>> >>> I think the option is misnamed. >>> >>> In the usual terminology, 'precision' is the total number of >>> significative digits, not only the ones past the decimal point. >>> >>> Cheers >>> P. >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From alexey.v.romanov@REDACTED Wed Jan 12 16:11:58 2011 From: alexey.v.romanov@REDACTED (Alexey Romanov) Date: Wed, 12 Jan 2011 18:11:58 +0300 Subject: string:to_float/1 behavior Message-ID: 1) Where can I find its source? It isn't defined in lib/stdlib-1.17.1/src/string.erl. I thought it must be a BIF, but "ack to_float /usr/local/lib/erlang/" doesn't seem to find anything relevant. The full output is at https://gist.github.com/776256, but 1) the only C function found is in lib/erl_interface-3.7.1/src/legacy/erl_marshal.c; 2) the Erlang code doesn't seem to define any mapping for this name. 2) The reason I went looking for its code is to check whether it is guaranteed to deal with both "." and "," as decimal separators, independent of locale? It does on my machine (with LC_NUMERIC="en_US.UTF-8") but the string module documentation doesn't say anything. Yours, Alexey Romanov From serge@REDACTED Wed Jan 12 18:13:34 2011 From: serge@REDACTED (Serge Aleynikov) Date: Wed, 12 Jan 2011 12:13:34 -0500 Subject: [erlang-questions] [patch] new float_to_list/2 In-Reply-To: References: <4D2D3FA8.1000009@aleynikov.org> <4D2DB65C.1090106@aleynikov.org> Message-ID: <4D2DE13E.7040404@aleynikov.org> I did a micro-benchmark on a slightly modified version of the BIF that accepts an integer as its options. The results shown below display a very insignificant difference between a call with no options and a call with an integer precision passed as the second argument: 1> test:test(). float_to_list(123.4, []) = {0.619512,"123.4000"} float_to_list(123.4, [{precision, 4}]) = {0.624895,"123.4000"} float_to_list(123.4, 4) = {0.622896,"123.4000"} The majority of time is actually spent in the printf(3) function, which takes longer to execute when given the "%.*f" argument compared to "%g" as in float_to_list/1 case. Did your patch rely on printf? On 1/12/2011 9:23 AM, Gleb Peregud wrote: > Some time ago I've submitted similar but simpler patch. It was a > float_to_list/2 with a second parameter being an integer specifying > precision. For me it was important to generate A LOT of floats as > strings as fast as possible with specified precision. Serge's version > has an overhead of inspecting proplist of the second parameter. So I > was wondering about introducing two versions of this function: with a > proplist as a second parameter and with a number as a second > parameter. Alternatively proplist version can be factored out into > float_to_list_opts/2. > > Just my 0.2 cents > > On Wed, Jan 12, 2011 at 15:10, Serge Aleynikov wrote: >> The reason I called it precision was to be consistent with the naming >> convention of the printf function. Below is the extract from "man 3 >> printf", which refers to the digits after the decimal point as "precision": >> >> f, F The double argument is rounded and converted to decimal >> notation in the style [-]ddd.ddd, where the number of >> digits after the decimal-point character is equal to the >> precision specification. If the precision is missing, >> it is taken as 6; if the precision is explicitly zero, >> no decimal-point character appears. If a decimal point >> appears, at least one digit appears before it. >> >> I don't have a very strong preference for calling it precision or scale, but >> I do have a strong preference for including this patch in the distribution, >> because the default behavior of float_to_list/1 hard-coded in C is >> deficient. >> >> >> On 1/12/2011 4:58 AM, nox wrote: >>> >>> Il should be called "scale", shouldn't it? >>> >>> Le 12 janv. 2011 ? 10:26, Pierpaolo Bernardi a ?crit >>> : >>> >>>> On Wed, Jan 12, 2011 at 06:44, Serge Aleynikov >>>> wrote: >>>>> >>>>> Attached please find a patch that adds a new float_to_list/2 BIF. The >>>>> patch >>>>> was created off of the master branch of https://github.com/erlang/otp. >>>>> >>>>> This BIF solves a problem of float_to_list/1 that doesn't allow >>>>> specifying >>>>> the number of digits after the decimal point when formatting floats. >>>>> >>>>> float_to_list(Float, Options) -> string() >>>>> >>>>> Float = float() >>>>> Options = [Option] >>>>> Option = {precision, Precision::integer()} | compact >>>>> >>>>> Text representation of a float formatted using given options >>>>> >>>>> Returns a string which corresponds to the text >>>>> representation of Float using fixed decimal point formatting. >>>>> When precision option is specified >>>>> the returned value will contain at most Precision number of >>>>> digits past the decimal point. When compact option is provided >>>>> the trailing zeros at the end of the list are truncated. >>>> >>>> I think the option is misnamed. >>>> >>>> In the usual terminology, 'precision' is the total number of >>>> significative digits, not only the ones past the decimal point. >>>> >>>> Cheers >>>> P. >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: test.erl URL: From serge@REDACTED Wed Jan 12 18:20:06 2011 From: serge@REDACTED (Serge Aleynikov) Date: Wed, 12 Jan 2011 12:20:06 -0500 Subject: [erlang-questions] string:to_float/1 behavior In-Reply-To: References: Message-ID: <4D2DE2C6.9050308@aleynikov.org> It is a BIF. Look here: emulator/beam/bif.tab It does handle both "." and "," cases - this is hardcoded in the implementation. On 1/12/2011 10:11 AM, Alexey Romanov wrote: > 1) Where can I find its source? It isn't defined in > lib/stdlib-1.17.1/src/string.erl. I thought it must be a BIF, but "ack > to_float /usr/local/lib/erlang/" doesn't seem to find anything > relevant. The full output is at https://gist.github.com/776256, but 1) > the only C function found is in > lib/erl_interface-3.7.1/src/legacy/erl_marshal.c; 2) the Erlang code > doesn't seem to define any mapping for this name. > > 2) The reason I went looking for its code is to check whether it is > guaranteed to deal with both "." and "," as decimal separators, > independent of locale? It does on my machine (with > LC_NUMERIC="en_US.UTF-8") but the string module documentation doesn't > say anything. > > Yours, Alexey Romanov > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From ericsetton@REDACTED Wed Jan 12 19:18:35 2011 From: ericsetton@REDACTED (Eric Setton (PA)) Date: Wed, 12 Jan 2011 13:18:35 -0500 Subject: Tango needs Erlang Engineers Message-ID: <32DA5B78-FF70-43F7-9D05-45ACA9150F3B@tango.me> I am the CTO/Founder of Tango, a mobile2mobile video calling service that allows iPhones, Androids, and other videophones to connect over the 3G network. We launched across the mobile app market at the end of September and we were already over million users in our first 10 days. We are now at 5+ million and it is just January. We are doing great. I am attempting to do some recruiting myself, and looking for Senior Erlang Engineers. I'm hoping you might be interested in helping us build mobile video communication software that could change how millions of people connect around the world. If you would be interested email me at ericsetton@REDACTED I look forward to hearing from you, Eric Setton a link to one of many articles regarding our meteoric growth: http://gigaom.com/2010/12/16/tango-snags-4m-users-as-video-chat-becomes-the-new-texting/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:+OmMalik+%28GigaOM:+Tech%29 From alexey.v.romanov@REDACTED Wed Jan 12 19:57:10 2011 From: alexey.v.romanov@REDACTED (Alexey Romanov) Date: Wed, 12 Jan 2011 21:57:10 +0300 Subject: [erlang-questions] string:to_float/1 behavior In-Reply-To: <4D2DE2C6.9050308@aleynikov.org> References: <4D2DE2C6.9050308@aleynikov.org> Message-ID: Thank you! Yours, Alexey Romanov On Wed, Jan 12, 2011 at 8:20 PM, Serge Aleynikov wrote: > It is a BIF. ?Look here: > > ? ? ? ?emulator/beam/bif.tab > > It does handle both "." and "," cases - this is hardcoded in the > implementation. > > On 1/12/2011 10:11 AM, Alexey Romanov wrote: >> >> 1) Where can I find its source? It isn't defined in >> lib/stdlib-1.17.1/src/string.erl. I thought it must be a BIF, but "ack >> to_float /usr/local/lib/erlang/" doesn't seem to find anything >> relevant. The full output is at https://gist.github.com/776256, but 1) >> the only C function found is in >> lib/erl_interface-3.7.1/src/legacy/erl_marshal.c; 2) the Erlang code >> doesn't seem to define any mapping for this name. >> >> 2) The reason I went looking for its code is to check whether it is >> guaranteed to deal with both "." and "," as decimal separators, >> independent of locale? It does on my machine (with >> LC_NUMERIC="en_US.UTF-8") but the string module documentation doesn't >> say anything. >> >> Yours, Alexey Romanov >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From kaiduanx@REDACTED Wed Jan 12 22:55:47 2011 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Wed, 12 Jan 2011 16:55:47 -0500 Subject: SOA with Erlang Message-ID: Hi, all, Can someone share the experience on using SOA (Service Oriented Architecture) with Erlang? Are there successful cases in industry? Thanks for help, /Kaiduan From gordon@REDACTED Thu Jan 13 00:05:46 2011 From: gordon@REDACTED (Gordon Guthrie) Date: Wed, 12 Jan 2011 23:05:46 +0000 Subject: [erlang-questions] SOA with Erlang In-Reply-To: References: Message-ID: Kaiduan Here's a link to the Hypernumbers White Paper I wrote a while back: http://www.scribd.com/doc/32838204/Hypernumbers-Whitepaper-Using-Erlang-In-Web-Startups Cheers Gordon On 12 January 2011 21:55, Kaiduan Xie wrote: > Hi, all, > > Can someone share the experience on using SOA (Service Oriented > Architecture) with Erlang? Are there successful cases in industry? > > Thanks for help, > > /Kaiduan > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669 From rmelton@REDACTED Thu Jan 13 02:56:05 2011 From: rmelton@REDACTED (Robert Melton) Date: Wed, 12 Jan 2011 20:56:05 -0500 Subject: [erlang-questions] SOA with Erlang In-Reply-To: References: Message-ID: On 12 January 2011 21:55, Kaiduan Xie wrote: > Can someone share the experience on using SOA (Service Oriented > Architecture) with Erlang? Are there successful cases in industry? I am in the process (just started) of doing this now on my first production product in Erlang, and it is, interesting and frustrating at the same time. In the past, I have built SOA platforms (this will be my fourth), focused on having a single communication point, the HTTP layer for everything. This had some nice advantages, load-balanced back and front end with off the shelf components, great scalability, great caching, great concurrency (thread the tip and allow any concurrent processes to be load balanced and run concurrently), and generally I didn't feel like I was giving up too much in the exchange. I had to deal with home grow the error handling to make complex requests fail gracefully, and deal with communication parsing and network latency, but generally the decision was fairly easy as the benefits greatly outweighed the cons. Erlang is a far different animal in this regard, so much of what I spent a lot of time building in the past is built in, and trivial in Erlang. Message passing between components in HTTP/(JSON or XML) seems insanely heavy, home grown error handling feels obnoxious and outdated, and pushing all the communication to the http layer feels like a net negative. However, pushing a lot out to HTTP, you do still get some benefits in regards to load balancing, high levels of scaling (hundreds of servers can be used to fulfill a single request). More than answering your questions, I am thinking aloud about my own issues, sorry for that. I am hunting for that balance point, should I continue to push everything up to HTTP using a tool like WebMachine, and then Erlang just becomes a far better local webserver. If I instead decide to use all of Erlang's wonderful features, where do I draw the line between Erlang internal clustering and process communication tools, and the web layer... and what do I do once I pass Erlang's ability to cluster (this may be a non-issue, but I have read repeatedly that you can't get into the hundreds of Erlang nodes without issues). /thread_hijack On Wed, Jan 12, 2011 at 6:05 PM, Gordon Guthrie wrote: > Here's a link to the Hypernumbers White Paper I wrote a while back: > http://www.scribd.com/doc/32838204/Hypernumbers-Whitepaper-Using-Erlang-In-Web-Startups Thanks Gordon, for taking the time to write that all up! -- Robert Melton | Contact Information @ http://robertmelton.com/contact From alain.odea@REDACTED Thu Jan 13 03:10:11 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Wed, 12 Jan 2011 22:40:11 -0330 Subject: [erlang-questions] SOA with Erlang In-Reply-To: References: Message-ID: <0FF97A15-DE9A-4472-BAB3-FDF7FB05B200@gmail.com> On 2011-01-12, at 18:25, Kaiduan Xie wrote: > Hi, all, > > Can someone share the experience on using SOA (Service Oriented > Architecture) with Erlang? Are there successful cases in industry? > > Thanks for help, > > /Kaiduan Most systems built in Erlang embody SOA principles. OTP is a very effective SOA platform. AFAIK most serious Erlang systems are written using OTP. There are likely as many Erlang SOA success stories as there are Erlang success stories as a whole :) Given that Erlang/OTP is a proven SOA platform I suggest picking up Erlang and OTP in Action http://manning.com/logan and working through the SOA project it uses as an example. From kaiduanx@REDACTED Thu Jan 13 04:40:06 2011 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Wed, 12 Jan 2011 22:40:06 -0500 Subject: [erlang-questions] SOA with Erlang In-Reply-To: <0FF97A15-DE9A-4472-BAB3-FDF7FB05B200@gmail.com> References: <0FF97A15-DE9A-4472-BAB3-FDF7FB05B200@gmail.com> Message-ID: Thanks a lot for the information. What I have now is an OTP structured application that runs great, but our group is pushing SOA hard, so I seek ideas on how to bring SOA into the picture so that Erlang can be accepted in the group. I have no idea of SOA before that :) Best regards, /Kaiduan On Wed, Jan 12, 2011 at 9:10 PM, Alain O'Dea wrote: > On 2011-01-12, at 18:25, Kaiduan Xie wrote: > >> Hi, all, >> >> Can someone share the experience on using SOA (Service Oriented >> Architecture) with Erlang? Are there successful cases in industry? >> >> Thanks for help, >> >> /Kaiduan > > Most systems built in Erlang embody SOA principles. ?OTP is a very effective SOA platform. ?AFAIK most serious Erlang systems are written using OTP. > > There are likely as many Erlang SOA success stories as there are Erlang success stories as a whole :) > > Given that Erlang/OTP is a proven SOA platform I suggest picking up Erlang and OTP in Action http://manning.com/logan and working through the SOA project it uses as an example. From bruce@REDACTED Thu Jan 13 04:58:34 2011 From: bruce@REDACTED (Bruce Fitzsimons) Date: Thu, 13 Jan 2011 16:58:34 +1300 Subject: [erlang-questions] SOA with Erlang In-Reply-To: References: <0FF97A15-DE9A-4472-BAB3-FDF7FB05B200@gmail.com> Message-ID: <4D2E786A.3000201@fitzsimons.org> On 13/01/2011 4:40 p.m., Kaiduan Xie wrote: > Thanks a lot for the information. What I have now is an OTP structured > application that runs great, but our group is pushing SOA hard, so I > seek ideas on how to bring SOA into the picture so that Erlang can be > accepted in the group. I have no idea of SOA before that :) > It would be remiss not to mention RabbitMQ (http://www.rabbitmq.org) as a basis for an SOA architecture. Built in Erlang but usable by pretty much any language, it is an AMQP (http://en.wikipedia.org/wiki/AMQP) broker that would provide an excellent basis for building services. Regards, Bruce From alain.odea@REDACTED Thu Jan 13 05:01:43 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Thu, 13 Jan 2011 00:31:43 -0330 Subject: [erlang-questions] SOA with Erlang In-Reply-To: References: <0FF97A15-DE9A-4472-BAB3-FDF7FB05B200@gmail.com> Message-ID: I should clarify my rather oblique argument than OTP is SOA. OTP very cleanly supports the notice of publishing and consuming services in the form of local and global gen_servers. One Erlang node may run a gen_server that provides session data, while another runs one that provides inventory information and yet another runs Zotonic (or something similar) to render a web-based UI. The beauty of Erlang/OTP is that all these services can transparently be on a single node or any number of nodes. Even better is that Erlang/OTP makes deciding how failures are handled and communicated manageable and clean. Start simple and gradually. This mailing list is a treasure trove of design patterns and best practices. Ask single focused questions about what you are looking for and you will get great and timely responses :) Erlang/OTP makes SOA easy. I hope you have as much fun applying it as I do :) On 2011-01-13, at 0:10, Kaiduan Xie wrote: > Thanks a lot for the information. What I have now is an OTP structured > application that runs great, but our group is pushing SOA hard, so I > seek ideas on how to bring SOA into the picture so that Erlang can be > accepted in the group. I have no idea of SOA before that :) > > Best regards, > > /Kaiduan > > On Wed, Jan 12, 2011 at 9:10 PM, Alain O'Dea wrote: >> On 2011-01-12, at 18:25, Kaiduan Xie wrote: >> >>> Hi, all, >>> >>> Can someone share the experience on using SOA (Service Oriented >>> Architecture) with Erlang? Are there successful cases in industry? >>> >>> Thanks for help, >>> >>> /Kaiduan >> >> Most systems built in Erlang embody SOA principles. OTP is a very effective SOA platform. AFAIK most serious Erlang systems are written using OTP. >> >> There are likely as many Erlang SOA success stories as there are Erlang success stories as a whole :) >> >> Given that Erlang/OTP is a proven SOA platform I suggest picking up Erlang and OTP in Action http://manning.com/logan and working through the SOA project it uses as an example. From torben.lehoff@REDACTED Thu Jan 13 10:11:58 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Thu, 13 Jan 2011 10:11:58 +0100 Subject: [erlang-questions] string:to_float/1 behavior In-Reply-To: References: <4D2DE2C6.9050308@aleynikov.org> Message-ID: If you want to find all the BIFs: xref:start(s). xref:add_release(s,"c:/Program\ Files/erl5.7.4/",[{builtins,true}]). {ok,Bifs} = xref:q(s,"B"). Bifs. Correct the path in the 2nd arg to xref:add_release/3 as appropriate. Cheers, Torben On Wed, Jan 12, 2011 at 19:57, Alexey Romanov wrote: > Thank you! > > Yours, Alexey Romanov > > > > On Wed, Jan 12, 2011 at 8:20 PM, Serge Aleynikov > wrote: > > It is a BIF. Look here: > > > > emulator/beam/bif.tab > > > > It does handle both "." and "," cases - this is hardcoded in the > > implementation. > > > > On 1/12/2011 10:11 AM, Alexey Romanov wrote: > >> > >> 1) Where can I find its source? It isn't defined in > >> lib/stdlib-1.17.1/src/string.erl. I thought it must be a BIF, but "ack > >> to_float /usr/local/lib/erlang/" doesn't seem to find anything > >> relevant. The full output is at https://gist.github.com/776256, but 1) > >> the only C function found is in > >> lib/erl_interface-3.7.1/src/legacy/erl_marshal.c; 2) the Erlang code > >> doesn't seem to define any mapping for this name. > >> > >> 2) The reason I went looking for its code is to check whether it is > >> guaranteed to deal with both "." and "," as decimal separators, > >> independent of locale? It does on my machine (with > >> LC_NUMERIC="en_US.UTF-8") but the string module documentation doesn't > >> say anything. > >> > >> Yours, Alexey Romanov > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- http://www.linkedin.com/in/torbenhoffmann From torben.lehoff@REDACTED Thu Jan 13 10:24:18 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Thu, 13 Jan 2011 10:24:18 +0100 Subject: [erlang-questions] We need a better marketing division :-) In-Reply-To: References: <24250519.271261294767163150.JavaMail.root@zimbra> <4D2D56B9.90103@lbox.cc> Message-ID: General observation on a fundamental difference between Scala and Erlang regarding the supporting organisations (as I see it, YMMV): - Scala was born out of an academic environment where there is a strong tradition for applying for grants to support research. - Erlang was born in a private company trying to solve a problem they had and was only later released as open source. And right now most Erlangers are busy trying to solve real world problems in order to make money so applying for this type of grants is not on the top of the agenda. I am - and would suggest to other to do the same - looking at areas where Erlang could be used to solve a fundamental problem in that domain and there are in fact a lot of grants and/or public funding hanging around. Right now Green IT - in particular Energy Management - seems like an area where Erlang would be a good tool and there are plenty of grants and public funding floating around... just check various EU pages. Cheers, Torben -- http://www.linkedin.com/in/torbenhoffmann From arnaud74130@REDACTED Thu Jan 13 11:06:17 2011 From: arnaud74130@REDACTED (Arnaud GARCIA) Date: Thu, 13 Jan 2011 11:06:17 +0100 Subject: Finding records in List Message-ID: Hello, (still learning erlang ;-) I would like to find all records matching criterion/criteria in a list of records. if you have one criteria, you can use: lists:keyfind(Name,#person.name,ListOfPersonRecords). => But, it will return just the first one who match the criteria =>Moreover with lists:keyfind I didn't find a way for multiple criteria like : find all person in list where Name= and Age= I saw an interesting post using record<->proplists... is it the way to do it ? http://stackoverflow.com/questions/447188/erlang-and-run-time-record-limitations Finally, I build my own function which parse the List and check if it match the criterion or criteria BUT I was asking if there is an erlang function to do this ? thanks, Arnaud From ulf.wiger@REDACTED Thu Jan 13 11:13:13 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 13 Jan 2011 11:13:13 +0100 Subject: picky dialyzer Message-ID: This has me confused: Compiled src/jobs_server.erl ==> jobs (analyze) jobs_server.erl:1251: The pattern {'queue', _, _, {'producer', _}, _, _, _, _, _, _, _, _, _, _} can never match the type #queue{mod::atom(),type::'fifo' | #action{a::'approve' | 'reject'} | #passive{type::'fifo'},group::atom(),...} The line in question is q_is_empty(#queue{type = #producer{}}) -> false; The #queue{} record definition is: -record(producer, {f :: mfa() | function()}). -record(passive , {type = fifo :: fifo}). -record(action , {a = approve :: approve | reject}). -record(queue, {name :: any(), mod :: atom(), type = fifo :: fifo | #producer{} | #passive{} | #action{}, group :: atom(), ?} Why does dialyzer think that #producer{} is not a valid member of the type def for the 'type' attribute? The funny thing is that if I rename the #producer{} record to e.g. #prod{} or #produce{}, dialyzer picks it up (but generates a bunch of other problems, partly because the change wasn't done consistently throughout). This (or practically the same warning) can be reproduced by pulling the latest version of http://github.com/esl/jobs and running ./rebar analyze BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From matthias@REDACTED Thu Jan 13 11:16:50 2011 From: matthias@REDACTED (Matthias Lang) Date: Thu, 13 Jan 2011 11:16:50 +0100 Subject: [erlang-questions] Finding records in List In-Reply-To: References: Message-ID: <20110113101650.GA4297@corelatus.se> On Thursday, January 13, Arnaud GARCIA wrote: > I would like to find all records matching criterion/criteria in a list of > records. > if you have one criteria, you can use: Take a look at lists:filter() (List comprehensions will also do what you want, but if you're just starting out, take a look at lists:filter() first) Matt From bob@REDACTED Thu Jan 13 11:20:28 2011 From: bob@REDACTED (Bob Ippolito) Date: Thu, 13 Jan 2011 18:20:28 +0800 Subject: [erlang-questions] Finding records in List In-Reply-To: References: Message-ID: On Thu, Jan 13, 2011 at 6:06 PM, Arnaud GARCIA wrote: > Hello, > (still learning erlang ;-) > > I would like to find all records matching criterion/criteria in a list of > records. > if you have one criteria, you can use: > > ?lists:keyfind(Name,#person.name,ListOfPersonRecords). > > => But, it will return just the first one who match the criteria > =>Moreover with lists:keyfind I didn't find a way for multiple criteria like > : find all person in list where Name= and Age= > > I saw an interesting post using record<->proplists... is it the way to do it > ? > http://stackoverflow.com/questions/447188/erlang-and-run-time-record-limitations > > Finally, I build my own function which parse the List and check if it match > the criterion or criteria BUT I was asking if there is an erlang function to > do this ? Well, it depends on if you know what these things are at compile time or not. If you are doing this at compile time, you can simply use a list comprehension: [P || P <- people(), P#person.name =:= Name, P#person.age =:= Age]. -bob From bengt.kleberg@REDACTED Thu Jan 13 11:20:42 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 13 Jan 2011 11:20:42 +0100 Subject: [erlang-questions] Finding records in List In-Reply-To: References: Message-ID: <1294914042.5279.15.camel@seasc0694> Greetings, It sounds as if you want to filter a list of person records (Persons). That means either lists:filter/2 or a list comprehension. Both approaches can use the following function: is_name_and_age( Person, Name, Age ) -> (Person#person.name =:= Name) andalso (Person#person.age =:= Age) and the list comprehension would be: [X || X <- Persons, is_name_and_age( X, Name, Age )] bengt On Thu, 2011-01-13 at 11:06 +0100, Arnaud GARCIA wrote: > Hello, > (still learning erlang ;-) > > I would like to find all records matching criterion/criteria in a list of > records. > if you have one criteria, you can use: > > lists:keyfind(Name,#person.name,ListOfPersonRecords). > > => But, it will return just the first one who match the criteria > =>Moreover with lists:keyfind I didn't find a way for multiple criteria like > : find all person in list where Name= and Age= > > I saw an interesting post using record<->proplists... is it the way to do it > ? > http://stackoverflow.com/questions/447188/erlang-and-run-time-record-limitations > > Finally, I build my own function which parse the List and check if it match > the criterion or criteria BUT I was asking if there is an erlang function to > do this ? > > thanks, > > Arnaud From ivan@REDACTED Thu Jan 13 10:23:57 2011 From: ivan@REDACTED (Ivan Uemlianin) Date: Thu, 13 Jan 2011 09:23:57 +0000 Subject: [erlang-questions] We need a better marketing division :-) In-Reply-To: References: <24250519.271261294767163150.JavaMail.root@zimbra> <4D2D56B9.90103@lbox.cc> Message-ID: <4D2EC4AD.9040701@llaisdy.com> On 12/01/2011 09:49, Ulf Wiger wrote: > One thing that comes to mind is work done by the USAF Cognitive > Modelling Group in Scottsdale, AZ on using Erlang for very-large- > scale cognitive modeling. Part of the work has been published > > http://iccm2010.cs.drexel.edu/proceedings/papers/Douglass.pdf That is a very interesting paper, thank you. From time to time I google around for natural language processing research using Erlang (esp. interested in speech recognition). I've yet to find any. If anyone here knows of any such work I'd love to hear of it. Best wishes Ivan -- ============================================================ Ivan A. Uemlianin Speech Technology Research and Development ivan@REDACTED www.llaisdy.com llaisdy.wordpress.com www.linkedin.com/in/ivanuemlianin "Froh, froh! Wie seine Sonnen, seine Sonnen fliegen" (Schiller, Beethoven) ============================================================ From kostis@REDACTED Thu Jan 13 12:09:51 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 13 Jan 2011 13:09:51 +0200 Subject: [erlang-questions] picky dialyzer In-Reply-To: References: Message-ID: <4D2EDD7F.5040409@cs.ntua.gr> Ulf Wiger wrote: > This has me confused: > > Compiled src/jobs_server.erl > ==> jobs (analyze) > jobs_server.erl:1251: The pattern {'queue', _, _, {'producer', _}, _, _, _, _, _, _, _, _, _, _} can never match the type #queue{mod::atom(),type::'fifo' | #action{a::'approve' | 'reject'} | #passive{type::'fifo'},group::atom(),...} > > The line in question is > > q_is_empty(#queue{type = #producer{}}) -> false; > > > The #queue{} record definition is: > > -record(producer, {f :: mfa() | function()}). > -record(passive , {type = fifo :: fifo}). > -record(action , {a = approve :: approve | reject}). > > -record(queue, {name :: any(), > mod :: atom(), > type = fifo :: fifo | #producer{} | #passive{} | #action{}, > group :: atom(), > ?} > > Why does dialyzer think that #producer{} is not a valid member of > the type def for the 'type' attribute? Because it is very clever! :D You are looking at the tree and you are missing the forest... You are concentrating at the type definitions you have added, when you should be examining the code you wrote. The code of q_is_empty/1 reads: q_is_empty(#queue{type = #producer{}}) -> false; q_is_empty(#queue{mod = Mod} = Q) -> Mod:is_empty(Q). and this is a module-local function, meaning it will only be used from within this module. The only place in the module where there are calls to this function is from; check_queue(#queue{type = #producer{}} = Q, TS, S) -> do_check_queue(Q, TS, S); check_queue(Q, TS, S) -> case q_is_empty(Q) of true -> ...; false -> ... end. At the point where the call to q_is_empty/1 is, dialyzer knows that Q will be a #queue{} record where type is not a #producer{}. In other words, there will never be any calls that will match the first clause of q_is_empty/1. So, this clause is unreachable. > The funny thing is that if I rename the #producer{} record to e.g. > #prod{} or #produce{}, dialyzer picks it up (but generates a bunch of > other problems, partly because the change wasn't done consistently throughout). > > This (or practically the same warning) can be reproduced by pulling the > latest version of http://github.com/esl/jobs and running ./rebar analyze Well, if I pull the latest version from this repository, I get many more dialyzer warnings when I analyze this file. For example, dialyzer complains that jobs_server.erl:991: The call erlang:spawn_monitor(M::atom(),F::atom(),A::byte()) will never return since it differs in the 3rd argument from the success typing arguments: (atom(),atom(),[any()]) and it is of course right. The problem is that your #producer{} definition is a buggy. It reads: -record(producer, {f = {erlang,error,[undefined_producer]} :: mfa() | function()}). mfa() is an alias for {atom(),atom(),byte()}. You want to write {atom(),atom(),[term()]} instead of mfa() there. Cheers, Kostis From ulf.wiger@REDACTED Thu Jan 13 13:34:16 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 13 Jan 2011 13:34:16 +0100 Subject: [erlang-questions] picky dialyzer In-Reply-To: <4D2EDD7F.5040409@cs.ntua.gr> References: <4D2EDD7F.5040409@cs.ntua.gr> Message-ID: <8548EADD-542F-42E9-9ED3-FC1FE5523BFF@erlang-solutions.com> On 13 Jan 2011, at 12:09, Kostis Sagonas wrote: > Because it is very clever! :D > > You are looking at the tree and you are missing the forest... You are concentrating at the type definitions you have added, when you should be examining the code you wrote. The code of q_is_empty/1 reads: Well, I didn't really question that dialyzer had a real complaint, only that the warning became a bit confusing when it obviously used the static type definition as a starting point, but then pruned it, rather than saying - as it actually does on occasion - that it was a derived type it was comparing to. > Well, if I pull the latest version from this repository, I get many more dialyzer warnings when I analyze this file. For example, dialyzer complains that *Many* more? I guess I'll have to try and rebuild the plt, or ask you what special dialyzer version you're using. :) I'm using OTP R14B here. (?with plt rebuilt, no extra warnings surfaced) > jobs_server.erl:991: The call erlang:spawn_monitor(M::atom(),F::atom(),A::byte()) will never return since it differs in the 3rd argument from the success typing arguments: (atom(),atom(),[any()]) > > and it is of course right. > > The problem is that your #producer{} definition is a buggy. It reads: > > -record(producer, {f = {erlang,error,[undefined_producer]} :: mfa() | function()}). > > mfa() is an alias for {atom(),atom(),byte()}. Ahh, as in {Module, Function, Arity}? In the System Limits of the Efficiency Guide, it says that you can have 256 arguments in a function or fun. Since dialyzer is never wrong, I will assume that the Efficiency Guide is? :) > You want to write {atom(),atom(),[term()]} instead of mfa() there. I did add a type mfa() that had exactly that definition, but dialyzer complained that mfa() couldn't be redefined. It didn't occur to me that it could be defined as something other than {Module, Function, Args}. Now I know. Thanks for your help. BR, Ulf Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From alexey.v.romanov@REDACTED Thu Jan 13 14:05:06 2011 From: alexey.v.romanov@REDACTED (Alexey Romanov) Date: Thu, 13 Jan 2011 16:05:06 +0300 Subject: Possible reasons for heart not terminating after saying it's going to? Message-ID: On startup of my application, I've got the message {"Kernel pid terminated",application_controller,"{application_start_failure,gmcontroller,{{{badmatch,{error,enoent}},[{eeprom,parse,1},{config,read,1},{gmc_root_sup,init,1},{supervisor,init,1},{gen_server,init_it,6},{proc_lib,init_p_do_apply,3}]},{gmcontroller,start,[normal,[]]}}}"} heart: Sat Jan 1 21:53:26 2000: Erlang has closed. heart: Sat Jan 1 21:53:26 2000: Would reboot. Terminating. and, then it didn't terminate (at least after waiting for 10 minutes). ^C stopped the process without problems. Yours, Alexey Romanov From kostis@REDACTED Thu Jan 13 14:09:27 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 13 Jan 2011 15:09:27 +0200 Subject: [erlang-questions] picky dialyzer In-Reply-To: <8548EADD-542F-42E9-9ED3-FC1FE5523BFF@erlang-solutions.com> References: <4D2EDD7F.5040409@cs.ntua.gr> <8548EADD-542F-42E9-9ED3-FC1FE5523BFF@erlang-solutions.com> Message-ID: <4D2EF987.2010706@cs.ntua.gr> Ulf Wiger wrote: > On 13 Jan 2011, at 12:09, Kostis Sagonas wrote: > .... >> mfa() is an alias for {atom(),atom(),byte()}. > > Ahh, as in {Module, Function, Arity}? > > In the System Limits of the Efficiency Guide, it says that you can have > 256 arguments in a function or fun. Since dialyzer is never wrong, I will > assume that the Efficiency Guide is? :) Let's see... %%========================================================= -module(f_256). -export([f/256]). f(_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_) -> ok. %%========================================================== % erlc f_256.erl % ~/HiPE/otp/bin/erl Erlang R14B02 (erts-5.8.3) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.3 (abort with ^G) 1> l(f_256). {"Kernel pid terminated",application_controller,"{application_terminated,kernel,shutdown}"} Crash dump was written to: erl_crash.dump Kernel pid terminated (application_controller) ({application_terminated,kernel,shutdown}) Well, what can I say? Dialyzer is never wrong! ;-) I guess the efficiency guide needs some fixing. Also, the compiler should better complain in such cases instead of generating a bogus .beam file. Kostis From kenji.rikitake@REDACTED Thu Jan 13 14:17:29 2011 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Thu, 13 Jan 2011 22:17:29 +0900 Subject: ACM Erlang Workshop 2011, 23-SEP-2011, Tokyo, Japan Message-ID: <20110113131729.GA14516@k2r.org> This is a very early announcement to let you know that there will be the Tenth ACM SIGPLAN Erlang Workshop on the 23rd September 2011, held as a colocated event of ICFP 2011, at National Institute of Informatics, Tokyo, Japan. The Call For Papers and more details will be coming soon. Regards, Kenji Rikitake From michael.eugene.turner@REDACTED Thu Jan 13 15:20:57 2011 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Thu, 13 Jan 2011 23:20:57 +0900 Subject: [erlang-questions] We need a better marketing division :-) In-Reply-To: References: <24250519.271261294767163150.JavaMail.root@zimbra> <4D2D56B9.90103@lbox.cc> Message-ID: Torben: "... right now most Erlangers are busy trying to solve real world problems in order to make money so applying for this type of grants is not on the top of the agenda." And I'm reminded of the software company (name forgotten) who said he spent a lot of time worrying about marketing until he figured out that his best marketing move was always to keep making the product better. When you've got people characterizing you as "the best-kept secret" in a niche, your marketing troubles are over -- everybody wants to know a best-kept secret, and they *will* find out. Torben again: "Right now Green IT - in particular Energy Management - seems like an area where Erlang would be a good tool and there are plenty of grants and public funding floating around... just check various EU pages." These do seem like good bets. Erlang has some potential for making IT greener simply because it can use a bunch of server hardware more efficiently than the typical LAMP stack. As for energy management, as power use gets smarter, it starts to resemble telephony -- probably, within a decade or so, many light switches in new buildings will contain low-power sensors and smarts for detecting an approaching hand, so that they can send a message to the intelligent grid like, "There's a 90% probability of needing x more Watts about a half-second from now." People don't realize that instant-on electricity has systemic costs, but it does. -michael turner On Thu, Jan 13, 2011 at 6:24 PM, Torben Hoffmann wrote: > > > General observation on a fundamental difference between Scala and Erlang > regarding the supporting organisations (as I see it, YMMV): > > - Scala was born out of an academic environment where there is a strong > tradition for applying for grants to support research. > - Erlang was born in a private company trying to solve a problem they had > and was only later released as open source. > > And right now most Erlangers are busy trying to solve real world problems > in > order to make money so applying for this type of grants is not on the top > of > the agenda. > > I am - and would suggest to other to do the same - looking at areas where > Erlang could be used to solve a fundamental problem in that domain and > there > are in fact a lot of grants and/or public funding hanging around. > > Right now Green IT - in particular Energy Management - seems like an area > where Erlang would be a good tool and there are plenty of grants and public > funding floating around... just check various EU pages. > > Cheers, > Torben > -- > http://www.linkedin.com/in/torbenhoffmann > From dmercer@REDACTED Thu Jan 13 16:17:22 2011 From: dmercer@REDACTED (David Mercer) Date: Thu, 13 Jan 2011 09:17:22 -0600 Subject: [erlang-questions] Finding records in List In-Reply-To: <1294914042.5279.15.camel@seasc0694> References: <1294914042.5279.15.camel@seasc0694> Message-ID: <010601cbb334$faea9520$f0bfbf60$@com> On Thursday, January 13, 2011, Bengt Kleberg wrote: > is_name_and_age( Person, Name, Age ) -> > (Person#person.name =:= Name) andalso (Person#person.age =:= Age) Question about style: is there any preference between what Bengt wrote, and: is_name_and_age(#person{name = Name, age = Age}, Name, Age) -> true; is_name_and_age(_,_,_) -> false. In fact, I think I like Bengt's better, since it is more clear about intent, but I'd have written it as the latter figuring that that was more idiomatic. Are records in match specifications out of favor? (To a certain extent, if they are, good.) Cheers, DBM From rzezeski@REDACTED Thu Jan 13 16:51:46 2011 From: rzezeski@REDACTED (Ryan Zezeski) Date: Thu, 13 Jan 2011 10:51:46 -0500 Subject: [erlang-questions] Re: error: Removing (timedout) connection In-Reply-To: <4D2D5E97.1000007@amberbio.com> References: <4D2D5E97.1000007@amberbio.com> Message-ID: http://www.erlang.org/cgi-bin/ezmlm-cgi?4:mss:53287:201009:fheblnkablgeikajmkoa I feel dumb now, because I made a comment on that thread :) Thanks Morten! -Ryan On Wed, Jan 12, 2011 at 2:56 AM, Morten Krogh wrote: > Ryan, see the thread "node to node message passing" from September last > year. > > Morten. > > > On 1/12/11 8:15 AM, Ryan Zezeski wrote: > >> Update: >> >> I know that a big part of my problem was running that map-reduce query on >> every node at the same time. That's an expensive operation, especially >> since I'm using the filesystem to back my luwak cluster. With that in >> mind, >> I went ahead and migrated to riak-search and wrote a custom >> extractor/schema >> to index my luwak files. The system, at least for the last 4+ hours, has >> shown much more stability than what it had previously. In fact, I've only >> seen 4 of the timed-out connection errors thus far. Furthermore, I >> believe >> I've pinpointed the cause of these errors. All 4 of them occurred in a >> particularly nasty piece of code where I call ets:tab2file, then read the >> file into a binary, then zlib:gzip it and then finally send the compressed >> binary to luwak via the native Erlang client. >> >> I'm wondering about that last part, sending it via Erlang external format >> as >> a binary. Even compressed, these binaries can be as large as 120M! Would >> this be a potential problem, possibly delaying the net kernel ticks >> between >> nodes and causing my timeouts? I imagine using the riak protocol buffers >> interface might be a better choice? >> >> Thanks, >> >> -Ryan >> >> On Mon, Jan 10, 2011 at 2:20 PM, Ryan Zezeski wrote: >> >> Hi guys/gals, >>> >>> Recently I've been converting my non-distributed Erlang app into a >>> distributed one and I ran into some troubles. If you want to skip >>> straight >>> to the question it's at the end, but I try to give some insight into what >>> I'm doing below. >>> >>> First off, I attached a PDF (sorry, PDF was not my choice) which contains >>> a >>> diagram I drew of the current setup. I apologize for my utter failure as >>> an >>> artist. In this diagram you'll see 3 vertical partitions representing 3 >>> different machines and a horizontal one representing the fact that each >>> machine has 2 Erland nodes on it. 3 of the Erlang nodes form a riak >>> cluster. The other 3 are the application (or should I say release) I >>> wrote, >>> and to distribute my app I utilized riak's underlying technology, >>> riak_core >>> (I use it as an easy way to persist cluster membership and use the ring >>> metadata to store some data). These six nodes are fully connected, i.e. >>> each node has connection to the other. >>> >>> Occasionally, I've noticed the following message on any one of the six >>> nodes: >>> >>> =ERROR REPORT==== ... >>> ** Node not responding ** >>> ** Removing (timedout) connection ** >>> >>> Furthermore, using net_kernel:monitor_nodes(true, [nodedown_reason]) I've >>> noticed messages like the following: >>> >>> {nodedown,, [{nodedown_reason, connection_closed}]} >>> >>> >>> You'll notice there is a system process running on machine A, and it >>> makes >>> a gen_server:cast to three processes to do some work, and these processes >>> each call link (L). Each of these three (gen_server) processes makes a >>> call >>> (at roughly the same time) to the riak cluster performing the _same >>> exact_ >>> map/reduce job. Sometimes I'll see errors where this map/reduce job >>> times >>> out on one of the nodes. So at lunch, I wondered, is it because there is >>> just too much communication going on between the nodes that the kernel >>> ticks >>> are getting lost or delayed? I wondered if each node was using the same >>> TCP >>> connection to talk to every other node. That could explain my symptoms, >>> right? A few netcats later and I realized that it's a dedicated conn for >>> each node, so that theory was blown. However, I still think that many >>> msgs >>> being passed back and forth could be the cause of the problem, and I >>> wondered if it blocks the VM in some way so that the kernel tick can't >>> get >>> through? >>> >>> >>> Q: Can a chatty cluster cause the kernel ticks to be lost/delayed thus >>> causing nodes to disconnect from each other? >>> >>> Thanks, >>> >>> -Ryan >>> >>> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From gijsbert.de.haan@REDACTED Thu Jan 13 16:52:23 2011 From: gijsbert.de.haan@REDACTED (Gijsbert) Date: Thu, 13 Jan 2011 07:52:23 -0800 (PST) Subject: how to patch a standard module Message-ID: <73cd3e8a-d6b4-4910-a2f2-9359252a3288@o4g2000yqd.googlegroups.com> Hi, I'd like to extend httpd_request.erl to handle the OPTIONS method. I've managed to do this by copying httpd_request.erl into my project and making sure it overrides the standard version. My question: is it possible to redefine a function in a standard module at runtime? Then I could define my own validate(Method, Uri, Version) function (just a few lines) and keep using the standard httdp_request.erl. Regards, Gijsbert From attila.r.nohl@REDACTED Thu Jan 13 18:40:00 2011 From: attila.r.nohl@REDACTED (Attila Rajmund Nohl) Date: Thu, 13 Jan 2011 18:40:00 +0100 Subject: String continuation Message-ID: Hello! Let's have a look at this little example code: -module('strc'). -export([f/0]). f() -> {["string1", "string2"], ["string3" "string4"]}. This compiles fine and returns a tuple with two lists, the first with 2 strings, the second with a single list. Of course, if there was only a typo in the second list (a missed comma), I wouldn't notice it until the code executes (which might happen in an inconvenient time in error handling code). I wonder that the possibility to omit the extra ++ from the end of "string3" line worth the problems what might be caused by the missing comma. This only bit me once, so it might not be that common... From xramtsov@REDACTED Thu Jan 13 19:07:30 2011 From: xramtsov@REDACTED (Evgeniy Khramtsov) Date: Fri, 14 Jan 2011 03:07:30 +0900 Subject: [erlang-questions] String continuation In-Reply-To: References: Message-ID: <4D2F3F62.2050804@gmail.com> 14.01.2011 02:40, Attila Rajmund Nohl wrote: > Hello! > > Let's have a look at this little example code: > > -module('strc'). > > -export([f/0]). > > f() -> > {["string1", > "string2"], > ["string3" > "string4"]}. > > This compiles fine and returns a tuple with two lists, the first with > 2 strings, the second with a single list. Of course, if there was only > a typo in the second list (a missed comma), I wouldn't notice it until > the code executes (which might happen in an inconvenient time in error > handling code). I wonder that the possibility to omit the extra ++ > from the end of "string3" line worth the problems what might be caused > by the missing comma. This only bit me once, so it might not be that > common... > ++ operator is slow. It is better to use dialyzer in order to track such errors. -- Regards, Evgeniy Khramtsov, ProcessOne. xmpp:xram@REDACTED From ulf.wiger@REDACTED Thu Jan 13 19:22:51 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 13 Jan 2011 19:22:51 +0100 Subject: [erlang-questions] String continuation In-Reply-To: <4D2F3F62.2050804@gmail.com> References: <4D2F3F62.2050804@gmail.com> Message-ID: On 13 Jan 2011, at 19:07, Evgeniy Khramtsov wrote: > ++ operator is slow. It is better to use dialyzer in order to track such errors. Efficiency Guide, Chapter 2.4 - Myth: '++' is always bad http://erlang.org/doc/efficiency_guide/myths.html#id54761 BR, Ulf W Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From jameschurchman@REDACTED Thu Jan 13 22:15:26 2011 From: jameschurchman@REDACTED (James Churchman) Date: Thu, 13 Jan 2011 21:15:26 +0000 Subject: [erlang-questions] String continuation In-Reply-To: References: Message-ID: <1149ED0F-F7B3-4307-B51F-B5C882FD821F@gmail.com> i think for what your are asking, its designed to allow strings to elegantly span lines. In JS for example its very common to want multiline strings (for html for example), and people do the following "really really long string part 1" + "really really long string part 2" + "really really long string part 3" to get around this limitation of multi line spanning strings ( you cant even do this for example, and have it include the line breakes "long line 1 long line 2" ) people noted however this is quite inefficient as it involves (potentially dozens) of string concatenations all at run time, often in loops most JS compressors now will go and optimise it into a huge one-liner, as this provides a good speed up & people will never read this optimised version i would personally rather just type "really really long string part 1" "really really long string part 2" "really really long string part 3" than the former but i guess its less common in erlang to have large chunks of text so its a bit less relevant.. still something i appreciate tho On 13 Jan 2011, at 17:40, Attila Rajmund Nohl wrote: > Hello! > > Let's have a look at this little example code: > > -module('strc'). > > -export([f/0]). > > f() -> > {["string1", > "string2"], > ["string3" > "string4"]}. > > This compiles fine and returns a tuple with two lists, the first with > 2 strings, the second with a single list. Of course, if there was only > a typo in the second list (a missed comma), I wouldn't notice it until > the code executes (which might happen in an inconvenient time in error > handling code). I wonder that the possibility to omit the extra ++ > from the end of "string3" line worth the problems what might be caused > by the missing comma. This only bit me once, so it might not be that > common... > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From robert.virding@REDACTED Thu Jan 13 22:16:23 2011 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 13 Jan 2011 21:16:23 +0000 (GMT) Subject: [erlang-questions] String continuation In-Reply-To: <1521212432.286221294953232480.JavaMail.root@zimbra> Message-ID: <288128613.286241294953383364.JavaMail.root@zimbra> ----- "Ulf Wiger" wrote: > On 13 Jan 2011, at 19:07, Evgeniy Khramtsov wrote: > > > ++ operator is slow. It is better to use dialyzer in order to track > such errors. > > > Efficiency Guide, Chapter 2.4 - Myth: '++' is always bad > http://erlang.org/doc/efficiency_guide/myths.html#id54761 It gets even better. The compiler will at compile time optimise "string3" ++ "string4" to "string3string4". So: "string3" "string4" --> "string3string4" during parsing "string3" ++ "string4" --> "string3string4" during compiling There is no escape. Robert -- Robert Virding, Erlang Solutions Ltd. From alexey.v.romanov@REDACTED Thu Jan 13 22:17:48 2011 From: alexey.v.romanov@REDACTED (Alexey Romanov) Date: Fri, 14 Jan 2011 00:17:48 +0300 Subject: [erlang-questions] String continuation In-Reply-To: <4D2F3F62.2050804@gmail.com> References: <4D2F3F62.2050804@gmail.com> Message-ID: ++ _between string literals_ is slow? I'd expect it to be optimized by the compiler just like "str1" "str2" is. Yours, Alexey Romanov On Thu, Jan 13, 2011 at 9:07 PM, Evgeniy Khramtsov wrote: > 14.01.2011 02:40, Attila Rajmund Nohl wrote: >> >> Hello! >> >> Let's have a look at this little example code: >> >> -module('strc'). >> >> -export([f/0]). >> >> f() -> >> ? ? {["string1", >> ? ? ? "string2"], >> ? ? ?["string3" >> ? ? ? "string4"]}. >> >> This compiles fine and returns a tuple with two lists, the first with >> 2 strings, the second with a single list. Of course, if there was only >> a typo in the second list (a missed comma), I wouldn't notice it until >> the code executes (which might happen in an inconvenient time in error >> handling code). I wonder that the possibility to omit the extra ++ >> from the end of "string3" line worth the problems what might be caused >> by the missing comma. This only bit me once, so it might not be that >> common... >> > > ++ operator is slow. It is better to use dialyzer in order to track such > errors. > > -- > Regards, > Evgeniy Khramtsov, ProcessOne. > xmpp:xram@REDACTED > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From mazen.harake@REDACTED Thu Jan 13 23:23:34 2011 From: mazen.harake@REDACTED (Mazen Harake) Date: Thu, 13 Jan 2011 23:23:34 +0100 Subject: [erlang-questions] String continuation In-Reply-To: References: Message-ID: <4D2F7B66.4040504@erlang-solutions.com> I think you are confusing the two. Writing two strings without anything in between is a literal and not a statement of concatenation of both. This means that: f() -> "Hello" "World". is the same as... f() -> "HelloWorld". and not f() -> "Hello" ++ "World". /Mazen On 13/01/2011 18:40, Attila Rajmund Nohl wrote: > Hello! > > Let's have a look at this little example code: > > -module('strc'). > > -export([f/0]). > > f() -> > {["string1", > "string2"], > ["string3" > "string4"]}. > > This compiles fine and returns a tuple with two lists, the first with > 2 strings, the second with a single list. Of course, if there was only > a typo in the second list (a missed comma), I wouldn't notice it until > the code executes (which might happen in an inconvenient time in error > handling code). I wonder that the possibility to omit the extra ++ > from the end of "string3" line worth the problems what might be caused > by the missing comma. This only bit me once, so it might not be that > common... > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From robert.virding@REDACTED Thu Jan 13 22:21:46 2011 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 13 Jan 2011 21:21:46 +0000 (GMT) Subject: [erlang-questions] Finding records in List In-Reply-To: <1049191250.286271294953556342.JavaMail.root@zimbra> Message-ID: <2138761059.286291294953706021.JavaMail.root@zimbra> ----- "David Mercer" wrote: > On Thursday, January 13, 2011, Bengt Kleberg wrote: > > > is_name_and_age( Person, Name, Age ) -> > > (Person#person.name =:= Name) andalso (Person#person.age =:= Age) > > Question about style: is there any preference between what Bengt > wrote, and: > > is_name_and_age(#person{name = Name, age = Age}, Name, Age) -> true; > is_name_and_age(_,_,_) -> false. > > In fact, I think I like Bengt's better, since it is more clear about > intent, > but I'd have written it as the latter figuring that that was more > idiomatic. > Are records in match specifications out of favor? (To a certain > extent, if > they are, good.) Hopefully records in matches are not out of favour, it is the most erlangy way of doing it. Otherwise it would be like using element/2 instead of matching tuples. Exactly like actually. The match version is probably more efficient. Robert -- Robert Virding, Erlang Solutions Ltd. From ponton@REDACTED Thu Jan 13 22:25:54 2011 From: ponton@REDACTED (Tomasz Maciejewski) Date: Thu, 13 Jan 2011 22:25:54 +0100 Subject: [erlang-questions] how to patch a standard module In-Reply-To: <73cd3e8a-d6b4-4910-a2f2-9359252a3288@o4g2000yqd.googlegroups.com> References: <73cd3e8a-d6b4-4910-a2f2-9359252a3288@o4g2000yqd.googlegroups.com> Message-ID: Dnia 13-01-2011 o 16:52:23 Gijsbert napisa?(a): > Hi, > > I'd like to extend httpd_request.erl to handle the OPTIONS method. > I've managed to do this by copying httpd_request.erl into my project > and making sure it overrides the standard version. > My question: is it possible to redefine a function in a standard > module at runtime? Then I could define my own validate(Method, Uri, > Version) function (just a few lines) and keep using the standard > httdp_request.erl. > > Regards, > Gijsbert If you like to extend httpd, you should write a module for it, instead of patching original http_request.erl. All the informations are here: http://www.erlang.org/doc/man/httpd.html. You can also see how mod_get.erl and mod_head.erl are done. For redefining function at runtime, you can compile and load modified module by typing "c(module)", or just load new beam by "l(module)". -- Tomasz Maciejewski From jameschurchman@REDACTED Thu Jan 13 22:41:17 2011 From: jameschurchman@REDACTED (James Churchman) Date: Thu, 13 Jan 2011 21:41:17 +0000 Subject: [erlang-questions] String continuation In-Reply-To: <288128613.286241294953383364.JavaMail.root@zimbra> References: <288128613.286241294953383364.JavaMail.root@zimbra> Message-ID: <23A78EF7-7DE0-4D08-AC7C-E763645CDC9F@gmail.com> nice :-) On 13 Jan 2011, at 21:16, Robert Virding wrote: > It gets even better. The compiler will at compile time optimise "string3" ++ "string4" to "string3string4". So: From ulf.wiger@REDACTED Thu Jan 13 22:47:55 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 13 Jan 2011 22:47:55 +0100 Subject: [erlang-questions] String continuation In-Reply-To: <4D2F7B66.4040504@erlang-solutions.com> References: <4D2F7B66.4040504@erlang-solutions.com> Message-ID: <2F728F5C-CE3A-4D7E-8C33-2F89C7706940@erlang-solutions.com> Actually, and you can verify this with erlc -S, they are _all_ the same, as Robert said: f() -> "a" "b". g() -> "ab". h() -> "a" ++ "b". compiles to: {function, f, 0, 2}. {label,1}. {func_info,{atom,strs},{atom,f},0}. {label,2}. {move,{literal,"ab"},{x,0}}. return. {function, g, 0, 4}. {label,3}. {func_info,{atom,strs},{atom,g},0}. {label,4}. {move,{literal,"ab"},{x,0}}. return. {function, h, 0, 6}. {label,5}. {func_info,{atom,strs},{atom,h},0}. {label,6}. {move,{literal,"ab"},{x,0}}. return. BR, Ulf W On 13 Jan 2011, at 23:23, Mazen Harake wrote: > I think you are confusing the two. Writing two strings without anything in between is a literal and not a statement of concatenation of both. This means that: > > f() -> "Hello" "World". > > is the same as... > > f() -> "HelloWorld". > > and not > > f() -> "Hello" ++ "World". > > > /Mazen > > On 13/01/2011 18:40, Attila Rajmund Nohl wrote: >> Hello! >> >> Let's have a look at this little example code: >> >> -module('strc'). >> >> -export([f/0]). >> >> f() -> >> {["string1", >> "string2"], >> ["string3" >> "string4"]}. >> >> This compiles fine and returns a tuple with two lists, the first with >> 2 strings, the second with a single list. Of course, if there was only >> a typo in the second list (a missed comma), I wouldn't notice it until >> the code executes (which might happen in an inconvenient time in error >> handling code). I wonder that the possibility to omit the extra ++ >> from the end of "string3" line worth the problems what might be caused >> by the missing comma. This only bit me once, so it might not be that >> common... >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From kostis@REDACTED Thu Jan 13 22:48:40 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Thu, 13 Jan 2011 23:48:40 +0200 Subject: [erlang-questions] Finding records in List In-Reply-To: <010601cbb334$faea9520$f0bfbf60$@com> References: <1294914042.5279.15.camel@seasc0694> <010601cbb334$faea9520$f0bfbf60$@com> Message-ID: <4D2F7338.4050404@cs.ntua.gr> David Mercer wrote: > On Thursday, January 13, 2011, Bengt Kleberg wrote: > >> is_name_and_age( Person, Name, Age ) -> >> (Person#person.name =:= Name) andalso (Person#person.age =:= Age) > > Question about style: is there any preference between what Bengt wrote, and: > > is_name_and_age(#person{name = Name, age = Age}, Name, Age) -> true; > is_name_and_age(_,_,_) -> false. > > In fact, I think I like Bengt's better, since it is more clear about intent, > but I'd have written it as the latter figuring that that was more idiomatic. Bengt's version, although more type-correct than yours, IMO is verbose without any particular reason. If you want a more compact version with only one clause, which is another advantage I see in Bengt's version, write the following: is_name_and_age(#person{name = N, age = A}, Name, Age) -> N =:= Name andalso A =:= Age. with or without parentheses around the andalso arguments. This version makes it crystal clear that the first argument has to be a #person{} record (and does not return 'false' by accident if called with something else in its first argument). It also has the advantage that the clause head can be read without having to pay special attention to the non-linearity of the clause head variables. Kostis From jameschurchman@REDACTED Thu Jan 13 22:39:47 2011 From: jameschurchman@REDACTED (James Churchman) Date: Thu, 13 Jan 2011 21:39:47 +0000 Subject: [erlang-questions] how to patch a standard module In-Reply-To: <73cd3e8a-d6b4-4910-a2f2-9359252a3288@o4g2000yqd.googlegroups.com> References: <73cd3e8a-d6b4-4910-a2f2-9359252a3288@o4g2000yqd.googlegroups.com> Message-ID: Lots of dif ways i can think of, but generally it's a bit of a bad idea as it goes down the multiple inheritance route of OO -include(?..). Usually works if : 1) your module is still named the same as the original 2) you are adding a function with a unique name/arity combination, and not changing, removing etc? any others 3) you set the include directory to the file, when you compile. So in your original module (that you name "httpd_request.erl" and have no module declaration in): -include("/usr/local/lib/erlang/lib/inets-5.5.1/src/http_server/httpd_request.erl"). %% this is dependant on your instalation, and i think include_lib wont work in this case, as it will try to include its self options(My_Input)->my_output. Then compile code with : compile:file("/Users/james/Desktop/httpd_request.erl",[{i,"/usr/local/lib/erlang/lib/inets-5.5.1/src/http_server/"}]). %% again dependant on your erlang installation and file location Modifying is more tricky and requires getting the AST source representation of the two modules ( probably in a parse transform of the 1st), removing one of the module names (as if you have two, it wont compile), then sorting all the functions to the bottom of the list then returning that to the parse transform. Sorting is required as erlang wont compile a file that has functions before attributes in the source / AST. This can be packaged up into a general thing however. Meck could also be used, which probably does a similar thing, but I think that it's only designed for testing and will revert to the old module on errors etc.. Is there defiantly no way to achieve what you want without extending the original code?? i would bet that there is! James On 13 Jan 2011, at 15:52, Gijsbert wrote: > Hi, > > I'd like to extend httpd_request.erl to handle the OPTIONS method. > I've managed to do this by copying httpd_request.erl into my project > and making sure it overrides the standard version. > My question: is it possible to redefine a function in a standard > module at runtime? Then I could define my own validate(Method, Uri, > Version) function (just a few lines) and keep using the standard > httdp_request.erl. > > Regards, > Gijsbert > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From paolo.negri@REDACTED Fri Jan 14 00:13:27 2011 From: paolo.negri@REDACTED (Paolo Negri) Date: Fri, 14 Jan 2011 00:13:27 +0100 Subject: about the use of parametrized modules Message-ID: Looking at the misultin source code [1] I noticed the use of a parametrized module, after googling about it I came to the conclusion that there's something controversial about them [2] and that it seems to be appealing mainly people coming from OO background [3]. I'm considering applying this technique but before doing so I'd like to know if isn't a deprecated approach and in general if it's considered a bad practice. I'm curious because I don't remember seeing parametrized modules used in Joe Armstron's or Francesco Cesarini's books. [1] http://github.com/ostinelli/misultin [2] http://stackoverflow.com/questions/1358230/erlang-module-attribute [3] http://myotherpants.com/2009/04/parameterized-modules-in-erlang Thanks for your help, Paolo From freza@REDACTED Fri Jan 14 00:26:52 2011 From: freza@REDACTED (Jachym Holecek) Date: Thu, 13 Jan 2011 23:26:52 +0000 Subject: [erlang-questions] about the use of parametrized modules In-Reply-To: References: Message-ID: <20110113232652.GA1737@hanele.lan> # Paolo Negri 2011-01-13: > Looking at the misultin source code [1] I noticed the use of a > parametrized module, after googling about it I came to the conclusion > that there's something controversial about them [2] and that it seems > to be appealing mainly people coming from OO background [3]. > > I'm considering applying this technique [...] Out of curiosity, what are your expactations from this move? IMVHO it only obfuscates flow of control without any real benefit. Regards, -- Jachym From gijsbert.de.haan@REDACTED Fri Jan 14 00:47:03 2011 From: gijsbert.de.haan@REDACTED (Gijsbert) Date: Thu, 13 Jan 2011 15:47:03 -0800 (PST) Subject: how to patch a standard module In-Reply-To: References: <73cd3e8a-d6b4-4910-a2f2-9359252a3288@o4g2000yqd.googlegroups.com> Message-ID: <216f04d8-c713-4f9f-9afd-b5c212311171@u32g2000yqe.googlegroups.com> On Jan 13, 10:25?pm, "Tomasz Maciejewski" wrote: > Dnia 13-01-2011 o 16:52:23 Gijsbert napisa?(a): > > > Hi, > > > I'd like to extend httpd_request.erl to handle the OPTIONS method. > > I've managed to do this by copying httpd_request.erl into my project > > and making sure it overrides the standard version. > > My question: is it possible to redefine a function in a standard > > module at runtime? Then I could define my own validate(Method, Uri, > > Version) function (just a few lines) and keep using the standard > > httdp_request.erl. > > > Regards, > > Gijsbert > > If you like to extend httpd, you should write a module for it, instead of patching original http_request.erl. All the informations are here:http://www.erlang.org/doc/man/httpd.html. You can also see how mod_get.erl and mod_head.erl are done. Alas, my module is never called because httpd_request:validate() does not have a clause for "OPTIONS" and returns {error, {not_implemented, ...}}. As far as I know I cannot extend the validation phase of httpd. > > For redefining function at runtime, you can compile and load modified module by typing "c(module)", or just load new beam by "l(module)". > > -- > Tomasz Maciejewski > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > Seehttp://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED From robert.virding@REDACTED Fri Jan 14 01:05:00 2011 From: robert.virding@REDACTED (Robert Virding) Date: Fri, 14 Jan 2011 00:05:00 +0000 (GMT) Subject: [erlang-questions] about the use of parametrized modules In-Reply-To: <1228820052.286871294963364227.JavaMail.root@zimbra> Message-ID: <770248544.286891294963500152.JavaMail.root@zimbra> ----- "Paolo Negri" wrote: > Looking at the misultin source code [1] I noticed the use of a > parametrized module, after googling about it I came to the conclusion > that there's something controversial about them [2] and that it seems > to be appealing mainly people coming from OO background [3]. > > I'm considering applying this technique but before doing so I'd like > to know if isn't a deprecated approach and in general if it's > considered a bad practice. > I'm curious because I don't remember seeing parametrized modules used > in Joe Armstron's or Francesco Cesarini's books. The main reason they don't mention it is because it is still only experimental and not part of the language. It is very controversial and often discussions about it pop up in erlang-questions. I personally don't like it. Robert -- Robert Virding, Erlang Solutions Ltd. From geoffrey.biggs@REDACTED Fri Jan 14 01:46:09 2011 From: geoffrey.biggs@REDACTED (Geoffrey Biggs) Date: Fri, 14 Jan 2011 09:46:09 +0900 Subject: [erlang-questions] UML or other modeling tools In-Reply-To: <2E275C30-C1EE-453B-8CE2-1C934F17A50F@erlang-solutions.com> References: <2E275C30-C1EE-453B-8CE2-1C934F17A50F@erlang-solutions.com> Message-ID: <4D2F9CD1.9030601@aist.go.jp> On 09/01/11 23:06, Ulf Wiger wrote: > - The things most interesting to model in an Erlang program are the > state machines and the interaction between processes, but UML > state machines have very complex - and most importantly, *different* > - semantics, making it quite probable that the design becomes subject > to compromises forced by the modeling tool, rather than reflecting the > power of the programming language. I wonder if you could expand on what these differences are. As a novice at both UML and Erlang, I don't want to fall into any traps caused by making false assumptions. Geoff From fred.hebert@REDACTED Fri Jan 14 01:50:08 2011 From: fred.hebert@REDACTED (=?iso-8859-1?Q?Fr=E9d=E9ric_Trottier-H=E9bert?=) Date: Thu, 13 Jan 2011 19:50:08 -0500 Subject: [erlang-questions] about the use of parametrized modules In-Reply-To: <770248544.286891294963500152.JavaMail.root@zimbra> References: <770248544.286891294963500152.JavaMail.root@zimbra> Message-ID: <67A49013-7102-455F-B3AD-C799BC436A91@erlang-solutions.com> On 2011-01-13, at 19:05 PM, Robert Virding wrote: > ----- "Paolo Negri" wrote: > >> Looking at the misultin source code [1] I noticed the use of a >> parametrized module, after googling about it I came to the conclusion >> that there's something controversial about them [2] and that it seems >> to be appealing mainly people coming from OO background [3]. >> >> I'm considering applying this technique but before doing so I'd like >> to know if isn't a deprecated approach and in general if it's >> considered a bad practice. >> I'm curious because I don't remember seeing parametrized modules used >> in Joe Armstron's or Francesco Cesarini's books. > > The main reason they don't mention it is because it is still only experimental and not part of the language. It is very controversial and often discussions about it pop up in erlang-questions. I personally don't like it. > > Robert > > -- > Robert Virding, Erlang Solutions Ltd. > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > I will echo the sentiment. My reasons for disliking parametrized modules have been written in a stackoverflow.com rant before (http://stackoverflow.com/questions/2291155/what-alternatives-are-there-to-parameterised-modules-in-erlang/2291394#2291394), but I'll rehash them here. As far as I know, parametrized modules were first added to the language as a way to deal with services that would take callbacks, to which you need to add your own state, except that: - The server doesn't allow to pass specific extra arguments - The server wants a module instead of a closure (frequently the case) - Meta-programming or hardcoding different variations is too much of a burden - Holding some kind of reference to global state is an annoyance and usually a code smell. (see: http://www.erlang.se/workshop/2003/paper/p29-carlsson.pdf) In these case, parametrized modules would allow you to encapsulate some state along with the callback module and would allow you to solve that problem. The current implementation, as far as I know, depends on the old way to call funs with a tuple and arguments that has been deprecated for quite a while now. The mechanisms to handle it were apparently still in the VM and used to let parametrized modules be. Whether parametrized modules are the right solution to that problem or not is not really my judgment call to make, given I can't think of a better approach at the moment. The problem I have with parametrized modules is that they are used in many ways except the one for which they were [unofficially] introduced into the language. A lot of people use them as a pseudo-object or global variables (I'm looking at you, chicagoboss), or as a way to have some syntactic sugar around passing a state parameter to callback functions (Now I'm looking at you, mochiweb). This, in my opinion, is shoehorning OO patterns into Erlang, in a way that confuses many newcomers (we see them from time to time on IRC). It also uselessly adds new semantics to the language: is the new/N function safe to use for other reasons? Why are there 3 ways to call modules now (one returns an updatable data structure, one doesn't because a process holds it, and one returns a new module instance)? It also causes ghosting issues , because the parametrized variables act like global variables in the module, which you might overwrite with function arguments or thought could be used safely in pattern matches. The 'case ... of' expression comes to mind right now -- people have complained a bit about how its scoping rules are weird; parametrized modules add to that, because you can now potentially pattern match on variables that might or might not work with your current patterns depending on how you called them, and this without any clue of where it's coming from (especially for large modules) unless you analyse it later. I have frequently avoided them, but earlier this week, someone on IRC complained that parametrized modules basically messed up a lot of contracts and angered Dialyzer because of it. If using them means type-checking your code is harder (or you have to accept errors all the time), I just want to pass my turn. In my opinion, they are used as a shortcut so you type less, but in the long run, I firmly believe it does it at the cost of readability, ease of understanding code and ease of maintenance while making Erlang more complex as a language. /Rant -- Fred H?bert http://www.erlang-solutions.com From kuleshovmail@REDACTED Fri Jan 14 07:13:42 2011 From: kuleshovmail@REDACTED (shk) Date: Thu, 13 Jan 2011 22:13:42 -0800 (PST) Subject: Compile all modules in emacs Message-ID: <1294985622095-3217190.post@n4.nabble.com> Hello, I user GNU/Emacs + Erlang-mode for erlang programming. I use C-c C-k for compiling current buffer. For example i have 3 *.erl files: test1.erl test2.erl test3.erl Theese file opened in emacs in deifferent buffer. How can i caompile all files at one times? Thank you. -- View this message in context: http://erlang.2086793.n4.nabble.com/Compile-all-modules-in-emacs-tp3217190p3217190.html Sent from the Erlang Questions mailing list archive at Nabble.com. From torben.lehoff@REDACTED Fri Jan 14 08:28:18 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Fri, 14 Jan 2011 08:28:18 +0100 Subject: [erlang-questions] UML or other modeling tools In-Reply-To: References: <2E275C30-C1EE-453B-8CE2-1C934F17A50F@erlang-solutions.com> <8483A9E6-5AB9-4A1E-9920-A07B8A305460@gmail.com> Message-ID: I am using OPM (http://en.wikipedia.org/wiki/Object_Process_Methodology) for my high level drawings. OPM has the nice aspect that it allows you to treat both objects and processes as 1st level citizens of the language. (Mind you the processes are not a 1:1 with Erlang processes!!). OPM has both a graphical and a textual representation that matches up 1:1 - I have not had the time to try out the OPCAT tool yet, so I cannot say how that works. So far I have used my own Dia stencil with OPM shapes to draw things. As a supplement I use MSCs, where http://plantuml.sourceforge.net/ is a nice tool to create them. I have been contemplating extending PlantUML with OPM or write a simple Dot based tool for OPM, but that is the kind of activity that requires a bit of time to get right and right now I am busy trying to make money, I am content with my Dia drawings right now. Cheers, Torben On Mon, Jan 10, 2011 at 15:11, Alain O'Dea wrote: > On 2011-01-10, at 5:35, Ulf Wiger wrote: > > > > > On 9 Jan 2011, at 22:30, Alain O'Dea wrote: > > > >> On 2011-01-09, at 10:36, Ulf Wiger > wrote: > >> > >>> - The FMC (http://fmc-modeling.org/) notation seems to complement > >>> Erlang fairly well, describing roughly the things that are most > interesting > >>> to describe visually in an Erlang program. > >> > >> Hi Ulf: > >> > >> What success stories can you share about using FMC? > > > > None, actually. It would be interesting to hear from someone who's > > actually tried it. :) > > > > The most ambitious story around FMC is The Apache Modelling Project > > > > > http://www.fmc-modeling.org/download/projects/apache/the_apache_modelling_project.pdf > > > > as far as I know. > > > > Last time I looked at it, the state of FMC-capable editors was a bit > > troubling. MS Visio supposedly supports FMC notation, as well as > > OpenOffice. Oryx is an Open Source project and an online editor, > > but it failed to convert me, at least. > > > > http://bpt.hpi.uni-potsdam.de/Oryx/FMC > > > > BR, > > Ulf W > > > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > > http://erlang-solutions.com > > FMC looks easy enough to draw. One key for a modeling system for me is the > ease with which it can be drawn on a flip-chart or whiteboard. > > Tools that understand the semantics are less important to me. I imagine > that OmniGraffle/Visio would work reasonably since they support the magnets > and connectors elements needed to make the editing comfortable. A more > purposed editor would be interesting, but in my experience they limit the > expression and annotation too much to be generally useful. -- http://www.linkedin.com/in/torbenhoffmann From sedrik@REDACTED Fri Jan 14 08:44:07 2011 From: sedrik@REDACTED (Fredrik Andersson) Date: Fri, 14 Jan 2011 08:44:07 +0100 Subject: [erlang-questions] Compile all modules in emacs In-Reply-To: <1294985622095-3217190.post@n4.nabble.com> References: <1294985622095-3217190.post@n4.nabble.com> Message-ID: I am no emacs user but having a proper Makefile you should be able to just bind a key combination to run make and have it detect and recompile files that needs to be recompiled. Is this not sufficient for you? On Fri, Jan 14, 2011 at 7:13 AM, shk wrote: > > Hello, > > I user GNU/Emacs + Erlang-mode for erlang programming. I use C-c C-k for > compiling current buffer. For example i have 3 *.erl files: > > test1.erl > test2.erl > test3.erl > > Theese file opened in emacs in deifferent buffer. How can i caompile all > files at one times? > > Thank you. > -- > View this message in context: > http://erlang.2086793.n4.nabble.com/Compile-all-modules-in-emacs-tp3217190p3217190.html > Sent from the Erlang Questions mailing list archive at Nabble.com. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From bengt.kleberg@REDACTED Fri Jan 14 09:03:56 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Fri, 14 Jan 2011 09:03:56 +0100 Subject: [erlang-questions] Finding records in List In-Reply-To: <010601cbb334$faea9520$f0bfbf60$@com> References: <1294914042.5279.15.camel@seasc0694> <010601cbb334$faea9520$f0bfbf60$@com> Message-ID: <1294992236.5122.7.camel@seasc1137> Greetings, I too consider the use of a record in the function head better. I apologise for any confusion my first example may have caused. bengt On Thu, 2011-01-13 at 16:17 +0100, David Mercer wrote: > On Thursday, January 13, 2011, Bengt Kleberg wrote: > > > is_name_and_age( Person, Name, Age ) -> > > (Person#person.name =:= Name) andalso (Person#person.age =:= Age) > > Question about style: is there any preference between what Bengt wrote, and: > > is_name_and_age(#person{name = Name, age = Age}, Name, Age) -> true; > is_name_and_age(_,_,_) -> false. > > In fact, I think I like Bengt's better, since it is more clear about intent, > but I'd have written it as the latter figuring that that was more idiomatic. > Are records in match specifications out of favor? (To a certain extent, if > they are, good.) > > Cheers, > > DBM > From raimo+erlang-questions@REDACTED Fri Jan 14 09:37:25 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Fri, 14 Jan 2011 09:37:25 +0100 Subject: [erlang-questions] String continuation In-Reply-To: <288128613.286241294953383364.JavaMail.root@zimbra> References: <1521212432.286221294953232480.JavaMail.root@zimbra> <288128613.286241294953383364.JavaMail.root@zimbra> Message-ID: <20110114083725.GA10216@erix.ericsson.se> On Thu, Jan 13, 2011 at 09:16:23PM +0000, Robert Virding wrote: > ----- "Ulf Wiger" wrote: > > > On 13 Jan 2011, at 19:07, Evgeniy Khramtsov wrote: > > > > > ++ operator is slow. It is better to use dialyzer in order to track > > such errors. > > > > > > Efficiency Guide, Chapter 2.4 - Myth: '++' is always bad > > http://erlang.org/doc/efficiency_guide/myths.html#id54761 > > It gets even better. The compiler will at compile time optimise "string3" ++ "string4" to "string3string4". So: > > "string3" "string4" --> "string3string4" during parsing > "string3" ++ "string4" --> "string3string4" during compiling And, "string3"++A is optimised into [$s,$t,$r,$i,$n,$g,$3|A]. > > There is no escape. > > Robert > > -- > Robert Virding, Erlang Solutions Ltd. > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From ola.a.andersson@REDACTED Fri Jan 14 10:32:05 2011 From: ola.a.andersson@REDACTED (Ola Andersson A) Date: Fri, 14 Jan 2011 10:32:05 +0100 Subject: [erlang-questions] Re: how to patch a standard module In-Reply-To: <216f04d8-c713-4f9f-9afd-b5c212311171@u32g2000yqe.googlegroups.com> References: <73cd3e8a-d6b4-4910-a2f2-9359252a3288@o4g2000yqd.googlegroups.com> <216f04d8-c713-4f9f-9afd-b5c212311171@u32g2000yqe.googlegroups.com> Message-ID: Hi, It is actually possible to to what you want. I have used a piece of code that could replace functions in existing modules by manipulating the beam code. In that project we only used it in eunit testing to stub functions in order to isolate the tested code. I'm not sure about using that method in a production system though. I don't have the code available any more so I can't provide any details about it but I think that Tomas Abrahamsson who sometimes contributes to this list could give some pointers on how to do it if he reads this. /OLA. > -----Original Message----- > From: erlang-questions@REDACTED > [mailto:erlang-questions@REDACTED] On Behalf Of Gijsbert > Sent: den 14 januari 2011 00:47 > To: erlang-questions@REDACTED > Subject: [erlang-questions] Re: how to patch a standard module > > > > On Jan 13, 10:25?pm, "Tomasz Maciejewski" wrote: > > Dnia 13-01-2011 o 16:52:23 Gijsbert > napisa?(a): > > > > > Hi, > > > > > I'd like to extend httpd_request.erl to handle the OPTIONS method. > > > I've managed to do this by copying httpd_request.erl into > my project > > > and making sure it overrides the standard version. > > > My question: is it possible to redefine a function in a standard > > > module at runtime? Then I could define my own > validate(Method, Uri, > > > Version) function (just a few lines) and keep using the standard > > > httdp_request.erl. > > > > > Regards, > > > Gijsbert > > > > If you like to extend httpd, you should write a module for > it, instead of patching original http_request.erl. All the > informations are > here:http://www.erlang.org/doc/man/httpd.html. You can also > see how mod_get.erl and mod_head.erl are done. > > Alas, my module is never called because > httpd_request:validate() does not have a clause for "OPTIONS" > and returns {error, {not_implemented, ...}}. As far as I know > I cannot extend the validation phase of httpd. > > > > > For redefining function at runtime, you can compile and > load modified module by typing "c(module)", or just load new > beam by "l(module)". > > > > -- > > Tomasz Maciejewski > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > Seehttp://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscr...@REDACTED > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From paolo.negri@REDACTED Fri Jan 14 11:10:05 2011 From: paolo.negri@REDACTED (Paolo Negri) Date: Fri, 14 Jan 2011 11:10:05 +0100 Subject: [erlang-questions] about the use of parametrized modules In-Reply-To: <20110114002255.GA3010@hanele.lan> References: <20110113232652.GA1737@hanele.lan> <20110114002255.GA3010@hanele.lan> Message-ID: I'm forwarding this interesting answer to the list, it went out of band because I hit reply instead of reply all. Paolo ---------- Forwarded message ---------- From: Jachym Holecek Date: Fri, Jan 14, 2011 at 1:22 AM Subject: Re: [erlang-questions] about the use of parametrized modules To: Paolo Negri # Paolo Negri 2011-01-14: > On Fri, Jan 14, 2011 at 12:26 AM, Jachym Holecek wrote: > > # Paolo Negri 2011-01-13: > >> Looking at the misultin source code [1] I noticed the use of a > >> parametrized module, after googling about it I came to the conclusion > >> that there's something controversial about them [2] and that it seems > >> to be appealing mainly people coming from OO background [3]. > >> > >> I'm considering applying this technique [...] > > > > Out of curiosity, what are your expactations from this move? IMVHO it only > > obfuscates flow of control without any real benefit. > > > > Regards, > > ? ? ? ?-- Jachym > > I have a couple of modules where i pass along over and over the same > variable to be able to extract information from it. > Really looking at the misultin_req module produces a valid example of > my case [1]. > But the whole point of asking here is to be aware of valid reasons not > to do so and to know which one would be instead a good approach. > > [1] https://github.com/ostinelli/misultin/blob/master/src/misultin_req.erl Understood. I think what misultin_req is doing is exactly The Right Thing (TM) to do -- it makes data flow crystal clear even without previous exposure to the codebase in question. If you were to pass more pieces of data this way, you could pack them into a record/proplist/whatever, but personally I only do that when there's embarassingly huge amount of data items (more than 10) and when these are intimately inter-related. To put it briefly, code clarity outweights the one-time effort of writing program text by several orders of magnitude. ;-) I think Robert Virding kind of answers the "officialy approved" status of this feature. Regards, ? ? ? ?-- Jachym PS: Feel free to forward this back to the list if you find it useful. From kuleshovmail@REDACTED Fri Jan 14 11:58:51 2011 From: kuleshovmail@REDACTED (shk) Date: Fri, 14 Jan 2011 02:58:51 -0800 (PST) Subject: Traverse dets and get all items Message-ID: <1295002731680-3217460.post@n4.nabble.com> Hello, I have dets table. Table view: {Time, Date, Space, Text} I need travers this table and put all data to list or another data structure. I find example: Fun = fun(X) -> io:format("~p~n", [X]), continue end, dets:traverse(MyTable,Fun). But it's only print dets items in console. How can i get all dets items and put it into list or something else? Thank you. -- View this message in context: http://erlang.2086793.n4.nabble.com/Traverse-dets-and-get-all-items-tp3217460p3217460.html Sent from the Erlang Questions mailing list archive at Nabble.com. From bengt.kleberg@REDACTED Fri Jan 14 12:17:01 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Fri, 14 Jan 2011 12:17:01 +0100 Subject: [erlang-questions] Traverse dets and get all items In-Reply-To: <1295002731680-3217460.post@n4.nabble.com> References: <1295002731680-3217460.post@n4.nabble.com> Message-ID: <1295003821.5122.13.camel@seasc1137> Greetings, To accumulate things you could use dets:foldl/3 bengt On Fri, 2011-01-14 at 11:58 +0100, shk wrote: > Hello, > > I have dets table. Table view: > > {Time, Date, Space, Text} > > I need travers this table and put all data to list or another data > structure. I find example: > > Fun = fun(X) -> io:format("~p~n", [X]), continue end, > dets:traverse(MyTable,Fun). > > But it's only print dets items in console. How can i get all dets items and > put it into list or something else? > > Thank you. From alessandro.sivieri@REDACTED Fri Jan 14 12:21:36 2011 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Fri, 14 Jan 2011 12:21:36 +0100 Subject: Building selected modules Message-ID: Hi all, when compiling Erlang sources, is there a way to specify which modules to build? -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From freza@REDACTED Fri Jan 14 12:31:14 2011 From: freza@REDACTED (Jachym Holecek) Date: Fri, 14 Jan 2011 11:31:14 +0000 Subject: [erlang-questions] about the use of parametrized modules In-Reply-To: References: <20110113232652.GA1737@hanele.lan> <20110114002255.GA3010@hanele.lan> Message-ID: <20110114113114.GA1138@hanele.lan> # Paolo Negri 2011-01-14: > > On Fri, Jan 14, 2011 at 12:26 AM, Jachym Holecek wrote: > > [...] > > I have a couple of modules where i pass along over and over the same > > variable to be able to extract information from it. > > Really looking at the misultin_req module produces a valid example of > > my case [1]. > > But the whole point of asking here is to be aware of valid reasons not > > to do so and to know which one would be instead a good approach. > > > > [1] https://github.com/ostinelli/misultin/blob/master/src/misultin_req.erl > > Understood. I think what misultin_req is doing is exactly The Right Thing (TM) > to do [...] Embarrassingly, I missed the fact that misultin_req actually uses parametrised modules -- but hopefully my point is clear from the rest of the message. So much for writing email late at night... Regards, -- Jachym From hynek@REDACTED Fri Jan 14 13:13:15 2011 From: hynek@REDACTED (Hynek Vychodil) Date: Fri, 14 Jan 2011 13:13:15 +0100 Subject: [erlang-questions] Building selected modules In-Reply-To: References: Message-ID: Look at module make http://erlang.org/doc/man/make.html On Fri, Jan 14, 2011 at 12:21 PM, Alessandro Sivieri wrote: > Hi all, > > when compiling Erlang sources, is there a way to specify which modules to > build? > > -- > Sivieri Alessandro > alessandro.sivieri@REDACTED > http://www.chimera-bellerofonte.eu/ > http://www.poul.org/ > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From alessandro.sivieri@REDACTED Fri Jan 14 14:00:22 2011 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Fri, 14 Jan 2011 14:00:22 +0100 Subject: [erlang-questions] Building selected modules In-Reply-To: References: Message-ID: On Fri, Jan 14, 2011 at 1:13 PM, Hynek Vychodil wrote: > Look at module make http://erlang.org/doc/man/make.html > > Thanks, but I mean Erlang and OTP platform sources :) -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From bgustavsson@REDACTED Fri Jan 14 15:40:35 2011 From: bgustavsson@REDACTED (=?UTF-8?Q?Bj=C3=B6rn_Gustavsson?=) Date: Fri, 14 Jan 2011 15:40:35 +0100 Subject: [erlang-questions] picky dialyzer In-Reply-To: <4D2EF987.2010706@cs.ntua.gr> References: <4D2EDD7F.5040409@cs.ntua.gr> <8548EADD-542F-42E9-9ED3-FC1FE5523BFF@erlang-solutions.com> <4D2EF987.2010706@cs.ntua.gr> Message-ID: On Thu, Jan 13, 2011 at 2:09 PM, Kostis Sagonas wrote: > Ulf Wiger wrote: >> >> On 13 Jan 2011, at 12:09, Kostis Sagonas wrote: >> .... >>> >>> mfa() is an alias for {atom(),atom(),byte()}. >> >> Ahh, as in {Module, Function, Arity}? >> >> In the System Limits of the Efficiency Guide, it says that you can have >> 256 arguments in a function or fun. Since dialyzer is never wrong, I will >> assume that the Efficiency Guide is? :) > > Let's see... > > %%========================================================= > -module(f_256). > > -export([f/256]). > > f(_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_) > -> ok. > %%========================================================== > > > % erlc f_256.erl > % ~/HiPE/otp/bin/erl > Erlang R14B02 (erts-5.8.3) [source] [smp:4:4] [rq:4] [async-threads:0] > [hipe] [kernel-poll:false] > > Eshell V5.8.3 (abort with ^G) > 1> l(f_256). > {"Kernel pid > terminated",application_controller,"{application_terminated,kernel,shutdown}"} > > Crash dump was written to: erl_crash.dump > Kernel pid terminated (application_controller) > ({application_terminated,kernel,shutdown}) > Works fine for me: $ erlc f_256.erl $ /ldisk/bjorn/otp/bin/erl Erlang R14B02 (erts-5.8.3) [source] [smp:4:4] [rq:4] [async-threads:0] [kernel-poll:false] Eshell V5.8.3 (abort with ^G) 1> l(f_256). {module,f_256} 2> If I try to load a function with 257 arguments, the loader will refuse to load the file: Erlang R14B02 (erts-5.8.3) [source] [smp:4:4] [rq:4] [async-threads:0] [kernel-poll:false] Eshell V5.8.3 (abort with ^G) 1> c(f_257). =ERROR REPORT==== 14-Jan-2011::15:34:45 === beam/beam_load.c(2115): Error loading function f_257:f/257: op i_func_info: IaaI: too many arguments: 257 {error,badfile} =ERROR REPORT==== 14-Jan-2011::15:34:45 === Loading of /home/bjorn/test/f_257.beam failed: badfile 2> (Hint: Note that HiPE is not enabled in the emulator I used.) -- Bj?rn Gustavsson, Erlang/OTP, Ericsson AB From ulf.wiger@REDACTED Fri Jan 14 15:56:34 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 14 Jan 2011 15:56:34 +0100 Subject: [erlang-questions] picky dialyzer In-Reply-To: References: <4D2EDD7F.5040409@cs.ntua.gr> <8548EADD-542F-42E9-9ED3-FC1FE5523BFF@erlang-solutions.com> <4D2EF987.2010706@cs.ntua.gr> Message-ID: <3B77805E-1772-48C1-816B-FC7887C8209B@erlang-solutions.com> Ouch! What say you, Kostis? Waiting for the comeback from Dialyzer. :) Of course now I'm also curious what leads to such a weird limit as 256 arguments? obviously, 0 args is allowed, so the range 0..256 does at least make my eyebrow twitch a little. Bj?rn? BR, Ulf On 14 Jan 2011, at 15:40, Bj?rn Gustavsson wrote: > On Thu, Jan 13, 2011 at 2:09 PM, Kostis Sagonas wrote: >> Ulf Wiger wrote: >>> >>> On 13 Jan 2011, at 12:09, Kostis Sagonas wrote: >>> .... >>>> >>>> mfa() is an alias for {atom(),atom(),byte()}. >>> >>> Ahh, as in {Module, Function, Arity}? >>> >>> In the System Limits of the Efficiency Guide, it says that you can have >>> 256 arguments in a function or fun. Since dialyzer is never wrong, I will >>> assume that the Efficiency Guide is? :) >> >> Let's see... >> >> %%========================================================= >> -module(f_256). >> >> -export([f/256]). >> >> f(_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_) >> -> ok. >> %%========================================================== >> >> >> % erlc f_256.erl >> % ~/HiPE/otp/bin/erl >> Erlang R14B02 (erts-5.8.3) [source] [smp:4:4] [rq:4] [async-threads:0] >> [hipe] [kernel-poll:false] >> >> Eshell V5.8.3 (abort with ^G) >> 1> l(f_256). >> {"Kernel pid >> terminated",application_controller,"{application_terminated,kernel,shutdown}"} >> >> Crash dump was written to: erl_crash.dump >> Kernel pid terminated (application_controller) >> ({application_terminated,kernel,shutdown}) >> > > Works fine for me: > > $ erlc f_256.erl > $ /ldisk/bjorn/otp/bin/erl > Erlang R14B02 (erts-5.8.3) [source] [smp:4:4] [rq:4] [async-threads:0] > [kernel-poll:false] > > Eshell V5.8.3 (abort with ^G) > 1> l(f_256). > {module,f_256} > 2> > > > If I try to load a function with 257 arguments, the loader will > refuse to load the file: > > Erlang R14B02 (erts-5.8.3) [source] [smp:4:4] [rq:4] [async-threads:0] > [kernel-poll:false] > > Eshell V5.8.3 (abort with ^G) > 1> c(f_257). > > =ERROR REPORT==== 14-Jan-2011::15:34:45 === > beam/beam_load.c(2115): Error loading function f_257:f/257: op > i_func_info: IaaI: > too many arguments: 257 > > {error,badfile} > > =ERROR REPORT==== 14-Jan-2011::15:34:45 === > Loading of /home/bjorn/test/f_257.beam failed: badfile > 2> > > > (Hint: Note that HiPE is not enabled in the > emulator I used.) > > -- > Bj?rn Gustavsson, Erlang/OTP, Ericsson AB Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From alessandro.sivieri@REDACTED Fri Jan 14 16:18:30 2011 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Fri, 14 Jan 2011 16:18:30 +0100 Subject: [erlang-questions] Building selected modules In-Reply-To: References: Message-ID: On Fri, Jan 14, 2011 at 2:14 PM, Tomasz Maciejewski wrote: > Each src directory should have its Makefile. > ? I can disable a few things, like ssl, jinterface, hipe or others using the configure script, but there are many others that cannot be disabled in that way. I saw a few projects compiling only kernel/stdlib/erts, so I think there is a way of doing it... -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From tony@REDACTED Fri Jan 14 16:24:30 2011 From: tony@REDACTED (Tony Rogvall) Date: Fri, 14 Jan 2011 16:24:30 +0100 Subject: [erlang-questions] picky dialyzer In-Reply-To: <3B77805E-1772-48C1-816B-FC7887C8209B@erlang-solutions.com> References: <4D2EDD7F.5040409@cs.ntua.gr> <8548EADD-542F-42E9-9ED3-FC1FE5523BFF@erlang-solutions.com> <4D2EF987.2010706@cs.ntua.gr> <3B77805E-1772-48C1-816B-FC7887C8209B@erlang-solutions.com> Message-ID: On 14 jan 2011, at 15.56, Ulf Wiger wrote: > Ouch! What say you, Kostis? Waiting for the comeback from Dialyzer. :) > > Of course now I'm also curious what leads to such a weird limit as 256 > arguments? obviously, 0 args is allowed, so the range 0..256 does at > least make my eyebrow twitch a little. Bj?rn? > Switch side now, are we ? ;-) /Tony > BR, > Ulf > > On 14 Jan 2011, at 15:40, Bj?rn Gustavsson wrote: > >> On Thu, Jan 13, 2011 at 2:09 PM, Kostis Sagonas wrote: >>> Ulf Wiger wrote: >>>> >>>> On 13 Jan 2011, at 12:09, Kostis Sagonas wrote: >>>> .... >>>>> >>>>> mfa() is an alias for {atom(),atom(),byte()}. >>>> >>>> Ahh, as in {Module, Function, Arity}? >>>> >>>> In the System Limits of the Efficiency Guide, it says that you can have >>>> 256 arguments in a function or fun. Since dialyzer is never wrong, I will >>>> assume that the Efficiency Guide is? :) >>> >>> Let's see... >>> >>> %%========================================================= >>> -module(f_256). >>> >>> -export([f/256]). >>> >>> f(_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_) >>> -> ok. >>> %%========================================================== >>> >>> >>> % erlc f_256.erl >>> % ~/HiPE/otp/bin/erl >>> Erlang R14B02 (erts-5.8.3) [source] [smp:4:4] [rq:4] [async-threads:0] >>> [hipe] [kernel-poll:false] >>> >>> Eshell V5.8.3 (abort with ^G) >>> 1> l(f_256). >>> {"Kernel pid >>> terminated",application_controller,"{application_terminated,kernel,shutdown}"} >>> >>> Crash dump was written to: erl_crash.dump >>> Kernel pid terminated (application_controller) >>> ({application_terminated,kernel,shutdown}) >>> >> >> Works fine for me: >> >> $ erlc f_256.erl >> $ /ldisk/bjorn/otp/bin/erl >> Erlang R14B02 (erts-5.8.3) [source] [smp:4:4] [rq:4] [async-threads:0] >> [kernel-poll:false] >> >> Eshell V5.8.3 (abort with ^G) >> 1> l(f_256). >> {module,f_256} >> 2> >> >> >> If I try to load a function with 257 arguments, the loader will >> refuse to load the file: >> >> Erlang R14B02 (erts-5.8.3) [source] [smp:4:4] [rq:4] [async-threads:0] >> [kernel-poll:false] >> >> Eshell V5.8.3 (abort with ^G) >> 1> c(f_257). >> >> =ERROR REPORT==== 14-Jan-2011::15:34:45 === >> beam/beam_load.c(2115): Error loading function f_257:f/257: op >> i_func_info: IaaI: >> too many arguments: 257 >> >> {error,badfile} >> >> =ERROR REPORT==== 14-Jan-2011::15:34:45 === >> Loading of /home/bjorn/test/f_257.beam failed: badfile >> 2> >> >> >> (Hint: Note that HiPE is not enabled in the >> emulator I used.) >> >> -- >> Bj?rn Gustavsson, Erlang/OTP, Ericsson AB > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > "Have run Make so many times I dunno what's installed anymore" From ulf.wiger@REDACTED Fri Jan 14 16:29:56 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 14 Jan 2011 16:29:56 +0100 Subject: [erlang-questions] picky dialyzer In-Reply-To: References: <4D2EDD7F.5040409@cs.ntua.gr> <8548EADD-542F-42E9-9ED3-FC1FE5523BFF@erlang-solutions.com> <4D2EF987.2010706@cs.ntua.gr> <3B77805E-1772-48C1-816B-FC7887C8209B@erlang-solutions.com> Message-ID: <310965E6-F077-4CA1-9A70-3235E0944A86@erlang-solutions.com> Oh, I'm just in it for the action. :) BR, Ulf On 14 Jan 2011, at 16:24, Tony Rogvall wrote: > > On 14 jan 2011, at 15.56, Ulf Wiger wrote: > >> Ouch! What say you, Kostis? Waiting for the comeback from Dialyzer. :) >> >> Of course now I'm also curious what leads to such a weird limit as 256 >> arguments? obviously, 0 args is allowed, so the range 0..256 does at >> least make my eyebrow twitch a little. Bj?rn? >> > > Switch side now, are we ? ;-) > > /Tony > >> BR, >> Ulf >> >> On 14 Jan 2011, at 15:40, Bj?rn Gustavsson wrote: >> >>> On Thu, Jan 13, 2011 at 2:09 PM, Kostis Sagonas wrote: >>>> Ulf Wiger wrote: >>>>> >>>>> On 13 Jan 2011, at 12:09, Kostis Sagonas wrote: >>>>> .... >>>>>> >>>>>> mfa() is an alias for {atom(),atom(),byte()}. >>>>> >>>>> Ahh, as in {Module, Function, Arity}? >>>>> >>>>> In the System Limits of the Efficiency Guide, it says that you can have >>>>> 256 arguments in a function or fun. Since dialyzer is never wrong, I will >>>>> assume that the Efficiency Guide is? :) >>>> >>>> Let's see... >>>> >>>> %%========================================================= >>>> -module(f_256). >>>> >>>> -export([f/256]). >>>> >>>> f(_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_,_) >>>> -> ok. >>>> %%========================================================== >>>> >>>> >>>> % erlc f_256.erl >>>> % ~/HiPE/otp/bin/erl >>>> Erlang R14B02 (erts-5.8.3) [source] [smp:4:4] [rq:4] [async-threads:0] >>>> [hipe] [kernel-poll:false] >>>> >>>> Eshell V5.8.3 (abort with ^G) >>>> 1> l(f_256). >>>> {"Kernel pid >>>> terminated",application_controller,"{application_terminated,kernel,shutdown}"} >>>> >>>> Crash dump was written to: erl_crash.dump >>>> Kernel pid terminated (application_controller) >>>> ({application_terminated,kernel,shutdown}) >>>> >>> >>> Works fine for me: >>> >>> $ erlc f_256.erl >>> $ /ldisk/bjorn/otp/bin/erl >>> Erlang R14B02 (erts-5.8.3) [source] [smp:4:4] [rq:4] [async-threads:0] >>> [kernel-poll:false] >>> >>> Eshell V5.8.3 (abort with ^G) >>> 1> l(f_256). >>> {module,f_256} >>> 2> >>> >>> >>> If I try to load a function with 257 arguments, the loader will >>> refuse to load the file: >>> >>> Erlang R14B02 (erts-5.8.3) [source] [smp:4:4] [rq:4] [async-threads:0] >>> [kernel-poll:false] >>> >>> Eshell V5.8.3 (abort with ^G) >>> 1> c(f_257). >>> >>> =ERROR REPORT==== 14-Jan-2011::15:34:45 === >>> beam/beam_load.c(2115): Error loading function f_257:f/257: op >>> i_func_info: IaaI: >>> too many arguments: 257 >>> >>> {error,badfile} >>> >>> =ERROR REPORT==== 14-Jan-2011::15:34:45 === >>> Loading of /home/bjorn/test/f_257.beam failed: badfile >>> 2> >>> >>> >>> (Hint: Note that HiPE is not enabled in the >>> emulator I used.) >>> >>> -- >>> Bj?rn Gustavsson, Erlang/OTP, Ericsson AB >> >> Ulf Wiger, CTO, Erlang Solutions, Ltd. >> http://erlang-solutions.com >> >> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > "Have run Make so many times I dunno what's installed anymore" > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From gleber.p@REDACTED Fri Jan 14 16:33:46 2011 From: gleber.p@REDACTED (Gleb Peregud) Date: Fri, 14 Jan 2011 16:33:46 +0100 Subject: [erlang-questions] [patch] new float_to_list/2 In-Reply-To: <4D2DE13E.7040404@aleynikov.org> References: <4D2D3FA8.1000009@aleynikov.org> <4D2DB65C.1090106@aleynikov.org> <4D2DE13E.7040404@aleynikov.org> Message-ID: Here's my patch: http://www.erlang.org/cgi-bin/ezmlm-cgi/4/43529 It uses double sprintf invocations. Your solution is clearly better (I didn't knew it is possible to specify precision in args). Your benchmark shows that difference is negligible. Between integer as a second parameter or a proplist (~0.4%). Though it might be a bit more "stable" in terms of running time, but still probably not worth considering it for performance reasons. It still may be useful in terms of simpler API, but I have no strong opinion on it. So generally +1 on including Serge's patch into Erlang. Serge, can you put it into GitHub pull request? As described here: https://github.com/erlang/otp/wiki/submitting-patches Best, Gleb On Wed, Jan 12, 2011 at 18:13, Serge Aleynikov wrote: > I did a micro-benchmark on a slightly modified version of the BIF that > accepts an integer as its options. ?The results shown below display a very > insignificant difference between a call with no options and a call with an > integer precision passed as the second argument: > > 1> test:test(). > float_to_list(123.4, []) ? ? ? ? ? ? ? = {0.619512,"123.4000"} > float_to_list(123.4, [{precision, 4}]) = {0.624895,"123.4000"} > float_to_list(123.4, 4) ? ? ? ? ? ? ? ?= {0.622896,"123.4000"} > > The majority of time is actually spent in the printf(3) function, which > takes longer to execute when given the "%.*f" argument compared to "%g" as > in float_to_list/1 case. > > Did your patch rely on printf? > > > On 1/12/2011 9:23 AM, Gleb Peregud wrote: >> >> Some time ago I've submitted similar but simpler patch. It was a >> float_to_list/2 with a second parameter being an integer specifying >> precision. For me it was important to generate A LOT of floats as >> strings as fast as possible with specified precision. Serge's version >> has an overhead of inspecting proplist of the second parameter. So I >> was wondering about introducing two versions of this function: with a >> proplist as a second parameter and with a number as a second >> parameter. Alternatively proplist version can be factored out into >> float_to_list_opts/2. >> >> Just my 0.2 cents >> >> On Wed, Jan 12, 2011 at 15:10, Serge Aleynikov >> ?wrote: >>> >>> The reason I called it precision was to be consistent with the naming >>> convention of the printf function. ?Below is the extract from "man 3 >>> printf", which refers to the digits after the decimal point as >>> "precision": >>> >>> ? ? ? f, F ? The ?double ?argument is rounded and converted to decimal >>> ? ? ? ? ? ? ?notation in the style [-]ddd.ddd, where the number of >>> ? ? ? ? ? ? ?digits after the decimal-point character is equal to the >>> ? ? ? ? ? ? ?precision specification. ?If the precision is missing, >>> ? ? ? ? ? ? ?it ?is taken as 6; if the precision is explicitly zero, >>> ? ? ? ? ? ? ?no decimal-point character appears. ?If a decimal point >>> ? ? ? ? ? ? ?appears, at least one digit appears before it. >>> >>> I don't have a very strong preference for calling it precision or scale, >>> but >>> I do have a strong preference for including this patch in the >>> distribution, >>> because the default behavior of float_to_list/1 hard-coded in C is >>> deficient. >>> >>> >>> On 1/12/2011 4:58 AM, nox wrote: >>>> >>>> Il should be called "scale", shouldn't it? >>>> >>>> Le 12 janv. 2011 ? 10:26, Pierpaolo Bernardi ? ?a >>>> ?crit >>>> : >>>> >>>>> On Wed, Jan 12, 2011 at 06:44, Serge Aleynikov >>>>> ?wrote: >>>>>> >>>>>> Attached please find a patch that adds a new float_to_list/2 BIF. ?The >>>>>> patch >>>>>> was created off of the master branch of https://github.com/erlang/otp. >>>>>> >>>>>> This BIF solves a problem of float_to_list/1 that doesn't allow >>>>>> specifying >>>>>> the number of digits after the decimal point when formatting floats. >>>>>> >>>>>> ? ? ? ?float_to_list(Float, Options) -> ? ?string() >>>>>> >>>>>> ? ? ? ?Float = float() >>>>>> ? ? ? ?Options = [Option] >>>>>> ? ? ? ?Option = {precision, Precision::integer()} | compact >>>>>> >>>>>> ? ? ? ?Text representation of a float formatted using given options >>>>>> >>>>>> ? ? ? ?Returns a string which corresponds to the text >>>>>> ? ? ? ?representation of Float using fixed decimal point formatting. >>>>>> ? ? ? ?When precision option is specified >>>>>> ? ? ? ?the returned value will contain at most Precision number of >>>>>> ? ? ? ?digits past the decimal point. ?When compact option is provided >>>>>> ? ? ? ?the trailing zeros at the end of the list are truncated. >>>>> >>>>> I think the option is misnamed. >>>>> >>>>> In the usual terminology, 'precision' is the total number of >>>>> significative digits, not only the ones past the decimal point. >>>>> >>>>> Cheers >>>>> P. >>>>> >>>>> ________________________________________________________________ >>>>> erlang-questions (at) erlang.org mailing list. >>>>> See http://www.erlang.org/faq.html >>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>>> >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >>> > From alessandro.sivieri@REDACTED Fri Jan 14 16:47:29 2011 From: alessandro.sivieri@REDACTED (Alessandro Sivieri) Date: Fri, 14 Jan 2011 16:47:29 +0100 Subject: [erlang-questions] Building selected modules In-Reply-To: References: Message-ID: Well, I just found [1], which seems to explain a few things... [1] https://github.com/esl/erlang-embedded -- Sivieri Alessandro alessandro.sivieri@REDACTED http://www.chimera-bellerofonte.eu/ http://www.poul.org/ From roberto@REDACTED Fri Jan 14 17:07:07 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 14 Jan 2011 17:07:07 +0100 Subject: [erlang-questions] about the use of parametrized modules In-Reply-To: References: <20110113232652.GA1737@hanele.lan> <20110114002255.GA3010@hanele.lan> Message-ID: i can only reply afa my personal experience is concerned. 1. there are certain variables that are totally useless to a developer using misultin, such as SocketPid in a request [1]. this is an internal variable used to provide streaming functionalities, which provides absolutely no added value to a developer. on the contrary, it could generate issues if misused. 2. a request, and all of its 'properties', is by concept very similar to an 'object' even though it still is a module underneath, in the same way 'records' actually are similar to 'named lists' and are lists 'in disguise'. it's just a matter of 'grouping' some small functionalities alltogether in the name of this same 'object'. point here is that the module functionalities are *all* so dependent on *the* very same variable [Req], that a module actually gets 'attached' to the variable itself. therefore, it's very simple: i've used parametrized modules in misultin because of a simple matter: _code useability_. for instance, in misultin the code to send out a file as an attachment is as follows: % callback on request received handle_http(Req) -> Req:file(attachment, "/usr/local/files/image1.png"). if i weren't using parametrized modules, it would look something similar to: % callback on request received handle_http(Req, SocketPid) -> Req:file(Req, SocketPid, attachment, "/usr/local/files/image1.png"). true, you would know that, in order for this to work, you need to pass both Req and a variable called SocketPid. but you wouldn't have the slightest clue on why the latter it's there, nor what can you do with it. i don't see any value added in making these variables explicit to the developer, instead of having them 'masked' by the use of parametrized modules. i do realize this can sound like a blasphemy to erlang purists, and that this can lead to incongruence in approaching code. as always, it's a matter of tradeoffs, and taste. my evaluation was that i by far preferred easing code readability of applications based on misultin. i haven't seen [i might be corrected, though] anywhere written that these parametrized modules present particular running issues when compared to 'explicit' code. just my 2 cents, apparently not popular ones since everybody seem to dislike parametrized modules :) r. [1] https://github.com/ostinelli/misultin/blob/master/src/misultin_req.erl From mazen.harake@REDACTED Fri Jan 14 18:14:50 2011 From: mazen.harake@REDACTED (Mazen Harake) Date: Fri, 14 Jan 2011 18:14:50 +0100 Subject: [erlang-questions] String continuation In-Reply-To: <2F728F5C-CE3A-4D7E-8C33-2F89C7706940@erlang-solutions.com> References: <4D2F7B66.4040504@erlang-solutions.com> <2F728F5C-CE3A-4D7E-8C33-2F89C7706940@erlang-solutions.com> Message-ID: <4D30848A.5010508@erlang-solutions.com> Actually that is exactly the method I used to check the literals. Didn't bother to check the other way but it is pretty obvious (when you think about it) that the compiler would optimise away that sort of things. /M On 13/01/2011 22:47, Ulf Wiger wrote: > Actually, and you can verify this with erlc -S, they are _all_ the same, > as Robert said: > > f() -> "a" "b". > > g() -> "ab". > > h() -> "a" ++ "b". > > compiles to: > > {function, f, 0, 2}. > {label,1}. > {func_info,{atom,strs},{atom,f},0}. > {label,2}. > {move,{literal,"ab"},{x,0}}. > return. > > {function, g, 0, 4}. > {label,3}. > {func_info,{atom,strs},{atom,g},0}. > {label,4}. > {move,{literal,"ab"},{x,0}}. > return. > > {function, h, 0, 6}. > {label,5}. > {func_info,{atom,strs},{atom,h},0}. > {label,6}. > {move,{literal,"ab"},{x,0}}. > return. > > BR, > Ulf W > > On 13 Jan 2011, at 23:23, Mazen Harake wrote: > >> I think you are confusing the two. Writing two strings without anything in between is a literal and not a statement of concatenation of both. This means that: >> >> f() -> "Hello" "World". >> >> is the same as... >> >> f() -> "HelloWorld". >> >> and not >> >> f() -> "Hello" ++ "World". >> >> >> /Mazen >> >> On 13/01/2011 18:40, Attila Rajmund Nohl wrote: >>> Hello! >>> >>> Let's have a look at this little example code: >>> >>> -module('strc'). >>> >>> -export([f/0]). >>> >>> f() -> >>> {["string1", >>> "string2"], >>> ["string3" >>> "string4"]}. >>> >>> This compiles fine and returns a tuple with two lists, the first with >>> 2 strings, the second with a single list. Of course, if there was only >>> a typo in the second list (a missed comma), I wouldn't notice it until >>> the code executes (which might happen in an inconvenient time in error >>> handling code). I wonder that the possibility to omit the extra ++ >>> from the end of "string3" line worth the problems what might be caused >>> by the missing comma. This only bit me once, so it might not be that >>> common... >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > From lukas@REDACTED Fri Jan 14 17:23:13 2011 From: lukas@REDACTED (Lukas Larsson) Date: Fri, 14 Jan 2011 17:23:13 +0100 Subject: [erlang-questions] Building selected modules In-Reply-To: References: Message-ID: <1295022193.5087.8.camel@ancalagon.du.uab.ericsson.se> Hi, If you do './otp_build setup && ./otp_build release' a small erlang release will be compiled and published into releases/$ARCH. As for only doing certain applications I do not know of any way, but you can always edit lib/Makefile which lists which applications to compile. This assumes that you are building from the github repo and not from the sources at erlang.org/download. There should however be a similar makefile there that you can edit. Lukas On Fri, 2011-01-14 at 16:47 +0100, Alessandro Sivieri wrote: > Well, I just found [1], which seems to explain a few things... > > [1] https://github.com/esl/erlang-embedded > From tomas.abrahamsson@REDACTED Fri Jan 14 17:42:54 2011 From: tomas.abrahamsson@REDACTED (Tomas Abrahamsson) Date: Fri, 14 Jan 2011 17:42:54 +0100 Subject: [erlang-questions] Re: how to patch a standard module In-Reply-To: References: <73cd3e8a-d6b4-4910-a2f2-9359252a3288@o4g2000yqd.googlegroups.com> <216f04d8-c713-4f9f-9afd-b5c212311171@u32g2000yqe.googlegroups.com> Message-ID: >> > > I'd like to extend httpd_request.erl to handle the OPTIONS method. >> > > I've managed to do this by copying httpd_request.erl into my project >> > > and making sure it overrides the standard version. >> > > My question: is it possible to redefine a function in a standard >> > > module at runtime? Then I could define my own validate(Method, Uri, >> > > Version) function (just a few lines) and keep using the standard >> > > httdp_request.erl. > It is actually possible to to what you want. I have used a piece of > code that could replace functions in existing modules by manipulating > the beam code. In that project we only used it in eunit testing to stub > functions in order to isolate the tested code. > I'm not sure about using that method in a production system though. > I don't have the code available any more so I can't provide any details > about it but I think that Tomas Abrahamsson who sometimes contributes to > this list could give some pointers on how to do it if he reads this. The stubber that Ola refers to is a bit similar to meck and some other mocking libraries. It provides the possibility to override all or only some functions of a module, while still making the rest of the functions available unchanged. The stubber has evolved a bit since: Klas Johansson has done a fair bit of useful work on it, to say the least, and I guess he might even make his work publicly available eventually(?) It is mainly intended for (e)unit testing, though. The way the existing modules were extended with new or changed functions, was by simply renaming the beam file, then provide a new beam module which either calls the renamed module for all non-changed functions, or uses the -extends(RenamedModule) feature (undocumented, I think). The new module was compiled to a binary and loaded with code:load_binary/3. See https://github.com/klajo/hacks/tree/master/beam at Klas' repository for some code to rename a beam module. See http://www.trapexit.org/Extend_Module for more info on the -extends attribute. BRs Tomas From hynek@REDACTED Fri Jan 14 17:44:20 2011 From: hynek@REDACTED (Hynek Vychodil) Date: Fri, 14 Jan 2011 17:44:20 +0100 Subject: [erlang-questions] about the use of parametrized modules In-Reply-To: References: <20110113232652.GA1737@hanele.lan> <20110114002255.GA3010@hanele.lan> Message-ID: On Fri, Jan 14, 2011 at 5:07 PM, Roberto Ostinelli wrote: > i can only reply afa my personal experience is concerned. > > 1. there are certain variables that are totally useless to a developer > using misultin, such as SocketPid in a request [1]. this is an > internal variable used to provide streaming functionalities, which > provides absolutely no added value to a developer. on the contrary, it > could generate issues if misused. > > 2. a request, and all of its 'properties', is by concept very similar > to an 'object' even though it still is a module underneath, in the > same way 'records' actually are similar to 'named lists' and are lists > 'in disguise'. it's just a matter of 'grouping' some small > functionalities alltogether in the name of this same 'object'. point > here is that the module functionalities are *all* so dependent on > *the* very same variable [Req], that a module actually gets 'attached' > to the variable itself. > > therefore, it's very simple: i've used parametrized modules in > misultin because of a simple matter: _code useability_. > > for instance, in misultin the code to send out a file as an attachment > is as follows: > > % callback on request received > handle_http(Req) -> > ? ?Req:file(attachment, "/usr/local/files/image1.png"). > > if i weren't using parametrized modules, it would look something similar to: > > % callback on request received > handle_http(Req, SocketPid) -> > ? ?Req:file(Req, SocketPid, attachment, "/usr/local/files/image1.png"). What prevent you add SockedPid to Req and above code would look like? % callback on request received handle_http(Req) -> ? ?misultin_req:file(Req, attachment, "/usr/local/files/image1.png"). Which is not so much different from % callback on request received handle_http(Req) -> ? ?Req:file(attachment, "/usr/local/files/image1.png"). It already contain socket and socket_mode, why not SockedPid? Problem with your code is, when I read % callback on request received handle_http(Req) -> ? ?Req:file(attachment, "/usr/local/files/image1.png"). and I want to see implementation of function file. Where I should look? I have to know *out of band* information that I should look to misultin_req module. I have to look for. I have to find. Why the hell is code encrypted? It same applies to mochiweb. I have to spend nontrivial time find out the module of implementation every time someone use parametrized module. It hurts. It hurts users of your code absolutely uselessly. > > true, you would know that, in order for this to work, you need to pass > both Req and a variable called SocketPid. but you wouldn't have the > slightest clue on why the latter it's there, nor what can you do with > it. i don't see any value added in making these variables explicit to > the developer, instead of having them 'masked' by the use of > parametrized modules. > > i do realize this can sound like a blasphemy to erlang purists, and > that this can lead to incongruence in approaching code. as always, > it's a matter of tradeoffs, and taste. my evaluation was that i by far > preferred easing code readability of applications based on misultin. i > haven't seen [i might be corrected, though] anywhere written that > these parametrized modules present particular running issues when > compared to 'explicit' code. > > just my 2 cents, apparently not popular ones since everybody seem to > dislike parametrized modules :) > > r. > > [1] https://github.com/ostinelli/misultin/blob/master/src/misultin_req.erl > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From roberto@REDACTED Fri Jan 14 18:05:57 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Fri, 14 Jan 2011 18:05:57 +0100 Subject: [erlang-questions] about the use of parametrized modules In-Reply-To: References: <20110113232652.GA1737@hanele.lan> <20110114002255.GA3010@hanele.lan> Message-ID: > What prevent you add SockedPid to Req > It already contain socket and socket_mode, why not SockedPid? the fact that SocketPid shouldn't be used by developers, the rest, yes. > and I want to see implementation of function file. Where I should > look? I have to know *out of band* information that I should look to > misultin_req module. I have to look for. I have to find. Why the hell > is code encrypted? It same applies to mochiweb. I have to spend > nontrivial time find out the module of implementation every time > someone use parametrized module. It hurts. It hurts users of your code > absolutely uselessly. i'm sorry to read that it did hurt you so bad. i'll be interested to read how many users of this code actually *use* it, and how many do spend time in walking through the code. i do get your point, though, and as i've said, it's a matter of tradeoffs. r. From pmm@REDACTED Fri Jan 14 18:21:57 2011 From: pmm@REDACTED (Peter Mechlenborg) Date: Fri, 14 Jan 2011 18:21:57 +0100 Subject: [erlang-questions] Compile all modules in emacs In-Reply-To: <1294985622095-3217190.post@n4.nabble.com> References: <1294985622095-3217190.post@n4.nabble.com> Message-ID: Hi You can also use an Emakefile, and then call: make:all([load]). from the Emacs shell. Best regards, -- Peter Mechlenborg On Fri, Jan 14, 2011 at 7:13 AM, shk wrote: > > Hello, > > I user GNU/Emacs + Erlang-mode for erlang programming. I use C-c C-k for > compiling current buffer. For example i have 3 *.erl files: > > test1.erl > test2.erl > test3.erl > > Theese file opened in emacs in deifferent buffer. How can i caompile all > files at one times? > > Thank you. > -- > View this message in context: > http://erlang.2086793.n4.nabble.com/Compile-all-modules-in-emacs-tp3217190p3217190.html > Sent from the Erlang Questions mailing list archive at Nabble.com. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From jameschurchman@REDACTED Fri Jan 14 18:01:26 2011 From: jameschurchman@REDACTED (James Churchman) Date: Fri, 14 Jan 2011 17:01:26 +0000 Subject: [erlang-questions] Re: how to patch a standard module In-Reply-To: References: <73cd3e8a-d6b4-4910-a2f2-9359252a3288@o4g2000yqd.googlegroups.com> <216f04d8-c713-4f9f-9afd-b5c212311171@u32g2000yqe.googlegroups.com> Message-ID: <3ABF5364-A445-4EB8-8AF6-48B2DC8C2FB5@gmail.com> I have 0 experience of beam? but that sounds much more tricky than the AST method, and only has the advantage of working without a copy of the original source? just now tried -extends, but it wont take a file path, only a module atom & i don't think it allows for the file it's extending to be of the same name! so in this exact case it wont work. The AST one can be done in less than 10 lines which i can dig out if of any use. In this very specific case tho the -include() with the include directory set does the job the simplest :-) On 14 Jan 2011, at 16:42, Tomas Abrahamsson wrote: >>>>> I'd like to extend httpd_request.erl to handle the OPTIONS method. >>>>> I've managed to do this by copying httpd_request.erl into my project >>>>> and making sure it overrides the standard version. >>>>> My question: is it possible to redefine a function in a standard >>>>> module at runtime? Then I could define my own validate(Method, Uri, >>>>> Version) function (just a few lines) and keep using the standard >>>>> httdp_request.erl. > >> It is actually possible to to what you want. I have used a piece of >> code that could replace functions in existing modules by manipulating >> the beam code. In that project we only used it in eunit testing to stub >> functions in order to isolate the tested code. >> I'm not sure about using that method in a production system though. >> I don't have the code available any more so I can't provide any details >> about it but I think that Tomas Abrahamsson who sometimes contributes to >> this list could give some pointers on how to do it if he reads this. > > The stubber that Ola refers to is a bit similar to meck > and some other mocking libraries. It provides the > possibility to override all or only some functions of a > module, while still making the rest of the functions > available unchanged. > > The stubber has evolved a bit since: Klas Johansson > has done a fair bit of useful work on it, to say the > least, and I guess he might even make his work publicly > available eventually(?) > > It is mainly intended for (e)unit testing, though. > > > The way the existing modules were extended with new or > changed functions, was by simply renaming the beam file, > then provide a new beam module which either calls the > renamed module for all non-changed functions, or uses the > -extends(RenamedModule) feature (undocumented, I think). > The new module was compiled to a binary and loaded with > code:load_binary/3. > > See https://github.com/klajo/hacks/tree/master/beam > at Klas' repository for some code to rename a beam > module. > > See http://www.trapexit.org/Extend_Module for more info > on the -extends attribute. > > BRs > Tomas > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From freza@REDACTED Fri Jan 14 19:25:31 2011 From: freza@REDACTED (Jachym Holecek) Date: Fri, 14 Jan 2011 18:25:31 +0000 Subject: [erlang-questions] Building selected modules In-Reply-To: <1295022193.5087.8.camel@ancalagon.du.uab.ericsson.se> References: <1295022193.5087.8.camel@ancalagon.du.uab.ericsson.se> Message-ID: <20110114182531.GA1751@hanele> # Lukas Larsson 2011-01-14: > If you do './otp_build setup && ./otp_build release' a small erlang > release will be compiled and published into releases/$ARCH. As for only > doing certain applications I do not know of any way, but you can always > edit lib/Makefile which lists which applications to compile. > > This assumes that you are building from the github repo and not from the > sources at erlang.org/download. There should however be a similar > makefile there that you can edit. I vaguely remember "touch lib//SKIP" did the trick at some point, a quick look at make/otp_subdir.mk suggests it might still work. This would let one build exactly the apps one needs -- perhaps that's one of underlying tricks of ESL? HTH, -- Jachym From karol.skocik@REDACTED Fri Jan 14 20:59:04 2011 From: karol.skocik@REDACTED (karol skocik) Date: Fri, 14 Jan 2011 20:59:04 +0100 Subject: [erlang-questions] hacked gmail account -> spam In-Reply-To: References: <20110111152628.GA15773@erix.ericsson.se> <79F3C0A8-9170-4EC8-8796-38564FAFBD8C@gmail.com> <1294812007.12584.9.camel@yang> Message-ID: Just a final note about the issue - Tuncer Ayaz asked me to mention, that Firefox 4 has a feature called 'compartments' which is also of interest for security and provides similar protection like tabs-as-processes. http://andreasgal.wordpress.com/2010/10/13/compartments/ Have a nice day, Karol On Wed, Jan 12, 2011 at 9:43 AM, karol skocik wrote: > Thank you both for suggestions. > Karol > > On Wed, Jan 12, 2011 at 7:00 AM, Boris M?hmer > wrote: >> Am Dienstag, den 11.01.2011, 14:51 -0330 schrieb Alain O'Dea: >>> I recently updated all my service passwords to independent 14-20 character long passwords with mixed case, numbers and symbols, etc. which I got from http://www.pctools.com/guides/password/ >>> >>> I generated them in batches of 3-5 and saved them in a keychain secure note for reference before changing the service passwords. >> >> At work, at home and all of my family use KeePass (1.x, >> http://keepass.info/) or its ports to Linux or Mac. >> >> The main feature of KeePass is, that the database file can be used on >> almost all the devices (there are even clients for some mobile phones). >> >> >> ?- boris >> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From alisdairsullivan@REDACTED Fri Jan 14 21:56:44 2011 From: alisdairsullivan@REDACTED (Alisdair Sullivan) Date: Fri, 14 Jan 2011 12:56:44 -0800 Subject: [erlang-questions] about the use of parametrized modules In-Reply-To: References: <20110113232652.GA1737@hanele.lan> <20110114002255.GA3010@hanele.lan> Message-ID: >> What prevent you add SockedPid to Req >> It already contain socket and socket_mode, why not SockedPid? > > the fact that SocketPid shouldn't be used by developers, the rest, yes. There is essentially no functional difference between 'Req:foo(bar, baz)' and 'req:foo(Req, bar, baz)'. In both examples Req is an opaque blob that we are calling a function on. The second, however, makes it much clearer at a cost of a few keystrokes. if you wish to hide state from users the correct abstraction is an opaque type or process - both of which have clearly defined interfaces - not a module. The popularity of parameterized modules is that they look familiar to people coming from python and ruby and java. That is not sufficient reason to use them. > From roberto@REDACTED Sat Jan 15 02:46:56 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Sat, 15 Jan 2011 02:46:56 +0100 Subject: [erlang-questions] about the use of parametrized modules In-Reply-To: References: <20110113232652.GA1737@hanele.lan> <20110114002255.GA3010@hanele.lan> Message-ID: there you have the answer to your original question, paolo: you should not consider using parametrized modules, because that's not the erlang way of doing things. maybe that's why, even after all these years and increased support to them, parametrized modules are still referred to as being 'experimental'. so, you should not use them. in fact, if you like them and are considering using them, you should probably stick to ruby, python, java or whatever language you come from, and forget erlang. [ot] well, i must say that i am not surprised that many readers of this newsgroup simply do not publicly intervene in many of the discussions that go on here. maybe it's just me, but i do often feel that i'm not reading opposable opinions of healthy discussions but THE TRUTH (TM), in such a way that you can only be slapped in the face for whatever you do which sounds a little out of schemes. there are languages born to mix OO programming with functional paradigms, such as scala. fortunately, the world, people and languages evolve, sometimes for the best, sometimes for the worse, but it's a fact that we are no longer speaking aztec, ancient greek nor latin currently in our countries. thanks to that, we can at least express ourselves in this newsgroup in a common language, english. thank you robert [virding] for saying "I personally don't like it", when referred to this technique. a true example of a respectful opinion, coming from a person whose knowledge and experience on erlang is barely imaginable by most of us all. i'd simply would love not having the impression that some things cannot be discussed in this list. there's no need to express harsh feelings or absolute positions and, most of all, creativity and communities need new blood, open code, libraries, and support. cheers to you all, and happy coding. [/ot] r. From bernie@REDACTED Sat Jan 15 03:39:13 2011 From: bernie@REDACTED (Bernard Duggan) Date: Sat, 15 Jan 2011 13:39:13 +1100 Subject: [erlang-questions] Traverse dets and get all items In-Reply-To: <1295003821.5122.13.camel@seasc1137> References: <1295002731680-3217460.post@n4.nabble.com> <1295003821.5122.13.camel@seasc1137> Message-ID: <4D3108D1.7060200@m5net.com> Or, if you wanted to be more circuitous, you could drop the dets table into an ets table with dets:to_ets/2, then use ets:tab2list/1. But on balance dets:foldl/3 is probably "righter" :) B On 14/01/2011 10:17 PM, Bengt Kleberg wrote: > Greetings, > > To accumulate things you could use dets:foldl/3 > > > bengt > > On Fri, 2011-01-14 at 11:58 +0100, shk wrote: >> Hello, >> >> I have dets table. Table view: >> >> {Time, Date, Space, Text} >> >> I need travers this table and put all data to list or another data >> structure. I find example: >> >> Fun = fun(X) -> io:format("~p~n", [X]), continue end, >> dets:traverse(MyTable,Fun). >> >> But it's only print dets items in console. How can i get all dets items and >> put it into list or something else? >> >> Thank you. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From rzezeski@REDACTED Sat Jan 15 04:40:16 2011 From: rzezeski@REDACTED (Ryan Zezeski) Date: Fri, 14 Jan 2011 22:40:16 -0500 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <1693153562.258211294521899230.JavaMail.root@zimbra> References: <992935859.258191294521884974.JavaMail.root@zimbra> <1693153562.258211294521899230.JavaMail.root@zimbra> Message-ID: Teach Yourself Programming in 10 Years! (or the next ones free) http://norvig.com/21-days.html Neat side story...For those of you who know the name Hunter S. Thompson you might be interested to know the he learned to write by typing out Hemingway word for word, letter for letter. I've used that fact as a reminder that the act of doing is the best teacher. If you're not failing everyday, then you're not trying hard enough. Success isn't nearly as insightful as failure. -Ryan On Sat, Jan 8, 2011 at 4:24 PM, Robert Virding < robert.virding@REDACTED> wrote: > This whole dscussion reminds a little of the Paul Graham essay "Beating the > Averages", http://www.paulgraham.com/avg.html , where he discusses the > Blub paradox. > > Robert > > From toby@REDACTED Sat Jan 15 05:17:54 2011 From: toby@REDACTED (Toby Thain) Date: Sat, 15 Jan 2011 01:17:54 -0300 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: References: <992935859.258191294521884974.JavaMail.root@zimbra> <1693153562.258211294521899230.JavaMail.root@zimbra> Message-ID: <4D311FF2.5040203@telegraphics.com.au> On 11-01-15 12:40 AM, Ryan Zezeski wrote: > Teach Yourself Programming in 10 Years! (or the next ones free) > > http://norvig.com/21-days.html > > Neat side story...For those of you who know the name Hunter S. Thompson you > might be interested to know the he learned to write by typing out Hemingway > word for word, letter for letter. Do you have a citation for that? Curious if true, but... I've used that fact as a reminder that > the act of doing is the best teacher. If you're not failing everyday, then > you're not trying hard enough. ...how can one "fail" at typing out Hemingway letter by letter, apart from occasionally fat fingering a key? I think one probably _can_ learn a lot about writing by thoughtfully copying out good writing. But there seems to be no "fail-retry-learn" loop in it. That comes when one tries WRITING... :) --Toby Success isn't nearly as insightful as > failure. > > -Ryan > > On Sat, Jan 8, 2011 at 4:24 PM, Robert Virding < > robert.virding@REDACTED> wrote: > >> This whole dscussion reminds a little of the Paul Graham essay "Beating the >> Averages", http://www.paulgraham.com/avg.html , where he discusses the >> Blub paradox. >> >> Robert >> >> > From rzezeski@REDACTED Sat Jan 15 05:58:38 2011 From: rzezeski@REDACTED (Ryan Zezeski) Date: Fri, 14 Jan 2011 23:58:38 -0500 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: <4D311FF2.5040203@telegraphics.com.au> References: <992935859.258191294521884974.JavaMail.root@zimbra> <1693153562.258211294521899230.JavaMail.root@zimbra> <4D311FF2.5040203@telegraphics.com.au> Message-ID: You're correct, the two ideas are different. I just tend to think of them both at the same time. My point was that practice (even if it's typing out something verbatim) is probably, pound for pound, the best way to learn. Furthermore, practice includes trying different things, even if they don't initially make sense because by doing that you find out what works and what doesn't. This means that you're going to fail, a lot. But with each failure you gain knowledge and insight. That's all I was trying to convey. That and the fact that you shouldn't expect to learn something in a few weeks, or even months with any great depth. As for the citation, it comes straight from the horse's mouth in the documentary "Buy the Ticket, Take the Ride." http://www.imdb.com/title/tt0775438/ -Ryan On Fri, Jan 14, 2011 at 11:17 PM, Toby Thain wrote: > On 11-01-15 12:40 AM, Ryan Zezeski wrote: > > Teach Yourself Programming in 10 Years! (or the next ones free) > > > > http://norvig.com/21-days.html > > > > Neat side story...For those of you who know the name Hunter S. Thompson > you > > might be interested to know the he learned to write by typing out > Hemingway > > word for word, letter for letter. > > Do you have a citation for that? Curious if true, but... > > I've used that fact as a reminder that > > the act of doing is the best teacher. If you're not failing everyday, > then > > you're not trying hard enough. > > ...how can one "fail" at typing out Hemingway letter by letter, apart > from occasionally fat fingering a key? > > I think one probably _can_ learn a lot about writing by thoughtfully > copying out good writing. But there seems to be no "fail-retry-learn" > loop in it. That comes when one tries WRITING... :) > > --Toby > > Success isn't nearly as insightful as > > failure. > > > > -Ryan > > > > On Sat, Jan 8, 2011 at 4:24 PM, Robert Virding < > > robert.virding@REDACTED> wrote: > > > >> This whole dscussion reminds a little of the Paul Graham essay "Beating > the > >> Averages", http://www.paulgraham.com/avg.html , where he discusses the > >> Blub paradox. > >> > >> Robert > >> > >> > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From ebegumisa@REDACTED Sat Jan 15 11:41:03 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Sat, 15 Jan 2011 21:41:03 +1100 Subject: [erlang-questions] We need a better marketing division :-) In-Reply-To: <4D2EC4AD.9040701@llaisdy.com> References: <24250519.271261294767163150.JavaMail.root@zimbra> <4D2D56B9.90103@lbox.cc> <4D2EC4AD.9040701@llaisdy.com> Message-ID: Seconded. This is indeed a fascinating paper :) I've spent my entire Saturday reading-up on the topics discussed in it. The possibilities... - Edmond - On Thu, 13 Jan 2011 20:23:57 +1100, Ivan Uemlianin wrote: > On 12/01/2011 09:49, Ulf Wiger wrote: >> One thing that comes to mind is work done by the USAF Cognitive >> Modelling Group in Scottsdale, AZ on using Erlang for very-large- >> scale cognitive modeling. Part of the work has been published >> >> http://iccm2010.cs.drexel.edu/proceedings/papers/Douglass.pdf > > That is a very interesting paper, thank you. > > From time to time I google around for natural language processing > research using Erlang (esp. interested in speech recognition). I've yet > to find any. If anyone here knows of any such work I'd love to hear of > it. > > Best wishes > > Ivan > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From watson.timothy@REDACTED Sat Jan 15 18:03:28 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Sat, 15 Jan 2011 17:03:28 +0000 Subject: [erlang-questions] about the use of parametrized modules In-Reply-To: References: <20110113232652.GA1737@hanele.lan> <20110114002255.GA3010@hanele.lan> Message-ID: > i'd simply would love not having the impression that some things > cannot be discussed in this list. there's no need to express harsh > feelings or absolute positions and, most of all, creativity and > communities need new blood, open code, libraries, and support. > You're right of course - room for healthy discussion and debate is important to any community and I don't think anyone here wants to discourage that. For my part, I'm very impressed with frameworks like misultin and mochiweb and have used both in production successfully without any major issues or downtime! If the OTP team supported the parameterised module feature fully, making sure to compensate for the issues mentioned (such as shadowing variables) and a few others (patching the cover and common_test tools to support them properly for example) then this discussion would probably be one of coding style rather than what is TTT (TM). Every application is going to have some "global state" in one sense or another, so I think getting a clear picture over whether this feature is going to be supported and encouraged in the long term (or not) is what we really need. Or perhaps alternative EEP suggestions about syntactic sugar for passing state to and from callback modules. Thanks for all your hard work on Misultin btw - whatever my opinion of parameterised modules, I'm a very happy user! Cheers, Tim From max.lapshin@REDACTED Sat Jan 15 19:23:10 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 15 Jan 2011 21:23:10 +0300 Subject: [erlang-questions] about the use of parametrized modules In-Reply-To: References: <20110113232652.GA1737@hanele.lan> <20110114002255.GA3010@hanele.lan> Message-ID: I can tell why I use parameterized modules in erlyvideo. This is list of module chain, where functions are called: {rtmp_handlers, [apps_login, apps_streaming]}. rtmp_session is calling in such manner: Module:connect(Session, Funcall) Now we need a configurable module: auth_users_limit. It requires somehow to store additional configuration variable: limit of users. The easies solution for me seems to store it as a parameter for module: {rtmp_handlers, [{auth_users_limit, 100}, apps_login, apps_streaming]} After that nothing is changed in rtmp_session: Module:connect(Session, Funcall), but Module now is just a tuple from config: {auth_users_limit, 100}. It happened to be very convenient. From freza@REDACTED Sat Jan 15 19:49:13 2011 From: freza@REDACTED (Jachym Holecek) Date: Sat, 15 Jan 2011 18:49:13 +0000 Subject: [erlang-questions] about the use of parametrized modules In-Reply-To: References: <20110114002255.GA3010@hanele.lan> Message-ID: <20110115184913.GA4510@hanele.lan> # Max Lapshin 2011-01-15: > I can tell why I use parameterized modules in erlyvideo. > > This is list of module chain, where functions are called: > > {rtmp_handlers, [apps_login, apps_streaming]}. > > rtmp_session is calling in such manner: Module:connect(Session, Funcall) > > Now we need a configurable module: auth_users_limit. It requires > somehow to store additional configuration variable: limit of users. > The easies solution for me seems to store it as a parameter for module: > > {rtmp_handlers, [{auth_users_limit, 100}, apps_login, apps_streaming]} > > After that nothing is changed in rtmp_session: > Module:connect(Session, Funcall), but Module now is just a tuple from > config: {auth_users_limit, 100}. > > It happened to be very convenient. The general point aside for a moment, making such a strong assumption about internal representation of parametrized modules seems like a very curageous attitude indeed ;-). Unless, of course, such a promise is made explicitely, like is done with record syntax. Whether that's the case or not I couldn't tell -- the PM paper only seems to be "available" from the ACM Borg Cube. Regards, -- Jachym From hynek@REDACTED Sat Jan 15 21:17:50 2011 From: hynek@REDACTED (Hynek Vychodil) Date: Sat, 15 Jan 2011 21:17:50 +0100 Subject: [erlang-questions] about the use of parametrized modules In-Reply-To: References: <20110113232652.GA1737@hanele.lan> <20110114002255.GA3010@hanele.lan> Message-ID: OTP stdlib module io is doing principally same and doesn't need parametrized module. On Sat, Jan 15, 2011 at 7:23 PM, Max Lapshin wrote: > I can tell why I use parameterized modules in erlyvideo. > > This is list of module chain, where functions are called: > > {rtmp_handlers, [apps_login, apps_streaming]}. > > rtmp_session is calling in such manner: ?Module:connect(Session, Funcall) > > Now we need a configurable module: ?auth_users_limit. It requires > somehow to store additional configuration variable: limit of users. > The easies solution for me seems to store it as a parameter for module: > > {rtmp_handlers, [{auth_users_limit, 100}, apps_login, apps_streaming]} > > After that nothing is changed in rtmp_session: > Module:connect(Session, Funcall), but Module now is just a tuple from > config: {auth_users_limit, 100}. > > It happened to be very convenient. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From ulf.wiger@REDACTED Sat Jan 15 21:36:26 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Sat, 15 Jan 2011 21:36:26 +0100 Subject: [erlang-questions] UML or other modeling tools In-Reply-To: <4D2F9CD1.9030601@aist.go.jp> References: <2E275C30-C1EE-453B-8CE2-1C934F17A50F@erlang-solutions.com> <4D2F9CD1.9030601@aist.go.jp> Message-ID: It's not an entirely easy question to answer, because there are (at least) three levels to consider: - what the UML standard says - *which version* of the UML standard (since there are semantic differences) - what the specific tool actually supports The UML standard doesn't specify in which order state machines should process messages, but in practice, this usually means FIFO. Messages are processed with run-to-completion semantics, so so far, it's pretty similar to Erlang's gen_server. However, there is no selective receive support in UML. You can specify an event as 'deferrable' in a given state (this is not exactly prominent in the spec, and not all tools support it). You can also explicitly defer and recall messages. One consequence of this is that it's near impossible to model a generic RPC* in UML. In UML 2.0, there is a synchronousCall() method, so you can get by to some extent anyway? * or gen_server:call() UML's method of tackling state machine complexity is to support hierarchical state machines, but this mainly gives some modularity assuming that the order of messages is ok; it cannot really help you if messages can arrive out of order or in undefined order. BR, Ulf W On 14 Jan 2011, at 01:46, Geoffrey Biggs wrote: > On 09/01/11 23:06, Ulf Wiger wrote: >> - The things most interesting to model in an Erlang program are the >> state machines and the interaction between processes, but UML >> state machines have very complex - and most importantly, *different* >> - semantics, making it quite probable that the design becomes subject >> to compromises forced by the modeling tool, rather than reflecting the >> power of the programming language. > > I wonder if you could expand on what these differences are. As a novice > at both UML and Erlang, I don't want to fall into any traps caused by > making false assumptions. > > Geoff > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From torben.lehoff@REDACTED Sat Jan 15 21:42:23 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Sat, 15 Jan 2011 21:42:23 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: References: <992935859.258191294521884974.JavaMail.root@zimbra> <1693153562.258211294521899230.JavaMail.root@zimbra> <4D311FF2.5040203@telegraphics.com.au> Message-ID: The only persons that do not fail are those that do nothing. The irony of that is that that in itself is a mistake and probably leads to failing... Or as Wayne Gretzky puts it: You miss 100% of the shots you don't take. Cheers to continuous failures! Torben On Sat, Jan 15, 2011 at 05:58, Ryan Zezeski wrote: > You're correct, the two ideas are different. I just tend to think of them > both at the same time. My point was that practice (even if it's typing out > something verbatim) is probably, pound for pound, the best way to learn. > Furthermore, practice includes trying different things, even if they don't > initially make sense because by doing that you find out what works and what > doesn't. This means that you're going to fail, a lot. But with each > failure you gain knowledge and insight. That's all I was trying to convey. > That and the fact that you shouldn't expect to learn something in a few > weeks, or even months with any great depth. > > As for the citation, it comes straight from the horse's mouth in the > documentary "Buy the Ticket, Take the Ride." > > http://www.imdb.com/title/tt0775438/ > > -Ryan > > > On Fri, Jan 14, 2011 at 11:17 PM, Toby Thain >wrote: > > > On 11-01-15 12:40 AM, Ryan Zezeski wrote: > > > Teach Yourself Programming in 10 Years! (or the next ones free) > > > > > > http://norvig.com/21-days.html > > > > > > Neat side story...For those of you who know the name Hunter S. Thompson > > you > > > might be interested to know the he learned to write by typing out > > Hemingway > > > word for word, letter for letter. > > > > Do you have a citation for that? Curious if true, but... > > > > I've used that fact as a reminder that > > > the act of doing is the best teacher. If you're not failing everyday, > > then > > > you're not trying hard enough. > > > > ...how can one "fail" at typing out Hemingway letter by letter, apart > > from occasionally fat fingering a key? > > > > I think one probably _can_ learn a lot about writing by thoughtfully > > copying out good writing. But there seems to be no "fail-retry-learn" > > loop in it. That comes when one tries WRITING... :) > > > > --Toby > > > > Success isn't nearly as insightful as > > > failure. > > > > > > -Ryan > > > > > > On Sat, Jan 8, 2011 at 4:24 PM, Robert Virding < > > > robert.virding@REDACTED> wrote: > > > > > >> This whole dscussion reminds a little of the Paul Graham essay > "Beating > > the > > >> Averages", http://www.paulgraham.com/avg.html , where he discusses > the > > >> Blub paradox. > > >> > > >> Robert > > >> > > >> > > > > > > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > -- http://www.linkedin.com/in/torbenhoffmann From kuleshovmail@REDACTED Sun Jan 16 06:59:21 2011 From: kuleshovmail@REDACTED (shk) Date: Sat, 15 Jan 2011 21:59:21 -0800 (PST) Subject: Erlang listen port Message-ID: <1295157561603-3219753.post@n4.nabble.com> Hello, I write simple pop3 server. I try to listen 110 port: -module(test). -export([listen/1]). -define(TCP_OPTIONS, [binary, {packet, 0}, {active, false}, {reuseaddr, true}]). % Call echo:listen(Port) to start the service. listen(Port) -> case gen_tcp:listen(Port, ?TCP_OPTIONS) of {ok, LSocket} -> accept(LSocket); {error, Reason} -> Reason end. % Wait for incoming connections and spawn the echo loop when we get one. accept(LSocket) -> {ok, Socket} = gen_tcp:accept(LSocket), spawn(fun() -> loop(Socket) end), accept(LSocket). % Echo back whatever data we receive on Socket. loop(Socket) -> case gen_tcp:recv(Socket, 0) of {ok, Data} -> gen_tcp:send(Socket, Data), loop(Socket); {error, closed} -> ok end. I run this code in eshell: 1>test:listen(110). eacces What's wrong? Thank you. -- View this message in context: http://erlang.2086793.n4.nabble.com/Erlang-listen-port-tp3219753p3219753.html Sent from the Erlang Questions mailing list archive at Nabble.com. From kuleshovmail@REDACTED Sun Jan 16 07:02:00 2011 From: kuleshovmail@REDACTED (shk) Date: Sat, 15 Jan 2011 22:02:00 -0800 (PST) Subject: Erlang listen port In-Reply-To: <1295157561603-3219753.post@n4.nabble.com> References: <1295157561603-3219753.post@n4.nabble.com> Message-ID: <1295157720706-3219755.post@n4.nabble.com> I try run it under root and it's work! How can i make that it works under user, without root. -- View this message in context: http://erlang.2086793.n4.nabble.com/Erlang-listen-port-tp3219753p3219755.html Sent from the Erlang Questions mailing list archive at Nabble.com. From andrew@REDACTED Sun Jan 16 07:22:03 2011 From: andrew@REDACTED (Andrew Tunnell-Jones) Date: Sun, 16 Jan 2011 17:22:03 +1100 Subject: [erlang-questions] Re: Erlang listen port In-Reply-To: <1295157720706-3219755.post@n4.nabble.com> References: <1295157561603-3219753.post@n4.nabble.com> <1295157720706-3219755.post@n4.nabble.com> Message-ID: <1274E004-65A2-4645-B0F4-0C2FA421E638@tj.id.au> On 16/01/2011, at 5:02 PM, shk wrote: > > I try run it under root and it's work! How can i make that it works under > user, without root. > Although geared towards yaws, this page covers all of the options for binding to a privileged port that I'm aware of: http://yaws.hyber.org/privbind.yaws ? Andrew http://andrew.tj.id.au/ From gijsbert.de.haan@REDACTED Sun Jan 16 11:33:29 2011 From: gijsbert.de.haan@REDACTED (Gijsbert de Haan) Date: Sun, 16 Jan 2011 11:33:29 +0100 Subject: [erlang-questions] Re: how to patch a standard module In-Reply-To: <3ABF5364-A445-4EB8-8AF6-48B2DC8C2FB5@gmail.com> References: <73cd3e8a-d6b4-4910-a2f2-9359252a3288@o4g2000yqd.googlegroups.com> <216f04d8-c713-4f9f-9afd-b5c212311171@u32g2000yqe.googlegroups.com> <3ABF5364-A445-4EB8-8AF6-48B2DC8C2FB5@gmail.com> Message-ID: James, the AST route sounds interesting, do you have that code available somewhere? I'll have a look at the -extends too. Thanks for all the suggestions! On Fri, Jan 14, 2011 at 6:01 PM, James Churchman wrote: > I have 0 experience of beam? but that sounds much more tricky than the AST method, and only has the advantage of working without a copy of the original source? just now tried -extends, but it wont take a file path, only a module atom & i don't think it allows for the file it's extending to be of the same name! so in this exact case it wont work. The AST one can be done in less than 10 lines which i can dig out if of any use. In this very specific case tho the -include() with the include directory set does the job the simplest :-) > > > On 14 Jan 2011, at 16:42, Tomas Abrahamsson wrote: > >>>>>> I'd like to extend httpd_request.erl to handle the OPTIONS method. >>>>>> I've managed to do this by copying httpd_request.erl into my project >>>>>> and making sure it overrides the standard version. >>>>>> My question: is it possible to redefine a function in a standard >>>>>> module at runtime? Then I could define my own validate(Method, Uri, >>>>>> Version) function (just a few lines) and keep using the standard >>>>>> httdp_request.erl. >> >>> It is actually possible to to what you want. I have used a piece of >>> code that could replace functions in existing modules by manipulating >>> the beam code. In that project we only used it in eunit testing to stub >>> functions in order to isolate the tested code. >>> I'm not sure about using that method in a production system though. >>> I don't have the code available any more so I can't provide any details >>> about it but I think that Tomas Abrahamsson who sometimes contributes to >>> this list could give some pointers on how to do it if he reads this. >> >> The stubber that Ola refers to is a bit similar to meck >> and some other mocking libraries. It provides the >> possibility to override all or only some functions of a >> module, while still making the rest of the functions >> available unchanged. >> >> The stubber has evolved a bit since: Klas Johansson >> has done a fair bit of useful work on it, to say the >> least, and I guess he might even make his work publicly >> available eventually(?) >> >> It is mainly intended for (e)unit testing, though. >> >> >> The way the existing modules were extended with new or >> changed functions, was by simply renaming the beam file, >> then provide a new beam module which either calls the >> renamed module for all non-changed functions, or uses the >> -extends(RenamedModule) feature (undocumented, I think). >> The new module was compiled to a binary and loaded with >> code:load_binary/3. >> >> See https://github.com/klajo/hacks/tree/master/beam >> at Klas' repository for some code to rename a beam >> module. >> >> See http://www.trapexit.org/Extend_Module for more info >> on the -extends attribute. >> >> BRs >> Tomas >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > From abdoo.mahran@REDACTED Sun Jan 16 15:19:17 2011 From: abdoo.mahran@REDACTED (Abdul Fattah Mahran) Date: Sun, 16 Jan 2011 16:19:17 +0200 Subject: [erlang-questions] Erlang listen port In-Reply-To: <1295157561603-3219753.post@n4.nabble.com> References: <1295157561603-3219753.post@n4.nabble.com> Message-ID: Hi all, The problem is you are using a reserved port. Thanks On Sun, Jan 16, 2011 at 7:59 AM, shk wrote: > > Hello, > > I write simple pop3 server. I try to listen 110 port: > > -module(test). > > -export([listen/1]). > > -define(TCP_OPTIONS, [binary, {packet, 0}, {active, false}, {reuseaddr, > true}]). > > % Call echo:listen(Port) to start the service. > listen(Port) -> > case gen_tcp:listen(Port, ?TCP_OPTIONS) of > {ok, LSocket} -> > accept(LSocket); > {error, Reason} -> > Reason > end. > > % Wait for incoming connections and spawn the echo loop when we get one. > accept(LSocket) -> > {ok, Socket} = gen_tcp:accept(LSocket), > spawn(fun() -> loop(Socket) end), > accept(LSocket). > > % Echo back whatever data we receive on Socket. > loop(Socket) -> > case gen_tcp:recv(Socket, 0) of > {ok, Data} -> > gen_tcp:send(Socket, Data), > loop(Socket); > {error, closed} -> > ok > end. > > I run this code in eshell: > 1>test:listen(110). > eacces > > What's wrong? > > Thank you. > > -- > View this message in context: > http://erlang.2086793.n4.nabble.com/Erlang-listen-port-tp3219753p3219753.html > Sent from the Erlang Questions mailing list archive at Nabble.com. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Thanks Best Regards, Abd El-Fattah Mahran From alain.odea@REDACTED Sun Jan 16 16:00:13 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Sun, 16 Jan 2011 11:30:13 -0330 Subject: [erlang-questions] Re: Erlang listen port In-Reply-To: <1274E004-65A2-4645-B0F4-0C2FA421E638@tj.id.au> References: <1295157561603-3219753.post@n4.nabble.com> <1295157720706-3219755.post@n4.nabble.com> <1274E004-65A2-4645-B0F4-0C2FA421E638@tj.id.au> Message-ID: <2A801C16-C1FE-418D-B7DF-4450A0586753@gmail.com> On 2011-01-16, at 2:52, Andrew Tunnell-Jones wrote: > On 16/01/2011, at 5:02 PM, shk wrote: > >> >> I try run it under root and it's work! How can i make that it works under >> user, without root. >> > > Although geared towards yaws, this page covers all of the options for binding to a privileged port that I'm aware of: > > http://yaws.hyber.org/privbind.yaws Thank you Andrew. That Yaws article is generally applicable for binding privileged ports in any program. I like the idea of using authbind or privbind for running Zotonic on 80 or 443. Unfortunately I got inpatient and fell back on NAT pre-routing with iptables (which I learned when setting up TeamCity). I'm going to give authbind and privbind another shot. Thanks again, Alain > > ? Andrew > http://andrew.tj.id.au/ From kuleshovmail@REDACTED Sun Jan 16 18:36:30 2011 From: kuleshovmail@REDACTED (shk) Date: Sun, 16 Jan 2011 09:36:30 -0800 (PST) Subject: Simple tcp server Message-ID: <1295199390279-3220218.post@n4.nabble.com> Hello, I'm new in erlang and try to write simple tcp server. Code : https://gist.github.com/781975 My Code I start server, then connect some client with help telnet. When I try to stop it i call stop() And after it thats CPU engaged 97-100 %. I think that i must close socket. How can I make it correctly? And just do the correct terminate server? Thank you. -- View this message in context: http://erlang.2086793.n4.nabble.com/Simple-tcp-server-tp3220218p3220218.html Sent from the Erlang Questions mailing list archive at Nabble.com. From kaiduanx@REDACTED Sun Jan 16 19:02:04 2011 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Sun, 16 Jan 2011 13:02:04 -0500 Subject: [erlang-questions] Simple tcp server In-Reply-To: <1295199390279-3220218.post@n4.nabble.com> References: <1295199390279-3220218.post@n4.nabble.com> Message-ID: The gen_server is the controlling process of the listen socket, when you terminate the gen_server, the gen_tcp:accept(Socket) will return error with reason closed, and it goes endless loop in your accept() process, so your CPU hits 97-100%. You can catch [error, closed} for gen_tcp:accept(), and close the listen socket. /Kaiduan On Sun, Jan 16, 2011 at 12:36 PM, shk wrote: > > Hello, > > I'm new in erlang and try to write simple tcp server. > > Code : ?https://gist.github.com/781975 ?My Code > > I start server, then connect some client with help telnet. When I try to > stop it i call stop() And after it thats CPU engaged 97-100 %. I think that > i must close socket. How can I make it correctly? And just do the correct > terminate server? > > Thank you. > > -- > View this message in context: http://erlang.2086793.n4.nabble.com/Simple-tcp-server-tp3220218p3220218.html > Sent from the Erlang Questions mailing list archive at Nabble.com. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From fernando.benavides@REDACTED Sun Jan 16 20:09:26 2011 From: fernando.benavides@REDACTED (Fernando Benavides) Date: Sun, 16 Jan 2011 16:09:26 -0300 Subject: [erlang-questions] Re: Erlang listen port In-Reply-To: <2A801C16-C1FE-418D-B7DF-4450A0586753@gmail.com> References: <1295157561603-3219753.post@n4.nabble.com> <1295157720706-3219755.post@n4.nabble.com> <1274E004-65A2-4645-B0F4-0C2FA421E638@tj.id.au> <2A801C16-C1FE-418D-B7DF-4450A0586753@gmail.com> Message-ID: You may need to use something like fdsrv. Using jungerl sources, I've created a project on github with that: http://github.com/elbrujohalcon/fd_server On Sun, Jan 16, 2011 at 12:00 PM, Alain O'Dea wrote: > On 2011-01-16, at 2:52, Andrew Tunnell-Jones wrote: > > > On 16/01/2011, at 5:02 PM, shk wrote: > > > >> > >> I try run it under root and it's work! How can i make that it works > under > >> user, without root. > >> > > > > Although geared towards yaws, this page covers all of the options for > binding to a privileged port that I'm aware of: > > > > http://yaws.hyber.org/privbind.yaws > > Thank you Andrew. That Yaws article is generally applicable for binding > privileged ports in any program. > > I like the idea of using authbind or privbind for running Zotonic on 80 > or 443. Unfortunately I got inpatient and fell back on NAT pre-routing > with iptables (which I learned when setting up TeamCity). I'm going to give > authbind and privbind another shot. > > Thanks again, > Alain > > > > > ? Andrew > > http://andrew.tj.id.au/ > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From jesper.louis.andersen@REDACTED Sun Jan 16 20:23:07 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Sun, 16 Jan 2011 20:23:07 +0100 Subject: [erlang-questions] Simple tcp server In-Reply-To: References: <1295199390279-3220218.post@n4.nabble.com> Message-ID: On Sun, Jan 16, 2011 at 19:02, Kaiduan Xie wrote: > The gen_server is the controlling process of the listen socket, when > you terminate the gen_server, the gen_tcp:accept(Socket) will return > error with reason closed, and it goes endless loop in your accept() > process, so your CPU hits 97-100%. > > You can catch [error, closed} for gen_tcp:accept(), and close the listen socket. An additional problem is that each accepted socket is controlled by the spawned acceptor process. So if it decides to die, all your sockets will be gone (A socket is automatically closed when its controlling process dies). You should, in time, consider moving the controller so it is controlled by the right process. -- J. From pablo.platt@REDACTED Mon Jan 17 02:19:14 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Sun, 16 Jan 2011 17:19:14 -0800 (PST) Subject: deb package from erlang app release Message-ID: <963495.11286.qm@web112616.mail.gq1.yahoo.com> Hi, I've created a release with rebar and now I'm trying to package it as deb to use in ubuntu. I'm creating a deb template with dh-make and building it with debuild. I'm getting several errors when the package is being tested with lintian. I'm not sure if it's related to the way I'm building the package, the way rebar creates the release or maybe incompatibility of deb packages and erlang releases. Do I need to break the release and spread it on several places on the file system or should I put everything under a specific folder? In the latter, which folder? var/lib/myapp, usr/share/myapp ? I think that RabbitMQ and Ejabberd deb packages don't use a release. Is there an example of a deb package that does use a release? Thanks From serge@REDACTED Mon Jan 17 04:28:34 2011 From: serge@REDACTED (Serge Aleynikov) Date: Sun, 16 Jan 2011 22:28:34 -0500 Subject: [erlang-questions] [patch] new float_to_list/2 In-Reply-To: References: <4D2D3FA8.1000009@aleynikov.org> <4D2DB65C.1090106@aleynikov.org> <4D2DE13E.7040404@aleynikov.org> Message-ID: <4D33B762.3080202@aleynikov.org> This implementation has been submitted as the pull request: https://github.com/erlang/otp/pull/9 This version is also optimized to run 5-10x faster than float_to_list/1 for floats under 2^52. Regards, Serge On 1/14/2011 10:33 AM, Gleb Peregud wrote: > Here's my patch: > > http://www.erlang.org/cgi-bin/ezmlm-cgi/4/43529 > > It uses double sprintf invocations. Your solution is clearly better (I > didn't knew it is possible to specify precision in args). > > Your benchmark shows that difference is negligible. Between integer as > a second parameter or a proplist (~0.4%). Though it might be a bit > more "stable" in terms of running time, but still probably not worth > considering it for performance reasons. It still may be useful in > terms of simpler API, but I have no strong opinion on it. > > So generally +1 on including Serge's patch into Erlang. Serge, can you > put it into GitHub pull request? As described here: > > https://github.com/erlang/otp/wiki/submitting-patches > > Best, > Gleb > > On Wed, Jan 12, 2011 at 18:13, Serge Aleynikov wrote: >> I did a micro-benchmark on a slightly modified version of the BIF that >> accepts an integer as its options. The results shown below display a very >> insignificant difference between a call with no options and a call with an >> integer precision passed as the second argument: >> >> 1> test:test(). >> float_to_list(123.4, []) = {0.619512,"123.4000"} >> float_to_list(123.4, [{precision, 4}]) = {0.624895,"123.4000"} >> float_to_list(123.4, 4) = {0.622896,"123.4000"} >> >> The majority of time is actually spent in the printf(3) function, which >> takes longer to execute when given the "%.*f" argument compared to "%g" as >> in float_to_list/1 case. >> >> Did your patch rely on printf? >> >> >> On 1/12/2011 9:23 AM, Gleb Peregud wrote: >>> >>> Some time ago I've submitted similar but simpler patch. It was a >>> float_to_list/2 with a second parameter being an integer specifying >>> precision. For me it was important to generate A LOT of floats as >>> strings as fast as possible with specified precision. Serge's version >>> has an overhead of inspecting proplist of the second parameter. So I >>> was wondering about introducing two versions of this function: with a >>> proplist as a second parameter and with a number as a second >>> parameter. Alternatively proplist version can be factored out into >>> float_to_list_opts/2. >>> >>> Just my 0.2 cents >>> >>> On Wed, Jan 12, 2011 at 15:10, Serge Aleynikov >>> wrote: >>>> >>>> The reason I called it precision was to be consistent with the naming >>>> convention of the printf function. Below is the extract from "man 3 >>>> printf", which refers to the digits after the decimal point as >>>> "precision": >>>> >>>> f, F The double argument is rounded and converted to decimal >>>> notation in the style [-]ddd.ddd, where the number of >>>> digits after the decimal-point character is equal to the >>>> precision specification. If the precision is missing, >>>> it is taken as 6; if the precision is explicitly zero, >>>> no decimal-point character appears. If a decimal point >>>> appears, at least one digit appears before it. >>>> >>>> I don't have a very strong preference for calling it precision or scale, >>>> but >>>> I do have a strong preference for including this patch in the >>>> distribution, >>>> because the default behavior of float_to_list/1 hard-coded in C is >>>> deficient. >>>> >>>> >>>> On 1/12/2011 4:58 AM, nox wrote: >>>>> >>>>> Il should be called "scale", shouldn't it? >>>>> >>>>> Le 12 janv. 2011 ? 10:26, Pierpaolo Bernardi a >>>>> ?crit >>>>> : >>>>> >>>>>> On Wed, Jan 12, 2011 at 06:44, Serge Aleynikov >>>>>> wrote: >>>>>>> >>>>>>> Attached please find a patch that adds a new float_to_list/2 BIF. The >>>>>>> patch >>>>>>> was created off of the master branch of https://github.com/erlang/otp. >>>>>>> >>>>>>> This BIF solves a problem of float_to_list/1 that doesn't allow >>>>>>> specifying >>>>>>> the number of digits after the decimal point when formatting floats. >>>>>>> >>>>>>> float_to_list(Float, Options) -> string() >>>>>>> >>>>>>> Float = float() >>>>>>> Options = [Option] >>>>>>> Option = {precision, Precision::integer()} | compact >>>>>>> >>>>>>> Text representation of a float formatted using given options >>>>>>> >>>>>>> Returns a string which corresponds to the text >>>>>>> representation of Float using fixed decimal point formatting. >>>>>>> When precision option is specified >>>>>>> the returned value will contain at most Precision number of >>>>>>> digits past the decimal point. When compact option is provided >>>>>>> the trailing zeros at the end of the list are truncated. >>>>>> >>>>>> I think the option is misnamed. >>>>>> >>>>>> In the usual terminology, 'precision' is the total number of >>>>>> significative digits, not only the ones past the decimal point. >>>>>> >>>>>> Cheers >>>>>> P. >>>>>> >>>>>> ________________________________________________________________ >>>>>> erlang-questions (at) erlang.org mailing list. >>>>>> See http://www.erlang.org/faq.html >>>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>>>> >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>>> >> From kuleshovmail@REDACTED Mon Jan 17 07:47:22 2011 From: kuleshovmail@REDACTED (Alexander Kuleshov) Date: Mon, 17 Jan 2011 06:47:22 +0000 Subject: [erlang-questions] Simple tcp server In-Reply-To: References: <1295199390279-3220218.post@n4.nabble.com> Message-ID: Thank you very much for your help. I remake my server. Code : https://gist.github.com/782566 Now it's not CPU 100%. But i'm new in erlang and i want to ask, maybe i can improve this code or server design, or maybe i don't see hide errors? Thank you. From nox@REDACTED Mon Jan 17 09:42:20 2011 From: nox@REDACTED (nox) Date: Mon, 17 Jan 2011 09:42:20 +0100 Subject: [erlang-questions] [patch] new float_to_list/2 In-Reply-To: References: <4D2D3FA8.1000009@aleynikov.org> <4D2DB65C.1090106@aleynikov.org> <4D2DE13E.7040404@aleynikov.org> Message-ID: <0CF9E046-9111-4825-A883-F3CE6A95071B@dev-extend.eu> Since when does OTP use Github's pull requests? Le 14 janv. 2011 ? 16:33, Gleb Peregud a ?crit : > So generally +1 on including Serge's patch into Erlang. Serge, can you > put it into GitHub pull request? As described here: > > https://github.com/erlang/otp/wiki/submitting-patches > > Best, > Gleb -- Anthony Ramine Dev:Extend http://dev-extend.eu From burbas@REDACTED Mon Jan 17 10:27:04 2011 From: burbas@REDACTED (Niclas Axelsson) Date: Mon, 17 Jan 2011 10:27:04 +0100 Subject: [erlang-questions] [patch] new float_to_list/2 In-Reply-To: <0CF9E046-9111-4825-A883-F3CE6A95071B@dev-extend.eu> References: <4D2D3FA8.1000009@aleynikov.org> <4D2DB65C.1090106@aleynikov.org> <4D2DE13E.7040404@aleynikov.org> <0CF9E046-9111-4825-A883-F3CE6A95071B@dev-extend.eu> Message-ID: <4D340B68.8020402@erlang.org> On 01/17/2011 09:42 AM, nox wrote: > Since when does OTP use Github's pull requests? > We don't :-). Thanks for your patch, Serge. I've looked at your otp-repository, and merged your 'float_to_list_2'-branch into 'pu'. It would be good if you could close the pull request since it's already merged into 'pu'. Thanks, Niclas Axelsson, Erlang/OTP From huss01@REDACTED Mon Jan 17 10:32:25 2011 From: huss01@REDACTED (=?ISO-8859-1?Q?H=E5kan_Huss?=) Date: Mon, 17 Jan 2011 10:32:25 +0100 Subject: [erlang-questions] Erlang and the learning curve In-Reply-To: References: <992935859.258191294521884974.JavaMail.root@zimbra> <1693153562.258211294521899230.JavaMail.root@zimbra> Message-ID: On Sat, Jan 15, 2011 at 04:40, Ryan Zezeski wrote: > Teach Yourself Programming in 10 Years! (or the next ones free) > > http://norvig.com/21-days.html > I prefer this way: http://abstrusegoose.com/249 Regards, /H?kan From gleber.p@REDACTED Mon Jan 17 10:33:06 2011 From: gleber.p@REDACTED (Gleb Peregud) Date: Mon, 17 Jan 2011 10:33:06 +0100 Subject: [erlang-questions] [patch] new float_to_list/2 In-Reply-To: <4D340B68.8020402@erlang.org> References: <4D2D3FA8.1000009@aleynikov.org> <4D2DB65C.1090106@aleynikov.org> <4D2DE13E.7040404@aleynikov.org> <0CF9E046-9111-4825-A883-F3CE6A95071B@dev-extend.eu> <4D340B68.8020402@erlang.org> Message-ID: On Mon, Jan 17, 2011 at 10:27, Niclas Axelsson wrote: > On 01/17/2011 09:42 AM, nox wrote: >> Since when does OTP use Github's pull requests? > We don't :-). Sorry for introducing confusion. I was too hasty and didn't double check that :) > Thanks for your patch, Serge. > I've looked at your otp-repository, and merged your 'float_to_list_2'-branch > into 'pu'. That's awesome this may be included into Erlang/OTP :) From pablo.platt@REDACTED Mon Jan 17 11:50:21 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Mon, 17 Jan 2011 02:50:21 -0800 (PST) Subject: [erlang-questions] deb package from erlang app release In-Reply-To: <1295259397.17254.16.camel@andre.mz.digirati.com.br> References: <963495.11286.qm@web112616.mail.gq1.yahoo.com> <1295259397.17254.16.camel@andre.mz.digirati.com.br> Message-ID: <499245.45314.qm@web112616.mail.gq1.yahoo.com> >On Sun, 2011-01-16 at 17:19 -0800, Pablo Platt wrote: >> Do I need to break the release and spread it on several places on the file >> system or should I put everything under a specific folder? >> In the latter, which folder? var/lib/myapp, usr/share/myapp ? > >When I created a deb package for my application I simply copied the >release to /opt and then created symlinks from the appropriate locations >to the release (e.g. /etc/init.d/foo -> /opt/foo/bin/foo). > >HTH, >Andre When I'm copying the release to /opt (in the dh-make install script) I'm getting many errors and warnings: W: myapp: executable-not-elf-or-script ./opt/myapp/lib/observer-0.9.8.4/priv/bin/getop.bat W: myapp: executable-not-elf-or-script ./opt/myapp/lib/observer-0.9.8.4/priv/bin/etop.bat W: myapp: executable-not-elf-or-script ./opt/myapp/lib/webtool-0.8.7/priv/bin/start_webtool.bat E: myapp: embedded-zlib ./opt/myapp/erts-5.8.2/bin/beam E: myapp: embedded-library ./opt/myapp/erts-5.8.2/bin/beam: pcre3 E: myapp: embedded-zlib ./opt/myapp/erts-5.8.2/bin/beam.smp E: myapp: embedded-library ./opt/myapp/erts-5.8.2/bin/beam.smp: pcre3 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /usr/ssl/lib E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /usr/local/lib E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /usr/local/openssl/lib64 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /usr/lib/ssl/lib E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /usr/openssl/lib E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /usr/pkg/lib64 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /usr/local/ssl/lib64 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /usr/local/openssl/lib E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /usr/local/ssl/lib E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /usr/sfw/lib E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /usr/lib E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /opt/local/lib E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /usr/lib64 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /usr/lib/ssl/lib64 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /usr/lib/openssl/lib64 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /usr/local/lib64 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /usr/openssl/lib64 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /opt/local/lib64 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /usr/pkg/lib E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /usr/sfw/lib64 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /usr/ssl/lib64 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /usr/lib/openssl/lib E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock /usr/ssl/lib E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock /usr/local/lib E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock /usr/local/openssl/lib64 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock /usr/lib/ssl/lib E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock /usr/openssl/lib E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock /usr/pkg/lib64 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock /usr/local/ssl/lib64 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock /usr/local/openssl/lib E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock /usr/local/ssl/lib E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock /usr/sfw/lib E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock /usr/lib E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock /opt/local/lib E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock /usr/lib64 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock /usr/lib/ssl/lib64 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock /usr/lib/openssl/lib64 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock /usr/local/lib64 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock /usr/openssl/lib64 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock /opt/local/lib64 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock /usr/pkg/lib E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock /usr/sfw/lib64 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock /usr/ssl/lib64 E: myapp: binary-or-shlib-defines-rpath ./opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock /usr/lib/openssl/lib E: myapp: dir-or-file-in-opt opt/myapp/ E: myapp: dir-or-file-in-opt opt/myapp/bin/ E: myapp: dir-or-file-in-opt opt/myapp/bin/myapp E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/ E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/bin/ E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/bin/beam E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/bin/beam.smp E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/bin/child_setup E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/bin/ct_run E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/bin/epmd E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/bin/erl E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/bin/erlc E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/bin/erlexec E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/bin/escript E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/bin/heart E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/bin/inet_gethost E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/bin/nodetool E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/bin/run_erl E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/bin/start E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/bin/to_erl E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/doc/ E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/ E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/driver_int.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/erl_driver.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/erl_drv_nif.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/erl_fixed_size_int_types.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/erl_int_sizes_config.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/erl_memory_trace_parser.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/erl_nif.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/erl_nif_api_funcs.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/ E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/README E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/erl_errno.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/erl_memory_trace_protocol.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/erl_misc_utils.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/erl_printf.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/erl_printf_format.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/erts_internal.mk E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/ethr_mutex.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/ethr_optimized_fallbacks.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/ethread.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/ethread.mk E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/ethread_header_config.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/gcc/ E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/gcc/ethr_atomic.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/gcc/ethread.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/i386/ E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/i386/atomic.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/i386/ethread.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/i386/rwlock.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/i386/spinlock.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/libatomic_ops/ E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/libatomic_ops/ethr_atomic.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/libatomic_ops/ethread.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/ppc32/ E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/ppc32/atomic.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/ppc32/ethread.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/ppc32/rwlock.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/ppc32/spinlock.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/pthread/ E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/pthread/ethr_event.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/sparc32/ E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/sparc32/atomic.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/sparc32/ethread.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/sparc32/rwlock.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/sparc32/spinlock.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/sparc64/ E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/sparc64/ethread.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/tile/ E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/tile/atomic.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/tile/ethread.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/win/ E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/win/ethr_atomic.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/win/ethr_event.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/win/ethread.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/x86_64/ E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/include/internal/x86_64/ethread.h E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/lib/ E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/lib/internal/ E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/lib/internal/README E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/lib/internal/liberts_internal.a E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/lib/internal/liberts_internal_r.a E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/lib/internal/libethread.a E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/lib/liberts.a E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/lib/liberts_r.a E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/man/ E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/src/ E: myapp: dir-or-file-in-opt opt/myapp/erts-5.8.2/src/setuid_socket_wrap.c E: myapp: dir-or-file-in-opt opt/myapp/etc/ E: myapp: dir-or-file-in-opt opt/myapp/etc/app.config E: myapp: dir-or-file-in-opt opt/myapp/etc/vm.args E: myapp: dir-or-file-in-opt opt/myapp/lib/ E: myapp: dir-or-file-in-opt opt/myapp/lib/compiler-4.7.2.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/crypto-2.0.2.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/crypto-2.0.2/ E: myapp: dir-or-file-in-opt opt/myapp/lib/crypto-2.0.2/priv/ E: myapp: dir-or-file-in-opt opt/myapp/lib/crypto-2.0.2/priv/lib/ E: myapp: dir-or-file-in-opt opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so E: myapp: dir-or-file-in-opt opt/myapp/lib/crypto-2.0.2/priv/obj/ E: myapp: dir-or-file-in-opt opt/myapp/lib/crypto-2.0.2/priv/obj/Makefile E: myapp: dir-or-file-in-opt opt/myapp/lib/crypto-2.0.2/priv/obj/crypto.o E: myapp: dir-or-file-in-opt opt/myapp/lib/edoc-0.7.6.8.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/edoc-0.7.6.8/ E: myapp: dir-or-file-in-opt opt/myapp/lib/edoc-0.7.6.8/include/ E: myapp: dir-or-file-in-opt opt/myapp/lib/edoc-0.7.6.8/include/edoc_doclet.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/edoc-0.7.6.8/priv/ E: myapp: dir-or-file-in-opt opt/myapp/lib/edoc-0.7.6.8/priv/edoc.dtd E: myapp: dir-or-file-in-opt opt/myapp/lib/edoc-0.7.6.8/priv/edoc_generate E: myapp: dir-or-file-in-opt opt/myapp/lib/edoc-0.7.6.8/priv/erlang.png E: myapp: dir-or-file-in-opt opt/myapp/lib/edoc-0.7.6.8/priv/stylesheet.css E: myapp: dir-or-file-in-opt opt/myapp/lib/emongo-1.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/emongo-1/ E: myapp: dir-or-file-in-opt opt/myapp/lib/emongo-1/include/ E: myapp: dir-or-file-in-opt opt/myapp/lib/emongo-1/include/emongo.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/erlydtl-0.6.1.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/erlydtl-0.6.1/ E: myapp: dir-or-file-in-opt opt/myapp/lib/erlydtl-0.6.1/priv/ E: myapp: dir-or-file-in-opt opt/myapp/lib/erlydtl-0.6.1/priv/custom_tags/ E: myapp: dir-or-file-in-opt opt/myapp/lib/erlydtl-0.6.1/priv/custom_tags/flashvideo E: myapp: dir-or-file-in-opt opt/myapp/lib/erts-5.8.2.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/et-1.4.2.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/et-1.4.2/ E: myapp: dir-or-file-in-opt opt/myapp/lib/et-1.4.2/include/ E: myapp: dir-or-file-in-opt opt/myapp/lib/et-1.4.2/include/et.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/eunit-2.1.6.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/eunit-2.1.6/ E: myapp: dir-or-file-in-opt opt/myapp/lib/eunit-2.1.6/include/ E: myapp: dir-or-file-in-opt opt/myapp/lib/eunit-2.1.6/include/eunit.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/gproc-0.01.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/gproc-0.01/ E: myapp: dir-or-file-in-opt opt/myapp/lib/gproc-0.01/include/ E: myapp: dir-or-file-in-opt opt/myapp/lib/gproc-0.01/include/gproc.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/gproc-0.01/priv/ E: myapp: dir-or-file-in-opt opt/myapp/lib/gproc-0.01/priv/sys.config E: myapp: dir-or-file-in-opt opt/myapp/lib/gs-1.5.13.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/gs-1.5.13/ E: myapp: dir-or-file-in-opt opt/myapp/lib/gs-1.5.13/priv/ E: myapp: dir-or-file-in-opt opt/myapp/lib/gs-1.5.13/priv/bitmap/ E: myapp: dir-or-file-in-opt opt/myapp/lib/gs-1.5.13/priv/bitmap/fup.bm E: myapp: dir-or-file-in-opt opt/myapp/lib/gs-1.5.13/priv/gs-xdefaults E: myapp: dir-or-file-in-opt opt/myapp/lib/gs-1.5.13/priv/gstk.tcl E: myapp: dir-or-file-in-opt opt/myapp/lib/hipe-3.7.8.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/inets-5.5.1.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/inets-5.5.1/ E: myapp: dir-or-file-in-opt opt/myapp/lib/inets-5.5.1/priv/ E: myapp: dir-or-file-in-opt opt/myapp/lib/inets-5.5.1/priv/bin/ E: myapp: dir-or-file-in-opt opt/myapp/lib/inets-5.5.1/priv/bin/runcgi.sh E: myapp: dir-or-file-in-opt opt/myapp/lib/kernel-2.14.2.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/kernel-2.14.2/ E: myapp: dir-or-file-in-opt opt/myapp/lib/kernel-2.14.2/include/ E: myapp: dir-or-file-in-opt opt/myapp/lib/kernel-2.14.2/include/file.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/kernel-2.14.2/include/inet.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/kernel-2.14.2/include/inet_sctp.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/mnesia-4.4.16.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/mnesia-4.4.16/ E: myapp: dir-or-file-in-opt opt/myapp/lib/mnesia-4.4.16/include/ E: myapp: dir-or-file-in-opt opt/myapp/lib/mochiweb-1.5.0.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/observer-0.9.8.4.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/observer-0.9.8.4/ E: myapp: dir-or-file-in-opt opt/myapp/lib/observer-0.9.8.4/include/ E: myapp: dir-or-file-in-opt opt/myapp/lib/observer-0.9.8.4/include/etop.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/observer-0.9.8.4/priv/ E: myapp: dir-or-file-in-opt opt/myapp/lib/observer-0.9.8.4/priv/bin/ E: myapp: dir-or-file-in-opt opt/myapp/lib/observer-0.9.8.4/priv/bin/etop E: myapp: dir-or-file-in-opt opt/myapp/lib/observer-0.9.8.4/priv/bin/etop.bat E: myapp: dir-or-file-in-opt opt/myapp/lib/observer-0.9.8.4/priv/bin/getop E: myapp: dir-or-file-in-opt opt/myapp/lib/observer-0.9.8.4/priv/bin/getop.bat E: myapp: dir-or-file-in-opt opt/myapp/lib/observer-0.9.8.4/priv/crashdump_viewer.tool E: myapp: dir-or-file-in-opt opt/myapp/lib/observer-0.9.8.4/priv/crashdump_viewer/ E: myapp: dir-or-file-in-opt opt/myapp/lib/observer-0.9.8.4/priv/crashdump_viewer/collapsd.gif E: myapp: dir-or-file-in-opt opt/myapp/lib/observer-0.9.8.4/priv/crashdump_viewer/exploded.gif E: myapp: dir-or-file-in-opt opt/myapp/lib/public_key-0.10.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/public_key-0.10/ E: myapp: dir-or-file-in-opt opt/myapp/lib/public_key-0.10/include/ E: myapp: dir-or-file-in-opt opt/myapp/lib/public_key-0.10/include/OTP-PUB-KEY.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/public_key-0.10/include/public_key.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/runtime_tools-1.8.4.1.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/runtime_tools-1.8.4.1/ E: myapp: dir-or-file-in-opt opt/myapp/lib/runtime_tools-1.8.4.1/include/ E: myapp: dir-or-file-in-opt opt/myapp/lib/runtime_tools-1.8.4.1/include/observer_backend.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/runtime_tools-1.8.4.1/priv/ E: myapp: dir-or-file-in-opt opt/myapp/lib/runtime_tools-1.8.4.1/priv/lib/ E: myapp: dir-or-file-in-opt opt/myapp/lib/runtime_tools-1.8.4.1/priv/lib/trace_file_drv.so E: myapp: dir-or-file-in-opt opt/myapp/lib/runtime_tools-1.8.4.1/priv/lib/trace_ip_drv.so E: myapp: dir-or-file-in-opt opt/myapp/lib/sasl-2.1.9.2.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/myapp-1.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/ssl-4.1.1.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/ssl-4.1.1/ E: myapp: dir-or-file-in-opt opt/myapp/lib/ssl-4.1.1/priv/ E: myapp: dir-or-file-in-opt opt/myapp/lib/ssl-4.1.1/priv/bin/ E: myapp: dir-or-file-in-opt opt/myapp/lib/ssl-4.1.1/priv/bin/ssl_esock E: myapp: dir-or-file-in-opt opt/myapp/lib/ssl-4.1.1/priv/obj/ E: myapp: dir-or-file-in-opt opt/myapp/lib/ssl-4.1.1/priv/obj/Makefile E: myapp: dir-or-file-in-opt opt/myapp/lib/ssl-4.1.1/priv/obj/debuglog.o E: myapp: dir-or-file-in-opt opt/myapp/lib/ssl-4.1.1/priv/obj/esock.o E: myapp: dir-or-file-in-opt opt/myapp/lib/ssl-4.1.1/priv/obj/esock_openssl.o E: myapp: dir-or-file-in-opt opt/myapp/lib/ssl-4.1.1/priv/obj/esock_osio.o E: myapp: dir-or-file-in-opt opt/myapp/lib/ssl-4.1.1/priv/obj/esock_poll.o E: myapp: dir-or-file-in-opt opt/myapp/lib/ssl-4.1.1/priv/obj/esock_posix_str.o E: myapp: dir-or-file-in-opt opt/myapp/lib/ssl-4.1.1/priv/obj/esock_utils.o E: myapp: dir-or-file-in-opt opt/myapp/lib/stdlib-1.17.2.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/stdlib-1.17.2/ E: myapp: dir-or-file-in-opt opt/myapp/lib/stdlib-1.17.2/include/ E: myapp: dir-or-file-in-opt opt/myapp/lib/stdlib-1.17.2/include/erl_bits.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/stdlib-1.17.2/include/erl_compile.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/stdlib-1.17.2/include/ms_transform.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/stdlib-1.17.2/include/qlc.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/stdlib-1.17.2/include/zip.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/syntax_tools-1.6.7.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/tools-2.6.6.2.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/tools-2.6.6.2/ E: myapp: dir-or-file-in-opt opt/myapp/lib/tools-2.6.6.2/priv/ E: myapp: dir-or-file-in-opt opt/myapp/lib/tools-2.6.6.2/priv/cover.tool E: myapp: dir-or-file-in-opt opt/myapp/lib/tools-2.6.6.2/priv/index.html E: myapp: dir-or-file-in-opt opt/myapp/lib/webtool-0.8.7.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/webtool-0.8.7/ E: myapp: dir-or-file-in-opt opt/myapp/lib/webtool-0.8.7/priv/ E: myapp: dir-or-file-in-opt opt/myapp/lib/webtool-0.8.7/priv/bin/ E: myapp: dir-or-file-in-opt opt/myapp/lib/webtool-0.8.7/priv/bin/start_webtool E: myapp: dir-or-file-in-opt opt/myapp/lib/webtool-0.8.7/priv/bin/start_webtool.bat E: myapp: dir-or-file-in-opt opt/myapp/lib/webtool-0.8.7/priv/root/ E: myapp: dir-or-file-in-opt opt/myapp/lib/webtool-0.8.7/priv/root/conf/ E: myapp: dir-or-file-in-opt opt/myapp/lib/webtool-0.8.7/priv/root/conf/mime.types E: myapp: dir-or-file-in-opt opt/myapp/lib/webtool-0.8.7/priv/root/doc/ E: myapp: dir-or-file-in-opt opt/myapp/lib/webtool-0.8.7/priv/root/doc/index.html E: myapp: dir-or-file-in-opt opt/myapp/lib/webtool-0.8.7/priv/root/doc/start_info.html E: myapp: dir-or-file-in-opt opt/myapp/lib/webtool-0.8.7/priv/root/doc/tool_management.html E: myapp: dir-or-file-in-opt opt/myapp/lib/wx-0.98.8.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/wx-0.98.8/ E: myapp: dir-or-file-in-opt opt/myapp/lib/wx-0.98.8/include/ E: myapp: dir-or-file-in-opt opt/myapp/lib/wx-0.98.8/include/gl.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/wx-0.98.8/include/glu.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/wx-0.98.8/include/wx.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/xmerl-1.2.7.ez E: myapp: dir-or-file-in-opt opt/myapp/lib/xmerl-1.2.7/ E: myapp: dir-or-file-in-opt opt/myapp/lib/xmerl-1.2.7/include/ E: myapp: dir-or-file-in-opt opt/myapp/lib/xmerl-1.2.7/include/xmerl.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/xmerl-1.2.7/include/xmerl_xlink.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/xmerl-1.2.7/include/xmerl_xpath.hrl E: myapp: dir-or-file-in-opt opt/myapp/lib/xmerl-1.2.7/include/xmerl_xsd.hrl E: myapp: dir-or-file-in-opt opt/myapp/log/ E: myapp: dir-or-file-in-opt opt/myapp/log/erlang.log.1 E: myapp: dir-or-file-in-opt opt/myapp/log/run_erl.log E: myapp: dir-or-file-in-opt opt/myapp/log/sasl-error.log E: myapp: dir-or-file-in-opt opt/myapp/log/sasl/ E: myapp: dir-or-file-in-opt opt/myapp/log/sasl/1 E: myapp: dir-or-file-in-opt opt/myapp/log/sasl/index E: myapp: dir-or-file-in-opt opt/myapp/releases/ E: myapp: dir-or-file-in-opt opt/myapp/releases/1/ E: myapp: dir-or-file-in-opt opt/myapp/releases/1/myapp.boot E: myapp: dir-or-file-in-opt opt/myapp/releases/1/myapp.rel E: myapp: dir-or-file-in-opt opt/myapp/releases/1/myapp.script E: myapp: dir-or-file-in-opt opt/myapp/releases/1/start_clean.boot E: myapp: dir-or-file-in-opt opt/myapp/releases/1/start_clean.rel E: myapp: dir-or-file-in-opt opt/myapp/releases/1/start_clean.script E: myapp: dir-or-file-in-opt opt/myapp/releases/start_erl.data Finished running lintian. ________________________________ From: Andre Nathan To: Pablo Platt Cc: erlang-questions@REDACTED Sent: Mon, January 17, 2011 12:16:37 PM Subject: Re: [erlang-questions] deb package from erlang app release On Sun, 2011-01-16 at 17:19 -0800, Pablo Platt wrote: > Do I need to break the release and spread it on several places on the file > system or should I put everything under a specific folder? > In the latter, which folder? var/lib/myapp, usr/share/myapp ? When I created a deb package for my application I simply copied the release to /opt and then created symlinks from the appropriate locations to the release (e.g. /etc/init.d/foo -> /opt/foo/bin/foo). HTH, Andre From andre@REDACTED Mon Jan 17 11:16:37 2011 From: andre@REDACTED (Andre Nathan) Date: Mon, 17 Jan 2011 08:16:37 -0200 Subject: [erlang-questions] deb package from erlang app release In-Reply-To: <963495.11286.qm@web112616.mail.gq1.yahoo.com> References: <963495.11286.qm@web112616.mail.gq1.yahoo.com> Message-ID: <1295259397.17254.16.camel@andre.mz.digirati.com.br> On Sun, 2011-01-16 at 17:19 -0800, Pablo Platt wrote: > Do I need to break the release and spread it on several places on the file > system or should I put everything under a specific folder? > In the latter, which folder? var/lib/myapp, usr/share/myapp ? When I created a deb package for my application I simply copied the release to /opt and then created symlinks from the appropriate locations to the release (e.g. /etc/init.d/foo -> /opt/foo/bin/foo). HTH, Andre From andre@REDACTED Mon Jan 17 12:19:21 2011 From: andre@REDACTED (Andre Nathan) Date: Mon, 17 Jan 2011 09:19:21 -0200 Subject: [erlang-questions] deb package from erlang app release In-Reply-To: <499245.45314.qm@web112616.mail.gq1.yahoo.com> References: <963495.11286.qm@web112616.mail.gq1.yahoo.com> <1295259397.17254.16.camel@andre.mz.digirati.com.br> <499245.45314.qm@web112616.mail.gq1.yahoo.com> Message-ID: <1295263161.17254.88.camel@andre.mz.digirati.com.br> On Mon, 2011-01-17 at 02:50 -0800, Pablo Platt wrote: > When I'm copying the release to /opt (in the dh-make install script) I'm getting > many errors and warnings: Here's my debian/rules file, in case it helps: #!/usr/bin/make -f %: dh $@ BUILD_DIR = $(CURDIR)/debian/myapp override_dh_auto_build: rebar compile mkdir -p rel cp debian/reltool.config rel rebar generate force=1 override_dh_auto_install: mkdir -p $(BUILD_DIR)/etc/init.d mkdir -p $(BUILD_DIR)/opt mkdir -p $(BUILD_DIR)/var/lib mkdir -p $(BUILD_DIR)/var/run mkdir -p $(BUILD_DIR)/var/log mv rel/myapp $(BUILD_DIR)/opt install -D -m0644 debian/vm.args $(BUILD_DIR)/opt/myapp/etc/vm.args install -D -m0644 debian/app.config $(BUILD_DIR)/opt/myapp/etc/app.config install -D -m0755 debian/myapp.runner $(BUILD_DIR)/opt/myapp/bin/myapp From vladdu55@REDACTED Mon Jan 17 13:43:35 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Mon, 17 Jan 2011 13:43:35 +0100 Subject: App supporting several Erlang releases Message-ID: Hi! This may be a dumb question, as I seem to have gotten caught in a "empty mind" state... Alternatively, the answer will come to me 5 seconds after pressing "send"... I have some code that I'd like to be able to run in R12 or later, depending on what the user has installed. The problem is that some OTP APIs have changed in R13 and R14, making the task not straightforward. Unfortunately it's not only code that is affected (that would be possible to fix by checking the actual version), but also hrl files have moved or been created in the meanwhile. The most acute example is about eunit, where for example in R13 eunit.hrl moved from eunit/src to eunit/include and the eunit_listener behaviour was added. Is there any way to address this in a more reasonable way than duplicating the affected module and fiddling with the code path at load time? best regards, Vlad From serge@REDACTED Mon Jan 17 14:26:38 2011 From: serge@REDACTED (Serge Aleynikov) Date: Mon, 17 Jan 2011 08:26:38 -0500 Subject: [erlang-questions] [patch] new float_to_list/2 In-Reply-To: <4D340B68.8020402@erlang.org> References: <4D2D3FA8.1000009@aleynikov.org> <4D2DB65C.1090106@aleynikov.org> <4D2DE13E.7040404@aleynikov.org> <0CF9E046-9111-4825-A883-F3CE6A95071B@dev-extend.eu> <4D340B68.8020402@erlang.org> Message-ID: <4D34438E.5010103@aleynikov.org> Thanks Niclas, Since this did create some confusion, what's the currently preferred way of submitting patches? On 1/17/2011 4:27 AM, Niclas Axelsson wrote: > On 01/17/2011 09:42 AM, nox wrote: >> Since when does OTP use Github's pull requests? > We don't :-). > > > Thanks for your patch, Serge. > I've looked at your otp-repository, and merged your > 'float_to_list_2'-branch > into 'pu'. It would be good if you could close the pull request since > it's already merged into 'pu'. > > > > Thanks, > Niclas Axelsson, Erlang/OTP From burbas@REDACTED Mon Jan 17 14:34:57 2011 From: burbas@REDACTED (Niclas Axelsson) Date: Mon, 17 Jan 2011 14:34:57 +0100 Subject: [erlang-questions] [patch] new float_to_list/2 In-Reply-To: <4D34438E.5010103@aleynikov.org> References: <4D2D3FA8.1000009@aleynikov.org> <4D2DB65C.1090106@aleynikov.org> <4D2DE13E.7040404@aleynikov.org> <0CF9E046-9111-4825-A883-F3CE6A95071B@dev-extend.eu> <4D340B68.8020402@erlang.org> <4D34438E.5010103@aleynikov.org> Message-ID: <4D344581.2090201@erlang.org> On 01/17/2011 02:26 PM, Serge Aleynikov wrote: > Thanks Niclas, > > Since this did create some confusion, what's the currently preferred > way of submitting patches? Hi again, Serge. We have a wiki-page on github that explains this: https://github.com/erlang/otp/wiki/Submitting-patches Regards, Niclas Axelsson, Erlang/OTP > > On 1/17/2011 4:27 AM, Niclas Axelsson wrote: >> On 01/17/2011 09:42 AM, nox wrote: >>> Since when does OTP use Github's pull requests? >> We don't :-). >> >> >> Thanks for your patch, Serge. >> I've looked at your otp-repository, and merged your >> 'float_to_list_2'-branch >> into 'pu'. It would be good if you could close the pull request since >> it's already merged into 'pu'. >> >> >> >> Thanks, >> Niclas Axelsson, Erlang/OTP > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From serge@REDACTED Mon Jan 17 14:43:08 2011 From: serge@REDACTED (Serge Aleynikov) Date: Mon, 17 Jan 2011 08:43:08 -0500 Subject: [erlang-questions] [patch] new float_to_list/2 In-Reply-To: <4D344581.2090201@erlang.org> References: <4D2D3FA8.1000009@aleynikov.org> <4D2DB65C.1090106@aleynikov.org> <4D2DE13E.7040404@aleynikov.org> <0CF9E046-9111-4825-A883-F3CE6A95071B@dev-extend.eu> <4D340B68.8020402@erlang.org> <4D34438E.5010103@aleynikov.org> <4D344581.2090201@erlang.org> Message-ID: <4D34476C.7030807@aleynikov.org> So Gleb did provide the right reference, but instead of GitHub pull requests you prefer emails with references to GitHub fetch instructions. Got it, thanks! On 1/17/2011 8:34 AM, Niclas Axelsson wrote: > On 01/17/2011 02:26 PM, Serge Aleynikov wrote: >> Thanks Niclas, >> >> Since this did create some confusion, what's the currently preferred >> way of submitting patches? > Hi again, Serge. > > We have a wiki-page on github that explains this: > https://github.com/erlang/otp/wiki/Submitting-patches > > Regards, > Niclas Axelsson, Erlang/OTP > >> >> On 1/17/2011 4:27 AM, Niclas Axelsson wrote: >>> On 01/17/2011 09:42 AM, nox wrote: >>>> Since when does OTP use Github's pull requests? >>> We don't :-). >>> >>> >>> Thanks for your patch, Serge. >>> I've looked at your otp-repository, and merged your >>> 'float_to_list_2'-branch >>> into 'pu'. It would be good if you could close the pull request since >>> it's already merged into 'pu'. >>> >>> >>> >>> Thanks, >>> Niclas Axelsson, Erlang/OTP >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From kuleshovmail@REDACTED Mon Jan 17 18:37:52 2011 From: kuleshovmail@REDACTED (shk) Date: Mon, 17 Jan 2011 09:37:52 -0800 (PST) Subject: Erlang connect by external ip address Message-ID: <1295285872458-3221240.post@n4.nabble.com> Hello, I write simple server application in erlang. Code of this application: https://gist.github.com/783117 I try to connect to it with telnet. When I connect to localhost or local ip addres which get from route it's ok, but when i try to connect to server with external ip it's don't work. What's wrong? Thank you. -- View this message in context: http://erlang.2086793.n4.nabble.com/Erlang-connect-by-external-ip-address-tp3221240p3221240.html Sent from the Erlang Questions mailing list archive at Nabble.com. From rtrlists@REDACTED Mon Jan 17 18:57:01 2011 From: rtrlists@REDACTED (Robert Raschke) Date: Mon, 17 Jan 2011 17:57:01 +0000 Subject: [erlang-questions] Erlang connect by external ip address In-Reply-To: <1295285872458-3221240.post@n4.nabble.com> References: <1295285872458-3221240.post@n4.nabble.com> Message-ID: On Mon, Jan 17, 2011 at 5:37 PM, shk wrote: > > I write simple server application in erlang. > > Code of this application: https://gist.github.com/783117 > > I try to connect to it with telnet. When I connect to localhost or local ip > addres which get from route it's ok, but when i try to connect to server > with external ip it's don't work. > > What's wrong? > > Not 100% sure, but I think inet or inet6 should be part of your gen_tcp:listen/2 option list. Robby From ARoe@REDACTED Mon Jan 17 19:19:27 2011 From: ARoe@REDACTED (Adrian Roe) Date: Mon, 17 Jan 2011 10:19:27 -0800 Subject: [erlang-questions] Erlang connect by external ip address In-Reply-To: References: <1295285872458-3221240.post@n4.nabble.com> Message-ID: <0697C1D63C68ED4DA4EE16BFBB156ACD0A40318F54@IE2RD2XVS071.red002.local> Again a little while since I?ve been here, but I suspect you need to explicitly listen on all IP addresses adding {ip, ?0.0.0.0?} to your options. That and make certain that you don?t have a firewall / routing issue blocking traffic to your public IP on your port. Adrian From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Robert Raschke Sent: 17 January 2011 17:57 To: shk Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] Erlang connect by external ip address On Mon, Jan 17, 2011 at 5:37 PM, shk > wrote: > > I write simple server application in erlang. > > Code of this application: https://gist.github.com/783117 > > I try to connect to it with telnet. When I connect to localhost or local ip > addres which get from route it's ok, but when i try to connect to server > with external ip it's don't work. > > What's wrong? > > > Not 100% sure, but I think inet or inet6 should be part of your gen_tcp:listen/2 option list. Robby ________________________________ No virus found in this message. Checked by AVG - www.avg.com Version: 10.0.1191 / Virus Database: 1435/3386 - Release Date: 01/17/11 From freza@REDACTED Mon Jan 17 20:22:21 2011 From: freza@REDACTED (Jachym Holecek) Date: Mon, 17 Jan 2011 19:22:21 +0000 Subject: [erlang-questions] App supporting several Erlang releases In-Reply-To: References: Message-ID: <20110117192221.GA2165@hanele> # Vlad Dumitrescu 2011-01-17: > I have some code that I'd like to be able to run in R12 or later, depending > on what the user has installed. The problem is that some OTP APIs have > changed in R13 and R14, making the task not straightforward. Unfortunately > it's not only code that is affected (that would be possible to fix by > checking the actual version), but also hrl files have moved or been created > in the meanwhile. The most acute example is about eunit, where for example > in R13 eunit.hrl moved from eunit/src to eunit/include and the > eunit_listener behaviour was added. > > Is there any way to address this in a more reasonable way than duplicating > the affected module and fiddling with the code path at load time? IIRC what a colleague did some time ago to solve similar problem was to extract OTP release from ${prefix}/lib/erlang/releases/RELEASES (or was it ERTS version? -- they're both there anyway) and use that to define a macro from erlc commandline, then add ifdefs to affected files that would include headers from the right location. I'm not sure this was actually used in the end, but it worked. So -- depends on your build system and the amount of dark trickery you're willing to tolerate in your codebase :-). Regards, -- Jachym From mazen.harake@REDACTED Tue Jan 18 09:23:22 2011 From: mazen.harake@REDACTED (Mazen Harake) Date: Tue, 18 Jan 2011 09:23:22 +0100 Subject: [erlang-questions] App supporting several Erlang releases In-Reply-To: <20110117192221.GA2165@hanele> References: <20110117192221.GA2165@hanele> Message-ID: <4D354DFA.6030507@erlang-solutions.com> IMHO. Trying to "cover" all versions sounds like the kind of nightmare I could wake up to in the middle of the night. Some call it "maintenance nightmare", I just call it "waste of time". If you update your software in anyway you will have to put in these quirks in every piece of code that isn't always backwards compatible. It feels a bit like handling python2 or python3 in your code I.e: wrong. My 2 cents? Just split the versions, one pre R13 and one R13+... or discard one (say only works with R13+ or whatever) /Mazen On 17/01/2011 20:22, Jachym Holecek wrote: > # Vlad Dumitrescu 2011-01-17: >> I have some code that I'd like to be able to run in R12 or later, depending >> on what the user has installed. The problem is that some OTP APIs have >> changed in R13 and R14, making the task not straightforward. Unfortunately >> it's not only code that is affected (that would be possible to fix by >> checking the actual version), but also hrl files have moved or been created >> in the meanwhile. The most acute example is about eunit, where for example >> in R13 eunit.hrl moved from eunit/src to eunit/include and the >> eunit_listener behaviour was added. >> >> Is there any way to address this in a more reasonable way than duplicating >> the affected module and fiddling with the code path at load time? > IIRC what a colleague did some time ago to solve similar problem was > to extract OTP release from ${prefix}/lib/erlang/releases/RELEASES > (or was it ERTS version? -- they're both there anyway) and use that > to define a macro from erlc commandline, then add ifdefs to affected > files that would include headers from the right location. I'm not > sure this was actually used in the end, but it worked. > > So -- depends on your build system and the amount of dark trickery > you're willing to tolerate in your codebase :-). > > Regards, > -- Jachym > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From vladdu55@REDACTED Tue Jan 18 08:59:57 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 18 Jan 2011 08:59:57 +0100 Subject: [erlang-questions] App supporting several Erlang releases In-Reply-To: References: <20110117192221.GA2165@hanele> Message-ID: Hi, On Tue, Jan 18, 2011 at 01:48, Joseph Wayne Norton < norton@REDACTED> wrote: > > A feature was recently added to rebar to support such trickery. > > For > example, here is one way to handle the differences between old and new > inets: > > %% Erlang > compiler options > {erl_opts, [debug_info, fail_on_warning > , {i, "../gmt_util/include/"} > , {i, "../partition_detector/include/"} > , {i, "../gdss_brick/include/"} > , {platform_define, "R13", 'old_inets'} > ]}. > > > -ifdef(old_inets). > -include_lib("inets/src/httpd.hrl"). > -else. > -include_lib("inets/src/http_server/httpd.hrl"). > Thanks for the suggestion! Using such defines per each changed API (instead of for the whole version) is definitely a better idea. regards, Vlad From vladdu55@REDACTED Tue Jan 18 09:05:17 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Tue, 18 Jan 2011 09:05:17 +0100 Subject: [erlang-questions] App supporting several Erlang releases In-Reply-To: <4D354DFA.6030507@erlang-solutions.com> References: <20110117192221.GA2165@hanele> <4D354DFA.6030507@erlang-solutions.com> Message-ID: Hi, On Tue, Jan 18, 2011 at 09:23, Mazen Harake < mazen.harake@REDACTED> wrote: > Trying to "cover" all versions sounds like the kind of nightmare I could > wake up to in the middle of the night. Some call it "maintenance nightmare", > I just call it "waste of time". > > If you update your software in anyway you will have to put in these quirks > in every piece of code that isn't always backwards compatible. > Yes, I know. The thing that would still make it work is that we might be able to just switch features on and off depending of the version, instead of having to provide alternative implementations. > It feels a bit like handling python2 or python3 in your code I.e: wrong. > My 2 cents? Just split the versions, one pre R13 and one R13+... or discard > one (say only works with R13+ or whatever) > Splitting would work too, but duplicating code isn't really a very nice approach either :-) Thanks! best regards, Vlad From mikma264@REDACTED Mon Jan 17 22:08:36 2011 From: mikma264@REDACTED (Mikael Magnusson) Date: Mon, 17 Jan 2011 22:08:36 +0100 Subject: [erlang-questions] deb package from erlang app release In-Reply-To: <499245.45314.qm@web112616.mail.gq1.yahoo.com> References: <963495.11286.qm@web112616.mail.gq1.yahoo.com> <1295259397.17254.16.camel@andre.mz.digirati.com.br> <499245.45314.qm@web112616.mail.gq1.yahoo.com> Message-ID: <4D34AFD4.4020901@gmail.com> On 01/17/2011 11:50 AM, Pablo Platt wrote: >> On Sun, 2011-01-16 at 17:19 -0800, Pablo Platt wrote: >>> Do I need to break the release and spread it on several places on the file >>> system or should I put everything under a specific folder? >>> In the latter, which folder? var/lib/myapp, usr/share/myapp ? I use /usr/lib/myapp for static beam files etc, and /var/lib/myapp for variable state information like mnesia databases etc. >> When I created a deb package for my application I simply copied the >> release to /opt and then created symlinks from the appropriate locations >> to the release (e.g. /etc/init.d/foo -> /opt/foo/bin/foo). >> >> HTH, >> Andre > When I'm copying the release to /opt (in the dh-make install script) I'm getting > many errors and warnings: Run lintian manually and use "--info" as command line parameter. Then each error will be explained. > W: myapp: executable-not-elf-or-script > ./opt/myapp/lib/observer-0.9.8.4/priv/bin/getop.bat I suppose it's for Microsoft's operating systems. There is no need to install them I guess. > E: myapp: embedded-zlib ./opt/myapp/erts-5.8.2/bin/beam > E: myapp: embedded-library ./opt/myapp/erts-5.8.2/bin/beam: pcre3 If you need to install your own copy of beam it needs to be linked against the system libraries, otherwise you will face problems if for example security vulnerabilities are found in the embedded libraries. > E: myapp: binary-or-shlib-defines-rpath > ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /usr/ssl/lib Refer to "lintian --info"E: myapp: dir-or-file-in-opt opt/myapp/ > E: myapp: dir-or-file-in-opt opt/myapp/bin/ Consider using /usr/lib/myapp instead. /Mikael From tony@REDACTED Tue Jan 18 11:21:11 2011 From: tony@REDACTED (Tony Rogvall) Date: Tue, 18 Jan 2011 11:21:11 +0100 Subject: [erlang-questions] Erlang connect by external ip address In-Reply-To: <0697C1D63C68ED4DA4EE16BFBB156ACD0A40318F54@IE2RD2XVS071.red002.local> References: <1295285872458-3221240.post@n4.nabble.com> <0697C1D63C68ED4DA4EE16BFBB156ACD0A40318F54@IE2RD2XVS071.red002.local> Message-ID: On 17 jan 2011, at 19.19, Adrian Roe wrote: > Again a little while since I?ve been here, but I suspect you need to explicitly listen on all IP addresses adding {ip, ?0.0.0.0?} to your options. That and make certain that you don?t have a firewall / routing issue blocking traffic to your public IP on your port. > No. This is default. It is not obvious from the documentation. But you can check this by: 24> {ok,L} = gen_tcp:listen(37891, []). {ok,#Port<0.817>} 25> inet:i(). Port Module Recv Sent Owner Local Address Foreign Address State 817 inet_tcp 0 0 <0.61.0> *:37891 *:* LISTEN Port Module Recv Sent Owner Local Address Foreign Address State Local address * means that listens on 'any' address. Compare with 28> {ok,L} = gen_tcp:listen(37891, [{ip,{127,0,0,1}}]). {ok,#Port<0.819>} 29> inet:i(). Port Module Recv Sent Owner Local Address Foreign Address State 819 inet_tcp 0 0 <0.67.0> localhost:37891 *:* LISTEN Port Module Recv Sent Owner Local Address Foreign Address State Here is the address bound to localhost (12.0.0.1). /Tony > Adrian > > From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Robert Raschke > Sent: 17 January 2011 17:57 > To: shk > Cc: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Erlang connect by external ip address > > > On Mon, Jan 17, 2011 at 5:37 PM, shk > wrote: > >> >> I write simple server application in erlang. >> >> Code of this application: https://gist.github.com/783117 >> >> I try to connect to it with telnet. When I connect to localhost or local ip >> addres which get from route it's ok, but when i try to connect to server >> with external ip it's don't work. >> >> What's wrong? > >> >> > Not 100% sure, but I think inet or inet6 should be part of your > gen_tcp:listen/2 option list. > > Robby > > ________________________________ > > No virus found in this message. > Checked by AVG - www.avg.com > Version: 10.0.1191 / Virus Database: 1435/3386 - Release Date: 01/17/11 "Have run Make so many times I dunno what's installed anymore" From pablo.platt@REDACTED Tue Jan 18 13:30:13 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Tue, 18 Jan 2011 04:30:13 -0800 (PST) Subject: [erlang-questions] deb package from erlang app release In-Reply-To: <4D34AFD4.4020901@gmail.com> References: <963495.11286.qm@web112616.mail.gq1.yahoo.com> <1295259397.17254.16.camel@andre.mz.digirati.com.br> <499245.45314.qm@web112616.mail.gq1.yahoo.com> <4D34AFD4.4020901@gmail.com> Message-ID: <250413.65799.qm@web112605.mail.gq1.yahoo.com> I've looked at the errors and I think it's because of putting files in the wrong place, like you said. I thought of two ways to package an erlang release as deb: 1. Break the release to several folders as recommended by debian. Practically copy the official erlang package structure. 2. Make my deb depend on the erlang deb. In my deb copy only the required erlang apps to /usr/lib/erlang/lib Copy the app boot file. In both cases the folders in app start script should be updated. I'm not sure if 1. will work and I can naively break the release to several folders. Anyone has a sane way to create a deb that follows the debian policy? ________________________________ From: Mikael Magnusson To: erlang-questions@REDACTED Sent: Mon, January 17, 2011 11:08:36 PM Subject: Re: [erlang-questions] deb package from erlang app release On 01/17/2011 11:50 AM, Pablo Platt wrote: >> On Sun, 2011-01-16 at 17:19 -0800, Pablo Platt wrote: >>> Do I need to break the release and spread it on several places on the file >>> system or should I put everything under a specific folder? >>> In the latter, which folder? var/lib/myapp, usr/share/myapp ? I use /usr/lib/myapp for static beam files etc, and /var/lib/myapp for variable state information like mnesia databases etc. >> When I created a deb package for my application I simply copied the >> release to /opt and then created symlinks from the appropriate locations >> to the release (e.g. /etc/init.d/foo -> /opt/foo/bin/foo). >> >> HTH, >> Andre > When I'm copying the release to /opt (in the dh-make install script) I'm >getting > many errors and warnings: Run lintian manually and use "--info" as command line parameter. Then each error will be explained. > W: myapp: executable-not-elf-or-script > ./opt/myapp/lib/observer-0.9.8.4/priv/bin/getop.bat I suppose it's for Microsoft's operating systems. There is no need to install them I guess. > E: myapp: embedded-zlib ./opt/myapp/erts-5.8.2/bin/beam > E: myapp: embedded-library ./opt/myapp/erts-5.8.2/bin/beam: pcre3 If you need to install your own copy of beam it needs to be linked against the system libraries, otherwise you will face problems if for example security vulnerabilities are found in the embedded libraries. > E: myapp: binary-or-shlib-defines-rpath > ./opt/myapp/lib/crypto-2.0.2/priv/lib/crypto.so /usr/ssl/lib Refer to "lintian --info"E: myapp: dir-or-file-in-opt opt/myapp/ > E: myapp: dir-or-file-in-opt opt/myapp/bin/ Consider using /usr/lib/myapp instead. /Mikael ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From kostis@REDACTED Tue Jan 18 13:44:33 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Tue, 18 Jan 2011 14:44:33 +0200 Subject: [erlang-bugs] Re: [erlang-questions] picky dialyzer In-Reply-To: References: <4D2EDD7F.5040409@cs.ntua.gr> <8548EADD-542F-42E9-9ED3-FC1FE5523BFF@erlang-solutions.com> <4D2EF987.2010706@cs.ntua.gr> <4D306A4A.5050005@cs.ntua.gr> Message-ID: <4D358B31.5010900@cs.ntua.gr> Bj?rn Gustavsson wrote: > 2011/1/14 Kostis Sagonas : > > The crash is in hipe_unified_loader. It crashes the code_server > process before it has found out that there is no native code > to load. > >> I can of course investigate and possibly correct this throughout but I am >> also wondering: is this 0..256 a sane choice? Why on earth one would have >> such a range? Wasn't Erlang exotic enough without this feature? :-) > > The 0..256 range was an arbitrary choice decided long before > there was a Dialyzer. > > Actually, I am willing to change the range to 0..255 in the next > release in order to keep the Dialyzer type simple and beautiful. > I don't think that there is any code depending on a function > having 256 arguments, so that change should be safe. Thanks Bjorn! I also think this is safe and in any case it's a change for the better. Actually, it's not dialyzer which is what's difficult to change here, but more finding out all places in the hipe compiler and loader where there might be implicit assumptions that the function's arity can always be stored in a byte. Kostis From mikpe@REDACTED Tue Jan 18 14:42:48 2011 From: mikpe@REDACTED (Mikael Pettersson) Date: Tue, 18 Jan 2011 14:42:48 +0100 Subject: [erlang-bugs] Re: [erlang-questions] picky dialyzer In-Reply-To: <4D358B31.5010900@cs.ntua.gr> References: <4D2EDD7F.5040409@cs.ntua.gr> <8548EADD-542F-42E9-9ED3-FC1FE5523BFF@erlang-solutions.com> <4D2EF987.2010706@cs.ntua.gr> <4D306A4A.5050005@cs.ntua.gr> <4D358B31.5010900@cs.ntua.gr> Message-ID: <19765.39128.437716.501328@pilspetsen.it.uu.se> Kostis Sagonas writes: > Bj??rn Gustavsson wrote: > > 2011/1/14 Kostis Sagonas : > > > > The crash is in hipe_unified_loader. It crashes the code_server > > process before it has found out that there is no native code > > to load. > > > >> I can of course investigate and possibly correct this throughout but I am > >> also wondering: is this 0..256 a sane choice? Why on earth one would have > >> such a range? Wasn't Erlang exotic enough without this feature? :-) > > > > The 0..256 range was an arbitrary choice decided long before > > there was a Dialyzer. > > > > Actually, I am willing to change the range to 0..255 in the next > > release in order to keep the Dialyzer type simple and beautiful. > > I don't think that there is any code depending on a function > > having 256 arguments, so that change should be safe. > > Thanks Bjorn! > > I also think this is safe and in any case it's a change for the better. > > Actually, it's not dialyzer which is what's difficult to change here, > but more finding out all places in the hipe compiler and loader where > there might be implicit assumptions that the function's arity can always > be stored in a byte. I distinctly remember that Bj?rn once said that the max arity was 255, though he may have said 256. I always interpreted it as 255, implying that a byte suffices, since 256 seemed like a totally arbitrary limit. The only things I _know_ break if the limit is 256 are: - hipe_{x86,amd64}.c emit "movb $Arity, ..." insns in the native code wrappers for BEAM functions, and hipe_arm.c similarly is limited to an 8-bit immediate for the arity - there are a few type checks on MFAs in the HiPE runtime system that check that the arity doesn't exceed 255 The first breakage is real and I'd hate to have to emit 32-bit immediates there. The other breakage is superficial and can easily be repaired if necessary. If Bj?rn's willing to lower the max arity from 256 to 255 then I'm all for that solution. /Mikael From dsc@REDACTED Tue Jan 18 14:54:50 2011 From: dsc@REDACTED (Dave Challis) Date: Tue, 18 Jan 2011 13:54:50 +0000 Subject: Any tools available to generate code for Erlang Ports? References: <4D359BAA.3000606@ecs.soton.ac.uk> Message-ID: I'm just curious - are there any tools or code generators out there for creating Erlang Ports to interface with C code? There are a bunch of old C libraries I'd like to be able to use from within Erlang, and it seems pretty time consuming to code interfaces to them from scratch. The only examples of this I've seen have been pretty basic though - are there any larger examples or tutorials on this out there? I'll admit that my C is a bit rusty, so the less time I have to spend on that side, the better! Thanks, -- Dave Challis dsc@REDACTED From alexey.v.romanov@REDACTED Tue Jan 18 15:10:52 2011 From: alexey.v.romanov@REDACTED (Alexey Romanov) Date: Tue, 18 Jan 2011 17:10:52 +0300 Subject: Passing arguments to valgrind via cerl -valgrind Message-ID: Is it possible to pass additional arguments to valgrind (e.g. --leak-check=full) when running a debug-enabled runtime system (http://www.erlang.org/doc/installation_guide/INSTALL.html#id68569)? Yours, Alexey Romanov From b.ghose@REDACTED Tue Jan 18 15:30:21 2011 From: b.ghose@REDACTED (Baishampayan Ghose) Date: Tue, 18 Jan 2011 20:00:21 +0530 Subject: Erlang project dependency management/build tools Message-ID: Hello, I am quite new to the Erlang world and I am trying to develop an OTP application (with releases in the future). I am looking for a tool that will let me manage the dependencies, build process and release engineering. I have found Rebar & Sinan (+ Faxien) so far but it seems Sinan doesn't do dependency management(?) and Rebar obviously lacks documentation. I would like to know the opinion of the Erlang community on this matter; I am looking for a simple tool that will fetch external dependencies (or otherwise let me put them in a directory), build/test/generate docs for the project and help with the OTP release creation process. Thanks in advance. Regards, BG -- Baishampayan Ghose b.ghose at gmail.com From erlang@REDACTED Tue Jan 18 15:52:15 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 18 Jan 2011 15:52:15 +0100 Subject: why is the error reason in a catch/try truncated? Message-ID: I have a program. When I run the "uncaught" version it prints a very nice error message like this: > compile_file(File, Args, ParseTree). {"init terminating in do_boot",{badarg,[{lists,member,[free_index,all_keys]},{ecc_db,add_to_list,2},{ecc_db,store,2},{ecc_pass1,start,2},{ecc_compile,compile_file0,3},{ecc_compile,compile_file,3},{ecc,batch,1},{init,start_it,1}]}} This tells be exactly were I bombed. In my "production" version of the code I add wrapper with a catch/try to trap any unwanted errors. So I write: compile_file(File, Args, ParseTree) -> case (catch compile_file0(File, Args, ParseTree)) of {'EXIT', Why} -> io:format("Compile file:~s aborted with:~p ~n",[File, Why]); _ -> ok end But when i run this > compile_file(File, Args, ParseTree). Compile file:first.c aborted with:error badarg So the Why in the error tuple has lost all the useful information that was in the untrapped code. To find the error I have to comment out the wrapped version and run the unwrapped version. This is because Why is just the atom badarg and not the more verbose {badarg, ...} I find this behavior very unhelpfull - am I the only one? /Joe From erlang@REDACTED Tue Jan 18 15:52:15 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 18 Jan 2011 15:52:15 +0100 Subject: why is the error reason in a catch/try truncated? Message-ID: I have a program. When I run the "uncaught" version it prints a very nice error message like this: > compile_file(File, Args, ParseTree). {"init terminating in do_boot",{badarg,[{lists,member,[free_index,all_keys]},{ecc_db,add_to_list,2},{ecc_db,store,2},{ecc_pass1,start,2},{ecc_compile,compile_file0,3},{ecc_compile,compile_file,3},{ecc,batch,1},{init,start_it,1}]}} This tells be exactly were I bombed. In my "production" version of the code I add wrapper with a catch/try to trap any unwanted errors. So I write: compile_file(File, Args, ParseTree) -> case (catch compile_file0(File, Args, ParseTree)) of {'EXIT', Why} -> io:format("Compile file:~s aborted with:~p ~n",[File, Why]); _ -> ok end But when i run this > compile_file(File, Args, ParseTree). Compile file:first.c aborted with:error badarg So the Why in the error tuple has lost all the useful information that was in the untrapped code. To find the error I have to comment out the wrapped version and run the unwrapped version. This is because Why is just the atom badarg and not the more verbose {badarg, ...} I find this behavior very unhelpfull - am I the only one? /Joe From steve@REDACTED Tue Jan 18 15:58:03 2011 From: steve@REDACTED (Steve Strong) Date: Tue, 18 Jan 2011 15:58:03 +0100 Subject: why is the error reason in a catch/try truncated? In-Reply-To: References: Message-ID: <9F056F05AE384A879D4795C2A9304BBE@srstrong.com> Nope, you're not the only one. Loosing exception information like that can be very frustrating. -- Steve Strong twitter.com/srstrong On Tuesday, January 18, 2011 at 3:52 PM, Joe Armstrong wrote: > I have a program. > > When I run the "uncaught" version it prints a very nice error message like > this: > > > > compile_file(File, Args, ParseTree). > > > > > > {"init terminating in > do_boot",{badarg,[{lists,member,[free_index,all_keys]},{ecc_db,add_to_list,2},{ecc_db,store,2},{ecc_pass1,start,2},{ecc_compile,compile_file0,3},{ecc_compile,compile_file,3},{ecc,batch,1},{init,start_it,1}]}} > > This tells be exactly were I bombed. > > In my "production" version of the code I add wrapper with a catch/try to > trap any unwanted errors. > So I write: > > compile_file(File, Args, ParseTree) -> > case (catch compile_file0(File, Args, ParseTree)) of > {'EXIT', Why} -> > io:format("Compile file:~s aborted with:~p ~n",[File, Why]); > _ -> > ok > end > > But when i run this > > > > compile_file(File, Args, ParseTree). > > > > > Compile file:first.c aborted with:error badarg > > So the Why in the error tuple has lost all the useful information that was > in > the untrapped code. > > To find the error I have to comment out the wrapped version and > run the unwrapped version. This is because Why is just the atom > badarg and not the more verbose {badarg, ...} > > I find this behavior very unhelpfull - am I the only one? > > > /Joe > > > From fernando.benavides@REDACTED Tue Jan 18 16:05:58 2011 From: fernando.benavides@REDACTED (Fernando Benavides) Date: Tue, 18 Jan 2011 12:05:58 -0300 Subject: [erlang-questions] Re: why is the error reason in a catch/try truncated? In-Reply-To: <9F056F05AE384A879D4795C2A9304BBE@srstrong.com> References: <9F056F05AE384A879D4795C2A9304BBE@srstrong.com> Message-ID: <1295363158.2863.2.camel@army.local> Did you try using erlang:get_stacktrace() Something along the lines of... compile_file(File, Args, ParseTree) -> case (catch compile_file0(File, Args, ParseTree)) of {'EXIT', Why} -> io:format("Compile file:~s aborted with:~p ~n \tStack: ~p~n",[File, Why, erlang:get_stacktrace()]); _ -> ok end ...may do the trick On Tue, 2011-01-18 at 15:58 +0100, Steve Strong wrote: > Nope, you're not the only one. Loosing exception information like that can be very frustrating. > From raimo+erlang-questions@REDACTED Tue Jan 18 16:46:21 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 18 Jan 2011 16:46:21 +0100 Subject: [erlang-questions] why is the error reason in a catch/try truncated? In-Reply-To: References: Message-ID: <20110118154621.GA19093@erix.ericsson.se> On Tue, Jan 18, 2011 at 03:52:15PM +0100, Joe Armstrong wrote: > I have a program. > > When I run the "uncaught" version it prints a very nice error message like > this: > > > compile_file(File, Args, ParseTree). > > {"init terminating in > do_boot",{badarg,[{lists,member,[free_index,all_keys]},{ecc_db,add_to_list,2},{ecc_db,store,2},{ecc_pass1,start,2},{ecc_compile,compile_file0,3},{ecc_compile,compile_file,3},{ecc,batch,1},{init,start_it,1}]}} > > This tells be exactly were I bombed. > > In my "production" version of the code I add wrapper with a catch/try to > trap any unwanted errors. > So I write: > > compile_file(File, Args, ParseTree) -> > case (catch compile_file0(File, Args, ParseTree)) of > {'EXIT', Why} -> > io:format("Compile file:~s aborted with:~p ~n",[File, Why]); > _ -> > ok > end > > But when i run this > > > compile_file(File, Args, ParseTree). > Compile file:first.c aborted with:error badarg > > So the Why in the error tuple has lost all the useful information that was > in > the untrapped code. > > To find the error I have to comment out the wrapped version and > run the unwrapped version. This is because Why is just the atom > badarg and not the more verbose {badarg, ...} > > I find this behavior very unhelpfull - am I the only one? You must be doing something wrong. $ cat test.erl -module(test). -export([foo/2]). foo(T, L) -> catch lists:member(T, L). $ erl Erlang R14B01 (erts-5.8.2) [64-bit] [smp:4:4] [rq:4] [async-threads:0] [kernel-poll:false] Eshell V5.8.2 (abort with ^G) 1> test:foo(d, [a,b,c|q]). {'EXIT',{badarg,[{lists,member,[d,[a,b,c|q]]}, {test,foo,2}, {erl_eval,do_apply,5}, {shell,exprs,7}, {shell,eval_exprs,7}, {shell,eval_loop,3}]}} The catch operator returns the whole stackdump. If you used the try..catch construct as your subject indicated but not your code, it will exclude the stacktrace, which is the intended behaviour since constructing the stacktrace will cost performance. To get the stacktrace in a try..catch you can use erlang:get_stacktrace(). See http://erlang.org/doc/reference_manual/expressions.html, section "Try" especially the example: Using try to emulate catch Expr: try Expr catch throw:Term -> Term; exit:Reason -> {'EXIT',Reason} error:Reason -> {'EXIT',{Reason,erlang:get_stacktrace()}} end > > > /Joe -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From raimo+erlang-questions@REDACTED Tue Jan 18 16:53:43 2011 From: raimo+erlang-questions@REDACTED (Raimo Niskanen) Date: Tue, 18 Jan 2011 16:53:43 +0100 Subject: [erlang-questions] why is the error reason in a catch/try truncated? In-Reply-To: <20110118154621.GA19093@erix.ericsson.se> References: <20110118154621.GA19093@erix.ericsson.se> Message-ID: <20110118155343.GA19793@erix.ericsson.se> On Tue, Jan 18, 2011 at 04:46:21PM +0100, Raimo Niskanen wrote: > On Tue, Jan 18, 2011 at 03:52:15PM +0100, Joe Armstrong wrote: > > I have a program. > > > > When I run the "uncaught" version it prints a very nice error message like > > this: > > > > > compile_file(File, Args, ParseTree). > > > > {"init terminating in > > do_boot",{badarg,[{lists,member,[free_index,all_keys]},{ecc_db,add_to_list,2},{ecc_db,store,2},{ecc_pass1,start,2},{ecc_compile,compile_file0,3},{ecc_compile,compile_file,3},{ecc,batch,1},{init,start_it,1}]}} > > > > This tells be exactly were I bombed. > > > > In my "production" version of the code I add wrapper with a catch/try to > > trap any unwanted errors. > > So I write: > > > > compile_file(File, Args, ParseTree) -> > > case (catch compile_file0(File, Args, ParseTree)) of > > {'EXIT', Why} -> > > io:format("Compile file:~s aborted with:~p ~n",[File, Why]); > > _ -> > > ok > > end > > > > But when i run this > > > > > compile_file(File, Args, ParseTree). > > Compile file:first.c aborted with:error badarg > > > > So the Why in the error tuple has lost all the useful information that was > > in > > the untrapped code. > > > > To find the error I have to comment out the wrapped version and > > run the unwrapped version. This is because Why is just the atom > > badarg and not the more verbose {badarg, ...} > > > > I find this behavior very unhelpfull - am I the only one? > > You must be doing something wrong. > > $ cat test.erl > -module(test). > -export([foo/2]). > foo(T, L) -> > catch lists:member(T, L). > > $ erl > Erlang R14B01 (erts-5.8.2) [64-bit] [smp:4:4] [rq:4] [async-threads:0] [kernel-poll:false] > > Eshell V5.8.2 (abort with ^G) > 1> test:foo(d, [a,b,c|q]). > {'EXIT',{badarg,[{lists,member,[d,[a,b,c|q]]}, > {test,foo,2}, > {erl_eval,do_apply,5}, > {shell,exprs,7}, > {shell,eval_exprs,7}, > {shell,eval_loop,3}]}} > > The catch operator returns the whole stackdump. > > If you used the try..catch construct as your subject indicated > but not your code, it will exclude the stacktrace, which is > the intended behaviour since constructing the stacktrace > will cost performance. To get the stacktrace in a try..catch > you can use erlang:get_stacktrace(). See > http://erlang.org/doc/reference_manual/expressions.html, section "Try" > especially the example: Using try to emulate catch Expr: > try Expr > catch > throw:Term -> Term; > exit:Reason -> {'EXIT',Reason} > error:Reason -> {'EXIT',{Reason,erlang:get_stacktrace()}} > end And see also "Programming Erlang" chapter "4 Exceptions" especially section "4.9 Stack Traces", by Joe Armstrong ;-) No, but seriously. It is an optimization so code that just want to catch an exception will no suffer from the emulator building the whole stacktrace just to throw it away without even looking at it. > > > > > > > /Joe > > -- > > / Raimo Niskanen, Erlang/OTP, Ericsson AB > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED -- / Raimo Niskanen, Erlang/OTP, Ericsson AB From egil@REDACTED Tue Jan 18 17:13:06 2011 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Tue, 18 Jan 2011 17:13:06 +0100 Subject: [erlang-questions] Passing arguments to valgrind via cerl -valgrind In-Reply-To: References: Message-ID: <4D35BC12.5020300@erix.ericsson.se> I think you can use VALGRIND_MISC_FLAGS, VALGRIND_MISC_FLAGS=--leak-check=full cerl -valgrind check cerl to see exactly // Bj?rn-Egil Erlang/OTP On 2011-01-18 15:10, Alexey Romanov wrote: > Is it possible to pass additional arguments to valgrind (e.g. > --leak-check=full) when running a debug-enabled runtime system > (http://www.erlang.org/doc/installation_guide/INSTALL.html#id68569)? > > Yours, Alexey Romanov > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From anton.krasovsky@REDACTED Tue Jan 18 17:31:32 2011 From: anton.krasovsky@REDACTED (Anton Krasovsky) Date: Tue, 18 Jan 2011 16:31:32 +0000 Subject: [erlang-questions] Simple tcp server In-Reply-To: References: <1295199390279-3220218.post@n4.nabble.com> Message-ID: Hi Alexander, you might find this tutorial useful: http://www.trapexit.org/Building_a_Non-blocking_TCP_server_using_OTP_principles Regards, Anton On Mon, Jan 17, 2011 at 6:47 AM, Alexander Kuleshov wrote: > Thank you very much for your help. I remake my server. Code : > https://gist.github.com/782566 > > Now it's not CPU 100%. But i'm new in erlang and i want to ask, maybe > i can improve this code or server design, or maybe i don't see hide > errors? > > Thank you. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From b.ghose@REDACTED Tue Jan 18 17:42:21 2011 From: b.ghose@REDACTED (Baishampayan Ghose) Date: Tue, 18 Jan 2011 22:12:21 +0530 Subject: [erlang-questions] Erlang project dependency management/build tools In-Reply-To: References: Message-ID: Eric, > I am more then happy to answer any questions you might have. Thanks for the help. As a matter of fact, I am right now on to the 11th Chapter of your wonderful book "Erlang/OTP in Action". I have learnt a great deal about Erlang/OTP from your book and I am loving it :) Let me explain my requirements a bit more clearly. I intend to create an OTP application which will have external dependencies (eg. Webmachine). I would ideally want to download the dependencies into a folder like deps/ inside my project root since that helps me in distributing the application/release cleanly. I would also like to keep on adding new dependencies in the near future. Can Faxien help me with that? So far, I have found Faxien to install libraries into a global location which is not something I desire. Rebar lets me do that at the moment but I would prefer to use Sinan for building instead because I like the default project layout that Sinan creates. I don't mind using a combination of Rebar and Sinan for my projects but at the moment it's not quite clear to me as to how I can achieve that. Thanks in advance. Regards, BG -- Baishampayan Ghose b.ghose at gmail.com From co7eb@REDACTED Tue Jan 18 18:05:47 2011 From: co7eb@REDACTED (=?iso-8859-1?Q?Gilberio_Carmenates_Garc=EDa?=) Date: Tue, 18 Jan 2011 12:05:47 -0500 Subject: Erlang / C# The Best Interface Message-ID: <002301cbb731$f5b5bd40$e12137c0$@co.cu> Hi all!!! There are any body here who really like C#!, and who want to do some stuffs combining this two wonderful technologies? ** New ExtendedVisualOtp release 1.6. ** All you dream as impossible is now possible. 1- Multiple server connections from the same client. 2- Real-time message passing (obviously thx to Erlang) between the C# client and Erlang server. 3- Request abort feature (a request aborted in the client, abort its resolver process in the server). 4- Multiple Sync and Async request at the same time to different servers form the same client. 5- Unknown Incoming messages treating in the client (the server can send massive messages to all the connected clients, useful for games server and chats), only using a little function like send_to_all(Description, Message) or send_to_others(Description, Message, ExceptionPid). 6- Other features like Abort a waiting for a reply request from another threat. A server disconnection makes all Waiting Requests to release and return null. 7- You can send objects of any kind to the server and to be received in other clients (like classes, images etc.) 8- You can make Sync requests without any fear of getting locks because if the answer for the request arrives before you call the WaitForReply function this function will throw an Error telling you that the request was already answered an received through the OnReceive Event of the async way. Example: IRequestInfo reqinfo = requestHanlder.Request(?Hola mundo?); Thread.Sleep(1000); // if the answer arrives before, through the OnReceive event because this fool! Then the next sentence will throw an error reqinfo.WaitForReply(); or reqinfo.WaitForReply(5000); You will never get locks using Sync requests. And if you are not sure use a timeout. The same way you can abort the request using reqinfo object: reqinfo.AbortRequest(); or know if the request was already answered or made using its properties. You can do many stuffs using IServerReply and IRequestInfo objects, example: using IServerReply object you can use a property to convert the Erlang terms to CSharp terms and vice versa. 9- You can program clients as server-client model of the most easy way (only if you don?t want to write any code in Erlang, just in C#) you can configure de clients and the server to act as intermediary between the clients of so easy way. Using the informing message that send the server to all the clients when a new client is connected or disconnected, sending if you want, its pid to all the other clients, then you can get that pid to send a message using the server as intermediary ?directly? to that other client or register that client in the server as a server prototype, this make possible to program the client-server model with the clients without write any line of code in Erlang. The automatic replier that brings with the framework ExtendedVisualOtp in the server part do all the dirty work for you. 10- You can send whatever you have been programming to your friends who are programming under ExtendedVisualOtp so, with a simple call to the function PublishClientImage(); of the ErlangServerRequest class. Then your program image in runtime it will be executed at the machines of your friends who are connected to the server so. Then if you are developing a chat, you can send the image of your program while your friend is programming other chat or program and see your creation, at the same time he can send to you his creation. The most powerful interface between languages ever known ExtendedVisualOtp. That and so much more EVO Release 1.6 Little examples: the C# part: erlangServerInterface1.Connect(); erlangServerRequest1.Request(new object[] {??insert_person??, ?Ivan?, 26, ?Male?}); the Erlang part: {?insert_person??, Name, Age, Sex}, Pid, RequestInfo}-> . Pid ! {reply, ?inserted_ok?, RequestInfo}; %% if you want to answer to the client who made the request. To receive the answer in the client If you use the sync way, add .WaitForReply().CSharpReply at the end of the request line before And if you want the async way just use the OnReceive event OnReceiveEvent..(IServerReply _reply) { If (_reply.Description == ?insert_person?) { MessageBox.Show(_reply.CSharpReply as string); } } As simple like it seems and powerful as it could not seems. ---------------------------------------------------------------------------- --------------------------------- Ing: Ivan Carmenates Garc?a Current project: ExtendedVisualOtp and Erlang Book: "Erlang Avanzado para principiantes" Ivanco Software Company in association with SPI Team spi_red32LogoSPI ======================================================================= Este mensaje ha sido enviado mediante el servicio de correo electr?nico que ofrece la Federaci?n de Radioaficionados de Cuba a sus miembros para respaldar el cumplimiento de los objetivos de la organizaci?n y su pol?tica informativa. La persona que env?a este correo asume el compromiso de usar el servicio a tales fines y cumplir con las regulaciones establecidas. -------------- next part -------------- An HTML attachment was scrubbed... URL: From garret.smith@REDACTED Tue Jan 18 18:28:44 2011 From: garret.smith@REDACTED (Garret Smith) Date: Tue, 18 Jan 2011 09:28:44 -0800 Subject: Gproc and gen_leader questions Message-ID: I am investigating gproc and gen_leader for further improving my distributed application that today uses dist_ac. I have been doing a bit of testing through interactive shells and some spotty code reading. 2 questions and a request for guidance... Based on some recent threads, I have been testing Abecciu's gen_leader from the gen_leader_revival repository. The warning in the doc mentioning that "only one gen_leader per node is permitted, if used in a diskless environment, incarnation must be adapted." was a little unclear. This warning only applies to diskless nodes, since incarnation uses the registered name as part of the incarnation file? Regarding gproc, I was hoping to use the global registration facility, but the warning in the overview has me worried. Since gproc comes bundled with an older version of gen_leader, will replacing the bundled version with the latest from Abecciu fix global registration, or is there more to be done? gproc_dist is starting gen_leader with [node()|nodes()] which means that only already-connected nodes will be included in the global registry. I am thinking of adding a gproc config setting that will allow to control this, so that nodes not online when gproc starts can be included in the global registry. Any guidance on this? Thanks! Garret Smith From serge@REDACTED Tue Jan 18 19:02:18 2011 From: serge@REDACTED (Serge Aleynikov) Date: Tue, 18 Jan 2011 13:02:18 -0500 Subject: [erlang-questions] Erlang / C# The Best Interface In-Reply-To: <002301cbb731$f5b5bd40$e12137c0$@co.cu> References: <002301cbb731$f5b5bd40$e12137c0$@co.cu> Message-ID: <4D35D5AA.6010000@aleynikov.org> Is this project a fork of OTP.NET (*)? (*) http://jungerl.cvs.sourceforge.net/viewvc/jungerl/jungerl/lib/otp.net On 1/18/2011 12:05 PM, Gilberio Carmenates Garc?a wrote: > Hi all!!! > > There are any body here who really like C#!, and who want to do some > stuffs combining this two wonderful technologies? > > ** New ExtendedVisualOtp release 1.6. ** > > All you dream as impossible is now possible. > > 1-Multiple server connections from the same client. > > 2-Real-time message passing (obviously thx to Erlang) between the C# > client and Erlang server. > > 3-Request abort feature (a request aborted in the client, abort its > resolver process in the server). > > 4-Multiple Sync and Async request at the same time to different servers > form the same client. > > 5-Unknown Incoming messages treating in the client (the server can send > massive messages to all the connected clients, useful for games server > and chats), > > only using a little function like send_to_all(Description, Message) or > send_to_others(Description, Message, ExceptionPid). > > 6-Other features like Abort a waiting for a reply request from another > threat. A server disconnection makes all Waiting Requests to release and > return null. > > 7-You can send objects of any kind to the server and to be received in > other clients (like classes, images etc.) > > 8-You can make Sync requests without any fear of getting locks because > if the answer for the request arrives before you call the WaitForReply > function this function will throw an Error telling you that the request > was already answered an received through the OnReceive Event of the > async way. > > Example: > > IRequestInfo reqinfo = requestHanlder.Request(?Hola mundo?); > > Thread.Sleep(1000); // if the answer arrives before, through the > OnReceive event because this fool! Then the next sentence will throw an > error > > reqinfo.WaitForReply(); or reqinfo.WaitForReply(5000); > > You will never get locks using Sync requests. And if you are not sure > use a timeout. > > The same way you can abort the request using reqinfo object: > reqinfo.AbortRequest(); or know if the request was already answered or > made using its properties. > > You can do many stuffs using IServerReply and IRequestInfo objects, > example: using IServerReply object you can use a property to convert the > Erlang terms to CSharp terms and vice versa. > > 9-You can program clients as server-client model of the most easy way > (only if you don?t want to write any code in Erlang, just in C#) you can > configure de clients > > and the server to act as intermediary between the clients of so easy > way. Using the informing message that send the server to all the clients > when a new client is connected or disconnected, sending if you want, its > pid to all the other clients, then you can get that pid to send a > message using the server as intermediary ?directly? to that other client > or register that client in the server as a server prototype, this make > possible to program the client-server model with the clients without > write any line of code in Erlang. The automatic replier that brings with > the framework ExtendedVisualOtp in the server part do all the dirty work > for you. > > 10-You can send whatever you have been programming to your friends who > are programming under ExtendedVisualOtp so, with a simple call to the > function > > PublishClientImage(); of the ErlangServerRequest class. Then your > program image in runtime it will be executed at the machines of your > friends who are connected to the > > server so. Then if you are developing a chat, you can send the image of > your program while your friend is programming other chat or program and > see your creation, at the > > same time he can send to you his creation. > > The most powerful interface between languages ever known > ExtendedVisualOtp. That and so much more EVO Release 1.6 > > Little examples: > > ?the C# part: > > erlangServerInterface1.Connect(); > > erlangServerRequest1.Request(new object[] {??insert_person??, ?Ivan?, > 26, ?Male?}); > > ?the Erlang part: > > {?insert_person??, Name, Age, Sex}, Pid, RequestInfo}-> > > ?. > > Pid ! {reply, ?inserted_ok?, RequestInfo}; %% if you want to > answer to the client who made the request. > > To receive the answer in the client > > If you use the sync way, add .WaitForReply().CSharpReply? at the end > of the request line before? > > And if you want the async way just use the OnReceive event > > OnReceiveEvent..(IServerReply _reply) > > { > > If (_/reply.Description == ?insert/_person?) { > > MessageBox.Show(_reply.CSharpReply as string); > > } > > } > > As simple like it seems and powerful as it could not seems. > > *-------------------------------------------------------------------------------------------------------------* > > *Ing: Ivan Carmenates Garc?a* > > Current project: ExtendedVisualOtp and Erlang Book: "Erlang Avanzado > para principiantes" > > Ivanco Software Company in association with SPI Team > > spi_red32LogoSPI > > > ======================================================================= > Este mensaje ha sido enviado mediante el servicio de correo electr?nico > que ofrece la Federaci?n de Radioaficionados de Cuba a sus miembros para > respaldar el cumplimiento de los objetivos de la organizaci?n y su > pol?tica informativa. La persona que env?a este correo asume el > compromiso de usar el servicio a tales fines y cumplir con las > regulaciones establecidas. From garret.smith@REDACTED Tue Jan 18 19:13:00 2011 From: garret.smith@REDACTED (Garret Smith) Date: Tue, 18 Jan 2011 10:13:00 -0800 Subject: Gproc and gen_leader questions In-Reply-To: References: Message-ID: Sorry for the last paragraph, it already works this way. I just had to dig a little deeper. On Tue, Jan 18, 2011 at 9:28 AM, Garret Smith wrote: > I am investigating gproc and gen_leader for further improving my > distributed application that today uses dist_ac. > > I have been doing a bit of testing through interactive shells and some > spotty code reading. ?2 questions and a request for guidance... > > Based on some recent threads, I have been testing Abecciu's gen_leader > from the gen_leader_revival repository. > The warning in the doc mentioning that "only one gen_leader per node > is permitted, if used in a diskless > environment, incarnation must be adapted." was a little unclear. ?This > warning only applies to diskless nodes, > since incarnation uses the registered name as part of the incarnation file? > > Regarding gproc, I was hoping to use the global registration facility, > but the warning in the overview has me worried. > Since gproc comes bundled with an older version of gen_leader, will > replacing the bundled version with the latest > from Abecciu fix global registration, or is there more to be done? > > gproc_dist is starting gen_leader with [node()|nodes()] which means > that only already-connected nodes will be > included in the global registry. ?I am thinking of adding a gproc > config setting that will allow to control this, so that > nodes not online when gproc starts can be included in the global > registry. ?Any guidance on this? > > Thanks! > Garret Smith > From watson.timothy@REDACTED Tue Jan 18 21:49:19 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 18 Jan 2011 20:49:19 +0000 Subject: [erlang-questions] Any tools available to generate code for Erlang Ports? In-Reply-To: References: <4D359BAA.3000606@ecs.soton.ac.uk> Message-ID: > The only examples of this I've seen have been pretty basic though - are > there any larger examples or tutorials on this out there? I probably should've mentioned that in terms of larger examples, there are lots out there. Some core functionality (i.e., in kernel/erts) is provided by linked in drivers, so the Erlang/OTP source code is a good place to start (https://github.com/erlang/otp). The spidermonkey driver used by riak is pretty mature and worth a look (https://github.com/basho/erlang_js), the ejabberd code base uses a number of drivers (https://github.com/processone/ejabberd) and there are plenty of other larger examples out there too (a couple of lua/erlang integrations for instance). Tim From watson.timothy@REDACTED Tue Jan 18 21:49:57 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 18 Jan 2011 20:49:57 +0000 Subject: [erlang-questions] Any tools available to generate code for Erlang Ports? In-Reply-To: References: <4D359BAA.3000606@ecs.soton.ac.uk> Message-ID: On 18 January 2011 13:54, Dave Challis wrote: > I'm just curious - are there any tools or code generators out there for > creating Erlang Ports to interface with C code? > > There are a bunch of old C libraries I'd like to be able to use from within > Erlang, and it seems pretty time consuming to code interfaces to them from > scratch. > There's the Erlang Driver Toolkit (http://www.snookles.com/erlang/edtk/) which is pretty good but documentation is a bit sparse. It looks pretty solid. Another one is DryvErl but I've not looked closely at that one. I came across another tool on github recently - https://github.com/tonyrog/eapi. This is completely undocumented apart from some example code, but looks very interesting. Finally, there was a guy on the list some time ago mentioning that he'd written some generic C/C++ stuff for generating driver code, but I've been unable to fish out the link to his post. You might want to try searching github, as there are probably a few others out there too. I'd also suggest looking into NIFs if you're library calls are likely to be short lived - as long as you do need complex state in the driver and can live with blocking semantics, they're a significantly easier options and not too hard to code yourself. Cheers, Tim From watson.timothy@REDACTED Tue Jan 18 21:53:07 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 18 Jan 2011 20:53:07 +0000 Subject: [erlang-questions] Any tools available to generate code for Erlang Ports? In-Reply-To: References: <4D359BAA.3000606@ecs.soton.ac.uk> Message-ID: On 18 January 2011 13:54, Dave Challis wrote: > I'm just curious - are there any tools or code generators out there for > creating Erlang Ports to interface with C code? Sorry that my last two response got completely scrambled - I *really* need to get some sleep! With regard to NIFs, I meant to say "as long as you do not" maintain state in the driver - for that sort of thing and for async processing, you want a port program or linked-in driver. From watson.timothy@REDACTED Tue Jan 18 21:58:38 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 18 Jan 2011 20:58:38 +0000 Subject: [erlang-questions] Erlang / C# The Best Interface In-Reply-To: <002301cbb731$f5b5bd40$e12137c0$@co.cu> References: <002301cbb731$f5b5bd40$e12137c0$@co.cu> Message-ID: Pretty cool. I like the dynamic feel of `erlangServerRequest1.Request(new object[] {??insert_person??, ?Ivan?, 26, ?Male?});` but couldn't you define the `Request` method using a param-array (i.e., Request(params object[] args) { ... }) so as to avoid the noisy object array instantiation at the call site? Silly me nit-picking though. Looks like a nice API - I'll have a play integrating Erlang with F# this weekend. :) From MARTIN.LOGAN@REDACTED Tue Jan 18 22:09:21 2011 From: MARTIN.LOGAN@REDACTED (Logan, Martin) Date: Tue, 18 Jan 2011 15:09:21 -0600 Subject: [erlang-questions] Erlang project dependency management/build tools In-Reply-To: Message-ID: Baishampayan, if you want to use faxien to install application dependencies into totally self directed locations try this faxien fetch-app [app-vsn] for example faxien fetch-app ibrowse $HOME/my_erlang_lib Does this help? Cheers, Martin On 1/18/11 10:42 AM, "Baishampayan Ghose" wrote: >Eric, > >> I am more then happy to answer any questions you might have. > >Thanks for the help. As a matter of fact, I am right now on to the >11th Chapter of your wonderful book "Erlang/OTP in Action". I have >learnt a great deal about Erlang/OTP from your book and I am loving it >:) > >Let me explain my requirements a bit more clearly. > >I intend to create an OTP application which will have external >dependencies (eg. Webmachine). I would ideally want to download the >dependencies into a folder like deps/ inside my project root since >that helps me in distributing the application/release cleanly. > >I would also like to keep on adding new dependencies in the near >future. Can Faxien help me with that? So far, I have found Faxien to >install libraries into a global location which is not something I >desire. > >Rebar lets me do that at the moment but I would prefer to use Sinan >for building instead because I like the default project layout that >Sinan creates. > >I don't mind using a combination of Rebar and Sinan for my projects >but at the moment it's not quite clear to me as to how I can achieve >that. > >Thanks in advance. > >Regards, >BG > >-- >Baishampayan Ghose >b.ghose at gmail.com > >________________________________________________________________ >erlang-questions (at) erlang.org mailing list. >See http://www.erlang.org/faq.html >To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From watson.timothy@REDACTED Tue Jan 18 22:20:04 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Tue, 18 Jan 2011 21:20:04 +0000 Subject: [erlang-questions] Erlang project dependency management/build tools In-Reply-To: References: Message-ID: You might also want to look at epm (https://github.com/JacobVorreuter/epm) and sutro (https://github.com/JacobVorreuter/sutro), which do similar stuff from a package management point of view. Personally I use rebar for builds because I only requires configuration where the 'default' OTP setup needs to be overridden. Sometimes I've combined this with epm, as you can see in a couple of builds: The hamcrest-erlang build uses a combination of epm and rebar to make sure dependencies are in place when building (https://github.com/hyperthunk/hamcrest-erlang/blob/master/Makefile). Another example where I've had to do this because rebar can't always build packages that aren't following standard OTP principles and/or require non-standard build steps (such as autoconf/autotools integrations) is in this project stub: https://github.com/hyperthunk/eventing-playpen/blob/master/Makefile. Personally I'd recommend sutro now instead of epm, as the support for pre/post build commands and ease of dependency management a la homebrew make it very simple to use. Both take advantage of OTP design principles in the same way rebar does. Neither requires that you package, build or configure your application in any other way. From nicolas@REDACTED Tue Jan 18 21:50:11 2011 From: nicolas@REDACTED (Nicolas Thauvin) Date: Tue, 18 Jan 2011 21:50:11 +0100 Subject: Eunit controls Message-ID: Hi, While trying to make use of Eunit controls in rebar (http://www.erlang.org/doc/apps/eunit/chapter.html#Control), some questions arose... My understanding is that all the test directives are recursive according to the @type tests() in eunit_data.erl. 1) Is it correct ? -> is {inorder, [{inparallel, [{timeout, 10, [{M,F}]}]}} a valid test specification? 2) Should modules be supported by the timeout control ? -> is {timeout, 10, [foo, bar]} valid ? => does not seem to work despite "A module can also be used to represent a test set [...]". As the rebar building tool is working with modules, that would be great if all directives could take modules as arguments... But it does not seem to be the case, at least for {timeout, N, ...}. For example, given two tests with basic timer:sleep(6000) : * eunit:test([{inparallel, [{timeout, 10, {foo, t1_test}}, {timeout, 10, {foo, t2_test}}]}], [verbose]). foo: t1_test...[6.005 s] ok foo: t2_test...[6.003 s] ok => Timeout rules are applied in the inparallel directive * eunit:test([{inparallel, [{timeout, 10, foo}]}], [verbose]). foo: t1_test...*timed out* => timeout is ignored What do you think ? Bug ? Feature ? -- Nicolas From rzezeski@REDACTED Tue Jan 18 23:24:31 2011 From: rzezeski@REDACTED (Ryan Zezeski) Date: Tue, 18 Jan 2011 17:24:31 -0500 Subject: linked-in driver blocks net_kernel tick? Message-ID: Hi everyone, Some of you may remember my latest question where I was having weird node timeout issues that I couldn't explain and I thought it might be related to the messages I was passing between my nodes. Well, I pinpointed the problem to a call to zlib:gzip/1. At first I was really surprised by this, as such a harmless line of code surely should have nothing to do with the ability for my nodes to communicate. However, as I dug further I realized gzip was implemented as a linked-in driver and I remember reading things about how one has to take care with them because they can trash the VM with them. I don't remember reading anything about them blocking code, and even if they do I fail to see why my SMP enabled node (16 cores) would allow this one thread to block the tick. It occurred to me that maybe the scheduler responsible for that process is the one blocked by the driver. Do processes have scheduler affinity? That would make sense, I guess. I've "fixed" this problem simply by using a plain port (i.e. run in it's own OS process). For my purposes, this actually makes more sense in the majority of the places I was making use of gzip. Can someone enlighten me as to exactly what is happening behind the scenes? To reproduce I create a random 1.3GB file: dd if=/dev/urandom of=rand bs=1048576 count=1365 Then start two named nodes 'foo' and 'bar', connect them, read in the file, and then compress said file. Sometime later (I think around 60+ seconds) the node 'bar' will claim that 'foo' is not responding. [progski@REDACTED ~/tmp_code/node_timeout] erl -name foo Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.1 (abort with ^G) (foo@REDACTED)1> net_adm:ping('bar@REDACTED'). pong (foo@REDACTED)2> nodes(). ['bar@REDACTED'] (foo@REDACTED)3> {ok,Data} = file:read_file("rand"). {ok,<<103,5,115,210,177,147,53,45,250,182,51,32,250,233, 39,253,102,61,73,242,18,159,45,185,232,80,33,...>>} (foo@REDACTED)4> zlib:gzip(Data). <<31,139,8,0,0,0,0,0,0,3,0,15,64,240,191,103,5,115,210, 177,147,53,45,250,182,51,32,250,233,...>> (foo@REDACTED)5> [progski@REDACTED ~/tmp_code/node_timeout] erl -name bar Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.1 (abort with ^G) (bar@REDACTED)1> nodes(). ['foo@REDACTED'] (bar@REDACTED)2> =ERROR REPORT==== 18-Jan-2011::17:16:10 === ** Node 'foo@REDACTED' not responding ** ** Removing (timedout) connection ** Thanks, -Ryan From alain.odea@REDACTED Wed Jan 19 00:04:03 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Tue, 18 Jan 2011 19:34:03 -0330 Subject: [erlang-questions] linked-in driver blocks net_kernel tick? In-Reply-To: References: Message-ID: On 2011-01-18, at 18:54, Ryan Zezeski wrote: > Hi everyone, > > Some of you may remember my latest question where I was having weird node > timeout issues that I couldn't explain and I thought it might be related to > the messages I was passing between my nodes. Well, I pinpointed the problem > to a call to zlib:gzip/1. At first I was really surprised by this, as such > a harmless line of code surely should have nothing to do with the ability > for my nodes to communicate. However, as I dug further I realized gzip was > implemented as a linked-in driver and I remember reading things about how > one has to take care with them because they can trash the VM with them. I > don't remember reading anything about them blocking code, and even if they > do I fail to see why my SMP enabled node (16 cores) would allow this one > thread to block the tick. It occurred to me that maybe the scheduler > responsible for that process is the one blocked by the driver. Do processes > have scheduler affinity? That would make sense, I guess. > > I've "fixed" this problem simply by using a plain port (i.e. run in it's own > OS process). For my purposes, this actually makes more sense in the > majority of the places I was making use of gzip. Can someone enlighten me > as to exactly what is happening behind the scenes? > > To reproduce I create a random 1.3GB file: > > dd if=/dev/urandom of=rand bs=1048576 count=1365 > > Then start two named nodes 'foo' and 'bar', connect them, read in the file, > and then compress said file. Sometime later (I think around 60+ seconds) > the node 'bar' will claim that 'foo' is not responding. > > [progski@REDACTED ~/tmp_code/node_timeout] erl -name foo > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] Your SMP node seems to be capped at smp:2:2 when it out to be smp:16. Some resource limit may be holding back the system. That said zlib should not ever cause this issue. > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.1 (abort with ^G) > (foo@REDACTED)1> net_adm:ping('bar@REDACTED'). > pong > (foo@REDACTED)2> nodes(). > ['bar@REDACTED'] > (foo@REDACTED)3> {ok,Data} = file:read_file("rand"). > {ok,<<103,5,115,210,177,147,53,45,250,182,51,32,250,233, > 39,253,102,61,73,242,18,159,45,185,232,80,33,...>>} > (foo@REDACTED)4> zlib:gzip(Data). > <<31,139,8,0,0,0,0,0,0,3,0,15,64,240,191,103,5,115,210, > 177,147,53,45,250,182,51,32,250,233,...>> > (foo@REDACTED)5> > > > [progski@REDACTED ~/tmp_code/node_timeout] erl -name bar > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.1 (abort with ^G) > (bar@REDACTED)1> nodes(). > ['foo@REDACTED'] > (bar@REDACTED)2> > =ERROR REPORT==== 18-Jan-2011::17:16:10 === > ** Node 'foo@REDACTED' not responding ** > ** Removing (timedout) connection ** > > > Thanks, > > -Ryan From rzezeski@REDACTED Wed Jan 19 00:53:33 2011 From: rzezeski@REDACTED (Ryan Zezeski) Date: Tue, 18 Jan 2011 18:53:33 -0500 Subject: [erlang-questions] linked-in driver blocks net_kernel tick? In-Reply-To: References: Message-ID: Apologies, the example I copied was run on my mac. This is what I have on the actual production machine: Erlang R14A (erts-5.8) [source] [64-bit] [smp:16:16] [rq:16] [async-threads:0] [hipe] [kernel-poll:false] To be certain, I ran the same example (except this time using two physical machines) and achieved the same result. Namely, the 'bar' node claims 'foo' is not responding and thus closes the connection. Whatever this is, I've now easily reproduced it on two different OSs, with 2 different Erlang versions. -Ryan On Tue, Jan 18, 2011 at 6:04 PM, Alain O'Dea wrote: > On 2011-01-18, at 18:54, Ryan Zezeski wrote: > > > Hi everyone, > > > > Some of you may remember my latest question where I was having weird node > > timeout issues that I couldn't explain and I thought it might be related > to > > the messages I was passing between my nodes. Well, I pinpointed the > problem > > to a call to zlib:gzip/1. At first I was really surprised by this, as > such > > a harmless line of code surely should have nothing to do with the ability > > for my nodes to communicate. However, as I dug further I realized gzip > was > > implemented as a linked-in driver and I remember reading things about how > > one has to take care with them because they can trash the VM with them. > I > > don't remember reading anything about them blocking code, and even if > they > > do I fail to see why my SMP enabled node (16 cores) would allow this one > > thread to block the tick. It occurred to me that maybe the scheduler > > responsible for that process is the one blocked by the driver. Do > processes > > have scheduler affinity? That would make sense, I guess. > > > > I've "fixed" this problem simply by using a plain port (i.e. run in it's > own > > OS process). For my purposes, this actually makes more sense in the > > majority of the places I was making use of gzip. Can someone enlighten > me > > as to exactly what is happening behind the scenes? > > > > To reproduce I create a random 1.3GB file: > > > > dd if=/dev/urandom of=rand bs=1048576 count=1365 > > > > Then start two named nodes 'foo' and 'bar', connect them, read in the > file, > > and then compress said file. Sometime later (I think around 60+ seconds) > > the node 'bar' will claim that 'foo' is not responding. > > > > [progski@REDACTED ~/tmp_code/node_timeout] erl -name foo > > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] > > Your SMP node seems to be capped at smp:2:2 when it out to be smp:16. Some > resource limit may be holding back the system. That said zlib should not > ever cause this issue. > > > [async-threads:0] [hipe] [kernel-poll:false] > > > > Eshell V5.8.1 (abort with ^G) > > (foo@REDACTED)1> net_adm:ping('bar@REDACTED'). > > pong > > (foo@REDACTED)2> nodes(). > > ['bar@REDACTED'] > > (foo@REDACTED)3> {ok,Data} = file:read_file("rand"). > > {ok,<<103,5,115,210,177,147,53,45,250,182,51,32,250,233, > > 39,253,102,61,73,242,18,159,45,185,232,80,33,...>>} > > (foo@REDACTED)4> zlib:gzip(Data). > > <<31,139,8,0,0,0,0,0,0,3,0,15,64,240,191,103,5,115,210, > > 177,147,53,45,250,182,51,32,250,233,...>> > > (foo@REDACTED)5> > > > > > > [progski@REDACTED ~/tmp_code/node_timeout] erl -name bar > > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] > > [async-threads:0] [hipe] [kernel-poll:false] > > > > Eshell V5.8.1 (abort with ^G) > > (bar@REDACTED)1> nodes(). > > ['foo@REDACTED'] > > (bar@REDACTED)2> > > =ERROR REPORT==== 18-Jan-2011::17:16:10 === > > ** Node 'foo@REDACTED' not responding ** > > ** Removing (timedout) connection ** > > > > > > Thanks, > > > > -Ryan > From garret.smith@REDACTED Wed Jan 19 01:21:39 2011 From: garret.smith@REDACTED (Garret Smith) Date: Tue, 18 Jan 2011 16:21:39 -0800 Subject: gproc patch Message-ID: Fixed a small issue with gproc in the 'await' function when no timeout is specified. see branch 'fix-await-notimeout' in the repository git://github.com/garret-smith/gproc.git or the patch: --- a/src/gproc.erl +++ b/src/gproc.erl @@ -312,7 +312,7 @@ request_wait({n,C,_} = Key, Timeout) when C==l; C==g -> WRef = call({await,Key,self()}, C), receive {gproc, WRef, registered, {_K, Pid, V}} -> - case Timeout of + case TRef of no_timer -> ignore; _ -> erlang:cancel_timer(TRef) end, I don't know of any guidelines for patch submission since this is not OTP, so let me know if there is a better way I can provide patches now or in the future. -G From co7eb@REDACTED Wed Jan 19 01:09:18 2011 From: co7eb@REDACTED (=?iso-8859-1?Q?Gilberio_Carmenates_Garc=EDa?=) Date: Tue, 18 Jan 2011 19:09:18 -0500 Subject: [erlang-questions] Erlang / C# The Best Interface Message-ID: <000b01cbb76d$1f9ecc10$5edc6430$@co.cu> Hi!, yes this project uses the Otp.Net core with some fixes in the data exchange. -----Mensaje original----- De: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] En nombre de Serge Aleynikov Enviado el: Martes, 18 de Enero de 2011 13:02 Para: Gilberio Carmenates Garc?a CC: erlang-questions@REDACTED Asunto: Re: [erlang-questions] Erlang / C# The Best Interface Is this project a fork of OTP.NET (*)? (*) http://jungerl.cvs.sourceforge.net/viewvc/jungerl/jungerl/lib/otp.net On 1/18/2011 12:05 PM, Gilberio Carmenates Garc?a wrote: > Hi all!!! > > There are any body here who really like C#!, and who want to do some > stuffs combining this two wonderful technologies? > > ** New ExtendedVisualOtp release 1.6. ** > > All you dream as impossible is now possible. > > 1-Multiple server connections from the same client. > > 2-Real-time message passing (obviously thx to Erlang) between the C# > client and Erlang server. > > 3-Request abort feature (a request aborted in the client, abort its > resolver process in the server). > > 4-Multiple Sync and Async request at the same time to different servers > form the same client. > > 5-Unknown Incoming messages treating in the client (the server can send > massive messages to all the connected clients, useful for games server > and chats), > > only using a little function like send_to_all(Description, Message) or > send_to_others(Description, Message, ExceptionPid). > > 6-Other features like Abort a waiting for a reply request from another > threat. A server disconnection makes all Waiting Requests to release and > return null. > > 7-You can send objects of any kind to the server and to be received in > other clients (like classes, images etc.) > > 8-You can make Sync requests without any fear of getting locks because > if the answer for the request arrives before you call the WaitForReply > function this function will throw an Error telling you that the request > was already answered an received through the OnReceive Event of the > async way. > > Example: > > IRequestInfo reqinfo = requestHanlder.Request(?Hola mundo?); > > Thread.Sleep(1000); // if the answer arrives before, through the > OnReceive event because this fool! Then the next sentence will throw an > error > > reqinfo.WaitForReply(); or reqinfo.WaitForReply(5000); > > You will never get locks using Sync requests. And if you are not sure > use a timeout. > > The same way you can abort the request using reqinfo object: > reqinfo.AbortRequest(); or know if the request was already answered or > made using its properties. > > You can do many stuffs using IServerReply and IRequestInfo objects, > example: using IServerReply object you can use a property to convert the > Erlang terms to CSharp terms and vice versa. > > 9-You can program clients as server-client model of the most easy way > (only if you don?t want to write any code in Erlang, just in C#) you can > configure de clients > > and the server to act as intermediary between the clients of so easy > way. Using the informing message that send the server to all the clients > when a new client is connected or disconnected, sending if you want, its > pid to all the other clients, then you can get that pid to send a > message using the server as intermediary ?directly? to that other client > or register that client in the server as a server prototype, this make > possible to program the client-server model with the clients without > write any line of code in Erlang. The automatic replier that brings with > the framework ExtendedVisualOtp in the server part do all the dirty work > for you. > > 10-You can send whatever you have been programming to your friends who > are programming under ExtendedVisualOtp so, with a simple call to the > function > > PublishClientImage(); of the ErlangServerRequest class. Then your > program image in runtime it will be executed at the machines of your > friends who are connected to the > > server so. Then if you are developing a chat, you can send the image of > your program while your friend is programming other chat or program and > see your creation, at the > > same time he can send to you his creation. > > The most powerful interface between languages ever known > ExtendedVisualOtp. That and so much more EVO Release 1.6 > > Little examples: > > the C# part: > > erlangServerInterface1.Connect(); > > erlangServerRequest1.Request(new object[] {??insert_person??, ?Ivan?, > 26, ?Male?}); > > the Erlang part: > > {?insert_person??, Name, Age, Sex}, Pid, RequestInfo}-> > > . > > Pid ! {reply, ?inserted_ok?, RequestInfo}; %% if you want to > answer to the client who made the request. > > To receive the answer in the client > > If you use the sync way, add .WaitForReply().CSharpReply at the end > of the request line before > > And if you want the async way just use the OnReceive event > > OnReceiveEvent..(IServerReply _reply) > > { > > If (_/reply.Description == ?insert/_person?) { > > MessageBox.Show(_reply.CSharpReply as string); > > } > > } > > As simple like it seems and powerful as it could not seems. > > *--------------------------------------------------------------------------- ----------------------------------* > > *Ing: Ivan Carmenates Garc?a* > > Current project: ExtendedVisualOtp and Erlang Book: "Erlang Avanzado > para principiantes" > > Ivanco Software Company in association with SPI Team > > spi_red32LogoSPI > > > ======================================================================= > Este mensaje ha sido enviado mediante el servicio de correo electr?nico > que ofrece la Federaci?n de Radioaficionados de Cuba a sus miembros para > respaldar el cumplimiento de los objetivos de la organizaci?n y su > pol?tica informativa. La persona que env?a este correo asume el > compromiso de usar el servicio a tales fines y cumplir con las > regulaciones establecidas. ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED ======================================================================= Este mensaje ha sido enviado mediante el servicio de correo electr?nico que ofrece la Federaci?n de Radioaficionados de Cuba a sus miembros para respaldar el cumplimiento de los objetivos de la organizaci?n y su pol?tica informativa. La persona que env?a este correo asume el compromiso de usar el servicio a tales fines y cumplir con las regulaciones establecidas. From norton@REDACTED Tue Jan 18 16:33:06 2011 From: norton@REDACTED (Joseph Wayne Norton) Date: Wed, 19 Jan 2011 00:33:06 +0900 Subject: [erlang-questions] Erlang project dependency management/build tools In-Reply-To: References: Message-ID: BG - I have found rebar to be a good tool for the job. The rebar team is doing a great job. Here is an example of applying the rebar tool along with Google's repo tool. http://hibari.github.com/hibari-doc/hibari-app-developer-guide.en.html#_building_hibari_from_source The repo tool is optional since rebar can also be used to automate the downloading of dependencies. However, the repo tool has some nice features for managing a set of Git repositories. The documentation effort is still a work in progress. Nevertheless, it might be helpful for your study. thanks, - Joe On 2011/01/18, at 23:30, Baishampayan Ghose wrote: > Hello, > I am quite new to the Erlang world and I am trying to develop an OTP > application (with releases in the future). I am looking for a tool > that will let me manage the dependencies, build process and release > engineering. > I have found Rebar & Sinan (+ Faxien) so far but it seems Sinan > doesn't do dependency management(?) and Rebar obviously lacks > documentation. > I would like to know the opinion of the Erlang community on this > matter; I am looking for a simple tool that will fetch external > dependencies (or otherwise let me put them in a directory), > build/test/generate docs for the project and help with the OTP release > creation process. > Thanks in advance. > Regards, > BG > --Baishampayan Ghose > b.ghose at gmail.com > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Joseph Wayne Norton norton@REDACTED From norton@REDACTED Tue Jan 4 11:30:15 2011 From: norton@REDACTED (Joseph Wayne Norton) Date: Tue, 04 Jan 2011 19:30:15 +0900 Subject: [JOB] Gemini Mobile Technologies, Inc. hiring in Tokyo, Japan Message-ID: Gemini Mobile Technologies, Inc. is hiring talented Junior and Senior Software Engineers in Tokyo, Japan. If you are an Erlang developer or a developer wanting to learn Erlang, check the following sites for information: http://www.geminimobile.com/company/careers.html http://www.geminimobile.jp/company/career.html If you have any questions, please email me at norton @ geminimobile . com regards, Joseph Wayne Norton From co7eb@REDACTED Wed Jan 19 01:42:48 2011 From: co7eb@REDACTED (=?iso-8859-1?Q?Gilberio_Carmenates_Garc=EDa?=) Date: Tue, 18 Jan 2011 19:42:48 -0500 Subject: [erlang-questions] Erlang / C# The Best Interface In-Reply-To: References: <002301cbb731$f5b5bd40$e12137c0$@co.cu> Message-ID: <000c01cbb771$ce15e540$6a41afc0$@co.cu> Hi, yes I already thought that but the problem is when you tend to make a request like this, imagine this three clauses in the receive statement in Erlang. {insert_person, Name, Age}-> . ; logoff-> ; {i_like_to_be_here}-> ; So for the first case you should do Request(new object[] { ??insert_person??, ?Ivan?, ?26?}); %% new object[] means new tuple { } So if you use Request(params object[]) -> Request(??insert_person??, ?Ivan?, ?26?) to make a tuple then how you send the simple ?logoff? atom? Like Request(? ?logoff? ?); So if you want to send {i_like_to_be_here} ----> Request(new object[] { ? ?i_like_to_be_here? ? }); How could I specify if I want to send a simple atom or a tuple with a simple atom inside using the same form Request(? ?i_like_to_be_here? ?); Other cool way could be, example: string Name = ?Ivan?; Request(? {?insert_person?, Name} ?); but hum seems to me I need to use a lot of ?Reflections? perhaps using F#. Using F# that would be nice to rebuild the component!! Using close pattern matching like Erlang in F# and Erlang expressions. That could be really fun!!!. Best, Ivan. De: Tim Watson [mailto:watson.timothy@REDACTED] Enviado el: Martes, 18 de Enero de 2011 15:59 Para: Gilberio Carmenates Garc?a CC: erlang-questions@REDACTED Asunto: Re: [erlang-questions] Erlang / C# The Best Interface Pretty cool. I like the dynamic feel of `erlangServerRequest1.Request(new object[] {??insert_person??, ?Ivan?, 26, ?Male?});` but couldn't you define the `Request` method using a param-array (i.e., Request(params object[] args) { ... }) so as to avoid the noisy object array instantiation at the call site? Silly me nit-picking though. Looks like a nice API - I'll have a play integrating Erlang with F# this weekend. :) ======================================================================= Este mensaje ha sido enviado mediante el servicio de correo electr?nico que ofrece la Federaci?n de Radioaficionados de Cuba a sus miembros para respaldar el cumplimiento de los objetivos de la organizaci?n y su pol?tica informativa. La persona que env?a este correo asume el compromiso de usar el servicio a tales fines y cumplir con las regulaciones establecidas. From mjtruog@REDACTED Wed Jan 19 02:20:59 2011 From: mjtruog@REDACTED (Michael Truog) Date: Tue, 18 Jan 2011 17:20:59 -0800 Subject: [erlang-questions] Any tools available to generate code for Erlang Ports? In-Reply-To: References: <4D359BAA.3000606@ecs.soton.ac.uk> Message-ID: <4D363C7B.6030406@gmail.com> On 01/18/2011 12:49 PM, Tim Watson wrote: > On 18 January 2011 13:54, Dave Challis wrote: > >> I'm just curious - are there any tools or code generators out there for >> creating Erlang Ports to interface with C code? >> >> There are a bunch of old C libraries I'd like to be able to use from within >> Erlang, and it seems pretty time consuming to code interfaces to them from >> scratch. >> >> > There's the Erlang Driver Toolkit > (http://www.snookles.com/erlang/edtk/) which is pretty good but > documentation is a bit sparse. It looks pretty solid. Another one is > DryvErl but I've not looked closely at that one. I came across another > tool on github recently - https://github.com/tonyrog/eapi. This is > completely undocumented apart from some example code, but looks very > interesting. Finally, there was a guy on the list some time ago > mentioning that he'd written some generic C/C++ stuff for generating > driver code, but I've been unable to fish out the link to his post. > You might want to try searching github, as there are probably a few > others out there too. > You might be thinking of GEP[D] (https://github.com/okeuday/generic-erlang-port--driver-). GEP[D] does everything EDTK does except the weird handling of file descriptors, but the code was written before NIFs, so it does port driver code (with async support). From co7eb@REDACTED Wed Jan 19 03:31:44 2011 From: co7eb@REDACTED (=?iso-8859-1?Q?Gilberio_Carmenates_Garc=EDa?=) Date: Tue, 18 Jan 2011 21:31:44 -0500 Subject: [erlang-questions] Erlang / C# The Best Interface In-Reply-To: <4D363D0A.5040103@aleynikov.org> References: <002301cbb731$f5b5bd40$e12137c0$@co.cu> <4D363D0A.5040103@aleynikov.org> Message-ID: <000301cbb781$05829820$1087c860$@co.cu> Hi, no I haven't provided it yet. I need a good page hosting to do it, not a repository, I need for example a redmine site hosting or something like that. I hate repositories. Regards, Ivan. -----Mensaje original----- De: Serge Aleynikov [mailto:serge@REDACTED] Enviado el: Martes, 18 de Enero de 2011 20:23 Para: Gilberio Carmenates Garc?a Asunto: Re: [erlang-questions] Erlang / C# The Best Interface Did I miss a link to the source? Or you haven't provided one? On 1/18/2011 12:05 PM, Gilberio Carmenates Garc?a wrote: > Hi all!!! > > There are any body here who really like C#!, and who want to do some > stuffs combining this two wonderful technologies? > > ** New ExtendedVisualOtp release 1.6. ** > > All you dream as impossible is now possible. > > 1-Multiple server connections from the same client. > > 2-Real-time message passing (obviously thx to Erlang) between the C# > client and Erlang server. > > 3-Request abort feature (a request aborted in the client, abort its > resolver process in the server). > > 4-Multiple Sync and Async request at the same time to different servers > form the same client. > > 5-Unknown Incoming messages treating in the client (the server can send > massive messages to all the connected clients, useful for games server > and chats), > > only using a little function like send_to_all(Description, Message) or > send_to_others(Description, Message, ExceptionPid). > > 6-Other features like Abort a waiting for a reply request from another > threat. A server disconnection makes all Waiting Requests to release and > return null. > > 7-You can send objects of any kind to the server and to be received in > other clients (like classes, images etc.) > > 8-You can make Sync requests without any fear of getting locks because > if the answer for the request arrives before you call the WaitForReply > function this function will throw an Error telling you that the request > was already answered an received through the OnReceive Event of the > async way. > > Example: > > IRequestInfo reqinfo = requestHanlder.Request(?Hola mundo?); > > Thread.Sleep(1000); // if the answer arrives before, through the > OnReceive event because this fool! Then the next sentence will throw an > error > > reqinfo.WaitForReply(); or reqinfo.WaitForReply(5000); > > You will never get locks using Sync requests. And if you are not sure > use a timeout. > > The same way you can abort the request using reqinfo object: > reqinfo.AbortRequest(); or know if the request was already answered or > made using its properties. > > You can do many stuffs using IServerReply and IRequestInfo objects, > example: using IServerReply object you can use a property to convert the > Erlang terms to CSharp terms and vice versa. > > 9-You can program clients as server-client model of the most easy way > (only if you don?t want to write any code in Erlang, just in C#) you can > configure de clients > > and the server to act as intermediary between the clients of so easy > way. Using the informing message that send the server to all the clients > when a new client is connected or disconnected, sending if you want, its > pid to all the other clients, then you can get that pid to send a > message using the server as intermediary ?directly? to that other client > or register that client in the server as a server prototype, this make > possible to program the client-server model with the clients without > write any line of code in Erlang. The automatic replier that brings with > the framework ExtendedVisualOtp in the server part do all the dirty work > for you. > > 10-You can send whatever you have been programming to your friends who > are programming under ExtendedVisualOtp so, with a simple call to the > function > > PublishClientImage(); of the ErlangServerRequest class. Then your > program image in runtime it will be executed at the machines of your > friends who are connected to the > > server so. Then if you are developing a chat, you can send the image of > your program while your friend is programming other chat or program and > see your creation, at the > > same time he can send to you his creation. > > The most powerful interface between languages ever known > ExtendedVisualOtp. That and so much more EVO Release 1.6 > > Little examples: > > the C# part: > > erlangServerInterface1.Connect(); > > erlangServerRequest1.Request(new object[] {??insert_person??, ?Ivan?, > 26, ?Male?}); > > the Erlang part: > > {?insert_person??, Name, Age, Sex}, Pid, RequestInfo}-> > > . > > Pid ! {reply, ?inserted_ok?, RequestInfo}; %% if you want to > answer to the client who made the request. > > To receive the answer in the client > > If you use the sync way, add .WaitForReply().CSharpReply at the end > of the request line before > > And if you want the async way just use the OnReceive event > > OnReceiveEvent..(IServerReply _reply) > > { > > If (_/reply.Description == ?insert/_person?) { > > MessageBox.Show(_reply.CSharpReply as string); > > } > > } > > As simple like it seems and powerful as it could not seems. > > *--------------------------------------------------------------------------- ----------------------------------* > > *Ing: Ivan Carmenates Garc?a* > > Current project: ExtendedVisualOtp and Erlang Book: "Erlang Avanzado > para principiantes" > > Ivanco Software Company in association with SPI Team > > spi_red32LogoSPI > > > ======================================================================= > Este mensaje ha sido enviado mediante el servicio de correo electr?nico > que ofrece la Federaci?n de Radioaficionados de Cuba a sus miembros para > respaldar el cumplimiento de los objetivos de la organizaci?n y su > pol?tica informativa. La persona que env?a este correo asume el > compromiso de usar el servicio a tales fines y cumplir con las > regulaciones establecidas. ======================================================================= Este mensaje ha sido enviado mediante el servicio de correo electr?nico que ofrece la Federaci?n de Radioaficionados de Cuba a sus miembros para respaldar el cumplimiento de los objetivos de la organizaci?n y su pol?tica informativa. La persona que env?a este correo asume el compromiso de usar el servicio a tales fines y cumplir con las regulaciones establecidas. From kuleshovmail@REDACTED Wed Jan 19 07:48:54 2011 From: kuleshovmail@REDACTED (shk) Date: Tue, 18 Jan 2011 22:48:54 -0800 (PST) Subject: string:strip/3 don't work right striping Message-ID: <1295419734190-3224632.post@n4.nabble.com> Hello, I receive data by socket in tcp: receive_loop(Socket) -> case gen_tcp:recv(Socket, 0) of {ok, Data} -> ReParseData = string:strip(string:to_lower(Data), both, $.), io:format(ReParseData, []), case ReParseData of "quit\r\n" -> gen_tcp:send(Socket, "quit \r\n"), gen_tcp:close(Socket); _ -> io:format(Data, []), receive_loop(Socket) end; {error, closed} -> ok end. if I send .........................Hello ...................................... I got in shell: >Hello ...................................... How can i correctly delete space from left and right in stirng? Thank you. -- View this message in context: http://erlang.2086793.n4.nabble.com/string-strip-3-don-t-work-right-striping-tp3224632p3224632.html Sent from the Erlang Questions mailing list archive at Nabble.com. From b.ghose@REDACTED Wed Jan 19 08:04:43 2011 From: b.ghose@REDACTED (Baishampayan Ghose) Date: Wed, 19 Jan 2011 12:34:43 +0530 Subject: [erlang-questions] Erlang project dependency management/build tools In-Reply-To: References: Message-ID: Martin, > if you want to use faxien to install application dependencies into > totally self directed locations try this > > faxien fetch-app [app-vsn] > > for example > > faxien fetch-app ibrowse $HOME/my_erlang_lib > > Does this help? Thanks, that works as expected. Now to the next step. Is there any way for Sinan to invoke Faxien with a custom directory directive (just like you suggested now)? I would prefer to have a config file with just the dependencies listed and ideally Sinan should just invoke Faxien to bring in those dependencies into a predetermined directory inside my code-base. That would vastly improve my workflow. In this respect I like the way how Rebar works, it lets me declare my dependencies in the config file and it brings them in nicely into my project directory. Regards, BG -- Baishampayan Ghose b.ghose at gmail.com From b.ghose@REDACTED Wed Jan 19 08:07:24 2011 From: b.ghose@REDACTED (Baishampayan Ghose) Date: Wed, 19 Jan 2011 12:37:24 +0530 Subject: [erlang-questions] Erlang project dependency management/build tools In-Reply-To: References: Message-ID: > For my own education, and to improve our products, why is the global > location an issue? OTP applications are all versioned and if you are > pulling them from a repo then there shouldn't be any problem with > multiple projects using the same local store. It's easier to have all your dependencies bundled with the release. That removes the necessity of root access on the server as well as having sinan/faxien itself installed on the server. It vastly simplifies deployment on a cloud infrastructure like Amazon EC2 where you'd like to start up machines on demand and deploy your Erlang applications. This is akin to having a statically linked executable :) Regards, BG -- Baishampayan Ghose b.ghose at gmail.com From gustav.simonsson@REDACTED Wed Jan 19 08:57:11 2011 From: gustav.simonsson@REDACTED (Gustav Simonsson) Date: Wed, 19 Jan 2011 07:57:11 +0000 (GMT) Subject: [erlang-questions] string:strip/3 don't work right striping In-Reply-To: <251575883.305041295423797559.JavaMail.root@zimbra> Message-ID: <1483092956.305061295423831877.JavaMail.root@zimbra> string:strip(string:to_lower("....Hello...."), both, $.). will evaluate to "hello". The reason you received dots in your shell might be because you print out the raw Data after you print out ReParseData, without anything printed between. Best Regards, Gustav Simonsson ----- Original Message ----- From: "shk" To: erlang-questions@REDACTED Sent: Wednesday, 19 January, 2011 07:48:54 GMT +01:00 Amsterdam / Berlin / Bern / Rome / Stockholm / Vienna Subject: [erlang-questions] string:strip/3 don't work right striping Hello, I receive data by socket in tcp: receive_loop(Socket) -> case gen_tcp:recv(Socket, 0) of {ok, Data} -> ReParseData = string:strip(string:to_lower(Data), both, $.), io:format(ReParseData, []), case ReParseData of "quit\r\n" -> gen_tcp:send(Socket, "quit \r\n"), gen_tcp:close(Socket); _ -> io:format(Data, []), receive_loop(Socket) end; {error, closed} -> ok end. if I send .........................Hello ...................................... I got in shell: >Hello ...................................... How can i correctly delete space from left and right in stirng? Thank you. -- View this message in context: http://erlang.2086793.n4.nabble.com/string-strip-3-don-t-work-right-striping-tp3224632p3224632.html Sent from the Erlang Questions mailing list archive at Nabble.com. ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From bengt.kleberg@REDACTED Wed Jan 19 09:00:31 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Wed, 19 Jan 2011 09:00:31 +0100 Subject: [erlang-questions] string:strip/3 don't work right striping In-Reply-To: <1295419734190-3224632.post@n4.nabble.com> References: <1295419734190-3224632.post@n4.nabble.com> Message-ID: <1295424031.5407.7.camel@seasc1137> Greetings, While I can not see all space that you want to delete in the input (it would be helpful if you provided " at the beginning and the end) I can guess that you have two \n at the end of each line. If you want to delete any leading and/or trailing blanks (space) then string:strip/1 would be a good choice. Given a better description of the input, and what you want to do with, it might result in a better suggestion. bengt On Wed, 2011-01-19 at 07:48 +0100, shk wrote: > Hello, > > I receive data by socket in tcp: > > receive_loop(Socket) -> > case gen_tcp:recv(Socket, 0) of > {ok, Data} -> > ReParseData = string:strip(string:to_lower(Data), both, $.), > io:format(ReParseData, []), > case ReParseData of > "quit\r\n" -> > gen_tcp:send(Socket, "quit \r\n"), > gen_tcp:close(Socket); > _ -> > io:format(Data, []), > receive_loop(Socket) > end; > {error, closed} -> > ok > end. > > if I send .........................Hello > ...................................... > I got in shell: > >Hello ...................................... > > How can i correctly delete space from left and right in stirng? > > Thank you. From alexey.v.romanov@REDACTED Wed Jan 19 10:08:20 2011 From: alexey.v.romanov@REDACTED (Alexey Romanov) Date: Wed, 19 Jan 2011 12:08:20 +0300 Subject: [erlang-questions] Passing arguments to valgrind via cerl -valgrind In-Reply-To: <4D35BC12.5020300@erix.ericsson.se> References: <4D35BC12.5020300@erix.ericsson.se> Message-ID: Yes, I expect that should work. Thank you! Yours, Alexey Romanov 2011/1/18 Bj?rn-Egil Dahlberg : > I think you can use VALGRIND_MISC_FLAGS, > > VALGRIND_MISC_FLAGS=--leak-check=full cerl -valgrind > > check cerl to see exactly > > // Bj?rn-Egil > ? Erlang/OTP > > > On 2011-01-18 15:10, Alexey Romanov wrote: >> >> Is it possible to pass additional arguments to valgrind (e.g. >> --leak-check=full) when running a debug-enabled runtime system >> (http://www.erlang.org/doc/installation_guide/INSTALL.html#id68569)? >> >> Yours, Alexey Romanov >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From jocke@REDACTED Wed Jan 19 10:23:56 2011 From: jocke@REDACTED (Joakim G.) Date: Wed, 19 Jan 2011 10:23:56 +0100 Subject: public_key:pem_decode/1 and public_key:verify/4 Message-ID: <4D36ADAC.3070509@tail-f.com> Hi all, It seems that public_key:pem_decode/1 can't decode public keys. I wanted use public_key:verify/4 to verify a digital signature and for that I used public_key:pem_decode/1 to extract the public key. 26> {ok, Bin} = file:read_file("key.pub"). {ok,<<"-----BEGIN RSA PUBLIC KEY-----\nMIGJAoGBAKhhZ8RrKaA0RWhjXJlSpTHd7/h6Luww9qeyUkEqBQyiCooZZGCYNle5C9TfRVJDUuxz"...>>} 27> rp(Bin). <<"-----BEGIN RSA PUBLIC KEY-----\nMIGJAoGBAKhhZ8RrKaA0RWhjXJlSpTHd7/h6Luww9qeyUkEqBQyiCooZZGCYNle5\nC9TfRVJDUuxzigvXuTh0tlba+9AyIe2hxAztjs43pSKy5VnnJlSjMBVDay8XmNjh\nlUxid/ZV2hDkKlwHDCnZjdZpeXqdtGbZ7o3N7wOBzJSRQ7kqzQHXAgMBAAE=\n-----END RSA PUBLIC KEY-----\n">> ok 28> public_key:pem_decode(Bin). [] That was a surprise. Cheers /Jocke From caox@REDACTED Wed Jan 19 10:49:57 2011 From: caox@REDACTED (caox) Date: Wed, 19 Jan 2011 17:49:57 +0800 Subject: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 In-Reply-To: <4D36ADAC.3070509@tail-f.com> References: <4D36ADAC.3070509@tail-f.com> Message-ID: <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> Maybe the asn1 type of your key file is not supported by public_key. You can check it and use openssl to convert the key format. ? 2011-1-19???5:23? Joakim G. ??? > Hi all, > It seems that public_key:pem_decode/1 can't decode public > keys. > > I wanted use public_key:verify/4 to verify a digital signature > and for that I used public_key:pem_decode/1 to extract the > public key. > > 26> {ok, Bin} = file:read_file("key.pub"). > {ok,<<"-----BEGIN RSA PUBLIC > KEY-----\nMIGJAoGBAKhhZ8RrKaA0RWhjXJlSpTHd7/h6Luww9qeyUkEqBQyiCooZZGCYNle5C9TfRVJDUuxz"...>>} > 27> rp(Bin). > <<"-----BEGIN RSA PUBLIC > KEY-----\nMIGJAoGBAKhhZ8RrKaA0RWhjXJlSpTHd7/h6Luww9qeyUkEqBQyiCooZZGCYNle5\nC9TfRVJDUuxzigvXuTh0tlba+9AyIe2hxAztjs43pSKy5VnnJlSjMBVDay8XmNjh\nlUxid/ZV2hDkKlwHDCnZjdZpeXqdtGbZ7o3N7wOBzJSRQ7kqzQHXAgMBAAE=\n-----END > RSA PUBLIC KEY-----\n">> > ok > 28> public_key:pem_decode(Bin). > [] > > That was a surprise. > > Cheers > /Jocke > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From axling@REDACTED Wed Jan 19 11:38:24 2011 From: axling@REDACTED (Erik Axling) Date: Wed, 19 Jan 2011 11:38:24 +0100 Subject: Can't get a resource in a nif Message-ID: Hello! I'm trying to write a NIF for the Fast Artificial Neural Network library that is written in C. But I'm running into a problem where I use a resource to allocate a pointer for a network that I have created. When I try to get that resource it fails for an unknown reason. I have used printfs to be able to know where the error occurs. My NIF returns a badarg. The C code is here: https://gist.github.com/785854 Thanks for any help. /Erik Axling From sverker@REDACTED Wed Jan 19 11:53:56 2011 From: sverker@REDACTED (Sverker Eriksson) Date: Wed, 19 Jan 2011 11:53:56 +0100 Subject: [erlang-questions] Can't get a resource in a nif In-Reply-To: References: Message-ID: <4D36C2C4.2060405@erix.ericsson.se> Erik Axling wrote: > Hello! > > I'm trying to write a NIF for the Fast Artificial Neural Network library > that is written in C. But I'm running into a problem where I use a resource > to allocate a pointer for a network that I have created. When I try to get > that resource it fails for an unknown reason. I have used printfs to be > able to know where the error occurs. My NIF returns a badarg. > The C code is here: https://gist.github.com/785854 > > Thanks for any help. > /Erik Axling > > Just after a quick glance: static ERL_NIF_TERM create_standard_nif(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]) { int x, ret, tuple_size; const ERL_NIF_TERM ** tuple_array; unsigned int * converted_array; struct fann * ann; ERL_NIF_TERM result; ann = enif_alloc_resource(FANN_POINTER, sizeof(struct fann *)); if(enif_get_tuple(env, argv[0], &tuple_size, tuple_array)) { converted_array = malloc(tuple_size*sizeof(unsigned int)); if(check_and_convert_array(env, tuple_array, tuple_size, converted_array)) { ann = fann_create_standard_array(tuple_size, converted_array); You assign the resource pointer to 'ann' and then you assign the result of fann_create_standard_array() to 'ann'. Maybe you ment to say *ann = fann_create... /Sverker, Erlang/OTP From watson.timothy@REDACTED Wed Jan 19 12:35:30 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Wed, 19 Jan 2011 11:35:30 +0000 Subject: [erlang-questions] Erlang / C# The Best Interface In-Reply-To: <000c01cbb771$ce15e540$6a41afc0$@co.cu> References: <002301cbb731$f5b5bd40$e12137c0$@co.cu> <000c01cbb771$ce15e540$6a41afc0$@co.cu> Message-ID: > So for the first case you should do Request(new object[] { > ??insert_person??, ?Ivan?, ?26?});?? %% new object[]? means new tuple? { } > > So if you use Request(params object[])?? -> Request(??insert_person??, > ?Ivan?, ?26?) to make a tuple then how you send the simple ?logoff? atom? > How about this: Request(params ErlangObject Input) { // do the rpc } static ErlangTuple Tuple(this Object self, params object[] argv) { return new ErlangTuple(argv); } static ErlangAtom Atom(this Object self, string s) { return new ErlangAtom(s); } // etc And then you could write the call site like so (accounting for the fancy pants static import-alike in .NET): var result = req.Request(Atom("i_like_foo")); // or var result = req.Request(Tuple(Atom("insert_person"), "Ivan", 26)); // or var result = req.Request(Tuple(Atom("insert_person"), List("Ivan", 26))); Where strings and numbers and other .NET value types are handled automatically but tuples, lists and atoms take a constructor and are a wrapper type. Just a thought! Cheers, Tim From axling@REDACTED Wed Jan 19 13:40:19 2011 From: axling@REDACTED (Erik Axling) Date: Wed, 19 Jan 2011 13:40:19 +0100 Subject: [erlang-questions] Can't get a resource in a nif In-Reply-To: <4D36CB44.1080006@erix.ericsson.se> References: <4D36CB44.1080006@erix.ericsson.se> Message-ID: Actually the fann_create_standard_array(tuple_size, converted_array); returns a struct fann *. I've been working with erlang for too long so my C knowledge has totally deteriorated so I might be missing something basic. I also feel that the resource concept is a bit fuzzy so I might be doing it wrong. I figured that the resource pointer that I allocated pointed to some memory where I could in turn point the newly created network from fann_create_standard_array? I fixed the tuple_array but I am still getting the same error. I am compiling with gcc -shared -fpic -l fann -o fann_nif.so fann_nif.c -I $ERL_ROOT/usr/include. >From running erl: bash-3.2# erl Erlang R14B01 (erts-5.8.2) [source] [rq:1] [async-threads:0] [kernel-poll:false] Eshell V5.8.2 (abort with ^G) 1> c(fann). {ok,fann} 2> A=fann:create_standard({2,2,1}). <<>> 3> fann:train_on_file(A, "xor.data", 20, 3, 0.5). Before resource ** exception error: bad argument in function fann:train_on_file/5 called as fann:train_on_file(<<>>,"xor.data",20,3,0.5) 4> On Wed, Jan 19, 2011 at 12:30 PM, Sverker Eriksson wrote: > Erik Axling wrote: > >> Hello! >> >> I'm trying to write a NIF for the Fast Artificial Neural Network library >> that is written in C. But I'm running into a problem where I use a >> resource >> to allocate a pointer for a network that I have created. When I try to get >> that resource it fails for an unknown reason. I have used printfs to be >> able to know where the error occurs. My NIF returns a badarg. >> The C code is here: https://gist.github.com/785854 >> >> Thanks for any help. >> /Erik Axling >> >> >> > (Seems the dingo ate the new-lines in my first reply) > > > After a second glance I see you use your 'tuple_array' wrongly. > > You should define it as > > > const ERL_NIF_TERM * tuple_array; > > and then use &tuple_array as argument to enif_get_tuple(). It's an > "out-argument" just like &tuple_size. > > > /Sverker > > From ingela@REDACTED Wed Jan 19 14:21:53 2011 From: ingela@REDACTED (Ingela Andin) Date: Wed, 19 Jan 2011 14:21:53 +0100 Subject: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 In-Reply-To: <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> References: <4D36ADAC.3070509@tail-f.com> <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> Message-ID: Hi! All possible PEM-types are not supported by public_key, the currently supported types are mainly a result of what we need for the ssl-application. More types may of course be added if there is a need. You are welcome to contribute PEM-types that may be of common use. If you want the public part of your own rsa-key it is included in the private-key and you can decode that file with public_key and pick out the public part. Regards Ingela Erlang/OTP team - Ericsson AB 2011/1/19 caox : > Maybe the asn1 type of your key file is not supported by public_key. You can check it and use openssl to convert the key format. > > ? 2011-1-19???5:23? Joakim G. ??? > >> Hi all, >> It seems that public_key:pem_decode/1 can't decode public >> keys. >> >> I wanted use public_key:verify/4 to verify a digital signature >> and for that I used public_key:pem_decode/1 to extract the >> public key. >> >> 26> {ok, Bin} = file:read_file("key.pub"). >> {ok,<<"-----BEGIN RSA PUBLIC >> KEY-----\nMIGJAoGBAKhhZ8RrKaA0RWhjXJlSpTHd7/h6Luww9qeyUkEqBQyiCooZZGCYNle5C9TfRVJDUuxz"...>>} >> 27> rp(Bin). >> <<"-----BEGIN RSA PUBLIC >> KEY-----\nMIGJAoGBAKhhZ8RrKaA0RWhjXJlSpTHd7/h6Luww9qeyUkEqBQyiCooZZGCYNle5\nC9TfRVJDUuxzigvXuTh0tlba+9AyIe2hxAztjs43pSKy5VnnJlSjMBVDay8XmNjh\nlUxid/ZV2hDkKlwHDCnZjdZpeXqdtGbZ7o3N7wOBzJSRQ7kqzQHXAgMBAAE=\n-----END >> RSA PUBLIC KEY-----\n">> >> ok >> 28> public_key:pem_decode(Bin). >> [] >> >> That was a surprise. >> >> Cheers >> /Jocke >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From atessaro@REDACTED Wed Jan 19 14:31:47 2011 From: atessaro@REDACTED (Tessaro Alexej) Date: Wed, 19 Jan 2011 14:31:47 +0100 Subject: Mnesia Fragmented Tables, del_table_copy Message-ID: <1295443907.20811.23.camel@texhp> Hello, I have started two communicating mnesia nodes and created one fragmented table with fragments stored in RAM only (no other replicas) and schema stored on disk on both nodes: node1 : constResourceData_frag3 (ram), constResourceData_frag4 (ram), schema (disk) node2 : constResourceData (ram), constResourceData_frag2 (ram), schema (disk) Now, from node1, I tried to call mnesia:del_table_copy(constResourceData_frag2, node2) but it fails saying: node1> Mnesia(node1): Last replica deleted in table constResourceData_frag2 node1>{aborted,{no_exists,constResourceData_frag2,frag_properties, frag_hash}} node1> Mnesia(node1): Transaction {tid,75,<0.371.0>} calling #Fun with [] failed: {aborted,{no_exists,constResourceData_frag2,frag_properties,frag_hash}} Looking at the mnesia_schema.erl source code, I found that the failure seems to happen when it tries to delete the "whole_table" in make_delete_table/2 and it runs the following check: %% Check if it is a base table nesia_frag:lookup_frag_hash(Tab), Why is my attempt of removing the fragment failing? My erlang version is R12B. Thank you From b.ghose@REDACTED Wed Jan 19 15:38:05 2011 From: b.ghose@REDACTED (Baishampayan Ghose) Date: Wed, 19 Jan 2011 20:08:05 +0530 Subject: [erlang-questions] Erlang project dependency management/build tools In-Reply-To: References: Message-ID: > Sinan does this bundling already :) If you run 'sinan dist' in your > project will end up with a tarball in > _build/developement/tar/-.tar.gz. This is a fully > deployable otp tarball that contains all of your dependencies and > works with all of the otp tools. It doesn't actually matter where your > dependencies are sinan will package them up for you when you run that > 'dist' task. I may need to make that a bit more obvious in the > documentation, there a couple of things in the docs now that I think I > can make more clear. Thanks! That's awesome! Another (potentially stupid) question, how does Sinan discover my dependencies? Does it read any configuration file or does it discover those magically? Also, if I download my dependencies into project/deps folder via Faxien, will Sinan be able to build my project? Regards, BG -- Baishampayan Ghose b.ghose at gmail.com From jocke@REDACTED Wed Jan 19 16:42:48 2011 From: jocke@REDACTED (Joakim G.) Date: Wed, 19 Jan 2011 16:42:48 +0100 Subject: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 In-Reply-To: References: <4D36ADAC.3070509@tail-f.com> <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> Message-ID: <4D370678.60203@tail-f.com> Thanks, I see. I'm using keys as defined in: https://gitweb.torproject.org/tor.git?a=blob_plain;hb=HEAD;f=doc/spec/tor-spec.txt That is: "For a public-key cipher, we use RSA with 1024-bit keys and a fixed exponent of 65537. We use OAEP-MGF1 padding, with SHA-1 as its digest function. We leave the optional "Label" parameter unset. (For OAEP padding, see ftp://ftp.rsasecurity.com/pub/pkcs/pkcs-1/pkcs-1v2-1.pdf)" I only have the public keys to work with. Not a surprise. :-) I will see if I can find more info on this. Cheers /Jocke On 2011-01-19 14:21, Ingela Andin wrote: > Hi! > > All possible PEM-types are not supported by public_key, the currently > supported types are mainly a result of what we need > for the ssl-application. More types may of course be added if there is > a need. You are welcome to contribute PEM-types that may be of common > use. > > If you want the public part of your own rsa-key it is included in the > private-key and you can decode that file with public_key and > pick out the public part. > > Regards Ingela Erlang/OTP team - Ericsson AB > > > 2011/1/19 caox : >> Maybe the asn1 type of your key file is not supported by public_key. You can check it and use openssl to convert the key format. >> >> ? 2011-1-19???5:23? Joakim G. ??? >> >>> Hi all, >>> It seems that public_key:pem_decode/1 can't decode public >>> keys. >>> >>> I wanted use public_key:verify/4 to verify a digital signature >>> and for that I used public_key:pem_decode/1 to extract the >>> public key. >>> >>> 26> {ok, Bin} = file:read_file("key.pub"). >>> {ok,<<"-----BEGIN RSA PUBLIC >>> KEY-----\nMIGJAoGBAKhhZ8RrKaA0RWhjXJlSpTHd7/h6Luww9qeyUkEqBQyiCooZZGCYNle5C9TfRVJDUuxz"...>>} >>> 27> rp(Bin). >>> <<"-----BEGIN RSA PUBLIC >>> KEY-----\nMIGJAoGBAKhhZ8RrKaA0RWhjXJlSpTHd7/h6Luww9qeyUkEqBQyiCooZZGCYNle5\nC9TfRVJDUuxzigvXuTh0tlba+9AyIe2hxAztjs43pSKy5VnnJlSjMBVDay8XmNjh\nlUxid/ZV2hDkKlwHDCnZjdZpeXqdtGbZ7o3N7wOBzJSRQ7kqzQHXAgMBAAE=\n-----END >>> RSA PUBLIC KEY-----\n">> >>> ok >>> 28> public_key:pem_decode(Bin). >>> [] >>> >>> That was a surprise. >>> >>> Cheers >>> /Jocke >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From b.ghose@REDACTED Wed Jan 19 17:05:49 2011 From: b.ghose@REDACTED (Baishampayan Ghose) Date: Wed, 19 Jan 2011 21:35:49 +0530 Subject: [erlang-questions] Erlang project dependency management/build tools In-Reply-To: References: Message-ID: Eric, > Dependencies have to be declared in the *.app files of your OTP > Applications according to the OTP standard. Sinan uses this > information to build a list of dependencies ?for your project, and > resolve that to a the actual versioned dependencies. Right now it just > takes the newest version of the application available to it. This > works in most cases but sometimes you need more control, thats a > project I am working on right now. The ability to have finer grained > control over your dependency versions. That should come along here in > the next couple of weeks. Makes a lot of sense. I am going to use Sinan for my project now and I will let you know if I face any issues. Thanks a lot! Regards, BG -- Baishampayan Ghose b.ghose at gmail.com From b.ghose@REDACTED Wed Jan 19 17:41:44 2011 From: b.ghose@REDACTED (Baishampayan Ghose) Date: Wed, 19 Jan 2011 22:11:44 +0530 Subject: [erlang-questions] Erlang project dependency management/build tools In-Reply-To: References: Message-ID: Hi, > That said, let me know if you run into problems. I am currently pretty > active in the sinan code and can make changes. What is the best way to install packages which are not available in the erlware repository? I wanted to install emongo using Faxien, but it's not available. Regards, BG -- Baishampayan Ghose b.ghose at gmail.com From info@REDACTED Wed Jan 19 18:46:17 2011 From: info@REDACTED (info) Date: Wed, 19 Jan 2011 18:46:17 +0100 Subject: record updating with several functions Message-ID: <201101191846160683077@its3.ch> Hello, Given one record with multiple fields. Each fields will be updated by executing a lot of functions. How to pick up the fields in the main program ? Shall I pass as parameter the record ? probably yes. How to have the new state of the record after each calls ? -record (fields,{ch1,ch2,...}). main() -> fctA, %set ch1 fctB, %set ch2... display the field ch1,ch2,... . Other case: the function are cascaded (fctA call fctB which call fctC etc ...) -record (fields,{ch1,ch2,...}). main() -> fctA, %set ch1 ... display the field ch1,ch2,... . fctA()-> ... %set ch1 fctB() . fctB()-> ... % set ch2 fctC() . J-Ph. Constantin ITS3 Gen?ve www.its3.ch From ciprian.craciun@REDACTED Wed Jan 19 19:16:43 2011 From: ciprian.craciun@REDACTED (Ciprian Dorin Craciun) Date: Wed, 19 Jan 2011 20:16:43 +0200 Subject: Erlang static linked interpreter Message-ID: Hello all! I was just wondering if anyone was able to build a static linked Erlang interpreter? I've tried today to use `LDFLAGS=-static`, but unfortunately when linking it complains of finding `__sigaction` duplicated, once in Erlang's own source code, and once in `pthread` library. Any ideas? Thanks, Ciprian. From alain.odea@REDACTED Wed Jan 19 19:40:52 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Wed, 19 Jan 2011 15:10:52 -0330 Subject: [erlang-questions] record updating with several functions In-Reply-To: <201101191846160683077@its3.ch> References: <201101191846160683077@its3.ch> Message-ID: <781D5239-FFA3-497F-AA2D-04F197498F82@gmail.com> On 2011-01-19, at 14:16, "info" wrote: > Hello, > Given one record with multiple fields. > Each fields will be updated by executing a lot of functions. > How to pick up the fields in the main program ? > Shall I pass as parameter the record ? probably yes. > How to have the new state of the record after each calls ? > > -record (fields,{ch1,ch2,...}). > main() -> > fctA, %set ch1 > fctB, %set ch2... > display the field ch1,ch2,... > . > > Other case: the function are cascaded (fctA call fctB which call fctC etc ...) > > -record (fields,{ch1,ch2,...}). > main() -> > fctA, %set ch1 > ... > display the field ch1,ch2,... > . > fctA()-> > ... %set ch1 > fctB() > . > fctB()-> > ... % set ch2 > fctC() > . > > J-Ph. Constantin > ITS3 Gen?ve > www.its3.ch Listing of set_record_fields.erl: -module(set_record_fields). -record (fields,{ch1,ch2,...}). main() -> Record = fctB(fctA(#fields{})), io:format("~w~n" "ch1 is ~w~n" "ch2 is ~w~n", Record, Record#fields.ch1, Record#fields.ch2). fctA(Record) -> Record#fields{ch1 = "test ch1"}. fctB(Record) -> Record#fields{ch2 = "test ch2"}. That should work (untested, my apologies if it doesn't). I wrote it from bed on my iPhone :) From jocke@REDACTED Wed Jan 19 20:20:46 2011 From: jocke@REDACTED (=?utf-8?B?am9ja2VAdGFpbC1mLmNvbQ==?=) Date: Wed, 19 Jan 2011 20:20:46 +0100 Subject: =?utf-8?B?QW5nLjogW2VybGFuZy1xdWVzdGlvbnNdIHB1YmxpY19rZXk6cGVtX2RlY29kZS8xIGFuZCBwdWJsaWNfa2V5OnZlcmlmeS80?= Message-ID: <20110119192048.DFBC776C272@mail.tail-f.com> The standard doesn't say anything about the PEM format itself. ----- Reply message ----- Fr?n: "Joakim G." Datum: ons, jan 19, 2011 16:42 Rubrik: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 Till: Thanks, I see. I'm using keys as defined in: https://gitweb.torproject.org/tor.git?a=blob_plain;hb=HEAD;f=doc/spec/tor-spec.txt That is: "For a public-key cipher, we use RSA with 1024-bit keys and a fixed exponent of 65537. We use OAEP-MGF1 padding, with SHA-1 as its digest function. We leave the optional "Label" parameter unset. (For OAEP padding, see ftp://ftp.rsasecurity.com/pub/pkcs/pkcs-1/pkcs-1v2-1.pdf)" I only have the public keys to work with. Not a surprise. :-) I will see if I can find more info on this. Cheers /Jocke On 2011-01-19 14:21, Ingela Andin wrote: > Hi! > > All possible PEM-types are not supported by public_key, the currently > supported types are mainly a result of what we need > for the ssl-application. More types may of course be added if there is > a need. You are welcome to contribute PEM-types that may be of common > use. > > If you want the public part of your own rsa-key it is included in the > private-key and you can decode that file with public_key and > pick out the public part. > > Regards Ingela Erlang/OTP team - Ericsson AB > > > 2011/1/19 caox : >> Maybe the asn1 type of your key file is not supported by public_key. You can check it and use openssl to convert the key format. >> >> ? 2011-1-19???5:23? Joakim G. ??? >> >>> Hi all, >>> It seems that public_key:pem_decode/1 can't decode public >>> keys. >>> >>> I wanted use public_key:verify/4 to verify a digital signature >>> and for that I used public_key:pem_decode/1 to extract the >>> public key. >>> >>> 26> {ok, Bin} = file:read_file("key.pub"). >>> {ok,<<"-----BEGIN RSA PUBLIC >>> KEY-----\nMIGJAoGBAKhhZ8RrKaA0RWhjXJlSpTHd7/h6Luww9qeyUkEqBQyiCooZZGCYNle5C9TfRVJDUuxz"...>>} >>> 27> rp(Bin). >>> <<"-----BEGIN RSA PUBLIC >>> KEY-----\nMIGJAoGBAKhhZ8RrKaA0RWhjXJlSpTHd7/h6Luww9qeyUkEqBQyiCooZZGCYNle5\nC9TfRVJDUuxzigvXuTh0tlba+9AyIe2hxAztjs43pSKy5VnnJlSjMBVDay8XmNjh\nlUxid/ZV2hDkKlwHDCnZjdZpeXqdtGbZ7o3N7wOBzJSRQ7kqzQHXAgMBAAE=\n-----END >>> RSA PUBLIC KEY-----\n">> >>> ok >>> 28> public_key:pem_decode(Bin). >>> [] >>> >>> That was a surprise. >>> >>> Cheers >>> /Jocke >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From mikpe@REDACTED Wed Jan 19 20:57:49 2011 From: mikpe@REDACTED (Mikael Pettersson) Date: Wed, 19 Jan 2011 20:57:49 +0100 Subject: [erlang-questions] Erlang static linked interpreter In-Reply-To: References: Message-ID: <19767.16957.813160.505530@pilspetsen.it.uu.se> Ciprian Dorin Craciun writes: > Hello all! > > I was just wondering if anyone was able to build a static linked > Erlang interpreter? > > I've tried today to use `LDFLAGS=-static`, but unfortunately when > linking it complains of finding `__sigaction` duplicated, once in > Erlang's own source code, and once in `pthread` library. > > Any ideas? Thanks, If you're on x86, then disable HiPE. HiPE on x86 needs to override sigaction(), but that only works with dynamic linkage. From ciprian.craciun@REDACTED Wed Jan 19 21:10:19 2011 From: ciprian.craciun@REDACTED (Ciprian Dorin Craciun) Date: Wed, 19 Jan 2011 22:10:19 +0200 Subject: [erlang-questions] Erlang static linked interpreter In-Reply-To: <19767.16957.813160.505530@pilspetsen.it.uu.se> References: <19767.16957.813160.505530@pilspetsen.it.uu.se> Message-ID: On Wed, Jan 19, 2011 at 21:57, Mikael Pettersson wrote: > Ciprian Dorin Craciun writes: > ?> ? ? Hello all! > ?> > ?> ? ? I was just wondering if anyone was able to build a static linked > ?> Erlang interpreter? > ?> > ?> ? ? I've tried today to use `LDFLAGS=-static`, but unfortunately when > ?> linking it complains of finding `__sigaction` duplicated, once in > ?> Erlang's own source code, and once in `pthread` library. > ?> > ?> ? ? Any ideas? Thanks, > > If you're on x86, then disable HiPE. ?HiPE on x86 needs to override > sigaction(), but that only works with dynamic linkage. But if disabling HiPE, I'll have a performance penalty, or? (BTW what is the performance gain of using HiPE?) And I'm a little bit confused about overriding `sigaction`. Does the code inside Erlang call `sigaction` -- in which case then these references could be renamed -- or there is a need to subvert the `sigaction` calls in other system libraries. (Isn't there an option for GCC to override a symbol?) From taylor@REDACTED Thu Jan 20 00:03:56 2011 From: taylor@REDACTED (Taylor Venable) Date: Wed, 19 Jan 2011 18:03:56 -0500 Subject: Functions with same name and varying arity using apply Message-ID: Hi there, is the following possible in Erlang? If I use the following code: %% test.erl -module(test). -export([foo/1,foo/2,bar/2]). foo(A) -> A. foo(A, B) -> A + B. bar(_, []) -> []; bar(F, [X|Xs]) -> [apply(test, F, X) | bar(F, Xs)]. %% END 2> test:bar(foo, [[1], [2,3]]). [1,5] Is there a way to do the same thing without exporting foo/1 and foo/2? It seems not with apply/3, but maybe some technique with apply/2, or another way? Thanks for any ideas. -- Taylor C. Venable http://metasyntax.net/ From dirk.scharff@REDACTED Thu Jan 20 00:31:33 2011 From: dirk.scharff@REDACTED (Dirk Scharff) Date: Thu, 20 Jan 2011 00:31:33 +0100 Subject: [erlang-questions] Functions with same name and varying arity using apply In-Reply-To: References: Message-ID: <77F30428-2E39-43D4-B49E-BB4D598E0801@googlemail.com> Hi Taylor, I don't get what you want to do. From your example I assume that you want to apply a function (in your case addition) to each element of a list? You could to the same calculation as follows: lists:map(fun(X)->lists:sum(X) end,[[1],[2,3]]). However I'm not sure if I got your question and if this is helpful at all. Regards, Dirk. Am 20.01.2011 um 00:03 schrieb Taylor Venable: > Hi there, is the following possible in Erlang? If I use the following code: > > %% test.erl > > -module(test). > -export([foo/1,foo/2,bar/2]). > > foo(A) -> A. > foo(A, B) -> A + B. > > bar(_, []) -> []; > bar(F, [X|Xs]) -> [apply(test, F, X) | bar(F, Xs)]. > > %% END > > 2> test:bar(foo, [[1], [2,3]]). > [1,5] > > Is there a way to do the same thing without exporting foo/1 and foo/2? > It seems not with apply/3, but maybe some technique with apply/2, or > another way? Thanks for any ideas. > > -- > Taylor C. Venable > http://metasyntax.net/ > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From ryanhuffman@REDACTED Thu Jan 20 01:00:28 2011 From: ryanhuffman@REDACTED (Ryan Huffman) Date: Wed, 19 Jan 2011 16:00:28 -0800 Subject: NULL parameter for ODBC param_query Message-ID: Is it possible to use null values in a parameterized query? example: odbc:param_query("exec User_Read ?", [{sql_integer, Param}]). % Param is either an integer or null The odbc driver returns the 'null' atom when selecting rows, but I haven't found a value that allows me to specify a null parameter ('null', 'NULL', <<"null">>, etc...) Thanks! Ryan From taylor@REDACTED Thu Jan 20 03:44:59 2011 From: taylor@REDACTED (Taylor Venable) Date: Wed, 19 Jan 2011 21:44:59 -0500 Subject: [erlang-questions] Functions with same name and varying arity using apply In-Reply-To: <77F30428-2E39-43D4-B49E-BB4D598E0801@googlemail.com> References: <77F30428-2E39-43D4-B49E-BB4D598E0801@googlemail.com> Message-ID: On Wed, Jan 19, 2011 at 18:31, Dirk Scharff wrote: > I don't get what you want to do. > From your example I assume that you want to apply a function (in your case addition) to each element of a list? > You could to the same calculation as follows: > lists:map(fun(X)->lists:sum(X) end,[[1],[2,3]]). Sorry, I could have been more clear; I'm very early in the process of learning Erlang and was writing my first "real" program when I encountered this. Basically, I want to dynamically apply functions which I've defined inside a module, which have the same name but take different numbers of arguments. If possible, I want to do it without exporting those "helper" functions (in this case foo/1 and foo/2). Perhaps a better example would be if foo/2 accepted a list of options that affected its behaviour, and foo/1 is just calling foo/2 with an empty option list: %% test.erl -module(test). -export([foo/1,foo/2,bar/2]). foo(A) -> foo(A, []). foo(A, Options) -> case lists:member(double, Options) of true -> A * 2; false -> A end. bar(_, []) -> []; bar(F, [X|Xs]) -> [apply(test, F, X) | bar(F, Xs)]. %% END With this method, foo/1 and foo/2 have to be exported due to how apply/3 works (I assume; that's what the documentation says, and it fails if you don't export them). If I try using apply/2 then that takes a function, rather than an atom, so I have to do something like: bar(fun foo/1, [[1], [2, [double]]]) % from inside the module Which is not a solution because I have to declare up front whether to use foo/1 or foo/2, the opposite of what I'm trying to accomplish. So I was wondering if there was another approach to make it work. The context of discovering this is that in my program I receive the arguments to foo (in my example: both [1] and [2, [double]]) from the user by way of reading terms (I think that's the right word; I'm reading a list of atoms and tuples and such from a file and then using them to direct what the program does, like a very simple specification of commands; like what one does in Lisp reading a sexp). To make it easier, rather than forcing the user to specify an empty options list every time there are no specific options, I allow one to be present or absent. For this present situation, I could just transform the input to add an empty options list when none is present, but since this is a more general problem, I wanted to see if it was possible. Thanks, -- Taylor C. Venable http://metasyntax.net/ From vinoski@REDACTED Thu Jan 20 04:51:03 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Wed, 19 Jan 2011 22:51:03 -0500 Subject: [erlang-questions] Functions with same name and varying arity using apply In-Reply-To: References: <77F30428-2E39-43D4-B49E-BB4D598E0801@googlemail.com> Message-ID: On Wed, Jan 19, 2011 at 9:44 PM, Taylor Venable wrote: > > Sorry, I could have been more clear; I'm very early in the process of > learning Erlang and was writing my first "real" program when I > encountered this. Basically, I want to dynamically apply functions > which I've defined inside a module, which have the same name but take > different numbers of arguments. If possible, I want to do it without > exporting those "helper" functions (in this case foo/1 and foo/2). > Perhaps a better example would be if foo/2 accepted a list of options > that affected its behaviour, and foo/1 is just calling foo/2 with an > empty option list: > > %% test.erl > > -module(test). > -export([foo/1,foo/2,bar/2]). > > foo(A) -> foo(A, []). > foo(A, Options) -> > ? ?case lists:member(double, Options) of > ? ? ? ?true -> A * 2; > ? ? ? ?false -> A > ? ?end. > > bar(_, []) -> []; > bar(F, [X|Xs]) -> [apply(test, F, X) | bar(F, Xs)]. > > %% END > > With this method, foo/1 and foo/2 have to be exported due to how > apply/3 works (I assume; that's what the documentation says, and it > fails if you don't export them). If you don't export them, the compiler will think they're unused and won't include them in the generated beam file, in which case they obviously won't be around to call by any means. > If I try using apply/2 then that > takes a function, rather than an atom, so I have to do something like: > > bar(fun foo/1, [[1], [2, [double]]]) % from inside the module > > Which is not a solution because I have to declare up front whether to > use foo/1 or foo/2, the opposite of what I'm trying to accomplish. So > I was wondering if there was another approach to make it work. You might consider just always passing a single list argument to foo, so there's just one to export. It could work like this: -module(test). -export([foo/1, bar/2]). foo([A]) -> foo([A, []]); foo([A, Options]) -> case lists:member(double, Options) of true -> A * 2; false -> A end. bar(_, []) -> []; bar(F, [X|Xs]) -> [?MODULE:F(X) | bar(F, Xs)]. Then you could call it like this: test:bar(foo, [[1], [2, [double]]]). which is the same as you showed earlier. I get [1, 4] from this call, as expected. > The context of discovering this is that in my program I receive the > arguments to foo (in my example: both [1] and [2, [double]]) from the > user by way of reading terms (I think that's the right word; I'm > reading a list of atoms and tuples and such from a file Using file:consult/1, I hope? > and then using > them to direct what the program does, like a very simple specification > of commands; like what one does in Lisp reading a sexp). Another option along the Lisp reader line would be to read the user's terms, and based on those terms first generate "code as data" in Abstract Format as a set of terms, then compile it and load the resulting beam on the fly, and finally invoke one of the newly-loaded functions. Look up Abstract Format in the erts manual and also see the compile module and the beam_lib module. Whether it's worth it to go to this extent depends on how often the code would be invoked -- if you're writing some sort of DSL that's read once and then invoked numerous times, this approach could be worth it. --steve From ryanhuffman@REDACTED Thu Jan 20 06:59:49 2011 From: ryanhuffman@REDACTED (Ryan Huffman) Date: Wed, 19 Jan 2011 21:59:49 -0800 Subject: [erlang-questions] Functions with same name and varying arity using apply In-Reply-To: References: <77F30428-2E39-43D4-B49E-BB4D598E0801@googlemail.com> Message-ID: If you really don't want to export, you could try something like this: -module(test). -export([bar/2]). -define(BARRABLE_FUNS, [{{foo, 1}, fun foo/1}, {{foo, 2}, fun foo/2}]). get_fun(F, X) -> proplists:get_value({F, length(X)}, ?BARRABLE_FUNS). foo(A) -> A. foo(A, B) -> A + B. bar(_, []) -> []; bar(F, [X|Xs]) -> [apply(get_fun(F, X), X) | bar(F, Xs)]. On Wed, Jan 19, 2011 at 7:51 PM, Steve Vinoski wrote: > On Wed, Jan 19, 2011 at 9:44 PM, Taylor Venable wrote: >> >> Sorry, I could have been more clear; I'm very early in the process of >> learning Erlang and was writing my first "real" program when I >> encountered this. Basically, I want to dynamically apply functions >> which I've defined inside a module, which have the same name but take >> different numbers of arguments. If possible, I want to do it without >> exporting those "helper" functions (in this case foo/1 and foo/2). >> Perhaps a better example would be if foo/2 accepted a list of options >> that affected its behaviour, and foo/1 is just calling foo/2 with an >> empty option list: >> >> %% test.erl >> >> -module(test). >> -export([foo/1,foo/2,bar/2]). >> >> foo(A) -> foo(A, []). >> foo(A, Options) -> >> ? ?case lists:member(double, Options) of >> ? ? ? ?true -> A * 2; >> ? ? ? ?false -> A >> ? ?end. >> >> bar(_, []) -> []; >> bar(F, [X|Xs]) -> [apply(test, F, X) | bar(F, Xs)]. >> >> %% END >> >> With this method, foo/1 and foo/2 have to be exported due to how >> apply/3 works (I assume; that's what the documentation says, and it >> fails if you don't export them). > > If you don't export them, the compiler will think they're unused and > won't include them in the generated beam file, in which case they > obviously won't be around to call by any means. > >> If I try using apply/2 then that >> takes a function, rather than an atom, so I have to do something like: >> >> bar(fun foo/1, [[1], [2, [double]]]) % from inside the module >> >> Which is not a solution because I have to declare up front whether to >> use foo/1 or foo/2, the opposite of what I'm trying to accomplish. So >> I was wondering if there was another approach to make it work. > > You might consider just always passing a single list argument to foo, > so there's just one to export. It could work like this: > > -module(test). > -export([foo/1, bar/2]). > > foo([A]) -> foo([A, []]); > foo([A, Options]) -> > ? case lists:member(double, Options) of > ? ? ? true -> A * 2; > ? ? ? false -> A > ? end. > > bar(_, []) -> []; > bar(F, [X|Xs]) -> [?MODULE:F(X) | bar(F, Xs)]. > > Then you could call it like this: > > test:bar(foo, [[1], [2, [double]]]). > > which is the same as you showed earlier. I get [1, 4] from this call, > as expected. > >> The context of discovering this is that in my program I receive the >> arguments to foo (in my example: both [1] and [2, [double]]) from the >> user by way of reading terms (I think that's the right word; I'm >> reading a list of atoms and tuples and such from a file > > Using file:consult/1, I hope? > >> and then using >> them to direct what the program does, like a very simple specification >> of commands; like what one does in Lisp reading a sexp). > > Another option along the Lisp reader line would be to read the user's > terms, and based on those terms first generate "code as data" in > Abstract Format as a set of terms, then compile it and load the > resulting beam on the fly, and finally invoke one of the newly-loaded > functions. Look up Abstract Format in the erts manual and also see the > compile module and the beam_lib module. Whether it's worth it to go to > this extent depends on how often the code would be invoked -- if > you're writing some sort of DSL that's read once and then invoked > numerous times, this approach could be worth it. > > --steve > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From bengt.kleberg@REDACTED Thu Jan 20 07:54:21 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Thu, 20 Jan 2011 07:54:21 +0100 Subject: [erlang-questions] Erlang static linked interpreter In-Reply-To: References: <19767.16957.813160.505530@pilspetsen.it.uu.se> Message-ID: <1295506461.5407.19.camel@seasc1137> Greetings, This will not solve your main question, but instead try to help you the one that came up. If you want to know the performance gain from using HiPE you should measure the code execution time with and without HiPE. bengt On Wed, 2011-01-19 at 21:10 +0100, Ciprian Dorin Craciun wrote: > On Wed, Jan 19, 2011 at 21:57, Mikael Pettersson wrote: > > Ciprian Dorin Craciun writes: > > > Hello all! > > > > > > I was just wondering if anyone was able to build a static linked > > > Erlang interpreter? > > > > > > I've tried today to use `LDFLAGS=-static`, but unfortunately when > > > linking it complains of finding `__sigaction` duplicated, once in > > > Erlang's own source code, and once in `pthread` library. > > > > > > Any ideas? Thanks, > > > > If you're on x86, then disable HiPE. HiPE on x86 needs to override > > sigaction(), but that only works with dynamic linkage. > > > But if disabling HiPE, I'll have a performance penalty, or? (BTW > what is the performance gain of using HiPE?) > > And I'm a little bit confused about overriding `sigaction`. Does > the code inside Erlang call `sigaction` -- in which case then these > references could be renamed -- or there is a need to subvert the > `sigaction` calls in other system libraries. (Isn't there an option > for GCC to override a symbol?) > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From maruthavanan_s@REDACTED Thu Jan 20 09:22:46 2011 From: maruthavanan_s@REDACTED (maruthavanan s) Date: Thu, 20 Jan 2011 03:22:46 -0500 Subject: BIF for writing logs Message-ID: Hi,Do we have any module or function that would return string to me what ever I give as input like below.M:F(atom) should return "atom"M:F(9999) should return "9999"M:F("string") should return "string"M:F({tuple}) should return "{tuple}"M:F([1,2,3]) should return "[1,2,3]"I am asking this because I would like to write use this function to write the return values in a file.Thanks,Marutha From bob@REDACTED Thu Jan 20 09:32:16 2011 From: bob@REDACTED (Bob Ippolito) Date: Thu, 20 Jan 2011 16:32:16 +0800 Subject: [erlang-questions] BIF for writing logs In-Reply-To: References: Message-ID: On Thu, Jan 20, 2011 at 4:22 PM, maruthavanan s wrote: > > Hi,Do we have any module or function that would return string to me what ever I give as input like below.M:F(atom) should return "atom"M:F(9999) should return "9999"M:F("string") should return "string"M:F({tuple}) should return "{tuple}"M:F([1,2,3]) should return "[1,2,3]"I am asking this because I would like to write use this function to write the return values in a file.Thanks,Marutha io_lib:format/2. Not a BIF. 1> io_lib:format("~p", [{tuple}]). [[123,["tuple"],125]] 2> lists:flatten(io_lib:format("~p", [{tuple}])). "{tuple}" If you're writing it to a file, you should just use the output iolist directly (as shown in 1), because it will be more efficient. The second example is just to show equivalence. -bob From ciprian.craciun@REDACTED Thu Jan 20 10:10:00 2011 From: ciprian.craciun@REDACTED (Ciprian Dorin Craciun) Date: Thu, 20 Jan 2011 11:10:00 +0200 Subject: HiPE performance gain (Was: Re: [erlang-questions] Erlang static linked interpreter) Message-ID: On Thu, Jan 20, 2011 at 08:54, Bengt Kleberg wrote: > Greetings, > > This will not solve your main question, but instead try to help you the > one that came up. > > If you want to know the performance gain from using HiPE you should > measure the code execution time with and without HiPE. > > bengt I am perfectly aware that no technology is a silver bullet in all circumstances, and that you have to benchmark a particular technology in a particular case to see its effect. But in my original question (although I've not clearly stated so) I was curios what is the impact of HiPE in general, meaning what is the experience different people had with HiPE? More exactly what are the use-cases which are likely to benefit from HiPE? I would guess that in a CPU bound application HiPE would do better than without, but what about network I/O bound applications, or applications that deal mainly with strings (represented as lists), or applications that deal mostly with binary matching? Thanks, Ciprian. From ingela@REDACTED Thu Jan 20 10:37:15 2011 From: ingela@REDACTED (Ingela Andin) Date: Thu, 20 Jan 2011 10:37:15 +0100 Subject: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 In-Reply-To: <4D370678.60203@tail-f.com> References: <4D36ADAC.3070509@tail-f.com> <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> <4D370678.60203@tail-f.com> Message-ID: Hi again, The PKCS-1 asn-1 spec is part of public key. PEM-files are mainly base64-encoded asn1 DER data. Pem files looks something like this: -----BEGIN SOMETHING----- ... -----END SOMETHING----- Adding support for the something = RSA PUBLIC KEY should be fairly simple as the ASN1- spec is alredy in place. 2011/1/19 Joakim G. : > Thanks, I see. > > I'm using keys as defined in: > https://gitweb.torproject.org/tor.git?a=blob_plain;hb=HEAD;f=doc/spec/tor-spec.txt > > That is: > > "For a public-key cipher, we use RSA with 1024-bit keys and a fixed > exponent of 65537. We use OAEP-MGF1 padding, with SHA-1 as its digest > function. We leave the optional "Label" parameter unset. (For OAEP > padding, see ftp://ftp.rsasecurity.com/pub/pkcs/pkcs-1/pkcs-1v2-1.pdf)" > > I only have the public keys to work with. Not a surprise. :-) No not really, but you never know, with time I learnt not to take anything for granted ;) > > I will see if I can find more info on this. [...] I will put it on the wish-list, but we have got a lot of other things on our table so a contribution will proably speed things up. Regards Ingela Erlang/OTP team - Ericsson AB From dsc@REDACTED Thu Jan 20 11:12:58 2011 From: dsc@REDACTED (Dave Challis) Date: Thu, 20 Jan 2011 10:12:58 +0000 Subject: [erlang-questions] Any tools available to generate code for Erlang Ports? In-Reply-To: References: <4D359BAA.3000606@ecs.soton.ac.uk> <4D380AAA.7050907@ecs.soton.ac.uk> Message-ID: Thanks for all the links, there's plenty there to get me started I should think. Looking at some of the larger examples is helping to give an idea of how things are done in practice. On 18/01/11 20:53, Tim Watson wrote: > On 18 January 2011 13:54, Dave Challis wrote: >> I'm just curious - are there any tools or code generators out there for >> creating Erlang Ports to interface with C code? > > Sorry that my last two response got completely scrambled - I *really* > need to get some sleep! With regard to NIFs, I meant to say "as long > as you do not" maintain state in the driver - for that sort of thing > and for async processing, you want a port program or linked-in driver. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Dave Challis dsc@REDACTED From thomasl_erlang@REDACTED Thu Jan 20 12:01:39 2011 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Thu, 20 Jan 2011 03:01:39 -0800 (PST) Subject: [erlang-questions] HiPE performance gain (Was: Re: [erlang-questions] Erlang static linked interpreter) In-Reply-To: References: Message-ID: <871318.87934.qm@web111404.mail.gq1.yahoo.com> ----- Original Message ---- > From: Ciprian Dorin Craciun ... > More exactly what are the use-cases which are likely to benefit > from HiPE? I would guess that in a CPU bound application HiPE would do > better than without, but what about network I/O bound applications, or > applications that deal mainly with strings (represented as lists), or > applications that deal mostly with binary matching? As far as I'm aware, Hipe mostly can't optimize the runtime system. Sockets, drivers, etc, are just invoked. Some BIFs are inlined, but the more complex ones are just invoked. Binary matching is (mostly or entirely?) native compiled. List traversal should be fast, though the emulator cheats (or at least used to cheat) by invoking C code for some common operations. Pattern matching is generally fast, though (some?) guards may be invoked instead of inlined. Our experience on AXD 301 way back was ambiguous, though part of that was due to insufficient I-cache on the Ultrasparc -- cache misses ate the other performance gains. (This is one of the classic arguments for using an emulator instead of native compilation.) Perhaps modern hardware would make better use of the native code. Anecdotes I've heard from people working on other big code bases indicate no clear gains, basically. But I can't claim to comprehensively cover the whole field by any means. It would be interesting to hear some testimonials. (The core erlang app used at my current company is estimated to max out the network interfaces without native compilation, so we haven't tried it.) One system that might have gained is Wings3D, since Hipe spent a good deal of effort on floating point optimization. Haven't measured it myself, but it would be interesting to hear of any experiences. Finally, you can always have a look at the Hipe output for a function or module, if you want to know the gory details. Best regards, Thomas From michael.eugene.turner@REDACTED Thu Jan 20 12:50:49 2011 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Thu, 20 Jan 2011 20:50:49 +0900 Subject: [erlang-questions] HiPE performance gain (Was: Re: [erlang-questions] Erlang static linked interpreter) In-Reply-To: <871318.87934.qm@web111404.mail.gq1.yahoo.com> References: <871318.87934.qm@web111404.mail.gq1.yahoo.com> Message-ID: "Our experience on AXD 301 way back was ambiguous, though part of that was due to insufficient I-cache on the Ultrasparc -- cache misses ate the other performance gains. (This is one of the classic arguments for using an emulator instead of native compilation.) Perhaps modern hardware would make better use of the native code." (Thomas Lindgren) How the emulator is coded probably makes a difference too. In my few visits to the BEAM VM code, I was struck by the amount of inlining (and via long C macros, whoa, scary!) I began to wonder whether those macros really paid off, when you consider the more limited I-caches. If their bloat also tends to crowd out HiPE-generated native code, that's another performance ding, albeit more indirectly suffered. Wouldn't it make more sense to recode those macros as static functions and try various compiler optimization flags relevant to automatic inlining of functions? A smaller (in generated native code) VM that *seemed* to have more function call overhead might nevertheless actually be a win for HiPE-compiled code. Maybe a win more generally, at least on some architectures. For giving HiPE a chance, it might be worth an experiment or two. -michael turner On Thu, Jan 20, 2011 at 8:01 PM, Thomas Lindgren wrote: > > > > > ----- Original Message ---- > > From: Ciprian Dorin Craciun > ... > > More exactly what are the use-cases which are likely to benefit > > from HiPE? I would guess that in a CPU bound application HiPE would do > > better than without, but what about network I/O bound applications, or > > applications that deal mainly with strings (represented as lists), or > > applications that deal mostly with binary matching? > > > As far as I'm aware, Hipe mostly can't optimize the runtime system. > Sockets, > drivers, etc, are just invoked. Some BIFs are inlined, but the more complex > ones > are just invoked. Binary matching is (mostly or entirely?) native compiled. > List > traversal should be fast, though the emulator cheats (or at least used to > cheat) > by invoking C code for some common operations. Pattern matching is > generally > fast, though (some?) guards may be invoked instead of inlined. > > Our experience on AXD 301 way back was ambiguous, though part of that was > due to > insufficient I-cache on the Ultrasparc -- cache misses ate the other > performance > gains. (This is one of the classic arguments for using an emulator instead > of > native compilation.) Perhaps modern hardware would make better use of the > native > code. > > Anecdotes I've heard from people working on other big code bases indicate > no > clear gains, basically. But I can't claim to comprehensively cover the > whole > field by any means. It would be interesting to hear some testimonials. > > (The core erlang app used at my current company is estimated to max out the > network interfaces without native compilation, so we haven't tried it.) > > One system that might have gained is Wings3D, since Hipe spent a good deal > of > effort on floating point optimization. Haven't measured it myself, but it > would > be interesting to hear of any experiences. > > Finally, you can always have a look at the Hipe output for a function or > module, > if you want to know the gory details. > > Best regards, > Thomas > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From alain.odea@REDACTED Thu Jan 20 13:11:40 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Thu, 20 Jan 2011 08:41:40 -0330 Subject: [erlang-questions] BIF for writing logs In-Reply-To: References: Message-ID: <5D8E6FEF-18EC-44B5-BB47-2C65C67A2980@gmail.com> On 2011-01-20, at 5:02, Bob Ippolito wrote: > On Thu, Jan 20, 2011 at 4:22 PM, maruthavanan s > wrote: >> >> Hi,Do we have any module or function that would return string to me what ever I give as input like below.M:F(atom) should return "atom"M:F(9999) should return "9999"M:F("string") should return "string"M:F({tuple}) should return "{tuple}"M:F([1,2,3]) should return "[1,2,3]"I am asking this because I would like to write use this function to write the return values in a file.Thanks,Marutha > io_lib:format/2. Not a BIF. > > 1> io_lib:format("~p", [{tuple}]). > [[123,["tuple"],125]] > 2> lists:flatten(io_lib:format("~p", [{tuple}])). > "{tuple}" > > If you're writing it to a file, you should just use the output iolist > directly (as shown in 1), because it will be more efficient. The > second example is just to show equivalence. > > -bob You can also use ~w instead if you don't need nicely indented output files: io_lib:format("~w", [{tuple}]). An even leaner approach is the binary external format. IMHO, it should be used unless you need human readable files: http://www.erlang.org/doc/apps/erts/erl_ext_dist.html term_to_binary/1,2 binary_to_term/1 From ingham.k@REDACTED Thu Jan 20 17:01:04 2011 From: ingham.k@REDACTED (Igor Karymov) Date: Thu, 20 Jan 2011 22:01:04 +0600 Subject: Serious problems with some operations on the ARM platfor. Message-ID: On the last version of erlang (tested on r13b04 and r14a) I observ next behavior: (nmg@REDACTED)2> A = 85023954648017753633109926276377209232011432883926046723251747957496189689499554217508419200604908647245310645038654319411021235336416116239327092322283173571753199380814137374196214921340934814673139306620241639100528995020328973184941898694736203837639298925043282006637104161166612956247829863503707011335. 85023954648017753633109926276377209232011432883926046723251747957496189689499554217508419200604908647245310645038654319411021235336416116239327092322283173571753199380814137374196214921340934814673139306620241639100528995020328973184941898694736203837639298925043282006637104161166612956247829863503707011335 (nmg@REDACTED)3> B = A bsr 8. 21766132389892544930076141126752565563394926818285067961152447477119024560511885879682155315354856613694799525129895505769221436246122525757267735634504492434368819041488419167794231019863279312556323662494781859609735422725204217135345126065852468182435660524811080193699098665258652916799444445056948994901760 (nmg@REDACTED)4> C = A bsl 8. 21766132389892544930076141126752565563394926818285067961152447477119024560511885879682155315354856613694799525129895505769221436246122525757267735634504492434368819041488419167794231019863279312556323662494781859609735422725204217135345126065852468182435660524811080193699098665258652916799444445056948994901760 (nmg@REDACTED)5> A > B. true (nmg@REDACTED)6> C == B. true On older versions (r12b05) ?all are right. Compiler: $ arm-softfloat-linux-gnu-gcc --version arm-softfloat-linux-gnu-gcc (GCC) 3.4.5 Linux: Linux some 2.6.22.19-4.03.0-c300evm #123 Tue Sep 21 10:00:16 NOVST 2010 armv6l unknown Processor: CPU: ARMv6-compatible processor [4117b361] revision 1 (ARMv6TEJ) -------------- next part -------------- A non-text attachment was scrubbed... Name: config.h Type: text/x-chdr Size: 22635 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: cfg_arm.sh Type: application/x-sh Size: 449 bytes Desc: not available URL: From ingham.k@REDACTED Thu Jan 20 17:09:02 2011 From: ingham.k@REDACTED (Igor Karymov) Date: Thu, 20 Jan 2011 08:09:02 -0800 (PST) Subject: Serious problems with some operations on the ARM platfor. In-Reply-To: References: Message-ID: <67038228-2b57-4d53-9564-1262858a5b84@z5g2000yqb.googlegroups.com> And more: (nmg@REDACTED)2> 7 / 3. 0.00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000037206633229885703 (nmg@REDACTED)3> 8 / 3. {error_logger,{{1970,1,1},{2,54,21}},supervisor_report,[{supervisor, {<0.33.0>,user_sup}},{errorContext,child_terminated},{reason,{badarg, [{erlang,iolist_to_binary, [[1194530458031458476616085406908857401459880800089440447834545706654653339164860940792438620212730789936,48,46,48]]}, {user,io_request,3},{user,io_request,3},{user,do_io_request,5}, {user,server_loop,2},{user,catch_loop,3}]}},{offender,[{pid,<0.35.0>}, {mod,user_sup}]}]} {error_logger,{{1970,1,1},{2,54,21}},"** Generic server ~p terminating \n** Last message in was ~p~n** When Server state == ~p~n** Reason for termination == ~n** ~p~n",[<0.33.0>,{'EXIT',<0.35.0>,{badarg, [{erlang,iolist_to_binary, [[1194530458031458476616085406908857401459880800089440447834545706654653339164860940792438620212730789936,48,46,48]]}, {user,io_request,3},{user,io_request,3},{user,do_io_request,5}, {user,server_loop,2},{user,catch_loop,3}]}}, {state,user_sup,undefined,<0.35.0>,{<0.33.0>,user_sup}},{badarg, [{erlang,iolist_to_binary, [[1194530458031458476616085406908857401459880800089440447834545706654653339164860940792438620212730789936,48,46,48]]}, {user,io_request,3},{user,io_request,3},{user,do_io_request,5}, {user,server_loop,2},{user,catch_loop,3}]}]} {error_logger,{{1970,1,1},{2,54,21}},crash_report,[[{initial_call, {supervisor_bridge,user_sup,['Argument__1']}},{pid,<0.33.0>}, {registered_name,[]},{error_info,{exit,{badarg, [{erlang,iolist_to_binary, [[1194530458031458476616085406908857401459880800089440447834545706654653339164860940792438620212730789936,48,46,48]]}, {user,io_request,3},{user,io_request,3},{user,do_io_request,5}, {user,server_loop,2},{user,catch_loop,3}]},[{gen_server,terminate,6}, {proc_lib,init_p_do_apply,3}]}},{ancestors,[kernel_sup,<0.9.0>]}, {messages,[]},{links,[<0.10.0>]},{dictionary,[]},{trap_exit,true}, {status,running},{heap_size,610},{stack_size,24},{reductions,177}], []]} {error_logger,{{1970,1,1},{2,54,21}},supervisor_report,[{supervisor, {local,kernel_sup}},{errorContext,child_terminated},{reason,{badarg, [{erlang,iolist_to_binary, [[1194530458031458476616085406908857401459880800089440447834545706654653339164860940792438620212730789936,48,46,48]]}, {user,io_request,3},{user,io_request,3},{user,do_io_request,5}, {user,server_loop,2},{user,catch_loop,3}]}},{offender,[{pid,<0.33.0>}, {name,user},{mfa,{user_sup,start,[]}},{restart_type,temporary}, {shutdown,2000},{child_type,supervisor}]}]} From bainx028@REDACTED Thu Jan 20 17:36:05 2011 From: bainx028@REDACTED (Trevor Bain) Date: Thu, 20 Jan 2011 10:36:05 -0600 Subject: OTP Behaviors in C Nodes? Message-ID: Can I use OTP Behaviors in C Nodes? Right now I'm using RCP to call OTP functions on the connected node. Can I spoof a OTP messages (i.e. gen_event:notify) by sending a correctly formatted message from my C Node? How reliable are the message structures used by OTP - if I do spoof OTP messages will they be likely to change in the future or do they change under different circumstances? I have hard latency limit requirements; how does this effect my decision? Thanks, Trevor From ulf.wiger@REDACTED Thu Jan 20 18:02:01 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Thu, 20 Jan 2011 18:02:01 +0100 Subject: [erlang-questions] gproc patch In-Reply-To: References: Message-ID: Hi Garrett, Which version of gproc are you using? That particular bug was fixed on Jan 11 in http://github.com/esl/gproc (That is, quite recently. :) Sending pull requests to that project will work just fine. BR, Ulf W On 19 Jan 2011, at 01:21, Garret Smith wrote: > Fixed a small issue with gproc in the 'await' function when no timeout > is specified. > > see branch 'fix-await-notimeout' in the repository > git://github.com/garret-smith/gproc.git > > or the patch: > > --- a/src/gproc.erl > +++ b/src/gproc.erl > @@ -312,7 +312,7 @@ request_wait({n,C,_} = Key, Timeout) when C==l; C==g -> > WRef = call({await,Key,self()}, C), > receive > {gproc, WRef, registered, {_K, Pid, V}} -> > - case Timeout of > + case TRef of > no_timer -> ignore; > _ -> erlang:cancel_timer(TRef) > end, > > > I don't know of any guidelines for patch submission since this is not > OTP, so let me know if there is a better way I can provide patches now > or in the future. > > -G > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From garret.smith@REDACTED Thu Jan 20 18:42:55 2011 From: garret.smith@REDACTED (Garret Smith) Date: Thu, 20 Jan 2011 09:42:55 -0800 Subject: [erlang-questions] gproc patch In-Reply-To: References: Message-ID: I was using the version at https://github.com/uwiger/gproc.git which looks identical except for that patch :) -G On Thu, Jan 20, 2011 at 9:02 AM, Ulf Wiger wrote: > > Hi Garrett, > > Which version of gproc are you using? > > That particular bug was fixed on Jan 11 in http://github.com/esl/gproc > (That is, quite recently. :) > > Sending pull requests to that project will work just fine. > > BR, > Ulf W > > On 19 Jan 2011, at 01:21, Garret Smith wrote: > >> Fixed a small issue with gproc in the 'await' function when no timeout >> is specified. >> >> see branch 'fix-await-notimeout' in the repository >> git://github.com/garret-smith/gproc.git >> >> or the patch: >> >> --- a/src/gproc.erl >> +++ b/src/gproc.erl >> @@ -312,7 +312,7 @@ request_wait({n,C,_} = Key, Timeout) when C==l; C==g -> >> ? ? WRef = call({await,Key,self()}, C), >> ? ? receive >> ? ? ? ? {gproc, WRef, registered, {_K, Pid, V}} -> >> - ? ? ? ? ? case Timeout of >> + ? ? ? ? ? case TRef of >> ? ? ? ? ? ? ? ?no_timer -> ignore; >> ? ? ? ? ? ? ? ?_ -> erlang:cancel_timer(TRef) >> ? ? ? ? ? ?end, >> >> >> I don't know of any guidelines for patch submission since this is not >> OTP, so let me know if there is a better way I can provide patches now >> or in the future. >> >> -G >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > From christophe.romain@REDACTED Thu Jan 20 18:15:25 2011 From: christophe.romain@REDACTED (Christophe Romain) Date: Thu, 20 Jan 2011 18:15:25 +0100 Subject: [erlang-questions] Re: Serious problems with some operations on the ARM platfor. In-Reply-To: <67038228-2b57-4d53-9564-1262858a5b84@z5g2000yqb.googlegroups.com> References: <67038228-2b57-4d53-9564-1262858a5b84@z5g2000yqb.googlegroups.com> Message-ID: <20110120171525.GI1127@localhost> I dayly use ARM platform (TI-OMAP) ARMv7 Processor rev 3 (v7l) running R14A erlang binary from debian >(nmg@REDACTED)2> 7 / 3. >0.00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000037206633229885703 1> 7/3. 2.3333333333333335 >(nmg@REDACTED)3> 8 / 3. >{error_logger,{{1970,1,1},{2,54,21}},supervisor_report,[{supervisor, >{<0.33.0>,user_sup}},{errorContext,child_terminated},{reason,{badarg, >[{erlang,iolist_to_binary, ..... 2> 8/3. 2.6666666666666665 so, your problem is not related to ARM, but more likely to your build process (gcc optimization flag?) which recent erlang version don't cope with i guess. From mikpe@REDACTED Thu Jan 20 21:03:45 2011 From: mikpe@REDACTED (Mikael Pettersson) Date: Thu, 20 Jan 2011 21:03:45 +0100 Subject: [erlang-questions] Serious problems with some operations on the ARM platfor. In-Reply-To: References: Message-ID: <19768.38177.770249.788865@pilspetsen.it.uu.se> Igor Karymov wrote: > ... > Compiler: > $ arm-softfloat-linux-gnu-gcc --version > arm-softfloat-linux-gnu-gcc (GCC) 3.4.5 > > Linux: > Linux some 2.6.22.19-4.03.0-c300evm #123 Tue Sep 21 10:00:16 NOVST > 2010 armv6l unknown > > Processor: > CPU: ARMv6-compatible processor [4117b361] revision 1 (ARMv6TEJ) You're using an ancient unsupported compiler, apparently the obsolete and no longer supported OABI, which has furthermore been hacked from HardFPA to SoftFP. If your softfloat toolchain uses the legacy FPA format, then it's known to be incompatible with lots of SW including Erlang/OTP, if it uses the VFP format then that aspect should be Ok. Do yourself a favour and update to: a) gcc-4.4.5 or newer, and b) ARM EABI, i.e. arm-unknown-linux-gnueabi. FWIW, OTP R14B02 works fine on my ARM machines (ARMv5TE, EABI, gcc-4.4.6 (heavily updated), glibc-2.10.1 (2.11.x soon), kernel 2.6.37). From kostis@REDACTED Thu Jan 20 23:33:20 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Fri, 21 Jan 2011 00:33:20 +0200 Subject: [erlang-questions] HiPE performance gain (Was: Re: [erlang-questions] Erlang static linked interpreter) In-Reply-To: References: Message-ID: <4D38B830.808@cs.ntua.gr> Ciprian Dorin Craciun wrote: > On Thu, Jan 20, 2011 at 08:54, Bengt Kleberg wrote: >> Greetings, >> >> This will not solve your main question, but instead try to help you the >> one that came up. >> >> If you want to know the performance gain from using HiPE you should >> measure the code execution time with and without HiPE. >> >> bengt > > > I am perfectly aware that no technology is a silver bullet in all > circumstances, and that you have to benchmark a particular technology > in a particular case to see its effect. > > But in my original question (although I've not clearly stated so) > I was curios what is the impact of HiPE in general, meaning what is > the experience different people had with HiPE? I will provide some answers to your questions below, but since I've read/heard this asked many times before, I think I mostly agree with Bengt's suggestion here. Suppose you are told that other people experience a 10% slowdown or a 3 times speedup in their application when using HiPE, what exactly does that tell you about how HiPE will perform on *your* application? (which presumably you care most about) We've spent a lot of effort to smoothly integrate the HiPE compiler in Erlang/OTP and it's just a matter of adding +native in the ERLC options in your Makefile to try HiPE out. So why don't you try this and see for yourself? Anyway, some answers to these questions: > More exactly what are the use-cases which are likely to benefit > from HiPE? I would guess that in a CPU bound application HiPE would do > better than without, but what about network I/O bound applications, or > applications that deal mainly with strings (represented as lists), or > applications that deal mostly with binary matching? Where HiPE really shines is in programs containing binaries. In the past, we've observed speedups of 8-10 times in such programs. But this is a speedup that is difficult to achieve in the general case. I would guess that in applications which are CPU bound one can get a speedup that on average is in the range 50% to 2.5 times faster than BEAM. If our application is mainly network bound, then you can probably forget about native code compilation. But as I wrote, it's one's *own* use case that matters most! I for once, I am spending most of my time these days running dialyzer on various code bases. In large code bases, I would definitely not like it if dialyzer was not insisting on using native code by default: $ dialyzer --build_plt --output_plt ~/.dialyzer_plt-nn --apps erts kernel stdlib mnesia Compiling some key modules to native code... done in 0m0.09s Creating PLT /home/kostis/.dialyzer_plt-nn ... ... done in 8m21.87s Compare with: $ dialyzer --no_native --build_plt --output_plt ~/.dialyzer_plt-nn --apps erts kernel stdlib mnesia Creating PLT /home/kostis/.dialyzer_plt-nn ... ... done in 17m12.80s I am using dialyzer as an example, because it's not a toy benchmark. Instead it is a very complex program (about 30KLOCs of Erlang code) where all sorts of things are involved (I/O for reading files, accessing ETS tables, big data structures, uses of many stdlib files, etc.) But don't just take my word, try it for yourself. Kostis From jocke@REDACTED Thu Jan 20 23:40:05 2011 From: jocke@REDACTED (Joakim G.) Date: Thu, 20 Jan 2011 23:40:05 +0100 Subject: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 In-Reply-To: References: <4D36ADAC.3070509@tail-f.com> <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> <4D370678.60203@tail-f.com> Message-ID: <4D38B9C5.9030002@tail-f.com> I updated pubkey_pem.html a tiny little bit and it seems to work: jocke@REDACTED:~/installs/otp_src_R14B01/lib/public_key/src$ diff -bwt pubkey_pem.erl.orig pubkey_pem.erl 157a158,159 > join_entry([<<"-----END RSA PUBLIC KEY-----", _/binary>>| Lines], Entry) -> > {lists:reverse(Entry), Lines}; 212a215,216 > pem_start('RSAPublicKey') -> > <<"-----BEGIN RSA PUBLIC KEY-----">>; 221a226,227 > pem_end(<<"-----BEGIN RSA PUBLIC KEY-----">>) -> > <<"-----END RSA PUBLIC KEY-----">>; 232a239,240 > asn1_type(<<"-----BEGIN RSA PUBLIC KEY-----">>) -> > 'RSAPublicKey'; Thanks /Jocke On 2011-01-20 10:37, Ingela Andin wrote: > Hi again, > > The PKCS-1 asn-1 spec is part of public key. PEM-files are mainly > base64-encoded asn1 DER data. > Pem files looks something like this: > > > -----BEGIN SOMETHING----- > > > ... > -----END SOMETHING----- > > > Adding support for the something = RSA PUBLIC KEY should be fairly > simple as the ASN1- spec is alredy in place. > > 2011/1/19 Joakim G. : >> Thanks, I see. >> >> I'm using keys as defined in: >> https://gitweb.torproject.org/tor.git?a=blob_plain;hb=HEAD;f=doc/spec/tor-spec.txt >> >> That is: >> >> "For a public-key cipher, we use RSA with 1024-bit keys and a fixed >> exponent of 65537. We use OAEP-MGF1 padding, with SHA-1 as its digest >> function. We leave the optional "Label" parameter unset. (For OAEP >> padding, see ftp://ftp.rsasecurity.com/pub/pkcs/pkcs-1/pkcs-1v2-1.pdf)" >> >> I only have the public keys to work with. Not a surprise. :-) > > No not really, but you never know, with time I learnt not to take > anything for granted ;) > >> >> I will see if I can find more info on this. > > [...] > > I will put it on the wish-list, but we have got a lot of other things > on our table so a contribution > will proably speed things up. > > Regards Ingela Erlang/OTP team - Ericsson AB > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From jocke@REDACTED Thu Jan 20 23:41:10 2011 From: jocke@REDACTED (Joakim G.) Date: Thu, 20 Jan 2011 23:41:10 +0100 Subject: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 In-Reply-To: <4D38B9C5.9030002@tail-f.com> References: <4D36ADAC.3070509@tail-f.com> <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> <4D370678.60203@tail-f.com> <4D38B9C5.9030002@tail-f.com> Message-ID: <4D38BA06.3080408@tail-f.com> lol. pubkey_pem.erl /Jocke On 2011-01-20 23:40, Joakim G. wrote: > I updated pubkey_pem.html a tiny little bit and it seems to > work: > > jocke@REDACTED:~/installs/otp_src_R14B01/lib/public_key/src$ diff -bwt > pubkey_pem.erl.orig pubkey_pem.erl > 157a158,159 >> join_entry([<<"-----END RSA PUBLIC KEY-----", _/binary>>| Lines], > Entry) -> >> {lists:reverse(Entry), Lines}; > 212a215,216 >> pem_start('RSAPublicKey') -> >> <<"-----BEGIN RSA PUBLIC KEY-----">>; > 221a226,227 >> pem_end(<<"-----BEGIN RSA PUBLIC KEY-----">>) -> >> <<"-----END RSA PUBLIC KEY-----">>; > 232a239,240 >> asn1_type(<<"-----BEGIN RSA PUBLIC KEY-----">>) -> >> 'RSAPublicKey'; > > Thanks > /Jocke > > On 2011-01-20 10:37, Ingela Andin wrote: >> Hi again, >> >> The PKCS-1 asn-1 spec is part of public key. PEM-files are mainly >> base64-encoded asn1 DER data. >> Pem files looks something like this: >> >> >> -----BEGIN SOMETHING----- >> >> >> ... >> -----END SOMETHING----- >> >> >> Adding support for the something = RSA PUBLIC KEY should be fairly >> simple as the ASN1- spec is alredy in place. >> >> 2011/1/19 Joakim G. : >>> Thanks, I see. >>> >>> I'm using keys as defined in: >>> https://gitweb.torproject.org/tor.git?a=blob_plain;hb=HEAD;f=doc/spec/tor-spec.txt >>> >>> That is: >>> >>> "For a public-key cipher, we use RSA with 1024-bit keys and a fixed >>> exponent of 65537. We use OAEP-MGF1 padding, with SHA-1 as its digest >>> function. We leave the optional "Label" parameter unset. (For OAEP >>> padding, see ftp://ftp.rsasecurity.com/pub/pkcs/pkcs-1/pkcs-1v2-1.pdf)" >>> >>> I only have the public keys to work with. Not a surprise. :-) >> >> No not really, but you never know, with time I learnt not to take >> anything for granted ;) >> >>> >>> I will see if I can find more info on this. >> >> [...] >> >> I will put it on the wish-list, but we have got a lot of other things >> on our table so a contribution >> will proably speed things up. >> >> Regards Ingela Erlang/OTP team - Ericsson AB >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From seth@REDACTED Thu Jan 20 23:48:05 2011 From: seth@REDACTED (Seth Falcon) Date: Thu, 20 Jan 2011 14:48:05 -0800 Subject: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 In-Reply-To: References: <4D36ADAC.3070509@tail-f.com> <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> <4D370678.60203@tail-f.com> Message-ID: Hi there, I'm very interested in adding support for RSA public key decoding to the public_key module. My use case is similar in that I need to be able to verify/decrypt when all I have is the RSA public key in PEM format. 2011/1/20 Ingela Andin : > The PKCS-1 asn-1 spec is part of public key. PEM-files are mainly > base64-encoded ?asn1 DER data. [snip] > Adding support for the something = RSA PUBLIC KEY should be fairly > simple as the ASN1- spec is alredy in place. I started working on such a patch and you are right that the changes to read in the pem data are fairly simple. However, once I have the DER format, I'm getting stuck with an error coming out of public_key:der_decode. Not sure if I'm doing it wrong or if there is something wrong with the asn1 definition for RSA public keys. Here's what I'm trying (skipping the pem conversion for now): shell: openssl genrsa > mykey.private openssl rsa -in mykey.private -outform der -pubout > mykey.public.der erl: {ok, PubDer} = file:read_file("mykey.public.der"). public_key:der_decode('RSAPublicKey', PubDer) ** exception error: no match of right hand side value {error, {asn1, {function_clause, [{'OTP-PUB-KEY',decode_integer, [{error,{asn1,{wrong_tag,{16,2}}}}]}, {'OTP-PUB-KEY',decode_integer,3}, {'OTP-PUB-KEY',dec_RSAPublicKey,2}, {'OTP-PUB-KEY',decode,2}, {public_key,der_decode,2}, {erl_eval,do_apply,5}, {shell,exprs,7}, {shell,eval_exprs,7}]}}} in function public_key:der_decode/2 Aside: when I use openssl to extract the public key, the output I get looks like: -----BEGIN PUBLIC KEY----- MFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBALyPPyTS+HyTV3oLf2z6W+H1NOKLos3K ztRytqJoIdvbzNFYsnJqFhabzhQx2DdPvTiHLZM1QHQ1Ma5yw5hjaQcCAwEAAQ== -----END PUBLIC KEY----- Note that it only says "PUBLIC KEY", not "RSA PUBLIC KEY". You will get identical header values when extracting a DSA public key. So I'm unsure about how one can detect what type of public key one has. Anyone know the details of how that's supposed to work? This throws a wrinkle in pem_decode/1. It may be that one needs to have pem_decode/2 for public keys where the user specifies what type of key is expected. If I can get a bit further I will put together a patch. Thanks, + seth -- Seth Falcon | @sfalcon | http://userprimary.net/ From seth@REDACTED Fri Jan 21 00:03:04 2011 From: seth@REDACTED (Seth Falcon) Date: Thu, 20 Jan 2011 15:03:04 -0800 Subject: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 In-Reply-To: <4D38B9C5.9030002@tail-f.com> References: <4D36ADAC.3070509@tail-f.com> <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> <4D370678.60203@tail-f.com> <4D38B9C5.9030002@tail-f.com> Message-ID: 2011/1/20 Joakim G. : > I updated pubkey_pem.html a tiny little bit and it seems to > work: > > jocke@REDACTED:~/installs/otp_src_R14B01/lib/public_key/src$ diff -bwt > pubkey_pem.erl.orig pubkey_pem.erl > 157a158,159 >> join_entry([<<"-----END RSA PUBLIC KEY-----", _/binary>>| Lines], > Entry) -> >> ? ? {lists:reverse(Entry), Lines}; > 212a215,216 >> pem_start('RSAPublicKey') -> >> ? ? <<"-----BEGIN RSA PUBLIC KEY-----">>; > 221a226,227 >> pem_end(<<"-----BEGIN RSA PUBLIC KEY-----">>) -> >> ? ? <<"-----END RSA PUBLIC KEY-----">>; > 232a239,240 >> asn1_type(<<"-----BEGIN RSA PUBLIC KEY-----">>) -> >> ? ? 'RSAPublicKey'; > > Thanks Our mails crossed. When you say "this seems to work", were you able to actually use the public key for verification/decryption? From rzezeski@REDACTED Fri Jan 21 02:48:45 2011 From: rzezeski@REDACTED (Ryan Zezeski) Date: Thu, 20 Jan 2011 20:48:45 -0500 Subject: [erlang-questions] linked-in driver blocks net_kernel tick? In-Reply-To: References: Message-ID: So...can anyone explain to me why zlib:gzip/1 is causing the net_kernel tick to be blocked? Do linked-in drivers block it's scheduler like NIFs? I'm really curious on this one :) -Ryan On Tue, Jan 18, 2011 at 6:53 PM, Ryan Zezeski wrote: > Apologies, the example I copied was run on my mac. > > This is what I have on the actual production machine: > > Erlang R14A (erts-5.8) [source] [64-bit] [smp:16:16] [rq:16] > [async-threads:0] [hipe] [kernel-poll:false] > > To be certain, I ran the same example (except this time using two physical > machines) and achieved the same result. Namely, the 'bar' node claims 'foo' > is not responding and thus closes the connection. Whatever this is, I've > now easily reproduced it on two different OSs, with 2 different Erlang > versions. > > -Ryan > > On Tue, Jan 18, 2011 at 6:04 PM, Alain O'Dea wrote: > >> On 2011-01-18, at 18:54, Ryan Zezeski wrote: >> >> > Hi everyone, >> > >> > Some of you may remember my latest question where I was having weird >> node >> > timeout issues that I couldn't explain and I thought it might be related >> to >> > the messages I was passing between my nodes. Well, I pinpointed the >> problem >> > to a call to zlib:gzip/1. At first I was really surprised by this, as >> such >> > a harmless line of code surely should have nothing to do with the >> ability >> > for my nodes to communicate. However, as I dug further I realized gzip >> was >> > implemented as a linked-in driver and I remember reading things about >> how >> > one has to take care with them because they can trash the VM with them. >> I >> > don't remember reading anything about them blocking code, and even if >> they >> > do I fail to see why my SMP enabled node (16 cores) would allow this one >> > thread to block the tick. It occurred to me that maybe the scheduler >> > responsible for that process is the one blocked by the driver. Do >> processes >> > have scheduler affinity? That would make sense, I guess. >> > >> > I've "fixed" this problem simply by using a plain port (i.e. run in it's >> own >> > OS process). For my purposes, this actually makes more sense in the >> > majority of the places I was making use of gzip. Can someone enlighten >> me >> > as to exactly what is happening behind the scenes? >> > >> > To reproduce I create a random 1.3GB file: >> > >> > dd if=/dev/urandom of=rand bs=1048576 count=1365 >> > >> > Then start two named nodes 'foo' and 'bar', connect them, read in the >> file, >> > and then compress said file. Sometime later (I think around 60+ >> seconds) >> > the node 'bar' will claim that 'foo' is not responding. >> > >> > [progski@REDACTED ~/tmp_code/node_timeout] erl -name foo >> > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] >> >> Your SMP node seems to be capped at smp:2:2 when it out to be smp:16. >> Some resource limit may be holding back the system. That said zlib should >> not ever cause this issue. >> >> > [async-threads:0] [hipe] [kernel-poll:false] >> > >> > Eshell V5.8.1 (abort with ^G) >> > (foo@REDACTED)1> net_adm:ping('bar@REDACTED'). >> > pong >> > (foo@REDACTED)2> nodes(). >> > ['bar@REDACTED'] >> > (foo@REDACTED)3> {ok,Data} = file:read_file("rand"). >> > {ok,<<103,5,115,210,177,147,53,45,250,182,51,32,250,233, >> > 39,253,102,61,73,242,18,159,45,185,232,80,33,...>>} >> > (foo@REDACTED)4> zlib:gzip(Data). >> > <<31,139,8,0,0,0,0,0,0,3,0,15,64,240,191,103,5,115,210, >> > 177,147,53,45,250,182,51,32,250,233,...>> >> > (foo@REDACTED)5> >> > >> > >> > [progski@REDACTED ~/tmp_code/node_timeout] erl -name bar >> > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] >> > [async-threads:0] [hipe] [kernel-poll:false] >> > >> > Eshell V5.8.1 (abort with ^G) >> > (bar@REDACTED)1> nodes(). >> > ['foo@REDACTED'] >> > (bar@REDACTED)2> >> > =ERROR REPORT==== 18-Jan-2011::17:16:10 === >> > ** Node 'foo@REDACTED' not responding ** >> > ** Removing (timedout) connection ** >> > >> > >> > Thanks, >> > >> > -Ryan >> > > From watson.timothy@REDACTED Fri Jan 21 04:27:21 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 21 Jan 2011 03:27:21 +0000 Subject: [erlang-questions] Any tools available to generate code for Erlang Ports? In-Reply-To: <4D363C7B.6030406@gmail.com> References: <4D359BAA.3000606@ecs.soton.ac.uk> <4D363C7B.6030406@gmail.com> Message-ID: > > You might be thinking of GEP[D] > (https://github.com/okeuday/generic-erlang-port--driver-). ?GEP[D] does > everything EDTK does except the weird handling of file descriptors, but > the code was written before NIFs, so it does port driver code (with > async support). > Yep, that was the one. :) From taylor@REDACTED Fri Jan 21 04:46:44 2011 From: taylor@REDACTED (Taylor Venable) Date: Thu, 20 Jan 2011 22:46:44 -0500 Subject: [erlang-questions] Functions with same name and varying arity using apply In-Reply-To: References: <77F30428-2E39-43D4-B49E-BB4D598E0801@googlemail.com> Message-ID: On Wed, Jan 19, 2011 at 22:51, Steve Vinoski wrote: > On Wed, Jan 19, 2011 at 9:44 PM, Taylor Venable wrote: >> With this method, foo/1 and foo/2 have to be exported due to how >> apply/3 works (I assume; that's what the documentation says, and it >> fails if you don't export them). > > If you don't export them, the compiler will think they're unused and > won't include them in the generated beam file, in which case they > obviously won't be around to call by any means. I'm trying to call them from inside the module; foo/1 and foo/2 are helper functions inside the module that are only to be used within the module. >> If I try using apply/2 then that >> takes a function, rather than an atom, so I have to do something like: >> >> bar(fun foo/1, [[1], [2, [double]]]) % from inside the module >> >> Which is not a solution because I have to declare up front whether to >> use foo/1 or foo/2, the opposite of what I'm trying to accomplish. So >> I was wondering if there was another approach to make it work. > > You might consider just always passing a single list argument to foo, > so there's just one to export. It could work like this: Yeah, doing some searching seems to indicate this is the common way of doing variadic functions in Erlang. No point in trying to do something complicated when this method will work. >> The context of discovering this is that in my program I receive the >> arguments to foo (in my example: both [1] and [2, [double]]) from the >> user by way of reading terms (I think that's the right word; I'm >> reading a list of atoms and tuples and such from a file > > Using file:consult/1, I hope? Yup. >> and then using >> them to direct what the program does, like a very simple specification >> of commands; like what one does in Lisp reading a sexp). > > Another option along the Lisp reader line would be to read the user's > terms, and based on those terms first generate "code as data" in > Abstract Format as a set of terms, then compile it and load the > resulting beam on the fly, and finally invoke one of the newly-loaded > functions. Look up Abstract Format in the erts manual and also see the > compile module and the beam_lib module. Whether it's worth it to go to > this extent depends on how often the code would be invoked -- if > you're writing some sort of DSL that's read once and then invoked > numerous times, this approach could be worth it. That's interesting, and a lot for me to digest there. This is just a small program mostly for learning purposes, so I'm going to keep it as simple as I can, but that is something that will definitely merit curious perusal later. -- Taylor C. Venable http://metasyntax.net/ From taylor@REDACTED Fri Jan 21 05:02:33 2011 From: taylor@REDACTED (Taylor Venable) Date: Thu, 20 Jan 2011 23:02:33 -0500 Subject: Shell is terminated (and then restarts) mysteriously Message-ID: Hi there. I'm trying to learn Erlang by working on some simple programs, and as I'm working with the shell I'm noticing some (what to me at least is) odd behaviour. Every once in a while, after compiling some code, it will state that the shell process was terminated. Here's a real example of it: 44> c(tblxform). ./tblxform.erl:90: Warning: variable 'Destination' is unused ./tblxform.erl:94: Warning: variable 'Destination' is unused ./tblxform.erl:94: Warning: variable 'Destination' shadowed in 'fun' ./tblxform.erl:94: Warning: variable 'Source' is unused ./tblxform.erl:94: Warning: variable 'Source' shadowed in 'fun' ./tblxform.erl:106: Warning: variable 'Destination' is unused ./tblxform.erl:107: Warning: variable 'F' is unused ./tblxform.erl:124: Warning: variable 'Destination' is unused ./tblxform.erl:138: Warning: variable 'Destination' is unused ./tblxform.erl:155: Warning: variable 'Destination' is unused ./tblxform.erl:163: Warning: variable 'Fold2' is unused ./tblxform.erl:180: Warning: variable 'Destination' is unused ./tblxform.erl:180: Warning: variable 'Source' is unused ./tblxform.erl:213: Warning: variable 'Destination' is unused ./tblxform.erl:213: Warning: variable 'Options' is unused ./tblxform.erl:236: Warning: variable 'Options' is unused ./tblxform.erl:236: Warning: variable 'Source' is unused *** ERROR: Shell process terminated! *** Eshell V5.8.1 (abort with ^G) 1> This has happened to me three times now while hacking away over the last couple nights, while working on two different pieces of code, in different Erlang VM instances (in case my terminology is wrong, by that I mean running separate "erl" commands in different xterms). After the shell restarts, I can compile and test my code just like I had been doing in the old shell that was killed, so the problem is neither persistent nor reproducible. But when it happens, it's fairly disruptive since anything I had done in the old shell is now gone. I asked for some advice last night on #erlang and posted the code I was compiling then at http://ideone.com/weJTP (just a simple program I'm trying to use to learn some things about Erlang). Obviously the output here doesn't tell me anything about what might have gone wrong; is there any way to get more information about why the shell process was terminated? Here's the banner information from my Erlang installation: Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] -- Taylor C. Venable http://metasyntax.net/ From taylor@REDACTED Fri Jan 21 05:30:15 2011 From: taylor@REDACTED (Taylor Venable) Date: Thu, 20 Jan 2011 23:30:15 -0500 Subject: Shell is terminated (and then restarts) mysteriously In-Reply-To: References: Message-ID: Sorry, I forgot to say that this is on Linux, specifically Ubuntu 10.04; here's my `uname -a`: Linux zareason 2.6.32-27-generic #49-Ubuntu SMP Thu Dec 2 00:51:09 UTC 2010 x86_64 GNU/Linux -- Taylor C. Venable http://metasyntax.net/ From ebegumisa@REDACTED Fri Jan 21 05:34:55 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Fri, 21 Jan 2011 15:34:55 +1100 Subject: [erlang-questions] Erlang Intro/Evangelism Presentation -- Starbucks! In-Reply-To: References: <511794668.258121294521466549.JavaMail.root@zimbra> <179103885.258141294521619929.JavaMail.root@zimbra> Message-ID: When I first read Paul's essay many years ago I thought he was nuts. I agreed with most of Graham's essays except this one. I was initially with Joel Spolsky on this. I've since changed my mind :) I, like many, focused too much on the Lisp element of Paul's argument (probably coz I don't "get" Lisp -- I still don't get it!) Yet the important part of his essay was on questioning popularity. It took me a while to see this. We have this vague notion of "industry standard" which has become just a fancy way of saying "popular". Java, .Net, SQL-RDBMSs, curly brace syntax are "industry standards" because they are popular. Everything else is "weird" and dangerous. Paul's core argument (which I initially missed) was that popularity is never a good reason to do anything, especially if you're trying to innovate. If you do it how most people do it you're more likely to get what most people get -- mediocrity. If I had listened to Paul all those years ago I would have discovered Erlang sooner. I wound have searched for alternatives to RDMBSs sooner. Instead, I chose the misguided safety umbrella of sticking to "industry standards." I wasted many years under that umbrella when I should have been scattering through the rain trying find less crowded shelter elsewhere. But then, save for consulting jobs, I've always worked with small teams in de facto start-up scenarios (a special case as you note). So I'm probably incapable of fully appreciating the other side of the argument: finding interchangeable programmers to maintain/upgrade code in "corporate" environments like Yahoo! Steve Yegge once lamented about how you can't use Erlang or Python at Google (a company many consider to be innovative)... "...a company where there's an engineer with ten years of experience, an architect, who's in your face screaming at you, with spittle flying on you, because you suggested using, you know... D. Or Haskell. Or Lisp, or Erlang, or take your pick... at Google, when I first got there, I was all idealistic. I'm like, wow, well Google hires all these great computer scientists, and so they must all be completely language-agnostic, and ha, ha, little do I know..." Transcript: http://steve-yegge.blogspot.com/2008/05/dynamic-languages-strike-back.html Video: http://ee380.stanford.edu/cgi-bin/videologger.php?target=080507-ee380-300.asx - Edmond - On Sun, 09 Jan 2011 15:34:16 +1100, Michael Turner wrote: > Paul Graham's Beating the Averages can always bear another re-reading > (notwithstanding it's various exaggerations -- I'm sorry, Paul, but > Cobol IS > closer to Python than to assembly language.) You could probably > substitute > "Erlang" for "Lisp" everywhere in the essay, without doing violence to an > excellent point. > > The problem is, Graham is addressing a special case: the startup context, > when there is (almost by definition, from the VC point of view) a market > window for a new class of application. The overwhelming bulk of > production > code in the world does not match that description. Faced with the > problem > of maintaining Yahoo! Stores as production code, Yahoo! rewrote it in C++ > and Perl. Paul Graham opines that they did this only out of Lispophobia, > but I really wonder: if he'd been golden-handcuffed to Yahoo! and put in > a > position to hire underlings, he'd be in an awkward position, wouldn't he? > "Gee, I have to hire quite a few Lisp hackers who are both (a) more or > less > in my league, and (b) all hot to work at Yahoo! on something like Viaweb, > rather than, say, some interesting problem in computational linguistics. > And, uh-oh, in the meantime, I'm getting a million resumes from people > who > have written a few hundred lines of bad Autolisp." I think the pickings > would be slim. In an interesting thread of commentary on Paul Graham's > essay, none other than Dan Weinreb, one of the preeminent Lisp hackers of > our time (or any time) writes: > > "... it would take a lot, or require special circumstances, to persuade > me > to choose Lisp for a major software project, if I were in charge of > making the choice." > > http://discuss.fogcreek.com/joelonsoftware/default.asp?cmd=show&ixPost=31402 > > -michael > turner > > On Sun, Jan 9, 2011 at 6:20 AM, Robert Virding < > robert.virding@REDACTED> wrote: > >> ----- "Edmond Begumisa" wrote: >> >> > Probably true. But I am reminded of something very interesting Ulf has >> > >> > just pointed me to... >> > >> > "The Innovator's Dilemma (C. Christensen) as quoted by Todd A. >> > Proebsting >> > ... >> > >> > "? companies [languages] that did everything right---were in tune with >> > >> > their competition, listened to their customers, and invested >> > aggressively >> > in new technologies---*still* lost their market leadership when >> > confronted >> > with disruptive changes in technology?" >> > >> > http://ll2.ai.mit.edu/talks/proebsting.ppt >> >> This is definitely a very interesting book to read (the Innovator's >> Dilemma). Also interesting is Paul Graham's essay "Beating the >> Averages", >> http://www.paulgraham.com/avg.html , which is also about disruptive >> technologies. >> >> Robert >> >> -- >> Robert Virding, Erlang Solutions Ltd. >> -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From fritchie@REDACTED Fri Jan 21 05:38:33 2011 From: fritchie@REDACTED (Scott Lystig Fritchie) Date: Thu, 20 Jan 2011 22:38:33 -0600 Subject: [erlang-questions] SOA with Erlang In-Reply-To: Message of "Thu, 13 Jan 2011 00:31:43 -0330." Message-ID: <96731.1295584713@snookles.snookles.com> FWIW, while I was at Gemini Mobile Technologies, we had very good luck using UBF(*) to glue together SOA'ish services using components written in JavaScript, Erlang, and C++. It's quite doable and very testable(**). -Scott (*) https://github.com/norton/ubf (**) http://www.erlang-factory.com/upload/presentations/180/ErlangUserConference2009-NortonandFritchie.pdf From ebegumisa@REDACTED Fri Jan 21 06:36:16 2011 From: ebegumisa@REDACTED (Edmond Begumisa) Date: Fri, 21 Jan 2011 16:36:16 +1100 Subject: [erlang-questions] SOA with Erlang In-Reply-To: <96731.1295584713@snookles.snookles.com> References: <96731.1295584713@snookles.snookles.com> Message-ID: UBF should be used more. I've had a very pleasant experience with it. - Edmond - On Fri, 21 Jan 2011 15:38:33 +1100, Scott Lystig Fritchie wrote: > FWIW, while I was at Gemini Mobile Technologies, we had very good luck > using UBF(*) to glue together SOA'ish services using components written > in JavaScript, Erlang, and C++. It's quite doable and very > testable(**). > > -Scott > > (*) https://github.com/norton/ubf > (**) > http://www.erlang-factory.com/upload/presentations/180/ErlangUserConference2009-NortonandFritchie.pdf > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ From ulf.wiger@REDACTED Fri Jan 21 08:19:05 2011 From: ulf.wiger@REDACTED (Ulf Wiger) Date: Fri, 21 Jan 2011 08:19:05 +0100 Subject: [erlang-questions] gproc patch In-Reply-To: References: Message-ID: How funny! I checked by simply substituting 'uwiger' for 'esl' in the commit URL: https://github.com/uwiger/gproc/commit/2742634c2ec608c341d425827eef7f6421849ea5 ?and there it was. Is this some weird artefact of how github interprets its URIs? Anyway, I consider github.com/esl/gproc to be the official one, even though I will strive to keep the uwiger version in synch. BR, Ulf On 20 Jan 2011, at 18:42, Garret Smith wrote: > I was using the version at https://github.com/uwiger/gproc.git which > looks identical except for that patch :) > > -G > > On Thu, Jan 20, 2011 at 9:02 AM, Ulf Wiger > wrote: >> >> Hi Garrett, >> >> Which version of gproc are you using? >> >> That particular bug was fixed on Jan 11 in http://github.com/esl/gproc >> (That is, quite recently. :) >> >> Sending pull requests to that project will work just fine. >> >> BR, >> Ulf W >> >> On 19 Jan 2011, at 01:21, Garret Smith wrote: >> >>> Fixed a small issue with gproc in the 'await' function when no timeout >>> is specified. >>> >>> see branch 'fix-await-notimeout' in the repository >>> git://github.com/garret-smith/gproc.git >>> >>> or the patch: >>> >>> --- a/src/gproc.erl >>> +++ b/src/gproc.erl >>> @@ -312,7 +312,7 @@ request_wait({n,C,_} = Key, Timeout) when C==l; C==g -> >>> WRef = call({await,Key,self()}, C), >>> receive >>> {gproc, WRef, registered, {_K, Pid, V}} -> >>> - case Timeout of >>> + case TRef of >>> no_timer -> ignore; >>> _ -> erlang:cancel_timer(TRef) >>> end, >>> >>> >>> I don't know of any guidelines for patch submission since this is not >>> OTP, so let me know if there is a better way I can provide patches now >>> or in the future. >>> >>> -G >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> Ulf Wiger, CTO, Erlang Solutions, Ltd. >> http://erlang-solutions.com >> >> >> >> Ulf Wiger, CTO, Erlang Solutions, Ltd. http://erlang-solutions.com From gordon@REDACTED Fri Jan 21 09:09:32 2011 From: gordon@REDACTED (Gordon Guthrie) Date: Fri, 21 Jan 2011 08:09:32 +0000 Subject: [erlang-questions] Re: Shell is terminated (and then restarts) mysteriously In-Reply-To: References: Message-ID: My erlang shell has started doing this (very) recently (R14B01 and Ubunutu 10.10) and I found it a bit odd too... On 21 January 2011 04:30, Taylor Venable wrote: > Sorry, I forgot to say that this is on Linux, specifically Ubuntu > 10.04; here's my `uname -a`: Linux zareason 2.6.32-27-generic > #49-Ubuntu SMP Thu Dec 2 00:51:09 UTC 2010 x86_64 GNU/Linux > > -- > Taylor C. Venable > http://metasyntax.net/ > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- Gordon Guthrie CEO hypernumbers http://hypernumbers.com t: hypernumbers +44 7776 251669 From sedrik@REDACTED Fri Jan 21 09:12:54 2011 From: sedrik@REDACTED (Fredrik Andersson) Date: Fri, 21 Jan 2011 09:12:54 +0100 Subject: How do you do TDD in vim? Message-ID: Hi all I am a Vim user that is looking for ways to improve my work flow. I am working hard to adopt TDD but have not been able to find too much good information for a good Vim setup to do it with. Am I the only Vim user? If you are a Vim user, what is your setup? Thanks PS: Please keep the Emacs vs Vim flamewar out of this. From mpalmer@REDACTED Fri Jan 21 09:44:53 2011 From: mpalmer@REDACTED (Matthew Palmer) Date: Fri, 21 Jan 2011 19:44:53 +1100 Subject: How do you do TDD in vim? In-Reply-To: References: Message-ID: <20110121084453.GM3846@hezmatt.org> On Fri, Jan 21, 2011 at 09:12:54AM +0100, Fredrik Andersson wrote: > I am a Vim user that is looking for ways to improve my work flow. I am > working hard to adopt TDD but have not been able to find too much good > information for a good Vim setup to do it with. > > Am I the only Vim user? If you are a Vim user, what is your setup? I'm not a vim user, but I do use a regular text editor. What you want is screen, with tabs. Then you have editor tabs and compliation tabs and test run tabs and whatever else you need. - Matt From ingela.andin@REDACTED Fri Jan 21 09:48:01 2011 From: ingela.andin@REDACTED (Ingela Andin) Date: Fri, 21 Jan 2011 09:48:01 +0100 Subject: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 In-Reply-To: References: <4D36ADAC.3070509@tail-f.com> <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> <4D370678.60203@tail-f.com> Message-ID: Hi! Well PEM-files that only BEGIN PUBLIC KEY and not BEGIN RSA PUBLIC KEY are using another ASN1 spec and can contain both RSA and DSA keys. It seems that it is the ASN1-spec SubjectPublicKeyInfo from PKIXExplicit88.asn1 also part of public_key. Regards Ingela Erlang/OTP team - Ericsson AB P.S Some time ago someone asked for PKCS-8 http://tools.ietf.org/html/rfc5208 support, which ASN1 specs are not currently part of the public_key application and therefore needs to be added, in order to support them. I do not know if this is intrests you it is only so you know. 2011/1/20 Seth Falcon : > Hi there, > > I'm very interested in adding support for RSA public key decoding to > the public_key module. ?My use case is similar in that I need to be > able to verify/decrypt when all I have is the RSA public key in PEM > format. > > 2011/1/20 Ingela Andin : >> The PKCS-1 asn-1 spec is part of public key. PEM-files are mainly >> base64-encoded ?asn1 DER data. > [snip] >> Adding support for the something = RSA PUBLIC KEY should be fairly >> simple as the ASN1- spec is alredy in place. > > I started working on such a patch and you are right that the changes > to read in the pem data are fairly simple. > > However, once I have the DER format, I'm getting stuck with an error > coming out of public_key:der_decode. ?Not sure if I'm doing it wrong > or if there is something wrong with the asn1 definition for RSA public > keys. > > Here's what I'm trying (skipping the pem conversion for now): > > shell: > > ?openssl genrsa > mykey.private > ?openssl rsa -in mykey.private -outform der -pubout > mykey.public.der > > erl: > > ?{ok, PubDer} = file:read_file("mykey.public.der"). > ?public_key:der_decode('RSAPublicKey', PubDer) > > ** exception error: no match of right hand side value {error, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? {asn1, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{function_clause, > > [{'OTP-PUB-KEY',decode_integer, > > [{error,{asn1,{wrong_tag,{16,2}}}}]}, > > {'OTP-PUB-KEY',decode_integer,3}, > > {'OTP-PUB-KEY',dec_RSAPublicKey,2}, > > {'OTP-PUB-KEY',decode,2}, > > {public_key,der_decode,2}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{erl_eval,do_apply,5}, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?{shell,exprs,7}, > > {shell,eval_exprs,7}]}}} > ? ? in function ?public_key:der_decode/2 > > > > Aside: when I use openssl to extract the public key, the output I get > looks like: > > -----BEGIN PUBLIC KEY----- > MFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBALyPPyTS+HyTV3oLf2z6W+H1NOKLos3K > ztRytqJoIdvbzNFYsnJqFhabzhQx2DdPvTiHLZM1QHQ1Ma5yw5hjaQcCAwEAAQ== > -----END PUBLIC KEY----- > > Note that it only says "PUBLIC KEY", not "RSA PUBLIC KEY". ?You will > get identical header values when extracting a DSA public key. > So I'm unsure about how one can detect what type of public key one > has. ?Anyone know the details of how that's supposed to work? ?This > throws a wrinkle in pem_decode/1. ?It may be that one needs to have > pem_decode/2 for public keys where the user specifies what type of key > is expected. > > > If I can get a bit further I will put together a patch. > > Thanks, > > ?+ seth > > -- > Seth Falcon | @sfalcon | http://userprimary.net/ > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From fangee@REDACTED Fri Jan 21 10:52:47 2011 From: fangee@REDACTED (franz) Date: Fri, 21 Jan 2011 10:52:47 +0100 Subject: mnesia memory usage on 32 and 64 bits machines In-Reply-To: <20110101163402.GA4251@talktalkplc.com> References: <4D1DC27A.80507@tiscali.it> <20110101163402.GA4251@talktalkplc.com> Message-ID: <4D39576F.1080704@tiscali.it> I admit I was tempted by the 32-bits architecture. As I'm developing a mnesia-centric distributed application I thought that using 32-bits machine would have reduced the ram consumption and the access speed. So I developed some tests to be run on two identical servers, one running a 32-bits kernel and erlang, the other 64-bits. And everything went fine, until I bumped into the "maximum heap size for 32-bit machines", which I was unaware of: when a mnesia table exceeds 3GB erlang crashes saying <>. In conclusion, for anyone wondering, I think I'm going 64. Regards, franz From n.decker@REDACTED Fri Jan 21 11:03:28 2011 From: n.decker@REDACTED (Decker, Nils) Date: Fri, 21 Jan 2011 11:03:28 +0100 Subject: AW: [erlang-questions] Serious problems with some operations on the ARM platfor. In-Reply-To: References: Message-ID: Hello, I had the same problem two weeks ago when cross compiling erlang to an embedded arm system. The toolchain by the vendor uses an old compiler too: gcc-Version 3.4.3 (MontaVista 3.4.3-25.0.30.0501131 2005-07-23) The bug caused almost all snmp queries to use 100% cpu with a totally unresponsive emulator. Keypresses in the erlang shell took minutes! Using gdb on the emulator i found that the compiler did something funny with erst/emulator/big.c:I_lshift. The shift left branch was taken even when y < 0. This turned a shift right on a bignum to a shift left. This lead to a small loop shifting bytes from a bignum into an infinite loop that built ever growing bignums. (snmp_pdus:eint/2) The workaround for me was a simple CFLAGS=-O1 in my erl-xcomp-xscale_be-linux.conf. One thing i found, is that a single process can completely block a (non-smp) emulator by building large bignums. loop(N) -> _ = N div 3, % some work needed on my desktop but not needed on arm. loop(N bsl 8). The reason is, that one bignum operation consumes just one reduction by the scheduler, but the work done is depends on the size of the bignum. Maybe the size (in bytes) of the bignum should be used for the number reductions consumed by bignum operations? Nils > -----Urspr?ngliche Nachricht----- > Von: erlang-questions@REDACTED > [mailto:erlang-questions@REDACTED] Im Auftrag von Igor Karymov > Gesendet: Donnerstag, 20. Januar 2011 17:01 > An: erlang-questions@REDACTED > Betreff: [erlang-questions] Serious problems with some > operations on the ARM platfor. > > On the last version of erlang (tested on r13b04 and r14a) I > observ next behavior: > > (nmg@REDACTED)2> A = > (nmg@REDACTED)3> B = A bsr 8. > (nmg@REDACTED)4> C = A bsl 8. > (nmg@REDACTED)5> A > B. > true > (nmg@REDACTED)6> C == B. > true > > On older versions (r12b05) ?all are right. > > Compiler: > $ arm-softfloat-linux-gnu-gcc --version > arm-softfloat-linux-gnu-gcc (GCC) 3.4.5 > > Linux: > Linux some 2.6.22.19-4.03.0-c300evm #123 Tue Sep 21 10:00:16 > NOVST 2010 armv6l unknown > > Processor: > CPU: ARMv6-compatible processor [4117b361] revision 1 (ARMv6TEJ) > Nils Decker Software-Ingenieur Projektierung Studio Hamburg Media Consult International (MCI) GmbH Jenfelder Allee 80 | Haus R 2. OG / Raum 257 22039 Hamburg Telefon: +49 (40) 6688-3437 n.decker@REDACTED www.mci-systems.de www.mci-products.de www.mci-werkstaetten.de www.mci-shop.de ................................................................................. Gesch?ftsf?hrung: Ralf Schimmel (Vorsitzender) Maximilian Below Prokuristen: J?rn Denneborg Kerstin G?ing J?rg Pankow Amtsgericht Hamburg: HRB 70454 Bitte denken Sie an die Umwelt, bevor Sie diese E-Mail drucken. From dgud@REDACTED Fri Jan 21 11:16:18 2011 From: dgud@REDACTED (Dan Gudmundsson) Date: Fri, 21 Jan 2011 11:16:18 +0100 Subject: [erlang-questions] linked-in driver blocks net_kernel tick? In-Reply-To: References: Message-ID: All c-calls blocks a schedulers, if they are not pushed out to a thread. In this case it's a bug in the zlib module (probably by me) gzip should chunk up the input before invoking the driver. What happens is that all schedulers go to sleep because there is no work to do, except the one invoking the driver, a ping is received and wakes up the "distribution" process which gets queued up on only scheduler that is awake, but that scheduler is blocked in an "eternal" call. The pings never become processed and the distributions times out. You can wait for a patch or use zlib api to chunk up compression your self, see implementation of gzip in zlib module. /Dan On Fri, Jan 21, 2011 at 2:48 AM, Ryan Zezeski wrote: > So...can anyone explain to me why zlib:gzip/1 is causing the net_kernel tick > to be blocked? ?Do linked-in drivers block it's scheduler like NIFs? ?I'm > really curious on this one :) > > -Ryan > > On Tue, Jan 18, 2011 at 6:53 PM, Ryan Zezeski wrote: > >> Apologies, the example I copied was run on my mac. >> >> This is what I have on the actual production machine: >> >> Erlang R14A (erts-5.8) [source] [64-bit] [smp:16:16] [rq:16] >> [async-threads:0] [hipe] [kernel-poll:false] >> >> To be certain, I ran the same example (except this time using two physical >> machines) and achieved the same result. ?Namely, the 'bar' node claims 'foo' >> is not responding and thus closes the connection. ?Whatever this is, I've >> now easily reproduced it on two different OSs, with 2 different Erlang >> versions. >> >> -Ryan >> >> On Tue, Jan 18, 2011 at 6:04 PM, Alain O'Dea wrote: >> >>> On 2011-01-18, at 18:54, Ryan Zezeski wrote: >>> >>> > Hi everyone, >>> > >>> > Some of you may remember my latest question where I was having weird >>> node >>> > timeout issues that I couldn't explain and I thought it might be related >>> to >>> > the messages I was passing between my nodes. ?Well, I pinpointed the >>> problem >>> > to a call to zlib:gzip/1. ?At first I was really surprised by this, as >>> such >>> > a harmless line of code surely should have nothing to do with the >>> ability >>> > for my nodes to communicate. ?However, as I dug further I realized gzip >>> was >>> > implemented as a linked-in driver and I remember reading things about >>> how >>> > one has to take care with them because they can trash the VM with them. >>> ?I >>> > don't remember reading anything about them blocking code, and even if >>> they >>> > do I fail to see why my SMP enabled node (16 cores) would allow this one >>> > thread to block the tick. ?It occurred to me that maybe the scheduler >>> > responsible for that process is the one blocked by the driver. ?Do >>> processes >>> > have scheduler affinity? ?That would make sense, I guess. >>> > >>> > I've "fixed" this problem simply by using a plain port (i.e. run in it's >>> own >>> > OS process). ?For my purposes, this actually makes more sense in the >>> > majority of the places I was making use of gzip. ?Can someone enlighten >>> me >>> > as to exactly what is happening behind the scenes? >>> > >>> > To reproduce I create a random 1.3GB file: >>> > >>> > dd if=/dev/urandom of=rand bs=1048576 count=1365 >>> > >>> > Then start two named nodes 'foo' and 'bar', connect them, read in the >>> file, >>> > and then compress said file. ?Sometime later (I think around 60+ >>> seconds) >>> > the node 'bar' will claim that 'foo' is not responding. >>> > >>> > [progski@REDACTED ~/tmp_code/node_timeout] erl -name foo >>> > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] >>> >>> Your SMP node seems to be capped at smp:2:2 when it out to be smp:16. >>> ?Some resource limit may be holding back the system. That said zlib should >>> not ever cause this issue. >>> >>> > [async-threads:0] [hipe] [kernel-poll:false] >>> > >>> > Eshell V5.8.1 ?(abort with ^G) >>> > (foo@REDACTED)1> net_adm:ping('bar@REDACTED'). >>> > pong >>> > (foo@REDACTED)2> nodes(). >>> > ['bar@REDACTED'] >>> > (foo@REDACTED)3> {ok,Data} = file:read_file("rand"). >>> > {ok,<<103,5,115,210,177,147,53,45,250,182,51,32,250,233, >>> > ? ? ?39,253,102,61,73,242,18,159,45,185,232,80,33,...>>} >>> > (foo@REDACTED)4> zlib:gzip(Data). >>> > <<31,139,8,0,0,0,0,0,0,3,0,15,64,240,191,103,5,115,210, >>> > ?177,147,53,45,250,182,51,32,250,233,...>> >>> > (foo@REDACTED)5> >>> > >>> > >>> > [progski@REDACTED ~/tmp_code/node_timeout] erl -name bar >>> > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] >>> > [async-threads:0] [hipe] [kernel-poll:false] >>> > >>> > Eshell V5.8.1 ?(abort with ^G) >>> > (bar@REDACTED)1> nodes(). >>> > ['foo@REDACTED'] >>> > (bar@REDACTED)2> >>> > =ERROR REPORT==== 18-Jan-2011::17:16:10 === >>> > ** Node 'foo@REDACTED' not responding ** >>> > ** Removing (timedout) connection ** >>> > >>> > >>> > Thanks, >>> > >>> > -Ryan >>> >> >> > From robert.virding@REDACTED Fri Jan 21 12:04:07 2011 From: robert.virding@REDACTED (Robert Virding) Date: Fri, 21 Jan 2011 11:04:07 +0000 (GMT) Subject: AW: [erlang-questions] Serious problems with some operations on the ARM platfor. In-Reply-To: <2133665806.320471295607165155.JavaMail.root@zimbra> Message-ID: <1731514594.320521295607847061.JavaMail.root@zimbra> ----- "Nils Decker" wrote: > Hello, > > ... > > One thing i found, is that a single process can completely block a > (non-smp) emulator by building large bignums. > loop(N) -> > _ = N div 3, % some work needed on my desktop but not needed on > arm. > loop(N bsl 8). > > The reason is, that one bignum operation consumes just one reduction > by the scheduler, but the work done is depends on the size of the > bignum. Maybe the size (in bytes) of the bignum should be used for the > number reductions consumed by bignum operations? This would be a good modification for all BIFs and internal operations irrespective of SMP or no-SMP, though I believe that it is already done in many places. Robert -- Robert Virding, Erlang Solutions Ltd. From jocke@REDACTED Fri Jan 21 12:47:11 2011 From: jocke@REDACTED (Joakim G.) Date: Fri, 21 Jan 2011 12:47:11 +0100 Subject: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 In-Reply-To: References: <4D36ADAC.3070509@tail-f.com> <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> <4D370678.60203@tail-f.com> <4D38B9C5.9030002@tail-f.com> Message-ID: <4D39723F.3060507@tail-f.com> With this patch public_key:pem_decode/1 returns a RSAPublicKey record as defined in public_key.hrl (instead of just []). I haven't used the output yet though. :-) I will experiment with public_key:verify/4 over the weekend. Cheers /Jocke On 2011-01-21 00:03, Seth Falcon wrote: > 2011/1/20 Joakim G. : >> I updated pubkey_pem.html a tiny little bit and it seems to >> work: >> >> jocke@REDACTED:~/installs/otp_src_R14B01/lib/public_key/src$ diff -bwt >> pubkey_pem.erl.orig pubkey_pem.erl >> 157a158,159 >>> join_entry([<<"-----END RSA PUBLIC KEY-----", _/binary>>| Lines], >> Entry) -> >>> {lists:reverse(Entry), Lines}; >> 212a215,216 >>> pem_start('RSAPublicKey') -> >>> <<"-----BEGIN RSA PUBLIC KEY-----">>; >> 221a226,227 >>> pem_end(<<"-----BEGIN RSA PUBLIC KEY-----">>) -> >>> <<"-----END RSA PUBLIC KEY-----">>; >> 232a239,240 >>> asn1_type(<<"-----BEGIN RSA PUBLIC KEY-----">>) -> >>> 'RSAPublicKey'; >> >> Thanks > > Our mails crossed. When you say "this seems to work", were you able > to actually use the public key for verification/decryption? > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From hynek@REDACTED Fri Jan 21 13:02:24 2011 From: hynek@REDACTED (Hynek Vychodil) Date: Fri, 21 Jan 2011 13:02:24 +0100 Subject: [erlang-questions] How do you do TDD in vim? In-Reply-To: References: Message-ID: I'm vim user and I don't use any special setting. On Fri, Jan 21, 2011 at 9:12 AM, Fredrik Andersson wrote: > Hi all > > I am a Vim user that is looking for ways to improve my work flow. I am > working hard to adopt TDD but have not been able to find too much good > information for a good Vim setup to do it with. > > Am I the only Vim user? If you are a Vim user, what is your setup? > > Thanks > > PS: Please keep the Emacs vs Vim flamewar out of this. > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From hynek@REDACTED Fri Jan 21 13:06:42 2011 From: hynek@REDACTED (Hynek Vychodil) Date: Fri, 21 Jan 2011 13:06:42 +0100 Subject: [erlang-questions] Re: mnesia memory usage on 32 and 64 bits machines In-Reply-To: <4D39576F.1080704@tiscali.it> References: <4D1DC27A.80507@tiscali.it> <20110101163402.GA4251@talktalkplc.com> <4D39576F.1080704@tiscali.it> Message-ID: If I remember right There was announced nice optimization of Erlang process heap using 32 "pointers" on 64-bit architecture on SFO Erlang Factory last year and I believe it is included in R14B version. Anyway I'm not sure and didn't verify. On Fri, Jan 21, 2011 at 10:52 AM, franz wrote: > I admit I was tempted by the 32-bits architecture. As I'm developing a > mnesia-centric distributed application I thought that using 32-bits machine > would have reduced the ram consumption and the access speed. So I developed > some tests to be run on two identical servers, one running a 32-bits kernel > and erlang, the other 64-bits. > And everything went fine, until I bumped into the "maximum heap size for > 32-bit machines", which I was unaware of: when a mnesia table exceeds 3GB > erlang crashes saying < memory (of type "heap")>>. > In conclusion, for anyone wondering, I think I'm going 64. > Regards, > franz > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- --Hynek (Pichi) Vychodil Analyze your data in minutes. Share your insights instantly. Thrill your boss.? Be a data hero! Try GoodData now for free: www.gooddata.com From matthias@REDACTED Fri Jan 21 13:13:44 2011 From: matthias@REDACTED (Matthias Lang) Date: Fri, 21 Jan 2011 13:13:44 +0100 Subject: how do I replace the Erlang shell with my own shell? Message-ID: <20110121121343.GA3672@corelatus.se> I want to write my own shell for Erlang. I want to be able to switch back and forth between my shell and the regular one without restarting the VM. I started off writing my own autocomplete. That's easy: just call io:setopts([{expand_fun, Fun}]). Details are in the 'io' manpage. But replacing the shell seems much harder. Two unsatifying approaches: 1. Write a module called shell.erl, i.e. same name as the one shipped by erlang.org, and use code loading to switch between my version and the normal one. That would switch _all_ shell instances at the same time. 2. Hack shell.erl and make it provide two variants of shell behaviour. I'll then have to merge changes forever. There must be a better way. Digging through how input comes in, I find: First, user_sup.erl starts either of two descriptively-named modules: user_drv.erl or user.erl, depending on options. The default is user_drv, but any of '-noshell', '-oldshell', '-noinput' cause 'user' to be started. There's also an undocumented switch, '-user' which lets you specify a module. Actual input comes from the 'tty_sl' port. user_drv.erl seems to be 600 lines of fiddling. The only feature it seems to implement is the ^G menu. The module name 'shell' is hardcoded into all of the variants of user_drv:start() which can be called from user_sup. user.erl seems to be 800 lines of roughly the same thing as user_drv.erl. Conclusion: If I want to write my own shell, I also have to write a replacement for user_drv. Right? Matt From matthias@REDACTED Fri Jan 21 13:25:50 2011 From: matthias@REDACTED (Matthias Lang) Date: Fri, 21 Jan 2011 13:25:50 +0100 Subject: yet another way to communicate between processes Message-ID: <20110121122550.GB3672@corelatus.se> How many ways can processes communicate? - Message passing, obviously. - Going outside Erlang, e.g. through files, sockets, pipes, etc. - ETS - Hipe lets you destructively update binaries Today, I saw a new one: (ab)use erlang:process_info(Pid, dictionary) to read another process' process dictionary. user_drv:interfaces/1 does it. Matt From robert.virding@REDACTED Fri Jan 21 13:31:59 2011 From: robert.virding@REDACTED (Robert Virding) Date: Fri, 21 Jan 2011 12:31:59 +0000 (GMT) Subject: [erlang-questions] how do I replace the Erlang shell with my own shell? In-Reply-To: <20110121121343.GA3672@corelatus.se> Message-ID: <1345550962.321711295613119594.JavaMail.root@zimbra> I use this method for LFE. I have an lfe_boot.erl which looks like: -module(lfe_boot). -export([start/0]). start() -> user_drv:start(['tty_sl -c -e',{lfe_shell,start,[]}]). I then start erlang with script lfe: #! /bin/bash # Run LFE shell by default. Can add -pa if necessary. erl "$@" -noshell -noinput -s lfe_boot start This works well on Mac/Linux but not on Windows. One problem is that 'user_drv' does not remember which shell it was given as an argument so 'shell' is always the default. I will fix that one of these days. User_drv handles ^G and the ability to start multiple "jobs" and switch user io between them. Robert ----- "Matthias Lang" wrote: > I want to write my own shell for Erlang. I want to be able to switch > back and forth between my shell and the regular one without > restarting > the VM. > > I started off writing my own autocomplete. That's easy: just call > io:setopts([{expand_fun, Fun}]). Details are in the 'io' manpage. > > But replacing the shell seems much harder. Two unsatifying > approaches: > > 1. Write a module called shell.erl, i.e. same name as the one > shipped by erlang.org, and use code loading to switch between my > version and the normal one. That would switch _all_ shell > instances at the same time. > > 2. Hack shell.erl and make it provide two variants of shell > behaviour. I'll then have to merge changes forever. > > There must be a better way. Digging through how input comes in, I > find: > > First, user_sup.erl starts either of two descriptively-named > modules: user_drv.erl or user.erl, depending on options. The > default > is user_drv, but any of '-noshell', '-oldshell', '-noinput' cause > 'user' to be started. There's also an undocumented switch, '-user' > which lets you specify a module. Actual input comes from the > 'tty_sl' port. > > user_drv.erl seems to be 600 lines of fiddling. The only feature it > seems to implement is the ^G menu. The module name 'shell' is > hardcoded > into all of the variants of user_drv:start() which can be called > from > user_sup. > > user.erl seems to be 800 lines of roughly the same thing as > user_drv.erl. > > Conclusion: If I want to write my own shell, I also have to write a > replacement for user_drv. Right? > > Matt > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED -- Robert Virding, Erlang Solutions Ltd. From kostis@REDACTED Fri Jan 21 13:42:36 2011 From: kostis@REDACTED (Kostis Sagonas) Date: Fri, 21 Jan 2011 14:42:36 +0200 Subject: [erlang-questions] yet another way to communicate between processes In-Reply-To: <20110121122550.GB3672@corelatus.se> References: <20110121122550.GB3672@corelatus.se> Message-ID: <4D397F3C.9010004@cs.ntua.gr> Matthias Lang wrote: > How many ways can processes communicate? > > - Message passing, obviously. > > - ... > > - Hipe lets you destructively update binaries I understand the point of Matt's message, but the phrasing of the last item on his list is a bit exaggerated. HiPE does NOT let you destructively update binaries! HiPE implements (mostly for its own consumption) a byte/bit array interface that just happens to currently use binaries as its underlying representation. The term which is returned is really opaque and cannot be forged. 42> hipe_bifs:array(42,0). 1341051852164310958483470496723442421146438490410398599188212234810775184469773903675250096414324892498977265047852129698138433859883549051251117502043807487620028142015641359445529065620292765912887098241208265312044730234611304175737605272347266134465468817112674654480500333272504281215210377271949900335044555573928768066406428898637749039097161515519403482064915448724323249585518139188183055 You can not possibly pass a random binary to the appropriate BIFs and destructively update their contents. 43> Bin = << <<1>> || _ <- lists:seq(1,42)>>. <<1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, 1,...>> 44> hipe_bifs:array_update(Bin, 2, 0). ** exception error: bad argument in function hipe_bifs:array_update/3 called as hipe_bifs:array_update(<<1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1, 1,...>>, 2,0) Of course, Matt is right that this mechanism could be used for communication: if one process does pass this array handle in a message to some other process, the other process can use this to read from or write to this array. Cheers, Kostis From mk@REDACTED Fri Jan 21 13:56:25 2011 From: mk@REDACTED (Morten Krogh) Date: Fri, 21 Jan 2011 13:56:25 +0100 Subject: [erlang-questions] yet another way to communicate between processes In-Reply-To: <20110121122550.GB3672@corelatus.se> References: <20110121122550.GB3672@corelatus.se> Message-ID: <4D398279.5080902@amberbio.com> Mutating Nifs, if that wasn't included in your list somehow. Cpu usage: Process A and B wants to communicate a bit. A starts a long computation a little before a predefined time if the bit is 1. B probes the CPU by timing known functions, and can hence see if A is using the CPU. This requires that A and B share CPU and that no other process starts a heavy job. The last one can be fixed by a more elaborate scheme. A, either do or don't start a computation at various predefined times, that no other process randomly would hit. This also gives error correction. Morten. On 1/21/11 1:25 PM, Matthias Lang wrote: > How many ways can processes communicate? > > - Message passing, obviously. > > - Going outside Erlang, e.g. through files, sockets, pipes, etc. > > - ETS > > - Hipe lets you destructively update binaries > > Today, I saw a new one: (ab)use erlang:process_info(Pid, dictionary) to > read another process' process dictionary. user_drv:interfaces/1 does it. > > Matt > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From billiob@REDACTED Fri Jan 21 14:50:25 2011 From: billiob@REDACTED (Boris 'billiob' Faure) Date: Fri, 21 Jan 2011 14:50:25 +0100 Subject: [erlang-questions] How do you do TDD in vim? In-Reply-To: References: Message-ID: On Fri, Jan 21, 2011 at 09:12, Fredrik Andersson wrote: > [?] > Am I the only Vim user? If you are a Vim user, what is your setup? I use vimerl: https://github.com/oscarh/vimerl -- Boris 'billiob' Faure From sedrik@REDACTED Fri Jan 21 14:58:11 2011 From: sedrik@REDACTED (Fredrik Andersson) Date: Fri, 21 Jan 2011 14:58:11 +0100 Subject: [erlang-questions] How do you do TDD in vim? In-Reply-To: References: Message-ID: On Fri, Jan 21, 2011 at 2:50 PM, Boris 'billiob' Faure wrote: > On Fri, Jan 21, 2011 at 09:12, Fredrik Andersson > wrote: > > [?] > > Am I the only Vim user? If you are a Vim user, what is your setup? > > I use vimerl: https://github.com/oscarh/vimerl > Nice it was things like this I was looking for. If anyone has more stuff like this that is worth trying I am interesting, would love something eunit/quickcheck/proper/dialyzer specific stuff aswell > -- > Boris 'billiob' Faure > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From jesper.louis.andersen@REDACTED Fri Jan 21 15:31:21 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 21 Jan 2011 15:31:21 +0100 Subject: [erlang-questions] How do you do TDD in vim? In-Reply-To: References: Message-ID: On Fri, Jan 21, 2011 at 14:58, Fredrik Andersson wrote: > > Nice it was things like this I was looking for. If anyone has more stuff > like this that is worth trying I am interesting, would love something > eunit/quickcheck/proper/dialyzer specific stuff aswell > The rebar tool on the command line is a nice helper for running tests and so on. It also has the distinct advantage that it is ${EDITOR} agnostic. So should you some day decide that acme[1] is a better editor, you can easily change to it. And your project is easier for others to pick up, whether or not they use the same editor as you. [1] http://en.wikipedia.org/wiki/Acme_(text_editor) -- J. From b.ghose@REDACTED Fri Jan 21 16:51:20 2011 From: b.ghose@REDACTED (Baishampayan Ghose) Date: Fri, 21 Jan 2011 21:21:20 +0530 Subject: [erlang-questions] Erlang project dependency management/build tools In-Reply-To: References: Message-ID: Eric, > ?There are two ways, the first is you can just publish the package > yourself, then everyone benefits or the easiest way is just to do a > local install with faxien. That would look like > > faxien ia > > and then its available to everything. Is there any way for me to publish webmachine to the Erlware repository? Regards, BG -- Baishampayan Ghose b.ghose at gmail.com From silent_vendetta@REDACTED Fri Jan 21 17:06:08 2011 From: silent_vendetta@REDACTED (Chris Hicks) Date: Fri, 21 Jan 2011 08:06:08 -0800 Subject: [erlang-questions] Re: Shell is terminated (and then restarts) mysteriously In-Reply-To: References: ,, Message-ID: I've noticed this happening on Windows 7 as well, and always when I've compiled a module multiple times within a relatively short span of time. > Date: Fri, 21 Jan 2011 08:09:32 +0000 > From: gordon@REDACTED > To: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Re: Shell is terminated (and then restarts) mysteriously > > My erlang shell has started doing this (very) recently (R14B01 and Ubunutu > 10.10) and I found it a bit odd too... > > On 21 January 2011 04:30, Taylor Venable wrote: > > > Sorry, I forgot to say that this is on Linux, specifically Ubuntu > > 10.04; here's my `uname -a`: Linux zareason 2.6.32-27-generic > > #49-Ubuntu SMP Thu Dec 2 00:51:09 UTC 2010 x86_64 GNU/Linux > > > > -- > > Taylor C. Venable > > http://metasyntax.net/ > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > > -- > Gordon Guthrie > CEO hypernumbers > > http://hypernumbers.com > t: hypernumbers > +44 7776 251669 From dujinfang@REDACTED Fri Jan 21 17:10:12 2011 From: dujinfang@REDACTED (Seven Du) Date: Sat, 22 Jan 2011 00:10:12 +0800 Subject: mysql driver in erlang support stored procedure Message-ID: Hi, I was using erlang_mysql_driver for a while, but it seems doesn't support stored procedure. Then I found another driver called EMysql https://github.com/JacobVorreuter/emysql , on test it does support call procedure, however, it returns ok_packets sometimes on select only procedures, and it happens also for simple selects(where it should always be result_packet I think): fetch4() -> SQL = "select * from test;", Result = emysql:execute(mysql2, SQL), io:format("~p~n", [Result]), Fields = Result#result_packet.field_list, Rows = Result#result_packet.rows, io:format("~p~n", [Fields]), io:format("~p~n", [Rows]). > mytest:fetch4(). {ok_packet,28,0,0,2,0,[]} ** exception error: {badrecord,result_packet} in function m_telsystem:fetch/0 I should report an issue on github but the only issue there is still in unread state and it's not been updated since 2009. There are other clones of the project, but not sure which one will has support. Any one has experience on this or other options? erlang_mysql_driver is quite state besides I haven't got it support stored procedure. Thanks. -- About: http://about.me/dujinfang Blog: http://www.dujinfang.com Proj:? http://www.freeswitch.org.cn From alain.odea@REDACTED Fri Jan 21 18:01:36 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Fri, 21 Jan 2011 13:31:36 -0330 Subject: [erlang-questions] gproc patch In-Reply-To: References: Message-ID: <3C5F9F26-8310-4E0E-A3FA-8CF2AE3FC95F@gmail.com> It's more likely that the commit is present in both repositories, but one does not have it on the master branch. On 2011-01-21, at 3:49, Ulf Wiger wrote: > > How funny! I checked by simply substituting 'uwiger' for 'esl' > in the commit URL: > > https://github.com/uwiger/gproc/commit/2742634c2ec608c341d425827eef7f6421849ea5 > > ?and there it was. > > Is this some weird artefact of how github interprets its URIs? > > Anyway, I consider github.com/esl/gproc to be the official one, > even though I will strive to keep the uwiger version in synch. > > BR, > Ulf > > On 20 Jan 2011, at 18:42, Garret Smith wrote: > >> I was using the version at https://github.com/uwiger/gproc.git which >> looks identical except for that patch :) >> >> -G >> >> On Thu, Jan 20, 2011 at 9:02 AM, Ulf Wiger >> wrote: >>> >>> Hi Garrett, >>> >>> Which version of gproc are you using? >>> >>> That particular bug was fixed on Jan 11 in http://github.com/esl/gproc >>> (That is, quite recently. :) >>> >>> Sending pull requests to that project will work just fine. >>> >>> BR, >>> Ulf W >>> >>> On 19 Jan 2011, at 01:21, Garret Smith wrote: >>> >>>> Fixed a small issue with gproc in the 'await' function when no timeout >>>> is specified. >>>> >>>> see branch 'fix-await-notimeout' in the repository >>>> git://github.com/garret-smith/gproc.git >>>> >>>> or the patch: >>>> >>>> --- a/src/gproc.erl >>>> +++ b/src/gproc.erl >>>> @@ -312,7 +312,7 @@ request_wait({n,C,_} = Key, Timeout) when C==l; C==g -> >>>> WRef = call({await,Key,self()}, C), >>>> receive >>>> {gproc, WRef, registered, {_K, Pid, V}} -> >>>> - case Timeout of >>>> + case TRef of >>>> no_timer -> ignore; >>>> _ -> erlang:cancel_timer(TRef) >>>> end, >>>> >>>> >>>> I don't know of any guidelines for patch submission since this is not >>>> OTP, so let me know if there is a better way I can provide patches now >>>> or in the future. >>>> >>>> -G >>>> >>>> ________________________________________________________________ >>>> erlang-questions (at) erlang.org mailing list. >>>> See http://www.erlang.org/faq.html >>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>> >>> >>> Ulf Wiger, CTO, Erlang Solutions, Ltd. >>> http://erlang-solutions.com >>> >>> >>> >>> > > Ulf Wiger, CTO, Erlang Solutions, Ltd. > http://erlang-solutions.com > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From michael.eugene.turner@REDACTED Fri Jan 21 17:59:58 2011 From: michael.eugene.turner@REDACTED (Michael Turner) Date: Sat, 22 Jan 2011 01:59:58 +0900 Subject: [erlang-questions] Erlang Intro/Evangelism Presentation -- Starbucks! In-Reply-To: References: <511794668.258121294521466549.JavaMail.root@zimbra> <179103885.258141294521619929.JavaMail.root@zimbra> Message-ID: "Paul's core argument (which I initially missed) was that popularity is never a good reason to do anything, especially if you're trying to innovate. If you do it how most people do it you're more likely to get what most people get -- mediocrity." True, modulo programmer talent. But programmer talent varies by an order of magnitude. So it still comes down to programmer ability, mainly. And ability to do what, exactly, in software? Most work in software is still where it's been since the 1960s: in maintenance of existing systems. If a large Java system has a bug, you're not going to fix it in Ruby. If a large PHP/MySql system needs a modest new feature, you're not going add it in Erlang. There are times when you should question popularity, and Paul Graham's essay makes a good case that people don't question it enough, at those times -- and that, because of that, if you do question it, at those times, you can win. But when you're maintaining a large, sprawling, legacy system, you're not doing it in that system's (main) language because the language is popular -- I've even had to do maintenance in Fortran, a language that, as far as I can tell, was unpopular almost the day it was born. No, you're doing it because to do otherwise is either impossible or so awkward at the interfaces as to not be worth it, and/or because you're asking other programmers to learn and like the new language you choose even when it is possible to incrementally change the code base in the new language, and/or because to start rewriting the system from scratch is nowhere near an option, financially, for the profit-making (you hope) entity you work for. This is an aspect of the software profession that I suspect Paul Graham, for all his other deep insights, has never really been exposed to, and which he doesn't really understand at the level of migraine headaches, unlike most programmers. Near as I can make out from his own bio, he went from CS grad school to art school to writing Viaweb from scratch in Lisp to cashing out big-time from the Yahoo! acquisition of Viaweb to becoming a VC. Nice work (and play) if you can get it. -michael turner On Fri, Jan 21, 2011 at 1:34 PM, Edmond Begumisa < ebegumisa@REDACTED> wrote: > When I first read Paul's essay many years ago I thought he was nuts. I > agreed with most of Graham's essays except this one. I was initially with > Joel Spolsky on this. > > I've since changed my mind :) I, like many, focused too much on the Lisp > element of Paul's argument (probably coz I don't "get" Lisp -- I still don't > get it!) Yet the important part of his essay was on questioning popularity. > It took me a while to see this. > > We have this vague notion of "industry standard" which has become just a > fancy way of saying "popular". Java, .Net, SQL-RDBMSs, curly brace syntax > are "industry standards" because they are popular. Everything else is > "weird" and dangerous. Paul's core argument (which I initially missed) was > that popularity is never a good reason to do anything, especially if you're > trying to innovate. If you do it how most people do it you're more likely to > get what most people get -- mediocrity. > > If I had listened to Paul all those years ago I would have discovered > Erlang sooner. I wound have searched for alternatives to RDMBSs sooner. > Instead, I chose the misguided safety umbrella of sticking to "industry > standards." I wasted many years under that umbrella when I should have been > scattering through the rain trying find less crowded shelter elsewhere. > > But then, save for consulting jobs, I've always worked with small teams in > de facto start-up scenarios (a special case as you note). So I'm probably > incapable of fully appreciating the other side of the argument: finding > interchangeable programmers to maintain/upgrade code in "corporate" > environments like Yahoo! > > Steve Yegge once lamented about how you can't use Erlang or Python at > Google (a company many consider to be innovative)... > > "...a company where there's an engineer with ten years of experience, an > architect, who's in your face screaming at you, with spittle flying on you, > because you suggested using, you know... D. Or Haskell. Or Lisp, or Erlang, > or take your pick... at Google, when I first got there, I was all > idealistic. I'm like, wow, well Google hires all these great computer > scientists, and so they must all be completely language-agnostic, and ha, > ha, little do I know..." > > Transcript: > http://steve-yegge.blogspot.com/2008/05/dynamic-languages-strike-back.html > Video: > http://ee380.stanford.edu/cgi-bin/videologger.php?target=080507-ee380-300.asx > > - Edmond - > > > > On Sun, 09 Jan 2011 15:34:16 +1100, Michael Turner < > michael.eugene.turner@REDACTED> wrote: > > Paul Graham's Beating the Averages can always bear another re-reading >> (notwithstanding it's various exaggerations -- I'm sorry, Paul, but Cobol >> IS >> closer to Python than to assembly language.) You could probably >> substitute >> "Erlang" for "Lisp" everywhere in the essay, without doing violence to an >> excellent point. >> >> The problem is, Graham is addressing a special case: the startup context, >> when there is (almost by definition, from the VC point of view) a market >> window for a new class of application. The overwhelming bulk of >> production >> code in the world does not match that description. Faced with the problem >> of maintaining Yahoo! Stores as production code, Yahoo! rewrote it in C++ >> and Perl. Paul Graham opines that they did this only out of Lispophobia, >> but I really wonder: if he'd been golden-handcuffed to Yahoo! and put in a >> position to hire underlings, he'd be in an awkward position, wouldn't he? >> "Gee, I have to hire quite a few Lisp hackers who are both (a) more or >> less >> in my league, and (b) all hot to work at Yahoo! on something like Viaweb, >> rather than, say, some interesting problem in computational linguistics. >> And, uh-oh, in the meantime, I'm getting a million resumes from people >> who >> have written a few hundred lines of bad Autolisp." I think the pickings >> would be slim. In an interesting thread of commentary on Paul Graham's >> essay, none other than Dan Weinreb, one of the preeminent Lisp hackers of >> our time (or any time) writes: >> >> "... it would take a lot, or require special circumstances, to persuade me >> to choose Lisp for a major software project, if I were in charge of >> making the choice." >> >> >> http://discuss.fogcreek.com/joelonsoftware/default.asp?cmd=show&ixPost=31402 >> >> < >> http://discuss.fogcreek.com/joelonsoftware/default.asp?cmd=show&ixPost=31402 >> >-michael >> turner >> >> On Sun, Jan 9, 2011 at 6:20 AM, Robert Virding < >> robert.virding@REDACTED> wrote: >> >> ----- "Edmond Begumisa" wrote: >>> >>> > Probably true. But I am reminded of something very interesting Ulf has >>> > >>> > just pointed me to... >>> > >>> > "The Innovator's Dilemma (C. Christensen) as quoted by Todd A. >>> > Proebsting >>> > ... >>> > >>> > "? companies [languages] that did everything right---were in tune with >>> > >>> > their competition, listened to their customers, and invested >>> > aggressively >>> > in new technologies---*still* lost their market leadership when >>> > confronted >>> > with disruptive changes in technology?" >>> > >>> > http://ll2.ai.mit.edu/talks/proebsting.ppt >>> >>> This is definitely a very interesting book to read (the Innovator's >>> Dilemma). Also interesting is Paul Graham's essay "Beating the Averages", >>> http://www.paulgraham.com/avg.html , which is also about disruptive >>> technologies. >>> >>> Robert >>> >>> -- >>> Robert Virding, Erlang Solutions Ltd. >>> >>> > > -- > Using Opera's revolutionary e-mail client: http://www.opera.com/mail/ > From thomasl_erlang@REDACTED Thu Jan 20 18:31:02 2011 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Thu, 20 Jan 2011 09:31:02 -0800 (PST) Subject: [erlang-questions] HiPE performance gain (Was: Re: [erlang-questions] Erlang static linked interpreter) In-Reply-To: References: <871318.87934.qm@web111404.mail.gq1.yahoo.com> Message-ID: <352094.70491.qm@web111413.mail.gq1.yahoo.com> ----- Original Message ---- > From: Michael Turner ... > How the emulator is coded probably makes a difference too. In my few visits > to the BEAM VM code, I was struck by the amount of inlining (and via long C > macros, whoa, scary!) I began to wonder whether those macros really paid > off, when you consider the more limited I-caches. If their bloat also tends > to crowd out HiPE-generated native code, that's another performance ding, > albeit more indirectly suffered. Wouldn't it make more sense to recode > those macros as static functions and try various compiler optimization flags > relevant to automatic inlining of functions? A smaller (in generated native > code) VM that *seemed* to have more function call overhead might > nevertheless actually be a win for HiPE-compiled code. Maybe a win more > generally, at least on some architectures. For giving HiPE a chance, it > might be worth an experiment or two. It might be interesting to do a trace based simulation and comparison of the I-cache performance of the emulator in various scenarios. Or just check whether there are any palpable problems on a local setup, of course. I think Mattias Lang has noted that the emulator has grown quite a bit over the years, is that a problem for instance? Best regards, Thomas From garret.smith@REDACTED Fri Jan 21 18:40:27 2011 From: garret.smith@REDACTED (Garret Smith) Date: Fri, 21 Jan 2011 09:40:27 -0800 Subject: [erlang-questions] gproc patch In-Reply-To: <3C5F9F26-8310-4E0E-A3FA-8CF2AE3FC95F@gmail.com> References: <3C5F9F26-8310-4E0E-A3FA-8CF2AE3FC95F@gmail.com> Message-ID: Thanks, I'll start tracking the esl version. -G On Fri, Jan 21, 2011 at 9:01 AM, Alain O'Dea wrote: > It's more likely that the commit is present in both repositories, but one does not have it on the master branch. > > > > On 2011-01-21, at 3:49, Ulf Wiger wrote: > >> >> How funny! I checked by simply substituting 'uwiger' for 'esl' >> in the commit URL: >> >> https://github.com/uwiger/gproc/commit/2742634c2ec608c341d425827eef7f6421849ea5 >> >> ?and there it was. >> >> Is this some weird artefact of how github interprets its URIs? >> >> Anyway, I consider github.com/esl/gproc to be the official one, >> even though I will strive to keep the uwiger version in synch. >> >> BR, >> Ulf >> >> On 20 Jan 2011, at 18:42, Garret Smith wrote: >> >>> I was using the version at https://github.com/uwiger/gproc.git which >>> looks identical except for that patch :) >>> >>> -G >>> >>> On Thu, Jan 20, 2011 at 9:02 AM, Ulf Wiger >>> wrote: >>>> >>>> Hi Garrett, >>>> >>>> Which version of gproc are you using? >>>> >>>> That particular bug was fixed on Jan 11 in http://github.com/esl/gproc >>>> (That is, quite recently. :) >>>> >>>> Sending pull requests to that project will work just fine. >>>> >>>> BR, >>>> Ulf W >>>> >>>> On 19 Jan 2011, at 01:21, Garret Smith wrote: >>>> >>>>> Fixed a small issue with gproc in the 'await' function when no timeout >>>>> is specified. >>>>> >>>>> see branch 'fix-await-notimeout' in the repository >>>>> git://github.com/garret-smith/gproc.git >>>>> >>>>> or the patch: >>>>> >>>>> --- a/src/gproc.erl >>>>> +++ b/src/gproc.erl >>>>> @@ -312,7 +312,7 @@ request_wait({n,C,_} = Key, Timeout) when C==l; C==g -> >>>>> ? ?WRef = call({await,Key,self()}, C), >>>>> ? ?receive >>>>> ? ? ? ?{gproc, WRef, registered, {_K, Pid, V}} -> >>>>> - ? ? ? ? ? case Timeout of >>>>> + ? ? ? ? ? case TRef of >>>>> ? ? ? ? ? ? ? no_timer -> ignore; >>>>> ? ? ? ? ? ? ? _ -> erlang:cancel_timer(TRef) >>>>> ? ? ? ? ? end, >>>>> >>>>> >>>>> I don't know of any guidelines for patch submission since this is not >>>>> OTP, so let me know if there is a better way I can provide patches now >>>>> or in the future. >>>>> >>>>> -G >>>>> >>>>> ________________________________________________________________ >>>>> erlang-questions (at) erlang.org mailing list. >>>>> See http://www.erlang.org/faq.html >>>>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>>>> >>>> >>>> Ulf Wiger, CTO, Erlang Solutions, Ltd. >>>> http://erlang-solutions.com >>>> >>>> >>>> >>>> >> >> Ulf Wiger, CTO, Erlang Solutions, Ltd. >> http://erlang-solutions.com >> >> >> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > From dgud@REDACTED Fri Jan 21 18:52:56 2011 From: dgud@REDACTED (Dan Gudmundsson) Date: Fri, 21 Jan 2011 18:52:56 +0100 Subject: [erlang-questions] Re: Shell is terminated (and then restarts) mysteriously In-Reply-To: References: Message-ID: erlang can hold 2 copies of a module, when you can compile (and load) a module the oldest version will be deleted, processes having references to that module will be killed. So what probably happens is that the shell process have a reference to the code, it will be killed when the module unloads. /Dan On Fri, Jan 21, 2011 at 5:06 PM, Chris Hicks wrote: > > I've noticed this happening on Windows 7 as well, and always when I've compiled a module multiple times within a relatively short span of time. > >> Date: Fri, 21 Jan 2011 08:09:32 +0000 >> From: gordon@REDACTED >> To: erlang-questions@REDACTED >> Subject: Re: [erlang-questions] Re: Shell is terminated (and then restarts) mysteriously >> >> My erlang shell has started doing this (very) recently (R14B01 and Ubunutu >> 10.10) and I found it a bit odd too... >> >> On 21 January 2011 04:30, Taylor Venable wrote: >> >> > Sorry, I forgot to say that this is on Linux, specifically Ubuntu >> > 10.04; here's my `uname -a`: Linux zareason 2.6.32-27-generic >> > #49-Ubuntu SMP Thu Dec 2 00:51:09 UTC 2010 x86_64 GNU/Linux >> > >> > -- >> > Taylor C. Venable >> > http://metasyntax.net/ >> > >> > ________________________________________________________________ >> > erlang-questions (at) erlang.org mailing list. >> > See http://www.erlang.org/faq.html >> > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > >> > >> >> >> -- >> Gordon Guthrie >> CEO hypernumbers >> >> http://hypernumbers.com >> t: hypernumbers >> +44 7776 251669 > From garret.smith@REDACTED Fri Jan 21 19:23:21 2011 From: garret.smith@REDACTED (Garret Smith) Date: Fri, 21 Jan 2011 10:23:21 -0800 Subject: gen_leader receiving 2 handle_DOWN messages when the leader dies in a 2-process setup Message-ID: I'm using the latest gen_leader from https://github.com/abecciu/gen_leader_revival.git When the number of candidates is down to 2, 1 candidate and 1 leader, and the leader dies, the last candidate receives 2 handle_DOWN messages with the same Node parameter. I have tried this starting with 3 candidates or 2 candidates. In either case, killing the leader when there are 2 candidates alive will generate 2 handle_DOWN callbacks on the survivor with the same Node parameter. The callback order on the last node alive is: elected(state, Elected, Node = undefined) handle_DOWN() handle_DOWN() Is this expected behavior? If so, what is the reason for 2 handle_DOWN callbacks? If not, I'll be happy to provide a lot more details on what I'm seeing. Thanks, Garret Smith From hd2010@REDACTED Fri Jan 21 19:45:58 2011 From: hd2010@REDACTED (=?utf-8?B?aGQyMDEwQGVvbmJsYXN0LmNvbQ==?=) Date: Fri, 21 Jan 2011 19:45:58 +0100 Subject: =?utf-8?B?UmU6IFtlcmxhbmctcXVlc3Rpb25zXSBteXNxbCBkcml2ZXIgaW4gZXJsYW5nIHN1cHBvcnQgc3RvcmVkIHByb2NlZHVyZQ==?= Message-ID: Hi Seven, I'll be looking into it. The Eonblast fork is what I am working at today. If you have more info on what the unexpected behavior of SP in Emysql is, please send it my way. Thanks, Henning From matthias@REDACTED Fri Jan 21 20:50:34 2011 From: matthias@REDACTED (Matthias Lang) Date: Fri, 21 Jan 2011 20:50:34 +0100 Subject: [erlang-questions] HiPE performance gain (Was: Re: [erlang-questions] Erlang static linked interpreter) In-Reply-To: <352094.70491.qm@web111413.mail.gq1.yahoo.com> References: <871318.87934.qm@web111404.mail.gq1.yahoo.com> <352094.70491.qm@web111413.mail.gq1.yahoo.com> Message-ID: <20110121195034.GA2248@corelatus.se> On Thursday, January 20, Thomas Lindgren wrote: > there are any palpable problems on a local setup, of course. I think > Mattias Lang has noted that the emulator has grown quite a bit over > the years, is that a problem for instance? Version CPU Stripped Emulator size ----------------------------------------------------------------- R7B01 PPC (MPC860) 688364 R8B PPC 793372 R9C PPC 931804 R10B MIPS (Au1000) 1275960 R11B MIPS 1299212 R13B04 MIPS 1691188 R14B01 MIPS 1865192 Over the years I've used different compiler versions and different options. For a while, I remember that hacking the Makefile for the emulator to use -0s instead of -02 gave a decent performance _increase_. Usually, the application runs a bit _slower_ on each new version of Erlang. Sometimes 5% slower. Sometimes not slower at all. I don't recall ever seeing a speedup. We do our performance-critical work in FPGAs or DSPs, so this doesn't bother us, but we notice. I've assumed that what happens is that each new Erlang release is tuned for increasingly faster CPUs, while our hardware doesn't change as often. But I'm just guessing. The Au1000 has 16kByte I-cache and another 16kByte D-cache, i.e. tiny compared to what's in even modest PCs. Matt From tristan.sloughter@REDACTED Fri Jan 21 21:22:24 2011 From: tristan.sloughter@REDACTED (Tristan Sloughter) Date: Fri, 21 Jan 2011 14:22:24 -0600 Subject: Backreferences in PCRE Message-ID: The re modules documentation(http://www.erlang.org/doc/man/re.html) references backreferences like it supports them. However my attempts to use them failed (specifically the pattern '(.)\1\1' which works fine for me in other languages regex libraries) and I found old emails ( http://www.trapexit.org/forum/viewtopic.php?p=45770&sid=8217e3c278c4ad9178964f5c6fb9085c) to this list that say they are not supported and are purposely turned off in PCRE. So I have a few questions: Is this the case? Is there no chance of ever adding them? What is a good way to implement something like /(.)\1\1/ to match 3 repeated characters if not with backreferences? Thanks, Tristan From watson.timothy@REDACTED Fri Jan 21 22:05:06 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 21 Jan 2011 21:05:06 +0000 Subject: [erlang-questions] Erlang project dependency management/build tools In-Reply-To: References: Message-ID: > ?Snan has always built OTP Application based projects and produced > artifacts in the stardard OTP way, since the first working code some > time ago. Taking advantage of all that OTP metadata was why I wrote > it. ?I think the confusion may come in because, Sinan makes a clear > distinction between your area and its area. That is between the Yes, that was a confusing thing for me - do I have to configure the "project area" at all, or will Sinan just figure out how to process what's beneath the root directory? > Just out of curiosity, why did you think that it didn't build in a > standards compliant way? I suspect there I am missing some > documentation to clear that up. It was the issue above - the distinction between the project area and application area. From watson.timothy@REDACTED Fri Jan 21 22:10:13 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 21 Jan 2011 21:10:13 +0000 Subject: [erlang-questions] Erlang project dependency management/build tools In-Reply-To: References: Message-ID: > Eric, > >> Dependencies have to be declared in the *.app files of your OTP >> Applications according to the OTP standard. Sinan uses this >> information to build a list of dependencies ?for your project, and >> resolve that to a the actual versioned dependencies.... And this is something I don't understand. If I have a dependency on a library (say its called libutils) and I put a dependency in the {applications,[]} tuple of my project's app file - what happens!? The application cannot launch because the OTP application module does not understand that this is a library dependency (and not an application) and therefore tries to start the application, which fails because it's not an application. If you used a custom tuple in the project's .app file (say {libs, [...]} for example) then this would work fine and the OTP application module would just ignore it. From jesper.louis.andersen@REDACTED Fri Jan 21 22:11:38 2011 From: jesper.louis.andersen@REDACTED (Jesper Louis Andersen) Date: Fri, 21 Jan 2011 22:11:38 +0100 Subject: [erlang-questions] HiPE performance gain (Was: Re: [erlang-questions] Erlang static linked interpreter) In-Reply-To: <20110121195034.GA2248@corelatus.se> References: <871318.87934.qm@web111404.mail.gq1.yahoo.com> <352094.70491.qm@web111413.mail.gq1.yahoo.com> <20110121195034.GA2248@corelatus.se> Message-ID: On Fri, Jan 21, 2011 at 20:50, Matthias Lang wrote: > > I've assumed that what happens is that each new Erlang release is > tuned for increasingly faster CPUs, while our hardware doesn't change > as often. But I'm just guessing. > In my experience, the only way to combat performance regressions like these is to plot the development branch against the time to complete the benchmark suite. That way, the relative price of any change can be quickly valuated. A page with sparklines usually suffices. I did one for my Haskell/BitTorrent experiment: http://jlouis.github.com/combinatorrent/ I've been thinking of using the performance counters in modern CPUs to measure the amount of cache misses and plot these as well, but I never came around to do it. I imagined using the linux 'perf' tool which lives in the linux kernel tree under /tools/perf. -- J. From rzezeski@REDACTED Fri Jan 21 22:25:24 2011 From: rzezeski@REDACTED (Ryan Zezeski) Date: Fri, 21 Jan 2011 16:25:24 -0500 Subject: [erlang-questions] linked-in driver blocks net_kernel tick? In-Reply-To: References: Message-ID: Dan, Thanks for the reply, I'll be sure to chunk my data. I was using the gzip/1 call for convenience. That said, I'm still a little fuzzy on something you said. Why is it that the "distribution" process is scheduled on the same scheduler that's running the call to the driver? Why not schedule it on one of the 15 other schedulers that are currently sleeping? Does this mean any other message I send will also be blocked? Dare I ask, how does the scheduling work exactly? -Ryan On Fri, Jan 21, 2011 at 5:16 AM, Dan Gudmundsson wrote: > All c-calls blocks a schedulers, if they are not pushed out to a thread. > > In this case it's a bug in the zlib module (probably by me) gzip should > chunk up the input before invoking the driver. > > What happens is that all schedulers go to sleep because there is no work to > do, > except the one invoking the driver, a ping is received and wakes up > the "distribution" process > which gets queued up on only scheduler that is awake, but that > scheduler is blocked > in an "eternal" call. The pings never become processed and the > distributions times out. > > You can wait for a patch or use zlib api to chunk up compression your self, > see > implementation of gzip in zlib module. > > /Dan > > On Fri, Jan 21, 2011 at 2:48 AM, Ryan Zezeski wrote: > > So...can anyone explain to me why zlib:gzip/1 is causing the net_kernel > tick > > to be blocked? Do linked-in drivers block it's scheduler like NIFs? I'm > > really curious on this one :) > > > > -Ryan > > > > On Tue, Jan 18, 2011 at 6:53 PM, Ryan Zezeski > wrote: > > > >> Apologies, the example I copied was run on my mac. > >> > >> This is what I have on the actual production machine: > >> > >> Erlang R14A (erts-5.8) [source] [64-bit] [smp:16:16] [rq:16] > >> [async-threads:0] [hipe] [kernel-poll:false] > >> > >> To be certain, I ran the same example (except this time using two > physical > >> machines) and achieved the same result. Namely, the 'bar' node claims > 'foo' > >> is not responding and thus closes the connection. Whatever this is, > I've > >> now easily reproduced it on two different OSs, with 2 different Erlang > >> versions. > >> > >> -Ryan > >> > >> On Tue, Jan 18, 2011 at 6:04 PM, Alain O'Dea > wrote: > >> > >>> On 2011-01-18, at 18:54, Ryan Zezeski wrote: > >>> > >>> > Hi everyone, > >>> > > >>> > Some of you may remember my latest question where I was having weird > >>> node > >>> > timeout issues that I couldn't explain and I thought it might be > related > >>> to > >>> > the messages I was passing between my nodes. Well, I pinpointed the > >>> problem > >>> > to a call to zlib:gzip/1. At first I was really surprised by this, > as > >>> such > >>> > a harmless line of code surely should have nothing to do with the > >>> ability > >>> > for my nodes to communicate. However, as I dug further I realized > gzip > >>> was > >>> > implemented as a linked-in driver and I remember reading things about > >>> how > >>> > one has to take care with them because they can trash the VM with > them. > >>> I > >>> > don't remember reading anything about them blocking code, and even if > >>> they > >>> > do I fail to see why my SMP enabled node (16 cores) would allow this > one > >>> > thread to block the tick. It occurred to me that maybe the scheduler > >>> > responsible for that process is the one blocked by the driver. Do > >>> processes > >>> > have scheduler affinity? That would make sense, I guess. > >>> > > >>> > I've "fixed" this problem simply by using a plain port (i.e. run in > it's > >>> own > >>> > OS process). For my purposes, this actually makes more sense in the > >>> > majority of the places I was making use of gzip. Can someone > enlighten > >>> me > >>> > as to exactly what is happening behind the scenes? > >>> > > >>> > To reproduce I create a random 1.3GB file: > >>> > > >>> > dd if=/dev/urandom of=rand bs=1048576 count=1365 > >>> > > >>> > Then start two named nodes 'foo' and 'bar', connect them, read in the > >>> file, > >>> > and then compress said file. Sometime later (I think around 60+ > >>> seconds) > >>> > the node 'bar' will claim that 'foo' is not responding. > >>> > > >>> > [progski@REDACTED ~/tmp_code/node_timeout] erl -name foo > >>> > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] > >>> > >>> Your SMP node seems to be capped at smp:2:2 when it out to be smp:16. > >>> Some resource limit may be holding back the system. That said zlib > should > >>> not ever cause this issue. > >>> > >>> > [async-threads:0] [hipe] [kernel-poll:false] > >>> > > >>> > Eshell V5.8.1 (abort with ^G) > >>> > (foo@REDACTED)1> net_adm:ping('bar@REDACTED'). > >>> > pong > >>> > (foo@REDACTED)2> nodes(). > >>> > ['bar@REDACTED'] > >>> > (foo@REDACTED)3> {ok,Data} = file:read_file("rand"). > >>> > {ok,<<103,5,115,210,177,147,53,45,250,182,51,32,250,233, > >>> > 39,253,102,61,73,242,18,159,45,185,232,80,33,...>>} > >>> > (foo@REDACTED)4> zlib:gzip(Data). > >>> > <<31,139,8,0,0,0,0,0,0,3,0,15,64,240,191,103,5,115,210, > >>> > 177,147,53,45,250,182,51,32,250,233,...>> > >>> > (foo@REDACTED)5> > >>> > > >>> > > >>> > [progski@REDACTED ~/tmp_code/node_timeout] erl -name bar > >>> > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] > >>> > [async-threads:0] [hipe] [kernel-poll:false] > >>> > > >>> > Eshell V5.8.1 (abort with ^G) > >>> > (bar@REDACTED)1> nodes(). > >>> > ['foo@REDACTED'] > >>> > (bar@REDACTED)2> > >>> > =ERROR REPORT==== 18-Jan-2011::17:16:10 === > >>> > ** Node 'foo@REDACTED' not responding ** > >>> > ** Removing (timedout) connection ** > >>> > > >>> > > >>> > Thanks, > >>> > > >>> > -Ryan > >>> > >> > >> > > > From alain.odea@REDACTED Fri Jan 21 22:56:11 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Fri, 21 Jan 2011 18:26:11 -0330 Subject: [erlang-questions] mysql driver in erlang support stored procedure In-Reply-To: References: Message-ID: <912DC819-77A5-49C5-AEFB-58719CC81F9C@gmail.com> On 2011-01-21, at 12:40, Seven Du wrote: > Hi, > > I was using erlang_mysql_driver for a while, but it seems doesn't > support stored procedure. > > Then I found another driver called EMysql > https://github.com/JacobVorreuter/emysql , on test it does support > call procedure, however, it returns ok_packets sometimes on select > only procedures, and it happens also for simple selects(where it > should always be result_packet I think): > > fetch4() -> > SQL = "select * from test;", > Result = emysql:execute(mysql2, SQL), > io:format("~p~n", [Result]), > Fields = Result#result_packet.field_list, > Rows = Result#result_packet.rows, > > io:format("~p~n", [Fields]), > io:format("~p~n", [Rows]). > > >> mytest:fetch4(). > {ok_packet,28,0,0,2,0,[]} > ** exception error: {badrecord,result_packet} > in function m_telsystem:fetch/0 > > > I should report an issue on github but the only issue there is still > in unread state and it's not been updated since 2009. There are other > clones of the project, but not sure which one will has support. > > Any one has experience on this or other options? > > erlang_mysql_driver is quite state besides I haven't got it support > stored procedure. > > Thanks. > > -- > About: http://about.me/dujinfang > Blog: http://www.dujinfang.com > Proj: http://www.freeswitch.org.cn > PostgreSQL is largely equivalent to MySQL and has mature support in epgsql in Zotonic. If it is an option I recommend using PostgreSQL instead. From taylor@REDACTED Fri Jan 21 22:54:12 2011 From: taylor@REDACTED (Taylor Venable) Date: Fri, 21 Jan 2011 16:54:12 -0500 Subject: [erlang-questions] Re: Shell is terminated (and then restarts) mysteriously In-Reply-To: References: Message-ID: On Fri, Jan 21, 2011 at 12:52, Dan Gudmundsson wrote: > So what probably happens is that the shell process have a reference to > the code, it will be killed > when the module unloads. I've seen that cited a couple times in reference to this error message (for the few results I could find through Google). What kinds of things could those "references" into the module be? I'm not leaving any files opened, I don't leave any processes spawned, my code is pretty simple sequential stuff (or at least as far as I understand). It doesn't happen consistently, just occasionally, but often enough to be annoying; it ended up happening 8 times last night over the course of about 3 hours. If it's a problem with my code... well, maybe I'm just not familiar enough with Erlang but I would think that a problem in my code would manifest itself more regularly. Or maybe I'm not really understanding the purpose of the shell. I've been treating it like a REPL where I can compile, reload, and test many times over in quick succession. Is the shell maybe better suited to interacting with existing running code rather than as a testing playground? Thanks for any explanation. -- Taylor C. Venable http://metasyntax.net/ From watson.timothy@REDACTED Fri Jan 21 22:55:26 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 21 Jan 2011 21:55:26 +0000 Subject: [erlang-questions] Erlang project dependency management/build tools In-Reply-To: References: Message-ID: On 21 January 2011 21:39, Eric Merritt wrote: > Tim, > > ?OTP supports something called 'included_applications' that do exactly > what you want, I believe and Sinan fully supports them as > dependencies. The documentation is her > http://www.erlang.org/doc/design_principles/included_applications.html > > ?The included application still needs to follow the directory > layouts, have an app file etc, but its not expected to have an > application behavior and the OTP release system wont try to start it. > > Eric Aha - that sounds ideal. From the documentation it appears that an "including" application isn't obliged to start a supervision tree for an "included" application if doing so is unnecessary, so presumably libraries can be referenced in this way without further issue. Unless of course you've referenced both mochiweb and erlsom yet both of them (in an imagined world) happen to refer to log4erl, which sounds a likely scenario. Despite that prickly issue, this does look like an improvement. Overall, I really like what erlware is trying to do. I think the problem is getting the right level of adoption, so that you've got a situation like maven2 where a large proportion of packages can be pulled in without much ado, and pushing/deploying non-maven-built artefacts into the repository isn't too hard either. If I could opt-out of having the core OTP libraries installed via the erlware repo, then I might be inclined to move from rebar, where the dependency management has issues of its own. Is this still how erlware works? Last I remember, applications/libraries such as kernel and stdlib were coming down from the repo too, and whilst I understand the embedded system doctrine and agree with it in principle, the reality at work is that we install the JVM and other emulators system wide and that's not going to go away in the near future. Perhaps I've misunderstood this aspect of erlware's operation though. From matthias@REDACTED Sat Jan 22 00:15:43 2011 From: matthias@REDACTED (Matthias Lang) Date: Sat, 22 Jan 2011 00:15:43 +0100 Subject: [erlang-questions] HiPE performance gain (Was: Re: [erlang-questions] Erlang static linked interpreter) In-Reply-To: References: <871318.87934.qm@web111404.mail.gq1.yahoo.com> <352094.70491.qm@web111413.mail.gq1.yahoo.com> <20110121195034.GA2248@corelatus.se> Message-ID: <20110121231543.GA2824@corelatus.se> MML wrote: > > I've assumed that what happens is that each new Erlang release is > > tuned for increasingly faster CPUs, while our hardware doesn't change > > as often. But I'm just guessing. JL wrote: > In my experience, the only way to combat performance regressions like > these is to plot the development branch against the time to complete Did you interpret my comment to mean "The OTP guys aren't noticing 3% preformance regressions which hit everyone. They're not noticing because they get new, faster machines every year"? That's not what I meant. What I meant was "In 2004, a typical server CPU was clocked (say) 2x faster than an Au1000 and had (say) 8x as much cache. In 2010, the ratios are more like 5x and 64x. This increased difference might* have made it more likely for a given change to the VM to result in increased performance on a current server CPU while also causing a decrease on an Au1000." I have no reason to believe that performance gets worse from release to release when running on normal (e.g. x86 server) hardware. Matt * i.e. wild speculation, discussion would be interesting if someone has concrete examples or, better still, actual measurements. From masklinn@REDACTED Sat Jan 22 00:19:40 2011 From: masklinn@REDACTED (Masklinn) Date: Sat, 22 Jan 2011 00:19:40 +0100 Subject: [erlang-questions] HiPE performance gain (Was: Re: [erlang-questions] Erlang static linked interpreter) In-Reply-To: <20110121195034.GA2248@corelatus.se> References: <871318.87934.qm@web111404.mail.gq1.yahoo.com> <352094.70491.qm@web111413.mail.gq1.yahoo.com> <20110121195034.GA2248@corelatus.se> Message-ID: On 2011-01-21, at 20:50 , Matthias Lang wrote: > For a while, I remember that hacking the Makefile for > the emulator to use -0s instead of -02 gave a decent performance > _increase_. Not that uncommon, I believe (and not limited to Erlang either). Though it does generally require some testing as it's not systematic. From agssvensson@REDACTED Sat Jan 22 00:30:44 2011 From: agssvensson@REDACTED (Anders Svensson) Date: Sat, 22 Jan 2011 00:30:44 +0100 Subject: [erlang-questions] Backreferences in PCRE In-Reply-To: References: Message-ID: On Fri, Jan 21, 2011 at 9:22 PM, Tristan Sloughter wrote: > The re modules documentation(http://www.erlang.org/doc/man/re.html) > references backreferences like it supports them. However my attempts to use > them failed (specifically the pattern '(.)\1\1' which works fine for me in 1> re:run("goood", "(.)\\1\\1", [{capture, all, list}]). {match,["ooo","o"]} /Anders > other languages regex libraries) and I found old emails ( > http://www.trapexit.org/forum/viewtopic.php?p=45770&sid=8217e3c278c4ad9178964f5c6fb9085c) > to this list that say they are not supported and are purposely turned off in > PCRE. So I have a few questions: > > Is this the case? Is there no chance of ever adding them? What is a good way > to implement something like /(.)\1\1/ to match 3 repeated characters if not > with backreferences? > > Thanks, > Tristan > From watson.timothy@REDACTED Sat Jan 22 01:15:37 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Sat, 22 Jan 2011 00:15:37 +0000 Subject: [erlang-questions] Erlang project dependency management/build tools In-Reply-To: References: Message-ID: On 21 January 2011 22:46, Eric Merritt wrote: > Tim, > > ?Now whether or not faxien installs versions of the standard libraries > is all up to you, where you install faxien and whats already there. If > you install faxien on a clean system that doesn't have erlang it will > pull down everything it needs. However, if you install it on a system > that already has erlang installed, it wont pull down anything that is > already there. faxien and sinan don't actually care where the > libraries come from, as long as they are on the system. ?If the > libraries/versions that something requires are not there though, it > will pull them down. Thats a bit different then it was at the > beginning when faxien kind of managed its own little world, that is no > longer the case however. > That's excellent news, I shall have a play once more then. I have to admit that most of my open source projects are using rebar and it's doing a fine job, but binary artefact repositories are the "norm" at work and there's been a lot of suggestions about packaging Erlang/OTP releases up and deploying them to our maven repositories. The reason for this is that our automated deployment tooling (which is based on capistrano) can easily grab hold of stuff that is in a know location and pulling from source (especially external source repositories such as github) would be frowned upon (despite the fact that for most open source dependencies such as mochiweb, that's exactly where we've taken the code from to begin with). Anyway thanks for the update - I'll starting playing again. Tim From pablo.platt@REDACTED Sat Jan 22 03:49:07 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Fri, 21 Jan 2011 18:49:07 -0800 (PST) Subject: error in the example of binary:part/2 in the docs Message-ID: <295585.68111.qm@web112604.mail.gq1.yahoo.com> Couldn't find the actual place in github. Maybe it is possible to validate the examples in the docs as valid erlang terms? http://erldocs.com/R14B/stdlib/binary.html?i=38&search=bin#part/2 1> Bin = <<1,2,3,4,5,6,7,8,9,10>>. 2> binary:part(Bin,{byte_size(Bin), -5)). <<6,7,8,9,10>> in 2) should be } instead of ) binary:part(Bin,{byte_size(Bin), -5}). Thanks From taylor@REDACTED Sat Jan 22 04:12:38 2011 From: taylor@REDACTED (Taylor Venable) Date: Fri, 21 Jan 2011 22:12:38 -0500 Subject: Shell is terminated (and then restarts) mysteriously In-Reply-To: References: Message-ID: On Thu, Jan 20, 2011 at 23:02, Taylor Venable wrote: > Hi there. I'm trying to learn Erlang by working on some simple > programs, and as I'm working with the shell I'm noticing some (what to > me at least is) odd behaviour. Every once in a while, after compiling > some code, it will state that the shell process was terminated. OK, after much playing I finally have a working minimal example of how to reproduce this every single time, at least for my installation. But first, with R14B I hacked lib/kernel/src/user_drv.erl on line 202 to print out R, which I assume is the reason why the EXIT message was received. The value of R was the atom 'killed'. Don't know if that has any significance. Then I upgraded to an unmodified R14B01, but there was no difference in behaviour. After a few hours of coding, and about 10 shell restarts, I happened to notice a pattern. From that I worked my way to this example. Start with this in test.erl: -module(test). -export([test/0]). test() -> F = fun (_, []) -> ok; (F, [_|Xs]) -> F(F, Xs, xxx) end, F(F, [foo]). Note: the recursive invocation of F is an intentional bug. Fire up erl: $ erl Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.2 (abort with ^G) 1> c(test). {ok,test} 2> test:test(). ** exception error: test:'-test/0-fun-0-'/2 called with 3 arguments in function test:'-test/0-fun-0-'/2 OK, good, that's what I expect. Now change "F(F, Xs, xxx)" to "F(F, Xs)" and then compile and run again: 3> c(test). {ok,test} 4> test:test(). ok Again, good. Now put the buggy recursive invocation of F back (i.e. undo the fix we just made so we have the original file again). Compile and run again: 5> c(test). *** ERROR: Shell process terminated! *** Eshell V5.8.2 (abort with ^G) 1> There is the shell termination. Using this method, I get the termination every single time. And, if I use either result(1), or history(1), at the beginning, I do not get the termination. This sounds to me similar to what Dan Gudmundsson said about the shell (probably through the history or result list) keeping some kind of reference into the module's code. However, constructing a similar dynamic function call does NOT cause the problem: -module(test). -export([test/0]). test() -> F = fun (F) -> F(xxx) end, G = fun () -> ok end, F(G). 1> c(test). {ok,test} 2> test:test(). ** exception error: test:'-test/0-fun-1-'/0 called with one argument in function test:'-test/0-fun-0-'/1 This is what we expect, now remove the xxx and recompile. 3> c(test). {ok,test} 4> test:test(). ok OK, let's put the xxx back and try it again. 5> c(test). {ok,test} 6> test:test(). ** exception error: test:'-test/0-fun-1-'/0 called with one argument in function test:'-test/0-fun-0-'/1 To my admittedly ignorant eyes, it seems like exactly the same kind of error that I've created, but unlike before, this example does NOT break the shell. It seems to me that the only difference of much consequence is whether the buggy call is recursive or not. The recursive call breaks the shell, the non-recursive call does not. So I guess now the question for me is, is this an expected result? And if so, why? Thanks for reading. -- Taylor C. Venable http://metasyntax.net/ From robert.virding@REDACTED Sat Jan 22 05:55:17 2011 From: robert.virding@REDACTED (Robert Virding) Date: Sat, 22 Jan 2011 04:55:17 +0000 (GMT) Subject: [erlang-questions] Re: Shell is terminated (and then restarts) mysteriously In-Reply-To: <1865193994.325381295671819909.JavaMail.root@zimbra> Message-ID: <691920276.325401295672117209.JavaMail.root@zimbra> It can be simplified even more. We use this test.erl: -module(test). -export([test/0]). test() -> fun (_, []) -> ok; (F, [_|Xs]) -> F(F, Xs) %% F(F, Xs, xxx) end. And use it like: rv ~/erlang$ erl Erlang R14B (erts-5.8.1) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.1 (abort with ^G) 1> c(test). {ok,test} 2> test:test(). #Fun> Here we move the comment to use the incorrect recursive call and continue: 3> c(test). {ok,test} 4> c(test). *** ERROR: Shell process terminated! *** Eshell V5.8.1 (abort with ^G) 1> The cause of the behaviour is that the first call to test:test/0 returns a fun which is saved in the history list. Changing that fun and recompiling/reloading the file loads in a new version. The old fun now points to the old version of the module. Recompiling/reloading the file again means that the fun on the history list would now point to a non-existent module so the shell process is killed by the code server. In the case of an error the history list contains the actual exit reason. You can see this with h(). In your first case the exit reason contained the fun so the shell process was killed. Funs don't fit really well into code handling so it can sometimes be hard to detect to which version of the code it points, especially if the fun itself hasn't been modified or the module has been significantly modified. In your second case the fun in the exit reason wasn't the one you changed. Robert ----- "Taylor Venable" wrote: > On Thu, Jan 20, 2011 at 23:02, Taylor Venable > wrote: > > Hi there. I'm trying to learn Erlang by working on some simple > > programs, and as I'm working with the shell I'm noticing some (what > to > > me at least is) odd behaviour. Every once in a while, after > compiling > > some code, it will state that the shell process was terminated. > > OK, after much playing I finally have a working minimal example of > how > to reproduce this every single time, at least for my installation. > > But first, with R14B I hacked lib/kernel/src/user_drv.erl on line 202 > to print out R, which I assume is the reason why the EXIT message was > received. The value of R was the atom 'killed'. Don't know if that > has > any significance. Then I upgraded to an unmodified R14B01, but there > was no difference in behaviour. > > After a few hours of coding, and about 10 shell restarts, I happened > to notice a pattern. From that I worked my way to this example. Start > with this in test.erl: > > -module(test). > -export([test/0]). > > test() -> > F = fun (_, []) -> ok; > (F, [_|Xs]) -> > F(F, Xs, xxx) > end, > F(F, [foo]). > > Note: the recursive invocation of F is an intentional bug. Fire up > erl: > > $ erl > Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:2:2] [rq:2] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.2 (abort with ^G) > 1> c(test). > {ok,test} > 2> test:test(). > ** exception error: test:'-test/0-fun-0-'/2 called with 3 arguments > in function test:'-test/0-fun-0-'/2 > > OK, good, that's what I expect. Now change "F(F, Xs, xxx)" to "F(F, > Xs)" and then compile and run again: > > 3> c(test). > {ok,test} > 4> test:test(). > ok > > Again, good. Now put the buggy recursive invocation of F back (i.e. > undo the fix we just made so we have the original file again). > Compile > and run again: > > 5> c(test). > *** ERROR: Shell process terminated! *** > Eshell V5.8.2 (abort with ^G) > 1> > > There is the shell termination. Using this method, I get the > termination every single time. > > And, if I use either result(1), or history(1), at the beginning, I do > not get the termination. This sounds to me similar to what Dan > Gudmundsson said about the shell (probably through the history or > result list) keeping some kind of reference into the module's code. > > However, constructing a similar dynamic function call does NOT cause > the problem: > > -module(test). > -export([test/0]). > > test() -> > F = fun (F) -> F(xxx) end, > G = fun () -> ok end, > F(G). > > 1> c(test). > {ok,test} > 2> test:test(). > ** exception error: test:'-test/0-fun-1-'/0 called with one argument > in function test:'-test/0-fun-0-'/1 > > This is what we expect, now remove the xxx and recompile. > > 3> c(test). > {ok,test} > 4> test:test(). > ok > > OK, let's put the xxx back and try it again. > > 5> c(test). > {ok,test} > 6> test:test(). > ** exception error: test:'-test/0-fun-1-'/0 called with one argument > in function test:'-test/0-fun-0-'/1 > > To my admittedly ignorant eyes, it seems like exactly the same kind > of > error that I've created, but unlike before, this example does NOT > break the shell. It seems to me that the only difference of much > consequence is whether the buggy call is recursive or not. The > recursive call breaks the shell, the non-recursive call does not. So > I > guess now the question for me is, is this an expected result? And if > so, why? > > Thanks for reading. > > -- > Taylor C. Venable > http://metasyntax.net/ > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED From mikma264@REDACTED Sat Jan 22 11:05:37 2011 From: mikma264@REDACTED (Mikael Magnusson) Date: Sat, 22 Jan 2011 11:05:37 +0100 Subject: [erlang-questions] PPA for ubuntu? In-Reply-To: <1292048640.1892.8.camel@shanghai> References: <1291963922.13124.1.camel@shanghai> <1291979964.29857.0.camel@shanghai> <1292048640.1892.8.camel@shanghai> Message-ID: <4D3AABF1.7050505@gmail.com> On 12/11/2010 07:24 AM, Xiaopong Tran wrote: > It's true, that's why it would great if there is an official > build for different Linux distros, like the binary for Windows. > I guess erlang.org could build binaries for Debian/Ubuntu which install into /opt and don't put any libraries or executables in the usual places to not conflict with the packages installed by Debian and Ubuntu. But of course that would mean you probably end up with multiple copies of Erlang, at least if you run Ubuntu desktop which install Erlang by default. /Mikael > People can certainly build from source, I do it too when I > want to try out new features but the new version has not been > packaged into the distro yet. But it's always good to have > pre-built packages. > > Xiaopong From taylor@REDACTED Sat Jan 22 13:47:12 2011 From: taylor@REDACTED (Taylor Venable) Date: Sat, 22 Jan 2011 07:47:12 -0500 Subject: [erlang-questions] Re: Shell is terminated (and then restarts) mysteriously In-Reply-To: <691920276.325401295672117209.JavaMail.root@zimbra> References: <1865193994.325381295671819909.JavaMail.root@zimbra> <691920276.325401295672117209.JavaMail.root@zimbra> Message-ID: On Fri, Jan 21, 2011 at 23:55, Robert Virding wrote: > In the case of an error the history list contains the actual exit reason. You can see this with h(). In your first case the exit reason contained the fun so the shell process was killed. Oh, I see now! That makes perfect sense; thank you very much for the excellent explanation. -- Taylor C. Venable http://metasyntax.net/ From pozortucnak@REDACTED Sat Jan 22 15:08:59 2011 From: pozortucnak@REDACTED (pozortucnak) Date: Sat, 22 Jan 2011 15:08:59 +0100 Subject: interactive console program in erlang? Message-ID: Hi Can I run some interactive console program in erlang? I try os:run(nano) but that do not word. Thank From alain.odea@REDACTED Sat Jan 22 17:20:32 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Sat, 22 Jan 2011 12:50:32 -0330 Subject: [erlang-questions] interactive console program in erlang? In-Reply-To: References: Message-ID: <5E40B377-EAB9-4557-AAA2-8DE836816873@gmail.com> On 2011-01-22, at 10:38, pozortucnak wrote: > Hi > Can I run some interactive console program in erlang? I try os:run(nano) but > that do not word. > > Thank This is doable with open_port. It is not trivial though. Instead of STDIN and STDOUT you communicate with nano using messages. It would be pretty awkward. From dgud@REDACTED Sat Jan 22 17:46:23 2011 From: dgud@REDACTED (Dan Gudmundsson) Date: Sat, 22 Jan 2011 17:46:23 +0100 Subject: [erlang-questions] linked-in driver blocks net_kernel tick? In-Reply-To: References: Message-ID: Rickard who have implemented this should explain it. If I have understood it correctly, it works like this: If a scheduler do not have any work to do it will be disabled. It will be disabled until a live thread discovers it have to much work and wakes a sleeping scheduler. The run-queues are only checked when processes are scheduled. Since in this case the only living scheduler is busy for a very long time, no queue checking will be done and the all schedulers will be blocked until the call to the driver is complete. We had a long discussion during lunch about it, and we didn't agree how it should work. :-) I agree that zlib is broken and it should be fixed but I still believe that it breaks the rule about least astonishment, if I have 16 schedulers and one is blocked in a long function call I still expect other code to be invoked. Rickards thought is that such call should never happen and should be called through an async driver or a separate thread. I guess it will take a couple of more lunches to come to a conclusion :-) /Dan On Fri, Jan 21, 2011 at 10:25 PM, Ryan Zezeski wrote: > Dan, > > Thanks for the reply, I'll be sure to chunk my data. ?I was using the gzip/1 > call for convenience. > > That said, I'm still a little fuzzy on something you said. ?Why is it that > the "distribution" process is scheduled on the same scheduler that's running > the call to the driver? ?Why not schedule it on one of the 15 other > schedulers that are currently sleeping? ?Does this mean any other message I > send will also be blocked? ?Dare I ask, how does the scheduling work > exactly? > > -Ryan > > On Fri, Jan 21, 2011 at 5:16 AM, Dan Gudmundsson wrote: > >> All c-calls blocks a schedulers, if they are not pushed out to a thread. >> >> In this case it's a bug in the zlib module (probably by me) gzip should >> chunk up the input before invoking the driver. >> >> What happens is that all schedulers go to sleep because there is no work to >> do, >> except the one invoking the driver, a ping is received and wakes up >> the "distribution" process >> which gets queued up on only scheduler that is awake, but that >> scheduler is blocked >> in an "eternal" call. The pings never become processed and the >> distributions times out. >> >> You can wait for a patch or use zlib api to chunk up compression your self, >> see >> implementation of gzip in zlib module. >> >> /Dan >> >> On Fri, Jan 21, 2011 at 2:48 AM, Ryan Zezeski wrote: >> > So...can anyone explain to me why zlib:gzip/1 is causing the net_kernel >> tick >> > to be blocked? ?Do linked-in drivers block it's scheduler like NIFs? ?I'm >> > really curious on this one :) >> > >> > -Ryan >> > >> > On Tue, Jan 18, 2011 at 6:53 PM, Ryan Zezeski >> wrote: >> > >> >> Apologies, the example I copied was run on my mac. >> >> >> >> This is what I have on the actual production machine: >> >> >> >> Erlang R14A (erts-5.8) [source] [64-bit] [smp:16:16] [rq:16] >> >> [async-threads:0] [hipe] [kernel-poll:false] >> >> >> >> To be certain, I ran the same example (except this time using two >> physical >> >> machines) and achieved the same result. ?Namely, the 'bar' node claims >> 'foo' >> >> is not responding and thus closes the connection. ?Whatever this is, >> I've >> >> now easily reproduced it on two different OSs, with 2 different Erlang >> >> versions. >> >> >> >> -Ryan >> >> >> >> On Tue, Jan 18, 2011 at 6:04 PM, Alain O'Dea >> wrote: >> >> >> >>> On 2011-01-18, at 18:54, Ryan Zezeski wrote: >> >>> >> >>> > Hi everyone, >> >>> > >> >>> > Some of you may remember my latest question where I was having weird >> >>> node >> >>> > timeout issues that I couldn't explain and I thought it might be >> related >> >>> to >> >>> > the messages I was passing between my nodes. ?Well, I pinpointed the >> >>> problem >> >>> > to a call to zlib:gzip/1. ?At first I was really surprised by this, >> as >> >>> such >> >>> > a harmless line of code surely should have nothing to do with the >> >>> ability >> >>> > for my nodes to communicate. ?However, as I dug further I realized >> gzip >> >>> was >> >>> > implemented as a linked-in driver and I remember reading things about >> >>> how >> >>> > one has to take care with them because they can trash the VM with >> them. >> >>> ?I >> >>> > don't remember reading anything about them blocking code, and even if >> >>> they >> >>> > do I fail to see why my SMP enabled node (16 cores) would allow this >> one >> >>> > thread to block the tick. ?It occurred to me that maybe the scheduler >> >>> > responsible for that process is the one blocked by the driver. ?Do >> >>> processes >> >>> > have scheduler affinity? ?That would make sense, I guess. >> >>> > >> >>> > I've "fixed" this problem simply by using a plain port (i.e. run in >> it's >> >>> own >> >>> > OS process). ?For my purposes, this actually makes more sense in the >> >>> > majority of the places I was making use of gzip. ?Can someone >> enlighten >> >>> me >> >>> > as to exactly what is happening behind the scenes? >> >>> > >> >>> > To reproduce I create a random 1.3GB file: >> >>> > >> >>> > dd if=/dev/urandom of=rand bs=1048576 count=1365 >> >>> > >> >>> > Then start two named nodes 'foo' and 'bar', connect them, read in the >> >>> file, >> >>> > and then compress said file. ?Sometime later (I think around 60+ >> >>> seconds) >> >>> > the node 'bar' will claim that 'foo' is not responding. >> >>> > >> >>> > [progski@REDACTED ~/tmp_code/node_timeout] erl -name foo >> >>> > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] >> >>> >> >>> Your SMP node seems to be capped at smp:2:2 when it out to be smp:16. >> >>> ?Some resource limit may be holding back the system. That said zlib >> should >> >>> not ever cause this issue. >> >>> >> >>> > [async-threads:0] [hipe] [kernel-poll:false] >> >>> > >> >>> > Eshell V5.8.1 ?(abort with ^G) >> >>> > (foo@REDACTED)1> net_adm:ping('bar@REDACTED'). >> >>> > pong >> >>> > (foo@REDACTED)2> nodes(). >> >>> > ['bar@REDACTED'] >> >>> > (foo@REDACTED)3> {ok,Data} = file:read_file("rand"). >> >>> > {ok,<<103,5,115,210,177,147,53,45,250,182,51,32,250,233, >> >>> > ? ? ?39,253,102,61,73,242,18,159,45,185,232,80,33,...>>} >> >>> > (foo@REDACTED)4> zlib:gzip(Data). >> >>> > <<31,139,8,0,0,0,0,0,0,3,0,15,64,240,191,103,5,115,210, >> >>> > ?177,147,53,45,250,182,51,32,250,233,...>> >> >>> > (foo@REDACTED)5> >> >>> > >> >>> > >> >>> > [progski@REDACTED ~/tmp_code/node_timeout] erl -name bar >> >>> > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] >> >>> > [async-threads:0] [hipe] [kernel-poll:false] >> >>> > >> >>> > Eshell V5.8.1 ?(abort with ^G) >> >>> > (bar@REDACTED)1> nodes(). >> >>> > ['foo@REDACTED'] >> >>> > (bar@REDACTED)2> >> >>> > =ERROR REPORT==== 18-Jan-2011::17:16:10 === >> >>> > ** Node 'foo@REDACTED' not responding ** >> >>> > ** Removing (timedout) connection ** >> >>> > >> >>> > >> >>> > Thanks, >> >>> > >> >>> > -Ryan >> >>> >> >> >> >> >> > >> > From kuleshovmail@REDACTED Sat Jan 22 18:08:20 2011 From: kuleshovmail@REDACTED (shk) Date: Sat, 22 Jan 2011 09:08:20 -0800 (PST) Subject: Project for erlang beginers Message-ID: <1295716100303-3231595.post@n4.nabble.com> Hello, I'm erlang beginner programmer. The best theory it is practice. Ant that's i start new project for myself and maybe other erlang beginners. I want to write pop3/smtp server in erlang to increase my skills in erlang and i want to invite to take part in development/testing erlang beginner programmers as i to take more experience. The project - https://github.com/onlyshk/epmail epmail . Now i implemet simple connection to the pop3 server and pop3 command parsing. Now i make mail store. Now it'll be in Maildir format, but then i'll plan to add supporting some db for mail storage. I think we together can to make cool and usefull project. For erlang professional i want to say not strongly scold my code :) I learn erlang about month, but i constantly improve my erlang knowledge. I shall be very glad to criticism my code. In general who are interested, please feel free to join at project. Thank you. -- View this message in context: http://erlang.2086793.n4.nabble.com/Project-for-erlang-beginers-tp3231595p3231595.html Sent from the Erlang Questions mailing list archive at Nabble.com. From wallentin.dahlberg@REDACTED Sat Jan 22 19:12:04 2011 From: wallentin.dahlberg@REDACTED (Wallentin Dahlberg) Date: Sat, 22 Jan 2011 19:12:04 +0100 Subject: [erlang-questions] linked-in driver blocks net_kernel tick? In-Reply-To: References: Message-ID: I think his argument was that a driver or nif that does not use an async-thread for potentially blocking calls is a seriously broken driver. Consider the non-smp case. It will halt the beam and hinder important processes to be scheduled. I agree that in the smp case, one scheduler should not block the other schedulers in damaging calls. If a developer wants to destroy a scheduler with a broken driver, he should be free to do so. This was the fear with nifs. With nifs developers has an easy tool to really destroy the system in order to "increase performance" and implement 3rd party libs. There are several cases with different impact, 1) destroy soft-real-time properties - reduction count badness 2) destroy concurrency with blocking calls - scheduler badness 3) destroy the system with faulty drivers (seg fault) - pure badness Some of these issues can be mitigated if the developer implements async threads, i.e. schedules operations to the async-pool. I feel that this is not ideal and is a heritage of ancient times. The problem in this case is that time does not progress in the system. Time is measured in reductions and each call is a reduction. At least this is the case with normal code. There are som special cases too, for instance a message sent "bumbs" the reduction count of the sender. Since native code (nif, bifs and drivers) do not increase reductions during its call but instead penalize the process after the call, time does not progress during the execution (as opposed to beam code). When a process reaches the reduction-limit it is scheduled out. Why reduction counters instead of time slices? Supposedly it much faster (according to Bj?rn). It is a design decision with trade-offs. The solution is fast and nimble, it has certain characteristics that are favorable and has some characteristics that are less favorable. I would favor time-slices since i think it would be fairer to the system and potentially we could save a register. Exactly how it should be done is a question for a different time. The load balancing in the scheduler is checked when a certain reduction count is reached for that scheduler. We do not want to check this too often since it will then become a serialization point. But fear not, there is a (beautiful) solution that is being discussed in the erts-team. Hopefully we can agree on the details. // Bj?rn-Egil 2011/1/22 Dan Gudmundsson > Rickard who have implemented this should explain it. > > If I have understood it correctly, it works like this: > If a scheduler do not have any work to do it will be disabled. > It will be disabled until a live thread discovers it have to much work and > wakes a sleeping scheduler. The run-queues are only checked when processes > are scheduled. > > Since in this case the only living scheduler is busy for a very long time, > no queue checking will be done and the all schedulers will be blocked until > the call to the driver is complete. > > We had a long discussion during lunch about it, and we didn't agree > how it should > work. :-) > > I agree that zlib is broken and it should be fixed but I still believe that > it > breaks the rule about least astonishment, if I have 16 schedulers and > one is blocked > in a long function call I still expect other code to be invoked. > Rickards thought is that > such call should never happen and should be called through an async > driver or a separate > thread. I guess it will take a couple of more lunches to come to a > conclusion :-) > > /Dan > > On Fri, Jan 21, 2011 at 10:25 PM, Ryan Zezeski wrote: > > Dan, > > > > Thanks for the reply, I'll be sure to chunk my data. I was using the > gzip/1 > > call for convenience. > > > > That said, I'm still a little fuzzy on something you said. Why is it > that > > the "distribution" process is scheduled on the same scheduler that's > running > > the call to the driver? Why not schedule it on one of the 15 other > > schedulers that are currently sleeping? Does this mean any other message > I > > send will also be blocked? Dare I ask, how does the scheduling work > > exactly? > > > > -Ryan > > > > On Fri, Jan 21, 2011 at 5:16 AM, Dan Gudmundsson > wrote: > > > >> All c-calls blocks a schedulers, if they are not pushed out to a thread. > >> > >> In this case it's a bug in the zlib module (probably by me) gzip should > >> chunk up the input before invoking the driver. > >> > >> What happens is that all schedulers go to sleep because there is no work > to > >> do, > >> except the one invoking the driver, a ping is received and wakes up > >> the "distribution" process > >> which gets queued up on only scheduler that is awake, but that > >> scheduler is blocked > >> in an "eternal" call. The pings never become processed and the > >> distributions times out. > >> > >> You can wait for a patch or use zlib api to chunk up compression your > self, > >> see > >> implementation of gzip in zlib module. > >> > >> /Dan > >> > >> On Fri, Jan 21, 2011 at 2:48 AM, Ryan Zezeski > wrote: > >> > So...can anyone explain to me why zlib:gzip/1 is causing the > net_kernel > >> tick > >> > to be blocked? Do linked-in drivers block it's scheduler like NIFs? > I'm > >> > really curious on this one :) > >> > > >> > -Ryan > >> > > >> > On Tue, Jan 18, 2011 at 6:53 PM, Ryan Zezeski > >> wrote: > >> > > >> >> Apologies, the example I copied was run on my mac. > >> >> > >> >> This is what I have on the actual production machine: > >> >> > >> >> Erlang R14A (erts-5.8) [source] [64-bit] [smp:16:16] [rq:16] > >> >> [async-threads:0] [hipe] [kernel-poll:false] > >> >> > >> >> To be certain, I ran the same example (except this time using two > >> physical > >> >> machines) and achieved the same result. Namely, the 'bar' node > claims > >> 'foo' > >> >> is not responding and thus closes the connection. Whatever this is, > >> I've > >> >> now easily reproduced it on two different OSs, with 2 different > Erlang > >> >> versions. > >> >> > >> >> -Ryan > >> >> > >> >> On Tue, Jan 18, 2011 at 6:04 PM, Alain O'Dea > >> wrote: > >> >> > >> >>> On 2011-01-18, at 18:54, Ryan Zezeski wrote: > >> >>> > >> >>> > Hi everyone, > >> >>> > > >> >>> > Some of you may remember my latest question where I was having > weird > >> >>> node > >> >>> > timeout issues that I couldn't explain and I thought it might be > >> related > >> >>> to > >> >>> > the messages I was passing between my nodes. Well, I pinpointed > the > >> >>> problem > >> >>> > to a call to zlib:gzip/1. At first I was really surprised by > this, > >> as > >> >>> such > >> >>> > a harmless line of code surely should have nothing to do with the > >> >>> ability > >> >>> > for my nodes to communicate. However, as I dug further I realized > >> gzip > >> >>> was > >> >>> > implemented as a linked-in driver and I remember reading things > about > >> >>> how > >> >>> > one has to take care with them because they can trash the VM with > >> them. > >> >>> I > >> >>> > don't remember reading anything about them blocking code, and even > if > >> >>> they > >> >>> > do I fail to see why my SMP enabled node (16 cores) would allow > this > >> one > >> >>> > thread to block the tick. It occurred to me that maybe the > scheduler > >> >>> > responsible for that process is the one blocked by the driver. Do > >> >>> processes > >> >>> > have scheduler affinity? That would make sense, I guess. > >> >>> > > >> >>> > I've "fixed" this problem simply by using a plain port (i.e. run > in > >> it's > >> >>> own > >> >>> > OS process). For my purposes, this actually makes more sense in > the > >> >>> > majority of the places I was making use of gzip. Can someone > >> enlighten > >> >>> me > >> >>> > as to exactly what is happening behind the scenes? > >> >>> > > >> >>> > To reproduce I create a random 1.3GB file: > >> >>> > > >> >>> > dd if=/dev/urandom of=rand bs=1048576 count=1365 > >> >>> > > >> >>> > Then start two named nodes 'foo' and 'bar', connect them, read in > the > >> >>> file, > >> >>> > and then compress said file. Sometime later (I think around 60+ > >> >>> seconds) > >> >>> > the node 'bar' will claim that 'foo' is not responding. > >> >>> > > >> >>> > [progski@REDACTED ~/tmp_code/node_timeout] erl -name foo > >> >>> > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] > >> >>> > >> >>> Your SMP node seems to be capped at smp:2:2 when it out to be > smp:16. > >> >>> Some resource limit may be holding back the system. That said zlib > >> should > >> >>> not ever cause this issue. > >> >>> > >> >>> > [async-threads:0] [hipe] [kernel-poll:false] > >> >>> > > >> >>> > Eshell V5.8.1 (abort with ^G) > >> >>> > (foo@REDACTED)1> net_adm:ping('bar@REDACTED'). > >> >>> > pong > >> >>> > (foo@REDACTED)2> nodes(). > >> >>> > ['bar@REDACTED'] > >> >>> > (foo@REDACTED)3> {ok,Data} = file:read_file("rand"). > >> >>> > {ok,<<103,5,115,210,177,147,53,45,250,182,51,32,250,233, > >> >>> > 39,253,102,61,73,242,18,159,45,185,232,80,33,...>>} > >> >>> > (foo@REDACTED)4> zlib:gzip(Data). > >> >>> > <<31,139,8,0,0,0,0,0,0,3,0,15,64,240,191,103,5,115,210, > >> >>> > 177,147,53,45,250,182,51,32,250,233,...>> > >> >>> > (foo@REDACTED)5> > >> >>> > > >> >>> > > >> >>> > [progski@REDACTED ~/tmp_code/node_timeout] erl -name bar > >> >>> > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] > >> >>> > [async-threads:0] [hipe] [kernel-poll:false] > >> >>> > > >> >>> > Eshell V5.8.1 (abort with ^G) > >> >>> > (bar@REDACTED)1> nodes(). > >> >>> > ['foo@REDACTED'] > >> >>> > (bar@REDACTED)2> > >> >>> > =ERROR REPORT==== 18-Jan-2011::17:16:10 === > >> >>> > ** Node 'foo@REDACTED' not responding ** > >> >>> > ** Removing (timedout) connection ** > >> >>> > > >> >>> > > >> >>> > Thanks, > >> >>> > > >> >>> > -Ryan > >> >>> > >> >> > >> >> > >> > > >> > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From samerlang@REDACTED Sat Jan 22 20:10:49 2011 From: samerlang@REDACTED (Sam B) Date: Sat, 22 Jan 2011 12:10:49 -0700 Subject: getty Message-ID: The getty.erl program on http://www.erlang.org/examples/small_examples/ doesn't work on new versions of OTP, since it used user:server/1 to do the i/o. Anyone knows of a way of fixing this? From alain.odea@REDACTED Sat Jan 22 22:36:17 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Sat, 22 Jan 2011 18:06:17 -0330 Subject: [erlang-questions] Project for erlang beginers In-Reply-To: <1295716100303-3231595.post@n4.nabble.com> References: <1295716100303-3231595.post@n4.nabble.com> Message-ID: On 2011-01-22, at 13:38, shk wrote: > > Hello, > > I'm erlang beginner programmer. The best theory it is practice. Ant that's i > start new project for myself and maybe other erlang beginners. I want to > write pop3/smtp server in erlang to increase my skills in erlang and i want > to invite to take part in development/testing erlang beginner programmers as > i to take more experience. > > The project - https://github.com/onlyshk/epmail epmail . > > Now i implemet simple connection to the pop3 server and pop3 command > parsing. Now i make mail store. Now it'll be in Maildir format, but then > i'll plan to add supporting some db for mail storage. I think we together > can to make cool and usefull project. > > For erlang professional i want to say not strongly scold my code :) I learn > erlang about month, but i constantly improve my erlang knowledge. I shall be > very glad to criticism my code. > > In general who are interested, please feel free to join at project. > > Thank you. Implementing protocol parsing and servers for these protocols is a very good approach to learning Erlang. I think you will get great insight through this process. SMTP may already well-supported by several third-party Erlang applications, but the insights you gain in building your implementation will be very useful to improve or even replace the existing libraries. So keep at it :) I look forward to seeing your project grow and I wish you a happy coding experience along the way :) From alain.odea@REDACTED Sun Jan 23 01:17:21 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Sat, 22 Jan 2011 20:47:21 -0330 Subject: [erlang-questions] linked-in driver blocks net_kernel tick? In-Reply-To: References: Message-ID: <690421C6-98EC-48EF-9AEF-1D0381FCF093@gmail.com> I'm mostly happy so long as the standard distribution is very careful to have its BIFs, NIFs and Port Drivers not run long. It should be hard or impossible by design to supply inputs for these that cause them to run long. It would be even better if NIFs/Drivers were time-limited, not in that they could be stopped (I assume that is impractical), but in that their results would be thrown away and an error raised if they exceed the limit. This would make bad NIFs take the blame they deserve by being treated as errors when they take too long. I fear a future in which third-party applications with NIFs/drivers become commonplace dependencies for all applications (similar to the frameworks of the Java world), and that the NIFs/Drivers they contain break the soft realtime behavior of Erlang. Making bad NIFs and Drivers purposely unusable will avoid a gradual erosion of Erlang's soft realtime properties for many users. On 2011-01-22, at 14:42, Wallentin Dahlberg wrote: > I think his argument was that a driver or nif that does not use an async-thread for potentially blocking calls is a seriously broken driver. Consider the non-smp case. It will halt the beam and hinder important processes to be scheduled. I agree that in the smp case, one scheduler should not block the other schedulers in damaging calls. If a developer wants to destroy a scheduler with a broken driver, he should be free to do so. > > This was the fear with nifs. With nifs developers has an easy tool to really destroy the system in order to "increase performance" and implement 3rd party libs. There are several cases with different impact, > 1) destroy soft-real-time properties - reduction count badness > 2) destroy concurrency with blocking calls - scheduler badness > 3) destroy the system with faulty drivers (seg fault) - pure badness > > Some of these issues can be mitigated if the developer implements async threads, i.e. schedules operations to the async-pool. > > I feel that this is not ideal and is a heritage of ancient times. > > The problem in this case is that time does not progress in the system. Time is measured in reductions and each call is a reduction. At least this is the case with normal code. There are som special cases too, for instance a message sent "bumbs" the reduction count of the sender. Since native code (nif, bifs and drivers) do not increase reductions during its call but instead penalize the process after the call, time does not progress during the execution (as opposed to beam code). When a process reaches the reduction-limit it is scheduled out. Why reduction counters instead of time slices? Supposedly it much faster (according to Bj?rn). It is a design decision with trade-offs. The solution is fast and nimble, it has certain characteristics that are favorable and has some characteristics that are less favorable. I would favor time-slices since i think it would be fairer to the system and potentially we could save a register. Exactly how it should be done is a question for a different time. > > The load balancing in the scheduler is checked when a certain reduction count is reached for that scheduler. We do not want to check this too often since it will then become a serialization point. > > But fear not, there is a (beautiful) solution that is being discussed in the erts-team. Hopefully we can agree on the details. > > // Bj?rn-Egil > > > 2011/1/22 Dan Gudmundsson > Rickard who have implemented this should explain it. > > If I have understood it correctly, it works like this: > If a scheduler do not have any work to do it will be disabled. > It will be disabled until a live thread discovers it have to much work and > wakes a sleeping scheduler. The run-queues are only checked when processes > are scheduled. > > Since in this case the only living scheduler is busy for a very long time, > no queue checking will be done and the all schedulers will be blocked until > the call to the driver is complete. > > We had a long discussion during lunch about it, and we didn't agree > how it should > work. :-) > > I agree that zlib is broken and it should be fixed but I still believe that it > breaks the rule about least astonishment, if I have 16 schedulers and > one is blocked > in a long function call I still expect other code to be invoked. > Rickards thought is that > such call should never happen and should be called through an async > driver or a separate > thread. I guess it will take a couple of more lunches to come to a > conclusion :-) > > /Dan > > On Fri, Jan 21, 2011 at 10:25 PM, Ryan Zezeski wrote: > > Dan, > > > > Thanks for the reply, I'll be sure to chunk my data. I was using the gzip/1 > > call for convenience. > > > > That said, I'm still a little fuzzy on something you said. Why is it that > > the "distribution" process is scheduled on the same scheduler that's running > > the call to the driver? Why not schedule it on one of the 15 other > > schedulers that are currently sleeping? Does this mean any other message I > > send will also be blocked? Dare I ask, how does the scheduling work > > exactly? > > > > -Ryan > > > > On Fri, Jan 21, 2011 at 5:16 AM, Dan Gudmundsson wrote: > > > >> All c-calls blocks a schedulers, if they are not pushed out to a thread. > >> > >> In this case it's a bug in the zlib module (probably by me) gzip should > >> chunk up the input before invoking the driver. > >> > >> What happens is that all schedulers go to sleep because there is no work to > >> do, > >> except the one invoking the driver, a ping is received and wakes up > >> the "distribution" process > >> which gets queued up on only scheduler that is awake, but that > >> scheduler is blocked > >> in an "eternal" call. The pings never become processed and the > >> distributions times out. > >> > >> You can wait for a patch or use zlib api to chunk up compression your self, > >> see > >> implementation of gzip in zlib module. > >> > >> /Dan > >> > >> On Fri, Jan 21, 2011 at 2:48 AM, Ryan Zezeski wrote: > >> > So...can anyone explain to me why zlib:gzip/1 is causing the net_kernel > >> tick > >> > to be blocked? Do linked-in drivers block it's scheduler like NIFs? I'm > >> > really curious on this one :) > >> > > >> > -Ryan > >> > > >> > On Tue, Jan 18, 2011 at 6:53 PM, Ryan Zezeski > >> wrote: > >> > > >> >> Apologies, the example I copied was run on my mac. > >> >> > >> >> This is what I have on the actual production machine: > >> >> > >> >> Erlang R14A (erts-5.8) [source] [64-bit] [smp:16:16] [rq:16] > >> >> [async-threads:0] [hipe] [kernel-poll:false] > >> >> > >> >> To be certain, I ran the same example (except this time using two > >> physical > >> >> machines) and achieved the same result. Namely, the 'bar' node claims > >> 'foo' > >> >> is not responding and thus closes the connection. Whatever this is, > >> I've > >> >> now easily reproduced it on two different OSs, with 2 different Erlang > >> >> versions. > >> >> > >> >> -Ryan > >> >> > >> >> On Tue, Jan 18, 2011 at 6:04 PM, Alain O'Dea > >> wrote: > >> >> > >> >>> On 2011-01-18, at 18:54, Ryan Zezeski wrote: > >> >>> > >> >>> > Hi everyone, > >> >>> > > >> >>> > Some of you may remember my latest question where I was having weird > >> >>> node > >> >>> > timeout issues that I couldn't explain and I thought it might be > >> related > >> >>> to > >> >>> > the messages I was passing between my nodes. Well, I pinpointed the > >> >>> problem > >> >>> > to a call to zlib:gzip/1. At first I was really surprised by this, > >> as > >> >>> such > >> >>> > a harmless line of code surely should have nothing to do with the > >> >>> ability > >> >>> > for my nodes to communicate. However, as I dug further I realized > >> gzip > >> >>> was > >> >>> > implemented as a linked-in driver and I remember reading things about > >> >>> how > >> >>> > one has to take care with them because they can trash the VM with > >> them. > >> >>> I > >> >>> > don't remember reading anything about them blocking code, and even if > >> >>> they > >> >>> > do I fail to see why my SMP enabled node (16 cores) would allow this > >> one > >> >>> > thread to block the tick. It occurred to me that maybe the scheduler > >> >>> > responsible for that process is the one blocked by the driver. Do > >> >>> processes > >> >>> > have scheduler affinity? That would make sense, I guess. > >> >>> > > >> >>> > I've "fixed" this problem simply by using a plain port (i.e. run in > >> it's > >> >>> own > >> >>> > OS process). For my purposes, this actually makes more sense in the > >> >>> > majority of the places I was making use of gzip. Can someone > >> enlighten > >> >>> me > >> >>> > as to exactly what is happening behind the scenes? > >> >>> > > >> >>> > To reproduce I create a random 1.3GB file: > >> >>> > > >> >>> > dd if=/dev/urandom of=rand bs=1048576 count=1365 > >> >>> > > >> >>> > Then start two named nodes 'foo' and 'bar', connect them, read in the > >> >>> file, > >> >>> > and then compress said file. Sometime later (I think around 60+ > >> >>> seconds) > >> >>> > the node 'bar' will claim that 'foo' is not responding. > >> >>> > > >> >>> > [progski@REDACTED ~/tmp_code/node_timeout] erl -name foo > >> >>> > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] > >> >>> > >> >>> Your SMP node seems to be capped at smp:2:2 when it out to be smp:16. > >> >>> Some resource limit may be holding back the system. That said zlib > >> should > >> >>> not ever cause this issue. > >> >>> > >> >>> > [async-threads:0] [hipe] [kernel-poll:false] > >> >>> > > >> >>> > Eshell V5.8.1 (abort with ^G) > >> >>> > (foo@REDACTED)1> net_adm:ping('bar@REDACTED'). > >> >>> > pong > >> >>> > (foo@REDACTED)2> nodes(). > >> >>> > ['bar@REDACTED'] > >> >>> > (foo@REDACTED)3> {ok,Data} = file:read_file("rand"). > >> >>> > {ok,<<103,5,115,210,177,147,53,45,250,182,51,32,250,233, > >> >>> > 39,253,102,61,73,242,18,159,45,185,232,80,33,...>>} > >> >>> > (foo@REDACTED)4> zlib:gzip(Data). > >> >>> > <<31,139,8,0,0,0,0,0,0,3,0,15,64,240,191,103,5,115,210, > >> >>> > 177,147,53,45,250,182,51,32,250,233,...>> > >> >>> > (foo@REDACTED)5> > >> >>> > > >> >>> > > >> >>> > [progski@REDACTED ~/tmp_code/node_timeout] erl -name bar > >> >>> > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] > >> >>> > [async-threads:0] [hipe] [kernel-poll:false] > >> >>> > > >> >>> > Eshell V5.8.1 (abort with ^G) > >> >>> > (bar@REDACTED)1> nodes(). > >> >>> > ['foo@REDACTED'] > >> >>> > (bar@REDACTED)2> > >> >>> > =ERROR REPORT==== 18-Jan-2011::17:16:10 === > >> >>> > ** Node 'foo@REDACTED' not responding ** > >> >>> > ** Removing (timedout) connection ** > >> >>> > > >> >>> > > >> >>> > Thanks, > >> >>> > > >> >>> > -Ryan > >> >>> > >> >> > >> >> > >> > > >> > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From jws@REDACTED Sun Jan 23 01:06:50 2011 From: jws@REDACTED (Jeff Schultz) Date: Sun, 23 Jan 2011 11:06:50 +1100 Subject: [erlang-questions] linked-in driver blocks net_kernel tick? In-Reply-To: References: Message-ID: <20110123000650.GA9905@mulga.csse.unimelb.edu.au> On Sat, Jan 22, 2011 at 07:12:04PM +0100, Wallentin Dahlberg wrote: > But fear not, there is a (beautiful) solution that is being discussed in the > erts-team. Hopefully we can agree on the details. We've seen a few references to this forthcoming solution on this list. How about a hint? Jeff Schultz From mjtruog@REDACTED Sun Jan 23 08:56:16 2011 From: mjtruog@REDACTED (Michael Truog) Date: Sat, 22 Jan 2011 23:56:16 -0800 Subject: erlbench and GEP[D] (Generic Erlang Port [Driver]) version 0.7 Released Message-ID: <4D3BDF20.2030200@gmail.com> An Erlang data structures comparison now exists for data indexed by integers and data indexed by strings (lists of integers) here: https://github.com/okeuday/erlbench My latest results (non-HiPE/HiPE) are here: http://okeuday.livejournal.com/16941.html A new version (0.7) of GEP[D] (Generic Erlang Port [Driver]) is available at: https://github.com/okeuday/generic-erlang-port--driver- GEP[D] uses C preprocessor macro expansion to automatically create either an Erlang port or an Erlang port driver which generates the same output. A few differences exist, because there is the choice to use the asynchronous thread pool for the port driver functions which return as {Port, {async, Data}} instead of {Port, {data, Data}}. The generated port code now captures stdout and stderr which is sent as messages to the Erlang process. An example of the code configuration necessary to generate bindings is below: // specify all the functions to generate bindings for // __________________________________________________________________________ // || FUNCTION || ARITY/TYPES || RETURN TYPE || ASYNC CALL || #define PORT_DRIVER_FUNCTIONS \ ((sleep_test1, 1, (uint32_t), void, 0)) \ ((sleep_test2, 1, (uint32_t), void, 1)) \ ((integer_test1, 0, (), uint64_t, 0)) \ ((char_test1, 1, (char), char, 0)) \ ((char_test2, 1, (uchar), uchar, 0)) \ ((float_test1, 0, (), float, 0)) \ ((pchar_test1, 1, (pchar_len), pchar, 0)) \ ((time_test1, 1, (time_t), pchar, 0)) \ ((float_test2, 1, (double), float, 0)) \ ((integer_test2, 4, (int8_t,int16_t,int32_t,int64_t), int32_t, 0)) \ ((integer_test3, 4, (uint8_t,uint16_t,uint32_t,uint64_t), uint32_t, 0)) \ ((pchar_test2, 6, (pchar_len, char, pchar_len, char, \ pchar_len, char), pchar, 0)) // specify the name of the driver, as provided to driver initialization // (e.g., erl_ddll:load_driver/2, erlang:open_port/2, ErlDrvEntry, etc.) #define PORT_DRIVER_NAME_PREFIX test_functions_port_driver // specify the C include file with the functions that will be called // from within the Erlang code #define PORT_DRIVER_C_FUNCTIONS_HEADER_FILE "test_functions.h" I know NIFs are nifty, but I still like being able to switch between ports and port drivers without any code changes (PORT_DRIVER_FUNCTIONS can be used to generate a port also). NIFs are unable to use the Erlang asynchronous thread pool, though usage of the asynchronous thread pool can be a bad choice if you don't read about the problems already mentioned on the mailing list (mainly, driver gets locked and no shared job queue). The generated port driver code uses port-level locking, so you should be able to more fully utilize all the VM scheduler threads with more than one instance of the same port driver (if your code is thread-safe). From dujinfang@REDACTED Sun Jan 23 14:24:19 2011 From: dujinfang@REDACTED (Seven Du) Date: Sun, 23 Jan 2011 21:24:19 +0800 Subject: [erlang-questions] mysql driver in erlang support stored procedure In-Reply-To: <912DC819-77A5-49C5-AEFB-58719CC81F9C@gmail.com> References: <912DC819-77A5-49C5-AEFB-58719CC81F9C@gmail.com> Message-ID: Unfortunately we need to support legacy DBs. On Sat, Jan 22, 2011 at 5:56 AM, Alain O'Dea wrote: > On 2011-01-21, at 12:40, Seven Du wrote: > >> Hi, >> >> I was using erlang_mysql_driver for a while, but it seems doesn't >> support stored procedure. >> >> Then I found another driver called EMysql >> https://github.com/JacobVorreuter/emysql , on test it does support >> call procedure, however, it returns ok_packets sometimes on select >> only procedures, and it happens also for simple selects(where it >> should always be result_packet I think): >> >> fetch4() -> >> ? ?SQL = "select * from test;", >> ? ?Result = emysql:execute(mysql2, SQL), >> ? ?io:format("~p~n", [Result]), >> ? ?Fields = Result#result_packet.field_list, >> ? ?Rows = Result#result_packet.rows, >> >> ? ?io:format("~p~n", [Fields]), >> ? ?io:format("~p~n", [Rows]). >> >> >>> mytest:fetch4(). >> {ok_packet,28,0,0,2,0,[]} >> ** exception error: {badrecord,result_packet} >> ? ? in function ?m_telsystem:fetch/0 >> >> >> I should report an issue on github but the only issue there is still >> in unread state and it's not been updated since 2009. There are other >> clones of the project, but not sure which one will has support. >> >> Any one has experience on this or other options? >> >> erlang_mysql_driver is quite state besides I haven't got it support >> stored procedure. >> >> Thanks. >> >> -- >> About: http://about.me/dujinfang >> Blog: http://www.dujinfang.com >> Proj: ?http://www.freeswitch.org.cn >> > > PostgreSQL is largely equivalent to MySQL and has mature support in epgsql in Zotonic. ?If it is an option I recommend using PostgreSQL instead. -- About: http://about.me/dujinfang Blog: http://www.dujinfang.com Proj:? http://www.freeswitch.org.cn From dujinfang@REDACTED Sun Jan 23 14:25:02 2011 From: dujinfang@REDACTED (Seven Du) Date: Sun, 23 Jan 2011 21:25:02 +0800 Subject: [erlang-questions] mysql driver in erlang support stored procedure In-Reply-To: <4d39d451.523de30a.63a5.ffffe5fcSMTPIN_ADDED@mx.google.com> References: <4d39d451.523de30a.63a5.ffffe5fcSMTPIN_ADDED@mx.google.com> Message-ID: Thanks Henning, I reported an issue on github. On Sat, Jan 22, 2011 at 2:45 AM, hd2010@REDACTED wrote: > Hi Seven, I'll be looking into it. > > The Eonblast fork is what I am working at today. If you have more info on > what the unexpected behavior of SP in Emysql is, please send it my way. > > Thanks, > Henning > > > > -- About: http://about.me/dujinfang Blog: http://www.dujinfang.com Proj:? http://www.freeswitch.org.cn From robert.virding@REDACTED Sun Jan 23 17:04:51 2011 From: robert.virding@REDACTED (Robert Virding) Date: Sun, 23 Jan 2011 16:04:51 +0000 (GMT) Subject: [erlang-questions] linked-in driver blocks net_kernel tick? In-Reply-To: <359963688.325961295798642758.JavaMail.root@zimbra> Message-ID: <1045859263.325981295798691364.JavaMail.root@zimbra> There are really two different problems: 1. No run-queue checking if the only living scheduler (schedulers ?) is blocked. 2. zlib is written in a blocking way. Both should be fixed though the first is the more serious. It will also become serious as NIFs become more used. While "hardliner me" says that NIF writers have themselves to blame if they block the system and that they should RTFM, "softliner me" says that we should probably try to help them and make it easier to get it right. Robert ----- "Dan Gudmundsson" wrote: > Rickard who have implemented this should explain it. > > If I have understood it correctly, it works like this: > If a scheduler do not have any work to do it will be disabled. > It will be disabled until a live thread discovers it have to much work > and > wakes a sleeping scheduler. The run-queues are only checked when > processes > are scheduled. > > Since in this case the only living scheduler is busy for a very long > time, > no queue checking will be done and the all schedulers will be blocked > until > the call to the driver is complete. > > We had a long discussion during lunch about it, and we didn't agree > how it should > work. :-) > > I agree that zlib is broken and it should be fixed but I still believe > that it > breaks the rule about least astonishment, if I have 16 schedulers and > one is blocked > in a long function call I still expect other code to be invoked. > Rickards thought is that > such call should never happen and should be called through an async > driver or a separate > thread. I guess it will take a couple of more lunches to come to a > conclusion :-) > > /Dan > > On Fri, Jan 21, 2011 at 10:25 PM, Ryan Zezeski > wrote: > > Dan, > > > > Thanks for the reply, I'll be sure to chunk my data. ?I was using > the gzip/1 > > call for convenience. > > > > That said, I'm still a little fuzzy on something you said. ?Why is > it that > > the "distribution" process is scheduled on the same scheduler that's > running > > the call to the driver? ?Why not schedule it on one of the 15 other > > schedulers that are currently sleeping? ?Does this mean any other > message I > > send will also be blocked? ?Dare I ask, how does the scheduling > work > > exactly? > > > > -Ryan > > > > On Fri, Jan 21, 2011 at 5:16 AM, Dan Gudmundsson > wrote: > > > >> All c-calls blocks a schedulers, if they are not pushed out to a > thread. > >> > >> In this case it's a bug in the zlib module (probably by me) gzip > should > >> chunk up the input before invoking the driver. > >> > >> What happens is that all schedulers go to sleep because there is no > work to > >> do, > >> except the one invoking the driver, a ping is received and wakes > up > >> the "distribution" process > >> which gets queued up on only scheduler that is awake, but that > >> scheduler is blocked > >> in an "eternal" call. The pings never become processed and the > >> distributions times out. > >> > >> You can wait for a patch or use zlib api to chunk up compression > your self, > >> see > >> implementation of gzip in zlib module. > >> > >> /Dan > >> > >> On Fri, Jan 21, 2011 at 2:48 AM, Ryan Zezeski > wrote: > >> > So...can anyone explain to me why zlib:gzip/1 is causing the > net_kernel > >> tick > >> > to be blocked? ?Do linked-in drivers block it's scheduler like > NIFs? ?I'm > >> > really curious on this one :) > >> > > >> > -Ryan > >> > > >> > On Tue, Jan 18, 2011 at 6:53 PM, Ryan Zezeski > > >> wrote: > >> > > >> >> Apologies, the example I copied was run on my mac. > >> >> > >> >> This is what I have on the actual production machine: > >> >> > >> >> Erlang R14A (erts-5.8) [source] [64-bit] [smp:16:16] [rq:16] > >> >> [async-threads:0] [hipe] [kernel-poll:false] > >> >> > >> >> To be certain, I ran the same example (except this time using > two > >> physical > >> >> machines) and achieved the same result. ?Namely, the 'bar' node > claims > >> 'foo' > >> >> is not responding and thus closes the connection. ?Whatever this > is, > >> I've > >> >> now easily reproduced it on two different OSs, with 2 different > Erlang > >> >> versions. > >> >> > >> >> -Ryan > >> >> > >> >> On Tue, Jan 18, 2011 at 6:04 PM, Alain O'Dea > > >> wrote: > >> >> > >> >>> On 2011-01-18, at 18:54, Ryan Zezeski > wrote: > >> >>> > >> >>> > Hi everyone, > >> >>> > > >> >>> > Some of you may remember my latest question where I was > having weird > >> >>> node > >> >>> > timeout issues that I couldn't explain and I thought it might > be > >> related > >> >>> to > >> >>> > the messages I was passing between my nodes. ?Well, I > pinpointed the > >> >>> problem > >> >>> > to a call to zlib:gzip/1. ?At first I was really surprised by > this, > >> as > >> >>> such > >> >>> > a harmless line of code surely should have nothing to do with > the > >> >>> ability > >> >>> > for my nodes to communicate. ?However, as I dug further I > realized > >> gzip > >> >>> was > >> >>> > implemented as a linked-in driver and I remember reading > things about > >> >>> how > >> >>> > one has to take care with them because they can trash the VM > with > >> them. > >> >>> ?I > >> >>> > don't remember reading anything about them blocking code, and > even if > >> >>> they > >> >>> > do I fail to see why my SMP enabled node (16 cores) would > allow this > >> one > >> >>> > thread to block the tick. ?It occurred to me that maybe the > scheduler > >> >>> > responsible for that process is the one blocked by the > driver. ?Do > >> >>> processes > >> >>> > have scheduler affinity? ?That would make sense, I guess. > >> >>> > > >> >>> > I've "fixed" this problem simply by using a plain port (i.e. > run in > >> it's > >> >>> own > >> >>> > OS process). ?For my purposes, this actually makes more sense > in the > >> >>> > majority of the places I was making use of gzip. ?Can > someone > >> enlighten > >> >>> me > >> >>> > as to exactly what is happening behind the scenes? > >> >>> > > >> >>> > To reproduce I create a random 1.3GB file: > >> >>> > > >> >>> > dd if=/dev/urandom of=rand bs=1048576 count=1365 > >> >>> > > >> >>> > Then start two named nodes 'foo' and 'bar', connect them, > read in the > >> >>> file, > >> >>> > and then compress said file. ?Sometime later (I think around > 60+ > >> >>> seconds) > >> >>> > the node 'bar' will claim that 'foo' is not responding. > >> >>> > > >> >>> > [progski@REDACTED ~/tmp_code/node_timeout] erl -name > foo > >> >>> > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] > >> >>> > >> >>> Your SMP node seems to be capped at smp:2:2 when it out to be > smp:16. > >> >>> ?Some resource limit may be holding back the system. That said > zlib > >> should > >> >>> not ever cause this issue. > >> >>> > >> >>> > [async-threads:0] [hipe] [kernel-poll:false] > >> >>> > > >> >>> > Eshell V5.8.1 ?(abort with ^G) > >> >>> > (foo@REDACTED)1> net_adm:ping('bar@REDACTED'). > >> >>> > pong > >> >>> > (foo@REDACTED)2> nodes(). > >> >>> > ['bar@REDACTED'] > >> >>> > (foo@REDACTED)3> {ok,Data} = file:read_file("rand"). > >> >>> > {ok,<<103,5,115,210,177,147,53,45,250,182,51,32,250,233, > >> >>> > ? ? ?39,253,102,61,73,242,18,159,45,185,232,80,33,...>>} > >> >>> > (foo@REDACTED)4> zlib:gzip(Data). > >> >>> > <<31,139,8,0,0,0,0,0,0,3,0,15,64,240,191,103,5,115,210, > >> >>> > ?177,147,53,45,250,182,51,32,250,233,...>> > >> >>> > (foo@REDACTED)5> > >> >>> > > >> >>> > > >> >>> > [progski@REDACTED ~/tmp_code/node_timeout] erl -name > bar > >> >>> > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] > >> >>> > [async-threads:0] [hipe] [kernel-poll:false] > >> >>> > > >> >>> > Eshell V5.8.1 ?(abort with ^G) > >> >>> > (bar@REDACTED)1> nodes(). > >> >>> > ['foo@REDACTED'] > >> >>> > (bar@REDACTED)2> > >> >>> > =ERROR REPORT==== 18-Jan-2011::17:16:10 === > >> >>> > ** Node 'foo@REDACTED' not responding ** > >> >>> > ** Removing (timedout) connection ** > >> >>> > > >> >>> > > >> >>> > Thanks, > >> >>> > > >> >>> > -Ryan > >> >>> > >> >> > >> >> > >> > > >> > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED -- Robert Virding, Erlang Solutions Ltd. From rzezeski@REDACTED Sun Jan 23 19:49:15 2011 From: rzezeski@REDACTED (Ryan Zezeski) Date: Sun, 23 Jan 2011 13:49:15 -0500 Subject: [erlang-questions] linked-in driver blocks net_kernel tick? In-Reply-To: <1045859263.325981295798691364.JavaMail.root@zimbra> References: <359963688.325961295798642758.JavaMail.root@zimbra> <1045859263.325981295798691364.JavaMail.root@zimbra> Message-ID: Interestingly enough, my coworker and I got into a debate about whether this is good behavior or not. He (my coworker) focused on the fact that there is a misbehaving entity that is doing something it shouldn't be doing (and therefore a "node down" is the correct response) whereas I focused on the semantics of the net_kernel tick. To me, the tick is nothing more than doing a ping every N seconds. If so many pings go unanswered, then the node must be assumed to be down. This makes perfect sense to me. However, in my case I have two nodes, each running Erlang SMP, each with 16 schedulers. One node decides to compress a very large file using a linked-in driver all in one go causing the only live scheduler to be blocked. Meanwhile, the other node sends a tick every 15s waiting for a response. It turns out that the tick response process happens to be scheduled on the same scheduler as the long-running, blocking process even though there are 15 idle schedulers waiting in the wings. The compression happens to take S seconds, which turns out to be greater than the threshold T that the tick response process had to respond. This causes the other node to consider it down and the socket is killed. This is wrong because a) the node was never down, and in fact was still doing work but the tick response got stuck behing a long-running process in the queue (like getting your license at the MVA :) ) and b) the network between the two was never compromised. Both nodes are still very much alive and doing work yet they disconnected from each other, which doesn't make any sense to me. I guess the upshot is to be very careful with linked-in code, not only because it can crash the VM (which is the common warning) but because it can block critical proceses that will affect the system in unforeseen and obscure ways. -Ryan On Sun, Jan 23, 2011 at 11:04 AM, Robert Virding < robert.virding@REDACTED> wrote: > There are really two different problems: > > 1. No run-queue checking if the only living scheduler (schedulers ?) is > blocked. > 2. zlib is written in a blocking way. > > Both should be fixed though the first is the more serious. It will also > become serious as NIFs become more used. While "hardliner me" says that NIF > writers have themselves to blame if they block the system and that they > should RTFM, "softliner me" says that we should probably try to help them > and make it easier to get it right. > > Robert > > ----- "Dan Gudmundsson" wrote: > > > Rickard who have implemented this should explain it. > > > > If I have understood it correctly, it works like this: > > If a scheduler do not have any work to do it will be disabled. > > It will be disabled until a live thread discovers it have to much work > > and > > wakes a sleeping scheduler. The run-queues are only checked when > > processes > > are scheduled. > > > > Since in this case the only living scheduler is busy for a very long > > time, > > no queue checking will be done and the all schedulers will be blocked > > until > > the call to the driver is complete. > > > > We had a long discussion during lunch about it, and we didn't agree > > how it should > > work. :-) > > > > I agree that zlib is broken and it should be fixed but I still believe > > that it > > breaks the rule about least astonishment, if I have 16 schedulers and > > one is blocked > > in a long function call I still expect other code to be invoked. > > Rickards thought is that > > such call should never happen and should be called through an async > > driver or a separate > > thread. I guess it will take a couple of more lunches to come to a > > conclusion :-) > > > > /Dan > > > > On Fri, Jan 21, 2011 at 10:25 PM, Ryan Zezeski > > wrote: > > > Dan, > > > > > > Thanks for the reply, I'll be sure to chunk my data. I was using > > the gzip/1 > > > call for convenience. > > > > > > That said, I'm still a little fuzzy on something you said. Why is > > it that > > > the "distribution" process is scheduled on the same scheduler that's > > running > > > the call to the driver? Why not schedule it on one of the 15 other > > > schedulers that are currently sleeping? Does this mean any other > > message I > > > send will also be blocked? Dare I ask, how does the scheduling > > work > > > exactly? > > > > > > -Ryan > > > > > > On Fri, Jan 21, 2011 at 5:16 AM, Dan Gudmundsson > > wrote: > > > > > >> All c-calls blocks a schedulers, if they are not pushed out to a > > thread. > > >> > > >> In this case it's a bug in the zlib module (probably by me) gzip > > should > > >> chunk up the input before invoking the driver. > > >> > > >> What happens is that all schedulers go to sleep because there is no > > work to > > >> do, > > >> except the one invoking the driver, a ping is received and wakes > > up > > >> the "distribution" process > > >> which gets queued up on only scheduler that is awake, but that > > >> scheduler is blocked > > >> in an "eternal" call. The pings never become processed and the > > >> distributions times out. > > >> > > >> You can wait for a patch or use zlib api to chunk up compression > > your self, > > >> see > > >> implementation of gzip in zlib module. > > >> > > >> /Dan > > >> > > >> On Fri, Jan 21, 2011 at 2:48 AM, Ryan Zezeski > > wrote: > > >> > So...can anyone explain to me why zlib:gzip/1 is causing the > > net_kernel > > >> tick > > >> > to be blocked? Do linked-in drivers block it's scheduler like > > NIFs? I'm > > >> > really curious on this one :) > > >> > > > >> > -Ryan > > >> > > > >> > On Tue, Jan 18, 2011 at 6:53 PM, Ryan Zezeski > > > > >> wrote: > > >> > > > >> >> Apologies, the example I copied was run on my mac. > > >> >> > > >> >> This is what I have on the actual production machine: > > >> >> > > >> >> Erlang R14A (erts-5.8) [source] [64-bit] [smp:16:16] [rq:16] > > >> >> [async-threads:0] [hipe] [kernel-poll:false] > > >> >> > > >> >> To be certain, I ran the same example (except this time using > > two > > >> physical > > >> >> machines) and achieved the same result. Namely, the 'bar' node > > claims > > >> 'foo' > > >> >> is not responding and thus closes the connection. Whatever this > > is, > > >> I've > > >> >> now easily reproduced it on two different OSs, with 2 different > > Erlang > > >> >> versions. > > >> >> > > >> >> -Ryan > > >> >> > > >> >> On Tue, Jan 18, 2011 at 6:04 PM, Alain O'Dea > > > > >> wrote: > > >> >> > > >> >>> On 2011-01-18, at 18:54, Ryan Zezeski > > wrote: > > >> >>> > > >> >>> > Hi everyone, > > >> >>> > > > >> >>> > Some of you may remember my latest question where I was > > having weird > > >> >>> node > > >> >>> > timeout issues that I couldn't explain and I thought it might > > be > > >> related > > >> >>> to > > >> >>> > the messages I was passing between my nodes. Well, I > > pinpointed the > > >> >>> problem > > >> >>> > to a call to zlib:gzip/1. At first I was really surprised by > > this, > > >> as > > >> >>> such > > >> >>> > a harmless line of code surely should have nothing to do with > > the > > >> >>> ability > > >> >>> > for my nodes to communicate. However, as I dug further I > > realized > > >> gzip > > >> >>> was > > >> >>> > implemented as a linked-in driver and I remember reading > > things about > > >> >>> how > > >> >>> > one has to take care with them because they can trash the VM > > with > > >> them. > > >> >>> I > > >> >>> > don't remember reading anything about them blocking code, and > > even if > > >> >>> they > > >> >>> > do I fail to see why my SMP enabled node (16 cores) would > > allow this > > >> one > > >> >>> > thread to block the tick. It occurred to me that maybe the > > scheduler > > >> >>> > responsible for that process is the one blocked by the > > driver. Do > > >> >>> processes > > >> >>> > have scheduler affinity? That would make sense, I guess. > > >> >>> > > > >> >>> > I've "fixed" this problem simply by using a plain port (i.e. > > run in > > >> it's > > >> >>> own > > >> >>> > OS process). For my purposes, this actually makes more sense > > in the > > >> >>> > majority of the places I was making use of gzip. Can > > someone > > >> enlighten > > >> >>> me > > >> >>> > as to exactly what is happening behind the scenes? > > >> >>> > > > >> >>> > To reproduce I create a random 1.3GB file: > > >> >>> > > > >> >>> > dd if=/dev/urandom of=rand bs=1048576 count=1365 > > >> >>> > > > >> >>> > Then start two named nodes 'foo' and 'bar', connect them, > > read in the > > >> >>> file, > > >> >>> > and then compress said file. Sometime later (I think around > > 60+ > > >> >>> seconds) > > >> >>> > the node 'bar' will claim that 'foo' is not responding. > > >> >>> > > > >> >>> > [progski@REDACTED ~/tmp_code/node_timeout] erl -name > > foo > > >> >>> > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] > > >> >>> > > >> >>> Your SMP node seems to be capped at smp:2:2 when it out to be > > smp:16. > > >> >>> Some resource limit may be holding back the system. That said > > zlib > > >> should > > >> >>> not ever cause this issue. > > >> >>> > > >> >>> > [async-threads:0] [hipe] [kernel-poll:false] > > >> >>> > > > >> >>> > Eshell V5.8.1 (abort with ^G) > > >> >>> > (foo@REDACTED)1> net_adm:ping('bar@REDACTED'). > > >> >>> > pong > > >> >>> > (foo@REDACTED)2> nodes(). > > >> >>> > ['bar@REDACTED'] > > >> >>> > (foo@REDACTED)3> {ok,Data} = file:read_file("rand"). > > >> >>> > {ok,<<103,5,115,210,177,147,53,45,250,182,51,32,250,233, > > >> >>> > 39,253,102,61,73,242,18,159,45,185,232,80,33,...>>} > > >> >>> > (foo@REDACTED)4> zlib:gzip(Data). > > >> >>> > <<31,139,8,0,0,0,0,0,0,3,0,15,64,240,191,103,5,115,210, > > >> >>> > 177,147,53,45,250,182,51,32,250,233,...>> > > >> >>> > (foo@REDACTED)5> > > >> >>> > > > >> >>> > > > >> >>> > [progski@REDACTED ~/tmp_code/node_timeout] erl -name > > bar > > >> >>> > Erlang R14B (erts-5.8.1) [source] [64-bit] [smp:2:2] [rq:2] > > >> >>> > [async-threads:0] [hipe] [kernel-poll:false] > > >> >>> > > > >> >>> > Eshell V5.8.1 (abort with ^G) > > >> >>> > (bar@REDACTED)1> nodes(). > > >> >>> > ['foo@REDACTED'] > > >> >>> > (bar@REDACTED)2> > > >> >>> > =ERROR REPORT==== 18-Jan-2011::17:16:10 === > > >> >>> > ** Node 'foo@REDACTED' not responding ** > > >> >>> > ** Removing (timedout) connection ** > > >> >>> > > > >> >>> > > > >> >>> > Thanks, > > >> >>> > > > >> >>> > -Ryan > > >> >>> > > >> >> > > >> >> > > >> > > > >> > > > > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- > Robert Virding, Erlang Solutions Ltd. > > From alain.odea@REDACTED Sun Jan 23 22:54:56 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Sun, 23 Jan 2011 18:24:56 -0330 Subject: [erlang-questions] record updating with several functions In-Reply-To: <201101231305284154898@its3.ch> References: <201101191846160683077@its3.ch> <781D5239-FFA3-497F-AA2D-04F197498F82@gmail.com> <201101211024021303579@its3.ch> <201101220000570583005@its3.ch> <201101231305284154898@its3.ch> Message-ID: <85166252-9190-4D09-8CA1-B24C0DC263BE@gmail.com> On 2011-01-23, at 8:35, "info" wrote: > Hi Alain, > > According to your explanation, I modified my example: > > -record (field,{ch1,ch2}). > testrecord()-> > Record = functionA(#field{}), > io:format("M: ch1 is ~s ch2 is ~s~n",[Record#field.ch1,Record#field.ch2]). > functionA(RecA)-> > R1=RecA#field{ch1 = "hello"}, > io:format("FA: ch1 is ~s ch2 is ~s~n",[R1#field.ch1,R1#field.ch2]), > RecB = functionB(R1), > RecB. > functionB(RecB)-> > R2=RecB#field{ch2 = "boy"}, > io:format("FB: ch1 is ~s ch2 is ~s~n",[R2#field.ch1,R2#field.ch2]), > R2. > > Now it's ok but I find the solution "hard" (I don't find the good term in english ...). I suppose that a good erlang programmer will not solve the problem like this ? > Regards, > > J-Ph. Constantin > ITS3 Gen?ve > www.its3.ch Hi: The english term you are looking for is "awkward" ;) It certainly is awkward, but not far from what you'd do in production Erlang code. Normally, an experienced Erlang/OTP programmer would use OTP generic behaviours and callback modules. This is a pretty advanced topic, but there is a very good book by Eric Merritt, Martin Logan and Richard Carlsson called Erlang and OTP in Action http://manning.com/logan. When you use an OTP behaviour callback module each callback function is handed a request, a sender PID and the State. The State is like this fields record you are working with. It is an opaque data structure that your callback module owns the internal details of. Each of your callback functions returns a reply with a new version of the State (or just the old state if nothing needs to change). If you want to get serious about code structure in Erlang, then OTP is the idiomatic approach. However, I seriously suggest that you become comfortable with the syntax and the semantics of sequential Erlang before getting into OTP. Learn You Some Erlang at http://learnyousomeerlang.com/ is a great tutorial supported by Erlang Solutions. It has a sense of humour and it presents the topics in an easy to follow format. It flows well and it will really answer most of your questions about the language. Please work through Learn You Some Erlang and post any questions or problems you have back to erlang-questions so that one of the other members of the community or I can assist you :) When replying to Erlang-questions remember to Reply All if you want it on the mailing list. Also leave the message body intact and add your reply inline or below the body of the email you are replying to. This makes it easier to follow the thread ;) Best regards, Alain From seth@REDACTED Mon Jan 24 06:14:07 2011 From: seth@REDACTED (Seth Falcon) Date: Sun, 23 Jan 2011 21:14:07 -0800 Subject: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 In-Reply-To: References: <4D36ADAC.3070509@tail-f.com> <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> <4D370678.60203@tail-f.com> Message-ID: Hi, On Fri, Jan 21, 2011 at 12:48 AM, Ingela Andin wrote: > Well PEM-files that only BEGIN PUBLIC KEY and not BEGIN RSA PUBLIC > KEY are using another ASN1 spec and can contain both RSA and DSA > keys. It seems that it is the ASN1-spec SubjectPublicKeyInfo from > PKIXExplicit88.asn1 also part of public_key. Thanks, that was a very useful hint :-) I can now read RSA public key PEM files as generated by openssl. Here's the recipe (assuming stock public_key module): read_rsa_public_key(Key) -> Bin = erlang:iolist_to_binary(public_key_lines(re:split(Key, "\n"), [])), Spki = public_key:der_decode('SubjectPublicKeyInfo', base64:mime_decode(Bin)), {_, _, {0, KeyDer}} = Spki, public_key:der_decode('RSAPublicKey', KeyDer). public_key_lines([<<"-----BEGIN PUBLIC KEY-----">>|Rest], Acc) -> public_key_lines(Rest, Acc); public_key_lines([<<"-----END PUBLIC KEY-----">>|Rest], Acc) -> lists:reverse(Acc); public_key_lines([Line|Rest], Acc) -> public_key_lines(Rest, [Line|Acc]). I have a patch to the public_key module that makes this easier by introducing three new functions: public_key_type/1, pem_decode_public_key/1, and der_decode_public_key/1. public_key_type takes either a SubjectPublicKeyInfo record (as returned by der_decode/2 when given type 'SubjectPublicKeyInfo') or the algorithm id tuple that is a part of the 'SubjectPublicKeyInfo' record. It is a wrapper for pubkey_cert_records:supportedPublicKeyAlgorithms/1. pem_decode_public_key takes a PEM binary and in the case that pem_decode(PEM) returns 'SubjectPublicKeyInfo', it determines the public key type and does der_decode so that the return value is the public key in a form usable by functions like public_key:decrypt_public/2. If the PEM does not contain 'SubjectPublicKeyInfo', then it is passed through unchanged. I think this would be useful for Joakim's use case. der_decode_public_key behaves similarly, but is given the DER encoded version of the public key to start with. Here, as I understand it, the problem is a bit stickier because you can't know what type it is. So it tries each of ['SubjectPublicKeyInfo', 'RSAPublicKey', 'DSAPublicKey'] and returns similar to pem_decode_public_key. I've put the work-in-progress on a branch here: https://github.com/seth/otp/tree/sf/rsa_pub_key I'm not sure if this is going in the right direction, so wrote this up (sorry for the length) to get some feedback. I'm willing to make adjustments and spend time polishing a patch if there is some agreement on general direction. Best, + seth -- Seth Falcon | @sfalcon | http://userprimary.net/ From bengt.kleberg@REDACTED Mon Jan 24 08:40:27 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 24 Jan 2011 08:40:27 +0100 Subject: [erlang-questions] record updating with several functions In-Reply-To: <85166252-9190-4D09-8CA1-B24C0DC263BE@gmail.com> References: <201101191846160683077@its3.ch> <781D5239-FFA3-497F-AA2D-04F197498F82@gmail.com> <201101211024021303579@its3.ch> <201101220000570583005@its3.ch> <201101231305284154898@its3.ch> <85166252-9190-4D09-8CA1-B24C0DC263BE@gmail.com> Message-ID: <1295854827.5191.8.camel@seasc1137> Greeting, Just a comment about how to reply. If you add comments inline it quickly gets difficult to follow who wrote what. Also, sometimes bits and pieces of the original post gets cut away (by mistake or on purpose) which might change the meaning of somebody else's post. bengt On Sun, 2011-01-23 at 22:54 +0100, Alain O'Dea wrote: > On 2011-01-23, at 8:35, "info" wrote: > > > Hi Alain, > > > > According to your explanation, I modified my example: > > > > -record (field,{ch1,ch2}). > > testrecord()-> > > Record = functionA(#field{}), > > io:format("M: ch1 is ~s ch2 is ~s~n",[Record#field.ch1,Record#field.ch2]). > > functionA(RecA)-> > > R1=RecA#field{ch1 = "hello"}, > > io:format("FA: ch1 is ~s ch2 is ~s~n",[R1#field.ch1,R1#field.ch2]), > > RecB = functionB(R1), > > RecB. > > functionB(RecB)-> > > R2=RecB#field{ch2 = "boy"}, > > io:format("FB: ch1 is ~s ch2 is ~s~n",[R2#field.ch1,R2#field.ch2]), > > R2. > > > > Now it's ok but I find the solution "hard" (I don't find the good term in english ...). I suppose that a good erlang programmer will not solve the problem like this ? > > Regards, > > > > J-Ph. Constantin > > ITS3 Gen?ve > > www.its3.ch > > Hi: > > The english term you are looking for is "awkward" ;) > > It certainly is awkward, but not far from what you'd do in production Erlang code. > > Normally, an experienced Erlang/OTP programmer would use OTP generic behaviours and callback modules. This is a pretty advanced topic, but there is a very good book by Eric Merritt, Martin Logan and Richard Carlsson called Erlang and OTP in Action http://manning.com/logan. > > When you use an OTP behaviour callback module each callback function is handed a request, a sender PID and the State. The State is like this fields record you are working with. It is an opaque data structure that your callback module owns the internal details of. Each of your callback functions returns a reply with a new version of the State (or just the old state if nothing needs to change). > > If you want to get serious about code structure in Erlang, then OTP is the idiomatic approach. > > However, I seriously suggest that you become comfortable with the syntax and the semantics of sequential Erlang before getting into OTP. Learn You Some Erlang at http://learnyousomeerlang.com/ is a great tutorial supported by Erlang Solutions. It has a sense of humour and it presents the topics in an easy to follow format. It flows well and it will really answer most of your questions about the language. > > Please work through Learn You Some Erlang and post any questions or problems you have back to erlang-questions so that one of the other members of the community or I can assist you :) > > When replying to Erlang-questions remember to Reply All if you want it on the mailing list. Also leave the message body intact and add your reply inline or below the body of the email you are replying to. This makes it easier to follow the thread ;) > > Best regards, > Alain From jocke@REDACTED Mon Jan 24 09:23:24 2011 From: jocke@REDACTED (Joakim G.) Date: Mon, 24 Jan 2011 09:23:24 +0100 Subject: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 In-Reply-To: <4D39723F.3060507@tail-f.com> References: <4D36ADAC.3070509@tail-f.com> <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> <4D370678.60203@tail-f.com> <4D38B9C5.9030002@tail-f.com> <4D39723F.3060507@tail-f.com> Message-ID: <4D3D36FC.7000103@tail-f.com> On 2011-01-21 12:47, Joakim G. wrote: > With this patch public_key:pem_decode/1 returns a RSAPublicKey > record as defined in public_key.hrl (instead of just []). > > I haven't used the output yet though. :-) > > I will experiment with public_key:verify/4 over the weekend. To verify: my naive fix did what is was supposed to do, i.e. this works with the patch applied: [PemEntry] = public_key:pem_decode(<<"-----BEGIN RSA PUBLIC KEY-----\nMIGJAoGBAKhhZ8RrKaA0RWhjXJlSpTHd7/h6Luww9qeyUkEqBQyiCooZZGCYNle5\nC9TfRVJDUuxzigvXuTh0tlba+9AyIe2hxAztjs43pSKy5VnnJlSjMBVDay8XmNjh\nlUxid/ZV2hDkKlwHDCnZjdZpeXqdtGbZ7o3N7wOBzJSRQ7kqzQHXAgMBAAE=\n-----END RSA PUBLIC KEY-----">>), RSAPublicKey = public_key:pem_entry_decode(PemEntry), I could then call: == verify(PlainText, Signature, #'RSAPublicKey'{modulus = Mod, publicExponent = Exp}) -> PlainText == crypto:rsa_public_decrypt(Signature, [crypto:mpint(Exp), crypto:mpint(Mod)], rsa_pkcs1_padding). == to verify a signature. AFAICS public_key:verify/4 is broken. I'm happpy now. The work can continue. :-) /Jocke > Cheers > /Jocke > > On 2011-01-21 00:03, Seth Falcon wrote: >> 2011/1/20 Joakim G. : >>> I updated pubkey_pem.html a tiny little bit and it seems to >>> work: >>> >>> jocke@REDACTED:~/installs/otp_src_R14B01/lib/public_key/src$ diff -bwt >>> pubkey_pem.erl.orig pubkey_pem.erl >>> 157a158,159 >>>> join_entry([<<"-----END RSA PUBLIC KEY-----", _/binary>>| Lines], >>> Entry) -> >>>> {lists:reverse(Entry), Lines}; >>> 212a215,216 >>>> pem_start('RSAPublicKey') -> >>>> <<"-----BEGIN RSA PUBLIC KEY-----">>; >>> 221a226,227 >>>> pem_end(<<"-----BEGIN RSA PUBLIC KEY-----">>) -> >>>> <<"-----END RSA PUBLIC KEY-----">>; >>> 232a239,240 >>>> asn1_type(<<"-----BEGIN RSA PUBLIC KEY-----">>) -> >>>> 'RSAPublicKey'; >>> >>> Thanks >> >> Our mails crossed. When you say "this seems to work", were you able >> to actually use the public key for verification/decryption? >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From jocke@REDACTED Mon Jan 24 09:26:25 2011 From: jocke@REDACTED (Joakim G.) Date: Mon, 24 Jan 2011 09:26:25 +0100 Subject: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 In-Reply-To: References: <4D36ADAC.3070509@tail-f.com> <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> <4D370678.60203@tail-f.com> Message-ID: <4D3D37B1.7070407@tail-f.com> Thanks. A nice addition to the public_key module. /Jocke On 2011-01-24 06:14, Seth Falcon wrote: > Hi, > > On Fri, Jan 21, 2011 at 12:48 AM, Ingela Andin wrote: >> Well PEM-files that only BEGIN PUBLIC KEY and not BEGIN RSA PUBLIC >> KEY are using another ASN1 spec and can contain both RSA and DSA >> keys. It seems that it is the ASN1-spec SubjectPublicKeyInfo from >> PKIXExplicit88.asn1 also part of public_key. > > Thanks, that was a very useful hint :-) > > I can now read RSA public key PEM files as generated by openssl. > Here's the recipe (assuming stock public_key module): > > read_rsa_public_key(Key) -> > Bin = erlang:iolist_to_binary(public_key_lines(re:split(Key, > "\n"), [])), > Spki = public_key:der_decode('SubjectPublicKeyInfo', > base64:mime_decode(Bin)), > {_, _, {0, KeyDer}} = Spki, > public_key:der_decode('RSAPublicKey', KeyDer). > > public_key_lines([<<"-----BEGIN PUBLIC KEY-----">>|Rest], Acc) -> > public_key_lines(Rest, Acc); > public_key_lines([<<"-----END PUBLIC KEY-----">>|Rest], Acc) -> > lists:reverse(Acc); > public_key_lines([Line|Rest], Acc) -> > public_key_lines(Rest, [Line|Acc]). From ingela@REDACTED Mon Jan 24 10:48:00 2011 From: ingela@REDACTED (Ingela Andin) Date: Mon, 24 Jan 2011 10:48:00 +0100 Subject: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 In-Reply-To: <4D3D36FC.7000103@tail-f.com> References: <4D36ADAC.3070509@tail-f.com> <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> <4D370678.60203@tail-f.com> <4D38B9C5.9030002@tail-f.com> <4D39723F.3060507@tail-f.com> <4D3D36FC.7000103@tail-f.com> Message-ID: Hi! public_key:verify/4 is not broken, it is just that verifies signatures created by the algorithm implemented by public_key:sign/3 and not the ones created by the algorithm implemented by public_key:encrypt_private/2 as yours seems to be. You should try public_key:decrypt_public/3 instead. Regards Ingela Erlang/OTP team - Ericsson AB 2011/1/24 Joakim G. : > On 2011-01-21 12:47, Joakim G. wrote: >> With this patch public_key:pem_decode/1 returns a RSAPublicKey >> record as defined in public_key.hrl (instead of just []). >> >> I haven't used the output yet though. :-) >> >> I will experiment with public_key:verify/4 over the weekend. > > To verify: my naive fix did what is was supposed to do, i.e. this > works with the patch applied: > > ? ?[PemEntry] = public_key:pem_decode(<<"-----BEGIN RSA PUBLIC > KEY-----\nMIGJAoGBAKhhZ8RrKaA0RWhjXJlSpTHd7/h6Luww9qeyUkEqBQyiCooZZGCYNle5\nC9TfRVJDUuxzigvXuTh0tlba+9AyIe2hxAztjs43pSKy5VnnJlSjMBVDay8XmNjh\nlUxid/ZV2hDkKlwHDCnZjdZpeXqdtGbZ7o3N7wOBzJSRQ7kqzQHXAgMBAAE=\n-----END > RSA PUBLIC KEY-----">>), > ? ?RSAPublicKey = public_key:pem_entry_decode(PemEntry), > > I could then call: > > == > > verify(PlainText, Signature, > ? ? ? #'RSAPublicKey'{modulus = Mod, publicExponent = Exp}) -> > ? ?PlainText == > ? ? ? ?crypto:rsa_public_decrypt(Signature, > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?[crypto:mpint(Exp), crypto:mpint(Mod)], > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?rsa_pkcs1_padding). > > == > > to verify a signature. > > AFAICS public_key:verify/4 is broken. > > I'm happpy now. The work can continue. :-) > > /Jocke > >> Cheers >> /Jocke >> >> On 2011-01-21 00:03, Seth Falcon wrote: >>> 2011/1/20 Joakim G. : >>>> I updated pubkey_pem.html a tiny little bit and it seems to >>>> work: >>>> >>>> jocke@REDACTED:~/installs/otp_src_R14B01/lib/public_key/src$ diff -bwt >>>> pubkey_pem.erl.orig pubkey_pem.erl >>>> 157a158,159 >>>>> join_entry([<<"-----END RSA PUBLIC KEY-----", _/binary>>| Lines], >>>> Entry) -> >>>>> ? ? {lists:reverse(Entry), Lines}; >>>> 212a215,216 >>>>> pem_start('RSAPublicKey') -> >>>>> ? ? <<"-----BEGIN RSA PUBLIC KEY-----">>; >>>> 221a226,227 >>>>> pem_end(<<"-----BEGIN RSA PUBLIC KEY-----">>) -> >>>>> ? ? <<"-----END RSA PUBLIC KEY-----">>; >>>> 232a239,240 >>>>> asn1_type(<<"-----BEGIN RSA PUBLIC KEY-----">>) -> >>>>> ? ? 'RSAPublicKey'; >>>> >>>> Thanks >>> >>> Our mails crossed. ?When you say "this seems to work", were you able >>> to actually use the public key for verification/decryption? >>> >>> ________________________________________________________________ >>> erlang-questions (at) erlang.org mailing list. >>> See http://www.erlang.org/faq.html >>> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >>> >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From ingela.andin@REDACTED Mon Jan 24 11:51:40 2011 From: ingela.andin@REDACTED (Ingela Andin) Date: Mon, 24 Jan 2011 11:51:40 +0100 Subject: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 In-Reply-To: References: <4D36ADAC.3070509@tail-f.com> <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> <4D370678.60203@tail-f.com> Message-ID: Hi Seth! Thank you very much for working on this. I have looked at your branch and my thoughts are the following. I think that the decoding of 'SubjectPublicKeyInfo', from pem_decode_public_key, should rather be handled in the function pem_entry_decode. As I think the normal thing is that you want the actual key and not a 'SubjectPublicKeyInfo', that would mean just a little more decoding is done by pem_entry_decode. (You can match on the 'SubjectPublicKeyInfo' in the function head to create a clause for this.) It will not then be a 100 % symmetric with pem_entry_encode but I think that is ok, it could be mentioned in the documentation. Another reason that this is a good idea is that it will not then be sensitive to that the public_key is the only entry in the file. PEM files may contain several entries which is not handled by your function. When it comes to the function der_decode_publick_key I am not sure it is needed as I think it is uncommon that you end up with a "DER-blob" that you do not know how it is encoded. Regards Ingela Erlang/OTP team - Ericssson AB 2011/1/24 Seth Falcon : > Hi, > > On Fri, Jan 21, 2011 at 12:48 AM, Ingela Andin wrote: >> Well PEM-files that only BEGIN PUBLIC KEY and not BEGIN RSA PUBLIC >> KEY are using another ASN1 spec and can contain both RSA and DSA >> keys. It seems that it is the ASN1-spec SubjectPublicKeyInfo from >> PKIXExplicit88.asn1 also part of public_key. > > Thanks, that was a very useful hint :-) > > I can now read RSA public key PEM files as generated by openssl. > Here's the recipe (assuming stock public_key module): > > ? ?read_rsa_public_key(Key) -> > ? ? ? ?Bin = erlang:iolist_to_binary(public_key_lines(re:split(Key, > "\n"), [])), > ? ? ? ?Spki = public_key:der_decode('SubjectPublicKeyInfo', > base64:mime_decode(Bin)), > ? ? ? ?{_, _, {0, KeyDer}} = Spki, > ? ? ? ?public_key:der_decode('RSAPublicKey', KeyDer). > > ? ?public_key_lines([<<"-----BEGIN PUBLIC KEY-----">>|Rest], Acc) -> > ? ? ? ?public_key_lines(Rest, Acc); > ? ?public_key_lines([<<"-----END PUBLIC KEY-----">>|Rest], Acc) -> > ? ? ? ?lists:reverse(Acc); > ? ?public_key_lines([Line|Rest], Acc) -> > ? ? ? ?public_key_lines(Rest, [Line|Acc]). > > I have a patch to the public_key module that makes this easier by > introducing three new functions: public_key_type/1, > pem_decode_public_key/1, and der_decode_public_key/1. > > public_key_type takes either a SubjectPublicKeyInfo record (as > returned by der_decode/2 when given type 'SubjectPublicKeyInfo') or > the algorithm id tuple that is a part of the 'SubjectPublicKeyInfo' > record. ?It is a wrapper for > pubkey_cert_records:supportedPublicKeyAlgorithms/1. > > > pem_decode_public_key > > ? ?takes a PEM binary and in the case that pem_decode(PEM) returns > ? ?'SubjectPublicKeyInfo', it determines the public key type and does > ? ?der_decode so that the return value is the public key in a form > ? ?usable by functions like public_key:decrypt_public/2. ?If the PEM > ? ?does not contain 'SubjectPublicKeyInfo', then it is passed through > ? ?unchanged. ?I think this would be useful for Joakim's use case. > > der_decode_public_key > > ? ?behaves similarly, but is given the DER encoded version of the > ? ?public key to start with. ?Here, as I understand it, the problem > ? ?is a bit stickier because you can't know what type it is. ?So it > ? ?tries each of ['SubjectPublicKeyInfo', 'RSAPublicKey', > ? ?'DSAPublicKey'] and returns similar to pem_decode_public_key. > > I've put the work-in-progress on a branch here: > > ? ?https://github.com/seth/otp/tree/sf/rsa_pub_key > > I'm not sure if this is going in the right direction, so wrote this up > (sorry for the length) to get some feedback. ?I'm willing to make > adjustments and spend time polishing a patch if there is some > agreement on general direction. > > Best, > > + seth > > > -- > Seth Falcon | @sfalcon | http://userprimary.net/ > From kenji.rikitake@REDACTED Mon Jan 24 13:14:54 2011 From: kenji.rikitake@REDACTED (Kenji Rikitake) Date: Mon, 24 Jan 2011 21:14:54 +0900 Subject: ACM SIGPLAN Erlang Workshop 2011 Call For Papers Message-ID: <20110124121454.GA80316@k2r.org> CALL FOR PAPERS =============== Tenth ACM SIGPLAN Erlang Workshop --------------------------------- National Institute of Informatics (NII), Tokyo Japan Friday, September 23, 2011 A satellite event of the 16th ACM SIGPLAN International Conference on Functional Programming (ICFP) Erlang is a concurrent, distributed functional programming language aimed at systems with requirements on massive concurrency, soft real time response, fault tolerance, and high availability. It has been available as open source for over 10 years, creating a community that actively contributes to its already existing rich set of libraries and applications. Originally created for telecom applications, its usage has spread to other domains including e-commerce, banking, databases, and computer telephony and messaging. Erlang programs are today among the largest applications written in any functional programming language. These applications offer new opportunities to evaluate functional programming and functional programming methods on a very large scale and suggest new problems for the research community to solve. This workshop will bring together the open source, academic, and industrial programming communities of Erlang. It will enable participants to familiarize themselves with recent developments on new techniques and tools tailored to Erlang, novel applications, draw lessons from users' experiences and identify research problems and common areas relevant to the practice of Erlang and functional programming. We invite three sorts of submissions. 1. Technical papers describing language extensions, critical discussions of the status quo, formal semantics of language constructs, program analysis and transformation, virtual machine extensions and compilation techniques, implementations and interfaces of Erlang in/with other languages, and new tools (profilers, tracers, debuggers, testing frameworks, etc.). The maximum length for technical papers is restricted to 12 pages. 2. Practice and application papers describing uses of Erlang in the "real-world", Erlang libraries for specific tasks, experiences from using Erlang in specific application domains, reusable programming idioms and elegant new ways of using Erlang to approach or solve a particular problem. The maximum length for the practice and application papers is restricted to 6 pages. Papers in this category may be allocated less time for their talk and instead be given the opportunity for the poster presentations during the workshop. 3. Poster presentations describing topics related to the workshop goals. Each of them includes max 2 pages of the abstract and summary. Presentations in this category will be given an hour of shared simultaneous demonstration time. Workshop Chair -------------- * Kenji Rikitake, Kyoto University, Japan Program Chair ------------- * Erik Stenman, Klarna AB, Sweden Program Committee ----------------- (Note: the Workshop and Program Chairs are also committee members) * Clara Benac-Earle, Universidad Politecnica de Madrid (UPM), Spain * Johan Bevemyr, Tail-f Systems, Sweden * Scott Lystig Fritchie, Basho Technologies, USA * John Hughes, Quviq, Sweden * Takeru Inoue, NTT Network Innovation Laboratories, Japan * Kenneth Lundin, Ericsson, Sweden * Simon Thompson, University of Kent, UK * Robert Virding, Erlang Solutions, UK * Phil Wadler, University of Edinburgh, UK Important Dates --------------- * Submission deadline: Friday, June 3, 2011 * Author notification: Friday, June 17, 2011 * Final submission for the publisher: Friday, July 13, 2011 * Workshop date: Friday, September 23, 2011 Instructions to authors ----------------------- Papers must be submitted online via EasyChair (via the "Erlang2011" event). Submitted papers should be in portable document format (PDF), formatted using the ACM SIGPLAN style guidelines. Each submission must adhere to SIGPLAN's republication policy. Violation risks summary rejection of the offending submission. Accepted papers will be published by the ACM and will appear in the ACM Digital Library. Paper submissions will be considered for poster submission in the case that they are not accepted as full papers. Venue & Registration Details ---------------------------- * For registration, please see the ICFP 2011 web site at: Related Links ------------- * ICFP 2011 web site: * Past ACM SIGPLAN Erlang workshops: * Open Source Erlang: * EasyChair submission site: * Author Information for SIGPLAN Conferences: * Access map to NII: From zeno490@REDACTED Mon Jan 24 16:01:40 2011 From: zeno490@REDACTED (Nicholas Frechette) Date: Mon, 24 Jan 2011 10:01:40 -0500 Subject: [erlang-questions] Re: mnesia memory usage on 32 and 64 bits machines In-Reply-To: References: <4D1DC27A.80507@tiscali.it> <20110101163402.GA4251@talktalkplc.com> <4D39576F.1080704@tiscali.it> Message-ID: The 32 bit vs 64 bit dilemma is pretty old now. You can see 64bit taking generally more memory pretty much everywhere. I remember testing ubuntu 32 bit vs 64 bit and the 64 bit version used a good 1.5-1.7 times more memory on a vanilla install (1year ago). But as mentioned in the thread before, this can be mitigated to some extent if you manage the types properly at runtime, incurring the extra cost only on pointers and the extra alignment/padding required by them. Since my server is a fairly old amd 64bit with only 1.5gb ram, the extra memory I save from running in 32bit helps quite a bit. With 2gb+ ram, the line gets blurry and it isn't really usefull to stick with 32bit. Obviously, with 4gb+ you need 64 bit... Your choice should depend on your hardware and on the applications you run on it. Simple as that, profile and see. Depending on your data layout and what you do with it, you might or might not see a noticeable performance difference (by that I mean +- 5%). I believe, generally, 64bit to be a bit slower due to the extra memory cost generating more cache misses. Nicholas On Fri, Jan 21, 2011 at 7:06 AM, Hynek Vychodil wrote: > If I remember right There was announced nice optimization of Erlang > process heap using 32 "pointers" on 64-bit architecture on SFO Erlang > Factory last year and I believe it is included in R14B version. Anyway > I'm not sure and didn't verify. > > On Fri, Jan 21, 2011 at 10:52 AM, franz wrote: > > I admit I was tempted by the 32-bits architecture. As I'm developing a > > mnesia-centric distributed application I thought that using 32-bits > machine > > would have reduced the ram consumption and the access speed. So I > developed > > some tests to be run on two identical servers, one running a 32-bits > kernel > > and erlang, the other 64-bits. > > And everything went fine, until I bumped into the "maximum heap size for > > 32-bit machines", which I was unaware of: when a mnesia table exceeds 3GB > > erlang crashes saying < > memory (of type "heap")>>. > > In conclusion, for anyone wondering, I think I'm going 64. > > Regards, > > franz > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > > > -- > --Hynek (Pichi) Vychodil > > Analyze your data in minutes. Share your insights instantly. Thrill > your boss. Be a data hero! > Try GoodData now for free: www.gooddata.com > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From seth@REDACTED Mon Jan 24 17:12:45 2011 From: seth@REDACTED (Seth Falcon) Date: Mon, 24 Jan 2011 08:12:45 -0800 Subject: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 In-Reply-To: References: <4D36ADAC.3070509@tail-f.com> <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> <4D370678.60203@tail-f.com> Message-ID: On Mon, Jan 24, 2011 at 2:51 AM, Ingela Andin wrote: > I think that the decoding of ?'SubjectPublicKeyInfo', ?from > pem_decode_public_key, should rather be handled in the > function pem_entry_decode. As I think the normal thing is that you > want the actual key and not a 'SubjectPublicKeyInfo', that would > mean just a little more decoding is done by pem_entry_decode. (You can > match on the 'SubjectPublicKeyInfo' in the function head to > create a clause for this.) Sure. I agree that the common case is that you want the public key in a form that can be used for encrypt/decrypt as easily as possible. I'll try reworking things so that this happens inside pem_entry_decode -- and then remove the pem_decode_public_key function. Can you point me at docs on how to run the tests for the public_key module? > It will not then be a 100 % symmetric with pem_entry_encode but I > think ?that is ok, it could be mentioned in the documentation. > Another reason that this is a good idea is that it will not then be > sensitive to that the public_key is the only entry in the file. > PEM files may contain several entries which is not handled by your > function. Do you think it should be possible to reproduce the SubjectPublicKeyInfo-style PEM file given an 'RSAPublicKey' record type? I think this has some value, aside from symmetry, because it is the default output of openssl. I admit that I haven't yet worked through the encoding cases in any detail, so perhaps I'm not clear on how this is supposed to work. > When it comes to the function der_decode_publick_key I am not sure it > is needed as I think it > is uncommon that you end up with a "DER-blob" that you do not know how > it is encoded. ok, will remove. + seth -- Seth Falcon | @sfalcon | http://userprimary.net/ From paolo.negri@REDACTED Mon Jan 24 18:43:22 2011 From: paolo.negri@REDACTED (Paolo Negri) Date: Mon, 24 Jan 2011 18:43:22 +0100 Subject: pushing a dict in the constant pool Message-ID: Dear list, I would like to know if there's a way to get a dict to be stored in the constant pool For a list I would do get_useful_list() -> [my, useful, ubiquitous, list] . But if I would like to have a similar function to get a dict (with up to few hundreds entries), how would it look like? My reason to look into this is in order to be able to efficiently distribute constant and immutable configuration data to thousands of processes and provide constant time lookup. Thanks for your help. Paolo From n.oxyde@REDACTED Mon Jan 24 19:01:21 2011 From: n.oxyde@REDACTED (nox) Date: Mon, 24 Jan 2011 19:01:21 +0100 Subject: [erlang-questions] pushing a dict in the constant pool In-Reply-To: References: Message-ID: You may want to use ct_expand: http://dukesoferl.blogspot.com/2009/08/metaprogramming-with-ctexpand.html Le 24 janv. 2011 ? 18:43, Paolo Negri a ?crit : > Dear list, > > I would like to know if there's a way to get a dict to be stored in > the constant pool > > For a list I would do > > get_useful_list() -> > [my, useful, ubiquitous, list] > . > > But if I would like to have a similar function to get a dict (with up > to few hundreds entries), how would it look like? > My reason to look into this is in order to be able to efficiently > distribute constant and immutable configuration data to thousands of > processes and provide constant time lookup. > > Thanks for your help. > > Paolo > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From egil@REDACTED Mon Jan 24 19:08:40 2011 From: egil@REDACTED (=?ISO-8859-1?Q?Bj=F6rn-Egil_Dahlberg?=) Date: Mon, 24 Jan 2011 19:08:40 +0100 Subject: [erlang-questions] pushing a dict in the constant pool In-Reply-To: References: Message-ID: <4D3DC028.7060206@erix.ericsson.se> Hi, Generate a module on the fly, compile and load it. Something like, new(Module, Terms) -> Forms = [{attribute,1,module,Module}, {attribute,2,export,[{data,0}]}, {function,3,data,0,[{clause,4,[],[], [erl_parse:abstract(Terms)]}]}], {ok, Module, Code} = compile:forms(Forms, []), {module,Module} = code:load_binary(Module, Module, Code), Module. Use, new(my_module, SomeDict). my_module:data() -> SomeDict. // Bj?rn-Egil On 2011-01-24 18:43, Paolo Negri wrote: > Dear list, > > I would like to know if there's a way to get a dict to be stored in > the constant pool > > For a list I would do > > get_useful_list() -> > [my, useful, ubiquitous, list] > . > > But if I would like to have a similar function to get a dict (with up > to few hundreds entries), how would it look like? > My reason to look into this is in order to be able to efficiently > distribute constant and immutable configuration data to thousands of > processes and provide constant time lookup. > > Thanks for your help. > > Paolo > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From anthonym@REDACTED Mon Jan 24 19:10:50 2011 From: anthonym@REDACTED (Anthony Molinaro) Date: Mon, 24 Jan 2011 10:10:50 -0800 Subject: [erlang-questions] pushing a dict in the constant pool In-Reply-To: References: Message-ID: <20110124181050.GB1708@alumni.caltech.edu> You could use ct_expand in the parse_trans package https://github.com/esl/parse_trans and have something like get_my_dict () -> ct_expand:term ( dict:from_list ([{foo, bar}, {baz, bop}]) ). Then your dictionary gets expanded at compile time. This is assuming you can define the dictionary at compile time. -Anthony On Mon, Jan 24, 2011 at 06:43:22PM +0100, Paolo Negri wrote: > Dear list, > > I would like to know if there's a way to get a dict to be stored in > the constant pool > > For a list I would do > > get_useful_list() -> > [my, useful, ubiquitous, list] > . > > But if I would like to have a similar function to get a dict (with up > to few hundreds entries), how would it look like? > My reason to look into this is in order to be able to efficiently > distribute constant and immutable configuration data to thousands of > processes and provide constant time lookup. > > Thanks for your help. > > Paolo > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- ------------------------------------------------------------------------ Anthony Molinaro From ingela.andin@REDACTED Mon Jan 24 22:59:44 2011 From: ingela.andin@REDACTED (Ingela Andin) Date: Mon, 24 Jan 2011 22:59:44 +0100 Subject: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 In-Reply-To: References: <4D36ADAC.3070509@tail-f.com> <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> <4D370678.60203@tail-f.com> Message-ID: Hi Seth, 2011/1/24 Seth Falcon : > On Mon, Jan 24, 2011 at 2:51 AM, Ingela Andin wrote: >> I think that the decoding of ?'SubjectPublicKeyInfo', ?from >> pem_decode_public_key, should rather be handled in the >> function pem_entry_decode. As I think the normal thing is that you >> want the actual key and not a 'SubjectPublicKeyInfo', that would >> mean just a little more decoding is done by pem_entry_decode. (You can >> match on the 'SubjectPublicKeyInfo' in the function head to >> create a clause for this.) > Sure. ?I agree that the common case is that you want the public key in > a form that can be used for encrypt/decrypt as easily as possible. > I'll try reworking things so that this happens inside pem_entry_decode > -- and then remove the pem_decode_public_key function. > > Can you point me at docs on how to run the tests for the public_key > module? https://github.com/erlang/otp/wiki/Running-tests I think this should still work, but we are in the process of migrating from one testfrontend to another. So maybe Lucas can point you eleswhere if needed. >> It will not then be a 100 % symmetric with pem_entry_encode but I >> think ?that is ok, it could be mentioned in the documentation. >> Another reason that this is a good idea is that it will not then be >> sensitive to that the public_key is the only entry in the file. >> PEM files may contain several entries which is not handled by your >> function. > > Do you think it should be possible to reproduce the > SubjectPublicKeyInfo-style PEM file given an 'RSAPublicKey' record > type? ?I think this has some value, aside from symmetry, because it is > the default output of openssl. > > I admit that I haven't yet worked through the encoding cases in any > detail, so perhaps I'm not clear on how this is supposed to work. > I thought some more about this and I feel the right way to go is have several clauses in pem_entry_encode for the asn1-type 'SubjectPublicKeyInfo' some thing along the lines pem_entry_decode( 'SubjectPublicKeyInfo', #'RSAPublicKey'{}) -> From ingela@REDACTED Mon Jan 24 23:05:19 2011 From: ingela@REDACTED (Ingela Andin) Date: Mon, 24 Jan 2011 23:05:19 +0100 Subject: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 In-Reply-To: References: <4D36ADAC.3070509@tail-f.com> <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> <4D370678.60203@tail-f.com> Message-ID: Ah, sorry accidently hit send before finished ... [...] > I thought some more about this and I feel the right way to go is have > several clauses in pem_entry_encode for the asn1-type ?'SubjectPublicKeyInfo' > some thing along the lines > > pem_entry_decode(, ?#'RSAPublicKey'{} = Key) -> SubjectPublicKeyInfo = create_subject_public_key_info_from_rsa_public_key(Key), pem_entry_decode( 'SubjectPublicKeyInfo', Key). Regards Ingela Erlang/OTP team - Ericsson AB From seth@REDACTED Mon Jan 24 23:42:48 2011 From: seth@REDACTED (Seth Falcon) Date: Mon, 24 Jan 2011 14:42:48 -0800 Subject: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 In-Reply-To: References: <4D36ADAC.3070509@tail-f.com> <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> <4D370678.60203@tail-f.com> Message-ID: On Mon, Jan 24, 2011 at 2:05 PM, Ingela Andin wrote: > Ah, sorry accidently hit send before finished ... > > [...] > >> I thought some more about this and I feel the right way to go is have >> several clauses in pem_entry_encode for the asn1-type ?'SubjectPublicKeyInfo' >> some thing along the lines >> >> pem_entry_decode(, ?#'RSAPublicKey'{} = Key) -> > ? ? ? ?SubjectPublicKeyInfo = > create_subject_public_key_info_from_rsa_public_key(Key), > ? ? ? ?pem_entry_decode( 'SubjectPublicKeyInfo', Key). > If you meant pem_entry_encode in the lines above, then I think I follow you. Otherwise, I'm confused about whether we are talking about encode or decode :-) From kaiduanx@REDACTED Tue Jan 25 00:54:04 2011 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Mon, 24 Jan 2011 18:54:04 -0500 Subject: Distributed publish/subscribe system Message-ID: Hi, Any one has experience to build distributed publish/subscribe system? The system has the following requirements, 1. Subscriber subscribes to the interested topic 2. Publisher publishes event related to the topic 3. System delivers the event to all the users subscribed to this topic. If subscriber is connected, the event is delivered immediately. If the subscriber is not connected, system should persist the event. When the user next time becomes online, system delivers the event to the user. Can you share the experience? Best regards, /Kaiduan From jack@REDACTED Tue Jan 25 01:14:14 2011 From: jack@REDACTED (Jack Moffitt) Date: Mon, 24 Jan 2011 17:14:14 -0700 Subject: [erlang-questions] Distributed publish/subscribe system In-Reply-To: References: Message-ID: > 1. Subscriber subscribes to the interested topic > 2. Publisher publishes event related to the topic > 3. System delivers the event to all the users subscribed to this > topic. If subscriber is connected, the event is delivered immediately. > ? ?If the subscriber is not connected, system should persist the > event. When the user next time becomes online, system delivers the > event to the user. You get most (if not all of this) from either XMPP (via ejabberd's mod_pubsub) or rabbitmq. I've used both for various things and was quite happy. jack. From tony.arcieri@REDACTED Tue Jan 25 01:26:48 2011 From: tony.arcieri@REDACTED (Tony Arcieri) Date: Mon, 24 Jan 2011 17:26:48 -0700 Subject: [erlang-questions] Distributed publish/subscribe system In-Reply-To: References: Message-ID: On Mon, Jan 24, 2011 at 5:14 PM, Jack Moffitt wrote: > You get most (if not all of this) from either XMPP (via ejabberd's > mod_pubsub) or rabbitmq. I've used both for various things and was > quite happy. Note that if you're looking for a simpler solution than either XMPP or RabbitMQ, Redis Cluster will eventually support distributed pubsub whenever it's released. -- Tony Arcieri Medioh! Kudelski From toby@REDACTED Tue Jan 25 03:25:39 2011 From: toby@REDACTED (Toby Thain) Date: Tue, 25 Jan 2011 00:25:39 -0200 Subject: [erlang-questions] Distributed publish/subscribe system In-Reply-To: References: Message-ID: <4D3E34A3.4010305@telegraphics.com.au> On 11-01-24 9:54 PM, Kaiduan Xie wrote: > Hi, > > Any one has experience to build distributed publish/subscribe system? > The system has the following requirements, > > 1. Subscriber subscribes to the interested topic > 2. Publisher publishes event related to the topic > 3. System delivers the event to all the users subscribed to this > topic. If subscriber is connected, the event is delivered immediately. > If the subscriber is not connected, system should persist the > event. When the user next time becomes online, system delivers the > event to the user. > > Can you share the experience? > I have done this with single publisher and single subscriber using pubsub module and ejabberd. The client was a Java application using Spark for the xmpp library. I would certainly prefer to use the existing xmpp design and ejabberd implementation over designing, building and testing my own. --Toby > Best regards, > > /Kaiduan > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From nraychaudhuri@REDACTED Tue Jan 25 04:23:41 2011 From: nraychaudhuri@REDACTED (Nilanjan Raychaudhuri) Date: Mon, 24 Jan 2011 22:23:41 -0500 Subject: Looking for an erlang speaker near Columbus area Message-ID: <1C907F9C-3E34-4CD8-A16C-85A7167C0A68@gmail.com> Hi, I run a meet up group called Columbus polyglot programmers meet up http://www.meetup.com/The-Columbus-Polyglot-Programmers-Meetup-Group/ We mainly get together and talk about various programming languages and tools. There is a very strong interest in functional programming languages. I am very interested to have someone talk about erlang and OTP. If anyone is interested to share his or her experience I would be happy to setup a meet up for you. I am really looking forward to hear back. Thanks Nilanjan From erlang@REDACTED Tue Jan 25 09:56:32 2011 From: erlang@REDACTED (Joe Armstrong) Date: Tue, 25 Jan 2011 09:56:32 +0100 Subject: [erlang-questions] Distributed publish/subscribe system In-Reply-To: References: Message-ID: Standard question: How big? - how scalable - what latency - fault tolerance requirements do you have. Distributed can mean 2 nodes - or 10^9 There might be ten channels or 10^12 How fault tolerant? - 2 replicas, 100 replicas. What the latency? - do you want the message to be delivered with 10 ms or 100 years What's the security model - who owns the data - is there centralized security or weak per-node security policies. The point is your requirements need to say things like this otherwise they are not much use. The architectures for these different cases will be *completely different* /Joe On Tue, Jan 25, 2011 at 12:54 AM, Kaiduan Xie wrote: > Hi, > > Any one has experience to build distributed publish/subscribe system? > The system has the following requirements, > > 1. Subscriber subscribes to the interested topic > 2. Publisher publishes event related to the topic > 3. System delivers the event to all the users subscribed to this > topic. If subscriber is connected, the event is delivered immediately. > If the subscriber is not connected, system should persist the > event. When the user next time becomes online, system delivers the > event to the user. > > Can you share the experience? > > Best regards, > > /Kaiduan > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From ingela@REDACTED Tue Jan 25 10:16:05 2011 From: ingela@REDACTED (Ingela Andin) Date: Tue, 25 Jan 2011 10:16:05 +0100 Subject: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 In-Reply-To: References: <4D36ADAC.3070509@tail-f.com> <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> <4D370678.60203@tail-f.com> Message-ID: Hi! Yes, encode it should be, late at night and copy paste does not work between my virtualbox and windows screen... maybe I should start running my browser in virtualbox too! ;) Regards Ingela - Erlang OTP team -Ericsson AB 2011/1/24 Seth Falcon : > On Mon, Jan 24, 2011 at 2:05 PM, Ingela Andin wrote: >> Ah, sorry accidently hit send before finished ... >> >> [...] >> >>> I thought some more about this and I feel the right way to go is have >>> several clauses in pem_entry_encode for the asn1-type ?'SubjectPublicKeyInfo' >>> some thing along the lines >>> >>> pem_entry_decode(, ?#'RSAPublicKey'{} = Key) -> >> ? ? ? ?SubjectPublicKeyInfo = >> create_subject_public_key_info_from_rsa_public_key(Key), >> ? ? ? ?pem_entry_decode( 'SubjectPublicKeyInfo', Key). >> > > If you meant pem_entry_encode in the lines above, then I think I > follow you. ?Otherwise, I'm confused about whether we are talking > about encode or decode :-) > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From thomasl_erlang@REDACTED Tue Jan 25 12:11:51 2011 From: thomasl_erlang@REDACTED (Thomas Lindgren) Date: Tue, 25 Jan 2011 03:11:51 -0800 (PST) Subject: [erlang-questions] Distributed publish/subscribe system In-Reply-To: References: Message-ID: <760093.50497.qm@web111413.mail.gq1.yahoo.com> ----- Original Message ---- > From: Kaiduan Xie ... > Any one has experience to build distributed publish/subscribe system? > The system has the following requirements, > > 1. Subscriber subscribes to the interested topic > 2. Publisher publishes event related to the topic > 3. System delivers the event to all the users subscribed to this > topic. If subscriber is connected, the event is delivered immediately. > If the subscriber is not connected, system should persist the > event. When the user next time becomes online, system delivers the > event to the user. > > Can you share the experience? I haven't tried it myself, but RabbitMQ sounds like it might be suitable: http://www.rabbitmq.com/tutorial-three-python.html I think you can have persistent queues too. Best regards, Thomas From jameschurchman@REDACTED Tue Jan 25 15:41:57 2011 From: jameschurchman@REDACTED (James Churchman) Date: Tue, 25 Jan 2011 14:41:57 +0000 Subject: [erlang-questions] Looking for an erlang speaker near Columbus area In-Reply-To: <1C907F9C-3E34-4CD8-A16C-85A7167C0A68@gmail.com> References: <1C907F9C-3E34-4CD8-A16C-85A7167C0A68@gmail.com> Message-ID: <1D0BA1B2-C799-4BF5-922B-91A9782F7750@gmail.com> Just an idea, but if it turns out to be impossible ( erlang is not exactly everywhere, yet :-) ) you could always get an erlang expert or two to give a 20 min talk over Skype video chat etc.. for your meetup I have seen this done at the odd conference or two, including a video stream back so that the audience can ask questions and participate. It's not perfect but worked better than i expected James On 25 Jan 2011, at 03:23, Nilanjan Raychaudhuri wrote: > Hi, > > I run a meet up group called Columbus polyglot programmers meet up > > http://www.meetup.com/The-Columbus-Polyglot-Programmers-Meetup-Group/ > > We mainly get together and talk about various programming languages and tools. There is a very strong interest in functional programming languages. I am very interested to have someone talk about erlang and OTP. If anyone is interested to share his or her experience I would be happy to setup a meet up for you. > > I am really looking forward to hear back. > > Thanks > Nilanjan > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From kaiduanx@REDACTED Tue Jan 25 16:00:56 2011 From: kaiduanx@REDACTED (Kaiduan Xie) Date: Tue, 25 Jan 2011 10:00:56 -0500 Subject: [erlang-questions] Distributed publish/subscribe system In-Reply-To: References: Message-ID: Thanks Jack, Tony, Toby, Joe and Thomas for the helpful comments and resources. Joe pointed out a lot of excellent points. The requirements are listed below, 1) The system should be able to support tens to hundreds of millions users spreading around the world, so the system will consist hundred of nodes located in different physical locations. 2) Any user can subscribe to the interested topic, but only topic owner can publish message to the group. We have no limit on the number of subscribers in each group. It can be huge, for example, the fans of Rolling Stone around the world. 3) The number of topics/groups in the system is unlimited. 4) User is not always connected to the system, not always connected to the same node in the system, and the message delivery should be guaranteed. 5) As to the latency, it should be in the range of 1 minute if subscriber is connected. I am doing research on this topic now, the other requirements are not sorted out yet. Thanks, /Kaiduan On Tue, Jan 25, 2011 at 3:56 AM, Joe Armstrong wrote: > Standard question: > > How big? - how scalable - what latency - fault tolerance requirements do you > have. > > Distributed can mean 2 nodes - or 10^9 > There might be ten channels or 10^12 > > How fault tolerant? - 2 replicas, 100 replicas. > > What the latency? - do you want the message to be delivered with 10 ms or > 100 years > > What's the security model - who owns the data - is there centralized > security or weak > per-node security policies. > > The point is your requirements need to say things like this otherwise they > are not much use. > > The architectures for these different cases will be *completely different* > > /Joe > > On Tue, Jan 25, 2011 at 12:54 AM, Kaiduan Xie wrote: >> >> Hi, >> >> Any one has experience to build distributed publish/subscribe system? >> The system has the following requirements, >> >> 1. Subscriber subscribes to the interested topic >> 2. Publisher publishes event related to the topic >> 3. System delivers the event to all the users subscribed to this >> topic. If subscriber is connected, the event is delivered immediately. >> ? ?If the subscriber is not connected, system should persist the >> event. When the user next time becomes online, system delivers the >> event to the user. >> >> Can you share the experience? >> >> Best regards, >> >> /Kaiduan >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > From olivier.boudeville@REDACTED Tue Jan 25 16:32:51 2011 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Tue, 25 Jan 2011 16:32:51 +0100 Subject: The display/1Gotcha Message-ID: Hi, Wouldn't it be nice if erlang:display/1 did not silently shadow any user-defined display/1? I have been bitten more than once by this. Currently, defining in a module display/1 and calling it from another function directly, without mentioning a module (ex: 'f(Term) -> display(Term).') will call erlang:display/1instead of calling the lesser-astonishing ?MODULE:display/1. As erlang:display/1is not so very well-known and display is a rather common verb, maybe this would deserve forcing its call with an explicit module prefix (i.e. 'erlang:display(Term)'; if it is possible not to auto-import it) or at least issue a warning like "warning: local function display/1 will be shadowed by erlang:display/1"? Best regards, Olivier Boudeville. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. From garazdawi@REDACTED Tue Jan 25 18:04:56 2011 From: garazdawi@REDACTED (Lukas Larsson) Date: Tue, 25 Jan 2011 18:04:56 +0100 Subject: [erlang-questions] The display/1Gotcha In-Reply-To: References: Message-ID: Hi, In the version of Erlang that I'm running right now (R14B02), display/1 without the erlang module prefix causes a compilation error. Also as far as I can tell by the documentation (if it is correct) it never has been auto imported, http://www.erlang.org/documentation/doc-5.0.1/lib/kernel-2.6.1/doc/html/erlang.html#erlang:display%1. What version of Erlang are you compiling with? are you doing any special preprocessing of the source code? Lukas On Tue, Jan 25, 2011 at 4:32 PM, Olivier BOUDEVILLE < olivier.boudeville@REDACTED> wrote: > Hi, > > Wouldn't it be nice if erlang:display/1 did not silently shadow any > user-defined display/1? I have been bitten more than once by this. > > Currently, defining in a module display/1 and calling it from another > function directly, without mentioning a module (ex: 'f(Term) -> > display(Term).') will call erlang:display/1instead of calling the > lesser-astonishing ?MODULE:display/1. > > As erlang:display/1is not so very well-known and display is a rather > common verb, maybe this would deserve forcing its call with an explicit > module prefix (i.e. 'erlang:display(Term)'; if it is possible not to > auto-import it) or at least issue a warning like "warning: local function > display/1 will be shadowed by erlang:display/1"? > > Best regards, > > Olivier Boudeville. > --------------------------- > Olivier Boudeville > > EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France > D?partement SINETICS, groupe ASICS (I2A), bureau B-226 > Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 > 65 27 13 > > > > Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont > ?tablis ? l'intention exclusive des destinataires et les informations qui y > figurent sont strictement confidentielles. Toute utilisation de ce Message > non conforme ? sa destination, toute diffusion ou toute publication totale > ou partielle, est interdite sauf autorisation expresse. > > Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de > le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou > partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de > votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace > sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir > imm?diatement l'exp?diteur par retour du message. > > Il est impossible de garantir que les communications par messagerie > ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute > erreur ou virus. > ____________________________________________________ > > This message and any attachments (the 'Message') are intended solely for > the addressees. The information contained in this Message is confidential. > Any use of information contained in this Message not in accord with its > purpose, any dissemination or disclosure, either whole or partial, is > prohibited except formal approval. > > If you are not the addressee, you may not copy, forward, disclose or use > any part of it. If you have received this message in error, please delete it > and all copies from your system and notify the sender immediately by return > message. > > E-mail communication cannot be guaranteed to be timely secure, error or > virus-free. > From johanmunk@REDACTED Tue Jan 25 21:43:26 2011 From: johanmunk@REDACTED (johan munk) Date: Tue, 25 Jan 2011 21:43:26 +0100 Subject: Fwd: erl crash on long PATH In-Reply-To: References: Message-ID: Had no luck sending to erlang-bugs. Trying erlang-questions ... ---------- Forwarded message ---------- From: johan munk Date: Tue, Jan 25, 2011 at 12:28 PM Subject: erl crash on long PATH To: erlang-bugs@REDACTED I just noticed the following crash of 'erl' when my PATH bash var gets longer: [johan@REDACTED:~] uname -a SunOS skuffe 5.11 snv_97 i86pc i386 i86pc [johan@REDACTED:~] erl Erlang R13B03 (erts-5.7.4) [source] [rq:1] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.4 ?(abort with ^G) 1> BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded ? ? ? (v)ersion (k)ill (D)b-tables (d)istribution [johan@REDACTED:~] export PATH=$PATH:$PATH [johan@REDACTED:~] export PATH=$PATH:$PATH [johan@REDACTED:~] export PATH=$PATH:$PATH [johan@REDACTED:~] export PATH=$PATH:$PATH [johan@REDACTED:~] erl Segmentation Fault (core dumped) [johan@REDACTED:~] From michael.santos@REDACTED Tue Jan 25 22:35:04 2011 From: michael.santos@REDACTED (Michael Santos) Date: Tue, 25 Jan 2011 16:35:04 -0500 Subject: [erlang-questions] Fwd: erl crash on long PATH In-Reply-To: References: Message-ID: <20110125213504.GA24105@ecn.lan> On Tue, Jan 25, 2011 at 09:43:26PM +0100, johan munk wrote: > I just noticed the following crash of 'erl' when my PATH bash var gets longer: > > [johan@REDACTED:~] uname -a > SunOS skuffe 5.11 snv_97 i86pc i386 i86pc > [johan@REDACTED:~] erl > Erlang R13B03 (erts-5.7.4) [source] [rq:1] [async-threads:0] [hipe] > [kernel-poll:false] > > Eshell V5.7.4 ?(abort with ^G) > 1> > BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded > ? ? ? (v)ersion (k)ill (D)b-tables (d)istribution > [johan@REDACTED:~] export PATH=$PATH:$PATH > [johan@REDACTED:~] export PATH=$PATH:$PATH > [johan@REDACTED:~] export PATH=$PATH:$PATH > [johan@REDACTED:~] export PATH=$PATH:$PATH > [johan@REDACTED:~] erl > Segmentation Fault (core dumped) > [johan@REDACTED:~] Should be fixed as of R14B01: $ export PATH=$PATH:$(perl -e 'print "x"x10000') $ erl Erlang R14B02 (erts-5.8.3) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.3 (abort with ^G) 1> halt(). From johanmunk@REDACTED Tue Jan 25 22:50:31 2011 From: johanmunk@REDACTED (Johan Munk) Date: Tue, 25 Jan 2011 22:50:31 +0100 Subject: [erlang-questions] Fwd: erl crash on long PATH In-Reply-To: <20110125213504.GA24105@ecn.lan> References: <20110125213504.GA24105@ecn.lan> Message-ID: <4D3F45A7.1090107@gmail.com> Ah - perfect, thanks. I've not yet had the time to upgrade to latest version. On 2011-01-25 22:35, Michael Santos wrote: > On Tue, Jan 25, 2011 at 09:43:26PM +0100, johan munk wrote: > >> I just noticed the following crash of 'erl' when my PATH bash var gets longer: >> >> [johan@REDACTED:~] uname -a >> SunOS skuffe 5.11 snv_97 i86pc i386 i86pc >> [johan@REDACTED:~] erl >> Erlang R13B03 (erts-5.7.4) [source] [rq:1] [async-threads:0] [hipe] >> [kernel-poll:false] >> >> Eshell V5.7.4 ?(abort with ^G) >> 1> >> BREAK: (a)bort (c)ontinue (p)roc info (i)nfo (l)oaded >> ? ? ? (v)ersion (k)ill (D)b-tables (d)istribution >> [johan@REDACTED:~] export PATH=$PATH:$PATH >> [johan@REDACTED:~] export PATH=$PATH:$PATH >> [johan@REDACTED:~] export PATH=$PATH:$PATH >> [johan@REDACTED:~] export PATH=$PATH:$PATH >> [johan@REDACTED:~] erl >> Segmentation Fault (core dumped) >> [johan@REDACTED:~] > Should be fixed as of R14B01: > > $ export PATH=$PATH:$(perl -e 'print "x"x10000') > $ erl > Erlang R14B02 (erts-5.8.3) [source] [smp:2:2] [rq:2] [async-threads:0] > [hipe] [kernel-poll:false] > > Eshell V5.8.3 (abort with ^G) > 1> halt(). > From alex.arnon@REDACTED Wed Jan 26 08:23:29 2011 From: alex.arnon@REDACTED (Alex Arnon) Date: Wed, 26 Jan 2011 09:23:29 +0200 Subject: [erlang-questions] Distributed publish/subscribe system In-Reply-To: References: Message-ID: >> As to the latency, it should be in the range of 1 minute if subscriber is connected. Well then, you don't need a software system at all - humans could do this for you :) On Tue, Jan 25, 2011 at 5:00 PM, Kaiduan Xie wrote: > Thanks Jack, Tony, Toby, Joe and Thomas for the helpful comments and > resources. Joe pointed out a lot of excellent points. The requirements > are listed below, > > 1) The system should be able to support tens to hundreds of millions > users spreading around the world, so the system will consist hundred > of nodes located in different physical locations. > > 2) Any user can subscribe to the interested topic, but only topic > owner can publish message to the group. We have no limit on the number > of subscribers in each group. It can be huge, for example, the fans of > Rolling Stone around the world. > > 3) The number of topics/groups in the system is unlimited. > > 4) User is not always connected to the system, not always connected to > the same node in the system, and the message delivery should be > guaranteed. > > 5) As to the latency, it should be in the range of 1 minute if > subscriber is connected. > > I am doing research on this topic now, the other requirements are not > sorted out yet. > > Thanks, > > /Kaiduan > > > On Tue, Jan 25, 2011 at 3:56 AM, Joe Armstrong wrote: > > Standard question: > > > > How big? - how scalable - what latency - fault tolerance requirements do > you > > have. > > > > Distributed can mean 2 nodes - or 10^9 > > There might be ten channels or 10^12 > > > > How fault tolerant? - 2 replicas, 100 replicas. > > > > What the latency? - do you want the message to be delivered with 10 ms or > > 100 years > > > > What's the security model - who owns the data - is there centralized > > security or weak > > per-node security policies. > > > > The point is your requirements need to say things like this otherwise > they > > are not much use. > > > > The architectures for these different cases will be *completely > different* > > > > /Joe > > > > On Tue, Jan 25, 2011 at 12:54 AM, Kaiduan Xie > wrote: > >> > >> Hi, > >> > >> Any one has experience to build distributed publish/subscribe system? > >> The system has the following requirements, > >> > >> 1. Subscriber subscribes to the interested topic > >> 2. Publisher publishes event related to the topic > >> 3. System delivers the event to all the users subscribed to this > >> topic. If subscriber is connected, the event is delivered immediately. > >> If the subscriber is not connected, system should persist the > >> event. When the user next time becomes online, system delivers the > >> event to the user. > >> > >> Can you share the experience? > >> > >> Best regards, > >> > >> /Kaiduan > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From jay@REDACTED Wed Jan 26 07:07:42 2011 From: jay@REDACTED (jay@REDACTED) Date: Tue, 25 Jan 2011 22:07:42 -0800 Subject: [erlang-questions] Distributed publish/subscribe system Message-ID: Kaiduan is looking for a PubSub solution: > 1) ... support tens to hundreds of millions > users spreading around the world ... > 2) ... no limit on the number of subscribers in each group ... > 3) The number of topics/groups in the system is unlimited. > 4) User is not always connected to the system... > 5) ... latency ... of 1 minute if subscriber is connected. > I am doing research on this topic now, the other requirements > are not sorted out yet. CouchDB is a possibility, and you could test it out using a service such as http://www.cloudant.com/ which uses quorum reads and writes to ensure data accuracy. The requirements I left in above are the ones that Couch would purport to satisfy, although on the scale of users you are talking about you would definitely need services distributed to different continents. jay From olivier.boudeville@REDACTED Wed Jan 26 11:14:51 2011 From: olivier.boudeville@REDACTED (Olivier BOUDEVILLE) Date: Wed, 26 Jan 2011 11:14:51 +0100 Subject: [erlang-questions] The display/1Gotcha In-Reply-To: Message-ID: Hi Kenneth, Hi Lukas, Thanks for your answers. At first I was not able to reproduce the issue on a minimal example, but, as you both hinted, original code was correct and should not have led to a local function being shadowed by a "built-in": apparently the culprit is the smart_exception parse transform. At least this is what is suggested by: """ -module(minimal_test). -export([ run/0, display/1 ]). run() -> display( "Smart exceptions do not seem to work properly." ). display( T ) -> io:format( "My function was called, really?!?~n" ), throw( {called_with,T} ). """ Then a vanilla compilation works as expected, whereas the one with smart_exception behaves wrongly: """ > erl Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:8:8] [rq:8] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.2 (abort with ^G) 1> c(minimal_test). {ok,minimal_test} 2> minimal_test:run(). My function was called, really?!? ** exception throw: {called_with,"Smart exceptions do not seem to work properly."} in function minimal_test:display/1 3> c(minimal_test,[{parse_transform, smart_exceptions}]). {ok,minimal_test} 4> minimal_test:run(). "Smart exceptions do not seem to work properly." true """ It looks like that, unless in my code I use ?MODULE:display/1, erlang:display/1 will be called instead if smart exceptions are used. (by the way, I am using the devel version of smart_exception that I downloaded a month ago, since there were an issue with the stable version as well, as described in http://erlang.2086793.n4.nabble.com/Problems-With-Smart-Exceptions-td3095862.html) So, unless I am mistaken, I suppose smart exceptions should better be disabled? This is a bit of a pity as it is a quite nice feature, useful to avoid additional intellectual puzzles. Best regards, Olivier Boudeville. --------------------------- Olivier Boudeville EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France D?partement SINETICS, groupe ASICS (I2A), bureau B-226 Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 65 27 13 kenneth.lundin@REDACTED 25/01/2011 17:09 A olivier.boudeville@REDACTED cc Objet Re: [erlang-questions] The display/1Gotcha Hi, I don't understand what you mean please provide a complete example. If we for example have this module -module(dtest). -export([f/0]). f() -> display("String"). display(String) -> {local_display_called_with,String}. Then in both R13B04 and in R14A (when we introduced a change in semantics so that autoimported BIFs no longer override local functions) the local function named display will be called when you call dtest:f(). Also note that erlang:display/1 is not an autoimported BIF. You must always call it giving the modulename erlang. /Kenneth Erlang/OTP Ericsson On Tue, Jan 25, 2011 at 4:32 PM, Olivier BOUDEVILLE wrote: > Hi, > > Wouldn't it be nice if erlang:display/1 did not silently shadow any > user-defined display/1? I have been bitten more than once by this. > > Currently, defining in a module display/1 and calling it from another > function directly, without mentioning a module (ex: 'f(Term) -> > display(Term).') will call erlang:display/1instead of calling the > lesser-astonishing ?MODULE:display/1. > > As erlang:display/1is not so very well-known and display is a rather > common verb, maybe this would deserve forcing its call with an explicit > module prefix (i.e. 'erlang:display(Term)'; if it is possible not to > auto-import it) or at least issue a warning like "warning: local function > display/1 will be shadowed by erlang:display/1"? > > Best regards, > > Olivier Boudeville. > --------------------------- > Olivier Boudeville > > EDF R&D : 1, avenue du G?n?ral de Gaulle, 92140 Clamart, France > D?partement SINETICS, groupe ASICS (I2A), bureau B-226 > Office : +33 1 47 65 59 58 / Mobile : +33 6 16 83 37 22 / Fax : +33 1 47 > 65 27 13 > > > > Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. > > Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. > > Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. > ____________________________________________________ > > This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. > > If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. > > E-mail communication cannot be guaranteed to be timely secure, error or virus-free. > Ce message et toutes les pi?ces jointes (ci-apr?s le 'Message') sont ?tablis ? l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme ? sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'?tes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez re?u ce Message par erreur, merci de le supprimer de votre syst?me, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions ?galement d'en avertir imm?diatement l'exp?diteur par retour du message. Il est impossible de garantir que les communications par messagerie ?lectronique arrivent en temps utile, sont s?curis?es ou d?nu?es de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. From B.Candler@REDACTED Wed Jan 26 11:30:30 2011 From: B.Candler@REDACTED (Brian Candler) Date: Wed, 26 Jan 2011 10:30:30 +0000 Subject: [erlang-questions] Distributed publish/subscribe system In-Reply-To: References: Message-ID: <20110126103030.GB9274@talktalkplc.com> On Tue, Jan 25, 2011 at 10:00:56AM -0500, Kaiduan Xie wrote: > 1) The system should be able to support tens to hundreds of millions > users spreading around the world, so the system will consist hundred > of nodes located in different physical locations. > > 2) Any user can subscribe to the interested topic, but only topic > owner can publish message to the group. We have no limit on the number > of subscribers in each group. It can be huge, for example, the fans of > Rolling Stone around the world. > > 3) The number of topics/groups in the system is unlimited. > > 4) User is not always connected to the system, not always connected to > the same node in the system, and the message delivery should be > guaranteed. > > 5) As to the latency, it should be in the range of 1 minute if > subscriber is connected. Sounds like NNTP to me! From erlang@REDACTED Wed Jan 26 14:00:51 2011 From: erlang@REDACTED (Joe Armstrong) Date: Wed, 26 Jan 2011 14:00:51 +0100 Subject: [erlang-questions] Distributed publish/subscribe system In-Reply-To: References: Message-ID: On Tue, Jan 25, 2011 at 4:00 PM, Kaiduan Xie wrote: > Thanks Jack, Tony, Toby, Joe and Thomas for the helpful comments and > resources. Joe pointed out a lot of excellent points. The requirements > are listed below, > > 1) The system should be able to support tens to hundreds of millions > users spreading around the world, so the system will consist hundred > of nodes located in different physical locations. > Who own the nodes? are they all owned by the same organisation? Who is legally responsible for the system? > > 2) Any user can subscribe to the interested topic, but only topic > owner can publish message to the group. We have no limit on the number > of subscribers in each group. It can be huge, for example, the fans of > Rolling Stone around the world. > > How do your stop spam - bogus groups etc? > 3) The number of topics/groups in the system is unlimited. > How do you stop name squatting - ie flooding the system with stupid channel names? > > 4) User is not always connected to the system, not always connected to > the same node in the system, and the message delivery should be > guaranteed. > Uuug - impossible - it's the Byzantine generals problem. How do you know they got a message? - send an ack - how do you know if they got the ack?- ack the ack - how do you know if they acked the ack, ack the acked ack ... /Joe > 5) As to the latency, it should be in the range of 1 minute if > subscriber is connected. > > I am doing research on this topic now, the other requirements are not > sorted out yet. > > Thanks, > > /Kaiduan > > > On Tue, Jan 25, 2011 at 3:56 AM, Joe Armstrong wrote: > > Standard question: > > > > How big? - how scalable - what latency - fault tolerance requirements do > you > > have. > > > > Distributed can mean 2 nodes - or 10^9 > > There might be ten channels or 10^12 > > > > How fault tolerant? - 2 replicas, 100 replicas. > > > > What the latency? - do you want the message to be delivered with 10 ms or > > 100 years > > > > What's the security model - who owns the data - is there centralized > > security or weak > > per-node security policies. > > > > The point is your requirements need to say things like this otherwise > they > > are not much use. > > > > The architectures for these different cases will be *completely > different* > > > > /Joe > > > > On Tue, Jan 25, 2011 at 12:54 AM, Kaiduan Xie > wrote: > >> > >> Hi, > >> > >> Any one has experience to build distributed publish/subscribe system? > >> The system has the following requirements, > >> > >> 1. Subscriber subscribes to the interested topic > >> 2. Publisher publishes event related to the topic > >> 3. System delivers the event to all the users subscribed to this > >> topic. If subscriber is connected, the event is delivered immediately. > >> If the subscriber is not connected, system should persist the > >> event. When the user next time becomes online, system delivers the > >> event to the user. > >> > >> Can you share the experience? > >> > >> Best regards, > >> > >> /Kaiduan > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > > > > > From igorrs@REDACTED Wed Jan 26 17:45:11 2011 From: igorrs@REDACTED (Igor Ribeiro Sucupira) Date: Wed, 26 Jan 2011 14:45:11 -0200 Subject: [erlang-questions] Mnesia Fragmented Tables, del_table_copy In-Reply-To: <1295443907.20811.23.camel@texhp> References: <1295443907.20811.23.camel@texhp> Message-ID: I haven't looked at it in detail, but I know it doesn't make much sense to delete the second fragment (leaving no other copies of it) in a table that has 4 fragments. It would break your fragmented table (writing keys would not be possible in some cases). Best regards. Igor. On Wed, Jan 19, 2011 at 11:31 AM, Tessaro Alexej wrote: > Hello, > > I have started two communicating mnesia nodes and created one fragmented > table with fragments stored in RAM only (no other replicas) and schema > stored on disk on both nodes: > > node1 : constResourceData_frag3 (ram), constResourceData_frag4 (ram), > schema (disk) > > node2 : constResourceData (ram), constResourceData_frag2 (ram), schema > (disk) > > Now, from node1, I tried to call > mnesia:del_table_copy(constResourceData_frag2, node2) > > but it fails saying: > > node1> Mnesia(node1): Last replica deleted in table > constResourceData_frag2 > node1>{aborted,{no_exists,constResourceData_frag2,frag_properties, > ? ? ? ? ? ? ? ? ? ?frag_hash}} > node1> Mnesia(node1): Transaction {tid,75,<0.371.0>} calling > #Fun with [] failed: > ?{aborted,{no_exists,constResourceData_frag2,frag_properties,frag_hash}} > > Looking at the mnesia_schema.erl source code, I found that the failure > seems to happen when it tries to delete the "whole_table" in > make_delete_table/2 and it runs the following check: > > %% Check if it is a base table > nesia_frag:lookup_frag_hash(Tab), > > Why is my attempt of removing the fragment failing? > > My erlang version is R12B. > > Thank you > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- "The secret of joy in work is contained in one word - excellence. To know how to do something well is to enjoy it." - Pearl S. Buck. From spence_m@REDACTED Wed Jan 26 23:49:33 2011 From: spence_m@REDACTED (Malcolm Spence) Date: Wed, 26 Jan 2011 16:49:33 -0600 Subject: [erlang-questions] Distributed publish/subscribe system Message-ID: <033001cbbdab$4c7f2790$e57d76b0$@ociweb.com> Interesting question. Just this last week we published a Middleware Newsbrief on combining Erlang with CORBA and DDS. http://mnb.ociweb.com/mnb/MiddlewareNewsBrief-201101 DDS is the OMG spec for a real time data distribution service (versus a queue). We developed an open source implementation for large scale messaging systems. It has the kind of QoS capabilities you need. Can have federated servers to help with FT and scaling etc. Has a Java interface and can be used as a JMS provider in a JBoss/ESB/SOA stck. regards Malcolm Malcolm D. Spence Director of Business Development OCI "Use our reach to exceed your grasp." Voting member of the OMG. (Phone 1- 314-579-0066 ext. 206 or FAX -0065) www.theaceorb.com www.ociweb.com From jameschurchman@REDACTED Thu Jan 27 02:41:58 2011 From: jameschurchman@REDACTED (James Churchman) Date: Thu, 27 Jan 2011 01:41:58 +0000 Subject: [erlang-questions] Distributed publish/subscribe system In-Reply-To: <033001cbbdab$4c7f2790$e57d76b0$@ociweb.com> References: <033001cbbdab$4c7f2790$e57d76b0$@ociweb.com> Message-ID: http://mnb.ociweb.com/mnb/MiddlewareNewsBrief-201101.html ** On 26 Jan 2011, at 22:49, Malcolm Spence wrote: > Interesting question. Just this last week we published a Middleware > Newsbrief on combining Erlang with CORBA and DDS. > > > > http://mnb.ociweb.com/mnb/MiddlewareNewsBrief-201101 > > > > DDS is the OMG spec for a real time data distribution service (versus a > queue). We developed an open source implementation for large scale messaging > systems. > > > > It has the kind of QoS capabilities you need. Can have federated servers to > help with FT and scaling etc. Has a Java interface and can be used as a JMS > provider in a JBoss/ESB/SOA stck. > > > > regards Malcolm > > > > > > Malcolm D. Spence > Director of Business Development > OCI "Use our reach to exceed your grasp." > Voting member of the OMG. > (Phone 1- 314-579-0066 ext. 206 or FAX -0065) > www.theaceorb.com www.ociweb.com > From yrashk@REDACTED Thu Jan 27 05:56:08 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Wed, 26 Jan 2011 20:56:08 -0800 (PST) Subject: [ANN] Agner, a rebar-friendly Erlang package index Message-ID: <90676862-9a7e-4ab3-8382-2dcb0cdd017f@d23g2000prj.googlegroups.com> Hi everybody, This is an announcement for an early release of Agner, a rebar- friendly Erlang package index, inspired by Clojars & Homebrew. You can find out more about it at https://github.com/agner/agner/blob/master/README.md Any feedback will be greatly appreciated. Please don't hesitate to ask questions, I am also available at #erlang IRC channel (my nick is yrashk). Thank you and have a great day! From tramjoe.merin@REDACTED Thu Jan 27 10:59:45 2011 From: tramjoe.merin@REDACTED (Jerome Martin) Date: Thu, 27 Jan 2011 10:59:45 +0100 Subject: [erlang-questions] Distributed publish/subscribe system In-Reply-To: <033001cbbdab$4c7f2790$e57d76b0$@ociweb.com> References: <033001cbbdab$4c7f2790$e57d76b0$@ociweb.com> Message-ID: Wouldn't using this defeat the whole purpose of moving beyond the RPC model, and leverage what Erlang has to offer, as described hereand there ? To re-quote Steve Vinoski after Joe Armstrong: *"What all those years of CORBA taught me, BTW, is that RPC, for a* *number of reasons, is generally A Really Bad Idea."* Or is it just me not understanding what years of Erlang can teach us regarding good practice, smart technology choices and mistakes of the past, sometimes leading to widespread adoption of products implementing probably the worse part of otherwise interesting research (like mostly all of OMG's so-called standards) ? On Wed, Jan 26, 2011 at 11:49 PM, Malcolm Spence wrote: > Interesting question. Just this last week we published a Middleware > Newsbrief on combining Erlang with CORBA and DDS. > > > > http://mnb.ociweb.com/mnb/MiddlewareNewsBrief-201101 > > > > DDS is the OMG spec for a real time data distribution service (versus a > queue). We developed an open source implementation for large scale > messaging > systems. > > > > It has the kind of QoS capabilities you need. Can have federated servers to > help with FT and scaling etc. Has a Java interface and can be used as a JMS > provider in a JBoss/ESB/SOA stck. > > > > regards Malcolm > > > > > > Malcolm D. Spence > Director of Business Development > OCI "Use our reach to exceed your grasp." > Voting member of the OMG. > (Phone 1- 314-579-0066 ext. 206 or FAX -0065) > www.theaceorb.com www.ociweb.com > > -- J?r?me Martin From andre@REDACTED Thu Jan 27 11:06:26 2011 From: andre@REDACTED (Andre Nathan) Date: Thu, 27 Jan 2011 08:06:26 -0200 Subject: Mnesia startup synchronization Message-ID: <1296122786.3789.9.camel@andre.mz.digirati.com.br> Hello In the "Erlang and OTP in Action" book, there's a section explaining Mnesia dynamic replication, where a node copies the tables from other cluster nodes (when there are any) or starts up on its own otherwise. In this section a race condition in this scheme is mentioned: two nodes starting up simultaneously may believe that the other one was the first, resulting in a deadlock where no initial schema is created. What would be a proper way to add synchronization to mnesia startup to ensure that a single node is always the first to run? Thanks in advance Andre From hm@REDACTED Thu Jan 27 13:12:18 2011 From: hm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Thu, 27 Jan 2011 13:12:18 +0100 Subject: [erlang-questions] Mnesia startup synchronization In-Reply-To: <1296122786.3789.9.camel@andre.mz.digirati.com.br> References: <1296122786.3789.9.camel@andre.mz.digirati.com.br> Message-ID: On Thu, Jan 27, 2011 at 11:06 AM, Andre Nathan wrote: > Hello > > In the "Erlang and OTP in Action" book, there's a section explaining > Mnesia dynamic replication, where a node copies the tables from other > cluster nodes (when there are any) or starts up on its own otherwise. > > In this section a race condition in this scheme is mentioned: two nodes > starting up simultaneously may believe that the other one was the first, > resulting in a deadlock where no initial schema is created. > > What would be a proper way to add synchronization to mnesia startup to > ensure that a single node is always the first to run? You should separate installation from normal startup. The installation is just run once and startup should assume that the schema is created. /H?kan From kuleshovmail@REDACTED Thu Jan 27 14:57:15 2011 From: kuleshovmail@REDACTED (Alexander Kuleshov) Date: Thu, 27 Jan 2011 13:57:15 +0000 Subject: EPmail-0.1 released Message-ID: Hello, Today i released epmail-0.1. EPmail - it's mailing system written in Erlang. Now it has only pop3 server. But in future i planned to do smtp/imap/pop3 server system in Erlang. EPmail - https://github.com/onlyshk/epmail EPmail Wiki - https://github.com/onlyshk/epmail/wiki EPMail status - https://github.com/onlyshk/epmail/wiki/Status I'm new in erlang and i think there are many bugs and defects, but i fix it with your help :) How to install and run epmail pop3 server written in Wiki or in README file. Now you can connect to it by telnet or full value mail client. I tested with thunderbird for example. Mail storage is simple text file for every user, but in future i'll plan to bind database for users. Now pop3 server worked only for 1 domain, but i fix it in next version. I waiting for your critics, because your opinion is very important for the project and for me as a beginner Erlang programmer. Please, feel free to fork it and improve project. Thank you. From tino.breddin@REDACTED Thu Jan 27 15:17:03 2011 From: tino.breddin@REDACTED (Tino Breddin) Date: Thu, 27 Jan 2011 15:17:03 +0100 Subject: [erlang-questions] Mnesia startup synchronization In-Reply-To: References: <1296122786.3789.9.camel@andre.mz.digirati.com.br> Message-ID: And you might find the 'setup' application useful for the separation of installation and system startup. See https://github.com/esl/setup Tino 2011/1/27 H?kan Mattsson > On Thu, Jan 27, 2011 at 11:06 AM, Andre Nathan > wrote: > > Hello > > > > In the "Erlang and OTP in Action" book, there's a section explaining > > Mnesia dynamic replication, where a node copies the tables from other > > cluster nodes (when there are any) or starts up on its own otherwise. > > > > In this section a race condition in this scheme is mentioned: two nodes > > starting up simultaneously may believe that the other one was the first, > > resulting in a deadlock where no initial schema is created. > > > > What would be a proper way to add synchronization to mnesia startup to > > ensure that a single node is always the first to run? > > You should separate installation from normal startup. The installation is > just run once and startup should assume that the schema is created. > > /H?kan > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From tuxofil@REDACTED Thu Jan 27 16:02:37 2011 From: tuxofil@REDACTED (Alex Morarash) Date: Thu, 27 Jan 2011 17:02:37 +0200 Subject: file:position/2 hangs up whole enode Message-ID: Hi all! Just another way to completely hang up a whole erlang node: open compressed file for read and move file position after actual size of uncompressed data. Stable reproduce in at least from R12B5 till R14B1. Here is the example. ----CUT HERE----------------------------------------- $ echo "All your bases are belong to us" | gzip > test.gz $ ls -l test.gz -rw-r--r-- 1 boa boa 52 Jan 27 16:43 test.gz $ erl Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.2 (abort with ^G) 1> {ok, H} = file:open("test.gz", [read, compressed]). {ok,<0.34.0>} 2> file:position(H, 10). {ok,10} 3> file:position(H, 40). ....HANGING....Even Ctrl+G doesn`t work. ----CUT HERE----------------------------------------- Regards, Alex Morarash, e-mail: tuxofil at gmail.com From spence_m@REDACTED Thu Jan 27 16:19:24 2011 From: spence_m@REDACTED (Malcolm Spence) Date: Thu, 27 Jan 2011 09:19:24 -0600 Subject: [erlang-questions] Distributed publish/subscribe system In-Reply-To: References: <033001cbbdab$4c7f2790$e57d76b0$@ociweb.com> Message-ID: <008b01cbbe35$94d285c0$be779140$@ociweb.com> Thanks for the comments Jerome. We are an open source software engineering shop. We integrate disparate systems for a living. If we did not have open source we would not be able to do a lot of what we do. We try to not to argue with clients over their past technology choices. We just try to make them work. This paper is example of us sharing our thoughts with others in case they run into similar situations. Our "Middleware Newsbrief" and our more OO language and tools oriented "Tech Trends" are ways in which we try to encourage fact based discussion with example code and observations. We are not advocating one technology over another. We like to understand the problem before offering a solution. And will readily admit no solution is perfect. In fact we do like to temper people's expectations when we first engage because we are pragmatists. regards Malcolm Malcolm D. Spence Director of Business Development OCI "Use our reach to exceed your grasp." Voting member of the OMG. (Phone 1- 314-579-0066 ext. 206 or FAX -0065) www.theaceorb.com www.ociweb.com From: Jerome Martin [mailto:tramjoe.merin@REDACTED] Sent: Thursday, January 27, 2011 4:00 AM To: Malcolm Spence Cc: erlang-questions@REDACTED Subject: Re: [erlang-questions] Distributed publish/subscribe system Wouldn't using this defeat the whole purpose of moving beyond the RPC model, and leverage what Erlang has to offer, as described here and there ? To re-quote Steve Vinoski after Joe Armstrong: "What all those years of CORBA taught me, BTW, is that RPC, for a number of reasons, is generally A Really Bad Idea." Or is it just me not understanding what years of Erlang can teach us regarding good practice, smart technology choices and mistakes of the past, sometimes leading to widespread adoption of products implementing probably the worse part of otherwise interesting research (like mostly all of OMG's so-called standards) ? On Wed, Jan 26, 2011 at 11:49 PM, Malcolm Spence wrote: Interesting question. Just this last week we published a Middleware Newsbrief on combining Erlang with CORBA and DDS. http://mnb.ociweb.com/mnb/MiddlewareNewsBrief-201101 DDS is the OMG spec for a real time data distribution service (versus a queue). We developed an open source implementation for large scale messaging systems. It has the kind of QoS capabilities you need. Can have federated servers to help with FT and scaling etc. Has a Java interface and can be used as a JMS provider in a JBoss/ESB/SOA stck. regards Malcolm Malcolm D. Spence Director of Business Development OCI "Use our reach to exceed your grasp." Voting member of the OMG. (Phone 1- 314-579-0066 ext. 206 or FAX -0065) www.theaceorb.com www.ociweb.com -- J?r?me Martin From daniel.goertzen@REDACTED Thu Jan 27 16:36:21 2011 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Thu, 27 Jan 2011 09:36:21 -0600 Subject: trouble debugging app in erlide Message-ID: I am using erlide, and I cannot get breakpoints to work on my app when the app is started automatically by my boot script. If I use the console to application:stop()/start() my app, breakpoints start working normally. But I am intensely lazy and want breakpoints to work right when I launch my Debug Config. Any advice? Thanks, Dan. From rzezeski@REDACTED Thu Jan 27 16:44:35 2011 From: rzezeski@REDACTED (Ryan Zezeski) Date: Thu, 27 Jan 2011 10:44:35 -0500 Subject: [erlang-questions] [ANN] Agner, a rebar-friendly Erlang package index In-Reply-To: <90676862-9a7e-4ab3-8382-2dcb0cdd017f@d23g2000prj.googlegroups.com> References: <90676862-9a7e-4ab3-8382-2dcb0cdd017f@d23g2000prj.googlegroups.com> Message-ID: I read the name "Anger" when I first read it. Hopefully I'm in the minority here :) -Ryan On Wed, Jan 26, 2011 at 11:56 PM, Yurii Rashkovskii wrote: > Hi everybody, > > This is an announcement for an early release of Agner, a rebar- > friendly Erlang package index, inspired by Clojars & Homebrew. You can > find out more about it at > https://github.com/agner/agner/blob/master/README.md > > Any feedback will be greatly appreciated. Please don't hesitate to ask > questions, I am also available at #erlang IRC channel (my nick is > yrashk). > > Thank you and have a great day! > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From tramjoe.merin@REDACTED Thu Jan 27 16:51:51 2011 From: tramjoe.merin@REDACTED (Jerome Martin) Date: Thu, 27 Jan 2011 16:51:51 +0100 Subject: [erlang-questions] Distributed publish/subscribe system In-Reply-To: <008b01cbbe35$94d285c0$be779140$@ociweb.com> References: <033001cbbdab$4c7f2790$e57d76b0$@ociweb.com> <008b01cbbe35$94d285c0$be779140$@ociweb.com> Message-ID: On Thu, Jan 27, 2011 at 4:19 PM, Malcolm Spence wrote: > Thanks for the comments Jerome. > Thank you for taking my comment so peacefully. > We are an open source software engineering shop. We integrate disparate > systems for a living. If we did not have open source we would not be able to > do a lot of what we do. We try to not to argue with clients over their past > technology choices. We just try to make them work. > I understand your position. It is an honorable business stance. Please note that my comment was not targeting OSS vs non OSS, but more the pervasiveness in the "industry" of methodologies and techniques for mostly the wrong reasons, from a purely technical standpoint (better marketing does not make a solution better at problem-solving). I realize that there are probably a lot of different reasons for one individual to adopt or support those, and it is not my intent to judge or criticize those. However, I have to admit I have an epidermic reaction to what I feel is a regression compared to the state of art in CS, and even more so when I see advocacy for such technology choice on this mailing list. > This paper is example of us sharing our thoughts with others in case they > run into similar situations. Our "Middleware Newsbrief" and our more OO > language and tools oriented "Tech Trends" are ways in which we try to > encourage fact based discussion with example code and observations. > Nice. If only you applied just that to technology that solves more problems than it creates (and no, I won't even attempt to prove that, it is an obvious exaggeration to make my point :-) ), I would be a big fan! > We are not advocating one technology over another. We like to understand > the problem before offering a solution. And will readily admit no solution > is perfect. In fact we do like to temper people's expectations when we first > engage because we are pragmatists. > Again, this is a most honorable stance. Regards, -- J?r?me Martin From gleber.p@REDACTED Thu Jan 27 16:59:24 2011 From: gleber.p@REDACTED (Gleb Peregud) Date: Thu, 27 Jan 2011 16:59:24 +0100 Subject: [erlang-questions] [ANN] Agner, a rebar-friendly Erlang package index In-Reply-To: References: <90676862-9a7e-4ab3-8382-2dcb0cdd017f@d23g2000prj.googlegroups.com> Message-ID: Is 2 still a minority? :) On Thu, Jan 27, 2011 at 16:44, Ryan Zezeski wrote: > I read the name "Anger" when I first read it. ?Hopefully I'm in the minority > here :) > > -Ryan > > On Wed, Jan 26, 2011 at 11:56 PM, Yurii Rashkovskii wrote: > >> Hi everybody, >> >> This is an announcement for an early release of Agner, a rebar- >> friendly Erlang package index, inspired by Clojars & Homebrew. You can >> find out more about it at >> https://github.com/agner/agner/blob/master/README.md >> >> Any feedback will be greatly appreciated. Please don't hesitate to ask >> questions, I am also available at #erlang IRC channel (my nick is >> yrashk). >> >> Thank you and have a great day! >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From tramjoe.merin@REDACTED Thu Jan 27 17:04:00 2011 From: tramjoe.merin@REDACTED (Jerome Martin) Date: Thu, 27 Jan 2011 17:04:00 +0100 Subject: [erlang-questions] [ANN] Agner, a rebar-friendly Erlang package index In-Reply-To: References: <90676862-9a7e-4ab3-8382-2dcb0cdd017f@d23g2000prj.googlegroups.com> Message-ID: I thought it was just me. That make three :-) On Thu, Jan 27, 2011 at 4:59 PM, Gleb Peregud wrote: > Is 2 still a minority? :) > > On Thu, Jan 27, 2011 at 16:44, Ryan Zezeski wrote: > > I read the name "Anger" when I first read it. Hopefully I'm in the > minority > > here :) > > > > -Ryan > > > > On Wed, Jan 26, 2011 at 11:56 PM, Yurii Rashkovskii >wrote: > > > >> Hi everybody, > >> > >> This is an announcement for an early release of Agner, a rebar- > >> friendly Erlang package index, inspired by Clojars & Homebrew. You can > >> find out more about it at > >> https://github.com/agner/agner/blob/master/README.md > >> > >> Any feedback will be greatly appreciated. Please don't hesitate to ask > >> questions, I am also available at #erlang IRC channel (my nick is > >> yrashk). > >> > >> Thank you and have a great day! > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > >> > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- J?r?me Martin From vinoski@REDACTED Thu Jan 27 17:04:05 2011 From: vinoski@REDACTED (Steve Vinoski) Date: Thu, 27 Jan 2011 11:04:05 -0500 Subject: [erlang-questions] Distributed publish/subscribe system In-Reply-To: References: <033001cbbdab$4c7f2790$e57d76b0$@ociweb.com> Message-ID: On Thu, Jan 27, 2011 at 4:59 AM, Jerome Martin wrote: > Wouldn't using this defeat the whole purpose of moving beyond the RPC model, > and leverage what Erlang has to offer, as described > hereand > there ? To re-quote > Steve Vinoski after Joe Armstrong: > > *"What all those years of CORBA taught me, BTW, is that RPC, for a* > *number of reasons, is generally A Really Bad Idea."* I couldn't agree more! :-) In Malcolm's defense, though, there's still a lot of CORBA out there, a fair amount of it in embedded systems, and it's not going anywhere anytime soon. For example, to the best of my knowledge certain telecommunications interfaces required by law are based on CORBA. A dozen years ago at the height of CORBA we always predicted it would still be in use in 2020, and so far it seems we might have been fairly accurate with that prediction. Malcolm and folks like him specialize in helping folks who have no choice but to maintain such systems try to fit it together with newer technologies changing and advancing around it. I'm happy to see Malcolm and his colleagues trying to bring Erlang into such systems in order to reduce maintenance costs, enhance functionality, and improve integration capabilities -- I think we should encourage more of that. --steve From stu.bailey@REDACTED Thu Jan 27 17:15:54 2011 From: stu.bailey@REDACTED (Stu Bailey) Date: Thu, 27 Jan 2011 08:15:54 -0800 Subject: [erlang-questions] Distributed publish/subscribe system In-Reply-To: References: <033001cbbdab$4c7f2790$e57d76b0$@ociweb.com> Message-ID: Folk might want to look at IF-MAP. It's a systems oriented pub/sub system with some interesting features like stateful aggregation and search. It looks like it would be simple to write an erlang client for it. http://www.if-map.com/en.html Stu On Thu, Jan 27, 2011 at 8:04 AM, Steve Vinoski wrote: > On Thu, Jan 27, 2011 at 4:59 AM, Jerome Martin > wrote: > > Wouldn't using this defeat the whole purpose of moving beyond the RPC > model, > > and leverage what Erlang has to offer, as described > > here< > http://armstrongonsoftware.blogspot.com/2008/05/road-we-didnt-go-down.html > >and > > there ? To > re-quote > > Steve Vinoski after Joe Armstrong: > > > > *"What all those years of CORBA taught me, BTW, is that RPC, for a* > > *number of reasons, is generally A Really Bad Idea."* > > I couldn't agree more! :-) > > In Malcolm's defense, though, there's still a lot of CORBA out there, > a fair amount of it in embedded systems, and it's not going anywhere > anytime soon. For example, to the best of my knowledge certain > telecommunications interfaces required by law are based on CORBA. A > dozen years ago at the height of CORBA we always predicted it would > still be in use in 2020, and so far it seems we might have been fairly > accurate with that prediction. Malcolm and folks like him specialize > in helping folks who have no choice but to maintain such systems try > to fit it together with newer technologies changing and advancing > around it. I'm happy to see Malcolm and his colleagues trying to bring > Erlang into such systems in order to reduce maintenance costs, enhance > functionality, and improve integration capabilities -- I think we > should encourage more of that. > > --steve > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From tramjoe.merin@REDACTED Thu Jan 27 17:18:21 2011 From: tramjoe.merin@REDACTED (Jerome Martin) Date: Thu, 27 Jan 2011 17:18:21 +0100 Subject: [erlang-questions] Distributed publish/subscribe system In-Reply-To: References: <033001cbbdab$4c7f2790$e57d76b0$@ociweb.com> Message-ID: On Thu, Jan 27, 2011 at 5:04 PM, Steve Vinoski wrote: > On Thu, Jan 27, 2011 at 4:59 AM, Jerome Martin > wrote: > > Wouldn't using this defeat the whole purpose of moving beyond the RPC > model, > > and leverage what Erlang has to offer, as described > > here< > http://armstrongonsoftware.blogspot.com/2008/05/road-we-didnt-go-down.html > >and > > there ? To > re-quote > > Steve Vinoski after Joe Armstrong: > > > > *"What all those years of CORBA taught me, BTW, is that RPC, for a* > > *number of reasons, is generally A Really Bad Idea."* > > I couldn't agree more! :-) > Why am I not surprised ? :-) > In Malcolm's defense, though, there's still a lot of CORBA out there, > a fair amount of it in embedded systems, and it's not going anywhere > anytime soon. Yes, and this is why I said I understand his business stance. Someone has to maintain these systems now I suppose. However, advising to create some more is a different thing. Well, that is unless if you specialize in maintenance of said systems :-) (just a humorous remark, I am not accusing anyone of anything there) > For example, to the best of my knowledge certain > telecommunications interfaces required by law are based on CORBA. A > dozen years ago at the height of CORBA we always predicted it would > still be in use in 2020, and so far it seems we might have been fairly > accurate with that prediction. Malcolm and folks like him specialize > in helping folks who have no choice but to maintain such systems try > to fit it together with newer technologies changing and advancing > around it. I'm happy to see Malcolm and his colleagues trying to bring > Erlang into such systems in order to reduce maintenance costs, enhance > functionality, and improve integration capabilities -- I think we > should encourage more of that. > Yes, in the same sense as I am happy to see functional idioms in python: this is far from ideal, and brings only an order of magnitude less than the benefits of pure FP. But it has the merit of exposing more people to FP, at the risk or diluting it, just like smalltalk ideas have been diluted (and the best ones mostly dropped) in "modern OO". -- J?r?me Martin From Mike.French@REDACTED Thu Jan 27 17:30:28 2011 From: Mike.French@REDACTED (French, Mike) Date: Thu, 27 Jan 2011 16:30:28 -0000 Subject: [erlang-questions] Distributed publish/subscribe system Message-ID: <3F8EEA01CF53D74DB4A9EC314D82B4F3024E2B5B@wells154924> > -----Original Message----- > From: erlang-questions@REDACTED > [mailto:erlang-questions@REDACTED]On > Behalf Of Malcolm Spence > Sent: 26 January 2011 22:50 > To: erlang-questions@REDACTED > Subject: Re: [erlang-questions] Distributed publish/subscribe system > > > Interesting question. Just this last week we published a Middleware > Newsbrief on combining Erlang with CORBA and DDS. Integration with DDS is interesting. DDS is definitely 'sympathetic' to The Erlang Way. Almost the only things it has in common with CORBA are reusing IDL as the message specification format and CDR as the on-the-wire binary data encoding. DDS is loosely-coupled publish-subscribe, not RPC. DDS is decentralized peer-to-peer, as opposed to the centralized hub-and-spoke topology for CORBA, and many other pub-sub message brokers, like AMQP. DDS specifies a binary wire protocol, unlike, say, JMS. DDS is designed for low-latency and high-throughput, and can operate in soft-realtime systems, but it goes to the next step and supports quality of service (QoS) parameters for individual pub-sub channels, so you can explicitly allocate a latency budget and configure what happens when messages are late. etc. etc. A native implementation of DDS in Erlang would be fascinating. Mike Thales UK Ltd (Wells) DISCLAIMER: The information contained in this e-mail is confidential. It may also be legally privileged. It is intended only for the stated addressee(s) and access to it by any other person is unauthorised. If you are not an addressee, you must not disclose, copy, circulate or in any other way use or rely on the information contained in this e-mail. Such unauthorised use may be unlawful. We may monitor all e-mail communications through our networks. If you have received this e-mail in error, please inform us immediately on sender's telephone number above and delete it and all copies from your system. We accept no responsibility for changes to any e-mail which occur after it has been sent. Attachments to this e-mail may contain software viruses which could damage your system. We therefore recommend you virus-check all attachments before opening. Thales UK Ltd. Registered Office: 2 Dashwood Lang Road, The Bourne Business Park, Addlestone, Weybridge, Surrey KT15 2NX Registered in England No. 868273 From vladdu55@REDACTED Thu Jan 27 17:33:32 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 27 Jan 2011 17:33:32 +0100 Subject: [erlang-questions] trouble debugging app in erlide In-Reply-To: References: Message-ID: Hi! On Thu, Jan 27, 2011 at 16:36, Daniel Goertzen wrote: > I am using erlide, and I cannot get breakpoints to work on my app when the > app is started automatically by my boot script. > > If I use the console to application:stop()/start() my app, breakpoints > start > working normally. But I am intensely lazy and want breakpoints to work > right when I launch my Debug Config. > > Any advice? > I think that the problem is that the application gets started before the debugger, thus before the breakpoints are set and the code is interpreted. I don't think we can do much about that, short of adding the erlide debugger application to the boot script, but that would require large changes to our framework. A suggestion would be to create a parameterless function that starts your application and does any other initialization, and specify it as startup function in the debug configuration. regards, Vlad From watson.timothy@REDACTED Thu Jan 27 17:42:11 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Thu, 27 Jan 2011 16:42:11 +0000 Subject: [erlang-questions] [ANN] Agner, a rebar-friendly Erlang package index In-Reply-To: <90676862-9a7e-4ab3-8382-2dcb0cdd017f@d23g2000prj.googlegroups.com> References: <90676862-9a7e-4ab3-8382-2dcb0cdd017f@d23g2000prj.googlegroups.com> Message-ID: On 27 January 2011 04:56, Yurii Rashkovskii wrote: > Hi everybody, > > This is an announcement for an early release of Agner, a rebar- > friendly Erlang package index, inspired by Clojars & Homebrew. You can > find out more about it at https://github.com/agner/agner/blob/master/README.md > > Any feedback will be greatly appreciated. Please don't hesitate to ask > questions, I am also available at #erlang IRC channel (my nick is > yrashk). > > Thank you and have a great day! > This looks very similar to https://github.com/JacobVorreuter/sutro. From antonio.fernandez@REDACTED Thu Jan 27 18:10:27 2011 From: antonio.fernandez@REDACTED (=?ISO-8859-1?Q?Antonio_Fern=E1ndez?=) Date: Thu, 27 Jan 2011 18:10:27 +0100 Subject: Shell - compile multiple files with ERLC in Windows Message-ID: <4D41A703.10309@jaraxa.com> Hi all ! I?m erlang beginner programmer. The following command works on Ubuntu Shell but don?t work on Windows : $ erlc --o ebin src/*.erl I tried with ..... c:/erlang>erlc -o ebin src\*.erl c:/erlang/src/*.erl:none: I/O error Thanks in advance, Antonio -- Antonio Miguel Fern?ndez Rodr?guez Jaraxa Software, SL Administrator, Software analyst Cel. +34 607 78 86 85 www.jaraxa.com From roberto@REDACTED Thu Jan 27 18:15:19 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 27 Jan 2011 18:15:19 +0100 Subject: [erlang-questions] Shell - compile multiple files with ERLC in Windows In-Reply-To: <4D41A703.10309@jaraxa.com> References: <4D41A703.10309@jaraxa.com> Message-ID: 2011/1/27 Antonio Fern?ndez > > Hi all ! > > I?m erlang beginner programmer. > > The following command works on Ubuntu Shell but don?t work on Windows : > > $ erlc --o ebin src/*.erl > > I tried with ..... > > c:/erlang>erlc -o ebin src\*.erl > c:/erlang/src/*.erl:none: I/O error > try: FOR %%f in (src\*.erl) DO erlc -W %command% -o ebin "%%f" r. From roberto@REDACTED Thu Jan 27 18:16:17 2011 From: roberto@REDACTED (Roberto Ostinelli) Date: Thu, 27 Jan 2011 18:16:17 +0100 Subject: [erlang-questions] Shell - compile multiple files with ERLC in Windows In-Reply-To: References: <4D41A703.10309@jaraxa.com> Message-ID: > > try: > > FOR %%f in (src\*.erl) DO erlc -W %command% -o ebin "%%f" > > r. > sorry for that, obviously %command% is a paste error. this should do it: FOR %%f in (src\*.erl) DO erlc -W -o ebin "%%f" From atessaro@REDACTED Thu Jan 27 18:18:56 2011 From: atessaro@REDACTED (Tessaro Alexej) Date: Thu, 27 Jan 2011 18:18:56 +0100 Subject: [erlang-questions] Mnesia Fragmented Tables, del_table_copy In-Reply-To: References: <1295443907.20811.23.camel@texhp> Message-ID: <1296148736.2059.19.camel@texhp> Thank you for your reply. In my case the table is dynamically growing (by adding new fragments when the existing ones reach some sort of threshold). Cause the contained data have no critical relevance (but real-time reading and writing times constraints), I thought to maintain ram only copies with no replicas. So, when a node disconnects from the cluster cloud, I would expect the data it was containing to disappear from the cloud (and the requests involving those records to fail). I expected mnesia_frag system to detect not running mnesia nodes, thus not considering them when calling mnesia activity functions on the whole table. Am I missing something? On Wed, 2011-01-26 at 14:45 -0200, Igor Ribeiro Sucupira wrote: > I haven't looked at it in detail, but I know it doesn't make much > sense to delete the second fragment (leaving no other copies of it) in > a table that has 4 fragments. It would break your fragmented table > (writing keys would not be possible in some cases). > > Best regards. > Igor. > > On Wed, Jan 19, 2011 at 11:31 AM, Tessaro Alexej wrote: > > Hello, > > > > I have started two communicating mnesia nodes and created one fragmented > > table with fragments stored in RAM only (no other replicas) and schema > > stored on disk on both nodes: > > > > node1 : constResourceData_frag3 (ram), constResourceData_frag4 (ram), > > schema (disk) > > > > node2 : constResourceData (ram), constResourceData_frag2 (ram), schema > > (disk) > > > > Now, from node1, I tried to call > > mnesia:del_table_copy(constResourceData_frag2, node2) > > > > but it fails saying: > > > > node1> Mnesia(node1): Last replica deleted in table > > constResourceData_frag2 > > node1>{aborted,{no_exists,constResourceData_frag2,frag_properties, > > frag_hash}} > > node1> Mnesia(node1): Transaction {tid,75,<0.371.0>} calling > > #Fun with [] failed: > > {aborted,{no_exists,constResourceData_frag2,frag_properties,frag_hash}} > > > > Looking at the mnesia_schema.erl source code, I found that the failure > > seems to happen when it tries to delete the "whole_table" in > > make_delete_table/2 and it runs the following check: > > > > %% Check if it is a base table > > nesia_frag:lookup_frag_hash(Tab), > > > > Why is my attempt of removing the fragment failing? > > > > My erlang version is R12B. > > > > Thank you > > > > > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > > -- Tessaro Alexej, Volunia Italia DISCLAIMER: This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you. From yrashk@REDACTED Thu Jan 27 18:38:13 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Thu, 27 Jan 2011 09:38:13 -0800 Subject: [erlang-questions] [ANN] Agner, a rebar-friendly Erlang package index In-Reply-To: References: <90676862-9a7e-4ab3-8382-2dcb0cdd017f@d23g2000prj.googlegroups.com> Message-ID: Dear Tim, It might be similar to sutro in general but there is a lot of differences in what it does and how it does it. You can probably get an idea of differences by glancing through agner's readme: https://github.com/agner/agner#readme -- there are certain differences in how specifications are organized, how indices can be layered, availability of 'installation' (which I believe in unnecessary as every project needs its own set of dependencies), rebar integration and so on. Hope this helps. On Thu, Jan 27, 2011 at 8:42 AM, Tim Watson wrote: > This looks very similar to https://github.com/JacobVorreuter/sutro. > From kuleshovmail@REDACTED Thu Jan 27 19:08:50 2011 From: kuleshovmail@REDACTED (Alexander Kuleshov) Date: Fri, 28 Jan 2011 00:08:50 +0600 Subject: gen_event help Message-ID: Hello, I try to write simple gen_event applcation in erlang. My code: -module(test). -behavior(gen_event). -export([notify/0]). %% API -export([start_link/0, add_handler/0, stop/0]). %% gen_event callbacks -export([init/1, handle_event/2, handle_call/2, handle_info/2, terminate/2, code_change/3]). -define(SERVER, ?MODULE). start_link() -> gen_event:start_link({local, ?SERVER}). add_handler() -> gen_event:add_handler(?SERVER, ?MODULE, []). stop() -> gen_event:stop(?MODULE). init([]) -> %add_handler(), {ok, null}. handle_event(_Event, State) -> {ok, State}; handle_event({test}, State) -> io:format("Test"), {ok, State}. handle_call(_Request, State) -> Reply = ok, {ok, Reply, State}. handle_info(_Info, State) -> {ok, State}. terminate(_Reason, _State) -> ok. code_change(_OldVsn, State, _Extra) -> {ok, State}. notify() -> gen_event:notify(?SERVER, {test}). When i try notify i expect that function handle_event({test}, State) execute but nothing occurs. Why? What's wrong? And where i can see simple example of gen_event? Thank you. From vladdu55@REDACTED Thu Jan 27 19:23:48 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 27 Jan 2011 19:23:48 +0100 Subject: [erlang-questions] gen_event help In-Reply-To: References: Message-ID: Hi, On Thu, Jan 27, 2011 at 19:08, Alexander Kuleshov wrote: > handle_event(_Event, State) -> > {ok, State}; > handle_event({test}, State) -> > io:format("Test"), > {ok, State}. > > The first clause matches and is executed. Try this instead: handle_event({test}, State) -> io:format("Test"), {ok, State}; handle_event(_Event, State) -> {ok, State}. regards, Vlad From fernando.benavides@REDACTED Thu Jan 27 19:26:19 2011 From: fernando.benavides@REDACTED (Fernando Benavides) Date: Thu, 27 Jan 2011 15:26:19 -0300 Subject: [erlang-questions] gen_event help In-Reply-To: References: Message-ID: <1296152779.3883.4.camel@army.local> Just a guess (i.e. not actually tested on console) but... this clause may match first... > handle_event(_Event, State) -> > {ok, State}; ...so nothing is shown on console. Cheers. On Fri, 2011-01-28 at 00:08 +0600, Alexander Kuleshov wrote: > Hello, > > I try to write simple gen_event applcation in erlang. > > My code: > > -module(test). > -behavior(gen_event). > > -export([notify/0]). > > %% API > -export([start_link/0, add_handler/0, stop/0]). > > %% gen_event callbacks > -export([init/1, handle_event/2, handle_call/2, > handle_info/2, terminate/2, code_change/3]). > > -define(SERVER, ?MODULE). > > start_link() -> > gen_event:start_link({local, ?SERVER}). > > add_handler() -> > gen_event:add_handler(?SERVER, ?MODULE, []). > > stop() -> > gen_event:stop(?MODULE). > > init([]) -> > %add_handler(), > {ok, null}. > > handle_event(_Event, State) -> > {ok, State}; > handle_event({test}, State) -> > io:format("Test"), > {ok, State}. > > handle_call(_Request, State) -> > Reply = ok, > {ok, Reply, State}. > > handle_info(_Info, State) -> > {ok, State}. > > terminate(_Reason, _State) -> > ok. > > code_change(_OldVsn, State, _Extra) -> > {ok, State}. > > notify() -> > gen_event:notify(?SERVER, {test}). > When i try notify i expect that function handle_event({test}, State) > execute but nothing occurs. Why? What's wrong? > > And where i can see simple example of gen_event? > > Thank you. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From ponton@REDACTED Thu Jan 27 19:20:48 2011 From: ponton@REDACTED (Tomasz Maciejewski) Date: Thu, 27 Jan 2011 19:20:48 +0100 Subject: [erlang-questions] gen_event help In-Reply-To: References: Message-ID: Dnia 27-01-2011 o 19:08:50 Alexander Kuleshov napisa?(a): > handle_event(_Event, State) -> > {ok, State}; > handle_event({test}, State) -> > io:format("Test"), > {ok, State}. > > (...) > > When i try notify i expect that function handle_event({test}, State) > execute but nothing occurs. Why? What's wrong? You catch every event by the first function clause, so the io:format/1 will never be called. -- Tomasz Maciejewski From daniel.goertzen@REDACTED Thu Jan 27 20:20:14 2011 From: daniel.goertzen@REDACTED (Daniel Goertzen) Date: Thu, 27 Jan 2011 13:20:14 -0600 Subject: [erlang-questions] trouble debugging app in erlide In-Reply-To: References: Message-ID: I setup the startup function as you suggest but it has to directly invoke my application start function to get debug capability; when I use application:start(myapp) I don't get debugging. My application callback module looks like this: -module(myapp_app). -behavior(application). -export([start/2, stop/1]). -export([start/0]). start() -> start(normal,[]). % invoked by erlide debug configuration %% start() -> application:start(myapp). %no good for debugging! start(normal, _Args) -> ... regular application startup Good enough for me; thank you again for your help. Dan. On Thu, Jan 27, 2011 at 10:33 AM, Vlad Dumitrescu wrote: > Hi! > > On Thu, Jan 27, 2011 at 16:36, Daniel Goertzen wrote: > >> I am using erlide, and I cannot get breakpoints to work on my app when the >> app is started automatically by my boot script. >> >> If I use the console to application:stop()/start() my app, breakpoints >> start >> working normally. But I am intensely lazy and want breakpoints to work >> right when I launch my Debug Config. >> >> Any advice? >> > > I think that the problem is that the application gets started before the > debugger, thus before the breakpoints are set and the code is interpreted. I > don't think we can do much about that, short of adding the erlide debugger > application to the boot script, but that would require large changes to our > framework. > > A suggestion would be to create a parameterless function that starts your > application and does any other initialization, and specify it as startup > function in the debug configuration. > > regards, > Vlad > > From vladdu55@REDACTED Thu Jan 27 20:28:40 2011 From: vladdu55@REDACTED (Vlad Dumitrescu) Date: Thu, 27 Jan 2011 20:28:40 +0100 Subject: [erlang-questions] trouble debugging app in erlide In-Reply-To: References: Message-ID: On Thu, Jan 27, 2011 at 20:20, Daniel Goertzen wrote: > I setup the startup function as you suggest but it has to directly invoke > my application start function to get debug capability; when I use > application:start(myapp) I don't get debugging. > That's good to know. > Good enough for me; thank you again for your help.? > You're welcome! /Vlad From robert.virding@REDACTED Thu Jan 27 20:32:16 2011 From: robert.virding@REDACTED (Robert Virding) Date: Thu, 27 Jan 2011 19:32:16 +0000 (GMT) Subject: [erlang-questions] Shell - compile multiple files with ERLC in Windows In-Reply-To: <728358952.360291296156652688.JavaMail.root@zimbra> Message-ID: <2057497181.360311296156736789.JavaMail.root@zimbra> The reason you need to do this in Windows is that in Linux the shell automatically expands src/*.erl to the names of all the files which match, i.e. all .erl files in src, while the Windows shell does not do this. There in Windows either the application has to do itself, which erlang doesn't, or you need to write an explicit loop which does the expansion as Roberto has done. Robert ----- "Roberto Ostinelli" wrote: > > > > try: > > > > FOR %%f in (src\*.erl) DO erlc -W %command% -o ebin "%%f" > > > > r. > > > > sorry for that, obviously %command% is a paste error. this should do > it: > > FOR %%f in (src\*.erl) DO erlc -W -o ebin "%%f" -- Robert Virding, Erlang Solutions Ltd. From alain.odea@REDACTED Thu Jan 27 22:19:49 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Thu, 27 Jan 2011 17:49:49 -0330 Subject: [erlang-questions] EPmail-0.1 released In-Reply-To: References: Message-ID: <7735F2E2-ACF9-4B68-80F3-1CCF83548CBD@gmail.com> On 2011-01-27, at 10:27, Alexander Kuleshov wrote: > Hello, > > Today i released epmail-0.1. EPmail - it's mailing system written in Erlang. > Now it has only pop3 server. But in future i planned to do > smtp/imap/pop3 server system in Erlang. > > EPmail - https://github.com/onlyshk/epmail > > EPmail Wiki - https://github.com/onlyshk/epmail/wiki > > EPMail status - https://github.com/onlyshk/epmail/wiki/Status > > I'm new in erlang and i think there are many bugs and defects, but i > fix it with your help :) > > How to install and run epmail pop3 server written in Wiki or in README > file. Now you can connect to it by telnet or full value mail client. I > tested with thunderbird for example. > Mail storage is simple text file for every user, but in future i'll > plan to bind database for users. Now pop3 server worked only for 1 > domain, but i fix it in next version. > > I waiting for your critics, because your opinion is very important for > the project and for me as a beginner Erlang programmer. > > Please, feel free to fork it and improve project. > > Thank you. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Nice work Alexander :) I noticed that you are using a receive loop in popd_listener. It may be easier to spawn a child gen_fsm for each connection. That would be an awesome next step. Mitchell Hashimoto's http://spawnlink.com/articles/an-introduction-to-gen_fsm-erlybanks-atm/ is a very solid introduction to gen_fsm. Overall his blog is an excellent intro to OTP. When you want to dig deeper you should pick up Erlang and OTP in Action http://manning.com/logan. Keep up the good work :) Sincerely, Alain From antonio.fernandez@REDACTED Thu Jan 27 22:21:28 2011 From: antonio.fernandez@REDACTED (=?UTF-8?B?QW50b25pbyBGZXJuw6FuZGV6?=) Date: Thu, 27 Jan 2011 22:21:28 +0100 Subject: [erlang-questions] Shell - compile multiple files with ERLC in Windows In-Reply-To: <2057497181.360311296156736789.JavaMail.root@zimbra> References: <2057497181.360311296156736789.JavaMail.root@zimbra> Message-ID: <4D41E1D8.2090306@jaraxa.com> Hi Robert ! I tried you solution but didn?t works : *FOR %%f in (src\*.erl) DO erlc -W -o ebin "%%f"* It works after a little fix ... *J:\erlang>FOR %f in (src\*.erl) DO erlc -W -o ebin "%f"* J:\erlang>erlc -W -o ebin "src\hello.erl" J:\erlang>erlc -W -o ebin "src\hello2.erl" I?m running Erlang over Windows7 home-premium ..... Thanks & nice weekend :D Antonio -- Antonio Miguel Fern?ndez Rodr?guez Jaraxa Software, SL www.jaraxa.com El 27/01/2011 20:32, Robert Virding escribi?: > The reason you need to do this in Windows is that in Linux the shell automatically expands src/*.erl to the names of all the files which match, i.e. all .erl files in src, while the Windows shell does not do this. There in Windows either the application has to do itself, which erlang doesn't, or you need to write an explicit loop which does the expansion as Roberto has done. > > Robert > > ----- "Roberto Ostinelli" wrote: > >>> try: >>> >>> FOR %%f in (src\*.erl) DO erlc -W %command% -o ebin "%%f" >>> >>> r. >>> >> sorry for that, obviously %command% is a paste error. this should do >> it: >> >> FOR %%f in (src\*.erl) DO erlc -W -o ebin "%%f" > -- > Robert Virding, Erlang Solutions Ltd. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > From hm@REDACTED Thu Jan 27 22:36:24 2011 From: hm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Thu, 27 Jan 2011 22:36:24 +0100 Subject: [erlang-questions] Mnesia Fragmented Tables, del_table_copy In-Reply-To: <1296148736.2059.19.camel@texhp> References: <1295443907.20811.23.camel@texhp> <1296148736.2059.19.camel@texhp> Message-ID: A fragmented table in Mnesia is a big distributed hash table where each bucket is implemented as a "normal" Mnesia table. When you insert a record in the fragmented table, a hash function is used to compute which bucket (table fragment) to put the record in. The linear hashing algorithm that is used allows the hash table to shrink by deleting the last bucket (table fragment) and dynamically rehash the records in that bucket and move them to other buckets. The algorithm does however not cope with deletion of arbitrary buckets. If you want to shrink the fragmented table (delete the last bucket) you need to use the function mnesia:change_table_frag(Tab, del_frag) as described in the user's guide: http://www.erlang.org/doc/apps/mnesia/Mnesia_chap5.html. Each bucket (table fragment) can be replicated to several nodes. Replicas are deleted with the function mnesia:del_table_copy(Frag, Node) and it is not possible to use this function to delete the last replica. In your case you had 4 fragments and you tried to delete the last replica of the second fragment. This is not allowed. But it would have been allowed to delete the fourth fragment (and move its records) by invoking mnesia:change_table_frag(constResourceData, del_frag) /H?kan PS. The function mnesia:change_table_frag/2 is only mentioned in the user's guide. It is not documented in the reference manual. Sorry for that. On Thu, Jan 27, 2011 at 6:18 PM, Tessaro Alexej wrote: > Thank you for your reply. > > In my case the table is dynamically growing (by adding new fragments > when the existing ones reach some sort of threshold). > > Cause the contained data have no critical relevance (but real-time > reading and writing times constraints), I thought to maintain ram only > copies with no replicas. > > So, when a node disconnects from the cluster cloud, I would expect the > data it was containing to disappear from the cloud (and the requests > involving those records to fail). > > I expected mnesia_frag system to detect not running mnesia nodes, thus > not considering them when calling mnesia activity functions on the whole > table. > > Am I missing something? > > On Wed, 2011-01-26 at 14:45 -0200, Igor Ribeiro Sucupira wrote: >> I haven't looked at it in detail, but I know it doesn't make much >> sense to delete the second fragment (leaving no other copies of it) in >> a table that has 4 fragments. It would break your fragmented table >> (writing keys would not be possible in some cases). >> >> Best regards. >> Igor. >> >> On Wed, Jan 19, 2011 at 11:31 AM, Tessaro Alexej wrote: >> > Hello, >> > >> > I have started two communicating mnesia nodes and created one fragmented >> > table with fragments stored in RAM only (no other replicas) and schema >> > stored on disk on both nodes: >> > >> > node1 : constResourceData_frag3 (ram), constResourceData_frag4 (ram), >> > schema (disk) >> > >> > node2 : constResourceData (ram), constResourceData_frag2 (ram), schema >> > (disk) >> > >> > Now, from node1, I tried to call >> > mnesia:del_table_copy(constResourceData_frag2, node2) >> > >> > but it fails saying: >> > >> > node1> Mnesia(node1): Last replica deleted in table >> > constResourceData_frag2 >> > node1>{aborted,{no_exists,constResourceData_frag2,frag_properties, >> > ? ? ? ? ? ? ? ? ? ?frag_hash}} >> > node1> Mnesia(node1): Transaction {tid,75,<0.371.0>} calling >> > #Fun with [] failed: >> > ?{aborted,{no_exists,constResourceData_frag2,frag_properties,frag_hash}} >> > >> > Looking at the mnesia_schema.erl source code, I found that the failure >> > seems to happen when it tries to delete the "whole_table" in >> > make_delete_table/2 and it runs the following check: >> > >> > %% Check if it is a base table >> > nesia_frag:lookup_frag_hash(Tab), >> > >> > Why is my attempt of removing the fragment failing? >> > >> > My erlang version is R12B. >> > >> > Thank you From boris.muehmer@REDACTED Thu Jan 27 22:28:52 2011 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Thu, 27 Jan 2011 22:28:52 +0100 Subject: Erlang and wxWidgets: how to help to improve the stability of wx (on Ubuntu)?!?! Message-ID: After some time of other work I wanted to do some tests with Erlang and wxWidgets again on my linux box at home, but my system failed on "wx:demo()" with a core dump. === output: begin === $ /opt/erlang-otp/bin/erl Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.2 (abort with ^G) 1> wx:demo(). beam.smp: malloc.c:3096: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. Aborted === output: end === This sort of behaviour (with erl core dumping) was already discussed two or three months ago ("Debugger fault at startup." and "wx and the problem that won't go away." by Michael Richter, end of September 2010). I had a brief look with gdb at the core dump, but it didn't give me a clue what went wrong. I even recompiled my erlang installation from the official tarball (R14B01) and the git repository, but all my builds core dumped on my home system. When I tried this at work, I could not reproduce this error. The main difference between those systems is the Ubuntu version installed: - home: 10.10 - work: 10.04 But I believe "wx:demo/()" worked with Ubuntu 10.10 last year, but I am not 100% sure, because reading "Erlang and OTP in Action" (for the last weeks) didn't involve any UI/wx programming. So what can I (or we)?do?to improve the stability of the wx environment? Is this just a problem with Ubuntu or do other (Linux, *BSD, Windows, etc.) systems/environments have the same or similar problem(s)? Is anyone interested in the core dump? ??- boris From jws@REDACTED Fri Jan 28 02:01:20 2011 From: jws@REDACTED (Jeff Schultz) Date: Fri, 28 Jan 2011 12:01:20 +1100 Subject: [erlang-questions] Shell - compile multiple files with ERLC in Windows In-Reply-To: <4D41A703.10309@jaraxa.com> References: <4D41A703.10309@jaraxa.com> Message-ID: <20110128010120.GA22320@mulga.csse.unimelb.edu.au> On Thu, Jan 27, 2011 at 06:10:27PM +0100, Antonio Fernndez wrote: > > Hi all ! > > I?m erlang beginner programmer. > > The following command works on Ubuntu Shell but don?t work on Windows : > > $ erlc --o ebin src/*.erl Possibly the easiest solution is to install http://cygwin.org/ (and put the Erlang bin directory in your path.) Jeff Schultz From watson.timothy@REDACTED Fri Jan 28 02:48:55 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 28 Jan 2011 01:48:55 +0000 Subject: [erlang-questions] [ANN] Agner, a rebar-friendly Erlang package index In-Reply-To: References: <90676862-9a7e-4ab3-8382-2dcb0cdd017f@d23g2000prj.googlegroups.com> Message-ID: On 27 January 2011 17:38, Yurii Rashkovskii wrote: > Dear Tim, > > It might be similar to sutro in general but there is a lot of > differences in what it does and how it does it. You can probably get > an idea of differences by glancing through agner's readme: > https://github.com/agner/agner#readme -- there are certain differences > in how specifications are organized, how indices can be layered, > availability of 'installation' (which I believe in unnecessary as > every project needs its own set of dependencies), rebar integration > and so on. Hope this helps. So are you saying the agner doesn't install packages, but it indexes, searches and fetches them? That's quite different to homebrew, which installs them - it was your comparison with homebrew that had me confused, but now I see that you're referring to the repository aspect of tools like it. I'm not entirely sure what you meant by "installation is [sic] unnecessary as every projects needs its own set of dependencies". Both sutro and its predecessor (epm) can handle transitive dependencies fine, and allow you get "specific" about custom pre/build/install commands where the default doesn't work. Maybe I've misunderstood? Nevertheless, this looks very interesting. I think it is good to have a choice about projects/approaches to various tasks, and this is becoming more common amongst the open source Erlang community. Perhaps this is a sign of its growing maturity (yes, I know the *platform/language* has been around forever - that's not what I mean). When managing dependencies and/or getting hold of Erlang *stuff*, I can now consider quite a few: - rebar (with get-deps) - Erlware (build tool, package manager and binary artefact repository) - Erlang Package Manager (epm - github/bitbucket) - Sutro (a la homebrew) - Agner (a la homebrew) From yrashk@REDACTED Fri Jan 28 02:54:31 2011 From: yrashk@REDACTED (Yurii Rashkovskii) Date: Thu, 27 Jan 2011 17:54:31 -0800 Subject: [erlang-questions] [ANN] Agner, a rebar-friendly Erlang package index In-Reply-To: References: <90676862-9a7e-4ab3-8382-2dcb0cdd017f@d23g2000prj.googlegroups.com> Message-ID: On Thu, Jan 27, 2011 at 5:48 PM, Tim Watson wrote: > So are you saying the agner doesn't install packages, but it indexes, > searches and fetches them? That's quite different to homebrew, which > installs them - it was your comparison with homebrew that had me > confused, but now I see that you're referring to the repository aspect > of tools like it. I can see where it can be confusing. What I was referring to is more of a publishing similarity to Homebrew ??a GitHub centered approach (albeit Agner and Homebrew have certain differences in how things are managed) > > I'm not entirely sure what you meant by "installation is [sic] > unnecessary as every projects needs its own set of dependencies". Both > sutro and its predecessor (epm) can handle transitive dependencies > fine, and allow you get "specific" about custom pre/build/install > commands where the default doesn't work. Maybe I've misunderstood? What I meant is that Agner is not intended to be used to install packages system or user-wise, it's primarily intended for fetching dependencies per-project, especially when using it in conjunction with rebar. > - rebar (with get-deps) > - Erlware (build tool, package manager and binary artefact repository) > - Erlang Package Manager (epm - github/bitbucket) > - Sutro (a la homebrew) > - Agner (a la homebrew) Agner is more of a conceptual mixture of homebrew and clojars, to be more precise And you don't have to choose between Agner & rebar, there is an agner-enabled rebar that can fetch {agner, ... references straight from deps defined in rebar.config; and that I believe is one of the very important aspects of Agner. From watson.timothy@REDACTED Fri Jan 28 03:14:22 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 28 Jan 2011 02:14:22 +0000 Subject: [erlang-questions] [ANN] Agner, a rebar-friendly Erlang package index In-Reply-To: References: <90676862-9a7e-4ab3-8382-2dcb0cdd017f@d23g2000prj.googlegroups.com> Message-ID: > > I can see where it can be confusing. What I was referring to is more > of a publishing similarity to Homebrew ??a GitHub centered approach > (albeit Agner and Homebrew have certain differences in how things are > managed) > Yes I can see that now. Thanks for clarification and good luck with your project! Cheers, Tim From watson.timothy@REDACTED Fri Jan 28 03:22:38 2011 From: watson.timothy@REDACTED (Tim Watson) Date: Fri, 28 Jan 2011 02:22:38 +0000 Subject: [erlang-questions] Shell - compile multiple files with ERLC in Windows In-Reply-To: <20110128010120.GA22320@mulga.csse.unimelb.edu.au> References: <4D41A703.10309@jaraxa.com> <20110128010120.GA22320@mulga.csse.unimelb.edu.au> Message-ID: 2011/1/28 Jeff Schultz : > On Thu, Jan 27, 2011 at 06:10:27PM +0100, Antonio Fernndez wrote: >> >> Hi all ! >> >> I? erlang beginner programmer. >> >> The following command works on Ubuntu Shell but don? work on Windows : >> >> $ erlc --o ebin src/*.erl > > Possibly the easiest solution is to install http://cygwin.org/ > (and put the Erlang bin directory in your path.) > Or just run `erl -make all` in the directory? Or go get a copy of rebar (https://github.com/basho/rebar), install it and then run `rebar compile` - I think it works on Windows fine. From kuleshovmail@REDACTED Fri Jan 28 07:06:02 2011 From: kuleshovmail@REDACTED (shk) Date: Thu, 27 Jan 2011 22:06:02 -0800 (PST) Subject: gen_event help In-Reply-To: References: Message-ID: <1296194762282-3243763.post@n4.nabble.com> Thank you for reply. I make it first: handle_event({test}, State) -> io:format("Test"), {ok, State}; handle_event(_Event, _State) -> {ok, _State}. But when i call notify/1 it's nothing output. -- View this message in context: http://erlang.2086793.n4.nabble.com/gen-event-help-tp3242753p3243763.html Sent from the Erlang Questions mailing list archive at Nabble.com. From kuleshovmail@REDACTED Fri Jan 28 07:07:34 2011 From: kuleshovmail@REDACTED (Alexander Kuleshov) Date: Fri, 28 Jan 2011 06:07:34 +0000 Subject: [erlang-questions] EPmail-0.1 released In-Reply-To: References: <7735F2E2-ACF9-4B68-80F3-1CCF83548CBD@gmail.com> Message-ID: Hello Alain, > It may be easier to spawn a child gen_fsm for each connection Yes, i think make it in next release. >Erlang and OTP in Action Good book, recently i started to read it. Thank you. From olivier.girondel@REDACTED Fri Jan 28 07:09:51 2011 From: olivier.girondel@REDACTED (Olivier Girondel) Date: Fri, 28 Jan 2011 07:09:51 +0100 Subject: [erlang-questions] Erlang and wxWidgets: how to help to improve the stability of wx (on Ubuntu)?!?! In-Reply-To: References: Message-ID: On Thu, Jan 27, 2011 at 10:28 PM, Boris M?hmer wrote: > After some time of other work I wanted to do some tests with Erlang > and wxWidgets again on my linux box at home, but my system failed on > "wx:demo()" with a core dump. > > === output: begin === > > $ /opt/erlang-otp/bin/erl > Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:2:2] [rq:2] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.8.2 ?(abort with ^G) > 1> wx:demo(). > beam.smp: malloc.c:3096: sYSMALLOc: Assertion `(old_top == (((mbinptr) > (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct > malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >>= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, > fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) > - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & > pagemask) == 0)' failed. > > ? ? ? ? ? ? Aborted > === output: end === > > This sort of behaviour (with erl core dumping) was already discussed > two or three months ago ("Debugger fault at startup." and "wx and the > problem that won't go away." by Michael Richter, end of September > 2010). > > I had a brief look with gdb at the core dump, but it didn't give me a > clue what went wrong. I even recompiled my erlang installation from > the official tarball (R14B01) and the git repository, but all my > builds core dumped on my home system. > > When I tried this at work, I could not reproduce this error. > > The main difference between those systems is the Ubuntu version installed: > - home: 10.10 > - work: 10.04 > > But I believe "wx:demo/()" worked with Ubuntu 10.10 last year, but I > am not 100% sure, because reading "Erlang and OTP in Action" (for the > last weeks) didn't involve any UI/wx programming. > > So what can I (or we)?do?to improve the stability of the wx > environment? Is this just a problem with Ubuntu or do other (Linux, > *BSD, Windows, etc.) systems/environments have the same or similar > problem(s)? Is anyone interested in the core dump? > > ??- boris Hi, I had the same problem on various platforms, this one is on Debian/squeeze, R13B04 So it might not be related to Ubuntu but rather to wx ? => ii libwxgtk2.8-dev 2.8.10.1-3+b1 ============================================================================= $ erl Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.7.5 (abort with ^G) 1> wx:demo(). beam.smp: malloc.c:3097: sYSMALLOc: Assertion `(old_top == (((mbinptr) (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & pagemask) == 0)' failed. Aborted ============================================================================= -- Olivier From dangud@REDACTED Fri Jan 28 07:32:56 2011 From: dangud@REDACTED (Dan Gudmundsson) Date: Fri, 28 Jan 2011 07:32:56 +0100 Subject: [erlang-questions] Shell - compile multiple files with ERLC in Windows In-Reply-To: <20110128010120.GA22320@mulga.csse.unimelb.edu.au> References: <4D41A703.10309@jaraxa.com> <20110128010120.GA22320@mulga.csse.unimelb.edu.au> Message-ID: I use msysgit-developer package for my windows development, with one install I get git, rxvt, bash, make and gcc. /Dan 2011/1/28 Jeff Schultz : > On Thu, Jan 27, 2011 at 06:10:27PM +0100, Antonio Fernndez wrote: >> >> Hi all ! >> >> I? erlang beginner programmer. >> >> The following command works on Ubuntu Shell but don? work on Windows : >> >> $ erlc --o ebin src/*.erl > > Possibly the easiest solution is to install http://cygwin.org/ > (and put the Erlang bin directory in your path.) > > > ? ?Jeff Schultz > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From dgud@REDACTED Fri Jan 28 07:41:26 2011 From: dgud@REDACTED (Dan Gudmundsson) Date: Fri, 28 Jan 2011 07:41:26 +0100 Subject: [erlang-questions] Erlang and wxWidgets: how to help to improve the stability of wx (on Ubuntu)?!?! In-Reply-To: References: Message-ID: I upgraded to Ubuntu-10.10 yesterday on my personal linux, and didn't have any problems. But I have a 32bit system. I will try to get hold of a Ubuntu 64 machine next week, but we run Sles-10 on most of our machines. You can email me (privatly?) the backtrace of the R14B01 crash? /Dan On Fri, Jan 28, 2011 at 7:09 AM, Olivier Girondel wrote: > On Thu, Jan 27, 2011 at 10:28 PM, Boris M?hmer > wrote: >> After some time of other work I wanted to do some tests with Erlang >> and wxWidgets again on my linux box at home, but my system failed on >> "wx:demo()" with a core dump. >> >> === output: begin === >> >> $ /opt/erlang-otp/bin/erl >> Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:2:2] [rq:2] >> [async-threads:0] [hipe] [kernel-poll:false] >> >> Eshell V5.8.2 ?(abort with ^G) >> 1> wx:demo(). >> beam.smp: malloc.c:3096: sYSMALLOc: Assertion `(old_top == (((mbinptr) >> (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct >> malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >>>= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, >> fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) >> - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & >> pagemask) == 0)' failed. >> >> ? ? ? ? ? ? Aborted >> === output: end === >> >> This sort of behaviour (with erl core dumping) was already discussed >> two or three months ago ("Debugger fault at startup." and "wx and the >> problem that won't go away." by Michael Richter, end of September >> 2010). >> >> I had a brief look with gdb at the core dump, but it didn't give me a >> clue what went wrong. I even recompiled my erlang installation from >> the official tarball (R14B01) and the git repository, but all my >> builds core dumped on my home system. >> >> When I tried this at work, I could not reproduce this error. >> >> The main difference between those systems is the Ubuntu version installed: >> - home: 10.10 >> - work: 10.04 >> >> But I believe "wx:demo/()" worked with Ubuntu 10.10 last year, but I >> am not 100% sure, because reading "Erlang and OTP in Action" (for the >> last weeks) didn't involve any UI/wx programming. >> >> So what can I (or we)?do?to improve the stability of the wx >> environment? Is this just a problem with Ubuntu or do other (Linux, >> *BSD, Windows, etc.) systems/environments have the same or similar >> problem(s)? Is anyone interested in the core dump? >> >> ??- boris > > Hi, > > I had the same problem on various platforms, this one is on > Debian/squeeze, R13B04 > > So it might not be related to Ubuntu but rather to wx ? > => ii ?libwxgtk2.8-dev ? ? ? ? ? ? ? ? ? ? ?2.8.10.1-3+b1 > > ============================================================================= > $ erl > Erlang R13B04 (erts-5.7.5) [source] [64-bit] [smp:2:2] [rq:2] > [async-threads:0] [hipe] [kernel-poll:false] > > Eshell V5.7.5 ?(abort with ^G) > 1> wx:demo(). > beam.smp: malloc.c:3097: sYSMALLOc: Assertion `(old_top == (((mbinptr) > (((char *) &((av)->bins[((1) - 1) * 2])) - __builtin_offsetof (struct > malloc_chunk, fd)))) && old_size == 0) || ((unsigned long) (old_size) >>= (unsigned long)((((__builtin_offsetof (struct malloc_chunk, > fd_nextsize))+((2 * (sizeof(size_t))) - 1)) & ~((2 * (sizeof(size_t))) > - 1))) && ((old_top)->size & 0x1) && ((unsigned long)old_end & > pagemask) == 0)' failed. > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?Aborted > ============================================================================= > > -- > Olivier > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From max.lapshin@REDACTED Fri Jan 28 09:10:57 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 28 Jan 2011 11:10:57 +0300 Subject: [erlang-questions] EPmail-0.1 released In-Reply-To: References: <7735F2E2-ACF9-4B68-80F3-1CCF83548CBD@gmail.com> Message-ID: Really, this code: accept(Socket) -> case gen_tcp:accept(Socket) of {ok, Sock} -> spawn(?MODULE, receive_loop, [Sock, [], []]), gen_tcp:send(Sock, "+OK POP3 server ready \r\n"), accept(Socket); {error, Reason} -> Reason end. isn't suitable for production. You should start all processes via supervisors to keep track of them. If you start just plain spawn/spawn_link, you risk to loose process and get memory leak. From atessaro@REDACTED Fri Jan 28 09:38:01 2011 From: atessaro@REDACTED (Tessaro Alexej) Date: Fri, 28 Jan 2011 09:38:01 +0100 Subject: [erlang-questions] Mnesia Fragmented Tables, del_table_copy In-Reply-To: References: <1295443907.20811.23.camel@texhp> <1296148736.2059.19.camel@texhp> Message-ID: <1296203881.1854.27.camel@texhp> Ok, so I can remove the last fragment using change_table_copy(Tab, del_frag), but what am I supposed to do if I loose one node (let's say by physical disconnection) and the fragments in it are last replicas.. In my opinion, the on-top application logic should continue to work but the data contained in the lost node fragments will not be considered...right? Well, for example, if I try to call (after loosing one node and his fragments): mnesia:activity(async_dirty, TableInfo, [Table, size], mnesia_frag) where TableInfo is TableInfo = fun(TName, TInfo) -> mnesia:table_info(TName, TInfo) end I am looking for the whole table size (the sum of the table fragments sizes), but the function aborts when considering the lost fragments. In my case I would like the mnesia table_info fun to return me the table size anyway (simply adding 0 records to the table size accumulator when considering not existing fragments). But this is only a personal need in a personal application context, where I prefer to maintain ram only copies with no replicas. On Thu, 2011-01-27 at 22:36 +0100, H?kan Mattsson wrote: > A fragmented table in Mnesia is a big distributed hash table where each bucket > is implemented as a "normal" Mnesia table. When you insert a record in the > fragmented table, a hash function is used to compute which bucket (table > fragment) to put the record in. The linear hashing algorithm that is used allows > the hash table to shrink by deleting the last bucket (table fragment) and > dynamically rehash the records in that bucket and move them to other buckets. > The algorithm does however not cope with deletion of arbitrary buckets. > > If you want to shrink the fragmented table (delete the last bucket) you need to > use the function mnesia:change_table_frag(Tab, del_frag) as described in the > user's guide: http://www.erlang.org/doc/apps/mnesia/Mnesia_chap5.html. > > Each bucket (table fragment) can be replicated to several nodes. Replicas > are deleted with the function mnesia:del_table_copy(Frag, Node) and it is > not possible to use this function to delete the last replica. > > In your case you had 4 fragments and you tried to delete the last replica > of the second fragment. This is not allowed. But it would have been allowed > to delete the fourth fragment (and move its records) by invoking > > mnesia:change_table_frag(constResourceData, del_frag) > > /H?kan > > PS. > The function mnesia:change_table_frag/2 is only mentioned in the user's > guide. It is not documented in the reference manual. Sorry for that. > > On Thu, Jan 27, 2011 at 6:18 PM, Tessaro Alexej wrote: > > Thank you for your reply. > > > > In my case the table is dynamically growing (by adding new fragments > > when the existing ones reach some sort of threshold). > > > > Cause the contained data have no critical relevance (but real-time > > reading and writing times constraints), I thought to maintain ram only > > copies with no replicas. > > > > So, when a node disconnects from the cluster cloud, I would expect the > > data it was containing to disappear from the cloud (and the requests > > involving those records to fail). > > > > I expected mnesia_frag system to detect not running mnesia nodes, thus > > not considering them when calling mnesia activity functions on the whole > > table. > > > > Am I missing something? > > > > On Wed, 2011-01-26 at 14:45 -0200, Igor Ribeiro Sucupira wrote: > >> I haven't looked at it in detail, but I know it doesn't make much > >> sense to delete the second fragment (leaving no other copies of it) in > >> a table that has 4 fragments. It would break your fragmented table > >> (writing keys would not be possible in some cases). > >> > >> Best regards. > >> Igor. > >> > >> On Wed, Jan 19, 2011 at 11:31 AM, Tessaro Alexej wrote: > >> > Hello, > >> > > >> > I have started two communicating mnesia nodes and created one fragmented > >> > table with fragments stored in RAM only (no other replicas) and schema > >> > stored on disk on both nodes: > >> > > >> > node1 : constResourceData_frag3 (ram), constResourceData_frag4 (ram), > >> > schema (disk) > >> > > >> > node2 : constResourceData (ram), constResourceData_frag2 (ram), schema > >> > (disk) > >> > > >> > Now, from node1, I tried to call > >> > mnesia:del_table_copy(constResourceData_frag2, node2) > >> > > >> > but it fails saying: > >> > > >> > node1> Mnesia(node1): Last replica deleted in table > >> > constResourceData_frag2 > >> > node1>{aborted,{no_exists,constResourceData_frag2,frag_properties, > >> > frag_hash}} > >> > node1> Mnesia(node1): Transaction {tid,75,<0.371.0>} calling > >> > #Fun with [] failed: > >> > {aborted,{no_exists,constResourceData_frag2,frag_properties,frag_hash}} > >> > > >> > Looking at the mnesia_schema.erl source code, I found that the failure > >> > seems to happen when it tries to delete the "whole_table" in > >> > make_delete_table/2 and it runs the following check: > >> > > >> > %% Check if it is a base table > >> > nesia_frag:lookup_frag_hash(Tab), > >> > > >> > Why is my attempt of removing the fragment failing? > >> > > >> > My erlang version is R12B. > >> > > >> > Thank you > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Tessaro Alexej, Volunia Italia DISCLAIMER: This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you. From hm@REDACTED Fri Jan 28 10:26:17 2011 From: hm@REDACTED (=?ISO-8859-1?Q?H=E5kan_Mattsson?=) Date: Fri, 28 Jan 2011 10:26:17 +0100 Subject: [erlang-questions] Mnesia Fragmented Tables, del_table_copy In-Reply-To: <1296203881.1854.27.camel@texhp> References: <1295443907.20811.23.camel@texhp> <1296148736.2059.19.camel@texhp> <1296203881.1854.27.camel@texhp> Message-ID: 2011/1/28 Tessaro Alexej : > Ok, so I can remove the last fragment using change_table_copy(Tab, del_frag), No. You should use the mnesia:change_table_frag/2 function. > but what am I supposed to do if I loose one node (let's say > by physical disconnection) and the fragments in it are last replicas.. > > In my opinion, the on-top application logic should continue to work but > the data contained in the lost node fragments will not be considered...right? If you want a fault tolerant system where you want the system to continue to work despite that some node has crashed, you need to ensure that you have redundant resources. For short you need to replicate your Mnesia tables. > Well, for example, if I try to call (after loosing one node and his fragments): > > mnesia:activity(async_dirty, TableInfo, [Table, size], mnesia_frag) > > where TableInfo is > > TableInfo = fun(TName, TInfo) ? -> > ? ? ? ?mnesia:table_info(TName, TInfo) > end > > I am looking for the whole table size (the sum of the table fragments > sizes), but the function aborts when considering the lost fragments. > In my case I would like the mnesia table_info fun to return me the table > size anyway (simply adding 0 records to the table size accumulator when > considering not existing fragments). It is simply so that as all fragments not are available, Mnesia cannot compute a correct table size and therefore the function will fail. Firstly the behavior is consistent with how ordinary Mnesia tables works: the table size can only be computed if at least one replica of the table is online. Secondly it is consistent with how hash tables works: the table size can only be computed if all buckets are available. > But this is only a personal need in a personal application context, > where I prefer to maintain ram only copies with no replicas. You are trying to use fragmented tables in a way that they not was intended to be used. I think that it would be better for you if you used ordinary Mnesia tables and rolled your own fragmention logic on top of them. In fact it I did not get the impression that you actually need a distributed hash table (fragmented table) as you wanted to remove arbitrary buckets (table fragments). That is a rather unusual use of a hash table. /H?kan > On Thu, 2011-01-27 at 22:36 +0100, H?kan Mattsson wrote: >> A fragmented table in Mnesia is a big distributed hash table where each bucket >> is implemented as a "normal" Mnesia table. When you insert a record in the >> fragmented table, a hash function is used to compute which bucket (table >> fragment) to put the record in. The linear hashing algorithm that is used allows >> the hash table to shrink by deleting the last bucket (table fragment) and >> dynamically rehash the records in that bucket and move them to other buckets. >> The algorithm does however not cope with deletion of arbitrary buckets. >> >> If you want to shrink the fragmented table (delete the last bucket) you need to >> use the function mnesia:change_table_frag(Tab, del_frag) as described in the >> user's guide: http://www.erlang.org/doc/apps/mnesia/Mnesia_chap5.html. >> >> Each bucket (table fragment) can be replicated to several nodes. Replicas >> are deleted with the function mnesia:del_table_copy(Frag, Node) and it is >> not possible to use this function to delete the last replica. >> >> In your case you had 4 fragments and you tried to delete the last replica >> of the second fragment. This is not allowed. But it would have been allowed >> to delete the fourth fragment (and move its records) by invoking >> >> ? mnesia:change_table_frag(constResourceData, del_frag) >> >> /H?kan >> >> PS. >> ? The function mnesia:change_table_frag/2 is only mentioned in the user's >> ? guide. It is not documented in the reference manual. Sorry for that. >> >> On Thu, Jan 27, 2011 at 6:18 PM, Tessaro Alexej wrote: >> > Thank you for your reply. >> > >> > In my case the table is dynamically growing (by adding new fragments >> > when the existing ones reach some sort of threshold). >> > >> > Cause the contained data have no critical relevance (but real-time >> > reading and writing times constraints), I thought to maintain ram only >> > copies with no replicas. >> > >> > So, when a node disconnects from the cluster cloud, I would expect the >> > data it was containing to disappear from the cloud (and the requests >> > involving those records to fail). >> > >> > I expected mnesia_frag system to detect not running mnesia nodes, thus >> > not considering them when calling mnesia activity functions on the whole >> > table. >> > >> > Am I missing something? >> > >> > On Wed, 2011-01-26 at 14:45 -0200, Igor Ribeiro Sucupira wrote: >> >> I haven't looked at it in detail, but I know it doesn't make much >> >> sense to delete the second fragment (leaving no other copies of it) in >> >> a table that has 4 fragments. It would break your fragmented table >> >> (writing keys would not be possible in some cases). >> >> >> >> Best regards. >> >> Igor. >> >> >> >> On Wed, Jan 19, 2011 at 11:31 AM, Tessaro Alexej wrote: >> >> > Hello, >> >> > >> >> > I have started two communicating mnesia nodes and created one fragmented >> >> > table with fragments stored in RAM only (no other replicas) and schema >> >> > stored on disk on both nodes: >> >> > >> >> > node1 : constResourceData_frag3 (ram), constResourceData_frag4 (ram), >> >> > schema (disk) >> >> > >> >> > node2 : constResourceData (ram), constResourceData_frag2 (ram), schema >> >> > (disk) >> >> > >> >> > Now, from node1, I tried to call >> >> > mnesia:del_table_copy(constResourceData_frag2, node2) >> >> > >> >> > but it fails saying: >> >> > >> >> > node1> Mnesia(node1): Last replica deleted in table >> >> > constResourceData_frag2 >> >> > node1>{aborted,{no_exists,constResourceData_frag2,frag_properties, >> >> > ? ? ? ? ? ? ? ? ? ?frag_hash}} >> >> > node1> Mnesia(node1): Transaction {tid,75,<0.371.0>} calling >> >> > #Fun with [] failed: >> >> > ?{aborted,{no_exists,constResourceData_frag2,frag_properties,frag_hash}} >> >> > >> >> > Looking at the mnesia_schema.erl source code, I found that the failure >> >> > seems to happen when it tries to delete the "whole_table" in >> >> > make_delete_table/2 and it runs the following check: >> >> > >> >> > %% Check if it is a base table >> >> > nesia_frag:lookup_frag_hash(Tab), >> >> > >> >> > Why is my attempt of removing the fragment failing? >> >> > >> >> > My erlang version is R12B. >> >> > >> >> > Thank you >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > -- > Tessaro Alexej, Volunia Italia > > DISCLAIMER: This e-mail and any attachment is for authorised use by > the intended recipient(s) only. It may contain proprietary material, > confidential information and/or be subject to legal privilege. It > should not be copied, disclosed to, retained or used by, any other > party. If you are not an intended recipient then please promptly > delete this e-mail and any attachment and all copies and inform > the sender. Thank you. > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From robert.virding@REDACTED Fri Jan 28 11:11:40 2011 From: robert.virding@REDACTED (Robert Virding) Date: Fri, 28 Jan 2011 10:11:40 +0000 (GMT) Subject: [erlang-questions] Re: gen_event help In-Reply-To: <1376553557.362151296209244444.JavaMail.root@zimbra> Message-ID: <840821384.362311296209500903.JavaMail.root@zimbra> Have you added the handler using test:add_handler(). The test:start_link() only starts the event manager, you need to give it something to do by explicitly adding handlers, even if they are in the same module. Which is not always good practice, even if alarm_handler does it. Two links about event handlers from the on-line docs: >From the Design Principles section: http://www.erlang.org/doc/design_principles/events.html And the gen_event module: http://www.erlang.org/doc/man/gen_event.html Robert ----- "shk" wrote: > Thank you for reply. > > I make it first: > > handle_event({test}, State) -> > io:format("Test"), > {ok, State}; > handle_event(_Event, _State) -> > {ok, _State}. > > But when i call notify/1 it's nothing output. > -- > View this message in context: > http://erlang.2086793.n4.nabble.com/gen-event-help-tp3242753p3243763.html > Sent from the Erlang Questions mailing list archive at Nabble.com. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED -- Robert Virding, Erlang Solutions Ltd. From dujinfang@REDACTED Fri Jan 28 13:54:29 2011 From: dujinfang@REDACTED (Seven Du) Date: Fri, 28 Jan 2011 20:54:29 +0800 Subject: [erlang-questions] mysql driver in erlang support stored procedure In-Reply-To: References: <4d39d451.523de30a.63a5.ffffe5fcSMTPIN_ADDED@mx.google.com> Message-ID: The Eonblast fork was fixed on this issue. Thanks. On Sun, Jan 23, 2011 at 9:25 PM, Seven Du wrote: > Thanks Henning, I reported an issue on github. > > On Sat, Jan 22, 2011 at 2:45 AM, hd2010@REDACTED > wrote: >> Hi Seven, I'll be looking into it. >> >> The Eonblast fork is what I am working at today. If you have more info on >> what the unexpected behavior of SP in Emysql is, please send it my way. >> >> Thanks, >> Henning >> >> >> >> > > > > -- > About: http://about.me/dujinfang > Blog: http://www.dujinfang.com > Proj:? http://www.freeswitch.org.cn > -- About: http://about.me/dujinfang Blog: http://www.dujinfang.com Proj:? http://www.freeswitch.org.cn From jodie.burch@REDACTED Fri Jan 28 13:56:41 2011 From: jodie.burch@REDACTED (Jodie Burch) Date: Fri, 28 Jan 2011 12:56:41 +0000 Subject: Free Erlang Factory Lite in Munich - Places still available Message-ID: Hi Just to let you know that there are still places available on the Free Erlang Factory Lite in Munich on Monday 31st January with speakers including Klaus Trainer, Francesco Cesarini and GeoCouch Creator, Volker Mische. The event starts at 1pm and is completely free of charge but registration is required. You can find all the information you need and register here: http://www.erlang-factory.com/conference/ErlangFactoryLiteMunich Any questions, drop me an email. Thanks Jodie From atessaro@REDACTED Fri Jan 28 15:16:44 2011 From: atessaro@REDACTED (Tessaro Alexej) Date: Fri, 28 Jan 2011 15:16:44 +0100 Subject: [erlang-questions] Mnesia Fragmented Tables, del_table_copy In-Reply-To: References: <1295443907.20811.23.camel@texhp> <1296148736.2059.19.camel@texhp> <1296203881.1854.27.camel@texhp> Message-ID: <1296224204.1854.28.camel@texhp> Thanks for your help! On Fri, 2011-01-28 at 10:26 +0100, H?kan Mattsson wrote: > 2011/1/28 Tessaro Alexej : > > Ok, so I can remove the last fragment using change_table_copy(Tab, del_frag), > > No. You should use the mnesia:change_table_frag/2 function. > > > but what am I supposed to do if I loose one node (let's say > > by physical disconnection) and the fragments in it are last replicas.. > > > > In my opinion, the on-top application logic should continue to work but > > the data contained in the lost node fragments will not be considered...right? > > If you want a fault tolerant system where you want the system to continue > to work despite that some node has crashed, you need to ensure that you > have redundant resources. For short you need to replicate your Mnesia > tables. > > > Well, for example, if I try to call (after loosing one node and his fragments): > > > > mnesia:activity(async_dirty, TableInfo, [Table, size], mnesia_frag) > > > > where TableInfo is > > > > TableInfo = fun(TName, TInfo) -> > > mnesia:table_info(TName, TInfo) > > end > > > > I am looking for the whole table size (the sum of the table fragments > > sizes), but the function aborts when considering the lost fragments. > > In my case I would like the mnesia table_info fun to return me the table > > size anyway (simply adding 0 records to the table size accumulator when > > considering not existing fragments). > > It is simply so that as all fragments not are available, Mnesia cannot compute > a correct table size and therefore the function will fail. Firstly the > behavior is > consistent with how ordinary Mnesia tables works: the table size can only be > computed if at least one replica of the table is online. Secondly it > is consistent > with how hash tables works: the table size can only be computed if all buckets > are available. > > > But this is only a personal need in a personal application context, > > where I prefer to maintain ram only copies with no replicas. > > You are trying to use fragmented tables in a way that they not was intended > to be used. I think that it would be better for you if you used ordinary Mnesia > tables and rolled your own fragmention logic on top of them. In fact > it I did not > get the impression that you actually need a distributed hash table (fragmented > table) as you wanted to remove arbitrary buckets (table fragments). That is a > rather unusual use of a hash table. > > /H?kan > > > On Thu, 2011-01-27 at 22:36 +0100, H?kan Mattsson wrote: > >> A fragmented table in Mnesia is a big distributed hash table where each bucket > >> is implemented as a "normal" Mnesia table. When you insert a record in the > >> fragmented table, a hash function is used to compute which bucket (table > >> fragment) to put the record in. The linear hashing algorithm that is used allows > >> the hash table to shrink by deleting the last bucket (table fragment) and > >> dynamically rehash the records in that bucket and move them to other buckets. > >> The algorithm does however not cope with deletion of arbitrary buckets. > >> > >> If you want to shrink the fragmented table (delete the last bucket) you need to > >> use the function mnesia:change_table_frag(Tab, del_frag) as described in the > >> user's guide: http://www.erlang.org/doc/apps/mnesia/Mnesia_chap5.html. > >> > >> Each bucket (table fragment) can be replicated to several nodes. Replicas > >> are deleted with the function mnesia:del_table_copy(Frag, Node) and it is > >> not possible to use this function to delete the last replica. > >> > >> In your case you had 4 fragments and you tried to delete the last replica > >> of the second fragment. This is not allowed. But it would have been allowed > >> to delete the fourth fragment (and move its records) by invoking > >> > >> mnesia:change_table_frag(constResourceData, del_frag) > >> > >> /H?kan > >> > >> PS. > >> The function mnesia:change_table_frag/2 is only mentioned in the user's > >> guide. It is not documented in the reference manual. Sorry for that. > >> > >> On Thu, Jan 27, 2011 at 6:18 PM, Tessaro Alexej wrote: > >> > Thank you for your reply. > >> > > >> > In my case the table is dynamically growing (by adding new fragments > >> > when the existing ones reach some sort of threshold). > >> > > >> > Cause the contained data have no critical relevance (but real-time > >> > reading and writing times constraints), I thought to maintain ram only > >> > copies with no replicas. > >> > > >> > So, when a node disconnects from the cluster cloud, I would expect the > >> > data it was containing to disappear from the cloud (and the requests > >> > involving those records to fail). > >> > > >> > I expected mnesia_frag system to detect not running mnesia nodes, thus > >> > not considering them when calling mnesia activity functions on the whole > >> > table. > >> > > >> > Am I missing something? > >> > > >> > On Wed, 2011-01-26 at 14:45 -0200, Igor Ribeiro Sucupira wrote: > >> >> I haven't looked at it in detail, but I know it doesn't make much > >> >> sense to delete the second fragment (leaving no other copies of it) in > >> >> a table that has 4 fragments. It would break your fragmented table > >> >> (writing keys would not be possible in some cases). > >> >> > >> >> Best regards. > >> >> Igor. > >> >> > >> >> On Wed, Jan 19, 2011 at 11:31 AM, Tessaro Alexej wrote: > >> >> > Hello, > >> >> > > >> >> > I have started two communicating mnesia nodes and created one fragmented > >> >> > table with fragments stored in RAM only (no other replicas) and schema > >> >> > stored on disk on both nodes: > >> >> > > >> >> > node1 : constResourceData_frag3 (ram), constResourceData_frag4 (ram), > >> >> > schema (disk) > >> >> > > >> >> > node2 : constResourceData (ram), constResourceData_frag2 (ram), schema > >> >> > (disk) > >> >> > > >> >> > Now, from node1, I tried to call > >> >> > mnesia:del_table_copy(constResourceData_frag2, node2) > >> >> > > >> >> > but it fails saying: > >> >> > > >> >> > node1> Mnesia(node1): Last replica deleted in table > >> >> > constResourceData_frag2 > >> >> > node1>{aborted,{no_exists,constResourceData_frag2,frag_properties, > >> >> > frag_hash}} > >> >> > node1> Mnesia(node1): Transaction {tid,75,<0.371.0>} calling > >> >> > #Fun with [] failed: > >> >> > {aborted,{no_exists,constResourceData_frag2,frag_properties,frag_hash}} > >> >> > > >> >> > Looking at the mnesia_schema.erl source code, I found that the failure > >> >> > seems to happen when it tries to delete the "whole_table" in > >> >> > make_delete_table/2 and it runs the following check: > >> >> > > >> >> > %% Check if it is a base table > >> >> > nesia_frag:lookup_frag_hash(Tab), > >> >> > > >> >> > Why is my attempt of removing the fragment failing? > >> >> > > >> >> > My erlang version is R12B. > >> >> > > >> >> > Thank you > >> > >> ________________________________________________________________ > >> erlang-questions (at) erlang.org mailing list. > >> See http://www.erlang.org/faq.html > >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > >> > > > > -- > > Tessaro Alexej, Volunia Italia > > > > DISCLAIMER: This e-mail and any attachment is for authorised use by > > the intended recipient(s) only. It may contain proprietary material, > > confidential information and/or be subject to legal privilege. It > > should not be copied, disclosed to, retained or used by, any other > > party. If you are not an intended recipient then please promptly > > delete this e-mail and any attachment and all copies and inform > > the sender. Thank you. > > > > > > ________________________________________________________________ > > erlang-questions (at) erlang.org mailing list. > > See http://www.erlang.org/faq.html > > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > > > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > -- Tessaro Alexej, Volunia Italia DISCLAIMER: This e-mail and any attachment is for authorised use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you. From antonio.fernandez@REDACTED Fri Jan 28 16:18:07 2011 From: antonio.fernandez@REDACTED (=?UTF-8?B?QW50b25pbyBGZXJuw6FuZGV6?=) Date: Fri, 28 Jan 2011 16:18:07 +0100 Subject: [erlang-questions] Shell - compile multiple files with ERLC in Windows In-Reply-To: References: <4D41A703.10309@jaraxa.com> <20110128010120.GA22320@mulga.csse.unimelb.edu.au> Message-ID: <4D42DE2F.8030903@jaraxa.com> Wow , this mailing list seems really active :_D !!! Thanks you all for this interesting and alternative solutions, it?s time to take an overview of every one and then choosing the best for my needs. Nice Weekend -- Antonio Miguel Fern?ndez Rodr?guez Jaraxa Software, SL www.jaraxa.com El 28/01/2011 7:32, Dan Gudmundsson escribi?: > I use msysgit-developer package for my windows development, with one install > I get git, rxvt, bash, make and gcc. > > /Dan > > 2011/1/28 Jeff Schultz: >> On Thu, Jan 27, 2011 at 06:10:27PM +0100, Antonio Fernndez wrote: >>> Hi all ! >>> >>> I? erlang beginner programmer. >>> >>> The following command works on Ubuntu Shell but don? work on Windows : >>> >>> $ erlc --o ebin src/*.erl >> Possibly the easiest solution is to install http://cygwin.org/ >> (and put the Erlang bin directory in your path.) >> >> >> Jeff Schultz >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From alain.odea@REDACTED Fri Jan 28 17:25:47 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Fri, 28 Jan 2011 12:55:47 -0330 Subject: [erlang-questions] EPmail-0.1 released In-Reply-To: References: <7735F2E2-ACF9-4B68-80F3-1CCF83548CBD@gmail.com> Message-ID: <00514DDD-4B36-4F00-8EA0-7ADEEE40EE15@gmail.com> On 2011-01-28, at 2:37, Alexander Kuleshov wrote: > Hello Alain, > >> It may be easier to spawn a child gen_fsm for each connection > Yes, i think make it in next release. > >> Erlang and OTP in Action > Good book, recently i started to read it. > > Thank you. My pleasure Alexander :) From alain.odea@REDACTED Fri Jan 28 17:29:36 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Fri, 28 Jan 2011 12:59:36 -0330 Subject: [erlang-questions] EPmail-0.1 released In-Reply-To: References: <7735F2E2-ACF9-4B68-80F3-1CCF83548CBD@gmail.com> Message-ID: On 2011-01-28, at 4:40, Max Lapshin wrote: > Really, this code: > > accept(Socket) -> > case gen_tcp:accept(Socket) of > {ok, Sock} -> > spawn(?MODULE, receive_loop, [Sock, [], []]), > gen_tcp:send(Sock, "+OK POP3 server ready \r\n"), > accept(Socket); > {error, Reason} -> > Reason > end. > > > isn't suitable for production. You should start all processes via > supervisors to keep track of them. > If you start just plain spawn/spawn_link, you risk to loose process > and get memory leak. Alexander has planned to address this in the next version by using a supervisor and possibly gen_event child processes. He is reading through Erlang and OTP in Action now :) Memory leaks are a another very important motivation for this refactor. Thank you Max :) From max.lapshin@REDACTED Fri Jan 28 17:59:09 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 28 Jan 2011 19:59:09 +0300 Subject: [erlang-questions] EPmail-0.1 released In-Reply-To: References: <7735F2E2-ACF9-4B68-80F3-1CCF83548CBD@gmail.com> Message-ID: On Fri, Jan 28, 2011 at 7:29 PM, Alain O'Dea wrote: > > Alexander has planned to address this in the next version by using a supervisor and possibly gen_event child processes. ?He is reading through Erlang and OTP in Action now :) > > Memory leaks are a another very important motivation for this refactor. ?Thank you Max :) > I have struggled a lot with different kinds of memory leaks in erlyvideo, so I'm willing to share experience. They are: 1) process lost 2) too fast buffering data in state (buffering faster than processing and flushing) 3) bad default OTP implementation of error logger system, that leads to whole server failure on dumping state and reason of failure in on process. From kuleshovmail@REDACTED Fri Jan 28 18:30:58 2011 From: kuleshovmail@REDACTED (Alexander Kuleshov) Date: Fri, 28 Jan 2011 17:30:58 +0000 Subject: [erlang-questions] EPmail-0.1 released In-Reply-To: References: <7735F2E2-ACF9-4B68-80F3-1CCF83548CBD@gmail.com> Message-ID: Hi to all, I startin to implement gen_fsm. When client connect to server by listen socket i started want to start new gen_fsm for client request handling. I think i must create supervisor and started gen_fsm child every time that i accept new connection. Is it true way? Thank you. From max.lapshin@REDACTED Fri Jan 28 18:39:08 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Fri, 28 Jan 2011 20:39:08 +0300 Subject: [erlang-questions] EPmail-0.1 released In-Reply-To: References: <7735F2E2-ACF9-4B68-80F3-1CCF83548CBD@gmail.com> Message-ID: You need to create child one_to_one_simple Just take a look in code, how is it implemented. From ARoe@REDACTED Fri Jan 28 19:51:59 2011 From: ARoe@REDACTED (Adrian Roe) Date: Fri, 28 Jan 2011 10:51:59 -0800 Subject: [erlang-questions] EPmail-0.1 released In-Reply-To: References: <7735F2E2-ACF9-4B68-80F3-1CCF83548CBD@gmail.com> Message-ID: <0697C1D63C68ED4DA4EE16BFBB156ACD0B1025E13F@IE2RD2XVS071.red002.local> I presume you meant simple_one_for_one worker? There?s a pretty good discussion from the list here (http://www.erlang.org/pipermail/erlang-questions/2005-July/015900.html) although the code sample isn?t there anymore. My advice is to throw yourself in at the deep end ? they make much more sense once you have one up and running and have experimented with creating / closing / killing worker nodes. Adrian From: erlang-questions@REDACTED [mailto:erlang-questions@REDACTED] On Behalf Of Max Lapshin Sent: 28 January 2011 17:39 To: Alexander Kuleshov Cc: erlang-questions Subject: Re: [erlang-questions] EPmail-0.1 released You need to create child one_to_one_simple Just take a look in code, how is it implemented. ________________________________________________________________ erlang-questions (at) erlang.org mailing list. See http://www.erlang.org/faq.html To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED ________________________________ No virus found in this message. Checked by AVG - www.avg.com Version: 10.0.1204 / Virus Database: 1435/3408 - Release Date: 01/28/11 From johan@REDACTED Fri Jan 28 21:02:15 2011 From: johan@REDACTED (=?ISO-8859-1?Q?Johan_W=E4rlander?=) Date: Fri, 28 Jan 2011 21:02:15 +0100 Subject: [erlang-questions] EPmail-0.1 released In-Reply-To: References: <7735F2E2-ACF9-4B68-80F3-1CCF83548CBD@gmail.com> Message-ID: Hi Alexander, I'm also fairly new to Erlang, and I've tried a couple of different models for accepting TCP connections. The model I eventually arrived at was this: 1) Setting up an "acceptor supervisor", which starts a listener process and an acceptor pool supervisor in one_for_all mode. 2) When I call listen(Port, Acceptors) on the listener gen_server process it will gen_tcp:listen() and then start a number of child processes in the acceptor pool, which all wait in gen_tcp:accept() for incoming connections. 3) The acceptor pool is a simple_one_for_one supervisor, which will restart any of its acceptor processes if they die. 4) The acceptor child processes themselves have only this job: to wait for an incoming connection, then start an actual worker process (in your case the gen_fsm), and hand off the socket to that process. They will then go back to listening for the next incoming connection. Not sure if this is too complex a model (certainly others here will have opinions about that - do share!), but the reason I did it this way is that to me, it seems to provide better fault tolerance. If my acceptor child processes die for some reason, after a while the acceptor pool will restart itself. Since the top acceptor supervisor uses one_for_all strategy, it will then also restart the listener, which in turn reopens the listening socket and starts new acceptor child processes in the pool, getting us back to a known state. Feel free to have a look here: https://bitbucket.org/jwarlander/erlymud/src/cbf836ae2b6c/lib/erlymud/src/ The files in question are em_acceptor_sup.erl (the top one_for_all acceptor supervisor), em_acceptor_pool.erl (the simple_one_for_one acceptor pool), em_acceptor.erl (acceptor child processes) and em_listener.erl (opening the listening socket and tells em_acceptor_pool to start acceptor child processes). Regards, Johan On Fri, Jan 28, 2011 at 6:30 PM, Alexander Kuleshov wrote: > Hi to all, > > I startin to implement gen_fsm. > > When client connect to server by listen socket i started want to start > new gen_fsm for client request handling. > > I think i must create supervisor and started gen_fsm child every time > that i accept new connection. > > Is it true way? > > Thank you. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From johan@REDACTED Fri Jan 28 21:20:28 2011 From: johan@REDACTED (=?ISO-8859-1?Q?Johan_W=E4rlander?=) Date: Fri, 28 Jan 2011 21:20:28 +0100 Subject: [erlang-questions] EPmail-0.1 released In-Reply-To: References: <7735F2E2-ACF9-4B68-80F3-1CCF83548CBD@gmail.com> Message-ID: Actually.. *ponder* Now that I describe what I was imagining would happen, I realize it's not the case entirely. Sorry about that. In case of a failure that actually causes the acceptor pool to restart (ie, an acceptor child restarts several times in a short span of time), I'll end up without acceptor children anyway because it's set up so that em_listener:listen() must be called in order to start those up, but it's not called automatically of course just because em_listener gets restarted. Either way it should be fairly stable as the acceptor child processes are very very simple, and if they suddenly die repeatedly then there's probably some other issue going on with the system. But now I'll need to do some thinking on how to make this work like I wanted it to, anyway, just so I can be on the safe side. //Johan On Fri, Jan 28, 2011 at 9:02 PM, Johan W?rlander wrote: > Hi Alexander, > > I'm also fairly new to Erlang, and I've tried a couple of different models > for accepting TCP connections. The model I eventually arrived at was this: > > 1) Setting up an "acceptor supervisor", which starts a listener process and > an acceptor pool supervisor in one_for_all mode. > 2) When I call listen(Port, Acceptors) on the listener gen_server process > it will gen_tcp:listen() and then start a number of child processes in the > acceptor pool, which all wait in gen_tcp:accept() for incoming connections. > 3) The acceptor pool is a simple_one_for_one supervisor, which will restart > any of its acceptor processes if they die. > 4) The acceptor child processes themselves have only this job: to wait for > an incoming connection, then start an actual worker process (in your case > the gen_fsm), and hand off the socket to that process. They will then go > back to listening for the next incoming connection. > > Not sure if this is too complex a model (certainly others here will have > opinions about that - do share!), but the reason I did it this way is that > to me, it seems to provide better fault tolerance. If my acceptor child > processes die for some reason, after a while the acceptor pool will restart > itself. Since the top acceptor supervisor uses one_for_all strategy, it will > then also restart the listener, which in turn reopens the listening socket > and starts new acceptor child processes in the pool, getting us back to a > known state. > > Feel free to have a look here: > https://bitbucket.org/jwarlander/erlymud/src/cbf836ae2b6c/lib/erlymud/src/ > > The files in question are em_acceptor_sup.erl (the top one_for_all acceptor > supervisor), em_acceptor_pool.erl (the simple_one_for_one acceptor pool), > em_acceptor.erl (acceptor child processes) and em_listener.erl (opening the > listening socket and tells em_acceptor_pool to start acceptor child > processes). > > Regards, > Johan > > > On Fri, Jan 28, 2011 at 6:30 PM, Alexander Kuleshov < > kuleshovmail@REDACTED> wrote: > >> Hi to all, >> >> I startin to implement gen_fsm. >> >> When client connect to server by listen socket i started want to start >> new gen_fsm for client request handling. >> >> I think i must create supervisor and started gen_fsm child every time >> that i accept new connection. >> >> Is it true way? >> >> Thank you. >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > From boris.muehmer@REDACTED Sat Jan 29 08:03:56 2011 From: boris.muehmer@REDACTED (=?UTF-8?Q?Boris_M=C3=BChmer?=) Date: Sat, 29 Jan 2011 08:03:56 +0100 Subject: Erlang and wxWidgets: how to help to improve the stability of wx (on Ubuntu)?!?! In-Reply-To: References: Message-ID: 2011/1/27 Boris M?hmer : > "wx:demo()" with a core dump. I just realized that the core dump can be generated just with "wx:new()."! The result ist the same. - boris From kuleshovmail@REDACTED Sat Jan 29 08:18:59 2011 From: kuleshovmail@REDACTED (Alexander Kuleshov) Date: Sat, 29 Jan 2011 07:18:59 +0000 Subject: [erlang-questions] EPmail-0.1 released In-Reply-To: References: <7735F2E2-ACF9-4B68-80F3-1CCF83548CBD@gmail.com> Message-ID: Thank you Johan for help. I try to supervisor start_child gen_gsm: accept(Socket) -> case gen_tcp:accept(Socket) of {ok, Sock} -> Child = {popd_fsm, {popd_fsm, start_link, [Sock, [], []]}, permanent, 2000, worker, [popd_fsm]}, supervisor:start_child(popd_sup, Child), gen_tcp:send(Sock, "+OK POP3 server ready \r\n"), accept(Socket); {error, Reason} -> Reason end. My popd_fsm src - https://gist.github.com/801635. It's work normal only with 1 client. When i connect 2 client and more it's crash. I add {next_state, autorization, State}. in the autorization/2 it's not crash, but work only for one client. From johan@REDACTED Sat Jan 29 08:54:14 2011 From: johan@REDACTED (=?ISO-8859-1?Q?Johan_W=E4rlander?=) Date: Sat, 29 Jan 2011 08:54:14 +0100 Subject: [erlang-questions] EPmail-0.1 released In-Reply-To: References: <7735F2E2-ACF9-4B68-80F3-1CCF83548CBD@gmail.com> Message-ID: I'll look closer at the code when I get to the office, but initially I can say that if your gen_fsm should receive data from the socket, you need to transfer ownership to the new process.. Something like: {ok, Pid} = supervisor:start_child(...), gen_tcp:controlling_process(Sock, Pid), ... Check docs for gen_tcp, not sure of that function call. Eh.. Tedious to type on the phone :) //Johan Den 29 jan 2011 08.21 skrev "Alexander Kuleshov" : > Thank you Johan for help. > > I try to supervisor start_child gen_gsm: > > accept(Socket) -> > case gen_tcp:accept(Socket) of > {ok, Sock} -> > Child = {popd_fsm, {popd_fsm, start_link, [Sock, [], []]}, > permanent, 2000, worker, [popd_fsm]}, > > supervisor:start_child(popd_sup, Child), > gen_tcp:send(Sock, "+OK POP3 server ready \r\n"), > accept(Socket); > {error, Reason} -> > Reason > end. > > My popd_fsm src - https://gist.github.com/801635. > > It's work normal only with 1 client. When i connect 2 client and more > it's crash. > > I add {next_state, autorization, State}. in the autorization/2 it's > not crash, but work only for one client. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From kuleshovmail@REDACTED Sat Jan 29 09:00:10 2011 From: kuleshovmail@REDACTED (Alexander Kuleshov) Date: Sat, 29 Jan 2011 08:00:10 +0000 Subject: [erlang-questions] EPmail-0.1 released In-Reply-To: References: <7735F2E2-ACF9-4B68-80F3-1CCF83548CBD@gmail.com> Message-ID: Thank you. If i add gen_tcp:controlling_process(Sock, Pid), When i connect with second client i get exception that's it already started: =ERROR REPORT==== 29-Jan-2011::13:58:20 === Error in process <0.103.0> on node 'emacs@REDACTED' with exit value: {{badmatch,{error,{already_started,<0.105.0>}}},[{popd_listener,accept,1}]} Thank you. From dangud@REDACTED Sat Jan 29 09:24:14 2011 From: dangud@REDACTED (Dan Gudmundsson) Date: Sat, 29 Jan 2011 09:24:14 +0100 Subject: [erlang-questions] Re: Erlang and wxWidgets: how to help to improve the stability of wx (on Ubuntu)?!?! In-Reply-To: References: Message-ID: Hmm, I don't do much in the wx:new() part, can you do a: gdb PATH/bin/beam.smp core.ID > bt /Dan On Sat, Jan 29, 2011 at 8:03 AM, Boris M?hmer wrote: > 2011/1/27 Boris M?hmer : >> "wx:demo()" with a core dump. > > I just realized that the core dump can be generated just with "wx:new()."! > > The result ist the same. > > > ?- boris > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From max.lapshin@REDACTED Sat Jan 29 09:28:41 2011 From: max.lapshin@REDACTED (Max Lapshin) Date: Sat, 29 Jan 2011 11:28:41 +0300 Subject: [erlang-questions] EPmail-0.1 released In-Reply-To: References: <7735F2E2-ACF9-4B68-80F3-1CCF83548CBD@gmail.com> Message-ID: Of course! It is because You are starting second example of singleton =)) simple_one_for_one is what you need. From kuleshovmail@REDACTED Sat Jan 29 13:44:17 2011 From: kuleshovmail@REDACTED (Alexander Kuleshov) Date: Sat, 29 Jan 2011 12:44:17 +0000 Subject: [erlang-questions] EPmail-0.1 released In-Reply-To: References: <7735F2E2-ACF9-4B68-80F3-1CCF83548CBD@gmail.com> Message-ID: On Sat, Jan 29, 2011 at 8:28 AM, Max Lapshin wrote: > Of course! > > It is because You are starting second example of singleton =)) > > simple_one_for_one is what you need. > Hello Max, I set simple_one_for_one in: init([]) -> {ok, {{simple_one_for_one, 5, 60}, [{popd_listner, {popd_listener, start_link, []}, permanent, brutal_kill, worker, [popd_sup]}]}}. insteat one_for_one. And now client can not connect to server. From freza@REDACTED Sat Jan 29 15:14:02 2011 From: freza@REDACTED (Jachym Holecek) Date: Sat, 29 Jan 2011 14:14:02 +0000 Subject: [erlang-questions] EPmail-0.1 released In-Reply-To: References: Message-ID: <20110129141402.GA1487@hanele.lan> # Alexander Kuleshov 2011-01-29: > Thank you Johan for help. > > I try to supervisor start_child gen_gsm: > > accept(Socket) -> > case gen_tcp:accept(Socket) of > {ok, Sock} -> Pid = popd_sup:add_socket(Sock), ok = gen_tcp:controlling_process(Sock, Pid), popd_fsm:kick(Pid), accept(Socket); [...] Where popd_sup exports: add_socket(Sock) -> Child = {Sock, {popd_fsm, start_link, [Sock, [], []]}, temporary, 2000, worker, [popd_fsm]}, {ok, Pid} = supervisor:start_child(popd_sup, Child), Pid. Note unique child Id this time and more suitable restart strategy. I'd keep the supervisor one_for_one, but that's just because I don't like simple_one_for_one much for no clear reason. And popd_fsm has something like: kick(Pid) -> catch gen_fsm:send_event(Pid, kick), ok. [...] initial(kick, #state{sock = Sock} = State) -> ok = inet:setopts(Sock, [{active, once}]), ok = gen_tcp:send(Sock, "+OK POP3 server ready \r\n"), {next_state, whatever, State}; Besides fixing bugs that others already pointed out, I suppose my point is to encapsulate functionality into modules that actually provide it instead of scattering "lowlevel" calls (supervisor:start_child/X, gen_server:call/X, gen_fsm:send_event/X and such) all around the place. Anyway, this is just a quick sketch written without much thought... HTH, -- Jachym From michael.santos@REDACTED Sat Jan 29 17:27:21 2011 From: michael.santos@REDACTED (Michael Santos) Date: Sat, 29 Jan 2011 11:27:21 -0500 Subject: [erlang-questions] file:position/2 hangs up whole enode In-Reply-To: References: Message-ID: <20110129162721.GA32223@ecn.lan> On Thu, Jan 27, 2011 at 05:02:37PM +0200, Alex Morarash wrote: > Hi all! > > Just another way to completely hang up a whole erlang node: > open compressed file for read and move file position after > actual size of uncompressed data. > > Stable reproduce in at least from R12B5 till R14B1. > > Here is the example. > > The gzio driver is spinning on the end of the file. This patch just checks if the file offset changes after reading from the stream: $ touch test.gz $ bin/erl 1> {ok,H} = file:open("test.gz", [read,compressed]). {ok,<0.34.0>} 2> file:position(H,10). {ok,0} diff --git a/erts/emulator/drivers/common/gzio.c b/erts/emulator/drivers/common/gzio.c index 5531a27..2b33f74 100644 --- a/erts/emulator/drivers/common/gzio.c +++ b/erts/emulator/drivers/common/gzio.c @@ -633,4 +633,5 @@ erts_gzseek(gzFile file, int offset, int whence) char buf[512]; int n; + int cur = s->position; n = pos - s->position; @@ -644,4 +645,5 @@ erts_gzseek(gzFile file, int offset, int whence) erts_gzwrite(file, buf, n); } + if (cur == s->position) break; } From pablo.platt@REDACTED Sun Jan 30 00:01:59 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Sat, 29 Jan 2011 15:01:59 -0800 (PST) Subject: print data to the shell during common tests Message-ID: <108348.56369.qm@web112601.mail.gq1.yahoo.com> Hi Is it possible to print info to the shell during a test when using common tests? io:format/2 doesn't work because ct redirect standard output. I'm running tests using rebar: /rebar ct Thanks From garazdawi@REDACTED Sun Jan 30 01:39:33 2011 From: garazdawi@REDACTED (Lukas Larsson) Date: Sun, 30 Jan 2011 01:39:33 +0100 Subject: [erlang-questions] print data to the shell during common tests In-Reply-To: <108348.56369.qm@web112601.mail.gq1.yahoo.com> References: <108348.56369.qm@web112601.mail.gq1.yahoo.com> Message-ID: You can use either ct:print/3 or ct:pal/3 to print to the shell within a testcase in common_test. Lukas On Sun, Jan 30, 2011 at 12:01 AM, Pablo Platt wrote: > Hi > > Is it possible to print info to the shell during a test when using common > tests? > io:format/2 doesn't work because ct redirect standard output. > I'm running tests using rebar: > /rebar ct > > Thanks > > > From pablo.platt@REDACTED Sun Jan 30 02:12:12 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Sat, 29 Jan 2011 17:12:12 -0800 (PST) Subject: [erlang-questions] print data to the shell during common tests In-Reply-To: References: <108348.56369.qm@web112601.mail.gq1.yahoo.com> Message-ID: <262276.69597.qm@web112610.mail.gq1.yahoo.com> I didn't see the ct module at http://erldocs.com The edoc of ct/print/2 probably need to be corrected, I'll file a bug: print(X1, X2) -> term() Equivalent to print(Category, Format, Args). Now I can print to the log with ct:pal/3 but can't see it in the shell. ________________________________ From: Lukas Larsson To: Pablo Platt Cc: erlang-questions@REDACTED Sent: Sun, January 30, 2011 2:39:33 AM Subject: Re: [erlang-questions] print data to the shell during common tests You can use either ct:print/3 or ct:pal/3 to print to the shell within a testcase in common_test. Lukas On Sun, Jan 30, 2011 at 12:01 AM, Pablo Platt wrote: > Hi > > Is it possible to print info to the shell during a test when using common > tests? > io:format/2 doesn't work because ct redirect standard output. > I'm running tests using rebar: > /rebar ct > > Thanks > > > From taylor@REDACTED Sun Jan 30 04:01:58 2011 From: taylor@REDACTED (Taylor Venable) Date: Sat, 29 Jan 2011 22:01:58 -0500 Subject: edoc: infinite loop Message-ID: Hi there, the following seems to produce an infinite loop in edoc (version 0.7.6.8): %% START FILE %% @doc %% ``` %% text %% ''' -module(test). -export([foo/0]). foo() -> ok. %% END FILE Example usage: Erlang R14B01 (erts-5.8.2) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false] Eshell V5.8.2 (abort with ^G) 1> edoc:run([], ['test.erl'], [{dir, doc}, {hidden, true}, {private, true}]). Also, I found a thread from September 2009 indicating that edoc does not support UTF-8 documentation in files; is this still true? (The thread is http://erlang.2086793.n4.nabble.com/UTF8-and-EDoc-td2088002.html - I get an error similar to that of the OP when trying to use UTF-8 encoded documentation in my own files with the invocation for edoc:run above.) Thanks, -- Taylor C. Venable http://metasyntax.net/ From kuleshovmail@REDACTED Sun Jan 30 14:05:56 2011 From: kuleshovmail@REDACTED (Alexander Kuleshov) Date: Sun, 30 Jan 2011 13:05:56 +0000 Subject: [erlang-questions] EPmail-0.1 released In-Reply-To: References: <20110129141402.GA1487@hanele.lan> Message-ID: Hello, it's again i. I make Root supervisor that start new supervisor that start my popd_listener: Sup1 - https://gist.github.com/802847 Sup2 - https://gist.github.com/802847 But i can't to think how can i create simple_one_for_one gen_fsm workers. I need to create 3 supervisor? Thank you From kuleshovmail@REDACTED Sun Jan 30 14:57:29 2011 From: kuleshovmail@REDACTED (shk) Date: Sun, 30 Jan 2011 05:57:29 -0800 (PST) Subject: Erlang supervisor start_child Message-ID: <1296395849734-3247153.post@n4.nabble.com> Hello, I write pop3 server. My application: root supervisor (one_for_one) -> listener supervisor(one_for_one) -> fsm supervisor (simple one_for_one) and listener gen_server I run all supervisors it's ok. When i try to start_child of fsm supervisor 2 or more times i get error: {error,{already_started,<0.324.0>}} How can i correctly design my supervisors that i can run many gen_fsm child by fsm supervisor. All code: https://github.com/onlyshk/epmail Thank you. -- View this message in context: http://erlang.2086793.n4.nabble.com/Erlang-supervisor-start-child-tp3247153p3247153.html Sent from the Erlang Questions mailing list archive at Nabble.com. From serge@REDACTED Sun Jan 30 15:30:42 2011 From: serge@REDACTED (Serge Aleynikov) Date: Sun, 30 Jan 2011 09:30:42 -0500 Subject: [erlang-questions] EPmail-0.1 released In-Reply-To: References: <20110129141402.GA1487@hanele.lan> Message-ID: <4D457612.6040008@aleynikov.org> You can find an example here: http://www.trapexit.org/Building_a_Non-blocking_TCP_server_using_OTP_principles On 1/30/2011 8:05 AM, Alexander Kuleshov wrote: > Hello, it's again i. > > I make Root supervisor that start new supervisor that start my popd_listener: > > Sup1 - https://gist.github.com/802847 > Sup2 - https://gist.github.com/802847 > > But i can't to think how can i create simple_one_for_one gen_fsm > workers. I need to create 3 supervisor? > > Thank you > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From alexey.v.romanov@REDACTED Sun Jan 30 16:12:17 2011 From: alexey.v.romanov@REDACTED (Alexey Romanov) Date: Sun, 30 Jan 2011 18:12:17 +0300 Subject: [erlang-questions] Erlang supervisor start_child In-Reply-To: <1296395849734-3247153.post@n4.nabble.com> References: <1296395849734-3247153.post@n4.nabble.com> Message-ID: >From documentation of gen_fsm:start_link: If FsmName={local,Name}, the gen_fsm is registered locally as Name using register/2. If FsmName={global,GlobalName}, the gen_fsm is registered globally as GlobalName using global:register_name/2. If no name is provided, the gen_fsm is not registered. Since you can't have two processes with same registered name on a node, you need to provide different names (or no names at all). Yours, Alexey Romanov On Sun, Jan 30, 2011 at 4:57 PM, shk wrote: > > Hello, > > I write pop3 server. My application: > > ? ? ? root supervisor (one_for_one) ?-> listener supervisor(one_for_one) -> > fsm supervisor (simple one_for_one) and listener gen_server > > I run all supervisors it's ok. When i try to start_child of fsm supervisor 2 > or more times i get error: {error,{already_started,<0.324.0>}} > > How can i correctly design my supervisors that i can run many gen_fsm child > by fsm supervisor. > > All code: https://github.com/onlyshk/epmail > > Thank you. > -- > View this message in context: http://erlang.2086793.n4.nabble.com/Erlang-supervisor-start-child-tp3247153p3247153.html > Sent from the Erlang Questions mailing list archive at Nabble.com. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From kuleshovmail@REDACTED Sun Jan 30 17:30:38 2011 From: kuleshovmail@REDACTED (Alexander Kuleshov) Date: Sun, 30 Jan 2011 16:30:38 +0000 Subject: [erlang-questions] Erlang supervisor start_child In-Reply-To: References: <1296395849734-3247153.post@n4.nabble.com> Message-ID: On Sun, Jan 30, 2011 at 4:13 PM, Alexey Romanov wrote: > Yours, Alexey Romanov > > > > On Sun, Jan 30, 2011 at 7:05 PM, Alexander Kuleshov > wrote: >>>Since you can't have two processes with same registered name on a >>>node, you need to provide different names (or no names at all). >> >> Hello, >> >> If i havent gen_fsm name, how can i gen_fsm:send_event to this gen_fsm? >> > > By the pid (which you can get from supervisor:which_children). > Thank you for reply. In popd_listener i try: pop_fsm_sup:start_child(Sock, [], []), [{_,_, Child, _}] = supervisor:which_children(pop_fsm_sup), When i connect with first client it's ok, but when i try to connect with second client i see error: Error in process <0.815.0> on node 'emacs@REDACTED' with exit value: {{badmatch,[{undefined,<0.818.0>,worker,[popd_fsm]},{undefined,<0.819.0>,worker,[popd_fsm]}]},[{popd_listener,accept,1}]} From alexey.v.romanov@REDACTED Sun Jan 30 17:37:09 2011 From: alexey.v.romanov@REDACTED (Alexey Romanov) Date: Sun, 30 Jan 2011 19:37:09 +0300 Subject: [erlang-questions] Erlang supervisor start_child In-Reply-To: References: <1296395849734-3247153.post@n4.nabble.com> Message-ID: [{_,_, Child, _}] = ... assumes that the right side only has one child. Yours, Alexey Romanov On Sun, Jan 30, 2011 at 7:30 PM, Alexander Kuleshov wrote: > On Sun, Jan 30, 2011 at 4:13 PM, Alexey Romanov > wrote: >> Yours, Alexey Romanov >> >> >> >> On Sun, Jan 30, 2011 at 7:05 PM, Alexander Kuleshov >> wrote: >>>>Since you can't have two processes with same registered name on a >>>>node, you need to provide different names (or no names at all). >>> >>> Hello, >>> >>> If i havent gen_fsm name, how can i gen_fsm:send_event to this gen_fsm? >>> >> >> By the pid (which you can get from supervisor:which_children). >> > > Thank you for reply. > > In popd_listener i try: > > pop_fsm_sup:start_child(Sock, [], []), > [{_,_, Child, _}] = supervisor:which_children(pop_fsm_sup), > > When i connect with first client it's ok, but when i try to connect > with second client i see error: > > Error in process <0.815.0> on node 'emacs@REDACTED' with exit value: > {{badmatch,[{undefined,<0.818.0>,worker,[popd_fsm]},{undefined,<0.819.0>,worker,[popd_fsm]}]},[{popd_listener,accept,1}]} > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From kuleshovmail@REDACTED Sun Jan 30 17:38:27 2011 From: kuleshovmail@REDACTED (Alexander Kuleshov) Date: Sun, 30 Jan 2011 16:38:27 +0000 Subject: [erlang-questions] Erlang supervisor start_child In-Reply-To: References: <1296395849734-3247153.post@n4.nabble.com> Message-ID: On Sun, Jan 30, 2011 at 4:37 PM, Alexey Romanov wrote: > [{_,_, Child, _}] = ... > > assumes that the right side only has one child. > > > Yours, Alexey Romanov > > > > On Sun, Jan 30, 2011 at 7:30 PM, Alexander Kuleshov > wrote: >> On Sun, Jan 30, 2011 at 4:13 PM, Alexey Romanov >> wrote: >>> Yours, Alexey Romanov >>> >>> >>> >>> On Sun, Jan 30, 2011 at 7:05 PM, Alexander Kuleshov >>> wrote: >>>>>Since you can't have two processes with same registered name on a >>>>>node, you need to provide different names (or no names at all). >>>> >>>> Hello, >>>> >>>> If i havent gen_fsm name, how can i gen_fsm:send_event to this gen_fsm? >>>> >>> >>> By the pid (which you can get from supervisor:which_children). >>> >> >> Thank you for reply. >> >> In popd_listener i try: >> >> pop_fsm_sup:start_child(Sock, [], []), >> [{_,_, Child, _}] = supervisor:which_children(pop_fsm_sup), >> >> When i connect with first client it's ok, but when i try to connect >> with second client i see error: >> >> Error in process <0.815.0> on node 'emacs@REDACTED' with exit value: >> {{badmatch,[{undefined,<0.818.0>,worker,[popd_fsm]},{undefined,<0.819.0>,worker,[popd_fsm]}]},[{popd_listener,accept,1}]} >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> >> > Now all works. From pablo.platt@REDACTED Sun Jan 30 22:06:49 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Sun, 30 Jan 2011 13:06:49 -0800 (PST) Subject: use a tcp socket server in common tests Message-ID: <587947.5519.qm@web112607.mail.gq1.yahoo.com> I'm trying to write a test for a tcp client. I've created a fake tcp server that send predefined packets. I can't run the server in the init_per_suite of the test. When I start the server from the shell and run the test or when I run a module that run the server and execute the same actions like in the test it works. I'm getting two types of errors: 1. timeout when calling the server. 2. {error,eaddrinuse} in the second time running the test because the end_per_suite didn't have chance to stop the server. Is there a different way I should start the tcp server in the common test? Is there a way to execute end_per_suite even if the test fails? A simplified version of the code I'm using: common test ---------------- init_per_suite(Config) -> myserver:start_link(?PORT), gen_fsm:send_event(myserver, accept). {ok, Conn} = myclient:start_link(?HOST, ?FAKEPORT), unlink(Conn), [{conn, Conn} | Config]. test(Config) -> Conn = ?config(conn, Config), Packet = <<1,0,0,0>>, <<2,0,0,0>> = gen_server:call(Conn, {packet, Packet}). server -------------- start_link(Port) -> gen_fsm:start_link({local, ?MODULE}, ?MODULE, Port, []). init(Port) -> {ok, LSocket} = gen_tcp:listen(Port, [binary, {active, true}]), {ok, waiting, LSocket}. waiting(accept, LSocket) -> {ok, _Socket} = gen_tcp:accept(LSocket), {next_state, waiting, LSocket}. handle_info({tcp, Socket, Data}, StateName, State) -> Resp = <<2,0,0,0>>, gen_tcp:send(Socket, Resp), {next_state, StateName, State}. client --------------- start_link(Host, Port) -> gen_server:start_link(?MODULE, [Host, Port], []). init([Host, Port]) -> {ok, Socket} = gen_tcp:connect(Host, Port, [binary, {active, true}]) {ok, {Socket, from}}. handle_call({packet, Packet}, From,{Socket, from}) -> gen_tcp:send(Socket, Packet), {noreply, {Socket, From}}. handle_info({tcp, _Socket, Data}, {Socket, From}) -> gen_server:reply(From, Data), {noreply, NewState}. From orionqwest@REDACTED Sun Jan 30 23:38:22 2011 From: orionqwest@REDACTED (Nav) Date: Mon, 31 Jan 2011 00:38:22 +0200 Subject: TCP flow control problem Message-ID: I have implemented two TCP servers using gen_fsm, and gen_server, respectively. I am using {active, once} option, after getting each message. When I try to send data from one side to another in a loop, the data gets corrupted (first 1 or 2 message arrive ok, but following gets corrupted) when it arrives at other side. However, if I put a timer:sleep(1000) before sending each message, then everything seems to go fast. What seems to go wrong by sending data too fast? Do I need to set some more socket options? Thanks, Nav From kaellis@REDACTED Mon Jan 31 03:48:04 2011 From: kaellis@REDACTED (Ken Ellis) Date: Sun, 30 Jan 2011 21:48:04 -0500 Subject: Broken error reporting with ODBC app, w/ patch Message-ID: The ODBC app does not report informative error messages for connection issues. It always reports "No SQL-driver information available", which is misleading since something as simple as attempting to connect to a database that does not exist will return this error. This is due to a bad handle being passed in db_connect in odbcserver.c -- incorrectly as a statement handle instead of a db connection handle. See patch below. With the patch you get: {error,"[unixODBC]Sending the authentication packet failed;\nCould not connect to remote server. Connection to database failed."} or {error,"[unixODBC][Driver Manager]Data source name not found, and no default driver specified Connection to database failed."} In the current version both of the above errors would result in the same message: {error,"No SQL-driver information available. Connection to database failed."} A google search suggests a reasonable amount of time could be saved for those trying to use this package: http://www.google.com/search?q=erlang+odbc+%22No+SQL-driver+information+available%22&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a -Ken diff otp_src_R14B01/lib/odbc/c_src/odbcserver.c otp_src_R14B01_working/lib/odbc/c_src/odbcserver.c 475c475 < diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); --- > diagnos = get_diagnos(SQL_HANDLE_DBC, connection_handle(state)); From zabrane3@REDACTED Mon Jan 31 08:05:31 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Mon, 31 Jan 2011 08:05:31 +0100 Subject: Android Tablet and Erlang (+Java) In-Reply-To: References: <1296395849734-3247153.post@n4.nabble.com> Message-ID: <01F08CDE-681F-4C02-A8AB-9470435B7F33@gmail.com> Does someone knows a good "Android tablet" than can be used for small Erlang development? From bengt.kleberg@REDACTED Mon Jan 31 08:49:15 2011 From: bengt.kleberg@REDACTED (Bengt Kleberg) Date: Mon, 31 Jan 2011 08:49:15 +0100 Subject: [erlang-questions] TCP flow control problem In-Reply-To: References: Message-ID: <1296460155.5188.3.camel@seasc1137> Greetings, TCP is a stream protocol. It does not preserve message boundaries. You have to handle that yourself. Sleeping one second is one way. When you do not sleep, how do you separate the messages? bengt On Sun, 2011-01-30 at 23:38 +0100, Nav wrote: > I have implemented two TCP servers using gen_fsm, and gen_server, respectively. > I am using {active, once} option, after getting each message. > > When I try to send data from one side to another in a loop, the data > gets corrupted (first 1 or 2 message arrive ok, but following gets > corrupted) when it arrives at other side. > > However, if I put a timer:sleep(1000) before sending each message, > then everything seems to go fast. > > What seems to go wrong by sending data too fast? Do I need to set some > more socket options? > > Thanks, Nav > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From torben.lehoff@REDACTED Mon Jan 31 09:17:14 2011 From: torben.lehoff@REDACTED (Torben Hoffmann) Date: Mon, 31 Jan 2011 09:17:14 +0100 Subject: [erlang-questions] EPmail-0.1 released In-Reply-To: References: <7735F2E2-ACF9-4B68-80F3-1CCF83548CBD@gmail.com> Message-ID: >From the supervisor documentation: *Important note on simple-one-for-one supervisors:* The dynamically created child processes of a simple-one-for-one supervisor are not explicitly killed, regardless of shutdown strategy, but are expected to terminate when the supervisor does (that is, when an exit signal from the parent process is received). But as long as you stay with OTP components the child will have this behaviour by default (as I read the manual). I had an issue where the simple_one_for_one did not work, but I cannot remember the details (sorry), it was just so painful that I have never used the simple_one_for_one again but instead implemented a function my_child_spec/* that returns a child spec that suits the problem at hand and the started the child with supervisor:start_child/2. One thing to note about the simple_one_for_one supervisors is that the children do not get an id, so when you call supervisor:which_children/1 you cannot tell them apart very easily. I tend to give the children an id that makes it easy for me to debug a running system, e.g., by putting something like a call reference in the id term. That makes it easy to figure out with pid a specific process has and then call sys:get_status/1 to get further information. Without an id for the children it becomes guess work. Cheers, Torben On Sat, Jan 29, 2011 at 09:28, Max Lapshin wrote: > Of course! > > It is because You are starting second example of singleton =)) > > simple_one_for_one is what you need. > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > -- http://www.linkedin.com/in/torbenhoffmann From tuxofil@REDACTED Mon Jan 31 10:23:23 2011 From: tuxofil@REDACTED (Alex Morarash) Date: Mon, 31 Jan 2011 11:23:23 +0200 Subject: [erlang-questions] file:position/2 hangs up whole enode In-Reply-To: <20110129162721.GA32223@ecn.lan> References: <20110129162721.GA32223@ecn.lan> Message-ID: Thanks! I applied this patch. Works fine. IMHO, need to post this to upstream. On 29 January 2011 18:27, Michael Santos wrote: > On Thu, Jan 27, 2011 at 05:02:37PM +0200, Alex Morarash wrote: >> Hi all! >> >> Just another way to completely hang up a whole erlang node: >> open compressed file for read and move file position after >> actual size of uncompressed data. >> >> Stable reproduce in at least from R12B5 till R14B1. >> >> Here is the example. >> >> > The gzio driver is spinning on the end of the file. This patch just > checks if the file offset changes after reading from the stream: > > $ touch test.gz > $ bin/erl > 1> {ok,H} = file:open("test.gz", [read,compressed]). > {ok,<0.34.0>} > 2> file:position(H,10). > {ok,0} > > > diff --git a/erts/emulator/drivers/common/gzio.c b/erts/emulator/drivers/common/gzio.c > index 5531a27..2b33f74 100644 > --- a/erts/emulator/drivers/common/gzio.c > +++ b/erts/emulator/drivers/common/gzio.c > @@ -633,4 +633,5 @@ erts_gzseek(gzFile file, int offset, int whence) > ? ? ? ?char buf[512]; > ? ? ? ?int n; > + ? ? ? int cur = s->position; > > ? ? ? ?n = pos - s->position; > @@ -644,4 +645,5 @@ erts_gzseek(gzFile file, int offset, int whence) > ? ? ? ? ? ?erts_gzwrite(file, buf, n); > ? ? ? ?} > + ? ? ? if (cur == s->position) break; > ? ? } > > > From zvi.avraham@REDACTED Mon Jan 31 13:20:28 2011 From: zvi.avraham@REDACTED (Zvi) Date: Mon, 31 Jan 2011 04:20:28 -0800 (PST) Subject: Erlang number of processes limit (erl +P XXXX) Message-ID: <2834f6fd-3d81-4329-978f-f7e460c8a6bb@v31g2000vbs.googlegroups.com> Hi, Is number of processes limit is per node (i.e. single beam process) or cluster of node (in case of distributed erlang)? I hope it's per node, otherwise why each pid has 3 indices/components? thanks in advance, Zvi From mihai@REDACTED Mon Jan 31 13:39:32 2011 From: mihai@REDACTED (Mihai Balea) Date: Mon, 31 Jan 2011 07:39:32 -0500 Subject: [erlang-questions] Erlang number of processes limit (erl +P XXXX) In-Reply-To: <2834f6fd-3d81-4329-978f-f7e460c8a6bb@v31g2000vbs.googlegroups.com> References: <2834f6fd-3d81-4329-978f-f7e460c8a6bb@v31g2000vbs.googlegroups.com> Message-ID: <14731371-6580-4691-A539-B323D8772744@hates.ms> On Jan 31, 2011, at 7:20 AM, Zvi wrote: > Hi, > Is number of processes limit is per node (i.e. single beam process) or > cluster of node (in case of distributed erlang)? The limit is per beam process. Mihai From garazdawi@REDACTED Mon Jan 31 16:05:03 2011 From: garazdawi@REDACTED (Lukas Larsson) Date: Mon, 31 Jan 2011 15:05:03 +0000 Subject: [erlang-questions] use a tcp socket server in common tests In-Reply-To: <587947.5519.qm@web112607.mail.gq1.yahoo.com> References: <587947.5519.qm@web112607.mail.gq1.yahoo.com> Message-ID: Hi! You should not have to do anything special to get common_test to test a TCP server. The eaddrinsure sound like it might be because you did not put the reuseaddr option in your gen_tcp:listen call. The timeout for the server might be because the process calling init_per_suite has died and because you use spawn_link when starting the server your server process died as well. Try doing it without linking and see if that works better. Lukas ** On Sun, Jan 30, 2011 at 9:06 PM, Pablo Platt wrote: > I'm trying to write a test for a tcp client. > I've created a fake tcp server that send predefined packets. > I can't run the server in the init_per_suite of the test. > When I start the server from the shell and run the test > or when I run a module that run the server and execute the same actions > like in > the test it works. > > I'm getting two types of errors: > 1. timeout when calling the server. > 2. {error,eaddrinuse} in the second time running the test because the > end_per_suite didn't have chance to stop the server. > > Is there a different way I should start the tcp server in the common test? > Is there a way to execute end_per_suite even if the test fails? > > A simplified version of the code I'm using: > > common test > ---------------- > init_per_suite(Config) -> > myserver:start_link(?PORT), > gen_fsm:send_event(myserver, accept). > {ok, Conn} = myclient:start_link(?HOST, ?FAKEPORT), > unlink(Conn), > [{conn, Conn} | Config]. > > test(Config) -> > Conn = ?config(conn, Config), > Packet = <<1,0,0,0>>, > <<2,0,0,0>> = gen_server:call(Conn, {packet, Packet}). > > server > -------------- > start_link(Port) -> > gen_fsm:start_link({local, ?MODULE}, ?MODULE, Port, []). > > init(Port) -> > {ok, LSocket} = gen_tcp:listen(Port, [binary, {active, true}]), > {ok, waiting, LSocket}. > > waiting(accept, LSocket) -> > {ok, _Socket} = gen_tcp:accept(LSocket), > {next_state, waiting, LSocket}. > > handle_info({tcp, Socket, Data}, StateName, State) -> > Resp = <<2,0,0,0>>, > gen_tcp:send(Socket, Resp), > {next_state, StateName, State}. > > client > --------------- > start_link(Host, Port) -> > gen_server:start_link(?MODULE, [Host, Port], []). > > init([Host, Port]) -> > {ok, Socket} = gen_tcp:connect(Host, Port, [binary, {active, true}]) > {ok, {Socket, from}}. > > handle_call({packet, Packet}, From,{Socket, from}) -> > gen_tcp:send(Socket, Packet), > {noreply, {Socket, From}}. > > handle_info({tcp, _Socket, Data}, {Socket, From}) -> > gen_server:reply(From, Data), > {noreply, NewState}. > > > From garazdawi@REDACTED Mon Jan 31 16:10:01 2011 From: garazdawi@REDACTED (Lukas Larsson) Date: Mon, 31 Jan 2011 15:10:01 +0000 Subject: [erlang-questions] Android Tablet and Erlang (+Java) In-Reply-To: <01F08CDE-681F-4C02-A8AB-9470435B7F33@gmail.com> References: <1296395849734-3247153.post@n4.nabble.com> <01F08CDE-681F-4C02-A8AB-9470435B7F33@gmail.com> Message-ID: Hi! A colleague of mine has gotten Erlang to work on the Samsung Galaxy tablet. Don't know if it is better or worse than any else, but it works anyways. Lukas On Mon, Jan 31, 2011 at 7:05 AM, Zabrane Mickael wrote: > Does someone knows a good "Android tablet" than can be used for small > Erlang development? > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From zabrane3@REDACTED Mon Jan 31 16:19:50 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Mon, 31 Jan 2011 16:19:50 +0100 Subject: [erlang-questions] Android Tablet and Erlang (+Java) In-Reply-To: References: <1296395849734-3247153.post@n4.nabble.com> <01F08CDE-681F-4C02-A8AB-9470435B7F33@gmail.com> Message-ID: <89EA73CE-11FC-4D23-88C9-20EC52820632@gmail.com> Thanks Lucas. Any other experiences guys? Le 31 janv. 2011 ? 16:10, Lukas Larsson a ?crit : > Hi! > > A colleague of mine has gotten Erlang to work on the Samsung Galaxy tablet. > Don't know if it is better or worse than any else, but it works anyways. > > Lukas From alain.odea@REDACTED Mon Jan 31 18:06:54 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Mon, 31 Jan 2011 13:36:54 -0330 Subject: [erlang-questions] Broken error reporting with ODBC app, w/ patch In-Reply-To: References: Message-ID: <82788BBB-A34D-4993-BCCD-0D92A3438054@gmail.com> On 2011-01-30, at 23:18, Ken Ellis wrote: > The ODBC app does not report informative error messages for connection > issues. It always reports "No SQL-driver information available", > which is misleading since something as simple as attempting to connect > to a database that does not exist will return this error. This is due > to a bad handle being passed in db_connect in odbcserver.c -- > incorrectly as a statement handle instead of a db connection handle. > See patch below. > > With the patch you get: > {error,"[unixODBC]Sending the authentication packet failed;\nCould not > connect to remote server. Connection to database failed."} > or > {error,"[unixODBC][Driver Manager]Data source name not found, and no > default driver specified Connection to database failed."} > > In the current version both of the above errors would result in the > same message: > {error,"No SQL-driver information available. Connection to database failed."} > > A google search suggests a reasonable amount of time could be saved > for those trying to use this package: > http://www.google.com/search?q=erlang+odbc+%22No+SQL-driver+information+available%22&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a > > -Ken > > > diff otp_src_R14B01/lib/odbc/c_src/odbcserver.c > otp_src_R14B01_working/lib/odbc/c_src/odbcserver.c > 475c475 > < diagnos = get_diagnos(SQL_HANDLE_STMT, statement_handle(state)); > --- >> diagnos = get_diagnos(SQL_HANDLE_DBC, connection_handle(state)); > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > Better error messages for the win :) Thank you for posting this Ken! You should send your patch on to erlang-patches@REDACTED for integration into OTP. For more information on contributing to OTP see: https://github.com/erlang/otp/wiki/submitting-patches Thanks again, Alain From kuleshovmail@REDACTED Mon Jan 31 18:22:46 2011 From: kuleshovmail@REDACTED (shk) Date: Mon, 31 Jan 2011 09:22:46 -0800 (PST) Subject: Erlang crossplatform Message-ID: <1296494566167-3249229.post@n4.nabble.com> Hello, In future i'll plan to port my erlang application in MS Windows. And i want to ask question: Erlang is corssplatform in full? Is anything in Erlang which not supporting Windows? Thank you. -- View this message in context: http://erlang.2086793.n4.nabble.com/Erlang-crossplatform-tp3249229p3249229.html Sent from the Erlang Questions mailing list archive at Nabble.com. From alain.odea@REDACTED Mon Jan 31 18:27:20 2011 From: alain.odea@REDACTED (Alain O'Dea) Date: Mon, 31 Jan 2011 13:57:20 -0330 Subject: [erlang-questions] TCP flow control problem In-Reply-To: <1296460155.5188.3.camel@seasc1137> References: <1296460155.5188.3.camel@seasc1137> Message-ID: Bengt is right. I suggest using a gen_server to collect the data and look for message boundaries. As it collects data it would prepend it to an iodata() in its own State structure. When it encounters a message boundary it would cast the data in its State plus the data up to and including the message boundary onto a delegate gen_server for processing. The data in the packet after the message boundary would then become the new partial message data in it's State. Below is pseudo-code for this: -record(state, {partial_message :: iodata(), delegate :: pid()}). handle_info({tcp, {data, Data}}, _From, #state{partial_message = Partial} = State) -> NextPartial = case boundary_split(Data) of {MsgEnd, NextMsg} -> tcp_message_handler:handle_message(State#state.delegate, [MsgEnd | Partial]), NextMsg; Data -> [Data | Partial] end, {noreply, State#state{partial_message = NextPartial}. On 2011-01-31, at 4:19, Bengt Kleberg wrote: > Greetings, > > TCP is a stream protocol. It does not preserve message boundaries. You > have to handle that yourself. Sleeping one second is one way. When you > do not sleep, how do you separate the messages? > > > bengt > > On Sun, 2011-01-30 at 23:38 +0100, Nav wrote: >> I have implemented two TCP servers using gen_fsm, and gen_server, respectively. >> I am using {active, once} option, after getting each message. >> >> When I try to send data from one side to another in a loop, the data >> gets corrupted (first 1 or 2 message arrive ok, but following gets >> corrupted) when it arrives at other side. >> >> However, if I put a timer:sleep(1000) before sending each message, >> then everything seems to go fast. >> >> What seems to go wrong by sending data too fast? Do I need to set some >> more socket options? >> >> Thanks, Nav >> >> ________________________________________________________________ >> erlang-questions (at) erlang.org mailing list. >> See http://www.erlang.org/faq.html >> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED >> > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > From fhamilton@REDACTED Mon Jan 31 18:58:37 2011 From: fhamilton@REDACTED (Felix Hamilton) Date: Mon, 31 Jan 2011 09:58:37 -0800 Subject: [erlang-questions] Android Tablet and Erlang (+Java) In-Reply-To: <89EA73CE-11FC-4D23-88C9-20EC52820632@gmail.com> References: <1296395849734-3247153.post@n4.nabble.com> <01F08CDE-681F-4C02-A8AB-9470435B7F33@gmail.com> <89EA73CE-11FC-4D23-88C9-20EC52820632@gmail.com> Message-ID: Recently got Ubuntu running on an Archos 10.1 'android tablet'. Havent actually tried to get erlang compiled natively on an android device, however ... On Mon, Jan 31, 2011 at 7:19 AM, Zabrane Mickael wrote: > Thanks Lucas. Any ?other experiences guys? > > Le 31 janv. 2011 ? 16:10, Lukas Larsson a ?crit : > >> Hi! >> >> A colleague of mine has gotten Erlang to work on the Samsung Galaxy tablet. >> Don't know if it is better or worse than any else, but it works anyways. >> >> Lukas > > > ________________________________________________________________ > erlang-questions (at) erlang.org mailing list. > See http://www.erlang.org/faq.html > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED > > From seth@REDACTED Mon Jan 31 19:27:54 2011 From: seth@REDACTED (Seth Falcon) Date: Mon, 31 Jan 2011 10:27:54 -0800 Subject: [erlang-questions] public_key:pem_decode/1 and public_key:verify/4 In-Reply-To: References: <4D36ADAC.3070509@tail-f.com> <9612AF2F-A145-45C3-9BD1-11ACE66BAD3E@lightpole.net> <4D370678.60203@tail-f.com> Message-ID: Hi Ingela, I've updated my patch: https://github.com/seth/otp/tree/sf/rsa_pub_key it now includes basic tests for decoding and encoding both RSA and DSA public keys in the SubjectPublicKeyInfo format. After moving all lib/FOO/test directories with references to wx, I was able to run the tests for public_key on my OS X laptop :-) Here's the summary of where I ended up: {ok, RSAPubPem} = file:read_file(filename:join(Datadir, "rsa_pub.pem")), [{'SubjectPublicKeyInfo', _, _} = PubEntry1] = public_key:pem_decode(RSAPubPem), RSAPubKey = public_key:pem_entry_decode(PubEntry1), true = check_entry_type(RSAPubKey, 'RSAPublicKey'), PubEntry1 = public_key:pem_entry_encode('RSAPublicKey', RSAPubKey), Notes: - I haven't merged in the trivial patch from Joakim to allow reading RSA public keys in "unwrapped" form where the PEM header has "RSA Public Key". - RSA public key records get encoded into the SubjectPublicKeyInfo wrapper form. So to create the form that Joakim is dealing with, one would have to use der_encode directly I believe. - When encoding RSAPublic keys, there is a parameter spot where it seems the right thing to put is <<5, 0>>, but I wanted to mention this in case I'm wrong about this being the right value for all RSA public keys in SPKI form. Let me know what else I can do to help get this patch into the public_key module :-) Best, + seth -- Seth Falcon | @sfalcon | http://userprimary.net/ From zabrane3@REDACTED Mon Jan 31 20:05:15 2011 From: zabrane3@REDACTED (Zabrane Mickael) Date: Mon, 31 Jan 2011 20:05:15 +0100 Subject: [erlang-questions] Android Tablet and Erlang (+Java) In-Reply-To: References: <1296395849734-3247153.post@n4.nabble.com> <01F08CDE-681F-4C02-A8AB-9470435B7F33@gmail.com> <89EA73CE-11FC-4D23-88C9-20EC52820632@gmail.com> Message-ID: <34072B4B-57BD-4C55-9CCD-2E565013CA50@gmail.com> Le 31 janv. 2011 ? 18:58, Felix Hamilton a ?crit : > Recently got Ubuntu running on an Archos 10.1 'android tablet'. Havent > actually tried to get erlang compiled natively on an android device, > however ... Cool. Thanks Felix! From pablo.platt@REDACTED Mon Jan 31 23:56:49 2011 From: pablo.platt@REDACTED (Pablo Platt) Date: Mon, 31 Jan 2011 14:56:49 -0800 (PST) Subject: [erlang-questions] use a tcp socket server in common tests In-Reply-To: References: <587947.5519.qm@web112607.mail.gq1.yahoo.com> Message-ID: <789021.73768.qm@web112613.mail.gq1.yahoo.com> reuseaddr fixed the first problem. start instead of start_link fixed the second. I've learned two new things. Thanks. ________________________________ From: Lukas Larsson To: Pablo Platt Cc: erlang-questions@REDACTED Sent: Mon, January 31, 2011 5:05:03 PM Subject: Re: [erlang-questions] use a tcp socket server in common tests Hi! You should not have to do anything special to get common_test to test a TCP server. The eaddrinsure sound like it might be because you did not put the reuseaddr option in your gen_tcp:listen call. The timeout for the server might be because the process calling init_per_suite has died and because you use spawn_link when starting the server your server process died as well. Try doing it without linking and see if that works better. Lukas On Sun, Jan 30, 2011 at 9:06 PM, Pablo Platt wrote: I'm trying to write a test for a tcp client. >I've created a fake tcp server that send predefined packets. >I can't run the server in the init_per_suite of the test. >When I start the server from the shell and run the test >or when I run a module that run the server and execute the same actions like in >the test it works. > >I'm getting two types of errors: >1. timeout when calling the server. >2. {error,eaddrinuse} in the second time running the test because the >end_per_suite didn't have chance to stop the server. > >Is there a different way I should start the tcp server in the common test? >Is there a way to execute end_per_suite even if the test fails? > >A simplified version of the code I'm using: > >common test >---------------- >init_per_suite(Config) -> > myserver:start_link(?PORT), > gen_fsm:send_event(myserver, accept). > {ok, Conn} = myclient:start_link(?HOST, ?FAKEPORT), > unlink(Conn), > [{conn, Conn} | Config]. > >test(Config) -> > Conn = ?config(conn, Config), > Packet = <<1,0,0,0>>, > <<2,0,0,0>> = gen_server:call(Conn, {packet, Packet}). > >server >-------------- >start_link(Port) -> > gen_fsm:start_link({local, ?MODULE}, ?MODULE, Port, []). > >init(Port) -> > {ok, LSocket} = gen_tcp:listen(Port, [binary, {active, true}]), > {ok, waiting, LSocket}. > >waiting(accept, LSocket) -> > {ok, _Socket} = gen_tcp:accept(LSocket), > {next_state, waiting, LSocket}. > >handle_info({tcp, Socket, Data}, StateName, State) -> > Resp = <<2,0,0,0>>, > gen_tcp:send(Socket, Resp), > {next_state, StateName, State}. > >client >--------------- >start_link(Host, Port) -> > gen_server:start_link(?MODULE, [Host, Port], []). > >init([Host, Port]) -> > {ok, Socket} = gen_tcp:connect(Host, Port, [binary, {active, true}]) > {ok, {Socket, from}}. > >handle_call({packet, Packet}, From,{Socket, from}) -> > gen_tcp:send(Socket, Packet), > {noreply, {Socket, From}}. > >handle_info({tcp, _Socket, Data}, {Socket, From}) -> > gen_server:reply(From, Data), > {noreply, NewState}. > > >