digest format is unreadable
Jacob Perkins
japerk@REDACTED
Tue Jun 9 18:17:18 CEST 2009
Hi, not sure if there's a better place to send this, so hopefully it gets to
the right person: The new digest email format is practically unusable.
Messages are only separated by a few lines of whitespace and theres no
separation between message topics. The top part of the email lists all the
messages with IDs, which could work if the IDs were listed with each message
below, but they aren't. I've included the latest digest below so you can see
for yourself. </endrant>
It'd be great if the format was something like googlegroups digests. I'd
really appreciate if someone could fix this, at least make it like it was
before.
Thanks,
Jacob
---------- Forwarded message ----------
From: <erlang-questions-digest-help@REDACTED>
Date: 2009/6/9
Subject: erlang-questions Digest 9 Jun 2009 16:03:22 -0000 Issue 17
To: erlang-questions@REDACTED
erlang-questions Digest 9 Jun 2009 16:03:22 -0000 Issue 17
Topics (messages 44436 through 44464):
Lightweight HTTP client
44436 by: Oscar Hellström
Request for enhancement: Sparse files
44437 by: Ville Silventoinen
44454 by: Alex Arnon
44456 by: Ville Silventoinen
import/1
44438 by: Joel Reymont
44439 by: Robert Virding
44440 by: Joel Reymont
44441 by: Kostis Sagonas
44442 by: Joel Reymont
44443 by: Maria Christakis
http:request issue
44444 by: Joel Reymont
44445 by: Joel Reymont
44446 by: Joel Reymont
When won't mnesia:restore/2 work?
44447 by: Sergey Samokhin
What is the best way to detect the conflict of mnesia schemas?
44448 by: Sergey Samokhin
http client and keep_alive
44449 by: Joel Reymont
44460 by: Adriano Bonat
NonBlocking TCP Server
44450 by: Allan Merolla
Mnesia -- "equals ignore case" equivalent?
44451 by: Steve Davis
44452 by: Christian
44453 by: Ulf Wiger
44455 by: Steve Davis
Erlang ODBC & Unicode
44457 by: Juhani Ränkimies
pipe messages sent between different nodes to boost speed
44458 by: Roberto Ostinelli
44464 by: Davide Marquês
Sending UDP messages without IP
44459 by: Lucas Robsahm
44461 by: David Sveningsson
Versioned variable names
44462 by: Attila Rajmund Nohl
44463 by: Paul Mineiro
Administrivia:
To subscribe to the digest, e-mail:
<erlang-questions-digest-subscribe@REDACTED>
To unsubscribe from the digest, e-mail:
<erlang-questions-digest-unsubscribe@REDACTED>
To post to the list, e-mail:
<erlang-questions@REDACTED>
----------------------------------------------------------------------
Hi All,
There has been a lot of talk about issues with HTTP clients in Erlang.
At ETC we developed a simple HTTP/1.1 client that doesn't have the
problems we've noticed in inets/ibrowse. We would now like to share this
with the community :) The client currently has some limitations, but the
code is quite simple and it shouldn't be hard to add the features you
need. You can find information on our open source page,
http://www.erlang-consulting.com/erlang/opensource.html, and on
bitbucket, http://bitbucket.org/etc/lhttpc/. You can report any issues
in the bitbucket tracker, and you can also send questions to me.
Best regards
--
Oscar Hellström, oscar@REDACTED
Office: +44 20 7655 0337
Mobile: +44 798 45 44 773
Erlang Training and Consulting Ltd
http://www.erlang-consulting.com/
Hi,
I've written a directory scanner in Erlang that calculates usages per
Unix user and group. It works well, except that the results are wrong
when it encounters sparse files (holes in the files):
http://en.wikipedia.org/wiki/Sparse_file. We have some users that have
files that seem to be 1-2 terabytes, but in reality occupy less than a
gigabyte on disk.
The C stat struct has two fields I'd need (shown by "stat" command in Unix):
blksize_t st_blksize; /* optimal I/O size */
blkcnt_t st_blocks; /*allocated 512-byte blocks */
If st_size > st_blocks * 512, the file is sparse. Unfortunately, the
read_file_info/read_link_info doesn't provide the blocks information.
Any chance this could be included in some future Erlang release?
Also, any chance file:copy would support copying sparse files? :-) I
tested, the target file becomes non-sparse.
Erlang has been great help in our environment, where simple rsync has
become too slow...
Thanks,
Ville
P.S. I tried to add blocks information to my R13B Erlang environment,
but I broke the build system somehow (something to do with the fact
that prim_file is preloaded? "make preloaded" got me a bit further,
but I gave up when compile:compile/3 became undef). Below are the
changes I made to otp_src_R13B sources:
# diff efile_drv.c efile_drv.c.original
1823,1827c1823
< put_int32(d->info.block_size, &resbuf[1 + (29 * 4)]);
< put_int32(d->info.blocks_high, &resbuf[1 + (30 * 4)]);
< put_int32(d->info.blocks_low, &resbuf[1 + (31 * 4)]);
<
< #define RESULT_SIZE (1 + (32 * 4))
---
> #define RESULT_SIZE (1 + (29 * 4))
# diff erl_efile.h erl_efile.h.original
105,107d104
< Uint32 block_size; /* Optimal I/O size. */
< Uint32 blocks_low; /* Allocated 512-byte blocks,
lower 32 bits. */
< Uint32 blocks_high; /* Allocated 512-byte blocks,
higher 32 bits. */
# diff unix_efile.c unix_efile.c.original
872d871
< pInfo->blocks_high = 0;
875d873
< pInfo->blocks_high = (Uint32)(statbuf.st_blocks >> 32);
878,879d875
< pInfo->blocks_low = (Uint32)statbuf.st_blocks;
< pInfo->block_size = (Uint32)statbuf.st_blksize;
# diff prim_file.erl prim_file.erl.original
1022,1024c1022
< [Mode, Links, Major, Minor, Inode, Uid, Gid, Access|Tail4] = Tail3,
< [BlockSize, HighBlocks, LowBlocks] = Tail4,
< Blocks = HighBlocks * 16#100000000 + LowBlocks,
---
> [Mode, Links, Major, Minor, Inode, Uid, Gid, Access] = Tail3,
1038,1040c1036
< gid = Gid,
< block_size = BlockSize,
< blocks = Blocks}.
---
> gid = Gid}.
# diff file.hrl file.hrl.original
60,62c60
< gid :: integer(), % Group id for owner.
< block_size :: non_neg_integer(), % On Unix, optimal I/O size.
< blocks :: non_neg_integer()}). % On Unix, allocated
512-byte blocks.
---
> gid :: integer()}). % Group id for owner.
Can you tell where actual data blocks reside, using application logic?
(I know this might not be solving your problem, but introducing such
nonportable features into the standard distribution probably has small
chance of being incorporated).
On Mon, Jun 8, 2009 at 6:00 PM, Ville Silventoinen <
ville.silventoinen@REDACTED> wrote:
> Hi,
>
> I've written a directory scanner in Erlang that calculates usages per
> Unix user and group. It works well, except that the results are wrong
> when it encounters sparse files (holes in the files):
> http://en.wikipedia.org/wiki/Sparse_file. We have some users that have
> files that seem to be 1-2 terabytes, but in reality occupy less than a
> gigabyte on disk.
>
> The C stat struct has two fields I'd need (shown by "stat" command in
> Unix):
> blksize_t st_blksize; /* optimal I/O size */
> blkcnt_t st_blocks; /*allocated 512-byte blocks */
>
> If st_size > st_blocks * 512, the file is sparse. Unfortunately, the
> read_file_info/read_link_info doesn't provide the blocks information.
> Any chance this could be included in some future Erlang release?
>
> Also, any chance file:copy would support copying sparse files? :-) I
> tested, the target file becomes non-sparse.
>
> Erlang has been great help in our environment, where simple rsync has
> become too slow...
>
> Thanks,
> Ville
>
> P.S. I tried to add blocks information to my R13B Erlang environment,
> but I broke the build system somehow (something to do with the fact
> that prim_file is preloaded? "make preloaded" got me a bit further,
> but I gave up when compile:compile/3 became undef). Below are the
> changes I made to otp_src_R13B sources:
>
> # diff efile_drv.c efile_drv.c.original
> 1823,1827c1823
> < put_int32(d->info.block_size, &resbuf[1 + (29 *
> 4)]);
> < put_int32(d->info.blocks_high, &resbuf[1 + (30 *
> 4)]);
> < put_int32(d->info.blocks_low, &resbuf[1 + (31 *
> 4)]);
> <
> < #define RESULT_SIZE (1 + (32 * 4))
> ---
> > #define RESULT_SIZE (1 + (29 * 4))
>
> # diff erl_efile.h erl_efile.h.original
> 105,107d104
> < Uint32 block_size; /* Optimal I/O size. */
> < Uint32 blocks_low; /* Allocated 512-byte blocks,
> lower 32 bits. */
> < Uint32 blocks_high; /* Allocated 512-byte blocks,
> higher 32 bits. */
>
> # diff unix_efile.c unix_efile.c.original
> 872d871
> < pInfo->blocks_high = 0;
> 875d873
> < pInfo->blocks_high = (Uint32)(statbuf.st_blocks >> 32);
> 878,879d875
> < pInfo->blocks_low = (Uint32)statbuf.st_blocks;
> < pInfo->block_size = (Uint32)statbuf.st_blksize;
>
> # diff prim_file.erl prim_file.erl.original
> 1022,1024c1022
> < [Mode, Links, Major, Minor, Inode, Uid, Gid, Access|Tail4] = Tail3,
> < [BlockSize, HighBlocks, LowBlocks] = Tail4,
> < Blocks = HighBlocks * 16#100000000 + LowBlocks,
> ---
> > [Mode, Links, Major, Minor, Inode, Uid, Gid, Access] = Tail3,
> 1038,1040c1036
> < gid = Gid,
> < block_size = BlockSize,
> < blocks = Blocks}.
> ---
> > gid = Gid}.
>
> # diff file.hrl file.hrl.original
> 60,62c60
> < gid :: integer(), % Group id for owner.
> < block_size :: non_neg_integer(), % On Unix, optimal I/O
> size.
> < blocks :: non_neg_integer()}). % On Unix, allocated
> 512-byte blocks.
> ---
> > gid :: integer()}). % Group id for owner.
>
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
>
>
Hi Alex,
What do you mean by application logic? I can write an Erlang driver
that gets the actual block size of a file, is that then part of an
application?
I would think the blocks information would be useful to Unix, Linux
and Mac users who have to deal with files.
How is providing two more fields in file_info record non-portable?
What needs porting? You can just ignore them on Windows (until they
decide to become Unix-based as well ;-), or don't provide the fields
at all (like Python stat).
Cheers,
Ville
On Tue, Jun 9, 2009 at 10:26 AM, Alex Arnon<alex.arnon@REDACTED> wrote:
> Can you tell where actual data blocks reside, using application logic?
> (I know this might not be solving your problem, but introducing such
> nonportable features into the standard distribution probably has small
> chance of being incorporated).
>
>
> On Mon, Jun 8, 2009 at 6:00 PM, Ville Silventoinen
> <ville.silventoinen@REDACTED> wrote:
>>
>> Hi,
>>
>> I've written a directory scanner in Erlang that calculates usages per
>> Unix user and group. It works well, except that the results are wrong
>> when it encounters sparse files (holes in the files):
>> http://en.wikipedia.org/wiki/Sparse_file. We have some users that have
>> files that seem to be 1-2 terabytes, but in reality occupy less than a
>> gigabyte on disk.
>>
>> The C stat struct has two fields I'd need (shown by "stat" command in
>> Unix):
>> blksize_t st_blksize; /* optimal I/O size */
>> blkcnt_t st_blocks; /*allocated 512-byte blocks */
>>
>> If st_size > st_blocks * 512, the file is sparse. Unfortunately, the
>> read_file_info/read_link_info doesn't provide the blocks information.
>> Any chance this could be included in some future Erlang release?
>>
>> Also, any chance file:copy would support copying sparse files? :-) I
>> tested, the target file becomes non-sparse.
>>
>> Erlang has been great help in our environment, where simple rsync has
>> become too slow...
>>
>> Thanks,
>> Ville
>>
>> P.S. I tried to add blocks information to my R13B Erlang environment,
>> but I broke the build system somehow (something to do with the fact
>> that prim_file is preloaded? "make preloaded" got me a bit further,
>> but I gave up when compile:compile/3 became undef). Below are the
>> changes I made to otp_src_R13B sources:
>>
>> # diff efile_drv.c efile_drv.c.original
>> 1823,1827c1823
>> < put_int32(d->info.block_size, &resbuf[1 + (29 *
>> 4)]);
>> < put_int32(d->info.blocks_high, &resbuf[1 + (30 *
>> 4)]);
>> < put_int32(d->info.blocks_low, &resbuf[1 + (31 *
>> 4)]);
>> <
>> < #define RESULT_SIZE (1 + (32 * 4))
>> ---
>> > #define RESULT_SIZE (1 + (29 * 4))
>>
>> # diff erl_efile.h erl_efile.h.original
>> 105,107d104
>> < Uint32 block_size; /* Optimal I/O size. */
>> < Uint32 blocks_low; /* Allocated 512-byte blocks,
>> lower 32 bits. */
>> < Uint32 blocks_high; /* Allocated 512-byte blocks,
>> higher 32 bits. */
>>
>> # diff unix_efile.c unix_efile.c.original
>> 872d871
>> < pInfo->blocks_high = 0;
>> 875d873
>> < pInfo->blocks_high = (Uint32)(statbuf.st_blocks >> 32);
>> 878,879d875
>> < pInfo->blocks_low = (Uint32)statbuf.st_blocks;
>> < pInfo->block_size = (Uint32)statbuf.st_blksize;
>>
>> # diff prim_file.erl prim_file.erl.original
>> 1022,1024c1022
>> < [Mode, Links, Major, Minor, Inode, Uid, Gid, Access|Tail4] = Tail3,
>> < [BlockSize, HighBlocks, LowBlocks] = Tail4,
>> < Blocks = HighBlocks * 16#100000000 + LowBlocks,
>> ---
>> > [Mode, Links, Major, Minor, Inode, Uid, Gid, Access] = Tail3,
>> 1038,1040c1036
>> < gid = Gid,
>> < block_size = BlockSize,
>> < blocks = Blocks}.
>> ---
>> > gid = Gid}.
>>
>> # diff file.hrl file.hrl.original
>> 60,62c60
>> < gid :: integer(), % Group id for owner.
>> < block_size :: non_neg_integer(), % On Unix, optimal I/O
>> size.
>> < blocks :: non_neg_integer()}). % On Unix, allocated
>> 512-byte blocks.
>> ---
>> > gid :: integer()}). % Group id for owner.
>>
>> ________________________________________________________________
>> erlang-questions mailing list. See http://www.erlang.org/faq.html
>> erlang-questions (at) erlang.org
>>
>
>
Is there a particular reason why -import(foo). does not work?
The compiler does not complain but no functions seem to be imported.
Thanks, Joel
---
Mac hacker with a performance bent
http://www.linkedin.com/in/joelreymont
When using import you have to explicitly state which functions you are
importing from a module. The compiler should complain here but I guess it is
interpreting it as
-import(foo, []).
i.e. import no functions from foo.
Note that import does very little! All it does is to implicitly add the
module when calling an imported function. So after doing
-import(foo, [bar/1]).
a call to bar, for example bar(42), will be expanded to foo:bar(42). That is
all. It has no semantic meaning, or checks, or looking up the module, or
anything. It only saves some typing. This why you have to explicitly state
which functions to import.
Personally I only use it for importing functions from some very basic
modules like lists.
Robert
2009/6/8 Joel Reymont <joelr1@REDACTED>
> Is there a particular reason why -import(foo). does not work?
>
> The compiler does not complain but no functions seem to be imported.
>
> Thanks, Joel
>
> ---
> Mac hacker with a performance bent
> http://www.linkedin.com/in/joelreymont
>
>
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
>
>
On Jun 8, 2009, at 4:57 PM, Robert Virding wrote:
Personally I only use it for importing functions from some very basic
> modules like lists.
>
I was trying to import all functions from a module to simplify a DSL.
Alas, it seems like I either need to use prefixes or import all functions
individually.
---
Mac hacker with a performance bent
http://www.linkedin.com/in/joelreymont
Joel Reymont wrote:
>
> On Jun 8, 2009, at 4:57 PM, Robert Virding wrote:
>
> Personally I only use it for importing functions from some very basic
>> modules like lists.
>>
>
>
> I was trying to import all functions from a module to simplify a DSL.
>
What does it mean "to import all functions from a module" in a language with
hot-code loading when this set of functions can change at anytime?
(and the compiler currently looks only at one module at a time)
Kostis
On Jun 8, 2009, at 5:03 PM, Kostis Sagonas wrote:
What does it mean "to import all functions from a module" in a language
> with hot-code loading when this set of functions can change at anytime?
> (and the compiler currently looks only at one module at a time)
>
The functions exported from the module at compilation time.
I can't see how -import() can work at runtime.
---
Mac hacker with a performance bent
http://www.linkedin.com/in/joelreymont
I think import/1 is used like this:
"If a module from another package is used repeatedly in a module, an import
declaration can make life easier:
-module(foo.bar.m1).
-export([f/1, g/1]).
-import(fee.fie.foe.m2).
f(X) -> m2:g(X).
g(X) -> m2:h(X).
will make the calls to m2 refer to fee.fie.foe.m2."
Maria
Joel Reymont wrote:
>
> On Jun 8, 2009, at 4:57 PM, Robert Virding wrote:
>
> Personally I only use it for importing functions from some very basic
>> modules like lists.
>>
>
>
> I was trying to import all functions from a module to simplify a DSL.
>
> Alas, it seems like I either need to use prefixes or import all functions
> individually.
>
> ---
> Mac hacker with a performance bent
> http://www.linkedin.com/in/joelreymont
>
>
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
>
>
inets:start().
http:request(post, {"http://biggie/publish", [],
"application/x-www-form-urlencoded",
"topic=events&event=test_event&data=%22test%22"}, [], []).
It does not matter what I put in the URL above, the request just hangs
forever when I invoke the above in the shell. It's never hitting the server.
Any suggestions?
Thanks, Joel
---
Mac hacker with a performance bent
http://www.linkedin.com/in/joelreymont
I think it's a module conflict issue since http:request works fine when I
don't supply a path to my ebin. Please disregard my previous message!
---
Mac hacker with a performance bent
http://www.linkedin.com/in/joelreymont
To be more precise, I happened to call one of my modules http_transport I
should have listened when Erlang told me that http_util:to_lower was
undefined :-(.
---
Mac hacker with a performance bent
http://www.linkedin.com/in/joelreymont
Hello!
The documentation to mnesia:restore/2 says:
"If the database is huge, it may not be possible to restore it online.
In such cases, the old database must be restored by installing a
fallback and then restart."
How big the database must be for the mnesia:restore/2 to fail? I
couldn't reproduce it with 150MB backup.
--
Sergey Samokhin
Hello.
I'm trying to create a cluster based on mnesia. And there is a problem
with connecting a node (which may have schema previously stored on
disk) to a cluster.
Schema stored on disk by mnesia may contain one schema whereas node
I'm trying to connect to by mnesia:change_config(extra_db_nodes,
[Node]) another one. How should I detect this problem and say the user
that schemas are incompatible? mnesia:start() doesn't return anything
like {error, schemas_conflict} which could be used to exactly detect
such a problem. Also I didn't see any useful events sent to
error_logger.
One way to detect it is to compare schema's cookies before mnesia is
started. Unfortunatelly, I can't get such a cookie until mnesia is
started. And as soon as mnesia is started, error may happen if there
are two different schemas.
Let's suppose that emulator is started by "erl -kernel auto_dist_connect
never":
mnesia:start(), % there won't be any auto-connects even if schema
contains info about another nodes
Cookie = mnesia:table_info(schema, cookie), % will fail if mnesia isn't
started
Node = another_node@REDACTED, % The node I want to connect to
% <skipped> Here I'm checking if Cookie is the same as Node has
% If cookies are the same, I would like to connect:
application:set_env(kernel, auto_dist_connect, always),
mnesia:change_config(extra_db_nodes, [Node])
% If cookies are different, I want to crash:
exit(schemas_conflict_detected)
I've found that changing the kernel option "auto_dist_connect" by calling:
application:set_env(kernel, auto_dist_connect, Type), where Type =
once | never | always
...works fine.
Is it safe to change this option on the fly without restarting an emulator?
Of course, schema stored on disk by mnesia may contain information
about nodes which no longer have the same schema (they may have been
taken away from one cluster and put into another one). In this case
one can compare the current schema with all the schemas from nodes
obtained by mnesia:system_info(db_nodes). It's look too tricky...
What do you use to detect such a problems? Do you let mnesia crash
when starting (in this case, how do you detect this problem and write
error messages to a log)?
--
Sergey Samokhin
Is there a way to disable keep_alive for the http client? I really want it
to keep a single connection to the server and close it after each request.
I tried this set of options to http:request/4 but they don't do the trick.
http_client_opts() ->
[{max_sessions, 1},
{max_pipeline_length, 1},
{max_keepalive_length, 1},
{pipeline_timeout, 1},
{keep_alive_timeout, 0}].
Thanks, Joel
---
Mac hacker with a performance bent
http://www.linkedin.com/in/joelreymont
You can provide a header {"Connection", "close"} to the request, or
use HTTP/1.0, the manual says:
version
Can be used to make the client act as an HTTP/1.0 or HTTP/0.9 client.
By default this is an HTTP/1.1 client. When using HTTP/1.0 persistent
connections will not be used.
Regards.
On Mon, Jun 8, 2009 at 8:00 PM, Joel Reymont<joelr1@REDACTED> wrote:
> Is there a way to disable keep_alive for the http client? I really want it
> to keep a single connection to the server and close it after each request.
>
> I tried this set of options to http:request/4 but they don't do the trick.
>
> http_client_opts() ->
> [{max_sessions, 1},
> {max_pipeline_length, 1},
> {max_keepalive_length, 1},
> {pipeline_timeout, 1},
> {keep_alive_timeout, 0}].
>
> Thanks, Joel
>
> ---
> Mac hacker with a performance bent
> http://www.linkedin.com/in/joelreymont
>
>
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
>
>
Hello All,
I've been writing a TCP server in erlang and after studying some OTP
principals found this blog on how to build an TCP server using OTP. My
question is: is there anything that has changed in the latest release builds
of the erlang VM that would stop this code from working. I find that the
gen_fsm does not receive any messages when the {active,once} is set.
http://www.trapexit.org/Building_a_Non-blocking_TCP_server_using_OTP_principles
Thanks in Advance.
Allan Merolla
Hi,
I can't see any advice previously offered for the best way to query a
(large) mnesia table with a query comparable to "equals ignore case".
My first thought is to storing an indexed lower or upper "case"
version of the data in an additional field, but with nearly 20 million
records that doesn't seem very attractive.
TIA for any thoughts,
Regards,
Steve
You could hash on the (up/low) uni-cased string, and perhaps get
smaller overhead for that extra field.
The cost is that you have to sort out potential collisions at lookup.
Also: 20 million records sounds like peanuts.
On Tue, Jun 9, 2009 at 10:45, Steve Davis<steven.charles.davis@REDACTED>
wrote:
> My first thought is to storing an indexed lower or upper "case"
> version of the data in an additional field, but with nearly 20 million
> records that doesn't seem very attractive.
I once wrote some code to convert a regexp to a
match specification. Given that you cannot recurse
in match specs, it has some severe limitations. If you
can live with those, you might get some mileage out of
this approach.
http://jungerl.cvs.sourceforge.net/viewvc/jungerl/jungerl/lib/rdbms/src/rdbms_ms.erl?revision=1.1&view=markup
Eshell V5.7.1 (abort with ^G)
1> rdbms_ms:re_match(["ulf","Ulf","ULF","abc"],"[uU][lL][fF]").
["ulf","Ulf","ULF"]
3> rdbms_ms:re("[uU][lL][fF]").
[{['$1','$2','$3'],
[{'andalso',
{'andalso',
{'=/=','$1',[]},
{'orelse',{'==','$1',85},{'==','$1',117}}},
{'andalso',
{'andalso',
{'=/=','$2',[]},
{'orelse',{'==','$2',76},{'==','$2',108}}},
{'andalso',
{'andalso',
{'=/=','$3',[]},
{'orelse',{'==','$3',70},{'==','$3',102}}},
true}}}],
['$_']}]
At least it beats writing the same thing by hand. :)
One obvious problem is that, since the key part will be unbound,
it will result in a linear search. Not knowing your specific
problem, this may or may not be a killer. :/
BR,
Ulf W
Steve Davis wrote:
> Hi,
>
> I can't see any advice previously offered for the best way to query a
> (large) mnesia table with a query comparable to "equals ignore case".
>
> My first thought is to storing an indexed lower or upper "case"
> version of the data in an additional field, but with nearly 20 million
> records that doesn't seem very attractive.
>
> TIA for any thoughts,
>
> Regards,
> Steve
>
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
>
>
--
Ulf Wiger
CTO, Erlang Training & Consulting Ltd
http://www.erlang-consulting.com
Thanks both :)
-- I think I'm beginning to prefer the un-dry original plan (despite
the fact that the extra peanuts will mean I can't use ram_copies).
Thanks again
/s
Hi,
On Tue, Jul 29, 2008 at 10:50 AM, Ingela Anderton Andin <
ingela@REDACTED> wrote:
> I put support for "SQL_UNICODE-types" corresponding types for nvarchar
> and friends on the wish list
> for odbc. Even though it is not a very big job it does not come on top
> of the priority list, so if anyone
> wants it badly I would like to encourage you to make a
> patch-contribution. Otherwise we will get around to it sometime ...
> probably in a somewhat distant future.
>
This bit me pretty badly, so I took a stab at it.
I modified odbc module to support SQL_WVARCHAR (NVARCHAR) fields and it
would appear to work on XP with MS SQL Server 2008.
Unicode text has to be provided as little endian utf16 binaries.
odbctest.erl shows the extent (lack) of testing. Below, is the output....
-------------------------------------------------------------
Erlang R13B (erts-5.7.1) [source] [rq:1] [async-threads:0]
Eshell V5.7.1 (abort with ^G)
1> c(odbctest).
{ok,odbctest}
2> odbc:start().
ok
3> odbctest:test().
=INFO REPORT==== 9-Jun-2009::14:00:34 ===
test_describe: {ok,[{"a",{sql_wvarchar,50}}]}
test_select: []
test_insert: {updated,12}
test_select: [<<"ÖÄÅÄÖÅäöå">>,<<"testasdf">>,<<"Row 3">>,<<"Row 4">>,
<<"Row 5">>,<<"Row 6">>,<<"Row 7">>,<<"Row 8">>,<<"Row
9">>,
<<"Row 10">>,<<"Row 11">>,<<"Row 12">>]
ok
4>
-------------------------------------------------------------
Regards,
Juhani
dear all,
two months ago i've started an experiment to increase speed of messages sent
between different erlang nodes. i seem to have found a way to considerably
increase this speed up to 3 times the native erlang speed, and would love to
hear your feedback on this.
the idea is quite simple: queuing all messages sent from a node to another
node, and sending them by groups. the whole concept is therefore to have a
gen_server, called 'qr', running on every node where message passing takes
place. a single process from a node A sends a message to a process of node
B, being relayed by the two 'qr':
process on node A => 'qr' on node A => 'qr' on node B => process on node B.
this is something that is generally taken care of at lower level of
implementations (tcp), with algorithms such as nagle, but i've decided to
try a pipe/queuing mechanism at erlang application level too, to see if i
could get any improvements.
a detailed explanation of the tests and benchmarks that i've performed are
available here:
http://www.ostinelli.net/boost-message-passing-between-erlang-nodes and a
new updated code is available here:
http://www.ostinelli.net/wp-content/uploads/2009/06/erlang_mq_boost_2.zip for
you to try it out on your machine.
thanks to ulf wiger writing a note on the post above, i've also tried out
the undocumented dist_nodelay kernel option, which did provide improvements,
but still far from the ones i'm experimenting with the 'qr' pipe mechanism.
please note that i'm in no way pretending to have found something great and
new. i'm posting this here just because reactions to my linked post have
mainly pointed towards telling me to perform additional tcp optimization,
but i've personally been unable to find a way to reproduce the results of
the 'qr' pipe mechanism by mere tcp optimization. also, the benchmarking
test that i've used is quite specific, since it sends 200,000 messages in
parallel first, which are then processed sequentially on the recipient node.
this is because this test reflects a real need of an application i'm
developing, where loads of client threads have to go through a bottleneck of
a single registered process.
therefore, any opinions on this are warmly welcome, so as to have a better
understanding on what is going on and hopefully produce better software.
thank you in advance those of you who took the time to read till here, and
more to the ones who will [hopefully] give me some feedback.
cheers,
r.
Hi all,
Roberto: I did read your blog post and tried out the code on a windows
machine with very positive results! :)
I find Roberto's approach really interesting in that in adition to speeding
up message delivery it opens the door for a mailman-like message passing
system!
My take is that Erlang could benefit from such a tool in that it would help
support applications with looser inter-process communication requirements.
Instead of having lots of processes "monitoring" their communication
partners the processes could simply deliver the message to the mailman and
delegate the efective message delivery to that entity. This is turn could
turn the whole software more amenable to handle network
partitions/disconnected operations than the current erlang tool set (which
favours consistency and availability over partition-tolerance).
Thumbs up to Roberto for this great little experience! :)
Cheers,
:Davide
On Tue, Jun 9, 2009 at 1:13 PM, Roberto Ostinelli <roberto@REDACTED
>wrote:
> dear all,
>
> two months ago i've started an experiment to increase speed of messages
> sent between different erlang nodes. i seem to have found a way to
> considerably increase this speed up to 3 times the native erlang speed,
and
> would love to hear your feedback on this.
>
> the idea is quite simple: queuing all messages sent from a node to another
> node, and sending them by groups. the whole concept is therefore to have a
> gen_server, called 'qr', running on every node where message passing takes
> place. a single process from a node A sends a message to a process of node
> B, being relayed by the two 'qr':
>
> process on node A => 'qr' on node A => 'qr' on node B => process on node
B.
>
> this is something that is generally taken care of at lower level of
> implementations (tcp), with algorithms such as nagle, but i've decided to
> try a pipe/queuing mechanism at erlang application level too, to see if i
> could get any improvements.
>
> a detailed explanation of the tests and benchmarks that i've performed are
> available here:
> http://www.ostinelli.net/boost-message-passing-between-erlang-nodes and a
> new updated code is available here:
> http://www.ostinelli.net/wp-content/uploads/2009/06/erlang_mq_boost_2.zipfor
> you to try it out on your machine.
>
> thanks to ulf wiger writing a note on the post above, i've also tried out
> the undocumented dist_nodelay kernel option, which did provide
improvements,
> but still far from the ones i'm experimenting with the 'qr' pipe
mechanism.
>
> please note that i'm in no way pretending to have found something great
and
> new. i'm posting this here just because reactions to my linked post have
> mainly pointed towards telling me to perform additional tcp optimization,
> but i've personally been unable to find a way to reproduce the results of
> the 'qr' pipe mechanism by mere tcp optimization. also, the benchmarking
> test that i've used is quite specific, since it sends 200,000 messages in
> parallel first, which are then processed sequentially on the recipient
node.
> this is because this test reflects a real need of an application i'm
> developing, where loads of client threads have to go through a bottleneck
of
> a single registered process.
>
> therefore, any opinions on this are warmly welcome, so as to have a better
> understanding on what is going on and hopefully produce better software.
>
> thank you in advance those of you who took the time to read till here,
and
> more to the ones who will [hopefully] give me some feedback.
>
> cheers,
>
> r.
>
>
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
>
>
Hi!
I'm trying to get a dhcp-server
(http://code.google.com/p/erlang-dhcp-server/) to work but right now I'm
not able to answer on the broadcasted dhcp-discover message that's sent
from IP 0.0.0.0.
When I use a sniffer on the network-devices dhcp-offer message is sent
from 127.0.0.1 to 127.0.0.1, I guess that's the way Erlang or Linux
handle messages to 0.0.0.0.
Is there any way to send UDP messages from Erlang to a computer without
IP-address set?
Is it possible to modify the ARP? I guess that would solve my problem
somehow.
//Lucas
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Lucas Robsahm wrote:
> Hi!
>
> I'm trying to get a dhcp-server
> (http://code.google.com/p/erlang-dhcp-server/) to work but right now I'm
> not able to answer on the broadcasted dhcp-discover message that's sent
> from IP 0.0.0.0.
> When I use a sniffer on the network-devices dhcp-offer message is sent
> from 127.0.0.1 to 127.0.0.1, I guess that's the way Erlang or Linux
> handle messages to 0.0.0.0.
>
> Is there any way to send UDP messages from Erlang to a computer without
> IP-address set?
> Is it possible to modify the ARP? I guess that would solve my problem
> somehow.
>
> //Lucas
>
Yes, using UDP broadcast.
{ok, Socket} = gen_udp:open(0, [binary, {broadcast, true}]),
gen_udp:send(Socket, {255,255,255,255}, ?PORT, Data),
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.11 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkoudSsACgkQ6pa1H/H5pqUikgCgnlFEPecZjK/VsvnYos5B0GWl
bgYAn2o3qIBplwLeZs+7mZhQ3Q4bzzdj
=OKqE
-----END PGP SIGNATURE-----
Hello!
I think there wasn't any grumbling this month about the immutable
local variables in Erlang, so here's real world code I've found just
today:
% Take away underscore and replace with hyphen
MO1 = re:replace(MO, "_", "-", [{return, list}, global]),
MO2 = toupper(MO1),
% Replace zeros
MO3 = re:replace(MO2, "RX0", "RXO", [{return, list}, global]),
% Insert hyphen if missing
MO5 = case re:run(MO3, "-", [{capture, none}]) of
nomatch ->
insert_hyphen(MO3);
match ->
MO3
end,
I think it's fairly clumsy to use MOx (MO for managed object) in the
code. MO4 was removed during the regexp->re refactoring step. How to
eliminate the versioned variable names? The
MOAfterUnderscoresWereReplaced, UpperCaseMO, MOAfterRX0WasReplaced,
etc. variablenames are really ugly. It used to use regexp, so at that
point it wasn't possible to easily nest the whole into one call, but
that would be still ugly. So any other ideas?
I like the foldl way:
------
lists:foldl (fun (Fun, Acc) -> Fun (Acc) end,
M0,
[ fun (X) -> re:replace (X, "_", "-", [ { return, list }, global
]) end,
fun to_upper/1,
fun (X) -> re:replace (X, "RX0", "RX0", [ { return, list },
global ]) end,
fun (X) -> case re:run (X, "-", [ { capture, none } ]) of
nomatch ->
insert_hyphen (X);
match ->
X
end
end ]).
------
Cheers,
-- p
On Tue, 9 Jun 2009, Attila Rajmund Nohl wrote:
> Hello!
>
> I think there wasn't any grumbling this month about the immutable
> local variables in Erlang, so here's real world code I've found just
> today:
>
> % Take away underscore and replace with hyphen
> MO1 = re:replace(MO, "_", "-", [{return, list}, global]),
> MO2 = toupper(MO1),
> % Replace zeros
> MO3 = re:replace(MO2, "RX0", "RXO", [{return, list}, global]),
> % Insert hyphen if missing
> MO5 = case re:run(MO3, "-", [{capture, none}]) of
> nomatch ->
> insert_hyphen(MO3);
> match ->
> MO3
> end,
>
> I think it's fairly clumsy to use MOx (MO for managed object) in the
> code. MO4 was removed during the regexp->re refactoring step. How to
> eliminate the versioned variable names? The
> MOAfterUnderscoresWereReplaced, UpperCaseMO, MOAfterRX0WasReplaced,
> etc. variablenames are really ugly. It used to use regexp, so at that
> point it wasn't possible to easily nest the whole into one call, but
> that would be still ugly. So any other ideas?
>
> ________________________________________________________________
> erlang-questions mailing list. See http://www.erlang.org/faq.html
> erlang-questions (at) erlang.org
>
>
Well for these men if they succeed; well also, though not so well, if
they fail, given only that they have nobly ventured, and have put forth
all their heart and strength.
-- Theodore Roosevelt
More information about the erlang-questions
mailing list