Erlang does have problems

[Bob Ippolito]

Storing text as lists-of-integers is suboptimal for storage. It
doesn't particularly bother me, because I'm simply not working with
much text. The text I am working with in my current application is
relatively static, so I can treat most of it as lists of UTF8
binaries... but that also means I have to write most of my own
functions for operating on it. That's disappointing, but it doesn't
bother me much because the strings module doesn't really provide much
functionality either so I really am not missing much.

iodata-of-binaries just means that such a binary-strings module
could/should be able to handle lists-of-binaries in addition to raw
binaries.. just like nearly all of the IO functions can deal with
iodata. That way you could avoid unnecessary copying because you can
use [<<"foo">>, <<"bar">>] to concatenate instead of
iolist_to_binary([<<"foo">>, <<"bar">>]).

Splitting binaries should already be faster than splitting lists in
many cases, because the VM can choose to create views on the larger
binary. Concatenation of binaries requires copying, so that's why it
may be better to allow iodata instead of just pure strings, since
copying can be avoided.

I'm also not terribly convinced that lists are better for
concatenation, because adding data to the end of a list requires a lot
of copying and that's the most common operation in my experience. It
really depends on how many operations you're doing and how many
intermediate binaries you need to create. The binary copying might
take less memory and be faster than all of the copying incurred by
lists:reverse or whatever.

In any case, what's needed is a prototype and some benchmarks that can
be tested for performance on several kinds of machines to see what
works well under which circumstances. The current implementation is
good enough for me, so I'm not inclined to put in that kinda effort.

-bob

On 8/30/06, Joel Reymont <joelr1@REDACTED> wrote:
> Bob,
>
> Can you please elaborate on the iodata-of-binaries bit and on what
> you see as wrong right now?
>
> On Aug 30, 2006, at 9:29 PM, Bob Ippolito wrote:
>
> > Inserting and deleting pieces could be fast operations if
> > iodata-of-just-binaries were allowed instead of requiring binaries.
> > All of the slicing and dicing should be memory-friendly since the
> > majority of the time it will be a view on some larger binary in
> > memory...
> --
> http://wagerlabs.com/
>
>
>
>
>
>