[erlang-questions] A bug? - bloom filters - and loadable BIFs

[Jachym Holecek]

Hello,

# Joe Armstrong 2009-04-29:
> So he sent me a program, which crashes his machine (out of memory)
> which he thought should work - to my eye it looks ok - It will not
> be blindingly fast, but it looks properly tail-recursive so the
> garbage collector should not have any problems.
> 
> I've appended at the end of this mail - so you can see for yourself.
> It is a GC bug or a user error?

UNIX heap size limit is certainly one of the factors. During the
test (doesn't crash on my system -- my user is allowed to consume
up to 2GB RAM) 'erl' consumes anywhere between 20MB to 1GB of RAM.

> What I'd like is a "linked in BIF" - it seems to me to be a bit of a
> pain in the whatnot to have to mess around with the Erlang internals
> every time you want to write a new BIF.

Doesn't the FFI EEP solve this?

> ------ cut --- below this line is from Steve's post-----
> [...]
> Nor does it talk about why this code fails, though I'd imagine it is
> copying and gc'ing large binaries like crazy every time you change 1
> bit.

It seems more like it's building new binaries with references to the
old binaries, the structure seems to be flattened on GC. Indeed,
adding a call to erlang:garbage_collect/0 into bitmap:set_bits/2
makes the test run in pretty much constant 30MB of RAM.

Mostly guessing here (based on bitmap.beam disassembly), but I'm
sure someone will correct me if I'm wrong ;-).

Regards,
	-- Jachym