[erlang-questions] Market data, trading strategies and dropping the last element of a list

[Scott Lystig Fritchie]

>>>>> "tl" == Thomas Lindgren <thomasl_erlang@REDACTED> writes:

tl> A simple but somewhat nasty solution is to keep the window in a
tl> destructively updated circular buffer, implemented with an ets
tl> table.

My functional answer ... is to use the file system.  :-)

Create your log file with a "hole" by starting to write at an offset
of (for example) 2 billion bytes (to avoid possible problems with
2^31-limited file systems).  The log grows *downward*.  The file
system(*) only allocates blocks for data that's actually written, no
blocks allocated in the "hole".

To remove old entries, use truncate(2).

As an added bonus, no changes required if the total data size >
physical RAM.  If the total data size is << physical RAM, the disk
blocks should all be cached to reduce the pain of the file I/O and
still keep you in soft real-time bounds.

-Scott

(*) With the possible exception of a really naive/silly/dumb file
systems that would allocate *all* blocks for you....