[erlang-questions] Coming Back (maybe improving lists:reverse/1)

[Richard A. O'Keefe]

On 9/10/2015, at 4:40 pm, Ivan Carmenates Garcia <co7eb@REDACTED> wrote:

> Regards,
> 
> Well seems to me that there is no more optimizations for the algorithm
> (except for yours 'my_reverse_join' see below), I tested yours and mine and
> they both take exactly the same for the final result the unparsed "string",
> well mine is 200 millisecond faster in 1 million of iterations, yours take
> 64XX main 62XX approx. The problem is that using lists:flatten is the hell
> slow,

The problem is that I suggested *NOT* using lists:flatten/1.

By the way, you really do not *have* to use any particular built in
function.  A few minutes' coding produced this result for flattening
a tree with a million elements:

- Using lists:flatten/1 : 123,939 microseconds
- Using my own code     :  63,134 microseconds

The effect of my code isn't precisely the same as that of
lists:flatten/1, but for a tree of strings that doesn't matter.

my_flatten([A,B,C|Xs], R) ->
    my_flatten(A, my_flatten(B, my_flatten(C, my_flatten(Xs, R))));
my_flatten([A|Xs], R) ->
    my_flatten(A, my_flatten(Xs, R));
my_flatten([], R) ->
    R;
my_flatten(X, R) ->
    [X|R].


> I also tested your algorithm just after the io lists return and it is
> very faster its took 14XX milliseconds, so when you use the final
> lists:flatten/1 everything goes to crap, sorry about the word. So it is
> amazing who faster it is and then lists:flatten by its own take the another
> 5 seconds, I did knew that because I read Erlang session about 7 myths off
> bla bla .. and optimizations stuffs some time ago and they say lists:flatten
> is slow also I tested it when I was constructing the algorithm first time I
> did avoid constructing the io lists because of that.

No, the whole point of io lists is to *NOT* flatten them.

The advice, in short, was to structure the rest of your program
so that you don't NEED to do the flattening.
> 
> I did clear some things like unparse well I need another name I don't like
> unparse either and parse is wrong, I will come up with something.

Maybe you don't *like* "unparse", but it *is* the standard
technical term.  Type "unparsing" into the search box of your
browser and read!
> I also have questions for you if you are so kind to answer.
> 
> Regards proplists, well, it is necessary to make lists of options as
> proplists?

No, of course not.  Even when you encode a list of options as
something *like* a proplist, the structure you want will usually
be stricter.  In any case, what you have here is simply not a
list of options.  The elements are not options, they are
descriptions of columns you want to get back from some data base
query.

> yes, proplists:compact/1 and expand/1 for [a] to [{a, true}] and
> all those stuffs but what I was trying to do is to make it simple for the
> user, because more tuples are more complicate to write in the case of
> {users, name, alias}

(a) Remember, one of the essential things about *being* a proplist
is that the consumer is going to IGNORE anything that is not an atom
or a pair.  If you are not going to do that, you are not treating
the data *as* a proplist, and it is "lying to the user" to call it one.

(b) Far from making things simpler for the user, you made it
much more confusing.  There are places where an alias *can* be
supplied and places where it *can't*, and there doesn't seem to
be any reason for that.  Well, it confused the heck out of *me*.

> well that could be right adding more brackets like
> {users, {name, alias}} because it organizes the thing. But i.e.: I have
> another algorithm for match specs in which I can do {name | {table, name} |
> value, op, name | {table, name} | value } also [{..., op, ...}, ...],
> logic_op, ... so it will be easy for the user to build something like 
> [
>    [{id, '==', 1}, {age, '==', 31}],
>    'or',
>    {username, '==', "john"} 
> ]

There are programming languages in which building queries as
data structures is easy.

  In Lisp,
    (make-query '(or (and (= id 1) (= age 31)) (= username "john")))
  is easy because that's *exactly* what an expression would look
  like in Lisp.

  In R,
    make_query(id = 1 && age == 31 || username == "john")
  is easy because R functions evaluate their arguments lazily and
  can get at the abstract syntax trees of those arguments, so again
  the syntax is exactly what an expression normally looks like.

Erlang really isn't one of those languages, but there's a feature
of Erlang that means you can get a lot closer.  You could use a
parse transform to take something like

   make_query(Id == 1 andalso Age == 31 orelse UserName == "John")

which the parser turns into an abstract syntax tree, and then your
parse transform could take make_query(...) and turn that into whatever
data structure you like.  It's hardER in Erlang than it is in Lisp,
but it's still much easier than it is with C or Java.  (And of course
LFE would do it exactly the way Lisp does it.)

If I were a user of your system, I would be somewhere between
baffled and outraged at the claim that it would be *easy* for
me to construct a query in either of the forms you mention.

This is perhaps the ideal time to mention the idea of
STAGED computation, also known as partial evaluation or partial
execution.  The function we started discussing looks like a
textbook case of something you probably shouldn't be doing at
run time anyway.