[erlang-questions] Bug, feature or limitation of erl_parse?

[Robert Virding]

It is a problem of *where* the conversion of "test" ++ Else to
[$t,$e,$s,$t|Else] is done. It has nothing to do with whether the list
is a proper list or not.

The parser just parses the input and doesn't not attempt any
optimisation or transformation of the input. This I think is a Good
Thing as if were to attempt something like this it would be trying to
guess how its results were going to be used, and this never works. The
parser should return the AST formed from the input, no more no less.

That it works in shell is a completely different matter. After the
input has been parsed it is passed to an erlang interpreter, erl_eval,
which then evaluates the input. It is here where *expressions* like
"foo"++'_' below are evaluated to [$f,$o,$o|'_'] and passed in to
ets:match. It is also here where similar constructs in *patterns* are
handled in the pattern matching so as if they were converted and the
result is the same as with the compiler.

It means that if you want to be able to read in terms from strings,
parse them and send them to mnesia:select you will have to do it
yourself. Unfortunately. It is not really difficult if you are
restricted to patterns.

Robert


On 25 August 2010 17:10, Hynek Vychodil <hynek@REDACTED> wrote:
> I still don't understand.
>
> 1> T = ets:new(foo,[private]).
> 20496
> 2> ets:insert(T, [{"foo", 1}, {"foobar", 2}, {"bar", 3}, {"foobaz", 4}]).
> true
> 3> Prefix = "foo".
> "foo"
> 4> catch ets:match(T,{Prefix++'$1', '$2'}).
> [["bar",2],[[],1],["baz",4]]

Here the expression Prefix++'$1' is evaluated to the select spec
[$f,$o,$o|'$1'] by the shell evaluator so it works. Writing the same
thing in compiled code basically results in the same evaluation though
the compiler specially recognises this and can do some of the work at
compile time.

> 5> catch ets:select(T,[{{Prefix++'_', '_'}, [], ['$_']}]).
> [{"foobar",2},{"foo",1},{"foobaz",4}]

Same as previous expression.

> On Wed, Aug 25, 2010 at 4:47 PM,  <bile@REDACTED> wrote:
>> Try that using erl_scan and erl_parse. That's what doesn't work and
>> that's what I'm trying to solve. I am reading in strings of erlang
>> terms for running selects on mnesia. "prefix" ++ '_' is easier for my
>> customer to understand and deal with than [$p,$r,$e,$f,$i,$x|'_']. I
>> can make a preprocessor which does the substitution but I wanted to
>> know if I was doing something wrong.
>>
>> On Wed, 25 Aug 2010 15:38:21 +0200
>> Hynek Vychodil <hynek@REDACTED> wrote:
>>
>>> What is problem?
>>>
>>> 1> L = "test".
>>> "test"
>>> 2> Match = L ++ '_'.
>>> [116,101,115,116|'_']
>>>
>>>
>>> On Wed, Aug 25, 2010 at 3:22 PM, Musumeci, Antonio S
>>> <Antonio.Musumeci@REDACTED> wrote:
>>> > http://www.erlang.org/doc/apps/erts/match_spec.html
>>> >
>>> > It shows that [$t|'_'] is a proper match pattern... And it does
>>> > work. Since otherwise "t" ++ Else is a shortcut for [$t|Else] I was
>>> > trying to find out why I appear unable to parse a string of the
>>> > former to use in a matchspec as the latter one does work.
>>> >
>>> > -----Original Message-----
>>> > From: erlang-questions@REDACTED
>>> > [mailto:erlang-questions@REDACTED] On Behalf Of Robert Raschke
>>> > Sent: Wednesday, August 25, 2010 8:52 AM To:
>>> > erlang-questions@REDACTED Subject: Re: [erlang-questions] Bug,
>>> > feature or limitation of erl_parse?
>>> >
>>> > Hmm, [$t,$e,$s,$t|'_'] is not a proper list.
>>> > Do you mean [$t,$e,$s,$t,'_'] (or the equivalent
>>> > [$t,$e,$s,$t|['_']])?
>>> >
>>> > If yes, then "test"++['_'] will do the match:
>>> >
>>> > Eshell V5.6.5  (abort with ^G)
>>> > 1> L=[$t,$e,$s,$t|['_']].
>>> > [116,101,115,116,'_']
>>> > 2> "test"++['_']=L.
>>> > [116,101,115,116,'_']
>>> >
>>> > But I'm pretty sure I am not understanding your question exactly.
>>> >
>>> > Robby
>>> >
>>> > PS Interesting, why does this work:
>>> > 3> "test"++'_'.
>>> > [116,101,115,116|'_']
>>> > ???
>>> >
>>> > On Wed, Aug 25, 2010 at 1:17 PM, Musumeci, Antonio S <
>>> > Antonio.Musumeci@REDACTED> wrote:
>>> >
>>> >> Understood. Is there no way to turn "test" ++ '_' into
>>> >> [$t,$e,$s,$t|'_'] from a string for use in a matchspec?
>>> >>
>>> >> -----Original Message-----
>>> >> From: Robert Virding [mailto:rvirding@REDACTED]
>>> >> Sent: Tuesday, August 24, 2010 5:01 PM
>>> >> To: Musumeci, Antonio S (Enterprise Infrastructure)
>>> >> Cc: erlang-questions@REDACTED
>>> >> Subject: Re: [erlang-questions] Bug, feature or limitation of
>>> >> erl_parse?
>>> >>
>>> >> Erl_parse does recognise '"test" ++ Else', but it is not a term it
>>> >> is an expression. It is later inside the compiler that the
>>> >> expression is converted into it's shortcut form. The parser just
>>> >> parses the expression. When you did it by hand, so to speak, you
>>> >> got a term which erl_parse could parse as a term. So:
>>> >>
>>> >> 17> f(S), {ok,S,_} = erl_scan:string("\"test\" ++ Else. ").
>>> >> {ok,[{string,1,"test"},{'++',1},{var,1,'Else'},{dot,1}],1}
>>> >> 18> erl_parse:parse_exprs(S).
>>> >> {ok,[{op,1,'++',{string,1,"test"},{var,1,'Else'}}]}
>>> >>
>>> >> N.B. it it not the string "term" which causes it to be an
>>> >> expression but the operator '++'.
>>> >>
>>> >> Robert
>>> >>
>>> >> On 24 August 2010 22:08, Musumeci, Antonio S <
>>> >> Antonio.Musumeci@REDACTED> wrote:
>>> >> > It doesn't appear to understand the "test" ++ Else shortcut for
>>> >> [$t,$e,$s,$t|Else]. Reason I noticed was I'm reading in terms as
>>> >> strings and passing them to mnesia:select. erl_parse works fine
>>> >> with [$t|'_'] but not "t" ++ '_'.
>>> >> >
>>> >> >> S = "[$t,$e,$s,$t|'_']".
>>> >> > "[$t,$e,$s,$t|'_']"
>>> >> >> T = "\"test\" ++ '_'".
>>> >> > "\"test\" ++ '_'"
>>> >> >> [$t,$e,$s,$t|'_'] = "test" ++ '_'.
>>> >> > [116,101,115,116|'_']
>>> >> >> {ok,S2,_} = erl_scan:string(S).
>>> >> > {ok,[{'[',1},
>>> >> >     {char,1,116},
>>> >> >     {',',1},
>>> >> >     {char,1,101},
>>> >> >     {',',1},
>>> >> >     {char,1,115},
>>> >> >     {',',1},
>>> >> >     {char,1,116},
>>> >> >     {'|',1},
>>> >> >     {atom,1,'_'},
>>> >> >     {']',1}],
>>> >> >    1}
>>> >> >> {ok,T2,_} = erl_scan:string(T).
>>> >> > {ok,[{string,1,"test"},{'++',1},{atom,1,'_'}],1}
>>> >> >> erl_parse:parse_term(S2++[{dot,1}]).
>>> >> > {ok,[116,101,115,116|'_']}
>>> >> >> erl_parse:parse_term(T2++[{dot,1}]).
>>> >> > {error,{1,erl_parse,"bad term"}}
>>> >> >
>>> >>
>>> > ________________________________________________________________
>>> > erlang-questions (at) erlang.org mailing list.
>>> > See http://www.erlang.org/faq.html
>>> > To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>>> >
>>> >
>>>
>>>
>>>
>>
>>
>
>
>
> --
> --Hynek (Pichi) Vychodil
>
> Analyze your data in minutes. Share your insights instantly. Thrill
> your boss.  Be a data hero!
> Try GoodData now for free: www.gooddata.com
>
> ________________________________________________________________
> erlang-questions (at) erlang.org mailing list.
> See http://www.erlang.org/faq.html
> To unsubscribe; mailto:erlang-questions-unsubscribe@REDACTED
>
>