[erlang-questions] re:split not splitting at pipe symbol

[Jesper Louis Andersen]

On Sat, Jan 8, 2011 at 21:37, Alain O'Dea <alain.odea@REDACTED> wrote:
> It can be better to use string:tokens/2 over re:split/2 if you don't need to know the position of a token.  The problem is that string:tokens/2 drops empty tokens silently.
>
> If you have a record like:
> GivenName|MiddleName|Surname
>
> And some records don't have middle names like:
> Alain||O'Dea
>
> Then string:tokens/2 will give you ["Alain","O'Dea"]

Perhaps it would be worth looking at the standard ml basis library in
this respect:

- String.tokens (fn c => c = #"|") "Alain||O'Dea";
val it = ["Alain","O'Dea"] : string list
- String.fields (fn c => c = #"|") "Alain||O'Dea";
val it = ["Alain","","O'Dea"] : string list

It is worth having both variants I think. The patch shouldn't be hard
to provide against the stdlib.

And I am a firm believer that a regular expression is almost always
the wrong thing to do. They are nice for lexers, but keep them there
:)



-- 
J.