filelib:fold_files/5

[Chris Pressey]

Sorry to reply to my own message, but I've given this some thought, and
at this point I think it's more question than bug report so I'm
forwarding it to erlang-questions.

To recap, filelib:fold_files/5...

- is a great idea
- is documented
- does not work at all :(
- is written to only visit plain files, never directories
- requires a global regexp
- requires a global recursion flag

I think the way to maximize the utility of this function is to...

- make it visit both files and directories and pass an IsDir flag
- make it not require a global regexp (make the fun check, itself)
- let the fun return a local recursion flag (fine-grained recursion)

This way, you can (say) use different regexps on plain files and
directories, and you can skip entire subtrees that you do not want to
scan.  This is much more powerful.  My proposal is a completely
rewritten function, fold_files/3.  fold_files/5 can easily be rewritten
as a wrapper around fold_files/3.  I've included them both below, for
your perusal.



%% @spec fold_files(dir(), fun(), term()) -> term()
%% @doc Folds the function Fun(F, IsDir, Acc) -> {Recurse, Acc1} over
%% all files F in Dir that match the regular expression RegExp.
%% If Recurse is true all sub-directories of F are processed.
%% (This function is a modified version of that from filelib.erl)

fold_files(Dir, Fun, Acc) ->
  case file:list_dir(Dir) of
    {ok, Files} ->
      fold_files0(Files, Dir, Fun, Acc);
    {error, _} ->
      Acc
  end.

fold_files0([File | Tail], Dir, Fun, Acc) ->
  FullName = filename:join([Dir, File]),
  IsDir = filelib:is_dir(FullName),
  {Recurse, NewAcc} = Fun(FullName, IsDir, Acc),
  fold_files0(FullName, Tail, Dir, Fun, IsDir, Recurse, NewAcc);
fold_files0([], Dir, Fun, Acc) ->
  Acc.

fold_files0(FullName, Tail, Dir, Fun, true, true, Acc) ->
  NewAcc = fold_files(FullName, Fun, Acc),
  fold_files0(Tail, Dir, Fun, NewAcc);
fold_files0(FullName, Tail, Dir, Fun, _, _, Acc) ->
  fold_files0(Tail, Dir, Fun, Acc).

%% @spec fold_files(dir(), regexp(), bool(), fun(), term()) -> term()
%% Wrapper for the original fold_files/5 behaviour.

fold_files(Dir, RegExp, Recursive, Fun, InitialAcc) ->
  {ok, CompiledRegExp} = regexp:parse(RegExp),
  Wrapper = fun
    (FullName, false, Acc) ->
      NewAcc = case regexp:match(FullName, CompiledRegExp) of
        {match, _, _}  -> 
          Fun(FullName, Acc);
        _ ->
          Acc
      end,
      {Recursive, NewAcc};
    (_, true, Acc) ->
      {Recursive, Acc}
  end,
  fold_files(Dir, Wrapper, InitialAcc).



On Tue, 16 Sep 2003 16:00:15 -0700
Chris Pressey <cpressey@REDACTED> wrote:

> Hi,
> 
> No offense to Mr. Gustavsson, but filelib:fold_files/5 looks like it
> was written while he was asleep :)
> 
> Attached is a patch which makes it (at least) work.  But I think it
> would be good to consider a change to its definition first.
> 
> With the present behaviour, the callback never sees directory entries
> themselves.  So, for example, empty directories are never detected.  I
> think the behaviour would be more useful if it called the fun for both
> plain files and directories.  The callback could also accept a flag,
> like fun(Filename, IsDir, Acc), so that it doesn't have to do the test
> itself (since fold_files/5 already does it anyway.)
> 
> What do you think?
> 
> Other than that one issue, I like the filelib module a lot!  Keep up
> the good work.
> 
> -Chris
>