<br><br><div class="gmail_quote">On Wed, May 25, 2011 at 5:34 AM, Edmond Begumisa <span dir="ltr"><<a href="mailto:ebegumisa@hysteria-tech.com">ebegumisa@hysteria-tech.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
Hello Joe,<br>
<br>
I've personally wanted an easy way to load modules from different places.<br>
<br>
If you started with making modules easy to load from different places, then later provide support for functions, I imagine this would be easier to implement immediately and easier for people to make use of immediately ... where "people" refers to me :)<br>
<br>
I've got two suggestions (mostly inspired by the Mozilla Framework)...<br>
<br>
1. How about letting the programmer choose where the functions come from (kv store, fs, web-server, etc)...<br>
<br>
For instance, a module could start...<br>
<br>
-import(["file://misc/collect_int", "<a href="http://erlang.org/lists/*" target="_blank">http://erlang.org/lists/*</a>", "mnesia://db1/misc/merge_kv"]).<br>
<br>
By default, the code loader would know what to do when it encounters "file://", "http://", "https://", "ftp://", "sftp://" and "mnesia://" URI schemes. Programmers could be allowed define their own URI schemes by providing callbacks for loading code. For instance, one would be able to define "ubf://" and have the code loader call his/her code when it encounters "ubf://foo/bar"<br>
<br>
2. Expanding on this: letting the programmer centralise the possible locations of modules/functions and use this to define namespaces...<br>
<br>
One possible way of doing this is to steal from the "chrome registration" system that the Mozilla Framework uses. A programmer could define one-level deep namespaces in a configuration file which points to targets that could come from anywhere, then use that in her modules. A module could then start...<br>
<br>
-import(["erlang://misc/collect_int", "erlang://couch/mapreduce"]).<br>
<br>
Where "misc" and "couch" are namespaces and "erlang://" is a special URI scheme telling the code loader to refer to the configuration file "erlang.manifest" which would contain...<br>
<br>
[{"misc", "file://./ebin/misc.beam"},<br>
{"couch", "<a href="http://localhost:7777/ebin/" target="_blank">http://localhost:7777/ebin/</a>"}]<br>
<br>
Where "couch" points to a yaws server serving couch beams, or something along those lines. Then she can easy switch between different code sources using one file.<br>
<br>
A code-signing system could also be introduced.<br></blockquote><div><br>Did this a while back - you can also xform everything - all private modules can have their<br>names changed to MD5SUM(Content) of the module IFYSWIM<br>
<br>/Joe<br><br> </div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<br>
- Edmond -<div><div></div><div class="h5"><br>
<br>
<br>
On Tue, 24 May 2011 18:06:19 +1000, Joe Armstrong <<a href="mailto:erlang@gmail.com" target="_blank">erlang@gmail.com</a>> wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
Why do we need modules at all?<br>
<br>
This is a brain-dump-stream-of-consciousness-thing. I've been<br>
thinking about this for a while.<br>
<br>
I'm proposing a slightly different way of programming here<br>
The basic idea is<br>
<br>
- do away with modules<br>
- all functions have unique distinct names<br>
- all functions have (lots of) meta data<br>
- all functions go into a global (searchable) Key-value database<br>
- we need letrec<br>
- contribution to open source can be as simple as<br>
contributing a single function<br>
- there are no "open source projects" - only "the open source<br>
Key-Value database of all functions"<br>
- Content is peer reviewed<br>
<br>
These are discussed in no particular order below:<br>
<br>
Why does Erlang have modules?<br>
<br>
There's a good an bad side to modules:<br>
<br>
Good: Provides a unit of compilation, a unit of code<br>
distribution. unit of code replacement<br>
<br>
Bad: It's very difficult to decide which module to put an individual<br>
function in. Break encapsulation (see later)<br>
<br>
Aside: lib_misc.erl<br>
<br>
When I'm programming I often get to the point were I say there should<br>
a function foo/2 in lists.erl but their isn't. There should be but<br>
there isn't - foo/2 is a small self contained thing. Why should it be<br>
in lists.erl because it "feels right".<br>
<br>
Strings are lists, so why do we have two modules lists.erl and<br>
string.erl how should I decide in which module my new string/list<br>
processing function should go.<br>
<br>
To avoid all mental anguish when I need a small function that<br>
should be somewhere else and isn't I stick it in<br>
a module elib1_misc.erl.<br>
<br>
My elib1_misc exports the following:<br>
<br>
added_files/2 make_challenge/0<br>
as_bits/1 make_response/2<br>
as_bits_test/0 make_response_test/0<br>
bdump/2 make_test_strings/1<br>
bin2hex/1 make_test_strings_test/0<br>
bin2hex_test/0 make_tmp_filename/2<br>
check_io_list/1 merge_kv/1<br>
collect_atom/1 merge_kv_test/0<br>
collect_atom_test/0 mini_shell/0<br>
collect_int/1 module_info/0<br>
collect_int_test/0 module_info/1<br>
collect_string/1 ndots/1<br>
collect_string_test/0 nibble_to_hex_char/1<br>
collect_word/1 nibble_to_hex_char_test/0<br>
complete/2 odd/1<br>
complete_test/0 on_exit/2<br>
dos2unix/1 out_of_date/2<br>
downcase_char/1 outfile/2<br>
dump/2 padd/2<br>
dump_tmp/2 perms/1<br>
duplicates/1 perms_test/0<br>
ensure_started/2 pmap/2<br>
eval_file/1 pmap1/2<br>
eval_file_test/0 pmap1_test/0<br>
eval_string/1 pmap_test/0<br>
eval_string_test/0 priority_receive/0<br>
every/3 random_seed/0<br>
expand_env_vars/1 random_string/1<br>
expand_file_template/3 random_string/2<br>
expand_string_template/2 read_at_most_n_lines/2<br>
expand_tabs/1 read_at_most_n_lines_test/0<br>
expand_tabs_test/0 remove_duplicates/1<br>
expand_template/2 remove_duplicates_test/0<br>
extract_attribute/2 remove_leading_and_trailing_whitespace/1<br>
extract_attribute_test/0 remove_leading_and_trailing_whitespace_test/0<br>
extract_prefix/2 remove_leading_whitespace/1<br>
fetch/2 remove_prefix/2<br>
fetch_test/0 remove_prefix_test/0<br>
file2lines/1 remove_trailing_whitespace/1<br>
file2lines_test/0 replace/3<br>
file2md5/1 root_dir/0<br>
file2numberedlines/1 rpc/2<br>
file2numberedlines_test/0 safe/1<br>
file2paras/1 show_loaded/1<br>
file2stream/1 signed_byte_to_hex_string/1<br>
file2string/1 signed_byte_to_hex_string_test/0<br>
file2template/1 skip_blanks/1<br>
file2term/1 skip_blanks_test/0<br>
file_size_and_type/1 skip_to_nl/1<br>
find_src/1 skip_to_nl_test/0<br>
first/1 sleep/1<br>
flatten_io_list/1 spawn_monitor/3<br>
flush_buffer/0 split_at_char/2<br>
for/3 split_at_char_test/0<br>
force/1 split_list/2<br>
foreach_chunk_in_file/3 split_list_test/0<br>
foreach_word_in_file/2 string2exprs/1<br>
foreach_word_in_string/2 string2exprs_test/0<br>
forever/0 string2html/1<br>
get_erl_section/2 string2latex/1<br>
get_line/1 string2lines/1<br>
get_line/2 string2lines_test/0<br>
have_common_prefix/1 string2stream/1<br>
have_common_prefix_test/0 string2stream_test/0<br>
hex2bin/1 string2template/1<br>
hex2bin_test/0 string2template_test/0<br>
hex2list/1 string2term/1<br>
hex2list_test/0 string2term_test/0<br>
hex_nibble2int/1 string2toks/1<br>
hex_nibble2int_test/0 string2toks_test/0<br>
id/1 sub_binary/3<br>
include_dir/0 template2file/3<br>
include_file/1 term2file/2<br>
interleave/2 term2string/1<br>
is_alphanum/1 test/0<br>
is_blank_line/1 test1_test/0<br>
is_prefix/2 test_function_over_substrings/2<br>
is_prefix_test/0 tex2pdf/1<br>
is_response_correct/3 time_fun/2<br>
keep_alive/2 time_stamp/0<br>
lines2para/1 to_lower/1<br>
list2frequency_distribution/1 to_lower_test/0<br>
list2frequency_distribution_tetrim/1<br>
longest_common_prefix/1 trim_test/0<br>
longest_common_prefix_test/0 unconsult/2<br>
lookup/2 unsigned_byte_to_hex_string/1<br>
lorem/1 unsigned_byte_to_hex_string_test/0<br>
ls/1 which/1<br>
which_added/1<br>
<br>
Now I find this very convenient when I write a new small utility function<br>
I stick in in elib1_misc.erl - no mental anguish in choosing a module<br>
name is involved.<br>
<br>
The observation that I find this very-convenient is telling me something<br>
about modules - I like my elib1_misc it feels right.<br>
<br>
(aside - It seems many development projects have their own private<br>
lib_miscs ...)<br>
<br>
Which brings me to the point of my question.<br>
<br>
Do we need module's at all? Erlang programs are composed of lots of small<br>
functions, the only place where modules seem useful is to hide a letrec.<br>
<br>
The classic example is fibonacci. We want to expose fib/1 but hide the<br>
helper function fib/3. Using modules we say<br>
<br>
-module(math).<br>
-export([fib/1]).<br>
<br>
fib(N) -><br>
fib(N, 1, 0).<br>
<br>
fib(N, A, B) when N < 2 -> A;<br>
fib(N, A, B) -> fib(N-1, A+B, A).<br>
<br>
The downside is we have had to *invent* one module name math - whose *only*<br>
purpose is to hide the definition of fib/3 which we don't want to be made<br>
callable.<br>
<br>
If we put a second function into the module math, then this second function<br>
could call fib/3 which breaks the encapsulation of fib/3.<br>
<br>
We could say:<br>
<br>
let fib = fun(N) -> fib(N, 1, 0) end<br>
in<br>
fib(N, A, B) when N < 2 -> A;<br>
fib(N, A, B) -> fib(N-1, A+B, A).<br>
end.<br>
<br>
I hardly dare suggest a syntax for this since I've been following<br>
another thread in this forum where syntax discussion seem to encourage<br>
much comment.<br>
<br>
** Please do suggest alternative syntax's here - but do not comment on<br>
other peoples suggestions ...<br>
<br>
I would like to just talk about why we have modules.<br>
<br>
Another question:<br>
<br>
Does the idea of a module come from the idea that functions have to be<br>
stored somewhere, so we store them in a file, and we slurp the<br>
file (as a unit) into the system, so the file becomes a module?<br>
<br>
If all the files were store by themselves in a database would this<br>
change things.<br>
<br>
I am thinking more and more that if would be nice to have *all* functions in<br>
a key_value database with unique names.<br>
<br>
lookup(foo,2) would get the definition foo foo/2 from a database.<br>
<br>
The unique names bit is interesting - is this a good idea. Qualified<br>
names (ie names like xxx:foo/2) or (a.b.c.foo/2) sounds like a good<br>
idea but but when I'm programming I have to invent the xxx or the<br>
a.b.c which is very difficult. It also involves the "decision problem"<br>
if the namespaces xxx and a.b.c already exist I have to *choose* which<br>
to put my new function in.<br>
<br>
I think there might be a case for alises here joe:foo/2 could be used<br>
while developing "joe" would expand to a horrible random local string the<br>
real name being ab123aZwerasch123123_foo/2 but I would not be able to<br>
publish my code or make it available to a third_part before I had<br>
chosen a sensible name.<br>
<br>
(( managing namespaces seems really tricky, a lot of peoople seem<br>
to thing that the problem goes away by adding "." 's to the name<br>
but managing a namespace with namees like foo.bar.baz.z is just as complex<br>
as managing a namespace with names like foo_bar_baz_z or names like<br>
0x3af312a78a3f1ae123 - the problem is that we have to go from a symbolic<br>
name like www.a.b to a reference like 123.45.23.12 - but how do we discover<br>
the initial name www.a.b? - there are two answers - a) we are given the name<br>
(ie we click on a link) - we do not know the name but we search fo it ))<br>
<br>
<br>
When programs are small we can live with "just the code" in "a few<br>
modules" the ratio of code to meta data is high.<br>
<br>
When programs are large we need a lot of meta-data to understand them.<br>
<br>
I would like to see all functions with all meta-data in a data base.<br>
<br>
I'd like to say:<br>
<br>
lookup(foo,2,Attribute) when Attribute =<br>
<br>
code|source|documentation|type signatures|revision history|authors|...<br>
<br>
The more I think about it the more I think program development should<br>
viewed as changing the state of a Key-Value database.<br>
<br>
So I imagine:<br>
<br>
1) all functions have unique names<br>
2) there are no modules<br>
3) we discover the name of a function by searching metadata<br>
describing the function in a database<br>
4) all public functions (think open source) are in the same<br>
database<br>
<br>
We could make a system to do this.<br>
<br>
I think this would make open-source projects easier, since the<br>
granularity of contribution goes down. You could contribute<br>
a single function - not an entire application.<br>
<br>
(( A problem with GUT style open source projects is there is<br>
not one database of functions, I often what one function from<br>
this project, another function from another project -- the<br>
granularity of reusable parts should be the individual function.<br>
<br>
functions are really easy to reuse<br>
modules are more difficult to reuse<br>
entire applications are very difficult to reuse<br>
(Unless there are isolated through a communication channel))<br>
<br>
Possible extensions.<br>
<br>
1) Voting for promotion<br>
2) A review process<br>
<br>
Given a raw database will *all* functions in it - we could derive an<br>
"approved" functions database.<br>
<br>
Popular functions could be moved to the approved database - the<br>
review process would need to be discussed - so kind of peer-review/wiki<br>
stuff.<br>
<br>
Comments?<br>
<br>
Volunteers?<br>
<br>
/Joe<br>
</blockquote>
<br>
<br></div></div><font color="#888888">
-- <br>
Using Opera's revolutionary e-mail client: <a href="http://www.opera.com/mail/" target="_blank">http://www.opera.com/mail/</a><br>
</font></blockquote></div><br>