rfc: rdbms - new type system

[ke.han]

Ulf,	
In general I would like to see more primitives for the most common cases:
float, coercedFloat, number, integer
intervals of float, number, integer
enums of any specified list
accept undefined
the attr is a list of another defined type
etc...

This would make the declarations easier to read and would probably make 
the constraint runtime checking more efficient.
	
your example:

[{{typedef, t_age},
  {'and',[{'>=',0},{'=<',150}]}},
  {{attr, age, type},
   {'or',[{type,t_age},{'==',undefined}]}}

  is too messy as allowing undefined is very common.  You are correct 
that undefined messes up user's app code.  I am attempting to remedy 
this in my MVC framework (erlang JSON encoding and javascript code) 
where I handle type mapping between internal and external representation 
and handle missing (undefined) values.

I would prefer something like:

{{attr, age, type},
  {'and', [{'>=', 0},
           {'=<', 150}]},
  {'or', undefined}}

  or perhaps these examples (changing your syntax even more):

  {{attr, age,
  	[{integer, {interval, 0, 150},
	undefined]}

   {{attr, time,
   	[integer,
	{undefined, fun() -> erlang:now() end}]}  % if the value is undefined, 
set the value to be the current time.


0..1 and 0..n relationships are the most common cardinality in 
mainstream apps.  The app spec may say the cardinality should be 1..1 or 
1..n, but the db needs to be able to handle empty lists and undefined 
relation prior to the app code enforcing these rules.

I am imagining high-level declarations such as:

{attr, person,
	{record, person}}  % must be a record of type person
	
{attr, person,
	[{record, person}, 	% must be a record of type person
	undefined]} 		% undefined is acceptable	

{attr, people, {list, person}}  % people is a list of person type and if 
uninitialized will give an empty list. i.e. never undefined

{attr, month,
	{enum, ["June", "July", August"]}} % any value in the list is acceptable

Need to be able to set a number type which accepts both float and 
integers.  This is an annoyance when using io_lib and others that expect 
a float and pass it 1 instead of 1.0.  This causes me to write 
conversion code in my MVC framework.

Something like this would work:

{attr, price, coercedFloat} % ensures that if setting 1 instead of 1.0, 
you always get back 1.0
vs.
{attr, price, float} % only accepts float
	
or

{attr, price, number} %% accepts integers and float but does not coerce; 
  so you get back exactly what you put in


That my feedback for now.
So I see from your example how your defined types.
How are you defining relations?  Any relation integrity constraints?
thanks, ke han


Ulf Wiger (AL/EAB) wrote:
> Rdbms is asymptotically approaching beta status. (:
> 
> Some comments on the following are most welcome.
> 
> I started doing some cleanups and documentation.
> Currently, I'm in some kind of test-doc-driven
> development mode (code a little, write test suites,
> redesign, document, redesign to comply with the 
> documentation, and so on.) I'm using a tiddlyWiki
> for the documentation, which I think fits very
> nicely with this way of working.
> 
> The current holdup is that I'm completely redesigning
> the type system (after all, no-one demanded backwards
> compatibility...) I'm just getting ready to throw out
> the 'required' and 'bounds' properties, as they can be
> expressed as types.
> 
> I removed some old types that were just... messy
> (e.g. 'compound'), as well as 'record' and 'tuples'
> which are now redundant.
> 
> 
> Current approach:
> 
> Typedefs: you can specify {typedef, Name, Type}
> as a table property or a global property. Table
> typedefs shadow global typedefs. It is not 
> possible to shadow predefined types.
> 
> So, 
> 
> [{{attr,age, type}, integer},
>  {{attr,age,bounds}, {inclusive, 0, 100}},
>  {{attr,age,required}, true}]
> 
> now becomes:
> 
> {{attr, age, type},
>  {'and', [{'>=', 0},
>           {'=<', 150}]}}
> 
> In this case, 'integer' and 'required' are implicit.
> 
> In order to allow the value undefined,
> one could do this:
> 
> [{{typedef, t_age},
>  {'and',[{'>=',0},{'=<',150}]}},
>  {{attr, age, type},
>   {'or',[{type,t_age},{'==',undefined}]}}
> 
> ...which is messier than before, but mixing
> 'undefined' and integers in the same attribute
> also messes up the user's code. (:
> 
> It certainly simplifies the 'rdbms' code a lot.
> 
> The remaining other attribute property
> is 'default', which is used e.g. in 
> cascading updates.
> 
> 
> The following type-checking code may serve
> to illustrate the type system
> (which compiles, but hasn't been tested yet):
> 
> %%% check_type(Type, Value) -> boolean()
> %%%
> check_type(false, _) -> false;
> check_type(true, _) -> true;
> check_type(any, _) -> true;
> check_type({tuple, Arity, Ts}, X)
>   when is_tuple(X), size(X) == Arity ->
>     Elems = tuple_to_list(X),
>     lists:zipwith(
>       fun check_type/2, Ts, Elems);
> check_type({tuple, Arity}, X)
>   when is_tuple(X), size(X) == Arity -> true;
> check_type(tuple, X) when is_tuple(X) -> true;
> check_type(integer, X) when is_integer(X) -> true;
> check_type(float, X) when is_float(X) -> true;
> check_type(number, X)
>   when is_integer(X); is_float(X) -> true;
> check_type(atom, X) when is_atom(X) -> true;
> check_type(pid, X) when is_pid(X) -> true;
> check_type(port, X) when is_port(X) -> true;
> check_type(binary, X) when is_binary(X) -> true;
> check_type(string, X) ->
>     try list_to_binary(X) of 
> 	_ -> true
>     catch
> 	error:_ -> false
>     end;
> check_type(list, X) when is_list(X) -> true;
> check_type(nil, [])  -> true;
> check_type({list,false},_) ->
>     false; % optimization
> check_type({list, T}, [_|_])
>   when T==any; T==undefined -> true;
> check_type({list, T}, X) when is_list(X) ->
>     while(true, fun(Elem) ->
>                    check_type(T, Elem)
>                 end, X);
> check_type(function, X)
>   when is_function(X) -> true;
> check_type({function,Arity}, X)
>   when is_function(X, Arity) -> true;
> check_type({'and',Ts}, X) ->
>     lists:foldl(fun(T,Acc) ->
> 		     Acc and check_type(T,X)
> 		end, true, Ts);
> check_type({'andalso',Ts}, X) ->
>     while(true, fun(T) -> check_type(T, X) end, Ts);
> check_type({'or',Ts}, X) ->
>     lists:foldl(fun(T,Acc) ->
> 			Acc or check_type(T,X)
> 		end, false, Ts);
> check_type({'orelse', Ts}, X) ->
>     while(false, fun(T) -> check_type(T, X) end, Ts);
> check_type({'not', T}, X) ->
>     not(check_type(T, X));
> check_type({O,V}, X)
>   when O=='==';O='=/=';O=='<';O=='>';O=='>=';O=='=<' ->
>     erlang:O(X,V);
> check_type(_, _) ->
>     false.
>     
> while(Bool, F, L) when is_list(L) ->
>     Not = not(Bool),
>     try lists:foreach(
> 	  fun(Elem) ->
> 		  case F(Elem) of
> 		      Not ->  throw(Not);
> 		      Bool -> Bool
> 		  end
> 	  end, X) of
> 	_ -> Bool
>     catch
> 	throw:Not ->
> 	    Not
>     end.
> 
> When types are added, I run some code that 
> tries to simplify the types a bit. This code
> is naive at best, and I'm hoping to get some
> help with this later on. The main candidate
> is the {list, T} type. If T can be simplified
> into any of the values 'true', 'false', 'any',
> or 'undefined', then the list won't have to be
> traversed. 
> 
> While doing this, I also had to ask myself 
> what {list, any} actually is supposed to mean.
> Currently, it's 'any non-empty list', as opposed
> to 'nil'. I'm perfectly willing to revisit this,
> since it has the unfortunate side effect that 
> a typed list which includes [] becomes more 
> difficult to specify:
> 
>   {'or', [nil, {list, integer}]}
> 
> Perhaps it would be better to force those 
> who want a non-empty list to write:
> 
>   {'andalso', [{'not',nil}, {list, any}]}
> 
> (Formulating the question, I think I have
> decided to go for the latter. The test-doc-
> driven development process at work again. :)
> 
> BTW, why would one want both 'and' and 
> 'andalso' in this context? Shouldn't it
> always be 'andalso'?
> 
> Regards,
> Ulf W
>