When performance is of highest priority and one is interested in a limited part of the ASN.1 encoded message, before one decide what to do with the rest of it, one may want to decode only this small part. The situation may be a server that has to decide to which addressee it will send a message. The addressee may be interested in the entire message, but the server may be a bottleneck that one want to spare any unnecessary load. Instead of making two complete decodes (the normal case of decode), one in the server and one in the addressee, it is only necessary to make one specialized decode(in the server) and another complete decode(in the addressee). The following specialized decodes exclusive decode and selected decode support to solve this and similar problems.
So far this functionality is only provided when using the optimized BER_BIN version, that is when compiling with the options ber_bin and optimize. It does also work using the driver option. We have no intent to make this available on the default BER version, but maybe in the PER_BIN version (per_bin).
The basic idea with exclusive decode is that you specify which parts of the message you want to exclude from being decoded. These parts remain encoded and are returned in the value structure as binaries. They may be decoded in turn by passing them to a certain decode_part/2 function. The performance gain is high when the message is large and you can do an exclusive decode and later on one or several decodes of the parts or a second complete decode instead of two or more complete decodes.
In order to make exclusive decode work you have to do the following:
The run-time user interface for exclusive decode consists of two different functions. First, the function for an exclusive decode, whose name the user decides in the configuration file. Second, the compiler generates a decode_part/2 function when exclusive decode is chosen. This function decodes the parts that were left undecoded during the exclusive decode. Both functions are described below.
If the exclusive decode function has for example got the name decode_exclusive and an ASN.1 encoded message Bin shall be exclusive decoded, the call is:
{ok,Excl_Message} = 'MyModule':decode_exclusive(Bin)
The result Excl_Message has the same structure as an complete decode would have, except for the parts of the top-type that were not decoded. The undecoded parts will be on their place in the structure on the format {Type_Key,Undecoded_Value}.
Each undecoded part that shall be decoded must be fed into the decode_part/2 function,like:
{ok,Part_Message} = 'MyModule':decode_part(Type_Key,Undecoded_Value)
This instruction is written in the configuration file on the format:
Exclusive_Decode_Instruction = {exclusive_decode,{Module_Name,Decode_Instructions}}. Module_Name = atom() Decode_Instructions = [Decode_Instruction]+ Decode_Instruction = {Exclusive_Decode_Function_Name,Type_List} Exclusive_Decode_Function_Name = atom() Type_List = [Top_Type,Element_List] Element_List = [Element]+ Element = {Name,parts} | {Name,undecoded} | {Name,Element_List} Top_Type = atom() Name = atom()
Observe that the instruction must be a valid Erlang term ended by a dot.
In the Type_List the "path" from the top type to each undecoded sub-components is described. The top type of the path is an atom, the name of it. The action on each component/type that follows will be described by one of {Name,parts}, {Name,undecoded}, {Name,Element_List}
The use and effect of the actions are:
Name in the actions above may be a component name of a SEQUENCE or a SET or a name of an alternative in a CHOICE.
In the examples below we use the definitions from the following ASN.1 spec:
GUI DEFINITIONS AUTOMATIC TAGS ::= BEGIN Action ::= SEQUENCE { number INTEGER DEFAULT 15, handle [0] Handle DEFAULT {number 12, on TRUE} } Key ::= [11] EXPLICIT Button Handle ::= [12] Key Button ::= SEQUENCE { number INTEGER, on BOOLEAN } Window ::= CHOICE { vsn INTEGER, status E } Status ::= SEQUENCE { state INTEGER, buttonList SEQUENCE OF Button, enabled BOOLEAN OPTIONAL, actions CHOICE { possibleActions SEQUENCE OF Action, noOfActions INTEGER } } END
If Button is a top type and we want to exclude component number from decode the Type_List in the instruction in the configuration file will be ['Button',[{number,undecoded}]]. If we call the decode function decode_Button_exclusive the Decode_Instruction will be {decode_Button_exclusive,['Button',[{number,undecoded}]]}.
We also have another top type Window whose sub component actions in type Status and the parts of component buttonList shall be left undecoded. For this type we name the function decode__Window_exclusive. The whole Exclusive_Decode_Instruction configuration is as follows:
{exclusive_decode,{'GUI', [{decode_Window_exclusive,['Window',[{status,[{buttonList,parts},{actions,undecoded}]}]]}, {decode_Button_exclusive,['Button',[{number,undecoded}]]}]}}.
Compiling GUI.asn including the configuration file is done like:
unix> erlc -bber_bin +optimize +asn1config GUI.asn erlang> asn1ct:compile('GUI',[ber_bin,optimize,asn1config]).
The module can be used like:
1> Button_Msg = {'Button',123,true}. {'Button',123,true} 2> {ok,Button_Bytes} = 'GUI':encode('Button',Button_Msg). {ok,[<<48>>, [6], [<<128>>, [1], 123], [<<129>>, [1], 255]]} 3> {ok,Exclusive_Msg_Button} = 'GUI':decode_Button_exclusive(list_to_binary(Button_Bytes)). {ok,{'Button',{'Button_number',<<28,1,123>>}, true}} 4> 'GUI':decode_part('Button_number',<<128,1,123>>). {ok,123} 5> Window_Msg = {'Window',{status,{'Status',35, [{'Button',3,true}, {'Button',4,false}, {'Button',5,true}, {'Button',6,true}, {'Button',7,false}, {'Button',8,true}, {'Button',9,true}, {'Button',10,false}, {'Button',11,true}, {'Button',12,true}, {'Button',13,false}, {'Button',14,true}], false, {possibleActions,[{'Action',16,{'Button',17,true}}]}}}}. {'Window',{status,{'Status',35, [{'Button',3,true}, {'Button',4,false}, {'Button',5,true}, {'Button',6,true}, {'Button',7,false}, {'Button',8,true}, {'Button',9,true}, {'Button',10,false}, {'Button',11,true}, {'Button',12,true}, {'Button',13,false}, {'Button',14,true}], false, {possibleActions,[{'Action',16,{'Button',17,true}}]}}}} 6> {ok,Window_Bytes}='GUI':encode('Window',Window_Msg). {ok,[<<161>>, [127], [<<128>>, ... 8> {ok,{status,{'Status',Int,{Type_Key_SeqOf,Val_SEQOF}, BoolOpt,{Type_Key_Choice,Val_Choice}}}}= 'GUI':decode_Window_status_exclusive(list_to_binary(Window_Bytes)). {ok,{status,{'Status',35, {'Status_buttonList',[<<48,6,128,1,3,129,1,255>>, <<48,6,128,1,4,129,1,0>>, <<48,6,128,1,5,129,1,255>>, <<48,6,128,1,6,129,1,255>>, <<48,6,128,1,7,129,1,0>>, <<48,6,128,1,8,129,1,255>>, <<48,6,128,1,9,129,1,255>>, <<48,6,128,1,10,129,1,0>>, <<48,6,128,1,11,129,1,255>>, <<48,6,128,1,12,129,1,255>>, <<48,6,128,1,13,129,1,0>>, <<48,6,128,1,14,129,1,255>>]}, false, {'Status_actions', <<163,21,160,19,48,17,2,1,16,160,12,172,10,171,8,48,6,128,1,...>>}}}} 10> 'GUI':decode_part(Type_Key_SeqOf,Val_SEQOF). {ok,[{'Button',3,true}, {'Button',4,false}, {'Button',5,true}, {'Button',6,true}, {'Button',7,false}, {'Button',8,true}, {'Button',9,true}, {'Button',10,false}, {'Button',11,true}, {'Button',12,true}, {'Button',13,false}, {'Button',14,true}]} 11> 'GUI':decode_part(Type_Key_SeqOf,hd(Val_SEQOF)). {ok,{'Button',3,true}} 12> 'GUI':decode_part(Type_Key_Choice,Val_Choice). {ok,{possibleActions,[{'Action',16,{'Button',17,true}}]}}
This specialized decode decodes one single subtype of a constructed value. It is the fastest method to extract one sub value. The typical use of this decode is when one want to inspect, for instance a version number,to be able to decide what to do with the entire value. The result is returned as {ok,Value} or {error,Reason}.
The following steps are necessary:
The only new user interface function is the one provided by the user in the configuration file. You can invoke that function by the ModuleName:FunctionName notation.
So, if you have the following spec {selective_decode,{'ModuleName',[{selected_decode_Window,TypeList}]}} in the con-fig file, you do the selective decode by {ok,Result}='ModuleName':selected_decode_Window(EncodedBinary).
It is possible to describe one or many selective decode functions in a configuration file, you have to use the following notation:
Selective_Decode_Instruction = {selective_decode,{Module_Name,Decode_Instructions}}. Module_Name = atom() Decode_Instructions = [Decode_Instruction]+ Decode_Instruction = {Selective_Decode_Function_Name,Type_List} Selective_Decode_Function_Name = atom() Type_List = [Top_Type|Element_List] Element_List = Name|List_Selector Name = atom() List_Selector = [integer()]
Observe that the instruction must be a valid Erlang term ended by a dot.
The Module_Name is the same as the name of the ASN.1 spec, but without the extension. A Decode_Instruction is a tuple with your chosen function name and the components from the top type that leads to the single type you want to decode. Notice that you have to choose a name of your function that will not be the same as any of the generated functions. The first element of the Type_List is the top type of the encoded message. In the Element_List it is followed by each of the component names that leads to selected type. Each of the names in the Element_List must be constructed types except the last name, which can be any type.
The List_Selector makes it possible to choose one of the encoded components in a SEQUENCE OF/ SET OF. It is also possible to go further in that component and pick a sub type of that to decode. So in the Type_List: ['Window',status,buttonList,[1],number] the component buttonList has to be a SEQUENCE OF or SET OF type. In this example component number of the first of the encoded elements in the SEQUENCE OF buttonList is selected. This apply on the ASN.1 spec above.
In this example we use the same ASN.1 spec as above. A valid selective decode instruction is:
{selective_decode, {'GUI', [{selected_decode_Window1, ['Window',status,buttonList, [1], number]}, {selected_decode_Action, ['Action',handle,number]}, {selected_decode_Window2, ['Window', status, actions, possibleActions, [1], handle,number]}]}}.
The first Decode_Instruction, {selected_decode_Window1,['Window',status,buttonList,[1],number]} is commented in the previous section. The instruction {selected_decode_Action,['Action',handle,number]} picks the component number in the handle component of the type Action. If we have the value ValAction = {'Action',17,{'Button',4711,false}} the internal value 4711 should be picked by selected_decode_Action. In an Erlang terminal it looks like:
ValAction = {'Action',17,{'Button',4711,false}}. {'Action',17,{'Button',4711,false}} 7> {ok,Bytes}='GUI':encode('Action',ValAction). ... 8> BinBytes = list_to_binary(Bytes). <<48,18,2,1,17,160,13,172,11,171,9,48,7,128,2,18,103,129,1,0>> 9> 'GUI':selected_decode_Action(BinBytes). {ok,4711} 10>
The third instruction, ['Window',status,actions,possibleActions,[1],handle,number], which is a little more complicated,
The following figures shows which components are in the TypeList ['Window',status,actions,possibleActions,[1],handle,number]. And which part of a message that will be decoded by selected_decode_Window2.
With the following example you can examine that both selected_decode_Window2 and selected_decode_Window1 decodes the intended sub-value of the value Val
1> Val = {'Window',{status,{'Status',12, [{'Button',13,true}, {'Button',14,false}, {'Button',15,true}, {'Button',16,false}], true, {possibleActions,[{'Action',17,{'Button',18,false}}, {'Action',19,{'Button',20,true}}, {'Action',21,{'Button',22,false}}]}}}} 2> {ok,Bytes}='GUI':encode('Window',Val). ... 3> Bin = list_to_binary(Bytes). <<161,101,128,1,12,161,32,48,6,128,1,13,129,1,255,48,6,128,1,14,129,1,0,48,6,128,1,15,129,...>> 4> 'GUI':selected_decode_Window1(Bin). {ok,13} 5> 'GUI':selected_decode_Window2(Bin). {ok,18}
Observe that the value feed into the selective decode functions must be a binary.
To give an indication on the possible performance gain using the specialized decodes, some measures have been performed. The relative figures in the outcome between selective, exclusive and complete decode (the normal case) depends on the structure of the type, the size of the message and on what level the selective and exclusive decodes are specified.
The specs GUI and MEDIA-GATEWAY-CONTROL was used in the test.
For the GUI spec the configuration looked like:
{selective_decode, {'GUI', [{selected_decode_Window1, ['Window', status,buttonList, [1], number]}, {selected_decode_Window2, ['Window', status, actions, possibleActions, [1], handle,number]}]}}. {exclusive_decode, {'GUI', [{decode_Window_status_exclusive, ['Window', [{status, [{buttonList,parts}, {actions,undecoded}]}]]}]}}.
The MEDIA-GATEWAY-CONTROL configuration was:
{exclusive_decode, {'MEDIA-GATEWAY-CONTROL', [{decode_MegacoMessage_exclusive, ['MegacoMessage', [{authHeader,undecoded}, {mess, [{mId,undecoded}, {messageBody,undecoded}]}]]}]}}. {selective_decode, {'MEDIA-GATEWAY-CONTROL', [{decode_MegacoMessage_selective, ['MegacoMessage',mess,version]}]}}.
The corresponding values were:
{'Window',{status,{'Status',12, [{'Button',13,true}, {'Button',14,false}, {'Button',15,true}, {'Button',16,false}, {'Button',13,true}, {'Button',14,false}, {'Button',15,true}, {'Button',16,false}, {'Button',13,true}, {'Button',14,false}, {'Button',15,true}, {'Button',16,false}], true, {possibleActions, [{'Action',17,{'Button',18,false}}, {'Action',19,{'Button',20,true}}, {'Action',21,{'Button',22,false}}, {'Action',17,{'Button',18,false}}, {'Action',19,{'Button',20,true}}, {'Action',21,{'Button',22,false}}, {'Action',17,{'Button',18,false}}, {'Action',19,{'Button',20,true}}, {'Action',21,{'Button',22,false}}, {'Action',17,{'Button',18,false}}, {'Action',19,{'Button',20,true}}, {'Action',21,{'Button',22,false}}, {'Action',17,{'Button',18,false}}, {'Action',19,{'Button',20,true}}, {'Action',21,{'Button',22,false}}, {'Action',17,{'Button',18,false}}, {'Action',19,{'Button',20,true}}, {'Action',21,{'Button',22,false}}]}}}} {'MegacoMessage',asn1_NOVALUE, {'Message',1, {ip4Address, {'IP4Address',[125,125,125,111],55555}}, {transactions, [{transactionReply, {'TransactionReply',50007,asn1_NOVALUE, {actionReplies, [{'ActionReply',0,asn1_NOVALUE,asn1_NOVALUE, [{auditValueReply,{auditResult,{'AuditResult', {'TerminationID',[],[255,255,255]}, [{mediaDescriptor, {'MediaDescriptor',asn1_NOVALUE, {multiStream, [{'StreamDescriptor',1, {'StreamParms', {'LocalControlDescriptor', sendRecv, asn1_NOVALUE, asn1_NOVALUE, [{'PropertyParm', [0,11,0,7], [[52,48]], asn1_NOVALUE}]}, {'LocalRemoteDescriptor', [[{'PropertyParm', [0,0,176,1], [[48]], asn1_NOVALUE}, {'PropertyParm', [0,0,176,8], [[73,78,32,73,80,52,32,49,50,53,46,49, 50,53,46,49,50,53,46,49,49,49]], asn1_NOVALUE}, {'PropertyParm', [0,0,176,15], [[97,117,100,105,111,32,49,49,49,49,32, 82,84,80,47,65,86,80,32,32,52]], asn1_NOVALUE}, {'PropertyParm', [0,0,176,12], [[112,116,105,109,101,58,51,48]], asn1_NOVALUE}]]}, {'LocalRemoteDescriptor', [[{'PropertyParm', [0,0,176,1], [[48]], asn1_NOVALUE}, {'PropertyParm', [0,0,176,8], [[73,78,32,73,80,52,32,49,50,52,46,49,50, 52,46,49,50,52,46,50,50,50]], asn1_NOVALUE}, {'PropertyParm', [0,0,176,15], [[97,117,100,105,111,32,50,50,50,50,32,82, 84,80,47,65,86,80,32,32,52]], asn1_NOVALUE}, {'PropertyParm', [0,0,176,12], [[112,116,105,109,101,58,51,48]], asn1_NOVALUE}]]}}}]}}}, {packagesDescriptor, [{'PackagesItem',[0,11],1}, {'PackagesItem',[0,11],1}]}, {statisticsDescriptor, [{'StatisticsParameter',[0,12,0,4],[[49,50,48,48]]}, {'StatisticsParameter',[0,11,0,2],[[54,50,51,48,48]]}, {'StatisticsParameter',[0,12,0,5],[[55,48,48]]}, {'StatisticsParameter',[0,11,0,3],[[52,53,49,48,48]]}, {'StatisticsParameter',[0,12,0,6],[[48,46,50]]}, {'StatisticsParameter',[0,12,0,7],[[50,48]]}, {'StatisticsParameter',[0,12,0,8],[[52,48]]}]}]}}}]}]}}}]}}}
The size of the encoded values was 458 bytes for GUI and 464 bytes for MEDIA-GATEWAY-CONTROL.
The ASN.1 specs in the test are compiled with the options ber_bin, optimize, driver and asn1config. If the driver option had been omitted there should have been higher values for decode and decode_part.
The test program runs 10000 decodes on the value, resulting in a printout with the elapsed time in microseconds for the total number of decodes.
Function | Time(microseconds) | Kind of Decode | ASN.1 spec | % of time vs. complete decode |
decode_MegacoMessage_selective/1 | 374045 | selective | MEDIA-GATEWAY-CONTROL | 8.3 |
decode_MegacoMessage_exclusive/1 | 621107 | exclusive | MEDIA-GATEWAY-CONTROL | 13.8 |
decode/2 | 4507457 | complete | MEDIA-GATEWAY-CONTROL | 100 |
selected_decode_Window1/1 | 449585 | selective | GUI | 7.6 |
selected_decode_Window2/1 | 890666 | selective | GUI | 15.1 |
decode_Window_status_exclusive/1 | 1251878 | exclusive | GUI | 21.3 |
decode/2 | 5889197 | complete | GUI | 100 |
Another interesting question is what the relation is between a complete decode, an exclusive decode followed by decode_part of the excluded parts and a selective decode followed by a complete decode. Some situations may be compared to this simulation, e.g. inspect a sub-value and later on look at the entire value. The following table shows figures from this test. The number of loops and time unit is the same as in the previous test.
Actions | Function | Time(microseconds) | ASN.1 spec | % of time vs. complete decode |
complete | decode/2 | 4507457 | MEDIA-GATEWAY-CONTROL | 100 |
selective and complete | decode_MegacoMessage_selective/1 | 4881502 | MEDIA-GATEWAY-CONTROL | 108.3 |
exclusive and decode_part | decode_MegacoMessage_exclusive/1 | 5481034 | MEDIA-GATEWAY-CONTROL | 112.3 |
complete | decode/2 | 5889197 | GUI | 100 |
selective and complete | selected_decode_Window1/1 | 6337636 | GUI | 107.6 |
selective and complete | selected_decode_Window2/1 | 6795319 | GUI | 115.4 |
exclusive and decode_part | decode_Window_status_exclusive/1 | 6249200 | GUI | 106.1 |
Other ASN.1 types and values can differ much from these figures. Therefore it is important that you, in every case where you intend to use either of these decodes, perform some tests that shows if you will benefit your purpose.
Generally speaking the gain of selective and exclusive decode in advance of complete decode is greater the bigger value and the less deep in the structure you have to decode. One should also prefer selective decode instead of exclusive decode if you are interested in just one single sub-value.
Another observation is that the exclusive decode followed by decode_part decodes is very attractive if the parts will be sent to different servers for decoding or if one in some cases not is interested in all parts.
The fastest selective decode are when the decoded type is a primitive type and not so deep in the structure of the top type. The selected_decode_Window2 decodes a big constructed value, which explains why this operation is relatively slow.
It may vary from case to case which combination of selective/complete decode or exclusive/part decode is the fastest.