[erlang-bugs] ssh2_msg_channel_failure

Ingela Anderton Andin ingela@REDACTED
Wed May 2 12:30:45 CEST 2012


Hi!

I ran your code and I think the problem is that your application is not 
running as root and you do not specify a system_dir. The default 
system_dir usually requiers root
privilages. I also think we need to have some better handling of this 
error in ssh, we probably need to move the check so that it will be 
noticed earlier and give an errormessage when starting the server rather 
than getting an internal error when someone tries to connect.

Regards Ingela Erlang OTP team -Ericsson AB**

Ingela Anderton Andin wrote:
> Hi!
>
> Fyodor Ustinov wrote:
>> Hi!
>>
>> putty say "Server send disconnect message type 11 (by application): 
>> "Internal error"
>>
>> Erlang say:
>>
> Interesting! It seems like the server is trying to handle a request 
> for a channel that has for some reason been closed.
> If i modify the patch like this ...
>
> diff --git a/lib/ssh/src/ssh_connection.erl 
> b/lib/ssh/src/ssh_connection.erl
> index 8574e08..fbd4967 100644
> --- a/lib/ssh/src/ssh_connection.erl
> +++ b/lib/ssh/src/ssh_connection.erl
> @@ -723,11 +723,14 @@ 
> handle_msg(#ssh_msg_channel_request{recipient_channel = ChannelId,
>                                    want_reply = WantReply},  
> #connection{channel_cache = Cache} = Connection,
>           ConnectionPid, _) ->
>     if WantReply == true ->
> -           #channel{remote_id = RemoteId} = ChannelId =
> -               ssh_channel:cache_lookup(Cache, ChannelId),
> -           FailMsg = channel_failure_msg(RemoteId),
> -           {{replies, [{connection_reply, ConnectionPid, FailMsg}]},
> -            Connection};
> +               case ssh_channel:cache_lookup(Cache, ChannelId) of
> +                   #channel{remote_id = RemoteId} = ChannelId ->
> +                       FailMsg = channel_failure_msg(RemoteId),
> +                       {{replies, [{connection_reply, ConnectionPid, 
> FailMsg}]},
> +                        Connection};
> +                   undefined -> %% Chanel has been closed
> +                       {noreply, Connection}
> +               end;
>        true ->
>            {noreply, Connection}
>     end;
>
> ... you should not get the below crash. But I am wondering if your 
> application will
> now behave as you expect?  Can you give me any more information on the 
> senario
> of your application. It would like to be able to write a minimal test 
> case that provokes
> the problem.
>
>
>> =ERROR REPORT==== 25-Apr-2012::20:03:09 ===
>> ** Generic server <0.156.0> terminating
>> ** Last message in was {ssh_msg,<0.157.0>,
>> <<98,0,0,0,0,0,0,0,34,119,105,110,97,100,106,
>>                                   
>> 64,112,117,116,116,121,46,112,114,111,106,
>>                                   
>> 101,99,116,115,46,116,97,114,116,97,114,117,
>>                                   115,46,111,114,103,1>>}
>> ** When Server state == {state,server,undefined,undefined,<0.157.0>,
>>                             {connection,[],315439,[],1,
>>                                 {ssh_cli,[#Fun<sshd.0.34972026>]},
>>                                 {10,128,0,10},
>>                                 2222,
>>                                 [{role,server},
>>                                  {pwdfun,#Fun<sshd.pwcheck.2>},
>>                                  {shell,#Fun<sshd.0.34972026>}],
>>                                 undefined,<0.153.0>,undefined},
>>                             0,
>>                             [{address,{10,128,0,10}},
>>                              {port,2222},
>>                              {role,server},
>>                              {socket_opts,[inet,{ip,{10,128,0,10}}]},
>>                              {ssh_opts,
>>                                  [{role,server},
>>                                   {pwdfun,#Fun<sshd.pwcheck.2>},
>>                                   {shell,#Fun<sshd.0.34972026>}]}],
>>                             undefined,true}
>> ** Reason for termination ==
>> ** {{badmatch,
>>         {disconnect,
>>             {11,"Internal error"},
>>             {{replies,[{channel_data,<0.159.0>,{closed,0}}]},
>>              {connection,[],315439,[],1,
>>                  {ssh_cli,[#Fun<sshd.0.34972026>]},
>>                  {10,128,0,10},
>>                  2222,
>>                  [{role,server},
>>                   {pwdfun,#Fun<sshd.pwcheck.2>},
>>                   {shell,#Fun<sshd.0.34972026>}],
>>                  undefined,<0.153.0>,undefined}}}},
>>     [{ssh_connection_manager,handle_call,3,
>>          [{file,"../src/ssh_connection_manager.erl"},{line,271}]},
>>      {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,588}]},
>>      {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]}
>>
>> =ERROR REPORT==== 25-Apr-2012::20:03:09 ===
>> Erlang ssh connection handler failed with reason: {{{badmatch,
>>                                                      {disconnect,
>>                                                       {11,"Internal 
>> error"},
>>                                                       {{replies,
>>                                                         [{channel_data,
>> <0.159.0>,
>>                                                           {closed,0}}]},
>>                                                        
>> {connection,[],315439,
>>                                                         [],1,
>>                                                         {ssh_cli,
>>                                                          
>> [#Fun<sshd.0.34972026>]},
>>                                                         {10,128,0,10},
>>                                                         2222,
>>                                                         [{role,server},
>>                                                          {pwdfun,
>>                                                           
>> #Fun<sshd.pwcheck.2>},
>>                                                          {shell,
>>                                                           
>> #Fun<sshd.0.34972026>}],
>>                                                         
>> undefined,<0.153.0>,
>>                                                         undefined}}}},
>>                                                     
>> [{ssh_connection_manager,
>>                                                       handle_call,3,
>>                                                       [{file,
>>                                                         
>> "../src/ssh_connection_manager.erl"},
>>                                                        {line,271}]},
>>                                                      
>> {gen_server,handle_msg,
>>                                                       5,
>>                                                       
>> [{file,"gen_server.erl"},
>>                                                        {line,588}]},
>>                                                      {proc_lib,
>>                                                       init_p_do_apply,3,
>>                                                       
>> [{file,"proc_lib.erl"},
>>                                                        {line,227}]}]},
>>                                                    {gen_server,call,
>>                                                     [<0.156.0>,
>>                                                      {ssh_msg,<0.157.0>,
>> <<93,0,0,0,0,0,0,32,106>>},
>>                                                      infinity]}}
>> , Stacktace: [{gen_server,call,3,[{file,"gen_server.erl"},{line,188}]},
>>               {ssh_connection_manager,call,3,
>>                   
>> [{file,"../src/ssh_connection_manager.erl"},{line,583}]},
>>               {ssh_connection_handler,generate_event,4,
>>                   [{file,"ssh_connection_handler.erl"},{line,804}]},
>>               {gen_fsm,handle_msg,7,[{file,"gen_fsm.erl"},{line,494}]},
>>               
>> {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]
>> please report this to erlang-bugs@REDACTED
>> =ERROR REPORT==== 25-Apr-2012::20:03:09 ===
>> ** State machine <0.157.0> terminating
>> ** Last message in was {tcp,#Port<0.1845>,<<>>}
>> ** When State == connected
>> **      Data  == {state,tcp,gen_tcp,tcp_closed,
>>                      {ssh,server,
>>                          {undefined,{{10,128,0,1},1076}},
>>                          {2,0},
>>                          {2,0},
>>                          "SSH-2.0-PuTTY_Release_0.62","SSH-2.0-Erlang",
>> <<20,231,36,36,12,195,57,51,47,209,229,164,97,204,116,
>>                            
>> 116,90,0,0,0,154,100,105,102,102,105,101,45,104,101,
>>                            
>> 108,108,109,97,110,45,103,114,111,117,112,45,101,
>>                            
>> 120,99,104,97,110,103,101,45,115,104,97,50,53,54,44,
>>                            
>> 100,105,102,102,105,101,45,104,101,108,108,109,97,
>>                            
>> 110,45,103,114,111,117,112,45,101,120,99,104,97,110,
>>                            
>> 103,101,45,115,104,97,49,44,100,105,102,102,105,101,
>>                            
>> 45,104,101,108,108,109,97,110,45,103,114,111,117,
>>                            
>> 112,49,52,45,115,104,97,49,44,100,105,102,102,105,
>>                            
>> 101,45,104,101,108,108,109,97,110,45,103,114,111,
>>                            
>> 117,112,49,45,115,104,97,49,44,114,115,97,50,48,52,
>>                            
>> 56,45,115,104,97,50,53,54,44,114,115,97,49,48,50,52,
>>                            
>> 45,115,104,97,49,0,0,0,15,115,115,104,45,114,115,97,
>>                            
>> 44,115,115,104,45,100,115,115,0,0,0,159,97,101,115,
>>                            
>> 50,53,54,45,99,116,114,44,97,101,115,50,53,54,45,99,
>>                            
>> 98,99,44,114,105,106,110,100,97,101,108,45,99,98,99,
>>                            
>> 64,108,121,115,97,116,111,114,46,108,105,117,46,115,
>>                            
>> 101,44,97,101,115,49,57,50,45,99,116,114,44,97,101,
>>                            
>> 115,49,57,50,45,99,98,99,44,97,101,115,49,50,56,45,
>>                            
>> 99,116,114,44,97,101,115,49,50,56,45,99,98,99,44,98,
>>                            
>> 108,111,119,102,105,115,104,45,99,116,114,44,98,108,
>>                            
>> 111,119,102,105,115,104,45,99,98,99,44,51,100,101,
>>                            
>> 115,45,99,116,114,44,51,100,101,115,45,99,98,99,44,
>>                            
>> 97,114,99,102,111,117,114,50,53,54,44,97,114,99,102,
>>                            
>> 111,117,114,49,50,56,0,0,0,159,97,101,115,50,53,54,
>>                            
>> 45,99,116,114,44,97,101,115,50,53,54,45,99,98,99,44,
>>                            
>> 114,105,106,110,100,97,101,108,45,99,98,99,64,108,
>>                            
>> 121,115,97,116,111,114,46,108,105,117,46,115,101,44,
>>                            
>> 97,101,115,49,57,50,45,99,116,114,44,97,101,115,49,
>>                            
>> 57,50,45,99,98,99,44,97,101,115,49,50,56,45,99,116,
>>                            
>> 114,44,97,101,115,49,50,56,45,99,98,99,44,98,108,
>>                            
>> 111,119,102,105,115,104,45,99,116,114,44,98,108,111,
>>                            
>> 119,102,105,115,104,45,99,98,99,44,51,100,101,115,
>>                            
>> 45,99,116,114,44,51,100,101,115,45,99,98,99,44,97,
>>                            
>> 114,99,102,111,117,114,50,53,54,44,97,114,99,102,
>>                            
>> 111,117,114,49,50,56,0,0,0,31,104,109,97,99,45,115,
>>                            
>> 104,97,49,44,104,109,97,99,45,115,104,97,49,45,57,
>>                            
>> 54,44,104,109,97,99,45,109,100,53,0,0,0,31,104,109,
>>                            
>> 97,99,45,115,104,97,49,44,104,109,97,99,45,115,104,
>>                            
>> 97,49,45,57,54,44,104,109,97,99,45,109,100,53,0,0,0,
>>                            
>> 9,110,111,110,101,44,122,108,105,98,0,0,0,9,110,111,
>>                            
>> 110,101,44,122,108,105,98,0,0,0,0,0,0,0,0,0,0,0,0,0>>,
>> <<20,68,106,202,126,189,248,249,152,175,237,102,43,
>>                            
>> 239,21,11,93,0,0,0,26,100,105,102,102,105,101,45,
>>                            
>> 104,101,108,108,109,97,110,45,103,114,111,117,112,
>>                            
>> 49,45,115,104,97,49,0,0,0,15,115,115,104,45,114,115,
>>                            
>> 97,44,115,115,104,45,100,115,115,0,0,0,19,97,101,
>>                            
>> 115,49,50,56,45,99,98,99,44,51,100,101,115,45,99,98,
>>                            
>> 99,0,0,0,19,97,101,115,49,50,56,45,99,98,99,44,51,
>>                            
>> 100,101,115,45,99,98,99,0,0,0,9,104,109,97,99,45,
>>                            
>> 115,104,97,49,0,0,0,9,104,109,97,99,45,115,104,97,
>>                            
>> 49,0,0,0,9,110,111,110,101,44,122,108,105,98,0,0,0,
>>                            
>> 9,110,111,110,101,44,122,108,105,98,0,0,0,0,0,0,0,0,
>>                            0,0,0,0,0>>,
>>                          
>> undefined,'diffie-hellman-group1-sha1','ssh-rsa',
>>                          ssh_file,ssh_io,'hmac-sha1',
>> <<92,89,171,169,113,136,6,188,156,238,58,144,38,214,
>>                            101,198,247,166,205,208>>,
>>                          20,'hmac-sha1',
>> <<24,206,173,99,5,252,206,70,166,228,30,17,101,89,36,
>>                            241,175,116,238,86>>,
>>                          20,'aes128-cbc',
>> <<45,208,196,236,205,82,244,222,24,132,67,151,22,62,
>>                            250,150>>,
>>                          16,
>> <<219,113,112,85,250,220,170,249,194,198,230,158,76,
>>                            122,39,155>>,
>>                          'aes128-cbc',
>> <<44,157,215,194,90,86,202,31,228,49,188,97,227,97,
>>                            233,144>>,
>>                          16,
>> <<196,110,206,199,93,111,201,100,74,194,231,242,179,
>>                            201,15,112>>,
>>                          none,undefined,none,undefined,none,none,true,
>>                          infinity,
>>                          
>> 132151717501655075704740428602371821313660310182519774905429178494543592682738802201444120687073445805819575914576401425536104160589136387731343596760889931670361900328999551747697001054684538185039596790225261438983095509229017954492065022996307622720613396529860286665422597229362632923823896919144441708303, 
>>
>> <<163,238,219,114,10,240,95,30,207,178,78,138,146,58,
>>                            87,102,43,69,173,214>>,
>> <<163,238,219,114,10,240,95,30,207,178,78,138,146,58,
>>                            87,102,43,69,173,214>>,
>>                          [{address,{10,128,0,10}},
>>                           {port,2222},
>>                           {role,server},
>>                           {pwdfun,#Fun<sshd.pwcheck.2>},
>>                           {shell,#Fun<sshd.0.34972026>}],
>>                          114,49,
>>                          
>> {{49985839305237737688589908016292087537304670681285167039328910635484947391794443232164854289884214947929742793900078775832939310627327467626874637085242037758449972175725827907772272533385534014381436171929411436926043326068534386929776710296045274502307637113679098707841881585433086106245829282665980090815, 
>>
>>                            
>> 8395950144135694221446766804944646771011612746974327140375413192360614631043395924085970461957169944119762874929925969231067866506208529309578676567265288403413833138477256106374779295193693244061192404399083583730265790053008130870886892278024924477046645863115470835406736499812828883753405586277633495326}, 
>>
>>                           {2,
>>                            
>> 179769313486231590770839156793787453197860296048756011706444423684197180216158519368947833795864925541502180565485980503646440548199239100050792877003355816639229553136239076508735759914822574862575007425302077447712589550957937778424442426617334727629299387668709205606050270810842907692932019128194467627007}}, 
>>
>>                          undefined,undefined,"ssh-connection",undefined,
>>                          
>> "publickey,keyboard_interactive,password",undefined,
>>                          undefined,
>>                          ["ssh-rsa","ssh-dss"]},
>>                      #Port<0.1845>,<<>>,
>> <<107,0,185,229,147,59,116,109,92,16,81,210,0,243,152,220,
>>                        
>> 153,112,133,2,156,113,5,213,202,236,150,200,184,190,61,
>>                        
>> 227,37,31,157,1,53,98,136,245,199,18,165,166,214,36,210,
>>                        12,92,130,17,174>>,
>>                      undefined,
>>                      {ssh_msg_kexinit,
>> <<126,96,184,235,216,208,69,175,215,35,94,180,214,20,
>>                            246,141>>,
>>                          ["diffie-hellman-group1-sha1"],
>>                          ["ssh-rsa","ssh-dss"],
>>                          ["aes128-cbc","3des-cbc"],
>>                          ["aes128-cbc","3des-cbc"],
>>                          ["hmac-sha1"],
>>                          ["hmac-sha1"],
>>                          ["none","zlib"],
>>                          ["none","zlib"],
>>                          [],[],false,0},
>>                      false,<0.156.0>,undefined,undefined,undefined,
>>                      [{address,{10,128,0,10}},
>>                       {port,2222},
>>                       {role,server},
>>                       {pwdfun,#Fun<sshd.pwcheck.2>},
>>                       {shell,#Fun<sshd.0.34972026>}]}
>> ** Reason for termination =
>> ** {{{badmatch,
>>          {disconnect,
>>              {11,"Internal error"},
>>              {{replies,[{channel_data,<0.159.0>,{closed,0}}]},
>>               {connection,[],315439,[],1,
>>                   {ssh_cli,[#Fun<sshd.0.34972026>]},
>>                   {10,128,0,10},
>>                   2222,
>>                   [{role,server},
>>                    {pwdfun,#Fun<sshd.pwcheck.2>},
>>                    {shell,#Fun<sshd.0.34972026>}],
>>                   undefined,<0.153.0>,undefined}}}},
>>      [{ssh_connection_manager,handle_call,3,
>>           [{file,"../src/ssh_connection_manager.erl"},{line,271}]},
>>       {gen_server,handle_msg,5,[{file,"gen_server.erl"},{line,588}]},
>>       {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,227}]}]},
>>     {gen_server,call,
>>         
>> [<0.156.0>,{ssh_msg,<0.157.0>,<<93,0,0,0,0,0,0,32,106>>},infinity]}}
>>
>>
>>
>>> Hi!
>>>
>>> Daniel Goertzen wrote:
>>>> The channel number on the putty side is 0x100, and on the Erlang 
>>>> side it is 0x0.  Erlang is responding to SSH_MSG_CHANNEL_REQUEST 
>>>> using 0x0, however RFC4254 says the responses should be...
>>>>
>>>>
>>>>       byte      SSH_MSG_CHANNEL_SUCCESS
>>>>       uint32    recipient channel
>>>>
>>>>
>>>>       byte      SSH_MSG_CHANNEL_FAILURE
>>>>       uint32    recipient channel
>>>>
>>>>
>>>> ...and I interpret "recipient channel" as being the Putty side, 
>>>> which is 0x100.  Putty seems to cope when Erlang returns SUCCESS, 
>>>> but blows up when Erlang returns a FAILURE.
>>>>
>>>
>>> Yes that is correct I remember correcting such a bug a long time 
>>> ago, checking the code however there seems to be one error handling 
>>> case that I missed.  Could you check if
>>> the following patch resolves your issue Fyodor?
>>>
>>> diff --git a/lib/ssh/src/ssh_connection.erl 
>>> b/lib/ssh/src/ssh_connection.erl
>>> index 46f0c7e..8574e08 100644
>>> --- a/lib/ssh/src/ssh_connection.erl
>>> +++ b/lib/ssh/src/ssh_connection.erl
>>> @@ -720,10 +720,12 @@ 
>>> handle_msg(#ssh_msg_channel_request{request_type = "env"},
>>>
>>> handle_msg(#ssh_msg_channel_request{recipient_channel = ChannelId,
>>>                                    request_type = _Other,
>>> -                                   want_reply = WantReply}, 
>>> Connection,
>>> +                                   want_reply = WantReply},  
>>> #connection{channel_cache = Cache} = Connection,
>>>           ConnectionPid, _) ->
>>>     if WantReply == true ->
>>> -           FailMsg = channel_failure_msg(ChannelId),
>>> +           #channel{remote_id = RemoteId} = ChannelId =
>>> +               ssh_channel:cache_lookup(Cache, ChannelId),
>>> +           FailMsg = channel_failure_msg(RemoteId),
>>>            {{replies, [{connection_reply, ConnectionPid, FailMsg}]},
>>>             Connection};
>>>        true ->
>>>
>>>
>>>> I could imagine this passing automated tests when both ends number 
>>>> their channels starting from 0.  I bet randomizing the channel 
>>>> numbers would reveal this and maybe other issues.
>>>>
>>>> Dan.
>>>>
>>> I think you have a valid point however we have to change the 
>>> implementation in one way or an other to do that and that has not 
>>> been prioritized.
>>>
>>>
>>> Regards Ingela Erlang/OTP team -Ericsson AB
>>>
>>>>
>>>> On Tue, Apr 24, 2012 at 11:22 AM, Fyodor Ustinov <ufm@REDACTED 
>>>> <mailto:ufm@REDACTED>> wrote:
>>>>
>>>>     Hi!
>>>>
>>>>     I added this patch - did not help.
>>>>
>>>>     http://blog.ufm.su/putty.log - this putty log.
>>>>
>>>>
>>>>         Hi!
>>>>
>>>>         This might be a symptom of the "close race-condition"-bug
>>>>          reported by Daniel. The patch for that bug is
>>>>
>>>>         diff --git a/lib/ssh/src/ssh_channel.erl
>>>>         b/lib/ssh/src/ssh_channel.erl
>>>>         index 7b600ed..1938858 100644
>>>>         --- a/lib/ssh/src/ssh_channel.erl
>>>>         +++ b/lib/ssh/src/ssh_channel.erl
>>>>         @@ -1,7 +1,7 @@
>>>>         %%
>>>>         %% %CopyrightBegin%
>>>>         %%
>>>>         -%% Copyright Ericsson AB 2008-2011. All Rights Reserved.
>>>>         +%% Copyright Ericsson AB 2008-2012. All Rights Reserved.
>>>>         %%
>>>>         %% The contents of this file are subject to the Erlang Public
>>>>         License,
>>>>         %% Version 1.1, (the "License"); you may not use this file
>>>>         except in
>>>>         @@ -215,7 +215,7 @@ handle_info({ssh_cm, ConnectionManager,
>>>>         {closed, ChannelId}},
>>>>                          close_sent = false} = State) ->
>>>>            %% To be on the safe side, i.e. the manager has already
>>>>         been terminated.
>>>>            (catch ssh_connection:close(ConnectionManager, ChannelId)),
>>>>         -    {stop, normal, State};
>>>>         +    {stop, normal, State#state{close_sent = true}};
>>>>
>>>>         handle_info({ssh_cm, _, _} = Msg, #state{cm = 
>>>> ConnectionManager,
>>>>                               channel_cb = Module,
>>>>         diff --git a/lib/ssh/src/ssh_connection_manager.erl
>>>>         b/lib/ssh/src/ssh_connection_manager.erl
>>>>         index e993f59..8c1f88e 100644
>>>>         --- a/lib/ssh/src/ssh_connection_manager.erl
>>>>         +++ b/lib/ssh/src/ssh_connection_manager.erl
>>>>         @@ -384,9 +384,10 @@ handle_call({close, ChannelId}, _,
>>>>                   #state{connection = Pid, connection_state =
>>>>                          #connection{channel_cache = Cache}} = 
>>>> State) ->
>>>>            case ssh_channel:cache_lookup(Cache, ChannelId) of
>>>>         -       #channel{remote_id = Id} ->
>>>>         +       #channel{remote_id = Id} = Channel ->
>>>>                   send_msg({connection_reply, Pid,
>>>>                             ssh_connection:channel_close_msg(Id)}),
>>>>         +           ssh_channel:cache_update(Cache,
>>>>         Channel#channel{sent_close = true}),
>>>>                   {reply, ok, State};
>>>>               undefined ->
>>>>                   {reply, ok, State}
>>>>
>>>>
>>>>         Regards Ingela Erlang/OTP  team - Ericsson AB
>>>>
>>>>         Daniel Goertzen wrote:
>>>>
>>>>             Under Putty Session->Logging can you turn on "SSH Packets"
>>>>             and post the log showing the error?
>>>>
>>>>             Dan.
>>>>
>>>>             On Sun, Apr 22, 2012 at 3:58 AM, Fyodor Ustinov
>>>> <ufm@REDACTED <mailto:ufm@REDACTED> <mailto:ufm@REDACTED
>>>> <mailto:ufm@REDACTED>>> wrote:
>>>>
>>>>                Hi!
>>>>
>>>>                Windows ssh client PuTTY show message  "Disconnected:
>>>>             Received
>>>>                SSH2_MSG_CHANNEL_FAILURE for nonexistent channel 0"
>>>>             while working
>>>>                with erlang ssh daemon.
>>>>
>>>>                Erlang R15B01 (erts-5.9.1) [source] [64-bit] [smp:8:8]
>>>>                [async-threads:0] [kernel-poll:false]
>>>>
>>>>                WBR,
>>>>                   Fyodor.
>>>>
>>>>                _______________________________________________
>>>>                erlang-bugs mailing list
>>>>                erlang-bugs@REDACTED <mailto:erlang-bugs@REDACTED>
>>>> <mailto:erlang-bugs@REDACTED
>>>> <mailto:erlang-bugs@REDACTED>>
>>>>                http://erlang.org/mailman/listinfo/erlang-bugs
>>>>
>>>>
>>>>
>>>>
>>>>             --             *Daniel Goertzen | Senior Software 
>>>> Engineer*
>>>>             Office: 828.610.4596 <tel:828.610.4596> | Fax:
>>>>             828.322.5294 <tel:828.322.5294> |
>>>>             dang@REDACTED
>>>> <mailto:dang@REDACTED>
>>>> <mailto:dang@REDACTED
>>>> <mailto:dang@REDACTED>>
>>>>             *Network Integrity Systems | We Bring Security To Light™*
>>>>
>>>>             1937 Tate Blvd. SE
>>>>
>>>>             Hickory, North Carolina, USA 28602
>>>>
>>>>             /Network Integrity Systems’ INTERCEPTOR™ Optical Network
>>>>             Security System is a Smart-PDS™ that ensures superior
>>>>             protection and cost effectiveness of classified networks.
>>>>              For more information, visit our website
>>>>             at:www.networkintegritysystems.com
>>>> <http://www.networkintegritysystems.com>
>>>> <http://www.networkintegritysystems.com/>./
>>>>
>>>>
>>>>             __________________________________________
>>>>             INTERCEPTOR™ Optical Network Security System is made in
>>>>             the USA for the USA. Although not an export controlled
>>>>             item, because of the role it plays in the assurance of the
>>>>             safety and integrity of National Security Information,
>>>>             Network Integrity Systems (NIS) is committed to compliance
>>>>             with the U.S. Export Administration Act. Accordingly, NIS
>>>>             will not ship INTERCEPTOR products to certain foreign
>>>>             government end users without U.S. government approval and
>>>>             will refuse transactions with individuals or entities that
>>>>             have been denied export privileges.
>>>>
>>>>
>>>>             
>>>> ------------------------------------------------------------------------ 
>>>>
>>>>
>>>>             _______________________________________________
>>>>             erlang-bugs mailing list
>>>>             erlang-bugs@REDACTED <mailto:erlang-bugs@REDACTED>
>>>>             http://erlang.org/mailman/listinfo/erlang-bugs
>>>>
>>>>
>>>>
>>>>     _______________________________________________
>>>>     erlang-bugs mailing list
>>>>     erlang-bugs@REDACTED <mailto:erlang-bugs@REDACTED>
>>>>     http://erlang.org/mailman/listinfo/erlang-bugs
>>>>
>>>>
>>>>
>>>>
>>>> -- 
>>>> *Daniel Goertzen | Senior Software Engineer*
>>>> Office: 828.610.4596 | Fax: 828.322.5294 | 
>>>> dang@REDACTED 
>>>> <mailto:dang@REDACTED>
>>>> *Network Integrity Systems | We Bring Security To Light™*
>>>>
>>>> 1937 Tate Blvd. SE
>>>>
>>>> Hickory, North Carolina, USA 28602
>>>>
>>>> /Network Integrity Systems’ INTERCEPTOR™ Optical Network Security 
>>>> System is a Smart-PDS™ that ensures superior protection and cost 
>>>> effectiveness of classified networks.  For more information, visit 
>>>> our website at:www.networkintegritysystems.com 
>>>> <http://www.networkintegritysystems.com/>./
>>>>
>>>>
>>>> __________________________________________
>>>> INTERCEPTOR™ Optical Network Security System is made in the USA for 
>>>> the USA. Although not an export controlled item, because of the 
>>>> role it plays in the assurance of the safety and integrity of 
>>>> National Security Information, Network Integrity Systems (NIS) is 
>>>> committed to compliance with the U.S. Export Administration Act. 
>>>> Accordingly, NIS will not ship INTERCEPTOR products to certain 
>>>> foreign government end users without U.S. government approval and 
>>>> will refuse transactions with individuals or entities that have 
>>>> been denied export privileges.
>>>>
>>>>
>>>
>>
>
> _______________________________________________
> erlang-bugs mailing list
> erlang-bugs@REDACTED
> http://erlang.org/mailman/listinfo/erlang-bugs
>




More information about the erlang-bugs mailing list