[erlang-questions] how search and search_quit are handled by user_drv?

[Fred Hebert]

On 05/21, Leo Liu wrote:
>Press key C-r and then TAB in the eshell one can observe the prompt
>changes to history `search' and back to normal again.
>
>I try to understand how it works in details but my understanding does
>not match what I observe above.
>
>The following is what I think should have happened where `|' indicates
>the cursor and indented lines are messages the user_drv process receives
>which cause the prompt to change from state 1 to 5.
>

Hi,

I implemented the search functionality so I'm in a good position to 
explain how it works.

The thing to understand about the Erlang shell is how it is being split 
up [1]:
- user.erl (or a tty program) ends up hadnling stdio
- user_sup.erl handles initialization of the whole system
- user_drv handles translation between the internal Erlang IO protocol 
  and the driver (tty) stuff. It also handles job management (^G) and 
  choosing which shell is active.
- group.erl has an instance for each individual job/shell, does line 
  buffering (until they're ready to be evaluated), handles up/down 
  arrows (to switch lines), and so on.
- edlin.erl handles each individual *line* for cursor movement, 
  character deletion, and so on. It also handles things like 
  interpreting keyboard shortcuts (^A, ^H, etc.)
- shell.erl handles evaluation of Erlang code

So when functionality like ^r search gets implemented, it needs to work 
both with group.erl (to cycle through lines, buffer the input, etc.) and 
edlin.erl (to interpret ^R and ^S). The stuff in user_drv takes place at 
an earlier time, and shell.erl at a latter time; both aren't connected 
to search.

The other thing to understand is that while vi and vim are the most 
known modal editors in the wild, the Erlang shell uses similar modes to 
understand how to handle its stuff internally, even through emacs mode; 
as such, <Esc>+B (backwards word) is represented in two characters: $\e 
(escape), which switches the shell in 'meta' mode, and 'B', which is 
interpreted as a special sequence under that mode rather than regular 
ASCII input.

So the implementation works a bit as follows:

1. When ^R (backward search) or ^S (forward search) is seen in group.erl 
and we're in the normal mode, we store the current line prompt ('1>') 
and create a new one, then set edlin into the 'search' mode[2]

2. The search mode is enabled in edlin, which will handle interpreting 
what to do in there. Edlin has a character mapping table[3] that lets 
search interpret which mode the shell should be in. This means special 
sequences to mean quitting search mode, ways to specify whether to look 
back up or down again (^S or ^R has been pressed a second time), or 
whether the character is just valid. This portion is really about 
selecting the right *mode*.

3. The mapping is used to switch on operations [4] and identify proper 
transformations to the visual representation of the buffer than needs to 
be done [5]. This is because there's a difference between what we have 
as an internal state, and how we represent it. This bit is about 
*representation*.

4. Whenever the shell is in search mode, group.erl sends its data 
through edlin, handles the result, and looks at characters that came in, 
the current line buffer, and performs a search on each line, either 
backwards or forwards according to the search direction [6]. It then 
takes whatever line it has found (if any) and sends it to the driver to 
output (bypassing or adding to what edlin had done before) to get the 
right *representation* for the user.

This bit is tricky because while we display old user text in the line, 
this text never makes it to edlin.erl -- what we do is tell the driver 
"show this", while edlin holds the current search buffer in memory. The 
trick here is that group.erl stores the current line stack position 
(where in the list of line is the cursor?) so that user search doesn't 
conflict with what we show on screen.

5. Whenever the user is satisfied with a result, they press <Enter>, 
which edlin interprets as the 'search_found' state; group.erl interprets 
this to fetch the current line buffer (which should be in a position 
where a satisfactory entry was found!), copy it in the current topmost 
position in the line buffer, and revert the prompt to normal with the 
input ready to go [7].

6. If on the other hand the search is cancelled, the shell falls back 
into edit mode, but at the current stack position [8] to replicate 
existing bash/zsh behaviour. This lets you edit a result you had found 
previously, in place.

7. Any other special meta modes (like search_meta_left_sq_bracket, which 
handles fancy escape sequences while in shell mode) are tunneled through 
to edlin because they have nothing to do with multiline business 
group.erl handles, and everything to do with mode switching.

This is all rather complex, but respects (as well as possible) the 
previously established separation of concerns of the Erlang shell 
between line editing and buffer management.

Regards,
Fred.

[1]: http://ferd.ca/repl-a-bit-more-and-less-than-that.html
[2]: 
https://github.com/erlang/otp/blob/maint/lib/stdlib/src/edlin.erl#L234
[3]: 
https://github.com/erlang/otp/blob/maint/lib/stdlib/src/edlin.erl#L234
[4]: 
https://github.com/erlang/otp/blob/maint/lib/stdlib/src/edlin.erl#L131
[5]: 
https://github.com/erlang/otp/blob/maint/lib/stdlib/src/edlin.erl#L268-L318
[6]: 
https://github.com/erlang/otp/blob/maint/lib/kernel/src/group.erl#L581-L605
[7]: 
https://github.com/erlang/otp/blob/maint/lib/kernel/src/group.erl#L558-L564
[8]: 
https://github.com/erlang/otp/blob/maint/lib/kernel/src/group.erl#L565-L580