Better handle asynchronous Eldoc sources

This is a backward compatible redesign of significant parts of the
eldoc.el library.

Previously, Eldoc clients (major/minor modes setting its documentation
gathering variables) needed to directly call eldoc-message, an
internal function, to display the docstring to the user.  When more
asynchronous sources are involved, this is hard to do or even breaks
down.

Now, an Eldoc backend may return any non-nil, non-string value and
call a callback afterwards.  This restores power to Eldoc over how
(and crucially also when) to display the docstrings to the user.

Among other things, this fixes so called "doc blinking", or the very
short-lived display of a lower priority Eldoc message.  This would
happen if a particular producer of documentation finishes shortly
before a higher priority one, like in the LSP engine Eglot as reported
by Andrii Kolomoiets <andreyk.mad@gmail.com> and Dmitry Gutov
<dgutov@yandex.ru>.

Gathering docstrings is now delegated to the variable
eldoc-documentation-strategy, which is the new name for the
now-obsolete eldoc-documentation-function, and still accepts the
so-called "old protocol".  Examples of the new strategies enabled are
codified in functions such as eldoc-documentation-enthusiast,
eldoc-documentation-compose-eagerly, along with the existing
eldoc-documentation-compose and eldoc-documentation-default.

The work of displaying and formatting docstrings is shifted almost
fully to Eldoc itself and is delegated to the internal function
eldoc--handle-docs.  Among other improvements, it handles most of
eldoc-echo-area-use-multiline-p and outputs documentation to a
temporary *eldoc* buffer.

The manual and NEWS are updated to mention the new Eldoc features.

* lisp/emacs-lisp/eldoc.el (eldoc-documentation-functions):
Overhaul docstring.
(eldoc-documentation-compose, eldoc-documentation-default): Handle
non-nil, non-string values of elements of
eldoc-documentation-functions.  Use eldoc--handle-multiline.
(eldoc-print-current-symbol-info): Honour non-nil, non-string
values returned by eldoc-documentation-callback.
(eldoc--make-callback): Now also a function.
(eldoc-documentation-default, eldoc-documentation-compose): Tweak docstring.
(eldoc-documentation-enthusiast, eldoc-documentation-compose-eagerly):
New functions.
(eldoc-echo-area-use-multiline-p): Add new semantics.
(eldoc--handle-docs): Handle some of eldoc-echo-area-use-multiline-p.
(eldoc-doc-buffer): New command.
(eldoc-prefer-doc-buffer): New defcustom.
(eldoc--enthusiasm-curbing-timer): New variable.
(eldoc-documentation-strategy): Rename from eldoc-documentation-function.
(eldoc--supported-p): Use eldoc-documentation-strategy
(eldoc-highlight-function-argument)
(eldoc-argument-case, global-eldoc-mode)
(turn-on-eldoc-mode): Mention eldoc-documentation-strategy.
(eldoc-message-function): Mention eldoc--message.
(eldoc-message): Made obsolete.
(eldoc--message): New helper.

* lisp/hexl.el (hexl-print-current-point-info): Adjust to new
eldoc-documentation-functions protocol.

* lisp/progmodes/cfengine.el (cfengine3-documentation-function):
Adjust to new eldoc-documentation-functions protocol.

* lisp/progmodes/elisp-mode.el
(elisp-eldoc-documentation-function): Adjust to new
eldoc-documentation-functions protocol.

* lisp/progmodes/octave.el (octave-eldoc-function): Adjust to new
eldoc-documentation-functions protocol.

* lisp/progmodes/python.el (python-eldoc-function): Adjust to new
eldoc-documentation-functions protocol.

(eldoc-print-current-symbol-info): Rework with cl-labels.

* doc/emacs/programs.texi (Lisp Doc): Mention
eldoc-documentation-strategy.

* doc/lispref/modes.texi (Major Mode Conventions): Mention
eldoc-documentation-functions.

* etc/NEWS: Mention eldoc-documentation-strategy.

1 files changed, 6 insertions, 4 deletions

diff --git a/doc/emacs/programs.texi b/doc/emacs/programs.texi
index 865a3a67d56..2757c84b53a 100644
--- a/doc/emacs/programs.texi
+++ b/doc/emacs/programs.texi
@@ -1273,17 +1273,19 @@ Eldoc mode, which is turned on by default, and affects buffers whose
 major mode sets the variables described below.  Use @w{@kbd{M-x
 global-eldoc-mode}} to turn it off globally.
 
-@vindex eldoc-documentation-function
+@vindex eldoc-documentation-strategy
 @vindex eldoc-documentation-functions
   These variables can be used to configure ElDoc mode:
 
 @table @code
-@item eldoc-documentation-function
+@item eldoc-documentation-strategy
 This variable holds the function which is used to retrieve
 documentation for the item at point from the functions in the hook
 @code{eldoc-documentation-functions}.  By default,
-@code{eldoc-documentation-function} returns the first documentation
-string produced by the @code{eldoc-documentation-functions} hook.
+@code{eldoc-documentation-strategy} returns the first documentation
+string produced by the @code{eldoc-documentation-functions} hook, but
+it may be customized to compose those functions' results in other
+ways.
 
 @item eldoc-documentation-functions
 This abnormal hook holds documentation functions.  It acts as a