Regression: Initial filter kicks in late

[GfEW]

The Problem

There are three kinds of unexpected behavior with initial filters in v696 and up (could be v693 and up). This issue is named after the most substantial of the three, but it also describes the other two, because they appear to be closely related.

How to reproduce?

Setup:

for ((i=1;i<=50;i++)); do
  echo $((i));
  sleep 1 || break;
done |
LESSNOCONFIG='LESS_LINES' LESS_LINES=10 less -M --use-backslash $'+&[02468]\$\nF'

1. Watch lines 1 through 8 coming in.

• 1a. Notice the filter is not active.
• 1b. Notice that all lines are printed in order, from the top down to the bottom.
• 1c. Notice the F mode prompt line is missing.

2. Watch lines 9 through 17 coming in.

• 2a. Notice the filter kicks in at the moment line 9 is being read, i. e. when there are enough unfiltered lines to fill the content area.
• 2b. Notice there's a sudden jump in the movement of content lines on screen at that same moment.
• 2c. Notice the F mode prompt line is still missing.

3. Watch lines 18 and up coming in.

• 3a. (OK)
• 3b. Notice that content lines start to scroll upwards (as expected) at the moment line 20 is being read.
• 3c. Notice the F mode prompt line appears at the moment line 18 is being read, i. e. when there are enough filtered lines to fill the content area.

3. Type ^Xq to quit less.

What should happen instead?

Short answer:

Test and see less v692.

Long answer:

• 1a. The filter should be active from the start.
• 2b. There should be no "jumps" (larger than single line scrolling steps, that is) in the movement of content lines on screen, as they keep coming in.
• 3c. The Waiting for data ... prompt line should be visible from the start.
  (I'm not sure about this part, but v692 does display the F mode prompt immediately while the content area is being filled, which feels more consistent and informative than a blank prompt.)

Additional Context

• Whilst the combination of LESS_LINES and F mode produces the symptoms described above, variants of the latter are also present with only one or none of those options.
• Apparently, the problem got introduced between v692 and v696, as v692 behaves correctly in all three regards (1a, 2b, 3c).

[gwsw]

Should be fixed in 7f5bb27.

This was broken in 8f10702 which was an attempted fix for #752 which you reported. The new fix should be better.

[GfEW]

I can confirm 7f5bb27 behaves correctly with what originally were reproducers in both #752 and #759 (in all three regards).

It's amazing to laymen like me just how small a change of code can fix broken behavior if addressed properly. Feels like magic, not for the first time.

Given the commit comment, this is one of the rare occasions where I've actually searched for remaining issues not encountered already. There's one edge case I'd like to bring to your attention, as it might hint at incorrect ordering somewhere during initialisation. However, we're getting into nerdy territory here. Please don't hesitate to ignore/dismiss if considered too niche.

for ((i=1;i<=50;i++)); do
  echo $((i));
  sleep .5;     # sped up because nothing special here
  ((i>=18)) && sleep 1.5;
done |
LESSNOCONFIG='LESS_LINES' LESS_LINES=10 less -JNM --status-line --use-backslash $'+&[02468]\$\n20mmF'

Observed

1. Watch lines 1 through 18 coming in.

• 1c. Notice there's no F mode prompt (nor any other).
• 1d. Notice how display height builds up correctly with every line.

2. Watch line 19 through 21 coming in.

• 2c. Notice there's still no F mode prompt (nor any other).
• 2d. Notice that suddenly with line 19 being read, the content displayed so far gets duplicated below, extending the overall height of less to 19 lines (i. e. 9 + 9 + 1, including the cursor / empty prompt).
• 2e. Notice that, when line 20 is being read, the mark is set as expected, and overall height extends to 20 lines (i. e. 9 + 10 + 1, including the cursor / empty prompt).

3. Watch lines 22 and up coming in.

• 3c. Notice the F mode prompt pops up when line 22 is being read.
• 3d. Notice the extended display collapses to proper height at that same moment.
• 3e. Notice the mark is maintained, and moved up as further lines arrive, correctly.

4. Type ^Xq to quit less.

Expected

Still, in this particular configuration:

• 3c. The Waiting for data ... prompt line should be visible from the start.
  (if that's to be expected with -M while F mode is active)

as well as previously undiscovered:

• 3d. Display height should not exceed the value of LESS_LINES.

[gwsw]

It's amazing to laymen like me just how small a change of code can fix broken behavior if addressed properly.

Many years ago I jocularly proposed a principle that the harder a bug is to find, the smaller the change needed to fix it. I have on occasion spent many days tracking down a bug, after which I was able to fix it by changing one character in the code.

Anyway most of the issues in your latest example are due to an incomplete implementation of LESS_LINES, and should be fixed in d377035. You will see that there is still no prompt line displayed during the initial screen fill. That is because it is being done by the m command and not the F command. The F command prints the "Waiting for data" message immediately, but in other cases, that message is deferred until no data has been received for 4 seconds. There was a discussion of this in #337, in which most users felt that the message was intrusive and shouldn't be displayed immediately. You can adjust the delay by setting LESS_DATA_DELAY to a small number of milliseconds. For example, adding LESS_DATA_DELAY=10 will cause the prompt to display during the initial screen fill.