windows: increase WIN32textout internal buffer#562
Merged
Conversation
By default (*), the main console output (from win_flush using WIN32textout) is converted from UTF-8 to wchar_t. The conversion uses a fixed-size buffer which was 1024, and now is 8K. The size increase is required because win_flush can output the whole buffer in one call, especially with -Da where the internal SGR processor is bypassed - obuf is sent unmodified to WIN32textout. obuf was changed recently from 1024 to 8K in commit 5854746 (windows: SGR processor: increase buffer size for less issues), and because the WIN32textout buffer did not increase till now, it could result in failed conversion due to insufficient result buffer, and no output. Before 5854746, OUTBUF_SIZ was 1024 and used only for obuf, while the WIN32textout buffer was independently and luckily also 1024. Now both obuf and WIN32textout use the same size OUTBUF_SIZE (which is not used elsewhere), and its 8K value is set in defines.wn . (*) If the internal "less" charset is UTF8 (default on Windows), and the console output CP is not UTF8 (also default), then conversion from UTF8 to wchar_t happens before using WriteConsoleW.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
By default (*), the main console output (from win_flush using WIN32textout) is converted from UTF-8 to wchar_t.
The conversion uses a fixed-size buffer which was 1024, and now is 8K.
The size increase is required because win_flush can output the whole buffer in one call, especially with -Da where the internal SGR processor is bypassed - obuf is sent unmodified to WIN32textout.
obuf was changed recently from 1024 to 8K in commit 5854746 (windows: SGR processor: increase buffer size for less issues), and because the WIN32textout buffer did not increase till now, it could result in failed conversion due to insufficient result buffer, and no output.
Before 5854746, OUTBUF_SIZ was 1024 and used only for obuf, while the WIN32textout buffer was independently and luckily also 1024.
Now both obuf and WIN32textout use the same size OUTBUF_SIZE (which is not used elsewhere), and its 8K value is set in defines.wn .
(*) If the internal "less" charset is UTF8 (default on Windows), and
the console output CP is not UTF8 (also default), then conversion
from UTF8 to wchar_t happens before using WriteConsoleW.