Logger improvements

[gavv]

This PR contains two commits:

• minor refactoring in log.go
• decoupling of C and Go logging threads

More details about the second commit:

I noticed that when user calls SetLogger/SetLoggerFunc to provide custom log handler, or configures standard logger to use custom output writer, it may happen that our C threads invoke arbitrary complex user's go code.

I feel that invoking random user's go code on random C threads may be dangerous, at best it can confuse go scheduler and probably cause performance issues, though I don't have specific examples on top of my head.

Anyway, I just decoupled this threads: C callback now writes messages to a channel, and user's go code is invoked in a separate goroutine which reads messages from that channel. Now we can be sure that user's callback stay in "go world" and callbacks invoked on C thread is small and minimal and is completely controlled by us.

[coveralls]

Coverage: 88.592% (+0.6%) from 87.971% when pulling 63bb26c on gavv:main into ec43d2b on roc-streaming:main.

[ortex]

LGTM

slightly related stuff:
Need to fix comments for LogInfo and LogTrace levels.

	// LogError enables informational messages and above.
	LogInfo LogLevel = 2
...
	// LogDebug enables extra verbose logging, which may hurt performance
	// and should not be used in production.
	LogTrace LogLevel = 4

[gavv]

Need to fix comments for LogInfo and LogTrace levels.

Fixed.

[gavv]

I feel that invoking random user's go code on random C threads may be dangerous, at best it can confuse go scheduler and probably cause performance issues, though I don't have specific examples on top of my head.

So I dug into it a bit more.

It works as follows:

• C thread invokes Go entry point

• Go attaches temporary M to the C thread, switches to M's stack, and invokes LockOSThread (pin goroutine to system thread)

• Go invokes our callback

• our callback is executed on the C thread and Go stack; it's allowed to block; it's not allowed to switch goroutines; when it blocks, instead of scheduling another goroutine, it's OS thread is suspended on a futex until woke up from other Go threads;

Conclusion:

• my feeling was right, it's not OK to invoke arbitrary user code in logger callback, because executing code in a goroutine after LockOSThread may have performance penalties and user may not expect that their code will run in such environment

• luckily, it's OK to block in Go callback on C thread, so a blocking write to a channel that I'm doing in this PR is acceptable; the C thread will just sleep, as intended

Reference:

• https://go.dev/src/runtime/HACKING
• https://github.com/golang/go/blob/0d82f584c223fb045496f3e063bf93416a24e0b0/src/runtime/cgocall.go#L51-L68
• https://github.com/golang/go/blob/0d82f584c223fb045496f3e063bf93416a24e0b0/src/runtime/asm_amd64.s#L933-L938
• https://github.com/golang/go/blob/0d82f584c223fb045496f3e063bf93416a24e0b0/src/runtime/proc.go#L1857-L1895
• https://github.com/golang/go/blob/7c47a6b15782b13ecb76fd3c6c18e5f1edc34733/src/runtime/proc.go#L3339
• https://github.com/golang/go/blob/7c47a6b15782b13ecb76fd3c6c18e5f1edc34733/src/runtime/proc.go#L2561
• https://github.com/golang/go/blob/7c47a6b15782b13ecb76fd3c6c18e5f1edc34733/src/runtime/lock_futex.go#L148

Test example: https://gist.github.com/gavv/71c60e9aedd2aeb9ce036ee24d94d0c7