Description Paul Pazderski 2020-01-15 16:07:58 EST

This arise from bug 421303 comment 8 and others.

A quick summary how the (relevant) parts of Eclipse Console work.
Content is appended to console by writing to a IOConsoleOutputStream.
Writing to console is allowed from any thread.
The content is not directly inserted in Console but queued for processing.
Processing is done by UI thread. One reason is that it need to update UI elements.
It includes some kind of thread control. Processing is delayed a bit to aggregate multiple small writes. If more is added the delay is skipped. If the next threshold is reached pending content is either directly processed (if currently in UI thread) or thread blocked to give UI time for processing.
This can lead to a deadlock and was recently changed to wait only a second.

The current state is still open to unfortunate blocking situations in some setups. Here the shortened scenario from bug 421303.
UI thread is producing a lot of log events. The log events are passed into a fixed size buffer. Another thread is reading new events from this buffer and write the log messages to console.
Now if the buffer is full UI thread need to wait. The consumer is writing log events to console but once he reached the 'to much pending' limit it will block to give UI time to process. But since UI thread is not done writing log events it is also blocked. This is a deadlock in 4.14, a one second delay in 4.15. But a delay for every write once the limit is reached until UI start processing.


My main goal is to bring a dense summary of this potential problem into a separate bug. I have no estimations how or if this problem will be solved.

Here are possible options for this bug I could imagine:

1. Ignore. We warn to write large amounts to console in UI thread and even if it is not the UI thread writing in this case we state that the UI thread is necessary for processing. And with bug 421303 it is at least not deadlocking anymore.
(no offence Sergey, I just list any option which came to my mind)

2. Increase the limits and hope no one is hitting them anymore in such a bad situation. Overall those limits for 'schedule without delay' which is  2000 byte pending and block thread from further appending which is a bit over 300kb are old. They were introduced in 2005/2006 and average available memory has changed since then.

3. Add yet another level of thread control. As mentioned above we have currently: delay 50ms to aggregate small writes, schedule without delay if 1000 characters are pending and block for 1 second if 160k characters are pending.
The block is added to prevent to much memory consumption and give UI time to process. A more complex logic would be to wait shorter, e.g. 100ms, and if still pending after multiple waits do not block anymore until a much higher limit is reached. This would solve the problem leading to this bug but might open the possibility of new problems. (and might need to be adjusted if UI is starving from many other threads writing stuff)

4. The luxurious and maybe perfect solution. Remove the blocking of other threads and limitation on UI thread. A main reason to block threads from writing is memory consumption. The console content can be limited, the pending content not. If we do processing in a separate thread owned by us and blocked from nothing and including the console trimming this problem should vanish.
But at the same time it is the, by far!, most complex solution. I already did a quick test in this direction in the past because I disliked that console processing is doing so much in UI thread but without positive result. IMO the main problems are the dependency on the underlying console document which is UI bound and (apart from synchronizing in general) synchronizing with user input.
Bug 551745 was a good start in this direction but even this change did not go as smoothly as I had hoped.

I'm open for alternative proposals. I may have worked to much on console stuff and became blinkered for an obvious solution.