Bug 538656 - Make JDT CompletionProcessors not require UI Thread
Summary: Make JDT CompletionProcessors not require UI Thread
Status: RESOLVED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 4.5   Edit
Hardware: All All
: P3 enhancement (vote)
Target Milestone: 4.14 M3   Edit
Assignee: Mickael Istria CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on: 547683
Blocks: 531061
  Show dependency tree
 
Reported: 2018-09-05 11:40 EDT by Mickael Istria CLA
Modified: 2019-11-18 10:19 EST (History)
15 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Mickael Istria CLA 2018-09-05 11:40:30 EDT
Most JDT content assist processors part of org.eclipse.jdt.ui actually don't require the UI Thread for their logic. However, they do all rely on the same super class that uses context.getViewer().getSelectedRange() which triggers a UI call (need to run in UI Thread). Some content-assist processors directly reference this chain.

We should try to avoid requiring UI Thread for such requests, by storing the selectedRange -and its modifications- in the context and providing an API to get it directly from there instead of going through the viewer.
Also, we should put a comment on context.getViewer() to remind extenders that by doing this, they have big risk of requiring UI Thread, leading to impossible parallel computation and guaranteed UI latencies.

Then, once we fixed the JDT classes, we should declare them as non-UI Thread compatible (bug 538630)
Comment 1 Mickael Istria CLA 2018-09-07 12:41:10 EDT
One major issue I'm facing is that there seems to be no good way to just get the selection from a ITextViewer without requiring the UI Thread. Indeed, getViewer().getSelectedRange() call a checkWidget() and fails if not in UI Thread, getViewer().getSelectionProvider().getSelection() also invoke control methods calling checkWidget()...
Does anyone know a trick to just get the selection without requiring UI Thread? It's really a must-have if we want to make the UI more reactive.
Comment 2 Eclipse Genie CLA 2018-09-10 03:57:38 EDT
New Gerrit change created: https://git.eclipse.org/r/129010
Comment 3 Mickael Istria CLA 2018-09-10 13:04:01 EDT
One positive side-effect with this change is that by removing adherance to UI Thread, it may make the code easy to reuse or port to JDT-LS.
Comment 4 Noopur Gupta CLA 2018-10-29 07:53:15 EDT
(In reply to Mickael Istria from comment #1)
> One major issue I'm facing is that there seems to be no good way to just get
> the selection from a ITextViewer without requiring the UI Thread. Indeed,
> getViewer().getSelectedRange() call a checkWidget() and fails if not in UI
> Thread, getViewer().getSelectionProvider().getSelection() also invoke
> control methods calling checkWidget()...
> Does anyone know a trick to just get the selection without requiring UI
> Thread? It's really a must-have if we want to make the UI more reactive.
I see this issue still exists with the patch and we don't have a solution to get the selection without wrapping/running the code in a UI thread. Will it be OK to do so internally even though the new extension attribute and preference option say that UI thread is not required?

Also, the patch changes the signature of org.eclipse.jdt.internal.ui.text.template.contentassist.TemplateEngine.complete(...) method in terms of its parameters. Though it is an internal method, I can see that it is being used by JDT Debug at several locations. It might also be used by other clients like Code Recommenders.

Dani, do you have any suggestions on how to proceed here?
Comment 5 Andrey Loskutov CLA 2018-10-29 08:11:45 EDT
(In reply to Noopur Gupta from comment #4)
> (In reply to Mickael Istria from comment #1)
> > One major issue I'm facing is that there seems to be no good way to just get
> > the selection from a ITextViewer without requiring the UI Thread. Indeed,
> > getViewer().getSelectedRange() call a checkWidget() and fails if not in UI
> > Thread, getViewer().getSelectionProvider().getSelection() also invoke
> > control methods calling checkWidget()...
> > Does anyone know a trick to just get the selection without requiring UI
> > Thread? 

Not really, except one adds some new API to retrieve *last* selection from widget, or something like comment 0.

> It's really a must-have if we want to make the UI more reactive.
> I see this issue still exists with the patch and we don't have a solution to
> get the selection without wrapping/running the code in a UI thread. Will it
> be OK to do so internally even though the new extension attribute and
> preference option say that UI thread is not required?

I personally would not do this, my fear is deadlocks etc. If the javadoc says no UI any client can take any number of locks assuming it will not hurt anyone, but syncExec() call from inside such code *is* an access to potentially locked resource (UI thread).

You probably mean JavaContentAssistInvocationContext.getViewerSelection()? The code as is will not work anyway, getViewer().getTextWidget().getDisplay() will throw SWT error outside UI thread.
Comment 6 Angelo ZERR CLA 2018-10-29 08:15:48 EDT
It out of the scope of this bug, but please note that I had the same problem when I try to manage templates with GenericEditor async completion. See my gerrit patch at https://git.eclipse.org/r/#/c/119801/1/org.eclipse.jface.text/src/org/eclipse/jface/text/templates/TemplateCompletionProcessor.java
Comment 7 Noopur Gupta CLA 2018-10-29 08:45:40 EDT
(In reply to Andrey Loskutov from comment #5)
Thanks, Andrey for taking a look. Yes, I was referring to the same (also, in the context of patches from bug 538630 and bug 531061).

Mickael, please revisit the approach for this bug.
Comment 8 Julian Honnen CLA 2019-05-21 10:55:03 EDT
(In reply to Andrey Loskutov from comment #5)
> I personally would not do this, my fear is deadlocks etc. If the javadoc
> says no UI any client can take any number of locks assuming it will not hurt
> anyone, but syncExec() call from inside such code *is* an access to
> potentially locked resource (UI thread).

There doesn't seem to be any other way with the current design, since the IContentAssistProcessor never gets a chance to do some preprocessing on the UI thread first.

@Mickael: Shouldn't the jface API be separated in two stages?

IAsyncContentAssistProcessor {

  // called from UI
  Context createContext(ITextViewer viewer, int offset)

  // may be called async
  ICompletionProposal[] computeCompletionProposals(Context context)
}


createContext() could then memorize the viewer selection. That also has the benefit that the selection would stay the same during the computation.
Comment 9 Mickael Istria CLA 2019-05-21 16:57:28 EDT
(In reply to Julian Honnen from comment #8)
> @Mickael: Shouldn't the jface API be separated in two stages?

This might work, but initial idea was to avoid creating new type and stick with IContentAssistProcessor to ease adoption. Introducing new classes involves adding extra maintenance cost, so unless it's really worth it, we should avoid it and try to deal with existing APIs. What you're suggesting is a new API + some heavy and error-prone refactoring, I'd like to make sure we consider other alternatives that remain internal to JDT.
Comment 10 Julian Honnen CLA 2019-05-22 03:54:22 EDT
(In reply to Mickael Istria from comment #9)
> I'd like to make sure
> we consider other alternatives that remain internal to JDT.

I would have thought that other projects have similar problems, but I couldn't find any getSelection() (or similar) calls in PDE and CDT content assists.


What I did find are more reasons for a new jface API:

1) progress monitor/cancellation: both JDT and CDT create a NullProgressMonitor in computeCompletionProposals() and then seem to report sensible progress on it.
The ability to cancel previous computations seems like a nice benefit.

2) IContentAssistProcessors can't really be executed concurrently. JDT/CDT maintain a state (error message, # of results) which is cleared first in compute proposals. That statefulness is required by the current jface API (IContentAssistProcessor::getErrorMessage)
Comment 11 Mickael Istria CLA 2019-05-22 05:22:30 EDT
Some other projects (IIRC LSP4E, PDT, DLTK..) successfully use the current API wothout trouble.
Why does JDT need UI Thread and getSelection()? Couldn't this be.replaced by other approaches like listeners or filters?
Comment 12 Julian Honnen CLA 2019-05-22 07:10:28 EDT
The TemplateEngine uses the selected text for selection variables (and skips templates containing selection variables when nothing is selected).

Otherwise the selected range is used to override the replacement length in JavaCompletionProposal. Not sure exactly what that does. I tried hardcoding some different values without seeing an immediate difference.

WDYT about some lower impact API change like a computeCompletionProposals(ITextViewer, ITextSelection) overload?
Comment 13 Lars Vogel CLA 2019-05-23 08:56:56 EDT
(In reply to Mickael Istria from comment #9)
> (In reply to Julian Honnen from comment #8)
> > @Mickael: Shouldn't the jface API be separated in two stages?
> 
> This might work, but initial idea was to avoid creating new type and stick
> with IContentAssistProcessor to ease adoption.

Looking at this thread, I think the JDT use case justifies new API. (In reply to Julian Honnen from comment #8)

> @Mickael: Shouldn't the jface API be separated in two stages?
> 
> IAsyncContentAssistProcessor {
> 
>   // called from UI
>   Context createContext(ITextViewer viewer, int offset)
> 
>   // may be called async
>   ICompletionProposal[] computeCompletionProposals(Context context)
> }

Could we add these methods to IContentAssistProcessor as default methods? Adding new default methods to an interface should not break API.
Comment 14 Julian Honnen CLA 2019-05-23 09:23:52 EDT
(In reply to Lars Vogel from comment #13)
> Could we add these methods to IContentAssistProcessor as default methods?
> Adding new default methods to an interface should not break API.

I'll submit a gerrit with an API proposal. Let's discuss what maintenance/compatibility issues it would cause there.
Comment 15 Eclipse Genie CLA 2019-05-23 10:02:42 EDT
New Gerrit change created: https://git.eclipse.org/r/142669
Comment 16 Julian Honnen CLA 2019-05-23 10:12:16 EDT
Proposal here: https://git.eclipse.org/r/142669
Comment 17 Mickael Istria CLA 2019-05-27 06:57:45 EDT
I think fixing bug 547683 should resolve the main issues as it would allow to invoke context.getViewer().getSelection() from whatever thread.
Comment 18 Mickael Istria CLA 2019-11-05 05:45:10 EST
Tentative for 4.13, patch is waiting for review (Jeff already +1'd it but would like Andrey and/or Noopur to make the final call).
Comment 19 Noopur Gupta CLA 2019-11-05 06:13:04 EST
(In reply to Mickael Istria from comment #18)
> Tentative for 4.13, patch is waiting for review (Jeff already +1'd it but
> would like Andrey and/or Noopur to make the final call).

https://git.eclipse.org/r/#/c/129010/

I see ITextViewerExtension9#getLastKnownSelection being used now. So, I believe this will take care of Andrey's concerns. Please confirm.
Comment 20 Mickael Istria CLA 2019-11-08 03:56:03 EST
(In reply to Noopur Gupta from comment #19)
> I see ITextViewerExtension9#getLastKnownSelection being used now. So, I
> believe this will take care of Andrey's concerns. Please confirm.

Sorry, I thought you were asking Andrey.
Yes, I confirm that usage of getLastKnownSelection is here to remove the risk of deadlocks, which was Andrey's main concern.
Comment 22 Jeff Johnston CLA 2019-11-13 13:32:21 EST
Released for 4.14M3
Comment 23 Eclipse Genie CLA 2019-11-13 18:56:32 EST
New Gerrit change created: https://git.eclipse.org/r/152623
Comment 24 Eclipse Genie CLA 2019-11-13 22:51:14 EST
New Gerrit change created: https://git.eclipse.org/r/152627
Comment 25 Vikas Chandra CLA 2019-11-13 22:57:02 EST
See Bug 553023
Comment 27 Vikas Chandra CLA 2019-11-13 23:29:19 EST
Please ensure https://git.eclipse.org/r/129010  & https://git.eclipse.org/r/#/c/152623 go in
together after review.
Comment 28 Vikas Chandra CLA 2019-11-13 23:30:06 EST
Reopened for review and re-committing the patches.
Comment 29 Sarika Sinha CLA 2019-11-14 00:26:16 EST
(In reply to Jeff Johnston from comment #22)
> Released for 4.14M3

It will be good to check x-friends list while changing internal methods:
org.eclipse.jdt.internal.ui.text.template.contentassist;x-friends:="org.eclipse.jdt.debug.ui",
Comment 30 Eclipse Genie CLA 2019-11-14 10:44:34 EST
New Gerrit change created: https://git.eclipse.org/r/152673
Comment 31 Jeff Johnston CLA 2019-11-14 10:48:08 EST
(In reply to Sarika Sinha from comment #29)
> (In reply to Jeff Johnston from comment #22)
> > Released for 4.14M3
> 
> It will be good to check x-friends list while changing internal methods:
> org.eclipse.jdt.internal.ui.text.template.contentassist;x-friends:="org.
> eclipse.jdt.debug.ui",

Point taken.  I have just posted a modified patch for review with the old TemplateEngine.complete() method that JDT debug uses marked as deprecated.