Bug 542089 - Cannot set conditional debug breakpoint or evaluate expressions in external maven sources (like Spring)
Summary: Cannot set conditional debug breakpoint or evaluate expressions in external m...
Status: ASSIGNED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Debug (show other bugs)
Version: 4.8   Edit
Hardware: PC Mac OS X
: P3 normal with 4 votes (vote)
Target Milestone: ---   Edit
Assignee: Igor Fedorenko CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords:
Depends on:
Blocks:
 
Reported: 2018-12-04 08:54 EST by Phillip Verheyden CLA
Modified: 2024-03-25 17:49 EDT (History)
10 users (show)

See Also:


Attachments
Failing prjoect (8.97 KB, application/zip)
2018-12-04 08:54 EST, Phillip Verheyden CLA
no flags Details
simple jdt project that demonstrates the problem (157.63 KB, application/zip)
2018-12-18 22:14 EST, Igor Fedorenko CLA
no flags Details
Screenshot for the issue in 2023-09 (926.29 KB, image/png)
2023-09-26 14:41 EDT, Yurii Kartsev CLA
no flags Details
Screenshot of the bug in version 2023-09 (272.65 KB, image/jpeg)
2023-09-26 14:51 EDT, Yurii Kartsev CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Phillip Verheyden CLA 2018-12-04 08:54:00 EST
Created attachment 276810 [details]
Failing prjoect

Final output: https://user-images.githubusercontent.com/684275/49231352-61318480-f3b7-11e8-8345-5dad4a5856fe.png. Text: Conditional Breakpoint has Compilation error(s). Reason: Unable to compile conditional breakpoint - missing Java project context.

Reproduction steps:

1. Download the latest Eclipse 2018-09 R from https://www.eclipse.org/downloads/packages/release/2018-09/r/eclipse-ide-java-ee-developers
2. Imported the attached project as File -> Import -> Existing Maven Project
3. CMD + Shift + T to open `DefaultListableBeanFactory`, set a breakpoint in `registerBeanDefinition()` to hit when the `BeanFromTest` is registered
4. Hit 'Debug as Junit Test' to debug the test

The breakpoint hits on every single invocation because of the "Unable to compile conditional breakpoint - Missing Java Project Context". Also if you look in the screenshot, you'll see that 2 instances of `DefaultListableBeanFactory` are opened: 1 that has the little Java bytecode argument (which is where I originally set the breakpoint) and one that just has a `J`, which was opened when the breakpoint hit. The `J` icon usually means that the project is not a Java project.

Also might be interesting is that before running the JUnit test at all, I had previously downloaded the sources for `spring-context` so that was already in my `~/.m2/repository`. It did not need to download it on-demand.

I also cannot do things I normally can in Java debugging, like right-click and "Inspect" with 'Unable to evaluate the selected expression' (even though hovering works):

Screen recording of 'Inspect' problem: https://user-images.githubusercontent.com/684275/49232448-8f17c880-f3b9-11e8-993d-b9727b94b844.gif


Originally opened at https://github.com/spring-projects/sts4/issues/152 within STS, validated broken in core Eclipse.
Comment 1 Sarika Sinha CLA 2018-12-04 22:18:59 EST
Moving to Maven for comment.
Comment 2 Igor Fedorenko CLA 2018-12-09 08:23:12 EST
I just imported the provided project into fresh/clean workspace and cannot find DefaultListableBeanFactory class using Java search. My _guess_, the class is loaded dynamically by spring-boot and JDT simply does not have enough information about the class to provide conditional breakpoints there.

You should be able to get conditional breakpoints in the class by importing the artifact containing DefaultListableBeanFactory as Maven Binary Project (action available from right-click on the stack frame in the debug view.
Comment 3 Phillip Verheyden CLA 2018-12-09 16:33:51 EST
@Igor I'm not really sure what you were doing, but I was easily able to find DefaultListableBeanFactory using the exact same methods you were. Here is a GIF recording: https://cl.ly/82ea9969282e

As you can see, the class is not loaded dynamically by Spring Boot it is in spring-beans-5.1.2.RELEASE.jar on the classpath from Maven.

Also, if it was loaded dynamically, how would I have been able to set a breakpoint to begin with?
Comment 4 Phillip Verheyden CLA 2018-12-09 16:44:08 EST
@Igor I did take your advice about the right-click on the stack frame. If I go to 'Source Lookup Info' I see this: https://cl.ly/3fce557c9895 (ExternalArchiveSourceContainer). If I then take your advice and right click on the stack frame entry and do 'Import Binary Project', then it looks like the conditional breakpoints work as I expect. I also did the 'Source Lookup Info' again and the info is different: https://cl.ly/2b3cb06eb342 it's now a 'Java Project Source Container'.

Also, if I go to my project explorer I now see another project called org.springframework_spring-beans_5.1.2.RELEASE.

I suppose this does work as a workaround, so definitely thanks for that! However I personally see 2 issues with this that makes the solution feel incomplete:

1. The fact that conditional breakpoints in maven dependencies no longer work by default (they did in previous Eclipse releases without having to do a manual 'Import Binary Project' from the debugger view)
2. Another project is added to my workspace that I then need to filter out

Maybe something changed in how projects and sources are determined with Maven projects? Is this now the only way to keep sources and binaries in sync with the JDT?

Thanks again for your help!
Comment 5 Phillip Verheyden CLA 2018-12-09 16:47:34 EST
Another thing: sometimes I actually had to 'Import Binary Project' twice. The first time, the source in the editor changed to 'Source Not Found'. If I did 'Import Binary Project' one more time on the same class, everything turned fully operational.

It now looks like the binaries and sources are in sync (see the icon in the editor, it's the Java binary symbol, not a 'J'): https://cl.ly/2f25bc65b75
Comment 6 Igor Fedorenko CLA 2018-12-09 20:23:14 EST
(In reply to Phillip Verheyden from comment #3)
> @Igor I'm not really sure what you were doing, 

I was at an airport and most likely network there wasn't good enough for m2e. I can reproduce the problem now.

> Also, if it was loaded dynamically, how would I have been able to set a
> breakpoint to begin with?

Starting with Eclipse 4.8, jdt+m2e are able to discover sources of dynamically loaded classes in many cases (in nutshells, jdt knows jar file of the class and m2e knows how to match the jar to a maven artifact)

> Another thing: sometimes I actually had to 'Import Binary Project' twice.

There is a bug somewhere in jdt ui cache. I never was motivated enough to find and fix it. Source Lookup Info dialog has "Refresh" button you can use instead of re-importing binary project.

> 1. The fact that conditional breakpoints in maven dependencies no longer
> work by default (they did in previous Eclipse releases without having to do
> a manual 'Import Binary Project' from the debugger view)

Agree this is a regression, but want to clarify it only affects small-ish subset of codepaths, in most cases source lookup for maven projects works as expected. I need to think how to improve the regressed scenario, don't see how to solve it properly yet.
Comment 7 Igor Fedorenko CLA 2018-12-15 12:27:34 EST
There are two problems at play here. 

First, the advanced source lookup does not consider the java project associated with the launch configured to be "in the context" of the source lookup. I submitted https://git.eclipse.org/r/c/134050/ to address this, which should address the immediate problem reported in this bugzilla.

Second, there is currently no way to associate source lookup information with existing "source-less" IPackageFragmentRoot, so if debugger stops at such classpath entry we have to choose between showing the sources or supporting conditional breakpoints (and other debugger features that require java project context). The proper fix requires changes to jdt core. I have rough idea what needs to be done there, but would like to get confirmation from jdt developers they will be able to work with me on the changes before I start working on this.
Comment 8 Sarika Sinha CLA 2018-12-16 23:16:25 EST
I will review https://git.eclipse.org/r/#/c/134050/

Adding Jay and Stephan from JDT Core.
Comment 9 Igor Fedorenko CLA 2018-12-18 22:14:45 EST
Created attachment 276955 [details]
simple jdt project that demonstrates the problem

bug542089.zip is a simple java project that can be used to demonstrate the problems:

* import the project into existing fresh/clean eclipse workspace
* open junit.framework.TestSuite using Command-Shift-T java search dialog and set conditional breakpoint at line 203
* debug bug542089.Bug542089Test as junit test

Observe that breakpoint condition evaluation fails when using R4_10_maintenance code and works as expected with the changes from https://git.eclipse.org/r/c/134050/.
Comment 10 Sarika Sinha CLA 2019-05-17 06:59:20 EDT
Not sure, why the gerrit was not attached, will look at this in early 4.13.
Comment 11 Stephan Herrmann CLA 2019-08-20 05:10:38 EDT
Sarika Sinha changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Target Milestone|4.13                        |4.13 M1

I assume you meant a *future* M1 :)
Comment 12 Sarika Sinha CLA 2019-08-20 05:27:41 EDT
(In reply to Stephan Herrmann from comment #11)
> Sarika Sinha changed:
> 
>            What    |Removed                     |Added
> ----------------------------------------------------------------------------
>    Target Milestone|4.13                        |4.13 M1
> 
> I assume you meant a *future* M1 :)

Yes :)
Comment 13 Sarika Sinha CLA 2019-11-13 07:18:53 EST
(In reply to Igor Fedorenko from comment #9)
> Created attachment 276955 [details]
> simple jdt project that demonstrates the problem
> 
> bug542089.zip is a simple java project that can be used to demonstrate the
> problems:
> 
> * import the project into existing fresh/clean eclipse workspace
> * open junit.framework.TestSuite using Command-Shift-T java search dialog
> and set conditional breakpoint at line 203
> * debug bug542089.Bug542089Test as junit test
> 
> Observe that breakpoint condition evaluation fails when using
> R4_10_maintenance code and works as expected with the changes from
> https://git.eclipse.org/r/c/134050/.

Even without the gerrit patch, I see the breakpoint condition evaluation succeeds with 4.13.
Comment 14 Igor Fedorenko CLA 2019-12-10 10:22:06 EST
(In reply to Sarika Sinha from comment #13)
> (In reply to Igor Fedorenko from comment #9)
> > Created attachment 276955 [details]
> > simple jdt project that demonstrates the problem
> > 
> > bug542089.zip is a simple java project that can be used to demonstrate the
> > problems:
> > 
> > * import the project into existing fresh/clean eclipse workspace
> > * open junit.framework.TestSuite using Command-Shift-T java search dialog
> > and set conditional breakpoint at line 203
> > * debug bug542089.Bug542089Test as junit test
> > 
> > Observe that breakpoint condition evaluation fails when using
> > R4_10_maintenance code and works as expected with the changes from
> > https://git.eclipse.org/r/c/134050/.
> 
> Even without the gerrit patch, I see the breakpoint condition evaluation
> succeeds with 4.13.

Advanced source lookup falls back to the legacy behaviour, i.e. full workspace scanning, if it can't find context-specific source code. This fall-back masks the problem, I believe. You should be able to reproduce the problem if you comment out [1].

The fallback is suboptimal for two reasons. First, it can pick wrong source code if workspace includes multiple versions of the same library, which is rather common for large multi-module maven projects. Second, full workspace scan is also slow for large workspaces.

Hope this helps.


[1] https://git.eclipse.org/c/jdt/eclipse.jdt.debug.git/tree/org.eclipse.jdt.launching/launching/org/eclipse/jdt/internal/launching/sourcelookup/advanced/AdvancedSourceLookupDirector.java?h=S4_14_0_RC2#n51
Comment 15 Jeremy Krieg CLA 2021-01-15 01:41:21 EST
I have just hit this same bug, but in Bndtools rather than m2e. Seems to be the same root issue - the Bndtools can launch with jars that are not on any project path or can be different versions of the jars that were used for building, so it uses its own custom "Bnd Dependencies" source container that uses ExternalArchiveSourceContainer. This stops the conditional breakpoints from compiling. See https://github.com/bndtools/bnd/issues/4506

A workaround is to start a new project with the launch file in there, add all of the run-time dependencies to the project's build-path, and circumvent Bndtool's own built-in source lookup functionality.
Comment 16 Jeremy Krieg CLA 2021-01-15 01:42:09 EST
(In reply to Jeremy Krieg from comment #15)
> I have just hit this same bug, but in Bndtools rather than m2e. Seems to be
> the same root issue - the Bndtools can launch with jars that are not on any
> project path or can be different versions of the jars that were used for
> building, so it uses its own custom "Bnd Dependencies" source container that
> uses ExternalArchiveSourceContainer. This stops the conditional breakpoints
> from compiling. See https://github.com/bndtools/bnd/issues/4506
> 
> A workaround is to start a new project with the launch file in there, add
> all of the run-time dependencies to the project's build-path, and circumvent
> Bndtool's own built-in source lookup functionality.

Sorry, I forgot to add that this happened with Eclipse 2020-09.
Comment 17 Eclipse Genie CLA 2023-02-02 09:26:14 EST
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.
Comment 18 Witold Markowski CLA 2023-07-28 05:37:15 EDT
The simple test project from attachment 276955 [details] is not reproducible since Eclipse 2023-03-R so it looks like the issue is already fixed since that release. It also works with 2023-06-R.

Can somebody please recheck and confirm that it works? More info about the test case in comment #9.
Comment 19 Yurii Kartsev CLA 2023-09-26 14:41:38 EDT
Created attachment 289187 [details]
Screenshot for the issue in 2023-09
Comment 20 Yurii Kartsev CLA 2023-09-26 14:46:17 EDT
Hi. I have just faced the same issue with Version: 2023-09 (4.29.0), Build id: 20230907-1323. The issue faced when putting a conditional breakpoint in Cucumber source: attachment 289187 [details]

Not sure if this is related to the same bug, but I think it's useful to report it... Not sure how to resolve it.

Thank you.
Comment 21 Yurii Kartsev CLA 2023-09-26 14:51:17 EDT
Created attachment 289188 [details]
Screenshot of the bug in version 2023-09
Comment 22 Yurii Kartsev CLA 2023-09-26 14:52:55 EDT
(In reply to Yurii Kartsev from comment #20)
> Hi. I have just faced the same issue with Version: 2023-09 (4.29.0), Build
> id: 20230907-1323. The issue faced when putting a conditional breakpoint in
> Cucumber source: attachment screenshot
> 
> Not sure if this is related to the same bug, but I think it's useful to
> report it... Not sure how to resolve it.
> 
> Thank you.

Updated screenshot: attachment 289188 [details]
Comment 23 Steven Schlansker CLA 2024-03-25 17:49:21 EDT
Still experiencing this issue in 2024-03 (4.31.0)