Bug 568978 - Writing "continue" in run commands without any breakpoint locks up the debug session
Summary: Writing "continue" in run commands without any breakpoint locks up the debug ...
Status: NEW
Alias: None
Product: CDT
Classification: Tools
Component: cdt-debug-hardware (show other bugs)
Version: Next   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: cdt-debug-inbox@eclipse.org CLA
QA Contact: John Dallaway CLA
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-11-20 03:05 EST by Torbjörn Svensson CLA
Modified: 2021-03-15 11:59 EDT (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Torbjörn Svensson CLA 2020-11-20 03:05:48 EST
If there is no breakpoint set and a simple "continue" command is added to the run commands input field of the "GDB Hardware Debugging" launch configuration type, the debug session will start, but there is no way to get control over the target using the UI controls. The only way I've found to gain control again is to switch to the "Debugger console" view and issue the command "interrupt", but users should not be expected to do this hack...

If a session with no breakpoint and the "continue" command is suspended using the pause button, the following is logged:

!ENTRY org.eclipse.cdt.dsf 4 10005 2020-11-19 22:59:26.973
!MESSAGE Request for monitor: 'RequestMonitor (org.eclipse.cdt.dsf.debug.ui.actions.DsfSuspendCommand$4$1@454d1b1f): Status ERROR: org.eclipse.cdt.dsf.gdb code=10004 Suspend operation timeout. null' resulted in an error.
!SUBENTRY 1 org.eclipse.cdt.dsf.gdb 4 10004 2020-11-19 22:59:26.973
!MESSAGE Suspend operation timeout.

In the GDB traces, the last few lines are:
756,400 26continue
756,401 27-stack-info-depth --thread 1 11
756,402 &"continue\n"
756,402 ~"Continuing.\n"
756,402 26^running
756,402 *running,thread-id="all"
756,402 (gdb) 
756,404 28-break-list
762,719 29-exec-interrupt --thread-group i1

When I write the "interrupt" command in the "Debugger console" view, the following is added to the GDB traces (just adding the first few lines since the rest is irrelevant to the error):
803,189 ~"\nProgram"
803,189 ~" received signal SIGINT, Interrupt.\n"
803,194 ~"main () at ../src/main.c:9\n"
803,194 ~"9\t\twhile(1);\n"
803,194 *stopped,reason="signal-received",signal-name="SIGINT",signal-meaning="Interrupt",frame={add\
r="0x0800044c",func="main",args=[],file="../src/main.c",fullname="/home/azoff/eclipse/2020-09/runtim\
e-2020-09-1/f4/src/main.c",line="9",arch="arm"},thread-id="1",stopped-threads="all",core="0"
803,195 27^done,depth="1"


If I instead of issuing the "interrupt" command clocks on the GDB node in the debug tree and clicks on the stop button to terminate, the following is printed in the runtime console:
!ENTRY org.eclipse.cdt.dsf 4 -1 2020-11-19 22:59:50.007
!MESSAGE Invocation exception when calling a service event handler method
!STACK 0
java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.eclipse.cdt.dsf.service.DsfSession.doDispatchEvent(DsfSession.java:541)
	at org.eclipse.cdt.dsf.service.DsfSession$3.run(DsfSession.java:408)
	at org.eclipse.cdt.dsf.concurrent.DefaultDsfExecutor$TracingWrapperRunnable.run(DefaultDsfExecutor.java:381)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.IllegalStateException: RequestMonitor: RequestMonitor (org.eclipse.cdt.dsf.debug.ui.actions.DsfSuspendCommand$4$1@10d4894b): Status ERROR: org.eclipse.cdt.dsf.gdb code=10001 Connection is shut down null, done() method called more than once
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.done(RequestMonitor.java:288)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.done(RequestMonitor.java:345)
	at org.eclipse.cdt.dsf.gdb.service.GDBRunControl_7_12$3.handleFailure(GDBRunControl_7_12.java:116)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.handleCompleted(RequestMonitor.java:393)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor$2.run(RequestMonitor.java:317)
	at org.eclipse.cdt.dsf.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:67)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.done(RequestMonitor.java:314)
	at org.eclipse.cdt.dsf.mi.service.command.AbstractMIControl.cancelRxCommands(AbstractMIControl.java:335)
	at org.eclipse.cdt.dsf.mi.service.command.AbstractMIControl.stopCommandProcessing(AbstractMIControl.java:312)
	at org.eclipse.cdt.dsf.gdb.service.command.GDBControl_7_0.eventDispatched(GDBControl_7_0.java:130)
	... 13 more
Root exception:
java.lang.IllegalStateException: RequestMonitor: RequestMonitor (org.eclipse.cdt.dsf.debug.ui.actions.DsfSuspendCommand$4$1@10d4894b): Status ERROR: org.eclipse.cdt.dsf.gdb code=10001 Connection is shut down null, done() method called more than once
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.done(RequestMonitor.java:288)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.done(RequestMonitor.java:345)
	at org.eclipse.cdt.dsf.gdb.service.GDBRunControl_7_12$3.handleFailure(GDBRunControl_7_12.java:116)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.handleCompleted(RequestMonitor.java:393)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor$2.run(RequestMonitor.java:317)
	at org.eclipse.cdt.dsf.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:67)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.done(RequestMonitor.java:314)
	at org.eclipse.cdt.dsf.mi.service.command.AbstractMIControl.cancelRxCommands(AbstractMIControl.java:335)
	at org.eclipse.cdt.dsf.mi.service.command.AbstractMIControl.stopCommandProcessing(AbstractMIControl.java:312)
	at org.eclipse.cdt.dsf.gdb.service.command.GDBControl_7_0.eventDispatched(GDBControl_7_0.java:130)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.eclipse.cdt.dsf.service.DsfSession.doDispatchEvent(DsfSession.java:541)
	at org.eclipse.cdt.dsf.service.DsfSession$3.run(DsfSession.java:408)
	at org.eclipse.cdt.dsf.concurrent.DefaultDsfExecutor$TracingWrapperRunnable.run(DefaultDsfExecutor.java:381)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

!ENTRY org.eclipse.cdt.dsf 4 -1 2020-11-19 22:59:50.013
!MESSAGE Uncaught exception in DSF executor thread
!STACK 0
java.lang.AssertionError: Exception thrown by a IServiceEventListener.ServiceHandlerMethod method
	at org.eclipse.cdt.dsf.service.DsfSession.doDispatchEvent(DsfSession.java:549)
	at org.eclipse.cdt.dsf.service.DsfSession$3.run(DsfSession.java:408)
	at org.eclipse.cdt.dsf.concurrent.DefaultDsfExecutor$TracingWrapperRunnable.run(DefaultDsfExecutor.java:381)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Uncaught exception in session executor thread: java.lang.AssertionError: Exception thrown by a IServiceEventListener.ServiceHandlerMethod method
	at org.eclipse.cdt.dsf.service.DsfSession.doDispatchEvent(DsfSession.java:549)
	at org.eclipse.cdt.dsf.service.DsfSession$3.run(DsfSession.java:408)
	at org.eclipse.cdt.dsf.concurrent.DefaultDsfExecutor$TracingWrapperRunnable.run(DefaultDsfExecutor.java:381)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

!ENTRY org.eclipse.cdt.dsf 4 10005 2020-11-19 22:59:50.016
!MESSAGE Request for monitor: 'RequestMonitor (org.eclipse.cdt.dsf.mi.service.MIBreakpointsSynchronizer$3@6edbe4d8): Status ERROR: org.eclipse.cdt.dsf.gdb code=10001 Connection is shut down null' resulted in an error.
!SUBENTRY 1 org.eclipse.cdt.dsf.gdb 4 10001 2020-11-19 22:59:50.017
!MESSAGE Connection is shut down


All the this was verified using GDB version 8.2.50.20181213-git (GNU Tools for Arm Embedded Processors 8-2018-q4-major) on Linux.
Comment 1 Jonah Graham CLA 2020-11-20 11:42:21 EST
There are two (possibly interconnected) bugs here. I haven't tried to reproduce yet, but here is my understanding:

### -exec-interrupt ###

The -exec-interrupt not working is sort of "user" error (i.e can't put continue) or a GDB bug in async mode. The continue is issued in on the CLI without & (so in the foreground). Therefore the way to interrupt it must be by sending a Ctrl-C/SIGINT. But as the MI session is in async mode, CDT sends the -exec-interrupt.

I know there was a similar error in GDB 8.1 as in the GDB 7.12 to 9.0 timeframe a lot of refactoring was done on this area of GDB.

Are you able to test in a newer or older GDB? 7.11 and earlier won't use async mode, so the CDT will send a Ctrl-C/SIGINT. With any luck this is fixed in 9.0 or later.

If not fixed, I think we need to engage the GDB community on the correct way to interrupt in this case. i.e. how does CDT detect that we should be sending a signal to interrupt in this case? If the user types continue in the debugger console then the MI stream's -exec-interrupt works (this assumption should be tested too).

The other possibility is that this problem goes away if CDT issues these commands wrapped in -interpreter-exec console, which will make the continue synchronous on the console stream instead of synchronous on the MI stream.


### Errors on shutdown ####

This problem may stop being reproducible if the above problem is fixed. Although it will still be there lurking for another way to expose itself.

I would test this without assertions enabled (no -ea command line option) to see if the assertion exceptions are getting in the way of an otherwise clean shutdown. I would also stick a breakpoint on the line in request monitor that checks for done being called twice to find out if there is a simple error that means done is incorrectly being called twice.
Comment 2 John Dallaway CLA 2020-11-20 12:02:22 EST
Jonah, I agree that wrapping "continue" in "-interpreter-exec" using a version of GDB where the reported issue is triggered would be a useful experiment. Also use of "-exec-continue" in the Run Commands box.
Comment 3 Torbjörn Svensson CLA 2020-11-20 16:47:49 EST
I can quite easily check a few different version of GDB as long as Arm has a toolchain including that version of GDB. I've been testing the following versions:

7.10.1.20160923-cvs:
400,049 26continue
400,050 &"continue\n"
400,050 ~"Continuing.\n"
400,050 26^running
400,050 *running,thread-id="all"
400,050 (gdb) 
400,051 27-break-list
411,609 ~"\nProgram received signal "
411,610 ~"SIGINT, Interrupt.\n"
411,610 ~"main () at ../src/main.c:9\n"
411,610 ~"9\t\twhile(1);\n"
Works!

7.12.0.20161204-git:
508,325 27continue
508,325 &"continue\n"
508,325 ~"Continuing.\n"
508,325 27^running
508,326 *running,thread-id="all"
508,326 (gdb) 
508,327 28-break-list
519,482 29-exec-interrupt --thread-group i1
Hangs!

10.0.50.20200226-git
586,978 27continue
586,978 &"continue\n"
586,978 ~"Continuing.\n"
586,978 27^running
586,978 *running,thread-id="all"
586,978 (gdb) 
586,979 28-break-list
600,775 29-exec-interrupt --thread-group i1
Hangs!


So, this concludes that with just "continue" in the run commands will hang GDB 7.12 and later versions.


Wrapping the "continue" command does not help (tested with 10.0.50.20200226-git).
10.0.50.20200226-git:
824,913 27-interpreter-exec console continue
824,913 ~"Continuing.\n"
824,913 27^running
824,913 *running,thread-id="all"
824,913 (gdb) 
824,914 28-break-list
827,086 29-exec-interrupt --thread-group i1
Hangs...

Maybe CDT could track outstanding MI commands and if there is one pending with for more than 100 msec, it switches from "-exec-interrupt" to using the signal or maybe signals wont work over MI?






I do not see any obvious difference without -ea. Here is what's printed in the console after selecting the thread, clicking suspend and getting the error dialog and thenselecting the GDB node and clicking terminate. This is the same sequence I did in comment 0.

!ENTRY org.eclipse.cdt.dsf 4 -1 2020-11-20 22:37:57.696
!MESSAGE Invocation exception when calling a service event handler method
!STACK 0
java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.eclipse.cdt.dsf.service.DsfSession.doDispatchEvent(DsfSession.java:541)
	at org.eclipse.cdt.dsf.service.DsfSession$3.run(DsfSession.java:408)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.IllegalStateException: RequestMonitor: RequestMonitor (org.eclipse.cdt.dsf.debug.ui.actions.DsfSuspendCommand$4$1@40ff3c5a): Status ERROR: org.eclipse.cdt.dsf.gdb code=10001 Connection is shut down null, done() method called more than once
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.done(RequestMonitor.java:288)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.done(RequestMonitor.java:345)
	at org.eclipse.cdt.dsf.gdb.service.GDBRunControl_7_12$3.handleFailure(GDBRunControl_7_12.java:116)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.handleCompleted(RequestMonitor.java:393)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor$2.run(RequestMonitor.java:317)
	at org.eclipse.cdt.dsf.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:67)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.done(RequestMonitor.java:314)
	at org.eclipse.cdt.dsf.mi.service.command.AbstractMIControl.cancelRxCommands(AbstractMIControl.java:335)
	at org.eclipse.cdt.dsf.mi.service.command.AbstractMIControl.stopCommandProcessing(AbstractMIControl.java:312)
	at org.eclipse.cdt.dsf.gdb.service.command.GDBControl_7_0.eventDispatched(GDBControl_7_0.java:130)
	... 12 more
Root exception:
java.lang.IllegalStateException: RequestMonitor: RequestMonitor (org.eclipse.cdt.dsf.debug.ui.actions.DsfSuspendCommand$4$1@40ff3c5a): Status ERROR: org.eclipse.cdt.dsf.gdb code=10001 Connection is shut down null, done() method called more than once
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.done(RequestMonitor.java:288)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.done(RequestMonitor.java:345)
	at org.eclipse.cdt.dsf.gdb.service.GDBRunControl_7_12$3.handleFailure(GDBRunControl_7_12.java:116)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.handleCompleted(RequestMonitor.java:393)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor$2.run(RequestMonitor.java:317)
	at org.eclipse.cdt.dsf.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:67)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.done(RequestMonitor.java:314)
	at org.eclipse.cdt.dsf.mi.service.command.AbstractMIControl.cancelRxCommands(AbstractMIControl.java:335)
	at org.eclipse.cdt.dsf.mi.service.command.AbstractMIControl.stopCommandProcessing(AbstractMIControl.java:312)
	at org.eclipse.cdt.dsf.gdb.service.command.GDBControl_7_0.eventDispatched(GDBControl_7_0.java:130)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.eclipse.cdt.dsf.service.DsfSession.doDispatchEvent(DsfSession.java:541)
	at org.eclipse.cdt.dsf.service.DsfSession$3.run(DsfSession.java:408)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

!ENTRY org.eclipse.cdt.dsf 4 10005 2020-11-20 22:37:57.741
!MESSAGE Request for monitor: 'RequestMonitor (org.eclipse.cdt.dsf.mi.service.MIBreakpointsSynchronizer$3@bc0d20f): Status ERROR: org.eclipse.cdt.dsf.gdb code=10001 Connection is shut down null' resulted in an error.
!SUBENTRY 1 org.eclipse.cdt.dsf.gdb 4 10001 2020-11-20 22:37:57.741
!MESSAGE Connection is shut down



If I instead just launch the debug session and then immediately click on the GDB node and then click on terminate, all I get in the console is this:

!ENTRY org.eclipse.cdt.dsf 4 10005 2020-11-20 22:43:23.651
!MESSAGE Request for monitor: 'RequestMonitor (org.eclipse.cdt.dsf.mi.service.MIBreakpointsSynchronizer$3@2e44b2ac): Status ERROR: org.eclipse.cdt.dsf.gdb code=10001 Connection is shut down null' resulted in an error.
!SUBENTRY 1 org.eclipse.cdt.dsf.gdb 4 10001 2020-11-20 22:43:23.651
!MESSAGE Connection is shut down

My guess is that this error is related to the queued "-break-list" command that is never returning.



Is there anything more you want me to try?
Comment 4 Jonah Graham CLA 2020-11-20 16:59:41 EST
Can you try "continue &"?
Comment 5 Jonah Graham CLA 2020-11-20 17:07:48 EST
(In reply to Torbjörn Svensson from comment #3)
> Maybe CDT could track outstanding MI commands and if there is one pending
> with for more than 100 msec, it switches from "-exec-interrupt" to using the
> signal or maybe signals wont work over MI?

CDT already monitors the interrupt - so it can fall back to trying to send a signal to see if that works. Let me try to put together a quick patch you can test.
Comment 6 Torbjörn Svensson CLA 2020-11-20 17:13:51 EST
(In reply to Jonah Graham from comment #5)
> (In reply to Torbjörn Svensson from comment #3)
> > Maybe CDT could track outstanding MI commands and if there is one pending
> > with for more than 100 msec, it switches from "-exec-interrupt" to using the
> > signal or maybe signals wont work over MI?
> 
> CDT already monitors the interrupt - so it can fall back to trying to send a
> signal to see if that works. Let me try to put together a quick patch you
> can test.

I did a quick test having GDBRunControl_7_12 peeking AbstractMIControl.fRxCommands, and if it was not empty, it would simply run super.suspend... but it did not help.
Comment 7 Jonah Graham CLA 2020-11-20 17:18:34 EST
(In reply to Torbjörn Svensson from comment #6)
> I did a quick test having GDBRunControl_7_12 peeking
> AbstractMIControl.fRxCommands, and if it was not empty, it would simply run
> super.suspend... but it did not help.

Oh well - that is basically what I was going to try. So if it sounds like GDB is providing no way to interrupt the target. Then it sounds like a GDB bug and something to report to them. Perhaps someone there will have a suggestion of how to workaround it.

I will to make a reproducer that does not works on x86_64 Linux as that is most likely to be easy for others to test.
Comment 8 Eclipse Genie CLA 2020-11-20 17:19:47 EST
New Gerrit change created: https://git.eclipse.org/r/c/cdt/org.eclipse.cdt/+/172599
Comment 9 Jonah Graham CLA 2020-11-20 17:20:17 EST
(In reply to Eclipse Genie from comment #8)
> New Gerrit change created:
> https://git.eclipse.org/r/c/cdt/org.eclipse.cdt/+/172599

I saved this here as WIP, not sure if it is useful but didn't want to delete it yet.
Comment 10 Torbjörn Svensson CLA 2020-11-20 17:22:45 EST
(In reply to Jonah Graham from comment #4)
> Can you try "continue &"?

This sort-of works...

470,160 27continue &
470,162 &"continue &\n"
470,162 ~"Continuing.\n"
470,162 27^running
470,162 *running,thread-id="all"
470,162 (gdb) 
470,167 28-break-list
470,168 28^done,BreakpointTable={nr_rows="0",nr_cols="6",hdr=[{width="7",alignment="-1",col_name="nu\
mber",colhdr="Num"},{width="14",alignment="-1",col_name="type",colhdr="Type"},{width="4",alignment="\
-1",col_name="disp",colhdr="Disp"},{width="3",alignment="-1",col_name="enabled",colhdr="Enb"},{width\
="10",alignment="-1",col_name="addr",colhdr="Address"},{width="40",alignment="2",col_name="what",col\
hdr="What"}],body=[]}
470,168 (gdb) 
476,843 29-exec-interrupt --thread-group i1
476,844 29^done
476,844 (gdb) 

So, no longer stuck, but then I have this in the "Debugger console" view:


(gdb) /mnt/workspace/workspace/GCC-10-pipeline/jenkins-GCC-10-pipeline-13_20200625_1593044217/src/gdb/gdb/inferior.c:279: internal-error: inferior* find_inferior_pid(process_stratum_target*, int): Assertion `pid != 0' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Quit this debugging session? (y or n)     


Rolling back to 8.3.1.20191211-git solves the crash and I have control over the debug session. 8.3.1.20191211-git is included the last stable release from Arm ( 9-2020-q2-update package).
Comment 11 Jonah Graham CLA 2020-11-20 17:26:02 EST
(In reply to Jonah Graham from comment #7)
> I will to make a reproducer that does not works on x86_64 Linux as that is
> most likely to be easy for others to test.

I managed to reproduce with gdbserver on x86_64 Linux AFAICT - but I have to run now.
Comment 12 Jonah Graham CLA 2020-11-20 18:11:05 EST
(In reply to Torbjörn Svensson from comment #6)
> I did a quick test having GDBRunControl_7_12 peeking
> AbstractMIControl.fRxCommands, and if it was not empty, it would simply run
> super.suspend... but it did not help.

The issue with this, as well as my patch, is that the SIGINT is being processed by the wrong stream. The CLI stream is processing it, probably because the CLI stream is the primary stream connected to GDB :(
Comment 13 Jonah Graham CLA 2020-11-24 12:39:58 EST
I think the solution to the continue problem is to document it. Maybe even scan for continue in the dialog and warn. It was a reasonable test to add continue in there for Bug 568228, but that bug could be tested with -exec-continue instead.

As for the exceptions, that needs to be resolved though.
Comment 14 Jonah Graham CLA 2020-11-24 12:42:08 EST
(In reply to Jonah Graham from comment #13)
> As for the exceptions, that needs to be resolved though.

I think this may be a duplicate of Bug 569123, but has a proposed fix already. Torbjörn if you get a chance can you test this use case against the fix in Bug 569123.
Comment 15 Torbjörn Svensson CLA 2020-11-25 03:41:18 EST
(In reply to Jonah Graham from comment #14)
> (In reply to Jonah Graham from comment #13)
> > As for the exceptions, that needs to be resolved though.
> 
> I think this may be a duplicate of Bug 569123, but has a proposed fix
> already. Torbjörn if you get a chance can you test this use case against the
> fix in Bug 569123.

I tried the gerrit in bug 569123 on top of the gerrit for bug 568228.

1. Enter "continue" into the run commands
2. Launch the debug session
3. Select the "gdb" node in the "Debug" view.
4. Click on terminate button

With this sequence, I get the following in the GDB traces:
838,781 26continue
838,781 &"continue\n"
838,781 ~"Continuing.\n"
838,781 26^running
838,781 *running,thread-id="all"
838,781 (gdb) 
838,783 27-break-list
848,347 28-gdb-exit

The corresponding output in the runtime console is:
!ENTRY org.eclipse.cdt.dsf 4 10005 2020-11-25 09:27:30.356
!MESSAGE Request for monitor: 'RequestMonitor (org.eclipse.cdt.dsf.mi.service.MIBreakpointsSynchronizer$3@5c5ed399): Status ERROR: org.eclipse.cdt.dsf.gdb code=10001 Connection is shut down null' resulted in an error.
!SUBENTRY 1 org.eclipse.cdt.dsf.gdb 4 10001 2020-11-25 09:27:30.356
!MESSAGE Connection is shut down
850,347 =thread-group-exited,id="i1"


Now I try a slightly different sequence.

1. Enter "continue" into the run commands
2. Launch the debug session
3. Select "f4.elf" (my application)
4. Click on suspend and wait for the timeout. Close the error dialog regarding the timeout.
6. Select the "gdb" node in the "Debug" view.
7. Click on terminate button

With this sequence, I get the following in the GDB traces:
333,849 26continue
333,849 &"continue\n"
333,849 ~"Continuing.\n"
333,849 26^running
333,849 *running,thread-id="all"
333,849 (gdb) 
333,850 27-break-list
340,282 28-exec-interrupt --thread-group i1
353,517 29-gdb-exit
355,517 =thread-group-exited,id="i1"

The corresponding output in the runtime console is:
!ENTRY org.eclipse.cdt.dsf 4 10005 2020-11-25 09:35:45.333
!MESSAGE Request for monitor: 'RequestMonitor (org.eclipse.cdt.dsf.debug.ui.actions.DsfSuspendCommand$4$1@60deebdb): Status ERROR: org.eclipse.cdt.dsf.gdb code=10004 Suspend operation timeout. null' resulted in an error.
!SUBENTRY 1 org.eclipse.cdt.dsf.gdb 4 10004 2020-11-25 09:35:45.334
!MESSAGE Suspend operation timeout.

!ENTRY org.eclipse.cdt.dsf 4 -1 2020-11-25 09:35:55.520
!MESSAGE Invocation exception when calling a service event handler method
!STACK 0
java.lang.reflect.InvocationTargetException
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.eclipse.cdt.dsf.service.DsfSession.doDispatchEvent(DsfSession.java:541)
	at org.eclipse.cdt.dsf.service.DsfSession$3.run(DsfSession.java:408)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.IllegalStateException: RequestMonitor: RequestMonitor (org.eclipse.cdt.dsf.debug.ui.actions.DsfSuspendCommand$4$1@60deebdb): Status ERROR: org.eclipse.cdt.dsf.gdb code=10001 Connection is shut down null, done() method called more than once
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.done(RequestMonitor.java:288)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.done(RequestMonitor.java:345)
	at org.eclipse.cdt.dsf.gdb.service.GDBRunControl_7_12$3.handleFailure(GDBRunControl_7_12.java:116)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.handleCompleted(RequestMonitor.java:393)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor$2.run(RequestMonitor.java:317)
	at org.eclipse.cdt.dsf.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:67)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.done(RequestMonitor.java:314)
	at org.eclipse.cdt.dsf.mi.service.command.AbstractMIControl.cancelRxCommands(AbstractMIControl.java:341)
	at org.eclipse.cdt.dsf.mi.service.command.AbstractMIControl.stopCommandProcessing(AbstractMIControl.java:313)
	at org.eclipse.cdt.dsf.gdb.service.command.GDBControl_7_0.eventDispatched(GDBControl_7_0.java:130)
	... 12 more
Root exception:
java.lang.IllegalStateException: RequestMonitor: RequestMonitor (org.eclipse.cdt.dsf.debug.ui.actions.DsfSuspendCommand$4$1@60deebdb): Status ERROR: org.eclipse.cdt.dsf.gdb code=10001 Connection is shut down null, done() method called more than once
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.done(RequestMonitor.java:288)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.done(RequestMonitor.java:345)
	at org.eclipse.cdt.dsf.gdb.service.GDBRunControl_7_12$3.handleFailure(GDBRunControl_7_12.java:116)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.handleCompleted(RequestMonitor.java:393)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor$2.run(RequestMonitor.java:317)
	at org.eclipse.cdt.dsf.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:67)
	at org.eclipse.cdt.dsf.concurrent.RequestMonitor.done(RequestMonitor.java:314)
	at org.eclipse.cdt.dsf.mi.service.command.AbstractMIControl.cancelRxCommands(AbstractMIControl.java:341)
	at org.eclipse.cdt.dsf.mi.service.command.AbstractMIControl.stopCommandProcessing(AbstractMIControl.java:313)
	at org.eclipse.cdt.dsf.gdb.service.command.GDBControl_7_0.eventDispatched(GDBControl_7_0.java:130)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.eclipse.cdt.dsf.service.DsfSession.doDispatchEvent(DsfSession.java:541)
	at org.eclipse.cdt.dsf.service.DsfSession$3.run(DsfSession.java:408)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

!ENTRY org.eclipse.cdt.dsf 4 10005 2020-11-25 09:35:55.522
!MESSAGE Request for monitor: 'RequestMonitor (org.eclipse.cdt.dsf.mi.service.MIBreakpointsSynchronizer$3@385fbb1a): Status ERROR: org.eclipse.cdt.dsf.gdb code=10001 Connection is shut down null' resulted in an error.
!SUBENTRY 1 org.eclipse.cdt.dsf.gdb 4 10001 2020-11-25 09:35:55.522
!MESSAGE Connection is shut down


The first log entry is from the suspend operation, the rest is from the terminate operation.


I have not analysed the stack traces in depth, but it looks like the issue remains even if the fRxCommands is cleared before calling done on the CommandHandle instances.

@Jonah: If you want me to test something different, please ping me.
Comment 16 Jonah Graham CLA 2021-03-15 11:59:07 EDT
Removing target milestone.