Bug 560889 - [formatter] Unneeded wraps with "Format edited lines" save action
Summary: [formatter] Unneeded wraps with "Format edited lines" save action
Status: VERIFIED FIXED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 4.15   Edit
Hardware: All All
: P3 major (vote)
Target Milestone: 4.16 M1   Edit
Assignee: Mateusz Matela CLA
QA Contact:
URL:
Whiteboard:
Keywords: regression
: 561441 561813 562604 (view as bug list)
Depends on:
Blocks:
 
Reported: 2020-03-07 17:16 EST by Garret Wilson CLA
Modified: 2020-07-14 17:29 EDT (History)
6 users (show)

See Also:


Attachments
Eclipse Java format configuration to reproduce bug. (40.75 KB, text/xml)
2020-03-15 18:49 EDT, Garret Wilson CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Garret Wilson CLA 2020-03-07 17:16:47 EST
I'm using Eclipse Enterprise 2020-03 M3 (4.15.0M3) on Windows 10.

I have a custom formatting profile that indents using tabs and only wraps at 160 characters. (I've been using this profile for years and years.) In addition I have my Java editor save actions to format the source code, but only edited lines.

I have the following code (note the tabbed indentation):

```
	/**
	 * Loads an excerpt from the given source input stream and returns it as a document fragment.
	 * <p>
	 * The document fragment must be in XHTML using the HTML namespace.
	 * </p>
	 * @implNote The returned document fragment will not yet have been processed. For example, no expressions will have been evaluated and links will still
	 *           reference source paths. This will likely be changed or otherwise improved in the future.
	 * @param context The context of static site generation.
	 * @param inputStream The input stream from which to to load the excerpt.
	 * @param name The full source identifier of the document, such as a path or URL.
	 * @return A document fragment providing an excerpt, if available, of the source content of the artifact to generate.
	 * @throws IOException if there is an error loading and/or converting the source file contents.
	 * @throws DOMException if there is some error manipulating the XML document object model.
	 */
	public Optional<DocumentFragment> loadSourceExcerpt(@Nonnull MummyContext context, @Nonnull InputStream inputStream, @Nonnull final String name) throws IOException, DOMException;
```

If I hit `Ctrl+Shift+F`, it wraps the method signature (as it should), as that line otherwise goes beyond 160 characters:

```
	/**
	 * Loads an excerpt from the given source input stream and returns it as a document fragment.
	 * <p>
	 * The document fragment must be in XHTML using the HTML namespace.
	 * </p>
	 * @implNote The returned document fragment will not yet have been processed. For example, no expressions will have been evaluated and links will still
	 *           reference source paths. This will likely be changed or otherwise improved in the future.
	 * @param context The context of static site generation.
	 * @param inputStream The input stream from which to to load the excerpt.
	 * @param name The full source identifier of the document, such as a path or URL.
	 * @return A document fragment providing an excerpt, if available, of the source content of the artifact to generate.
	 * @throws IOException if there is an error loading and/or converting the source file contents.
	 * @throws DOMException if there is some error manipulating the XML document object model.
	 */
	public Optional<DocumentFragment> loadSourceExcerpt(@Nonnull MummyContext context, @Nonnull InputStream inputStream, @Nonnull final String name)
			throws IOException, DOMException;
```

Wonderful.

But then if I hit `Ctrl+S` and save the file, it unnecessarily wraps the method signature:

```
	/**
	 * Loads an excerpt from the given source input stream and returns it as a document fragment.
	 * <p>
	 * The document fragment must be in XHTML using the HTML namespace.
	 * </p>
	 * @implNote The returned document fragment will not yet have been processed. For example, no expressions will have been evaluated and links will still
	 *           reference source paths. This will likely be changed or otherwise improved in the future.
	 * @param context The context of static site generation.
	 * @param inputStream The input stream from which to to load the excerpt.
	 * @param name The full source identifier of the document, such as a path or URL.
	 * @return A document fragment providing an excerpt, if available, of the source content of the artifact to generate.
	 * @throws IOException if there is an error loading and/or converting the source file contents.
	 * @throws DOMException if there is some error manipulating the XML document object model.
	 */
	public Optional<DocumentFragment> loadSourceExcerpt(@Nonnull MummyContext context, @Nonnull InputStream inputStream,
			@Nonnull final String name)
			throws IOException, DOMException;
```

There are two problems here. First, the line was not over 160 character wide, so it should not have been wrapped. Secondly, if it was already formatted one way using `Ctrl+Shift+F`, why was it formatted differently when performing the save actions?

I can't say for sure, but I'm pretty sure this is a new bug in the 4.15 series.
Comment 1 Garret Wilson CLA 2020-03-07 17:19:18 EST
Just to clarify, the manual formatting places the `throws` clause `throws IOException, DOMException;` on the next line. But the save action then goes back and inappropriately breaks the line _again_ to place `@Nonnull final String name)` on a separate line all by itself.
Comment 2 Garret Wilson CLA 2020-03-07 17:42:01 EST
Oh, no, is the formatter broken (again!) across the board in 2020-03 M3?

I have the following code:

```																		getLogger().warn("Invalid S3 object fingerprint metadata `{}` for key `{}`: {}", METADATA_CONTENT_FINGERPRINT, key, illegalArgumentException.getLocalizedMessage(), illegalArgumentException);
```

This will look odd here (because Bugzilla formatting is horrible and) because the line is indented nine places (using tabs, as I mentioned).

I do `Ctrl+Shift+F` and Eclipse formats the line to:

```
									getLogger().warn("Invalid S3 object fingerprint metadata `{}` for key `{}`: {}", METADATA_CONTENT_FINGERPRINT, key,
											illegalArgumentException.getLocalizedMessage(), illegalArgumentException);
```

Wonderful! The second part of the wrapped line is indented 11 times (two more times than the line above it), as it should be.

But then I hit `Ctrl+S`.

```
									getLogger().warn("Invalid S3 object fingerprint metadata `{}` for key `{}`: {}", METADATA_CONTENT_FINGERPRINT, key,
													illegalArgumentException.getLocalizedMessage(), illegalArgumentException);
```

Now the second part of the line is indented 13 times!! The formatter moved the second line over two more indention spaces.

Peeps, this is affecting my existing code. If I make the tiniest of edits somewhere, this formatter bug screws up the formatting, messes up my diffs, etc.

I had waited to switch from 2019-12 because 2020-03 M1 was so buggy. But now I've switched to 2020-03 M3 and I didn't realize this other big bug until now. Please fix this soon to keep from messing up my existing content.

(Have you noticed a trend that so many Eclipse releases over the past few years break the formatting over and over? Could we perhaps improve the QA regression test suite to catch these before they affect the users? Thank you.)
Comment 3 Mateusz Matela CLA 2020-03-15 18:09:28 EDT
Cannot reproduce using your examples on 4.15RC2 with a default formatter profile (with line width changed to 160 chars).
Can you check what is needed to see the effect starting with a fresh workspace? It may be your full formatter profile or a more complete code sample. Are you sure there are no 3rd party plugins that may interfere here?
Comment 4 Garret Wilson CLA 2020-03-15 18:11:48 EDT
Did you try it using the tab character (represented by two spaces) for indentation?

Are you using the Enterprise package unzipped? Don't use the Oomph installer or whatever it is, and don't use the plain distribution. Use the Enterprise one for Windows x64.

If you still can't reproduce it, I'll attach the custom formatting configuration I'm using.
Comment 5 Garret Wilson CLA 2020-03-15 18:13:11 EDT
> Cannot reproduce using your examples on 4.15RC2 …

I just looked at my bug report and it appears I'm using M3, not M2.

Have you tried reproducing this with the version indicated in the bug report?
Comment 6 Mateusz Matela CLA 2020-03-15 18:30:13 EDT
(In reply to Garret Wilson from comment #4)
> Did you try it using the tab character (represented by two spaces) for
> indentation?

Yes, the default profile indents with tabs (what do you mean "represented by two spaces"? did you set tab with to 2?)

> 
> Are you using the Enterprise package unzipped? Don't use the Oomph installer
> or whatever it is, and don't use the plain distribution. Use the Enterprise
> one for Windows x64.

I use a version from here: https://download.eclipse.org/eclipse/downloads/drops4/S-4.15RC2-202003050155/
Where do you find the EE package of M3?

Are you certain that the problem doesn't occur with other installation methods? This would be the main clue for finding the root problem. I've never encountered a situation like that, the JDT jars are the same as long as the release version number is the same.

> 
> If you still can't reproduce it, I'll attach the custom formatting
> configuration I'm using.

It would save time if you attached it right away.


(In reply to Garret Wilson from comment #5)
> > Cannot reproduce using your examples on 4.15RC2 …
> 
> I just looked at my bug report and it appears I'm using M3, not M2.
> 
> Have you tried reproducing this with the version indicated in the bug report?

RC2 is more recent than M3. So if the bug is no longer there, we can close the bug as WORKSFORME.



(In reply to Garret Wilson from comment #2)
> I had waited to switch from 2019-12 because 2020-03 M1 was so buggy. But now
> I've switched to 2020-03 M3 and I didn't realize this other big bug until
> now. Please fix this soon to keep from messing up my existing content.
> 
> (Have you noticed a trend that so many Eclipse releases over the past few
> years break the formatting over and over? Could we perhaps improve the QA
> regression test suite to catch these before they affect the users? Thank
> you.)

The milestone builds (M*) are not advertised as production ready AFAIK. So it's great if you use them to help us find errors, but you shouldn't expect it to work perfectly.
And any community effort on improving QA is always welcome :)
Comment 7 Garret Wilson CLA 2020-03-15 18:42:10 EDT
> Where do you find the EE package of M3?

https://www.eclipse.org/downloads/packages/release/2020-03/m3

> RC2 is more recent than M3. So if the bug is no longer there, we can close the bug as WORKSFORME.

But the "if the bug is no longer there" is your assumption.

Just because you can't reproduce it doesn't meant it isn't there. I mean, it will be great if it isn't there, but you won't know for sure until you _are_ able to reproduce it under the reporting conditions.

> The milestone builds (M*) are not advertised as production ready AFAIK. So it's
> great if you use them to help us find errors, but you shouldn't expect it to
> work perfectly.

I found a problem, and I am reporting it so that hopefully it will get fixed before final release.

I know you didn't mean it so badly, but this whole attitude of "well, it's a milestone, what do you expect" is a little frustrating from where I'm coming from. If you read the comments of Bug 558434. Basically the story goes like this:

1. In Bug 558434, Eclipse 2019-12 _final release_ (not a milestone) broke something that had been broken time and time again. I reported this.
2. They supposedly fixed the problem, and was like, "please, please install a milestone to verify that it was fixed."
3. I'm like, "I'm busy, and the milestones break all over the place, and it will waste a huge chunk of my time."
4. The response was, "come on, it can't take that long; install a milestone".
5. So fine, I installed the milestone, and sure enough, it broke all over the place and I wasted my time filing Bug 559629 and Bug 559638 for example, the first of which I might add has not even been addressed yet. So I had to roll back to 2019-12 just to get work done.
6. So now I install M3 just to see if all those bugs in M2 have been fixed, and I find another bug, and I report it so that it won't get into GA, and the response I get is, "duh, it's a milestone, what did you expect"?

Yeah, so that is not the response I was looking for. I hope you understand where I'm coming from.
Comment 8 Garret Wilson CLA 2020-03-15 18:49:29 EDT
Created attachment 282116 [details]
Eclipse Java format configuration to reproduce bug.

> It would save time if you attached it right away.

Done.
Comment 9 Mateusz Matela CLA 2020-03-16 17:58:16 EDT
Using the exact same version, the same formatter profile, still no luck.
Maybe it's something in the Save Actions config - can you share a screenshot?

(In reply to Garret Wilson from comment #7)
> "duh, it's a milestone, what did you expect"?

Oh come on, I didn't mean anything like that. It's just that your additional rants left me with the impression that you did expect milestone builds to be production ready, so I wanted to make sure there's no misunderstanding.
Comment 10 Michael Tiller CLA 2020-03-19 06:39:18 EDT
Hello,
I'm having the same issue. I started from 2020-03-M2 but updated a couple of time, and I now have the following versions:

  Buildship: Eclipse Plug-ins for Gradle	3.1.3.v20191118-1057
  EclEmma Java Code Coverage	3.1.3.202003132300
  Eclipse Java Development Tools	3.18.300.v20200305-0155
  Eclipse Java Web Developer Tools	3.15.0.v201908261515
  Eclipse Platform	4.15.0.I20200305-0155
  Git integration for Eclipse	5.7.0.202003110725-r
  Kotlin	0.8.20.v20200316-1305
  Marketplace Client	1.8.2.v20200309-0038
  SonarLint for Eclipse	5.0.0.15138
  Yaml Editor	1.6.2
  Yet another Terminate Button (YaTB)	1.0.0.201603281533

Formatter: https://stuff.stooit.com/d/1/5e734a35aea9a/formatter.xml
Class example: https://stuff.stooit.com/d/1/5e734a360d7f5/StationRegistry.java
I reduced my save actions to the minimum: https://stuff.stooit.com/d/1/5e734b34e625a/save-actions.png

If line 28 you remove the linebreak between "this" and ".getStation" to have "this.getStation(stationId)". Then when you save you end-up with: https://stuff.stooit.com/d/1/5e734b34d1d49/issue.png
Where if you run the formatter manually, you have : https://stuff.stooit.com/d/1/5e734ba63481d/no-issue.png

Hope this is getting fixed before release because it is really annoying on day to day use

Michael
Comment 11 Garret Wilson CLA 2020-03-20 10:27:41 EDT
I just retested this on Eclipse Enterprise 2020-03 (4.15.0) (final release, released yesterday) on Windows 10. The formatting bugs I reported seem to to longer occur. (I noticed a slight variation in formatting in a separate case, but it may not be related and it doesn't rise to bug level.) Perhaps Michael Tiller can confirm this.

> The milestone builds (M*) are not advertised as production ready AFAIK.
> So it's great if you use them to help us find errors, but you shouldn't
> expect it to work perfectly.

You're conflating two categories of errors here.

No, of course I don't expect new features to be bug-free or to work perfectly. That's why this is a milestone — a milestone towards the end goal of adding a new feature.

On the other hand, for the most part I do expect existing functionality to continue to work bug-free. That's why we should have regression tests. For example, here is a set of unit tests for my [XML DOM serializer and formatter](https://github.com/globalmentor/globalmentor-web/blob/master/xml/src/main/java/com/globalmentor/xml/XMLSerializer.java):

https://github.com/globalmentor/globalmentor-web/blob/master/xml/src/test/java/com/globalmentor/xml/XMLSerializerTest.java

If I decide to add a new feature to my serializer/formatter and I break existing functionality, the regression tests will tell me immediately.

The Eclipse formatter has broken in quite a few releases over the past few years; this suggests to me that perhaps it needs more regression tests.

In any case, above all please stay safe with the current COVID-19 crisis, and I wish everyone the best as we weather this storm for a while.
Comment 12 Michael Tiller CLA 2020-03-20 11:59:27 EDT
I updated to the latest version, and I still have the exact same behavior on my end
Comment 13 Garret Wilson CLA 2020-03-21 11:59:37 EDT
Oh, noooo! This is still broken, and worse than ever.

Manually formatting with `Ctrl+Shift+F` gives me:

```
	protected List<Map.Entry<URI, Object>> loadSourceMetadata(@Nonnull MummyContext context, @Nonnull InputStream inputStream, @Nonnull final String name)
			throws IOException {
```

But then saving with `Ctrl+S` produces:

```
	protected List<Map.Entry<URI, Object>> loadSourceMetadata(@Nonnull MummyContext context, @Nonnull InputStream inputStream,
			@Nonnull final String name)
			throws IOException {
```

But it gets worse. I have this:

```
		final Element templateHeadElement = findHtmlHeadElement(templateDocument) //add a <html><head> if not present
				.orElseThrow(() -> new IllegalArgumentException("Template missing <head> element."));
```


I decide to remove the comment:

```
		final Element templateHeadElement = findHtmlHeadElement(templateDocument)
				.orElseThrow(() -> new IllegalArgumentException("Template missing <head> element."));
```

But as soon as I hit `Ctrl+S`, it changes it to this:

```
		final Element templateHeadElement = findHtmlHeadElement(
				templateDocument)
				.orElseThrow(() -> new IllegalArgumentException("Template missing <head> element."));
```

I see that I filed a similar Bug 559638 back for 2020-03 M1, and it was marked as a duplicate and marked as fixed. This looks like the same sort of thing, but it certainly isn't fixed.

This is totally screwing up my source files, even if I barely modify them! I can't go for weeks, messing up the format for all my files, waiting for some 2020-06 M1 that may or may not solve the problem (and which will introduce all sorts of other bugs, as you already emphasized to me on this ticket).

I'll have to try to downgrade to 2019-12. This is disappointing and frustrating.
Comment 14 Garret Wilson CLA 2020-03-21 12:33:13 EDT
(Whew, thankfully I still had a copy of my 2019-12 workspace.) I can confirm that this bug was not present in 2019-12 R, which means it was probably already introduced before 2020-03 M1, because that is when I filed Bug 559638 for similar behavior.
Comment 15 Mateusz Matela CLA 2020-03-22 16:36:00 EDT
Fixed with https://git.eclipse.org/c/jdt/eclipse.jdt.core.git/commit/?id=8c29a0f2ca923455e4c772743b032a9fdf573af8 (missed the word "Bug" in commit message so it wasn't linked automatically...)

The problem manifests when "Format edited lines" save action is enabled and a line that is wrapped at the end gets modified. The formatter acts like it must add another wrap inside that line.
This was another problem introduced with the fix for bug 250656.
It's kind of weird that the first one - bug 559006 about wrong indentation - was spotted quickly in M1 by multiple people while this one was under the radar until right after the release :(
Comment 16 Mateusz Matela CLA 2020-03-26 05:52:11 EDT
*** Bug 561441 has been marked as a duplicate of this bug. ***
Comment 17 Jay Arthanareeswaran CLA 2020-04-08 00:56:01 EDT
Verified for 4.16 M1 using build I20200407-1800
Comment 18 Mateusz Matela CLA 2020-04-13 06:48:45 EDT
*** Bug 561813 has been marked as a duplicate of this bug. ***
Comment 19 Garret Wilson CLA 2020-04-18 10:19:53 EDT
Good morning. This bug is screwing up code that I edit at work. The bug has supposedly been fixed in 2020-06 (4.16) M1; unfortunately Content Assist is broken in that release; see Bug 562282.

I'd like to downgrade to 2019-12, because that's the last version that didn't have this bug or a related bug. Could someone please give me a hint in how to force Eclipse 2019-12 to use my workspace that 2020-03 upgraded before I realized this bug hadn't been fixed yet? Is there a simple metadata value I can change? Thanks in advance.
Comment 20 Mateusz Matela CLA 2020-05-01 09:10:05 EDT
*** Bug 562604 has been marked as a duplicate of this bug. ***
Comment 21 Garret Wilson CLA 2020-05-28 12:58:24 EDT
Could someone please look at Bug 563487 and make sure it is assigned to the correct person? That bug is a variation of this bug (or maybe it is caused by this bug), but it has received absolutely no response.

I'm just worried that Bug 563487 won't be fixed before the release of Eclipse 2020-06, which means that all the releases since Eclipse 2019-12 will be broken.

I'm trying to make sure the bug has visibility, because Comment 15 here indicated that this similar bug went "under the radar" until after release (although in truth I filed it for M3 before the release). I don't want Bug 563487 to go "under the radar" either or we'll have half a year of broken Eclipse releases forcing us to keep using 2019-12 for three more months.

Thanks in advance for your attention.
Comment 22 Eclipse Genie CLA 2020-07-14 17:25:56 EDT
New Gerrit change created: https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/166297