Bug 575408 - Closing and reopen a project casue the resets of language provider settings
Summary: Closing and reopen a project casue the resets of language provider settings
Status: NEW
Alias: None
Product: CDT
Classification: Tools
Component: cdt-build-managed (show other bugs)
Version: 10.3.0   Edit
Hardware: PC Linux
: P3 major with 2 votes (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact: Jonah Graham CLA
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-08-14 10:11 EDT by Alessandro Fardin CLA
Modified: 2022-05-04 12:45 EDT (History)
5 users (show)

See Also:


Attachments
Don't delete resources on project close (3.43 KB, patch)
2022-04-25 06:11 EDT, John Moule CLA
no flags Details | Diff
screenshot for comment 21 (50.01 KB, image/png)
2022-04-27 21:17 EDT, Jonah Graham CLA
no flags Details
error log file for opening cdt project with git auto sahre (11.15 KB, text/plain)
2022-05-03 17:52 EDT, Alessandro Fardin CLA
no flags Details
error log file for closing CDT project (415.78 KB, text/plain)
2022-05-03 17:54 EDT, Alessandro Fardin CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Alessandro Fardin CLA 2021-08-14 10:11:57 EDT
The project provider settings are reset to default after closing and reopening the project.

The issue seem arose when project are big:
When project has a small amount of files, all work right, if more and more file will be added, at some point the issue will arise.


STEP TO REPLICATE
1) Download https://www.php.net/distributions/php-7.2.34.tar.gz
1) Create a folder for the project
2) Extract the php sources inside the folder
4) From eclipse : select menu file/new/Makefile Project with existing code
5) Select the created folder as Existing Code Location
6) Select Cross GCC for ToolChain for index settings
7) select both C and C++
8) confirm the project creation.
7) From menu project settings select Preprocessor include paths, macros etc / providers
9)Adds only these 3 providers:
- CDT User Setting

- CDT GCC Build Output 
  and modify in (.*g?cc)|(.*[gc]\+\+)|(clang)

- CDT Cross GCC Build-in Compiler settings
  and modify with relevant cross tool for example:
  /opt/exorintos/5.0/sysroots/x86_64-pokysdk-linux/usr/bin/arm-poky-linux-gnueabi/arm-poky-linux-gnueabi-${COMMAND} ${FLAGS} -E -P -v -dD "${INPUTS}"

10) Click on apply and close.
11) Wait for indexer to finish.
12) Open the file .settings/language.settings.xml and verify that setings correspond the settings.
13) Close the file
14) Close thw project
15) Reopen the project

ACTUAL RESULTS:

Open the file .settings/language.settings.xml, now file has lost the settings and have the default ${COMMAND} ${FLAGS} -E -P -v -dD "${INPUTS}"
From the eclipse UI the settings is seems OK, but closing and reopend the project again, settings will be lost on both file and UI.
The indexer restart using the default (wrong) settings.


EXPTECTED RESULT:

Provider settings shall be maintained after close and reopen projects.



  

All test are done with a vanilla CDT without any other plugin
just download CDT package, extract it and run eclipse.

Ubuntu 20.04
Eclipse IDE for C/C++ Developers

Version: 2021-06 (4.20.0)
Build id: 20210612-2011
Comment 1 Alessandro Fardin CLA 2021-10-13 16:49:59 EDT
Hi all, is there any chance to have this bug fixed ?
For me and my team this is very annoing bug, because we use multi project workspace and we need to close project in which we are not working.

thank you
Comment 2 Alessandro Fardin CLA 2021-12-18 19:18:31 EST
Hi,
Any news about It?
Comment 3 Jonah Graham CLA 2021-12-18 19:44:47 EST
Hi Alessandro, No one is looking at this at the moment as far as I know. You can try to increase the visibility by sending an email to cdt-dev (https://accounts.eclipse.org/mailing-list/cdt-dev) - there is some support available to review a patch on this topic - start with https://wiki.eclipse.org/CDT/contributing
Comment 4 Randall Becker CLA 2022-04-18 19:04:23 EDT
I am experiencing a similar issue with language.settings.xml changing each time ECLIPSE 2021-03 and 2021-09 is restarted. We are a heavy git shop and this is really annoying because the change is specifically a hash value local to developer machines and changes different for everyone everyday.
Comment 5 Alessandro Fardin CLA 2022-04-22 12:08:55 EDT
Yes I know the hashing changing in the file language.settings.xml, but for what I have understood, this is the expected behaviour.

As you I'm usng git from my C/C++ project and this is how I arrange to resolve the problem:

I've a copy of all eclipse project files on a eclipse/projectfile folder:

myCProject
-->folder1
-->folder2
-->ecc
-->eclipse
   -->projectfile
      --> .project
      --> .cproject
      --> .settings
           --> language.settings.xml
           --> org.eclipse.cdt.core.prefs
           --> ....

I get a checkout of my project, then I lauch a script that copy all the eclipse files in the project root folder.
the eclipse files in the root directory are in my .gitignore so that they are not considered by git
Comment 6 Randall Becker CLA 2022-04-22 12:30:08 EDT
Although it may appear to be "expected", if the .language.settings.xml file is not managed by git, the build configurations are not preserved across clones. I understand that it may be that build configurations are not intended to be shared, but this is rather disconcerting and leaves each developer on their own for build purposes. We wrap our builds in Makefiles so that the general build settings are managed independently of ECLIPSE, but is the expectation that build configurations are workspace-specific? They should then be subject to import/export explicitly.
Comment 7 Randall Becker CLA 2022-04-22 14:50:20 EDT
I should point out something critical here. You must store

.settings/org.eclipse.core.resources.prefs

in your git repo. This file provides encodings for ECLIPSE. If this file is not managed, you could get yourself into a situation where content becomes unreadable. For example, your default container may be US-ASCII but a specific file is cp1292. If you do not preserve the encoding info, characters inside the cp1292 file will not be usable. The contents of .settings are crucial.
Comment 8 Alessandro Fardin CLA 2022-04-22 16:02:02 EDT
(In reply to Randall Becker from comment #7)
> I should point out something critical here. You must store
> 
> .settings/org.eclipse.core.resources.prefs
> 
> in your git repo. This file provides encodings for ECLIPSE. If this file is
> not managed, you could get yourself into a situation where content becomes
> unreadable. For example, your default container may be US-ASCII but a
> specific file is cp1292. If you do not preserve the encoding info,
> characters inside the cp1292 file will not be usable. The contents of
> .settings are crucial.

All eclipse projects files:
.project
.cproject
.settings folder with all its files 

are stored in the git repo but not in the project root directory, they are in a dedicated folder.

When devs need a new clone of the project they:
1. clone and check out the project from the repo
2. invoke a make set_Eclipse_project


that make target set_eclipse_project: copy all the eclipse project files (and the .settings folder) from the dedicated folder into the project root folder
so at this point eclipse IDE CDT find its project files and all work as expected.

In this way we shared in the git repo a common eclipse configuration for all the dev teams. avoiding the annoyng problem of git complaining about modified .cproject and .settings/language.settings.xml.


If I need to change the configuration:
1. change whethever I need, 
2. copy all roject files from the root folder into the dedicated folder
3. commit in the repo the dedicated folder
4. the dev checkout from repo
5. run the make set_Eclipse_project

and they have the new eclipse configuration up and running.

the only things is that you need to run the make set_eclipse_project the first time you clone the project and every time the eclipse project is changed.

I hope to give you a clear explanation.

I havn't the file encoding problem, because we work in linux and the default file encoding is UTF8. and as said before we had the .settings/org.eclipse.core.resources.prefs shared in the repo.

let me know if it is more clear now
Comment 9 Randall Becker CLA 2022-04-22 17:06:44 EDT
I understand what you are doing, but I do not agree that this is desirable. It assumes that your team's knowledge of project cloning is maintained forever. It precludes rational headless building using the ECLIPSE project structure without having to adjust the project.

The more I hear about this issue, the more I am convinced it is a design defect in CDT. I'm sorry to be forceful about this issue but the point of git is to make things quickly and easily sharable, comparable, mergeable, and understood without mucking about and maintaining a reconfiguration knowledge across everyone in the team on each clone. CDT appears to go against this.

Items that are supposed to vary from one workstation to another should be separate from those that are sharable. It is not the case with the .language.settings.xml file.

If I had another vote to use on fixing this, I would.
Comment 10 Alessandro Fardin CLA 2022-04-22 17:18:11 EDT
Hi Randal,
I agree with you, this is not a desiderable behaviour, I found this workaround to avoid  git complaining about eclipse modified files.


Also I still have my original problem if i close and reopen the project in eclipe, the .language.settings.xml looses my configuration cross compiler setings as stated in my original comment.

So for sure you have my vote!!!


(In reply to Randall Becker from comment #9)
> I understand what you are doing, but I do not agree that this is desirable.
> It assumes that your team's knowledge of project cloning is maintained
> forever. It precludes rational headless building using the ECLIPSE project
> structure without having to adjust the project.
> 
> The more I hear about this issue, the more I am convinced it is a design
> defect in CDT. I'm sorry to be forceful about this issue but the point of
> git is to make things quickly and easily sharable, comparable, mergeable,
> and understood without mucking about and maintaining a reconfiguration
> knowledge across everyone in the team on each clone. CDT appears to go
> against this.
> 
> Items that are supposed to vary from one workstation to another should be
> separate from those that are sharable. It is not the case with the
> .language.settings.xml file.
> 
> If I had another vote to use on fixing this, I would.
Comment 11 John Moule CLA 2022-04-25 06:11:42 EDT
Created attachment 288469 [details]
Don't delete resources on project close

Hi,
A complete hunch on my part, but could this issue be caused by a problem in:

org.eclipse.cdt.internal.core.build.CBuildConfigurationManager.resourceChanged(IResourceChangeEvent)?

I've noticed that when a project is closed, the project's preferences are removed. Also some scanner info files are deleted. I was coming at this problem due to an issue with CMake build returning "Build not configured correctly" when project was closed and then opened, so not directly related to language settings.

This patch adds a project deleted clause and moves the preference/scanner info delete into that code path. When project is merely closed the delete does not happen.
Comment 12 Torbjörn Svensson CLA 2022-04-25 06:52:13 EDT
Hello John

(In reply to John Moule from comment #11)
> This patch adds a project deleted clause and moves the preference/scanner
> info delete into that code path. When project is merely closed the delete
> does not happen.

Can you please provide this patch as a gerrit instead of attaching it to the bugzilla?
Comment 13 John Moule CLA 2022-04-26 06:14:49 EDT
Hi Torbjörn,

(In reply to Torbjörn Svensson from comment #12)
> Hello John
> 
> (In reply to John Moule from comment #11)
> > This patch adds a project deleted clause and moves the preference/scanner
> > info delete into that code path. When project is merely closed the delete
> > does not happen.
> 
> Can you please provide this patch as a gerrit instead of attaching it to the
> bugzilla?

Yes, although not immediately as I first need to setup a new CDT environment. Does this guide still have the latest setup info?

https://wiki.eclipse.org/Getting_started_with_CDT_development
Comment 14 Jonah Graham CLA 2022-04-26 08:52:55 EDT
(In reply to John Moule from comment #13)
> Yes, although not immediately as I first need to setup a new CDT
> environment. Does this guide still have the latest setup info?
> 
> https://wiki.eclipse.org/Getting_started_with_CDT_development

Yes.
Comment 15 Eclipse Genie CLA 2022-04-27 06:28:30 EDT
New Gerrit change created: https://git.eclipse.org/r/c/cdt/org.eclipse.cdt/+/192946
Comment 16 Jonah Graham CLA 2022-04-27 11:53:40 EDT
I tried to reproducing the problem as described in Comment 0 so I could review https://git.eclipse.org/r/c/cdt/org.eclipse.cdt/+/192946 properly. 

I want to add some observations:

- Running steps in Comment 0 it looks like the issue happens on re-opening the project. Restarting the IDE between closing and re-opening the project does not expose the problem AFAICT (see next point though)
- I cannot reproduce this reliably, sometimes it works as expected.
- I could never reproduce it with a simple Makefile project, only with the php example

It looks like the project description is being cached in memory somewhere which causes the problems. There are two problems:
1) The "old" settings are kept based on some steps
2) The UI accesses the new settings
Comment 17 Randall Becker CLA 2022-04-27 12:32:04 EDT
This happens in my environment (2021-09) when I have multiple build configurations and include at least one GCC build. I think the root here is the attributes being persisted in settings, not just that the overwrite is happening.
Comment 18 Alessandro Fardin CLA 2022-04-27 14:34:18 EDT
I confirm that in our projects we have multiple configurazione.
If It can help I can post my project files.
Comment 19 Jonah Graham CLA 2022-04-27 14:58:56 EDT
On a related note for this step:

> - CDT GCC Build Output 
>   and modify in (.*g?cc)|(.*[gc]\+\+)|(clang)

Starting in CDT 10.7.0 prefixed toolchains should work without this change anymore due to the default value changing. See Bug 579756.
Comment 20 Alessandro Fardin CLA 2022-04-27 18:12:21 EDT
Yes, I see, that's a good solutions, because covers all the compilers and cross compilers variants Invocations.
Thank you

(In reply to Jonah Graham from comment #19)
> On a related note for this step:
> 
> > - CDT GCC Build Output 
> >   and modify in (.*g?cc)|(.*[gc]\+\+)|(clang)
> 
> Starting in CDT 10.7.0 prefixed toolchains should work without this change
> anymore due to the default value changing. See Bug 579756.
Comment 21 Jonah Graham CLA 2022-04-27 21:17:29 EDT
Created attachment 288483 [details]
screenshot for comment 21

@Alessandro Please try something for me if you can. I have found a problem, but not sure it is actually the same problem you have.

Please repeat your test case (either Comment 0, or your own real world use cases) with "Automatically share projects located in a Git repository" turned off (Preferences -> Version Control -> Git -> Projects). You can manually share the project once it is open with Context menu on project -> Team -> Share Project

---

Behind the scenes. What is going on is that the "Automatically share" code triggers some of the project loading code, but at this point the project and its files are not fully refreshed into the workspace. Therefore the code in LanguageSettingsProvidersSerializer.loadLanguageSettings that checks if .settings/language.settings.xml exists actually returns false, causing CDT to recreate it from defaults.

By disabling the autoshare option, the project will be refreshed before the project is loaded into memory and the .settings/language.settings.xml file will be seen.
Comment 22 Jonah Graham CLA 2022-04-27 21:37:24 EDT
Looking around the code there are other areas that can cause a similar race condition even without git. What is weird (to me) is that adding

    storeInPrjArea.refreshLocal(IResource.DEPTH_ZERO, null);

right before checking if the file exists didn't resolve the problem, indicating that the project wasn't properly open yet. In fact if I look at the internal data structures in this case the only file that Eclipse's filesystem says is there is the .project file.
Comment 23 Alessandro Fardin CLA 2022-04-28 04:38:05 EDT
If it still apply I can try to disable git.
But I can remember that when I did my test with the php tarball, I didn't use git.
Comment 24 Jonah Graham CLA 2022-04-28 08:47:45 EDT
(In reply to Alessandro Fardin from comment #23)
> If it still apply I can try to disable git.
> But I can remember that when I did my test with the php tarball, I didn't
> use git.

The disabling of git auto share is just a workaround that I hope addresses the most common case you are hitting. As I mentioned in Comment 22, there are definitely other such race conditions, as I reproduced the php problem once without it being a git project.

Note, once the project is open and you manually share it, all the git functionality in Eclipse works, it is just the automatic sharing that seems to be one of the problems because it runs too early.
Comment 25 Alessandro Fardin CLA 2022-05-03 17:25:05 EDT
(In reply to comment #24)
> (In reply to Alessandro Fardin from comment #23)
> > If it still apply I can try to disable git.
> > But I can remember that when I did my test with the php tarball, I didn't
> > use git.
> 
> The disabling of git auto share is just a workaround that I hope addresses the
> most common case you are hitting. As I mentioned in Comment 22, there are
> definitely other such race conditions, as I reproduced the php problem once
> without it being a git project.
> 
> Note, once the project is open and you manually share it, all the git
> functionality in Eclipse works, it is just the automatic sharing that seems to
> be one of the problems because it runs too early.

I've tried , as suggested to disable the git auto share, but is seem to not resolve the problem, sometimes the settings are resetted to default.

what I see is that with autoshare disabled the project is opened faster and give no errors in the Error log window, but some time the described problem is still there.
The rate at which happened is the same with or without git auto share
Comment 26 Alessandro Fardin CLA 2022-05-03 17:52:21 EDT
Created attachment 288495 [details]
error log file for opening cdt project with git auto sahre

I don't know if it can help to investigate problem, but here you are the error log.
when I disable git autoshare, no errors are logged
Comment 27 Alessandro Fardin CLA 2022-05-03 17:54:06 EDT
Created attachment 288496 [details]
error log file for closing CDT project

each time I close the project with or without git autoshare enabled, I've got these errors on the Error log
Comment 28 Jonah Graham CLA 2022-05-04 12:45:06 EDT
Yes that does help:

	at org.eclipse.cdt.core.model.CoreModel.isNewStyleProject(CoreModel.java:1457)
	at org.eclipse.cdt.ui.newui.ManageConfigSelector.getProjects(ManageConfigSelector.java:104)
	at org.eclipse.cdt.internal.ui.workingsets.HasManagedCdtProjectSelection.hasManagedCdtProjectSelection(HasManagedCdtProjectSelection.java:43)
	at org.eclipse.cdt.internal.ui.workingsets.HasManagedCdtProjectSelection.test(HasManagedCdtProjectSelection.java:33)
	at org.eclipse.core.internal.expressions.Property.test(Property.java:65)

indicates another of those race condition paths. I think that HasManagedCdtProjectSelection is querying the project before it is fully loaded too.