I'd like to add a short note on needs for file content caching:
Shelby absolutely right that we have AST caching which is pretty enough for all caching needs. However we introduced source caching to support current TCL implementations:
Right now TCL implementation work with 2 different parsers, both reading file content which decrease DLTK significantly in networked environments as Alex mentioned. We may meet same cases if will use different parsers for other languages something like fast structure parser, and detailed AST parser. From other side content caching seems to be useless for locally hosted workspaces...
I think we shall add possiblity for adopters to enable/disable content caching depending on their build/parse structure and environment.
Kind Regards,
Andrey
----- Original Message -----
From: "Alex Panchenko" <alex@xxxxxxxxx>
To: "DLTK Developer Discussions" <dltk-dev@xxxxxxxxxxx>
Sent: Thursday, October 2, 2008 1:26:44 PM GMT +06:00 Almaty, Novosibirsk
Subject: Re: [Dltk-dev] Buffer Syncronization After IFile.setContents()
Shelby,
In the SourceCodeCache class there is inner ChangeListener class to
handle resource change events.
The listener is added to the workspace in the constructor of the
SourceCodeCache.
Would you please test in the debugger if the events are delivered to it
in you setup?
Probably some of the previously registered listeners throws an
exception, so cache listener is not called?
The cache was added to minimize file reads in networked environments,
you can disable it in your release, but I really would like to know why
it does not work as expected in your environment.
Regards,
Alex
Shelby Sanders wrote:
> Alex,
>
> See responses inline.
>
> Thank You,
> Shelby Sanders
>
>
> On Oct 1, 2008, at 01:28AM, Alex Panchenko wrote:
>
>> Hi Shelby,
>>
>> One of the recent changes (last Friday I think) was introduction of the
>> IFile content caching.
>> On change event the modified file is removed from cache.
>> I have just committed the tests in the
>> org.eclipse.dltk.ruby.core.tests.resources.SourceCacheTests class.
>> The test illustrates that after IFile.setContents() and IFile.delete()
>> cache item is removed.
>
> I've updated to the latest sources from CVS HEAD. I assume all of the
> tests are passing for you.
>
> SourceCacheTests.testResourceChange() is failing for me with the
> following trace, which matches the behavior I'm seeing in our tests:
> junit.framework.ComparisonFailure: expected:<class [Class]001
> #
> #
> #
> ...> but was:<class [Resource]001
> #
> #
> #
> ...>
> at junit.framework.Assert.assertEquals(Assert.java:81)
> at junit.framework.Assert.assertEquals(Assert.java:87)
> at
> org.eclipse.dltk.ruby.core.tests.resources.SourceCacheTests.testResourceChange(SourceCacheTests.java:72)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:597)
> at junit.framework.TestCase.runTest(TestCase.java:164)
> at junit.framework.TestCase.runBare(TestCase.java:130)
> at junit.framework.TestResult$1.protect(TestResult.java:106)
> at junit.framework.TestResult.runProtected(TestResult.java:124)
> at junit.framework.TestResult.run(TestResult.java:109)
> at junit.framework.TestCase.run(TestCase.java:120)
> at junit.framework.TestSuite.runTest(TestSuite.java:230)
> at
> org.eclipse.dltk.core.tests.model.SuiteOfTestCases$Suite.runTest(SuiteOfTestCases.java:151)
>
> at junit.framework.TestSuite.run(TestSuite.java:225)
> at
> org.eclipse.dltk.core.tests.model.SuiteOfTestCases$Suite.superRun(SuiteOfTestCases.java:132)
>
> at
> org.eclipse.dltk.core.tests.model.SuiteOfTestCases$Suite$2.protect(SuiteOfTestCases.java:118)
>
> at junit.framework.TestResult.runProtected(TestResult.java:124)
> at
> org.eclipse.dltk.core.tests.model.SuiteOfTestCases$Suite.run(SuiteOfTestCases.java:128)
>
> at junit.framework.TestSuite.runTest(TestSuite.java:230)
> at junit.framework.TestSuite.run(TestSuite.java:225)
> at
> org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:130)
>
> at
> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
>
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460)
>
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)
>
> at
> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)
>
> at
> org.eclipse.pde.internal.junit.runtime.RemotePluginTestRunner.main(RemotePluginTestRunner.java:58)
>
> at
> org.eclipse.pde.internal.junit.runtime.UITestApplication$1.run(UITestApplication.java:122)
>
> at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
> at
> org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:123)
>
> at
> org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:3659)
> at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3296)
> at
> org.eclipse.ui.internal.Workbench.runEventLoop(Workbench.java:2389)
> at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2353)
> at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2219)
> at org.eclipse.ui.internal.Workbench$4.run(Workbench.java:466)
> at
> org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:289)
>
> at
> org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:461)
>
> at
> org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
> at
> org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:106)
>
> at
> org.eclipse.pde.internal.junit.runtime.UITestApplication.start(UITestApplication.java:52)
>
> at
> org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:169)
>
> at
> org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:106)
>
> at
> org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:76)
>
> at
> org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:363)
>
> at
> org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:176)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:508)
> at org.eclipse.equinox.launcher.Main.basicRun(Main.java:447)
> at org.eclipse.equinox.launcher.Main.run(Main.java:1173)
> at org.eclipse.equinox.launcher.Main.main(Main.java:1148)
>
>> Another possible explanation is that you have active IBuffer for the
>> SourceModule, so getSource() returns the buffer contents and
>> modifications of the underlying resource are ignored.
>
> I don't think there should be an active buffer, because there isn't an
> editor open, since this happens in an automated test.
>
> Are there other cases where a buffer would be active?
>
>> You can temporary turn off the caching to check if it is related to your
>> test failures (comment out Map.put or use null instead of Map.get - in
>> the future we are going to make pluggable cache providers for testing
>> purposes, but at the moment the code modifications are necessary).
>
> Yes, if I comment out the put() calls in SourceCodeCache, our tests
> all pass again.
>
> Note, with the put() calls commented, I can get SourceCacheTests to
> pass by commenting out both
> "assertNotNull(cache.getContentsIfCached(file));" lines.
>
>> Could you please provide the code of the failing tests, so we can check
>> if it fails for us?
>
> I can't really provide the actual test code, because it depends on
> quite a bit of our internal infrastructure. However, the general
> sequence of events is:
> In TestCase.setUp():
> Use IFile.setContents() to update the contents of an already
> existing *.rb file
> Call IProject.refreshLocal(IResource.DEPTH_INFINITE, new
> NullProgressMonitor());
> Call
> ResourcesPlugin.getWorkspace().build(IncrementalProjectBuilder.INCREMENTAL_BUILD,
> new NullProgressMonitor());
> Wait for the build to finish, via calls to
> AbstractModelTests.waitForAutoBuild() and
> Job.getJobManager().join(ResourcesPlugin.FAMILY_MANUAL_BUILD, null);
> Wait for the indexer to finish, via a call to
> AbstractModelTests.waitUntilIndexesReady()
> Then in the test() method:
> Retrieve the IFile
> Retrieve the IModelElement for the IFile via
> DLTKCore.create(file)
> Retrieve the AST for the file via
> SourceParserUtil.getModuleDeclaration((ISourceModule)modelElement, null);
> At this point we run some checks on the AST, which fail
> because it is based on the old file contents. I've confirmed this by
> calling ISourceModule.getSource().
>
> As you can see the logic is roughly the same as
> SourceCacheTests.testResourceChange() with some additional logic to
> retrieve and process the AST.
>
> Do you have any suggestions regarding why SourceCacheTests is failing
> for me and/or how to proceed with tracking this down?
>
> What part of the cache removal process is supposed to cause the
> ISourceModule to reload from disk?
>
>> Regards,
>> Alex
>>
>> Shelby Sanders wrote:
>>> All,
>>>
>>> Something changed in the last week which is causing an interesting
>>> issue in our automated unit tests.
>>>
>>> We use IFile.setContents() to update the content of a Ruby file, and
>>> then run various tests which access the ISourceModule for that file.
>>>
>>> This was all happily working and everything stayed in sync, until I
>>> updated our code to use the latest sources from CVS HEAD as of
>>> yesterday.
>>>
>>> Now, ISourceModule.getSource() still reports the old contents of the
>>> file, even after all the events have fired and the index is ready.
>>> Also, calling ISourceModule.makeConsistent() doesn't help, because
>>> ISourceModule.isConsistent() returns true causing it to return without
>>> doing anything.
>>>
>>> I've temporarily worked around the issue by calling
>>> ISourceModule.close() then ISourceModule.open() after changing the
>>> file contents.
>>>
>>> However, I'm guessing this is really just exposing a bigger issue. In
>>> general, shouldn't ISourceModules detect when the underlying IFile
>>> changes outside of the scope of DLTK, and update themselves
>>> accordingly?
>>>
>>> Thank You,
>>> Shelby Sanders
>>> _______________________________________________
>>> dltk-dev mailing list
>>> dltk-dev@xxxxxxxxxxx
>>> https://dev.eclipse.org/mailman/listinfo/dltk-dev
>> _______________________________________________
>> dltk-dev mailing list
>> dltk-dev@xxxxxxxxxxx
>> https://dev.eclipse.org/mailman/listinfo/dltk-dev
>
> _______________________________________________
> dltk-dev mailing list
> dltk-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/dltk-dev
_______________________________________________
dltk-dev mailing list
dltk-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/dltk-dev
