Community
Participate
Working Groups
Support logical resources. The Eclipse Platform supports a strong physical view of projects, files, and folders in the workspace. However, there are many situations where a physical view is not the most salient or useful for many purposes. In some cases, multiple distinct objects happen to be stored in a single file, like an archive. Conversely, in other cases, something that is logically a single object is stored across multiple files. This discrepency between logical and physical creates problems for common operations such as searching, comparing, and versioning, which need to work in the physical realm. Eclipse should support some way of mapping between a logical view and the physical organization of files on disk. [Platform Core, Platform UI]
I've started collecting scenarios relating to this plan item. It's still reflects the fact that in the SDK the problems aren't as visible as when Eclipse is bundled with other plugins (modeling tools, database tools, ...). The document can be found at: http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-vcm-home/docs/online/team3.0/logical-physical.html
*** Bug 41139 has been marked as a duplicate of this bug. ***
Deferred to post-3.0. The Eclipse 3.0 plan will be updated to reflect this.
Because the current resources system is file oriented I haven't found a way to make it work for database or network (passing objects over RMI to a database) use cases. It would be nice if these cases were considered when making the resources system more flexible.
i really like the idea of ressources other than plain files. no matter if you use RMI or s.th. else to transport data (XML) or if you build an object during run-time, you should be able to handle all objects (why isn't it possible with IRessource) as Ressource. Usecase: <> items in the Navigator-View send by a server on demand (=on expand) that are retrieved from a parent/child-relation in a database
Reopening for 3.1 draft plan.
From the proposal: "Naturally another possible solution would be to change IResource to not be analogous to file system resources. However this scope of change is beyond the 3.0 timeframe and would cause, in my opinion, non-justifiable breaking changes to all plugins." Personally I think thats exactly what we need to do. If it wasn't for that fact that one gains much for free using IResource this wouldn't be such an issue. As for breaking all plugins, wouldn't you do just as was done for the Equinox project merge, provide a new API that new plugins can use...
Assigning to Inbox user
Jean-Michel, an interesting use case for this logical-physical work would be PDE and its handling of plug-in files. The plug-in data is spread across the plugin.xml, manifest.mf and build.properties files, while the PDE editor shows a unified view of the plug- in and serializes the data into the right files.
I really hope this "logical resource" feature will support 0 physical files as well (RMI, Database etc backed systems).
Morton, could you expand on that comment? As the problem description states, we had scoped this item to handling mapping from logical to physical resources, if your data model doesn't care about files then this plan item doesn't help you. Maybe what you are looking for is a modeling framework for your data objects? Have you looked at EMF?
It wasn't very clear was it :) No, What I'd like is to unmarry resources from physical files all together. It is mentioned in Comment #4 and Comment #5. Use cases and examples would be many. Any old-fashion client-server system could be an example (with Eclipse being the client). So I'd like to be able to use (and extend) IResource where my physical resource is not a physical file at all (a N -> 0 mapping between logical -> physical files). Now we're already using Eclipse/GEF to edit and modify server "objects" (gotten currently through RMI from an application server. But any protocol will do, that should be up to the resource provider/plugin). Show the objects in our own view and open them into custom GEF editors.. That works in a way fine, but what is missing is a good integration capabiltiy with the Eclipse resource system. I'd like to be able to-do: 1. Use the standard navigator to define "Projects" for server connections (I'm guessing using natures or some other project settings). 2. With 1), get the ability to use all the navigators features like open/close projects, working sets etc. 3. Annotation Markers (IMarker) on server objects. Less important, but still nice to have: 4. Integrate with search (maybe even abstract it so I can run the actual search on the server side). 5. Import/Export integration. I admit I know too little about the resource system in Eclipse to really say whether supporting this is even possible, mostly because we cannot use it. But I know Netbeans does it. And I also know there are several (at least commercial) plugins out there that uses similar techniques as we do, and would benefit greatly from being able to integrate with the Eclipse resource system.
When I made Comment #4 I was trying to use the resource system to notify listeners of changes to my objects when they were saved from an editor. I haven't looked at resources since and don't know much about it but I recall that it was tied to the notion of a physical file. That's obviously a problem for editors that don't edit files. I ended up creating a custom listener interface to my persistence service (sends objects across RMI to be saved in a database) that my plugins use instead of Eclipse's resources system.
"logical" means "not physical". If there is still a required tie to some physical file on disk somewhere then in my mind that's not fully executing the vision of logical resources and is Not Worth Doing. As a use case, SAS has a concept of libnames, which can be either the union of one or more concatenated directories or can be a virtual view created by code out of whole cloth. Inside a libname there are datasets and catalogs. Datasets can be real files or just connections to data sources or can be generated by code. Catalogs are generally real files on some server machine. Inside catalogs are entries, which can have different names and types. For example an entry can be a dataset, or it can be SAS source code. I can write Java code to enumerate, read, write, rename, etc. catalogs, datasets, and catalog entries. What I want to be able to do is have all of these things participate in the Eclipse resources framework. I want to show them in the standard views like Resources and Package Explorer, I want to do checkin/checkout, I want to refactor>rename, I want to cut-n-paste, I want to do drag-n-drop, I want to double-click on something to edit it in the real honest to God IDE Text Editor or Java editor or SAS program editor or whatever with full functionality, I want the entities to participate in refactoring. I want to single step through code that is contained in one of these entities with source in another, I want to set breakpoints, I want to do searches, I want to create working sets, I want to set bookmarks, I want to manipulate entities that are OUTSIDE of any Project just as easily as those inside, I want to create a Project inside of a catalog, I want to use these entities in another Project, I want to do compares, I want to use normal file associations and adaptables to control click behavior and menus, I want to use the normal decoration mechanism, and so forth. I DON'T want to have some kind of synchronized local copy of ANY of this data. I DON'T want to have certain editors work and other editors not work. I DON'T want the user to be able to tell that these resources are different than any other resources they might be using like local files. I DON'T want the user to have to use some special SAS-hacked navigator view that will go out of date as soon as the next Eclipse version is installed.
Getting rid of the association from a resource to a file would also allow to provide things like an email or IM editor, that doesn't store its data in files, probably not even locally.
Where in the plan item's description does the 'vision' of logical resources not include physical resources? Quite the contrary, the vision of this plan item is stated succinctly: "Eclipse should support some way of mapping between a logical view and the physical organization of files on disk". Nevertheless, I've heard from many the need for a data modeling framework that would allow information to be modelled independently from their binary and storage locations supported by all the bells and whistles in the workbench. It is a valid request. However, the resources plug-in doesn't do this and wasn't designed to do this and THIS plan item won't address that issue. BTW, have you looked at the haystack framework based on RDF (http://haystack.lcs.mit.edu/) or maybe even EMF? It has potential but I wonder about the performance? The best approach to getting this technology into Eclipse is by starting a technology project, in the same vein as what Equinox did. I'm not aware of any groups actively working on this tough. For an updated description of the proposed solution to this plan item for 3.1, please have a look at the following two documents: http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-vcm-home/docs/online/team3.1/logical-physical-public.html http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-vcm-home/docs/online/team3.1/logical-physical-adaptables.html
Comment #14 and Comment #15 are very nice descriptions of what I was trying to say rather clumsily. I don't know what the "vision" for this is/was of course, I'm just a poor end user :) But I'm sure I read from reports from EclipseCon 2004 that "logical resources", the way Ed was describing them, was going to come very soon.
As I read it more closely, it looks like the plan description doesn't adequately cover the vision of logical resources. If it's taken literally (i.e., map something to one or more physical files on disk) it wouldn't cover our use cases. Take my word for it that this is NOT what most people think you mean when you say you want to support logical resources. Comment #17 supports this, and a quick search would turn up many other references expressing what the users mean and what they're expecting with this item. I looked over the documents provided. I'm not sure I understand the need for ITraversal given the Visitor pattern that everything else uses but that's a minor thing. IModelContext seems to be a round-about way to have resources that aren't exactly files, but they're defined based on local files anyway (for example fetchContents takes an IFile). IStorage is the kind of thing we need more of. It defines some content with methods to read and write it but doesn't dictate that it's a disk file or part of a disk file. The External file editor uses IStorage but as you know it has some problems. Viewing a remote version of a file from a CVS branch uses IStorage as well. But you can't, say, debug a program that was compiled with an old revision of the source and step through it and set breakpoints in that remote view; for one reason all that support requires an IResource. I read somewhere that Haystack had to jump through all kinds of hoops to support its virtualized resource concept. If Eclipse had real logical/virtual resources I'm sure it would be much easier for them. As far as EMF, I don't think that's applicable here.
I agree that the plan item's title is vague, and open to interpretation. Sorry for that, maybe we should change it? And don't take me wrong, I DO understand the need for a logical model plug-in in the SDK, something that IStorage is a far cry from. If you look closer at the concepts introduced by the resources plug-in, deltas/listeners, builders, properties, and markers, they are generic features of any model. What I am hearing from Eclipse users is that they want help modeling their domain information with a model that provides these features. Then the base Eclipse UI pieces are then built on this ubiquitous model: problems view, navigator, tasks. Now EMF does support change listeners, properties, and such so I wonder why you say it is not applicable? Ignoring some of the performance issues with EMF, one of its design goals is fine grain data integration in Eclipse. If users are writing Eclipse plug-ins and need to model non-resource based data, they should take a look at EMF. I consider the ITraversal closer to the IAdaptable pattern that instead of returning an Object returns a bit more. Also, the context is mostly used to implement advanced features when working with files that are remote. For example, let's say a repository tool has an update command enabled in a logical model view and the user picks update on a class diagram. The traversal for the class diagram would return the file(s) the class diagram is stored in and any other associated files. But what if there is a new version in the repository, that depends on new files that aren't local yet. The update command would update the files listed in the traversal but leave the new files on the server. The local class file could then be invalid given the missing files. The model context allows the model element to construct a traversal that would ensure that an update will retrieve a consistent set of files to ensure that the local model is valid. This can be applied to merging as well. I don't expect all logical models to support this advanced feature, but we have many request this. Of course, we haven't actually implemented this yet, at this point we are bouncing ideas around.
As far as I'm concerned this is an equal rights issue. Resources should be able to reap the rewards of the Eclipse Resource framework, not just files. I know Jean-Michel believes this is not his problem domain, but I for one (and it seems others here) was under the impression that this issue was solving the problem in general. Jean has reduced this problem scope to files only, personally I don't agree with this. I would hate for us to be spending lots of calories fixing the wrong issue. I think there is a general misconception among developers about what 'Support logical resources' would be. As for EMF, isn't this just a workaround to the issue that Eclipse doesn't support logical resources? as far as I'm aware EMF isn't part of the platform nor supported by the default navigators, editors etc..
I'm pretty sure the community is expecting what we're talking about and not just a 1->N / N->1 mapping between physical and logical resources.. A quick look at websites found this quote: "dj_houghton writes: ?Deferred to post-3.0. The Eclipse 3.0 plan will be updated to reflect this.? This was one I was hoping would make it since it would simplify handling of non-physical resources like remote files, database tables, and so on." http://www.eclipsepowered.org/archives/2004/04/15/logical-resources-dropped-from-30/
See also bug 54900. In this case the user is editing a file not in the workspace but because of the strong association with IFile to physical files in the workspace some features like the ruler and code folding and breakpoint display don't work. A bugzilla search turns up many problems with external files that probably spring from this basic limitation. Are these problems covered by this plan item or not? If not, then I suggest closing this plan item and creating two or three new ones with more clearly defined scopes.
Yes and No. The bug in question is specific to how JDT displays annotations in external files. The text infrastructure does support creating text editors to non-file things; and it can have it's own annotation model. The problem is that some want to re-use the IMarker annotation model which is IResource based. I would like to keep this item open to track the overral discussions about logical resource support, but I do agree that the scope of the problem is too large. I've summarized the work we are doing in 3.1 related to logical resources on the following page: http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-vcm-home/docs/online/team3.1/plan-logical.html
Hate to bug people, but is there any progress on this item yet in the 3.1 Milestones?
The resource-mapping APIs are in the org.eclipse.core.resources plugin as of 3.1 M4. We are planning on having CVS using these mappings by 3.1 M5.
Support for ResourceMapping adaptability is available in 3.1 M5. This includes: 1) Object contributions can now be made to ResourceMapping instead of IResource for tools that want to operation on resources 2) CVS is now mapping aware (e.g. CVS operations on Java packages are now shallow) The following document describes this from the standpoint of both a logical model and an object contributor (e.g. repository provider) http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-vcm- home/docs/online/team3.1/logical-physical-mappings.html
I know it's a little close to EclipseCON, but is there any talks, code-camps or similar planed for "logical resources"?
Do you mean "logical resources" in the broader sense or the narrower sense as laid out in the documents linked in comment 23 and comment 26. If you are referring to the former, than there are a couple of tutorials and talks on EMF (Eclipse Modeling Framework) that may touch on this area. If you mean the later, there is nothing formally plan but I will be at EclipseCon and would be happy to talk to anyone who is interested.
I meant the later :) Maybe I'll see you there then
Work on this plan item is completed for 3.1. A follow-on item may be added to the 3.2 plan for future work in this area.
Re-opening to cover ongoing work in R3.2. Summary set to match work item from the R3.2 plan. New text for the item is: The Eclipse Platform supports a strong physical view of projects containing resources (files and folders) in the workspace. However, there are many situations where plug-in developers would prefer to work with a logical model that is appropriate to their domain. Eclipse should ease the task for plug-in developers who want to make logical model-aware plug-ins. To do this, Eclipse should provide more flexible mechanisms for contributing actions on models that do not have a one-to-one mapping to files on the users hard disk. This would, for example, allow a team provider's repository operations to be made available on logical artifacts. In addition, existing views like the navigator and problems view should be generalized to handle logical artifacts and, in general, there should be better control over what is displayed in views and editors based on the logical models that the end user is working on. [Platform Core, UI, Team]
I suggest using URIs (nomen omen right?) to map an IResource to a namespace provider. Plugins then contribute backends to supply the data. Default URIs like platform: could be mapped to the file: provider while other like soap:, dav:, cvs:, etc... would be taken care of by different plugins.
Re comment #32 - This is not in the scope of this plan item. However, your suggestion is exactly right for plan item bug 106176. This particular plan item is for facilitating the integration of logical models with the IResource layer. In many cases you have a logical model that is contained within files, and you need to be able to deal with both the logical model and the file level. One tool may operate on the logical model, but CVS, for example, only knows how to deal with "physical" things like files and directories. Thus a mechanism is needed for mapping between these two layers.
Why shouldn't these two efforts be merged? Sometimes my logical model is in memory or on remote servers as well which is what Bug 106176 covers.
The intent of this item is to make it easier for application writers that want to use logical models to integrate *their* work into Eclipse. It covers such things as making it easier to enable operations that are provided for the (physical) Resources that Eclipse uses to be made available on the model objects that map to those resources. It also covers presentation level issues, such as enabling the problems view to display alternative (logical) descriptions for problem locations rather than the "Resource and line number" indicators that it currently uses. It is important to understand that this work does not include fundamentally redefining what it means to be a Resource. The cost of doing that, both in terms of performance and backwards compatibility issues, and even just the sheer amount of raw work involved, is not something that we believe the community would accept. Please see comment #31 for the current description of the plan item.
With that Comment (Comment 35), it sounds like you're saying that Bug 106176 will never be done because it is does want to redefine what Resources are and that is too dangerous, slow and costly to do. Not trying to be rude, just trying to understand what we can expect of support and what to plan to work around.
The point I was trying to make was that, from the point of view of Eclipse, resources still represent "files" and "folders", not objects in the application's logical domain, and that the common currency of Eclipse is still that notion of resource. This bug is talking about enabling logical models built on top of those resources to be well integrated with Eclipse. Bug 106176 is talking about making it possible to provide a different transport layer, below the level of the resource (i.e. it's a virtual file system model). If a particular transport layer has significantly different performance characteristics, then there is most definitely potential for performance problems. We need to understand how sensitive various Eclipse features are to this, and do what we can to alleviate them. The trick of course is that, if you happen to be based on a real file system, then you will perform like a real file system, but in addition, you can potentially cover use cases that you might not have otherwise (assuming that the performance is acceptable for that particular use). There are also side-benefits, like allowing for richer mappings between Eclipse's notion of resources and the *actual* shape of the folders and files. If that's closer to what you were talking about, then I apologize for misunderstanding you. (Also, John was right: 106176 is the place to have that discussion.) In any case, I hope you see that the answer to your earlier question, is that the reason why these are being treated separately is that there really is a separation of concerns here: This bug is talking about how particular model implementors can map their model object to resources and integrate into the various Eclipse mechanisms. Bug 106176 is talking about how the resources in turn map to the underlying file system (or whatever <g>).
In that case, these two bugs should be renamed and restated (again). I've read everything here and in the other bug and I'm still not clear on the distinction. How's this: bug 37723: "Provide a way to map logical objects to files" bug 106176: "Provide a way to read and write files from different places other than the file system (for example ftp)" Neither of these is addressing what I think is the *real* problem, which is a superset of these and other problems. Is there a bug for: "Base Eclipse completely on logical model objects independent of projects and file systems." Description: A logical model object could be a project, or it could be a source file, class in a source file, reference to a remote database, column in a database, an article in an RSS feed, an Action on a Menu, POJOs, or anything at all you want it to be. Logical model objects should be the fundamental modeling structure displayed by the project explorer and other views. They should be compatible with or extend or replace EMF model objects. This would replace or extend the resource concept in Eclipse 3.1. See also the part about Nodes in http://www.eclipsezone.com/java/forums/t44963.html . Every time we think we have a bugzilla entry that means that, we find out it means something else :). Should we enter a new one?
Ed, you are correct that this bug is not about what you propose. You are welcome to enter a different bug report, and we can debate its merits there without further cluttering this bug.
I know I'm not supposed to "clutter" this bug report anymore, but what Ed was referring to is exactly why I suggested to merge this with Bug 106176. Fixing this for "real" by having an abstraction like Netbeans have with their Nodes (and IntelliJ for that matter) solves all these patch bugs to the Eclipse resource system once and for all because it wouldn't put any restrictions on mapping or location of resources.
I'll make one last attempt at explaining this, although after quickly reviewing the history of this bug I see most of this debate has happened before. Let's say we had some abstract model. Some tools, such as Ant and CVS, ONLY know about physical files on disk. They will NEVER understand an Eclipse-specified abstract model. Thus, a mechanism is sometimes needed to translate the abstract model into physical files. That is what this bug report is about. This problem is completely unrelated to whether or not Eclipse has some common abstract model. If you think Eclipse needs another abstract model separate from EMF, please enter a different bug report.
Ok, Please see/cc/vote for bug 109194.
Here is a link to a description of the code that is available in 3.2 M3 that supports model based merging of files and projects. The bug also summarizes what we would like to accomplish in the rest of 3.2. Interested parties should have a look, try it out and provide feedback in this bug. http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-vcm- home/docs/online/team3.2/logical-physical-mappings.html
Here's a shorter link to the same document http://dev.eclipse.org/viewcvs/index.cgi/platform-vcm- home/docs/online/team3.2/logical-physical-mappings.html
Editor opening is another issue we need to support. For one scenario, see bug 42643 comment 3.
Bookmarks (or in general IMarkers) in logical resources should also be considered. See bug 22284 comment 6.
The document at the following URL has been updated to summarize the state of the Team API as of 3.2 M4. http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-vcm-home/docs/online/team3.2/logical-physical-mappings.html Interested parties should read the document. Changes since the previous versions are summarized at the bottom of the document.
With regards to comment 46, there is currently no plan to generalize markers. However, there is a new Eclipse File System (EFS) API in 3.2 that would allow the contents of zips and jars to be treated as IResource and hence targets for IMarkers. I do not know what plans JDT has is this area (or whether it is even practical). You may want to ask on the JDT newsgroup or log a bug against JDT/Core.
In the doc "team support for local model integration", you say <i>"Model objects must implement the IAdaptable interface". But in fact, I want to keep my model independent of Eclipse classes, as I want to use that very same model e.g. in batch jobs. In the latter case, the model is filled through an other dataprovider than the eclipse workspace. How can I make this completely eclipse-independent model available in this framework ?
The example model we are using does not implement IAdaptable and all the new support we are adding does not require that the model elements implement IAdaptable. I will update the document to reflect this. The problem is that there is exisiting funtionality in the workbench that does require model elements to implement IAdaptable and we do not have time in 3.2 to fix all these places. So, although it will be possible for you to use the new features we are adding if your model objects do not implement IAdapatable, there may be existing features that do not work.
Our current requirements are to be able to register actions on jar entries (xml files), and place our own markers on these files (and line-numbers) contained in the jars. I'm reading comment #48 to mean that this may become available in 3.2, while I may be wrong as there's lot of information on this page (and I just added some clutter myself). I'd be interested in testing this out to make sure it will work for us if and when it becomes available in one of the future milestone builds.
Re comment #51: Rather than generalizing IMarker to allow them to be added elsewhere, we have generalized IResource to allow resources to be created in an arbitrary backing storage system. This makes it theoreticaly possible, for example, to create an IResource for a single entry within a JAR file. While the basic infrastructure to make this possible will be in place, specific support for IResource in zip/jar will not be in 3.2. Another option is to use the approach employed by breakpoints, which can be created on files in zips/jars. To do this, I believe breakpoints are added to the IFile for the JAR file itself, and a property is added to the marker indicating what entry in the JAR it corresponds to. See also bug 22284.
Support for logical model integration has been added to the Platform in 3.2. The doc for this item is still forthcoming. Here are links to the latest documentation (as of M6) http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-vcm-home/docs/online/team3.2/repoRoadMap.html http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/platform-vcm-home/docs/online/team3.2/modelRoadMap.html
