[cme-dev] Loader extensibility and some related architectural issues for

[Stan Sutton <suttons@xxxxxxxxxx>]

Hi All,

I've previously brought up the topic
of making the CME UI more extensible with respect to top-level loaders
that might serve as additions to (or possibly even replacements for) the
current demo loader.  My initial motivation for this was that, acting
in the role of a tool-building CME user, I wanted to add another top-level
loader (one for the Cosmos schema).  I discovered that there was no
way to do this short of hacking the UI code.  That wasn't too difficult,
but it struck me as an obvious limitation that there would be no easy,
nonintrusive way to extend such a significant piece of functionality in
an environment that is intended (and advertised) to support experimentation
and adaptation.

In looking into this issue it also struck
me that the UI tangles together elements that address a number of concerns,
some of which (like those relating to display and interaction) are more
UI-ish, and some of which (like those relating to loaders) are less UI-ish.
 Thus, it seems to me that we might also wish to make loader-related
functionality more separable from the UI just as a way to better separate
concerns in our system.  (This should also contribute to extensibility
of loading functionality.)

With this message I'd like to initiate
an in-depth discussion of how we might proceed to address these issues.
 Below in an outline form are points relating to the motivation, current
situation, some aspects of separation, and recommendations for specific
approaches and steps.  (I'm just trying to be helpful :-) )

Please let me know your questions and
comments!

Thanks and regards,

Stan


Why would we want to support additional
or alternative top-level loaders?

• To allow different principal views of the same artifacts
  • e.g., workspace in terms of JDT resource model instead of core resource
  • To allow multiple concurrent views of the same artifacts
    • e.g., a CIT view (or views) "overlying" some workspace view (may want one or more CIT type spaces in a workspace)
    • To better support staged loading of different kinds of elements into the model
      • e.g., of artifacts, relationships, concerns, constraints, and different kinds of each
      • For loading of artifacts from outside of the workspace
        • may help with integration with non-Eclipse-based systems
        • For loading of concerns, concern-modeling schemas, etc. (i.e., alternatives to "Features")
          • these generally won't depend on an artifact model (or loader) and will be orthogonal (and additional) to such
          
          
          What is the situation in the current UI with respect to loading?
          • Contains parts that serve as loaders (for workspace, projects)
            • Loads from the core resource model (not, for example, the JDT Java model)
            • Constructs the default workspace loader
              • "Preselected" by UI (that is, fixed)
              • Not itself extensible
              • Sets up its own structure in the workspace concern model
              • Contains parts that drive loading (invoke loading of specific artifacts)
                • Loads them into the project concern
                  • Does not build substructure for projects
                  • Structure that appears in our concern model is introduced by our Java loader
                  • Addresses one loader-related extension point ("artifact loaders")
                    • Artifact loaders are added to concern space (not coordinated with other loaders)
                    • They are driven by UI, so (from perspective of UI) they only load files into project concerns (although what they may do internally is not constrained)
                    • As noted, no support for addition of top-level loaders
                    • Supports interactive addition of concerns at the top level
                    
                    
                    If we want to get more analytical, we can identify a number of basic, loader-related functions that the UI combines in various ways:
                    • Loader construction and initialization
                    • Artifact discovery
                    • Loader invocation
                    • Concern model construction (direct and via loaders)
                    • Loader-extension integration
                    We can look at these as functional concerns in the loader concern space.
                    
                    So, separating "loading" from the UI isn't just a matter of separating loaders (although that's part of it).  There's other functionality to consider ...
                    
                    For instance, we could separate the "driving" of loaders:
                    • "Driving" of loaders is the discovery (or stipulation) of artifacts and invocation of loaders to load them
                    • Currently the UI does this when the user pushes the (re)build workspace button
                      • Finds the workspace and projects and directly loads concern-model elements for them
                      • Finds files in projects and invokes loaders on them
                      • How might this be approached more separably and extensibly?
                        • Drivers might be associated with particular loaders (e.g., to load the CIUniverse for a workspace)
                          • May have a default driver for each loader (e.g., replicating something like the current UI loading behavior)
                          • Perhaps allow multiple alternative drivers to be associated with any loader
                          • Drivers might be invoked manually by users or automatically by the UI (e.g., when a button is pushed)
                          • Manual driving by users might be supported through the UI, i.e., enabling the user to invoke specific loaders on specific artifacts (possibly with specific controlling parameters)
                          • Supports extensibility if
                            • Loader extension points can incorporate the provision of associated drivers in addition to loaders (I don't know if this is even possible)
                            • Loader-drivers are given their own extension point
                            Note:  Drivers may also incorporate functionality to create and initialize loaders.  This has to be done somewhere, and it might or might not be combined with drivers.
                            
                            
                            My recommendations on an approach for separating loaders from the UI and promoting loader extensibility:
                            • Support extensibility through the plugin mechanism
                            • Adopt two or three extension points for different categories of loader
                              • E.g. (or i.e.), artifact loaders, top-level loaders, ???
                              • Make all loaders plugins (including the demo loader)
                              • Provide one or more default loader plugins to assure that the UI has something like built-in loading capability that it has now
                              
                              
                              Some steps to take initially:
                              1. Retain the "artifact loaders" extension point (although we should probably think more systematically about how we envision such artifact loaders interacting with other loaders)
                              2. Add an extension point to the UI or CME for top-level loaders
                              • Aside:  The "artifact loaders" extension point is in the CME plugin, not the UI plugin.  This seems okay to me, but putting loader extension points into the UI also seems to makes sense, and I'm not sure which is preferable
                              • Separate the current built-in loader from the core UI and make it a plugin (like user-contributed loaders in general would be)
                                • This would have the effect of simplifying parts of the UI and making them more of an integrating mechanism for loaders as opposed to a means of creating and invoking loaders
                                • Add an extension point (or some such mechanism) to enable the provision of loader drivers
                                • Add support to the UI to enable users to invoke top-level loaders (or loader drivers) selectably
                                1. Move from having button for one loader to having a menu for multiple loaders?
                                I think the above points should all be fairly easy to implement (although I don't know anything about adding to menus in Eclipse).
                                
                                Some steps after that:
                                1. Provide a mechanism whereby users can indicate which artifacts they want to load
                                1. This is an aspect of the "driving" issue, i.e., what drives the loading of particular artifacts by particular loaders
                                2. Perhaps users could be given the option of activating/deactivating artifact loaders for specific loads (e.g., a workspace load with or without loading of Ant artifacts)
                                3. ...