| Re: [eclipse-incubator-e4-dev] What I dislike about using EMF for e4... |
Michael,
Comments below.
Ed Merks/Toronto/IBM@IBMCA
mailto: merks@xxxxxxxxxx
905-413-3265 (t/l 313)
eclipse-incubator-e4-dev-bounces@xxxxxxxxxxx wrote on 04/24/2008 09:32:15 PM:
> Ed,
>
> Comments below
>
> Ed Merks wrote:
> > > Ed Merks wrote:
> > > > I would also assert that that EObjects really do feel just like DOM.
> > >
> > > Yes it does. But as Eric pointed out, there are a already DOM-like
> > > interfaces in eclipse:
> > >
> > > org.eclipse.ui.IMemento
> > > org.eclipse.core.runtime.IConfigurationElement
> > > org.eclipse.jface.preference.IPreferenceStore
> > > org.eclipse.jface.dialogs.IDialogSettings
> > > org.eclipse.debug.core.ILaunchConfiguration
> > > org.eclipse.core.databinding.observable.value.IObservableValue
> > > ... (I am sure I forgot some)
> > >
> > > And I would add reflective EMF to that list. Only EMF and
> > IObservableValue
> > > support change propagation.
> >
> > Do you think we have enough yet, or would another one or two more on the
> > list be good? :-P
> >
> > >
> > > I am a great fan of reflective data-structures. And I would love to see
> > > a unification of all the different DOM like data-structures. If we can
> > > adapt all of the data-structures to EObject then we would have a common
> > > reflective API. Unfortunately the EObject API is not as easy to use as
> > > say IMemento. One of the problem is that you have to have a metamodel
> > > in order to create instances. But even IMemento is hard to use.
> >
> > This easy to use thing is something that's all too often like beauty,
> > i.e., it's in the eye of the beholder. As with architecture choices in
> > general, what's unfortunate from one perspective ends up being fortunate
> > from another. If this were not the case, the simple obvious solution
> > would have arrived years ago and the interesting conversations would
> > have moved to a different topic.
>
> We have to distinguish different uses of models. IMemento,
> IConfigurationElement,
> IPreferenceStore, IDialogSettings, ILaunchConfiguration all solve one
> single problem using different syntax: the problem is persistence of
> arbitrary data structures: you as a user define a "ad-hoc" data-structure
> and you can read and write those. There is no meta-model behind the
> data (well except for IConfigurationElement). It is ad-hoc. Like
> the famous hash-maps, JSON etc.
In my twisted way of thinking, there is always a metamodel. Think of it this way. Any of this can be serialized as XML so at worst the metamodel is xsd:anyType. Probably most are more structured than that, but there is a range.
>
> The question is: if EMF would have been available at the time, would
> it have solved the problem? Or, is EMF the solution for the replacement
> of those data-structures? Honestly, I don't know the answer.
EMF did exist way back in those days, though it's grown significantly in capabilities since then. Likely there would have appeared to be good reasons why it would have been hard to use EMF.
>
> If we have to support those in the future, and we decide to use
> EMF, the question is if the EMF reflective API is the right
> way to go. I have my doubts that EMF as it is today with no
> abstraction layer is usable.
I don't know what you mean by no abstraction layer nor. Perhaps you're refering back to the is-a thing, but even if that's the case, EObjects can act as adapters and EStore-based object need not directly carry any data (an perfect example of a facace that reflects what it hides). We's even less clear how the EMF's reflective API relates to these things. I imagine folks could have defined an explicit data model they wanted to persist and EMF would have just persisted it as XML as it does for all data models. It could be some very plastic API, like you get with XML schema's wildcards or something more tightly constrained. The clients themselves would not have written any serialization logic and yet also would not have needed to write anything reflectively. And the skills learned to model something are highly reusable; so few applications don't rely on a data model...
>
>
> > >
> > > When to use reflective style and when java interfaces?
> > > ------------------------------------------------------
> > >
> > > Reflective API is great for:
> > > - serialization
> > > - generic UI
> > > - databinding
> > > - data driven UI
> > > - scripting
> > > - verification ,
> > > - code generation (based on a model)
> > > -...
> > >
> > > Reflective access gets very difficult if it is used in hand-written
> > > code. In those cases interfaces are much better.
> >
> > Things like "very difficult" is just not quantifiable. I agree that
> > reflective is good for generic applications and, in constrast, when you
> > know specifically what you are dealing with, then specific APIs end up
> > being much nicer and more intutive. An important principle is to
> > support both, preferrably without one detracting from the other.
>
> This is exactly my point!
Excellent. After all, Java itself is a good example of how you can get both.
>
> > >
> > >
> > > MagicInterface
> > > --------------
> > >
> > > Some years ago when I figured out that there are so many reflective
> > > interfaces in eclipse I thought, how can I unify them and make all
> > > of them type safe. Reflective interfaces are great for generic
> > > tools like my generic editor (http://tinyurl.com/4rbe5j). However
> > > using reflective interfaces "by hand" is annoying. It is much better
> > > to use real interfaces. But with EMF, you have to create a meta-model
> > > and then generate a whole bunch of code. Which is also inconvenient.
> >
> > It's inconvient to be specific when you don't want to feel bounded by
> > today's choices. Keep in mind that EMF doesn't require you to generate
> > any code for your model.
>
> YES! YES! YES! This is exactly my point. Some years ago I wrote
> a parser foe something like JSON (which knows only maps and lists,
> where some maps had a "type" attribute). The trick was to first
> read the (unknown) data and build up a meta-model and in the second
> pass read the real data.
Marcelo's been playing with this too. The lack of a decent model description in JSON makes it not terribly useful for round tripping arbitrary strongly typed data models. The complete lack of support for non-containment references is also a major limitation. In any case, pretty syntax doesn't impress me; a pretty abstract model, that's what I like to "see". If syntax is all that matters, it's easy enough to populate an instance of AnyType with equivalent data.
>
>
> > That's your choice when you've decided your
> > model is static enough to warrent specific APIs. Those APIs are
> > something the specific clients our your model are generally going to
> > prefer. As you pointed out, generic interfaces are not so fun to use.
> > Otherwise we'd all be ecstatic to have DOM or a hash map and never want
> > for more.
>
> Exactly. And what makes it worse that there are so many different DOMs.
> As I pointed out in my original mail, the openArchitectureWare have created
> a pretty cool DOM abstraction that works with EMF and some other metamodels.
> I think they did it, because when they started they wanted to support
> multiple modeling frameworks. EMF came later. I am not suggesting using this
> model nor my model BUT we should have *one* DOM model and not the many I
> listed in the beginning.
We're kind of stuck with what we have for binary compatibility API reasons. APIs are a one way trip that leaves a lasting trail, barring a healthy dose of no-slip teflon. That is my concern about having yet another one. The pattern I see goes like this. Given that architecture involves tradeoffs, optimal is not at an extreme end of some simple curve. There are saddle points as well as various hills, dips, and valleys. So not only is optimal difficult to find, it's difficult to define. So, a design is ultimately arrived at that makes specific tradeoffs. Then someone new comes along and finds a specific tradeoff they feel is less than optimal. They call this a fatal flaw and it represents the reason for a new expedition to explore the territories to find the top of a higher mountain. They'll likely only find another hill and likely it will be no higher (since height is ill defined), but most certainly it will be in a different location, thereby ensuring that the process will always repeat. Given enough exploration, I can hope folks will see the similarities between the hills.
>
> > >
> > > So, I invented what I called MagicInterface.
> >
> > I feel a slight of hand coming on! :-P
> >
> > > The idea is that you
> > > write the data-structure as interfaces and there is some magic
> > > that binds the interface to the underlying DOM like representation.
> >
> > I wonder if this will be like an EClass whose behavior you can emulate
> > dynamically or for which you can generate an interface and an
> > implementation class?
>
> Since most (all) the underlying implementations do not have a meta-model,
> I do not require a meta-model (see attached file). I have no EMF
> implementation
> of IDataStore, but it should not be too difficult.
I so doubt this claim of a lack of a metamodel. It's like saying there is no coherent structure at all, just chaos.
>
> > >
> > > Let's stick with IMemento. Some code from
> > org.eclipse.ui.internal.FastViewBar:
> > >
> > > public void saveState(IMemento memento) {
> > > memento.putInteger(IWorkbenchConstants.TAG_FAST_VIEW_SIDE,
> > > getSide());
> > >
> > > Iterator iter = viewOrientation.keySet().iterator();
> > > while (iter.hasNext()) {
> > > String next = (String) iter.next();
> > > IMemento orientation = memento
> > > .createChild(IWorkbenchConstants.
> > > TAG_FAST_VIEW_ORIENTATION);
> > >
> > > orientation.putString(IWorkbenchConstants.TAG_VIEW, next);
> > > orientation.putInteger(IWorkbenchConstants.TAG_POSITION,
> > > ((Integer) viewOrientation.get(next)).intValue());
> > > }
> > > }
> > >
> > > With MagicInterface you'd create an ad-hock interface that represents
> > > you data-structure (very much like the EMF annotated java):
> >
> > Or perhaps it's like the demand created meta data describe in
> >
> > http://www.theserverside.com/tt/articles/article.tss?l=BindingXMLJava
> >
> > I.e., just create an "element" or "attributes" to fit into a "wildcard"
> > as the need arises.
>
>
>
> > >
> > > interface FastViewState extends MagicInterface {
> > > //MagicInterface is empty marker interface
> > > int getFastViewSide();
> > > void setFastViewSide(int side);
> > > Orientation[] getOrientation();
> > > Orientation addOrientation();
> > > };
> > > interface Orientation extends MagicInterface {
> > > String getView();
> > > void setView(String view);
> > > int getPosition();
> > > void setPosition(int pos);
> > > };
> > >
> > > With the magic interface and the memento you go to the MagicFactory
> > > (today I'd call it MagigAdapter) and it will create you an
> > > implementation (based on java.lang.reflect.Proxy):
> >
> > I've played with creating implementation classes using Java dynamic
> > proxies on the fly, but at the time they were horribly slow. It's a
> > good way to avoid the static byte code for impl classes though...
>
> Yes, it is OK for those ad-hoc data-structures mostly used for
> persistence but they are *not* a replacement for generated code....
If there's actually no need to specialize the generated code, it is in fact a replacement for the Impl classes. That's the whole point of a dynamic model, i.e., if there is no need for generated classes, why generate them. Our webservices space is filled with code that no human ever needs to read or modify. So an important principle to keep in mind is: if you can use a model to generate it, why can't you use a model to emulate it dynamically. The answer should be of course you can. You should generate it only if you have a need for specific APIs against which to hand write code. My lack of interest in the Java dynamic proxies Impl classes was the poor performance they would have...
>
> That is: you have the interface and some magic that adapts the
> interface to you "ugly" DOM.
I suspect most folks would get turned off by "magic". Obviously we should only trust technology, not magic...
>
> > >
> > > public void saveState(IMemento memento) {
> > > FastViewState state=(FastViewState )MagicFactory.
> > > cast(FastViewState.class, memento);
> > > state.setFastViewSide(getSide());
> > >
> > > Iterator iter = viewOrientation.keySet().iterator();
> > > while (iter.hasNext()) {
> > > String next = (String) iter.next();
> > > Orientation orientation = state.addOrientation();
> > >
> > > orientation.setView) next);
> > > orientation.setPosition( ((Integer) viewOrientation.
> > > get(next)).intValue());
> > > }
> > > }
> > >
> > > The advantage is, that you have real interfaces (refactoring is your
> > friend)
> > > but no overhead in generating a model. Sure, this does not scale, but it
> > > makes your code a bit more readable. Instead of ad-hoc string properties,
> > > it uses ad-hoc interfaces.
> >
> > It sounds much like XML Schema wildcards and then after the fact
> > deducing a more specific type based on the actual properties set. I
> > wonder though at what point you'll have decide what properties are
> > enough? I wonder if the overhead in generating a model something that
> > needs to be avoided in the first place. There are definitely
> > performance benefits to having one.
> >
> > >
> > > Well and if you want to save the state as IDialogSettings, just use
> > > the MagicFactory to convert the dialog settings to magic interfaces.
> > > Any of the DOM mentioned above are all used in the same way. To add
> > > support for a new DOM type, all you have to do is to implement my
> > > special version of a simple interface called IDataStore.
> >
> > It's almost like XMLTypePackage's AnyType, which is the analog of XML
> > Schema's xsd:any. It's just like DOM and you can stick pretty anything
> > into it. It could be data for which there is no schema, or data for
> > which you have well-defined schema...
>
>
>
>
> > >
> > > The implementation is based on naming conventions. I also had a version
> > > where you could give additional hints to the underlying implementation
> > > using final attributes (I did this long before jdk1.5 and annotations
> > existed)
> > > using fluent interface (http://en.wikipedia.org/wiki/Fluent_interface)
> > >
> > > final MagicMember name=new
> > MagicMember().setDefault("default-value");
> > > String getString();
> > >
> > > But my colleagues thought this is too much complexity, so I removed it.
> >
> > Didn't anyone get annoyed by it being Magic? :-P I know people have
> > even found the "E" annoying.
> >
> > >
> > > Summary
> > > -------
> > >
> > > Reflective access to data-structures is great when used by frameworks in
> > > a generic fashion. In hand written code, should use interfaces instead.
> >
> > This assumes that you don't write hand written code that does generic
> > things that applies for all models.
>
> Come on, you know what I mean!
Hey, if you're going to pick holes in the work of others, make sure you don't have any obvious ones yourself. :-P You'd be surprised how many useful things can be done reflectively; actually you shouldn't be, you did all the cool reflective editor stuff!
>
> > So it's not generally true. In the
> > modeling world, people do serialization, search, compare, equality,
> > change recording, and an endless set of cool things using generics.
> > They also do an endless set of things against specific APIs that have
> > full knowledge of the model. Both things are useful and needed.
>
> Absolutely! The same object is either used as specific APIs or reflective
> depending on the context. We are absolutely on the same page. Unfortunately,
> the interfaces like IMemento are great
It certainly looks like something for which a single underlying "plastic" model could be created. One that would implement EObject and would be ammenable to EMF reflection while also supporting this API.
>
>
> > > EMF is great for both. Unfortunately, it is not really simple to be used
> > > in an ad-hoc way (like Magic Interfaces)
> >
> > I'm not sure I agree with that. I firmly believe that simplicity is
> > like beauty.
>
> I agree! Look at IDataStore, it is a abstraction for all the refective
> APIs mentioned in the beginning of my mail. And the MagicInterface in
> one way to turn them into specific APIs.
>
> > It's just not easy to pin down.
>
> That's the reason for this mail. I want to reduce the number of
> reflective APIs in e4....
Note that I don't question your mail's good intent. It's exceedingly important that we discuss these things in detail and that folks raise all their concrete concerns as well as constructive suggestions at the level of detail as you've done.
With regard to this specific issue of reducing the relfective APIs, we can't get rid of ones that exist, so unfortunately we can't reduce anything. At best we can try to unify them and try to avoid creating yet another one.
>
> > > and it is too hard to be used
> > > in in a reflective way, because you have to define a meta-model first.
> >
> > This isn't true either, and it mixes the issue of who is defining the
> > meta model and who is writing the specific or generic code. If you want
> > generated interfaces, you definitely have to define a meta model.
>
> ...and as of now that is the only way to get interfaces with EMF...
As I said, defining an interface is defining a model. Jave is the metamodel and there is a mapping between Java and Ecore.
>
> > In
> > fact, writing an interface is writing the metamodel, so you're pretty
> > much doing both at once.
>
> Yes. The pieces of meta-model are locally defined and as IMemento expresses
> it is up to the client to define the meta-model of his memento. And even
> if you use IMemento in a HashMap style you implicitly define a meta-model.
The way I see IMemento, it's effectively like DOM (as you've pointed out) and what the clients are doing is writing explicit serialization and deserialization code by mapping their data model onto the IMemento data model. The process of doing this is not significantly diffeferent from hand writing code to convert to and from DOM. The point being that the client likely has a data model already and is needing to use a second data model to serialize and deserialize the first...
>
> > Reflective code assumes some type of meta
> > model.
>
> And sometimes there is no *explicit* metamodel. As in the snippet from
> org.eclipse.ui.internal.FastViewBar. When I created the magic interfaces
> for the FastViewBar memento I turned the implicit meta-model into a
> more explicit form. But the created IMemento is exactly the same.
Yes, just a given bit of XML syntax could map to many different abstract structures. And just like there are utilities to deduce a schema given an XML instance...
>
>
> > The more metadata you have, the more constrained the types of
> > things you can do. If those constraints prevent you from doing
> > meaningless or incorrect things then they are good. Making it easy to
> > produce nonsense data is a disserivce. So as always, it's a balance act
> > between extremes. DOM lets you make syntatic nonsense easily. With
> > strongly typed Java it's much more difficult to use reflection to create
> > an instance, but the instance you do create is inherently more structured.
>
> Yes, MagicInterface is one step into this direction. The nice thing
> is that you can use the normal java refactoring tools to change it.
Imagine refactoring tools that raise the level of abstract and what they could accomplish to avoid all the rote hand written code that ties us up today. But that's the Model Driven Development approach and not everyone's bought into that, yet. Here, have a little drink of this kool-aid :-P