[ve-dev] [Design] Init string parsing

[Rich Kulp <richkulp@xxxxxxxxxx>]

At this time JVE is limited in its support
for complex expressions for property setting or initialization.  The
target VM has an init parser that is responsible to evaluate expressions
from strings.
For example:

#import  com.foo.bar.Style;
#import  com.foo.bar.StyleFactory;
          ...
x.setFoo(new Bar(getJPanel(), StyleFactory.getMyTemplateStyleFlags
|= Style.Horizontal));

The target VM's "init" parser
evaluate this string as is.  This implies: parsing, resolving, and
evaluating the property Foo.   Driving this effort completely on the
target VM is inherently limited, as the target VM has no context to resolve
_expression_ elements (e.g., qualification, getters, msg. receivers etc.)
and it is inherently hard to maintain.

It hould be noted that parsing/resolving/evaluating
is common to property expressions, instance construction (with argument
based constructor), as well as method invocation (e.g., x.SomeMethod(_expression_)).


The parsing/resolving effort is already
done today at the CodeGen level, using the JDT's AST.  The goal of
this effort is to make this information available (from the model), so
that to simplify the total support for evaluating expressions to to set
properties, instantiate bean instances, and introspecting beans properly.
  In the example above, it is expected that at some point, the property
Sheet will denote the "orientation" property to be set to Horizontal.
     


Problem 1 - InitString evaluation
for statements scoped outside a single _expression_
The current JEM model

To construct a target VM live instance
the initString is sent to the target VM and evaluated.  This works
great for expressions like
java.awt.Color.red
new java.awt.Font("Arial","12","Bold");
"Dog"
but not so great for stuff like
display.getSystemColor(SWT.COLOR_RED);
new Button(composite,SWT.LEFT  SWT.CHECK);
bundle.getString("Dog");

This is an example of JCM with a red
button whose size is 115 by 71

<?xml version="1.0"
encoding="ASCII"?>
<org.eclipse.ve.internal.jcm:BeanSubclassComposition
xmi:version="2.0"
xmlns:xmi="http://www.omg.org/XMI"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:ecore="http://www.eclipse.org/emf/2002/Ecore"
    xmlns:java.awt="java:/java.awt"
xmlns:javax.swing="java:/javax.swing"
xmlns:org.eclipse.ve.internal.cdm="http:///org/eclipse/ve/internal/cdm.ecore"
    xmlns:org.eclipse.ve.internal.jcm="http:///org/eclipse/ve/internal/jcm.ecore"
components="//@members.0">
  <diagrams id="org.eclipse.ve.internal.cdm.primarydiagram"
visualInfos="//@annotations.0/@visualInfos.0"/>
  <annotations xsi:type="org.eclipse.ve.internal.cdm:AnnotationEMF"
annotates="//@members.0">
    <keyedValues xsi:type="ecore:EStringToStringMapEntry"
key="org.eclipse.ve.internal.cde.core.nameincomposition"
        value="jButton"/>
    <visualInfos diagram="//@diagrams.0">
      <keyedValues xsi:type="org.eclipse.ve.internal.cdm:KeyedConstraint"
value="42,55,-1,-1"
key="org.eclipse.ve.internal.cdm.model.visualconstraintkey"/>
    </visualInfos>
  </annotations>
  <members xsi:type="javax.swing:JButton"
initializationString="new
javax.swing.JButton()" size="//@methods.0/@properties.0"

      background="">"//@methods.0/@properties.1"/>
  <methods name="getJButton" initializes="//@members.0" return="//@members.0">
    <properties xsi:type="java.awt:Dimension" initializationString="new java.awt.Dimension(115,71)"/>
    <properties xsi:type="java.awt:Color" initializationString="java.awt.Color.red"/>
  </methods>
</org.eclipse.ve.internal.jcm:BeanSubclassComposition>

Right now we partially handle special initString cases for String, Dialog and others but we need a better way to deal with these in a generic fashion.

Also - we do double parsing, because the target VM has to parse the string in order to tokenize it and evaluate it into a valid set of java.lang.reflect operations that can be used to create the Java instance.  This parsing has already been done to a certain level by the IDE VM with the AST model.  We have issues where certain types of expressions cannot be dealt with by the target VM (such as multi dimensional arrays) and to avoid being in the parsing _expression_ business it'd be favorable to let the JDT do the parsing although we still need to evaluation

Another issue with initParsing right now is how it looks up class names.  Right now the target VM has no knowledge of the import statements on the class so it can't deal with unqualified class names, e.g.
new com.mycorp.MyClass(AnotherClass.FIELD, AnotherClass.staticMethod());
The references to AnotherClass result in ClassNotFound because they can't be resolved.  Under some circumstances the initString is manipulated prior to going from the source into the JavaObjectInstance by code parsing (this occurs for unqualified class names with static method calls), but static fields and class names in method arguments are not qualified out in this way.

Solution 1
Change the model so that elements of the initialization string are held in a parsed fashion.  This parsing would be done by taking the AST _expression_ for the initString that contains nodes representing the statement segments.  This then gets visited so that every node that references a field is replaced.  The AST hierarchy is

Object
   Statement
      _expression_
         Reference

Take this structure and turn it into a tree that is not AST but can easily be sent over to the remote JVM to interpret.

This structure needs to replace the initString in the model, and also we need to be able to evaluate it on the target VM to instantiate the Java object

VE model
Because of different ways of instanting Java objects (initStrings which we will keep, XML serialized expressions, the AST tree, implicit objects (that are retrieved rather than created) instead of having separate fields on JavaObjectInstance instead have a delegate strategy called JavaAllocation

(Note: Change ASTConstructor to a generic VEExpression because it won't be AST at this time).

Have to decide whether the tree of expressions is EMF'd or not. If they are EMF'd, they would easy to create by hand, but it would be overhead because they would then need to be converted to a form that can be shipped to remote VM. If not EMF'd, then either they can't be recreated from an XMI document, or we would need a string that is not so readable that could reconstruct the tree. This needs more thought.

The BeanProxyAdapter would delegate all the construction to this object.  Parsing would put the VEExpression there having visited it and replaced the nodes (with JavaObjectInstanceReference)

Property editors already construct the Instance object.  They will get their allocationExpression by asking a factory, e.g.

JavaAllocationFactory.getDefault().getAllocationExpression(String initString);

For jem.core this will return an instance of InitializationStringConstructor.  The evaluation of this will work by using the existing InitializationStringParser class on the target VM

CodeGen will register an ASTAllocation factory.  What this does is when given the initString is parse it using using the InitializationStringParser.

Both InitializationStringAllocation and VEExpression therefore hold the parsed tree of nodes which are able to be transmitted to the target VM.  These will be called the initStringNodes (to distinguish them from AST nodes).  They are basically the same objects in the current org.eclipse.jem.internal.proxy.initParser package.

Serialization of both allocation objects will use the tree of nodes to create an initString.  There is no need to serialize the nodes any other way, as default creation from the XMI to a JavaObject is done by evaulating the string.

 <methods name="getButton" initializes="//@members.0" return="//@members.0">
    <properties xsi:type="java.awt:Dimension">
      <allocationExpression xsi:type="org.eclipse.ve.internal.InitializationStringConstructor" initString="new java.awt.Dimension(115,71)"/>
    </properties>
    <properties xsi:type="java.awt:Color">
      <allocationExpression xsi:type="org.eclipse.ve.internal.VEExpression" initString="java.awt.Color.red"/>
    </properties>
  </methods>


InitParsing
From code parsing bottom up the stages would be
Walk the AST nodes on the target to create a parallel set of our own nodes (the initStringNodes) that are concerned with evaluation, knowing the return type, how to use java.lang.reflect to return objects etc..  
Send the nodes down the wire to the target that decodes them using standard io serialization
Target VM evaluates them.  For each of the JavaObjectInstanceReference nodes the information contained about how to locate the target VM proxied JavaBean would be contained so these references could be properly resolved
Target VM returns the new object, BeanProxy stores it and continues as before

There are scenarios where the property editor doesn't create the IBeanProxyObject for us (such as the IDE property editors like number, String, boolean, Image, etc...) and we rely on being able to set the initString into the JavaObjectInstance and let the initParser evaluate it.  These should be changed whereever possible to also create the object on the target VM and put this into the bean proxy, just to avoid the overhead of having to parse the string and evaluate it when we already have the object right there.  For the primitives + String this should be straightforward because we have the IDE value and there are factory helper classes to create target VM objects.


Step 1:

We will first create just the init string constructor object to get the framework layed out. And then design more the _expression_ form.