Re: [vtp-dev] Questions, directions, techie stuff

[nino martinez wael <nino.martinez.wael@xxxxxxxxx>]

Ok thanks for the hints.. And thanks for fixing the svn link.. Now if
I could only get some time todo the stuff :)

regards Nino

2009/4/21 Trip Gilman <trip@xxxxxxxxxxxxxxx>:
> Nino,
>
>    Absolutely.  Take a look at the exporter code.  That would probably be
> the best spot to start with.  Extracting the design XML document from the
> visual editor would probably not be very fun, but certainly doable.
>
> On another note, I have fixed the SVN link on the VTP web site.  Let me know
> if you have any problems with it.
>
> Trip Gilman
>
>
> On 4/17/09 2:59 PM, "nino martinez wael" <nino.martinez.wael@xxxxxxxxx>
> wrote:
>
>> Thanks Trip for the enlightenment. Very good answer. I like the idea
>> with VTP being so visual, but I also like the idea of having a
>> modifiable framework, in my world it would be best if it spewed out
>> some java, and keept it in synch with the visual layout, thus making
>> it a strong framework for developing java in and extending and not
>> "just" a UI, do not misunderstand this. However Im probably not the
>> common user for this project, so I understand where you have ended
>> up.. And I thing it's very good that you have taken steps to ease
>> integration with version control, something that commercial providers
>> does not seem to do.
>>
>> If I may, i'd like to borrow the visual interface and write my own
>> runtime engine behind it instead (I cant remember how liberal the
>> eclipse license are, probably Apache compatible). At this stage I've
>> just begun wondering howto set things up. And this forum are probably
>> not the right place to discuss further on the topic..
>>
>> regards Nino
>>
>> 2009/4/16 Trip Gilman <trip@xxxxxxxxxxxxxxx>:
>>> Nino,
>>>
>>>     You are absolutely right that there are drawbacks with the single file
>>> storage approach.  I'll try to outline why we decided to go that direction,
>>> as our original design actually used multiple files.  Most of the issues
>>> we¹ve seen when multiple developers make changes to an application can be
>>> attributed to two primary causes: the information is stored in XML format
>>> and there is a separation between the storage format and the medium familiar
>>> to the developer.  Let¹s take a look a closer look at these problems.
>>>
>>> The XML format is fabulous for certain use cases.  Storing highly structured
>>> data that contains large amounts of text that needs to be accessible on
>>> multiple platforms and manageable by humans is one of them.  And this is
>>> exactly the type of data that makes up our application definitions.  The
>>> problem is that is gives the diff engine of most source control systems
>>> fits.  Nearly by definition, all data markers in XML are variable length in
>>> both individual composition (attributes) or number and order (sub elements).
>>>  In my experience the most usual culprits of incorrect difference
>>> calculation are elements with multiple attributes, especially when the
>>> number or which attributes are dynamic, and the end tags of elements with
>>> children.  To help avoid the havoc caused by these instances, the
>>> application document generated by the VTP follows some strict rules:
>>>
>>> XML indentation is fully enforced on all elements and levels of nesting
>>> Start tag pre-ambles are always placed on their own line
>>> Element attributes are always placed on their own line
>>> Element attributes always occur in the same order regardless of presence
>>> Element end tags are always placed on their own line
>>> Element collections are always written in the same order they were read or
>>> created unless there is an overriding ordering mechanism
>>>
>>> However, even with all these rules in place, there are still instances of
>>> conflict when merging changes from independent sources.  This brings us to
>>> the other basic driver; the developer is not familiar with the XML format.
>>>  Our development environment presents the developer with a 100% graphical
>>> representation of all information present in the XML document.  This goes so
>>> far as the editor doesn¹t even bother with the underlying file until a
>>> change needs to be committed to storage.  Often times, the developer doesn¹t
>>> even have experience with XML or other central technologies employed by the
>>> VTP let alone a working knowledge of the schema and nuances of our specific
>>> XML language.  Eclipse is great in that it happily displays the head to head
>>> conflict resolution UI.  Given the target audience, this can result in even
>>> further data loss depending on their tendency to click buttons.
>>>
>>> I¹ve now done something I hate to do, describe an array of problems without
>>> really offering any solutions.  I¹m not sure if there are any ³good and
>>> easy² ones.  OK, back to why we migrated to single file over multiple.  The
>>> main reason was that using multiple files to store the data opened our
>>> project structure up to a possible inconsistent state, depending on user
>>> behavior.  In a traditional java project, having many different files with
>>> changes in them and various states of committal to the source repository is
>>> normal and often doesn¹t even result in an inconsistent state.  However, our
>>> applications are very integrated and represent a continuous flow of logic
>>> from the single entry point to its final states.  There are very few edit
>>> oriented operations that could be performed that wouldn¹t leave the entire
>>> project in an inconsistent state if all documents weren¹t checked in.
>>>  Although this type of situation is fairly common for those of us who are
>>> familiar with the standard software development lifecycle, it can be quite a
>>> learning curve for much of the target audience.
>>>
>>> Another reason we decided to perform the migration was all the other issues
>>> with XML were still present in the multiple file approach, so we decided to
>>> merge all the files into a single source.  To be honest, I¹m not entirely
>>> sold on this approach, but it solved more issues than it presented at the
>>> time.  Choices like these really resonate with what Kent Beck discussed in
>>> his Design is an Island piece on Planet Eclipse.  I would love to enable a
>>> greater liberty in changes being made by multiple developers.  I would like
>>> to mention that with a moderate amount of discipline, multiple developers
>>> can still edit the same project.  If the developers remain within say a
>>> single canvas or don¹t modify the same canvas, the likelihood of a conflict
>>> is greatly reduced thanks to the efforts I mentioned above.  Also, there is
>>> the ability to separate your application into several independent projects
>>> as application fragments.  These can obviously be edited without fear of
>>> collision.
>>>
>>> Ultimately, I think the only real solution is to provide a completely custom
>>> conflict resolution UI that allows the application designer to merge changes
>>> using the medium they are used to.  Showing the changes in a head-to-head
>>> visual display and simple point and click change acceptance.  This would
>>> almost be a project in and of itself and consequently has been a lower
>>> priority than other more easily reachable fruit.  I¹ve been recently playing
>>> around with a couple of ideas on how to make the XML rules we follow more
>>> effective.  My favorite one so far is to sort of watermark the end element
>>> tag so as to identify it uniquely even within a list of similar elements.
>>>  For example, most of our elements represent objects that have an
>>> identifying attribute like Œid¹.  The line containing the end tag would have
>>> an XML comment added just after the tag content that contained the
>>> identifier.  This would allow the line based diff engines to more easily
>>> match groups of lines together in cohesive segments.  I haven¹t fully worked
>>> out the details yet.
>>>
>>> Trip Gilman
>>>
>>>
>>> On 4/14/09 10:57 AM, "Nino Martinez" <nino.martinez.wael@xxxxxxxxx> wrote:
>>>
>>>> Hi Trip
>>>>
>>>> Now that you mention a thing about the runtime platform, as I see it the
>>>> interactions you describe in the visual layout all goes into one xml
>>>> container? Im wondering why it's done this way? Nortel's SCE, which im
>>>> heading into using everyday, also has this approach.
>>>>
>>>> Making it impossible to be more than one dev on projects. Since you
>>>> overlap changes in the xml file.. Or am I wrong about VTP (I hope so)..
>>>>
>>>> If it instead could be contained in some code generation or something
>>>> like it, maybe taking advantage of java7's scripting possibilities (when
>>>> it comes) or something else to leverage this problem.. However this
>>>> would probably impose a big code change..
>>>>
>>>> regards Nino
>>>>
>>>>
>>>> On Sun, 2009-04-12 at 19:27 -0500, Trip Gilman wrote:
>>>>> David
>>>>>
>>>>> First I'd like to thank you for your interest in the VTP and welcome you
>>>>> back to the industry.  I've provided a broad overview of where we are now
>>>>> as
>>>>> a project and also weaved some of my detailed thoughts on your questions
>>>>> into your original text below.
>>>>>
>>>>> You are correct in that the current VTP is a complete departure from the
>>>>> codebase originally contributed by IBM.  I would like to take a moment to
>>>>> point out some of the fundamental changes between the original vision and
>>>>> our current direction and then discuss the reasoning for our approach.
>>>>>
>>>>> First, the xml editors for GRXML grammars have been replaced with the
>>>>> generic xml editor provided by the basic eclipse development framework.
>>>>>  The
>>>>> original VTP utilized a customized xml editor based on those provided by
>>>>> the
>>>>> Web Tools Project.  The removal of this dependency reduced the overall
>>>>> size
>>>>> of our software by nearly 100M and eliminated several layers of
>>>>> complexity
>>>>> for no real loss of functionality.
>>>>>
>>>>> Second, and probably the most centrally important, the VTP employs a
>>>>> common
>>>>> runtime architecture as opposed to code generation.  I don't want to
>>>>> focus
>>>>> on the choice between the two, but I would like to illustrate how we
>>>>> leverage our choice of a runtime architecture.
>>>>>
>>>>> The voice tools project has almost become a misnomer.  Our development
>>>>> and
>>>>> runtime frameworks are no longer just for VXML, but have evolved into a
>>>>> generic platform for describing interactions.  The core of the
>>>>> development
>>>>> environment is blissfully unaware that the user intends to use the
>>>>> application they are creating as the source of a VXML interaction.  The
>>>>> runtime environment executes the interaction within a process engine that
>>>>> emits basic interactive commands that are interpreted into VXML at the
>>>>> final
>>>>> moment.  This is important to understand as this philosophy is pervasive
>>>>> throughout the codebase.
>>>>>
>>>>> The user creates applications by adding Actions to a canvas and then
>>>>> defining the transitions between those actions.  The most common Actions,
>>>>> such as a Menu or a Prompt(Output), are standard components provided by
>>>>> the
>>>>> framework.  However, third-party developers can easily and quickly create
>>>>> their own action libraries for users to use.  I want to reiterate that
>>>>> although these actions may share the names of the elements that make up
>>>>> VXML, they are not the same.
>>>>>
>>>>> The designer has an abstraction framework that models the nuances of
>>>>> interaction types on these actions, allowing interaction specific
>>>>> configuration.  For example, the VTP comes with an implementation of the
>>>>> Voice (VXML) interaction type.  This provides interaction specific UIs to
>>>>> manage VXML settings such as barge-in and dtmf/speech entry.
>>>>>
>>>>> This framework has been used to create other interaction types such as
>>>>> Instant Messaging.  An application can support any combination of
>>>>> interaction types the user has available to them.  This is very powerful
>>>>> in
>>>>> that the same application deployed once, could drive the interactions of
>>>>> both telephone and IM based customers.
>>>>>
>>>>> On the runtime side, things are equally abstracted.  As I mentioned
>>>>> earlier,
>>>>> the artifacts produced by our designer (upon export) are actually process
>>>>> meta-language documents.  These documents are executed by the process
>>>>> engine
>>>>> at the core of the runtime.  As the process is executed, interactive
>>>>> commands are generated.  These commands are then filtered through a
>>>>> platform
>>>>> abstraction layer and evaluated.  The VTP includes a platform layer
>>>>> designed
>>>>> for VXML interactions that is capable of transforming the interactive
>>>>> commands into their representative VXML documents.
>>>>>
>>>>> The platform abstraction layer is powerful on two fronts.  First, it
>>>>> allows
>>>>> the VXML generation process to be customized to the specific VXML Browser
>>>>> platform requesting the document.  This means that a single application
>>>>> instance can service multiple VXML platforms simultaneously without
>>>>> re-export or compilation.  The VXML session could even bounce back and
>>>>> forth
>>>>> between the platforms without issue.
>>>>>
>>>>> Second, the platform abstraction is open to third-parties to add
>>>>> implementations for additional platform types and output mechanisms.  The
>>>>> VXML platform generates XML documents and processes requests and
>>>>> responses
>>>>> over HTTP.  Other implementations could simply execute direct API calls
>>>>> to
>>>>> other software or hardware, never producing a document artifact.  They
>>>>> could
>>>>> also support transports other than HTTP.
>>>>>
>>>>> I hope this helps illuminate the project for you.
>>>>>
>>>>> Trip Gilman
>>>>>
>>>>>
>>>>> On 4/10/09 8:33 PM, "David Reich" <David.Reich@xxxxxxxxxxx> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> First, let me (re)introduce myself.  David Reich, formerly of IBM.  I
>>>>>> was
>>>>>> involved in the creation of VoiceXML and was the guy who was the initial
>>>>>> driver behind all of the Eclipse-based voice tooling, first from IBM's
>>>>>> toolkit, and it's subsequent donation to Eclipse and VTP.  I've since
>>>>>> left
>>>>>> IBM
>>>>>> and as you know, IBM's involvement in voice has waned.  But, I am now
>>>>>> with a
>>>>>> new company, happily back in VoiceXML, CCXML and SRGS, and we are doing
>>>>>> a
>>>>>> lot
>>>>>> with VoiceXML and CCXML and I was brought in to drive our development
>>>>>> team,
>>>>>> and one of the areas I am looking at is the development of our
>>>>>> application
>>>>>> artifacts, and VoiceXML application generation.
>>>>>>
>>>>>> With that, I've spen the last couple of days figuring out where things
>>>>>> are
>>>>>> with this project, how we can use it, how we might give back to it and
>>>>>> so
>>>>>> on.
>>>>>> From what I've been able to see thus far, it looks like the current code
>>>>>> base
>>>>>> bears little to no resemblance to the VTP 1.0 code that came in from my
>>>>>> former
>>>>>> team, but, on the other hand, there is a nicely-coming-along visual
>>>>>> editor
>>>>>> without which all VTP would be is the XML editor with DTDs plugged in
>>>>>> and
>>>>>> while nice, not really compelling.
>>>>>>
>>>>>> I was hoping to provoke some discussion and see about some combination
>>>>>> of
>>>>>> code
>>>>>> to create an even more gener(ic)ally useful environment.  With that,
>>>>>> some
>>>>>> thoughts and the hope we can put some other things together.  In no
>>>>>> specific
>>>>>> order:
>>>>>>
>>>>>>
>>>>>>  *   The original IBM code seems to be gone.  When things moved to SVN,
>>>>>> the
>>>>>> CVS repository went away, so unless someone has that source code, it's
>>>>>> gone
>>>>>> forever (I've checked with the (remnants of) the IBM team, and without
>>>>>> some
>>>>>> serious effort, it's not readily available.  So, how can we get that
>>>>>> back,
>>>>>> even if we only want to take small bits (or even none?) of it?  There's
>>>>>> got
>>>>>> to
>>>>>> be useful things there, and we should still look at that.
>>>>>
>>>>> The original IBM source still exists, however it is not readily available
>>>>> as
>>>>> it has been archived from CVS by the webmasters to reclaim space and keep
>>>>> the repositories current.  If you would like a copy of the archive, I
>>>>> would
>>>>> be more than happy to make the request for you.
>>>>>
>>>>>>  *   In the original tools, of which the VTP 1.0 code was a part, the
>>>>>> vision
>>>>>> was a call flow builder similar to what WebMethods has been working on,
>>>>>> but
>>>>>> rather than generate a metalanguage, have it generate VoiceXML and have
>>>>>> a
>>>>>> tabbed view (kinda like FrontPage) where you see the call flow view, and
>>>>>> the
>>>>>> VoiceXML source view, and the editor would allow you to tweak the
>>>>>> VoiceXML
>>>>>> in
>>>>>> source edit mode, affording finer-grained control of the VoiceXML.
>>>>>
>>>>> Very early on we abandoned code generation in favor of a runtime
>>>>> architecture to overcome several challenges we were faced with when
>>>>> developing applications in the field.  Each time we encountered a new
>>>>> VXML
>>>>> platform, there were always a myriad of minute differences or interesting
>>>>> interpretations of the specification.  We considered just coding these
>>>>> changes into the code generation piece until we ran into a client that
>>>>> had
>>>>> multiple platforms in service.  The solution was to export the
>>>>> application
>>>>> twice, once for each platform.  This instance was one of the driving
>>>>> forces
>>>>> in the transition to a runtime architecture.
>>>>>
>>>>> Another driving factor of the move, was the realization that we were
>>>>> already
>>>>> building a runtime, just not officially and not effectively.  We had
>>>>> developed a loose library of utility functions and were using them over
>>>>> and
>>>>> over again in all our applications.  More and more functionality was
>>>>> being
>>>>> encoded into this framework in an effort to hasten development.  Making
>>>>> the
>>>>> runtime a first class citizen in the architecture allowed us to leverage
>>>>> the
>>>>> advances we had made while removing most of the hurdled we were facing.
>>>>>
>>>>>>  *   Linkages to the Eclipse framework for task-list items in
>>>>>> validation,
>>>>>> warnings, errors, etc.
>>>>>
>>>>> Absolutely, integration to the tagging systems in Eclipse is currently on
>>>>> the wish list.
>>>>>
>>>>>>  *   Enabling (as Randy has told me is the direction) a derivative or
>>>>>> extension plugin to generate code that spews forth VoiceXML.  This could
>>>>>> take
>>>>>> the form of the OpenMethods servlet, or even perhaps an extension one
>>>>>> could
>>>>>> write
>>>>>>
>>>>>> to do JSPs, or other types of markup/tag/metalanguage since not everyone
>>>>>> will
>>>>>> want just the VoiceXML from the OpenMethods servlet, or wants to make
>>>>>> further
>>>>>> tweaks to the VoiceXML.
>>>>>
>>>>> I won't say that it would be impossible to generate the resulting VXML
>>>>> from
>>>>> an application statically, but at some point you'd end up recreating the
>>>>> logic inherent in the runtime in JSP or some other language.  In the case
>>>>> of
>>>>> multi-modal applications, simply duplicate the application several times
>>>>> over.
>>>>>
>>>>> On a side note, the modification of generated code after export can be a
>>>>> very tempting practice as many problems can be solved quickly this way.
>>>>> However, I never recommend the use of this.  It places a process
>>>>> requirement
>>>>> on the development and deployment staff to always re-apply the changes to
>>>>> the code each time there is an update.  What happens if changes made to
>>>>> the
>>>>> code require the workaround to be updated?  This can easily balloon into
>>>>> a
>>>>> maintenance nightmare.  I would say that it should be the goal of any
>>>>> system, whether code generation or runtime, to avoid the need of this at
>>>>> all
>>>>> costs.
>>>>>
>>>>>>  *   A broader property sheet for each call element in the palette on
>>>>>> the
>>>>>> canvas to specify items such as barge-in, SSML tags or information,
>>>>>> prompt
>>>>>> audio files to be played and so on. Specifically, the block would play
>>>>>> one
>>>>>> of
>>>>>> a set of prompts based on current state (such as "Welcome to..."  or
>>>>>> "Welcome
>>>>>> back to..." based on some global variables in the ECMAscript for
>>>>>> example.
>>>>>> This tool should should hopefully allow for the export of this property
>>>>>> sheet
>>>>>> data in different formats, but then again, as open source, we can work
>>>>>> together on extensions for our specific needs and write the hooks and
>>>>>> extension code for different needs.
>>>>>
>>>>> The designer already supports this type of configuration in several ways.
>>>>> At a base level, the developer can use Javascript variables to indicate
>>>>> prompt content.  SSML can also be included in prompt content simply as
>>>>> text.
>>>>> At a higher level, the designer employs a system called branding to allow
>>>>> very detailed configuration for different user segments.  Branding can be
>>>>> used for personalization, line of business differentiation, or even
>>>>> multi-tenanting of an application.
>>>>>
>>>>> On the export aspect of this, I assume you intend to use the artifacts
>>>>> within a static JSP/VXML document.
>>>>>
>>>>>>  *   Have the palette be extensible where one can take prebuilt modules
>>>>>> (say
>>>>>> an authentication subflow) add them to the palette, and drop it into a
>>>>>> new
>>>>>> or
>>>>>> existing application.
>>>>>
>>>>> The designer is fully extensible.  Third-parties can easily publish
>>>>> additional modules for use.  Developers are also able to package up
>>>>> common
>>>>> call flow elements into reusable components called dialogs or even
>>>>> significant portions of a call flow into a fragment.
>>>>>
>>>>>> That's enough or now, but you get the general idea.   I'm hoping we can
>>>>>> make
>>>>>> this a bit more granular and extensible, and some of that is in the
>>>>>> earlier
>>>>>> code (yes, I also have a fondness for my baby ;-) and marrying that with
>>>>>> the
>>>>>> newer code, I think we can do a lot here.
>>>>>>
>>>>>> Comments?
>>>>>>
>>>>>> Thanks....
>>>>>>
>>>>>> David Reich
>>>>>> AIT Architect,  Adeptra, Inc.
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> vtp-dev mailing list
>>>>>> vtp-dev@xxxxxxxxxxx
>>>>>> https://dev.eclipse.org/mailman/listinfo/vtp-dev
>>>>>
>>>>> _______________________________________________
>>>>> vtp-dev mailing list
>>>>> vtp-dev@xxxxxxxxxxx
>>>>> https://dev.eclipse.org/mailman/listinfo/vtp-dev
>>>>
>>>> _______________________________________________
>>>> vtp-dev mailing list
>>>> vtp-dev@xxxxxxxxxxx
>>>> https://dev.eclipse.org/mailman/listinfo/vtp-dev
>>>
>>> _______________________________________________
>>> vtp-dev mailing list
>>> vtp-dev@xxxxxxxxxxx
>>> https://dev.eclipse.org/mailman/listinfo/vtp-dev
>>>
>>>
>> _______________________________________________
>> vtp-dev mailing list
>> vtp-dev@xxxxxxxxxxx
>> https://dev.eclipse.org/mailman/listinfo/vtp-dev
>
> _______________________________________________
> vtp-dev mailing list
> vtp-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/vtp-dev
>