Having been absent in the middle of a lot (in on the beginning and then away for a bit and now back to cause trouble ;-), my concern is primarily on the metalanguage, and taking focus away from the industry standard language(s); VoiceXML (and CCXML and SRGS). I appreciate well the stated goal of a more generic "interaction" model rather than just voice, but the goal of the V in VTP was for Voice, and it seems that we've departed from that, and now rely on a generator of the VoiceXML from a metalanguage that may or may not meet needs of the marketplace. The interaction model is good, but we've taken our eye off the ball in the process and IMHO, have made some significant sacrifices.
As an example, my company's platform, while making extensive use of VoiceXML and CCXML, relies on a proprietary dynamic infrastructure for the end-game markup generation based on Freemarker (analagous to JSPs). My desire is to have the tooling be able to generate this Freemarker/VoiceXML/CCXML so I can bring it into my infrastructure for dynamic rendering. I wanted to be able to write a type of hook, and write my generator for Freemarker/VoiceXML/CCXML (FVC for short for now ;-). With this metalanguage, and this other generator, there's not a lot I can do here. The tool only puts out metalanguage, and I can't do much with it. I'm still not completely clear where this generator in your graphics lives, whether it is open source, a server-runtime component, a plugin into VTP or what. In a list form, some questions I have here are:
1) Why is VTP as in Eclipse pretty much unusuable? Why can we only get a working build of it as openVXML from OpenMethods? (I got this info right form you guys in the newsgroup ;-) This needs to be fixed.
2) Is this renderer for the metalanguage open source? If not, I submit it must be because the metalanguage is all VTP will generate today, and to not be able to do what we want with it, IMHO, goes against the spirit of the project.
3) Whether or not the renderer is open source (it should be) if it stays in the big picture, it needs to have extension points to make it render whatever we want. If for no other reason, the visual builder only outputs metalanguage, we need to be able to do what we want with what comes out of the visual interaction builder.
4) By what mechanism is this metalanguage extensible or definable? IOW, what can I do if there are things I want to be able to capture/do with the metalanguage that it can't do today? It's XML, which should be a self-defining language, but how can we do more with it all today?
5) There is no editor for anything in a source mode. There should be a way to edit what gets output by the communication flow builder, and the generator on the back end. At a minimum, there should be an XML editor with the appropriate DTDs set up with the right file extension links put into the perspective.
6) Just a nit, but why is this now "openVMXL" and not VTP? Is it just because I got it from OpenMethods and it's OpenMethod's branding of the tool? (See #1)
I'm not sure the course for this project is a consistent one, or that it embodies the spirit of the project, or a natural evolution of it. I'm willing to concede I may still not "get it" completely, but I think there are some holes here.
I'd be happy to host a call to iron out the questions I and others might have.
David Reich
AIT Architect, Adeptra, Inc.
________________________________
From: Trip Gilman <trip@xxxxxxxxxxxxxxx>
Reply-To: Voice Tools general developers <vtp-dev@xxxxxxxxxxx>
Date: Thu, 16 Apr 2009 15:24:58 -0400
To: Vtp-Dev <vtp-dev@xxxxxxxxxxx>
Subject: Re: [vtp-dev] Questions, directions, techie stuff
Mike,
You are correct in that this mailing list is provided by the Eclipse foundation for our project to use to discuss issues of design and architecture and to iron out implementation details. The design decisions that I have recently described were not discussed in this forum as they were made prior to OpenMethods contributing what is now the majority of the VTP codebase.
The codebase has been stable from a design and architecture standpoint for quite some time. If you look at the recent Bugzilla and source commit activity for this project you'll see that there has been quite a number of bugs and minor usability enhancements that have been put in place. Sam Hopkins has made significant contributions in both usability and defect resolution. Adam Berry has done a great job bringing the VTP into the common Eclipse build infrastructure. We will soon have consistent and frequent builds to offer consumers of this project. Randy Childers has been invaluable in testing and documentation updates. Dirk has been implementing the launcher configuration and bridge for JVoiceXML. I have finished work on the media defaults system and merged it into the trunk for general consumption.
I will be kicking off a discussion of the "what's next" variety soon for the next major development effort, but for the most part, the current design and architecture has been able to adequately support the expanding feature set. I would suggest that if anyone has suggestions or feature requests that they post them to either this list, the newsgroup, or preferably directly into Bugzilla. I have tried to be open about the process behind the decisions made on this project, even those that occurred pre-contribution. I am more than willing to discuss any decision made in the past or those that will be made in the future. Are there any design or architecture elements in particular you would like to discuss?
Trip Gilman
On 4/16/09 8:08 AM, "Mike Greenawalt" <mike@xxxxxxxxxxxxxx> wrote:
Isn't this mailing list the place where all these design issues and
choices should have been discussed and debated long since? It seems odd
that this info appears only now, after 2+ years of the project.
Thanks, David, for stirring the pot.
-- Mike
Trip Gilman wrote:
Nino,
You are absolutely right that there are drawbacks with the single file
storage approach. I'll try to outline why we decided to go that
direction, as our original design actually used multiple files. Most
of the issues we've seen when multiple developers make changes to an
application can be attributed to two primary causes: the information
is stored in XML format and there is a separation between the storage
format and the medium familiar to the developer. Let's take a look a
closer look at these problems.
The XML format is fabulous for certain use cases. Storing highly
structured data that contains large amounts of text that needs to be
accessible on multiple platforms and manageable by humans is one of
them. And this is exactly the type of data that makes up our
application definitions. The problem is that is gives the diff engine
of most source control systems fits. Nearly by definition, all data
markers in XML are variable length in both individual composition
(attributes) or number and order (sub elements). In my experience the
most usual culprits of incorrect difference calculation are elements
with multiple attributes, especially when the number or which
attributes are dynamic, and the end tags of elements with children. To
help avoid the havoc caused by these instances, the application
document generated by the VTP follows some strict rules:
* XML indentation is fully enforced on all elements and levels of
nesting
* Start tag pre-ambles are always placed on their own line
* Element attributes are always placed on their own line
* Element attributes always occur in the same order regardless of
presence
* Element end tags are always placed on their own line
* Element collections are always written in the same order they
were read or created unless there is an overriding ordering
mechanism
However, even with all these rules in place, there are still instances
of conflict when merging changes from independent sources. This brings
us to the other basic driver; the developer is not familiar with the
XML format. Our development environment presents the developer with a
100% graphical representation of all information present in the XML
document. This goes so far as the editor doesn't even bother with the
underlying file until a change needs to be committed to storage. Often
times, the developer doesn't even have experience with XML or other
central technologies employed by the VTP let alone a working knowledge
of the schema and nuances of our specific XML language. Eclipse is
great in that it happily displays the head to head conflict resolution
UI. Given the target audience, this can result in even further data
loss depending on their tendency to click buttons.
I've now done something I hate to do, describe an array of problems
without really offering any solutions. I'm not sure if there are any
"good and easy" ones. OK, back to why we migrated to single file over
multiple. The main reason was that using multiple files to store the
data opened our project structure up to a possible inconsistent state,
depending on user behavior. In a traditional java project, having many
different files with changes in them and various states of committal
to the source repository is normal and often doesn't even result in an
inconsistent state. However, our applications are very integrated and
represent a continuous flow of logic from the single entry point to
its final states. There are very few edit oriented operations that
could be performed that wouldn't leave the entire project in an
inconsistent state if all documents weren't checked in. Although this
type of situation is fairly common for those of us who are familiar
with the standard software development lifecycle, it can be quite a
learning curve for much of the target audience.
Another reason we decided to perform the migration was all the other
issues with XML were still present in the multiple file approach, so
we decided to merge all the files into a single source. To be honest,
I'm not entirely sold on this approach, but it solved more issues than
it presented at the time. Choices like these really resonate with what
Kent Beck discussed in his /Design is an Island/ piece on Planet
Eclipse. I would love to enable a greater liberty in changes being
made by multiple developers. I would like to mention that with a
moderate amount of discipline, multiple developers can still edit the
same project. If the developers remain within say a single canvas or
don't modify the same canvas, the likelihood of a conflict is greatly
reduced thanks to the efforts I mentioned above. Also, there is the
ability to separate your application into several independent projects
as application fragments. These can obviously be edited without fear
of collision.
Ultimately, I think the only real solution is to provide a completely
custom conflict resolution UI that allows the application designer to
merge changes using the medium they are used to. Showing the changes
in a head-to-head visual display and simple point and click change
acceptance. This would almost be a project in and of itself and
consequently has been a lower priority than other more easily
reachable fruit. I've been recently playing around with a couple of
ideas on how to make the XML rules we follow more effective. My
favorite one so far is to sort of watermark the end element tag so as
to identify it uniquely even within a list of similar elements. For
example, most of our elements represent objects that have an
identifying attribute like 'id'. The line containing the end tag would
have an XML comment added just after the tag content that contained
the identifier. This would allow the line based diff engines to more
easily match groups of lines together in cohesive segments. I haven't
fully worked out the details yet.
Trip Gilman
On 4/14/09 10:57 AM, "Nino Martinez" <nino.martinez.wael@xxxxxxxxx> wrote:
Hi Trip
Now that you mention a thing about the runtime platform, as I see it the
interactions you describe in the visual layout all goes into one xml
container? Im wondering why it's done this way? Nortel's SCE, which im
heading into using everyday, also has this approach.
Making it impossible to be more than one dev on projects. Since you
overlap changes in the xml file.. Or am I wrong about VTP (I hope so)..
If it instead could be contained in some code generation or something
like it, maybe taking advantage of java7's scripting possibilities (when
it comes) or something else to leverage this problem.. However this
would probably impose a big code change..
regards Nino
On Sun, 2009-04-12 at 19:27 -0500, Trip Gilman wrote:
David
First I'd like to thank you for your interest in the VTP and welcome you
back to the industry. I've provided a broad overview of where we are
now as
a project and also weaved some of my detailed thoughts on your questions
into your original text below.
You are correct in that the current VTP is a complete departure from the
codebase originally contributed by IBM. I would like to take a moment to
point out some of the fundamental changes between the original
vision and
our current direction and then discuss the reasoning for our approach.
First, the xml editors for GRXML grammars have been replaced with the
generic xml editor provided by the basic eclipse development
framework. The
original VTP utilized a customized xml editor based on those
provided by the
Web Tools Project. The removal of this dependency reduced the
overall size
of our software by nearly 100M and eliminated several layers of
complexity
for no real loss of functionality.
Second, and probably the most centrally important, the VTP employs a
common
runtime architecture as opposed to code generation. I don't want to
focus
on the choice between the two, but I would like to illustrate how we
leverage our choice of a runtime architecture.
The voice tools project has almost become a misnomer. Our
development and
runtime frameworks are no longer just for VXML, but have evolved into a
generic platform for describing interactions. The core of the
development
environment is blissfully unaware that the user intends to use the
application they are creating as the source of a VXML interaction. The
runtime environment executes the interaction within a process engine
that
emits basic interactive commands that are interpreted into VXML at
the final
moment. This is important to understand as this philosophy is pervasive
throughout the codebase.
The user creates applications by adding Actions to a canvas and then
defining the transitions between those actions. The most common Actions,
such as a Menu or a Prompt(Output), are standard components provided
by the
framework. However, third-party developers can easily and quickly create
their own action libraries for users to use. I want to reiterate that
although these actions may share the names of the elements that make up
VXML, they are not the same.
The designer has an abstraction framework that models the nuances of
interaction types on these actions, allowing interaction specific
configuration. For example, the VTP comes with an implementation of the
Voice (VXML) interaction type. This provides interaction specific UIs to
manage VXML settings such as barge-in and dtmf/speech entry.
This framework has been used to create other interaction types such as
Instant Messaging. An application can support any combination of
interaction types the user has available to them. This is very
powerful in
that the same application deployed once, could drive the interactions of
both telephone and IM based customers.
On the runtime side, things are equally abstracted. As I mentioned
earlier,
the artifacts produced by our designer (upon export) are actually
process
meta-language documents. These documents are executed by the process
engine
at the core of the runtime. As the process is executed, interactive
commands are generated. These commands are then filtered through a
platform
abstraction layer and evaluated. The VTP includes a platform layer
designed
for VXML interactions that is capable of transforming the interactive
commands into their representative VXML documents.
The platform abstraction layer is powerful on two fronts. First, it
allows
the VXML generation process to be customized to the specific VXML
Browser
platform requesting the document. This means that a single application
instance can service multiple VXML platforms simultaneously without
re-export or compilation. The VXML session could even bounce back
and forth
between the platforms without issue.
Second, the platform abstraction is open to third-parties to add
implementations for additional platform types and output mechanisms. The
VXML platform generates XML documents and processes requests and
responses
over HTTP. Other implementations could simply execute direct API
calls to
other software or hardware, never producing a document artifact.
They could
also support transports other than HTTP.
I hope this helps illuminate the project for you.
Trip Gilman
On 4/10/09 8:33 PM, "David Reich" <David.Reich@xxxxxxxxxxx> wrote:
Hi,
First, let me (re)introduce myself. David Reich, formerly of IBM. I was
involved in the creation of VoiceXML and was the guy who was the
initial
driver behind all of the Eclipse-based voice tooling, first from IBM's
toolkit, and it's subsequent donation to Eclipse and VTP. I've
since left
IBM
and as you know, IBM's involvement in voice has waned. But, I am
now with a
new company, happily back in VoiceXML, CCXML and SRGS, and we are
doing a
lot
with VoiceXML and CCXML and I was brought in to drive our
development team,
and one of the areas I am looking at is the development of our
application
artifacts, and VoiceXML application generation.
With that, I've spen the last couple of days figuring out where
things are
with this project, how we can use it, how we might give back to it
and so
on.
From what I've been able to see thus far, it looks like the current
code
base
bears little to no resemblance to the VTP 1.0 code that came in
from my
former
team, but, on the other hand, there is a nicely-coming-along visual
editor
without which all VTP would be is the XML editor with DTDs plugged
in and
while nice, not really compelling.
I was hoping to provoke some discussion and see about some
combination of
code
to create an even more gener(ic)ally useful environment. With that,
some
thoughts and the hope we can put some other things together. In no
specific
order:
* The original IBM code seems to be gone. When things moved to SVN, the
CVS repository went away, so unless someone has that source code,
it's gone
forever (I've checked with the (remnants of) the IBM team, and
without some
serious effort, it's not readily available. So, how can we get that
back,
even if we only want to take small bits (or even none?) of it?
There's got
to
be useful things there, and we should still look at that.
The original IBM source still exists, however it is not readily
available as
it has been archived from CVS by the webmasters to reclaim space and
keep
the repositories current. If you would like a copy of the archive, I
would
be more than happy to make the request for you.
* In the original tools, of which the VTP 1.0 code was a part, the
vision
was a call flow builder similar to what WebMethods has been working
on, but
rather than generate a metalanguage, have it generate VoiceXML and
have a
tabbed view (kinda like FrontPage) where you see the call flow
view, and the
VoiceXML source view, and the editor would allow you to tweak the
VoiceXML
in
source edit mode, affording finer-grained control of the VoiceXML.
Very early on we abandoned code generation in favor of a runtime
architecture to overcome several challenges we were faced with when
developing applications in the field. Each time we encountered a new
VXML
platform, there were always a myriad of minute differences or
interesting
interpretations of the specification. We considered just coding these
changes into the code generation piece until we ran into a client
that had
multiple platforms in service. The solution was to export the
application
twice, once for each platform. This instance was one of the driving
forces
in the transition to a runtime architecture.
Another driving factor of the move, was the realization that we were
already
building a runtime, just not officially and not effectively. We had
developed a loose library of utility functions and were using them
over and
over again in all our applications. More and more functionality was
being
encoded into this framework in an effort to hasten development.
Making the
runtime a first class citizen in the architecture allowed us to
leverage the
advances we had made while removing most of the hurdled we were facing.
* Linkages to the Eclipse framework for task-list items in validation,
warnings, errors, etc.
Absolutely, integration to the tagging systems in Eclipse is
currently on
the wish list.
* Enabling (as Randy has told me is the direction) a derivative or
extension plugin to generate code that spews forth VoiceXML. This
could
take
the form of the OpenMethods servlet, or even perhaps an extension
one could
write
to do JSPs, or other types of markup/tag/metalanguage since not
everyone
will
want just the VoiceXML from the OpenMethods servlet, or wants to make
further
tweaks to the VoiceXML.
I won't say that it would be impossible to generate the resulting
VXML from
an application statically, but at some point you'd end up recreating the
logic inherent in the runtime in JSP or some other language. In the
case of
multi-modal applications, simply duplicate the application several times
over.
On a side note, the modification of generated code after export can be a
very tempting practice as many problems can be solved quickly this way.
However, I never recommend the use of this. It places a process
requirement
on the development and deployment staff to always re-apply the
changes to
the code each time there is an update. What happens if changes made
to the
code require the workaround to be updated? This can easily balloon
into a
maintenance nightmare. I would say that it should be the goal of any
system, whether code generation or runtime, to avoid the need of
this at all
costs.
* A broader property sheet for each call element in the palette on the
canvas to specify items such as barge-in, SSML tags or information,
prompt
audio files to be played and so on. Specifically, the block would
play one
of
a set of prompts based on current state (such as "Welcome to..." or
"Welcome
back to..." based on some global variables in the ECMAscript for
example.
This tool should should hopefully allow for the export of this
property
sheet
data in different formats, but then again, as open source, we can work
together on extensions for our specific needs and write the hooks and
extension code for different needs.
The designer already supports this type of configuration in several
ways.
At a base level, the developer can use Javascript variables to indicate
prompt content. SSML can also be included in prompt content simply
as text.
At a higher level, the designer employs a system called branding to
allow
very detailed configuration for different user segments. Branding can be
used for personalization, line of business differentiation, or even
multi-tenanting of an application.
On the export aspect of this, I assume you intend to use the artifacts
within a static JSP/VXML document.
* Have the palette be extensible where one can take prebuilt
modules (say
an authentication subflow) add them to the palette, and drop it
into a new
or
existing application.
The designer is fully extensible. Third-parties can easily publish
additional modules for use. Developers are also able to package up
common
call flow elements into reusable components called dialogs or even
significant portions of a call flow into a fragment.
That's enough or now, but you get the general idea. I'm hoping we
can make
this a bit more granular and extensible, and some of that is in the
earlier
code (yes, I also have a fondness for my baby ;-) and marrying that
with the
newer code, I think we can do a lot here.
Comments?
Thanks....
David Reich
AIT Architect, Adeptra, Inc.
_______________________________________________
vtp-dev mailing list
vtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/vtp-dev
_______________________________________________
vtp-dev mailing list
vtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/vtp-dev
_______________________________________________
vtp-dev mailing list
vtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/vtp-dev
------------------------------------------------------------------------
_______________________________________________
vtp-dev mailing list
vtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/vtp-dev
_______________________________________________
vtp-dev mailing list
vtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/vtp-dev
_______________________________________________
vtp-dev mailing list
vtp-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/vtp-dev
