| AW: [smila-dev] Message Resequencer :: change to Agent Interface |
|
Hi Tom, I share Daniel’s
opinion on both issues. Before you start programming
(I see that you’ve already opened a dedicated branch in repository for the
resequencer), please let’s discuss the problem and do some conceptual
work. BTW: What is the use
case that you’re trying to cover with your resequencer? Regards Igor Von:
smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-bounces@xxxxxxxxxxx] Im
Auftrag von Daniel.Stucky@xxxxxxxxxxx Hi Tom, I see two drawbacks
of your proposed solution(s): 1) it will only work on one machine. In a
distributed environment it is not guaranteed, that a Resequencer will get all
the relevant messages concerning one Record. And as each Resequencer has
it’s own map of Ids and sequence numbers two competing operations would
not be recognized as such. The map has to be shared across all Resequencer instances
(e.g. by using another Queue, or a database). The initial idea of the Buffer component in Connectivity
was to filter out and resolve competing operations before they enter the
“system”, that is before they are processed. Of course this Buffer
would also have to share its internal state across all instances (At the moment
a Agent/Crawler is bound to one instance of Connectivity, so this distribution
is not relevant, yet). In either case, the processing is left totally untouched
by introducing the Buffer component., which leads me to my second issue: 2) The workflow has to be adapted to architecture
changes. So in order to benefit from the Resequencer business logic, workflows
have to be designed in special ways (first do some processing, second store thee
processed data). I think this is hard to grasp by users. BTW: would the actual
storing be configurable, I mean will the Resequencer execute a BPEL pipeline or
is the LuceneIndexing hardcoded ? The latter is of course no valid scenario, we
have to be flexible in this regard, as users may want to store their data in
arbitrary stores/indexes/whatsoever Perhaps you could
elaborate about your concerns with our initial Buffer idea ? Bye, Daniel Von:
smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-bounces@xxxxxxxxxxx] Im
Auftrag von Thomas Menzel oops, http://wiki.eclipse.org/SMILA/Specifications/ProcessingMessageResequencer PS: all along writing
this draft I had this mail open but still managed to forget to add link…. Kind regards Thomas Menzel @ brox IT-Solutions GmbH From: smila-dev-bounces@xxxxxxxxxxx
[mailto:smila-dev-bounces@xxxxxxxxxxx] On Behalf Of Igor.Novakovic@xxxxxxxxxxx Hi Thomas, Could you please
provide us the link to your specification draft? Cheers Igor Von:
smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-bounces@xxxxxxxxxxx] Im
Auftrag von Thomas Menzel Hi, I wrote a
specification draft for this change. plz feel free to comment. in order for this to
implement I will need to change the interface of the agent. Kind regards Thomas Menzel @ brox IT-Solutions GmbH From: smila-dev-bounces@xxxxxxxxxxx
[mailto:smila-dev-bounces@xxxxxxxxxxx] On Behalf Of Thomas Menzel hi folks, just wanted to announce and inform you that
I will be working on the problem that messages don’t get out of sync when
there are changes in close succession this change will be tracked thru the bug https://bugs.eclipse.org/bugs/show_bug.cgi?id=289995 Kind regards Thomas Menzel @ brox IT-Solutions GmbH |