RE: [smila-dev] RE: FYI :: new feature :: Message Resequencer

[Thomas Menzel <tmenzel@xxxxxxx>]

Hi igor,

> > The principle is **very trivial**.
> That may be true, but the way you elaborated it is certainly not.

That is why I rewrote it this afternoon and this mail seems to be based on an older version.
Hopefully it is clearer now.

> By concentrating on something you call processing target.
> Quote:
> " the processing piplines are as normal, but: 1. w/o the step of
> calling the processing target and #  they add the result to a new
> queue, Q2"
> You assume here that the first call of the pipeline is cheap. This is
> never the case!

As I have said before: 
first comes how a requirement may be met, however complicated or slow it makes smila.
Then comes the evaluation in regard to need and costs.

BTW: This has changed now, it is still possible but not necessary, see the diff. setup cases

> Sorry, but I have a problem with the very first sentence:
> Quote:
> " the following applies to all PRs in need of resequencing:
>    1. the first thing it needs to do, is to be REGISTERED with the
> SRS."
> The processing request needs to register itself to/with SRS??
> How should this happen?
> Please explain this.

My fault. Should not have used an active case here, was just tired using passive. This is fixed now on the wiki.
Also the introduction sentence is redundant as we only talk about cases where we need resequencing.

> Before a write a novel which explains the details of the buffer concept
> Do you agree with them?
> If yes, that I can go deeper and present the technical details that
> came across my mind.

I guess I understand the buffer concept. I agree with it so far that it is a useful solution with merits and weaknesses, as does mine. So by all means, edit the wiki page for the buffer concept as you see fit.

> > I'm thinking of giving each solution an own page, e.g. one for
> buffer, FSR and
> > SRS.
> > What do you think of that?
> I have no problem with that.
> Please also do not forget do add one for Jürgen's idea.

Done. 

For jürgen's idea i only have a page and links to the mails thus far.
Any idea for a good name? Skip Pipelet doesn't quite cut it IMO. 

Kind regards
Thomas Menzel @ brox IT-Solutions GmbH


> -----Original Message-----
> From: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-
> bounces@xxxxxxxxxxx] On Behalf Of Igor.Novakovic@xxxxxxxxxxx
> Sent: Mittwoch, 7. Oktober 2009 16:33
> To: smila-dev@xxxxxxxxxxx
> Subject: AW: [smila-dev] RE: FYI :: new feature :: Message Resequencer
> 
> Hi Tom,
> 
> > > But I am not solely to blame on that since:
> > > 	- Your proposal is everything else but trivial. Actually you
> > > proposed 2 things...
> > > 	- The description is 8k plain text long.
> > > 	- You introduced almost a dozen of new terms and
> > > abbreviations...
> >
> > The principle is **very trivial**.
> That may be true, but the way you elaborated it is certainly not.
> 
> > I just take time and space to explain it and see if all requirements
> are met.
> > Sorry, if that is too much for you, but in order to be precise and
> complete these
> > things just need to happen.
> I'm not the only one having trouble to understand your elaboration...
> It would be helpful if you could present your ideas more concise - like
> for example Jürgen did.
> If the principle that you suggest is that trivial, than you could
> explain it in 3 sentences. Right?
> 
> 
> > > 1. Your approach is not general. You concentrate on something you
> call
> > "processing targets"
> > Sorry, but my approach is _very_ general IMO, or what do you think
> limits it to a
> > specific use case?
> >
> > > and completely neglect the price of preprocessing.
> > Where do I do this?
> By concentrating on something you call processing target.
> Quote:
> " the processing piplines are as normal, but: 1. w/o the step of
> calling the processing target and #  they add the result to a new
> queue, Q2"
> You assume here that the first call of the pipeline is cheap. This is
> never the case!
> 
> 
> 
> > > 2. Both "full" and "smart" resequencer share the same problem: In a
> > > cluster environment you will have at least one resequencer instance
> per
> > > node. How will those instances share their ID-SN-Maps in a
> performant
> > > way?
> >
> > I just updated that section to explain that better.
> Sorry, but I have a problem with the very first sentence:
> Quote:
> " the following applies to all PRs in need of resequencing:
>    1. the first thing it needs to do, is to be REGISTERED with the
> SRS."
> The processing request needs to register itself to/with SRS??
> How should this happen?
> Please explain this.
> 
> 
> > > Done. Please take a look at " Idea 1 - Connectivity
> > > Consolidation/Resequencer Buffer" section of
> > >
> http://wiki.eclipse.org/SMILA/Specifications/ProcessingMessageResequenc
> >
> > I did. Unfortunately you only left a few comments on the things I
> pointed out.
> > What I actually had asked you, is to explain for each listed
> requirement, i.e.
> > these:
> >
> > # 2.1 Basic Operations
> > # 2.2 compound management, splitting of records
> >     * 2.2.1 Composition
> >     * 2.2.2 Aggregation
> >     * 2.2.3 Parent/Descendants Ordering Requirement
> > # 2.3 support >1 processing targets
> > # 2.4 clustering, complex processing chain
> > # 2.5 single point of failure
> > # 2.6 scalability and performance
> >
> > ...how the buffer will solve it. For this you will have to rework the
> section a
> > bit.
> Before a write a novel which explains the details of the buffer concept
> I wanted to present some major ideas.
> Do you agree with them?
> If yes, that I can go deeper and present the technical details that
> came across my mind.
> 
> > Other subject:
> > I'm thinking of giving each solution an own page, e.g. one for
> buffer, FSR and
> > SRS.
> > What do you think of that?
> I have no problem with that.
> Please also do not forget do add one for Jürgen's idea.
> 
> Regards
> Igor
> _______________________________________________
> smila-dev mailing list
> smila-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/smila-dev
> 
> No virus found in this incoming message.
> Checked by AVG - www.avg.com
> Version: 8.5.420 / Virus Database: 270.14.4/2417 - Release Date:
> 10/07/09 05:18:00