Hi folks,

Has someone of you encountered this problem before?

… I know, I know, this is an old version - but it runs at a customer.

2012-04-12 10:52:07,552 ERROR [QueueWorker-Listener@552327108               ]

listener.ListenerWorker                       -

org.eclipse.smila.connectivity.queue.worker.ListenerException:

[Add_SuchkernPipeline, thread:0] processMessage

        at

org.eclipse.smila.connectivity.queue.worker.internal.listener.ListenerWorker.parseMessage(ListenerWorker.java:295)

        at

org.eclipse.smila.connectivity.queue.worker.internal.listener.ListenerWorker.run(ListenerWorker.java:123)

        at java.lang.Thread.run(Thread.java:662)

Caused by: org.eclipse.smila.datamodel.xml.DeserializationException:

java.io.StreamCorruptedException: invalid stream header: 00000000

        at

org.eclipse.smila.datamodel.xml.XmlSerializationUtils.deserialize(XmlSerializationUtils.java:200)

        at

org.eclipse.smila.connectivity.queue.worker.internal.listener.ListenerWorker.parseMessage(ListenerWorker.java:261)

        ... 2 more

Caused by: java.io.StreamCorruptedException: invalid stream header: 00000000

        at

java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:782)

        at java.io.ObjectInputStream.<init>(ObjectInputStream.java:279)

        at

org.eclipse.smila.datamodel.xml.XmlSerializationUtils.deserialize(XmlSerializationUtils.java:196)

        ... 3 more

It appears that under processing load the internal AMQ contains an illegal JMS entry, i.e. one that just contains zeros and no record and hence the exception when parsing.

The result is that the message is never remove from the AMQ and smila then re-tries it again and again, till kingdom come, causing 99% load on the server.

I will create a fix in the branch for this by just logging an ERROR here and pushing the message into the DLQ but I was wondering what might create this message - hence the question.

PS: manual resolution is to shutdown smila and kill the AMQ store.

Thomas Menzel @ brox IT-Solutions GmbH

ah thx juergen,

so the <sequence> is like the {} in java  and defines a block.

cheers

Hi Thomas,

the problem is in the BPEL: two consecutive pipelet invocations must be placed inside a <sequence> element,

or ODE will execute only the first one. The <if> structure is something like

  <if>

    <condition>…</condition>

    &activity;

  </if>

and &activity; is either ONE extensionActicity (or some other basic BPEL activity) or a structured activity like <sequence>.

On the other hand you can omit the <sequence> around the <if> because the <if> is the only activity inside the <forEach><scope>.

(Yeah, BPEL is fun … ;-)

And finally, when using the index variable in the pipelet invocation, you probably don’t want to use the output variable, because it

then will contain only one record ID from the input variable selected by the index, but not the complete record list processed by the

pipeline. Of course, yu must also return the original “request” variable in the <reply> activity, because the “result” variable does is

not set anymore. That’s OK because it’s only the record IDs in the BPEL variables that matter, the record content is taken from the

blackboard.

The way to use output and index together is shown in the attached “testpipeline3-alternative.bpel”: Inside the <forEach> scope you

can declare a new variable that will hold only the single record that is processed in <forEach> scope. The first pipelet invocation then

writes the record “request[index]” to variable “currentRecord” and the second pipelet can process just “currentRecord” without

using the index variable. That should be a bit faster, especially if the many records are processed in a single pipeline run, or if lots of

pipelets are invoked in the sequence of the <forEach> loop.

Hope that helps (-; At least, here the test is green with both variants.

Cheers,

Juergen

Hi Thomas,

the problem is in the BPEL: two consecutive pipelet invocations must be placed inside a <sequence> element,

or ODE will execute only the first one. The <if> structure is something like

  <if>

    <condition>…</condition>

    &activity;

  </if>

and &activity; is either ONE extensionActicity (or some other basic BPEL activity) or a structured activity like <sequence>.

On the other hand you can omit the <sequence> around the <if> because the <if> is the only activity inside the <forEach><scope>.

(Yeah, BPEL is fun … ;-)

And finally, when using the index variable in the pipelet invocation, you probably don’t want to use the output variable, because it

then will contain only one record ID from the input variable selected by the index, but not the complete record list processed by the

pipeline. Of course, yu must also return the original “request” variable in the <reply> activity, because the “result” variable does is

not set anymore. That’s OK because it’s only the record IDs in the BPEL variables that matter, the record content is taken from the

blackboard.

The way to use output and index together is shown in the attached “testpipeline3-alternative.bpel”: Inside the <forEach> scope you

can declare a new variable that will hold only the single record that is processed in <forEach> scope. The first pipelet invocation then

writes the record “request[index]” to variable “currentRecord” and the second pipelet can process just “currentRecord” without

using the index variable. That should be a bit faster, especially if the many records are processed in a single pipeline run, or if lots of

pipelets are invoked in the sequence of the <forEach> loop.

Hope that helps (-; At least, here the test is green with both variants.

Cheers,

Juergen

Hi folks,

I have come over a probable bug of ODE/BPEL,  in that it doesn’t instantiate a 2^nd instance of the same Pipelet in certain cases.

Plz have a look @ org.eclipse.smila.processing.worker.test.BpelWorkerTestPipelet3 and the associated pipeline3, which very simple and in this it works, as the test chose when run.

I have attached another simple bpel which test fails, even though semantically it is the same, as I just do

if (1=1) {

                instance 1

                instance 2

}

For quick testing just replace the content of the pipline3 with this one and run the test -> it fails.

If you put a breakpoint into the configure() of the Pipelet it will only halt once but obviously it should be 2x!

Any one has a clue if I make smth. Wrong in my bpel or if this is a genuine bug?

Thomas Menzel @ brox IT-Solutions GmbH

Hi,

Me too.

Bye

Andreas

Hi,

Ok with me.

Cheers,

Juergen.

Hi folks,

I was just looking over the HL spec and was wondering if we should change it a bit for the 1.1 release.

ATM we just ask for a list of attributes for which HL shall be turned on, but there is no room to spec anything else.

Hence I propose to change the HL config to be similar to the other configs on field level to be contained in a map, like so:

<Val key="query">meaning of life</Val>

<Seq key="highlight">

  <Map>

    <Val key='attribute'>content</Val>

    <Map key='nativeParameters'>

      ...

    </Map>

  </Map>

</Seq>

For now I would just put into the general smila search spec, that this map contains the attribute and the nativeParams map.

If any of you think we should add any common HL arguments into the general spec

-          Length

-          startMarker

-          endMarker

-          â

plz answer to this mail but at this time I wouldnât want to do this.

Result wise I would leave it at the current general spec that the highlighted text (if present at all) is contained in the _highlight-Map for each result item containing a nested map having the attributeâs name and a nested text-Val at minimum. If an impl. wants to add anything else it is free to do so.

<Seq key="records">

  <Map>

    <Val key="_weight" type="double">0.95</Val>

    <Val key="_recordid">file:hamlet</Val>

    <Val key="title">Hamlet</Val>

    <Val key="author">Shakespeare</Val>

    <Map key="_highlight">

      <Map key="content">

        <Val key="text">... To *be* or not to *be* ...</Val>

      </Map>

    </Map>

    ...

  </Map>

Are you ok with that?

What do you think?

If there are no objections we would impl. that then for solr.

Thomas Menzel @ brox IT-Solutions GmbH

Hi,

Ok with me.

Cheers,

Juergen.

Hi folks,

I was just looking over the HL spec and was wondering if we should change it a bit for the 1.1 release.

ATM we just ask for a list of attributes for which HL shall be turned on, but there is no room to spec anything else.

Hence I propose to change the HL config to be similar to the other configs on field level to be contained in a map, like so:

<Val key="query">meaning of life</Val>

<Seq key="highlight">

  <Map>

    <Val key='attribute'>content</Val>

    <Map key='nativeParameters'>

      ...

    </Map>

  </Map>

</Seq>

For now I would just put into the general smila search spec, that this map contains the attribute and the nativeParams map.

If any of you think we should add any common HL arguments into the general spec

-          Length

-          startMarker

-          endMarker

-          â

plz answer to this mail but at this time I wouldnât want to do this.

Result wise I would leave it at the current general spec that the highlighted text (if present at all) is contained in the _highlight-Map for each result item containing a nested map having the attributeâs name and a nested text-Val at minimum. If an impl. wants to add anything else it is free to do so.

<Seq key="records">

  <Map>

    <Val key="_weight" type="double">0.95</Val>

    <Val key="_recordid">file:hamlet</Val>

    <Val key="title">Hamlet</Val>

    <Val key="author">Shakespeare</Val>

    <Map key="_highlight">

      <Map key="content">

        <Val key="text">... To *be* or not to *be* ...</Val>

      </Map>

    </Map>

    ...

  </Map>

Are you ok with that?

What do you think?

If there are no objections we would impl. that then for solr.

Thomas Menzel @ brox IT-Solutions GmbH