Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [henshin-dev] benchmark results

Based on the results of the two examples, there must be cases where the "temporaryDomain=null" is required, and other cases where it must not be done.

Unfortunately we don't know when it should not be done. We could try to debug the Java2StateMachine example but based on the complexity I have my doubts that we find the problem. So can we think of a scenario where the "temporaryDomain=null" should *not* be executed in DomainSlot.clear() ? Or should it be somewhere else? Or...?

Cheers,
Christian


On 06/11/2012 08:05 PM, Christian Krause wrote:
Hi Enrico,

On 06/11/2012 06:53 PM, Enrico Biermann wrote:
Hi,

No. If a variable which restricts A gets unlocked, the objects which where removed from the temporary domain of A because of B needs to be readded.


I think I get it.

However I don't know how it is currently done, because the storage of these removed objects was changed when you implemented the new change API (now the objects are stored in a DomainChange container, but I don't know whether this is the only change or not)


I did not change anything in the implementation of the match finder. The old and the new change model as used for applying rules is not and was not used by the match finder. I don't know when or who introduced the DomainChange class. So, again, I did not touch any of the internals of the match finder, because I know how tricky it is ;). But I am also keen to know how exactly the DomainChanges are used by the match finder.

As for the second point: Yes, calling resetVariables in eval() is fine. The prior line with findGraph() does a complete matching of its graph and leaves the domain slots in a dirty state (either with all slots locked and a valid match or all slots unlocked with empty domains because no solution was found).
The eval() function is called recursively for each graph in an application condition tree. So for your example a match for the lhs would include a valid (non)-match for the NAC. However each call to the eval() function of the NAC is done with a different match in the lhs and the NAC domain slots must therefore be clean.


Thanks for the explanation. That sounds reasonable. But somehow we must be missing a scenario where something goes wrong. Anybody got any ideas how we can solve this problem?

Cheers,
Christian

Cheers,
Enrico

On 11.06.2012 09:43, Christian Krause wrote:
Hi Enrico,

ok. Does that mean that the temporary domain for a variable A should be reset whenever a variable B that restricts the domain of A gets unlocked?

I noticed also that resetVariables() is called in ApplicationCondition.eval(). So that means whenever we check whether a found match for a NAC is valid, we reset the variables and delete all temporary domains? Why are the domain slots cleared at this point and should the temporary domains really be reset here?

Cheers,
Christian


On 06/11/2012 07:44 AM, Enrico Biermann wrote:
Hi,

Assume a lhs with B->A.
If a variable is not initialized its domain is initialized by all possible values for its type, e.g. a graph has 5 nodes of type A, so the domain is set to 5. In the above example when matching B, the domain of a A has to be restricted too (to all outgoing edges from the fixed node B in the graph to A nodes). This information is stored in the temporary domain. If the temporary domain of A is already set when the domain is initialized. The status of the temporary domain is used instead of the type constraint to set the domain.

This has other consequences, too. For example if 2 different constraints to A restrict the temporary domain to empty, you can immediatly start to backtrack because you will never be able to initialize the domain of A.

Cheers,
Enrico

On 10.06.2012 21:01, Christian Krause wrote:
Hi,

I guess then we have to figure out when exactly the temporaryDomain should be cleared and when not (or how to fix it otherwise). Enrico: if I understand it correctly, the temporaryDomain is used to restrict the domain further when some of the variables have been matched already (?). Initially, I thought the temporaryDomain is just used for performance reasons, but that seems not to be the case. Can you tell us a bit about its purpose?

cheers,
Christian


On 06/10/2012 02:18 PM, Stefan Jurack wrote:
The reference model is correct! It has been published along the ttc2011 workshop and other approaches led to the same solution too.

Stefan

Am 09.06.2012 15:11, schrieb Christian Krause:
Either the reference model is incorrect, or the bugfix introduced another bug.

Cheers,
Christian

On 06/09/2012 02:56 PM, Christian Krause wrote:
Hi,

I looked into the Java2StateMachine example. It seems like one Transition with action="" and trigger="FIN" is not generated and some of the transitions are not correctly wired up. Are we sure the reference model is correct?

Cheers,
Christian



On 06/09/2012 11:35 AM, Enrico Biermann wrote:
Hi,

I found the bug. The temporary domain of a domain slot was not reset after a failed match attampt.
So the scenario was the following:
1) LHS matched
2) NAC did not match, but the temporary domain of the forbidden node still contained an element
3) LHS matched again
4) When checking whether a specific edge would still be possible, a reference constraint failed because the old temporary domain did not contain possible target values for the reference.

I attached a patch. However if you apply it, one of the tests (testJava2StateMachine) will fail. Still, I think that not resetting the temporary domain is a bug and possibly the test is in error in this case.

Cheers,
Enrico

On 09.06.2012 11:06, Enrico Biermann wrote:
Hi,

I debugged the example.
The error occurs when trying to match a "ver" edge. Based on the fact that the target domain slot is not initialized I guess that it is the edge from the top right node to the bottom right forbidden node (in the diagram file).

The reason that the NAC does not match is because the domain of the forbidden node is empty. I don't know yet why but I will see if I can find out.

Cheers,
Enrico

On 08.06.2012 14:21, Christian Krause wrote:
I created a minimal example of the error. Copy the two files into the 'combpattern' package and run it. It should not find a match. The graph has this structure:

o---o
|     |
o---o
|     |
o---o

On 06/08/2012 02:05 PM, Christian Krause wrote:
Hi Enrico,

I still don't know where the problem is. I know exactly when and where it happens (step #7) but I don't know why. No match for the NAC is found. The graph is correct at this point and the rule is also fine as far as I can tell. The debugging is really difficult. I did not really understand what you meant by making a rule for each constraint type. Can you maybe give me some more hints how to debug this?

Cheers,
Christian

On 06/03/2012 11:06 AM, Enrico Biermann wrote:
Hi,

true, the optimization is not the problem, however it probably uncovered a bug in the constraints. you could make a rule for each constraint type present in the application condition of addNewColumn and then switching the order of variables (a nice side benefit is that these rules would also make great unit tests).
At least one variable order of one rule should fail to match.

Regards,
Enrico

On 03.06.2012 10:53, Christian Krause wrote:
Hi,

I couldn't find a bug in the example. The optimization is not the problem. I changed the order of the nodes in the NAC and now I get the error with and without the optimization.

From the log I can see that the NAC was not correctly checked. There does exist an object and two edge, so the rule should not be applicable to this match. The interpreter prints waring now that edges are deleted due to a side effect. I still don't know why the match finder has a problem.

Cheers
Christian

On 06/03/2012 10:38 AM, Enrico Biermann wrote:
Hi,

The order of variables should be irrelevant (for correct results).
If it is not, then it points to either a missing constraint or an incorrect handling of either initialized or uninitialized domain slots for an existing constraint.

The order of nodes for the application condition of addNewColumn was changed from 1, 2, 3, 4, 5 (order of creation) to 3, 1, 4, 5, 2. So a constraint of 3, 4 or 5 should be responsible, but someone with knowledge of the example should continue with the analysis.

Regards,
Enrico

On 02.06.2012 18:07, Christian Krause wrote:
Hi,

if you want to see why the optimization is important: check out the latest version of the examples and run the STSBenchmark with and without the optimization (order of magnitude faster with optimization).

Enrico: does the MatchFinder assume any particular order of the variables in the value lists returned by ruleInfo.getVariableInfo().getGraph2variables() ? All I was doing is to sort the lists.

Cheers,
Christian

On 06/02/2012 11:06 AM, Christian Krause wrote:
Hi,

I added the examples to the SVN. My last optimization causes the comp-example to fail. I commented out the optimization in line 342 of EngineImpl therefore. So for now it works, but we should try to fix it so that the optimization can be used. It seems like the NAC of addNewColumn is not properly checked when the optimization is used (nodes are added in every rule application for the same match).

I turned on the logging. So if you want to try it out, do this:

1) run CombBenchmarkManyMatches (works)
2) comment line 343 and uncomment 344-350 in EngineImpl and execute it again (will not stop).

Cheers,
Christian

On 06/02/2012 10:06 AM, Dmitry Zakharov wrote:
Hi Christian,

I am giving you the short summery of all test cases. With red I highlight those where hanshin is slower.

1) STS: Henshin is faster

2) LTS:
                                        Henshin     StoryDiagram
N=20    R=10000            50541 ms      76562 ms   
N=500  R=1                   120765 ms    21505 ms      

3) ALAP: Henshin and SD have time same time

Nodes    HENSHIN        StoryDiagram
1000       468 ms           571 ms
5000       6973 ms         6751 ms
10000      29319 ms     30936 ms

4) Program Understanding : Henshin is faster

5) Comb Many-Matches
                                                                       Henshin     Story Diagram
Grid 50, pattern size 10 (2009 matches)       48725       1098 ms
Grid 50, pattern size  50 (49 matches)           46747       748 ms

6) Comb No-Match

                                           Henshin     Story Diagram
Grid 200, pattern 10        38551 ms        1478 ms
Grid 200, pattern 50        54934 ms     3109 ms


regards, dmitry

on 02.06.2012 9:19, Christian Krause wrote:
Hi Dmitry,

I have one more request: could you briefly summarize where Henshin was 
slower than SD and also give us a hint how much slower it was? This 
would be very helpful for the profiling and optimization.

Cheers,
Christian





_______________________________________________
henshin-dev mailing list
henshin-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/henshin-dev


_______________________________________________
henshin-dev mailing list
henshin-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/henshin-dev



_______________________________________________
henshin-dev mailing list
henshin-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/henshin-dev


_______________________________________________
henshin-dev mailing list
henshin-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/henshin-dev




_______________________________________________
henshin-dev mailing list
henshin-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/henshin-dev


_______________________________________________
henshin-dev mailing list
henshin-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/henshin-dev


_______________________________________________
henshin-dev mailing list
henshin-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/henshin-dev




_______________________________________________
henshin-dev mailing list
henshin-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/henshin-dev



_______________________________________________
henshin-dev mailing list
henshin-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/henshin-dev



_______________________________________________
henshin-dev mailing list
henshin-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/henshin-dev


_______________________________________________
henshin-dev mailing list
henshin-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/henshin-dev



_______________________________________________
henshin-dev mailing list
henshin-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/henshin-dev


_______________________________________________
henshin-dev mailing list
henshin-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/henshin-dev



Back to the top