Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [henshin-dev] benchmark results

I think it is a bug in the example. I try to locate it...

On 06/02/2012 06:07 PM, Christian Krause wrote:
Hi,

if you want to see why the optimization is important: check out the latest version of the examples and run the STSBenchmark with and without the optimization (order of magnitude faster with optimization).

Enrico: does the MatchFinder assume any particular order of the variables in the value lists returned by ruleInfo.getVariableInfo().getGraph2variables() ? All I was doing is to sort the lists.

Cheers,
Christian

On 06/02/2012 11:06 AM, Christian Krause wrote:
Hi,

I added the examples to the SVN. My last optimization causes the comp-example to fail. I commented out the optimization in line 342 of EngineImpl therefore. So for now it works, but we should try to fix it so that the optimization can be used. It seems like the NAC of addNewColumn is not properly checked when the optimization is used (nodes are added in every rule application for the same match).

I turned on the logging. So if you want to try it out, do this:

1) run CombBenchmarkManyMatches (works)
2) comment line 343 and uncomment 344-350 in EngineImpl and execute it again (will not stop).

Cheers,
Christian

On 06/02/2012 10:06 AM, Dmitry Zakharov wrote:
Hi Christian,

I am giving you the short summery of all test cases. With red I highlight those where hanshin is slower.

1) STS: Henshin is faster

2) LTS:
                                        Henshin     StoryDiagram
N=20    R=10000            50541 ms      76562 ms   
N=500  R=1                   120765 ms    21505 ms      

3) ALAP: Henshin and SD have time same time

Nodes    HENSHIN        StoryDiagram
1000       468 ms           571 ms
5000       6973 ms         6751 ms
10000      29319 ms     30936 ms

4) Program Understanding : Henshin is faster

5) Comb Many-Matches
                                                                       Henshin     Story Diagram
Grid 50, pattern size 10 (2009 matches)       48725       1098 ms
Grid 50, pattern size  50 (49 matches)           46747       748 ms

6) Comb No-Match

                                           Henshin     Story Diagram
Grid 200, pattern 10        38551 ms        1478 ms
Grid 200, pattern 50        54934 ms     3109 ms


regards, dmitry

on 02.06.2012 9:19, Christian Krause wrote:
Hi Dmitry,

I have one more request: could you briefly summarize where Henshin was 
slower than SD and also give us a hint how much slower it was? This 
would be very helpful for the profiling and optimization.

Cheers,
Christian





Back to the top