Community
Participate
Working Groups
Build 20021217 1. Create simple project S 2. Create 2 folders /S/f1 and /S/f2 3. Create simple file /S/f1/read.txt 4. Move read.txt from f1 to f2 back and forth (using drag and drop) Observe: On the 10th move, it takes 15 seconds to do the move. The 11th move takes 35s, the 12th move takes more than 1min30s.
This is a bug (?) which has been around for a while but has just surfaced because of recent changes with copying local history automatically. I believe that we would see the same behaviour if we called #setContents 1000's of times on the same file. The deal is that the local history for a file is expanding exponentially. In the history store we check some of the Policy attributes (file size) but not others (max number of states per file). We only check the number of states when we clean the history store which is currently on shutdown. We will not fix this for M4. We don't feel that normal users will run into this problem frequently. We will investigate our application of Policy items. Perhaps we should be doing it during the copyHistory method. Marking as an M5 item.
At the end of the copyHistory method, we now check to see if we have exceeded the maximum number of states for a file. If this maximum has been exceeded, we will purge down to the maximum (keeping the states with the most recent last modified times). New test added to the history store tests to output timings on the move/copy of a particular file. Time to move/copy is no longer exponential.
*** Bug 28952 has been marked as a duplicate of this bug. ***
I'm not convinced this is fixed... If you move file1 > file2 > file1, you end up with 2-3 times as many states in the history store as when you started, all of which are exact duplicates of the original states. We now ensure this doesn't exceed the limit, but if someone has set their limit much higher than the default it will still be a problem. We should be able to detect when copying states if the destination already has those states available (won't the UUID be the same?). Also, the new test is just printing things to the console. The stdouts should be removed and it should be making assertions about the number of states instead.
Reopening to look into not adding duplicate states.
I have commented out the System.out.println statements in the test (HistoryStoreTest#testBug28603). This made it hard to see test results when running junit tests.
The question arises: What is more important - fewer states in the history store or faster time to move/copy a file? I have done some time tests as follows: - create a file (in a project, in a folder). Call it 'file1'. - give the file some contents and change the contents so the file has multiple states - repeat 50 times: - move 'file1' to 'file2' - move 'file2' to 'file1' (so there is a total of 100 moves for each test). For each test record the following: - how long each move takes - how much time out of each move is spent in the logic to copy state information - how many states are copied At the end of each test, output the averages of each of these numbers (i.e. keep a running total and divide by 100). In this test, the maximum number of states allowed is 50 states. The following data was gathered: Don't do any checking for duplicate state information: Avg. Move Time Avg. Time to Copy State Info Avg. # of States Moved --------------------------------------------------------------------------- 92ms 59ms 49 states 94 61 49 94 64 49 95 63 49 87 57 49 90 59 49 90 60 49 88 58 49 -- -- -- Avg. 91.25 60.125 49 Check for duplicate states using IHistoryStoreVisitor logic: Avg. Move Time Avg. Time to Copy State Info Avg. # of States Moved --------------------------------------------------------------------------- 100ms 69ms 1 state 99 67 1 101 69 1 100 69 1 104 73 1 101 68 1 100 70 1 100 68 1 -- -- -- Avg. 100.625 69.125 1 Check for duplicate states, without using a visitor, but accessing the index for the history store directly: Avg. Move Time Avg. Time to Copy State Info Avg. # of States Moved --------------------------------------------------------------------------- 104ms 73ms 1 state 107 74 1 107 76 1 101 71 1 105 73 1 108 76 1 106 73 1 105 74 1 -- -- -- Avg. 105.375 73.75 1 Conclusions: The average time to move a file will increase by approximately 10% if we check for duplicate history states. The number of states stored is greatly decreased if we check for duplicates and if the use case is such that a moved/copied file is likely to be moved/copied back to its original position. It should be noted that these timing tests do not take into account any performance benefit that would be derived from having fewer states in the history store.
These changes for integration build 20030121+ We now check for duplicate history store states using the IHistoryStoreVisitor logic. Tests which output information on timings (for performance) have been moved into the test package org.eclipse.core.tests.resources.perf so these print statements will not pollute information from the test suite run with the nightly build. The test for this bug, left in the test package org.eclipse.core.tests.internal.localstore is now just concerned with ensuring that: - we don't add duplicate states when a copy/move is done - is we will exceed the maximum number of states, we prune the history store for this file
