Summary: | [index] Lots of unbuffered sequential reads in DiskIndex | ||||||||
---|---|---|---|---|---|---|---|---|---|
Product: | [Eclipse Project] JDT | Reporter: | Brock Janiczak <brockj> | ||||||
Component: | Core | Assignee: | Frederic Fusier <frederic_fusier> | ||||||
Status: | VERIFIED FIXED | QA Contact: | |||||||
Severity: | normal | ||||||||
Priority: | P3 | CC: | daniel_megert | ||||||
Version: | 3.3 | Keywords: | performance | ||||||
Target Milestone: | 3.4 M1 | ||||||||
Hardware: | All | ||||||||
OS: | All | ||||||||
Whiteboard: | |||||||||
Attachments: |
|
Description
Brock Janiczak
2007-04-07 05:55:17 EDT
Sounds a reasonable request. We can apply same optimization in this method than the one made for bug 168384. However, please note that this method represents an really small part of the Java Tooling Initialization. Using yourkit to look at method counts and time, here are the numbers: Stack of calls Time % Count BasicSearchEngine.searchAllTypeNames 4,402 1 ... + DiskIndex.initialize(boolean) 1,301 29.6% 105 + RandomAccessFile.<init>(File, String) 949 21.6% 105 + DiskIndex.readHeaderInfo(RandomAccessFile) 144 3.3% 100 + RandomAccessFile.readInt() 54 1.2% 1,421 + ClassLoader.loadClassInternal(String) 51 1.2% 2 + RandomAccessFile.readUTF() 7 <0.01% 797 + HashtableOfIntValues.put(char[], int) 3 <0.01% 1,594 + SimpleSetOfCharArray.get(char[]) 3 <0.01% 797 + String.toCharArray() 1 <0.01% 797 + RandomAccessFile.readUTF() 46 1.0% 105 + RandomAccessFile.close() 2 0 % 105 + File.exists() 1 0 % 105 I wasn't expecting much of an improvement. It may be more noticeable when you have more indexes. I have a fairly small workspace and have about 160. My workspace at work is about 3x the size (400 indexes). Did you use sampling or tracing to get your profile? The almost 1s for creating RandomAccessFiles looks a bit odd. I wrote a simple program to create a new RandomAccessFile on each file in my index and then close it. Total time was about 7ms (this was not under a profiler). Although it may be hitting a FS cache or something. public class Main { public static void main(String[] args) { File f = new File("D:\\data\\eclipse\\workspace\\.metadata\\.plugins\\org.eclipse.jdt.core"); File[] children = f.listFiles(); long start = System.nanoTime(); for (int i = 0; i < children.length; i++) { try { RandomAccessFile a = new RandomAccessFile(children[i], "r"); a.close(); } catch (Exception e) { } } } } Will continue to work on this improvement in next release... Created attachment 71307 [details] Proposed patch This patch replace the RandomAccessFile usage by a FileInputStream and so can take advantage of the optimization done in DiskIndex while fixing bug 171653... I forgot to precise that posted patch passes all JDT/Core and JDT/UI tests + FullSourceWorkspaceCompleteSearchTests + several massive requests in a 3.3 and Europa full binaries/source workspaces... Released for 3.4M1 in HEAD stream. Reopening since this change caused bug 195091. Patch as been reverted in HEAD. Problem came from the fact that DiskIndex internal chars buffer index was not correctly set at the beginning. Reading of index file signature was invalid and so always generate the excpetion of wrong format hence the indexing job at each startup... :-( Created attachment 72969 [details]
New proposed patch
Released for 3.4M1 in HEAD stream. Verified (based on code) for 3.4M1 using build I20070802-0800. |