platform-core-home/documents/3.1/large_scale_issues.html

Parent Directory Parent Directory | Revision Log Revision Log

Revision 1.5 - (view) (download) (as text)

1 : johna 1.1 <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
2 :     <html>
3 :     <head>
4 :     <title>Large-scale development issues</title>
5 :     <link rel="stylesheet" href="http://dev.eclipse.org/default_style.css" type="text/css">
6 :     </head>
7 :     <body text="#000000" bgcolor="#FFFFFF">
8 :    
9 :     <table width="100%">
10 :     <tr><td style="background:#0080C0"><b><span style="color:white">Large-scale development issues</span></b></td></tr>
11 :     </table>
12 : johna 1.5 <font size="-1">Last modified: December 13, 2004</font>
13 : johna 1.1 <p>
14 :     One of the major development themes for Eclipse 3.1 is to improve support for
15 :     "Large-scale development" in Eclipse. This includes improving collaboration for
16 :     large, distributed teams, but it also encompasses support for large workspaces.
17 :     This document captures requirements submitted in bug reports, mailing lists,
18 :     and other discussions from people using Eclipse for large-scale project development.
19 :     Not all of these issues are committed to be solved in Eclipse 3.1, but this list presents,
20 :     in no particular order, a problem scope from which work items can be chosen. Some
21 :     of these items are already present on the Eclipse 3.1 plan, but are included here
22 :     for completeness.
23 :     </p>
24 :     <h3>1. Memory footprint</h3>
25 :     <p>
26 :     Eclipse imposes a significant RAM footprint when working with a large workspace.
27 :     Identify principal areas of memory consumption and explore opportunities to
28 :     reduce current footprint.
29 :     <ul>
30 : johna 1.2 <li><img src="../../images/ok.gif" height="10" width="10" border="0">
31 : johna 1.1 Extension registry footprint. Eclipse maintains the registry of plug-ins, extensions,
32 :     and extension points in memory. As the number of plug-ins and extensions
33 :     grows, so does this footprint. Convert the registry to a cache structure that
34 :     stores infrequently referenced portions on disk and brings extensions into memory
35 :     in a lazy and transient manner.
36 :     </li>
37 : johna 1.2 <li><img src="../../images/glass.gif" height="10" width="10" border="0">
38 : johna 1.1 Workspace tree footprint. The workspace is represented in memory as a tree
39 :     containing various data such as resource names, attributes, markers, etc.
40 :     Explore reducing the amount of data stored in memory for each resource, and
41 :     other optimizations such as uniquification of strings.
42 : johna 1.2 <li><img src="../../images/glass.gif" height="10" width="10" border="0">
43 : johna 1.1 Team/CVS metadata footprint. The CVS plug-ins store significant information
44 :     in memory about the synchronization and "dirty" state of each resource. Explore
45 :     reducing the footprint of this data or using lazy caching to only bring this information
46 :     into memory when needed.
47 : johna 1.2 <li><img src="../../images/glass.gif" height="10" width="10" border="0">
48 : johna 1.1 Message bundles. Most plug-ins store translated strings in <tt>ResourceBundle</tt>
49 :     objects. These bundles are not space-efficient, and often use lengthy string-based
50 :     keys for message lookup. Explore a more efficient representation, integer-based keys,
51 :     or a disk-based bundle for infrequently used messages.
52 :     </li>
53 :     </ul>
54 :     <h3>2. Performance of I/O-bound operations</h3>
55 :     <p>
56 :     Large teams often store development artifacts (code, diagrams, documentation)
57 :     on a network file system in order to increase reliability, facilitate backup and restoring of data,
58 :     and to simplify integration and building. I/O-bound operations in Eclipse are typically
59 :     much slower in such environments. Explore optimization of I/O-bound operations,
60 :     and moving lengthy operations into a background thread.
61 :     <ul>
62 : johna 1.2 <li><img src="../../images/ok.gif" height="10" width="10" border="0">
63 : johna 1.1 Project creation. Creating a project at a file system location that contains
64 :     a large number of existing files and folders requires significant I/O to discover
65 :     all the files and to gather local information such as time stamps. This project
66 :     discovery can be moved into a background thread.
67 :     </li>
68 : johna 1.2 <li><img src="../../images/progress.gif" height="5" width="14" border="0">
69 : johna 1.1 Resource copy, move, and delete. Most operations that act on trees of resources
70 :     still cannot be run in the background. When these operations take a long time,
71 :     the user is forced to wait until they complete. These should be converted to "user" jobs
72 :     that can optionally be run in the background.
73 :     </li>
74 :     <li>Recursive deletion. Java provides no API for recursively deleting a directory
75 :     containing files and other directories. This means deleting a large resource tree
76 :     requires two native I/O calls per directory (one to list the children, one to delete), and
77 :     one native I/O call per file. This particularly impacts compilation, which often needs
78 :     to delete large trees of resources in the output (bin) folder. Consider adding a native
79 :     method to improve recursive deletion performance.
80 :     </li>
81 :     </ul>
82 :     </p>
83 :     <h3>3. Project interchange</h3>
84 :     <p>
85 :     Eclipse has always emphasized first-class support for integration of repository tools,
86 :     and has treated repositories as the primary vehicle for code sharing among team members.
87 :     This leaves behind groups that either don't use a repository, or don't use a repository
88 :     that has Eclipse integration plug-ins. The Import/Export wizards are typically used
89 :     by such groups to share code. Some improvements to the import and export tools
90 :     would them more powerful as a project interchange (sharing) mechanism.
91 :     <ul>
92 : johna 1.4 <li><img src="../../images/ok.gif" height="10" width="10" border="0">
93 : johna 1.1 Import multiple projects at once
94 :     (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=22698">22698</a>).
95 :     If you have unzipped, untarred, or checked out a large group of projects from a repository,
96 :     there is no way to load them all into a workspace at once. The current "existing project"
97 :     import wizard only allows importing one project at a time.
98 :     </li>
99 :     <li>
100 :     Import project inside zip file
101 :     (bugs <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=66798">66798</a>,
102 :     <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=67808">67808</a>).
103 :     The Export wizard allows you to export an entire project into a ZIP or JAR file. The corresponding
104 :     Import wizards don't allow you to import that ZIP or JAR file back into a workspace
105 :     as a top-level project. The user has to unzip the file and import as existing project,
106 :     or create a new project with the same name and import the contents. This should
107 :     be made easier. Similarly, it should be possible to import a ZIP containing multiple
108 :     projects.
109 :     </li>
110 : johna 1.2 <li><img src="../../images/progress.gif" height="5" width="14" border="0">
111 : johna 1.1 Rename project on import
112 :     (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=40493">40493</a>).
113 :     The "import existing project" wizard doesn't allow you to import a project but
114 :     pick a different name for the project in the workspace. This is often needed
115 :     by users who check out projects from a repository into the file system, and then
116 :     want to call them something different in the workspace (one common example
117 :     is when working on multiple streams of the same project in a single workspace).
118 :     </li>
119 :     </ul>
120 :     </p>
121 :     <h3>4. Support for non-incremental builders</h3>
122 : johna 1.2 <p><img src="../../images/glass.gif" height="10" width="10" border="0">
123 : johna 1.1 The workspace builder infrastructure is designed primarily with efficient
124 :     incremental compilers in mind. Auto-build is turned on by default, and this
125 :     is only realistic for fast builders. The workspace should have support
126 :     for inherently non-incremental or slower builders (such as C compilers and Ant-based builders).
127 :     In particular, we need to support users working in a heterogeneous environment with
128 :     some fast incremental builders and some slow non-incremental builders, sometimes
129 :     with both on the same project
130 :     (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=60803">60803</a>).
131 : johna 1.5 Read the <a href="large_scale_builders.html">proposal</a>.
132 : johna 1.1 </p>
133 :     <h3>5. Improved working sets</h3>
134 :     <p>
135 :     With very large workspaces, working sets are often used to filter the amount of
136 :     information showed in various views, and for scoping long-running tasks such
137 :     as builds and searches. The current working set support has some problems:
138 :     <ul>
139 :     <li>Shared notion of a working set
140 :     (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=22328">22328</a>).
141 :     Each view has to be explicitly and manually
142 :     scoped to a given working set. Each long running search or build also needs to have
143 :     the working set manually chosen. One particularly bad example is the Java browsing
144 :     perspective, which has four views that each needs to have its working set specified manually.
145 :     Consider adding a global notion of a current working set, or a current working set
146 :     per window.
147 :     </li>
148 :     <li>
149 :     Dynamic working sets. A working set is defined as a static list of elements.
150 :     Consider adding mechanisms to make working sets more flexible, such as
151 :     wildcards (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=62646">62646</a>),
152 :     exclusion (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=22362">22362</a>),
153 :     and tracking project creations (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=15941">15941</a>)
154 : johna 1.3 and moves (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=15938">15938</a>).
155 : johna 1.1 Aggregating or nesting working sets would also be useful for very large scale workspaces.
156 :     </li>
157 :     </ul>
158 :     </p>
159 :    
160 :     <!-- LEGEND -->
161 :     <table border="0" cellspacing="5" cellpadding="2" width="100%">
162 :     <tbody>
163 :     <tr>
164 :     <td align="left" valign="top" colspan="4" bgcolor="#0080c0"
165 :     width="100%"><b><font face="Arial,Helvetica"><font color="#ffffff">Legend
166 :     </font></font></b></td>
167 :     </tr>
168 :     <tr>
169 : johna 1.2 <td valign="top"><img src="../../images/progress.gif" height="5" width="14" border="0"></td>
170 : johna 1.1 <td width="50%">item is under development.</td>
171 : johna 1.2 <td valign="top"><img src="../../images/glass.gif" height="10" width="10" border="0"></td>
172 : johna 1.1 <td width="50%">item is under investigation.</td>
173 :     </tr>
174 :     <tr>
175 : johna 1.2 <td valign="top"><img src="../../images/ok.gif" height="10" width="10" border="0"></td>
176 : johna 1.1 <td width="50%">item is finished.</td>
177 :     <td valign="top">( )</td>
178 :     <td width="50%">item is time permitted.</td>
179 :     </tr>
180 :     <tr>
181 :     <td valign="top"><font class="indexsub" color="#8080ff">[&gt;3.1]</font></td>
182 :     <td width="50%">item is deferred.</td>
183 : johna 1.2 <td valign="top"><font class="indexsub"><img border="0" src="../../images/new.gif"></font></td>
184 : johna 1.1 <td width="50%">new</td>
185 :     </tr>
186 :     </tbody>
187 :     </table>
188 :     </body>
189 :     </html>