platform-core-home/documents/3.1/large_scale_issues.html
Parent Directory
| 
Revision Log
Revision 1.9 - (view) (download) (as text)
| 1 : | johna | 1.1 | <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> |
| 2 : | <html> | ||
| 3 : | <head> | ||
| 4 : | <title>Large-scale development issues</title> | ||
| 5 : | <link rel="stylesheet" href="http://dev.eclipse.org/default_style.css" type="text/css"> | ||
| 6 : | </head> | ||
| 7 : | <body text="#000000" bgcolor="#FFFFFF"> | ||
| 8 : | |||
| 9 : | <table width="100%"> | ||
| 10 : | <tr><td style="background:#0080C0"><b><span style="color:white">Large-scale development issues</span></b></td></tr> | ||
| 11 : | </table> | ||
| 12 : | johna | 1.8 | <font size="-1">Last modified: February 24, 2005</font> |
| 13 : | johna | 1.1 | <p> |
| 14 : | One of the major development themes for Eclipse 3.1 is to improve support for | ||
| 15 : | "Large-scale development" in Eclipse. This includes improving collaboration for | ||
| 16 : | large, distributed teams, but it also encompasses support for large workspaces. | ||
| 17 : | This document captures requirements submitted in bug reports, mailing lists, | ||
| 18 : | and other discussions from people using Eclipse for large-scale project development. | ||
| 19 : | Not all of these issues are committed to be solved in Eclipse 3.1, but this list presents, | ||
| 20 : | in no particular order, a problem scope from which work items can be chosen. Some | ||
| 21 : | of these items are already present on the Eclipse 3.1 plan, but are included here | ||
| 22 : | for completeness. | ||
| 23 : | </p> | ||
| 24 : | <h3>1. Memory footprint</h3> | ||
| 25 : | <p> | ||
| 26 : | Eclipse imposes a significant RAM footprint when working with a large workspace. | ||
| 27 : | Identify principal areas of memory consumption and explore opportunities to | ||
| 28 : | reduce current footprint. | ||
| 29 : | <ul> | ||
| 30 : | johna | 1.2 | <li><img src="../../images/ok.gif" height="10" width="10" border="0"> |
| 31 : | johna | 1.1 | Extension registry footprint. Eclipse maintains the registry of plug-ins, extensions, |
| 32 : | and extension points in memory. As the number of plug-ins and extensions | ||
| 33 : | grows, so does this footprint. Convert the registry to a cache structure that | ||
| 34 : | stores infrequently referenced portions on disk and brings extensions into memory | ||
| 35 : | in a lazy and transient manner. | ||
| 36 : | </li> | ||
| 37 : | johna | 1.6 | <li><img src="../../images/ok.gif" height="10" width="10" border="0"> |
| 38 : | johna | 1.1 | Workspace tree footprint. The workspace is represented in memory as a tree |
| 39 : | containing various data such as resource names, attributes, markers, etc. | ||
| 40 : | Explore reducing the amount of data stored in memory for each resource, and | ||
| 41 : | other optimizations such as uniquification of strings. | ||
| 42 : | johna | 1.9 | <li><img src="../../images/ok.gif" height="10" width="10" border="0"> |
| 43 : | johna | 1.1 | Team/CVS metadata footprint. The CVS plug-ins store significant information |
| 44 : | in memory about the synchronization and "dirty" state of each resource. Explore | ||
| 45 : | reducing the footprint of this data or using lazy caching to only bring this information | ||
| 46 : | into memory when needed. | ||
| 47 : | johna | 1.8 | <li><img src="../../images/ok.gif" height="10" width="10" border="0"> |
| 48 : | Message bundles (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=37712">37712</a>). | ||
| 49 : | Most plug-ins store translated strings in <tt>ResourceBundle</tt> | ||
| 50 : | johna | 1.1 | objects. These bundles are not space-efficient, and often use lengthy string-based |
| 51 : | keys for message lookup. Explore a more efficient representation, integer-based keys, | ||
| 52 : | or a disk-based bundle for infrequently used messages. | ||
| 53 : | </li> | ||
| 54 : | </ul> | ||
| 55 : | <h3>2. Performance of I/O-bound operations</h3> | ||
| 56 : | <p> | ||
| 57 : | Large teams often store development artifacts (code, diagrams, documentation) | ||
| 58 : | on a network file system in order to increase reliability, facilitate backup and restoring of data, | ||
| 59 : | and to simplify integration and building. I/O-bound operations in Eclipse are typically | ||
| 60 : | much slower in such environments. Explore optimization of I/O-bound operations, | ||
| 61 : | and moving lengthy operations into a background thread. | ||
| 62 : | <ul> | ||
| 63 : | johna | 1.2 | <li><img src="../../images/ok.gif" height="10" width="10" border="0"> |
| 64 : | johna | 1.1 | Project creation. Creating a project at a file system location that contains |
| 65 : | a large number of existing files and folders requires significant I/O to discover | ||
| 66 : | all the files and to gather local information such as time stamps. This project | ||
| 67 : | discovery can be moved into a background thread. | ||
| 68 : | </li> | ||
| 69 : | johna | 1.8 | <li><img src="../../images/ok.gif" height="10" width="10" border="0"> |
| 70 : | johna | 1.1 | Resource copy, move, and delete. Most operations that act on trees of resources |
| 71 : | still cannot be run in the background. When these operations take a long time, | ||
| 72 : | the user is forced to wait until they complete. These should be converted to "user" jobs | ||
| 73 : | that can optionally be run in the background. | ||
| 74 : | </li> | ||
| 75 : | johna | 1.8 | <li><font class="indexsub" color="#8080ff">[>3.1]</font> |
| 76 : | Recursive deletion (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=10628">10628</a>). | ||
| 77 : | Java provides no API for recursively deleting a directory | ||
| 78 : | johna | 1.1 | containing files and other directories. This means deleting a large resource tree |
| 79 : | requires two native I/O calls per directory (one to list the children, one to delete), and | ||
| 80 : | one native I/O call per file. This particularly impacts compilation, which often needs | ||
| 81 : | to delete large trees of resources in the output (bin) folder. Consider adding a native | ||
| 82 : | method to improve recursive deletion performance. | ||
| 83 : | </li> | ||
| 84 : | </ul> | ||
| 85 : | </p> | ||
| 86 : | <h3>3. Project interchange</h3> | ||
| 87 : | <p> | ||
| 88 : | Eclipse has always emphasized first-class support for integration of repository tools, | ||
| 89 : | and has treated repositories as the primary vehicle for code sharing among team members. | ||
| 90 : | This leaves behind groups that either don't use a repository, or don't use a repository | ||
| 91 : | that has Eclipse integration plug-ins. The Import/Export wizards are typically used | ||
| 92 : | by such groups to share code. Some improvements to the import and export tools | ||
| 93 : | would them more powerful as a project interchange (sharing) mechanism. | ||
| 94 : | <ul> | ||
| 95 : | johna | 1.4 | <li><img src="../../images/ok.gif" height="10" width="10" border="0"> |
| 96 : | johna | 1.1 | Import multiple projects at once |
| 97 : | (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=22698">22698</a>). | ||
| 98 : | If you have unzipped, untarred, or checked out a large group of projects from a repository, | ||
| 99 : | there is no way to load them all into a workspace at once. The current "existing project" | ||
| 100 : | import wizard only allows importing one project at a time. | ||
| 101 : | </li> | ||
| 102 : | johna | 1.9 | <li><img src="../../images/ok.gif" height="10" width="10" border="0"> |
| 103 : | johna | 1.1 | Import project inside zip file |
| 104 : | (bugs <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=66798">66798</a>, | ||
| 105 : | <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=67808">67808</a>). | ||
| 106 : | The Export wizard allows you to export an entire project into a ZIP or JAR file. The corresponding | ||
| 107 : | Import wizards don't allow you to import that ZIP or JAR file back into a workspace | ||
| 108 : | as a top-level project. The user has to unzip the file and import as existing project, | ||
| 109 : | or create a new project with the same name and import the contents. This should | ||
| 110 : | be made easier. Similarly, it should be possible to import a ZIP containing multiple | ||
| 111 : | projects. | ||
| 112 : | </li> | ||
| 113 : | johna | 1.9 | <li><font class="indexsub" color="#8080ff">[>3.1]</font> |
| 114 : | johna | 1.1 | Rename project on import |
| 115 : | (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=40493">40493</a>). | ||
| 116 : | The "import existing project" wizard doesn't allow you to import a project but | ||
| 117 : | pick a different name for the project in the workspace. This is often needed | ||
| 118 : | by users who check out projects from a repository into the file system, and then | ||
| 119 : | want to call them something different in the workspace (one common example | ||
| 120 : | is when working on multiple streams of the same project in a single workspace). | ||
| 121 : | </li> | ||
| 122 : | </ul> | ||
| 123 : | </p> | ||
| 124 : | <h3>4. Support for non-incremental builders</h3> | ||
| 125 : | johna | 1.7 | <p><img src="../../images/ok.gif" height="10" width="10" border="0"> |
| 126 : | johna | 1.1 | The workspace builder infrastructure is designed primarily with efficient |
| 127 : | incremental compilers in mind. Auto-build is turned on by default, and this | ||
| 128 : | is only realistic for fast builders. The workspace should have support | ||
| 129 : | for inherently non-incremental or slower builders (such as C compilers and Ant-based builders). | ||
| 130 : | In particular, we need to support users working in a heterogeneous environment with | ||
| 131 : | some fast incremental builders and some slow non-incremental builders, sometimes | ||
| 132 : | with both on the same project | ||
| 133 : | (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=60803">60803</a>). | ||
| 134 : | johna | 1.5 | Read the <a href="large_scale_builders.html">proposal</a>. |
| 135 : | johna | 1.1 | </p> |
| 136 : | <h3>5. Improved working sets</h3> | ||
| 137 : | <p> | ||
| 138 : | With very large workspaces, working sets are often used to filter the amount of | ||
| 139 : | information showed in various views, and for scoping long-running tasks such | ||
| 140 : | as builds and searches. The current working set support has some problems: | ||
| 141 : | <ul> | ||
| 142 : | <li>Shared notion of a working set | ||
| 143 : | (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=22328">22328</a>). | ||
| 144 : | Each view has to be explicitly and manually | ||
| 145 : | scoped to a given working set. Each long running search or build also needs to have | ||
| 146 : | the working set manually chosen. One particularly bad example is the Java browsing | ||
| 147 : | perspective, which has four views that each needs to have its working set specified manually. | ||
| 148 : | Consider adding a global notion of a current working set, or a current working set | ||
| 149 : | per window. | ||
| 150 : | </li> | ||
| 151 : | <li> | ||
| 152 : | Dynamic working sets. A working set is defined as a static list of elements. | ||
| 153 : | Consider adding mechanisms to make working sets more flexible, such as | ||
| 154 : | wildcards (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=62646">62646</a>), | ||
| 155 : | exclusion (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=22362">22362</a>), | ||
| 156 : | and tracking project creations (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=15941">15941</a>) | ||
| 157 : | johna | 1.3 | and moves (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=15938">15938</a>). |
| 158 : | johna | 1.1 | Aggregating or nesting working sets would also be useful for very large scale workspaces. |
| 159 : | </li> | ||
| 160 : | </ul> | ||
| 161 : | </p> | ||
| 162 : | |||
| 163 : | <!-- LEGEND --> | ||
| 164 : | <table border="0" cellspacing="5" cellpadding="2" width="100%"> | ||
| 165 : | <tbody> | ||
| 166 : | <tr> | ||
| 167 : | <td align="left" valign="top" colspan="4" bgcolor="#0080c0" | ||
| 168 : | width="100%"><b><font face="Arial,Helvetica"><font color="#ffffff">Legend | ||
| 169 : | </font></font></b></td> | ||
| 170 : | </tr> | ||
| 171 : | <tr> | ||
| 172 : | johna | 1.2 | <td valign="top"><img src="../../images/progress.gif" height="5" width="14" border="0"></td> |
| 173 : | johna | 1.1 | <td width="50%">item is under development.</td> |
| 174 : | johna | 1.2 | <td valign="top"><img src="../../images/glass.gif" height="10" width="10" border="0"></td> |
| 175 : | johna | 1.1 | <td width="50%">item is under investigation.</td> |
| 176 : | </tr> | ||
| 177 : | <tr> | ||
| 178 : | johna | 1.2 | <td valign="top"><img src="../../images/ok.gif" height="10" width="10" border="0"></td> |
| 179 : | johna | 1.1 | <td width="50%">item is finished.</td> |
| 180 : | <td valign="top">( )</td> | ||
| 181 : | <td width="50%">item is time permitted.</td> | ||
| 182 : | </tr> | ||
| 183 : | <tr> | ||
| 184 : | <td valign="top"><font class="indexsub" color="#8080ff">[>3.1]</font></td> | ||
| 185 : | <td width="50%">item is deferred.</td> | ||
| 186 : | johna | 1.2 | <td valign="top"><font class="indexsub"><img border="0" src="../../images/new.gif"></font></td> |
| 187 : | johna | 1.1 | <td width="50%">new</td> |
| 188 : | </tr> | ||
| 189 : | </tbody> | ||
| 190 : | </table> | ||
| 191 : | </body> | ||
| 192 : | </html> |
| help@eclipse.org | ViewVC Help |
| Powered by ViewVC 1.0.3 |