platform-core-home/documents/3.1/large_scale_issues.html

Parent Directory Parent Directory | Revision Log Revision Log

Revision 1.3 - (download) (as text) (annotate)
Tue Nov 30 19:27:07 2004 UTC (4 years, 11 months ago) by johna
Branch: MAIN
Changes since 1.2: +1 -1 lines 
fixed broken link

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <title>Large-scale development issues</title>
  <link rel="stylesheet" href="http://dev.eclipse.org/default_style.css" type="text/css">
</head>
<body text="#000000" bgcolor="#FFFFFF">

<table width="100%">
<tr><td style="background:#0080C0"><b><span style="color:white">Large-scale development issues</span></b></td></tr>
</table>
<font size="-1">Last modified: November 30, 2004</font> 
<p>
One of the major development themes for Eclipse 3.1 is to improve support for 
"Large-scale development" in Eclipse. This includes improving collaboration for 
large, distributed teams, but it also encompasses support for large workspaces.
This document captures requirements submitted in bug reports, mailing lists,
and other discussions from people using Eclipse for large-scale project development.
Not all of these issues are committed to be solved in Eclipse 3.1, but this list presents,
in no particular order, a problem scope from which work items can be chosen. Some
of these items are already present on the Eclipse 3.1 plan, but are included here
for completeness.
</p>
<h3>1. Memory footprint</h3>
<p>
Eclipse imposes a significant RAM footprint when working with a large workspace.
Identify principal areas of memory consumption and explore opportunities to 
reduce current footprint.
<ul>
<li><img src="../../images/ok.gif"  height="10" width="10" border="0">
	Extension registry footprint.  Eclipse maintains the registry of plug-ins, extensions,
	and extension points in memory.  As the number of plug-ins and extensions
	grows, so does this footprint.  Convert the registry to a cache structure that
	stores infrequently referenced portions on disk and brings extensions into memory
	in a lazy and transient manner.
</li>
<li><img src="../../images/glass.gif"  height="10" width="10" border="0">
	Workspace tree footprint.  The workspace is represented in memory as a tree
	containing various data such as resource names, attributes, markers, etc.
	Explore reducing the amount of data stored in memory for each resource, and
	other optimizations such as uniquification of strings.
<li><img src="../../images/glass.gif"  height="10" width="10" border="0">
	Team/CVS metadata footprint.  The CVS plug-ins store significant information
	in memory about the synchronization and "dirty" state of each resource. Explore
	reducing the footprint of this data or using lazy caching to only bring this information
	into memory when needed.
<li><img src="../../images/glass.gif"  height="10" width="10" border="0">
	Message bundles. Most plug-ins store translated strings in <tt>ResourceBundle</tt>
	objects.  These bundles are not space-efficient, and often use lengthy string-based
	keys for message lookup.  Explore a more efficient representation, integer-based keys,
	or a disk-based bundle for infrequently used messages.
</li>
</ul>
<h3>2. Performance of I/O-bound operations</h3>
<p>
Large teams often store development artifacts (code, diagrams, documentation)
on a network file system in order to increase reliability, facilitate backup and restoring of data,
and to simplify integration and building. I/O-bound operations in Eclipse are typically
much slower in such environments.  Explore optimization of I/O-bound operations, 
and moving lengthy operations into a background thread.
<ul>
<li><img src="../../images/ok.gif"  height="10" width="10" border="0">
	Project creation. Creating a project at a file system location that contains
	a large number of existing files and folders requires significant I/O to discover
	all the files and to gather local information such as time stamps. This project
	discovery can be moved into a background thread.
</li>
<li><img src="../../images/progress.gif" height="5" width="14" border="0">
	Resource copy, move, and delete.  Most operations that act on trees of resources
	still cannot be run in the background.  When these operations take a long time,
	the user is forced to wait until they complete. These should be converted to "user" jobs
	that can optionally be run in the background.
</li>
<li>Recursive deletion.  Java provides no API for recursively deleting a directory
containing files and other directories. This means deleting a large resource tree
requires two native I/O calls per directory (one to list the children, one to delete), and
one native I/O call per file.  This particularly impacts compilation, which often needs
to delete large trees of resources in the output (bin) folder. Consider adding a native
method to improve recursive deletion performance.
</li>
</ul>
</p>
<h3>3. Project interchange</h3>
<p>
Eclipse has always emphasized first-class support for integration of repository tools,
and has treated repositories as the primary vehicle for code sharing among team members.
This leaves behind groups that either don't use a repository, or don't use a repository
that has Eclipse integration plug-ins. The Import/Export wizards are typically used
by such groups to share code.  Some improvements to the import and export tools
would them more powerful as a project interchange (sharing) mechanism.
<ul>
<li>
	Import multiple projects at once 
	(bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=22698">22698</a>).
	If you have unzipped, untarred, or checked out a large group of projects from a repository,
	there is no way to load them all into a workspace at once.  The current "existing project"
	import wizard only allows importing one project at a time.
</li>
<li>
	Import project inside zip file 
	(bugs <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=66798">66798</a>, 
		<a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=67808">67808</a>).
	The Export wizard allows you to export an entire project into a ZIP or JAR file.  The corresponding
	Import wizards don't allow you to import that ZIP or JAR file back into a workspace
	as a top-level project.  The user has to unzip the file and import as existing project,
	or create a new project with the same name and import the contents.  This should
	be made easier.  Similarly, it should be possible to import a ZIP containing multiple
	projects.
</li>
<li><img src="../../images/progress.gif" height="5" width="14" border="0">
	Rename project on import 
	(bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=40493">40493</a>).
	The "import existing project" wizard doesn't allow you to import a project but
	pick a different name for the project in the workspace.  This is often needed
	by users who check out projects from a repository into the file system, and then
	want to call them something different in the workspace (one common example
	is when working on multiple streams of the same project in a single workspace).
</li>
</ul>
</p>
<h3>4. Support for non-incremental builders</h3>
<p><img src="../../images/glass.gif"  height="10" width="10" border="0">
	The workspace builder infrastructure is designed primarily with efficient
	incremental compilers in mind.  Auto-build is turned on by default, and this
	is only realistic for fast builders.  The workspace should have support
	for inherently non-incremental or slower builders (such as C compilers and Ant-based builders).
	In particular, we need to support users working in a heterogeneous environment with 
	some fast incremental builders and some slow non-incremental builders, sometimes
	with both on the same project
	(bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=60803">60803</a>).
</p>
<h3>5. Improved working sets</h3>
<p>
With very large workspaces, working sets are often used to filter the amount of
information showed in various views, and for scoping long-running tasks such
as builds and searches. The current working set support has some problems:
<ul>
<li>Shared notion of a working set
	(bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=22328">22328</a>).  
	Each view has to be explicitly and manually 
	scoped to a given working set.  Each long running search or build also needs to have
	the working set manually chosen.  One particularly bad example is the Java browsing
	perspective, which has four views that each needs to have its working set specified manually.
	Consider adding a global notion of a current working set, or a current working set 
	per window.
</li>
<li>
	Dynamic working sets.  A working set is defined as a static list of elements.
	Consider adding mechanisms to make working sets more flexible, such as 
	wildcards (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=62646">62646</a>),
	exclusion (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=22362">22362</a>),
	and tracking project creations (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=15941">15941</a>)
	and moves (bug <a href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=15938">15938</a>).
	Aggregating or nesting working sets would also be useful for very large scale workspaces.
</li>
</ul>
</p>

<!-- LEGEND -->
<table border="0" cellspacing="5" cellpadding="2" width="100%">
	<tbody>
		<tr>
			<td align="left" valign="top" colspan="4" bgcolor="#0080c0"
				width="100%"><b><font face="Arial,Helvetica"><font color="#ffffff">Legend
			</font></font></b></td>
		</tr>
		<tr>
			<td valign="top"><img src="../../images/progress.gif" height="5" width="14" border="0"></td>
			<td width="50%">item is under development.</td>
			<td valign="top"><img src="../../images/glass.gif"  height="10" width="10" border="0"></td>
			<td width="50%">item is under investigation.</td>
		</tr>
		<tr>
			<td valign="top"><img src="../../images/ok.gif"  height="10" width="10" border="0"></td>
			<td width="50%">item is finished.</td>
			<td valign="top">( )</td>
			<td width="50%">item is time permitted.</td>
		</tr>
		<tr>
			<td valign="top"><font class="indexsub" color="#8080ff">[&gt;3.1]</font></td>
			<td width="50%">item is deferred.</td>
			<td valign="top"><font class="indexsub"><img border="0" src="../../images/new.gif"></font></td>
			<td width="50%">new</td>
		</tr>
	</tbody>
</table>
</body>
</html>