Bug 13069 - [CVS Core] Sending large files may result in excessive memory usage
Summary: [CVS Core] Sending large files may result in excessive memory usage
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Team (show other bugs)
Version: 2.0   Edit
Hardware: PC Windows 2000
: P3 normal (vote)
Target Milestone: 2.1 M5   Edit
Assignee: Boris Shingarov CLA
QA Contact:
URL:
Whiteboard:
Keywords: investigate, performance
Depends on:
Blocks:
 
Reported: 2002-04-09 10:37 EDT by Jeff Brown CLA
Modified: 2003-01-20 13:14 EST (History)
0 users

See Also:

Attachments
Purely streaming (size=O(1)) sendFile() (11.90 KB, patch)
2003-01-19 22:12 EST, Boris Shingarov CLA 		no flags Details | Diff
Better formatting of the above code (10.37 KB, patch)
2003-01-20 12:59 EST, Boris Shingarov CLA 		no flags Details | Diff
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Jeff Brown CLA 2002-04-09 10:37:57 EDT 
Session.sendFile() buffers entire files in memory when converting
line delimiters in text files, and/or GZIP'ing text or binary files.
Should place an upper bound on how large a file can be buffered.
Files larger than this size should be read from disk twice,
once to compute the final size, and once to send the contents.
Comment 1 Kevin McGuire CLA 2002-04-10 19:08:32 EDT 
post 2.0
Comment 2 Michael Valenta CLA 2002-09-08 20:12:39 EDT 
Reopening
Comment 3 Michael Valenta CLA 2002-09-08 20:14:41 EDT 
Since most text files are not huge (I've seen some as large as 100KB and some 
may be bigger but this is rare) this is not a pressing issue but it should 
still be addressed.
Comment 4 Dr. David J. Knowles CLA 2002-11-18 15:04:11 EST 
This is causing out of memory errors when comparing large binary files (>~1MB)
as the project contains even the binary files (jars etc) when using CVS to store
the entire project this means that it is not possible to determine if a file has
changed should the "consider file contents in comparisons" be checked in team
preferences.
Comment 5 Michael Valenta CLA 2002-11-18 15:58:13 EST 
We will investigate for 2.1. Does turning off compression on the CVS 
preferences page fix the problem?
Comment 6 Boris Shingarov CLA 2003-01-19 22:12:24 EST 
Created attachment 3036 [details]
Purely streaming (size=O(1)) sendFile()
Comment 7 Boris Shingarov CLA 2003-01-20 12:59:27 EST 
Created attachment 3045 [details]
Better formatting of the above code
Comment 8 Michael Valenta CLA 2003-01-20 13:14:53 EST 
Fix released to HEAD