Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cross-project-issues-dev] Eclipse SDK I build mirrors?


We run I-Builds on Tuesdays at 8am.  If some cases there are rebuilds later in the week to address issues identified in the first build. During milestone week, we run four or five on build days.

Martin, if you are interested in downloading an IBuild, another alternative would be for you  to implement an rsync script that downloads the I-build automatically and stores it in a common location.  Using the exclude or include you could only download the drops you need for testing.  This is what we do for our teams in Europe.

Kim



Denis Roy <denis.roy@xxxxxxxxxxx>
Sent by: cross-project-issues-dev-bounces@xxxxxxxxxxx

03/07/2007 09:27 AM

Please respond to
Cross project issues <cross-project-issues-dev@xxxxxxxxxxx>

To
Cross project issues <cross-project-issues-dev@xxxxxxxxxxx>
cc
Subject
Re: [cross-project-issues-dev] Eclipse SDK I build mirrors?





Kim, what is the frequency of an I-build?  I see it's about 2.2G to 2.7G in size.

Martin,fetching with SFTP/SCP will definitely be faster as it's part of the SSH QoS. We don't typically have any interactive users on eclipse.org, so this shouldn't be a problem.  Besides, if you're only puling the SDK, it shouldn't take too long.  Go ahead and use SFTP or SCP.

Also, as we have recently increased our total bandwidth, I'll increase the portion of bandwidth available for the two QoS pools.

Denis

Oberhuber, Martin wrote:

Hi Denis,

Can you point us to better ways of accessing I-builds?

It's been disucssed in a recent Europa meeting that projects should
run automated tests against both Platform Milestones and Platform
I-builds.

Downloading an I-build for testing can be really sluggish occasionally.
When I was trying to get I20070228 recently, it took more than 30
Minutes just to get eclipse-SDK-win32. I used the manual download
>From the web browser for this, and tend to use wget from scripts.

Such slow access to the I-builds can be a real problem when the
Test window is narrow during a milestone week, so I'm looking
For better ways - for instance:
- other protocol with QoS? Sftp may not be appropriate since
 it would hinder interactive users on eclipse.org, but perhaps
 FTP could be privileged since FTP to download.eclipse.org
 seems to be used by insiders only? Perhaps FTP access could
 even be limited to I-builds only, but included tools and
 other projects in addition to Platform only?
- Bittorrents for I-builds? Does anybody know how these could
 be accessed by a non-interactive scripted client?
- rsync or similar?

Currently, I'm more looking at an improved way of "pulling"
Parts of an I-build efficiently, rather than "pushing" the
Entire 2.6GB dataset to us since we'll typically only need
Small parts (and not from every I-build).

Thanks,
Martin


-----Original Message-----
From:
cross-project-issues-dev-bounces@xxxxxxxxxxx
[
mailto:cross-project-issues-dev-bounces@xxxxxxxxxxx] On Behalf Of Denis
Roy
Sent: Sunday, March 04, 2007 5:17 PM
To: Cross project issues
Subject: Re: [cross-project-issues-dev] Eclipse SDK I build mirrors?

Kim is right - we had a noticeable bandwidth *decrease* when we applied
the Nightly/I-build filter for the mirrors - especially during weeks
where Platform (or any other project) runs builds back-to-back.

> Should *some* I builds be mirrored, then?

It depends to whom. The IBM and a few other internal mirrors get the
complete dataset. I assume they are the largest consumers of N and I
builds, so we're saving bandwidth there.  I also assme that when an
individual consumes a nightly build, they're not downloading the entire
2.6GB dataset, but only a fraction of it -- therefore, many, many
downloads need to occur before mirroring 2.6GB to X mirrors becomes
profitable.

D.



Kim Moir wrote:
 

I don't know.  Denis would have to look at the numbers .  Each of our
I builds is about 2.6GB of with perf test results so that is quite a
bit of data to send the mirrors.

Kim




*Nick Boldt/Toronto/IBM@IBMCA*
Sent by:
cross-project-issues-dev-bounces@xxxxxxxxxxx

03/04/2007 07:50 PM
Please respond to
Cross project issues
<cross-project-issues-dev@xxxxxxxxxxx>


               
To
                "Cross project issues"
<cross-project-issues-dev@xxxxxxxxxxx>
cc
               
Subject
                Re: [cross-project-issues-dev] Eclipse SDK I build mirrors?


               





That's what I figured... though at the same time, there's recently
been the suggestion that projects should be testing w/ the platform
more often than simply milestone to milestone in order to avoid
   

respins.
 

Should *some* I builds be mirrored, then?

N

On 3/4/07, *Kim Moir* <_Kim_Moir@xxxxxx.com_
<mailto:Kim_Moir@xxxxxxxxxx>> wrote:

Nick,

Denis  determined a while ago that the bandwidth cost of sending out
all the N and I builds to the mirrors was not  worth it.  Not that
many people download the I and N builds from the mirrors compared to
the cost of sending all that date out every day.  I think he announced
   


 

it on one of his webmaster updates.

Kim



*Nick Boldt/Toronto/IBM@IBMCA*
Sent by: _cross-project-issues-dev-bounces@xxxxxxxxxxx _
<mailto:cross-project-issues-dev-bounces@xxxxxxxxxxx>

03/04/2007 07:20 PM
Please respond to
Cross project issues <_cross-project-issues-dev@eclipse.org_
<mailto:cross-project-issues-dev@xxxxxxxxxxx>>

               
To
                "Cross project issues" <_cross-project-issues-dev@xxxxxxxxxxx _
<mailto:cross-project-issues-dev@xxxxxxxxxxx>>
cc
               
Subject
                [cross-project-issues-dev] Eclipse SDK I build mirrors?



               






Forgive me if things have always been this way, but I just noticed
that there are no mirrors for Eclipse SDK I builds.

For example:
_
__
http://www.eclipse.org/downloads/download.php?file=/eclipse/download
s/drops/I20070228-0930/eclipse-SDK-I20070228-0930-linux-gtk.tar.gz
_
<http://www.eclipse.org/downloads/download.php?file=/eclipse/downloads
/drops/I20070228-0930/eclipse-SDK-I20070228-0930-linux-gtk.tar.gz>
_
__
http://www.eclipse.org/downloads/download.php?file=/eclipse/download
s/drops/I20070228-0930/eclipse-SDK-I20070228-0930-win32.zip
_
<http://www.eclipse.org/downloads/download.php?file=/eclipse/downloads
/drops/I20070228-0930/eclipse-SDK-I20070228-0930-win32.zip>


Rather than open a bug, I thought I'd just ask a question here.

Why aren't there mirrors for I builds? And is the traffic hitting_
__eclipse.org_
<http://eclipse.org/> for these non-Milestone,
non-Release builds small enough that it really doesn't matter?

Cheers,

--
Nick Boldt :: Release Engineer, IBM Toronto Lab Eclipse Modeling ::
_
http://www.eclipse.org/modeling__
__
http://wiki.eclipse.org/index.php/User:Nickb_
_______________________________________________
cross-project-issues-dev mailing list_
__cross-project-issues-dev@eclipse.org_
<mailto:cross-project-issues-dev@xxxxxxxxxxx>_
__
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_


_______________________________________________
cross-project-issues-dev mailing list_
__cross-project-issues-dev@xxxxxxxxxxx _
<mailto:cross-project-issues-dev@xxxxxxxxxxx>_
__
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_

_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev


----------------------------------------------------------------------
--

_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
   


_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev

 


--
Denis Roy
Manager, IT Infrastructure
Eclipse Foundation, Inc.  --  
http://www.eclipse.org/
Office: 613.224.9461 x224 (Eastern time)
Cell: 819.210.6481
denis.roy@xxxxxxxxxxx_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev


Back to the top