Community
Participate
Working Groups
In the past we've had some good reasons not to enable automatic updates by default, e.g. from my own memory - it always opens a network connection without asking the user - it creates a lot of network traffic/requests, for the user, but also for the eclipse.org servers - there's no control of what other bundles are installed by end users, and all those bundles introduce additional dependencies; this may lead very often to situations where an update isn't possible at all - we want the users have full control - ...and probably many more reasons. Some of these arguments are still valid, but with p2 being stable and with Pascal's changes in 4.3/Kepler I think it is time to think about it again. Others (e.g. Firefox, Linux Distros, ...) are automatically looking for updates successfully, and Eclipse shouldn't be different here. I'd propose to enable "Automatically find new updates and notify me" in all EPP packages. What's not clear to me are the other settings, especially the schedule. I'm tempted to try it with "Look for updates each time Eclipse is started" but that may impact server performance too much.
Created attachment 237477 [details] downloads 2013-10-29 This statistics was a surprise to me. I tried to assemble some download numbers at the time of EclipseCon Europe. The interesting part is the nearly invisible yellow bar: It is the number of packages that has been updated from Kepler Release to Kepler SR1. My expectation was that it is about half of the blue bar, but it seems to me that no one updates Eclipse, but everyone downloads a fresh copy (SR1).
More information about the preferences and their possible settings - see [1] Adding the following snippet to the RCP/RAP package has been tested successfully by myself, but I wouldn't use such aggressive reminder settings; in my opinion "Notify me once about updates" should be enough. # automatic update options are defined # in org.eclipse.equinox.p2.sdk.scheduler.PreferenceConstants # check for updates on startup org.eclipse.equinox.p2.ui.sdk.scheduler/enabled=true org.eclipse.equinox.p2.ui.sdk.scheduler/schedule=on-startup # remind the user every 4 hours org.eclipse.equinox.p2.ui.sdk.scheduler/remindOnSchedule=true # see AutomaticUpdatesPopup, values can be "30 minutes", "Hour", "4 Hours" org.eclipse.equinox.p2.ui.sdk.scheduler/remindElapsedTime=4 Hours # download updates before notifying the user #org.eclipse.equinox.p2.ui.sdk.scheduler/download=true [1] http://help.eclipse.org/kepler/index.jsp?topic=%2Forg.eclipse.platform.doc.isv%2Fguide%2Fp2_director.html
I'm trying to get feedback from the Platform team... if it would be possible to change the default setting in the Platform itself. And maybe the p2 team and the webmasters have additional feedback? If it is not possible to be changed in Platform, I'd be willing to change it in EPP, because I believe that every application should have auto-update enabled by default.
Thanks for the CC. > Others (e.g. Firefox, Linux Distros, ...) are automatically looking for > updates successfully, and Eclipse shouldn't be different here. I agree with that assessment -- nowadays it's quite standard for apps to inform their users that there are updates available. > I'd propose to enable "Automatically find new updates and notify me" in all > EPP packages. +1 > What's not clear to me are the other settings, especially the schedule. I'm > tempted to try it with "Look for updates each time Eclipse is started" but > that may impact server performance too much. I'd be concerned about that too. What does "look for updates" mean? What does it fetch? What other scheduling options are there?
(In reply to Denis Roy from comment #4) > > I'd propose to enable "Automatically find new updates and notify me" in all > > EPP packages. > > +1 +1
(Note that enabling automatic updates does not solve the problem that only root IUs are updated (bug 311319). But the good news is that we've re-started a discussion in bug 332989 on how to mitigate and/or solve this problem by restructuring the EPP packages.)
FYI - The p2 automatic update is a very expensive operation, even if turned of. See Bug 443809.
> FYI - The p2 automatic update is a very expensive operation, even if turned > of. See Bug 443809. So you are making the case to active auto-update right?
(In reply to Pascal Rapicault from comment #8) > > FYI - The p2 automatic update is a very expensive operation, even if turned > > of. See Bug 443809. > So you are making the case to active auto-update right? I have not measured if the execution of the code takes longer if activated, but I would assume so.
(In reply to Denis Roy from comment #4) > > What's not clear to me are the other settings, especially the schedule. I'm > > tempted to try it with "Look for updates each time Eclipse is started" but > > that may impact server performance too much. > > I'd be concerned about that too. What does "look for updates" mean? What > does it fetch? What other scheduling options are there? p2 team, please correct if I'm wrong, or add if I'm missing details. What does "look for udpates" mean? What does it fetch? -> When Eclipse is looking for updates it would download the p2 repository metadata of all referenced, activated p2 repositories first, i.e. it would use GET (or HEAD if cached) requests for the files p2.index, content.jar, content.xml, compositeContent.jar, compositeContent.xml until it successfully downloads one of the *ontent* files. When this is done it does the same for artifacts.jar, artifacts.xml, compositeArtifacts.jar, and finally compositeArtifacts.xml. Everything is aggressively cached, e.g. the p2.index files are only read once. -> By default the minimal set of p2 repositories in the EPP packages is * /releases/luna which is a composite repository pointing to R/SR1/SR1a/... repositories plus the EPP p2 repository (which is aggregated into a single one for performance reasons). * /eclipse/updates/4.4 which is another composite repository for all service releases in that stream. -> If I'm counting right this means (without caching!): * after the first 'R' release in June: 11 + 9 = 20 requests * after SR1: (11+4) + (9+4) = 28 requests * after SR2: (11+4+4) + (9+4+4) = 36 requests or in the case of Luna (again, without caching) that sums up to 3676 kilobytes for 'R' and 7644 kilobytes for SR1. In other words every single release of these two p2 repositories adds another 4 MB. These requests do not use the Eclipse mirrors, only download.eclipse.org. That's for sure an amount that we need to take into account even if lots of these requests are cached, especially because every user adds more p2 repositories to Eclipse over time. What other scheduling options are there? -> Look for updates each time Eclipse is started -> Look for updates... every day at a specified time -> Look for updates... every week on a specified day and time I'd like to see an option that searches for updates once a week, but not all Eclipse installations of the world on the same weekday at the same time together.
> p2 team, please correct if I'm wrong, or add if I'm missing details. > > What does "look for udpates" mean? What does it fetch? When looking for an update, p2 only downloads the p2.index and the metadata repositories (composite content, content). The artifact repositories are only downloaded when artifacts need to be downloaded, so in this case this would be only when the installation needs to be updated. Also, as you mention, these files (to the exception of the p2.index) are cached locally. So they should only be retrieved once. > or in the case of Luna (again, without caching) that sums up to 3676 > kilobytes for 'R' and 7644 kilobytes for SR1. In other words every single > release of these two p2 repositories adds another 4 MB. These requests do > not use the Eclipse mirrors, only download.eclipse.org. That's for sure an > amount that we need to take into account even if lots of these requests are > cached, especially because every user adds more p2 repositories to Eclipse > over time. Note, that when we ship SR1, we could try to pre-cache the SR0 files in the distro thus off-loading work from the server. The other thing that could be tried is for the foundation to automatically re-route those requests to a mirror. However, this has been tried a couple years ago and caused a variety of problems, so if we wanted to be able to put that in place, we would have to do modifications to p2 to checksum the various files, or only mirror to sites that we can trust have pristine copies of the files (e.g. we could try bintray). Btw, do you know if this question has been brought up to the attention of the people working on the creation of the Eclipse installer? > I'd like to see an option that searches for updates once a week, but not all > Eclipse installations of the world on the same weekday at the same time > together. We could try something like the first time you start eclipse during a week? wdyt?
Thanks for your quick answers, Pascal! (In reply to Pascal Rapicault from comment #11) > ... Also, as you mention, these files (to the exception of the p2.index) are > cached locally. So they should only be retrieved once. When is the cache invalidated? Is this a TTL setting sent from the HTTP server? Or does it always start with a new HEAD request? > Note, that when we ship SR1, we could try to pre-cache the SR0 files in > the distro thus off-loading work from the server. First thought: Cool, very cool. Second thought: Does it really change the overall download size? Yes, to some degree. The packages would be a few MBytes larger, but this additional amount of data would be downloaded from the Eclipse mirrors, not from the main download server. > The other thing that could be tried is for the foundation to automatically > re-route those requests to a mirror. If content.jar and artifacts.jar were cryptographically signed this would be *the* solution. I don't know enough about bintray to judge if this could be a solution for offloading parts or all traffic. Statistics etc. shouldn't be an issue because for that reason we still have the HEAD requests. > Btw, do you know if this question has been brought up to the attention of > the people working on the creation of the Eclipse installer? No, I'm adding Ed and Eike. > > I'd like to see an option that searches for updates once a week, but not all > > Eclipse installations of the world on the same weekday at the same time > > together. > We could try something like the first time you start eclipse during a > week? wdyt? That will leave a peak on Mondays (my assumption...) but that would certainly help. Another solution has been realized in Jenkins (and maybe in Hudson, too). It allows to specify kind of a time interval without defining the exact minute in a crontab-like style: # every fifteen minutes (perhaps at :07, :22, :37, :52) H/15 * * * * In Eclipse we could do it in a similar way: Sleep for a week after the last update, e.g. if the last search-for-update was on a Wednesday, start the next search on the next Wednesday at the earliest.
> When is the cache invalidated? Is this a TTL setting sent from the HTTP > server? Or does it always start with a new HEAD request? It is a HEAD request. > > Note, that when we ship SR1, we could try to pre-cache the SR0 files in > > the distro thus off-loading work from the server. > > First thought: Cool, very cool. Second thought: Does it really change the > overall download size? Yes, to some degree. The packages would be a few > MBytes larger, but this additional amount of data would be downloaded from > the Eclipse mirrors, not from the main download server. It will likely inflate the size of the download by the size of metadata and artifact repos since the repos are already compressed. > > The other thing that could be tried is for the foundation to automatically > > re-route those requests to a mirror. > > If content.jar and artifacts.jar were cryptographically signed this would be > *the* solution. I don't know enough about bintray to judge if this could be > a solution for offloading parts or all traffic. Statistics etc. shouldn't be > an issue because for that reason we still have the HEAD requests. Stats are not based on the download of metadata files but the actual download of the artifacts, and those are reported against a URL specified in the artifacts.xml. > > > I'd like to see an option that searches for updates once a week, but not all > > > Eclipse installations of the world on the same weekday at the same time > > > together. > > We could try something like the first time you start eclipse during a > > week? wdyt? > > That will leave a peak on Mondays (my assumption...) but that would > certainly help. Another solution has been realized in Jenkins (and maybe in > Hudson, too). It allows to specify kind of a time interval without defining > the exact minute in a crontab-like style: > > # every fifteen minutes (perhaps at :07, :22, :37, :52) > H/15 * * * * > > In Eclipse we could do it in a similar way: Sleep for a week after the last > update, e.g. if the last search-for-update was on a Wednesday, start the > next search on the next Wednesday at the earliest. I'm not a fan of the cron route just because it brings more complexity to the UI and provides more opportunities to cause load on the servers by misconfigured entries. I think we could get rid of the current options (every <day> at <time>) which I don't think are that realistic for the usecase of an IDE, and replace this by the options you suggest like "once a day", "once a week", "once a month" all activated on startup only. wdyt?
if(date_last_check > 1 month) { check_now(ask_questions_later); } else if(date_last_check > 1 week) { sleep(random(86400) seconds); check_now(); }
While we debate this (and maybe add new options), does it make sense to enable this in the milestones using the 'Once a Week' option? We haven't used this option on a large scale before, and it would be good to get as much testing as possible (and as early as possible).
(In reply to Pascal Rapicault from comment #13) > I'm not a fan of the cron route... I meant that as an example to distribute load evenly. While I like cron, I'm not a fan of exposing it in the UI either. ;-) I've created a separate feature request for p2 based on the suggestions in comment #13 and comment #14 in bug 457375. (In reply to Ian Bull from comment #15) > ...does it make sense to enable this in the milestones... Yes, that's what I had in mind and why I restarted the discussion on this bug again in December. I'm re-assigning the bug from Platform to EPP again because I assume the Platform team won't change their defaults in the near future based on the discussion in bug 443809. If you disagree please let me know. I will create a change for the next Mars milestone that enables weekly automatic updates in all packages. Do we have a preferred day in the week? Unless we implement a new strategy with bug 457375 this is probably the only option that we can use for a start.
(In reply to Markus Knauer from comment #16) > (In reply to Ian Bull from comment #15) > > ...does it make sense to enable this in the milestones... > > Yes, that's what I had in mind and why I restarted the discussion on this > bug again in December. I'm re-assigning the bug from Platform to EPP again > because I assume the Platform team won't change their defaults in the near > future based on the discussion in bug 443809. If you disagree please let me > know. +1 from me. Too often this issue gets deferred because it's too late in the cycle. Let's start this now, with the full expectation that this might get turned back off before Mars. But we'll learn a lot in the process. > > I will create a change for the next Mars milestone that enables weekly > automatic updates in all packages. Do we have a preferred day in the week? > Unless we implement a new strategy with bug 457375 this is probably the only > option that we can use for a start. Without any data, my suggestion is late in the week (Thursday or Friday). Monday mornings suck as it is, adding more excitement to Mondays won't help :-).
> > I will create a change for the next Mars milestone that enables weekly > > automatic updates in all packages. Do we have a preferred day in the week? > > Unless we implement a new strategy with bug 457375 this is probably the only > > option that we can use for a start. > > Without any data, my suggestion is late in the week (Thursday or Friday). > Monday mornings suck as it is, adding more excitement to Mondays won't help > :-). I would vote for Friday's 12pm. This way if your install gets busted you can blame Eclipse to take the afternoon off :)
(I was thinking about Monday... honestly.) I submitted change https://git.eclipse.org/r/39539 to Gerrit for review. The change modifies the initial preference in all packages. After thinking a bit about it I decided to use Tuesday 10m for our first experiments with these settings enabled. The reason for that is quite simple: I was looking for a weekday (i.e. not on weekends) that is in the beginning of the week but not Monday. Usually we are releasing our milestones, release candidates and releases on Friday. Looking for updates on Tuesdays seems to be a good balance between searching too early (Friday) and too late (Thursday). (I had another thought... we could use different days and times in the packages, e.g. the CPP package searches on Mondays for updates, the Java package on Tuesday, etc. - this would be simple to "implement" and could help reducing the server load on a single day. Just another idea.)
(In reply to Markus Knauer from comment #19) > (I had another thought... we could use different days and times in the > packages, e.g. the CPP package searches on Mondays for updates, the Java > package on Tuesday, etc. - this would be simple to "implement" and could > help reducing the server load on a single day. Just another idea.) I was thinking about this too. It's not as distributed as random would be, but it's pretty good. If you consider that many people also have different time zones too, then this approach might just spread things out as much as any other strategy we come up with. The next question is time of day? I'm not sure how the schedule works if Eclipse isn't running during the scheduled time. Does Eclipse check on the next start if it misses it's window? Does it wait until the next scheduled event? Will it happen randomly at some point?
Can someone confirm that it is working at all? I tried to test this and Eclipse did... nothing. My test setup: * Eclipse Luna SR1 Java package https://www.eclipse.org/downloads/packages/release/luna/sr1 * Modification of "Automatic Updates" preferences * manual from IDE: - 'Automatically find...' enabled - Look for updates on the following schedule: Every Friday, 10am - everything else: default * changing default by modifying plugin_customization.ini in plugins/org.eclipse.epp.package.java_4.4.1.20140925-1820/ The expected result is that Eclipse finds an update to Luna SR1a (which is the case if I'm searching for updates manually).
(In reply to Markus Knauer from comment #21) > Can someone confirm that it is working at all? I tried to test this and > Eclipse did... nothing. > I tried the setting in the UI and got the same thing... nothing! But then I decided to change the time (to noon) and restart Eclipse. At 12:00pm I was prompted with an update. @Markus, did you try restarting Eclipse after you set the schedule? While I don't know for sure, I wouldn't be surprised if the scheduling was done on startup.
(In reply to Markus Knauer from comment #21) > * changing default by modifying plugin_customization.ini in > plugins/org.eclipse.epp.package.java_4.4.1.20140925-1820/ Retesting was successful... in this case I started with a pristine copy of Luna SR1 packages (Java + RCP/RAP), manually modified the plugin_customization.ini, started Eclipse, and waited. Now everything seemed to work. Ian, maybe you are right with your assumption that the scheduling is done at Eclipse startup only. I'll keep an eye open when we activate these settings in the packages.
Where are we with this proposal.. +1 for Java EE package assuming the overhead and server load won't be overwhelming :)
I am scared.
For my understanding, dies this option also performs release updates, e.g., from Luna to Mars? If yes, how is the new update site made available?
No, this is only about enabling this preference.
(I don't want to scare anyone...) I pushed Gerrit change https://git.eclipse.org/r/39539 to master, see comment #19 for the details of this change. Please note that this is *NOT* a guarantee that this preference will stay enabled in Mars/4.5 as there are too many loose ends up to now. I see this more as a field-test in the next milestones in order to get some experience with this feature and to tweak its settings!
(In reply to Markus Knauer from comment #27) > No, this is only about enabling this preference. Does this preference setting also result in a release update (Luna -> Mars)?
(In reply to Lars Vogel from comment #29) > (In reply to Markus Knauer from comment #27) > > No, this is only about enabling this preference. > > Does this preference setting also result in a release update (Luna -> Mars)? Yes. But one would have to add the repository.
(In reply to Pascal Rapicault from comment #30) > > Does this preference setting also result in a release update (Luna -> Mars)? > Yes. But one would have to add the repository. Yes... and no... it depends. If we change the structure of the packages (like described in bug 332989) chances are high that there will be an upgrade from one version to another but the results are not as expected. E.g. if this upgrade replaces only the root features (currently one) with a stripped down feature after a restructuring, the user will get a packages with less functionality than expected. Anyway, upgrades did work in the past (I always tested this) and they will work with some manual interaction in the future, but we've never advertised this because too many things could go wrong.
Closing this bug as "fixed". Check for updates is enabled in all packages since Mars M5 every Tuesday at 10am (LT) by default.
*sigh* I know I've asked this before, but can someone remind me of the files that are touched when a computer "checks for updates" automatically?
(In reply to Denis Roy from comment #33) > *sigh* > > I know I've asked this before, but can someone remind me of the files that > are touched when a computer "checks for updates" automatically? The following files should be checked when Eclipse is looking for updates - p2.index - compositeContent.jar - content.jar
Thanks, Pascal. I've written this down on a sticky note on my monitor.
Bug 498116 has some follow-up caused by this..
