Community
Participate
Working Groups
As commented in the the forum thread [1], it would be good that at the presence of any kind of error during the qualifier replacement step, buckminster scaled up the exception to notify said error, for instance, to hudson. I've blocked the following build [2] because there are some buckminster warnings related to this issue due to the lack of a proper p2 repository in the artifacts of the lastSuccessfulBuild (in the times the build was executed) so that there was not any repository reference to compare with. The build was successful, however no qualifier replacement took place[3]. From this build I understand that the reference.repository plays some kind of role, indeed. On the one hand, I still full of doubts about how the qualifier replacement works, does it need a repository reference to compare with ? Does the source code management system also participate in somehow during said replacement step ? On the other hamd, I see a couple of actions: 1. The easy one: Turning said buckminster warnings into errors, to make hudson build fail. 2. The hard one: Investigating if there are other places where any kind of exception may be caught. (Why did I have some plugins qualifier segments replaced and others were not replaced? - Sorry the faulty builds commented in the forum were lost, as soon as the replacement started to work again - ) [1] http://www.eclipse.org/forums/index.php/m/695394/#msg_695394 [2] https://hudson.eclipse.org/hudson/job/buckminster-mdt-ocl-3.1-maintenance/613/console [3] https://hudson.eclipse.org/hudson/job/buckminster-mdt-ocl-3.1-maintenance/613/artifact/MDT-OCL.p2.repository/plugins/ Regards, Adolfo.
The short story: A missing reference repository from the last build should not have any effect on the qualifier generation. A warning should be printed but nothing more since this is a normal condition. The repository will not exist the first time the build is executed. The fix for this is to catch the ProvisionException, print a (one line) warning, and continue. The version qualifier has already been computed when that exception is thrown so the generator will then succeed. The fix is committed: http://git.eclipse.org/c/buckminster/buckminster.git/commit/?id=14c49be1b3e1f6560b058344b9f29830e8b07f1e Since you wanted more details on how things work, here's the long story: The version qualifier will first consult the SCM (cvs, svn, or git) to find the latest modification pertaining to the project. It then tries to figure out if a bundle with the exact same version (with qualifier) can be found in the last successful build. If that's the case, and if that component has a generated build.id stored as a property then some further action needs to be taken. Only plug-ins with an about.mappings file will have receive this property. The about.mappings file is generated by the build and it contains the build.id so when the build.id changes the version qualifier must also change. The current build.id and the stored build.id is therefore compared and if they differ, then the qualifier is regenerated using the current date rather than the timestamp of the latest source modification.
Thomas. Thanks for the info... It sounds very interesting and enlightening... So I understand that both reference repository and SCM info are used to generate qualifiers. I guess that when you say "can be found in the last successful build", you mean the reference repositor( which usually we configure to be used by our last hudson sucessful build)... Things start to make sense to me, now... thanks More comments in line below. (In reply to comment #1) > The short story: > A missing reference repository from the last build should not have any effect on > the qualifier generation. A warning should be printed but nothing more since > this is a normal condition. The repository will not exist the first time the > build is executed. > > The fix for this is to catch the ProvisionException, print a (one line) warning, > and continue. The version qualifier has already been computed when that > exception is thrown so the generator will then succeed. > But if there is a missing repository, how will the qualifiers be replaced ?. Will the current date be used ? Will the timestamp of last modification be used instead ?. If it's the former case, I think it's better to raise and error and tell the user to use a proper repository or no using anyone, instead of silently generating undesirable qualifiers replacement... > The fix is committed: > http://git.eclipse.org/c/buckminster/buckminster.git/commit/?id=14c49be1b3e1f6560b058344b9f29830e8b07f1e > I'll do some experiments in local after updating my local Eclipse installation with the new version of buckminster. To ensure that no providing reference repository, it properly generates qualifiers. > Since you wanted more details on how things work, here's the long story: > > The version qualifier will first consult the SCM (cvs, svn, or git) to find the > latest modification pertaining to the project. It then tries to figure out if a > bundle with the exact same version (with qualifier) can be found in the last > successful build. If that's the case, and if that component has a generated > build.id stored as a property then some further action needs to be taken. > > Only plug-ins with an about.mappings file will have receive this property. The > about.mappings file is generated by the build and it contains the build.id so > when the build.id changes the version qualifier must also change. The current > build.id and the stored build.id is therefore compared and if they differ, then > the qualifier is regenerated using the current date rather than the timestamp of > the latest source modification. This also explains why in spite of not making any change to the source code (I'm doing releng changes), some plugins qualifiers are being replaced with the current date of the build... which I find it an inconvenience, and I'll try to explain that. These suspicious plugins are the tipical plugins which work as branding plugins (used by the features), containing the about information (which includes about.mappings with the build.id). This build information is quite useful to see in our features, and I understand that if the buildId changes, then the plugin is different, then we should change the qualifier to use the current build date. However changing the qualifier, when doing a future update, make P2 to download no-necessarily a lightweight plugin again, just for updating a simple buildID. IMHO, an inconvenience from the point of view of an updating process. Some solutions: 1- No including build information in our features. 2- Splitting out the about information in a different branding plugin, so that it's just a lightweight branding plugin which gets downloaded during an update process. 3- Making this qualifier replacement behaviour, configurable from buckminster. Although I'm not sure that this is the right solution... 4- Living with overhead in the update process ;P An interesting discussion to have with my project leader, probably with more cross-project-dev list Cheers, Adolfo.
(In reply to comment #2) > But if there is a missing repository, how will the qualifiers be replaced ?. > Will the current date be used ? Will the timestamp of last modification be used > instead ?. The latter. > Some solutions: > 1- No including build information in our features. > 2- Splitting out the about information in a different branding plugin, so that > it's just a lightweight branding plugin which gets downloaded during an update > process. > 3- Making this qualifier replacement behaviour, configurable from buckminster. > Although I'm not sure that this is the right solution... > 4- Living with overhead in the update process ;P > I would advocate #2. Use a dedicated branding plug-in. #3 is not an option since it's extremely bad practice to retain the same qualifier when the bits change.
(In reply to comment #3) > (In reply to comment #2) > > But if there is a missing repository, how will the qualifiers be replaced ?. > > Will the current date be used ? Will the timestamp of last modification be > used > > instead ?. > > The latter. > Thinking a little bit more on this issue, the latter may also produce undesirable qualifiers... Think on a plugin with such about.mappings so that with every build its qualifier is replaced with the current date 1. first build (no reference repository), lets say v20110303 (timestamp of the last modification) 2. second build (we have reference repository), we would have v20110714 (current date, because the buildId in the about.mappings file gets updated). 3. third build (no reference repository, we miss it by somehow), we would have v20110303 again. This would imply an undesirable qualifier downgrade in a plugin of our generated repository. It's also true, that the problem would be fixed with the next build (with a valid reference repository), however we would have "silently" (to me a warning, it's not revealing, specially in automated environments) producing incorrect p2 repositories... Perhaps, making the build process fail because we want to provide a reference repository, and the repository is missing or it's corrupted, could be more revealing for the release engineer... Anyway, it's just an opinion, and I understand that this may be considered a "breaking" change, and I guess I could live without it... so it's up to you to adopt it ;) Cheers, Adolfo.
BTW, Buckminster - http://download.eclipse.org/tools/buckminster/updates-3.7 still shows the update from Tuesday. Waiting for Wednesday one to verify the fix. Cheers, Adolfo.
(In reply to comment #4) > ... Perhaps, making the build > process fail because we want to provide a reference repository, and the > repository is missing or it's corrupted, could be more revealing for the > release engineer... > Ideally, we could trap the reason why the repository is unavailable. If we were 100% sure that the repository was in fact missing, then that could render a warning while all other causes for not being able to access the repository would be an error. That way, we would support the normal use-case (the very first build is missing the reference repository) while also throwing exceptions on other types of errors. Unfortunately, I don't think we can extract the reason in a safe enough manner to do that. So the choice is between always throw an exception if the reference repository cannot be loaded, and thereby invalidating the normal "first build" use-case, or to print a warning when that repository cannot be accessed. I chose the latter.
They are simply opinions... As a modest release engineer, I prefer having an error in my first build, and in every one in which a non-proper reference repository is used, instead of silently generating undesirable qualifiers, therefore p2 repositories... Anyway, it's OK to me ;) BTW, my last build (on master server) produced the abnormal qualifier replacement again: https://hudson.eclipse.org/hudson/job/buckminster-mdt-ocl-core-3.2-master/392/artifact/MDT-OCL.p2.repository/plugins/ I don't have any idea about why this happens. My feelings, some kind of server fault provokes some buckminster abnormal behavior. Cheers, Adolfo.
Thommas, I haven't been able to verify this. In my local Eclipe installation using the last buckminster 3.7 (Buckminster Core - v1.4.0.v20110712-1350 ): I've included the following property in the properties file I'm going to use: buckminster.reference.repository=C:/temp Providing that the folder is empty, after executing the action which produces my p2.repository the following occurs (I'll attatch an screenshot): - Features don't get the forth segment qualifier version replaced. Some feature sources get the qualifier segment replaced. - Plugins don't get the forth segment qualifier version replace. Some plugins sources get the qualifier segment replaced. A lot of "Unable to qualify version" errors appear in the console. For example: [start org.eclipse.ocl:osgi.bundle$3.1.0.qualifier#manifest] Unable to qualify version [end org.eclipse.ocl:osgi.bundle$3.1.0.qualifier#manifest] [start org.eclipse.ocl:osgi.bundle$3.1.0.qualifier#about.mappings] [end org.eclipse.ocl:osgi.bundle$3.1.0.qualifier#about.mappings] [start org.eclipse.ocl:osgi.bundle$3.1.0.qualifier#bundle.jar] [end org.eclipse.ocl:osgi.bundle$3.1.0.qualifier#bundle.jar] If I remove that property, all qualifier are properly generated. Regards, Adolfo.
Created attachment 200590 [details] Demonstrating screenshot I nearly forget the SS :)
