Re: [epp-dev] Package maintainers are you testing your packages?

[Jonah Graham <jonah@xxxxxxxxxxxxxxxx>]

On Thu, 5 Mar 2020 at 10:51, Mickael Istria <mistria@xxxxxxxxxx> wrote:

On Thu, Mar 5, 2020 at 4:39 PM Arthur van Dorp <Arthur.vanDorp@xxxxxxxxxxxxxxxx> wrote:

On another note I am always surprised by how fast some projects have their +1s ready. How do you do that? Our builds go through automatic testing which is (relatively) fast. But the manual testing of EPP builds takes us quite some time and sometimes it’s not immediately clear if an issue found has anything to do with the build or is just some local problem of the tester. Any secret sauce you are using from which we might profit too?

FWIW, for Rust and _javascript_, as the components have good test coverage on their end and are usually already released before joining EPP, I simply run a few very small scenarios and simple which cover about everything, basically:

1. Create/import a project with dedicated wizard

2. edit some files to see validation, completion, hover, outline working (I trust the rest is tested by each brick)

3. Try the Run As/Debug As and ensure those work

Unless I'm stuck in any of those tasks, I vote +1 (even if there is a known bug, as long as it's non-blocking and reported).

That allows to spend less time on specific EPP testing, and my experience is that in order to avoid bugs downstream (in EPP or other) then keeping focus on quality in the actual upstream components (Corrosion, Wild Web Developer) with regular builds, good test suites, managing dependency range at best... makes that a simple 5/10 minutes test for EPP seems enough to me.

ie the more trust you build on the component itself, the less downstream consumers like EPP are likely to require QA.

This is the process I have inherited for CPP too. With EPP I am focused on the integration being in a working state and letting individual projects ensure their contributions to SimRel are good enough. I think the biggest hole in testing is the cross platform testing. Most packages (AFAIK) are only tested on a single platform, but every platform seems to be tested by at least one package tester.

There have been a few bugs in the past with non-trivial integration issues - i.e. things that don't appear until a specific feature is tested in detail. This level of testing is simply not done at the EPP level.

So, for example, I do not test that egit which is part of most (all?) the packages works in the CPP package.

> Thanks for asking. For Scout: Yes, we are testing. So far the results have been good enough for not sending a -1 and some issues are under investigation so no one sent a +1… And then there’s been the understanding that +1s are now assumed and only -1s are sent if necessary.

Thank you for replying - I am trying to make sure we know and have recorded what +1s/-1s mean properly. You are correct that +1 are essentially assumed - i.e. the packages will be published regardless. The occasional +1 is also an indication that the maintainers are paying attention and that the packages are not completely broken.

 

_______________________________________________
epp-dev mailing list
epp-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://www.eclipse.org/mailman/listinfo/epp-dev