Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [rt-pmc] Revisiting blanket and test/build CQs

Just to summarise a brief phone conversation with Jeff McAffer...

The Virgo test dependencies currently being considered fall into two groups:

1. There is an optional dependency relationship from Spring framework (which is shipped with Virgo) to the 3rd party library. We have tests with such libraries to make sure Spring and the library function normally in the Virgo environment.
2. There are no dependencies whatsoever from the Virgo runtime to the 3rd party library. For example, a 3rd party library used by Virgo applications uses thread context class loading. We have tests with such libraries to make sure Virgo's thread context class loading support works correctly.

It is arguable, at least for the second group, that no CQ is necessary at all.

Anyway, it seems this whole topic should be discussed again on a future RT PMC call to reach the necessary clarity and consensus. I will then adjust the Virgo CQs if necessary. Meanwhile, I'll put off raising more CQs along these lines until we have agreement (although I reserve the right to time out when my schedule starts to bite).

Glyn
On 12 May 2010, at 15:45, Joakim Erdfelt wrote:

Also lurking. :-)

I have to say that the explicit and implicit mental categorization of dependencies
is too rigidly black and white.  There are more and more examples of gray areas
in today's modern projects.  We need to accept this reality and find a new language
to describe these.

Examples: taken from Jesse's scope explanation.
compile = explicit
provided = gray (explicit defined) (implicit use, never deployed or installed by project using it.  Can refer to required non-project controlled dependencies, or even optional dependencies for more functionality)
runtime = implicit
test = gray (mostly explicit, but good testing will simulate real world situations, which often means dependencies that falls into implicit category)
system = gray (explicit defined and used) (implicit use, as user is required to install manually before project can be used)

In the end, any discussion needs to not view a dependency as a simple implicit and explicit only, that just leads to confusion.

How I view dependencies is like this ...

# First I ask how its used to create (realize) the project
To-Build - a boolean indicating the need of the dependency by the build systems
For-Testing - a boolean indicating the need of the dependency by the testing & integration systems
# Then I ask how its "shipped"
Shipped-Packaged - a boolean indicating that the dependency is a shipped with the projects own core packages.
Shipped-Optional - a boolean indicating that the project is making the dependency available as an optional add-on / plugin / downloadable, possibly packaged together with some of the projects own classes / configuration files / etc ...
Shipped-Provided - a boolean indicating that the core project needs this dependency to function properly, but the project expects the dependency to arrive via a means prior to this package's installation (the jvm classes, the servlet api, etc ...)

My $0.02 on this subject ...

- Joakim Erdfelt

On Wed, May 12, 2010 at 7:05 AM, Mike Milinkovich <mike.milinkovich@xxxxxxxxxxx<mailto:mike.milinkovich@xxxxxxxxxxx>> wrote:
Since I am lurking on this list, I might as well chime in.

I think that Doug's points on explicit and implicit dependencies are spot
on.

That said, the primary motivation for CQs and our due diligence process is
to capture information relevant to the use of code distributed from the
Eclipse Foundation. In particular, to provide information and mitigate risk
relevant to the adoption and use of Eclipse projects in commercial products.


There are certainly some ambiguities and corner cases, but let's not lose
sight of what drives our collective behaviour.

Given that, test-time or build-time dependencies that are simply an artifact
of our production processes likely require IP processes which vary somewhere
between minimal and none. I believe that the challenge for the PMC is to
make the determination as to whether something really is just a
test/build-time dependency or whether it really is an implicit dependency
relevant to a consumer of the technology. I recognize that upon occasion
those distinctions can be subtle.


> -----Original Message-----
> From: rt-pmc-bounces@xxxxxxxxxxx<mailto:rt-pmc-bounces@xxxxxxxxxxx> [mailto:rt-pmc-bounces@xxxxxxxxxxx<mailto:rt-pmc-bounces@xxxxxxxxxxx>] On
> Behalf Of Glyn Normington
> Sent: May-11-10 10:33 PM
> To: Runtime Project PMC mailing list
> Subject: Re: [rt-pmc] Revisiting blanket and test/build CQs
>
> I agree it is good to nail this subject down while it is still unclear in
> people's minds. Thanks for raising it Doug.
>
> >From my perspective, CQ 4083 (in the case of LGPL) provides a formal
> record that Virgo build/test dependencies which are neither checked in at,
> nor distributed from, eclipse.org<http://eclipse.org/>: (a) are agreed to be "works with"
> dependencies, and (b) are benign from an IP perspective.
>
> Such CQs also solve the practical problem that we don't have an automatic
> means to synchronise a detailed list of build/test dependency CQs with the
> dependencies encoded in the build system. We also can't generate the list
> from the transitive dependencies of the runtime code as in many cases
there
> is no such transitive dependency (even when the complete generality of the
> Spring framework is taken into account).
>
> So the only way we could verify we had raised the correct set of CQs would
> be to do a laborious and error-prone trawl before each release (unlike
> checked in or distributed dependencies which are relatively bounded in
> number and trivial to scan for).
>
> But suppose for the sake of argument that we had such a system and every
> time we added a new build/test dependency, we were prompted to raise a new
> CQ. What value would this provide?
>
> Each new CQ would log a build/test dependency on third party library xxx
at
> version nnn with license yyy. I believe the thinking as far as PMC
approval
> would be: the new CQ is yet another in the long list of build/test
> dependencies so there is no problem in approving it. I also believe the
> thinking in the legal team would be: if the new CQ has the same license as
> some of the other build/test dependencies, again there is no problem in
> approving it. If that analysis is correct, the new CQ would *only* add
real
> value if it introduced a build/test dependency with a previously unused
> license. Well, that is precisely the same value that CQ 4083 (in the case
> of LGPL) is providing, but at a much lower net cost to everyone involved.
>
> So, personally, I have yet to understand what value a detailed list of
> build/test dependency CQs would add, even if it was practical to maintain
> such a list. I'd be grateful if someone could enlighten me.
>
> Thanks,
> Glyn
> On 11 May 2010, at 21:10, Jesse McConnell wrote:
>
> > When jetty was going through the review process we had the
> > understanding that test dependencies were no different then anything
> > else in terms of the code being checked into svn.  If it was required
> > to build and/or run at all it had to have a CQ and go through the
> > proper channels.  That being said we still use the artifacts from the
> > maven central repo for the actual testing but we have solid IPLog
> > representation for everything (case by case) and we purposefully left
> > code of jetty on The Codehaus because it would have been problematic
> > to pass dependencies through the CQ process or the code implementing
> > it through the IP Validation/Verification process.  When we move to
> > distribute anything from eclipse.org<http://eclipse.org/> it uses everything from Orbit
> > like it should.
> >
> > One case I would like to add for clarification is an artifact that is
> > never going to be downloaded from eclipse.org<http://eclipse.org/>, like for example a
> > maven plugin.  It has firm dependencies on things like the maven
> > plugin api and stuff like that but since its never going to be
> > download from eclipse.org<http://eclipse.org/>, much less will its dependencies see the
> > light of orbit....are we still banned from having that in our svn @
> > eclipse?  We would release it into the maven central repository same
> > as everything else we release to maven central...
> >
> > Anyway, I am encouraged to seem further light shown on the issues.
> >
> > cheers,
> > jesse
> >
> > --
> > jesse mcconnell
> > jesse.mcconnell@xxxxxxxxx<mailto:jesse.mcconnell@xxxxxxxxx>
> >
> >
> >
> > On Tue, May 11, 2010 at 14:13, Jeff McAffer <jeff@xxxxxxxxxxxxxxxxx<mailto:jeff@xxxxxxxxxxxxxxxxx>>
> wrote:
> >> Great topic Doug.  Thanks for the context. I agree with the overall
> concern
> >> around blanket CQs. I am also interested in people's thoughts on the
> >> position of test code in the IP landscape.
> >> There has been an increase in the number of libs needed only for
> testing. In
> >> the past we have not drawn much of a distinction between coded needed
> for
> >> testing and other dependencies.  Things like EasyMock are somewhat
> obvious
> >> and unproblematic but, what about, *for example*, using Hibernate for
> >> testing? Raises questions like:
> >> - What are you testing and won't users have to get hibernate too?
> >> - If hibernate is just one of the options, can we pick one that would
be
> >> less problematic?
> >> - how does testing code relate to incubating code in the eyes of the IP
> >> system?
> >> As with much of the IP policy, I am quite open to different positions
> but
> >> feel strongly that we have a clear and simple set of guidelines so we
> can
> >> stop having this kind of discussion :-)
> >> Jeff
> >>
> >> On 2010-05-11, at 2:38 PM, Douglas Clarke wrote:
> >>
> >> RT PMC,
> >>
> >> I would like to revisit the discussion on blanket CQs for test/build
> such as
> >> CQ 4083 recently approved for Virgo. I apologize that I was unable to
> >> participate in the earlier discussion but after looking at this CQ I
> believe
> >> we need to revisit the issue to ensure we are all clear on how we
handle
> >> these situations.
> >>
> >> My concerns include:
> >>
> >> The distinction between build/test and code that is contained in the
> Eclipse
> >> code repositories and the code/binaries that are distributed from the
> >> projects.
> >>
> >> I firmly believe that all code that is included in the
> distribution/release
> >> from a project that has a 3rd party dependency either explicitly (i.e.
> >> import) or implicitly (functionality requires the library must be on
> >> classpath to use) should have a separate CQ listing the individual
> >> dependency and explaining how it meets the works-with requirements and
> that
> >> CQ be backed up with approval on the PMC mailing list.
> >>
> >> Additionally there are some who believe that all code in the
repository,
> >> including test cases, are part of the project's 'release' even if not
> >> distributed in bundles/zip. We should discuss where we draw the line
> between
> >> build/test/release and how these CQs for 3rd party dependencies are
> handled.
> >> I am concerned that a blanket CQ is too broad and does not offer much
> value
> >> from an IP due diligence perspective. I would prefer to see individual
> CQs
> >> or at a minimum a list of projects and versions and possibly jar files
> that
> >> the CQ applies to. In the case of 4083 I believe the intent was for
> testing
> >> utilities only and not to any Virgo code that would be shipped but the
> broad
> >> nature of the CQ makes it unclear what it applies to and could lead to
> >> committers inadvertently assuming they can include LGPL dependencies
> without
> >> the proper CQs.
> >>
> >> I am happy to get the discussion started here and then continue on next
> >> week's PMC call.
> >>
> >> Cheers,
> >>
> >> Doug
> >>
> >> For reference the policy is:
> >>
>
http://www.eclipse.org/org/documents/Eclipse_Policy_and_Procedure_for_3rd_P
> arty_Dependencies_Final.pdf
> >> _______________________________________________
> >> rt-pmc mailing list
> >> rt-pmc@xxxxxxxxxxx<mailto:rt-pmc@xxxxxxxxxxx>
> >> https://dev.eclipse.org/mailman/listinfo/rt-pmc
> >>
> >>
> >> _______________________________________________
> >> rt-pmc mailing list
> >> rt-pmc@xxxxxxxxxxx<mailto:rt-pmc@xxxxxxxxxxx>
> >> https://dev.eclipse.org/mailman/listinfo/rt-pmc
> >>
> >>
> > _______________________________________________
> > rt-pmc mailing list
> > rt-pmc@xxxxxxxxxxx<mailto:rt-pmc@xxxxxxxxxxx>
> > https://dev.eclipse.org/mailman/listinfo/rt-pmc
>
> _______________________________________________
> rt-pmc mailing list
> rt-pmc@xxxxxxxxxxx<mailto:rt-pmc@xxxxxxxxxxx>
> https://dev.eclipse.org/mailman/listinfo/rt-pmc

_______________________________________________
rt-pmc mailing list
rt-pmc@xxxxxxxxxxx<mailto:rt-pmc@xxxxxxxxxxx>
https://dev.eclipse.org/mailman/listinfo/rt-pmc

<ATT00001..txt>



Back to the top