Re: [rt-pmc] 08 Sep 2010 - RT PMC call follow-up

[Glyn Normington <gnormington@xxxxxxxxxx>]

One way to formalise "released content" would be to focus on the download area and produce a tool which automatically guarantees all archive content and other files in the download area match CQs for the relevant project. Content destined for the download area would be placed there using the tool and would need to pass an automated check first. A failed check would return a report listing the problem files.

Apart from formalising released content, such an approach would have the benefit of removing a source of human error and would reduce the manual checking that is presumably needed when a project is released. Admittedly this would take some investment as such a tool would be probably be a significant addition to IPZilla, but I think it would pay for itself over time. Just a thought...

Regards,
Glyn

On 18 Sep 2010, at 16:44, Jeff McAffer wrote:

> Thanks for the update Janet.  I do have a few questions/clarifications.
>
> - When you say "test suite" here what do you mean? Typically the test suite is written by the Eclipse project team so its a bit odd to have Eclipse originating code be an exempt prereq. What we are really talking about is random libraries that the Eclipse project test suite might need in the course of running the tests. While there likely are cases where we get whole test suites from a third party (e.g., the OSGi tests), that is not the case we are talking about here.
>
> - Are you specifically focussing on test-related code? The general problem we are facing relates to prereqs that do not "ship". Testing is certainly one of those cases but there are others. Build-time prereqs (e.g., annotation processors and the like) may also fall in this category.
>
> - Given that the subject of this effort (build, test and similar code) is not actually consumed by the broader community, is there a chance that listing these libs as exempt prereqs actually mislead consumers into believing they need more than they really do?
>
> - These seem to point to a missing a concept in the IP policy. In a previous post I mentioned the idea of "Released Content" (and its complement). While I am not wed to that precise idea, it does seem like we are skirting the issue.  For example in documenting the exempt prereqs that arise out of your proposed approach, we will need to identify the prereqs as "only for xxx" (where xxx = build, test, ...) as in "you won't need this to actually use the project's output". It seems that we should reify that concept into something well-defined.
>
> - A step like this is good progress but we also need clarifications on what kinds of third party use qualify. We started this process with a basic confusion over how various uses should be classified and handled.  This gives us a means to handle with some types of uses but we need to nail down very concretely what those uses are.
>
> - Can you comment on the workflows, roles and responsibilities involved here? As I understand the policy, the EMO has to approve exempt prereqs. Does there need to be a CQ?  A PMC mailing list discussion? Other? If there are multiple steps, what is the order?
>
> - Why is a license compatibility check needed? If for example a project wanted to use MySQL (GNU licensed) to test against should this matter? Assuming of course that there are no runtime or other requirements directly on MySQL but rather "a database".  Going back to the motivation for the IP policy, our end-user community will know like know or care what database is used.
>
> Jeff
>
> On 2010-09-17, at 2:47 PM, Janet Campbell wrote:
>
>> A quick update for everyone.  We’ve reviewed our Policy – Guidelines for the Review of Third party Dependencies [1] – and had a follow on discussion with Jesse that was quite helpful.  What I would like to propose for discussion is that we introduce a modification to the definition of an exempt prerequisite under the Policy to include test suites.   While it would still be necessary to document the use of the suites, the approval of the use of them would be quite straightforward and no due diligence beyond a license compatibility check would be required.
>>
>> We believe the proposed change would address many of the concerns raised while maintaining the risk balance we seek to maintain at Eclipse.   I plan to introduce this proposed change at the next IP Advisory Committee meeting on Wednesday of next week so if you have any concerns regarding this approach, please let me know as soon as possible.
>>
>> Regards,
>> Janet
>>
>> [1] http://www.eclipse.org/org/documents/Eclipse_Policy_and_Procedure_for_3rd_Party_Dependencies_Final.pdf
>>
>>
>> -----Original Message-----
>> From: Janet Campbell [mailto:janet.campbell@xxxxxxxxxxx]
>> Sent: Tuesday, September 14, 2010 5:08 PM
>> To: 'Jesse McConnell'; 'Runtime Project PMC mailing list'
>> Cc: 'emo-ip-team@xxxxxxxxxxx'
>> Subject: RE: [rt-pmc] 08 Sep 2010 - RT PMC call follow-up
>>
>> We appreciate everyone’s comments, understand  your pain and are actively looking into how we can alleviate the burden while maintaining the balance of risk at Eclipse.
>>
>> We want to make sure that we find the right solution and so we ask for your patience while we figure out how to best address your concerns.
>>
>> Regards,
>> Janet
>>
>> Janet Campbell
>> Director, Intellectual Property
>> Eclipse Foundation, Inc.
>>
>> -----Original Message-----
>> From: Content-filter at foundation.eclipse.org [mailto:postmaster@xxxxxxxxxxxxxxxxxxxxxx] On Behalf Of Jesse McConnell
>> Sent: Tuesday, September 14, 2010 4:26 PM
>> To: Runtime Project PMC mailing list
>> Cc: emo-ip-team@xxxxxxxxxxx
>> Subject: Re: [rt-pmc] 08 Sep 2010 - RT PMC call follow-up
>>
>> Jeff, I think we are totally on the same page here..imo we just need a
>> definitive statement that our basic view of the world can be
>> effectively reconciled with IP policy and we should be good to go...
>>
>> so much of this is (I think) pretty common sense, its just making sure
>> that what I/we think of as common sense passes the legal litmus test
>> sufficiently :)
>>
>> fwiw I am tickled to see these distinctions being discussed
>>
>> cheers,
>> jesse
>>
>> --
>> jesse mcconnell
>> jesse.mcconnell@xxxxxxxxx
>>
>>
>>
>> On Tue, Sep 14, 2010 at 15:15, Jeff McAffer <jeff@xxxxxxxxxxxxxxxxx> wrote:
>>> Thanks for the update Jesse. I really think that we are dealing with a
>>> conceptual and terminological mismatch here.
>>> A bit of context. The IP policy is in place largely
>>> "to provide information and mitigate risk relevant to the
>>> adoption and use of Eclipse projects in commercial products."[1]
>>> Looking at the current IP policy [0] there appears to be a gap.  We have the
>>> following terms:
>>> “Content” is copyrightable material, including without limitation software,
>>> documentation, articles, whitepapers, and presentation materials.
>>> “Distributed Content” is Content which is distributed by the Eclipse
>>> Foundation via its Repository or other means in a manner consistent with
>>> this Intellectual Property Policy.
>>> “Eclipse Content” is Distributed Content intended to be developed or
>>> modified by one or more Eclipse Projects (as that term is defined by the
>>> Eclipse Development Process), regardless of the license or licenses that
>>> govern the use of that Content.
>>> “Non-Eclipse Content” is Distributed Content which is not Eclipse Content.
>>>
>>> The notion of "Released Content" (potential new term) does not appear here.
>>> Released Content is the Distributed Content that we intend or expect our
>>> adopters and users to consume. In the absence of this term there is an
>>> implicit assumption that any and all Distributed Content is intended to be
>>> "used" by our end-users. Many of the cases at hand here relate to
>>> Distributed Content that is not yet (or will never be) released and is not
>>> intended to be used by anyone other than the committers/contributors. For
>>> example, tests. In [2] Barb signal towards this distinction by acknowledging
>>> that some people will get code from an "Eclipse repository before it is
>>> “officially” released by a project". That is, there is Distributed Content
>>> that is Released and Distributed Content that is not.
>>> In the absence of this distinction, Section IV of the policy has no choice
>>> but to say that all Distributed Content has to go through the IP process.
>>> This is the problem.
>>> In [1] Mike M recognizes this challenge when he says
>>> "...test-time or build-time dependencies that are simply an artifact
>>> of our production processes likely require IP processes which vary somewhere
>>> between minimal and none."
>>> Similarly, speaking of the yet to be released Content in [2], Barb
>>> acknowledges agrees that "this small risk would not justify the adverse
>>> impact such additional controls would have on the community".
>>> Summary: Recognizing that the constituents for whom the policy is in place
>>> likely don't know or care what is used to test the things they consume, it
>>> is hard to motivate considerable additional work for the IP team or the
>>> committers to deal with Content that "will never see the light of day" as
>>> Released Content.
>>> To be clear, I am in now way suggesting that all test code be
>>> IP-process-free.  I am however suggesting that there are clearly some chunks
>>> of code or dependencies that matter only at points in time that have little
>>> to do with our adopter community. The IP policy should capture this.
>>> Jeff
>>> p.s., While INAL I thought it would be fun to put forward a definition of
>>> Released Content:
>>> "Released Content" is Distributed Content that is intended by the producing
>>> projects to be adopted or used by their end-user community. Released Content
>>> would include but not be limited to Distributed Content that has been or
>>> will be the subject of a Release Review.
>>> [0] http://eclipse.org/org/documents/Eclipse_IP_Policy.pdf
>>> [1] http://dev.eclipse.org/mhonarc/lists/rt-pmc/msg01595.html
>>> [2] http://dev.eclipse.org/mhonarc/lists/rt-pmc/msg02007.html
>>> On 2010-09-14, at 3:12 PM, Jesse McConnell wrote:
>>>
>>> I had a talk with Barb about this a bit and wanted to follow up on
>>> this thread with this blurb from our conversation...
>>>
>>> my current frustration with the IP policy/procedure is the existence
>>> of a 'safe on the build server' classification for some things.  In my
>>> little slice of heaven we write software that builds straight out of
>>> svn...and the current definition of 'distribution' includes what is in
>>> svn...which means that I am unable to make use of this 'safe on the
>>> build server' classification of IP.  From what I hear the whole 'safe
>>> on the build server' exists because there is an explicit understanding
>>> that the builds produced on the build server are not 'eclipse
>>> distributions'.
>>>
>>> so I don't understand how there exists a case where a build occurring
>>> on the eclipse build server is not a distribution but somehow software
>>> being built from something checked out from SVN is some how an
>>> 'eclipse distribution'.  I understand that is absurd and its not
>>> 'really' but if I want to open a CQ to use something like apache
>>> directory service its basically a rubber stamp if I have it on the
>>> build server and use it there, but if my build system has an actual
>>> 'dependency' declared on it for usage in a test case (not going into
>>> my distribution at _all_, just a unit test) it is suddenly a full
>>> fledged dependency (because there is no distinguishing between test
>>> scope or distribution scope).
>>>
>>> cheers,
>>> jesse
>>>
>>> --
>>> jesse mcconnell
>>> jesse.mcconnell@xxxxxxxxx
>>>
>>>
>>>
>>> On Mon, Sep 13, 2010 at 15:45, Jeff McAffer <jeff@xxxxxxxxxxxxxxxxx> wrote:
>>>
>>> Hey Barb,
>>>
>>> Thanks for the additional info.  Comments embedded.
>>>
>>> Jeff
>>>
>>> On 2010-09-13, at 2:11 PM, Barb Cochrane wrote:
>>>
>>> 1)       Eclipse-developed test code deposited in the repository calling
>>>
>>> third party test code which does not reside on Eclipse servers.
>>>
>>>
>>>
>>> Ø       In the cases described, third party code could be characterized as a
>>>
>>> “workswith” or perhaps “exempt pre-req”
>>>
>>> Can you clarify "could"?  Is it "should" or "must"?
>>>
>>> As I recall from the discussion there were various shades here. There are
>>>
>>> a) test setups that just need some other function to be present but do not
>>>
>>> reference this function directly in any code.
>>>
>>> b) test setups that need specific function to be present but still do not
>>>
>>> reference the function directly
>>>
>>> c) test setups that directly reference the third party libraries but that
>>>
>>> function is not part of the actual released output of the project
>>>
>>> Other scenarios where the third party references come from (to be) released
>>>
>>> project output are easier to handle as classical dependencies.  The subtlety
>>>
>>> in the above comes from the absence/presence of direct references and the
>>>
>>> fact that test code is not generally released so is not typical consumable
>>>
>>> project output. That is, we don't expect any significant number of people to
>>>
>>> get this code (see case 2 below). Under that model, scenarios a-c above
>>>
>>> would be instances of case 2.
>>>
>>> Ø       The process outlined in the Guidelines for the Review of Third Party
>>>
>>> Dependencies would apply (as there is no distinction made based on the type
>>>
>>> of code involved)
>>>
>>> Ø       It is our understanding that the dynamic nature of calls to these
>>>
>>> third party packages would make the task of creating CQs for each package an
>>>
>>> extraordinarily heavy burden
>>>
>>> Ø       The IP team is in the process of reviewing the IP Policy to try to
>>>
>>> identify what we can do to alleviate this burden.  Janet will raise any
>>>
>>> suggested changes to the Policy to the IP Advisory Committee as necessary.
>>>
>>>
>>>
>>> 2)       A risk was identified in that there are a small number of
>>>
>>> bleeding-edge developers (low single percentage of users) who may download
>>>
>>> code from the build server or from the Eclipse repository before it is
>>>
>>> “officially” released by a project.  In either case, the downloader may
>>>
>>> consume code that has not been fully reviewed / approved by the IP team.
>>>
>>> We appreciate this risk being highlighted but feel the additional controls
>>>
>>> necessary to eliminate this small risk would not justify the adverse impact
>>>
>>> such additional controls would have on the community.
>>>
>>>
>>>
>>> 3)       There was a question raised as to why CQs are required for
>>>
>>> “workswith” dependencies, but not their further dependencies.  In the case
>>>
>>> of dependencies, we are trying to highlight to our downstream consumers the
>>>
>>> Eclipse Project dependencies in such a way so that they may investigate
>>>
>>> further and make decisions accordingly.  In so doing, we must determine what
>>>
>>> level of information is reasonable to provide given our resource
>>>
>>> constraints.  It was felt that the first level of information regarding
>>>
>>> dependencies struck a reasonable balance given our current staffing levels.
>>>
>>>
>>>
>>>
>>> I hope that helps.   If an additional conference call would be useful,
>>>
>>> please let me know.  I hope to update you further on the first topic in the
>>>
>>> next couple of weeks.
>>>
>>>
>>>
>>> Cheers,
>>>
>>>
>>>
>>> Barb Cochrane
>>>
>>> Eclipse Foundation, Inc.
>>>
>>> Phone 613.224.9461 ext 232
>>>
>>>
>>>
>>> Eclipse Summit Europe, Nov 2-4
>>>
>>> http://www.eclipsecon.org/summiteurope2010/
>>>
>>>
>>>
>>> _______________________________________________
>>>
>>> rt-pmc mailing list
>>>
>>> rt-pmc@xxxxxxxxxxx
>>>
>>> https://dev.eclipse.org/mailman/listinfo/rt-pmc
>>>
>>>
>>> _______________________________________________
>>>
>>> rt-pmc mailing list
>>>
>>> rt-pmc@xxxxxxxxxxx
>>>
>>> https://dev.eclipse.org/mailman/listinfo/rt-pmc
>>>
>>>
>>> _______________________________________________
>>> rt-pmc mailing list
>>> rt-pmc@xxxxxxxxxxx
>>> https://dev.eclipse.org/mailman/listinfo/rt-pmc
>>>
>>>
>>> _______________________________________________
>>> rt-pmc mailing list
>>> rt-pmc@xxxxxxxxxxx
>>> https://dev.eclipse.org/mailman/listinfo/rt-pmc
>>>
>>>
>>
>> _______________________________________________
>> rt-pmc mailing list
>> rt-pmc@xxxxxxxxxxx
>> https://dev.eclipse.org/mailman/listinfo/rt-pmc
>
> _______________________________________________
> rt-pmc mailing list
> rt-pmc@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/rt-pmc