| Re: [science-iwg] IDataSet in EAVP |
Werner
Send science-iwg mailing list submissions to
science-iwg@xxxxxxxxxxx
To subscribe or unsubscribe via the World Wide Web, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
or, via email, send a message with subject or body 'help' to
science-iwg-request@xxxxxxxxxxx
You can reach the person managing the list at
science-iwg-owner@xxxxxxxxxxx
When replying, please edit your Subject line so it is more specific
than "Re: Contents of science-iwg digest..."
Today's Topics:
1. Re: IDataSet in EAVP (Philip Wenig)
----------------------------------------------------------------------
Message: 1
Date: Wed, 2 Sep 2015 14:26:26 +0200
From: Philip Wenig <philip.wenig@xxxxxxxxxxxxx>
To: science-iwg@xxxxxxxxxxx
Subject: Re: [science-iwg] IDataSet in EAVP
Message-ID: <55E6EAF2.3030903@xxxxxxxxxxxxx>
Content-Type: text/plain; charset="windows-1252"; Format="flowed"
I'll check if it works with an older version too. But having a newer one
would be nice :-).
Am 02.09.2015 um 13:30 schrieb Jay Jay Billings:
>
> Philip,
>
> I use Apache Commons Math from Orbit already. Is it just that you need
> a newer version?
>
> Jay
>
> On Sep 2, 2015 7:26 AM, "Philip Wenig" <philip.wenig@xxxxxxxxxxxxx
> <mailto:philip.wenig@xxxxxxxxxxxxx>> wrote:
>
> Hi Peter,
>
> Apache Commons Math is great library ... but it's hard getting it
> through the IP review process. I've already file a CQ to bring
> Commons Math 3.5 to Eclipse Orbit:
> http://dev.eclipse.org/ipzilla/show_bug.cgi?id=9724
>
> Somehow, "AbstractWell.java" can't be accepted. Removing causes a
> plethora of subsequent problems. I'm trying to resolve that issue
> at the moment.
>
>
> Best,
> Philip
>
>
> Am 02.09.2015 um 13:09 schrieb Peter.Chang@xxxxxxxxxxxxx
> <mailto:Peter.Chang@xxxxxxxxxxxxx>:
>>
>> Hi all,
>>
>> In terms of bikeshedding a package name, I don?t think it is a
>> big deal even for Diamond. Searching through our scripts finds
>> less than 100 of them import of the api and impl dataset packages.
>>
>> Most of the other issues are eminently solvable. The exception
>> would be dependencies: if the jars are separated out then this
>> should not be an issue. Or have I misunderstood and there are
>> more problems? (Why is AC Maths bad?)
>>
>> Regards,
>>
>> Peter
>>
>> *From:*science-iwg-bounces@xxxxxxxxxxx
>> <mailto:science-iwg-bounces@xxxxxxxxxxx>
>> [mailto:science-iwg-bounces@xxxxxxxxxxx] *On Behalf Of *Jonah Graham
>> *Sent:* 01 September 2015 18:14
>> *To:* Science Industry Working Group
>> *Subject:* Re: [science-iwg] IDataSet in EAVP
>>
>> Hi all,
>>
>> I am currently moving the plug-ins to a new repository (actually
>> not really a different one, but a fork with the non-relevant code
>> deleted, that allows for maintaining the history). I am trying to
>> make this repo the minimum that enables people to use datasets
>> without requiring any additional dependencies. It would be great
>> to get this as part of a top level project.
>>
>> By having a repo with a smaller code base, hopefully it can be
>> API versioned sooner and IP reviewed in such a way to solve the
>> EAVP dependency issue.
>>
>> There is still how to manage that code base back in Diamond, in
>> particular moving datasets to a new package has a cost to Diamond
>> because there are many Jython scripts in the wild that refer to them.
>>
>> Jonah
>>
>> On 1 September 2015 at 17:55, Torkild Ulv?y Resheim
>> <torkildr@xxxxxxxxx <mailto:torkildr@xxxxxxxxx>> wrote:
>>
>> Hi all,
>>
>> I wonder if it would make any sense to move the two plug-ins in
>> question to a separate repository and namespace and do the
>> IP-review on those. Moving these should also rid them of
>> dependencies such as Apache Commons Math which i understand is
>> problematic. I like the namespace ?org.eclipse.dataset? but it
>> may be a bit to generic. In preparation for a top level science
>> project we may want to call it ?org.eclipse.science.dataset?.
>>
>> Best regards,
>> Torkild
>>
>>
>> > 1. sep. 2015 kl. 18.39 skrev Jonah Graham
>> <jonah@xxxxxxxxxxxxxxxx <mailto:jonah@xxxxxxxxxxxxxxxx>>:
>> >
>> > Hi all,
>> >
>> > If I can chime in with our 2 cents.
>> >
>> > Following the SWG meeting in Trondheim it was great to see the
>> community get behind the need for a standardised data api. Based
>> on what I know of IDataset (and the way it matches numpy) I am
>> convinced that is the way forward. However, I do see there is
>> work to be done to get there.
>> >
>> > The issues we see in particular are:
>> > 1. There are precompiled jars in the current dataset plugin -
>> this must be resolved so it can be all source.
>> > 2. There are dependencies on non-orbit approved 3rd party
>> packages- we started discussing in Trondheim how some of these
>> can be resolve.
>> > 3. There are 2 plugins & the api plugin comes with lots of
>> other (non-relevant) apis
>> > 4. The code has not been IP-review and established as IP clean.
>> >
>> > These are all very surmountable, and in fact we (kichwa coders)
>> would like to go ahead and do something about it.
>> > Jonah is currently working on a fork of IDataset, which will be
>> a subset of the current functionality available in the DAWNSci
>> plug-ins. We need to work with the folks at Diamond about the
>> best way to structure this so it can be reintegrated with DAWNSci
>> & other Diamond projects, but still be re-usable for SWG.
>> >
>> > The goal will be to get this into a stage where it can be
>> easily IP-checked and meets this criteria
>> > - available in an Eclipse build infrastructure
>> > - has a reproducible build from source
>> > - IP checked & clean
>> > - generates a p2 site
>> > - can be reusued in other projects in different formats (jars,
>> oomph installers, the whole works)
>> >
>> > Where it is made available is not nearly as relevant as that it
>> is made relevant (but FWIW, we would favour a top-level science
>> project).
>> >
>> > We'd like to work with all in the community to push this
>> forward so will send a pointer to the fork by the end of the week
>> and welcome comments.
>> >
>> > Oh, and our working title is org.eclipse.dataset (because we
>> think this is key for any data whatsoever, even beyond science).
>> >
>> > Regards,
>> > Tracy
>> >
>> >
>> >
>> > ~~~
>> > Jonah Graham
>> > Kichwa Coders Ltd.
>> > www.kichwacoders.com <http://www.kichwacoders.com>
>> > jonah@xxxxxxxxxxxxxxxx <mailto:jonah@xxxxxxxxxxxxxxxx>
>> > +44 (0) 1494523014 <tel:%2B44%20%280%29%201494523014>
>> > +44 (0) 7988836817 <tel:%2B44%20%280%29%207988836817>
>> >
>> > On 1 September 2015 at 17:10, <Matt.Gerring@xxxxxxxxxxxxx
>> <mailto:Matt.Gerring@xxxxxxxxxxxxx>> wrote:
>> > Jay,
>> >
>> >
>> >
>> > What are the IP issues? How do official releases vs. other
>> kinds of releases help? J
>> >
>> >
>> >
>> > Now that the two plugins are in DAWNSci* does the rest of
>> DAWNSci have to pass IP checking? That should be ok, there is not
>> much which is non-eclipse and the CQ?s are raised. If EAVP was
>> waiting for release on DAWNSci it should be possible to do that
>> with little risk.
>> >
>> >
>> >
>> > Matt
>> >
>> >
>> >
>> > *(Which state of affairs I was pushed into somewhat, if you
>> remember I did say that they should be a community project.)
>> >
>> >
>> >
>> > From: science-iwg-bounces@xxxxxxxxxxx
>> <mailto:science-iwg-bounces@xxxxxxxxxxx>
>> [mailto:science-iwg-bounces@xxxxxxxxxxx
>> <mailto:science-iwg-bounces@xxxxxxxxxxx>] On Behalf Of Jay Jay
>> Billings
>> > Sent: 01 September 2015 16:44
>> >
>> >
>> > To: Science Industry Working Group
>> > Subject: Re: [science-iwg] IDataSet in EAVP
>> >
>> >
>> >
>> > Matt,
>> >
>> > The biggest concern, in my mind, with using it from DAWNSci is
>> not the license, but the IP issues. If you are planning an
>> official release of DAWNSci, then all those issues (eventually)
>> go away. However, if not, then it isn't clear how we could
>> "legally" (procedural sense) use it.
>> >
>> > Jay
>> >
>> > On Sep 1, 2015 11:12 AM, <Matt.Gerring@xxxxxxxxxxxxx
>> <mailto:Matt.Gerring@xxxxxxxxxxxxx>> wrote:
>> >
>> > Jay,
>> >
>> >
>> >
>> > >> the whole community, me included, really wants to be able to
>> use IDataSet with EAVP
>> >
>> > Perfect! What is the blocker in using it right now?
>> >
>> >
>> >
>> > I suggest to put it in EAVP from where it is, it is there on
>> the eclipse github and EPL licensed. Import statements can be
>> changed later and eclipse projects can depend on one another
>> after all.
>> >
>> >
>> >
>> > Tim suggested sciDataset, so perhaps the eventual
>> package/project could be org.eclipse.scidataset? But before we do
>> that, I would like to quantify the need from the community of its
>> location.
>> >
>> >
>> >
>> > Given that it is two plugins, it should be pretty easy to
>> create a project for it faster than EAVP and ensure that EAVP can
>> make its targets for releasing with the Eclipse train; if that is
>> the blocker.
>> >
>> >
>> >
>> > Best Regards,
>> >
>> >
>> >
>> > Matt
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > From: science-iwg-bounces@xxxxxxxxxxx
>> <mailto:science-iwg-bounces@xxxxxxxxxxx>
>> [mailto:science-iwg-bounces@xxxxxxxxxxx
>> <mailto:science-iwg-bounces@xxxxxxxxxxx>] On Behalf Of Jay Jay
>> Billings
>> > Sent: 01 September 2015 14:56
>> > To: Science Industry Working Group
>> > Subject: Re: [science-iwg] IDataSet in EAVP
>> >
>> >
>> >
>> > Matt,
>> >
>> > Will you please clarify your response and be a little more
>> definitive please? My feeling is that the the whole community, me
>> included, really wants to be able to use IDataSet with EAVP. I am
>> essentially trying to figure out if this is something that we can
>> plan and push for as part of our 1.0 release in 6-12 months or if
>> it needs to wait until a 2.0 release sometime after that.
>> >
>> > So, are you saying that you don't know what you're going to do
>> with it yet? Or are you saying that you want to propose a new
>> "data structures" project that we could wait on?
>> >
>> > Even a statement of "I need more time to investigate this"
>> would be fine, just as long as you say it unequivocally, because
>> then I can put it on the potential feature set for the EAVP 2.0
>> release next year and leave it out of 1.0.
>> >
>> > Jay
>> >
>> >
>> >
>> > On Tue, Sep 1, 2015 at 6:04 AM, <Matt.Gerring@xxxxxxxxxxxxx
>> <mailto:Matt.Gerring@xxxxxxxxxxxxx>> wrote:
>> >
>> > Hello Jay and other folks,
>> >
>> >
>> >
>> > >> Alex noted that you felt like the "moving IDataSet" option
>> would make EAVP a competitor to DAWNSci.
>> >
>> > To clarify, I said was that at the point where EAVP does not
>> use IDataset, it becomes a competitor to DAWNSci J This is
>> because the numerics become less compatible but the scope is the
>> same. If the numerics are compatible it becomes a collaborator
>> with a similar scope.
>> >
>> >
>> >
>> > I think our vision for IDataset is not a visualization specific
>> project but something more general, just as numpy is for python.
>> IDataset is two plugins with few dependencies. Suggestions about
>> where it sits not specific to any single use case (visualization,
>> numerics, scripting, etc) are welcome.
>> >
>> >
>> >
>> > If we do move, we might need a name like numjy, jDataset etc.
>> for the project, again suggestions welcome! There would also be
>> some Diamond internal work for me to consider and agree, likely
>> not a blocker although I might ask external people register
>> specific interest in how they will reuse it and why they want it
>> moved.
>> >
>> >
>> >
>> > I also reported in the meeting that IDataset is complete but
>> can be faster for some operations. When/if speed issues arise,
>> there are a number of strategies which can be used already or new
>> code added J It can deal with data much larger than available
>> memory, which is a powerful way at looking at large data on small
>> computers.
>> >
>> >
>> >
>> > Matt
>> >
>> >
>> >
>> > From: science-iwg-bounces@xxxxxxxxxxx
>> <mailto:science-iwg-bounces@xxxxxxxxxxx>
>> [mailto:science-iwg-bounces@xxxxxxxxxxx
>> <mailto:science-iwg-bounces@xxxxxxxxxxx>] On Behalf Of Jay Jay
>> Billings
>> > Sent: 31 August 2015 20:41
>> > To: Science Industry Working Group
>> > Subject: Re: [science-iwg] IDataSet in EAVP
>> >
>> >
>> >
>> > Everyone,
>> >
>> > I should clarify that EAVP is short for "the Eclipse Advanced
>> Visualization Project" for those who haven't heard of it before.
>> It is the viz project that we have been discussing for awhile.
>> >
>> > Jau
>> >
>> > On Aug 31, 2015 2:01 PM, "Jay Jay Billings"
>> <jayjaybillings@xxxxxxxxx <mailto:jayjaybillings@xxxxxxxxx>> wrote:
>> >
>> > Matt,
>> >
>> > I met with Alex this morning and he told me how awesome the
>> Norway meeting was. He also told me that there is broad support
>> for using IDataSet in EAVP. I would like to discuss how we can
>> move forward on this. I strongly support the use of IDataSet in
>> EAVP and I really want to see a tight integration because it is
>> technology that would be very valuable to what we are doing at
>> ORNL and, I imagine, other institutions in this group. There is a
>> very big technical question that we need to answer though: Where
>> is IDataSet going to be released?
>> >
>> >
>> > In one of our previous steering committee calls you said that
>> DIAMOND has no intention of doing a "full Eclipse release" of
>> DAWNSci since your goal with the project was only to release the
>> source code, not binaries. That makes it very challenging to use
>> anything in DAWNSci in any other SWG project because it lacks
>> that Eclipse "technical goodness" that many of our projects
>> depend on.
>> >
>> > So, if you don't mind, let's start there. What are your release
>> plans for DAWNSci? If you are planning an official, Eclipse.org
>> release then I think we can use IDataSet as it is and just wait
>> on your release. However, if not or if not reasonably soon, then
>> I think you should seriously consider contributing it to EAVP so
>> that the community can use it and help develop it with you.
>> >
>> > Alex noted that you felt like the "moving IDataSet" option
>> would make EAVP a competitor to DAWNSci. I don't personally see
>> that since no other Eclipse projects are competitors to DAWNSci.
>> I urge you not to look at the project that way either. It is
>> meant to be an open, community effort that we all work on - and
>> use - together.
>> >
>> > Please let me know what you think and let's keep this
>> discussion in the open so that we can get good thoughts and ideas
>> from the community. I'm certain there is a workable path forward
>> on this.
>> >
>> > Jay
>> >
>> >
>> > --
>> >
>> > Jay Jay Billings
>> >
>> > Oak Ridge National Laboratory
>> >
>> > Twitter Handle: @jayjaybillings
>> >
>> >
>> >
>> > --
>> >
>> > This e-mail and any attachments may contain confidential,
>> copyright and or privileged material, and are for the use of the
>> intended addressee only. If you are not the intended addressee or
>> an authorised recipient of the addressee please notify us of
>> receipt by returning the e-mail and do not use, copy, retain,
>> distribute or disclose the information in or attached to the e-mail.
>> > Any opinions expressed within this e-mail are those of the
>> individual and not necessarily of Diamond Light Source Ltd.
>> > Diamond Light Source Ltd. cannot guarantee that this e-mail or
>> any attachments are free from viruses and we cannot accept
>> liability for any damage which you may sustain as a result of
>> software viruses which may be transmitted in or with the message.
>> > Diamond Light Source Limited (company no. 4375679). Registered
>> in England and Wales with its registered office at Diamond House,
>> Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11
>> 0DE, United Kingdom
>> >
>> >
>> >
>> > _______________________________________________
>> > science-iwg mailing list
>> > science-iwg@xxxxxxxxxxx <mailto:science-iwg@xxxxxxxxxxx>
>> > To change your delivery options, retrieve your password, or
>> unsubscribe from this list, visit
>> > https://dev.eclipse.org/mailman/listinfo/science-iwg
>> >
>> >
>> >
>> >
>> > --
>> >
>> > Jay Jay Billings
>> >
>> > Oak Ridge National Laboratory
>> >
>> > Twitter Handle: @jayjaybillings
>> >
>> >
>> >
>> > --
>> >
>> > This e-mail and any attachments may contain confidential,
>> copyright and or privileged material, and are for the use of the
>> intended addressee only. If you are not the intended addressee or
>> an authorised recipient of the addressee please notify us of
>> receipt by returning the e-mail and do not use, copy, retain,
>> distribute or disclose the information in or attached to the e-mail.
>> > Any opinions expressed within this e-mail are those of the
>> individual and not necessarily of Diamond Light Source Ltd.
>> > Diamond Light Source Ltd. cannot guarantee that this e-mail or
>> any attachments are free from viruses and we cannot accept
>> liability for any damage which you may sustain as a result of
>> software viruses which may be transmitted in or with the message.
>> > Diamond Light Source Limited (company no. 4375679). Registered
>> in England and Wales with its registered office at Diamond House,
>> Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11
>> 0DE, United Kingdom
>> >
>> >
>> >
>> > _______________________________________________
>> > science-iwg mailing list
>> > science-iwg@xxxxxxxxxxx <mailto:science-iwg@xxxxxxxxxxx>
>> > To change your delivery options, retrieve your password, or
>> unsubscribe from this list, visit
>> > https://dev.eclipse.org/mailman/listinfo/science-iwg
>> >
>> >
>> > --
>> >
>> > This e-mail and any attachments may contain confidential,
>> copyright and or privileged material, and are for the use of the
>> intended addressee only. If you are not the intended addressee or
>> an authorised recipient of the addressee please notify us of
>> receipt by returning the e-mail and do not use, copy, retain,
>> distribute or disclose the information in or attached to the e-mail.
>> > Any opinions expressed within this e-mail are those of the
>> individual and not necessarily of Diamond Light Source Ltd.
>> > Diamond Light Source Ltd. cannot guarantee that this e-mail or
>> any attachments are free from viruses and we cannot accept
>> liability for any damage which you may sustain as a result of
>> software viruses which may be transmitted in or with the message.
>> > Diamond Light Source Limited (company no. 4375679). Registered
>> in England and Wales with its registered office at Diamond House,
>> Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11
>> 0DE, United Kingdom
>> >
>> >
>> >
>> > _______________________________________________
>> > science-iwg mailing list
>> > science-iwg@xxxxxxxxxxx <mailto:science-iwg@xxxxxxxxxxx>
>> > To change your delivery options, retrieve your password, or
>> unsubscribe from this list, visit
>> > https://dev.eclipse.org/mailman/listinfo/science-iwg
>> >
>> > _______________________________________________
>> > science-iwg mailing list
>> > science-iwg@xxxxxxxxxxx <mailto:science-iwg@xxxxxxxxxxx>
>> > To change your delivery options, retrieve your password, or
>> unsubscribe from this list, visit
>> > https://dev.eclipse.org/mailman/listinfo/science-iwg
>>
>>
>> _______________________________________________
>> science-iwg mailing list
>> science-iwg@xxxxxxxxxxx <mailto:science-iwg@xxxxxxxxxxx>
>> To change your delivery options, retrieve your password, or
>> unsubscribe from this list, visit
>> https://dev.eclipse.org/mailman/listinfo/science-iwg
>>
>> --
>>
>> This e-mail and any attachments may contain confidential,
>> copyright and or privileged material, and are for the use of the
>> intended addressee only. If you are not the intended addressee or
>> an authorised recipient of the addressee please notify us of
>> receipt by returning the e-mail and do not use, copy, retain,
>> distribute or disclose the information in or attached to the e-mail.
>> Any opinions expressed within this e-mail are those of the
>> individual and not necessarily of Diamond Light Source Ltd.
>> Diamond Light Source Ltd. cannot guarantee that this e-mail or
>> any attachments are free from viruses and we cannot accept
>> liability for any damage which you may sustain as a result of
>> software viruses which may be transmitted in or with the message.
>> Diamond Light Source Limited (company no. 4375679). Registered in
>> England and Wales with its registered office at Diamond House,
>> Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11
>> 0DE, United Kingdom
>>
>>
>>
>> _______________________________________________
>> science-iwg mailing list
>> science-iwg@xxxxxxxxxxx <mailto:science-iwg@xxxxxxxxxxx>
>> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
>> https://dev.eclipse.org/mailman/listinfo/science-iwg
>
> --
> ~~~~~~~~~~~~~~~~~~~~~~~~
> OpenChrom - the open source alternative for chromatography / mass spectrometry
> Dr. Philip Wenig ? Founder ?philip.wenig@xxxxxxxxxxxxx <mailto:philip.wenig@xxxxxxxxxxxxx> ?http://www.openchrom.net
> ~~~~~~~~~~~~~~~~~~~~~~~~
>
>
> _______________________________________________
> science-iwg mailing list
> science-iwg@xxxxxxxxxxx <mailto:science-iwg@xxxxxxxxxxx>
> To change your delivery options, retrieve your password, or
> unsubscribe from this list, visit
> https://dev.eclipse.org/mailman/listinfo/science-iwg
>
>
>
> _______________________________________________
> science-iwg mailing list
> science-iwg@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> https://dev.eclipse.org/mailman/listinfo/science-iwg
--
~~~~~~~~~~~~~~~~~~~~~~~~
OpenChrom - the open source alternative for chromatography / mass spectrometry
Dr. Philip Wenig ? Founder ? philip.wenig@xxxxxxxxxxxxx ? http://www.openchrom.net
~~~~~~~~~~~~~~~~~~~~~~~~
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://dev.eclipse.org/mailman/private/science-iwg/attachments/20150902/75a16fbe/attachment.html>
------------------------------
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
End of science-iwg Digest, Vol 32, Issue 22
*******************************************