Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [science-iwg] IDataSet in EAVP

Peter is awesome! Yes, we welcome his input & vision.

On 2 September 2015 at 09:55, <Matt.Gerring@xxxxxxxxxxxxx> wrote:

Hello,

 

It is not my call what IDataset plugins should be called, although it might cause some cost to the core team. I suggest that the community would be best served by Peter Chang being the eclipse project lead for the dataset plugins, you will notice he is the original author on the vast majority of the classes. He has consistently chosen powerful designs which have been computationally efficient and powerful when I use them.

 

In terms of forks or clean ups or renames, I suggest that Peter be involved. Anyone whom forks away from what Peter plans for future developments would be at a disadvantage.

 

Best Regards,

 

Matt

 

From: science-iwg-bounces@xxxxxxxxxxx [mailto:science-iwg-bounces@xxxxxxxxxxx] On Behalf Of Philip Wenig
Sent: 02 September 2015 07:24
To: science-iwg@xxxxxxxxxxx


Subject: Re: [science-iwg] IDataSet in EAVP

 

Hi folks,

I would also prefer using the IDN "org.eclipse.science.dataset".


Best,
Philip

Am 01.09.2015 um 18:55 schrieb Torkild Ulvøy Resheim:

Hi all,
 
I wonder if it would make any sense to move the two plug-ins in question to a separate repository and namespace and do the IP-review on those. Moving these should also rid them of dependencies such as Apache Commons Math which i understand is problematic. I like the namespace “org.eclipse.dataset” but it may be a bit to generic. In preparation for a top level science project we may want to call it “org.eclipse.science.dataset”.
 
Best regards,
Torkild
 
1. sep. 2015 kl. 18.39 skrev Jonah Graham <jonah@xxxxxxxxxxxxxxxx>:
 
Hi all,
 
If I can chime in with our 2 cents. 
 
Following the SWG meeting in Trondheim it was great to see the community get behind the need for a standardised data api. Based on what I know of IDataset (and the way it matches numpy) I am convinced that is the way forward. However, I do see there is work to be done to get there. 
 
The issues we see in particular are:
1. There are precompiled jars in the current dataset plugin - this must be resolved so it can be all source.
2. There are dependencies on non-orbit approved 3rd party packages- we started discussing in Trondheim how some of these can be resolve. 
3. There are 2 plugins & the api plugin comes with lots of other (non-relevant) apis
4. The code has not been IP-review and established as IP clean. 
 
These are all very surmountable, and in fact we (kichwa coders) would like to go ahead and do something about it. 
Jonah is currently working on a fork of IDataset, which will be a subset of the current functionality available in the DAWNSci plug-ins. We need to work with the folks at Diamond about the best way to structure this so it can be reintegrated with DAWNSci & other Diamond projects, but still be re-usable for SWG. 
 
The goal will be to get this into a stage where it can be easily IP-checked and meets this criteria 
- available in an Eclipse build infrastructure
- has a reproducible build from source
- IP checked & clean
- generates a p2 site
- can be reusued in other projects in different formats (jars, oomph installers, the whole works)
 
Where it is made available is not nearly as relevant as that it is made relevant (but FWIW, we would favour a top-level science project).
 
We'd like to work with all in the community to push this forward so will send a pointer to the fork by the end of the week and welcome comments. 
 
Oh, and our working title is org.eclipse.dataset (because we think this is key for any data whatsoever, even beyond science).
 
Regards,
Tracy
 
 
 
~~~
Jonah Graham
Kichwa Coders Ltd.
www.kichwacoders.com
jonah@xxxxxxxxxxxxxxxx
+44 (0) 1494523014
+44 (0) 7988836817
 
On 1 September 2015 at 17:10, <Matt.Gerring@xxxxxxxxxxxxx> wrote:
Jay,
 
 
 
What are the IP issues? How do official releases vs. other kinds of releases help? J
 
 
 
Now that the two plugins are in DAWNSci* does the rest of DAWNSci have to pass IP checking? That should be ok, there is not much which is non-eclipse and the CQ’s are raised. If EAVP was waiting for release on DAWNSci it should be possible to do that with little risk.
 
 
 
Matt
 
 
 
*(Which state of affairs I was pushed into somewhat, if you remember I did say that they should be a community project.)
 
 
 
From: science-iwg-bounces@xxxxxxxxxxx [mailto:science-iwg-bounces@xxxxxxxxxxx] On Behalf Of Jay Jay Billings
Sent: 01 September 2015 16:44
 
 
To: Science Industry Working Group
Subject: Re: [science-iwg] IDataSet in EAVP
 
 
 
Matt,
 
The biggest concern, in my mind, with using it from DAWNSci is not the license, but the IP issues. If you are planning an official release of DAWNSci, then all those issues (eventually) go away. However, if not, then it isn't clear how we could "legally" (procedural sense) use it.
 
Jay
 
On Sep 1, 2015 11:12 AM, <Matt.Gerring@xxxxxxxxxxxxx> wrote:
 
Jay,
 
 
 
the whole community, me included, really wants to be able to use IDataSet with EAVP
 
Perfect! What is the blocker in using it right now?
 
 
 
I suggest to put it in EAVP from where it is, it is there on the eclipse github and EPL licensed. Import statements can be changed later and eclipse projects can depend on one another after all.
 
 
 
Tim suggested sciDataset, so perhaps the eventual package/project could be org.eclipse.scidataset? But before we do that, I would like to quantify the need from the community of its location.
 
 
 
Given that it is two plugins, it should be pretty easy to create a project for it faster than EAVP and ensure that EAVP can make its targets for releasing with the Eclipse train; if that is the blocker.
 
 
 
Best Regards,
 
 
 
Matt
 
 
 
 
 
 
 
From: science-iwg-bounces@xxxxxxxxxxx [mailto:science-iwg-bounces@xxxxxxxxxxx] On Behalf Of Jay Jay Billings
Sent: 01 September 2015 14:56
To: Science Industry Working Group
Subject: Re: [science-iwg] IDataSet in EAVP
 
 
 
Matt,
 
Will you please clarify your response and be a little more definitive please? My feeling is that the the whole community, me included, really wants to be able to use IDataSet with EAVP. I am essentially trying to figure out if this is something that we can plan and push for as part of our 1.0 release in 6-12 months or if it needs to wait until a 2.0 release sometime after that.
 
So, are you saying that you don't know what you're going to do with it yet? Or are you saying that you want to propose a new "data structures" project that we could wait on?
 
Even a statement of "I need more time to investigate this" would be fine, just as long as you say it unequivocally, because then I can put it on the potential feature set for the EAVP 2.0 release next year and leave it out of 1.0.
 
Jay
 
 
 
On Tue, Sep 1, 2015 at 6:04 AM, <Matt.Gerring@xxxxxxxxxxxxx> wrote:
 
Hello Jay and other folks,
 
 
 
Alex noted that you felt like the "moving IDataSet" option would make EAVP a competitor to DAWNSci.
 
To clarify, I said was that at the point where EAVP does not use IDataset, it becomes a competitor to DAWNSci J This is because the numerics become less compatible but the scope is the same. If the numerics are compatible it becomes a collaborator with a similar scope.
 
 
 
I think our vision for IDataset is not a visualization specific project but something more general, just as numpy is for python. IDataset is two plugins with few dependencies. Suggestions about where it sits not specific to any single use case (visualization, numerics, scripting, etc) are welcome.
 
 
 
If we do move, we might need a name like numjy, jDataset etc. for the project, again suggestions welcome! There would also be some Diamond internal work for me to consider and agree, likely not a blocker although I might ask external people register specific interest in how they will reuse it and why they want it moved.
 
 
 
I also reported in the meeting that IDataset is complete but can be faster for some operations. When/if speed issues arise, there are a number of strategies which can be used already or new code added J It can deal with data much larger than available memory, which is a powerful way at looking at large data on small computers.
 
 
 
Matt
 
 
 
From: science-iwg-bounces@xxxxxxxxxxx [mailto:science-iwg-bounces@xxxxxxxxxxx] On Behalf Of Jay Jay Billings
Sent: 31 August 2015 20:41
To: Science Industry Working Group
Subject: Re: [science-iwg] IDataSet in EAVP
 
 
 
Everyone,
 
I should clarify that EAVP is short for "the Eclipse Advanced Visualization Project" for those who haven't heard of it before. It is the viz project that we have been discussing for awhile.
 
Jau
 
On Aug 31, 2015 2:01 PM, "Jay Jay Billings" <jayjaybillings@xxxxxxxxx> wrote:
 
Matt,
 
I met with Alex this morning and he told me how awesome the Norway meeting was. He also told me that there is broad support for using IDataSet in EAVP. I would like to discuss how we can move forward on this. I strongly support the use of IDataSet in EAVP and I really want to see a tight integration because it is technology that would be very valuable to what we are doing at ORNL and, I imagine, other institutions in this group. There is a very big technical question that we need to answer though: Where is IDataSet going to be released?
 
 
In one of our previous steering committee calls you said that DIAMOND has no intention of doing a "full Eclipse release" of DAWNSci since your goal with the project was only to release the source code, not binaries. That makes it very challenging to use anything in DAWNSci in any other SWG project because it lacks that Eclipse "technical goodness" that many of our projects depend on.
 
So, if you don't mind, let's start there. What are your release plans for DAWNSci? If you are planning an official, Eclipse.org release then I think we can use IDataSet as it is and just wait on your release. However, if not or if not reasonably soon, then I think you should seriously consider contributing it to EAVP so that the community can use it and help develop it with you. 
 
Alex noted that you felt like the "moving IDataSet" option would make EAVP a competitor to DAWNSci. I don't personally see that since no other Eclipse projects are competitors to DAWNSci. I urge you not to look at the project that way either. It is meant to be an open, community effort that we all work on - and use - together.
 
Please let me know what you think and let's keep this discussion in the open so that we can get good thoughts and ideas from the community. I'm certain there is a workable path forward on this.
 
Jay
 
 
--
 
Jay Jay Billings
 
Oak Ridge National Laboratory
 
Twitter Handle: @jayjaybillings
 
 
 
-- 
 
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. 
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
 
 
 
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
 
 
 
 
--
 
Jay Jay Billings
 
Oak Ridge National Laboratory
 
Twitter Handle: @jayjaybillings
 
 
 
-- 
 
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. 
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
 
 
 
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
 
 
-- 
 
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. 
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
 
 
 
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
 
_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg
 




_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg



-- 
~~~~~~~~~~~~~~~~~~~~~~~~
OpenChrom - the open source alternative for chromatography / mass spectrometry
Dr. Philip Wenig » Founder » philip.wenig@xxxxxxxxxxxxx » http://www.openchrom.net
~~~~~~~~~~~~~~~~~~~~~~~~

 

-- 

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
 


_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg


Back to the top