Apache Commons Math is great library ... but it's hard
getting it through the IP review process. I've already file
a CQ to bring Commons Math 3.5 to Eclipse Orbit: http://dev.eclipse.org/ipzilla/show_bug.cgi?id=9724
Somehow, "AbstractWell.java" can't be accepted. Removing
causes a plethora of subsequent problems. I'm trying to
resolve that issue at the moment.
In
terms of bikeshedding a package name, I don’t think
it is a big deal even for Diamond. Searching through
our scripts finds less than 100 of them import of
the api and impl dataset packages.
Most
of the other issues are eminently solvable. The
exception would be dependencies: if the jars are
separated out then this should not be an issue. Or
have I misunderstood and there are more problems?
(Why is AC Maths bad?)
I am currently moving the
plug-ins to a new repository (actually not
really a different one, but a fork with the
non-relevant code deleted, that allows for
maintaining the history). I am trying to make
this repo the minimum that enables people to use
datasets without requiring any additional
dependencies. It would be great to get this as
part of a top level project.
By having a repo with a
smaller code base, hopefully it can be API
versioned sooner and IP reviewed in such a way
to solve the EAVP dependency issue.
There is still how to manage
that code base back in Diamond, in particular
moving datasets to a new package has a cost to
Diamond because there are many Jython scripts in
the wild that refer to them.
Jonah
On 1 September 2015 at
17:55, Torkild Ulvøy Resheim <torkildr@xxxxxxxxx>
wrote:
Hi all,
I wonder if it would make any sense to move
the two plug-ins in question to a separate
repository and namespace and do the IP-review
on those. Moving these should also rid them of
dependencies such as Apache Commons Math which
i understand is problematic. I like the
namespace “org.eclipse.dataset” but it may be
a bit to generic. In preparation for a top
level science project we may want to call it
“org.eclipse.science.dataset”.
Best regards,
Torkild
> 1. sep. 2015 kl. 18.39 skrev Jonah
Graham <jonah@xxxxxxxxxxxxxxxx>:
>
> Hi all,
>
> If I can chime in with our 2 cents.
>
> Following the SWG meeting in
Trondheim it was great to see the
community get behind the need for a
standardised data api. Based on what I
know of IDataset (and the way it matches
numpy) I am convinced that is the way
forward. However, I do see there is work
to be done to get there.
>
> The issues we see in particular are:
> 1. There are precompiled jars in the
current dataset plugin - this must be
resolved so it can be all source.
> 2. There are dependencies on
non-orbit approved 3rd party packages- we
started discussing in Trondheim how some
of these can be resolve.
> 3. There are 2 plugins & the api
plugin comes with lots of other
(non-relevant) apis
> 4. The code has not been IP-review
and established as IP clean.
>
> These are all very surmountable, and
in fact we (kichwa coders) would like to
go ahead and do something about it.
> Jonah is currently working on a fork
of IDataset, which will be a subset of the
current functionality available in the
DAWNSci plug-ins. We need to work with the
folks at Diamond about the best way to
structure this so it can be reintegrated
with DAWNSci & other Diamond projects,
but still be re-usable for SWG.
>
> The goal will be to get this into a
stage where it can be easily IP-checked
and meets this criteria
> - available in an Eclipse build
infrastructure
> - has a reproducible build from
source
> - IP checked & clean
> - generates a p2 site
> - can be reusued in other projects in
different formats (jars, oomph installers,
the whole works)
>
> Where it is made available is not
nearly as relevant as that it is made
relevant (but FWIW, we would favour a
top-level science project).
>
> We'd like to work with all in the
community to push this forward so will
send a pointer to the fork by the end of
the week and welcome comments.
>
> Oh, and our working title is
org.eclipse.dataset (because we think this
is key for any data whatsoever, even
beyond science).
>
> Regards,
> Tracy
>
>
>
> ~~~
> Jonah Graham
> Kichwa Coders Ltd.
> www.kichwacoders.com
> jonah@xxxxxxxxxxxxxxxx
> +44 (0) 1494523014
> +44 (0) 7988836817
>
> On 1 September 2015 at 17:10, <Matt.Gerring@xxxxxxxxxxxxx>
wrote:
> Jay,
>
>
>
> What are the IP issues? How do
official releases vs. other kinds of
releases help? J
>
>
>
> Now that the two plugins are in
DAWNSci* does the rest of DAWNSci have to
pass IP checking? That should be ok, there
is not much which is non-eclipse and the
CQ’s are raised. If EAVP was waiting for
release on DAWNSci it should be possible
to do that with little risk.
>
>
>
> Matt
>
>
>
> *(Which state of affairs I was pushed
into somewhat, if you remember I did say
that they should be a community project.)
>
>
>
> From: science-iwg-bounces@xxxxxxxxxxx
[mailto:science-iwg-bounces@xxxxxxxxxxx]
On Behalf Of Jay Jay Billings
> Sent: 01 September 2015 16:44
>
>
> To: Science Industry Working Group
> Subject: Re: [science-iwg] IDataSet
in EAVP
>
>
>
> Matt,
>
> The biggest concern, in my mind, with
using it from DAWNSci is not the license,
but the IP issues. If you are planning an
official release of DAWNSci, then all
those issues (eventually) go away.
However, if not, then it isn't clear how
we could "legally" (procedural sense) use
it.
>
> Jay
>
> On Sep 1, 2015 11:12 AM, <Matt.Gerring@xxxxxxxxxxxxx>
wrote:
>
> Jay,
>
>
>
> >> the whole community, me
included, really wants to be able to use
IDataSet with EAVP
>
> Perfect! What is the blocker in using
it right now?
>
>
>
> I suggest to put it in EAVP from
where it is, it is there on the eclipse
github and EPL licensed. Import statements
can be changed later and eclipse projects
can depend on one another after all.
>
>
>
> Tim suggested sciDataset, so perhaps
the eventual package/project could be
org.eclipse.scidataset? But before we do
that, I would like to quantify the need
from the community of its location.
>
>
>
> Given that it is two plugins, it
should be pretty easy to create a project
for it faster than EAVP and ensure that
EAVP can make its targets for releasing
with the Eclipse train; if that is the
blocker.
>
>
>
> Best Regards,
>
>
>
> Matt
>
>
>
>
>
>
>
> From: science-iwg-bounces@xxxxxxxxxxx
[mailto:science-iwg-bounces@xxxxxxxxxxx]
On Behalf Of Jay Jay Billings
> Sent: 01 September 2015 14:56
> To: Science Industry Working Group
> Subject: Re: [science-iwg] IDataSet
in EAVP
>
>
>
> Matt,
>
> Will you please clarify your response
and be a little more definitive please? My
feeling is that the the whole community,
me included, really wants to be able to
use IDataSet with EAVP. I am essentially
trying to figure out if this is something
that we can plan and push for as part of
our 1.0 release in 6-12 months or if it
needs to wait until a 2.0 release sometime
after that.
>
> So, are you saying that you don't
know what you're going to do with it yet?
Or are you saying that you want to propose
a new "data structures" project that we
could wait on?
>
> Even a statement of "I need more time
to investigate this" would be fine, just
as long as you say it unequivocally,
because then I can put it on the potential
feature set for the EAVP 2.0 release next
year and leave it out of 1.0.
>
> Jay
>
>
>
> On Tue, Sep 1, 2015 at 6:04 AM, <Matt.Gerring@xxxxxxxxxxxxx>
wrote:
>
> Hello Jay and other folks,
>
>
>
> >> Alex noted that you felt
like the "moving IDataSet" option would
make EAVP a competitor to DAWNSci.
>
> To clarify, I said was that at the
point where EAVP does not use IDataset, it
becomes a competitor to DAWNSci J This is
because the numerics become less
compatible but the scope is the same. If
the numerics are compatible it becomes a
collaborator with a similar scope.
>
>
>
> I think our vision for IDataset is
not a visualization specific project but
something more general, just as numpy is
for python. IDataset is two plugins with
few dependencies. Suggestions about where
it sits not specific to any single use
case (visualization, numerics, scripting,
etc) are welcome.
>
>
>
> If we do move, we might need a name
like numjy, jDataset etc. for the project,
again suggestions welcome! There would
also be some Diamond internal work for me
to consider and agree, likely not a
blocker although I might ask external
people register specific interest in how
they will reuse it and why they want it
moved.
>
>
>
> I also reported in the meeting that
IDataset is complete but can be faster for
some operations. When/if speed issues
arise, there are a number of strategies
which can be used already or new code
added J It can deal with data much larger
than available memory, which is a powerful
way at looking at large data on small
computers.
>
>
>
> Matt
>
>
>
> From: science-iwg-bounces@xxxxxxxxxxx
[mailto:science-iwg-bounces@xxxxxxxxxxx]
On Behalf Of Jay Jay Billings
> Sent: 31 August 2015 20:41
> To: Science Industry Working Group
> Subject: Re: [science-iwg] IDataSet
in EAVP
>
>
>
> Everyone,
>
> I should clarify that EAVP is short
for "the Eclipse Advanced Visualization
Project" for those who haven't heard of it
before. It is the viz project that we have
been discussing for awhile.
>
> Jau
>
> On Aug 31, 2015 2:01 PM, "Jay Jay
Billings" <jayjaybillings@xxxxxxxxx>
wrote:
>
> Matt,
>
> I met with Alex this morning and he
told me how awesome the Norway meeting
was. He also told me that there is broad
support for using IDataSet in EAVP. I
would like to discuss how we can move
forward on this. I strongly support the
use of IDataSet in EAVP and I really want
to see a tight integration because it is
technology that would be very valuable to
what we are doing at ORNL and, I imagine,
other institutions in this group. There is
a very big technical question that we need
to answer though: Where is IDataSet going
to be released?
>
>
> In one of our previous steering
committee calls you said that DIAMOND has
no intention of doing a "full Eclipse
release" of DAWNSci since your goal with
the project was only to release the source
code, not binaries. That makes it very
challenging to use anything in DAWNSci in
any other SWG project because it lacks
that Eclipse "technical goodness" that
many of our projects depend on.
>
> So, if you don't mind, let's start
there. What are your release plans for
DAWNSci? If you are planning an official,
Eclipse.org release then I think we can
use IDataSet as it is and just wait on
your release. However, if not or if not
reasonably soon, then I think you should
seriously consider contributing it to EAVP
so that the community can use it and help
develop it with you.
>
> Alex noted that you felt like the
"moving IDataSet" option would make EAVP a
competitor to DAWNSci. I don't personally
see that since no other Eclipse projects
are competitors to DAWNSci. I urge you not
to look at the project that way either. It
is meant to be an open, community effort
that we all work on - and use - together.
>
> Please let me know what you think and
let's keep this discussion in the open so
that we can get good thoughts and ideas
from the community. I'm certain there is a
workable path forward on this.
>
> Jay
>
>
> --
>
> Jay Jay Billings
>
> Oak Ridge National Laboratory
>
> Twitter Handle: @jayjaybillings
>
>
>
> --
>
> This e-mail and any attachments may
contain confidential, copyright and or
privileged material, and are for the use
of the intended addressee only. If you are
not the intended addressee or an
authorised recipient of the addressee
please notify us of receipt by returning
the e-mail and do not use, copy, retain,
distribute or disclose the information in
or attached to the e-mail.
> Any opinions expressed within this
e-mail are those of the individual and not
necessarily of Diamond Light Source Ltd.
> Diamond Light Source Ltd. cannot
guarantee that this e-mail or any
attachments are free from viruses and we
cannot accept liability for any damage
which you may sustain as a result of
software viruses which may be transmitted
in or with the message.
> Diamond Light Source Limited (company
no. 4375679). Registered in England and
Wales with its registered office at
Diamond House, Harwell Science and
Innovation Campus, Didcot, Oxfordshire,
OX11 0DE, United Kingdom
>
>
>
>
_______________________________________________
> science-iwg mailing list
> science-iwg@xxxxxxxxxxx
> To change your delivery options,
retrieve your password, or unsubscribe
from this list, visit
>
https://dev.eclipse.org/mailman/listinfo/science-iwg
>
>
>
>
> --
>
> Jay Jay Billings
>
> Oak Ridge National Laboratory
>
> Twitter Handle: @jayjaybillings
>
>
>
> --
>
> This e-mail and any attachments may
contain confidential, copyright and or
privileged material, and are for the use
of the intended addressee only. If you are
not the intended addressee or an
authorised recipient of the addressee
please notify us of receipt by returning
the e-mail and do not use, copy, retain,
distribute or disclose the information in
or attached to the e-mail.
> Any opinions expressed within this
e-mail are those of the individual and not
necessarily of Diamond Light Source Ltd.
> Diamond Light Source Ltd. cannot
guarantee that this e-mail or any
attachments are free from viruses and we
cannot accept liability for any damage
which you may sustain as a result of
software viruses which may be transmitted
in or with the message.
> Diamond Light Source Limited (company
no. 4375679). Registered in England and
Wales with its registered office at
Diamond House, Harwell Science and
Innovation Campus, Didcot, Oxfordshire,
OX11 0DE, United Kingdom
>
>
>
>
_______________________________________________
> science-iwg mailing list
> science-iwg@xxxxxxxxxxx
> To change your delivery options,
retrieve your password, or unsubscribe
from this list, visit
>
https://dev.eclipse.org/mailman/listinfo/science-iwg
>
>
> --
>
> This e-mail and any attachments may
contain confidential, copyright and or
privileged material, and are for the use
of the intended addressee only. If you are
not the intended addressee or an
authorised recipient of the addressee
please notify us of receipt by returning
the e-mail and do not use, copy, retain,
distribute or disclose the information in
or attached to the e-mail.
> Any opinions expressed within this
e-mail are those of the individual and not
necessarily of Diamond Light Source Ltd.
> Diamond Light Source Ltd. cannot
guarantee that this e-mail or any
attachments are free from viruses and we
cannot accept liability for any damage
which you may sustain as a result of
software viruses which may be transmitted
in or with the message.
> Diamond Light Source Limited (company
no. 4375679). Registered in England and
Wales with its registered office at
Diamond House, Harwell Science and
Innovation Campus, Didcot, Oxfordshire,
OX11 0DE, United Kingdom
>
>
>
>
_______________________________________________
> science-iwg mailing list
> science-iwg@xxxxxxxxxxx
> To change your delivery options,
retrieve your password, or unsubscribe
from this list, visit
>
https://dev.eclipse.org/mailman/listinfo/science-iwg
>
>
_______________________________________________
> science-iwg mailing list
> science-iwg@xxxxxxxxxxx
> To change your delivery options,
retrieve your password, or unsubscribe
from this list, visit
>
https://dev.eclipse.org/mailman/listinfo/science-iwg
This e-mail and any attachments may
contain confidential, copyright and or privileged
material, and are for the use of the intended addressee
only. If you are not the intended addressee or an
authorised recipient of the addressee please notify us
of receipt by returning the e-mail and do not use, copy,
retain, distribute or disclose the information in or
attached to the e-mail.
Any opinions expressed within this e-mail are those of
the individual and not necessarily of Diamond Light
Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this
e-mail or any attachments are free from viruses and we
cannot accept liability for any damage which you may
sustain as a result of software viruses which may be
transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679).
Registered in England and Wales with its registered
office at Diamond House, Harwell Science and Innovation
Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
--
~~~~~~~~~~~~~~~~~~~~~~~~
OpenChrom - the open source alternative for chromatography / mass spectrometry
Dr. Philip Wenig » Founder » philip.wenig@xxxxxxxxxxxxx » http://www.openchrom.net
~~~~~~~~~~~~~~~~~~~~~~~~
--
~~~~~~~~~~~~~~~~~~~~~~~~
OpenChrom - the open source alternative for chromatography / mass spectrometry
Dr. Philip Wenig » Founder » philip.wenig@xxxxxxxxxxxxx » http://www.openchrom.net
~~~~~~~~~~~~~~~~~~~~~~~~