Re: [january-dev] Support for sparse arrays?

[<Peter.Chang@xxxxxxxxxxxxx>]

No, there is no sparse support.

A sparse implementation of IDataset is entirely possible using something like a sparse array (https://developer.android.com/reference/android/util/SparseArray.html) to store the data.

In the meantime, filling the empty dataset with NaNs is a standard practice for marking missing data. The statistical methods allow you to ignore the NaNs.

Regards,
 Peter


-----Original Message-----
From: january-dev-bounces@xxxxxxxxxxx [mailto:january-dev-bounces@xxxxxxxxxxx] On Behalf Of Ian Mayo
Sent: 15 June 2017 10:03
To: january developer discussions <january-dev@xxxxxxxxxxx>
Subject: [january-dev] Support for sparse arrays?

Hi all,
I wish to store some measurements in a 2-D array.

We can imagine these measurements as being a series of fuel consumption measurements, each accompanied by the measurements of speed and engine revs.  I wish to bin this data, and produce a grid of measured fuel consumption (litre/sec) indexed by speed (m/sec) in one axis and engine revs (/sec) in the other.

The January dataset constructors seem to assume that values are present for all cells.

But, my measurements are infrequent, resulting in a relatively sparse array.

I could provide zeros for the cells that don't have measurements, but does January offer any alternative strategies/working practices?

Note 1: My default strategy would be to create an empty DoubleDataset of the correct dimensions, then work through my measurements and use .setItem(double value, int I, int j) to set the value of specific cells.

Note 2: I don't have concerns about the memory wastage that are resolved through Sparse Matrix libraries, my arrays remain quite small.

Thanks,
Ian
_______________________________________________
january-dev mailing list
january-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit https://dev.eclipse.org/mailman/listinfo/january-dev

-- 
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. 
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom