Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [science-iwg] Eclipse 'Scanning' Project

There’s some documents for getting started at the top of the project:

https://github.com/eclipse/scanning/blob/master/GETTINGSTARTED.pdf

https://github.com/eclipse/scanning/blob/master/SCANANATOMY.pdf

 

Still have to get permission for Toulouse but expect to be there. It is in a new financial year for us so things can always change.

 

M

 

From: science-iwg-bounces@xxxxxxxxxxx [mailto:science-iwg-bounces@xxxxxxxxxxx] On Behalf Of Philip Wenig
Sent: 07 March 2017 07:08
To: science-iwg@xxxxxxxxxxx
Subject: Re: [science-iwg] Eclipse 'Scanning' Project

 

Matt,

very nice. I'd like to learn more about this.
Are you going to the EclipseCon France?


Best,
Philip

Am 06.03.2017 um 10:33 schrieb Matt.Gerring@xxxxxxxxxxxxx:

Hi Philip,

 

We have had the code working for the synchrotron for some time including 3D scans. (height, x, y) is a common scenario, microscopes normally produce images so running a scan would produce a 5D HDF5 dataset written to the NeXus standard including metadata and information such as demand and achieved axes.

 

Diamond Light Source also hosts four cryoEM:

http://www.diamond.ac.uk/Home/Corporate-Literature/newsletter/Autumn2015/News/CryoEM.html

http://www.diamond.ac.uk/Science/Science/Integrated-facilities/eBIC

 

Alun Ashton’s group and others have been working on getting the next generation analysis codes running on the cluster I think. The goal with many projects like this is to integrate scanning with analysis which NeXus helps with considerably. However of course many analysis codes cannot work with HDF5.

 

Matt

 

From: science-iwg-bounces@xxxxxxxxxxx [mailto:science-iwg-bounces@xxxxxxxxxxx] On Behalf Of Philip Wenig
Sent: 06 March 2017 08:14
To: science-iwg@xxxxxxxxxxx
Subject: Re: [science-iwg] Eclipse 'Scanning' Project

 

Matt,

your proposal sounds very challenging :-).

Am I right, it would be also possible to control e.g. a microscope as far as it supports an interface to control its 3d coordinates (height, x, y)? If yes, a combination of Eclipse Scanning and ImageJ could become an essential time saver for Biologists :-)!


Best,
Philip

Am 03.03.2017 um 16:36 schrieb Matt.Gerring@xxxxxxxxxxxxx:

Hello,

 

I would like to introduce you to a new open source project for scanning hardware and completing scientific experiments. It has been donated to the Eclipse Foundation as an incubation project and has number of interesting features. The original project proposal can be read here: https://projects.eclipse.org/proposals/scanning

 

However, I would like to point out some of the things it can do to you, as a subscriber to this list or person who has expressed an interest in the past with the project.

Solstice.png
Scan Paths

The start of a scan is the scan path. It is a series of n-Dimensional motor/scalar positions provided by a generator pattern - we call them scan point generators. The possible paths are created using a factory, with each path requiring a model. The generators may be nested, limited only by their ability to be addressed. The paths scale to hundreds of millions of points because not all points are created in memory at the start of the scan, so really large scans are possible. The paths are defined in Python to make it easy for scientists to create custom generators on the fly, in the unlikely event that they cannot achieve the type of scan required. Unlimited regions of interest are allowed so the user may for instance create a grid and then draw various shapes to filter the scan points. A simple example of a scan path is a snake scan running over a grid of 2D positions of a stage motors for x and y.

Runnable Devices and Scannable Devices

When it comes time to run the scan, the motors and detectors are represented in code by the simple interfaces IRunnableDevice and IScannable. These classes are each managed by an OSGi service which allows devices to be added by a Spring layer or by creating and registering them with the server in Python. Once they are added, they provide the link to the hardware such as Area Detector or EPICS PVs, you can then start to run scans.

Scanning Algorithms

The scanning algorithm is also part of a factory pattern and allows users to override or create their own. The default algorithm uses the power of Java Executor Service. It splits up the devices by a concept known as level and uses a fully multi-threaded design to move motors in the most efficient way. Motors are moved while detectors are read out and the file is written. For those motors in a 2D scan which have a fast and slow axis on the same EPICS IOC, the power of the scan path design means that the code can look forwards to figure out each fast path segment and run lines close to the theoretical maximum.

User Interface

The project comes with a complete user interface for creating and running scans written in Eclipse RCP. The bundles which contribute the UI are modular and may be removed for those people without an RCP front end. The user interface interacts seamlessly with the python layer giving the user either the ability to submit without scripting or, for advanced users, the ability to copy the scan command into Python. It is easy to create Java-Swing, Python-QT or any front end because the scanning system is driven externally by JSON.

Python and Malcolm

The Scanning project is partly written in Python and partly Java. Most of the C-python device layer is contributed in a separate project called pymalcolm and the project interacts with Malcolm devices (which are IRunnableDevice) using the EPICSv4 protocol, http://epics-pvdata.sourceforge.net/.

Eclipse January, NeXus HDF5 -SWMR

Scanning includes the implementation of the January concept “ILazyWriteableDataset”. This means that if you are creating a device, you only need depend on January and your control layer (EPICS for instance). The device layer is modular and multi-threaded so it allows the full speed of HDF5-SWMR to be utilised without the need to mix up HDF5 and device code.

Online Analysis

For the first time, online analysis has been considered and designed into a scanning system from the start. This means that online analysis codes running on the cluster can run processing SWMR files on the data during the live data collection. The user interface allows the choose between different analysis or even create your own graphically using DAWN and then deploy the analysis with the scan. This is done efficiently using the new Single Write Multiple Read (SWMR) HDF5 library.

Builds and Tests

Scanning does not skimp on tests, it has hundreds of tests executing different scanning scenarios and different device options. It checks operation of the algorithms with machine topup and beam dump scenarios, it writes HDF5 SWMR files and checks the supported scan point generators. It gives the user interface composites a workover using SWTBot. For the first time this layer has be abstracted into a separately built (travisCi+maven) and tested(travisCI+junit+sonarqube) project which enables anyone to participate and contribute.

Getting Started

If you want to check out ‘Scanning’ and run it, providing you are experienced with targets and products it is easy to run by following these instructions:


https://github.com/eclipse/scanning/blob/master/GETTINGSTARTED.pdf

 

Sincerely,

 

Matthew Gerring

 

-- 

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
 





_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg




-- 
~~~~~~~~~~~~~~~~~~~~~~~~
OpenChrom - the open source alternative for chromatography / mass spectrometry
Dr. Philip Wenig » Founder » philip.wenig@xxxxxxxxxxxxx » http://www.openchrom.net
~~~~~~~~~~~~~~~~~~~~~~~~

 

-- 

This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
 




_______________________________________________
science-iwg mailing list
science-iwg@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/science-iwg



-- 
~~~~~~~~~~~~~~~~~~~~~~~~
OpenChrom - the open source alternative for chromatography / mass spectrometry
Dr. Philip Wenig » Founder » philip.wenig@xxxxxxxxxxxxx » http://www.openchrom.net
~~~~~~~~~~~~~~~~~~~~~~~~

Back to the top