CODE - Mobile and Flexible Instrumentation for full-field X-ray Diffraction Imaging
According to the requirements derived during the methodological development and inspired by the experience of test experiments, the STROBOS-CODE partners specify, design, construct, commission, and test instrumentation for full-field X-ray diffraction imaging. The station will be suited for operation at the IMAGE beamline of the synchrotron radiation source at KIT, but its mobile and flexible layout will allow its application for user experiments at the experimental end-stations of different state-of-the-art synchrotrons like PETRA lll and ESRF.
Sample Manipulation Module
A main module of the CODE-station is a sample-manipulator system for general full-field X-ray diffraction imaging, with particular focus on suitability for in situ X-ray diffraction laminography (XDL).
In order to provide all degrees of freedom required by XDL, a solution consisting of three components was designed and realized (from bottom to top): 1) a tilt-stage θ to provide Bragg condition for the selected reciprocal lattice vector, which also defines the laminographic/tomographic rotation axis, 2) a highly precise rotary table for the rotation about the selected reciprocal lattice vector ρ, and 3) a high-precission miniatur parallel kinematic (3 translations and 3 rotations) for fine-alignment of the sample with respect to the components below and selecting the sample's region of interest.
Two complementary realizations were worked out (enabling a flexible adjustment of the configuration, considering the required precision, stability, angular range, available space, etc.) and the required components were delivered and commissioned throughout 2017 and 2018. At present, the sample-manipulator system is undergoing final testing and characterization.
Variation 1: The first realization is a light weight and compact solution, which can easily be mounted on any given sub-structure. It employs a cradle-like so-called 1-circle-element for the laminographic tilt with a angular travel range of θ = ±15°.
Variation 2: The second design's main advantage is the unlimited angular range in θ. This, in combination with sophisticated sample holders, allows selecting reciprocal lattice vectors for the investigation that were previously inaccesible.
The rotary stages of the two implementations are interchangeable and thus enable compiling the sample-stack according to the experiment’s specific requirements (which can be influenced by the sample material, beam-properties at the given synchrotron facility, etc.). All configurations employ a so-called SpaceFab mounted on top of the system. It provides six degrees of freedom which not only enables a fast semi-automated sample fine-alignment, i.e. ensuring that ρ║hhkl, but also choosing the region of interest in the sample.
Latest test measurements demonstrated that the possible sample-stack compilations have angular and spatial errors below 10-4° and 2 µm, respectively, with an overall weight of ca. 25 kg (1-circle-segment solution) and ca. 30 kg (solution with unlimited θ-range).
Detector Manipulation Module
Since in the case of X-ray diffraction imaging the propagation direction of the detected (diffracted) beam profile is inclined by twice the Bragg angle from the original incident beam direction, the cooperation partners plan to implement a suitable detector manipulation module, in the ideal case enabling the flexible positioning of 2D detector at any desired position in the upper hemisphere, meaning that the detector may face the sample from any direction above the sample itself. Within the recent measurement campaigns, promissing devices for detector movement were already tested.
STROBOS - Camera Development
High-speed image-based process control requires continuous recording and analysis of image streams.
For this purpose we have developed the modular FPGA-based scientific streaming camera platform intended for rapid development of scientific data acquisition systems. The camera platform provides fast interconnects, customizable image-based triggers and embedded process control logic. All these features are not available in commercial cameras. It is based on various CMOS active pixel sensors, mounted on mezzanine daughter cards. The daughter card is connected to the main readout board. For further data analysis the camera platform is seamlessly integrated in the UFO parallel processing framework.
The main benefits of the high-throughput camera platform are:
- Continuous data acquisition at full speed
- On-line image-based self-event trigger architecture (e.g. for fast rejection of use-less image)
- Automatic region-of-interest readout strategies
- Easily extendable to any available CMOS image sensor
- Fully programmable and controllable
For the UFO camera platform several image sensor boards are existing with a spatial resolution of 1 to 20 Megapixel and frame rates of up to 5000 frames per second in full sensor resolution. Even higher rates are possible at reduced or interpolated resolution by using the fast reject logic. Frame rate and resolution are only limited by the current sensor data throughput of about 6.5 GByte/s. The readout architecture itself is open for other sensors in order to match the specific requirements of scientific experiments. Due to its modular structure the camera electronics is easily extendable, e.g. for optical data transmission links.
Within STROBOS-CODE the UFO high-throughput camera platform for continuous data streaming has been adopted to the requirements of diffraction imaging. The fast data link enabled tight coupling of programmable camera electronics and the UFO computing infrastructure. This heterogeneous DAQ setup represents concept for future data acquisition systems in general. First examples of embedded logic have been realized like real-time data reconstruction and fast feedback loops. During the course of the project, thee sensor modules have been developed. The list of modules in-clude two high-resolution cameras respectively of 2 and 4 Mpixels with a pixel pitch of 5.5 µm and frame rate of up to 330 fps, and one wide-field-of-view camera with 20 Mpixels pixel pitch of 6.4µm and frame rate of up to 30 fps.
To optimize continuous data streaming from the cameras to GPU-based computing infrastructure for fast 3D reconstruction, a flexible high-performance readout card has been developed. The readout card is integrated into the UFO camera platform and provides FMC connectors for images sensor boards like the 20 Mpixels sensor board shown in Figure 2. The readout card is equipped with a Xilinx Virtex-7 FPGA and is connected to the computing infrastructure by PCIe Gen 3. The readout card has been designed as multi-purpose module and is foreseen for a wide field of DAQ applications. Two high pin count connectors are FMC VITA-57 compliant and provide a large number of high-speed in-terconnects to the application specific FMC card. DDR3 memories with 4 GB have been integrated and the firmware was optimized to operate with up to 15 GB/s.
In a traditional DMA architecture, data is at first written to the PC main system memory and in a se-cond step distributed to the included GPUs for processing. Typically additional copy operations in the main memory are performed. Throughput and latency of the system are therefore limited by performance and load of the involved components and operating system. For the 5000 fps camera system the required data throughput exceeds 6 GB/s. Therefore new concepts have to be evaluated to ensure that high throughput with minimal load on the system architecture. We have developed a custom readout architecture that realizes fast communication between FPGA, CPUs and GPUs via Remote Direct Memory Access (RDMA). Our solution includes a Direct Memory Access (DMA) engine compatible with the Xilinx PCI-Express core, and appropriate Linux drivers. To enable direct low-latency communication between FPGA and GPU, a module for the UFO framework has been developed to manage memory transfers using AMD’s “Direct GMA” technology. The UFO camera is fully integrated in the UFO computing infrastructure. The camera abstraction library LibUCA allows to easily adopt for different sensor modules or pre-processing algorithms.
Cooling of the images sensor reduces noise and so improves the SNR. Together with the partners from the Shubnikov Crystallography Institute (SCI) a cooling system for the UFO cameras has been developed. The camera is housing designed, realized, and accomplished in SCI.