Research Article: Diffraction-geometry refinement in the DIALS framework

Date Published: April 01, 2016

Publisher: International Union of Crystallography

Author(s): David G. Waterman, Graeme Winter, Richard J. Gildea, James M. Parkhurst, Aaron S. Brewster, Nicholas K. Sauter, Gwyndaf Evans.


A comprehensive description of the methods used within the DIALS framework for diffraction-geometry refinement using predicted reflection centroids is given. Examples of the advanced features of the software are provided.

Partial Text

The successful integration of single-crystal diffraction data depends on the accurate prediction of Bragg spot locations on area-detector images. An initial model for the diffraction geometry may be constructed from metadata provided with the diffraction images (Parkhurst et al., 2014 ▸) or provided by the user. This starting model is completed by estimating crystal parameters, which are usually derived from data by an autoindexing procedure, such as that of Steller et al. (1997 ▸). This model is rarely sufficient for accurate prediction throughout a data set. Thus, a crucial step in data processing is to refine the geometrical model by procedures that minimize the discrepancies between spot locations observed on the image data and their locations as predicted from the model.

A central tenet of the DIALS framework is modularity, such that algorithms may be exchanged to alter or extend the capability of the software. A key issue this raises is the scope of each task. For instance, the geometry refinement should not include models that naturally fall within the scope of integration and hence become dependent on them. Those aspects of the global model that affect spot size and shape, such as the crystal mosaicity and beam divergence, are more appropriately dealt with by the part of the software that either learns or otherwise constructs models of the reflection profiles. This is a useful distinction, because for rotation-scan data these parameters do not alter the central impacts determining the recorded reflection position, only the general impacts that determine the reflection extent (Duisenberg et al., 2003 ▸). This distinction does not hold for data sets consisting of still shots, because in general the reflecting condition for a central impact is not met on a still. In this case, the distribution of observed spots, and their partiality, is inescapably a function of general impact parameters. In this work, we restrict our description of geometry refinement to rotation experiments, and thereby to refinement of parameters that affect the reflection centroids, whilst making no assumption about parameters that contribute to spot size and shape.

The general diffraction geometry for central impacts is shown in Fig. 1 ▸. The use of arbitrary vectors to describe the experimental geometry avoids limitations caused by adherence to an idealized geometry or coordinate system for a particular experimental method. The description that follows is a useful abstraction for correctly capturing the geometries found in any experimental diffractometer. Also, the implementation details of particular pieces of hardware play no role in the description of diffraction in these general terms. This design dates back to the seminal workshops at which a device-independent version of the program MADNES was planned (Bricogne, 1986a ▸,b ▸, 1987 ▸), and directly influenced the dxtbx software used by DIALS (Parkhurst et al., 2014 ▸). Here, we follow closely the reports associated with that workshop and repeat some of the relevant mathematical working here for clarity, as unfortunately these documents are not widely available.

The DIALS refinement module makes an explicit distinction between the experimental models (beam, crystal, goniometer and detector) and the parameterizations of these models. The models are used throughout the DIALS framework and are abstract descriptions of the physical components that they represent, intended to be generally applicable to a wide variety of experiments and to be used by all algorithms written within the DIALS framework. A model parameterization, in contrast, is relevant only within refinement. Each parameterization attaches to its model for the duration of the refinement procedure, providing a means to express the state of the model from the values of its parameters, to calculate first derivatives of this state and to update the model after each step of the refinement algorithm. This distinction enables great flexibility, as alternative parameterizations may be applied to a particular model to control the behaviour of refinement. This may be used to represent different levels of prior knowledge about an experiment. For example, the default detector parameterization in DIALS represents the position and orientation of each detector plane with six degrees of freedom. For some instruments it may be appropriate to restrict relative offsets of individual planes to translations, or to specify certain known mechanical axes about which operations of translation or rotation may take place. Such cases would be accommodated by providing an alternative parameterization to attach to the core detector model. This alternative parameterization would be guaranteed to affect only the behaviour of refinement in DIALS, because the core detector model remains unchanged.

Minimization requires three entities: a model function, a target function and a minimization algorithm. The model function for centroid refinement is the vector-valued reflection-prediction equation, which for some integer index vector h calculates the reflection centroid according to the model states s0, U, B and d, and a Boolean flag e ∈ {true, false} that determines whether the passage of the reciprocal-lattice point is entering or exiting the Ewald sphere,

The least-squares method is neither robust nor resistant, which means that the refined parameter estimates can be highly sensitive to extreme values in the data. These values may result either from long-tailed error distributions or the presence of outliers in the data (Prince & Collins, 2006 ▸). The squaring of residuals in the target function (25) magnifies the effect of extreme data points, such that the inclusion of even a single severe outlier can dominate the minimization procedure. As it is the form of the target function that holds this property, altering the choice of minimization algorithm cannot improve robustness or resistance.

A common mode of operation in current software, as exemplified by the programs MOSFLM (Leslie & Powell, 2007 ▸) and XDS (Kabsch, 2010a ▸), is to index and refine an initial model and then process a data set in a linear fashion from beginning to end of the sweep, alternating between refinement and integration tasks. The integration model is localized in φ by refinement tasks that are specific for a small wedge of data. The model may not be appropriate outside of this φ window owing to changes in the diffraction geometry such as the crystal setting angles. However, the refined model is adequate for spot prediction within this wedge of images and is taken as the starting point for refinement within the next φ window. This method may also ‘correct’ for deficiencies in the geometrical model that preclude a general representation for the full data set, such as enforced ideal geometry of the rotation method (with the spindle axis and detector plane both at right angles to the beam) where this is not strictly appropriate. This approach of alternating local refinement and integration was developed when typical data-collection experiments were slower than the computational processing, and analysis would start before data collection was complete. Processing only small wedges also ensured a reduced memory requirement, which was a key issue in the design of software for older hardware.

The approach of global refinement presented in §7 is applicable to data from throughout a single-crystal data set. For special cases, the idiom may be extended to a higher level to encompass data from multiple experiments in a single joint refinement. Here, the term ‘experiment’ has a precise meaning within the DIALS framework, and refers to a set of unique experimental models necessary to satisfy the diffraction condition and produce a consistent set of measured intensities. An experiment must therefore contain exactly one beam, one crystal and one detector model. It may also contain one goniometer and one scan model if the experiment is a rotation scan. It is possible to jointly refine multiple experiments if those experiments share one or more models. The commonest example is that of multi-lattice data, in which the experiments differ only in their crystal models. The use of DIALS multi-lattice refinement during the indexing of small wedges of multi-crystal data has previously been presented (Gildea et al., 2014 ▸). Fig. 3 ▸ shows how the correlation between crystal and detector parameters for one of these small wedges is reduced when multiple lattices are refined together, and Fig. 4 ▸ illustrates the set of models used for joint refinement.

During data collection the crystal unit cell may change owing to radiation-induced structural changes. Changes in the illuminated crystal volume during rotation or translation may introduce undamaged regions into the beam, thereby impacting the effective unit-cell parameters. In addition, crystal movements or goniometer defects may lead to changes in the crystal orientation. Where these changes occur smoothly over the course of a rotation scan, the benefits of global refinement of parameters can still be obtained by explicitly including a smoothly varying parameterization of the crystal, expressed as a function of position in the contiguous series of images forming the scan. For this purpose a Gaussian smoother based upon code from AIMLESS (Evans & Murshudov, 2013 ▸) is used. This provides a simple and fast way to calculate smoothly changing values and gradients for an arbitrary parameter without assuming any particular functional form for the behaviour of that parameter. To set up the smoother, a number of equally spaced points are chosen throughout the scan, based on a user-configurable interval width.

When any nonlinear least-squares algorithm is used to minimize the target function, the errors on the refined parameters may be estimated by the standard procedure of inverting the normal matrix. As these parameters may not necessarily be relevant outside of refinement, it is more useful to convert these to error estimates of the model states (see Table 1 ▸ for definitions). This is achieved by the usual procedure for error propagation, as detailed in Appendix C, and for the crystal model estimated standard deviations of the real space unit-cell parameters may be derived.

A comprehensive description of diffraction-geometry refinement within the DIALS framework has been given and the tools made available within the command-line program dials.refine. This program builds upon a simple but general model for the prediction of reflection central impacts and minimization of a least-squares residual by providing flexible parameterization, choices of minimization algorithm and outlier-rejection method, global scan-varying refinement of the crystal and joint refinement of multiple experiments. Examples of these advanced features have been shown, as well as an analysis of the propagation of errors using a novel method for drawing statistical replicate data sets from real experimental data. The design of the software is explicitly intended to facilitate extensibility and modification, and to provide a solid platform from which future research into advanced methods of diffraction-geometry modelling may be performed.




Leave a Reply

Your email address will not be published.