Data Formats | Contributions received prior to canSAS-3

The replies here are reflex responses primarily from Ron Ghosh; the main intention may be to stimulate other replies from potential participants.

NeXus and sasCIF for TOF-SANS (Steve King, ISIS)

Steve has produced a discussion paper here which should greatly help improving compatibility between NeXus and sasCIF dictionaries and our use of terms in general.

Uncertainty in Q for fourth column

Rather than equate this as an error term it would be better to define it as a resolution width, in which case it should be clearly defined as being FWHM or standard deviation. (Adrian Rennie)

Disquiet over the future of HDF

Even today only a limited number of packages offer direct interfaces to HDF4. What can be expected if HDF5 is now an incompatible, and at present little used variant? (John Barnes, Ron Ghosh et al.)

This was a point of prime importance at the NeXus workshop, and the software developers showed willingness to push for more general integration of the new standard HDF5, (already, for example, it is a standard component of Octave, a gnuish-lookalike for Matlab).

The design of the NeXus software does allow several options for rendering the HDF version transparent for the Users. Similarly at a more fundamental level HDF5 code is orthogonal to HDF4, and both can co-exist in the same package.

There remain vast volumes of data stored in HDF4 format, which will never be rewritten in the newer format. In implementing the rewrite for HDF5 NCSA is recognising that usage of HDF now extends beyond what the earlier format could satisfy.

Synchrotrons use commercially available detectors, with their own data formats

The solution at RIKEN/SPring-8 is to use networking software to provide a uniform interface for users to treat the different data types. (Tetsuro Fujisawa).

This would appear to be ripe for discussion in future canSAS meetings when we take up the problems of formatting raw data (having solved those of treated data). An important point in favour of HDF5 now appears to be the high performance which can be attained in performing I/O operations.

XML offers a well-structured basis for an ASCII data format

XML, extensible markup language, has all the components of data element definitions and attributes; this would offer a future-proof standard for defining SAS data. (John Barnes) There was a presentation on XML at NoBugs-3 at Daresbury last year, and another review at the PSI-NeXus meeting. XML appears to be the ideal language for verifying structure and integrity of NeXus files, and might even help automatic generation of reading and writing software. At present tools remain firmly in the domain of computer scientists.