2012 Data Discussion

Minutes

Saturday Afternoon Working Group Session

• Start with defining scope, what is reduced data?
  • Eliminate all instrument geometry and detector effects.
  • Will still need wavelength for anomalous x-rays
  • Required:
    • Intensity either in absolute units of cross section or directly convertible by a scaling constant
    • Data described as a function of Q vector
    • Uncertainty in intensity
    • Wavelength and type of the probe radiation
  • Resolution may be provided - how to represent it up for broader discussion.
  • Keywords and verification tools - checking that correct tag names/attribute names have been used. Smarter than just saying yes/no.
  • Defined a proposed minimum spec outline for discussion:
    • http://www.smallangles.net/wgwiki/images/archive/0/0a/20120729121131%212012-minimum.png
  • Defined a proposed minimum recommended spec outline for discussion:
    • http://www.smallangles.net/wgwiki/images/archive/1/14/20120729121223%212012-recommended-minimum.png

• (Compare with the 1D XML standard: media:cansas1d-v1-10-minimum.png)

Sunday Morning Discussion Session

• How many implementations?
  • Need at least one reference implementation by end of meeting
  • Shouldn't preclude multiple possible implementations. RG noted that if they follow the same data structure then they should be "trivially" interchangeable.
• SESANS? Spin-Echo? XPCS?
  • Out of scope for our remit.
• Next step -> science examples.
• How to incorporate complimentary data e.g. uv spectra
• Space for calibration information
• Space for process data

Sunday Afternoon Working Group Session

• Based on morning discussion...
  • Redefined a proposed minimum spec outline:
    • 
  • Redefined a proposed minimum recommended spec outline:
    • 

• Generated a proposal for usage and format.

See: 2012_Data_Discussion_Examples

Agenda with Discussion

• The following is the agenda of work posted under business for canSAS-2012. Please add comments in the appropriate section below:

1D Format

Agree a proposed extension of the current 1D standard

• This is an extension proposed by Steve King (ISIS) that is required to enable, for example, t-o-f instruments to store auxillary wavelength-dependent non-I(Q) data in the same output files
  • see Section 1.6.5 in here
  • and look at this example
  • or this one

Agree resultant changes to the "official" schema/stylesheet

SMK -  

For clarity, this extension was proposed after adoption of the existing version of the standard. Ratification of this extension at CanSAS-2012 would give facilities/software developers the necessary confidence to implement it.

SMK -  

This proposed extension would allow more-or-less any non-I(Q) data to be included in a CanSAS-1D data file under a suitable foreign namespace. That is remarkable flexibility, and it does not require any significant revision of the existing standard. However, it could be argued that certain types of non-I(Q) data are so integral to a SAS experiment that they should instead be given explicit SASXML tags within the standard? Transmission data is a good example of this: the existing standard explicitly includes a tag for the transmission value from a monochromatic measurement (SASsample\transmission) but makes no provision for transmission values from white beam measurements. The proposed foreign namespace extension will make such provision possible but only as an extension, not as a formal part of the standard. Is that acceptable? The downside of explicitly adding new SASXML tags to the existing standard is that it would require issuing a new version of the standard.

ARJN -  

I would recommend that during this meeting that the current format then be 'parked', the new 2D format should be flexible enough to handle 1D and 2D data.

SMK -  

Agreed.

ARR -  

Perhaps we should be cautious about suggesting no further development for the existing 1-D format. Unless there is scope for adapting to future needs, it will be seen as not suitable for new uses. This, of course, does not imply that there have to be active plans for further changes at present.

AJJ -  

As to this particular case, I suggest that since SMK and I have it done, that we accept the additional tags, call it v1.1 and move on. If other extensions arise later and become "widely" used, then we can incorporate them as they arise. This is the key benefit of using an extensible format.

ARJN -  

Just as a matter of interest, why is it necessary to include transmission measurements for TOF instruments? Does the analysis depend on these values? Not saying it's not necessary, just wondering about why it is useful.

2D Format

Define minimum information necessary for reduced data

• definition of what reduced data is and hence the problems we should address.
• review previous discussions on 2D
• considerations specific to 2D reduced data
• consider forward looking issues such as
  • grazing incidence
  • event mode analysis

Suggest format framework

• NeXus extension, canSAS 1D extension, other, with brief discussion of the reason for the choice (including options considered, pros and cons of each, and final weighing)
  • compare with NeXus definitions for:
    • NXsas
    • NXiqproc
    • NXsqom (for S(Q,OM) or S(Q,E) data)

---

• A proposal for a 2D version of the canSAS 1D data format has been posted for discussion: 2D Data Format Proposal
• NeXus has a definition for multi-dimensional data:
  • http://download.nexusformat.org/doc/html/ClassDefinitions-Application.html#NXsas
  • http://svn.nexusformat.org/definitions/trunk/applications/NXsas.nxdl.xml

ARJN -  

Having looked at both those Nexus definitions it is not clear to me how:

• • multiple detectors are handled.
  • What happens if the detector pixels are not grid shaped?
  • how does one stuff multiple datasets (e.g. from event mode acquisition)?

Remember, all we need is instrument independent information in this file, nothing else.

Andyfaff 12:45, 27 July 2012 (CDT)

---

• James Hester (an expert in the CIF format) has compelling ideas on the subject of common data files. CIF format notes Hester

• Andrew Nelson has commented via email exchange with Adrian Rennie : Notes on Reduced Data formats with examples from reflectometry

---

• Ron Ghosh has laid out details of a proposal to use HDF5/Nexus : HDF5 Notes Ghosh

---

• Peter Boesecke misses definitions of parameters that are needed to describe small-angle and wide-angle scattering experiments. Parameters like "Distance", "Center", "Rotation" have only a meaning when they are part of a geometrical model, otherwise they are useless. He has collected parameters that are required for SAS/WAS experiments with position sensitive detectors (1D and 2D). When saving experimental data it should be checked that these parameters are either implicitly or explicitly described by the supplied metadata: SX_parametrization.

---

• Jerome Kieffer sent the following to consider: Jerome Kieffer's Mail