canSAS - User contributions [en]

cansas1d binding Fortran

2008-06-03T08:03:27Z

130.246.132.26: /* canSAS 1-D SAS XML v1.0 support */

The development of the FORTRAN language, so beloved of scientists, pre-dates the development of XML. And it shows. FORTRAN is not a language that manipulates strings with ease, and this makes parsing XML decidedly awkward. So unless you ''really'' have to use FORTRAN, you are probably better off with C/C++ (or something else more 'modern'), see for example Daniel Veillard's LIBXML2 library at http://xmlsoft.org/ or Frank van den Berghen's parser at http://www.applied-mathematics.net/tools/xmlParser.html.

If you have to use a dialect earlier than FORTRAN-90 (F90), then the chances are you will have to code your own parser.

== Software Development Kits ==
For later dialects, there are some SDK's available on the Web:

* F90:
** XMLPARSE - by Arjen Markus at http://xml-fortran.sourceforge.net/
** FoX - by Toby White & others at http://uszla.me.uk/space/software/FoX/
* For F95:
** XML - by Mart Rentmeester at http://nn-online.org/code/xml/

== canSAS 1-D SAS XML v1.0 support ==
Steve King[mailto:s.m.king@rl.ac.uk] (ISIS) has provided a F77 routine ([http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/fortran/SASXML_G77.F SASXML_G77.F]) that will read CanSAS XML v1.0 files.

cansas1d binding Fortran

2008-05-19T12:00:21Z

130.246.132.26:

The development of the FORTRAN language, so beloved of scientists, pre-dates the development of XML. And it shows. FORTRAN is not a language that manipulates strings with ease, and this makes parsing XML decidely awkward. So unless you ''really'' have to use FORTRAN, you are probably better off with C/C++ (or something else more 'modern'), see for example Daniel Veillard's LIBXML2 library at http://xmlsoft.org/ or Frank van den Berghen's parser at http://www.applied-mathematics.net/tools/xmlParser.html.

If you have to use a dialect earlier than FORTRAN-90 (F90), then the chances are you will have to code your own parser.

For later dialects, there are some SDK's available on the Web:

For F90:

XMLPARSE - by Arjen Markus at http://xml-fortran.sourceforge.net/

FoX - by Toby White & others at http://uszla.me.uk/space/software/FoX/

For F95:

XML - by Mart Rentmeester at http://nn-online.org/code/xml/

Steve King[mailto:s.m.king@rl.ac.uk] (ISIS) has a VMS F77 routine that will read CanSAS XML v1.0 files.

cansas1d binding Fortran

2008-05-19T10:47:59Z

130.246.132.26:

The development of the FORTRAN language, so beloved of scientists, pre-dates the development of XML. And it shows. FORTRAN is not a language that manipulates strings with ease, and this makes parsing XML decidely awkward. So unless you ''really'' have to use FORTRAN, you are probably better off with C/C++ (or something else more 'modern'), e.g. Daniel Veillard's LIBXML2 at http://xmlsoft.org/ or Frank van den Berghen's parser at http://www.applied-mathematics.net/tools/xmlParser.html.

If you have to use a dialect earlier than FORTRAN-90 (F90), then the chances are you will have to code your own parser.

For later dialects, there are some SDK's available on the Web:

For F90:

XMLPARSE - by Arjen Markus at http://xml-fortran.sourceforge.net/

FoX - by Toby White & others at http://uszla.me.uk/space/software/FoX/

For F95:

XML - by Mart Rentmeester at http://nn-online.org/code/xml/

cansas1d binding Fortran

2008-05-19T10:45:17Z

130.246.132.26:

The development of the FORTRAN language, so beloved of scientists, pre-dates the development of XML. And it shows. FORTRAN is not a language that manipulates strings with ease, and this makes parsing XML decidely awkward. So unless you ''really'' have to use FORTRAN, you are probably better off with C/C++ (or something else more 'modern'), e.g. Daniel Veillard's LIBXML2 at http://xmlsoft.org/ or Frank van den Berghen's parser at http://www.applied-mathematics.net/tools/xmlParser.html.

If you have to use a dialect earlier than FORTRAN-90 (F90), then the chances are you will have to code your own parser.

For later dialects, there are some SDK's available on the Web:

For F90:

XMLPARSE - by Arjen Markus at http://xml-fortran.sourceforge.net/

FoX - by Toby White & others at http://uszla.me.uk/space/software/FoX/

For F95:

XML - by Mart Rentmeester at http://nn-online.org/code/xml/

cansas1d binding Fortran

2008-05-19T10:42:13Z

130.246.132.26:

cansas1d binding Fortran

2008-05-19T10:41:34Z

130.246.132.26:

cansas1d binding Fortran

2008-05-19T10:41:01Z

130.246.132.26:

The development of the FORTRAN language, so beloved of scientists, pre-dates the development of XML. And it shows. FORTRAN is not a language that manipulates strings with ease, and this makes parsing XML decidely awkward. So unless you ''really'' have to use FORTRAN, you are probably better off with C/C++ (or something else more 'modern'), e.g. Daniel Veillard's LIBXML2 at http://xmlsoft.org/[http://xmlsoft.org/].

If you have to use a dialect earlier than FORTRAN-90 (F90), then the chances are you will have to code your own parser.

For later dialects, there are some SDK's available on the Web:

For F90:
XMLPARSE - by Arjen Markus at http://xml-fortran.sourceforge.net/

FoX - by Toby White & others at http://uszla.me.uk/space/software/FoX/[http://uszla.me.uk/space/software/FoX/]

For F95:
XML - by Mart Rentmeester at http://nn-online.org/code/xml/[http://nn-online.org/code/xml/]

cansas1d binding Fortran

2008-05-19T10:39:56Z

130.246.132.26: New page: The development of the FORTRAN language, so beloved of scientists, pre-dates the development of XML. And it shows. FORTRAN is not a language that manipulates strings with ease, and this ...

The development of the FORTRAN language, so beloved of scientists, pre-dates the development of XML. And it shows. FORTRAN is not a language that manipulates strings with ease, and this makes parsing XML decidely awkward. So unless you ''really'' have to use FORTRAN, you are probably better off with C/C++ (or something else more 'modern'), e.g. Daniel Veillard's LIBXML2 at http://xmlsoft.org/[http://xmlsoft.org/].

If you have to use a dialect earlier than FORTRAN-90 (F90), then the chances are you will have to code your own parser.

For later dialects, there are some SDK's available on the Web:

For F90:
XMLPARSE - by Arjen Markus at http://xml-fortran.sourceforge.net/[http://xml-fortran.sourceforge.net/]

FoX - by Toby White & others at http://uszla.me.uk/space/software/FoX/[http://uszla.me.uk/space/software/FoX/]

For F95:
XML - by Mart Rentmeester at http://nn-online.org/code/xml/[http://nn-online.org/code/xml/]

cansas1d documentation

2008-05-19T10:08:39Z

130.246.132.26: /* Support for ForTran */

= Disclaimer =

This description is meant to inform the community how to layout
the information within the XML files. However, should the
information in this document and the [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xsd cansas1d/1.0 SAS XML Schema]
differ, the XML Schema will be deemed to have the most correct description
of the standard.

= Objective =

One of the first aims of the canSAS (Collective Action for Nomadic Small-Angle Scatterers) forum of users, software developers, and facility staff was to discuss better sharing of SAS data analysis software. [http://www.smallangles.net/canSAS/ CanSAS] identified that a significant need within the SAS community can be satisfied by a robust, self-describing, text-based, standard format to communicate reduced one-dimensional small-angle scattering data, ''I(Q)'', between users of our facilities. Our goal has been to define such a format that leaves the data file instantly human-readable, editable in the simplest of editors, and importable by simple text import filters in programs that need not recognise advanced structure in the file nor require advanced programming interfaces. The file should contain both the primary data of ''I(Q)'' and also any other descriptive information (metadata) about the sample, measurement, instrument, processing, or analysis steps.

The cansas1d/1.0 standard meets the objectives for a 1D standard, incorporating metadata about the measurement, parameters and results of processing or analysis steps. Even multiple measurements (related or unrelated) may be included within a single XML file.

= General Layout of the XML Data =

The canSAS 1-D standard for reduced 1-D SAS data is implemented using XML files.
A single file can contain SAS data from a single experiment or multiple experiments.
All types of relevant data (''I(Q)'', metadata) are described for each experiment.
More details are provided below.

== Overview ==

[[Image:cansas1d-v1-10-minimum.png|thumb|block diagram of minimum elements required for cansas1d/1.0 standard]]

The basic elements of the cansas1d/1.0 standard are shown in the following table.
After an XML header, the root element of the file is [[cansas1d_SASroot | SASroot]] which
contains one or more [[cansas1d_SASentry | SASentry]] elements, each of which describes
a single experiment (data set, time-slice, step in a series, new
sample, etc.). Details of the [[cansas1d_SASentry | SASentry]] element are also
shown in the next figure. Refer to the [[cansas1d_block_diagrams | block diagrams]]
for alternative depictions. See [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xml cansas1d.xml] for an example XML file.
Examples, Case Studies, and other background information are below. More discussion can be found on the [[1D_Data_Formats_Working_Group|canSAS 1D Data Formats Working Group]] page and its [[Talk:1D_Data_Formats_Working_Group|discussion]] page. Details about
each specific field (XPath string, XML elements and attributes) are described
on the [[cansas1d_definition_of_terms]] page.

* [[cansas1d_SASroot | '''SASroot''': the root element of the file (after the XML header)]]
* [[cansas1d_SASentry | '''SASentry''': describes a single experiment (data set, time-slice, step in a series, new sample, etc.)]]
* [[cansas1d_block_diagrams | block diagrams]]
* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xml cansas1d.xml] example XML file
* discussion of this format: [[1D_Data_Formats_Working_Group | basic]] [[Talk:1D_Data_Formats_Working_Group | more]]
* [[# Help_for_XML | Seek outside help for XML]]
* [[cansas1d_definition_of_terms | '''Definition of terms''': Details about each specific field (XPath string, XML elements and attributes)]]

=== Basic elements of the cansas1d/1.0 standard ===

{|
|-
! {{Headcellstyle}} | element
||  
! {{Headcellstyle}} | description
|-
||
[[cansas1d_XML_header | XML header]]
||
||descriptive info required at the start of every XML file
|-
||
:[[cansas1d_SASroot | SASroot]]
||
||
|-
||
::[[cansas1d_SASentry | SASentry]]
||
||data set, time-slice, step in a series, new sample, etc.
|-
||
:::Title
||
||for this particular SASentry
|-
||
:::Run
||
||run number or ID number of experiment
|-
||
:::[[cansas1d_any | {any}]]
||
||any non-cansas1d/1.0 element can be used at this point
|-
||
:::[[cansas1d_SASdata | SASdata]]
||
||this is where the reduced 1-D SAS data is stored
|-
||
::::[[cansas1d_SASdata | Idata]]
||
||a single data point in the dataset
|-
||
:::[[cansas1d_any | {any}]]
||
||any non-cansas1d/1.0 element can be used at this point
|-
||
:::[[cansas1d_SASsample | SASsample]]
||
|| description of the sample
|-
||
:::[[cansas1d_SASinstrument | SASinstrument]]
||
|| description of the instrument
|-
||
::::[[cansas1d_SASsource | SASsource]]
||
|| description of the source
|-
||
::::[[cansas1d_SAScollimation | SAScollimation]]
||
|| description of the collimation
|-
||
::::[[cansas1d_SASdetector | SASdetector]]
||
|| description of the detector
|-
||
:::[[cansas1d_SASprocess | SASprocess]]
||
||for each processing or analysis step
|-
||
:::[[cansas1d_SASnote | SASnote]]
||
||anything at all
|}

=== Required XML file header ===
<pre>
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="example.xsl" ?>
<SASroot version="1.0"
xmlns="cansas1d/1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="cansas1d/1.0 http://svn.smallangles.net/svn/canSAS/1dwg/trunk/cansas1d.xsd"
>
</pre>

== Rules ==

[[Image:Q-geometry.jpg|thumb|definition of Q geometry for small-angle scattering]]

[[Image:Translation-orientation-geometry-2.jpg|thumb|definition of translation and orientation geometry]]

# canSAS1d/1.0 XML data files will adhere to the standard if they can successfully [[cansas1d_documentation#Validation_of_XML_against_the_Schema | validate]] against the established XML Schema ([http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xsd cansas1d.xsd])
# Q=(4 π / λ) sin(θ) where λ is the wavelength of the radiation and 2θ is the angle through which the detected radiation has been scattered.
# units to be given in standard SI abbreviations (eg, m, cm, mm, nm, K) with the following exceptions:
##um=micrometres
##C=celsius
##A=Angstroms
##percent=%.
##fraction
##a.u.=arbitrary units
##none=no units are relevant (such as dimensionless)
# where reciprocal units need to be quoted the format shall be "1/abbreviation"
# when raised to a power, use similar to "A^3" or "1/m^4" (and not "A3" or "m-4")
# axes:
##z is along the flight path (positive value in the direction of the detector)
##x is orthogonal to z in the horizontal plane (positive values increase to the right when viewed towards the incoming radiation)
##y is orthogonal to z and x in the vertical plane (positive values increase upwards)
#orientation (angles):
##roll is about z
##pitch is about x
##yaw is about y
# Unicode characters MUST NOT be used
# Binary data is not supported

=== Compatibility of Geometry Definitions ===

Note: translation and orientation geometry used by canSAS are consistent with:
#http://en.wikipedia.org/wiki/Cartesian_coordinate_system
#http://en.wikipedia.org/wiki/Right-hand_rule
#http://www.nexusformat.org/Coordinate_Systems
#http://mcstas.risoe.dk/documentation/tutorial/node6.html
#http://webhost5.nts.jhu.edu/reza/book/kinematics/kinematics.htm

The translation and orientation geometry definitions used
here are different than those used by
'''SHADOW''' ([http://www.nanotech.wisc.edu/shadow/ http://www.nanotech.wisc.edu/shadow/])
where the ''y'' and ''z'' axes are swapped
and the direction of ''x'' is changed.

= Documentation and Definitions =

* '''Documentation''': [[cansas1d_documentation]] (this page)
* '''Definitions''': [[cansas1d_definition_of_terms]]
* '''Block diagrams''': [[cansas1d_block_diagrams]]

== XML Schema ==

'''XML Schema''': The [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xsd cansas1d.xsd] [http://www.w3schools.com/xsd XML Schema] defines the rules for the XML file format ([http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xsd TRAC], [http://svn.smallangles.net/svn/canSAS/1dwg/trunk/cansas1d.xsd SVN]) and is used to validate
any XML file for adherence to the format.

== XML Stylesheets ==

* '''example.xsl''': [http://www.w3schools.com/xsl/ XSLT stylesheets] can be used to extract metadata or to convert into another file format. The default canSAS stylesheet [[http://svn.smallangles.net/svn/canSAS/1dwg/trunk/example.xsl | example.xsl]] should be copied into the each folder with canSAS XML data file(s). It can be used to display the data in a supporting WWW browser (such as Firefox or Internet Explorer) or to import into Microsoft Excel (with the added XML support in Excel). By default, MS Windows binds '''*.xml''' files to start Internet Explorer. Double-clicking on a canSAS XML data file with the [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/example.xsl '''example.xsl'''] stylesheet in the same directory will produce a WWW page with the SAS data and selected metadata.

== Examples and Case Studies ==

* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xml cansas1d.xml] basic example: Note that, for clarity, only one row of data is shown. This is probably a very good example to use as a starting point for creating XML files with a text editor.
* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/bimodal-test1.xml bimodal-test1.xml]: Simulated SAS data to test size distribution calculation routines.
* [[cansas1d_casestudy_collagen | dry chick collagen]]: illustrates the minimum information necessary to meet the requirements of the standard format
* [[cansas1d_casestudy_af1410 | AF1410 steel]]: SANS study using magnetic contrast variation (with multiple samples and multiple data sets for each sample), the files can be viewed from TRAC (no description yet): [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/examples/af1410/ http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/examples/af1410/]
* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d-template.xml cansas1d-template.xml]: This is used to test all the rules in the XML Schema. This is probably not a very good example to use as a starting point for creating XML files with a text editor since it tests many of the special-case rules.

=== XML layout for multiple experiments ===

Each experiment is described with a single '''SASentry''' element.
The fragment below shows how multiple experiments can be included
in a single XML file. Full examples of canSAS XML files with multiple experiments include:
* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/W1W2.XML ISIS LOQ SANS instrument]
* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cs_af1410.xml NIST SANS data]

<pre>
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="example.xsl" ?>
<SASroot version="1.0"
xmlns="cansas1d/1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="cansas1d/1.0 http://svn.smallangles.net/svn/canSAS/1dwg/trunk/cansas1d.xsd"
>
<SASentry name="071121.dat#S22">

</SASentry>
<SASentry name="example temperature series">

<Title>title of this series</Title>
<Run name="run1">42-001</Run>
<Run name="run2">42-002</Run>
<SASdata name="run1">

</SASdata>
<SASdata name="run2">

</SASdata>

</SASentry>
<SASentry name="other sample">


</SASentry>
</SASroot>
</pre>

== Foreign Elements ==

To allow for inclusion of elements that are not defined by the cansas1d.xsd XML Schema,
XML ''foreign elements'' are permitted at select locations in the cansas1d/1.0 format.
Please refer to the references (and others) [[#Help_for_XML | below]] for deeper discussions on foreign elements.

An example of the use of foreign elements in the cansas1d/1.0 format is given by
* http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/s81-polyurea.xml

== Support tools for Visualization & Analysis software ==

(under development as of May 2008)

=== Binding for IgorPro ===

An import tool (a.k.a. binding) for [http://www.wavemetrics.com/ IgorPro] has been created
([http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/IgorPro/cansasXML.ipf cansasXML.ipf]).
[[cansas1d_binding_IgorPro | Documentation]] is available.


'''Note''' that this tool is not a true [http://en.wikipedia.org/wiki/Binding_%28computer_science%29 binding] in that the structure of the XML file is not replicated in IgorPro data structures. This tool reads the vectors of 1-D SAS data (Q, I, ...) into IgorPro waves (Qsas, Isas, ...). The tool also reads most of the metadata into an IgorPro textWave for use by other support in IgorPro.


Status:
* Capabilities
** test suite of XML files developed
** import the XML files into IgorPro
* open TRAC tickets (as of 2008-05-15)
** http://svn.smallangles.net/trac/canSAS/ticket/3
** http://svn.smallangles.net/trac/canSAS/ticket/10
* Further
** conversation with Andrew Nelson shows more efficiency can be gained from minor redesign.
** Development of a GUI (to support the [http://usaxs.xor.aps.anl.gov/staff/ilavsky/irena.html Irena] package) has begun
** Development to add export capabilities (from IgorPro) back to the cansas1d/1.0 format has begun

=== Binding for Java ===

A [http://java.sun.com/developer/technicalArticles/WebServices/jaxb/ JAXB] [http://en.wikipedia.org/wiki/Binding_%28computer_science%29 binding] for Java has been created.
(http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/java/cansas1d).
[[cansas1d_binding_Java | Documentation]] is in progress.


'''Note''' that this tool replicates the structure of the XML file into Java data structures. Additional Java software must be applied to convert the /SASdata/Idata/* elements into vectors of ''I(Q)'' data.


Status:
* Capabilities
** import the XML files into Java
** tested by use in collimation-correction (desmearing) software (not publicly available at this time)
* Further
** need to test export capabilities

=== Support for Python ===

Specific support for the cansas1d/1.0 data standard in Python has not been developed.
However, some constructive suggestions are being gathered on a specific
[[cansas1d_binding_Python | Python Documentation]] page.

=== Support for FORTRAN ===

Specific support for the cansas1d/1.0 data standard in FORTRAN (remember? It stands for FORmula TRANslation) has not been developed by CanSAS.
However, some information is gathered on a specific
[[cansas1d_binding_Fortran | Fortran Documentation]] page.

== Software repositories ==

* '''TRAC''': [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk]
* '''Subversion''': [http://svn.smallangles.net/svn/canSAS/1dwg http://svn.smallangles.net/svn/canSAS/1dwg] (<nowiki>svn checkout http://svn.smallangles.net/svn/canSAS/1dwg/trunk/ cansas-1dwg</nowiki>)

= Validation of XML against the Schema =
# open browser to: http://www.xmlvalidation.com/
# paste content of candidate XML file (with reference in the header to the XML Schema as shown above) into the form
# press <validate>
# paste content of [http://svn.smallangles.net/svn/canSAS/1dwg/trunk/cansas1d.xsd cansas1d.xsd] XSD file into form and press <continue validation>
# check the results

= Help for XML =
; XML: EXtensible Markup Language http://www.w3schools.com/xml/ http://www.w3.org/XML/ http://en.wikipedia.org/wiki/XML http://www.zvon.org/xxl/XPathTutorial/General/examples.html
; XSL (or XSLT): EXtensible Stylesheet Language http://www.w3schools.com/xsl/ http://www.w3.org/Style/XSL/ http://en.wikipedia.org/wiki/Extensible_Stylesheet_Language http://en.wikipedia.org/wiki/XSLT
; XPath: XPath is a language for finding information in an XML document. http://www.w3schools.com/xpath/ http://www.w3.org/Style/XSL/ http://en.wikipedia.org/wiki/XPath
; Schema: An XML Schema describes the structure of an XML document. http://www.w3schools.com/schema/ http://www.w3.org/XML/Schema http://en.wikipedia.org/wiki/XSD
; XML Namespaces: XML namespaces are used for providing uniquely named elements and attributes in an XML instance. http://www.zvon.org/xxl/NamespaceTutorial/Output http://en.wikipedia.org/wiki/XML_namespaces http://www.w3schools.com/XML/xml_namespaces.asp
; XML Foreign Elements: Inclusion of elements, at select locations, that are not defined by the cansas1d.xsd XML Schema http://books.xmlschemata.org/relaxng/relax-CHP-11-SECT-4.html http://www.w3.org/TR/SVG/extend.html http://www.google.com/search?q=XML+foreign+elements

cansas1d documentation

2008-05-19T10:07:44Z

130.246.132.26: /* Support for ForTran */

= Disclaimer =

This description is meant to inform the community how to layout
the information within the XML files. However, should the
information in this document and the [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xsd cansas1d/1.0 SAS XML Schema]
differ, the XML Schema will be deemed to have the most correct description
of the standard.

= Objective =

One of the first aims of the canSAS (Collective Action for Nomadic Small-Angle Scatterers) forum of users, software developers, and facility staff was to discuss better sharing of SAS data analysis software. [http://www.smallangles.net/canSAS/ CanSAS] identified that a significant need within the SAS community can be satisfied by a robust, self-describing, text-based, standard format to communicate reduced one-dimensional small-angle scattering data, ''I(Q)'', between users of our facilities. Our goal has been to define such a format that leaves the data file instantly human-readable, editable in the simplest of editors, and importable by simple text import filters in programs that need not recognise advanced structure in the file nor require advanced programming interfaces. The file should contain both the primary data of ''I(Q)'' and also any other descriptive information (metadata) about the sample, measurement, instrument, processing, or analysis steps.

The cansas1d/1.0 standard meets the objectives for a 1D standard, incorporating metadata about the measurement, parameters and results of processing or analysis steps. Even multiple measurements (related or unrelated) may be included within a single XML file.

= General Layout of the XML Data =

The canSAS 1-D standard for reduced 1-D SAS data is implemented using XML files.
A single file can contain SAS data from a single experiment or multiple experiments.
All types of relevant data (''I(Q)'', metadata) are described for each experiment.
More details are provided below.

== Overview ==

[[Image:cansas1d-v1-10-minimum.png|thumb|block diagram of minimum elements required for cansas1d/1.0 standard]]

The basic elements of the cansas1d/1.0 standard are shown in the following table.
After an XML header, the root element of the file is [[cansas1d_SASroot | SASroot]] which
contains one or more [[cansas1d_SASentry | SASentry]] elements, each of which describes
a single experiment (data set, time-slice, step in a series, new
sample, etc.). Details of the [[cansas1d_SASentry | SASentry]] element are also
shown in the next figure. Refer to the [[cansas1d_block_diagrams | block diagrams]]
for alternative depictions. See [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xml cansas1d.xml] for an example XML file.
Examples, Case Studies, and other background information are below. More discussion can be found on the [[1D_Data_Formats_Working_Group|canSAS 1D Data Formats Working Group]] page and its [[Talk:1D_Data_Formats_Working_Group|discussion]] page. Details about
each specific field (XPath string, XML elements and attributes) are described
on the [[cansas1d_definition_of_terms]] page.

* [[cansas1d_SASroot | '''SASroot''': the root element of the file (after the XML header)]]
* [[cansas1d_SASentry | '''SASentry''': describes a single experiment (data set, time-slice, step in a series, new sample, etc.)]]
* [[cansas1d_block_diagrams | block diagrams]]
* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xml cansas1d.xml] example XML file
* discussion of this format: [[1D_Data_Formats_Working_Group | basic]] [[Talk:1D_Data_Formats_Working_Group | more]]
* [[# Help_for_XML | Seek outside help for XML]]
* [[cansas1d_definition_of_terms | '''Definition of terms''': Details about each specific field (XPath string, XML elements and attributes)]]

=== Basic elements of the cansas1d/1.0 standard ===

{|
|-
! {{Headcellstyle}} | element
||  
! {{Headcellstyle}} | description
|-
||
[[cansas1d_XML_header | XML header]]
||
||descriptive info required at the start of every XML file
|-
||
:[[cansas1d_SASroot | SASroot]]
||
||
|-
||
::[[cansas1d_SASentry | SASentry]]
||
||data set, time-slice, step in a series, new sample, etc.
|-
||
:::Title
||
||for this particular SASentry
|-
||
:::Run
||
||run number or ID number of experiment
|-
||
:::[[cansas1d_any | {any}]]
||
||any non-cansas1d/1.0 element can be used at this point
|-
||
:::[[cansas1d_SASdata | SASdata]]
||
||this is where the reduced 1-D SAS data is stored
|-
||
::::[[cansas1d_SASdata | Idata]]
||
||a single data point in the dataset
|-
||
:::[[cansas1d_any | {any}]]
||
||any non-cansas1d/1.0 element can be used at this point
|-
||
:::[[cansas1d_SASsample | SASsample]]
||
|| description of the sample
|-
||
:::[[cansas1d_SASinstrument | SASinstrument]]
||
|| description of the instrument
|-
||
::::[[cansas1d_SASsource | SASsource]]
||
|| description of the source
|-
||
::::[[cansas1d_SAScollimation | SAScollimation]]
||
|| description of the collimation
|-
||
::::[[cansas1d_SASdetector | SASdetector]]
||
|| description of the detector
|-
||
:::[[cansas1d_SASprocess | SASprocess]]
||
||for each processing or analysis step
|-
||
:::[[cansas1d_SASnote | SASnote]]
||
||anything at all
|}

=== Required XML file header ===
<pre>
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="example.xsl" ?>
<SASroot version="1.0"
xmlns="cansas1d/1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="cansas1d/1.0 http://svn.smallangles.net/svn/canSAS/1dwg/trunk/cansas1d.xsd"
>
</pre>

== Rules ==

[[Image:Q-geometry.jpg|thumb|definition of Q geometry for small-angle scattering]]

[[Image:Translation-orientation-geometry-2.jpg|thumb|definition of translation and orientation geometry]]

# canSAS1d/1.0 XML data files will adhere to the standard if they can successfully [[cansas1d_documentation#Validation_of_XML_against_the_Schema | validate]] against the established XML Schema ([http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xsd cansas1d.xsd])
# Q=(4 π / λ) sin(θ) where λ is the wavelength of the radiation and 2θ is the angle through which the detected radiation has been scattered.
# units to be given in standard SI abbreviations (eg, m, cm, mm, nm, K) with the following exceptions:
##um=micrometres
##C=celsius
##A=Angstroms
##percent=%.
##fraction
##a.u.=arbitrary units
##none=no units are relevant (such as dimensionless)
# where reciprocal units need to be quoted the format shall be "1/abbreviation"
# when raised to a power, use similar to "A^3" or "1/m^4" (and not "A3" or "m-4")
# axes:
##z is along the flight path (positive value in the direction of the detector)
##x is orthogonal to z in the horizontal plane (positive values increase to the right when viewed towards the incoming radiation)
##y is orthogonal to z and x in the vertical plane (positive values increase upwards)
#orientation (angles):
##roll is about z
##pitch is about x
##yaw is about y
# Unicode characters MUST NOT be used
# Binary data is not supported

=== Compatibility of Geometry Definitions ===

Note: translation and orientation geometry used by canSAS are consistent with:
#http://en.wikipedia.org/wiki/Cartesian_coordinate_system
#http://en.wikipedia.org/wiki/Right-hand_rule
#http://www.nexusformat.org/Coordinate_Systems
#http://mcstas.risoe.dk/documentation/tutorial/node6.html
#http://webhost5.nts.jhu.edu/reza/book/kinematics/kinematics.htm

The translation and orientation geometry definitions used
here are different than those used by
'''SHADOW''' ([http://www.nanotech.wisc.edu/shadow/ http://www.nanotech.wisc.edu/shadow/])
where the ''y'' and ''z'' axes are swapped
and the direction of ''x'' is changed.

= Documentation and Definitions =

* '''Documentation''': [[cansas1d_documentation]] (this page)
* '''Definitions''': [[cansas1d_definition_of_terms]]
* '''Block diagrams''': [[cansas1d_block_diagrams]]

== XML Schema ==

'''XML Schema''': The [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xsd cansas1d.xsd] [http://www.w3schools.com/xsd XML Schema] defines the rules for the XML file format ([http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xsd TRAC], [http://svn.smallangles.net/svn/canSAS/1dwg/trunk/cansas1d.xsd SVN]) and is used to validate
any XML file for adherence to the format.

== XML Stylesheets ==

* '''example.xsl''': [http://www.w3schools.com/xsl/ XSLT stylesheets] can be used to extract metadata or to convert into another file format. The default canSAS stylesheet [[http://svn.smallangles.net/svn/canSAS/1dwg/trunk/example.xsl | example.xsl]] should be copied into the each folder with canSAS XML data file(s). It can be used to display the data in a supporting WWW browser (such as Firefox or Internet Explorer) or to import into Microsoft Excel (with the added XML support in Excel). By default, MS Windows binds '''*.xml''' files to start Internet Explorer. Double-clicking on a canSAS XML data file with the [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/example.xsl '''example.xsl'''] stylesheet in the same directory will produce a WWW page with the SAS data and selected metadata.

== Examples and Case Studies ==

* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xml cansas1d.xml] basic example: Note that, for clarity, only one row of data is shown. This is probably a very good example to use as a starting point for creating XML files with a text editor.
* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/bimodal-test1.xml bimodal-test1.xml]: Simulated SAS data to test size distribution calculation routines.
* [[cansas1d_casestudy_collagen | dry chick collagen]]: illustrates the minimum information necessary to meet the requirements of the standard format
* [[cansas1d_casestudy_af1410 | AF1410 steel]]: SANS study using magnetic contrast variation (with multiple samples and multiple data sets for each sample), the files can be viewed from TRAC (no description yet): [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/examples/af1410/ http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/examples/af1410/]
* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d-template.xml cansas1d-template.xml]: This is used to test all the rules in the XML Schema. This is probably not a very good example to use as a starting point for creating XML files with a text editor since it tests many of the special-case rules.

=== XML layout for multiple experiments ===

Each experiment is described with a single '''SASentry''' element.
The fragment below shows how multiple experiments can be included
in a single XML file. Full examples of canSAS XML files with multiple experiments include:
* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/W1W2.XML ISIS LOQ SANS instrument]
* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cs_af1410.xml NIST SANS data]

<pre>
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="example.xsl" ?>
<SASroot version="1.0"
xmlns="cansas1d/1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="cansas1d/1.0 http://svn.smallangles.net/svn/canSAS/1dwg/trunk/cansas1d.xsd"
>
<SASentry name="071121.dat#S22">

</SASentry>
<SASentry name="example temperature series">

<Title>title of this series</Title>
<Run name="run1">42-001</Run>
<Run name="run2">42-002</Run>
<SASdata name="run1">

</SASdata>
<SASdata name="run2">

</SASdata>

</SASentry>
<SASentry name="other sample">


</SASentry>
</SASroot>
</pre>

== Foreign Elements ==

To allow for inclusion of elements that are not defined by the cansas1d.xsd XML Schema,
XML ''foreign elements'' are permitted at select locations in the cansas1d/1.0 format.
Please refer to the references (and others) [[#Help_for_XML | below]] for deeper discussions on foreign elements.

An example of the use of foreign elements in the cansas1d/1.0 format is given by
* http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/s81-polyurea.xml

== Support tools for Visualization & Analysis software ==

(under development as of May 2008)

=== Binding for IgorPro ===

An import tool (a.k.a. binding) for [http://www.wavemetrics.com/ IgorPro] has been created
([http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/IgorPro/cansasXML.ipf cansasXML.ipf]).
[[cansas1d_binding_IgorPro | Documentation]] is available.


'''Note''' that this tool is not a true [http://en.wikipedia.org/wiki/Binding_%28computer_science%29 binding] in that the structure of the XML file is not replicated in IgorPro data structures. This tool reads the vectors of 1-D SAS data (Q, I, ...) into IgorPro waves (Qsas, Isas, ...). The tool also reads most of the metadata into an IgorPro textWave for use by other support in IgorPro.


Status:
* Capabilities
** test suite of XML files developed
** import the XML files into IgorPro
* open TRAC tickets (as of 2008-05-15)
** http://svn.smallangles.net/trac/canSAS/ticket/3
** http://svn.smallangles.net/trac/canSAS/ticket/10
* Further
** conversation with Andrew Nelson shows more efficiency can be gained from minor redesign.
** Development of a GUI (to support the [http://usaxs.xor.aps.anl.gov/staff/ilavsky/irena.html Irena] package) has begun
** Development to add export capabilities (from IgorPro) back to the cansas1d/1.0 format has begun

=== Binding for Java ===

A [http://java.sun.com/developer/technicalArticles/WebServices/jaxb/ JAXB] [http://en.wikipedia.org/wiki/Binding_%28computer_science%29 binding] for Java has been created.
(http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/java/cansas1d).
[[cansas1d_binding_Java | Documentation]] is in progress.


'''Note''' that this tool replicates the structure of the XML file into Java data structures. Additional Java software must be applied to convert the /SASdata/Idata/* elements into vectors of ''I(Q)'' data.


Status:
* Capabilities
** import the XML files into Java
** tested by use in collimation-correction (desmearing) software (not publicly available at this time)
* Further
** need to test export capabilities

=== Support for Python ===

Specific support for the cansas1d/1.0 data standard in Python has not been developed.
However, some constructive suggestions are being gathered on a specific
[[cansas1d_binding_Python | Python Documentation]] page.

=== Support for ForTran ===

Specific support for the cansas1d/1.0 data standard in FORTRAN (remember? It stands for FORmula TRANslation) has not been developed.
However, some information is gathered on a specific
[[cansas1d_binding_Fortran | Fortran Documentation]] page.

== Software repositories ==

* '''TRAC''': [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk]
* '''Subversion''': [http://svn.smallangles.net/svn/canSAS/1dwg http://svn.smallangles.net/svn/canSAS/1dwg] (<nowiki>svn checkout http://svn.smallangles.net/svn/canSAS/1dwg/trunk/ cansas-1dwg</nowiki>)

= Validation of XML against the Schema =
# open browser to: http://www.xmlvalidation.com/
# paste content of candidate XML file (with reference in the header to the XML Schema as shown above) into the form
# press <validate>
# paste content of [http://svn.smallangles.net/svn/canSAS/1dwg/trunk/cansas1d.xsd cansas1d.xsd] XSD file into form and press <continue validation>
# check the results

= Help for XML =
; XML: EXtensible Markup Language http://www.w3schools.com/xml/ http://www.w3.org/XML/ http://en.wikipedia.org/wiki/XML http://www.zvon.org/xxl/XPathTutorial/General/examples.html
; XSL (or XSLT): EXtensible Stylesheet Language http://www.w3schools.com/xsl/ http://www.w3.org/Style/XSL/ http://en.wikipedia.org/wiki/Extensible_Stylesheet_Language http://en.wikipedia.org/wiki/XSLT
; XPath: XPath is a language for finding information in an XML document. http://www.w3schools.com/xpath/ http://www.w3.org/Style/XSL/ http://en.wikipedia.org/wiki/XPath
; Schema: An XML Schema describes the structure of an XML document. http://www.w3schools.com/schema/ http://www.w3.org/XML/Schema http://en.wikipedia.org/wiki/XSD
; XML Namespaces: XML namespaces are used for providing uniquely named elements and attributes in an XML instance. http://www.zvon.org/xxl/NamespaceTutorial/Output http://en.wikipedia.org/wiki/XML_namespaces http://www.w3schools.com/XML/xml_namespaces.asp
; XML Foreign Elements: Inclusion of elements, at select locations, that are not defined by the cansas1d.xsd XML Schema http://books.xmlschemata.org/relaxng/relax-CHP-11-SECT-4.html http://www.w3.org/TR/SVG/extend.html http://www.google.com/search?q=XML+foreign+elements

cansas1d documentation

2008-05-19T10:06:25Z

130.246.132.26: /* Support for ForTran */

= Disclaimer =

This description is meant to inform the community how to layout
the information within the XML files. However, should the
information in this document and the [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xsd cansas1d/1.0 SAS XML Schema]
differ, the XML Schema will be deemed to have the most correct description
of the standard.

= Objective =

One of the first aims of the canSAS (Collective Action for Nomadic Small-Angle Scatterers) forum of users, software developers, and facility staff was to discuss better sharing of SAS data analysis software. [http://www.smallangles.net/canSAS/ CanSAS] identified that a significant need within the SAS community can be satisfied by a robust, self-describing, text-based, standard format to communicate reduced one-dimensional small-angle scattering data, ''I(Q)'', between users of our facilities. Our goal has been to define such a format that leaves the data file instantly human-readable, editable in the simplest of editors, and importable by simple text import filters in programs that need not recognise advanced structure in the file nor require advanced programming interfaces. The file should contain both the primary data of ''I(Q)'' and also any other descriptive information (metadata) about the sample, measurement, instrument, processing, or analysis steps.

The cansas1d/1.0 standard meets the objectives for a 1D standard, incorporating metadata about the measurement, parameters and results of processing or analysis steps. Even multiple measurements (related or unrelated) may be included within a single XML file.

= General Layout of the XML Data =

The canSAS 1-D standard for reduced 1-D SAS data is implemented using XML files.
A single file can contain SAS data from a single experiment or multiple experiments.
All types of relevant data (''I(Q)'', metadata) are described for each experiment.
More details are provided below.

== Overview ==

[[Image:cansas1d-v1-10-minimum.png|thumb|block diagram of minimum elements required for cansas1d/1.0 standard]]

The basic elements of the cansas1d/1.0 standard are shown in the following table.
After an XML header, the root element of the file is [[cansas1d_SASroot | SASroot]] which
contains one or more [[cansas1d_SASentry | SASentry]] elements, each of which describes
a single experiment (data set, time-slice, step in a series, new
sample, etc.). Details of the [[cansas1d_SASentry | SASentry]] element are also
shown in the next figure. Refer to the [[cansas1d_block_diagrams | block diagrams]]
for alternative depictions. See [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xml cansas1d.xml] for an example XML file.
Examples, Case Studies, and other background information are below. More discussion can be found on the [[1D_Data_Formats_Working_Group|canSAS 1D Data Formats Working Group]] page and its [[Talk:1D_Data_Formats_Working_Group|discussion]] page. Details about
each specific field (XPath string, XML elements and attributes) are described
on the [[cansas1d_definition_of_terms]] page.

* [[cansas1d_SASroot | '''SASroot''': the root element of the file (after the XML header)]]
* [[cansas1d_SASentry | '''SASentry''': describes a single experiment (data set, time-slice, step in a series, new sample, etc.)]]
* [[cansas1d_block_diagrams | block diagrams]]
* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xml cansas1d.xml] example XML file
* discussion of this format: [[1D_Data_Formats_Working_Group | basic]] [[Talk:1D_Data_Formats_Working_Group | more]]
* [[# Help_for_XML | Seek outside help for XML]]
* [[cansas1d_definition_of_terms | '''Definition of terms''': Details about each specific field (XPath string, XML elements and attributes)]]

=== Basic elements of the cansas1d/1.0 standard ===

{|
|-
! {{Headcellstyle}} | element
||  
! {{Headcellstyle}} | description
|-
||
[[cansas1d_XML_header | XML header]]
||
||descriptive info required at the start of every XML file
|-
||
:[[cansas1d_SASroot | SASroot]]
||
||
|-
||
::[[cansas1d_SASentry | SASentry]]
||
||data set, time-slice, step in a series, new sample, etc.
|-
||
:::Title
||
||for this particular SASentry
|-
||
:::Run
||
||run number or ID number of experiment
|-
||
:::[[cansas1d_any | {any}]]
||
||any non-cansas1d/1.0 element can be used at this point
|-
||
:::[[cansas1d_SASdata | SASdata]]
||
||this is where the reduced 1-D SAS data is stored
|-
||
::::[[cansas1d_SASdata | Idata]]
||
||a single data point in the dataset
|-
||
:::[[cansas1d_any | {any}]]
||
||any non-cansas1d/1.0 element can be used at this point
|-
||
:::[[cansas1d_SASsample | SASsample]]
||
|| description of the sample
|-
||
:::[[cansas1d_SASinstrument | SASinstrument]]
||
|| description of the instrument
|-
||
::::[[cansas1d_SASsource | SASsource]]
||
|| description of the source
|-
||
::::[[cansas1d_SAScollimation | SAScollimation]]
||
|| description of the collimation
|-
||
::::[[cansas1d_SASdetector | SASdetector]]
||
|| description of the detector
|-
||
:::[[cansas1d_SASprocess | SASprocess]]
||
||for each processing or analysis step
|-
||
:::[[cansas1d_SASnote | SASnote]]
||
||anything at all
|}

=== Required XML file header ===
<pre>
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="example.xsl" ?>
<SASroot version="1.0"
xmlns="cansas1d/1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="cansas1d/1.0 http://svn.smallangles.net/svn/canSAS/1dwg/trunk/cansas1d.xsd"
>
</pre>

== Rules ==

[[Image:Q-geometry.jpg|thumb|definition of Q geometry for small-angle scattering]]

[[Image:Translation-orientation-geometry-2.jpg|thumb|definition of translation and orientation geometry]]

# canSAS1d/1.0 XML data files will adhere to the standard if they can successfully [[cansas1d_documentation#Validation_of_XML_against_the_Schema | validate]] against the established XML Schema ([http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xsd cansas1d.xsd])
# Q=(4 π / λ) sin(θ) where λ is the wavelength of the radiation and 2θ is the angle through which the detected radiation has been scattered.
# units to be given in standard SI abbreviations (eg, m, cm, mm, nm, K) with the following exceptions:
##um=micrometres
##C=celsius
##A=Angstroms
##percent=%.
##fraction
##a.u.=arbitrary units
##none=no units are relevant (such as dimensionless)
# where reciprocal units need to be quoted the format shall be "1/abbreviation"
# when raised to a power, use similar to "A^3" or "1/m^4" (and not "A3" or "m-4")
# axes:
##z is along the flight path (positive value in the direction of the detector)
##x is orthogonal to z in the horizontal plane (positive values increase to the right when viewed towards the incoming radiation)
##y is orthogonal to z and x in the vertical plane (positive values increase upwards)
#orientation (angles):
##roll is about z
##pitch is about x
##yaw is about y
# Unicode characters MUST NOT be used
# Binary data is not supported

=== Compatibility of Geometry Definitions ===

Note: translation and orientation geometry used by canSAS are consistent with:
#http://en.wikipedia.org/wiki/Cartesian_coordinate_system
#http://en.wikipedia.org/wiki/Right-hand_rule
#http://www.nexusformat.org/Coordinate_Systems
#http://mcstas.risoe.dk/documentation/tutorial/node6.html
#http://webhost5.nts.jhu.edu/reza/book/kinematics/kinematics.htm

The translation and orientation geometry definitions used
here are different than those used by
'''SHADOW''' ([http://www.nanotech.wisc.edu/shadow/ http://www.nanotech.wisc.edu/shadow/])
where the ''y'' and ''z'' axes are swapped
and the direction of ''x'' is changed.

= Documentation and Definitions =

* '''Documentation''': [[cansas1d_documentation]] (this page)
* '''Definitions''': [[cansas1d_definition_of_terms]]
* '''Block diagrams''': [[cansas1d_block_diagrams]]

== XML Schema ==

'''XML Schema''': The [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xsd cansas1d.xsd] [http://www.w3schools.com/xsd XML Schema] defines the rules for the XML file format ([http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xsd TRAC], [http://svn.smallangles.net/svn/canSAS/1dwg/trunk/cansas1d.xsd SVN]) and is used to validate
any XML file for adherence to the format.

== XML Stylesheets ==

* '''example.xsl''': [http://www.w3schools.com/xsl/ XSLT stylesheets] can be used to extract metadata or to convert into another file format. The default canSAS stylesheet [[http://svn.smallangles.net/svn/canSAS/1dwg/trunk/example.xsl | example.xsl]] should be copied into the each folder with canSAS XML data file(s). It can be used to display the data in a supporting WWW browser (such as Firefox or Internet Explorer) or to import into Microsoft Excel (with the added XML support in Excel). By default, MS Windows binds '''*.xml''' files to start Internet Explorer. Double-clicking on a canSAS XML data file with the [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/example.xsl '''example.xsl'''] stylesheet in the same directory will produce a WWW page with the SAS data and selected metadata.

== Examples and Case Studies ==

* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xml cansas1d.xml] basic example: Note that, for clarity, only one row of data is shown. This is probably a very good example to use as a starting point for creating XML files with a text editor.
* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/bimodal-test1.xml bimodal-test1.xml]: Simulated SAS data to test size distribution calculation routines.
* [[cansas1d_casestudy_collagen | dry chick collagen]]: illustrates the minimum information necessary to meet the requirements of the standard format
* [[cansas1d_casestudy_af1410 | AF1410 steel]]: SANS study using magnetic contrast variation (with multiple samples and multiple data sets for each sample), the files can be viewed from TRAC (no description yet): [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/examples/af1410/ http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/examples/af1410/]
* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d-template.xml cansas1d-template.xml]: This is used to test all the rules in the XML Schema. This is probably not a very good example to use as a starting point for creating XML files with a text editor since it tests many of the special-case rules.

=== XML layout for multiple experiments ===

Each experiment is described with a single '''SASentry''' element.
The fragment below shows how multiple experiments can be included
in a single XML file. Full examples of canSAS XML files with multiple experiments include:
* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/W1W2.XML ISIS LOQ SANS instrument]
* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cs_af1410.xml NIST SANS data]

<pre>
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="example.xsl" ?>
<SASroot version="1.0"
xmlns="cansas1d/1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="cansas1d/1.0 http://svn.smallangles.net/svn/canSAS/1dwg/trunk/cansas1d.xsd"
>
<SASentry name="071121.dat#S22">

</SASentry>
<SASentry name="example temperature series">

<Title>title of this series</Title>
<Run name="run1">42-001</Run>
<Run name="run2">42-002</Run>
<SASdata name="run1">

</SASdata>
<SASdata name="run2">

</SASdata>

</SASentry>
<SASentry name="other sample">


</SASentry>
</SASroot>
</pre>

== Foreign Elements ==

To allow for inclusion of elements that are not defined by the cansas1d.xsd XML Schema,
XML ''foreign elements'' are permitted at select locations in the cansas1d/1.0 format.
Please refer to the references (and others) [[#Help_for_XML | below]] for deeper discussions on foreign elements.

An example of the use of foreign elements in the cansas1d/1.0 format is given by
* http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/s81-polyurea.xml

== Support tools for Visualization & Analysis software ==

(under development as of May 2008)

=== Binding for IgorPro ===

An import tool (a.k.a. binding) for [http://www.wavemetrics.com/ IgorPro] has been created
([http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/IgorPro/cansasXML.ipf cansasXML.ipf]).
[[cansas1d_binding_IgorPro | Documentation]] is available.


'''Note''' that this tool is not a true [http://en.wikipedia.org/wiki/Binding_%28computer_science%29 binding] in that the structure of the XML file is not replicated in IgorPro data structures. This tool reads the vectors of 1-D SAS data (Q, I, ...) into IgorPro waves (Qsas, Isas, ...). The tool also reads most of the metadata into an IgorPro textWave for use by other support in IgorPro.


Status:
* Capabilities
** test suite of XML files developed
** import the XML files into IgorPro
* open TRAC tickets (as of 2008-05-15)
** http://svn.smallangles.net/trac/canSAS/ticket/3
** http://svn.smallangles.net/trac/canSAS/ticket/10
* Further
** conversation with Andrew Nelson shows more efficiency can be gained from minor redesign.
** Development of a GUI (to support the [http://usaxs.xor.aps.anl.gov/staff/ilavsky/irena.html Irena] package) has begun
** Development to add export capabilities (from IgorPro) back to the cansas1d/1.0 format has begun

=== Binding for Java ===

A [http://java.sun.com/developer/technicalArticles/WebServices/jaxb/ JAXB] [http://en.wikipedia.org/wiki/Binding_%28computer_science%29 binding] for Java has been created.
(http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/java/cansas1d).
[[cansas1d_binding_Java | Documentation]] is in progress.


'''Note''' that this tool replicates the structure of the XML file into Java data structures. Additional Java software must be applied to convert the /SASdata/Idata/* elements into vectors of ''I(Q)'' data.


Status:
* Capabilities
** import the XML files into Java
** tested by use in collimation-correction (desmearing) software (not publicly available at this time)
* Further
** need to test export capabilities

=== Support for Python ===

Specific support for the cansas1d/1.0 data standard in Python has not been developed.
However, some constructive suggestions are being gathered on a specific
[[cansas1d_binding_Python | Python Documentation]] page.

=== Support for ForTran ===

Specific support for the cansas1d/1.0 data standard in FORTRAN (remember? FORTRAN stands for FORmula TRANslation) has not been developed.
However, some constructive suggestions could be gathered on a specific
[[cansas1d_binding_Fortran | Fortran Documentation]] page.

== Software repositories ==

* '''TRAC''': [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk]
* '''Subversion''': [http://svn.smallangles.net/svn/canSAS/1dwg http://svn.smallangles.net/svn/canSAS/1dwg] (<nowiki>svn checkout http://svn.smallangles.net/svn/canSAS/1dwg/trunk/ cansas-1dwg</nowiki>)

= Validation of XML against the Schema =
# open browser to: http://www.xmlvalidation.com/
# paste content of candidate XML file (with reference in the header to the XML Schema as shown above) into the form
# press <validate>
# paste content of [http://svn.smallangles.net/svn/canSAS/1dwg/trunk/cansas1d.xsd cansas1d.xsd] XSD file into form and press <continue validation>
# check the results

= Help for XML =
; XML: EXtensible Markup Language http://www.w3schools.com/xml/ http://www.w3.org/XML/ http://en.wikipedia.org/wiki/XML http://www.zvon.org/xxl/XPathTutorial/General/examples.html
; XSL (or XSLT): EXtensible Stylesheet Language http://www.w3schools.com/xsl/ http://www.w3.org/Style/XSL/ http://en.wikipedia.org/wiki/Extensible_Stylesheet_Language http://en.wikipedia.org/wiki/XSLT
; XPath: XPath is a language for finding information in an XML document. http://www.w3schools.com/xpath/ http://www.w3.org/Style/XSL/ http://en.wikipedia.org/wiki/XPath
; Schema: An XML Schema describes the structure of an XML document. http://www.w3schools.com/schema/ http://www.w3.org/XML/Schema http://en.wikipedia.org/wiki/XSD
; XML Namespaces: XML namespaces are used for providing uniquely named elements and attributes in an XML instance. http://www.zvon.org/xxl/NamespaceTutorial/Output http://en.wikipedia.org/wiki/XML_namespaces http://www.w3schools.com/XML/xml_namespaces.asp
; XML Foreign Elements: Inclusion of elements, at select locations, that are not defined by the cansas1d.xsd XML Schema http://books.xmlschemata.org/relaxng/relax-CHP-11-SECT-4.html http://www.w3.org/TR/SVG/extend.html http://www.google.com/search?q=XML+foreign+elements

cansas1d documentation

2008-05-19T10:05:20Z

130.246.132.26: /* Support for ForTran */

= Disclaimer =

This description is meant to inform the community how to layout
the information within the XML files. However, should the
information in this document and the [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xsd cansas1d/1.0 SAS XML Schema]
differ, the XML Schema will be deemed to have the most correct description
of the standard.

= Objective =

One of the first aims of the canSAS (Collective Action for Nomadic Small-Angle Scatterers) forum of users, software developers, and facility staff was to discuss better sharing of SAS data analysis software. [http://www.smallangles.net/canSAS/ CanSAS] identified that a significant need within the SAS community can be satisfied by a robust, self-describing, text-based, standard format to communicate reduced one-dimensional small-angle scattering data, ''I(Q)'', between users of our facilities. Our goal has been to define such a format that leaves the data file instantly human-readable, editable in the simplest of editors, and importable by simple text import filters in programs that need not recognise advanced structure in the file nor require advanced programming interfaces. The file should contain both the primary data of ''I(Q)'' and also any other descriptive information (metadata) about the sample, measurement, instrument, processing, or analysis steps.

The cansas1d/1.0 standard meets the objectives for a 1D standard, incorporating metadata about the measurement, parameters and results of processing or analysis steps. Even multiple measurements (related or unrelated) may be included within a single XML file.

= General Layout of the XML Data =

The canSAS 1-D standard for reduced 1-D SAS data is implemented using XML files.
A single file can contain SAS data from a single experiment or multiple experiments.
All types of relevant data (''I(Q)'', metadata) are described for each experiment.
More details are provided below.

== Overview ==

[[Image:cansas1d-v1-10-minimum.png|thumb|block diagram of minimum elements required for cansas1d/1.0 standard]]

The basic elements of the cansas1d/1.0 standard are shown in the following table.
After an XML header, the root element of the file is [[cansas1d_SASroot | SASroot]] which
contains one or more [[cansas1d_SASentry | SASentry]] elements, each of which describes
a single experiment (data set, time-slice, step in a series, new
sample, etc.). Details of the [[cansas1d_SASentry | SASentry]] element are also
shown in the next figure. Refer to the [[cansas1d_block_diagrams | block diagrams]]
for alternative depictions. See [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xml cansas1d.xml] for an example XML file.
Examples, Case Studies, and other background information are below. More discussion can be found on the [[1D_Data_Formats_Working_Group|canSAS 1D Data Formats Working Group]] page and its [[Talk:1D_Data_Formats_Working_Group|discussion]] page. Details about
each specific field (XPath string, XML elements and attributes) are described
on the [[cansas1d_definition_of_terms]] page.

* [[cansas1d_SASroot | '''SASroot''': the root element of the file (after the XML header)]]
* [[cansas1d_SASentry | '''SASentry''': describes a single experiment (data set, time-slice, step in a series, new sample, etc.)]]
* [[cansas1d_block_diagrams | block diagrams]]
* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xml cansas1d.xml] example XML file
* discussion of this format: [[1D_Data_Formats_Working_Group | basic]] [[Talk:1D_Data_Formats_Working_Group | more]]
* [[# Help_for_XML | Seek outside help for XML]]
* [[cansas1d_definition_of_terms | '''Definition of terms''': Details about each specific field (XPath string, XML elements and attributes)]]

=== Basic elements of the cansas1d/1.0 standard ===

{|
|-
! {{Headcellstyle}} | element
||  
! {{Headcellstyle}} | description
|-
||
[[cansas1d_XML_header | XML header]]
||
||descriptive info required at the start of every XML file
|-
||
:[[cansas1d_SASroot | SASroot]]
||
||
|-
||
::[[cansas1d_SASentry | SASentry]]
||
||data set, time-slice, step in a series, new sample, etc.
|-
||
:::Title
||
||for this particular SASentry
|-
||
:::Run
||
||run number or ID number of experiment
|-
||
:::[[cansas1d_any | {any}]]
||
||any non-cansas1d/1.0 element can be used at this point
|-
||
:::[[cansas1d_SASdata | SASdata]]
||
||this is where the reduced 1-D SAS data is stored
|-
||
::::[[cansas1d_SASdata | Idata]]
||
||a single data point in the dataset
|-
||
:::[[cansas1d_any | {any}]]
||
||any non-cansas1d/1.0 element can be used at this point
|-
||
:::[[cansas1d_SASsample | SASsample]]
||
|| description of the sample
|-
||
:::[[cansas1d_SASinstrument | SASinstrument]]
||
|| description of the instrument
|-
||
::::[[cansas1d_SASsource | SASsource]]
||
|| description of the source
|-
||
::::[[cansas1d_SAScollimation | SAScollimation]]
||
|| description of the collimation
|-
||
::::[[cansas1d_SASdetector | SASdetector]]
||
|| description of the detector
|-
||
:::[[cansas1d_SASprocess | SASprocess]]
||
||for each processing or analysis step
|-
||
:::[[cansas1d_SASnote | SASnote]]
||
||anything at all
|}

=== Required XML file header ===
<pre>
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="example.xsl" ?>
<SASroot version="1.0"
xmlns="cansas1d/1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="cansas1d/1.0 http://svn.smallangles.net/svn/canSAS/1dwg/trunk/cansas1d.xsd"
>
</pre>

== Rules ==

[[Image:Q-geometry.jpg|thumb|definition of Q geometry for small-angle scattering]]

[[Image:Translation-orientation-geometry-2.jpg|thumb|definition of translation and orientation geometry]]

# canSAS1d/1.0 XML data files will adhere to the standard if they can successfully [[cansas1d_documentation#Validation_of_XML_against_the_Schema | validate]] against the established XML Schema ([http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xsd cansas1d.xsd])
# Q=(4 π / λ) sin(θ) where λ is the wavelength of the radiation and 2θ is the angle through which the detected radiation has been scattered.
# units to be given in standard SI abbreviations (eg, m, cm, mm, nm, K) with the following exceptions:
##um=micrometres
##C=celsius
##A=Angstroms
##percent=%.
##fraction
##a.u.=arbitrary units
##none=no units are relevant (such as dimensionless)
# where reciprocal units need to be quoted the format shall be "1/abbreviation"
# when raised to a power, use similar to "A^3" or "1/m^4" (and not "A3" or "m-4")
# axes:
##z is along the flight path (positive value in the direction of the detector)
##x is orthogonal to z in the horizontal plane (positive values increase to the right when viewed towards the incoming radiation)
##y is orthogonal to z and x in the vertical plane (positive values increase upwards)
#orientation (angles):
##roll is about z
##pitch is about x
##yaw is about y
# Unicode characters MUST NOT be used
# Binary data is not supported

=== Compatibility of Geometry Definitions ===

Note: translation and orientation geometry used by canSAS are consistent with:
#http://en.wikipedia.org/wiki/Cartesian_coordinate_system
#http://en.wikipedia.org/wiki/Right-hand_rule
#http://www.nexusformat.org/Coordinate_Systems
#http://mcstas.risoe.dk/documentation/tutorial/node6.html
#http://webhost5.nts.jhu.edu/reza/book/kinematics/kinematics.htm

The translation and orientation geometry definitions used
here are different than those used by
'''SHADOW''' ([http://www.nanotech.wisc.edu/shadow/ http://www.nanotech.wisc.edu/shadow/])
where the ''y'' and ''z'' axes are swapped
and the direction of ''x'' is changed.

= Documentation and Definitions =

* '''Documentation''': [[cansas1d_documentation]] (this page)
* '''Definitions''': [[cansas1d_definition_of_terms]]
* '''Block diagrams''': [[cansas1d_block_diagrams]]

== XML Schema ==

'''XML Schema''': The [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xsd cansas1d.xsd] [http://www.w3schools.com/xsd XML Schema] defines the rules for the XML file format ([http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xsd TRAC], [http://svn.smallangles.net/svn/canSAS/1dwg/trunk/cansas1d.xsd SVN]) and is used to validate
any XML file for adherence to the format.

== XML Stylesheets ==

* '''example.xsl''': [http://www.w3schools.com/xsl/ XSLT stylesheets] can be used to extract metadata or to convert into another file format. The default canSAS stylesheet [[http://svn.smallangles.net/svn/canSAS/1dwg/trunk/example.xsl | example.xsl]] should be copied into the each folder with canSAS XML data file(s). It can be used to display the data in a supporting WWW browser (such as Firefox or Internet Explorer) or to import into Microsoft Excel (with the added XML support in Excel). By default, MS Windows binds '''*.xml''' files to start Internet Explorer. Double-clicking on a canSAS XML data file with the [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/example.xsl '''example.xsl'''] stylesheet in the same directory will produce a WWW page with the SAS data and selected metadata.

== Examples and Case Studies ==

* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d.xml cansas1d.xml] basic example: Note that, for clarity, only one row of data is shown. This is probably a very good example to use as a starting point for creating XML files with a text editor.
* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/bimodal-test1.xml bimodal-test1.xml]: Simulated SAS data to test size distribution calculation routines.
* [[cansas1d_casestudy_collagen | dry chick collagen]]: illustrates the minimum information necessary to meet the requirements of the standard format
* [[cansas1d_casestudy_af1410 | AF1410 steel]]: SANS study using magnetic contrast variation (with multiple samples and multiple data sets for each sample), the files can be viewed from TRAC (no description yet): [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/examples/af1410/ http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/examples/af1410/]
* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cansas1d-template.xml cansas1d-template.xml]: This is used to test all the rules in the XML Schema. This is probably not a very good example to use as a starting point for creating XML files with a text editor since it tests many of the special-case rules.

=== XML layout for multiple experiments ===

Each experiment is described with a single '''SASentry''' element.
The fragment below shows how multiple experiments can be included
in a single XML file. Full examples of canSAS XML files with multiple experiments include:
* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/W1W2.XML ISIS LOQ SANS instrument]
* [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/cs_af1410.xml NIST SANS data]

<pre>
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="example.xsl" ?>
<SASroot version="1.0"
xmlns="cansas1d/1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="cansas1d/1.0 http://svn.smallangles.net/svn/canSAS/1dwg/trunk/cansas1d.xsd"
>
<SASentry name="071121.dat#S22">

</SASentry>
<SASentry name="example temperature series">

<Title>title of this series</Title>
<Run name="run1">42-001</Run>
<Run name="run2">42-002</Run>
<SASdata name="run1">

</SASdata>
<SASdata name="run2">

</SASdata>

</SASentry>
<SASentry name="other sample">


</SASentry>
</SASroot>
</pre>

== Foreign Elements ==

To allow for inclusion of elements that are not defined by the cansas1d.xsd XML Schema,
XML ''foreign elements'' are permitted at select locations in the cansas1d/1.0 format.
Please refer to the references (and others) [[#Help_for_XML | below]] for deeper discussions on foreign elements.

An example of the use of foreign elements in the cansas1d/1.0 format is given by
* http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/s81-polyurea.xml

== Support tools for Visualization & Analysis software ==

(under development as of May 2008)

=== Binding for IgorPro ===

An import tool (a.k.a. binding) for [http://www.wavemetrics.com/ IgorPro] has been created
([http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/IgorPro/cansasXML.ipf cansasXML.ipf]).
[[cansas1d_binding_IgorPro | Documentation]] is available.


'''Note''' that this tool is not a true [http://en.wikipedia.org/wiki/Binding_%28computer_science%29 binding] in that the structure of the XML file is not replicated in IgorPro data structures. This tool reads the vectors of 1-D SAS data (Q, I, ...) into IgorPro waves (Qsas, Isas, ...). The tool also reads most of the metadata into an IgorPro textWave for use by other support in IgorPro.


Status:
* Capabilities
** test suite of XML files developed
** import the XML files into IgorPro
* open TRAC tickets (as of 2008-05-15)
** http://svn.smallangles.net/trac/canSAS/ticket/3
** http://svn.smallangles.net/trac/canSAS/ticket/10
* Further
** conversation with Andrew Nelson shows more efficiency can be gained from minor redesign.
** Development of a GUI (to support the [http://usaxs.xor.aps.anl.gov/staff/ilavsky/irena.html Irena] package) has begun
** Development to add export capabilities (from IgorPro) back to the cansas1d/1.0 format has begun

=== Binding for Java ===

A [http://java.sun.com/developer/technicalArticles/WebServices/jaxb/ JAXB] [http://en.wikipedia.org/wiki/Binding_%28computer_science%29 binding] for Java has been created.
(http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk/java/cansas1d).
[[cansas1d_binding_Java | Documentation]] is in progress.


'''Note''' that this tool replicates the structure of the XML file into Java data structures. Additional Java software must be applied to convert the /SASdata/Idata/* elements into vectors of ''I(Q)'' data.


Status:
* Capabilities
** import the XML files into Java
** tested by use in collimation-correction (desmearing) software (not publicly available at this time)
* Further
** need to test export capabilities

=== Support for Python ===

Specific support for the cansas1d/1.0 data standard in Python has not been developed.
However, some constructive suggestions are being gathered on a specific
[[cansas1d_binding_Python | Python Documentation]] page.

=== Support for ForTran ===

Specific support for the cansas1d/1.0 data standard in ForTran (remember? ForTran stands for Formula Translation) has not been developed.
However, some constructive suggestions could be gathered on a specific
[[cansas1d_binding_Fortran | Fortran Documentation]] page.

== Software repositories ==

* '''TRAC''': [http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk http://svn.smallangles.net/trac/canSAS/browser/1dwg/trunk]
* '''Subversion''': [http://svn.smallangles.net/svn/canSAS/1dwg http://svn.smallangles.net/svn/canSAS/1dwg] (<nowiki>svn checkout http://svn.smallangles.net/svn/canSAS/1dwg/trunk/ cansas-1dwg</nowiki>)

= Validation of XML against the Schema =
# open browser to: http://www.xmlvalidation.com/
# paste content of candidate XML file (with reference in the header to the XML Schema as shown above) into the form
# press <validate>
# paste content of [http://svn.smallangles.net/svn/canSAS/1dwg/trunk/cansas1d.xsd cansas1d.xsd] XSD file into form and press <continue validation>
# check the results

= Help for XML =
; XML: EXtensible Markup Language http://www.w3schools.com/xml/ http://www.w3.org/XML/ http://en.wikipedia.org/wiki/XML http://www.zvon.org/xxl/XPathTutorial/General/examples.html
; XSL (or XSLT): EXtensible Stylesheet Language http://www.w3schools.com/xsl/ http://www.w3.org/Style/XSL/ http://en.wikipedia.org/wiki/Extensible_Stylesheet_Language http://en.wikipedia.org/wiki/XSLT
; XPath: XPath is a language for finding information in an XML document. http://www.w3schools.com/xpath/ http://www.w3.org/Style/XSL/ http://en.wikipedia.org/wiki/XPath
; Schema: An XML Schema describes the structure of an XML document. http://www.w3schools.com/schema/ http://www.w3.org/XML/Schema http://en.wikipedia.org/wiki/XSD
; XML Namespaces: XML namespaces are used for providing uniquely named elements and attributes in an XML instance. http://www.zvon.org/xxl/NamespaceTutorial/Output http://en.wikipedia.org/wiki/XML_namespaces http://www.w3schools.com/XML/xml_namespaces.asp
; XML Foreign Elements: Inclusion of elements, at select locations, that are not defined by the cansas1d.xsd XML Schema http://books.xmlschemata.org/relaxng/relax-CHP-11-SECT-4.html http://www.w3.org/TR/SVG/extend.html http://www.google.com/search?q=XML+foreign+elements

Talk:Data Formats Working Group

2007-12-18T11:32:33Z

130.246.132.26: /* In the Idata tag */

----

==Which way to declare the units?==
--[[User:Jemian|Jemian]] 18:06, 17 December 2007 (EST)
Cast your vote by attaching initials to one of these three methods.

===In the Idata tag===
votes: PRJ, ARJN, SMK
<pre>
<SASdata>
<Idata>
<Q units="1/A"></Q>

<Qdev units="1/A"></Qdev>
<Idev units="1/cm"></Idev>
<Qfwhm units="1/A"></Qfwhm>
<Qmean units="1/A"></Qmean>
<ShadowFactor units="1/A"></ShadowFactor>
</Idata>
</SASdata>
</pre>

===As attributes of the SASdata tag===
votes:
<pre>
<SASdata
entries="Q:I:Qdev:Idev:Qfwhm:Qmean:ShadowFactor"
units="1/A:1/cm:1/A:1/cm:1/A:1/A:1/A">
<Idata>
<Q></Q>

<Qdev></Qdev>
<Idev></Idev>
<Qfwhm><! Qfwhm is optional ></Qfwhm>
<Qmean></Qmean>
<ShadowFactor></ShadowFactor>
</Idata>
</SASdata>
</pre>

===As axis declarations within the SASdata tag===
votes:
<pre>
<SASdata>
<axis name="Q" units="1/A" type="Idata" />
<axis name="I" units="1/cm" type="Idata" />
<axis name="Qdev" units="1/A" type="constant" >0.000085</axis>
<axis name="Idev" units="1/cm" type="Idata" />
<axis name="Qfwhm" units="1/A" type="constant" >0.000191</axis>
<axis name="Qmean" units="1/A" type="undefined" />
<axis name="ShadowFactor" units="1/A" type="undefined" />
<Idata>
<Q></Q>

<Qdev></Qdev>
<Idev></Idev>
<Qfwhm><! Qfwhm is optional ></Qfwhm>
<Qmean></Qmean>
<ShadowFactor></ShadowFactor>
</Idata>
</SASdata>
</pre>

----
--[[User:Jemian|Jemian]] 12:01, 14 December 2007 (EST) items moved from main page
==Considerations==
* a key point of what we discussed at NIST:
namely that our goal is to agree a format which that whilst using as much best XML practice as is reasonable, leaves the file instantly human-readable, editable in the simplest of editors, and importable by simple text import filters in programs that don't recognise the XML.
* document what we decide
**1DWG will take care of documenting the format it defines.
*** make that definition with a schema (for absolute validation of any proposed XML file against the standard)
*** instructions on how to use that schema
*** XSL style sheets to present the XML contents in various forms (also serves as examples)
*** a couple of examples
*** maybe also some words.
** move some of this discussion to
*** discussion page
*** other wiki pages
*** /dev/null after its usefulness has been exhausted
* coordinate with other communities
** [http://www.nexusformat.org NeXus (http://www.nexusformat.org)]
** reflectivity
** powder diffraction
*** [http://www.xrdml.com XRDML (http://www.xrdml.com)]
* should we consider a file naming convention?
* should we consider a SAS scan naming convention?
** sequential run number from facility
** convention set by the detector software provider
* XML representation of the I vs. Q data
** tabular format
** vector format
* general XML coding style
** readability by humans
*** with lots of computer skills
*** with rudimentary computer skills
** readability by computers
*** standard XML libraries
*** generic visualization tools
*** common software such as MS Excel or Open Office or [ftp://ftp.ill.fr/pub/cs/rxml XMLPLO source code for windows/linux/OSX86]
*** A plugin for IGOR has been written, [http://www.igorexchange.com/project/XMLutils XMLutils.xop], that can handle XML data. The [[sasXML and IGOR]] page gives details on the IGOR code required to write the prototype sasXML file.
** availability of style sheets
* scalability of XML format to 2D data?
* What is required?
* What is optional?
* Use the same tags again in similar contexts
** X,Y pairs for example, whether detector position, beam center, sample position
{| width="100%" border="1" style="border:1px solid navy; border-collapse: collapse"
|-
! width="50%" | inconsistent
! width="50%" | consistent
|-
| <pre>
<beam_size axis="x" units="mm">12.00</beam_size>
<beam_size axis="y" units="mm">12.00</beam_size>
<x0 units="mm">322.64</x0>
<y0 units="mm">327.68</y0>
<pixel_x units="mm">5.00</pixel_x>
<pixel_y units="mm">5.00</pixel_y>
</pre>
| <pre>
<beam_size axis="x" units="mm">12.00</beam_size>
<beam_size axis="y" units="mm">12.00</beam_size>
<beam_center axis="x" units="mm">322.64</beam_center>
<beam_center axis="y" units="mm">327.68</beam_center>
<pixel_size axis="x" units="mm">5.00</pixel_size>
<pixel_size axis="y" units="mm">5.00</pixel_size>
</pre>
|}

===Points for Discussion===

*Do we want to advocate/recommend particular names for particular tags; eg, SASdata, SASsample, Idata, etc.?
** which ones?
* provide for (optional) inclusion of sample prep details
* provide for (optional) inclusion of other (non-SAS) data in the XML
* Need to allow for more than a single SAS data set in one .xml file

===Other Points===
* It's not clear how to specify that multiple runs were reduced together
**(AJJ) Assuming that those multiple runs were first stored as XML then referencing the individual files would give all that back information (a la Ghosh suggestion). At NIST we take absolute I vs Q files and combine them to produce an absolute I vs Q file thus that is reasonable here. What about elsewhere?
* How does one include the instrument information of the many runs that we used to make up the composite file
* If we have reduction information, then everything needs to be in there, i.e. the run numbers for the can, the standard, the uniform field, etc.
* Information on the averaging, is it radial, sector, rectangular, etc.