JATS - JAT-based test suite
===========================

Version
-------

v0.0 -- proof-of-concept phase

**Disclaimer:** Version number of this specification starts with zero, which
signifies *proof-of-concept phase*; many aspects including the basic ones
**WILL CHANGE**.


Suite
-----

JATS test suite consists of one or more JAT-based tests.  Each test is
a directory, containing:

    test
    meta.yml

*test* is main body of the test.  *meta.yml* is a YAML file containing
test meta-data such as human-readable description, dependencies and
coverage information.


### Root directory ###

Root directory of a test suite is designated by placement of file *format*
in sub-directory *.jats*.  This file MUST contain version number in `X.Y`
format, where X is major format version and `Y` is minor format version.

A tool declaring support for JATS test suite MUST recognize one or more
compatible format versions and refuse to work with suite if no supported
versions match suite version.


### Suite meta-data ###

Test suite meta-data is stored under *.jats* directory under respective
filename:


 *  *domain* - Domain under which test suite exists.  A public JATS test suite
    SHOULD refer to a valid domain name owned by test suite owner.

    Default value is `jats.example.org`.

 *  *ns* - Namespace under which rules for test organization are defined.
    MUST consist of valid Path as defined in [RFC 3986][rfc3986path].

    Usually this contains hierarchical elements such as work group name and
    project name.

    Default value is `anonns`.

 *  *sut_pname* -  Plain name (or "package name") of System Under Test. MUST
    consist of lowercase ASCII letters and dashes.  MUST NOT start with dash.

    Default value is `anonsut`.

 *  *sut_url* - Valid HTTP(s) URL leading to more information about System Under
    Test.   Default value is `https://example.org/anonsut`,

 *  *url* - Valid HTTP(s) URL leading to more information about Test Suite
    itself.  Default value is composed in following way:

        https://<DOMAIN>/<NS>

    where `<DOMAIN>` and `<NS>` are contents of files *domain* and *ns* as above,
    respectively.

  [rfc3986path]: https://tools.ietf.org/html/rfc3986#section-3.3



Test
----

By default, the format of the test is a source-able Bash code using
*shellfu-bash-jat* library.  A trivial example of *test* code could
be:

    jat__pstartt "example test phase"
        jat__cmd -h "true is still true" \
            true
        jat__cmp -h "five is still greater than three" \
            5 gt 3
        jat__cmp -h "foo still contains oo" \
            foo re oo
    jat__pend


Meta-data and test properties
-----------------------------

Properties of a test are inferred from directory structure and complemented
by test meta-data.  Following test properties are defined:


### Test id ###

Test id is determined from available data and has format of:

    jats://<DOMAIN>/<NS>/<SUT_PNAME>/<PATH>

where:

 *  `jats://` is a constant pseudo-protocol string,

 *  `<DOMAIN>`, `<NS>`, and `<SUT_PNAME>` are contents of respective
    files from *.jats* meta-data directory,

 *  and `<PATH>` is path of test directory relative to suite root directory.

For example, following tree:

    foosuite
      +-- .jats
      |       +-- domain        # contaning "my.example.org"
      |       +-- ns            # contaning "myns"
      |       +-- sut_pname     # contaning "mysut"
      |       +-- version
      +-- foo
      |     +-- test
      +-- bar
           +-- baz-simple
           |     +-- test
           +-- baz-advanced
                 +-- test

contains following tests:

    jats://my.example.org/myns/mysut/foo
    jats://my.example.org/myns/mysut/bar/baz-simple
    jats://my.example.org/myns/mysut/bar/baz-advanced