demo test suite for JATS
========================

This is an independent test suite for fictional compoment 'demo'.  The suite
is intended to be reference example for JATS - Just A Test System.

JATS defines test suite format, which is a directory tree consisting of
individual tests.  These individual tests can in principle be implemented
in any language, but for now *shellfu-bash-jat* is the only available
library.


Dependencies
------------

Direct dependencies are:

 *  [shellfu-bash-jat][sbj] - the main JAT library,
 *  [shellfu-bash-xcase][sbx] - test library to add data-driven testing support,
 *  [jattool][jt] - harness and toolkit,
 *  [jats-demo][jd] - this test suite

All of the above have their own dependencies, so if you want to install everything
manually, follow instructions on their pages.

  [sbj]: https://github.com/AloisMahdal/shellfu-bash-jat
  [sbx]: https://github.com/AloisMahdal/shellfu-bash-xcase
  [jt]: https://github.com/AloisMahdal/jattool
  [jd]: https://github.com/AloisMahdal/jats-demo


Installation
------------

Most tests should work without installation.   That is, just cloning this
repo should be enough.  OTOH, in production environment--if you want to
ensure integrity of the tests and eg. have them be able to report their
own version, instaling the suite is recommended.

### ...manually

Manual installation consists of:

    make
    sudo make install


### ...via yum on Fedora and similar

For Fedora-like distros, this suite and its dependencies are available
in COPR projects [netvor/jats][nj] and [netvor/shellfu][ns].

  [nj]: https://copr.fedorainfracloud.org/coprs/netvor/jats/
  [ns]: https://copr.fedorainfracloud.org/coprs/netvor/shellfu/

    yum install yum-plugin-copr
    yum copr enable netvor/shellfu
    yum copr enable netvor/jats
    yum install jats-demo jattool

Note that older systems like RHEL-6 don't have the COPR plugin, so
you will need to add repositories manually.  There's a [script][jel6]
in GitHb Gist to help you with this step on CentOS-6 & RHEL-6.

  [jel6]: https://gist.github.com/AloisMahdal/89e422b4af54f5f65b87f70f7e339594

Also note that in order to minimize dependencies on testing machine,
install just *jattool-minimal* package, which contains only the bare
minimum needed for running tests.


### ... on Debian

Sorry, but since author of the tools does not know how to create at least
remotely proper Debian repository, .deb packages are available on-demand
only.  See Contact section in main README.


Running tests
-------------

To show list of installed tests:

    jattool tfind /usr/share/jats

Each JAT test is a directory, so above should give you list of directories: you
can then run any of them using `jattool runtest`, eg.:

    jattool runtest /usr/share/jats/suite/demo/foo

should give you something like this:

    test.INFO: id: jats://my.example.org/foodept/barpkg//foo
    test.INFO: version: 0.0.1

    session.START

    phase.START 'example test phase'
      test.INFO: CMD: true
      assert.PASS: true is still true
      assert.PASS: five is still greater than three
      assert.PASS: foo still ends with oo
    phase.END.PASS 'example test phase'

    session.FINALIZE.PASS


Exploring (and hacking on) tests
--------------------------------

I won't go into details; I'll tell you just enough to get you started,
but be aware that **(mostly) everything is (mostly) documented** and
primary ways to get help (actually more like reference, but helps)
using *sfdoc* tool, which should be on your workstation by now.

Tests normally only consist of one important file, named ... you
guessed it...  *test*.  (There's also *main.fmf*, but that only contains
meta-data, which is not even really used these days.)

    $ tree src/api/binary_req
    src/api/binary_req
    ├── main.fmf
    └── test

So open that file:  it's a Bash script with extra "enhancements":  you'll see things
like:

    shellfu import jat
    shellfu import preupg
    shellfu import ...
    shellfu import xcase

Shellfu platform brings modularity to Bash, so the names above are names
of modules.  `jat` is a module name.  And you can get reference on the module

    sfdoc jat

Since this is Bash, modules contain just functions and variables.  And since
Bash has no namespaces, an important convention for (almost) all modules
is in place:

 *  any functions defined by module `foo` must be named by `foo__` prefix,
 *  any variables defined by module `foo` must be named by `FOO__` prefix.

Eg. an initializing function from module *bar* would have to be called
`bar__init()`.


Creating new tests
------------------

**NOTE:** At the time of this writing, the test creation kit is not finished,
so if you really need new test, you can just copy another one.

In order to avoid duplication of work and unnecessary frustration,
take following steps:

 1. Before you start, **contact maintainer** and inform them that you
    are planning to write a test.  You can do this by filing an issue
    in Github repo or by contacting the maintainer directly.

 2. Read [Shellfu coding style][sfcs]; tests need to follow this.

 3. Be aware that the JATS project is in early stage (`v0.0.*`), and
    some (most?) pieces of coding style and conventions are not yet
    written down.  A more consistent documentation is expected to arrive
    before `v0.1.0`.

  [sfcs]: https://github.com/AloisMahdal/shellfu/blob/last/notes/style.md