jats-demo/README.md

demo test suite for JATS

This is an independent test suite for fictional compoment 'demo'. The suite is intended to be reference example for JATS - Just A Test System.

JATS defines test suite format, which is a directory tree consisting of individual tests. These individual tests can in principle be implemented in any language, but for now shellfu-bash-jat is the only available library.

Dependencies

Direct dependencies are:

• shellfu-bash-jat - the main JAT library,
• shellfu-bash-xcase - test library to add data-driven testing support,
• jattool - harness and toolkit,
• jats-demo - this test suite

All of the above have their own dependencies, so if you want to install everything manually, follow instructions on their pages.

Installation

Most tests should work without installation. That is, just cloning this repo should be enough. OTOH, in production environment--if you want to ensure integrity of the tests and eg. have them be able to report their own version, instaling the suite is recommended.

...manually

Manual installation consists of:

make
sudo make install

...via yum on Fedora and similar

For Fedora-like distros, this suite and its dependencies are available in COPR projects netvor/jats and netvor/shellfu.

yum install yum-plugin-copr
yum copr enable netvor/shellfu
yum copr enable netvor/jats
yum install jats-demo jattool

Note that older systems like RHEL-6 don't have the COPR plugin, so you will need to add repositories manually. There's a script in GitHb Gist to help you with this step on CentOS-6 & RHEL-6.

Also note that in order to minimize dependencies on testing machine, install just jattool-minimal package, which contains only the bare minimum needed for running tests.

... on Debian

Sorry, but since author of the tools does not know how to create at least remotely proper Debian repository, .deb packages are available on-demand only. See Contact section in main README.

Running tests

To show list of installed tests:

jattool tfind /usr/share/jats

Each JAT test is a directory, so above should give you list of directories: you can then run any of them using jattool runtest, eg.:

jattool runtest /usr/share/jats/suite/demo/foo

should give you something like this:

test.INFO: id: jats://my.example.org/foodept/barpkg//foo
test.INFO: version: 0.0.1

session.START

phase.START 'example test phase'
  test.INFO: CMD: true
  assert.PASS: true is still true
  assert.PASS: five is still greater than three
  assert.PASS: foo still ends with oo
phase.END.PASS 'example test phase'

session.FINALIZE.PASS

Exploring (and hacking on) tests

I won't go into details; I'll tell you just enough to get you started, but be aware that (mostly) everything is (mostly) documented and primary ways to get help (actually more like reference, but helps) using sfdoc tool, which should be on your workstation by now.

Tests normally only consist of one important file, named ... you guessed it... test. (There's also main.fmf, but that only contains meta-data, which is not even really used these days.)

$ tree src/api/binary_req
src/api/binary_req
├── main.fmf
└── test

So open that file: it's a Bash script with extra "enhancements": you'll see things like:

shellfu import jat
shellfu import preupg
shellfu import ...
shellfu import xcase

Shellfu platform brings modularity to Bash, so the names above are names of modules. jat is a module name. And you can get reference on the module

sfdoc jat

Since this is Bash, modules contain just functions and variables. And since Bash has no namespaces, an important convention for (almost) all modules is in place:

• any functions defined by module foo must be named by foo__ prefix,
• any variables defined by module foo must be named by FOO__ prefix.

Eg. an initializing function from module bar would have to be called bar__init().

Creating new tests

NOTE: At the time of this writing, the test creation kit is not finished, so if you really need new test, you can just copy another one.

In order to avoid duplication of work and unnecessary frustration, take following steps:

1. Before you start, contact maintainer and inform them that you are planning to write a test. You can do this by filing an issue in Github repo or by contacting the maintainer directly.

2. Read Shellfu coding style; tests need to follow this.

3. Be aware that the JATS project is in early stage (v0.0.*), and some (most?) pieces of coding style and conventions are not yet written down. A more consistent documentation is expected to arrive before v0.1.0.