jattool

JATTOOL

JATS--Just A Test System is a test system that is intended to be extensible, compatible across the most common GNU/Linux distros and most importantly, oriented towards Quality Assurance needs: simple API, cleaner focus on System Under Test, ability to write robust tests on all levels (from Unit to System to Integration), usable and extensible result format and built-in support for things like granular test and test case identification and specification reference management.

This repository contains early version of jattool, which is a toolset designed to work with JATS tests: suite management and packaging, test development, test planning, test running (harness) and result management.

Project Status and Versioning

• Development stage of this tool and most of its libraries is very early, that is, most parts of intended functionality are missing, buggy and/or subject to change.

• Branch last always contains the last released version.

• See Notes section for details.

Installation

Requirements

Apart from core things like Bash and coreutils, and minus things I'm forgetting right now, jattool uses:

• Shellfu for internal function management and some of its core libraries,

• Saturnin Shellfu library for support of configuration, versioning and meta-command structure,

• jat Shellfu library for wrapping and running tests (currently only Shellfu/Bash tests are supported,

• python2-jinja2 for exporting test results to other formats such as HTML.

Installation -- Fedora and similar

Fedora and CentOS builds are available in Fedora COPRs amahdal/shellfu and amahdal/jats.

On Fedora, following should work:

# dnf copr enable amahdal/shellfu
# dnf copr enable amahdal/jats
# dnf install jattool

Older distros like CentOS 6 and 7, don't have dnf copr or yum copr plugin, so you'll have to pick correct .repo file from COPR site and download it under /etc/yum.repos.d. (You can use this convenience script, though.)

Note that on testing machines, where you only want to run test (but not eg. export to HTML), you only need to install jattool-minimal:

# yum install jattool-minimal

This will help you avoid excess dependencies. Test suites should also normally depend only on jattool-minimal.

Installation -- Debian and similar

Debian builds are available per-request; there's no repository, though, and auhor of jattool is not aware of easy (and reasonably proper) way to create one.

Installation -- Manually

To install manually, clone this repository and use "old-school" make commands:

git clone https://github.com/AloisMahdal/jattool
make
sudo make install

Note that you will need to do this also for dependencies; all Shellfu dependencies use the same method (make && sudo make install), for the rest, use appropriate method.

Usage

Synopsis:

jattool [common-options] SUBCOMMAND [subcommand-options]

Common options for jattool are --version, -d for debugging and -v for verbosity. Note that these are options are for jattool meta-command only, ie. jattool -d foo and jattool foo -d are completely different things!

Here we'll just briefly introduce the available subcommands:

• jattool sfind, jattool tfind - discover test suites and individual tests.

• jattool qrun - quick&dirty in-place test run.

• jattool runtest - proper test run, ie. copy test to temporary directory and run it with identification data.

• jattool export - export test report ftom YAML log to other formats such as HTML.

Please refer to jattool --help for general help and or jattool SUBCOMMAND --help (some subcommands).

Running tests

You first need to obtain some tests. At this moment, jattool does not provide necessary tools to create tests, however, it's recommended to start with demo test suite called jats-demo. This is available in form of builds; in same repositories as jattool itself:

sudo dnf install jats-demo

Of course you can just clone it:

git clone https://github.com/AloisMahdal/jats-demo
jattool tfind jats-demo
jattool runtest jats-demo/src/foo

Follow jats-demo README for details about how typical JATS test suite is organized.

Notes

On versioning

We're committed to following SemVer 2.0 scheme. However, note that for early stages when Major version is zero, SemVer allows for custom rules. We fill this "gap" with SemVer extension, which (at this stage) means:

• x=0, y=0, only z is changing; ie. version is v0.0.*.

• Breaking API changes are allowed between any released versions.

Once the project reaches 0.1.0, breaking API changes will only be allowed on the Y level (e.g. 0.1.5 -> 0.2.0 may not be safe) but only safe, additive ones are allowed on Z level (e.g. 0.1.5 -> 0.1.6 should be safe).

Contacts

For filing bug requests, use the Issue tracker on project's Github page, or ping the auhor directly:

Alois Mahdal <amahdal+jats@redhat.com>
https://netvor.info/