|
@@ -1,4 +1,177 @@
|
1
|
1
|
JATTOOL
|
2
|
2
|
=======
|
3
|
3
|
|
4
|
|
-Shell-level testing framework
|
|
4
|
+JATS--Just A Test System is a test system that is intended to be extensible,
|
|
5
|
+compatible across the most common GNU/Linux distros and most importantly, oriented
|
|
6
|
+towards Quality Assurance needs: simple API, cleaner focus on System Under Test,
|
|
7
|
+ability to write robust tests on all levels (from Unit to System to Integration),
|
|
8
|
+usable and extensible result format and built-in support for things like granular
|
|
9
|
+test and test case identification and specification reference management.
|
|
10
|
+
|
|
11
|
+This repository contains early version of *jattool*, which is a toolset designed
|
|
12
|
+to work with JATS tests: suite management and packaging, test development, test
|
|
13
|
+planning, test running (harness) and result management.
|
|
14
|
+
|
|
15
|
+
|
|
16
|
+Project Status and Versioning
|
|
17
|
+-----------------------------
|
|
18
|
+
|
|
19
|
+ * Development stage of this tool and most of its libraries is **very
|
|
20
|
+ early**, that is, most parts of intended functionality are missing,
|
|
21
|
+ buggy and/or subject to change.
|
|
22
|
+
|
|
23
|
+ * Branch *last* always contains the last released version.
|
|
24
|
+
|
|
25
|
+ * See Notes section for details.
|
|
26
|
+
|
|
27
|
+
|
|
28
|
+Installation
|
|
29
|
+------------
|
|
30
|
+
|
|
31
|
+### Requirements ###
|
|
32
|
+
|
|
33
|
+Apart from core things like Bash and coreutils, and minus things I'm
|
|
34
|
+forgetting right now, *jattool* uses:
|
|
35
|
+
|
|
36
|
+ * [Shellfu][sf] for internal function management and some of its
|
|
37
|
+ core libraries,
|
|
38
|
+
|
|
39
|
+ * [Saturnin][st] Shellfu library for support of configuration,
|
|
40
|
+ versioning and meta-command structure,
|
|
41
|
+
|
|
42
|
+ * [jat][jat] Shellfu library for wrapping and running tests (currently
|
|
43
|
+ only Shellfu/Bash tests are supported,
|
|
44
|
+
|
|
45
|
+ * [python2-jinja2][ji2] for exporting test results to other formats such
|
|
46
|
+ as HTML.
|
|
47
|
+
|
|
48
|
+ [sf]: https://github.com/AloisMahdal/shellfu
|
|
49
|
+ [st]: https://github.com/AloisMahdal/shellfu-bash-saturnin
|
|
50
|
+ [jat]: https://github.com/AloisMahdal/shellfu-bash-jat
|
|
51
|
+ [ji2]: http://jinja.pocoo.org/
|
|
52
|
+
|
|
53
|
+
|
|
54
|
+### Installation -- Fedora and similar ###
|
|
55
|
+
|
|
56
|
+Fedora and CentOS builds are available in Fedora COPRs [amahdal/shellfu][cas]
|
|
57
|
+and [amahdal/jats][caj].
|
|
58
|
+
|
|
59
|
+ [cas]: https://copr.fedorainfracloud.org/coprs/amahdal/shellfu/
|
|
60
|
+ [caj]: https://copr.fedorainfracloud.org/coprs/amahdal/jats/
|
|
61
|
+
|
|
62
|
+On Fedora, following should work:
|
|
63
|
+
|
|
64
|
+ # dnf copr enable amahdal/shellfu
|
|
65
|
+ # dnf copr enable amahdal/jats
|
|
66
|
+ # dnf install jattool
|
|
67
|
+
|
|
68
|
+Older distros like CentOS 6 and 7, don't have `dnf copr` or `yum copr` plugin,
|
|
69
|
+so you'll have to pick correct *.repo* file from COPR site and download it
|
|
70
|
+under */etc/yum.repos.d*.
|
|
71
|
+
|
|
72
|
+Also, older distros also necessarily have all dependencies available, so if you
|
|
73
|
+only want to *run* test (but not eg. export to HTML), you only need to install
|
|
74
|
+*jattool-minimal*:
|
|
75
|
+
|
|
76
|
+ # yum install jattool-minimal
|
|
77
|
+
|
|
78
|
+
|
|
79
|
+### Installation -- Debian and similar ###
|
|
80
|
+
|
|
81
|
+Debian builds are available per-request; there's no repository, though, and
|
|
82
|
+auhor of *jattool* is not aware of easy (and reasonably proper) way to create
|
|
83
|
+one.
|
|
84
|
+
|
|
85
|
+
|
|
86
|
+### Installation -- Manually ###
|
|
87
|
+
|
|
88
|
+To install manually, clone this repository and use "old-school" make commands:
|
|
89
|
+
|
|
90
|
+ git clone https://github.com/AloisMahdal/jattool
|
|
91
|
+ make
|
|
92
|
+ sudo make install
|
|
93
|
+
|
|
94
|
+Note that you will need to do this also for dependencies; all Shellfu dependencies
|
|
95
|
+use the same method (`make && sudo make install`), for the rest, use appropriate
|
|
96
|
+method.
|
|
97
|
+
|
|
98
|
+
|
|
99
|
+Usage
|
|
100
|
+-----
|
|
101
|
+
|
|
102
|
+Synopsis:
|
|
103
|
+
|
|
104
|
+ jattool [common-options] SUBCOMMAND [subcommand-options]
|
|
105
|
+
|
|
106
|
+Common options for *jattool* are `--version`, `-d` for debugging and `-v`
|
|
107
|
+for verbosity. Note that these are options are for `jattool` meta-command
|
|
108
|
+only, ie. `jattool -d foo` and `jattool foo -d` are completely different things!
|
|
109
|
+
|
|
110
|
+Here we'll just briefly introduce the available subcommands:
|
|
111
|
+
|
|
112
|
+ * *jattool sfind*, *jattool tfind* - discover test suites and individual
|
|
113
|
+ tests.
|
|
114
|
+
|
|
115
|
+ * *jattool qrun* - quick&dirty **in-place** test run.
|
|
116
|
+
|
|
117
|
+ * *jattool runtest* - proper test run, ie. copy test to temporary directory
|
|
118
|
+ and run it with identification data.
|
|
119
|
+
|
|
120
|
+ * *jattool export* - export test report ftom YAML log to other formats such
|
|
121
|
+ as HTML.
|
|
122
|
+
|
|
123
|
+Please refer to `jattool --help` for general help and or `jattool
|
|
124
|
+SUBCOMMAND --help` (some subcommands).
|
|
125
|
+
|
|
126
|
+
|
|
127
|
+### Running tests ###
|
|
128
|
+
|
|
129
|
+You first need to obtain some tests. At this moment, *jattool* does
|
|
130
|
+not provide necessary tools to create tests, however, it's recommended
|
|
131
|
+to start with demo test suite called *jats-demo*. This is available in
|
|
132
|
+form of builds; in same repositories as *jattool* itself:
|
|
133
|
+
|
|
134
|
+ sudo dnf install jats-demo
|
|
135
|
+
|
|
136
|
+Of course you can just clone it:
|
|
137
|
+
|
|
138
|
+ git clone https://github.com/AloisMahdal/jats-demo
|
|
139
|
+ jattool tfind jats-demo
|
|
140
|
+ jattool runtest jats-demo/src/foo
|
|
141
|
+
|
|
142
|
+Follow [jats-demo README][jdr] for details about how typical JATS test
|
|
143
|
+suite is organized.
|
|
144
|
+
|
|
145
|
+ [jdr]: https://github.com/AloisMahdal/jats-demo/blob/master/README.md
|
|
146
|
+
|
|
147
|
+
|
|
148
|
+Notes
|
|
149
|
+-----
|
|
150
|
+
|
|
151
|
+### On versioning ###
|
|
152
|
+
|
|
153
|
+We're committed to following [SemVer 2.0][sv] scheme. However, note that for
|
|
154
|
+early stages when Major version is zero, SemVer allows for custom rules.
|
|
155
|
+We fill this "gap" with [SemVer extension][sv00], which (at this stage)
|
|
156
|
+means:
|
|
157
|
+
|
|
158
|
+ * *x=0*, *y=0*, only *z* is changing; ie. version is `v0.0.*`.
|
|
159
|
+
|
|
160
|
+ * Breaking API changes are allowed between any released versions.
|
|
161
|
+
|
|
162
|
+Once the project reaches 0.1.0, breaking API changes will only be allowed
|
|
163
|
+on the Y level (e.g. 0.1.5 -> 0.2.0 may not be safe) but only safe,
|
|
164
|
+additive ones are allowed on Z level (e.g. 0.1.5 -> 0.1.6 should be safe).
|
|
165
|
+
|
|
166
|
+ [sv]: http://semver.org/spec/v2.0.0
|
|
167
|
+ [sv00]: https://github.com/semver/semver/issues/363
|
|
168
|
+
|
|
169
|
+
|
|
170
|
+Contacts
|
|
171
|
+--------
|
|
172
|
+
|
|
173
|
+For filing bug requests, use the Issue tracker on project's Github page,
|
|
174
|
+or ping the auhor directly:
|
|
175
|
+
|
|
176
|
+ Alois Mahdal <amahdal+jats@redhat.com>
|
|
177
|
+ https://netvor.info/
|