|
@@ -0,0 +1,169 @@
|
|
1
|
+demo test suite for JATS
|
|
2
|
+========================
|
|
3
|
+
|
|
4
|
+This is an independent test suite for fictional compoment 'demo'. The suite
|
|
5
|
+is intended to be reference example for JATS - Just A Test System.
|
|
6
|
+
|
|
7
|
+JATS defines test suite format, which is a directory tree consisting of
|
|
8
|
+individual tests. These individual tests can in principle be implemented
|
|
9
|
+in any language, but for now *shellfu-bash-jat* is the only available
|
|
10
|
+library.
|
|
11
|
+
|
|
12
|
+
|
|
13
|
+Dependencies
|
|
14
|
+------------
|
|
15
|
+
|
|
16
|
+Direct dependencies are:
|
|
17
|
+
|
|
18
|
+ * [shellfu-bash-jat][sbj] - the main JAT library,
|
|
19
|
+ * [shellfu-bash-xcase][sbx] - test library to add data-driven testing support,
|
|
20
|
+ * [jattool][jt] - harness and toolkit,
|
|
21
|
+ * [jats-demo][jd] - this test suite
|
|
22
|
+
|
|
23
|
+All of the above have their own dependencies, so if you want to install everything
|
|
24
|
+manually, follow instructions on their pages.
|
|
25
|
+
|
|
26
|
+ [sbj]: https://github.com/AloisMahdal/shellfu-bash-jat
|
|
27
|
+ [sbx]: https://github.com/AloisMahdal/shellfu-bash-xcase
|
|
28
|
+ [jt]: https://github.com/AloisMahdal/jattool
|
|
29
|
+ [jd]: https://github.com/AloisMahdal/jats-demo
|
|
30
|
+
|
|
31
|
+
|
|
32
|
+Installation
|
|
33
|
+------------
|
|
34
|
+
|
|
35
|
+Most tests should work without installation. That is, just cloning this
|
|
36
|
+repo should be enough. OTOH, in production environment--if you want to
|
|
37
|
+ensure integrity of the tests and eg. have them be able to report their
|
|
38
|
+own version, instaling the suite is recommended.
|
|
39
|
+
|
|
40
|
+### ...manually
|
|
41
|
+
|
|
42
|
+Manual installation consists of:
|
|
43
|
+
|
|
44
|
+ make
|
|
45
|
+ sudo make install
|
|
46
|
+
|
|
47
|
+
|
|
48
|
+### ...via yum on Fedora and similar
|
|
49
|
+
|
|
50
|
+For Fedora-like distros, this suite and its dependencies are available
|
|
51
|
+in COPR projects [amahdal/jats][aj] and [amahdal/shellfu][as].
|
|
52
|
+
|
|
53
|
+ [aj]: https://copr.fedorainfracloud.org/coprs/amahdal/jats/
|
|
54
|
+ [as]: https://copr.fedorainfracloud.org/coprs/amahdal/shellfu/
|
|
55
|
+
|
|
56
|
+ yum install yum-plugin-copr
|
|
57
|
+ yum copr enable amahdal/shellfu
|
|
58
|
+ yum copr enable amahdal/jats
|
|
59
|
+ yum install jats-demo jattool
|
|
60
|
+
|
|
61
|
+Note that older systems like RHEL-6 don't have the COPR plugin, so
|
|
62
|
+you will need to add repositories manually. There's a [script][jel6]
|
|
63
|
+in GitHb Gist to help you with this step on CentOS-6 & RHEL-6.
|
|
64
|
+
|
|
65
|
+ [jel6]: https://gist.github.com/AloisMahdal/89e422b4af54f5f65b87f70f7e339594
|
|
66
|
+
|
|
67
|
+Also note that in order to minimize dependencies on testing machine,
|
|
68
|
+install just *jattool-minimal* package, which contains only the bare
|
|
69
|
+minimum needed for running tests.
|
|
70
|
+
|
|
71
|
+
|
|
72
|
+### ... on Debian
|
|
73
|
+
|
|
74
|
+Sorry, but since author of the tools does not know how to create at least
|
|
75
|
+remotely proper Debian repository, .deb packages are available on-demand
|
|
76
|
+only. See Contact section in main README.
|
|
77
|
+
|
|
78
|
+
|
|
79
|
+Running tests
|
|
80
|
+-------------
|
|
81
|
+
|
|
82
|
+To show list of installed tests:
|
|
83
|
+
|
|
84
|
+ jattool tfind /usr/share/jats
|
|
85
|
+
|
|
86
|
+Each JAT test is a directory, so above should give you list of directories: you
|
|
87
|
+can then run any of them using `jattool runtest`, eg.:
|
|
88
|
+
|
|
89
|
+ jattool runtest /usr/share/jats/suite/demo/foo
|
|
90
|
+
|
|
91
|
+should give you something like this:
|
|
92
|
+
|
|
93
|
+ test.INFO: id: jats://my.example.org/foodept/barpkg//foo
|
|
94
|
+ test.INFO: version: 0.0.1
|
|
95
|
+
|
|
96
|
+ session.START
|
|
97
|
+
|
|
98
|
+ phase.START 'example test phase'
|
|
99
|
+ test.INFO: CMD: true
|
|
100
|
+ assert.PASS: true is still true
|
|
101
|
+ assert.PASS: five is still greater than three
|
|
102
|
+ assert.PASS: foo still ends with oo
|
|
103
|
+ phase.END.PASS 'example test phase'
|
|
104
|
+
|
|
105
|
+ session.FINALIZE.PASS
|
|
106
|
+
|
|
107
|
+
|
|
108
|
+Exploring (and hacking on) tests
|
|
109
|
+--------------------------------
|
|
110
|
+
|
|
111
|
+I won't go into details; I'll tell you just enough to get you started,
|
|
112
|
+but be aware that **(mostly) everything is (mostly) documented** and
|
|
113
|
+primary ways to get help (actually more like reference, but helps)
|
|
114
|
+using *sfdoc* tool, which should be on your workstation by now.
|
|
115
|
+
|
|
116
|
+Tests normally only consist of one important file, named ... you
|
|
117
|
+guessed it... *test*. (There's also *main.fmf*, but that only contains
|
|
118
|
+meta-data, which is not even really used these days.)
|
|
119
|
+
|
|
120
|
+ $ tree src/api/binary_req
|
|
121
|
+ src/api/binary_req
|
|
122
|
+ ├── main.fmf
|
|
123
|
+ └── test
|
|
124
|
+
|
|
125
|
+So open that file: it's a Bash script with extra "enhancements": you'll see things
|
|
126
|
+like:
|
|
127
|
+
|
|
128
|
+ shellfu import jat
|
|
129
|
+ shellfu import preupg
|
|
130
|
+ shellfu import ...
|
|
131
|
+ shellfu import xcase
|
|
132
|
+
|
|
133
|
+Shellfu platform brings modularity to Bash, so the names above are names
|
|
134
|
+of modules. `jat` is a module name. And you can get reference on the module
|
|
135
|
+
|
|
136
|
+ sfdoc jat
|
|
137
|
+
|
|
138
|
+Since this is Bash, modules contain just functions and variables. And since
|
|
139
|
+Bash has no namespaces, an important convention for (almost) all modules
|
|
140
|
+is in place:
|
|
141
|
+
|
|
142
|
+ * any functions defined by module `foo` must be named by `foo__` prefix,
|
|
143
|
+ * any variables defined by module `foo` must be named by `FOO__` prefix.
|
|
144
|
+
|
|
145
|
+Eg. an initializing function from module *bar* would have to be called
|
|
146
|
+`bar__init()`.
|
|
147
|
+
|
|
148
|
+
|
|
149
|
+Creating new tests
|
|
150
|
+------------------
|
|
151
|
+
|
|
152
|
+**NOTE:** At the time of this writing, the test creation kit is not finished,
|
|
153
|
+so if you really need new test, you can just copy another one.
|
|
154
|
+
|
|
155
|
+In order to avoid duplication of work and unnecessary frustration,
|
|
156
|
+take following steps:
|
|
157
|
+
|
|
158
|
+ 1. Before you start, **contact maintainer** and inform them that you
|
|
159
|
+ are planning to write a test. You can do this by filing an issue
|
|
160
|
+ in Github repo or by contacting the maintainer directly.
|
|
161
|
+
|
|
162
|
+ 2. Read [Shellfu coding style][sfcs]; tests need to follow this.
|
|
163
|
+
|
|
164
|
+ 3. Be aware that the JATS project is in early stage (`v0.0.*`), and
|
|
165
|
+ some (most?) pieces of coding style and conventions are not yet
|
|
166
|
+ written down. A more consistent documentation is expected to arrive
|
|
167
|
+ before `v0.1.0`.
|
|
168
|
+
|
|
169
|
+ [sfcs]: https://github.com/AloisMahdal/shellfu/blob/last/notes/style.md
|