|
@@ -24,7 +24,7 @@ The harness, though, assumes that:
|
24
|
24
|
* basename of this directory becomes the name of the test,
|
25
|
25
|
|
26
|
26
|
* and return code from running the executable is reported
|
27
|
|
- as result of the test, according to "Exit status" chapter below.
|
|
27
|
+ as result of the test, according to "Exit status" section below.
|
28
|
28
|
|
29
|
29
|
|
30
|
30
|
Naming
|
|
@@ -96,7 +96,7 @@ Framework
|
96
|
96
|
|
97
|
97
|
This part is not intended to be used in tests, but rather contains
|
98
|
98
|
functions that help govern test discovery, preparation and execution as
|
99
|
|
-is described in previous chapters. Feel free to poke around, of course.
|
|
99
|
+is described in previous sections. Feel free to poke around, of course.
|
100
|
100
|
|
101
|
101
|
|
102
|
102
|
### subtest.sh ###
|
|
@@ -125,6 +125,7 @@ The minimal *TF_RUN* with two subtests could look like this:
|
125
|
125
|
case $1 in
|
126
|
126
|
test1) myprog foo ;;
|
127
|
127
|
test2) myprog bar ;;
|
|
128
|
+ test3) myprog baz ;;
|
128
|
129
|
esac
|
129
|
130
|
}
|
130
|
131
|
|
|
@@ -257,7 +258,7 @@ As you may have noticed, there are two ways how to skip a test:
|
257
|
258
|
return prematurely with `TF_ES_BAILOUT`, or suppress enumeration in
|
258
|
259
|
`tf_enum_subtests`. The problem is that the latter does not do anything
|
259
|
260
|
to inform upper in the stack that a test has been skipped, which seems to
|
260
|
|
-break the principle described in the previous chapters.
|
|
261
|
+break the principle described in previous sections.
|
261
|
262
|
|
262
|
263
|
Don't confuse these mechanisms, though. Each is supposed to be used
|
263
|
264
|
for distinct purpose. Compare: by using the `tf_enum_subtests` you are
|
|
@@ -271,19 +272,47 @@ A few common cases if that helps you:
|
271
|
272
|
carried out (e.g. an external resource is not available, or
|
272
|
273
|
something outside the SUT is broken), use `TF_ES_BAILOUT`.
|
273
|
274
|
|
274
|
|
- * If you want to disable the test because for some long-term condition,
|
275
|
|
- e.g. a known bug outside SUT but preventing execution of the test
|
276
|
|
- is not fixed, use `tf_enum_subtests`.
|
277
|
|
-
|
278
|
|
- * If you want to filter out some sub-tests to only for some platforms,
|
279
|
|
- e.g. 64-bit architecture, (IOW, you can safely check that a
|
280
|
|
- sub-test would be totally pointless if run on this box), use
|
281
|
|
- `tf_enum_subtests`.
|
|
275
|
+ tf_enum_subtests() {
|
|
276
|
+ echo test1
|
|
277
|
+ echo test2
|
|
278
|
+ echo test3
|
|
279
|
+ }
|
|
280
|
+
|
|
281
|
+ tf_do_subtest() {
|
|
282
|
+ case $1 in
|
|
283
|
+ test1) do_stuff ;;
|
|
284
|
+ test2) do_other_stuff ;;
|
|
285
|
+ test3) curl -s http://www.example.com/ >file \
|
|
286
|
+ || return $TF_ES_BAILOUT
|
|
287
|
+ do_stuff_with file ;;
|
|
288
|
+ esac
|
|
289
|
+ }
|
|
290
|
+
|
|
291
|
+ * If you want to filter out some sub-tests for some platforms, e.g. a
|
|
292
|
+ test for only 64-bit architectures, or a test only for Mac OS (IOW,
|
|
293
|
+ you can safely say that running this sub-test would be totally
|
|
294
|
+ pointless on this box), use `tf_enum_subtests`--just omit this test
|
|
295
|
+ from enumeration.
|
|
296
|
+
|
|
297
|
+ tf_enum_subtests() {
|
|
298
|
+ echo test1
|
|
299
|
+ echo test2
|
|
300
|
+ if this_is_macos_x; then
|
|
301
|
+ echo test3
|
|
302
|
+ fi
|
|
303
|
+ }
|
282
|
304
|
|
283
|
305
|
* If you want to disable (comment out test) that you might not have
|
284
|
306
|
implemented yet or is broken (and for some reason you still want
|
285
|
|
- it to haunt the test code), use `tf_enum_subtests` and properly
|
286
|
|
- comment the reasons in code.
|
|
307
|
+ it to haunt the test code) or something else outside SUT is broken
|
|
308
|
+ and prevents you from running the test, use `tf_enum_subtests` and
|
|
309
|
+ properly comment the reasons in code.
|
|
310
|
+
|
|
311
|
+ tf_enum_subtests() {
|
|
312
|
+ echo test1
|
|
313
|
+ echo test2
|
|
314
|
+ # echo test3 #FIXME: implement after bz1234
|
|
315
|
+ }
|
287
|
316
|
|
288
|
317
|
* If in doubt, use `TF_ES_BAILOUT`.
|
289
|
318
|
|
|
@@ -313,7 +342,7 @@ important. Follow me as I try to explain:
|
313
|
342
|
by the script, albeit exiting "in a hurry"--without proper clean up.
|
314
|
343
|
|
315
|
344
|
Unfortunately there will be cases like above but with the error code less
|
316
|
|
-than four. Example is a bash script syntax error, which returns 2, or
|
|
345
|
+than four. Example is a Bash script syntax error, which returns 2, or
|
317
|
346
|
Python exception which returns 1. Yes, in such cases the information
|
318
|
347
|
conveyed by the exit status is wrong and you should do everything to
|
319
|
348
|
avoid it.
|