Browse Source

Initial commit

Already v0.1.0 since it's been used for ages as part of BMO (my other
project, not released yet.)
Alois Mahdal 6 years ago
commit
875ecf88eb

+ 3
- 0
.gitignore View File

@@ -0,0 +1,3 @@
1
+.mkit/autoclean
2
+shellfu-bash-uripecker-*.tar.gz
3
+artifacts

+ 503
- 0
LICENSE.md View File

@@ -0,0 +1,503 @@
1
+### GNU LESSER GENERAL PUBLIC LICENSE
2
+
3
+Version 2.1, February 1999
4
+
5
+    Copyright (C) 1991, 1999 Free Software Foundation, Inc.
6
+    51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
7
+    
8
+    Everyone is permitted to copy and distribute verbatim copies
9
+    of this license document, but changing it is not allowed.
10
+
11
+    [This is the first released version of the Lesser GPL.  It also counts
12
+     as the successor of the GNU Library Public License, version 2, hence
13
+     the version number 2.1.]
14
+
15
+### Preamble
16
+
17
+The licenses for most software are designed to take away your freedom
18
+to share and change it. By contrast, the GNU General Public Licenses
19
+are intended to guarantee your freedom to share and change free
20
+software--to make sure the software is free for all its users.
21
+
22
+This license, the Lesser General Public License, applies to some
23
+specially designated software packages--typically libraries--of the
24
+Free Software Foundation and other authors who decide to use it. You
25
+can use it too, but we suggest you first think carefully about whether
26
+this license or the ordinary General Public License is the better
27
+strategy to use in any particular case, based on the explanations
28
+below.
29
+
30
+When we speak of free software, we are referring to freedom of use,
31
+not price. Our General Public Licenses are designed to make sure that
32
+you have the freedom to distribute copies of free software (and charge
33
+for this service if you wish); that you receive source code or can get
34
+it if you want it; that you can change the software and use pieces of
35
+it in new free programs; and that you are informed that you can do
36
+these things.
37
+
38
+To protect your rights, we need to make restrictions that forbid
39
+distributors to deny you these rights or to ask you to surrender these
40
+rights. These restrictions translate to certain responsibilities for
41
+you if you distribute copies of the library or if you modify it.
42
+
43
+For example, if you distribute copies of the library, whether gratis
44
+or for a fee, you must give the recipients all the rights that we gave
45
+you. You must make sure that they, too, receive or can get the source
46
+code. If you link other code with the library, you must provide
47
+complete object files to the recipients, so that they can relink them
48
+with the library after making changes to the library and recompiling
49
+it. And you must show them these terms so they know their rights.
50
+
51
+We protect your rights with a two-step method: (1) we copyright the
52
+library, and (2) we offer you this license, which gives you legal
53
+permission to copy, distribute and/or modify the library.
54
+
55
+To protect each distributor, we want to make it very clear that there
56
+is no warranty for the free library. Also, if the library is modified
57
+by someone else and passed on, the recipients should know that what
58
+they have is not the original version, so that the original author's
59
+reputation will not be affected by problems that might be introduced
60
+by others.
61
+
62
+Finally, software patents pose a constant threat to the existence of
63
+any free program. We wish to make sure that a company cannot
64
+effectively restrict the users of a free program by obtaining a
65
+restrictive license from a patent holder. Therefore, we insist that
66
+any patent license obtained for a version of the library must be
67
+consistent with the full freedom of use specified in this license.
68
+
69
+Most GNU software, including some libraries, is covered by the
70
+ordinary GNU General Public License. This license, the GNU Lesser
71
+General Public License, applies to certain designated libraries, and
72
+is quite different from the ordinary General Public License. We use
73
+this license for certain libraries in order to permit linking those
74
+libraries into non-free programs.
75
+
76
+When a program is linked with a library, whether statically or using a
77
+shared library, the combination of the two is legally speaking a
78
+combined work, a derivative of the original library. The ordinary
79
+General Public License therefore permits such linking only if the
80
+entire combination fits its criteria of freedom. The Lesser General
81
+Public License permits more lax criteria for linking other code with
82
+the library.
83
+
84
+We call this license the "Lesser" General Public License because it
85
+does Less to protect the user's freedom than the ordinary General
86
+Public License. It also provides other free software developers Less
87
+of an advantage over competing non-free programs. These disadvantages
88
+are the reason we use the ordinary General Public License for many
89
+libraries. However, the Lesser license provides advantages in certain
90
+special circumstances.
91
+
92
+For example, on rare occasions, there may be a special need to
93
+encourage the widest possible use of a certain library, so that it
94
+becomes a de-facto standard. To achieve this, non-free programs must
95
+be allowed to use the library. A more frequent case is that a free
96
+library does the same job as widely used non-free libraries. In this
97
+case, there is little to gain by limiting the free library to free
98
+software only, so we use the Lesser General Public License.
99
+
100
+In other cases, permission to use a particular library in non-free
101
+programs enables a greater number of people to use a large body of
102
+free software. For example, permission to use the GNU C Library in
103
+non-free programs enables many more people to use the whole GNU
104
+operating system, as well as its variant, the GNU/Linux operating
105
+system.
106
+
107
+Although the Lesser General Public License is Less protective of the
108
+users' freedom, it does ensure that the user of a program that is
109
+linked with the Library has the freedom and the wherewithal to run
110
+that program using a modified version of the Library.
111
+
112
+The precise terms and conditions for copying, distribution and
113
+modification follow. Pay close attention to the difference between a
114
+"work based on the library" and a "work that uses the library". The
115
+former contains code derived from the library, whereas the latter must
116
+be combined with the library in order to run.
117
+
118
+### TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
119
+
120
+**0.** This License Agreement applies to any software library or other
121
+program which contains a notice placed by the copyright holder or
122
+other authorized party saying it may be distributed under the terms of
123
+this Lesser General Public License (also called "this License"). Each
124
+licensee is addressed as "you".
125
+
126
+A "library" means a collection of software functions and/or data
127
+prepared so as to be conveniently linked with application programs
128
+(which use some of those functions and data) to form executables.
129
+
130
+The "Library", below, refers to any such software library or work
131
+which has been distributed under these terms. A "work based on the
132
+Library" means either the Library or any derivative work under
133
+copyright law: that is to say, a work containing the Library or a
134
+portion of it, either verbatim or with modifications and/or translated
135
+straightforwardly into another language. (Hereinafter, translation is
136
+included without limitation in the term "modification".)
137
+
138
+"Source code" for a work means the preferred form of the work for
139
+making modifications to it. For a library, complete source code means
140
+all the source code for all modules it contains, plus any associated
141
+interface definition files, plus the scripts used to control
142
+compilation and installation of the library.
143
+
144
+Activities other than copying, distribution and modification are not
145
+covered by this License; they are outside its scope. The act of
146
+running a program using the Library is not restricted, and output from
147
+such a program is covered only if its contents constitute a work based
148
+on the Library (independent of the use of the Library in a tool for
149
+writing it). Whether that is true depends on what the Library does and
150
+what the program that uses the Library does.
151
+
152
+**1.** You may copy and distribute verbatim copies of the Library's
153
+complete source code as you receive it, in any medium, provided that
154
+you conspicuously and appropriately publish on each copy an
155
+appropriate copyright notice and disclaimer of warranty; keep intact
156
+all the notices that refer to this License and to the absence of any
157
+warranty; and distribute a copy of this License along with the
158
+Library.
159
+
160
+You may charge a fee for the physical act of transferring a copy, and
161
+you may at your option offer warranty protection in exchange for a
162
+fee.
163
+
164
+**2.** You may modify your copy or copies of the Library or any
165
+portion of it, thus forming a work based on the Library, and copy and
166
+distribute such modifications or work under the terms of Section 1
167
+above, provided that you also meet all of these conditions:
168
+
169
+-   **a)** The modified work must itself be a software library.
170
+-   **b)** You must cause the files modified to carry prominent
171
+    notices stating that you changed the files and the date of
172
+    any change.
173
+-   **c)** You must cause the whole of the work to be licensed at no
174
+    charge to all third parties under the terms of this License.
175
+-   **d)** If a facility in the modified Library refers to a function
176
+    or a table of data to be supplied by an application program that
177
+    uses the facility, other than as an argument passed when the
178
+    facility is invoked, then you must make a good faith effort to
179
+    ensure that, in the event an application does not supply such
180
+    function or table, the facility still operates, and performs
181
+    whatever part of its purpose remains meaningful.
182
+
183
+    (For example, a function in a library to compute square roots has
184
+    a purpose that is entirely well-defined independent of
185
+    the application. Therefore, Subsection 2d requires that any
186
+    application-supplied function or table used by this function must
187
+    be optional: if the application does not supply it, the square
188
+    root function must still compute square roots.)
189
+
190
+These requirements apply to the modified work as a whole. If
191
+identifiable sections of that work are not derived from the Library,
192
+and can be reasonably considered independent and separate works in
193
+themselves, then this License, and its terms, do not apply to those
194
+sections when you distribute them as separate works. But when you
195
+distribute the same sections as part of a whole which is a work based
196
+on the Library, the distribution of the whole must be on the terms of
197
+this License, whose permissions for other licensees extend to the
198
+entire whole, and thus to each and every part regardless of who wrote
199
+it.
200
+
201
+Thus, it is not the intent of this section to claim rights or contest
202
+your rights to work written entirely by you; rather, the intent is to
203
+exercise the right to control the distribution of derivative or
204
+collective works based on the Library.
205
+
206
+In addition, mere aggregation of another work not based on the Library
207
+with the Library (or with a work based on the Library) on a volume of
208
+a storage or distribution medium does not bring the other work under
209
+the scope of this License.
210
+
211
+**3.** You may opt to apply the terms of the ordinary GNU General
212
+Public License instead of this License to a given copy of the Library.
213
+To do this, you must alter all the notices that refer to this License,
214
+so that they refer to the ordinary GNU General Public License, version
215
+2, instead of to this License. (If a newer version than version 2 of
216
+the ordinary GNU General Public License has appeared, then you can
217
+specify that version instead if you wish.) Do not make any other
218
+change in these notices.
219
+
220
+Once this change is made in a given copy, it is irreversible for that
221
+copy, so the ordinary GNU General Public License applies to all
222
+subsequent copies and derivative works made from that copy.
223
+
224
+This option is useful when you wish to copy part of the code of the
225
+Library into a program that is not a library.
226
+
227
+**4.** You may copy and distribute the Library (or a portion or
228
+derivative of it, under Section 2) in object code or executable form
229
+under the terms of Sections 1 and 2 above provided that you accompany
230
+it with the complete corresponding machine-readable source code, which
231
+must be distributed under the terms of Sections 1 and 2 above on a
232
+medium customarily used for software interchange.
233
+
234
+If distribution of object code is made by offering access to copy from
235
+a designated place, then offering equivalent access to copy the source
236
+code from the same place satisfies the requirement to distribute the
237
+source code, even though third parties are not compelled to copy the
238
+source along with the object code.
239
+
240
+**5.** A program that contains no derivative of any portion of the
241
+Library, but is designed to work with the Library by being compiled or
242
+linked with it, is called a "work that uses the Library". Such a work,
243
+in isolation, is not a derivative work of the Library, and therefore
244
+falls outside the scope of this License.
245
+
246
+However, linking a "work that uses the Library" with the Library
247
+creates an executable that is a derivative of the Library (because it
248
+contains portions of the Library), rather than a "work that uses the
249
+library". The executable is therefore covered by this License. Section
250
+6 states terms for distribution of such executables.
251
+
252
+When a "work that uses the Library" uses material from a header file
253
+that is part of the Library, the object code for the work may be a
254
+derivative work of the Library even though the source code is not.
255
+Whether this is true is especially significant if the work can be
256
+linked without the Library, or if the work is itself a library. The
257
+threshold for this to be true is not precisely defined by law.
258
+
259
+If such an object file uses only numerical parameters, data structure
260
+layouts and accessors, and small macros and small inline functions
261
+(ten lines or less in length), then the use of the object file is
262
+unrestricted, regardless of whether it is legally a derivative work.
263
+(Executables containing this object code plus portions of the Library
264
+will still fall under Section 6.)
265
+
266
+Otherwise, if the work is a derivative of the Library, you may
267
+distribute the object code for the work under the terms of Section 6.
268
+Any executables containing that work also fall under Section 6,
269
+whether or not they are linked directly with the Library itself.
270
+
271
+**6.** As an exception to the Sections above, you may also combine or
272
+link a "work that uses the Library" with the Library to produce a work
273
+containing portions of the Library, and distribute that work under
274
+terms of your choice, provided that the terms permit modification of
275
+the work for the customer's own use and reverse engineering for
276
+debugging such modifications.
277
+
278
+You must give prominent notice with each copy of the work that the
279
+Library is used in it and that the Library and its use are covered by
280
+this License. You must supply a copy of this License. If the work
281
+during execution displays copyright notices, you must include the
282
+copyright notice for the Library among them, as well as a reference
283
+directing the user to the copy of this License. Also, you must do one
284
+of these things:
285
+
286
+-   **a)** Accompany the work with the complete corresponding
287
+    machine-readable source code for the Library including whatever
288
+    changes were used in the work (which must be distributed under
289
+    Sections 1 and 2 above); and, if the work is an executable linked
290
+    with the Library, with the complete machine-readable "work that
291
+    uses the Library", as object code and/or source code, so that the
292
+    user can modify the Library and then relink to produce a modified
293
+    executable containing the modified Library. (It is understood that
294
+    the user who changes the contents of definitions files in the
295
+    Library will not necessarily be able to recompile the application
296
+    to use the modified definitions.)
297
+-   **b)** Use a suitable shared library mechanism for linking with
298
+    the Library. A suitable mechanism is one that (1) uses at run time
299
+    a copy of the library already present on the user's computer
300
+    system, rather than copying library functions into the executable,
301
+    and (2) will operate properly with a modified version of the
302
+    library, if the user installs one, as long as the modified version
303
+    is interface-compatible with the version that the work was
304
+    made with.
305
+-   **c)** Accompany the work with a written offer, valid for at least
306
+    three years, to give the same user the materials specified in
307
+    Subsection 6a, above, for a charge no more than the cost of
308
+    performing this distribution.
309
+-   **d)** If distribution of the work is made by offering access to
310
+    copy from a designated place, offer equivalent access to copy the
311
+    above specified materials from the same place.
312
+-   **e)** Verify that the user has already received a copy of these
313
+    materials or that you have already sent this user a copy.
314
+
315
+For an executable, the required form of the "work that uses the
316
+Library" must include any data and utility programs needed for
317
+reproducing the executable from it. However, as a special exception,
318
+the materials to be distributed need not include anything that is
319
+normally distributed (in either source or binary form) with the major
320
+components (compiler, kernel, and so on) of the operating system on
321
+which the executable runs, unless that component itself accompanies
322
+the executable.
323
+
324
+It may happen that this requirement contradicts the license
325
+restrictions of other proprietary libraries that do not normally
326
+accompany the operating system. Such a contradiction means you cannot
327
+use both them and the Library together in an executable that you
328
+distribute.
329
+
330
+**7.** You may place library facilities that are a work based on the
331
+Library side-by-side in a single library together with other library
332
+facilities not covered by this License, and distribute such a combined
333
+library, provided that the separate distribution of the work based on
334
+the Library and of the other library facilities is otherwise
335
+permitted, and provided that you do these two things:
336
+
337
+-   **a)** Accompany the combined library with a copy of the same work
338
+    based on the Library, uncombined with any other
339
+    library facilities. This must be distributed under the terms of
340
+    the Sections above.
341
+-   **b)** Give prominent notice with the combined library of the fact
342
+    that part of it is a work based on the Library, and explaining
343
+    where to find the accompanying uncombined form of the same work.
344
+
345
+**8.** You may not copy, modify, sublicense, link with, or distribute
346
+the Library except as expressly provided under this License. Any
347
+attempt otherwise to copy, modify, sublicense, link with, or
348
+distribute the Library is void, and will automatically terminate your
349
+rights under this License. However, parties who have received copies,
350
+or rights, from you under this License will not have their licenses
351
+terminated so long as such parties remain in full compliance.
352
+
353
+**9.** You are not required to accept this License, since you have not
354
+signed it. However, nothing else grants you permission to modify or
355
+distribute the Library or its derivative works. These actions are
356
+prohibited by law if you do not accept this License. Therefore, by
357
+modifying or distributing the Library (or any work based on the
358
+Library), you indicate your acceptance of this License to do so, and
359
+all its terms and conditions for copying, distributing or modifying
360
+the Library or works based on it.
361
+
362
+**10.** Each time you redistribute the Library (or any work based on
363
+the Library), the recipient automatically receives a license from the
364
+original licensor to copy, distribute, link with or modify the Library
365
+subject to these terms and conditions. You may not impose any further
366
+restrictions on the recipients' exercise of the rights granted herein.
367
+You are not responsible for enforcing compliance by third parties with
368
+this License.
369
+
370
+**11.** If, as a consequence of a court judgment or allegation of
371
+patent infringement or for any other reason (not limited to patent
372
+issues), conditions are imposed on you (whether by court order,
373
+agreement or otherwise) that contradict the conditions of this
374
+License, they do not excuse you from the conditions of this License.
375
+If you cannot distribute so as to satisfy simultaneously your
376
+obligations under this License and any other pertinent obligations,
377
+then as a consequence you may not distribute the Library at all. For
378
+example, if a patent license would not permit royalty-free
379
+redistribution of the Library by all those who receive copies directly
380
+or indirectly through you, then the only way you could satisfy both it
381
+and this License would be to refrain entirely from distribution of the
382
+Library.
383
+
384
+If any portion of this section is held invalid or unenforceable under
385
+any particular circumstance, the balance of the section is intended to
386
+apply, and the section as a whole is intended to apply in other
387
+circumstances.
388
+
389
+It is not the purpose of this section to induce you to infringe any
390
+patents or other property right claims or to contest validity of any
391
+such claims; this section has the sole purpose of protecting the
392
+integrity of the free software distribution system which is
393
+implemented by public license practices. Many people have made
394
+generous contributions to the wide range of software distributed
395
+through that system in reliance on consistent application of that
396
+system; it is up to the author/donor to decide if he or she is willing
397
+to distribute software through any other system and a licensee cannot
398
+impose that choice.
399
+
400
+This section is intended to make thoroughly clear what is believed to
401
+be a consequence of the rest of this License.
402
+
403
+**12.** If the distribution and/or use of the Library is restricted in
404
+certain countries either by patents or by copyrighted interfaces, the
405
+original copyright holder who places the Library under this License
406
+may add an explicit geographical distribution limitation excluding
407
+those countries, so that distribution is permitted only in or among
408
+countries not thus excluded. In such case, this License incorporates
409
+the limitation as if written in the body of this License.
410
+
411
+**13.** The Free Software Foundation may publish revised and/or new
412
+versions of the Lesser General Public License from time to time. Such
413
+new versions will be similar in spirit to the present version, but may
414
+differ in detail to address new problems or concerns.
415
+
416
+Each version is given a distinguishing version number. If the Library
417
+specifies a version number of this License which applies to it and
418
+"any later version", you have the option of following the terms and
419
+conditions either of that version or of any later version published by
420
+the Free Software Foundation. If the Library does not specify a
421
+license version number, you may choose any version ever published by
422
+the Free Software Foundation.
423
+
424
+**14.** If you wish to incorporate parts of the Library into other
425
+free programs whose distribution conditions are incompatible with
426
+these, write to the author to ask for permission. For software which
427
+is copyrighted by the Free Software Foundation, write to the Free
428
+Software Foundation; we sometimes make exceptions for this. Our
429
+decision will be guided by the two goals of preserving the free status
430
+of all derivatives of our free software and of promoting the sharing
431
+and reuse of software generally.
432
+
433
+**NO WARRANTY**
434
+
435
+**15.** BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO
436
+WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW.
437
+EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR
438
+OTHER PARTIES PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY
439
+KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE
440
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
441
+PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE
442
+LIBRARY IS WITH YOU. SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME
443
+THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
444
+
445
+**16.** IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN
446
+WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY
447
+AND/OR REDISTRIBUTE THE LIBRARY AS PERMITTED ABOVE, BE LIABLE TO YOU
448
+FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR
449
+CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE
450
+LIBRARY (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING
451
+RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A
452
+FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF
453
+SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
454
+DAMAGES.
455
+
456
+### END OF TERMS AND CONDITIONS
457
+
458
+### How to Apply These Terms to Your New Libraries
459
+
460
+If you develop a new library, and you want it to be of the greatest
461
+possible use to the public, we recommend making it free software that
462
+everyone can redistribute and change. You can do so by permitting
463
+redistribution under these terms (or, alternatively, under the terms
464
+of the ordinary General Public License).
465
+
466
+To apply these terms, attach the following notices to the library. It
467
+is safest to attach them to the start of each source file to most
468
+effectively convey the exclusion of warranty; and each file should
469
+have at least the "copyright" line and a pointer to where the full
470
+notice is found.
471
+
472
+    one line to give the library's name and an idea of what it does.
473
+    Copyright (C) year  name of author
474
+
475
+    This library is free software; you can redistribute it and/or
476
+    modify it under the terms of the GNU Lesser General Public
477
+    License as published by the Free Software Foundation; either
478
+    version 2.1 of the License, or (at your option) any later version.
479
+
480
+    This library is distributed in the hope that it will be useful,
481
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
482
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
483
+    Lesser General Public License for more details.
484
+
485
+    You should have received a copy of the GNU Lesser General Public
486
+    License along with this library; if not, write to the Free Software
487
+    Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
488
+
489
+Also add information on how to contact you by electronic and paper
490
+mail.
491
+
492
+You should also get your employer (if you work as a programmer) or
493
+your school, if any, to sign a "copyright disclaimer" for the library,
494
+if necessary. Here is a sample; alter the names:
495
+
496
+    Yoyodyne, Inc., hereby disclaims all copyright interest in
497
+    the library `Frob' (a library for tweaking knobs) written
498
+    by James Random Hacker.
499
+
500
+    signature of Ty Coon, 1 April 1990
501
+    Ty Coon, President of Vice
502
+
503
+That's all there is to it!

+ 10
- 0
Makefile View File

@@ -0,0 +1,10 @@
1
+# UriPecker - Peck them URIs out
2
+# See LICENSE.md file for copyright and license details.
3
+
4
+MKIT_DIR=utils/mkit
5
+include $(MKIT_DIR)/mkit.mk
6
+
7
+TF_DIR=utils/tfkit
8
+include $(TF_DIR)/tfkit.mk
9
+
10
+.PHONY: test

+ 5
- 0
README.md View File

@@ -0,0 +1,5 @@
1
+UriPecker
2
+=========
3
+
4
+
5
+Peck them URIs out

+ 34
- 0
mkit.ini View File

@@ -0,0 +1,34 @@
1
+[project]
2
+    version     = 0.1.0
3
+    name        = UriPecker
4
+    tagline     = Peck them URIs out
5
+    pkgname     = shellfu-bash-uripecker
6
+    maintainer  = Alois Mahdal <netvor+uripecker@vornet.cz>
7
+    vcs_browser = https://github.com/AloisMahdal/shellfu-bash-uripecker
8
+    relsrc      = master
9
+    reldst      = last
10
+
11
+[dist]
12
+    tarball  = LICENSE.md
13
+    tarball  = Makefile
14
+    tarball  = README.md
15
+    tarball  = mkit.ini
16
+    tarball  = packaging
17
+    tarball  = src
18
+    tarball  = tests
19
+    tarball  = utils
20
+    rpmstuff = packaging/template.spec
21
+    debstuff = packaging/debian
22
+
23
+[ENV]
24
+    PREFIX = /usr/local
25
+
26
+[roots]
27
+    doc   = [ENV:PREFIX]/share/doc/shellfu-bash-uripecker
28
+    sfmod = [ENV:PREFIX]/share/shellfu/include-bash
29
+
30
+[files]
31
+    doc   = README.md
32
+    sfmod = src/uripecker.sh
33
+
34
+#mkit version=0.0.24

+ 5
- 0
packaging/debian/changelog View File

@@ -0,0 +1,5 @@
1
+__MKIT_PROJ_PKGNAME__ (__MKIT_PROJ_VERSION__-1) UNRELEASED; urgency=medium
2
+
3
+  * Initial release. (Closes: #XXXXXX)
4
+
5
+ -- __MKIT_PROJ_MAINTAINER__  Thu, 10 Aug 2017 22:15:34 +0200

+ 1
- 0
packaging/debian/compat View File

@@ -0,0 +1 @@
1
+9

+ 26
- 0
packaging/debian/control View File

@@ -0,0 +1,26 @@
1
+Source: __MKIT_PROJ_PKGNAME__
2
+Maintainer: __MKIT_PROJ_MAINTAINER__
3
+Vcs-Browser: __MKIT_PROJ_VCS_BROWSER__
4
+Section: misc
5
+Priority: extra
6
+Standards-Version: 3.9.2
7
+Build-Depends:
8
+ debhelper (>= 9),
9
+ shellfu (>= 0.10),
10
+ shellfu-bash,
11
+ shellfu-bash-pretty,
12
+
13
+Package: __MKIT_PROJ_PKGNAME__
14
+Architecture: all
15
+Depends:
16
+ shellfu (>= 0.10),
17
+ shellfu-bash,
18
+ shellfu-bash-pretty,
19
+Description: __MKIT_PROJ_NAME__ - __MKIT_PROJ_TAGLINE__
20
+ UriPecker is Shellfu library of one function that takes a string on
21
+ standard output, and looks for things that look like could be URIs.
22
+ Besides real URIs, it can look for set of keywords and identifiers such
23
+ as 'build 1234', 'issue#45' or 'bz6789' and translate them based on a
24
+ simple INI file.
25
+
26
+# control file built with MKit __MKIT_SELF_VERSION__

+ 1
- 0
packaging/debian/copyright View File

@@ -0,0 +1 @@
1
+

+ 13
- 0
packaging/debian/rules View File

@@ -0,0 +1,13 @@
1
+#!/usr/bin/make -f
2
+
3
+%:
4
+
5
+	dh $@
6
+
7
+override_dh_auto_install:
8
+
9
+	make install PREFIX=/usr DESTDIR=debian/tmp
10
+
11
+override_dh_auto_test:
12
+	make test \
13
+		SHELLFU_PATH=$(shell pwd)/debian/tmp/usr/share/shellfu/include-bash

+ 1
- 0
packaging/debian/shellfu-bash-uripecker.install View File

@@ -0,0 +1 @@
1
+/usr/share/shellfu/include-bash/uripecker.sh

+ 1
- 0
packaging/debian/source/format View File

@@ -0,0 +1 @@
1
+3.0 (quilt)

+ 45
- 0
packaging/template.spec View File

@@ -0,0 +1,45 @@
1
+Name:		__MKIT_PROJ_PKGNAME__
2
+Version:	__MKIT_PROJ_VERSION__
3
+Release:	1%{?dist}
4
+Summary:	__MKIT_PROJ_NAME__ - __MKIT_PROJ_TAGLINE__
5
+URL:            __MKIT_PROJ_VCS_BROWSER__
6
+
7
+License:	LGPLv2
8
+Source0:	%{name}-%{version}.tar.gz
9
+
10
+BuildArch:	noarch
11
+BuildRequires:	coreutils
12
+BuildRequires:	shellfu => 0.10.0-1
13
+BuildRequires:	shellfu-bash-pretty
14
+Requires:	shellfu => 0.10.0-1
15
+Requires:	shellfu-bash
16
+Requires:	shellfu-bash-pretty
17
+
18
+%description
19
+UriPecker is Shellfu library of one function that takes a string on
20
+standard output, and looks for things that look like could be URIs.
21
+Besides real URIs, it can look for set of keywords and identifiers such
22
+as 'build 1234', 'issue#45' or 'bz6789' and translate them based on a
23
+simple INI file.
24
+
25
+%prep
26
+%setup -q
27
+
28
+%build
29
+make %{?_smp_mflags} PREFIX=/usr
30
+
31
+%install
32
+%make_install PREFIX=/usr
33
+
34
+%check
35
+make test \
36
+    SHELLFU_PATH=%{buildroot}/%{_datadir}/shellfu/include-bash
37
+
38
+%files
39
+%doc %{_docdir}/%{name}/README.md
40
+%{_datadir}/shellfu/include-bash/uripecker.sh
41
+
42
+
43
+%changelog
44
+
45
+# specfile built with MKit __MKIT_SELF_VERSION__

+ 273
- 0
src/uripecker.sh.skel View File

@@ -0,0 +1,273 @@
1
+#!/bin/bash
2
+
3
+shellfu import pretty
4
+
5
+
6
+uripecker() {
7
+    #
8
+    # Scan stdin for what looks like URI, ID or keyword
9
+    #
10
+    # Usage:
11
+    #     <SOURCE_TEXT uripecker MAP
12
+    #
13
+    # Search through SOURCE_TEXT and output in following order:
14
+    #
15
+    #  1. apparent URIs,
16
+    #  2. hash IDs ("bug#1234"),
17
+    #  3. equal sign expressions ("bug = 1234"),
18
+    #  4. "tags" ("bug1234")
19
+    #  5. keyword expressions ("BUG 1234" or "g hello world"),
20
+    #
21
+    # all (except the first one, obviously) converted to URI using mappings
22
+    # from MAP file.
23
+    #
24
+    # Note that keyword expressions (e.g. "bug 123") work only if they start
25
+    # the line;  rest of the line is taken as query argument and URL-quoted,
26
+    # so that "g what is bmo" would work as expected (given 'g' is defined
27
+    # as Google search).
28
+    #
29
+    # Apply this filter to args or clipboard, and either use head -1 or
30
+    # if you are brave, open all URIs.
31
+    #
32
+    # The MAP file can contain any number of query mappings in format:
33
+    #
34
+    #     NAME = URI_PATTERN
35
+    #
36
+    # where NAME is any string without spaces, equal sign or dot, and
37
+    # URI_PATTERN is a string with precisely one instance of '%s'.
38
+    #
39
+    # For example, given this MAP file:
40
+    #
41
+    #     issue = http://gitlab.example.org/issue?id=%s
42
+    #     faq = http://faq.example.org/faq/%s
43
+    #
44
+    # following text
45
+    #
46
+    #     issue = 1
47
+    #     faq#225
48
+    #     issue42
49
+    #     http://other.example.com/
50
+    #     faq 14
51
+    #     or faq 15
52
+    #
53
+    # would return
54
+    #
55
+    #     http://other.example.com/
56
+    #     http://faq.example.org/faq/225
57
+    #     http://gitlab.example.org/issue?id=1
58
+    #     http://gitlab.example.org/issue?id=42
59
+    #     http://faq.example.org/faq/14
60
+    #
61
+    # Note that the URI_PATTERN can be any kind of URI, such as ftp:// URI,
62
+    # (or any string, actually) but the '%s' is converted using HTTP URI rules.
63
+    #
64
+    local MapFile=$1; shift
65
+    test -n "$MapFile" || {
66
+        warn "usage: uripecker MAP"
67
+        return 2
68
+    }
69
+    local MapBody
70
+    local tmp   # temporary pipe storage
71
+    tmp=$(mktemp -d -t uripecker.XXXXXXXX)
72
+    local maybe_uris="$tmp/maybe_uris"
73
+    local maybe_ids="$tmp/maybe_ids"
74
+    local maybe_exps="$tmp/maybe_exps"
75
+    local maybe_tags="$tmp/maybe_tags"
76
+    local maybe_kws="$tmp/maybe_kws"
77
+    local uris="$tmp/uris"
78
+    local uris_from_ids="$tmp/uris_from_ids"
79
+    local uris_from_exps="$tmp/uris_from_exps"
80
+    local uris_from_tags="$tmp/uris_from_tags"
81
+    local uris_from_kws="$tmp/uris_from_kws"
82
+    MapBody=$(<"$MapFile") || {
83
+        warn "error reading map file: $MapFile"
84
+        return 3
85
+    }
86
+    ##
87
+    # heat up and fill pipes
88
+    #
89
+    mkfifo "$maybe_uris" "$maybe_ids" "$maybe_exps" "$maybe_tags" "$maybe_kws" \
90
+           "$uris" "$uris_from_ids" "$uris_from_exps" "$uris_from_tags" "$uris_from_kws"
91
+    __uripecker__minimize | tee "$maybe_uris" "$maybe_ids" "$maybe_exps" "$maybe_tags" "$maybe_kws" \
92
+        >/dev/null &
93
+    ##
94
+    # process each pipe *async* by different filter
95
+    #
96
+    < "$maybe_uris" __uripecker__flt_uris                          > "$uris" &
97
+    < "$maybe_ids"  __uripecker__flt_ids  | __uripecker__deref > "$uris_from_ids" &
98
+    < "$maybe_exps" __uripecker__flt_exps | __uripecker__deref > "$uris_from_exps" &
99
+    < "$maybe_tags" __uripecker__flt_tags | __uripecker__deref > "$uris_from_tags" &
100
+    < "$maybe_kws"  __uripecker__flt_kws  | __uripecker__deref > "$uris_from_kws" &
101
+    ##
102
+    # print result *sync* in correct order
103
+    #
104
+    {
105
+        cat "$uris"
106
+        cat "$uris_from_ids"
107
+        cat "$uris_from_exps"
108
+        cat "$uris_from_tags"
109
+        cat "$uris_from_kws"
110
+    } | grep .  # throw away empties; add missing LF
111
+    rm -rf "$tmp"
112
+}
113
+
114
+__uripecker__map_lookup() {
115
+    #
116
+    # Look up query $1 in $Map
117
+    #
118
+    local key=${1,,}
119
+    sed '
120
+            s/^ *//
121
+            s/ *$//
122
+            s/ *= */=/
123
+        ' <<<"$MapBody" \
124
+      | grep "^$key=" \
125
+      | cut -d= -f2-
126
+}
127
+
128
+__uripecker__deref() {
129
+    #
130
+    # Turn query (like "g hello" for google) to URI
131
+    #
132
+    local kw        # keyword part, eg. "g"             or "bug"
133
+    local query     # query part,   eg. "hello+dolly"   or "1234"
134
+    local fmt       # for queries: format string (from ini) to pass to printf
135
+    while read -r kw query;
136
+    do
137
+        debug -v kw query
138
+        fmt=$(__uripecker__map_lookup "$kw") || return 1
139
+        debug -v fmt
140
+        #shellcheck disable=SC2059
141
+        printf "$fmt\n" "$(__uripecker__urlquote "$query")"
142
+    done
143
+}
144
+
145
+__uripecker__flt_exps() {
146
+    #
147
+    # Hack expressions like bug = 123 out of the text
148
+    #
149
+    sed -e 's/(\d)\</\n/;' \
150
+      | perl -CS -ne '
151
+            next unless m/\b([a-zA-Z]\w*\s*=\s*[[:alnum:]][[:alnum:]_#-]*)\b/;
152
+            print "$1\n";
153
+        ' \
154
+      | sed -re 's/\s*=\s*/ /'
155
+}
156
+
157
+__uripecker__flt_ids() {
158
+    #
159
+    # Hack doer-like id's (ID#123) out of the text
160
+    #
161
+    tr ' ' '\n' \
162
+      | perl -CS -ne '
163
+            next unless m/\b([a-zA-Z]\w*#[[:alnum:]][[:alnum:]_#-]*)\b/;
164
+            print "$1\n";
165
+        ' \
166
+      | tr '#' ' '
167
+}
168
+
169
+__uripecker__flt_kws() {
170
+    #
171
+    # Hack out lines that look like kw expressions (word space text
172
+    #
173
+    # Eg. 'wiki hello world'
174
+    #
175
+    grep -Ee '^\s*[a-zA-Z][[:alpha:]_]*\s+[^=]'
176
+}
177
+
178
+__uripecker__flt_tags() {
179
+    #
180
+    # Hack "tags" like bug123 out of the text
181
+    #
182
+    tr -c '[:alnum:]' '\n' \
183
+      | grep -E '^[a-zA-Z]+[0-9]+$' \
184
+      | sed -r 's/([a-zA-Z]+)([0-9])/\1 \2/'
185
+}
186
+
187
+__uripecker__flt_uris() {
188
+    #
189
+    # Hack URIs out of the text.
190
+    #
191
+    # Uses URL parser regex Found as SO answer[1] and adapted to include
192
+    # comments from the original Immo's gist[2].
193
+    #
194
+    #   [1]: https://stackoverflow.com/a/30408189/835945
195
+    #   [2]: https://gist.github.com/imme-emosol/731338/810d83626a6d79f40a251f250ed4625cac0e731f
196
+    #
197
+    local uri_re    # Imme Emosol's URI pattern
198
+    local py_code   # code to do the matching
199
+    uri_re=$(
200
+        echo -n '\b'
201
+                    ## protocol identifier
202
+                    #
203
+        echo -n     '(?:(?:https?|ftp)://)'
204
+                    ## user:pass authentication
205
+                    #
206
+        echo -n     '(?:\S+(?::\S*)?@)?'
207
+        echo -n     '(?:'
208
+                        ## IP address dotted notation octets
209
+                        #
210
+        echo -n         '(?:[1-9]\d?|1\d\d|2[01]\d|22[0-3])'
211
+        echo -n         '(?:\.(?:1?\d{1,2}|2[0-4]\d|25[0-5])){2}'
212
+        echo -n         '(?:\.(?:[1-9]\d?|1\d\d|2[0-4]\d|25[0-4]))'
213
+        echo -n      '|'
214
+                        ## host name
215
+                        #
216
+        echo -n         '(?:(?:[a-z\u00a1-\uffff0-9]+-?)*[a-z\u00a1-\uffff0-9]+)'
217
+                        ## domain name
218
+                        #
219
+        echo -n         '(?:\.(?:[a-z\u00a1-\uffff0-9]+-?)*[a-z\u00a1-\uffff0-9]+)*'
220
+                        ## TLD identifier
221
+                        #
222
+        echo -n         '(?:\.(?:[a-z\u00a1-\uffff]{2,}))'
223
+        echo -n      '|'
224
+                        ## "localhost"
225
+                        #
226
+        echo -n         '(?:localhost)'
227
+        echo -n     ')'
228
+                    ## port number
229
+                    #
230
+        echo -n     '(?::\d{2,5})?'
231
+                    ## resource path
232
+                    #
233
+        echo -n     '(?:/[^\s]*)?'
234
+        echo -n '\b'
235
+    )
236
+    py_code=$(
237
+        echo 'import os'
238
+        echo 'import re'
239
+        echo 'import sys'
240
+        echo
241
+        echo 'for uri in re.findall(os.environ["URI_RE"], sys.stdin.read()):'
242
+        echo '    print uri'
243
+    )
244
+    URI_RE=$uri_re python -c "$py_code"
245
+}
246
+
247
+__uripecker__minimize() {
248
+    #
249
+    # Strip, squash spaces and replace tabs
250
+    #
251
+    sed '
252
+        s/\t/ /g
253
+        s/  */ /g
254
+        s/^ //
255
+        s/ $//
256
+    '
257
+}
258
+
259
+__uripecker__urlquote() {
260
+    #
261
+    # URL-quote query $1 and print result
262
+    #
263
+    local query=$1
264
+    debug -v query
265
+    #shellcheck disable=SC2028
266
+    python -c "$(
267
+        echo 'import urllib'
268
+        echo 'import sys'
269
+        echo 'print urllib.quote_plus(sys.argv[1])'
270
+    )" "$query"
271
+}
272
+
273
+#shellfu module-version: __MKIT_PROJ_VERSION__

+ 4
- 0
tests/TF_HEADER View File

@@ -0,0 +1,4 @@
1
+#!/bin/bash
2
+. "$(sfpath)" || exit 3
3
+echo -n "version: "
4
+shellfu version uripecker

+ 188
- 0
tests/uripecker/TF_RUN View File

@@ -0,0 +1,188 @@
1
+#!/bin/bash
2
+
3
+. "$TF_DIR/include/subtest.sh"
4
+. "$TF_DIR/include/tools.sh"
5
+
6
+tf_enum_subtests() {
7
+    echo uri
8
+    echo uris
9
+    echo id
10
+    echo ids
11
+    echo exp
12
+    echo exps
13
+    echo tag
14
+    echo tags
15
+    echo kw
16
+    echo kws
17
+    echo mixed
18
+    echo uris_in_lang
19
+}
20
+
21
+tf_do_subtest() {
22
+    local t_name=$1
23
+    local t_in="test/$t_name.stdin"
24
+    local o_out="oracle/$t_name.stdout"
25
+    {
26
+        echo "    bz = http://bugzilla.example.com/?query=%s"
27
+        echo "    g = http://google.example.com/search?q=%s"
28
+        echo "    storage = ftp://joe:secret@storage%s.files.example.com/"
29
+        echo "    issue = http://gitlab.example.org/issue?id=%s"
30
+        echo "    faq = http://faq.example.org/faq/%s"
31
+    } > test/example.map
32
+    mkdata test > "$t_in"
33
+    mkdata oracle > "$o_out"
34
+    tf_testflt -i "$t_in" -n "$t_name" -O "$o_out" uripecker test/example.map
35
+}
36
+
37
+mkdata() {
38
+    local what=$1
39
+    case $t_name:$what in
40
+        uri:test)
41
+            echo "http://distinct.uri1.example.com/"
42
+            ;;
43
+        uri:oracle)
44
+            echo "http://distinct.uri1.example.com"
45
+            ;;
46
+        uris:test)
47
+            echo "http://distinct.uri1.example.com/"
48
+            echo "also http://distinct.uri2.example.com/get"
49
+            echo "http://distinct.uri3.example.com/?x=1 as well"
50
+            echo "then there's http://distinct.uri4.example.com/?x=1&y=2 as well"
51
+            echo "and finally:"
52
+            echo "http://distinct.uri5.example.com/get?x=1&y=2&foo=bar+%20+baz"
53
+            ;;
54
+        uris:oracle)
55
+            echo "http://distinct.uri1.example.com"
56
+            echo "http://distinct.uri2.example.com/get"
57
+            echo "http://distinct.uri3.example.com/?x=1"
58
+            echo "http://distinct.uri4.example.com/?x=1&y=2"
59
+            echo "http://distinct.uri5.example.com/get?x=1&y=2&foo=bar+%20+baz"
60
+            ;;
61
+        id:test)
62
+            echo "bz#123456789"
63
+            ;;
64
+        id:oracle)
65
+            echo "http://bugzilla.example.com/?query=123456789"
66
+            ;;
67
+        ids:test)
68
+            echo "bz#1"
69
+            echo "also bz#2"
70
+            echo "also bz#345 as well"
71
+            echo "then there's bz#42 as well"
72
+            echo "and finally:"
73
+            echo "bz#6789"
74
+            ;;
75
+        ids:oracle)
76
+            echo "http://bugzilla.example.com/?query=1"
77
+            echo "http://bugzilla.example.com/?query=2"
78
+            echo "http://bugzilla.example.com/?query=345"
79
+            echo "http://bugzilla.example.com/?query=42"
80
+            echo "http://bugzilla.example.com/?query=6789"
81
+            ;;
82
+        exp:test)
83
+            echo "bz = 123456789"
84
+            ;;
85
+        exp:oracle)
86
+            echo "http://bugzilla.example.com/?query=123456789"
87
+            ;;
88
+        exps:test)
89
+            echo "bz = 1"
90
+            echo "also bz = 2"
91
+            echo "also bz = 345 as well"
92
+            echo "then there's bz = 42 as well"
93
+            echo "and finally:"
94
+            echo "bz = 6789"
95
+            ;;
96
+        exps:oracle)
97
+            echo "http://bugzilla.example.com/?query=1"
98
+            echo "http://bugzilla.example.com/?query=2"
99
+            echo "http://bugzilla.example.com/?query=345"
100
+            echo "http://bugzilla.example.com/?query=42"
101
+            echo "http://bugzilla.example.com/?query=6789"
102
+            ;;
103
+        tag:test)
104
+            echo "bz123456789"
105
+            ;;
106
+        tag:oracle)
107
+            echo "http://bugzilla.example.com/?query=123456789"
108
+            ;;
109
+        tags:test)
110
+            echo "bz1"
111
+            echo "bz345 as well"
112
+            echo "and finally:"
113
+            echo "bz6789"
114
+            ;;
115
+        tags:oracle)
116
+            echo "http://bugzilla.example.com/?query=1"
117
+            echo "http://bugzilla.example.com/?query=345"
118
+            echo "http://bugzilla.example.com/?query=6789"
119
+            ;;
120
+        kw:test)
121
+            echo "g foo"
122
+            ;;
123
+        kw:oracle)
124
+            echo "http://google.example.com/search?q=foo"
125
+            ;;
126
+        kws:test)
127
+            echo "g foo"
128
+            echo "g  bar"
129
+            echo "g baz qux quux"
130
+            echo "g learn 中文 in no time"
131
+        ;;
132
+        kws:oracle)
133
+            echo "http://google.example.com/search?q=foo"
134
+            echo "http://google.example.com/search?q=bar"
135
+            echo "http://google.example.com/search?q=baz+qux+quux"
136
+            echo "http://google.example.com/search?q=learn+%E4%B8%AD%E6%96%87+in+no+time"
137
+        ;;
138
+        mixed:test)
139
+            echo "issue = 1"
140
+            echo "http://other.example.com/"
141
+            echo "faq#225"
142
+            echo "https://secure.example.com/"
143
+            echo "issue42"
144
+            echo "https://secure.example.com/foo?id=2&name=bar+baz"
145
+            echo "faq 14"
146
+            echo "https://urlescape.example.com/foo?name=Jan%20%C5%A0varn%C3%BD%2C%20bytem%20%C5%98e%C4%8Dkovice%2017"
147
+            echo "or faq 15"
148
+            echo "http://anchor.example.org/foo/bar.html#baz"
149
+            echo "ftp://joe:secret@files.example.com/"
150
+            echo "storage 42"
151
+            ;;
152
+        mixed:oracle)
153
+            echo "http://other.example.com"
154
+            echo "https://secure.example.com"
155
+            echo "https://secure.example.com/foo?id=2&name=bar+baz"
156
+            echo "https://urlescape.example.com/foo?name=Jan%20%C5%A0varn%C3%BD%2C%20bytem%20%C5%98e%C4%8Dkovice%2017"
157
+            echo "http://anchor.example.org/foo/bar.html#baz"
158
+            echo "ftp://joe:secret@files.example.com"
159
+            echo "http://faq.example.org/faq/225"
160
+            echo "http://gitlab.example.org/issue?id=1"
161
+            echo "http://gitlab.example.org/issue?id=42"
162
+            echo "http://faq.example.org/faq/14"
163
+            echo "ftp://joe:secret@storage42.files.example.com/"
164
+            ;;
165
+        uris_in_lang:test)
166
+            echo 'Hello,'
167
+            echo ''
168
+            echo 'please download the new version here: https://build.example.com/new#build.'
169
+            echo 'Full changelog can be found at release notes page(http://release.example.com/cl/v1.2.0) within few minutes.'
170
+            echo
171
+            echo 'Cheers,'
172
+            echo 'Joe'
173
+            echo ''
174
+            echo '-- '
175
+            echo 'Joe the Builder'
176
+            echo 'http://joe.example.com/joe#contact'
177
+            ;;
178
+        uris_in_lang:oracle)
179
+            echo 'https://build.example.com/new#build'
180
+            echo 'http://release.example.com/cl/v1.2.0'
181
+            echo 'http://joe.example.com/joe#contact'
182
+            ;;
183
+    esac
184
+}
185
+
186
+mkdir -p test oracle
187
+
188
+tf_do_subtests

+ 5
- 0
tests/uripecker/TF_SETUP View File

@@ -0,0 +1,5 @@
1
+#!/bin/bash
2
+
3
+. "$(sfpath)" || exit 3
4
+
5
+shellfu import uripecker

+ 229
- 0
utils/mkit/include/build.sh View File

@@ -0,0 +1,229 @@
1
+#!/bin/bash
2
+# MKit - simple install helper
3
+# See LICENSE file for copyright and license details.
4
+
5
+mkit_import ini
6
+mkit_import facts
7
+
8
+
9
+_build1() {
10
+    #
11
+    # Process one skeleton $1 of type $3 (or guessed) to path $2
12
+    #
13
+    local srcpath=$1    # skeleton path
14
+    local dstpath=$2    # destination meaty animal path
15
+    local ftype=$3      # file/builder type
16
+    test -n "$dstpath"  || dstpath=${srcpath%.skel}
17
+    test -n "$ftype"    || ftype=$(_guess_ftype "$dstpath")
18
+    debug_var srcpath dstpath ftype
19
+    <"$srcpath" _build1_ftype "$ftype" >"$dstpath"
20
+    mkdir -p "$MKIT_LOCAL"
21
+    echo "$dstpath" >> "$MKIT_LOCAL/built.lst"
22
+}
23
+
24
+_build1_ftype() {
25
+    #
26
+    # Build a file of type $1; fom stdin to stdout
27
+    #
28
+    local ftype=$1      # file/builder type
29
+    case $ftype in
30
+        MKIT_COMMON)    _expand_tokens "tokens" ;;
31
+        markdown)       _expand_includes | _expand_tokens "tokens" ;;
32
+        rpmstuff)       _expand_tokens "tokens" "rpmstuff:tokens" ;;
33
+        debstuff)       _expand_tokens "tokens" "debstuff:tokens" ;;
34
+        *)              die "unknown file type: $ftype" ;;
35
+    esac
36
+}
37
+
38
+_expand_includes() {
39
+    #
40
+    # Expand include directives
41
+    #
42
+    # Expand e.g. `<!-- include4: foo.sh -->` to include code of foo.sh
43
+    #
44
+    perl -we '
45
+        use strict;
46
+        my $text;
47
+        while (<>) {
48
+            chomp;
49
+            if (m/<!-- include4: (\S+) -->/) {
50
+                open my $fh, $1 or warn "cannot find: $1";
51
+                my $text = do { local($/); <$fh> };
52
+                close $fh;
53
+                $text =~ s/^(.)/    $1/gm;
54
+                chomp $text;
55
+                print "$text\n";
56
+            } else {
57
+                print "$_\n";
58
+            }
59
+        }
60
+    '
61
+}
62
+
63
+_expand_tokens() {
64
+    #
65
+    # Read stdin, expanding tokens from sections $@
66
+    #
67
+    local script        # sed script cache
68
+    local section       # each section to expand tokens from
69
+    local varname       # each token name
70
+    local varvalue      # each token value
71
+    script=$(mktemp --tmpdir mkit-tmp.XXXXXXXXXX)
72
+    {
73
+        for section in "$@"; do
74
+            debug_var section
75
+            ini lskeys "$section" \
76
+              | while read -r varname; do
77
+                    varvalue="$(ini 1value "$section:$varname" | _qfs )"
78
+                    echo "s|$varname|$varvalue|g;"
79
+                    debug_var varname varvalue
80
+                done
81
+        done
82
+        echo "s|__MKIT_PROJ_NAME__|$(ini 1value project:name | _qfs)|g;"
83
+        echo "s|__MKIT_PROJ_CODENAME__|$(ini 1value project:codename | _qfs)|g;"
84
+        echo "s|__MKIT_PROJ_LICENSE__|$(ini 1value project:license | _qfs)|g;"
85
+        echo "s|__MKIT_PROJ_PKGNAME__|$(ini 1value project:pkgname | _qfs)|g;"
86
+        echo "s|__MKIT_PROJ_TAGLINE__|$(ini 1value project:tagline | _qfs)|g;"
87
+        echo "s|__MKIT_PROJ_MAINTAINER__|$(ini 1value project:maintainer | _qfs)|g;"
88
+        echo "s|__MKIT_PROJ_VCS_BROWSER__|$(ini 1value project:vcs_browser | _qfs)|g;"
89
+        echo "s|__MKIT_PROJ_GIT_LASTHASH__|$(git_lasthash | _qfs)|g;"
90
+        echo "s|__MKIT_PROJ_VERSION__|$(semver | _qfs)|g;"
91
+        echo "s|__MKIT_SELF_VERSION__|$MKIT_VERSION|g;"
92
+    } >> "$script"
93
+    sed -f "$script" || die "_expand_tokens failed"
94
+    rm "$script"
95
+}
96
+
97
+_guess_ftype() {
98
+    #
99
+    # Guess file type from destination path $1
100
+    #
101
+    local dstpath=$1    # destination path
102
+    case $dstpath in
103
+        *.md) echo markdown    ;;
104
+        *)    echo MKIT_COMMON ;;
105
+    esac
106
+}
107
+
108
+_qfs() {
109
+    #
110
+    # Quote for our sed scipt's RHS
111
+    #
112
+    sed '
113
+        s:\\:\\\\:g
114
+        s:|:\\|:g
115
+    '
116
+}
117
+
118
+build() {
119
+    #
120
+    # Add meat to all skeletons
121
+    #
122
+    local srcpath   # each source path
123
+    find -type f -name '*.skel' \
124
+     | while read -r srcpath; do
125
+           _build1 "$srcpath"
126
+       done
127
+}
128
+
129
+build_manpages() {
130
+    #
131
+    # Build manpages using ronn
132
+    #
133
+    local manfile   # each manual file listed in '[files:man]'
134
+    local mdfile    # each source markdown file
135
+    if command -v ronn >/dev/null; then
136
+        ini lskeys "files:man" \
137
+          | while read -r manfile; do
138
+                mdfile="$manfile.md"
139
+                ronn -r "$mdfile"
140
+                mkdir -p "$MKIT_LOCAL"
141
+                echo "$manfile" >> "$MKIT_LOCAL/built.lst"
142
+            done
143
+    else
144
+        echo "ronn is not installed"
145
+        return 1
146
+    fi
147
+}
148
+
149
+clean() {
150
+    #
151
+    # Clean up tree after building
152
+    #
153
+    test -f "$MKIT_LOCAL/built.lst" && {
154
+        <"$MKIT_LOCAL/built.lst" grep -v -e '\.\.' -e ^/ \
155
+          | xargs -r rm -rf
156
+        rm -f "$MKIT_LOCAL/built.lst"
157
+        rmdir --ignore-fail-on-non-empty "$MKIT_LOCAL"
158
+    }
159
+    true
160
+}
161
+
162
+debstuff() {
163
+    #
164
+    # Build Debian stuff (eamed tarball, debian dir)
165
+    #
166
+    local version       # package version
167
+    local debian_skel   # 'debian' folder skeleton
168
+    local dfsrc         # each source file from ^^
169
+    local dftgt         # each built packaging file
170
+    version=$(semver)
171
+
172
+    # tarball - we should already have by means of 'dist'
173
+    #
174
+    mv "${MKIT_PROJ_PKGNAME}-$version.tar.gz" \
175
+       "${MKIT_PROJ_PKGNAME}_$version.orig.tar.gz" \
176
+     || die "could not rename tarball"
177
+    echo "${MKIT_PROJ_PKGNAME}_$version.orig.tar.gz" >> "$MKIT_LOCAL/built.lst"
178
+
179
+    # read content of each mandatory file from debian_skel
180
+    #
181
+    debian_skel=$(ini 1value dist:debstuff)
182
+    test -n "$debian_skel" || die "dist:debstuff not specified"
183
+    test -d "$debian_skel" || die "debian directory template found: $debian_skel"
184
+    mkdir -p debian/source
185
+    find "$debian_skel" -type f \
186
+      | while read -r dfsrc; do
187
+            dftgt="debian/${dfsrc#$debian_skel}"
188
+            mkdir -p "$(dirname "$dftgt")"
189
+            _build1 "$dfsrc" "$dftgt" debstuff
190
+        done
191
+    echo debian >> "$MKIT_LOCAL/built.lst"
192
+}
193
+
194
+dist() {
195
+    #
196
+    # Create distributable tarball
197
+    #
198
+    #FIXME: lacking Makefile skills, we do this step twice fot
199
+    #       rpmstuff, hence -f hack for gzip
200
+    #
201
+    local version           # tarball version
202
+    local git_lasthash      # last git commit hash
203
+    local dirname           # directory and tarball name
204
+    version=$(semver)
205
+    dirname=$MKIT_PROJ_PKGNAME-$version
206
+    git_lasthash=$(git_lasthash)
207
+    mkdir -p "$dirname"
208
+    ini values "dist:tarball" | xargs -I DIST_ITEM cp -R DIST_ITEM "$dirname"
209
+    update_version "$version" "$dirname/mkit.ini"
210
+    mkdir -p "$dirname/.mkit"
211
+    echo -n "$git_lasthash" > "$dirname/.mkit/git_lasthash"
212
+    tar -cf "$dirname.tar" "$dirname"
213
+    gzip -f "$dirname.tar"      # see above FIXME
214
+    mkdir -p "$MKIT_LOCAL"
215
+    echo "$dirname.tar.gz" >> "$MKIT_LOCAL/built.lst"
216
+    rm -rf "$dirname"
217
+}
218
+
219
+rpmstuff() {
220
+    #
221
+    # Build specfile
222
+    #
223
+    local specname=$MKIT_PROJ_PKGNAME.spec      # .spec filename
224
+    local specsrc                               # source of specfile
225
+    specsrc="$(ini 1value "dist:rpmstuff")"
226
+    test -n "$specsrc" || die "dist:rpmstuff not specified"
227
+    test -f "$specsrc" || die "specfile template not found: $specsrc"
228
+    _build1 "$specsrc" "$specname" rpmstuff
229
+}

+ 129
- 0
utils/mkit/include/deploy.sh View File

@@ -0,0 +1,129 @@
1
+#!/bin/bash
2
+# MKit - simple install helper
3
+# See LICENSE file for copyright and license details.
4
+
5
+mkit_import ini
6
+
7
+_deploy_item() {
8
+    #
9
+    # Deploy item and make it look like wanted
10
+    #
11
+    # usage: _deploy_item src dst [mode]
12
+    #
13
+    # Both src and dst must be names of actual items[1],
14
+    # whereas dst must not exist.  On update, dst is
15
+    # usually to be replaced but that is uninstall's
16
+    # job!
17
+    #
18
+    #  [1] Ie. src=foo and dst=/foo/bar does *always*
19
+    #      mean that foo will become 'bar'.  This is
20
+    #      different than traditional `cp` behavior,
21
+    #      when this depends if 'bar' already exists
22
+    #      as a directory.
23
+    #
24
+    # If mode is omitted or empty, MKIT_DEFAULT_MODE is
25
+    # used instead.
26
+    #
27
+    # Directories are copied recursively, and mode is
28
+    # applied only to files.
29
+    #
30
+    local src=$1                            # source path
31
+    local dst=$2                            # destination path
32
+    local mode=${3:-$MKIT_DEFAULT_MODE}     # mode
33
+    local item                              # each in directory
34
+    if test -d "$src"; then
35
+        find "$src" -type f \
36
+          | while read -r item; do
37
+                [[ $item =~ .skel$ ]] \
38
+                 && grep -q "${item%.skel}" "$MKIT_LOCAL/built.lst" \
39
+                 && continue
40
+                _deploy_item "$item" "$dst${item#$src}" "$mode"
41
+            done
42
+    else
43
+        _maybe install -DTvm "$mode" "$src" "$dst"
44
+    fi
45
+}
46
+
47
+_get_dst() {
48
+    #
49
+    # Find out target path for src file $2 of group $1
50
+    #
51
+    local grp=$1        # deploy group
52
+    local src=$2        # each source
53
+    local dst=$3        # alternative destination name
54
+    test -n "$dst" || dst=${src##*/}
55
+    echo "$(_get_root "$grp")/$dst"
56
+}
57
+
58
+_get_root() {
59
+    #
60
+    # Find out target root for group $1
61
+    #
62
+    local grp=$1        # deploy group
63
+    local root          # root for this group
64
+    local destdir       # value of DESTDIR
65
+    root=$(ini 1value "roots:$grp")
66
+    destdir=$(ini 1value ENV:DESTDIR)
67
+    destdir=${destdir%/}
68
+    case $destdir:$root in
69
+        *:)     die "missing in config.ini: roots:$grp" ;;
70
+        :*)     echo "$root" ;;
71
+        *:*)    echo "$destdir/$root" ;;
72
+    esac
73
+}
74
+
75
+_maybe() {
76
+    #
77
+    # Call the deploy command $1 $@ unless in dry mode
78
+    #
79
+    debug "$@"
80
+    local cmd="$1"; shift
81
+    $MKIT_DRY && return
82
+    case "$cmd" in
83
+        cp|rm|rmdir|chmod|mkdir) $cmd "$@" ;;
84
+        install)                 command -p install "$@" ;;
85
+        *)                       die "bad command called";;
86
+    esac
87
+}
88
+
89
+install() {
90
+    #
91
+    # Install product
92
+    #
93
+    local group     # each deploy group
94
+    local mode      # mode (group-specific)
95
+    local src       # each source path
96
+    local dst       # each (final absolute) destination path
97
+    ini lskeys "files" \
98
+      | sort \
99
+      | uniq \
100
+      | while read -r group; do
101
+            mode=$(ini 1value "modes:$group")
102
+            ini values "files:$group" \
103
+              | while read -r src dst; do
104
+                    dst=$(_get_dst "$group" "$src" "$dst")
105
+                    _deploy_item "$src" "$dst" "$mode"
106
+                done
107
+        done
108
+    test -f "$MKIT_LOCAL/autoclean" && clean
109
+    true
110
+}
111
+
112
+uninstall() {
113
+    #
114
+    # Uninstall product
115
+    #
116
+    local group     # each deploy group
117
+    local src       # each source path
118
+    local dst       # each (final absolute) destination path
119
+    ini lskeys "files" \
120
+      | sort \
121
+      | uniq \
122
+      | while read -r group; do
123
+            ini values "files:$group" \
124
+              | while read -r src dst; do
125
+                    dst=$(_get_dst "$group" "$src" "$dst")
126
+                    _maybe rm -vrf "$dst"
127
+                done
128
+        done
129
+}

+ 182
- 0
utils/mkit/include/facts.sh View File

@@ -0,0 +1,182 @@
1
+#!/bin/bash
2
+# MKit - simple install helper
3
+# See LICENSE file for copyright and license details.
4
+
5
+mkit_import ini
6
+
7
+git_bool() {
8
+    #
9
+    # Get git bool (ie. exit status counts) $1
10
+    #
11
+    local bool_name=$1      # name of boolean to get
12
+    git_present || warn "can't give bool outside git repo: $bool_name"
13
+    case "$bool_name" in
14
+        dirty_files)
15
+            git diff-files | grep -qm 1 .
16
+            ;;
17
+        dirty_index)
18
+            git diff-index HEAD | grep -qm 1 .
19
+            ;;
20
+        dirty)
21
+            git_bool dirty_files || git_bool dirty_index
22
+            ;;
23
+        *)
24
+            warn "unknown git bool asked: $bool_name"
25
+            return 2
26
+            ;;
27
+    esac
28
+}
29
+
30
+git_fact() {
31
+    #
32
+    # Get git fact $1
33
+    #
34
+    local fact_name=$1      # name of fact to get
35
+    git_present || warn "can't give fact outside git repo: $fact_name"
36
+    case "$fact_name" in
37
+        latest_tag)
38
+            git log --format="%d" \
39
+              | sed 's/,/\n/g' \
40
+              | sed 's/^[[:blank:]]*//' \
41
+              | grep -E '^\(?tag' \
42
+              | tr -cd '[[:digit:]].v\n' \
43
+              | grep . -m 1
44
+            ;;
45
+        latest_version)
46
+            git_fact latest_tag | git_tag2ver
47
+            ;;
48
+        current_branch)
49
+            git rev-parse --abbrev-ref HEAD
50
+            ;;
51
+        reldiff)
52
+            git log --oneline "$(git_fact latest_tag)..HEAD" --name-only
53
+            ;;
54
+        latest_sha)
55
+            git log -1 --pretty=format:%h HEAD
56
+            ;;
57
+        *)
58
+            warn "unknown git fact asked: $fact_name"
59
+            ;;
60
+    esac
61
+}
62
+
63
+git_present() {
64
+    #
65
+    # True if we're in a git repo
66
+    #
67
+    git rev-parse HEAD >&/dev/null
68
+}
69
+
70
+git_tag2ver() {
71
+    #
72
+    # Convert tag to version
73
+    #
74
+    sed s/^v//
75
+}
76
+
77
+git_ver2tag() {
78
+    #
79
+    # Convert version to tag
80
+    #
81
+    sed s/^/v/
82
+}
83
+
84
+git_lasthash() {
85
+    #
86
+    # Show last commit hash (with .dirty suffix if needed)
87
+    #
88
+    # If outside git repo, get it from .mkit/git_lasthash, which
89
+    # should have been put there by dist target.  (I.e., this won't
90
+    # work if you got outside the git repo in other way than dist
91
+    # target, but that's actually expected.)
92
+    #
93
+    local last_hash     # last commit hash
94
+    if git_present; then    # we are in git repo
95
+        last_hash=$(git rev-parse HEAD)
96
+        echo -n "$last_hash"
97
+        git_bool dirty && echo -n ".dirty"
98
+    else    # we are outside (eg. distributor's build dir')
99
+        grep . .mkit/git_lasthash || {
100
+            echo UNKNOWN
101
+            warn "malformed source, could not determine git hash"
102
+        }
103
+    fi
104
+}
105
+
106
+semver() {
107
+    #
108
+    # Build semver version string with build metadata
109
+    #
110
+    # Build version string from available info using following
111
+    # logic:
112
+    #
113
+    #  1. use project.version (from mkit.ini)
114
+    #  2. if we are in git, override the version with last tag
115
+    #  3. if set, add project:prerl (from mkit.ini) as pre-release ID
116
+    #     (afer dash)
117
+    #  4. if we are at a later commit than the last tag, add branch
118
+    #     name and commit sha1 to build metadata (after plus sign)
119
+    #  5. if the tree is "dirty", i.e. has uncommited changes,
120
+    #     add "dirty" to build metadata
121
+    #
122
+    # The version is compatible with SemVer 2.0.0.
123
+    #
124
+    # Examples:
125
+    #
126
+    #     myprog v1.0.7                         # all clear
127
+    #     myprog v1.0.7-alpha                   # mkit.ini: project:prerl="alpha"
128
+    #     myprog v1.0.7-alpha+g1aef811.master   # ^^ + some commits after
129
+    #     myprog v1.0.7-alpha+gf14fc4f.api2     # ^^ + on a feature branch
130
+    #     myprog v1.0.7-alpha+gf14fc4f.api2.dirty  # ^^ + tree edited
131
+    #     myprog v1.0.7-alpha+dirty             # tag OK but tree edited
132
+    #     myprog v1.0.7+dirty                   # ^^ but no pre-release id
133
+    #
134
+    # Note that versions with "dirty" should be perceived as kind of
135
+    # dangerous outside developer's own machine.  Versions with sha1 are
136
+    # safer but must not be released.
137
+    #
138
+    # I have considered decorating the git commit refs to make them
139
+    # sort of sortable (e.g. "r1.g1aef811"), but on second thought,
140
+    # I don't think it's good idea to give *any* semantics to meta-data
141
+    # at all.  First, there is no rule that r1<r2<r3; a commit can be
142
+    # removing what other just added and one change can be split to
143
+    # multiple commits.  Also, the whole thing breaks anyway once you
144
+    # rebase your branch (no, it's not a sin).  The sole purpose of
145
+    # meta-data is to *identify* the code, and provide safe path back
146
+    # to tree; commit refs are already perfect for that.
147
+    #
148
+    # FIXME:  Using project:prerl for release IDs may not be compatible with
149
+    #         release strategy implemented in release.sh
150
+    #
151
+    local version       # version string (final result)
152
+    local prerl         # pre-release keyword (from mkit.ini, eg. 'alpha')
153
+    local latest_tag    # latest git tag
154
+    local commit        # commit indicator (CURRENT_BRANCH.gHASH)
155
+    local dirty         # 0 if dirty, 1 if clean
156
+    local suffix        # version suffix
157
+    version=$(ini 1value project:version)
158
+    prerl=$(ini 1value project:prerl)
159
+    grep ":" <<<"$prerl" \
160
+     && warn "colon in project:prerl may corrupt version data: $prerl"
161
+    if git_present;
162
+    then    # we are in git repo... so we can get smart
163
+        latest_tag=$(git_fact latest_tag)
164
+        if ! git describe --tags --exact-match HEAD >&/dev/null;
165
+        then    # we are at a later commit than the last tag
166
+            commit="$(git_fact current_branch).g$(git_fact latest_sha)"
167
+        fi
168
+        git_bool dirty; dirty=$?
169
+        test -n "$latest_tag" && version=${latest_tag:1}
170
+        case "$dirty:$commit" in
171
+            1:)  suffix=""               ;;
172
+            0:)  suffix="+dirty"         ;;
173
+            1:*) suffix="+$commit"       ;;
174
+            0:*) suffix="+$commit.dirty" ;;
175
+            *)   suffix=MKIT_BUG
176
+                 warn "MKIT_BUG: bad dirt/commit detection" ;;
177
+        esac
178
+        test -n "$prerl" && suffix="-$prerl$suffix"
179
+        version="$version$suffix"
180
+    fi
181
+    echo "$version"
182
+}

+ 145
- 0
utils/mkit/include/ini.sh View File

@@ -0,0 +1,145 @@
1
+#!/bin/bash
2
+# MKit - simple install helper
3
+# See LICENSE file for copyright and license details.
4
+
5
+_ini_cat() {
6
+    #
7
+    # A no-op for text stream
8
+    #
9
+    local line      # each line
10
+    while read -r line; do
11
+        printf -- "%s\n" "$line"
12
+    done
13
+}
14
+
15
+_ini_expand() {
16
+    #
17
+    # Expand reference value (prefix only)
18
+    #
19
+    local line      # each input line
20
+    local suffix    # tail of the line
21
+    local ref       # reference
22
+    local value     # value if reference
23
+    while read -r line; do                  # [foo:bar]/path
24
+        suffix="${line#\[*\]}"              # /path
25
+        ref="${line%$suffix}"               # [foo:bar]
26
+        ref="${ref%\]}"                     # [foo:bar
27
+        ref="${ref#\[}"                     # foo:bar
28
+        value="$(ini 1value "$ref")"        # foo_bar_value
29
+        printf -- "%s\n" "$value$suffix"    # foo_bar_value/path
30
+    done
31
+}
32
+
33
+_ini_grepkey() {
34
+    #
35
+    # Read key from a section
36
+    #
37
+    local wnt=$1    # wanted key
38
+    grep '.' \
39
+      | grep -v '\s*#' \
40
+      | sed -e 's/ *= */=/; s/ +$//; s/^//;' \
41
+      | grep -e "^$wnt=" \
42
+      | cut -d= -f2- \
43
+      | _ini_maybe_expand
44
+}
45
+
46
+_ini_greppath() {
47
+    #
48
+    # Read key from the right section
49
+    #
50
+    # E.g. `files:share:my/lib.sh` should read
51
+    #
52
+    #     [files:share]
53
+    #         my/lib.sh   = proj/my/lib.sh
54
+    #
55
+    local wnt=$1                    # wanted path
56
+    local wntkey=${wnt##*:}         # ^^ key part
57
+    local wntsec=${wnt%:$wntkey}    # ^^ section part
58
+    local override                  # ENV override (only ENV section)
59
+    if test "$wntsec" = 'ENV'; then
60
+        override=${!wntkey}
61
+        test -n "$override" \
62
+         && echo "$override" \
63
+         && return
64
+    fi
65
+    _ini_grepsec "$wntsec" | _ini_grepkey "$wntkey"
66
+}
67
+
68
+_ini_grepsec() {
69
+    #
70
+    # Read one INI section
71
+    #
72
+    local wnt=$1        # wanted section name
73
+    local ok=false      # are we in the section?
74
+    local line          # each input line
75
+    grep '.' \
76
+      | grep -v '\s*#' \
77
+      | while read -r line; do
78
+            case "$line" in
79
+                \[$wnt\]) ok=true;  continue ;;
80
+                \[*\])    ok=false; continue ;;
81
+            esac
82
+            $ok || continue
83
+            printf -- "%s\n" "$line"
84
+        done \
85
+      | sed -e 's/ *= */=/; s/ +$//; s/^//;'
86
+}
87
+
88
+_ini_lskeys() {
89
+    #
90
+    # List keys from a section
91
+    #
92
+    local sct=$1    # section of interest
93
+    _ini_grepsec "$sct" | cut -d= -f1 | awk '!x[$0]++'
94
+}
95
+
96
+_ini_maybe_expand() {
97
+    #
98
+    # Decide whether or not to expand
99
+    #
100
+    if test "$MKIT_INI_EXPAND" -gt 0; then
101
+        MKIT_INI_EXPAND=$(( --MKIT_INI_EXPAND )) _ini_expand
102
+    else
103
+        _ini_cat
104
+    fi
105
+}
106
+
107
+ini() {
108
+    #
109
+    # do ini operation
110
+    #
111
+    local op=$1             # operator
112
+    local arg=$2            # argument
113
+    local fn                # internal function implementing $op
114
+    local limit=_ini_cat    # limiting internal function
115
+    case $op in
116
+        lskeys) fn=_ini_lskeys   ;;
117
+        sec)    fn=_ini_grepsec  ;;
118
+        values) fn=_ini_greppath ;;
119
+        1value) fn=_ini_greppath; limit="tail -1" ;;
120
+        *)      die "incorrect use of \`ini()\`"
121
+    esac
122
+    <"$MKIT_INI" $fn "$arg" | $limit
123
+}
124
+
125
+update_version() {
126
+    #
127
+    # Change project.version in mkit.ini at path $2 to version $1
128
+    #
129
+    local version=$1    # new version
130
+    local inifile=$2    # mkit.ini path
131
+    local tmp           # mkit.ini cache
132
+    tmp=$(mktemp -t mkit.update_version.XXXXXXXX)
133
+    <"$inifile" perl -e '
134
+        my $hit = 0;
135
+        my $done = 0;
136
+        foreach (<STDIN>) {
137
+            if      ($done) { print; next; }
138
+            elsif   (m/\[project\]/) { $hit++; print; next; }
139
+            elsif   (m/\[/) { $hit = 0; print; next; }
140
+            elsif   ($hit) { s/\bversion\b( *)=( *).*/version$1=$2$ARGV[0]/ and $done++; print; }
141
+            else { print; next; }
142
+        }
143
+    ' "$version" >"$tmp" || die "failed to update version in mkit.ini"
144
+    mv "$tmp" "$inifile"
145
+}

+ 163
- 0
utils/mkit/include/mkit.sh View File

@@ -0,0 +1,163 @@
1
+#!/bin/bash
2
+# MKit - simple install helper
3
+# See LICENSE file for copyright and license details.
4
+
5
+die() {
6
+    #
7
+    # Exit with message and non-zero exit status
8
+    #
9
+    echo "fatal: $*" >&2
10
+    exit 4
11
+}
12
+
13
+mkit_import() {
14
+    #
15
+    # Import mkit module $1.sh
16
+    #
17
+    # Check for module, source it and die with reasonable message if needed.
18
+    #
19
+    local modname=$1
20
+    local modpath
21
+    modpath="$MKIT_DIR/include/$modname.sh"
22
+    test -f "$modpath" || die "no such module: $modpath"
23
+    bash -n "$modpath" || die "bad syntax: $modpath"
24
+    #shellcheck disable=SC1090
25
+    . "$modpath" || die "failed to import: $modname"
26
+}
27
+
28
+mkit_import build
29
+mkit_import deploy
30
+mkit_import release
31
+mkit_import ini
32
+
33
+_valid_targets() {
34
+    #
35
+    # List valid routes
36
+    #
37
+    echo build
38
+    echo build_manpages
39
+    echo clean
40
+    echo debstuff
41
+    echo dist
42
+    echo install
43
+    echo release
44
+    echo release_x
45
+    echo release_y
46
+    echo release_z
47
+    echo rpmstuff
48
+    echo uninstall
49
+    echo vbump
50
+    echo vbump_x
51
+    echo vbump_y
52
+    echo vbump_z
53
+}
54
+
55
+debug() {
56
+    #
57
+    # Print debug message
58
+    #
59
+    $MKIT_DEBUG || return 0
60
+    echo "MKIT_DEBUG: ${FUNCNAME[1]}()" "$@" >&2
61
+}
62
+
63
+debug_var() {
64
+    #
65
+    # Print debug message
66
+    #
67
+    $MKIT_DEBUG || return 0
68
+    local __mkit_debug_var_name__       # variable name to debug
69
+    for __mkit_debug_var_name__ in "$@"; do
70
+        {
71
+            echo -n "MKIT_DEBUG: ${FUNCNAME[1]}():"
72
+            echo -n " $__mkit_debug_var_name__"
73
+            echo -n "='${!__mkit_debug_var_name__}'"
74
+            echo
75
+        } >&2
76
+    done
77
+}
78
+
79
+_compver() {
80
+    #
81
+    # True if version $1 matches our version
82
+    #
83
+    # If our x is 0, check first two fragments, otherwise check just
84
+    # the x.  Fragments must equal.
85
+    #
86
+    local their_ver=$1      # their version
87
+    local our_x             # our X
88
+    local our_y             # our Y
89
+    local their_x           # their X
90
+    local their_y           # their Y
91
+    their_x=${their_ver%%.*}
92
+    their_y=${their_ver##$their_x.}
93
+    their_y=${their_y%%.*}
94
+    our_x=${MKIT_VERSION%%.*}
95
+    our_y=${MKIT_VERSION##$our_x.}
96
+    our_y=${our_y%%.*}
97
+    debug_var MKIT_VERSION our_x our_y their_ver their_x their_y
98
+    test "$their_x" -eq "$our_x" || return 1
99
+    test "$our_x" -eq 0 && {
100
+        test "$their_y" = "$our_y"
101
+        return $?
102
+    }
103
+    return 0
104
+}
105
+
106
+_chkiniversion() {
107
+    #
108
+    # Check if ini version is supported
109
+    #
110
+    # Look for "#mkit version=0.0.0" or similar in first or last
111
+    # 3 lines of the file; then check if the version is supported.
112
+    #
113
+    local ver_line      # line declaring version
114
+    local their_ver     # the declared version
115
+    ver_line=$(
116
+        {
117
+            head -3 "$MKIT_INI"
118
+            tail -3 "$MKIT_INI"
119
+        } | grep -m 1 -E '^# *mkit +version *= *v?[0-9]+\.[0-9]+\.[0-9]+'
120
+    )
121
+    test -n "$ver_line" \
122
+     || die "version mark ('#mkit version=x.y.z') not found in: $MKIT_INI"
123
+    their_ver="$(tr -d '[:blank:]v' <<<"${ver_line##*=}")"
124
+    _compver "$their_ver" \
125
+     || die "bad mkit.ini version: $their_ver does not match $MKIT_VERSION"
126
+}
127
+
128
+mkit_init() {
129
+    #
130
+    # Do basic initialization
131
+    #
132
+    # Check for ini file and some variables
133
+    #
134
+    $MKIT_DRY && MKIT_DEBUG=true
135
+    MKIT_PROJ_PKGNAME=$(ini 1value "project:pkgname")
136
+    test -f "$MKIT_INI" || die "cannot find mkit.ini: $MKIT_INI"
137
+    _chkiniversion
138
+    test -n "$(tr -d '[:space:]' <<<"$MKIT_LOCAL")" \
139
+     || die "MKIT_LOCAL must be non-blank: '$MKIT_LOCAL'"
140
+}
141
+
142
+route() {
143
+    #
144
+    # Call correct function based on $1
145
+    #
146
+    if _valid_targets | grep -qwx "^$1"; then
147
+        "$1"
148
+    else
149
+        {
150
+            echo "usage: $(basename "$0") TARGET"
151
+            echo
152
+            echo "valid targets:"
153
+            _valid_targets | sed 's/^/    /'
154
+        } >&2
155
+    fi
156
+}
157
+
158
+warn() {
159
+    #
160
+    # Print warning message
161
+    #
162
+    echo "$@" >&2
163
+}

+ 228
- 0
utils/mkit/include/release.sh View File

@@ -0,0 +1,228 @@
1
+#!/bin/bash
2
+# MKit - simple install helper
3
+# See LICENSE file for copyright and license details.
4
+
5
+mkit_import ini
6
+mkit_import facts
7
+
8
+_bump_version() {
9
+    #
10
+    # Bump version on stdin by level $1 (x, y or z)
11
+    #
12
+    local rlevel=$1     # release level
13
+    local old           # old version
14
+    local oldx          # ... X
15
+    local oldz          # ... Z
16
+    local oldy          # ... Y
17
+    local tmpy          # Y parsing cache
18
+    local new           # bumped version
19
+    read -r old
20
+    oldx=${old%.*.*}
21
+    oldz=${old#*.*.}
22
+    tmpy=${old%.*}
23
+    oldy=${tmpy#*.}
24
+    case $rlevel in
25
+        x) new="$((oldx+MKIT_BUMPSIZE)).0.0"            ;;
26
+        y) new="$oldx.$((oldy+MKIT_BUMPSIZE)).0"        ;;
27
+        z) new="$oldx.$oldy.$((oldz+MKIT_BUMPSIZE))"    ;;
28
+        *) die "invalid release level: $1"  ;;
29
+    esac
30
+    echo "$new"
31
+}
32
+
33
+_relck() {
34
+    #
35
+    # Die if blocking condition $1 is detected
36
+    #
37
+    local condition=$1      # condition name
38
+    local oracle            # expected value
39
+    case "$condition" in
40
+        git_present)
41
+            git rev-parse HEAD >&/dev/null\
42
+             || die "cannot do this outside git repo"
43
+            ;;
44
+        at_relsrc)
45
+            oracle=$(ini 1value project:relsrc)
46
+            git_fact current_branch \
47
+              | grep -qFx "$oracle" \
48
+             || die "you are not on release source branch: $oracle"
49
+            ;;
50
+        not_dirty)
51
+            git diff --shortstat 2>/dev/null \
52
+              | grep -q . \
53
+             && die "tree is dirty!"
54
+            ;;
55
+        tags_ok)
56
+            git_fact latest_tag \
57
+              | grep -q . \
58
+             || die "cannot find latest tag"
59
+            ;;
60
+        vbump_hot)
61
+            git diff-tree --no-commit-id --name-only -r HEAD \
62
+              | grep -qFx mkit.ini \
63
+             || die "last change must be version bump in mkit.ini"
64
+            ;;
65
+        no_wip)
66
+            git_fact reldiff \
67
+              | grep '^....... WIP ' \
68
+             && die "WIP commit since $(git_fact latest_tag)"
69
+            ;;
70
+        ini_version)
71
+            oracle=$(git_fact latest_version | _bump_version "$rlevel")
72
+            ini 1value project:version  \
73
+              | grep -qFx "$oracle" \
74
+             || die "new version not in mkit.ini: $oracle"
75
+            ;;
76
+        *)
77
+            die "unknown release check: $condition"
78
+            ;;
79
+    esac
80
+}
81
+
82
+_release() {
83
+    #
84
+    # Prepare release
85
+    #
86
+    # Span release routines: make a signed tag, check branch
87
+    # and update release branch
88
+    #
89
+    # FIXME: Using project:prerl as build.sh:semver() does may not be
90
+    #        compatible with this release strategy
91
+    #
92
+    local rlevel=$1     # release level (x, y or z)
93
+    local newtag        # new tag
94
+    local relsrc        # release source branch (if any)
95
+    local reldst        # release destination branch (if any)
96
+
97
+    _relck git_present
98
+    _relck at_relsrc
99
+    _relck not_dirty
100
+    _relck tags_ok
101
+    _relck vbump_hot
102
+    _relck no_wip
103
+    _relck ini_version
104
+
105
+    newtag=$(git_fact latest_version | _bump_version "$rlevel" | git_ver2tag )
106
+    set -e
107
+    debug_var newtag
108
+    $MKIT_DRY && return
109
+    git tag -m "$(_release_msg)" "$newtag"
110
+
111
+    relsrc=$(ini 1value project:relsrc)
112
+    reldst=$(ini 1value project:reldst)
113
+    debug_var relsrc reldst
114
+    if test -n "$reldst" && test "$reldst" != "$relsrc"; then
115
+        git branch -f "$reldst" "$newtag"
116
+    fi
117
+}
118
+
119
+_release_msg() {
120
+    #
121
+    # Generate message for annotated tag
122
+    #
123
+    # The last commit before issuing a release must be "Bump version" commit
124
+    # suggested by _vbump_gitmsg() and  manually edited by user.  Since the
125
+    # commit contains changelog, this function just uses the message body.
126
+    #
127
+    # The sort message (first line) is replaced with a nicer one (with project
128
+    # name, codename and version).
129
+    #
130
+    echo "$(ini 1value project:name) $newtag - $(ini 1value project:codename)"
131
+    echo
132
+    git show -s --format=%B \
133
+      | tail -n +3
134
+}
135
+
136
+_vbump() {
137
+    #
138
+    # Do version bump at level $1
139
+    #
140
+    # Perform checks, compute new version, update mkit.ini and initiate
141
+    # 'Bump version' commit with changelog template.
142
+    #
143
+    local rlevel=$1     # bump level (x, y or z)
144
+    local nextver       # version after the bump
145
+    _relck git_present
146
+    _relck at_relsrc
147
+    _relck not_dirty
148
+    nextver=$(ini 1value project:version | _bump_version "$rlevel")
149
+    debug_var nextver
150
+    $MKIT_DRY && return
151
+    update_version "$nextver" mkit.ini \
152
+      || die "failed to update version in mkit.ini"
153
+    git add mkit.ini \
154
+      || die "failed to add mkit.ini to the index"
155
+    git commit -e -m "$(_vbump_gitmsg)"
156
+}
157
+
158
+_vbump_gitmsg() {
159
+    #
160
+    # Compose git message template for 'Bump version' commit
161
+    #
162
+    echo "Bump version"
163
+    echo ""
164
+    echo "Overview of changes:"
165
+    echo ""
166
+    git_fact reldiff \
167
+      | sed '
168
+            s/^[a-f0-9]\{7\} / *  &/; t PATHS
169
+            s/^/        /
170
+            :PATHS
171
+        '
172
+}
173
+
174
+release() {
175
+    #
176
+    # Perform release on Z level
177
+    #
178
+    _release z
179
+}
180
+
181
+release_x() {
182
+    #
183
+    # Perform release on X level
184
+    #
185
+    _release x
186
+}
187
+
188
+release_y() {
189
+    #
190
+    # Perform release on Y level
191
+    #
192
+    _release y
193
+}
194
+
195
+release_z() {
196
+    #
197
+    # Perform release on Z level
198
+    #
199
+    _release z
200
+}
201
+
202
+vbump() {
203
+    #
204
+    # Perform version bump on Z level
205
+    #
206
+    _vbump z
207
+}
208
+
209
+vbump_x() {
210
+    #
211
+    # Perform version bump on X level
212
+    #
213
+    _vbump x
214
+}
215
+
216
+vbump_y() {
217
+    #
218
+    # Perform version bump on Y level
219
+    #
220
+    _vbump y
221
+}
222
+
223
+vbump_z() {
224
+    #
225
+    # Perform version bump on Z level
226
+    #
227
+    _vbump z
228
+}

+ 60
- 0
utils/mkit/include/vars.sh View File

@@ -0,0 +1,60 @@
1
+#!/bin/bash
2
+# MKit - simple install helper
3
+# See LICENSE file for copyright and license details.
4
+
5
+
6
+#
7
+# Bump size (for vbump_? and release_?)
8
+#
9
+MKIT_BUMPSIZE=${MKIT_BUMPSIZE:-1}
10
+
11
+#
12
+# Debug mode (true|false)
13
+#
14
+MKIT_DEBUG=${MKIT_DEBUG:-false}
15
+
16
+#
17
+# Default deploy mode for files
18
+#
19
+MKIT_DEFAULT_MODE="644"
20
+
21
+#
22
+# Dry mode (true|false)
23
+#
24
+# Set to true to not install anything. Implies MKIT_DEBUG.
25
+#
26
+MKIT_DRY=${MKIT_DRY:-false}
27
+
28
+#
29
+# Path to mkit.ini
30
+#
31
+MKIT_INI=${MKIT_INI:-mkit.ini}
32
+
33
+#
34
+# Limit ini expansion depth
35
+#
36
+# To avoid endless loops, this value is subtracted each
37
+# time ini() expands a reference; when zero is reached,
38
+# no more expansions happen.
39
+#
40
+MKIT_INI_EXPAND=2
41
+
42
+#
43
+# Path to MKit local config and temp
44
+#
45
+# Typically hidden in project root, here MKit can
46
+# save its temporary lists.
47
+#
48
+MKIT_LOCAL=${MKIT_LOCAL:-.mkit}
49
+
50
+#
51
+# Package name
52
+#
53
+# Used as base for tarball and in some default tokens.
54
+#
55
+MKIT_PROJ_PKGNAME=""
56
+
57
+#
58
+# This MKit version
59
+#
60
+MKIT_VERSION=0.0.24

+ 34
- 0
utils/mkit/make View File

@@ -0,0 +1,34 @@
1
+#!/bin/bash
2
+#shellcheck disable=SC2034
3
+# mkit - simple install helper
4
+# See LICENSE file for copyright and license details.
5
+
6
+init_core() {
7
+    #
8
+    # Load core module (or die)
9
+    #
10
+    #shellcheck disable=SC1090
11
+    . "$MKIT_DIR/include/mkit.sh" \
12
+     && . "$MKIT_DIR/include/vars.sh" \
13
+     && return 0
14
+    echo "failed to load core; check if MKIT_DIR is set properly: $MKIT_DIR" >&2
15
+    exit 9
16
+}
17
+
18
+#
19
+# Path to MKit dir (where 'include' is)
20
+#
21
+MKIT_DIR=${MKIT_DIR:-$(dirname "$0")}
22
+
23
+
24
+init_core
25
+
26
+case "$1" in
27
+    -V|--version-semver)    echo "$MKIT_VERSION"; exit 0 ;;
28
+    --version)              echo "Mkit (Simple Makefile target helper) $MKIT_VERSION"
29
+                            exit 0 ;;
30
+esac
31
+
32
+mkit_init
33
+
34
+route "$@"

+ 56
- 0
utils/mkit/mkit.mk View File

@@ -0,0 +1,56 @@
1
+# mkit - Simple Makefile target helper
2
+# See LICENSE file for copyright and license details.
3
+
4
+export MKIT_DIR
5
+
6
+all: build
7
+
8
+build:
9
+	@$(MKIT_DIR)/make build
10
+
11
+manpages: build
12
+	@$(MKIT_DIR)/make build_manpages
13
+
14
+clean:
15
+	@$(MKIT_DIR)/make clean
16
+
17
+debstuff: dist
18
+	@$(MKIT_DIR)/make debstuff
19
+
20
+dist: clean
21
+	@$(MKIT_DIR)/make dist
22
+
23
+rpmstuff: dist
24
+	@$(MKIT_DIR)/make rpmstuff
25
+
26
+install: all
27
+	@$(MKIT_DIR)/make install
28
+
29
+release:
30
+	@$(MKIT_DIR)/make release
31
+
32
+release_x:
33
+	@$(MKIT_DIR)/make release_x
34
+
35
+release_y:
36
+	@$(MKIT_DIR)/make release_y
37
+
38
+release_z:
39
+	@$(MKIT_DIR)/make release_z
40
+
41
+uninstall:
42
+	@$(MKIT_DIR)/make uninstall
43
+
44
+vbump:
45
+	@$(MKIT_DIR)/make vbump
46
+
47
+vbump_x:
48
+	@$(MKIT_DIR)/make vbump_x
49
+
50
+vbump_y:
51
+	@$(MKIT_DIR)/make vbump_y
52
+
53
+vbump_z:
54
+	@$(MKIT_DIR)/make vbump_z
55
+
56
+.PHONY: all clean dist rpmstuff install uninstall release release_x release_y release_z vbump vbump_x vbump_y vbump_z

+ 698
- 0
utils/mkit/stub View File

@@ -0,0 +1,698 @@
1
+#!/bin/bash
2
+# mkit - simple install helper
3
+# See LICENSE file for copyright and license details.
4
+
5
+init_core() {
6
+    #
7
+    # Load core modules (or die)
8
+    #
9
+    #shellcheck disable=SC1090
10
+    . "$MKIT_DIR/include/mkit.sh" \
11
+     && . "$MKIT_DIR/include/vars.sh" \
12
+     && return 0
13
+    echo "failed to load core; check if MKIT_DIR is set properly: $MKIT_DIR" >&2
14
+    exit 9
15
+}
16
+
17
+#
18
+# Path to MKit dir (where 'include' is)
19
+#
20
+MKIT_DIR=${MKIT_DIR:-$(dirname "$0")}
21
+
22
+init_core
23
+
24
+mkit_import ini
25
+
26
+declare -A MKIT_STUB_LICENSES
27
+MKIT_STUB_LICENSES[GPLv1]="http://www.gnu.org/licenses/old-licenses/gpl-1.0.md"
28
+MKIT_STUB_LICENSES[GPLv2]="http://www.gnu.org/licenses/old-licenses/gpl-2.0.md"
29
+MKIT_STUB_LICENSES[GPLv3]="http://www.gnu.org/licenses/gpl-3.0.md"
30
+MKIT_STUB_LICENSES[LGPLv2]="http://www.gnu.org/licenses/old-licenses/lgpl-2.1.md"
31
+MKIT_STUB_LICENSES[LGPLv3]="http://www.gnu.org/licenses/lgpl-3.0.md"
32
+MKIT_STUB_LICENSES[AGPLv3]="http://www.gnu.org/licenses/agpl-3.0.md"
33
+MKIT_STUB_LICENSES[FDLv1.3]="http://www.gnu.org/licenses/fdl-1.3.md"
34
+MKIT_STUB_LICENSES[FDLv1.2]="http://www.gnu.org/licenses/old-licenses/fdl-1.2.md"
35
+MKIT_STUB_LICENSES[FDLv1.1]="http://www.gnu.org/licenses/old-licenses/fdl-1.1.md"
36
+
37
+
38
+deploy() {
39
+    local file=$1       # which known file?
40
+    local any_name=${NiceName:-$PackageName}
41
+    local tmp
42
+    tmp=$(mktemp -t mkit.stub.deploy.XXXXXXX)
43
+    mkdir -p "$(dirname "$file")"
44
+    case $file in
45
+
46
+        Makefile)
47
+            echo -n "# $any_name"
48
+            test -n "$Tagline" && echo -n " - $Tagline"
49
+            echo
50
+            $MkLicense && echo '# See LICENSE.md file for copyright and license details.'
51
+            echo ''
52
+            echo 'MKIT_DIR=utils/mkit'
53
+            #shellcheck disable=SC2016
54
+            echo 'include $(MKIT_DIR)/mkit.mk'
55
+            ;;
56
+
57
+        README.md)
58
+            echo "$any_name"
59
+            tr -c '=\n' '=' <<<"$any_name"
60
+            echo ''
61
+            echo ''
62
+            if test -n "$Tagline"; then
63
+                echo "$Tagline"
64
+            else
65
+                echo "(Nothing to say about this project.)"
66
+            fi
67
+            ;;
68
+
69
+        */mkit.ini|mkit.ini)
70
+            echo "[project]"
71
+            echo "    version     = $Version"
72
+            test -n "$Codename"     && echo "    codename    = $Codename"
73
+            test -n "$NiceName"     && echo "    name        = $NiceName"
74
+            test -n "$Tagline"      && echo "    tagline     = $Tagline"
75
+            test -n "$PackageName"  && echo "    pkgname     = $PackageName"
76
+            test -n "$Maintainer"   && echo "    maintainer  = $Maintainer"
77
+            test -n "$VcsBrowser"   && echo "    vcs_browser = $VcsBrowser"
78
+            test -n "$RelSrc"       && echo "    relsrc      = $RelSrc"
79
+            test -n "$RelDst"       && echo "    reldst      = $RelDst"
80
+            if updating; then
81
+                remake_section dist
82
+                remake_section ENV
83
+                remake_section roots
84
+                remake_section tokens
85
+                remake_section modes
86
+                remake_section files
87
+            else
88
+                echo ""
89
+                echo "[dist]"
90
+                $MkLicense  && echo "    tarball = LICENSE.md"
91
+                $MkMakefile && echo "    tarball = Makefile"
92
+                $MkReadme   && echo "    tarball = README.md"
93
+                echo "    tarball = mkit.ini"
94
+                $MkPackaging && echo "    tarball = packaging"
95
+                echo "    tarball = src"
96
+                echo "    tarball = tests"
97
+                echo "    tarball = utils"
98
+                $MkPackaging && echo "    rpmstuff = packaging/template.spec"
99
+                $MkPackaging && echo "    debstuff = packaging/debian"
100
+                echo ""
101
+                echo "[ENV]"
102
+                echo "    PREFIX = /usr/local"
103
+                echo ""
104
+                echo "[roots]"
105
+                echo "    bin     = [ENV:PREFIX]/bin"
106
+                echo "    doc     = [ENV:PREFIX]/share/doc/$PackageName"
107
+                echo ""
108
+                echo "[tokens]"
109
+                echo "    __${PackageName^^}_FOO__ = Barr.."
110
+                echo ""
111
+                echo "[modes]"
112
+                echo "    bin     = 755"
113
+                echo "    doc     = 644"
114
+                echo ""
115
+                echo "[files]"
116
+                echo "    bin      = src/$PackageName"
117
+                $MkLicense && echo "    doc      = LICENSE.md"
118
+                $MkReadme  && echo "    doc      = README.md"
119
+            fi
120
+            echo ""
121
+            echo "#mkit version=$MKIT_VERSION"
122
+            ;;
123
+
124
+        packaging/template.spec)
125
+            echo 'Name:		__MKIT_PROJ_PKGNAME__'
126
+            echo 'Version:	__MKIT_PROJ_VERSION__'
127
+            echo 'Release:	1%{?dist}'
128
+            echo 'Summary:	__MKIT_PROJ_NAME__ - __MKIT_PROJ_TAGLINE__'
129
+            echo ''
130
+            $MkLicense && echo "License:	$License"
131
+            echo 'Source0:	%{name}-%{version}.tar.gz'
132
+            echo ''
133
+            echo 'BuildArch:	noarch'
134
+            echo 'BuildRequires:	coreutils git'
135
+            echo 'Requires:	MKIT_STUB_REQUIRES'
136
+            echo ''
137
+            echo '%description'
138
+            echo 'MKIT_STUB_DESCRIPTION'
139
+            echo ''
140
+            echo '%prep'
141
+            echo '%setup -q'
142
+            echo ''
143
+            echo '%build'
144
+            echo 'make %{?_smp_mflags} PREFIX=/usr'
145
+            echo ''
146
+            echo '%install'
147
+            echo '%make_install PREFIX=/usr'
148
+            echo ''
149
+            echo '%files'
150
+            echo 'MKIT_STUB_FILELIST'
151
+            echo ''
152
+            echo '%changelog'
153
+            echo ''
154
+            echo '# specfile built with MKit __MKIT_SELF_VERSION__'
155
+            ;;
156
+
157
+        packaging/debian/copyright)
158
+            echo ""
159
+            ;;
160
+
161
+        packaging/debian/control)
162
+            echo 'Source: __MKIT_PROJ_PKGNAME__'
163
+            echo 'Maintainer: __MKIT_PROJ_MAINTAINER__'
164
+            test -n "$VcsBrowser" && echo 'Vcs-Browser: __MKIT_PROJ_VCS_BROWSER__'
165
+            echo 'Section: misc'
166
+            echo 'Priority: extra'
167
+            echo 'Standards-Version: 3.9.2'
168
+            echo 'Build-Depends: debhelper (>= 9)'
169
+            echo ''
170
+            echo 'Package: __MKIT_PROJ_PKGNAME__'
171
+            echo 'Architecture: all'
172
+            echo 'Depends: MKIT_STUB_REQUIRES'
173
+            echo 'Description: __MKIT_PROJ_NAME__ - __MKIT_PROJ_TAGLINE__'
174
+            echo ' MKIT_STUB_DESCRIPTION'
175
+            echo ''
176
+            echo '# control file built with MKit __MKIT_SELF_VERSION__'
177
+            ;;
178
+
179
+        packaging/debian/changelog)
180
+            echo '__MKIT_PROJ_PKGNAME__ (__MKIT_PROJ_VERSION__-1) UNRELEASED; urgency=medium'
181
+            echo ''
182
+            echo '  * Initial release. (Closes: #XXXXXX)'
183
+            echo ''
184
+            echo " -- __MKIT_PROJ_MAINTAINER__  $(date -R)"
185
+            ;;
186
+
187
+        packaging/debian/compat)
188
+            echo 9
189
+            ;;
190
+
191
+        packaging/debian/rules)
192
+            echo '#!/usr/bin/make -f'
193
+            echo ''
194
+            echo '%:'
195
+            echo ''
196
+            echo '	dh $@'
197
+            echo ''
198
+            echo 'override_dh_auto_install:'
199
+            echo ''
200
+            echo '	make install PREFIX=/usr DESTDIR=debian/tmp'
201
+            echo ''
202
+            echo 'override_dh_usrlocal:'
203
+            echo ''
204
+            echo '	@true'
205
+            ;;
206
+
207
+        packaging/debian/source/format)
208
+            echo '3.0 (quilt)'
209
+            ;;
210
+
211
+        packaging/debian/*.install)
212
+            echo MKIT_STUB_FILELIST
213
+            ;;
214
+
215
+        src/*.skel)
216
+            echo 'echo "my version is: __MKIT_PROJ_VERSION__"'
217
+            echo 'echo "And that'"'"'s all, folks!"'
218
+            ;;
219
+
220
+        LICENSE.md)
221
+            local url   # license URL
222
+            url="${MKIT_STUB_LICENSES[$License]}"
223
+            curl -sf "$url" \
224
+             || die "failed to download license: $url"
225
+            ;;
226
+
227
+        .mkit/autoclean)
228
+            ;;
229
+
230
+        MKIT_STUB_README.md)
231
+            echo "FINISHING MKIT CONFIGURATION"
232
+            echo "============================"
233
+            echo ""
234
+            echo "Congratulations, your new project has been configured!"
235
+            echo ""
236
+            echo "However, the *stub* script is not able to figure out"
237
+            echo "everything, so few things still need to be done manually."
238
+            echo "This document will guide you throught the rest of the"
239
+            echo "process."
240
+
241
+            echo ""
242
+            echo ""
243
+            echo "Structure"
244
+            echo "---------"
245
+            echo ""
246
+            echo "First, let's go through the directory structure:"
247
+            echo ""
248
+            echo " *  *src* directory - here is your main place to store"
249
+            echo "    source files.  This includes also documents like user"
250
+            echo "    manuals---IOW, anything intended to end up on user's"
251
+            echo "    machine should be uder 'src'."
252
+            echo ""
253
+            echo "    Note that during build time, files named ending with"
254
+            echo "    '.skel' are subject to token expansion, see mkit.ini"
255
+            echo "    section below for details."
256
+            echo ""
257
+            echo " *  *notes* directory - here you shall store notes"
258
+            echo "    intended for people contributing to your project,"
259
+            echo "    for instance, guidelines, coding style documents,"
260
+            echo "    TODOs, ideas, plans..."
261
+            echo ""
262
+            echo " *  *utils* directory - here you shall store utilities"
263
+            echo "    and scripts that will help you with project maintenance,"
264
+            echo "    and that, unlike software like compilers or versioning"
265
+            echo "    systems, can (and should) be embedded inside the"
266
+            echo "    repository."
267
+            echo ""
268
+            echo "    MKit itself is one nice example. :)"
269
+
270
+            if $MkPackaging; then
271
+            echo ""
272
+            echo " *  *packaging* directory contains templates that enable"
273
+            echo "    MKit create raw stuffs used to create DEB or RPM"
274
+            echo "    packages.  Similar to '.skel' files in 'src', all files"
275
+            echo "    here are automatically considered for token expansion,"
276
+            echo "    no matter how they are named (see mkit.ini section"
277
+            echo "    below)."
278
+            echo ""
279
+            echo "    NOTE: these templates, as well as any packages created by"
280
+            echo "    them are intended only for experimental, private use and"
281
+            echo "    testing."
282
+            echo ""
283
+            echo "    Should you have ambition to create 'real' packages for"
284
+            echo "    OS distribution such as Debian or Fedora (what a great"
285
+            echo "    idea!), be prepared that you will need to follow their"
286
+            echo "    guidelines.  This will most probably mean huge changes"
287
+            echo "    to these packages or even changes to your workflow."
288
+
289
+            echo ""
290
+            echo ""
291
+            echo "Placeholders"
292
+            echo "------------"
293
+            echo ""
294
+            echo "At places where *stub* script did not have way to get all"
295
+            echo "information automatically, it has inserted placeholders."
296
+            echo "You will need to go through all of these placeholders and"
297
+            echo "replace them with proper data."
298
+            echo ""
299
+            echo "Please follow instructions:"
300
+            echo ""
301
+            echo " 1. Look for placeholders starting with \`MKIT_STUB_\`"
302
+            echo "    prefix by calling this command:"
303
+            echo ""
304
+            echo "        grep -l MKIT_STUB_ -r"
305
+            echo ""
306
+            echo " 2. Go through each file and locate the placeholder.  (You"
307
+            echo "    will also see placeholders like \`__MKIT_*__\`, you can"
308
+            echo "    ignore those."
309
+            echo ""
310
+            echo " 3. Replace placeholder with appropriate information:"
311
+            echo ""
312
+            echo "     *  \`MKIT_STUB_REQUIRES\` - Requirements of your"
313
+            echo "        project."
314
+            echo ""
315
+            echo "     *  \`MKIT_STUB_DESCRIPTION\` - Description of your"
316
+            echo "        project (few sentences to paragraphs)."
317
+            echo ""
318
+            echo "     *  \`MKIT_STUB_FILELIST\` - List of full paths to"
319
+            echo "        your files after installation."
320
+            echo ""
321
+            echo "        Note that in case of debian/package.install files,"
322
+            echo "        PREFIX based paths (eg. /usr/bin) in this file should"
323
+            echo "        should be as if PREFIX was /usr."
324
+            echo ""
325
+            echo "        In case of Fedora-based distro, you should make use"
326
+            echo "        of RPM macros:"
327
+            echo ""
328
+            echo "            https://fedoraproject.org/wiki/Packaging:RPMMacros"
329
+            echo ""
330
+            echo "    Refer to these documents for further details:"
331
+            echo ""
332
+            echo "        http://rpm-guide.readthedocs.io/"
333
+            echo "        https://www.debian.org/doc/manuals/maint-guide/"
334
+            fi
335
+
336
+            echo ""
337
+            echo ""
338
+            echo "mkit.ini"
339
+            echo "--------"
340
+            echo ""
341
+            echo "Most sections still need to be reviewed. In order to do"
342
+            echo "that, you will need to understand how MKit works:"
343
+            echo ""
344
+            echo " 1. First, you need to define *roles* your files will play"
345
+            echo "    when they are installed on user's file systems.  These"
346
+            echo "    roles then imply where and how the files should be"
347
+            echo "    deployed."
348
+            echo ""
349
+            echo "    Typical example of a role is e.g. 'bin' for commands"
350
+            echo "    (normally under '/usr/bin' or '/usr/local/bin'), 'doc'"
351
+            echo "    for documents or 'lib' for libraries."
352
+
353
+            echo ""
354
+            echo " 2. Next, in \`[roots]\` section, you have to set target"
355
+            echo "    root directory for each role.  However, in order to"
356
+            echo "    enable people to implement local conventions, it is"
357
+            echo "    considered a good manner to also respect PREFIX"
358
+            echo "    environment variable.  For this reason, most paths"
359
+            echo "    need to start with \`[ENV:PREFIX]\`."
360
+            echo ""
361
+
362
+            echo " 3. \`[files]\` section is where you assign actual files"
363
+            echo "    from your repository to their final paths.  The format"
364
+            echo "    is \`ROLE = REPOPATH [RENAMED]\`, where ROLE is file's"
365
+            echo "    role, REPOPATH is relative path to the file."
366
+            echo ""
367
+            echo "    Final path is then composed by taking path assigned to"
368
+            echo "    ROLE and appending file's name.  However, if you need"
369
+            echo "    the deployed file to have different name than in the"
370
+            echo "    codebase, you can specify the other name as RENAMED."
371
+            echo ""
372
+            echo "    Note that you don't need to address every single file"
373
+            echo "    individually, if in your repo you have a directory with"
374
+            echo "    100 files of the same role, you can add just path to the"
375
+            echo "    directory itself."
376
+
377
+            echo ""
378
+            echo " 4. If some roles require special permissions on your files,"
379
+            echo "    \`[modes]\` section is your friend.  Permissions here"
380
+            echo "    should be in UNIX octal format."
381
+
382
+            echo ""
383
+            echo " 5. Next, \`[tokens]\` section allows you to define own"
384
+            echo "    placeholders that will be replaced when your scripts are"
385
+            echo "    built.  Each file in 'src' directory that is named with"
386
+            echo "    '.skel' suffix, and each file from 'packaging' directory"
387
+            echo "    (no matter its name), can contain one or more of tokens"
388
+            echo "    defined here, plus range of tokens automatically defined"
389
+            echo "    by MKit.  During build, these tokens are replaced with"
390
+            echo "    their actual values."
391
+
392
+            echo ""
393
+            echo " 6. Less interesting, but important section is \`[dist]\`,"
394
+            echo "    which lists files in your codebase that will be added"
395
+            echo "    to distribution tarball (part of \"stuffs\" mentioned"
396
+            echo "    above).  Listing directory here will include all its"
397
+            echo "    contents, and normally it's OK to be very inclusive, so"
398
+            echo "    most of the time this section should be OK."
399
+
400
+            echo ""
401
+            echo " 7. Even less interesting is section \`[ENV]\`.  It is"
402
+            echo "    special in that it provides *default* value for an"
403
+            echo "    environment variable.  You almost never need to touch"
404
+            echo "    this."
405
+            echo ""
406
+
407
+            echo " 8. Finally, the most interesting section!  \`[project]\`,"
408
+            echo "    provides most general information for your project such"
409
+            echo "    as name and version."
410
+            echo ""
411
+            echo "    Note that the \`version\` key is another \"special"
412
+            echo "    snowflake\" -- it is now set to 0.0.0, and you **should"
413
+            echo "    not need** to change  it manually.  When you feel you"
414
+            echo "    a are ready to release next version of your evolving"
415
+            echo "    project, simply call \`make vbump\` and MKit will take"
416
+            echo "    care of everything!"
417
+
418
+            if $MkMakefile; then
419
+            echo ""
420
+            echo ""
421
+            echo "Makefile"
422
+            echo "--------"
423
+            echo ""
424
+            echo "*stub* script also created a Makefile for you, but all"
425
+            echo "it really does is include MKit's own mkit.mk, which in turn"
426
+            echo "only maps \`make\` targets to actual mkit script calls."
427
+            echo "Unless your project already uses GNU Make, you should not"
428
+            echo "need to change this file."
429
+            fi
430
+
431
+            if $MkReadme; then
432
+            echo ""
433
+            echo ""
434
+            echo "README.md"
435
+            echo "---------"
436
+            echo ""
437
+            echo "Each serious project needs a serious README.  Which is why"
438
+            echo "*stub* has created a 'stub' of one for you."
439
+            fi
440
+
441
+            echo ""
442
+            echo ""
443
+            echo "The final touch"
444
+            echo "---------------"
445
+            echo ""
446
+            echo "Once you have reviewed everything, just delete this file!"
447
+            ;;
448
+
449
+    esac >"$tmp"
450
+    cat "$tmp" > "$file"
451
+    rm "$tmp"
452
+}
453
+
454
+known_licenses() {
455
+    local key
456
+    local first=true
457
+    for key in "${!MKIT_STUB_LICENSES[@]}"; do
458
+        $first && echo "$key"  && continue
459
+        echo ", $key"
460
+    done
461
+}
462
+
463
+usage() {
464
+    {
465
+        echo "Usage:"
466
+        echo "   stub [options] new PKGNAME"
467
+        echo "   stub [options] update"
468
+        echo "   stub -L"
469
+        echo ""
470
+        echo "Options:"
471
+        echo ""
472
+        echo "    -c CODENAME   your project codename"
473
+        echo "    -t TAGLINE    your project tagline"
474
+        echo "    -b RELSRC     pre-release branch"
475
+        echo "    -B RELDST     post-release branch"
476
+        echo "    -n NICENAME   your project's 'nice' name"
477
+        echo "    -l LICENSE    your options's license (see -L)"
478
+        echo "    -m MAINT      project maintainer's name and e-mail"
479
+        echo "    -v URL        URL to public code browser"
480
+        echo "    -V VERSION    initial version (default: 0.0.0)"
481
+        echo "    -a            enable autoclean ('make clean' after"
482
+        echo "                  each 'make install')"
483
+        echo "    -g            make git commits before and adter"
484
+        echo "                  (implies -y)"
485
+        echo "    -y            don't ask, just do it"
486
+        echo "    -R            skip creating README.md"
487
+        echo "    -M            skip creating Makefile"
488
+        echo "    -P            skip creating packaging templates"
489
+        echo "    -L            list know licenses and exit"
490
+        echo ""
491
+        echo "PKGNAME should be packaging-friendly name, ie. consist"
492
+        echo "only of small letters, numbers, underscore and dash."
493
+        echo "For your 'real' name, use NICENAME, which can be any"
494
+        echo "string."
495
+    } >&2
496
+    exit 2
497
+}
498
+
499
+confirm() {
500
+    local response      # user's response to our warning
501
+    $Force && return 0
502
+    {
503
+        echo "Warning: This operation can be destructive for your"
504
+        echo "current codebase.  At least following files will be"
505
+        echo "created or overwritten:"
506
+        echo ""
507
+        $MkPackaging    && echo " *  'packaging' directory (pass -P to avoid)"
508
+        $MkMakefile     && echo " *  'Makefile' (pass -M to avoid)"
509
+        $MkReadme       && echo " *  'README.md' (pass -R to avoid)"
510
+        $MkLicense      && echo " *  'LICENSE.md' (omit -l to avoid)"
511
+        echo " *  'mkit.ini'"
512
+        echo ""
513
+        read -p "Type 'yes' to proceed: " -r response
514
+    } >&2
515
+    test "$response" == "yes" && return 0
516
+    warn "Aborting."
517
+    return 1
518
+}
519
+
520
+mkcommit_backup() {
521
+    git ls-files --others \
522
+      | grep -qv -e '^utils/mkit$' -e '^utils/mkit/' \
523
+     || { warn "nothing to back up"; return 0; }
524
+    git add .                           || return
525
+    git rm -r --cached utils/mkit       || return
526
+    git commit -m "WIP [mkit/stub] backup" || return
527
+}
528
+
529
+mkcommit_mkit_code() {
530
+    git ls-files --others \
531
+      | grep -q -e '^utils/mkit$' -e '^utils/mkit/' \
532
+     || return 0
533
+    git add utils/mkit || return
534
+    git commit -m "WIP [mkit/stub] Add MKit version v$MKIT_VERSION" || return
535
+}
536
+
537
+mkcommit_mkit_conf() {
538
+    local msg       # commit message (the important art
539
+    git add . || return
540
+    case $Action in
541
+        new)    msg="Add MKit configuration stub" ;;
542
+        update) msg="Update MKit configuration"   ;;
543
+    esac
544
+    git commit -m "WIP [mkit/stub] $msg" || return
545
+}
546
+
547
+deploy_packaging() {
548
+    rm -rf packaging
549
+    deploy packaging/template.spec
550
+    deploy packaging/debian/copyright
551
+    deploy packaging/debian/control
552
+    deploy packaging/debian/changelog
553
+    deploy packaging/debian/compat
554
+    deploy packaging/debian/rules
555
+    deploy packaging/debian/source/format
556
+    deploy packaging/debian/"$PackageName".install
557
+}
558
+
559
+init_from_existing() {
560
+    #
561
+    # Initialize variables from old mkit.ini
562
+    #
563
+    test -f "$MKIT_INI" \
564
+     || die "mkit.ini not found; aborting update: $MKIT_INI"
565
+    user_gave Codename    || Codename=$(ini_raw1v project:codename)
566
+    user_gave License     || License=$(ini_raw1v project:license)
567
+    user_gave Maintainer  || Maintainer=$(ini_raw1v project:maintainer)
568
+    user_gave NiceName    || NiceName=$(ini_raw1v project:name)
569
+    user_gave PackageName || PackageName=$(ini_raw1v project:pkgname)
570
+    user_gave RelDst      || RelDst=$(ini_raw1v project:reldst)
571
+    user_gave RelSrc      || RelSrc=$(ini_raw1v project:relsrc)
572
+    user_gave Tagline     || Tagline=$(ini_raw1v project:tagline)
573
+    user_gave VcsBrowser  || VcsBrowser=$(ini_raw1v project:vcs_browser)
574
+    user_gave Version     || Version=$(ini_raw1v project:version)
575
+}
576
+
577
+ini_raw1v() {
578
+    #
579
+    # Read raw scalar from mkit.ini
580
+    #
581
+    local path=$1
582
+    MKIT_INI_EXPAND=0 ini 1value "$path"
583
+}
584
+
585
+user_gave() {
586
+    #
587
+    # True if user gave value to variable $1
588
+    #
589
+    local var=$1    # name of the variable
590
+    { test "${UserGave[$var]}" == 1; } 2>/dev/null
591
+}
592
+
593
+updating() {
594
+    #
595
+    # Are we updating?
596
+    #
597
+    test "$Action" == update
598
+}
599
+
600
+remake_section() {
601
+    #
602
+    # Re-compose mkit.ini section $1
603
+    #
604
+    local section=$1
605
+    local key
606
+    local value
607
+    ini lskeys "$section" | grep -q . \
608
+     || return 1
609
+    echo ""
610
+    echo "[$section]"
611
+    ini lskeys "$section" \
612
+      | while read -r key; do
613
+            MKIT_INI_EXPAND=0 ini values "$section:$key" \
614
+              | while read -r value; do
615
+                    echo "$key = $value"
616
+                done
617
+        done \
618
+      | column -to' ' \
619
+      | sed 's/^/    /'
620
+}
621
+
622
+main() {
623
+    local NiceName          # human-readable project name
624
+    local PackageName       # machine-safe project (package) name
625
+    local RelSrc            # pre-release branch
626
+    local RelDst            # post-release branch
627
+    local Codename          # release codename
628
+    local Tagline           # project tagline
629
+    local Maintainer        # project maintainer (Name + e-mail)
630
+    local VcsBrowser        # VCS browser (eg. GitHub URL)
631
+    local Version=0.0.0     # project version
632
+    local AutoClean=false   # touch .mkit/autoclean?
633
+    local MkCommits=false   # create pre/post git commits?
634
+    local Force=false       # go without asking?
635
+    local MkReadme=true     # create README.md?
636
+    local MkMakefile=true   # create Makefile?
637
+    local MkPackaging=true  # create packaging templates?
638
+    local MkLicense=false   # create LICENSE.md file
639
+    local Action            # 'update' to respect existing, 'new' to force
640
+                            # rewrite incl. MKIT_STUB_* placeholders
641
+    declare -A UserGave
642
+    while true; do case $1 in
643
+        -n) NiceName=$2;   UserGave[NiceName]=1;   shift 2 || usage ;;
644
+        -b) RelSrc=$2;     UserGave[RelSrc]=1;     shift 2 || usage ;;
645
+        -B) RelDst=$2;     UserGave[RelDst]=1;     shift 2 || usage ;;
646
+        -c) Codename=$2;   UserGave[Codename]=1;   shift 2 || usage ;;
647
+        -t) Tagline=$2;    UserGave[Tagline]=1;    shift 2 || usage ;;
648
+        -l) License=$2;    UserGave[License]=1;    shift 2 || usage ;;
649
+        -m) Maintainer=$2; UserGave[Maintainer]=1; shift 2 || usage ;;
650
+        -v) VcsBrowser=$2; UserGave[VcsBrowser]=1; shift 2 || usage ;;
651
+        -V) Version=$2;    UserGave[Version]=1;    shift 2 || usage ;;
652
+        -M) MkMakefile=false;   shift ;;
653
+        -R) MkReadme=false;     shift ;;
654
+        -a) AutoClean=true;     shift ;;
655
+        -y) Force=true;         shift ;;
656
+        -g) MkCommits=true;     shift ;;
657
+        -P) MkPackaging=false;  shift ;;
658
+        -L) known_licenses | tr , '\n'; exit 0 ;;
659
+        -*) usage ;;
660
+        *)  break ;;
661
+    esac done
662
+    Action=$1; PackageName=$2
663
+    case $Action:$PackageName in
664
+        new:)     usage ;;
665
+        new:*)    :     ;;
666
+        update:)  :     ;;
667
+        update:*) usage ;;
668
+        *)        usage ;;
669
+    esac
670
+    updating && init_from_existing
671
+    if test -n "$License"; then
672
+        known_licenses | grep -qxF "$License" \
673
+         || die "unknown license (use -L to get list): $License"
674
+        MkLicense=true
675
+    fi
676
+    if $MkCommits; then
677
+        mkcommit_backup || die "failed creating backup commit"
678
+        Force=true
679
+    fi
680
+    confirm            || return 1
681
+    deploy "$MKIT_INI"
682
+    deploy src/"$PackageName".skel
683
+    $MkMakefile     && deploy Makefile
684
+    $MkReadme       && deploy README.md
685
+    $MkLicense      && deploy LICENSE.md
686
+    $AutoClean      && deploy .mkit/autoclean
687
+    $MkPackaging    && deploy_packaging
688
+    if $MkCommits; then
689
+        mkcommit_mkit_code || die "failed creating post-commit"
690
+        mkcommit_mkit_conf || die "failed creating post-commit"
691
+    fi
692
+    deploy MKIT_STUB_README.md
693
+    warn "Configuration stub built, follow instructions in"
694
+    warn "MKIT_STUB_README.md to finish configuration."
695
+    return 0
696
+}
697
+
698
+main "$@"

+ 389
- 0
utils/tfkit/doc/README.md View File

@@ -0,0 +1,389 @@
1
+Tests
2
+=====
3
+
4
+Running tests is handled by tfkit/runtests:
5
+
6
+    $ tfkit/runtest [filter]
7
+
8
+*filter* is a regular expression to be applied to sub-test name, running
9
+only the matching ones.  See below for details.
10
+
11
+
12
+Writing tests
13
+-------------
14
+
15
+Tests can be written in any scripting language, although the built-in
16
+framework, written in Bash, provides some useful features for writing
17
+certain kind of relatively simple tests.
18
+
19
+The harness, though, assumes that:
20
+
21
+ *  Any direct sub-directory of `$TF_SUITE` directory ("tests" by default)
22
+    that contains at least *TF_RUN* executable becomes a test,
23
+
24
+ *  basename of this directory becomes the name of the test,
25
+
26
+ *  and return code from running the executable is reported
27
+    as result of the test, according to "Exit status" section below.
28
+
29
+
30
+Naming
31
+------
32
+
33
+Test name should start with name of the module that is tested and
34
+underscore.  If module name contains dots, they should be replaced with
35
+underscores as well.
36
+
37
+    core_sanity
38
+    mod_submod_function
39
+    ini_iniread
40
+
41
+are valid test names.
42
+
43
+
44
+Data
45
+----
46
+
47
+Should the test need any data, just leave it around in the test directory
48
+along with *TF_RUN*.
49
+
50
+Note that before running, the whole test directory is automatically
51
+copied to a temporary location (one per test), and should the test fail,
52
+copied back as a debugging artifact.  For this reason, *do not store
53
+huge amounts of data here*.  If you really need huge data, consider
54
+obtaining it (and throwing it away) within runtime of *TF_RUN*.
55
+
56
+
57
+Exit status
58
+-----------
59
+
60
+We try hard to follow this semantic:
61
+
62
+ *  *Zero* means *OK* -- test has been run and passed.
63
+
64
+ *  *One* means *Failure* -- test has been run but failed (e.g. found
65
+     a bug).
66
+
67
+ *  *Two* means *Bailout* --  test has decided not to run at all.
68
+
69
+ *  *Three* means *Error* -- there was error detected during execution,
70
+     but script was able to clean up properly.
71
+
72
+ *  *Four* means *Panic* -- there was other error but script *was not*
73
+     able to clean up properly.
74
+
75
+ *  Anything else should indicate other uncaught errors, including those
76
+    outside control of the program such as segfaults in the test code
77
+    or test being SIGKILLed.
78
+
79
+Notice that the higher the value is, the worse situation it indicates.
80
+Thus, if a test is composed of several sub-tests, you need to make sure
81
+to always **exit with the highest value** (subtest.sh does take care
82
+of this).
83
+
84
+See *common.sh* for functions and variables to help with handling exit
85
+statuses with this semantic.
86
+
87
+Also see Notes section for more details on exit statuses, including
88
+cheat sheet and dscussuion.
89
+
90
+
91
+Framework
92
+---------
93
+
94
+
95
+### harness.sh ###
96
+
97
+This part is not intended to be used in tests, but rather contains
98
+functions that help govern test discovery, preparation and execution as
99
+is described in previous sections.  Feel free to poke around, of course.
100
+
101
+
102
+### subtest.sh ###
103
+
104
+As name suggests, this file defines few functions to handle subtests
105
+in *TF_RUN*.
106
+
107
+In order to make use of the subtests functionality, you will need to
108
+define two functions yourself:  `tf_enum_subtests` to enumerate names of
109
+tests you want to run, and `tf_do_subtest` with actual test
110
+implementation.
111
+
112
+The minimal *TF_RUN* with two subtests could look like this:
113
+
114
+    #!/bin/bash
115
+
116
+    . $TF_DIR/include/subtest.sh
117
+
118
+    tf_enum_subtests() {
119
+        echo test1
120
+        echo test2
121
+        something && echo test3
122
+    }
123
+
124
+    tf_do_subtest() {
125
+        case $1 in
126
+            test1)  myprog foo ;;
127
+            test2)  myprog bar ;;
128
+            test3)  myprog baz ;;
129
+        esac
130
+    }
131
+
132
+    tf_do_subtests
133
+
134
+At the end, `tf_do_subtests` acts as a launcher of the actual test.
135
+In short, it will
136
+
137
+ 1. run `tf_enum_subtests`, taking each line as name of a subtest;
138
+    for each subtest:
139
+
140
+     1. source *TF_SETUP*, if such file is found,
141
+     2. launch the `tf_do_subtest()` function with subtest name as
142
+        the only argument,
143
+     3. source *TF_CLEANUP*, if such file is found,
144
+
145
+ 2. and finally, report "worst" exit status encountered.
146
+
147
+Note that subtest names need to be single words (`[a-zA-Z0-9_]`).
148
+
149
+
150
+### tools.sh ###
151
+
152
+This file contains various tools and utilities to help with testing.
153
+
154
+Curently there is only one function, `tf_testflt` designed to help write
155
+tests for simple unix filters.
156
+
157
+
158
+#### tf_testflt ####
159
+
160
+The idea is that tester specifies
161
+
162
+ *  test name,
163
+ *  command to launch the system under test,
164
+ *  a data stream to use as STDIN,
165
+ *  and expected STDOUT, STDERR, and exit status.
166
+
167
+and tf_testflt launches the command, collects tha data and evaluates
168
+and reports the result using unified diff.
169
+
170
+In its simplest form:
171
+
172
+    tf_testflt -n foo my_command arg
173
+
174
+the function will run `my_command arg` (not piping anything to it),
175
+and will expect it to finish with exit status 0 and empty both STDERR
176
+and STDOUT.
177
+
178
+Example of full form,
179
+
180
+    tf_testflt -n foo -i foo.in -O foo.stdout -E foo.stderr -S 2 myprog
181
+
182
+will pipe foo.in into `myprog`, expecting exit status of 2, and STDOUT and
183
+STDERR as above.  Notice that parameters specifying expected values are
184
+uppercase, and those specifying input values are lowercase.
185
+
186
+Specifying name is mandatory, because it's used in reporting messages,
187
+and as a basis for naming temporary result files: these are saved in
188
+*results* subdirectory and kept for further reference.
189
+
190
+
191
+### common.sh ###
192
+
193
+This includes simple functions and variables shared between both mentioned
194
+libraries.
195
+
196
+First group is designed to help support the exit status semantic:
197
+
198
+ *  The functions are `tf_exit_pass`, `tf_exit_fail`, `tf_exit_bailout`,
199
+    `tf_exit_error` and `tf_exit_panic` and each take any number of
200
+    parameters that are printed on stderr.
201
+
202
+ *  The variables are `TF_ES_OK`, `TF_ES_FAIL`, `TF_ES_BAILOUT`,
203
+    `TF_ES_ERROR` and `TF_ES_PANIC` and are supposed to be used with
204
+    `return` builtin, e.g. to return from `tf_exit_error`.
205
+
206
+Second group is useful to better control output:  functions `tf_warn`,
207
+`tf_debug` and `tf_think` are used to print stuff on STDERR.  Use of
208
+`tf_warn` is apparent, just as `tf_debug`, the latter being muted if
209
+`TF_DEBUG` is set to `false` (set it to `true` to turn on debugging).
210
+
211
+`tf_think` is used for progress info, and is muted unless `TF_VERBOSE`
212
+is set to `true`, which is by default.
213
+
214
+
215
+### Setup and cleanup ###
216
+
217
+Special files *TF_SETUP* and *TF_CLEANUP* (one of them or both) can be
218
+added along with *TF_RUN*.  These will be sourced before (*TF_SETUP*)
219
+and after every subtest (*TF_CLEANUP*).
220
+
221
+First, if any of these files are missing, it is considered as if the
222
+respective phase succeeded.  Second, if setup phase fails, test will
223
+be skipped and subtest exit status will be *TF_ES_BAILOUT*.   Last,
224
+if cleanup fails (no matter result of setup), subtests aborts with
225
+*TF_ES_PANIC* returned.  Be aware that in this case the actual test
226
+status, albeit useful, is lost.
227
+
228
+When coming from other test frameworks, this may feel harsh, but note
229
+that this has been designed with the idea that if a cleanup fails,
230
+it may render all further tests are automatically unsafe, because the
231
+environment is not as expected.
232
+
233
+To cope with this behavior, try to bear in mind following advice:
234
+
235
+ 1. Make sure you write setup/cleanup procedures with extreme care and
236
+    test them well.
237
+
238
+ 2. Do not do complicated and risky things in the setup/cleanup phases.
239
+
240
+ 3. If you need to do such things, consider doing them in the *TF_RUN*
241
+    instead of doing them for all subtests.
242
+
243
+ 4. You don't need to clean up everything, the contents of the testing dir
244
+    will be moved out from the test system.
245
+
246
+ 5. If there are scenarios you can safely fix or ignore, handle them in
247
+    a robust manner.
248
+
249
+
250
+Notes
251
+-----
252
+
253
+
254
+### bailout vs. `tf_enum_subtests` ###
255
+
256
+One more note to claify relation of bailout and `tf_enum_subtests`.
257
+As you may have noticed, there are two ways how to skip a test:
258
+return prematurely with `TF_ES_BAILOUT`, or suppress enumeration in
259
+`tf_enum_subtests`.  The problem is that the latter does not do anything
260
+to inform upper in the stack that a test has been skipped, which seems to
261
+break the principle described in previous sections.
262
+
263
+Don't confuse these mechanisms, though. Each is supposed to be used
264
+for distinct purpose.  Compare: by using the `tf_enum_subtests` you are
265
+saying that you actually **did not even want** to run the test in the
266
+first place.  By using `TF_ES_BAILOUT`, you are saying that you **wanted**
267
+to run the test but could not.
268
+
269
+A few common cases if that helps you:
270
+
271
+ *  If during the test you find out that for some reason it can't be
272
+    carried out (e.g. an external resource is not available, or
273
+    something outside the SUT is broken), use `TF_ES_BAILOUT`.
274
+
275
+        tf_enum_subtests() {
276
+            echo test1
277
+            echo test2
278
+            echo test3
279
+        }
280
+
281
+        tf_do_subtest() {
282
+            case $1 in
283
+                test1) do_stuff  ;;
284
+                test2) do_other_stuff ;;
285
+                test3) curl -s http://www.example.com/ >file \
286
+                        || return $TF_ES_BAILOUT
287
+                       do_stuff_with file ;;
288
+            esac
289
+        }
290
+
291
+ *  If you want to filter out some sub-tests for some platforms, e.g. a
292
+    test for only 64-bit architectures, or a test only for Mac OS (IOW,
293
+    you can safely say that running this sub-test would be totally
294
+    pointless on this box), use `tf_enum_subtests`--just omit this test
295
+    from enumeration.
296
+
297
+        tf_enum_subtests() {
298
+            echo test1
299
+            echo test2
300
+            if this_is_macos_x; then
301
+                echo test3
302
+            fi
303
+        }
304
+
305
+ *  If you want to disable (comment out test) that you might not have
306
+    implemented yet or is broken (and for some reason you still want
307
+    it to haunt the test code) or something else outside SUT is broken
308
+    and prevents you from running the test, use `tf_enum_subtests` and
309
+    properly comment the reasons in code.
310
+
311
+        tf_enum_subtests() {
312
+            echo test1
313
+            echo test2
314
+        #   echo test3      #FIXME: implement after bz1234
315
+        }
316
+
317
+ *  If in doubt, use `TF_ES_BAILOUT`.
318
+
319
+
320
+### On exit statuses: three and above ###
321
+
322
+The difference in *error*, *panic* and higher values is subtle but
323
+important.  Follow me as I try to explain:
324
+
325
+ 1. If script has changed something on the system outside the working
326
+    directory, it is apparently expected to revert that change.
327
+
328
+ 2. Now if an error occurs, but the code responsible for cleaning up is
329
+    safely run, you can say there was *error but we have recovered*.
330
+
331
+ 3. But if the change can't be reverted safely, we know that we have
332
+    broken something and latter code may lead to weird results (including
333
+    masking bugs(!)), it's time to *panic* (in the code, not in real
334
+    life ;))
335
+
336
+ 4. And then there are corner cases like a bug in the script, OOM kill
337
+    or timeout when the status will be different and not really controlled
338
+    by the script.  Such cases will have to be treated the same way as
339
+    the "panic" case, but...
340
+
341
+ 5. the use of *panic* adds hint that the status has been set consciously
342
+    by the script, albeit exiting "in a hurry"--without proper clean up.
343
+
344
+Unfortunately there will be cases like above but with the error code less
345
+than four.   Example is a Bash script syntax error, which returns 2, or
346
+Python exception which returns 1.  Yes, in such cases the information
347
+conveyed by the exit status is wrong and you should do everything to
348
+avoid it.
349
+
350
+Possibilities like "test has passed but then something blew up" exist,
351
+but conveying this information is responsibility of the test output.
352
+
353
+Following table can be used as a cheat-sheet:
354
+
355
+    .---------------------------------------------------------------.
356
+    | e |    state of         |                                     |
357
+    | s |---------------------| script says                         |
358
+    |   | SUT   | environment |                                     |
359
+    |---|-------|-------------|-------------------------------------|
360
+    | 0 | OK    | safe        | test passed, everything worked fine |
361
+    | 1 | buggy | safe        | test failed, everything worked fine |
362
+    | 2 | ???   | safe        | I decided not to run the test       |
363
+    | 3 | ???   | safe        | Something blew up but I managed to  |
364
+    |   |       |             | clean up (I promise!)               |
365
+    | 4 | ???   | broken      | Something blew up and I rushed out  |
366
+    |   |       |             | in panic                            |
367
+    | * | ???   | broken      | ...nothing (is dead)                |
368
+    '---------------------------------------------------------------'
369
+
370
+As you can see, following this semantic allows us to see both the state
371
+of the system under test (SUT) *and* the environment.
372
+
373
+Following table illustrates how different statuses map to different
374
+scenarios with regard to test result as well as state of the environment:
375
+
376
+    .--------------------------------------------------.
377
+    | environment |  test result   |  test result      |
378
+    |             | pass fail unkn | pass fail unkn    |
379
+    |-------------|----------------|-------------------|
380
+    | clean(ed)   |  0    1    3   |  OK  FAIL ERROR   |
381
+    | untouched   |  ~    ~    2   |  ~    ~   BAILOUT |
382
+    | mess        |  ~    ~    4   |  ~    ~   PANIC   |
383
+    | ?! (trap)   |  ~    ~    5   |  ~    ~   ~       |
384
+    | ?! (sig 9)  |  ~    ~    137 |  ~    ~   ~       |
385
+    | ?! (aliens) |  ~    ~    ?   |  ~    ~   ~       |
386
+    '-------------|----------------|-------------------|
387
+                  |  exit status   |  human-readable   |
388
+                  |                |  name (TF_ES_*)   |
389
+                  '------------------------------------'

+ 26
- 0
utils/tfkit/doc/templates/grep_engine/TF_RUN View File

@@ -0,0 +1,26 @@
1
+#!/bin/bash
2
+
3
+. "$TF_DIR/include/subtest.sh"
4
+. "$TF_DIR/include/tools.sh"
5
+
6
+tf_enum_subtests() {
7
+    echo fixed
8
+    echo basic
9
+    echo extended
10
+#   echo perl       # TODO: write test
11
+}
12
+
13
+tf_name2cmd() {
14
+    local name=$1
15
+    local t_in="test/ALL.stdin"
16
+    local o_out="oracle/$name.stdout"
17
+    local args
18
+    case $name in
19
+        fixed)       args="-F 'The mask *.* matches all.'" ;;
20
+        basic)       args="-G 'he.*'" ;;
21
+        extended)    args="-P '.*og?g'" ;;
22
+    esac
23
+    echo "tf_testflt -n $name -i $t_in -O $o_out grep $args"
24
+}
25
+
26
+tf_do_subtests

+ 3
- 0
utils/tfkit/doc/templates/grep_engine/oracle/basic.stdout View File

@@ -0,0 +1,3 @@
1
+Linda and Nina work together.
2
+The mask *.* matches all.
3
+The mask *.docx matches Word documents.

+ 3
- 0
utils/tfkit/doc/templates/grep_engine/oracle/extended.stdout View File

@@ -0,0 +1,3 @@
1
+Alice uses Google.
2
+Bob wears goggles.
3
+Linda and Nina work together.

+ 1
- 0
utils/tfkit/doc/templates/grep_engine/oracle/fixed.stdout View File

@@ -0,0 +1 @@
1
+The mask *.* matches all.

+ 6
- 0
utils/tfkit/doc/templates/grep_engine/test/ALL.stdin View File

@@ -0,0 +1,6 @@
1
+Alice uses Google.
2
+Bob wears goggles.
3
+Joe and John are friends.
4
+Linda and Nina work together.
5
+The mask *.* matches all.
6
+The mask *.docx matches Word documents.

+ 110
- 0
utils/tfkit/include/common.sh View File

@@ -0,0 +1,110 @@
1
+#
2
+# Variables to hold exit status semantic
3
+#
4
+TF_ES_OK=0
5
+TF_ES_FAIL=1
6
+TF_ES_BAILOUT=2
7
+TF_ES_ERROR=3
8
+TF_ES_PANIC=4
9
+
10
+#
11
+# Color definition variables
12
+#
13
+TF_COLOR_BLACK="\033[0;30m"
14
+TF_COLOR_RED="\033[0;31m"
15
+TF_COLOR_GREEN="\033[0;32m"
16
+TF_COLOR_YELLOW="\033[0;33m"
17
+TF_COLOR_BLUE="\033[0;34m"
18
+TF_COLOR_MAGENTA="\033[0;35m"
19
+TF_COLOR_CYAN="\033[0;36m"
20
+TF_COLOR_WHITE="\033[0;37m"
21
+TF_COLOR_LBLACK="\033[1;30m"
22
+TF_COLOR_LRED="\033[1;31m"
23
+TF_COLOR_LGREEN="\033[1;32m"
24
+TF_COLOR_LYELLOW="\033[1;33m"
25
+TF_COLOR_LBLUE="\033[1;34m"
26
+TF_COLOR_LMAGENTA="\033[1;35m"
27
+TF_COLOR_LCYAN="\033[1;36m"
28
+TF_COLOR_LWHITE="\033[1;37m"
29
+TF_COLOR_NONE="\033[1;0m"
30
+
31
+
32
+tf_exit_ok() {
33
+    #
34
+    # Exit with OK status
35
+    #
36
+    exit $TF_ES_OK
37
+}
38
+
39
+tf_exit_fail() {
40
+    #
41
+    # Warn $1 and exit with status FAIL
42
+    #
43
+    tf_warn "$@"
44
+    exit $TF_ES_FAIL
45
+}
46
+
47
+tf_exit_bailout() {
48
+    #
49
+    # Warn $1 and exit with status FAIL
50
+    #
51
+    tf_warn "$@"
52
+    exit $TF_ES_BAILOUT
53
+}
54
+
55
+tf_exit_error() {
56
+    #
57
+    # Warn $1 and exit with status FAIL
58
+    #
59
+    tf_warn "$@"
60
+    exit $TF_ES_ERROR
61
+}
62
+
63
+tf_exit_panic() {
64
+    #
65
+    # Warn $1 and exit with status FAIL
66
+    #
67
+    tf_warn "$@"
68
+    exit $TF_ES_PANIC
69
+}
70
+
71
+tf_debug() {
72
+    #
73
+    # Emit debug message
74
+    #
75
+    $TF_DEBUG || return 0
76
+    local msg
77
+    for msg in "$@";
78
+    do
79
+        $TF_COLOR && echo -ne "$TF_COLOR_CYAN" >&2
80
+        echo "||| $msg" >&2;
81
+        $TF_COLOR && echo -ne "$TF_COLOR_NONE" >&2
82
+    done
83
+}
84
+
85
+tf_think() {
86
+    #
87
+    # Emit status/progress message
88
+    #
89
+    $TF_VERBOSE || return 0
90
+    local msg
91
+    for msg in "$@";
92
+    do
93
+        $TF_COLOR && echo -ne "$TF_COLOR_LBLACK" >&2
94
+        echo "$pfx$msg$sfx" >&2;
95
+        $TF_COLOR && echo -ne "$TF_COLOR_NONE" >&2
96
+    done
97
+}
98
+
99
+tf_warn() {
100
+    #
101
+    # Emit warning
102
+    #
103
+    local msg
104
+    for msg in "$@";
105
+    do
106
+        $TF_COLOR && echo -ne "$TF_COLOR_LRED" >&2
107
+        echo "$msg" >&2;
108
+        $TF_COLOR && echo -ne "$TF_COLOR_NONE" >&2
109
+    done
110
+}

+ 124
- 0
utils/tfkit/include/harness.sh View File

@@ -0,0 +1,124 @@
1
+#!/bin/bash
2
+# tfkit - Shellfu's movable test framework
3
+# See LICENSE file for copyright and license details.
4
+
5
+. "$TF_DIR/include/common.sh"
6
+
7
+
8
+__tf_collect_if_needed() {
9
+    #
10
+    # Collect artifact if exit status suggests it
11
+    #
12
+    # Use test exit status $1 to help decide if artifacts are
13
+    # needed, and collect them if so.
14
+    #
15
+    # If TF_COLLECT is set to "always", collect regardless of
16
+    # the status.  If set to "never", do not collect anything.
17
+    # The default setting, "auto" collects unless status is 0
18
+    # or 1 (pass or bailout); in that case do nothing.
19
+    #
20
+    local tes=$1    # test exit status
21
+    local will      # should we collect artifacts?
22
+    case "$TF_COLLECT:$tes" in
23
+        always:*)   will=true ;;
24
+        never:*)    will=false ;;
25
+        auto:0)     will=false ;;
26
+        auto:2)     will=false ;;
27
+        auto:*)     will=true ;;
28
+        *)          tf_exit_bailout "bad value of TF_COLLECT: $TF_COLLECT" ;;
29
+    esac
30
+    $will || return 0
31
+    mkdir -p "$artifact_dir/$stamp"
32
+    cp -r "$tmpdir"/* "$artifact_dir/$stamp"
33
+}
34
+
35
+__tf_header() {
36
+    #
37
+    # create header to add to output before test
38
+    #
39
+    local hdrline                       # each line from generator
40
+    local hdrgen="$TF_SUITE/TF_HEADER"  # header generator script
41
+    tf_think "#"
42
+    tf_think "# ---"
43
+    tf_think "# tfkit:"
44
+    if test -x "$hdrgen";
45
+    then
46
+        tf_think "#     sut:"
47
+        $hdrgen \
48
+          | while IFS= read -r hdrline;
49
+            do
50
+                test -n "$hdrline" || break
51
+                tf_think "#         $hdrline"
52
+            done
53
+    else
54
+        tf_think "#     hint: >"
55
+        tf_think "#         Add $hdrgen executable for own header."
56
+        tf_think "#         It should output YAML (\"key: value\" pairs) about"
57
+        tf_think "#         your SUT ('version' at least')."
58
+    fi
59
+    tf_think "#     run:"
60
+    tf_think "#         start_time: $(date -Iseconds)"
61
+    test -n "$TF_FILTER_TEST" \
62
+     && tf_think "#         TF_FILTER_TEST: $TF_FILTER_TEST"
63
+    test -n "$TF_FILTER_SUBTEST" \
64
+     && tf_think "#         TF_FILTER_SUBTEST: $TF_FILTER_SUBTEST"
65
+    tf_think "#"
66
+    tf_think ""
67
+}
68
+
69
+tf_enum_tests() {
70
+    #
71
+    # List what looks like test; relative to $TF_SUITE
72
+    #
73
+    tf_debug "TF_SUITE='$TF_SUITE'"
74
+    test -d "$TF_SUITE" || return 0
75
+    find -L \
76
+        "$TF_SUITE" \
77
+        -mindepth 2 \
78
+        -maxdepth 2 \
79
+        -type f \
80
+        -perm /111 \
81
+        -name TF_RUN \
82
+      | cut -d/ -f2
83
+}
84
+
85
+tf_run_tests() {
86
+    #
87
+    # Discover and run tests
88
+    #
89
+    local es=0          # overall exit status
90
+    local tmpdir=""     # test temporary dir
91
+    local tname=""      # test name
92
+    local tes=0         # test result
93
+    local stamp=""      # test stamp to use as artifact name
94
+    local tf_dir tf_suite   # to keep absolute paths for TF_RUN
95
+    local artifact_dir  # where to keep artifacts
96
+    artifact_dir="$(readlink -f "$TF_ARTIFACTS")"
97
+    __tf_header
98
+    tf_debug "TF_VERSION='$TF_VERSION'"
99
+    tf_dir="$(readlink -f "$TF_DIR")"
100
+    tf_suite="$(readlink -f "$TF_SUITE")"
101
+    es=0
102
+    for tname in $(tf_enum_tests | grep -e "$TF_FILTER_TEST");
103
+    do
104
+        tf_think "... $tname"
105
+        tmpdir=$(mktemp -d)
106
+        stamp=$(date "+artifacts-$tname-%Y%m%d-%H%M%S")
107
+        cp -r "$TF_SUITE/$tname/"* "$tmpdir"
108
+        pushd "$tmpdir" >/dev/null
109
+            TF_DIR="$tf_dir" TF_SUITE=$tf_suite TF_TNAME="$tname" \
110
+                ./TF_RUN
111
+            tes=$?
112
+            __tf_collect_if_needed $tes
113
+            test $tes -gt $es && es=$tes
114
+        popd >/dev/null
115
+        rm -rf "$tmpdir"
116
+        if test $tes -eq 0;
117
+        then
118
+            tf_think "''' $tname ($tes)"
119
+        else
120
+            tf_warn "??? $tname ($tes)"
121
+        fi
122
+    done
123
+    return $es
124
+}

+ 85
- 0
utils/tfkit/include/subtest.sh View File

@@ -0,0 +1,85 @@
1
+#!/bin/bash
2
+
3
+. "$TF_DIR/include/common.sh"
4
+
5
+tf_enum_subtests() {
6
+    #
7
+    # Stub: enumerate subtests
8
+    #
9
+    tf_warn "implement tf_enum_subtests()!"
10
+    return "$TF_ES_ERROR"
11
+}
12
+
13
+tf_do_subtest() {
14
+    #
15
+    # Stub: perform test named $1
16
+    #
17
+    tf_warn "implement tf_do_subtest()!"
18
+    return "$TF_ES_ERROR"
19
+}
20
+
21
+_tf_do_subtest() {
22
+    #
23
+    # Run single subtest inc. setup/cleanup if present
24
+    #
25
+    local subtname="$1"     # this subtest name
26
+    local ses=0             # subtest exit status
27
+    local setup=true        # setup command
28
+    local cleanup=true      # cleanup command
29
+    if test -f TF_SETUP;
30
+    then
31
+        setup=". TF_SETUP"
32
+        bash -n TF_SETUP || {
33
+            tf_warn "synax errors in TF_SETUP, skipping"
34
+            return "$TF_ES_ERROR"
35
+        }
36
+    fi
37
+    if test -f TF_CLEANUP;
38
+    then
39
+        setup=". TF_CLEANUP"
40
+        bash -n TF_CLEANUP || {
41
+            tf_warn "synax errors in TF_CLEANUP, skipping"
42
+            return "$TF_ES_ERROR"
43
+        }
44
+    fi
45
+    if $setup;
46
+    then
47
+        tf_do_subtest "$subtname"; ses=$?
48
+    else
49
+        tf_warn "setup phase failed, skipping: $subtname"
50
+        ses=$TF_ES_ERROR
51
+    fi
52
+    if ! $cleanup;
53
+    then
54
+        tf_warn "cleanup phase failed: $subtname"
55
+        ses=$TF_ES_PANIC
56
+    fi
57
+    return "$ses"
58
+}
59
+
60
+tf_do_subtests() {
61
+    #
62
+    # Run all subtests and return highest status
63
+    #
64
+    local es=0              # final exit status ("worst" of subtests)
65
+    local subtname=""       # one subtest name
66
+    local tes=""            # one subtest exit status
67
+    local enumd=TF_ENUMERATED_SUBTESTS
68
+    local fltrd=TF_FILTERED_SUBTESTS
69
+    tf_enum_subtests >$enumd    || { tf_warn "error enumerating subtests"; return "$TF_ES_BAILOUT"; }
70
+    test -s $enumd              || { tf_warn "no subtests enumerated";     return "$TF_ES_BAILOUT"; }
71
+    grep -e "$TF_FILTER_SUBTEST" $enumd > $fltrd
72
+    test -s $fltrd  || tf_debug "TF_FILTER_SUBTEST ate everything: $TF_FILTER_SUBTEST"
73
+
74
+    for subtname in $(<$fltrd);
75
+    do
76
+        export TF_SUBTNAME=$subtname
77
+        tf_think "::: $TF_TNAME::$TF_SUBTNAME"
78
+        _tf_do_subtest "$TF_SUBTNAME";
79
+        tes=$?
80
+        test $tes -gt $es              && es=$tes
81
+        test $tes -gt "$TF_ES_OK"      && tf_warn "!!! $TF_TNAME::$TF_SUBTNAME ($tes)"
82
+        test $tes -gt "$TF_ES_BAILOUT" && break
83
+    done
84
+    return $es
85
+}

+ 86
- 0
utils/tfkit/include/tools.sh View File

@@ -0,0 +1,86 @@
1
+#!/bin/bash
2
+
3
+. "$TF_DIR/include/common.sh"
4
+
5
+# 1. exec: [test] -> [result]
6
+# 2. eval:           [result] == [oracle]
7
+
8
+tf_testflt() {
9
+    #
10
+    # Run a simple test for a unix filter
11
+    #
12
+    #     tf_testflt -n foo [-i foo.stdin] \
13
+    #                [-O foo.stdout] [-E foo.stderr] [-S 3] \
14
+    #                cmd arg...
15
+    #
16
+    # Will drop *result/NAME.stdout* and *result/NAME.stderr* (intentionally
17
+    # not cleaning up).
18
+    #
19
+
20
+    # defaults
21
+    #
22
+    local t_in="/dev/null"      # test: stdin
23
+    local t_name=""             # test: name
24
+                                # command is "$@" after arg parsing
25
+    local t_es="0"              # final test exit status
26
+    local o_out="/dev/null"     # oracle: stdout
27
+    local o_err="/dev/null"     # oracle: stderr
28
+    local o_es="0"              # oralce: exit status
29
+    local r_out r_err r_es      # result: ^ ^ ^ those 3
30
+
31
+    local diff=diff
32
+    type colordiff >/dev/null 2>/dev/null && diff=colordiff
33
+
34
+    # get args
35
+    #
36
+    local orig_args="$0 $*"
37
+    tf_debug "orig_args=$orig_args"
38
+    local arg_err=false
39
+    while true; do case "$1" in
40
+        -i) t_in="$2";          shift 2 || { arg_err=true; break; } ;;
41
+        -n) t_name="$2";        shift 2 || { arg_err=true; break; } ;;
42
+        -O) o_out="$2";         shift 2 || { arg_err=true; break; } ;;
43
+        -E) o_err="$2";         shift 2 || { arg_err=true; break; } ;;
44
+        -S) o_es="$2";          shift 2 || { arg_err=true; break; } ;;
45
+        --)                     shift; break ;;
46
+        "")                            break ;;
47
+        -*) tf_warn "wrong testcli arg: $1"; return $TF_ES_BAILOUT ;;
48
+        *)                             break ;;
49
+    esac done
50
+    $arg_err && { tf_warn "error parsing arguments: $orig_args"; return $TF_ES_BAILOUT; }
51
+    tf_debug "t_in='$t_in'"
52
+    tf_debug "t_name='$t_name'"
53
+    tf_debug "o_out='$o_out'"
54
+    tf_debug "o_err='$o_err'"
55
+    tf_debug "o_es='$o_es'"
56
+    tf_debug "test command: $*"
57
+    test "$t_in" = "-" && t_in=/dev/stdin   # works better for check below
58
+    test -z "$t_name"  && { tf_warn "missing test name"             ; return $TF_ES_BAILOUT; }
59
+    test -z "$1"       && { tf_warn "missing test command"          ; return $TF_ES_BAILOUT; }
60
+    test -r "$t_in"    || { tf_warn "missing input file: $t_in"     ; return $TF_ES_BAILOUT; }
61
+    test -e "$o_out"   || { tf_warn "missing oracle stdout: $o_out" ; return $TF_ES_BAILOUT; }
62
+    test -e "$o_err"   || { tf_warn "missing oracle stderr: $o_err" ; return $TF_ES_BAILOUT; }
63
+    test "$o_es" -ge 0 || { tf_warn "invalid oracle status: $o_es"  ; return $TF_ES_BAILOUT; }
64
+
65
+    # prepare
66
+    #
67
+    mkdir -p result
68
+    r_out="result/$t_name.stdout"
69
+    r_err="result/$t_name.stderr"
70
+    tf_debug "r_out='$r_out'"
71
+    tf_debug "r_err='$r_err'"
72
+    touch "$r_out" || { tf_warn "cannot create tmp file: $r_out" ; return $TF_ES_BAILOUT; }
73
+    touch "$r_err" || { tf_warn "cannot create tmp file: $r_err" ; return $TF_ES_PANIC; }
74
+
75
+    # run
76
+    #
77
+    ( <"$t_in" eval "$@" >"$r_out" 2>"$r_err" ); r_es=$?
78
+    tf_debug "r_es='$r_es'"
79
+
80
+    # eval/report/exit
81
+    #
82
+    test $r_es = $o_es || { tf_warn "bad exit status: $r_es (need $o_es)" ; t_es=$TF_ES_FAIL; }
83
+    $diff -u "$o_err" "$r_err" || t_es=$TF_ES_FAIL
84
+    $diff -u "$o_out" "$r_out" || t_es=$TF_ES_FAIL
85
+    return $t_es
86
+}

+ 95
- 0
utils/tfkit/runtests View File

@@ -0,0 +1,95 @@
1
+#!/bin/bash
2
+# tfkit - Shellfu's movable test framework
3
+# See LICENSE file for copyright and license details.
4
+
5
+TF_VERSION="0.0.14"
6
+
7
+die() {
8
+    echo "$@" && exit 9
9
+}
10
+
11
+usage() {
12
+    echo "usage: $(basename "$0") [-c|-C] [-t tests_re] [-s subtest_re] [-p binpath] [-v] [-d]" >&2
13
+    exit 2
14
+}
15
+
16
+version() {
17
+    echo "TFKit (A trivial test kit) $TF_VERSION"
18
+    exit 0
19
+}
20
+
21
+LC_ALL=C
22
+
23
+#
24
+# Artifact directory path
25
+#
26
+TF_ARTIFACTS="${TF_ARTIFACTS:-artifacts}"
27
+
28
+#
29
+# Artifact collection mode
30
+#
31
+# 'always' to always collect, 'never` to never collect and 'auto'
32
+# to collect only on failed tests.
33
+#
34
+TF_COLLECT="${TF_COLLECT:-auto}"
35
+
36
+#
37
+# Enable color?
38
+#
39
+TF_COLOR=${TF_COLOR:-true}
40
+
41
+#
42
+# Turn on debug mode?
43
+#
44
+TF_DEBUG="${TF_DEBUG:-false}"
45
+
46
+#
47
+# Location of own directory
48
+#
49
+TF_DIR=${TF_DIR:-$(dirname "$0")}
50
+
51
+#
52
+# Regex (BRE) to filter subtests based on name
53
+#
54
+TF_FILTER_SUBTEST=${TF_FILTER_SUBTEST:-}
55
+
56
+#
57
+# Regex (BRE) to filter tests based on name
58
+#
59
+TF_FILTER_TEST=${TF_FILTER_TEST:-}
60
+
61
+#
62
+# Location of test suite
63
+#
64
+TF_SUITE="${TF_SUITE:-tests}"
65
+
66
+#
67
+# Turn on verbosity?
68
+#
69
+TF_VERBOSE=${TF_VERBOSE:-true}
70
+
71
+while true; do case "$1" in
72
+    -c|--collect)           TF_COLLECT=always;          shift ;;
73
+    -C|--no-collect)        TF_COLLECT=never;           shift ;;
74
+    -d|--debug)             TF_DEBUG=true;              shift ;;
75
+    -p|--prefix)            export PATH="$(readlink -f "$2")/bin:$PATH"
76
+                                                        shift 2 || usage ;;
77
+    -s|--filter-subtest)    TF_FILTER_SUBTEST="$2";     shift 2 || usage ;;
78
+    -t|--filter-test)       TF_FILTER_TEST="$2";        shift 2 || usage ;;
79
+    -v|--verbose)           TF_VERBOSE=true;            shift ;;
80
+    --version-semver)       echo "$TF_VERSION"; exit 0 ;;
81
+    --version)              version ;;
82
+    "") break ;;
83
+    *)  usage ;;
84
+esac done
85
+
86
+export LC_ALL
87
+export TF_DIR TF_SUITE TF_ARTIFACTS
88
+export TF_COLLECT TF_FILTER_SUBTEST TF_FILTER_TEST
89
+export TF_DEBUG TF_VERBOSE
90
+export TF_VERSION
91
+
92
+. "$TF_DIR/include/harness.sh" \
93
+ || die "cannot import harness; is TF_DIR set properly?: $TF_DIR"
94
+
95
+time tf_run_tests

+ 18
- 0
utils/tfkit/tfkit.mk View File

@@ -0,0 +1,18 @@
1
+# tfkit - Shellfu's movable test framework
2
+# See LICENSE file for copyright and license details.
3
+#
4
+# To use, set TF_DIR (make macro) in your Makefile and
5
+# include this file.
6
+#
7
+# Adds `test` target.
8
+#
9
+# Note that running "runtests" directly should have the same
10
+# effect but you will need to set TF_DIR and/or TF_SUITE properly
11
+# unless they are default (see README)
12
+
13
+export TF_DIR
14
+
15
+test:
16
+	@$(TF_DIR)/runtests
17
+
18
+.PHONY: test