minions ======= My favorite tiny scripts for development and testing. bin - UNIX scripts ------------------ ### autotest ### Script to make it easier to start developing your new Perl module under pressure of unit tests written in [Test::More][1] and the likes. [1]: (http://perldoc.perl.org/Test/More.html) Suggested use is to have it sit on the other monitor, run all *.t* files around and around and be annoying about the fact that they fail. Then the rule of thumb goes without saying: "no green, no commit". ### bd ### Binary dump. Reads STDIN, 4 bytes at a time and displays it in similar way as `hexdump(1)` does with `-C` option (Canonical hex+ASCII display) Example: $ echo "Hello world" | ./bd 00000000 01001000 01100101 01101100 01101100 |Hell| 00000004 01101111 00100000 01110111 01101111 |o wo| 00000008 01110010 01101100 01100100 00001010 |rld.| $ ### dissect\_url ### Split URLs into components and arguments. Output is useful to see more easily what is or is not in the URL and/or compare URls using standard tools like diff. $ dissect_url "proto://srv:port/a/query?par1=foo&par2=bar#joe&mary" proto://srv:port /a/query ? par1=foo& par2=bar # joe Note that by removing all whitespace from the dissected URL you should get the original URL. To enter multiple URLs, simply omit the argument; script will go into filter mode, where you can enter URLs one per line. Quit this mode by entering *EOF* (`Ctrl+D`) or an empty line. From this mode, output will be separated like this: $ dissect_url < two_urls === url 01 ============================================= url1 === url 02 ============================================= url2 ### mkexec ### mkexec [-f] [-e] name [type] [purpose] Make executable script, i.e. create new file, add shebang line and template, and mark it executable (0755). type (py, pl, bash...) and purpose (test or nothing) are used to choose the right template. If not supplied, attempt is made to guess from the filename. If the file already exist, gives up, unless *-f* option is passed. *-e* option causes default editor to be spawned with the new file loaded. ### pfile ### Wrapper around *file* utility. Reads STDIN, stores it in a temporary file (using Python's *tmpfile.mkstemp*), calls *file* on it and returns output. This is useful in cases like debugging a HTTP server with utility like *curl*, and when we don't want to see the actual output, yet still want to know what it looks like. Using pfile on pipe, we can easily combine the power of *file* with the simplicity of *curl*: us@here:~$ curl -4 -v http://www.example.com/ | pfile * About to connect() to www.example.com port 80 (#0) * Trying 1.2.3.4... * Connected to www.example.com (1.2.3.4) port 80 (#0) > GET / HTTP/1.1 > User-Agent: curl/7.26.0 > Host: www.example.com > Accept: */* > * additional stuff not fine transfer.c:1037: 0 0 * HTTP 1.1 or later with persistent connection, pipelining supported < HTTP/1.1 200 OK < Date: Tue, 22 Oct 2013 10:41:28 GMT < Server: Apache/2.2.22 (Debian) < Last-Modified: Wed, 17 Jul 2013 17:22:00 GMT < Accept-Ranges: bytes < Content-Length: 123 < Vary: Accept-Encoding < Content-Type: text/html < { [data not shown] 100 123 100 123 0 0 1169 0 --:--:-- --:--:-- --:--:-- 1369 * Connection #0 to host www.example.com left intact * Closing connection #0 /tmp/tmplZbSB4: ASCII text us@here:~$ ### pl2yml, yml2pl ### Read YAML/Perl data structure from a file and dump it in the other format to STDIN. Uses [YAML::Tiny][4] for YAML jobs. [4]: https://metacpan.org/module/YAML::Tiny ### se ### Translate to and from Czech using slovnik.cz service. se [options] word By default, outputs few Czech translations of it, one per line. Also supports other languages (about 10 in total). Most useful options are `--lines` (default is 25), `--long`, as shorthand to `--lines=50`, and `--direction`, which supports direction keyword in form "LNcz.cz" or "LNcz.LN", where LN is 2-letter code (**not** ISO) of the other language. Uses [www.slovnik.cz][2], so an Internet connection and [LWP::Simple][3] are needed. [2]: http://www.slovnik.cz/ [3]: http://search.cpan.org/~gaas/libwww-perl-6.05/lib/LWP/Simple.pm Has POD doc (`se --man` or `se --help`) worth looking at. ### timestamp ### Script to measure how long does one second take. For those that know how long one second takes, it can serve as a snippet for Perl `&stamp()`. ### vim-bx ### Wrapper around vim to store timestamps for certain files before edit and restore them afterwardds. This is designed for my bloxsom files, therefore it's hard-coded to look for files that contain my blosxom path in the pattern; however, this can easily be altered. The background is that Blosxom blogging system stores articles as plain files, and uses the file stamp as article date. I don't want articles to pop-up just because I did a minor typo-fix later. ### watchdump ### Prints a text file, clears the screen and pauses for 2s over and over. Designed mainly for use with `helper::dmup();` to enable you to see changes in your dumped data structure continuously, but obviously you can use it for any text file that will fit your screen. For improved visual feedack, it will pre-pend the file contents with the file path and an "animation". Display of the header can be controlled by options, see `--usage`. cgi-bin - CGI scripts --------------------- ### hello_host.cgi ### Prints overview of TCP connection details (IP addresses, ports, server user@host, time); 7 lines of plain text. ### htlog.cgi ### Collects GET requests with parameters *msg*, *tag* and *i* and logs them into single text file. * *msg* is actual message body. It can be used to contain parseable data like `name=john;age=32;state=il` * *tag* can be used to identify related messages like those from one test * use *i* if you tend to write tags like `test01-012` .. `test01-013` to store the iteration number, you'll be better off with this parameter as it won't break your ability to use tags ### randomfile.pl ### Send a 7-bit plain-text file via HTTP. One of these is sent: * random content of random length given by parameters *min* and *max* * EICAR test virus file Chance to receive EICAR is given by parameter *eicar* (0-100). lib - modules/libraries ----------------------- ### helper.pm ### Container module for some utility methods for Perl. Probably only `dmup()` is interesting—it bears a nice quick and dirty way for dumping Perl data. ### htlogger.pm, htlogr.py ### APIs to make usage of *htlog.cgi* in Perl and Python scripts even easier use htlogr; my $logger = htlogr::new('http://192.168.1.1/cgi-bin/htlog.cgi'); # we don't need tag nor iteration number, but it can be useful my $tag = "synopsis_test"; $logger->log("Commencing synopsis test", $tag); my $data = { foo => 1, bar => "Hello world" }; foreach my $i (1..1000) { # log normal messages--with I! $logger->log("next 10 done!", $tag, $i) unless ($num % 100); # or a simple one-level data structures $logger->data( my_func_returning_hashref($data), $tag, $i ); } Note that htlogr also supports passing of callable code instead of `i` or `tag`. Use this if you find yourself constructing them in a non-trivial way before every call. I'll illustrate this with Python API, but of course implementation and use is same in both APIs. Imagine situation when existence of a certain environment variable tells us context in which we are running (e.g. a specific test case) and its value is the iteration (e.g. Jenkins build number). This example examines environment for existence of such variable and then uses its name as tag and value as i. The code: import htlogr import os logger = htlogr('http://192.168.1.1/cgi-bin/htlog.cgi') def get_both(): for key in ['var1', 'var2']: try: return os.environ[key], key except KeyError: pass def get_i(): i, tag = get_both() return i def get_tag(): i, tag = get_both() return tag logger.log("hello", tag=get_tag, i=get_i) Now you can e.g. write a logging wrapper function in a trivial yet flexible way: # inside a class: def rmsg(self, mesage): self.logger.log(message, i=self.get_i, tag=self.name) def rwarn(self, mesage): self.logger.log('warning: ' message, i=self.get_i, tag=self.name) def rstats(self, stats): self.logger.data(stats, i=self.get_i, tag=self.name)