minions

minions

My favorite tiny scripts for development and testing.

bin - UNIX scripts

autotest

Script to make it easier to start developing your new Perl module under pressure of unit tests written in Test::More and the likes.

Suggested use is to have it sit on the other monitor, run all .t files around and around and be annoying about the fact that they fail. Then the rule of thumb goes without saying: "no green, no commit".

bd

Binary dump. Reads STDIN, 4 bytes at a time and displays it in similar way as hexdump(1) does with -C option (Canonical hex+ASCII display)

Example:

$ echo "Hello world" | ./bd
00000000  01001000 01100101 01101100 01101100  |Hell|
00000004  01101111 00100000 01110111 01101111  |o wo|
00000008  01110010 01101100 01100100 00001010  |rld.|
$

dissect_url

Split URLs into components and arguments. Output is useful to see more easily what is or is not in the URL and/or compare URls using standard tools like diff.

$ dissect_url "proto://srv:port/a/query?par1=foo&par2=bar#joe&mary"
proto://srv:port
    /a/query
    ?
        par1=foo&
        par2=bar
    #
        joe

Note that by removing all whitespace from the dissected URL you should get the original URL.

To enter multiple URLs, simply omit the argument; script will go into filter mode, where you can enter URLs one per line. Quit this mode by entering EOF (Ctrl+D) or an empty line. From this mode, output will be separated like this:

$ dissect_url < two_urls
=== url 01 =============================================
url1

=== url 02 =============================================
url2

mkexec

mkexec [-f] [-e] name [type] [purpose]

Make executable script, i.e. create new file, add shebang line and template, and mark it executable (0755). type (py, pl, bash...) and purpose (test or nothing) are used to choose the right template. If not supplied, attempt is made to guess from the filename.

If the file already exist, gives up, unless -f option is passed. -e option causes default editor to be spawned with the new file loaded.

pfile

Wrapper around file utility. Reads STDIN, stores it in a temporary file (using Python's tmpfile.mkstemp), calls file on it and returns output.

This is useful in cases like debugging a HTTP server with utility like curl, and when we don't want to see the actual output, yet still want to know what it looks like. Using pfile on pipe, we can easily combine the power of file with the simplicity of curl:

us@here:~$ curl -4 -v http://www.example.com/ | pfile
* About to connect() to www.example.com port 80 (#0)
*   Trying 1.2.3.4...
* Connected to www.example.com (1.2.3.4) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.26.0
> Host: www.example.com
> Accept: */*
>
* additional stuff not fine transfer.c:1037: 0 0
* HTTP 1.1 or later with persistent connection, pipelining supported
< HTTP/1.1 200 OK
< Date: Tue, 22 Oct 2013 10:41:28 GMT
< Server: Apache/2.2.22 (Debian)
< Last-Modified: Wed, 17 Jul 2013 17:22:00 GMT
< Accept-Ranges: bytes
< Content-Length: 123
< Vary: Accept-Encoding
< Content-Type: text/html
<
{ [data not shown]
100    123 100    123   0     0   1169      0 --:--:-- --:--:-- --:--:--  1369
* Connection #0 to host www.example.com left intact
* Closing connection #0
/tmp/tmplZbSB4: ASCII text
us@here:~$

pl2yml, yml2pl

Read YAML/Perl data structure from a file and dump it in the other format to STDIN. Uses YAML::Tiny for YAML jobs.

se

Translate to and from Czech using slovnik.cz service.

se [options] word

By default, outputs few Czech translations of it, one per line. Also supports other languages (about 10 in total).

Most useful options are --lines (default is 25), --long, as shorthand to --lines=50, and --direction, which supports direction keyword in form "LNcz.cz" or "LNcz.LN", where LN is 2-letter code (not ISO) of the other language.

Uses www.slovnik.cz, so an Internet connection and LWP::Simple are needed.

Has POD doc (se --man or se --help) worth looking at.

timestamp

Script to measure how long does one second take. For those that know how long one second takes, it can serve as a snippet for Perl &stamp().

vim-bx

Wrapper around vim to store timestamps for certain files before edit and restore them afterwardds.

This is designed for my bloxsom files, therefore it's hard-coded to look for files that contain my blosxom path in the pattern; however, this can easily be altered.

The background is that Blosxom blogging system stores articles as plain files, and uses the file stamp as article date. I don't want articles to pop-up just because I did a minor typo-fix later.

watchdump

Prints a text file, clears the screen and pauses for 2s over and over.

Designed mainly for use with helper::dmup(); to enable you to see changes in your dumped data structure continuously, but obviously you can use it for any text file that will fit your screen.

For improved visual feedack, it will pre-pend the file contents with the file path and an "animation". Display of the header can be controlled by options, see --usage.

cgi-bin - CGI scripts

hello_host.cgi

Prints overview of TCP connection details (IP addresses, ports, server user@host, time); 7 lines of plain text.

htlog.cgi

Collects GET requests with parameters msg, tag and i and logs them into single text file.

• msg is actual message body. It can be used to contain parseable data like name=john;age=32;state=il
• tag can be used to identify related messages like those from one test
• use i if you tend to write tags like test01-012 .. test01-013 to store the iteration number, you'll be better off with this parameter as it won't break your ability to use tags

randomfile.pl

Send a 7-bit plain-text file via HTTP. One of these is sent:

• random content of random length given by parameters min and max

• EICAR test virus file

Chance to receive EICAR is given by parameter eicar (0-100).

lib - modules/libraries

helper.pm

Container module for some utility methods for Perl. Probably only dmup() is interesting—it bears a nice quick and dirty way for dumping Perl data.

htlogger.pm, htlogr.py

APIs to make usage of htlog.cgi in Perl and Python scripts even easier

use htlogr;

my $logger = htlogr::new('http://192.168.1.1/cgi-bin/htlog.cgi');

# we don't need tag nor iteration number, but it can be useful
my $tag = "synopsis_test";

$logger->log("Commencing synopsis test", $tag);

my $data = {
    foo => 1,
    bar => "Hello world"
};

foreach my $i (1..1000) {

    # log normal messages--with I!
    $logger->log("next 10 done!", $tag, $i) unless ($num % 100);

    # or a simple one-level data structures
    $logger->data(
        my_func_returning_hashref($data),
        $tag,
        $i
    );

}

Note that htlogr also supports passing of callable code instead of i or tag. Use this if you find yourself constructing them in a non-trivial way before every call.

I'll illustrate this with Python API, but of course implementation and use is same in both APIs.

Imagine situation when existence of a certain environment variable tells us context in which we are running (e.g. a specific test case) and its value is the iteration (e.g. Jenkins build number).

This example examines environment for existence of such variable and then uses its name as tag and value as i.

The code:

import htlogr
import os

logger = htlogr('http://192.168.1.1/cgi-bin/htlog.cgi')

def get_both():
    for key in ['var1', 'var2']:
        try:
            return os.environ[key], key
        except KeyError:
            pass

def get_i():
    i, tag = get_both()
    return i

def get_tag():
    i, tag = get_both()
    return tag

logger.log("hello", tag=get_tag, i=get_i)

Now you can e.g. write a logging wrapper function in a trivial yet flexible way:

# inside a class:

def rmsg(self, mesage):
    self.logger.log(message, i=self.get_i, tag=self.name)

def rwarn(self, mesage):
    self.logger.log('warning: ' message, i=self.get_i, tag=self.name)

def rstats(self, stats):
    self.logger.data(stats, i=self.get_i, tag=self.name)