Shell Script Utils

Author

gentam

Date
2018-06-17
Category

snippet

Tag

shell-script, sed, awk

Updated

2018-12-27

These are shell scripts for very niche tasks.

Contents

conv2utf8

ファイルかディレクトリ下のファイルを丸ごとUTF-8に変換する. nkf のオプション が妙に覚えにくいので,ネットからダウンロードしてきたサンプルコードが全部 Shift-JISだったときなどに便利.

Convert character encodings of a file or all files under a directory to UTF-8, using nkf command.

https://gist.github.com/gentam/5758f8a46b48964aabceffdc11c45600

#!/usr/bin/env bash

opt='-w'
# lower case -> output / UPPER CASE -> input
#  -J  ISO-2022-JP (JIS code)
#  -S  Shift_JIS
#  -E  EUC-JP
#  -W  UTF-8

[ $# -eq 0 ] && echo "Usage: `basename $0` <filename | dirname>" \
             && exit 1

self="$0"
if [ -f "$@" ]; then
    # [ $(nkf -g "$@") = 'Shift_JIS' ] && \ # only convert specific encoding
    nkf --in-place $opt "$@"
    echo "converted $@"
elif [ -d "$@" ]; then
    # given directory, convert all files recursively
    find "$@" -type f -print0 | xargs -0 -n1 $self
else
    echo "$@: No such file or directory"
    exit 1
fi

find -print0 | xargs -0 is a useful combination to avoid a havoc caused by filename containing spaces.

changeExtension

Change file extension in bulk.

https://gist.github.com/gentam/b0836fc582a8578d578277b859f3bae9

#!/usr/bin/env bash

if [ $# -ne 2 ]; then
    echo "Usage: $(basename $0) oldExt newExt
    e.g. $(basename $0) JPEG jpg"
    exit 1
fi

oldExt=$1
newExt=$2

for file in *.$oldExt; do
    newName=${file%.$oldExt}.$newExt
    mv -i "${file}" "${newName}"
    echo $newName
done

fswap

It swaps two filenames. Maybe useful for situation such as handling *.conf.bak.

https://gist.github.com/gentam/b752df409a6ae91ae50fd9a2d644a8f1

#!/usr/bin/env bash

[ $# -ne 2 ] && echo "Usage: $(basename $0) fileA fileB" \
             && exit 1

tmp="tmp$(date +%s)"

mv -i $1 $tmp && \
mv $2 $1 && \
mv $tmp $2

codes2rst

I wrote this when I wanted to paste a set of codes to reStructredText. It adds .. code:: <extension> and indentations. It also deletes the comment line starts with // so it's not so general-purpose.

https://gist.github.com/gentam/5f47198f64f59d5193cd138b13e7b165

#!/usr/bin/env bash

[ $# -eq 0 ] && echo "Usage: `basename $0` [filename | *].extension [...]" \
             && exit 1

for f in $@; do
    ext=${f##*.}
    echo -e "$f\n\n.. code:: $ext"
    echo
    cat $f \
    | sed -e '/^\/\//d' \
    | awk '{print "  " $0}' \
    | sed  -e '1d' -e '$a\\n' \
    | tr -d '
'
done

The last command is actually tr -d '^M'

brewInstallMissing

brew doctor blamed me for some missing packages once, so I wrote this to automatically install missings. But I haven't got into the same situation since then.

https://gist.github.com/gentam/c61b2c63122884ce79876f70194e4598

#!/usr/bin/env bash

brew missing | awk 'BEGIN { print "Installing missing formulae..." }
    {
        split($0, a)
        for (i = 2; i <= NF; i++) {
            dep = a[i]
            if (!installed[dep]) {
                print "\n------------------------------------------------------------"
                print "Installing dependencies for " $1

                system("brew desc " dep)
                system("brew install " dep)
                installed[dep] = 1
            }
        }
    }
    END { print "\nDone" }'

find2html

This script prints the directory tree to HTML with links. I don't remember why I wrote this, but can be handy for listing a directory to the web.

https://gist.github.com/gentam/42bee765753fd8e9be4b8fb8277c7c55

#!/usr/bin/env bash

echo "<!DOCTYPE html><html lang=\"en\"><head><meta charset=\"UTF-8\">\
<title>Directory Tree</title></head><body>"

find . \
| tail -n +2 | sed -e '/\/\./d' \
| awk -F/ '
    BEGIN { preNF = 0 }
    {
        if (NF > preNF) print "<ul>"
        if (NF < preNF) {
            diff = preNF - NF
            while (diff--) print "</ul>"
        }
        printf "<li><a href=\"%s\">%s</a></li>\n", $0, $NF
        preNF = NF
    }
    END { while (--preNF) print "</ul>" }'

echo "</body></html>"

iwantmyname

iwantmyname DNS API wrapper

https://gist.github.com/gentam/874852602e0380af4c2a4eb1f3dc68cc

#!/usr/bin/env bash

## iwantmyname DNS API wrapper
## see <https://iwantmyname.com/developer/domain-dns-api>

usage() {
    echo "Usage: `basename $0` <RECORD_TYPE> <HOSTNAME> <RECORD_VALUE>
    RECORD_TYPE := A|AAAA|TXT|MX|CNAME|NAPTR|SRV

    e.g. `basename $0` A blog.example.com 123.456.78.90
         `basename $0` CNAME www.example.com example.com
         `basename $0` TXT _acme-challenge.example.com DmCKygRUkCIApIfHtpELeHy3f5JXcm_WNQccuADh1Do"
}

[ $# -ne 3 ] && usage && exit 1

read -p 'Email: ' USER
read -p 'Password: ' -s PASS
echo

TYPE="$1"
HOSTNAME="$2"
VAL="$3"

case $TYPE in
    a|aaaa|A|AAAA)
        curl -u "$USER:$PASS" "https://iwantmyname.com/basicauth/ddns?hostname=$HOSTNAME&myip=$VAL"
        ;;
    txt|mx|cname|naptr|srv|TXT|MX|CNAME|NAPTR|SRV)
        curl -u "$USER:$PASS" "https://iwantmyname.com/basicauth/ddns?hostname=$HOSTNAME&type=$TYPE&value=$VAL"
        ;;
    *)
        echo "Invalid record type: $1"
        usage
        exit 1;;
esac
echo

pingall

ping all IP addresses in the subnet and print the result

https://gist.github.com/gentam/69c41f37f0f04e189955617090291590

#!/usr/bin/env bash

NETADDR='192.168.1'

for ip in "$NETADDR".{1..254}; do
    echo -n $ip
    ping -c1 -W1 "$ip" > /dev/null 2>&1 && echo ' ok' || echo
    sleep 0.25
done

aax2mp4

convert Audible AAX file to MP4 for backup

  1. Use audible-activator <https://github.com/inAudible-NG/audible-activator> to obtain your activation code

  2. ./aax2mp source.aax then enter the code each time, or to set ACTIVATION_BYTES

https://gist.github.com/gentam/9dd0fd548518876437c1bea362759db8

#!/usr/bin/env bash

[ $# -ne 1 ] && { echo >&2 "Usage: `basename $0` audiobook.aax"; exit 1; }
type ffmpeg > /dev/null 2>&1 || { echo >&2 "Error: ffmpeg required"; exit 1; }

SRC="$1"
ACTIVATION_BYTES='' # can be obtained by audible-activator <https://github.com/inAudible-NG/audible-activator>
[ "$ACTIVATION_BYTES" = '' ] && read -p '4 byte activation secret (e.g. 1CEB00DA): ' ACTIVATION_BYTES

ffmpeg -activation_bytes "$ACTIVATION_BYTES" -i "$SRC" -vn -c:a copy "${SRC%.aax}.mp4"