Very often I login on machines with older mii-tool that reports 1Gbit/s cards as if they are connected on 100Mbit/s. This is normal as mii-tool is depricated. But a lot of old timers like me are used to its output. So I wrote a very simple awk script which can convert the ethtool output to the one liner output of mii-tool

if [ -z "$1" ]; then
    echo "Usage: mii-tool DEV"
    exit
fi
ethtool $1 2>&1|awk '
/Settings/{d=$3}
/Speed/{
    s=$2
    if (s !~ /Unknown/) {
        gsub(/Mb.*/,"",s)
        s=s"baseTx-"
    }
}
/Duplex/{
    if ($2 == "Full")
        s=s"FD"
    else
        s=s"HD"
}
/Link/{
    if ($3 == "yes")
        l="link ok"
    else
        l="no link"
}
/No data available/{err=1}
END{
    if (err) {
        gsub(/:/,"",d)
        print "SIOCGMIIPHY on \""d"\" failed: No such device"
        exit
    }
    if (s!~ /Unknown/)
        print d," "s", "l
    else
        print d, l
}'

And a shorter version for your .bashrc:
function mii-tool { ethtool $1|awk '/Settings/{d=$3}/Link/{if($3=="yes")l="link ok";else l="no link"}/Speed/{s=$2}/Duplex/{u=$2}END{if(s!~/Unknown/)print d," "s", "u", "l;else print d,l}'}

Posted by HackMan
Dated: 7th December 2015
Filled Under: Linux General, Technology
Comments: Post the 1st one!

Recently I wanted to check if I have reached the limit of my network connection. However I constatly had to login on my router and check the traffic on each interface… and I had to watch it continiously.
.
So I decided that I’ll write a tool which will collect the information and regulary update a DB, from which I will create graphs… a few minutes later I remembered a good old peace of software called vnstat. I immediatelly installed vnstat on the router and also added jsvnstat for a nice web interface to vnstat.
.
However a problem remained… I needed to see the traffic live and constant update of a DB and regeneration of graphs is not a very modern approach. So I decided to see what I can do to see the actual network traffic stats at the moment they are collected.
.
This is how Placky was born. Now I can see the live statistics directly from my browser… even on a phone :)
.
If you are interested I have left my home placky interface open to everyone.

Posted by HackMan
Dated: 15th October 2015
Filled Under: Technology
Comments: Post the 1st one!

Today I decided to join my co-located server to pool.ntp.org.
It was surprisingly easy and now I have my machine contributing to the big effort that is pool.ntp.org.

Statistics for it can be found here: http://www.pool.ntp.org/user/hackman

Posted by HackMan
Dated: 15th September 2014
Filled Under: Technology
Comments: Post the 1st one!

Since I started using Linux::Unshare after I created my Linux::Setns I found that unshare was missing a few tests.

I added them and sent a patch to the current maintainer Boris Sukholitko. However a few days later he wrote to me that he is no longer maintaining the module and proposed that I should take over maintainership.

So since today, I’m the maintainer of Linux::Unshare.

I’m going to release version 0.04 tomorrow.
I also created a GitHub repo for Linux::Unshare.

Posted by HackMan
Dated: 28th July 2014
Filled Under: Technology, Uncategorized
Comments: Post the 1st one!

I’m preparing a talk for YAPC::EU. The talk will be about managing Linux containers in Perl and for doing that I needed some functionality that was currently missing ;)
So I created a new Perl Module - Linux::Setns.

This is my first attempt at building and supporting modules on CPAN… but I hope I’ll add a few more in the same area soon.

The code is kept in GitHub.

Posted by HackMan
Dated: 28th July 2014
Filled Under: Technology, getClouder
Comments: Post the 1st one!

Requirements:

wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.12.24.tar.xz
wget http://hydra.azilian.net/3.12.24-config
cp 3.12.24-config .config
tar xfj linux-3.12.24.tar.xz
cd linux-3.12.24
git init
git add .
git commit -a -m 'initial'

Tasks:

  • Patch /proc/cpuinfo, so when you are not in the main CGroup, it will show you only the CPU cores you are allowed to use.
  • Patch /proc/stat, so when you are not in the main CGroup, it will show you only the CPU cores you are allowed to use.
  • Patch /proc/meminfo, so when you are not in the main CGroup, it will show you the total and free memory that is set to your current CGroup.
  • Patch /proc/partitions, so when you are not in the main CGroup, it will show you only the partitions/devices that you are allowed to use.

Tips:

  • File systems code is located under the ‘fs’ directory. So the proc file system code is located under ‘fs/proc’ directory.
  • When making your kernel use ‘-jX’ where X is the number of CPU cores you have +2. So if you have 4 cores, you should use -j6.
    make -jX
  • Control Groups are Linux mechanism to impose limits to group of processes instead of per-process as it is normal. We will use this functionalities, to set limits to some resources.
        cgroup - /
                 |- cpuset.cpus (used for the /proc/cpuinfo limit)
                 |- devices.allow (used to set the block device limit,
                 |   used for /proc/partitions limit)
                 |- devices.list (used to read the limits imposed using
                 |   devices.allow
                 |- memory.limit_in_bytes (used for the /proc/meminfo limit)
    
  • Finding the current control group you are in. The kernel exposes one global pointer ‘current’ which is a pointer to the process currently working. ‘current’ is of type task_struct. The task_struct structure is defined in ‘include/linux/sched.h’. You must find where the control groups code reside, but keep in mind that most of the functionality around control groups uses this type of structure ‘cgroup_subsys_state’ and it is regulary shortend to only ‘css’.

Testing your code

After successful compilation you should be able to boot a VM with your new kernel. There are a few commands that are the same for all tests:


mkdir /cgroup
mount -t cgroup none /cgroup
mkdir /cgroup/tt
echo 0 > /cgroup/tt/cpuset.cpus
echo 0 > /cgroup/tt/cpuset.mems
echo $$ > /cgroup/tt/tasks

Then for testing the cpuinfo:
cat /proc/cpuinfo

Then for testing the stat:
cat /proc/stat

Then for testing the meminfo:

echo 268435456 > /cgroup/tt/memory.limit_in_bytes
cat /proc/meminfo

Then for testing the partitions:

echo 'b 8:1 rw' > /cgroup/tt/devices.allow
cat /proc/partitions

Posted by HackMan
Dated: 21st July 2014
Filled Under: Teaching, Technology
Comments: Post the 1st one!

I needed an easy way to download videos I streamed to Twitch.tv, so I created this small script which downloads all parts of the video, converts them to mpegts and then combines them into a single mpeg flv video, ready for upload to YouTube.

#!/bin/bash

video_dir=~/twitch

if [ $# -ne 1 ]; then
        echo "Usage: $0 twitch_video_id"
        exit 0
fi

id=$1
video_urls=( $(curl http://api.justin.tv/api/broadcast/by_archive/${id}.xml?onsite=true | grep video_file_url | sed 's/.*url>\(http:.*\)< \/vid.*/\1/') )

if [ ! -d $video_dir ]; then
        mkdir $video_dir
fi

cd $video_dir

# download the videos
for i in ${video_urls[*]}; do
        wget -c $i
done

rm -f int*.ts
concat_list='concat:'
last_num=${#video_urls[*]}
let last_num--

# convert the videos to Mpeg TS video format
for i in $( seq 0 $last_num ); do
        ffmpeg -i ${video_urls[$i]/*\//} -c copy -bsf:v h264_mp4toannexb -f mpegts int${i}.ts
        if [ "$i" -eq 0 ]; then
                concat_list="${concat_list}int${i}.ts"
        else
                concat_list="$concat_list|int${i}.ts"
        fi
done

# merge the files togather
ffmpeg -f mpegts -i "$concat_list" -c copy for_youtube_${id}.flv
Posted by HackMan
Dated: 10th October 2013
Filled Under: Teaching, Technology
Comments: Post the 1st one!

Around OHM2013 I started using Xchat again to manage all the IRC networks I have to be on.
Pidgin simply could not do the job.

However I have to constantly authenticate my self against services. Unfortunately I have lost my ns-authenticate.pl plugin so I wrote a new one :)

#!/usr/bin/perl
use strict;
use warnings;

Xchat::register('NS Identify', '0.2', 'Identify against NickServ');
Xchat::hook_server( 'Notice', \&ns_identify );
Xchat::hook_print( 'Message Send', \&hide_nickserv ); 

my %pass = (
    'UniBG' => '',
    'FreeNode' => '',
    'OFTC' => ''
);

# Catch the notices from services and send passwords
sub ns_identify {
    return Xchat::EAT_NONE if !defined($_[0]);
    if ($_[0][0] =~ /^:NS!NickServ\@UniBG/ && $_[0][3] =~ /^:This/) {
        Xchat::command("msg NS identify $pass{'UniBG'}");
        return Xchat::EAT_NONE;
    }
    if ($_[0][0] =~ /^:NickServ!NickServ\@services./ && $_[0][3] =~ /^:This/) {
        Xchat::command("msg NickServ identify $pass{'FreeNode'}");
        return Xchat::EAT_NONE;
    }
    return Xchat::EAT_NONE;
}

# Catch the messages to NS and NickServ and do not show them in the windows and logs.
sub hide_nickserv {
    if (defined($_[0])) {
        if ($_[0][0] eq 'NS' || $_[0][0] =~ /NickServ/) {
            return Xchat::EAT_ALL;
        }
    }
    return Xchat::EAT_NONE;
}
Posted by HackMan
Dated: 9th August 2013
Filled Under: Uncategorized
Comments: Post the 1st one!

For a project that I’m working on I had to create MAC generator. Initially I decided that I will do it in BASH.

function gen_mac() {
        mac_vars=(0 1 2 3 4 5 6 7 8 9 a b c d e f)
        mac_base='52:53:54:'
        ret=''
        for i in {1..6}; do
                n=$RANDOM
                let 'n %= 16'
                ret="${ret}${mac_vars[$n]}"
                if [ $i -eq 2 ] || [ $i -eq 4 ]; then
                        ret="${ret}:"
                fi
        done
        echo "${mac_base}${ret}"
}

But unfortunately I had to rewrite it in plpgsql in order to prevent a stupid race condition that can appear.

CREATE OR REPLACE FUNCTION generate_mac() RETURNS text
    LANGUAGE plpgsql
    AS $$DECLARE
	mac TEXT;
	a CHAR;
	count INTEGER;
BEGIN
	mac='52:53:54:';
	FOR count IN 1..6 LOOP
		SELECT substring('0123456789abcdef' FROM (random()*16)::int + 1 FOR 1) INTO a;
		-- This fixes un issue, wehre the above SELECT returns NULL or empty string
		-- If for some reason we concatenate with a NULL string, the result will be NULL string
		WHILE  a IS NULL OR a = '' LOOP
			SELECT substring('0123456789abcdef' FROM (random()*16)::int + 1 FOR 1) INTO a;
		END LOOP;
		mac = mac || a;
		IF count = 2 OR count = 4 THEN
			mac = mac || ':';
		END IF;
	END LOOP;
	RETURN mac;
END;$$;
Posted by HackMan
Dated: 25th March 2013
Filled Under: Uncategorized
Comments: Post the 1st one!

Today I decided to play a bit with my Raspberry Pi and one of my Relay shields that I have for the Arduino.
I was surprised to see how easy it was to control and read the GPIOs on the Raspberry. So I decided to make it even easier by writing a simple bash script which I use to turn on and off relays from the shield.

This is the script:

#!/bin/bash

GPIO=11
if [ $# == 1 ]; then
        if ! [[ "$1" =~ ^[0-9]+$ ]]; then
                echo "Error: wrong pin format"
                echo "Usage: $0 [GPIO]"
                echo "GPIOs: [0-24]"
                exit
        fi
        GPIO=$1
fi

# export GPIO 11 to userspace if it isn't exported already
if [ ! -d /sys/class/gpio/gpio${GPIO} ]; then
        echo "Exporting GPIO $GPIO"
        echo $GPIO > /sys/class/gpio/export
        sleep 1
fi

if [ "$(</sys/class/gpio/gpio${GPIO}/direction)" != 'out' ]; then
        echo "Setting GPIO $GPIO to OUT"
        echo out > /sys/class/gpio/gpio${GPIO}/direction
fi

if [ "$(</sys/class/gpio/gpio${GPIO}/value)" == 1 ]; then
        echo "Set GPIO $GPIO OFF"
        echo 0 > /sys/class/gpio/gpio${GPIO}/value
else
        echo "Set GPIO $GPIO ON"
        echo 1 > /sys/class/gpio/gpio${GPIO}/value
fi

I decided to use GPIO 11 as default, because it is the closest to pin 25(ground). So it was easier to have the wires close to each other.

Posted by HackMan
Dated: 1st December 2012
Filled Under: Uncategorized
Comments: Post the 1st one!