Lubos Rendek

The only solution is determination.

Simple Way to Get an Average Internet Connection Speed With Bash

| Comments

Here is a simple way to get an average Internet connection speed over a given period of time. For example I would like to test my Internet connection speed every 10 minutes, log the result and calculate average. First run test ( ask you Internet provide to provide you with a local download file.

1
$ for i in $( seq 1 100); do wget http://www.tpg.com.au/downloads/tpgspeedtest.rar &>> download.txt; sleep 600; done

The above will run test 100 times every 10 minutes. It will take 16.6667 hours to complete. As my download file is approximately 8MB in size the whole test will eat up about 800MB.

Once the test is complete we can see download speed of each test with:

1
2
3
4
5
$ grep saved download.txt
2014-06-12 19:56:06 (306 KB/s) - ‘tpgspeedtest.rar.3’ saved [9096028/9096028]
2014-06-12 19:57:26 (320 KB/s) - ‘tpgspeedtest.rar.4’ saved [9096028/9096028]
...
...

From here we can also calculate average download speed.

1
2
$ grep saved download.txt | awk '{ print $3;}'  | sed 's/(//' | awk '{ t += $1; i++ } END { print t/i}'
313

Enable MP3 Layer1 Codec Support on Fedora Linux

| Comments

Due to patents and other copyrighted issues you will not find proprietary codes distributed using Fedora standard repository. In order to install proprietary codes one needs to use third party repository such as rpmfusion.org. After fresh install of Fedora the chances are high that you do not have MP3 support to play your music files. Here is a short howto to help you enable MP3 Layer1 codec support on Fedora Linux. First enable rpmfusion.org repository:

1
# yum localinstall --nogpgcheck http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm http://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm

What remains to be done next is to install codes:

1
# yum install gstreamer1-plugins-ugly gstreamer1-plugins-bad-freeworld

All done. Now you should be able to play MP3 files. Enjoy…!

Gorilla Password Install on Fedora Linux

| Comments

You will not find Gorilla Password application in the standard Fedora repository, but do not despair as you can always install it directly from Gorilla’s git repository. Let’s first install all prerequisites:

1
# yum install tcllib tk itcl git

At this stage we can clone Gorilla Password’s git repository.

1
2
$ cd /tmp/
$ git clone https://github.com/zdia/gorilla.wiki.git

At this point you should have /tmp/gorilla directory available. Let’s copy gorilla’s executable into our /opt:

1
# cp -r gorilla/sources/ /opt/gorilla

Create executable script:

1
2
# echo '/opt/gorilla/gorilla.tcl' > /usr/local/bin/gorilla
# chmod +x /usr/local/bin/gorilla

and the installation is done. Now open your terminal and execute:

1
$ gorilla

TROUBLESHOOTING:

  • make sure that /usr/local/bin is included in your PATH variable

Installation & Getting Started Tutorial With Sencha Ext JS 5 on Ubuntu/Debian Linux

| Comments

In this short guide we will install and build a simple default application with Sencha Ext JS CMD 5. This Getting Started guide is a rewrite of the official OS agnostic guide published on sencha.com. In the first step download Ext JS CMD 5 32bit or 64bit package. If unsure then download 32bit version. Once ready unzip it:

1
2
3
4
5
$ ls SenchaCmd-5.0.0.114-linux-x64.run.zip 
SenchaCmd-5.0.0.114-linux-x64.run.zip
$ unzip SenchaCmd-5.0.0.114-linux-x64.run.zip 
Archive:  SenchaCmd-5.0.0.114-linux-x64.run.zip
  inflating: SenchaCmd-5.0.0.114-linux-x64.run

The above will extract pre compiled SenchaCmd-5.0.0.114-linux-x64.run executable to be used to install Sencha Ext JS CMD 5 locally. In the next step we will install Ext JS CMD 5 in a user space without administrative privileges. First make your *.run file executable:

1
2
3
4
5
$ ls -l SenchaCmd-5.0.0.114-linux-x64.run
-rw-r--r-- 1 lubos lubos 43172707 Mar 28 16:00 SenchaCmd-5.0.0.114-linux-x64.run
$ chmod +x SenchaCmd-5.0.0.114-linux-x64.run
$ ls -l SenchaCmd-5.0.0.114-linux-x64.run
-rwxr-xr-x 1 lubos lubos 43172707 Mar 28 16:00 SenchaCmd-5.0.0.114-linux-x64.run

Now we can start the installation:

1
$ ./SenchaCmd-5.0.0.114-linux-x64.run

Choose installation directory. The default is ~/bin however feel free to choose any other directory within your home directory. The installation will actually create a new directory “Sencha” as a result it is fine to use your home root directory eg. /home/lubos: We are almost ready! Senha installation should modify your .bashrc file to include the following to lines:

1
2
3
$ grep Sencha ~/.bashrc 
export PATH=/home/lubos/Sencha/Cmd/5.0.0.114:$PATH
export SENCHA_CMD_3_0_0="/home/lubos/Sencha/Cmd/5.0.0.114"

The above lines will set our shell environment to make sure that Sencha EXT JS 5 CMD executables are available from a command line. Now either start new terminal or source your .bashrc file:

1
$ . .bashrc

At this point we should be ready to generate new app using sencha command:

1
2
3
4
5
6
7
8
9
10
11
12
13
$ sencha generate app -ext MyApp ./myApp
Sencha Cmd v5.0.0.114
[INF] Source File : http://cdn.sencha.com/cmd/packages/catalog.json
[INF] Downloading : ....................
[INF] Downloading ext package...
[INF] Source File : http://cdn.sencha.com/cmd/packages/ext/5.0.0.736/ext.pkg
[INF] Downloading : ....................
[INF] Extracting ext package...
[INF] Package is already local: ext/5.0.0.736
[INF] Extracting  : ....................
[INF] Loading app json manifest...
[INF] Writing content to /tmp/myApp/bootstrap.js
[INF] Writing content to /tmp/myApp/bootstrap.json

Next, check what java version is installed on your system. Sencha Ext JS 5 CMD requires java version => 1.7. Otherwise, we would would not be able to start a local web development server and we will see error message:

1
2
Sencha Cmd v5.0.0.114
[ERR] App Watch requires a minimum of java version 1.7 (current is 1.6.0.31).

Checking java version:

1
2
3
4
$ java -version
java version "1.6.0_31"
OpenJDK Runtime Environment (IcedTea6 1.13.3) (6b31-1.13.3-1~deb7u1)
OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode)

If your version is 1.7 or higher go ahead and start your development server with:

1
2
$ cd myApp/
$ sencha app watch

Otherwise if you are running Debian you are most likely to have java version 1.6. Next we will install java 1.7 and update system’s java alternatives:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
$ su
Password: 
# apt-get install openjdk-7-jdk
# update-alternatives --config java
There are 2 choices for the alternative java (providing /usr/bin/java).

  Selection    Path                                            Priority   Status
------------------------------------------------------------
* 0            /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java   1061      auto mode
  1            /usr/lib/jvm/java-6-openjdk-amd64/jre/bin/java   1061      manual mode
  2            /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java   1051      manual mode

Press enter to keep the current choice[*], or type selection number: 2
update-alternatives: using /usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java to provide /usr/bin/java (java) in manual mode
# java -version
java version "1.7.0_55"
OpenJDK Runtime Environment (IcedTea 2.4.7) (7u55-2.4.7-1~deb7u1)
OpenJDK 64-Bit Server VM (build 24.51-b03, mixed mode)

Now we are ready to start our development server:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
$ cd myApp
$ sencha app watch
Sencha Cmd v5.0.0.114
[INF] Loading app json manifest...
[INF] Writing content to /tmp/myApp/bootstrap.js
[INF] Writing content to /tmp/myApp/bootstrap.json
[INF] merging resources into /tmp/myApp/build/testing/MyApp/resources
[INF] merged 183 resources into /tmp/myApp/build/testing/MyApp/resources
[INF] merging resources into /tmp/myApp/build/testing/MyApp
[INF] merged 0 resources into /tmp/myApp/build/testing/MyApp
[INF] writing sass content to /tmp/myApp/build/temp/testing/MyApp/sass/MyApp-all.scss.tmp
[INF] appending sass content to /tmp/myApp/build/temp/testing/MyApp/sass/MyApp-all.scss.tmp
[INF] appending sass content to /tmp/myApp/build/temp/testing/MyApp/sass/MyApp-all.scss.tmp
[INF] writing sass content to /tmp/myApp/build/temp/testing/MyApp/sass/config.rb
[INF] executing compass using system installed ruby runtime
   create MyApp-all.css 
[INF] Mapping http://localhost:1841/ to /tmp/myApp...
[INF] ------------------------------------------------------------------
[INF] Starting web server at : http://localhost:1841
[INF] ------------------------------------------------------------------
[INF] Waiting for changes...

Now you are ready to see your first default sencha EXT JS 5 page by following http://localhost:1841 URL. Enjoy ! …

Don’t Get Ripped Off With Domain Name Registration Service CrazyDomains.com.au

| Comments

Here are few notes to read before you decide to register or move your domains to CrazzyDomains.com.au and why CrazyDomains are crazy. Here is how the story goes.

Back in 13 May 2011 I bought myself a co.uk domain with CrazyDomains.com.au for a price of $38 for 2 years. Back then I assumed that this fairly new company will provide me with cheap domain name registration options and since I already had some other domains registered with them I also acquired my .co.uk domain with them.

Two years later I have renewed my domain for $19 for another 1 year, which was exactly the same price as before: Today was a time for another renewal and the price tag CrazyDomains.com.au put on my renewal request just blow me away. They simple have no decency and respect to their customers as they asked me $49 for 1 year renewal. This is why I believe the word “Crazy” in their company name fits best and why they are called CrazyDomains.com.au. Once really needs to be crazy to buy domain with such a price tag. Their strategy is to put domains on sale however if you would like to renew the sales price tag does not apply to you: So the strategy of their business is not new and fairly obvious. They talk about nice prices when they try to catch new customers, but once you are locked in they charge you whatever they want since they can always put domains on sale for new customers. Instead of renewal of my domain name with CrazyDomains.com I have transfered it to my other registrant ntchosting.com which is part of Liquidnet Ltd.

Please note I’m not advocating that ntchosting.com is better domain name registrar and you should do the same! However, to me it seemed as the only option to transfer away from CrazyDomains.com.au. All what needs to be said is that before you decide to renew or purchase a domain with CrayDomain.com.au do your research first as there are probably thousands of other Domain Name registrars with better price as for example in this instance ntchosting.com. So for the same domain I got largely different prices. So for 9 year renewal with crazy domains I would have end up paying $441: Whereas with NTC Hosting for example I would pay $67.50 for 9 years renewal:

One year renewal price comparison:

1
2
3
4
CrazyDomains.com.au:
1 year .co.uk renewal - $49
NTC Hosting:
1 year .co.uk renewal/transfer - $7.50

Now this is what I call CrazyDomains price difference for the identical item.

Mount External With Full User Read/write Permissions

| Comments

By default mounted hard drive does not provide unprivileged used with write permissions. To ensure that users are able to write to a mounted hard drive use “mask=0” as a mounting option or in your fstab file. For example:

1
UUID=8AC2-B6F3  /home/lubos/mydisk            vfat    users,noatime,umask=0 0 0

Mount External USB Disk With Full User Read/write Permissions

| Comments

By default mounted hard drive does not provide unprivileged used with write permissions. To ensure that users are able to write to a mounted hard drive use “mask=0” as a mounting option or in your fstab file. For example:

1
UUID=8AC2-B6F3  /home/lubos/mydisk            vfat    users,noatime,umask=0 0 0

GNU-R MySQL Database Connection Examples With RMySQL Interface

| Comments

Here are few tips on how to get started with GNU-R and MySQL database. Before with can start using MySQL database with GNU-R first we need to make sure that we have a proper MySQL database interface installed. On Debian Linux this is easy as:

1
# apt-get install r-cran-rmysql

The above will install RMySQL nad MySQL database interface for GNU-R. Start R and confirm the installation:

1
2
3
4
> require("RMySQL")
Loading required package: RMySQL
Loading required package: DBI
> 

In the next step we need to create a database handle. Our database has name “mydata”, is located on “localhost” and the access to this database is given to user “lubos” with password “pass”.

1
dbh <- dbConnect(MySQL(), user="lubos", password="pass", dbname="mydata", host="localhost")

Our sample database contains only a single table “cost” with 3 columns:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
mysql> select * from cost;
+----+---------+---------+---------+
| id | column1 | column2 | column3 |
+----+---------+---------+---------+
|  1 |      54 |      68 |      74 |
|  2 |      54 |      67 |      77 |
|  3 |      54 |      67 |      78 |
|  4 |      54 |      67 |      76 |
|  5 |      54 |      67 |      79 |
|  6 |      54 |      67 |      74 |
|  7 |      54 |      67 |      75 |
|  8 |      54 |      67 |      68 |
|  9 |      54 |      67 |      63 |
| 10 |      54 |      65 |      63 |
+----+---------+---------+---------+
10 rows in set (0.01 sec)

To get more information about our database using GNU-R and RMySQL we can can use our previously created database handle “dbh”. First we can list tables:

1
2
> dbListTables(dbh)
[1] "cost"

Next we can also list all fields within our table:

1
2
> dbListFields(dbh, "cost")
[1] "id"      "column1" "column2" "column3"

Now that we know how our database looks like we can start to run some SQL queries. Let’s fetch all data from our “cost” table:

1
2
3
4
5
6
7
8
9
10
11
12
13
dbGetQuery(dbh, "SELECT * FROM cost;")
   id column1 column2 column3
1   1      54      68      74
2   2      54      67      77
3   3      54      67      78
4   4      54      67      76
5   5      54      67      79
6   6      54      67      74
7   7      54      67      75
8   8      54      67      68
9   9      54      67      63
10 10      54      65      63
>

In the next example we are going to make a sum of all values in column2:

1
2
> sum(dbGetQuery(dbh, "SELECT column2 FROM cost;"))
[1] 669

The above is the same as storing results in a matrix and perform sum only on column2:

1
2
3
4
5
6
> out = dbGetQuery(dbh, "SELECT * FROM cost;")
> sum(out$column2)
[1] 669
> sum(out$column3)
[1] 727
>

For a small data set like the one above dbGetQuery function is satisfactory. However, for a large data sets we may want to use dbSendQuery function. In contrast to dbGetQuery the dbSendQuery allows us to retrieve data in batches. Here is how dbSendQuery works. First we send mysql query:

1
> out = dbSendQuery(dbh, "SELECT * FROM cost;")

Now retrieve three separate batches of information. batchOne and batchTwo will hold first 6 results and batchThree we store rest of the results ( -1 ).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
> batchOne = fetch(out, n = 3)
> batchTwo = fetch(out, n = 3)
> batchThree = fetch(out, n = -1)
> batchOne
  id column1 column2 column3
1  1      54      68      74
2  2      54      67      77
3  3      54      67      78
> batchTwo
  id column1 column2 column3
4  4      54      67      76
5  5      54      67      79
6  6      54      67      74
> batchThree
   id column1 column2 column3
7   7      54      67      75
8   8      54      67      68
9   9      54      67      63
10 10      54      65      63
> 

To see how many rows have been fetched we can use dbGetRowCount function:

1
2
> dbGetRowCount(out)
[1] 10

To know whether query has completed use dbHasCompleted:

1
2
> dbHasCompleted(out)
[1] TRUE

In case that we no longer require our “out” results we can clear them with:

1
2
> dbClearResult(out)
[1] TRUE

Once we have finished with MySQL database we can disconnect from it using:

1
2
> dbDisconnect(dbh)
[1] TRUE

Using GNU R in scripts

If from some reason you need to run the above as a GNU R script in your shell you may want to install “little” the GNU R scripting front-end. On Linux the installation is simple as:

1
# apt-get install littler

At the point we are ready to put the above MySQL and GNU R lines into a script. Create a new file with some arbitrary name like gnurscript.R and include a following content:

1
2
3
4
5
#!/usr/bin/env r

require("RMySQL")
dbh <- dbConnect(MySQL(), user="lubos", password="pass", dbname="mydata", host="localhost")
print(dbGetQuery(dbh, "SELECT * FROM cost;"))

The first line will set your interpreter to GNU R using littler scripting front-end. Apart of the print() function the rest of the GNU R code is same as above. Make the script executable:

1
$ chmod +x gnurscript.R 

and finally run your script:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
$ ./gnurscript.R
Loading required package: RMySQL
Loading required package: methods
Loading required package: DBI
Loading required package: utils
   id column1 column2 column3
1   1      54      68      74
2   2      54      67      77
3   3      54      67      78
4   4      54      67      76
5   5      54      67      79
6   6      54      67      74
7   7      54      67      75
8   8      54      67      68
9   9      54      67      63
10 10      54      65      63

If you wish to suppress the messages upon loading GNU R libraries, packages etc. simply change the “require” line to:

1
suppressMessages(require("RMySQL"))

Simple Way to Schedule Torrent Download

| Comments

There are times that you want to schedule your torrent or any other download for a particular time. There maybe a number of reasons to do so and one of them maybe a peak and off peak download limitations enforced by your internet provider. If your application does not have a scheduled timer for download you can simply use iptables to block all the traffic and re-enable it at any given time with at command. Here is an example of scheduling download ( enabling network traffic ) at 2AM while your torrent client remains started at all times.

First block your network traffic with:

1
2
# iptables -P INPUT DROP
# iptables -P OUTPUT DROP

At this point you will notice that your network traffic whether incoming or outgoing is down. Next, create a iptables restore file:

1
2
3
4
5
6
7
cat restore-network
# Generated by iptables-save v1.4.14 on Tue Apr 22 19:21:25 2014
*filter
:INPUT ACCEPT [303:50630]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [3780:635312]
COMMIT

Create a at command text file with a commands you which to execute. In our case the file will contain only a single line:

1
2
# cat restore_traffic
/sbin/iptables-restore /root/restore-network

Once ready execute at command and include your commands list file:

1
2
3
# at 2AM < restore_traffic
warning: commands will be executed using /bin/sh
job 9 at Wed Apr 23 02:00:00 2014

Please note that you need to run at command as privileged root user. It is also possible to include command directly to at command.

Import Data From CSV File to MySQL With Bash Script

| Comments

Here is what you need to this simple demonstration on how to import data from CSV file to MySQL database. Fist we need a comma separated value file like:

1
2
3
4
5
6
7
8
9
10
11
$ cat input.csv 
54,68,74
54,67,77
54,67,78
54,67,76
54,67,79
54,67,74
54,67,75
54,67,68
54,67,63
54,65,63

Next, you need MySQL database with at least one table. Our database is called “mydata” and contains a single table named “cost”:

1
2
3
4
5
6
7
8
9
10
mysql> describe cost;
+---------+---------+------+-----+---------+----------------+
| Field   | Type    | Null | Key | Default | Extra          |
+---------+---------+------+-----+---------+----------------+
| id      | int(11) | NO   | PRI | NULL    | auto_increment |
| column1 | int(11) | NO   |     | NULL    |                |
| column2 | int(11) | NO   |     | NULL    |                |
| column3 | int(11) | NO   |     | NULL    |                |
+---------+---------+------+-----+---------+----------------+
4 rows in set (0.00 sec)

Lastly, we need a bash script to import our comma separated CSV file into our table. The file can look like the one below:

1
2
3
4
5
6
7
8
9
$ cat import-SQL.sh 
#!/bin/bash

IFS=,
while read column1 column2 column3
      do
        echo "INSERT INTO cost (column1,column2,column3) VALUES ('$column1', '$column2', '$column3');"

done < input.csv | mysql -u myusername -p mypassword mydata;

Most of the stuff above is self explanatory except IFS variable. IFS or Internal field separator is a bash buildin bash variable which is by default to set to contain space like “ ”. It is used to read input and since now our input is comma separated we set IFS to “,”. Ok, enough talk, let’s run the script:

1
$ ./import-SQL.sh

No ouptut, good output. Let’s check our database:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
mysql> select * from cost;
+----+---------+---------+---------+
| id | column1 | column2 | column3 |
+----+---------+---------+---------+
|  1 |      54 |      68 |      74 |
|  2 |      54 |      67 |      77 |
|  3 |      54 |      67 |      78 |
|  4 |      54 |      67 |      76 |
|  5 |      54 |      67 |      79 |
|  6 |      54 |      67 |      74 |
|  7 |      54 |      67 |      75 |
|  8 |      54 |      67 |      68 |
|  9 |      54 |      67 |      63 |
| 10 |      54 |      65 |      63 |
+----+---------+---------+---------+
10 rows in set (0.00 sec)