Browsing the archives for the Linux category.

Testing MySQL performance (SELECT query)

Applications, Coding, Linux, Web developing

Is SELECT column1,column2 FROM table faster than “SELECT * FROM table? Should I only pick data that I need, or just take everything?

I personally like to take everything, because it is much easier to develop applications that way, but many sites recommends to take only data that is needed because of speed.

So, I wanted to test this and see if there is a major speed difference.

On small databases (something like 10-1000 rows) there is almost none difference. I wanted to see the difference on a huge database, so I wrote little script that will fill the database with 1 million rows of random data. Then I coded another script for testing the speed differences on SELECT query.

Scripts used to build the test

Here is the database structure:

mysql> desc test;
+---------+--------------+------+-----+---------+----------------+
| Field   | Type         | Null | Key | Default | Extra          |
+---------+--------------+------+-----+---------+----------------+
| ID      | int(11)      | NO   | PRI | NULL    | auto_increment |
| field_a | varchar(500) | NO   |     | NULL    |                |
| field_b | varchar(500) | NO   |     | NULL    |                |
| field_c | varchar(500) | NO   |     | NULL    |                |
| field_d | varchar(500) | NO   |     | NULL    |                |
+---------+--------------+------+-----+---------+----------------+
5 rows in set (0.00 sec)

And here is the script that I used to fill the database:

<?php

echo("Connecting to the database...\n");
mysql_pconnect("localhost","lauri","") or die();
mysql_select_db("test") or die();

echo("Starting to fill the database...\n");

$amount = 1000000;
for($i = 0;$i < $amount;$i++){
$rand1 = random();
$rand2 = random();
$rand3 = random();
$rand4 = random();

query("INSERT INTO test(field_a,field_b,field_c,field_d) VALUES('$rand1','$rand2','$rand3','$rand4');");
echo("Value: $i \n");
}

--snip--

?>

(In the –snip– there is only the random() function creating random 9 char strings, and my query function that does normal querys to the database)

And here is the script that I used to test the SELECT speed:

<?php
echo("Connecting to the database...\n");
mysql_pconnect("localhost","lauri","") or die();
mysql_select_db("test") or die();

$stimer = explode( ' ', microtime() );
$stimer = $stimer[1] + $stimer[0];
echo("Executing full select\n");
$result = query("SELECT * FROM test");
$etimer = explode( ' ', microtime() );
$etimer = $etimer[1] + $etimer[0];

$diff = $etimer-$stimer;
echo("TIME: $diff\n");

$stimer = explode( ' ', microtime() );
$stimer = $stimer[1] + $stimer[0];
echo("Executing partial select\n");
$result = query("SELECT field_a FROM test");
$etimer = explode( ' ', microtime() );
$etimer = $etimer[1] + $etimer[0];

$diff = $etimer-$stimer;
echo("TIME: $diff\n");

--snip--
?>

(Snip contains the database query function, I stole found nice timer script from desilva)

Results

I executed the testing script against 1 million rows of random data, here is the results:

Connecting to the database...
Executing full select
TIME: 1.77005004883
Executing partial select
TIME: 1.03216195107

So there is only 0.7 second difference with million rows of data…

In my opinion, the 0.7 second difference is so small that if you are working on project that doesn’t have massive databases, you shouldn’t worry about selecting only fields that you need, but if you are working on something that has lots of users and huge databases, selecting only data you need may help speed of the script lot.

After the tests I realized that the varchar size of 500 is way bigger what I used… I changed the varchar size to 10 and executed the tests again. Results here with varchar(10):

Connecting to the database...
Executing full select
TIME: 1.71709799767
Executing partial select
TIME: 0.975940942764

Not huge difference either, but if you need all the speed you can get, make the sizes of columns right… Not way too big like I did.

System

I used my home-server with 3Ghz Pentium 4 processor with hyper-threading. 2 gigs of ram and normal 7200 rpm 80g hard-disk. Operating system was Debian with normal installation of PHP5 and MySQL via apt-get with default settings.

No Comments

Rackspacecloud VS Amazon EC2

Applications, Coding, Internet, Linux, Review

I have written about Amazon EC2 some time ago. I liked the EC2, but now I found new competitor that is better than EC2. Rackspace cloud is easier to use and has great support.

Prizing
Both Amazon and Rackspace offers virtually unlimited computing power/file storage for low cost pay-as-you-go prizing.

It looks like Rackspace offers cheaper servers. Rackspace also has small 256m server for 0.015$ and that is my most used server, because I usually don’t need big server.

Managing servers
Here is the point where the real difference becomes. Amazon has some kind of weird command line tools for managing the servers, firefox addon and a online  management console. The management console isn’t the best system, but it works. The main problem is that connecting to the server with ssh is damn difficult. It took 1hour for me first time to get the connection to the server…

Rackspace offers simple solution to connecting to the servers. They email the random server password for you. Then to can connect with normal ssh client to standart ports. No unnecessary key stuff etc. Rackspace also offers ajax console.

Support
I haven’t tried amazons support, because I haven’t a) needed help b) found where to get help

Rackspaces live support is awesome! There is always somebody to answer my questions.

In my opinion…

…Rackspacecloud is much better.

1 Comment

Uptime

Linux, iPod touch/iPhone apps

Uptime. Simple value that tells how long something has been running.

It is is way to see stability of the system/app, but It is very accurate way to check it.

Mostly uptime is used to show that someone has better uptime than someone else. With servers it is much easier to get hundreds days of uptimes, but with personal computer it is little harder.(?) There is more power outages and system is usually used daily, so changes that something goes wrong and screws up system. Updates are also very popular reason to reboot computer.

Most of computer owners shut downs the system for night. This is the most popular way to ruin uptime. I personally have to close my computer, because my hard-disc is bit noisy and I can’t sleep well while my computer is running…

I have currently 2 systems where I have semi-good uptimes. (imo)

  • Irssi 70d uptime and counting| last reason to close: server reboots
  • iPod Touch 18d uptime and counting| Last reason to boot: battery outage

IPods uptime may not be great, but the irssi uptime is one of the highest that I have seen. If you have seen better uptime, please tell me :) Mostly I only try to beat my own uptimes.

No Comments

Istat remote monitoring app

Applications, Linux, Mac, iPod touch/iPhone apps

Istat is a iPod touch/iPhone app developed by bjango. With istat you can remotely monitor ram/mem/disk usage of your home computers or remote servers.

Istat server is available for mac and Linux. Linux version must be compiled manually, but it isn’t very difficult task if you have compiled stuff before. One difference on Linux version is that you can’t track temperatures. There is no support for windows.

You can track:

  • Mem usage
  • CPU usage
  • Disk usage
  • Net usage
  • Uptime
  • Load
  • Temperatures

You can also view information from your iPod/iPhone. One cool feature is “free memory”. With free memory command you can clear iPods/iPhones memory and make it run faster. (?) You also find id number and mac address.

There is also good traceroute and ping tools. Only thing I would like to have is pinging range of ip:s.

This app is now on sale 1.5e and it is worth of that 1.5e

No Comments

How to do a quick backup at console

Linux, Tutorial

So, you are coding your project and you realize that you haven’t done any backups! What is the easiest and fastest way to do backup (I don’t know about speed and difficult of this method, but it works and I like this.)

Dump your databases! (if you have any)

Mysqldump --databases dbname > dump.SQL

Then move to parent directory

cd ..

And do a tar archive

tar cvvf project.tar project/

And voilà! Local backup is ready, let’s move it to safe place in your ftp server…

ftp ftp.example.com

Use ls and cd to find right folder on your server, then move local file with:

put project.tar

Whee! Now you have backups safely on your server! It is possible to write a cron script that backups and uploads your project every hour.

No Comments
« Older Posts