Adventures in Tech

Next Page

The mass of systemsI have recently changed by computer setup and I thought you might all be interested in having a look.

Click the photo to travel to the flickr


Gear shown in the photos is as follows:

Main computer:
AMD XP 2600 barton core
1GB Corsair DDR ram
3x300GB and 1x150GB hard drives giving 1.050TB
Leadtek Geforce FX 5600 video card
Sound blaster Audigy ZS sound card.
3x10/100 network cards
DVD burner

Server box
Celeron 333
256 MB sd ram
1x20GB hard drive.
3x10/100 network cards

Ibook
1.2Ghz G4
512MB DDR ram
20GB hard disk
DVD rom - CD burner

Dell - Latitude | D610
Pentium M 2.0GhZ
1GB ram
60GB hard disk
DVD burner

Switch
Level one FSW-1610TX
16 port 10/100 switch

Wireless access point
Linksys WAP54G

Belkin Surge Pro F9D800au-5M-ORG

AOC 17" LCD monitor 8MS


The Dell is the newer system in the setup, however that's not the interesting part.
I control all these systems from the one keyboard and mouse.
To switch from my main system to the ibook laptop I just move the mouse off the left edge of the main screen and it moves onto the laptop screen.
To move to the dell I move off the right side of the main monitor and onto the dell system.
Keyboard and mouse work prefect AND even the clipboard gets copied across, its fantastic! The program I use to do this is synergy http://synergy2.sourceforge.net/.
It runs as a server on my main box and the client system (dell and ibook) all connect in. Its totally cross platform and currently I have it running on OS X, Kubuntu linux and windows XP. I have also confirmed it works on vista.

Frankly I see this as better than dual monitors as it means I can hammer one system and still have a responsive system to use.
I have also noticed that synergys still works well when cpu load on the server system is 100%

Hope everyone had a happy new year and a nice christmas.

Later!

Caveman

---

Backups 101Backup scripts 101

Finally I get around to posting an entry about how my backup scripts work.
Firstly: I am going to explain how my backups get from the different systems on my network to the new backup box (see my last post for more info on the backup box).

Biocorporation (my main desktop box) is normally on 24/7 and goes for weeks at a time without a reboot. Normally it only gets rebooted if something horrible happens and it eats it tail or I do a heap of updates and think its "due".
On Biocorporation I have the following script:
START SCRIPT
#!/bin/bash

RSYNC=/usr/bin/rsync
{
cd /home/caveman/scripts/ $*

date > server-lastbackup.log
$RSYNC -rptgoDvL /data/backuplinks/ backupuser@10.1.1.11::data >> server-lastbackup.log&

}
END SCRIPT

Ok so go through the script.
The { cd section changes the script path, I will explain why in a sec.

Date > server-lastbackup.log puts the date into the top of a log file. This log file in stored in /home/caveman/scripts but we don't have to put that path because of the CD done above.

The rsync command is the business end of the script. I have folders all over the place, for example all my emails are in /home/caveman/Mail, all my pictures are in /data/photos. So instead of having to run a rsync command for each folder I link all the folders I want to backup in to /data/backuplinks then I use basically rsync -a with an option to FOLLOW system links.
NOTE: this would cause problems if using rsync to backup the whole system for that I recommend -a.
Now this copies all my data over to the backup box, I run this as a cron job around lunch time when I am not at my computer.

On the backup box I have the following 2 scripts.
START SCRIPT
#!/bin/sh

BACKDIR=/data/backup/caveman/daily
MV=/bin/mv
RM=/bin/rm
CP=gcp

if [ -d $BACKDIR/day7 ]; then
$RM -rf $BACKDIR/day7 ;
fi;

if [ -d $BACKDIR/day6 ]; then
$MV $BACKDIR/day6 $BACKDIR/day7 ;
fi;

if [ -d $BACKDIR/day5 ]; then
$MV $BACKDIR/day5 $BACKDIR/day6 ;
fi;

if [ -d $BACKDIR/day4 ]; then
$MV $BACKDIR/day4 $BACKDIR/day5 ;
fi;

if [ -d $BACKDIR/day3 ]; then
$MV $BACKDIR/day3 $BACKDIR/day4 ;
fi;

if [ -d $BACKDIR/day2 ]; then
$MV $BACKDIR/day2 $BACKDIR/day3 ;
fi;

if [ -d $BACKDIR/day1 ]; then
$MV $BACKDIR/day1 $BACKDIR/day2 ;
fi;

if [ -d $BACKDIR/day0 ]; then
$MV $BACKDIR/day0 $BACKDIR/day1 ;
fi;

if [ -d $BACKDIR/day1 ]; then
cd $BACKDIR/day1 && find . -print | cpio -dpl ../day0 ;
fi;

touch $BACKDIR/day0 ;

chown -R backupuser:caveman /data/backup/ ;
END SCRIPT

I am not going to explain this in detail however this site does a great job.

The basic idea is the first days backups go to day0 then on the second day they get moved to day1 and day0 gets recreated, however this is all done without using heaps of disk space. Due to the linking only changes have to be stored again, files that are the same as a past backup are just linked to. This is something you should understand before using these scripts, I am not going to cover it today as I will take be all day, however the link above does a good job.

I should note I use cd $BACKDIR/day1 && find . -print | cpio -dpl ../day0 ;
Instead of a cp -al command as I am using openbsd on the backup box which by default does not have the gnu copy util and I also found it to be just a touch faster.

Now the above script gives me a 7 day rolling backup, which is great.
However I also do a monthly backup.

START SCRIPT
#!/bin/sh

BACKDIR=/data/backup/caveman/monthly/
MV=/bin/mv
RM=/bin/rm
CP=gcp

if [ -d $BACKDIR/month3 ]; then
$RM -rf $BACKDIR/month3 ;
fi;

if [ -d $BACKDIR/month2 ]; then
$MV $BACKDIR/month2 $BACKDIR/month3 ;
fi;

if [ -d $BACKDIR/month1 ]; then
$MV $BACKDIR/month1 $BACKDIR/month2 ;
fi;

cd /data/backup/caveman/daily/day0/ && find . -print | cpio -dpl /data/backup/caveman/monthly/month1/ ;


touch $BACKDIR/month1 ;



chown -R backupuser:caveman /data/backup/ ;
echo "monthly caveman backup script ran" ;
END SCRIPT

This script takes the month 3 backup and removes it' then it makes month 2 month 3 and month 1 month 2.
Then it creates month1 from day0 backup.

Now the really cool part of all this is the linking.
If I have a 10GB file that never changes, using the above backup system it will only be stored on disk once.
Even if the file exists in month 3, month 2, month1 and days 7 to 0 as long as its unchanged the file will only be stored once.
If I make a change the file will then be stored twice until the old version of the file gets removed at the end of 3 months.

This backup system provides me with a daily backup for 7 days and monthly backup for 3 months.
Currently I have 32GB of data on my biocoporation deskop that needs to be backed up.
The current size of my whole backups (7 days + 3 months) comes in at 40GB.
Which means I have made 8GB of changes in data.

Now backing up the windows systems is not so simple, no rsync!
However it's still easy enough to do.
The same scripts are used on the backup server side, however in addition to these you need to have a samba share that points to the day0 folder and is access able by the windows system.

I then use a little tool called Karen's Replicator http://www.karenware.com/powertools/ptreplicator.asp

This tool will find changes in files and with the right settings can be programmed to remove files from the destination that don't exist on the source.
It can also be set to only copy files that have changed. One small fault with this is that it has a hard time with outlook and outlook express mail files. Even if you just open outlook and don't make any changes to file it will still copy the whole file over in the backup as it thinks it's changed.
I believe this is because of how the outlook data file works, I suspect it would be very hard to get around this without having code that could open the pst file.

The hard part was working out how I was going to run this file, the windows computers in the house are not on 24/7 and are on and off at odd times.
The best solution I could find was to run the backup are log off. Turns out this works really well. You shut your computer down, turn off the screen and go to bed. The backups will go and once done the computer will shutdown on its own.
I should also add that if Karen's Replicator can't find the samba share due to the network being down or the server offline it won't hang and crash logout. It just gracefully ends and goes to shutdown.

I run Karen's Replicator by creating a shortcut to the backup job, and popping that into the C:\WINDOWS\system32\GroupPolicy\User\Scripts\Logoff folder.
Even if you are not connected to a domain controller and hence not using a group policy anything in this folder will be executed on logoff.

And its done!
So what about you Mac people? Well OS X has rsync, do I need to say any more?

Hope this is useful to someone one day!

Caveman

---

Back from the lost land - And backups!SO.. Its been awhile since my last post.
But I have been busy, or that's the excuse this time.
I have lots of news to tell but instead of one huge post I will do this bit by
bit.
Firstly let me finish off what I ended up doing for my backup solution.

All the parts arrived and I put the system together. The mini-itx board is
surprisingly easy to get into a mini-itx case. The instruction booklet that
comes with the board is very wanting however. Which means one has to put some
thought into wiring up the USB ports.

I decided to give freenas a try and was less than impressed. I think freenas
would be fine if you wanted just a dumb nas box and that's all. However I was
looking for run things in cron which freebsd does not ship with and I also had
problems getting samba to authenticate share with username and password.

Next step was openbsd. I have been wanting to try openbsd for ages as I have
seen some very impressive things with its firewall. The other selling point was
one remote hole in 8 years.. Not bad.
Installing openbsd however is a bloody pain in the backside, I am sure its fine
if you have done it 50 times before but the first few times suck.
The main issues I had were with partitioning the disk and volume sizes.
However once past this the rest of the install was fairly easy. One word of
warning here: If you have never installed a linux distro before you will find
the openbsd install really hard!

Openbsd takes me back to my slackware days, with the easy to understand rc
scripts and none of the init.d stuff that makes my head hurt.
The default openbsd install is very basic, with almost nothing enabled by
default, which for my uses was fantastic!
It does have a few "security" items, which did throw me. Some demons wont run if
their config file has the wrong permissions this includes execute permissions as
root.
Once the system was installed I went and installed samba by following the
documentation on the openbsd web page.
Next I created the backup folders and started my backups using rsync over ssh.
It was then that I noticed that the network performance was really low (2-3MB/S)
over a 100MBIT link.
After some reading of the documentation I discovered that the key sizes on
openssh on the latest openbsd are larger and with my lower end hardware this was
causing the slow down. Instead of changing the key sizes I decided to run a
rsync server.
The rsync server is one of the simpler server demons out there. It also totally
fixed the speed problem and makes me recommend using the rsync demon unless you
need the security of ssh.
I would still use ssh for any transfers over the internet.

As for the backup scripts I am using? Well, that's coming in another post.

Later!

---

Mirco sized backup systems - The dilemma! - Part 2There has been progress since my last post!
I have decided that I will keep my current server/router/kitchen sink and get another system for the backups.

I figured out I can put this backup system on one of the shelves in the garage and then wire up a cat5 socket and a power point for it. The good part of this is its in a totally different location in the house and gets my backups off my server.

Hardware was a hard one. I decided to go with one of the VIA embedded mini-itx motherboards. It has a 600Mhz processor, one ddr ram slot, one pci slot, sound, vga, 10/100 and all the IDE stuff. The motherboard cooling is totally fanless.

All I had to do then was to decide on was a case or a build my own. Personally I hate building computer cases so I decided on Morex Venus 669 Case. I would have loved to go for the Travla C147 1U Rackmount Dual Mini-ITX Case which would have looked great sitting next to my rack mount switch, however at 169pounds, I thought better of it.

I am going to boot the operating system which will probably be some kind of BSD from a compact flash card using a compact flash to IDE adaptor. The aim of this is to allow the hard disks to spin down when not being used, which will save power and keep the heat down.

The motherboard comes with its own 10/100 network card, but I am thinking I might also get a gigabit card to go with that so I can have a super high speed link to my system and the 10/100 can go to the switch. Its not like I don't have enough network cable spare to run two wires to garage!
Unless I get a gigabit switch, but I think that might be overkill.

I am hoping that I might be able to get away with either no fans at all OR a 120mm slow speed temperature threshold activated fan.
Also looking at using wake-on-lan to run the system only during certain hours when its needed.

I am very interested to see how the compact flash part works out. I have read about a few people using this to run small embedded style operating systems and it seems to have worked well. The only thing I worry about is all the write accesses to the card that an operating system will generate killing the card really quickly.

As a side note: when I get around to building a new router box I am certainly looking at using a mini-itx board, a Morex Cubid 3688 Mini-ITX Case. a 2.5" hard disk and a VIA EPIA CL6000E motherboard. The end result would be tiny! The CL6000E has dual 10/100 network interfaces and is prefect for a tiny router based setup. The only catch might be at 65mm(H) x 210mm(D) x 258mm(W) it might become a cat toy!


Till part 3!

Caveman

---

Mirco sized backup systems - The dilemma! I must warm you this is a case of me trying to get my ideas into order! So its not the best reading material ever.

I have this poor long suffering Celeron 333Mhz, 256MB-ram system with just over 30GB of disk space in 2 drives.
The two fans that came with it (one for case air and the other in the power supply) both started making noises so I cut the power lines to them. The case was only ever designed to take one 3.5" disk drive, but I have managed to fit 2 in.
The system currently runs as my network router, email server, development box, vpn end point, back up server and is basically the kitchen sink. It runs 24 hours a day and never gets any maintenance to the hardware.
Despite all the abuse its had to take its still going strong and sitting with 70+ days uptime, not that update is anything to write home about.

But I have run into a problem, with both Mum and myself using it as a backup server I have run out of disk space. Putting another disk in is an option, although I worry that a bigger disk may well cause the whole thing to die from the load and heat. Another issue is this system is so old the IDE controllers only support 3MB/s transfers.

There are three options I am looking at currently.
1.
Get a new server, totally replace what I have and get something bigger and faster.
Advantages are that I get a faster server with new hardware
Disadvantage is that I again end up with all my backups being on my router box, which is not a great idea.

2.
Leave my current server as it is and get another system.
Advantages, backups are on a different box which could also be stored in a different location in the house.
Disadvantages, yet another computer in the house (we only have 6 now!), finding somewhere to put it as I don't have any room were the rest of my systems are and more power being used by yet another computer on 24/7.

3.
Get somekind of tiny NAS box.
Advantages, backups are on a different box which could also be stored in a different location in the house, small form factor, less power usage.
Disadvantages, most expensive option and not customisable.


With option 3 being my favourite I have been looking at build your own micro size nas box. Sounds like a relatively easy idea until you start trying!
My ideal design is this.

Super low power CPU (.5-1GHz), 100MB+ ram, passive cooling (ie no fans), 2x3.5" drive bays.
Operating system (probably freenas) booted from a compact flash card, 100 or 1000Mb ethernet.
Microsized case (shuttle size if not smaller).

Having browsed around the web for the best part of a week I have not found many solutions, I found a really cool product called LinkStation made by Buffalo tech. System runs a form of linux which is booted from a CF chip. Unfortunately its $250US which works out to $330AU and that's forgetting postage! As I have not been able to find an Australian retailer.

I also found the kurobox which is a linkstation without any disk installed, however again I seem only to be able to buy this in the US and ship out here. The lowest price I could find was something in the order of $220 and this is without a HD!! I did like the kurobox as there are a lot of linux modifications for it which is great and would make it ideal, but not for that price.

Next stop was the lower power, low cpu computer. Seems a lot of these only have room for 2.5" drives not 3.5".
mini-box make some very interesting stuff, I was impressed with their VoomPC Enclosure for Car Applications although its nothing a friends car stereo, which of course was discontinued, not that I am bitter or anything like that ;).
The Mini-Box-M300 looks great! Its around the right size and I believe it can fit one 3.5" drive despite what they say on the documentation, is aimed to take linux and is super lower power!. But again it looks like a import job and that starts to get expensive!

My last option is to do something like the guys at Mashie Design have done.
I am thinking I may go down this build it from scratch method.

However if you have done something like this before or know of a cool product I have missed be sure to kick me an email

Caveman

---

Story of the lost iPod - Serial number trackingWhen dad was coming home on the bus on Thursday we found a case lying just on the floor just as he was getting out.

When he got home we discovered this case had a 30GB black Video iPod. That's right the latest one!
It looked amazingly clean and I suspect it was very new with the name Brendan engraved on the back.
On Friday we ran the bus company but no one had reported a lost iPod.... So I decided to try something.

All iPods have a unique serial number on the back, therefore apple should be able to tell me at least where it was shipped.
Rang apple and got transferred to warranty department who took the details and said they would get in touch with the owner! I was impressed.
Less than 2 hours later I had a phone call from Brendan and just a couple of hours after that we dropped by and picked up the iPod!

Thumbs up to apple who clearly have a decent part tracking system!

So next time you find something, see if you can track the owner down by the serial number. And remember, if you are going to do crime make sure you don't drop your iPod!!

---

Next Page