Automated offsite backup.
Posted by Stian | Filed under Linux
In order to sleep better at night I’ve decided to be better at taking backups of important data. Knowing myself it would need to be automated, and also preferably offsite. Also didn’t want to depend on my own servers (since they are down from time to time, lately I’ve been fighting an ethernet switch that stops working until someone cycles the power.. very annoying, it’s getting replaced now..).
I had a look at amazon S3, which seems to be fairly popular these days. $0.15 per gb per month of storage, and a little bit for bandwidth. That’s basically free for my storage needs, especially considering the current dollar value (when are they going to rename it american pesos?). Sounds great, cheap offsite backup. However I want to encrypt my data, since I don’t trust anyone further than I can throw them in matters like this, and it’s hard to throw someone the size of amazon..
I had a look around and found duplicity, which does encrypted incremental backups. It’s made in python and uses librsync, it supports loads of different destinations - for example scp, rsync, ftp and even S3!
So I installed the newest version of duplicity and it’s dependencies ( boto - for S3 support, the rest can be found in most distributions ) and also made a GPG-key specifically for backup use (gpg –gen-key).
And made a little script:
#!/bin/bash
# Amazon S3 keys:
export AWS_ACCESS_KEY_ID="XXXXXXXXXXXXXXXXXX"
export AWS_SECRET_ACCESS_KEY="XXXXXXXXXXXXXXXXXXXXXXXXXXXX"
# GPG passphrase and key:
export PASSPHRASE=XXXXXXXX
GPG_KEY=XXXXXXXX
# MYSQL password:
MYSQLPW=XXXXXXXX
DATE=`date +%d`
SOURCE=/
DEST="s3+http://nameofyourbackupbucket"
mysqldump --all-databases --password=$MYSQLPW > /root/mysql/mysql-backup.sql
# Force a full backup twice a month..
if (( "$DATE" % 15 == "0" )) ; then
duplicity full \
--include-globbing-filelist /root/backuplist.txt \
--encrypt-key "$GPG_KEY" \
--sign-key "$GPG_KEY" \
--exclude=/** \
$SOURCE $DEST
# Don't really need more than 2 months of backup..
duplicity remove-older-than 2M $DEST
else
duplicity \
--include-globbing-filelist /root/backuplist.txt \
--encrypt-key "$GPG_KEY" \
--sign-key "$GPG_KEY" \
--exclude=/** \
$SOURCE $DEST
fi
export AWS_ACCESS_KEY_ID=
export AWS_SECRET_ACCESS_KEY=
export PASSPHRASE=
Everything i want to backup is listed in the file backuplist.txt, and the script runs once every night using cron…. This should cover my backup needs, if I only remember to backup my GPG-key :p
Updates..
Posted by Stian | Filed under Personlig
Current annoyances:
- Â Saying that I almost never get sick (not been sick for over a year), and then catch a bad cold the very next week..
- Umbrella-thieves, 2 in 3 days, come on.. are these guys organized or what?
- The kilo-price of passports..
- Showers without warm water..
- Running out of interesting courses to take next semester… I know I’ll take INF251 and one of INF235/INF236.. Not sure about the 3rd yet..
Bright sides:
- Almost done with this semester, 2 exams left, first exam went really well.
- Almost Christmas, heading back home some time after the 18th.
- Heading to Tenerife first thing next year with HÃ¥vard and Karl Trygve. Hoping for warm water and great weather
- Seems like Apple is releasing an ultraportable laptop early next year, with flash-disk and all! Doubt I’ll get the first edition, but it’s good news..
- Some hackers managed to get access to the GPU (RSX) on the PS3.
Ray-caster
Posted by Stian | Filed under Teknologi
Recently made a ray caster for volume rendering for one of the courses I’m taking this semester at the university. Turned out to be a very interesting and challenging assignment (I didn’t really have any computer graphics background). The results are also quite pleasing if you ask me, there’s really something different about producing 3d renderings of human heads compared to the usual “check out this implementation of <insert algorithm here>”..
Here are some sample renderings:
The two first are datasets from a CT-scanner. The last one is a simulation of hurricane Isabel from the National Center for Atmospheric Research.
EDIT: A small update of some of the final results:
Google Reader - Search!
Posted by Stian | Filed under Ukategorisert
Google reader now got a search function.. About fscking time…
Asterisk - Resolve callerid using online phone directories.
Posted by Stian | Filed under Linux, Teknologi
Here’s how to automatically resolve incoming callerids using a public online phone directory with a bit of PHP.
First you need a few packages installed.. php(4/5)-cli and php(4/5)-curl.. Copy the following code and save it as a file (’resolvnum.agi’) in your agi-bin directory, most likely /usr/share/asterisk/agi-bin. Make sure it’s executable: chmod +rx resolvnum.agi
#!/usr/bin/php -q
<?php
// Do not wait more than 4 secs..
ob_implicit_flush(false);
set_time_limit(5);
$stdin = fopen("php://stdin", "r");
$stdout = fopen("php://stdout", "w");
// Get all agi-variables from asterisk via stdin:
while (!feof($stdin)) {
$temp = fgets($stdin);
$temp = str_replace("n", "", $temp);
$s = explode(":", $temp);
$agivar[$s[0]] = trim($s[1]);
if (($temp == "") || ($temp == "n"))
break;
}
// Get callerid from agi-variables.
$callerid = $agivar['agi_callerid'];
if (!is_numeric($callerid)) exit(1);
// Set up curl:
$user_agent = "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)";
$ch = @curl_init("http://www.kvasir.no/telefonkatalog/searchresult.html?q=".$callerid."&x=0&y=0/");
@curl_setopt($ch, CURLOPT_USERAGENT, $user_agent);
@curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
@curl_setopt($ch, CURLOPT_TIMEOUT,4);
// Fetch us some data...
$html = str_replace("r", "", @curl_exec($ch));
curl_close($ch);
// Got nothing? Oh, bother...
if (!$html) exit(0);
// Replace Norwegian characters..
$html = str_replace(array(chr(230), chr(248), chr(229), chr(198), chr(216), chr(197)),
array("ae", "oe", "aa", "AE", "OE", "AA"), $html);
// Try to match some names..
preg_match_all("|<h4 class=\"forste_linje\">n(.*)n</h4>|U", $html, $names, PREG_SET_ORDER);
// Yay, we found something, set calleridname.. leave number alone..
if (isset($names[0][1])) {
fputs($stdout,"SET CALLERID \"".$names[0][1]." (".$callerid.") <".$callerid.">\"n");
fflush($stdout);
}
exit(0);
?>
Then add the agi to your incoming extensions in extensions.conf. Mine is set up like this:
[from-sip] exten => 85XXXXXX,1,AGI(nummeroppslag.agi) exten => 85XXXXXX,2,Macro(dialer,101,SIP/101&SIP/102,101) exten => 85XXXXXX,3,Hangup
It should now query www.kvasir.no/telefonkatalogen and set the callerid-name (which will show up on your phone) if it finds something. Since I’m pretty sure that kvasir won’t like this (I take no responsibility for your usage), I would recommend implementing a cache if you have a lot of traffic. You can also easily adapt it to other phone directories by changing the search URL and regexp.
Cell SDK on Ubuntu (Feisty Fawn).
Posted by Stian | Filed under Linux, Teknologi
I ended up installing ubuntu on my playstation 3, pretty straight forward installation. Getting the Cell SDK up and running on ubuntu turned out to be a bit of a challenge however. There’s really not much information out there about how to do it, the closest I got was Gammel’s “Installing Cell SDK under Ubuntu” - which almost worked. Most of the following is based on his recipe.
1. Install a few needed packages:
$ apt-get install rpm freeglut3 freeglut3-dev libxmu-dev libxext-dev \ build-essential perl rsync flex byacc tk8.4 tcl8.4 libelf1 gawk bash \ libnetpbm10 libnuma1
2. Make sure sh points to bash:
$ ls -la /bin/sh lrwxrwxrwx 1 root root 9 2007-08-25 07:33 /bin/sh -> /bin/bash
if it doesn’t then replace it:
$ rm /bin/sh && ln -s /bin/bash /bin/sh
3. Replace mawk with gawk:
$ update-alternatives --set awk /usr/bin/gawk
4. Add symlinks to a couple of libs:
$ ln -s /usr/lib/libtcl8.4.so.0 /usr/lib/libtcl8.4.so $ ln -s /usr/lib/libtk8.4.so.0 /usr/lib/libtk8.4.so
5. Download the Cell SDK (v2.1 is current version). Mount it and copy out the software.
$ mkdir /media/cell && mount -o loop CellSDK21.iso /media/cell $ cp -r /media/cell/software /tmp/
6. Fix the install script, i needed to add –ignorearch since it complained about the system not beeing ppc64 for some reason. Then run it..
$ cd /tmp/software $ sed -i 's/rpm -i/rpm -i --nodeps --ignorearch/g' cellsdk $ ./cellsdk install
7. Mount up the SPU-filesystem (if you get errors like “Unable to create SPE thread: Invalid argument” or “spu_create(): No such file or directory” while running your code this is probably why).
$ mkdir /spu $ echo "none /spu spufs defaults 0 0" >> /etc/fstab $ mount -a
Playstation 3.
Posted by Stian | Filed under Teknologi
Continued from ‘Return of the BlÃ¥gg‘…
I’ve bought a Playstation 3. This thing is a number crunching monster, the Cell CPU does Folding@Home about 10 times faster than a normal computer! The Cell CPU is basically a multithreaded core (PPE) backed up by 8 (!) smaller cpus (SPE) connected to a ring-bus (EIB). You don’t have to go many years back to find super-computers that this beast beats hands down on computing power (at least on floating point operations). I’ve installed linux on it (yellow dog, might change that soon - probably ubuntu) and you have access to the main PPE + 7 of the SPEs.
It’s also a very good DVD-player, which is one of the main reasons why I bought it. Upscales and transmits both image and sound to my receiver over a single cable (HDMI). A good upscaling DVD-player is around 3000NOK, so why not pay a little bit extra and get a number cruncher, (future) media center, gaming machine (there really aren’t many good games for it yet though) and blu-ray player included?
That aside, it was kinda a hard decision whether to buy it or not, I really don’t like supporting SONY (they got a really bad track record: the sony root kit, closing down lik sang, exploding batteries, etc, etc). I prefer HD-DVD over Blu-ray. Blu-ray is region-locked, HD-DVD is not, which means that buying movies from the states is harder with Blu-ray. The Blu-ray camp also got a nasty DRM trick up their sleeve called BD+, it’s basically a VM running on the player that executes code from the disc. Hopefully Sony sells the console at a loss.
I hope that it will be possible to attach a xbox360 HD-DVD player to the PS3 via USB and use that to play HD-DVDs in linux in the future.
Return of the Blågg.
Posted by Stian | Filed under Personlig
I’ve decided to revive my old BlÃ¥gg, in English this time around.
Summer is over and I’m back in Bergen, finishing the last year of my Bachelors degree. Spent most of the rainy summer in Haugesund, working at Diptel on an asterisk based IP-PBX along with my friend HÃ¥vard (same as last year really).
I did take a few days off and spent them hiking on the Hardangervidda mountain plateau with my neighbor and fellow student Mathias. We started out up north in Finse (train from Bergen) and ended up somewhere just outside Rjukan in the south (bus back to Haugesund). The weather forecast scared us a bit before leaving, luckily we ended up with 5 days of great weather. Also had a really strange run in with one of my lecturers this semester, Jan Arne Telle, he’s a professor at UiB - teaching algorithms . I didn’t know who he was at the time, but we ran into him a couple of times and started talking. Needless to say, I was quite surprised when I found out who he was, what’s the chance of running into your lecturer on Europe’s largest mountain plateau? Anyway, great trip! I want to go back there some other time, bringing a fishing rod and spending most of the days following one of the many rivers (remind me to bring a higher factor sunscreen the next time :p). Also want to visit Jotunheimen, Rondane and Femundsmarka.
As for school, this year I’m taking a look at the different Masters degree directions I’m considering - Algorithms, Cryptography and Visualization. The most likely direction atm is Parallel Algorithms. You might have noticed that the development of modern CPUs have slowed down considerably lately? They are adding more and more cores rather than improving the speed on each one. We’re slowly reaching the physical limits of CPUs, so they are forced to build lateral instead of just building higher and higher. You can get a quad core CPU these days for ~2000 NOK. So, how do we harness this parallel power? That’s what I want to help find out..
Which brings me to my next post….
Hvordan lure 550 millioner kroner fra norges befolkning.
Posted by Stian | Filed under Bog-skinke, Tanker
- Start en statlig eid kringkastingsorganisasjon, lovsatt finansiert av en kringkastingsavgift.
- Lur en sak forbi et sovende storting om flytting av lisensdato fra 1. mars til 1. januar.
- Krev dobbel lisens for disse 2 månedene (januar og februar er allerede betalt fra forrige lisens, og må nå betales for igjen).
- Sett styreleder Hallvard Bakke pÃ¥ saken med noen dÃ¥rlige unskyldninger, “Men har vi ikke allerede betalt for januar og februar?” - “Nei, det blir helt feil mÃ¥te Ã¥ se dette pÃ¥. Folk skal betale det samme som de alltid har gjort, sier Bakke.”. For 20 Ã¥r siden gikk visstnok NRK glipp av 2 mÃ¥neders lisensavgift nÃ¥r de justerte lisensdatoen motsatt vei, derfor er det bare pÃ¥ sin rett at NRK skal fÃ¥ disse pengene igjen med renter. “Det er vanntett, det, altsÃ¥.”
- Profitt, ca 550 millioner kroner ekstra inntekt..
Beklager, men det blir litt drøyt Ã¥ kreve inn 2 mÃ¥neders ekstra lisens nÃ¥ 20 Ã¥r etter at dere “glemte” Ã¥ kreve den inn. Hva med de som har blitt lisensbetalere de siste 20 Ã¥rene? Hvorfor skal de betale dobbelt? Hadde man krevd inn det lille ekstra som de faktisk mistet (justert etter konsumprisindeks) hadde det vært greit, men her skal man tydeligvis ha det igjen med renter. Norges befolkning er ikke en bank hvor du kan la være Ã¥ ta betalt, for sÃ¥ Ã¥ komme tilbake 20 Ã¥r senere og kreve tilbake “innskuddet” med renter.
Java (Sun) går open-source.
Posted by Stian | Filed under Ukategorisert
Ifølge flere kilder har Sun bestemt seg for å open-source Java. Når og hvordan (dvs, under hvilken lisens) dette skal skje er enda ikke bestemt. Håper de klarer å bestemme seg for en fornuftig lisens.
Sun har dessuten gitt ut en ny lisens som gjør det mye lettere å distribuere java (i linux og opensolaris). Forhåpentligvis betyr dette slutten på java med fetch-restrictions i f.eks gentoo.
