Technical stuff, mostly to remind myself when I forget in a couple of months' time.

Saturday, 25 February 2017

Ripping DVDs on Ubuntu 16.10

A recent reconfiguration of our living room caused a problem. The HDMI cable from the DVD player was not long enough to reach the television in its new position. We couldn't watch DVDs any more! "Get a new cable", you might say. Hah! No, as a man au fait and indeed au courant with technology, I knew what the answer was. It was time to rip all our DVDs and host them on a DLNA server.

I've actually attempted this before, but found it hard work, and I was never particularly satisfied with the results. Still, faced with the alternative of having to buy an entire cable - and possibly spend as much as £5 - it was time to try again.

Required software

There are a quite a lot of tools around but sticking with the free, and after a little experimentation, this is what I have ended up using:
  • VLC: everyone's favourite media player. If you haven't already got this installed, why not? VLC can play just about anything and is useful for checking which DVD title to extract (more on this later) and verifying the rip afterwards: sudo apt-get install vlc
  • Handbrake: the open source ripper of choice. Much improved since last time I tried it. Unfortunately the version in the standard Ubuntu repos is ancient, so we'll have to use a PPA.
  • libdvdcss2: the DVD decryption, needed because of the largely pointless industry practice of "encrypting" DVDs to prevent piracy. Because of the lack of clarity around whether ripping DVDs for private use is allowed, this isn't officially distributed by Ubuntu. But they do distribute a package that will install it for you, if you tell it to. So that's OK then. (standard disclaimer: consult a lawyer in your jurisdiction yadda yadda yadda. Yeah right)
  • ddrescue: a "data recovery tool", used for a couple of reasons: firstly, to allow us to queue up multiple DVDs to rip without having to sit up all night swapping disks in and out; and secondly to get round the more egregious copy protection on most Disney DVDs.
It's simple to install it all:
sudo add-apt-repository ppa:stebbins/handbrake-releases
sudo apt-get update
sudo apt-get install vlc handbrake-gtk libdvd-pkg gddrescue
sudo dpkg-reconfigure libdvd-pkg

The process

The simplest approach is to insert a disc, let Handbrake scan it (using the CSS decryption automatically), select a title to rip, and let it run. This has a couple of disadvantages though: it's slow, and while it's running you can't do anything else with the DVD drive. So you wait a couple of hours and then swap the disk, then do it all again. Clearly, this will take a while. There's another problem: Handbrake barfs on DVDs that have more advanced copy protection - primarily more recent Disney titles - because the DVDs have deliberately broken sectors and misreported file tables. (This also means older DVD players mess up on these discs. Thanks a bunch, Disney.)

Handbrake allows us to queue up multiple titles to rip, so what we'd like to be able to do is make an image of the disk, missing out the stupid broken sectors, onto the hard disk, and then rip from that image. That's where ddrescue comes in. It can make an ISO of the DVD and deal with (or skip) those sectors. Then once it's on the hard disk, we can get Handbrake to scan the ISO and queue up the title, or titles on that DVD. While Handbrake starts ripping that, we can put another DVD in and make an ISO from that. In this way, we can set up a queue of titles that can run overnight, or while out.

So, first step: copy the DVD to hard disk:
DISK_NAME=film-title; ddrescue -b 2048 -r 3 -Afnv /dev/dvd $DISK_NAME.iso $DISK_NAME.map
One of the things that ddrescue can do is do repeated passes over a disk, and those passes can even be on separate invocations. That's why there's a map file used as well. If the copy works well, the map file won't be needed, but if it gets interrupted for some reason, or the disk is having problems, then the map file will reduce the time needed for the next passes.

Next, play the ISO in VLC. By doing this, we can work out exactly which title to rip. Handbrake does a decent job of working out the right one, but on some it doesn't (particularly those irritating Disney DVDs, which report 99 titles, mostly fake). Also, if you want to rip some additional material, like extras, then you can work your way through the menus to find them too. When the title to rip is playing in VLC, choose the "Playback > Title" menu and it will show which title number is playing.

Now open Handbrake. Choose "Open Source" and select the ISO file you created. It will scan the disk image, create previews for the titles it finds and select the title that is most likely to be the correct one. Usually it chooses correctly, but if not, select it from the drop down. You can then change the various video, audio and subtitle settings. I usually first choose the preset "Super HQ 1080p30 Surround", then go to the "Video" tab and choose "Film" or "Animation". Adjust the audio and subtitle settings to taste. Then start encoding.

Some DVDs are a little more complex. I've found a few that will work only direct from the disc itself, and I have failed outright with a few: so far, "Monsters University", "Shaun The Sheep The Movie" and "Minority Report" (although I have a feeling this latter one is a damaged disk, even though I can't see any damage).

Sunday, 2 March 2014

Partitioning and formatting a 4TB disk

Amazing what just over a ton will get you these days - 4 TB of disk. I know it makes me sound old but I remember having a 10 MB disk and being pleased ...

Anyway, setting it up wasn't difficult but I didn't find a single consolidated guide to do what I wanted, which was to create two ext4 partitions, one normal and one encrypted with LUKS, for personal information. So here's what I did.

Partition the disk
Anything over 2TB is not supported by the venerable MBR, so the newer GPT partitioning scheme is needed, which in turn requires the use of a partitioning tool that supports GPT. I used parted (which I had to install first), like so:

# parted /dev/sdc
> unit %
> mklabel gpt
> mkpart primary 0 50
> mkpart primary 50 100
> print
> quit

Create file systems
1) The normal partition
# mkfs.ext4 /dev/sdc1

2) the LUKS parition
# cryptsetup -y -v luksFormat /dev/sdc2
# cryptsetup luksOpen /dev/sdc2 d2
# mkfs.ext4 /dev/mapper/d2

Reduce amount of system space
ext4 reserves 5% of the disk for system space by default, which on a 2TB volume is 100GB. This is excessive for volumes that just contain data and not any OS files. The following commands reduce this allocation to 1% and free up about 80GB of space.

# tune2fs -m 1 /dev/sdc1
# tune2fs -m 1 /dev/mapper/h2


Then mount in your preferred location and away you go!

Friday, 31 May 2013

Copying music files

My music files are stored on a server; sometimes I want to copy a selection onto sn SD card for playing in the car. The steps are:

  1. Make a playlist on the Squeezebox.
  2. Grab the playlist m3u file.
  3. Remove all the lines that start with the # character: sed '/#/d' playlist.m3u > music.txt
  4. Mount the Samba music share on my server: sudo mount -t cifs -o username=me //192.168.1.2/music /media/music
  5. Do a search and replace in a text editor on the text file so that the paths all reference /media/music
  6. Copy the files: first set IFS to a new line, then: for file in $(<music.txt); do cp -v "$file" music/; done
Then I run the files through Sound Converter because the player in my car can only deal with mp3 files.

Wednesday, 6 March 2013

"Editing" symlinks on Linux

Firstly, let's be clear. There *is* no way of editing symlinks. All you can do is delete them and recreate them. However, there is a way of automating this process.

I had lots of photos on one disk and then moved them. The links to the files needed changing en masse and I didn't want to have to redo it by hand. What I needed was a way of finding out where the link *used* to point to and use that to construct the new link.

The answer was readlink. Thus:
#!/bin/bash
for fn in *.jpg; do
    NEWFN=$(readlink $fn | sed 's,/media/d1,/media/d2,')
    ln -sf $NEWFN $fn
done
Pipe the data from readlink into sed, which is used to replace an occurrence of the string /media/d1 with /media/d2. Then recreate the link, using the -f flag to delete an existing link.

Easy. Full credit to this post on SuperUser.

Wednesday, 11 July 2012

Updating Debian Squeeze on PogoPlug

There's a nasty little bug in Debian Squeeze which means that if you update the kernel, IPv6 stops working.  The reason this is so annoying for me is that if the networking doesn't work, I can't get into the PogoPlug at all (there is no direct access, not even a serial console).  It's effectively bricked.

The answer is to run flash-kernel after running apt-get upgrade.  This updates the /boot/uImage and /boot/uInitrd files.  The bug is that this isn't run automatically.

The actual answer for me was to restore from a last known good backup from a couple of weeks ago. I attempted to look at the flash-kernel script and create these files by hand but it didn't work.

Friday, 22 June 2012

Check a Linux hard disk - command line

Although checking a ext2/3 disk from the command line is easy, there is surprisingly little concise info out there.  Here's what I did when faced with a disk that kept going into read-only mode (which is usually a sign that something is wrong):

sudo fsck -V -C /dev/sdc1 -- -y -f

I had originally thought that calling e2fsck was correct but fsck is a wrapper which calls the correct version for the file system.  It defaults to ext2.

The options I used are:
-V
Verbose
-C
Gives a progress bar
-y
Answers "yes" to all interactive questions
-f
Forces a check even if one has been done recently.
-c
Uses the badblocks utility to check for bad blocks (duh) and feeds this information into fsck in order to mark those blocks off. From what I read, you are better off specifying this parameter to fsck than running badblocks separately because it will work out block sizes for you this way.

Note that some of the options follow the device name.  These options are passed to the underlying program (e.g. fsck.ext2) without verification.

Tuesday, 22 November 2011

Logitech Media Server and media keys

I use squeezeslave to listen to music from my Logitech Media Server (LMS; previously known as Squeezebox Server).  The LMS has a pretty good web UI which can be used to control the player.  However, it's quicker to be able to use the media keys to do simple actions - pause/resume, previous, next.  It saves having to navigate to the browser window and move the mouse around.

Here's how I set this up on Ubuntu 11.04.

Firstly, you need a script to send commands directly to the LMS command line interface (CLI).  Luckily someone has written this.  The script is called sccmd and it's part of the squeezeslave codebase, I think.  However, I couldn't figure out how to get it from the main squeezeslave page at sourceforge and in fact I originally found it on github where someone has a mirror of the code.  Just copy the script to somewhere appropriate on your local drive (I put it in ~/bin which is automatically on the path in Ubuntu), modify it as necessary for your player and server (change the MAC address and host name) and make it executable.

Secondly, you need to map the keys.  Disable the media keys in the Keyboard Shortcuts settings panel. Then, if you're using Unity, then you can map the keys to the script using the CompizConfig Settings Manager (compizconfig-settings-manager in the Universe repository).  Go to the "Commands" plugin (enabling it if necessary) and define three command lines.  They are:

  • /home/user/bin/sscmd pause
  • /home/user/bin/sscmd "button jump_rew"
  • /home/user/bin/sscmd "button jump_fwd"

Then go to the second tab, Key Bindings, and define the keys.  This is a simple matter of clicking the "Disabled" button, clicking "Grab key combination" and pressing the appropriate key.

Finally, if you are using this remotely over ssh (as per my previous post) then you need to modify the ssh config to forward port 9090, which is the port used by the CLI (or change it in the LMS settings to a different port and forward that).  In this case, the host name in the sccmd file will be "localhost".

Update: in 12.10 I found that I needed to install the package "compiz-plugins" to get the "Commands" plugin.

Monday, 18 July 2011

Building a SqueezeBox server on PogoPlug

Quick summary for those in a hurry


That's it! Read on if you're interested in more detail and my ramblings ...

The concept

For some time I've been pondering how to solve a problem.  As I mentioned before, the machine I use to run Squeezebox Server (SBS) is old and noisy - and, unfortunately, the only place it can go at the moment is in our bedroom.  This means it can't be on all the time, which is a bit annoying sometimes.  One possible longer term solution was to move the whole kit and caboodle up into the loft, but that would require a lot of cabling and in any case I was a little worried about the temperature up there in summer.

I know people have installed SBS on NASes and the like although I'd never really looked into this. Then a few weeks ago I came across a post about setting up a SBS for under $100. That was quite interesting, but what really caught my attention was an addendum right at the end that mentioned the guy was setting up a server on a SheevaPlug - a whole computer in a (large, wall-wart type) plug! Why this should be surprising in these days of iPhones and so on I don't know, but it caught my attention.  The idea of replacing my big, old, noisy server with a small, low power, permanently on device for only a few tens of pounds was very appealing.

I started looking into the subject a bit more and found that there are a number of these "plug computers" around - albeit not literally built into a plug (even the SheevaPlug has a separate power cable in the non-US version).  The SheevaPlug is sold as a "development kit", the intention being that other manufacturers would then base their own products on it, but you can buy it as a consumer to, for about £100 in the UK.  This gives you a base computer with which to do what you will.  There is a branded consumer version too, called the TonidoPlug, which comes with some bundled apps and costs about $99 (although I can't find a UK outlet).

The SheevaPlug is now a couple of years old and there are newer versions from the same manufacturer but these start getting more expensive.  But there are other options too.  The Seagate Dockstar is a small device with a dock for Seagate disks and USB ports for other disks, plus a network connection.  It's based (I believe) on the PogoPlug (link is to v2 page), another "plug computer", which its v1 incarnation looked similar to the SheevaPlug.  Both of these allow you to attach disks to it, connect it to your home network and register it with my.pogoplug.com in order to be able to access the content on the disks from anywhere on the internet.  How these then differ from a NAS such as the LinkSys NSLU2 I don't know (apart from the external access).

What's interesting of course is that although these - the SheevaPlug apart - are sold as consumer, "plug 'n' play" devices, they are all fully fledged computers, running embedded Linux.  And there's no shortage of information on how to reconfigure them as general purpose servers - or even specifically as SqueezeBox Servers.  Having read around the subject for a few days, I decided to go for the cheapest option, the PogoPlug - only £50 on Amazon.  This seemed a reasonable amount of money to commit to an experiment - since, from what I'd read, there was some chance that the process of installing a new system might not work properly, and also there was a chance that it just wouldn't be powerful enough to do what I wanted (specifically, there was a passing mention on one the SqueezeBox forums that it used a single-threaded Perl, which would be an issue for SBS).

The implementation

The PogoPlug v2 is a small, nicely styled box about the same size as a router.  It has a Marvell Kirkwood ARM processor, four USB 2.0 ports, one ethernet connector and a power socket, 256 MB of RAM and 128 MB of internal storage (for the OS). It's larger than I expected (v1 really was the size of a plug) but still very unobtrusive, and tiny compared to my current server!

There are a couple of configuration options, drawing on the sterling work of one Jeff Doozan. He's figured out how to patch the embedded OS to boot from an attached USB drive, or even install on the internal storage. From then on, you can install any OS that supports the processor I guess, although I think in reality that means Linux. I found instructions to install ArchLinux and Debian (Jeff Doozan's instructions). Since I'm very familiar with Ubuntu, I chose Debian.

Most of the instructions I found concentrated on the Dockstar, presumably since this is a more common device.  For example, the instructions on the SqueezePlug wiki, which seemed complicated and in any case specifically aimed at Linux newbies.

Instead, I found some very helpful instructions which were specifically for the PogoPlug v2.  These go as far as installing Debian.  There's no point in reproducing them, so here's a couple of additional comments:

  • A number of sources mention that you will need to register at my.pogoplug.com in order to enable ssh, which is vital if you're going to get anywhere.  In fact, I found it was open anyway.  The only precaution I took was to disconnect my router from the internet until I'd disabled the auto-updating service that runs automatically.
  • Most of the instructions implicitly assume you'll be using a flash drive as the boot source but I didn't have one lying around.  I did have a spare 80GB USB disk though, so I used that.  It worked fine.
  • The script ran fine apart from a couple of files it couldn't find - in fact, the md5 files corresponding to the main files were missing off the server.  A minor correction to the script fixed this (and mehl has since resolved this).
  • I found that installing extra packages gave a warning about non-verified packages until I re-ran apt-get update.
  • The first packages to install were apt-utils and dialog, which are both used by some packages when installing.  I also installed sudo and nano, as per the instructions.
  • I followed mehl's instructions to set up a user account with sudo access and lock out root (as Ubuntu does), which I think makes sense on a semi-public facing server.
  • Finally, I configured ssh in the same way I did when I set up my first Squeezebox server.  mehl also has some instructions for configuring ssh.

Having done this, the next step was to shut down and backup the drive before I did anything stupid!

Installing SqueezeBox Server was pretty straightforward.  The SqueezePlug wiki also has some notes about manually installing SBS, which is what I followed.
  • First, I set up a mount point for the disk drive at /media/d1.
  • Then I plugged in the drive with the music on (a separate Seagate 1TB drive) and mounted it.
  • Next, I created a symlink from the root directory to the music directory on the Seagate drive.  This means I can configure SBS to look in a given directory and change the drive without reconfiguring SBS.
  • Then I installed SBS using the instructions in the wiki.

Having done this, I pointed my browser to the SBS web ui, did the minimal config necessary and let the music scan run.  It took a little under two hours - but I do have a reasonably large collection (over 28,000 tracks).

And that's it!  The performance when playing music is indistinguishable from that of the big old server.  The SBS web ui is a little slower perhaps, but not much.  The PogoPlug itself makes no noise at all; the disks are very quiet, albeit just about discernible in a quiet room.

Tuesday, 5 July 2011

Powering on Squeezebox Server remotely

Some time ago I wrote about using the Squeezebox Server remotely.  A minor detail, surprisingly overlooked in the post, was the fact that, if you want to do this, the server must be on.  This is all very well if you can leave the server on permanently.

Unfortunately, I can't.  It's a big old PC that makes a lot of noise and the associated disks (where the music is stored) have very bright lights.  And this all sits in our bedroom - unfortunately, the only room in the house left for such things now that the kids need their own rooms.

So what do I do if I have - once again - forgotten to switch it on before I left the house?  My options:

  • Send a text message to a remote device requesting boot up.  The remote device in this case is otherwise known as "C's mobile phone" and depended on her a) having it on; b) being in the house and c) being bothered.
  • Set the server to power on at set times.  I believe the BIOS supports this but I didn't look at it seriously because I wanted more manual control - and I didn't want it waking us up on the weekends (in the unlikely event that the kids hadn't already done so).
  • Use wake-on-LAN (WOL) to cause the server to boot up.
WOL it woz then!

The server's BIOS supports it, but this isn't the only configuration needed.  The OS needs to send some configuration to the network card (NIC) to put it into the right state.  This guide for Ubuntu explains what to do.  Once all set up, a simple command from another computer on the LAN sends a "magic packet" to the server's MAC address and - hey presto! - the server boots.  This was easily verified on the home LAN using the "wakeonlan" command in Ubuntu (and there are various other programs available for other OSs).

The tricky bit was doing this from a remote location.  What I want to be able to do is connect to the internet wherever I am, send a command to my home IP address (conveniently addressed via dynamic DNS) and have it just work.  However, it doesn't seem to be possible.  There's some suggestion that if you configure your router to route traffic on a nominated port (which one doesn't matter) to the LAN's broadcast address (e.g. 192.168.1.255) then this should work.  However, my router won't allow the LAN's broadcast address to be used like this.

There are also a couple of open-source router firmware packages that have built in WOL clients (DD-WRT and Tomato) but these didn't support my router and anyway I didn't fancy replacing the firmware on mine and possibly ending up with a bricked router.

So the only other possibility is to have another device in the house that is already on, ssh into that and send the WOL command from that.  It would have to be:

  • always on - since if I can't remember to switch the server on, I'm not going to remember to switch something else on;
  • no noise
  • capable of supporting ssh
  • capable of sending WOL magic packets
I don't have much hardware lying around and was at a loss as to what might fit the bill.  And then I realised - the Squeezebox Controller was just the thing!  It's the remote control for the SB Receiver (the pair being the SB Duet that I bought a couple of years ago) but it's not just a typical infrared, line of sight type device.  It's a capable device in its own right.  It runs embedded Linux (busybox), connects to the wireless network and can control multiple SB devices.  It's capable to waking up the server, it's small and silent - and it supports ssh as standard (using dropbear).  It wouldn't be a problem leaving this on - as long as it's in its cradle it stays on (when it's out it suspends after a couple of minutes).

My only concern was security.  My main server listens on a non-standard port instead of 22 and only allows public key authentication.  I followed this guide to enabling public key authentication on the Controller but try as I might I couldn't work out how to change the port or disable standard password based authentication.  So in the end I generated a very long complex password and used that (and saved it somewhere safe) because I'll never have to enter it because I'm using keys.

I set up my router to direct traffic on port 22 to the Controller, so now I can ssh in from a remote location.  And, after a bit of searching, I figured out the command to send a WOL packet - it's ether-wake.  I wrote a quick shell script on the Controller so that I didn't have to remember the server's MAC address - and we're done!

So now the server is off, just waiting for me to power it on remotely.  It's silent and dark.

Oh - and as for the dazzling blinkenlights on the USB disk drive, I discovered that the second hard disk (which acts as a backup for the first) has slightly more sophisticated power management - if the machine to which it is attached is off, it switches itself off, even though it has a separate power supply.  So I switched them round.

Of course, the right answer for this is to have a server I can leave on all the time - and I might have an answer for this soon ...


Friday, 15 January 2010

Installing Wine 1.2 in Ubuntu 9.10 (Karmic)

There is a known bug when installing Wine in Ubuntu 9.10 (Karmic), documented in this bug report.  From comments I've seen it affects upgraded installations of Ubuntu rather than fresh installations.  The log shows errors similar to this:

error: "sys.kernel.maps_protect" is an unknown key
error: "net.ipv4.ip_always_defrag" is an unknown key



Comment #11 on the above bug report solved the problem for me.  These are the steps I took:

  • Take a backup of the file /etc/sysctl.conf
  • Edit /etc/sysctl.conf (as root) and comment out the following lines by putting # in front of them (note that they are not all together in the file):

kernel.maps_protect = 1
net.ipv4.ip_always_defrag = 1
sys.kernel.maps_protect = 1


Then install Wine using your preferred method.  Once complete, uncomment the lines again and save.

I used to have Wine installed when this machine was running Jaunty but somehow Wine completely borked the upgrade from Jaunty to Karmic, necessitating a complete restore and removal of Wine.  I guess this was something to do with it.

Tuesday, 12 January 2010

Using Tomcat 6 on Xubuntu

Currently I'm doing some Java education.  In order to learn and understand some of the mechanics, I'm using a plain text editor instead of a sophisticated IDE and compiling on the command line.

I'm running in a VM, so I'm using Tomcat on Xubuntu (because it's reasonably lightweight) and Geany as my text editor.  This isn't as plain as using Mousepad (the default text editor in Xubuntu) but I'm only using it for syntax highlighting and outlining at the moment.  Once the compiling is done then testing is done running java manually from the command line and deployment is done by copying files manually into the Tomcat webapps directory.

It's all a bit different from my usual working environment, which is IBM Rational Application Developer (based on Eclipse) - auto-completing class names, automatically sorting out my imports, creating WAR files for me, and deploying to WebSphere Portal.  It took me a little while to figure out where things need to be deployed, and I'm not sure I'd ever run a Java program from the command line before!

So here's a reminder to myself:

  • web apps are deployed in /var/lib/tomcat6/webapps
  • compiling is done from the root development directory, like this:
~/webapps/MyWebApp$ javac -classpath classes:. -d classes src/com/example/model/TestModel.java

where the classpath is a colon-separated list of the parent directories of classes (i.e. "classes" has the directory structure com/example/etc underneath it) and the -d switch controls where the compiled output goes.
  • running a Java program (say, a test harness): 
~/webapps/MyWebApp$ java -cp src:classes com.example.model.test.TestModel param

where, again, the classpath (-cp) is the parent directory of necessary directory structures, and the class to be run is specified by its fully qualified name, not its path.

  • If you use the Tomcat web application manager to deploy a WAR file via the UI, the context root for the application is <war-name>/<root-mapping>. So if the mapping in web.xml is, say "/myservlet" and you WAR is called MyServlet.war, the correct place for the app is /MyServlet/myservlet. (This caught me out and cost me about an hour one time ...)


It might all seem simple stuff to you but when you're used to the tool doing it for you, it's easy to forget.  Took me a couple of goes to get it right!

Wednesday, 6 January 2010

Using Squeezebox Server remotely

My favourite gadget that I bought last year was my Squeezebox Duet.  But one of the best things about the whole Squeezebox package is actually the software.  The server is open source, which means I could play with it for a while before I splashed some cash on the hardware.  It's what won me over.

What's also won me over is this: with the server running at home, and a little set up, I can play music from my collection from wherever I am (with my PC and a network connection).  None of this is difficult (if you know how to tunnel TCP ports over SSH, you can probably go somewhere else now) but I wanted to document it so I can repeat it if/when necessary.

These instructions were originally for an Ubuntu 9.04 server, already running Squeezebox Server. I've now also used them with on a Pogoplug running Debian Squeeze. The client could be any Linux as it's all command line; I've used with Ubuntu 9.10, 10.04 and 11.04.  YMMV.  Etc.

Firstly, I set up a key on the client that to be used to authenticate with the server.  The Ubuntu documentation has a good guide, including an explanation of why a pass phrase is a good idea.  The steps I used were:
  • Open a terminal
  • Enter ssh-keygen -f ~/.ssh/id_rsa -t rsa -b 4096
  • Enter your pass phrase twice.
Then I used a USB key to copy the public key (id_rsa.pub) to the server.
Next, I prepared the server:

  1. Installed the SSH server by installing the openssh-server package.
  1. Configured SSH to only allow key-based authentication and change the port SSH listens on.  Both of these will make SSH more secure (remember that I am allowing external access to the server, so these measures are worthwhile).  Edit the SSH server's config file (/etc/ssh/sshd_config) and add or modify the following lines and restart SSH (more details):
PasswordAuthentication no
Port 1234
  1. Allowed the client to login to the server by taking the id_rsa.pub file from the client and appending its contents to the file /home/user/.ssh/authorized_keys, (where user is the user name on the server).
  1. My server is behind a firewall, (a cable modem/router thing), so I needed to forward the relevant port (1234, as configured above) from the router to the server.
  1. I use dyndns.org to set up a constant DNS name for my router, which can use this service automatically.  Then I don't have to worry about remembering what IP address my ISP has given me or whether it has changed.
Then I prepared the client.

  1. I checked that SSH was working on the local network before anything else.  In a terminal, enter
ssh -p 1234 user@192.168.0.10

Where 1234 is the configured SSH port and 192.168.0.10 is the server's IP address. The first time I did this I got a message telling me that the server's identity could not be verified.  I accepted this and entered the key's pass phrase.
  1. The SSH client now needed to be configured to forward a couple of ports to the remote server.  This can all be set up on the command line, but it's easier to put it in a config file.  I edited the file ~/.ssh/config and added the following lines:
Host home
    HostName myserver.dyndns.org
    Port 1234
    User user
    Compression yes
    LocalForward 3483 192.168.0.10:3483
    LocalForward 9000 192.168.0.10:9000
    LocalForward 9090 192.168.0.10:9090


(where 192.168.0.10 is the IP address of the SqueezeboxServer on the local network).  Now using SSH from a remote location is as simple as typing ssh home and all the relevant addresses and ports will be used.
  1. The client software I use is SqueezeSlave.  It's a headless (no UI) open-source software version of the Squeezebox.  Other players are available:
    • SoftSqueeze is a Java clone of the Squeezebox itself, with a UI, display and all.  I found it unreliable on Ubuntu and over-sensitive to what else is running.
    • SqueezePlay is ostensibly the replacement for SoftSqueeze but despite many attempts to compile it, it seems not to like PulseAudio on Ubuntu so although it functions as a controller, it won't produce any audio.
    • Any software player that can work with MP3 streams, using these instructions.  The difference between using this method and using SqueezeSlave is that the server has to transcode everything to an MP3 stream first, whereas SqueezeSlave receives the raw bytes and plays them directly, locally.
Once downloaded, I put the appropriate version of the squeezeslave executable in /usr/bin
  1. Now to run SqueezeSlave remotely, I first run SSH like this:
ssh -f home sleep 10

This runs ssh in the background and runs the sleep command.  This is a bit of a trick.  ssh will exit once it's run the command, *unless* the tunnel is being used.  So if you issue the next command within those 10 seconds, ssh won't exit until squeezeslave exits.
  1. Then I run SqueezeSlave like this:
squeezeslave -m 00:00:00:00:00:03 localhost

where the -m option specifies a fake MAC address, which is used by the SqueezeboxServer to distinguish between players, thereby remembering settings for each player. The "server" is localhost because SSH is tunneling the requests for ports 9000 and 3843 to the real server.
  1. I then control the player using the web UI from a browser at http://localhost:9000

Friday, 23 October 2009

Converting TS files to a DVD

A couple of years ago, we got a Humax PVR9200T (link is to current equivalent) - a Freeview, twin tuner hard disk recorder.  It's been - and still is - fantastic.  A hard disk recorder literally does change the way you view TV.  And I don't know why people pay a fortune every month for Sky+ when for one payment of £180 (now less), you can own a machine that does the same thing.

One of its features is a USB socket, from which you can copy recorded programmes onto a computer.  They come across as MPEG transmission stream files (*.ts).  They can be watched on your computer (good old VLC!) but if you want to put them on a playable DVD then a little work is required.

You need to demux the TS file into separate video and audio streams, correcting any errors from the transmission while you do so, edit the streams if you want (say, to remove ads), remux the streams into an MPEG file, create a DVD file structure with the MPEG file, create an ISO for the DVD and then burn the ISO onto a DVD.

I'm sure there are commercial programs out there to do all this for you, but I don't want to pay for this, so I did a little digging and, sure enough, on the UK Humax forum someone has very kindly told you what tools to use and put a batch file together to do the whole thing for you.

However, since I use Ubuntu rather than Windows, I decided to rewrite the batch file as a shell script, and change the tools since some of them were Windows only.  Linked below is the fruit of my labours.  There's a readme for more info, which I strongly recommend you read - you do need to install some software yourself.

Download the ZIP file

Hope it helps you and I'll be interested in any feedback.  Although of course you do use it at your own risk.  Don't blame me if it accidentally deletes your favourite film, wipes your hard disk, kicks your cat or insults your mother.

Your Host

Husband, father, pop music anorak, guitarist