Ripping DVDs on Ubuntu 16.10

A recent reconfiguration of our living room caused a problem. The HDMI cable from the DVD player was not long enough to reach the television in its new position. We couldn't watch DVDs any more! "Get a new cable", you might say. Hah! No, as a man au fait and indeed au courant with technology, I knew what the answer was. It was time to rip all our DVDs and host them on a DLNA server.

I've actually attempted this before, but found it hard work, and I was never particularly satisfied with the results. Still, faced with the alternative of having to buy an entire cable - and possibly spend as much as £5 - it was time to try again.

Required software

There are a quite a lot of tools around but sticking with the free, and after a little experimentation, this is what I have ended up using:

• VLC: everyone's favourite media player. If you haven't already got this installed, why not? VLC can play just about anything and is useful for checking which DVD title to extract (more on this later) and verifying the rip afterwards: sudo apt-get install vlc
• Handbrake: the open source ripper of choice. Much improved since last time I tried it. Unfortunately the version in the standard Ubuntu repos is ancient, so we'll have to use a PPA.
• libdvdcss2: the DVD decryption, needed because of the largely pointless industry practice of "encrypting" DVDs to prevent piracy. Because of the lack of clarity around whether ripping DVDs for private use is allowed, this isn't officially distributed by Ubuntu. But they do distribute a package that will install it for you, if you tell it to. So that's OK then. (standard disclaimer: consult a lawyer in your jurisdiction yadda yadda yadda. Yeah right)
• ddrescue: a "data recovery tool", used for a couple of reasons: firstly, to allow us to queue up multiple DVDs to rip without having to sit up all night swapping disks in and out; and secondly to get round the more egregious copy protection on most Disney DVDs.

It's simple to install it all:

sudo add-apt-repository ppa:stebbins/handbrake-releases
sudo apt-get update
sudo apt-get install vlc handbrake-gtk libdvd-pkg gddrescue
sudo dpkg-reconfigure libdvd-pkg

The process

The simplest approach is to insert a disc, let Handbrake scan it (using the CSS decryption automatically), select a title to rip, and let it run. This has a couple of disadvantages though: it's slow, and while it's running you can't do anything else with the DVD drive. So you wait a couple of hours and then swap the disk, then do it all again. Clearly, this will take a while. There's another problem: Handbrake barfs on DVDs that have more advanced copy protection - primarily more recent Disney titles - because the DVDs have deliberately broken sectors and misreported file tables. (This also means older DVD players mess up on these discs. Thanks a bunch, Disney.)

Handbrake allows us to queue up multiple titles to rip, so what we'd like to be able to do is make an image of the disk, missing out the stupid broken sectors, onto the hard disk, and then rip from that image. That's where ddrescue comes in. It can make an ISO of the DVD and deal with (or skip) those sectors. Then once it's on the hard disk, we can get Handbrake to scan the ISO and queue up the title, or titles on that DVD. While Handbrake starts ripping that, we can put another DVD in and make an ISO from that. In this way, we can set up a queue of titles that can run overnight, or while out.

So, first step: copy the DVD to hard disk:

DISK_NAME=film-title; ddrescue -b 2048 -r 3 -Afnv /dev/dvd $DISK_NAME.iso $DISK_NAME.map

One of the things that ddrescue can do is do repeated passes over a disk, and those passes can even be on separate invocations. That's why there's a map file used as well. If the copy works well, the map file won't be needed, but if it gets interrupted for some reason, or the disk is having problems, then the map file will reduce the time needed for the next passes.

Next, play the ISO in VLC. By doing this, we can work out exactly which title to rip. Handbrake does a decent job of working out the right one, but on some it doesn't (particularly those irritating Disney DVDs, which report 99 titles, mostly fake). Also, if you want to rip some additional material, like extras, then you can work your way through the menus to find them too. When the title to rip is playing in VLC, choose the "Playback > Title" menu and it will show which title number is playing.

Now open Handbrake. Choose "Open Source" and select the ISO file you created. It will scan the disk image, create previews for the titles it finds and select the title that is most likely to be the correct one. Usually it chooses correctly, but if not, select it from the drop down. You can then change the various video, audio and subtitle settings. I usually first choose the preset "Super HQ 1080p30 Surround", then go to the "Video" tab and choose "Film" or "Animation". Adjust the audio and subtitle settings to taste. Then start encoding.

Some DVDs are a little more complex. I've found a few that will work only direct from the disc itself, and I have failed outright with a few: so far, "Monsters University", "Shaun The Sheep The Movie" and "Minority Report" (although I have a feeling this latter one is a damaged disk, even though I can't see any damage).

Partitioning and formatting a 4TB disk

Amazing what just over a ton will get you these days - 4 TB of disk. I know it makes me sound old but I remember having a 10 MB disk and being pleased ...

Anyway, setting it up wasn't difficult but I didn't find a single consolidated guide to do what I wanted, which was to create two ext4 partitions, one normal and one encrypted with LUKS, for personal information. So here's what I did.

Partition the disk
Anything over 2TB is not supported by the venerable MBR, so the newer GPT partitioning scheme is needed, which in turn requires the use of a partitioning tool that supports GPT. I used parted (which I had to install first), like so:

# parted /dev/sdc
> unit %
> mklabel gpt
> mkpart primary 0 50
> mkpart primary 50 100
> print
> quit

Create file systems
1) The normal partition
# mkfs.ext4 /dev/sdc1

2) the LUKS parition
# cryptsetup -y -v luksFormat /dev/sdc2
# cryptsetup luksOpen /dev/sdc2 d2
# mkfs.ext4 /dev/mapper/d2

Reduce amount of system space
ext4 reserves 5% of the disk for system space by default, which on a 2TB volume is 100GB. This is excessive for volumes that just contain data and not any OS files. The following commands reduce this allocation to 1% and free up about 80GB of space.

# tune2fs -m 1 /dev/sdc1
# tune2fs -m 1 /dev/mapper/h2


Then mount in your preferred location and away you go!

Copying music files

My music files are stored on a server; sometimes I want to copy a selection onto sn SD card for playing in the car. The steps are:

1. Make a playlist on the Squeezebox.
2. Grab the playlist m3u file.
3. Remove all the lines that start with the # character: sed '/#/d' playlist.m3u > music.txt
4. Mount the Samba music share on my server: sudo mount -t cifs -o username=me //192.168.1.2/music /media/music
5. Do a search and replace in a text editor on the text file so that the paths all reference /media/music
6. Copy the files: first set IFS to a new line, then: for file in $(<music.txt); do cp -v "$file" music/; done

Then I run the files through Sound Converter because the player in my car can only deal with mp3 files.

"Editing" symlinks on Linux

Firstly, let's be clear. There *is* no way of editing symlinks. All you can do is delete them and recreate them. However, there is a way of automating this process.

I had lots of photos on one disk and then moved them. The links to the files needed changing en masse and I didn't want to have to redo it by hand. What I needed was a way of finding out where the link *used* to point to and use that to construct the new link.

The answer was readlink. Thus:

#!/bin/bash
for fn in *.jpg; do
    NEWFN=$(readlink $fn | sed 's,/media/d1,/media/d2,')
    ln -sf $NEWFN $fn
done

Pipe the data from readlink into sed, which is used to replace an occurrence of the string /media/d1 with /media/d2. Then recreate the link, using the -f flag to delete an existing link.

Easy. Full credit to this post on SuperUser.

Updating Debian Squeeze on PogoPlug

There's a nasty little bug in Debian Squeeze which means that if you update the kernel, IPv6 stops working.  The reason this is so annoying for me is that if the networking doesn't work, I can't get into the PogoPlug at all (there is no direct access, not even a serial console).  It's effectively bricked.

The answer is to run flash-kernel after running apt-get upgrade.  This updates the /boot/uImage and /boot/uInitrd files.  The bug is that this isn't run automatically.

The actual answer for me was to restore from a last known good backup from a couple of weeks ago. I attempted to look at the flash-kernel script and create these files by hand but it didn't work.

Check a Linux hard disk - command line

Although checking a ext2/3 disk from the command line is easy, there is surprisingly little concise info out there.  Here's what I did when faced with a disk that kept going into read-only mode (which is usually a sign that something is wrong):

sudo fsck -V -C /dev/sdc1 -- -y -f

I had originally thought that calling e2fsck was correct but fsck is a wrapper which calls the correct version for the file system.  It defaults to ext2.

The options I used are:

-V

Verbose

-C

Gives a progress bar

-y

Answers "yes" to all interactive questions

-f

Forces a check even if one has been done recently.

-c

Uses the badblocks utility to check for bad blocks (duh) and feeds this information into fsck in order to mark those blocks off. From what I read, you are better off specifying this parameter to fsck than running badblocks separately because it will work out block sizes for you this way.

Note that some of the options follow the device name.  These options are passed to the underlying program (e.g. fsck.ext2) without verification.

Logitech Media Server and media keys

I use squeezeslave to listen to music from my Logitech Media Server (LMS; previously known as Squeezebox Server).  The LMS has a pretty good web UI which can be used to control the player.  However, it's quicker to be able to use the media keys to do simple actions - pause/resume, previous, next.  It saves having to navigate to the browser window and move the mouse around.

Here's how I set this up on Ubuntu 11.04.

Firstly, you need a script to send commands directly to the LMS command line interface (CLI).  Luckily someone has written this.  The script is called sccmd and it's part of the squeezeslave codebase, I think.  However, I couldn't figure out how to get it from the main squeezeslave page at sourceforge and in fact I originally found it on github where someone has a mirror of the code.  Just copy the script to somewhere appropriate on your local drive (I put it in ~/bin which is automatically on the path in Ubuntu), modify it as necessary for your player and server (change the MAC address and host name) and make it executable.

Secondly, you need to map the keys.  Disable the media keys in the Keyboard Shortcuts settings panel. Then, if you're using Unity, then you can map the keys to the script using the CompizConfig Settings Manager (compizconfig-settings-manager in the Universe repository).  Go to the "Commands" plugin (enabling it if necessary) and define three command lines.  They are:

• /home/user/bin/sscmd pause
• /home/user/bin/sscmd "button jump_rew"
• /home/user/bin/sscmd "button jump_fwd"

Then go to the second tab, Key Bindings, and define the keys.  This is a simple matter of clicking the "Disabled" button, clicking "Grab key combination" and pressing the appropriate key.

Finally, if you are using this remotely over ssh (as per my previous post) then you need to modify the ssh config to forward port 9090, which is the port used by the CLI (or change it in the LMS settings to a different port and forward that).  In this case, the host name in the sccmd file will be "localhost".

Update: in 12.10 I found that I needed to install the package "compiz-plugins" to get the "Commands" plugin.