Frank W

Update: This post is out-dated and parts of it have been updated by Fully automated media centre using Flexget, emby, Trakt and IMDb.

I’m not really someone who watches most of the stuff on TV – I watch specific TV shows that interest me, the occasional movie, and I would watch news, documentary and some sports channels if they were available at a reasonable price without all the other crap.

I’ve therefore put my old computer to good use and created my own streaming TV service that shows me the TV shows and movies I  want to watch, when I want to watch them, and you can too. Think of it like a PVR over the internet. Basically your own private version of this rumoured streaming service from Rogers with the stuff you want. Also gives me an excuse to go the bar and watch certain sports 🙂

Hardware and software needed

Here is all the stuff you need:

1. A computer (or certain NAS devices) to store and stream from. You need a sizeable harddrive to store media on. I used two 1TB Western Digital Reds in mirrored RAID configuration (I’m tired of losing data).
2. A fast wireless router (best to get a 5.8GHz model these days to reduce interference).
3. Linux server – free. I used a Ubuntu Server 13.1 VM running on a Proxmox VE host, but you could use any Linux distro.
4. Serviio streaming media server – free or $25 if you need it. I wanted the MediaBrowser web interface so I can play content on my MacBook Air.
5. Flexget content automation tool – free.
6. Transmission bittorrent client – free.
7. Samba  file and print server – free. This is not required, but mounting the files on your laptop is also kind of useful.
8. An account on Trakt.tv – free. Use to select what TV shows you want, but it’s also a very useful site.
9. Some time and some knowledge of Linux (or a friend who does) – opportunity cost.

You can also add something like a DLNA-compatible TV to this mix, or an PS3 or Xbox (I found the Serviio Media Browser far preferable for browsing content due to the crap interface/functionality available on the Xbox).

Step by Step Instructions

First, let’s understand how this works.

1. You create a Linux machine and install all the software on it.
2. Flexget is the automation brain. It connects to Trakt, and gets your list of shows.
3. Flexget then connects to your RSS torrent sources, and grabs the lastest uploads.
4. When Flexget finds a new show that matches your Trakt list, in the quality you have chosen, it sends it to Transmission by RPC.
5. Transmission then downloads it and removes it when done.
6. When it’s downloaded, Serviio analyzes it and pulls down metadata (description and thumbnail).
7. It’s then available for you to view via your DLNA-compatible device (TV, Xbox, WD Live, etc) or via the Serviio MediaBrowser web interface (which I use and is like a personal Netflix).

I used an old computer running and put in in the coat closet out of the way – may sound strange, but it’s a headless server and is plugged directly into a GigE network port.

Step 1 – Set up Server

I’m not going to document how to install Linux as that is out of the scope of this guide, but I used Ubuntu Server 13.1 Server within an OpenVZ container running on a Proxmox VE 3.1 host. This is so I have the option to run other VMs on my physical hardware in the future for whatever reason.

You need to get the server running with package management (apt-get) and network connectivity all working smoothly. I created a user with a home directory and put all the downloads in there as I also wanted to mount the files on my laptop using Samba, but if you don’t then you don’t need to add a user.

adduser frank

Step 2 – Install Transmission

Transmission is what downloads your media files. This can be installed via the package manager (as root).

apt-get install transmission-cli transmission-common transmission-daemon

You also need to set up your transmission configuration file, mine is /etc/transmission-daemon/settings.json

{
    "alt-speed-down": 50,
    "alt-speed-enabled": false,
    "alt-speed-time-begin": 540,
    "alt-speed-time-day": 127,
    "alt-speed-time-enabled": false,
    "alt-speed-time-end": 1020,
    "alt-speed-up": 50,
    "bind-address-ipv4": "0.0.0.0",
    "bind-address-ipv6": "::",
    "blocklist-enabled": false,
    "blocklist-url": "http://www.example.com/blocklist",
    "cache-size-mb": 4,
    "dht-enabled": true,
    "download-dir": "/home/frank/Downloads",
    "download-limit": 100,
    "download-limit-enabled": 0,
    "download-queue-enabled": true,
    "download-queue-size": 5,
    "encryption": 1,
    "idle-seeding-limit": 30,
    "idle-seeding-limit-enabled": false,
    "incomplete-dir": "/home/debian-transmission/Downloads",
    "incomplete-dir-enabled": false,
    "lpd-enabled": false,
    "max-peers-global": 200,
    "message-level": 2,
    "peer-congestion-algorithm": "",
    "peer-id-ttl-hours": 6,
    "peer-limit-global": 300,
    "peer-limit-per-torrent": 100,
    "peer-port": 62222,
    "peer-port-random-high": 65535,
    "peer-port-random-low": 49152,
    "peer-port-random-on-start": false,
    "peer-socket-tos": "default",
    "pex-enabled": true,
    "port-forwarding-enabled": false,
    "preallocation": 1,
    "prefetch-enabled": 1,
    "queue-stalled-enabled": true,
    "queue-stalled-minutes": 30,
    "ratio-limit": 2,
    "ratio-limit-enabled": true,
    "rename-partial-files": true,
    "rpc-authentication-required": true,
    "rpc-bind-address": "0.0.0.0",
    "rpc-enabled": true,
    "rpc-password": "yourpasshere",
    "rpc-port": 9091,
    "rpc-url": "/transmission/",
    "rpc-username": "yourusername",
    "rpc-whitelist": "127.0.0.1",
    "rpc-whitelist-enabled": false,
    "scrape-paused-torrents-enabled": true,
    "script-torrent-done-enabled": false,
    "script-torrent-done-filename": "",
    "seed-queue-enabled": false,
    "seed-queue-size": 10,
    "speed-limit-down": 100,
    "speed-limit-down-enabled": false,
    "speed-limit-up": 100,
    "speed-limit-up-enabled": false,
    "start-added-torrents": true,
    "trash-original-torrent-files": false,
    "umask": 0,
    "upload-limit": 100,
    "upload-limit-enabled": 0,
    "upload-slots-per-torrent": 14,
    "utp-enabled": true
}

As with any software installation, you may need to tweak it for your system. Things to note here are:

1. I set my queue size to 5 active torrents max.
2. I set my download location.
3. I turned on encryption preferred.
4. I set my RPC username/password (transmission will hash the plaintext password you enter).
5. I set the umask to 0 (file permissions)

If you go type the address of your media server into your browser using the port you set in the config file (mine is http://media:9091/) then you should be able to see the Transmission web UI.

Step 3 – Create a Trakt account

I am using Trakt.tv to track which TV shows I watch and automate which shows are downloaded by Flexget. Rather than manually edit Flexget configuration files, you can get use Trakt to manage the shows you want.

Trakt is nifty for lots of things, and I invite you to explore their site. For the purposes of this guide, you want to create an account and add some of your favourite shows to the Watchlist (you can use a custom list but I didn’t bother, see the Trakt plugin page for more info).

Once you’ve added some shows, you need to get your API Key to put into your Flexget config. Make a note of this, you need it in step 3.

Step 4 – Install Flexget

Flexget is the brain of this entire thing.

1. Set up your server environment – you need Python 2.7 and PIP.
2. Install Flexget itself using PIP.

pip install flexget

This is going to compile a whole bunch of Python stuff, just let it do its thing. Look for errors, and if you see none, then verify Flexget install:

Successfully installed flexget SQLAlchemy jinja2 requests setuptools
Cleaning up...
root@media:/# flexget -V
1.2.60
You are on the latest release.

Before we continue, note that Flexget changes fairly rapidly and can be upgraded with the following command (run as root).

pip install --upgrade flexget

Great, now you have a working Flexget and just need to configure it. Create a config file somewhere, mine is in the .flexget directory of my user account /home/frank/.flexget/config.yml

templates:
  global:
    transmission:
      enabled: yes
      host: localhost
      port: 9091
      addpaused: no
      ratio: 1.00
      username: [[username]]
      password: [[password]]
    clean_transmission:
      host: localhost
      port: 9091
      username: [[username]]
      password: [[password]]
    email:
      active: true
      from: [[from@yourdomain.net]]
      to: [[you@yourdomain.net]]
      smtp_host: smtp.gmail.com
      smtp_port: 587
      smtp_login: true
      smtp_username: [[username]]
      smtp_password: [[password]]
      smtp_tls: true
    free_space:
      path: /home/frank/Downloads
      space: 50000
    content_filter:
      require:
        - '*.avi'
        - '*.mkv'
        - '*.mp4'
      reject:
        - 'password.txt'
        - '*.wmv'
        - '*.r0*'
    magnets: no
    pathscrub: windows
    verify_ssl_certificates: no

  tv-global:
    series:
      settings:
        720p:
          timeframe: 12 hours
          propers: yes

    thetvdb_lookup: yes
    # Impose reasonable size constraints
    content_size:
      max: 5000
      min: 20
    inputs:
      - rss: { url: 'http://showrss.info/feeds/all.rss', silent: no } # ShowRSS
      - rss: { url: 'https://publichd.se/rss.php', silent: no }       # Public HD
      - rss: { url: 'http://www.ezrss.it/feed/', silent: no }         # EZTV
    set:
      path: /home/frank/TV/{{series_name}} S{{series_season|pad(2)}}

tasks:
  tv-shows:
    priority: 1
    template:
      - global
      - tv-global
    configure_series:
      from:
        trakt_list:
          username: [[username]]
          api_key: [[yourapikey]]
          series: watchlist
          strip_dates: yes
      settings:
        quality: 720p

schedules:
  - tasks: tv-shows
    interval:
      minutes: 30

Things to note here (in order):

1. I have first defined my global settings. Transmission settings are defined, and I have enabled the (brand new) plugin clean_transmission, which removes completed files.
2. I have set up e-mail notifications sent via Gmail (thanks to lame port 25 SMTP blocking by my ISP Beanfield). This is useful because Flexget will send you a notification any time it downloads a new file!
3. There is a check for free space before we download anything else, and I have specified which types of files must be in the torrent for it to be accepted (no wmv’s thanks).
4. I have then set tv-global settings, where I say I want 720p and accept propers.
5. I am looking up the downloads on TheTVDB (provides meta data used to organize files into directories and do other funky stuff if you so desire).
6. I am providing 3 torrent sites that I like as inputs for torrents, and have listed their RSS feeds (you can use any site you want). I am then setting the path for my downloads and telling Flexget to tell Transmission that downloads should go into a directory for each season of each show.
7. Then I define tasks, where I tell Flexget to connect to Trakt, get my list of shows from my watchlist, and create series out of it using the configure_series plugin. 720p only.
8. Finally, I’m telling the new Flexget daemon to run my one task every 30 minutes continuously. (I haven’t go around to automating movies yet and not sure if I want to).

As of Flexget 1.2.0, the CLI has completely changed and a new daemon mode has been added. The config above has been updated for this new version, and I am now running Flexget in daemon mode rather than by CLI using crontab.

In light of this, we also now need to make the Flexget daemon start up when your server boots. I am using Ubuntu Upstart for this, so I created a script in /etc/init/flexget.conf

description "Flexget Daemon"
author "Frank W"
env USER=frank

start on started networking
stop on shutdown
respawn

env APP_ARGS="-c /home/frank/.flexget/config.yml daemon start"

exec start-stop-daemon --start  --chuid $USER --exec /usr/local/bin/flexget -- $APP_ARGS

This upstart script will start the server when the networking service starts (a good time to do it as Flexget needs a network…) You will need to set the Flexget arguments using a separate environment variable: I am pointing to my config file and telling Flexget to start in daemon mode as the user “frank”. Upstart manages everything else.

Before you start the daemon, you might want to check that everything is working well with Flexget by running it from the command line.

flexget --test -c ~/.flexget/config.yml execute

If there are no errors, then great, you can now start Flexget as a daemon.

service flexget start

You should now be able to see Flexget running in the background.

root@media:/home/frank# ps aux | grep flexget
frank     6563 12.8  1.9 269964 62228 ?        Ssl  17:49   0:03 /usr/bin/python /usr/local/bin/flexget -c /home/frank/.flexget/config.yml daemon start
root      6579  0.0  0.0   9604   960 pts/6    S+   17:49   0:00 grep --color=auto flexget

If you used the same configuration as me, then every 30 minutes Flexget will pull down the latest RSS feeds from the sites you have selected, and download any new torrents that match your Trakt list in 720p. You can check the status of the downloads using the Transmission Web UI. Once they are done, then it’s over to Serviio.

Step 5 – Install Serviio

So you have some files to watch! Now you need a media player to play them. I am using Serviio 1.4, which is free with a pro version available for $25. Installing Serviio is a bit of a pain in the ass to be honest, as you have to compile various supporting packages from source, including fdk-aac, ffmpeg, rtmpdump and x264. Probably the most difficult and most important of these is ffmpeg, which is what does all the transcoding of media files. If you don’t get this right, nothing works. A fair amount of Googling will be required, but the results are worth it. I tried other media servers and Serviio is the best IMHO.

I’m not going to go through the entire install process, as it is somewhat documented in the Serviio Getting Started guide. You should also refer to the ffmpeg Ubuntu compilation guide, which helped me a great deal. When compiling, use the version of ffmpeg in this post, if you compile directly from the git repo it will probably not work.

The output of the ffmpeg version command shows the ./configure parameters I used, along with version numbers of libraries that are installed (again, use the version above).

root@media:~/serviio_stuff# ffmpeg -version
ffmpeg version 1.1.git
built on Jan 21 2014 19:27:37 with gcc 4.8 (Ubuntu/Linaro 4.8.1-10ubuntu9)
configuration: --enable-libfdk-aac --enable-libass --enable-libmp3lame --enable-nonfree --enable-gpl --enable-version3 --enable-libfreetype --bindir=/usr/local/bin --enable-librtmp --enable-libx264
libavutil      52. 37.101 / 52. 37.101
libavcodec     55. 16.100 / 55. 16.100
libavformat    55.  9.100 / 55.  9.100
libavdevice    55.  2.100 / 55.  2.100
libavfilter     3. 77.101 /  3. 77.101
libswscale      2.  3.100 /  2.  3.100
libswresample   0. 17.102 /  0. 17.102
libpostproc    52.  3.100 / 52.  3.100

As I am lazy, I just extracted the Serviio tarball and ran the serviio.sh startup script. You can also automate this, again using Upstart. My /etc/init/serviio.conf file is shown below.

# Serviio - Media Streaming Server
#
# Serviio is a free media server. It allows you to stream your media files
# (music, video or images) to renderer devices (e.g. a TV set, Bluray
# player, games console or mobile phone) on your connected home network.

description "Serviio - Media Streaming Server"

env USER=root

start on started networking
start on mounted DEVICE=[/UL]*
stop on shutdown

respawn

exec start-stop-daemon --start --make-pidfile --pidfile /var/run/serviio.pid -c $USER --exec /root/serviio-1.4/bin/serviio.sh

You can then run Serviio by entering:

service serviio start

Now, you will need to download the Serviio client, called Serviio-Console, and set things up. Because we are using a headless server, we will have to download Serviio again from the download page, and this time get the version for our client machine.

You will then need to launch the client and  point it at the headless server by adding the command line switch serviio.remoteHost=yourhost. On MacOSX, this involves selecting the Serviio-Console file, and then selecting Open Package Contents from the menu. Then browse to the info.plist file in a text editor, scroll to the bottom and look for the VMOptions key. The line below that should look like this:

<string>-Xms5M -XX:+UseParNewGC -XX:MinHeapFreeRatio=5 -XX:MaxHeapFreeRatio=10 -Dserviio.remoteHost=media</string>

Close up this text file and launch the Serviio-Console application. If all goes well, it should connect to your media server. This client application is pretty self-explanatory, but things that are worth mentioning are.

Library – Shared Folders

In the Library, Shared folders tab you will need to select which folders you want Serviio to scan. You should set this to whatever path your downloads are saved in by Flexget. Serviio also supports photo and music collections, but I am only using it for video here.

Make sure to select the option to search for updates and to keep the library automatically refreshed (so that new downloads appear in Serviio automatically).

Delivery – Transcoding

The next important setting is the transcoding options. Here, I have enabled transcoding, which you will need to do to play files on multiple different devices. E.g. I need to transcode a 720p TV show to watch it on my iPhone while in the gym (yes I can do this). I have selected highest quality and the original audio.

On the next tab, you can also enabled subtitles, which is useful for some movies.

Metadata – Video

Meta data downloading is what makes Serviio so awesome, with Movie and TV Show thumbnails, and full descriptions of each movie and episode within a series. Very useful stuff.

Remote Access

Lastly, we need to set up some remote access settings. Firstly, you must set a password for the MediaBrowser web interface, which is a little annoying if you are only using Serviio within a LAN. Fortunately, Serviio 1.4 now allows you to save this password!

I have also set my default content quality to Original, to get video in full 720p. You can change this within the player.

Step 6 – Enjoy your streaming TV service

The last step is to enjoy your new streaming TV service. Quite honestly, there’s nothing like it. Nowhere else can you get ad-free, on demand versions of all the TV shows you want to watch, on any devices you want. Sure, you can get some shows from iTunes or watch it via your cable or satellite subscription. Other shows you can get from different TV channel websites that include adverts. Yet more behind IP-restricted paywalls. Missed a show and it’s not on your TV providers on-demand? Tough. Mobile? Not a chance.

With this setup you can watch what you want, when you want, on whatever device you want – all with zero adverts.

Here’s what all your hard work looks like.

E-mail notifications

Yep, you get an e-mail when something new downloads. In reality, new shows come out within minutes of them airing, within the hour for 720p. Some shows are available before they have even been on TV, depending on the iTunes release schedule, time-shifting, and of course your geographic location.

Serviio MediaBrowser web interface

This is what browsing the Series section of Serviio looks like. I promise you I personally do not watch all these shows 😉

Serviio MediaBrowser responsive web interface

Yes, it really works on mobile and has a responsive web interface since version 1.4, with more goodness to come in later versions I’m sure.

Disclaimer: This post is for educational reasons only. It is your responsibility to adhere to all relevant copyright laws in your country. Please review this post on the legality of TV show downloading that you have already received via your subscription to your TV provider.

In my personal opinion, downloading a show is the same thing as using a PVR to record it, or a computer to rip it yourself for viewing later, although this has not been decided in court.