FreeBSD 10.1 Guide

metadata

FreeBSD LogoThis is an installation and configuration guide for FreeBSD. I now use FreeBSD for my server which acts both as a NAS for all of my data and as a media server for my TV. I have managed to add some pretty neat features such as a Time Capsule for my MacBook and a PVR for BBC iPlayer. This guide describes the FreeBSD installation, ZFS configuration and user-land software installation and configuration. I include all of my configuration files so that you can see how I have set it all up.

  1. Installation Notes
  2. Configuration Notes
    1. General Setup
    2. Adding Users
    3. Installing Software
    4. Configuring Cron
    5. Configuring Software
      1. Time Machine
      2. Web Server
      3. Media Servers
      4. Samba
      5. SSH
      6. SMART Daemon
  3. Workarounds
  4. Usage Notes
  5. References
  6. Word Of Warning

§1 Installation Notes

I used a bootable USB image to install FreeBSD, you can grab one from the FreeBSD download page. During the first section of the installer I disabled “games” and “ports”. The automatic partitioner decided to go with:

DeviceSizeTypeMount Point
ada0149GB
ada0p164KBfreebsd-boot
ada0p2117GBfreebsd-ufs/
ada0p332GBfreebsd-swapnone

Further on in the process I decided to disable IPv6 and “dumpdev”. Once the process finished I decided to not immediately reboot into the new system, instead I opened a shell and typed gpart set -a active ada0 to set the drive as active before the first boot (which was needed to side-step a nasty bug which meant that you couldn’t boot into your nice new system).

§2 Configuration Notes

Now you should have a working fresh installation of FreeBSD. The majority of this tutorial covers the configuration of this fresh clean system. Firstly I talk about how I configured FreeBSD itself, then I add users, then I install user-land software, and finally I configure that user-land software.

§2.1 General Setup

Firstly, I tweaked the start-up configuration file, “/boot/loader.conf”, so that some additional kernel modules would be loaded during start-up. These modules allow the CPU temperature to be obtained (for my Intel CPUs), enable ACPI (for my ASUS motherboard) and finally enable disk encryption to be used (if the user wants). I did this with the following commands (as root) - the changes do not take effect until the system is rebooted, hence the final command.

1: echo 'coretemp_load="YES"' >> /boot/loader.conf
2: echo 'acpi_asus_load="YES"' >> /boot/loader.conf
3: echo 'geom_eli_load="YES"' >> /boot/loader.conf
4: reboot
You may also download “freebsd-10p1-cmds1.sh” directly or view “freebsd-10p1-cmds1.sh” on GitHub Gist.

Next, I added some lines to the other start-up configuration file so that some additional software would be loaded during start-up. The “/etc/rc.conf” file is very important in FreeBSD as it contains most of the custom configuration for my system. The commands (as root) I used are below - likewise, the changes do not take effect until the system is rebooted, however, I do not reboot immediately as some further tweaking can be performed during this boot.

The first command enables ZFS to be used (if the user wants); the second sets the default encryption settings for GELI; the third enables the inet daemon (which is often called the ‘super server’); the fourth clears “/tmp” during boot; the fifth enables the daemon to automatically mount removeable storage (such as USB drives); and the final command enables memory disks (a.k.a. RAM disks).

1: echo 'zfs_enable="YES"' >> /etc/rc.conf
2: echo 'geli_default_flags="-a HMAC/SHA256 -l 256 -s 4096"' >> /etc/rc.conf
3: echo 'inetd_enable="YES"' >> /etc/rc.conf
4: echo 'clear_tmp_enable="YES"' >> /etc/rc.conf
5: echo 'autofs_enable="YES"' >> /etc/rc.conf
6: echo 'md_enable="YES"' >> /etc/rc.conf
You may also download “freebsd-10p1-cmds2.sh” directly or view “freebsd-10p1-cmds2.sh” on GitHub Gist.

Next I configured the system to automatically mount removeable storage (such as USB drives). The instructions for this can be found in the section on USB disks in the FreeBSD handbook. Run vi /etc/auto_master and un-comment /media -media -nosuid. My only comment is that you don’t have to add all the lines to “/etc/devd.conf” as they are already there, just move the */ line further up the file and then they will not be commented out anymore. Run vi /etc/devd.conf and un-comment:

1: notify 100 {
2:     match "system" "GEOM";
3:     match "subsystem" "DEV";
4:     action "/usr/sbin/automount -c";
5: };
You may also download “freebsd-10p1-devd.conf” directly or view “freebsd-10p1-devd.conf” on GitHub Gist.

Before rebooting I edited “/etc/fstab” to create a RAM disk (to hold temporary copies of sensitive un-encrypted data) and to encrypt the swap partition. This final step ensures that any sensitive data that is written to swap (because the RAM is full) is unrecoverable if the machine is powered off (just like the data in RAM). Of course, the swap has no knowledge of which data it stores is sensitive and which isn’t: this method encrypts the whole partition. The downside to this is that your machine cannot be hibernated (which requires reading the swap during wake-up) or debugged completely (as memory dumps are sometimes written to swap). I believe that these two hindrances are acceptable.

1: # Make mount target
2: mkdir /mnt/volatile
3: # Create both file-systems
4: echo "md    /mnt/volatile    mfs    rw,-s8M    0    0" >> /etc/fstab
5: vi /etc/fstab
6:     # add ".eli" to end of /dev/ada0p3
7: # Setup "vim"
8: echo "syntax on" > ~/.vimrc
9: # Reboot
10: reboot
You may also download “freebsd-10p1-cmds3.sh” directly or view “freebsd-10p1-cmds3.sh” on GitHub Gist.

Next I cleaned the beginning of all of my data disks that were going to be my ZFS array. WARNING: This will make any existing data on the drive inaccessible. I then created the pool and created some useful partitions on the pool. You’ll notice that I enabled LZ4 compression and set the flags for snapshots on all partitions except “timecapsule” (which has its own snapshots in Mac OS X). The “autoexpand=on” line is needed if in future I decide to replace the disks (one-by-one) with larger ones.

01: # Remove partition table from the disk
02: gpart destroy -F /dev/ada1
03: gpart destroy -F /dev/ada2
04: gpart destroy -F /dev/ada3
05: gpart destroy -F /dev/ada4
06: gpart destroy -F /dev/ada5
07: # Clean the disks
08: dd if=/dev/zero of=/dev/ada1 bs=1m count=10
09: dd if=/dev/zero of=/dev/ada2 bs=1m count=10
10: dd if=/dev/zero of=/dev/ada3 bs=1m count=10
11: dd if=/dev/zero of=/dev/ada4 bs=1m count=10
12: dd if=/dev/zero of=/dev/ada5 bs=1m count=10
13: # Create the ZFS pool
14: zpool create pool1 raidz /dev/ada1 /dev/ada2 /dev/ada3 /dev/ada4 /dev/ada5
15: # Enable auto-expansion of the ZFS pool
16: zpool set autoexpand=on pool1
17: # Enable transparent compression
18: zfs set compression=lz4 pool1
19: # Create partitions
20: zfs create -p pool1/home
21: zfs create -p pool1/mnt/data
22: zfs create -p pool1/mnt/timecapsule
23: zfs create -p pool1/usr/local/share/doc
24: zfs create -p pool1/usr/local/www/thomasguymer.co.uk
25: zfs create -p pool1/usr/ports
26: zfs create -p pool1/usr/src
27: zfs create -p pool1/var/db
28: zfs create -p pool1/var/log
29: zfs create -p pool1/var/mail
30: # Mark them for snapshots
31: zfs set com.sun:auto-snapshot=true pool1/home
32: zfs set com.sun:auto-snapshot=true pool1/mnt/data
33: zfs set com.sun:auto-snapshot=true pool1/usr/local/share/doc
34: zfs set com.sun:auto-snapshot=true pool1/usr/local/www/thomasguymer.co.uk
35: zfs set com.sun:auto-snapshot=true pool1/usr/ports
36: zfs set com.sun:auto-snapshot=true pool1/usr/src
37: zfs set com.sun:auto-snapshot=true pool1/var/db
38: zfs set com.sun:auto-snapshot=true pool1/var/log
39: zfs set com.sun:auto-snapshot=true pool1/var/mail
You may also download “freebsd-10p1-cmds4.sh” directly or view “freebsd-10p1-cmds4.sh” on GitHub Gist.

Next I mounted the partitions so that they could be used straight away (whilst keeping copies of any data in the two targets that already existed).

01: # Move already existing versions
02: mv /usr/local/share/doc /usr/local/share/TEMPdoc
03: mv /var/log /var/TEMPlog
04: # Make mount targets
05: mkdir /home
06: mkdir /mnt/data
07: mkdir /mnt/timecapsule
08: mkdir /usr/local/share/doc
09: mkdir /usr/local/www/thomasguymer.co.uk
10: mkdir /usr/ports
11: mkdir /var/db
12: mkdir /var/log
13: # Mount the partitions
14: zfs set mountpoint=/home pool1/home
15: zfs set mountpoint=/mnt/data pool1/mnt/data
16: zfs set mountpoint=/mnt/timecapsule pool1/mnt/timecapsule
17: zfs set mountpoint=/usr/local/share/doc pool1/usr/local/share/doc
18: zfs set mountpoint=/usr/local/www/thomasguymer.co.uk pool1/usr/local/www/thomasguymer.co.uk
19: zfs set mountpoint=/usr/ports pool1/usr/ports
20: zfs set mountpoint=/usr/src pool1/usr/src
21: zfs set mountpoint=/var/db pool1/var/db
22: zfs set mountpoint=/var/log pool1/var/log
23: zfs set mountpoint=/var/mail pool1/var/mail
24: # Move already existing files into new versions
25: mv /usr/local/share/TEMPdoc/* /usr/local/share/doc/
26: mv /var/TEMPlog/* /var/log/
27: # Remove old versions
28: rmdir /usr/local/share/TEMPdoc
29: rmdir /var/TEMPlog
30: # Reboot
31: reboot
You may also download “freebsd-10p1-cmds5.sh” directly or view “freebsd-10p1-cmds5.sh” on GitHub Gist.

I like using BASH as my shell, which is not included in the default install of FreeBSD. I installed BASH using the port system, which means that I had to set up the port system first. The following command (as root) will download the port tree and extract it to the local directory: portsnap fetch extract.

Once that is done I created the “/etc/make.conf” file so that all ports were installed using the settings I wished. Use the Vi editor (by typing the command vi) to create the file and make it look like mine below.

1: # Set build options
2: BUILD_OPTIMIZED=yes
3: CPUTYPE=native
4: NO_INET6=yes
5: WITH_PKGNG=yes
6: 
7: # Set configure options
8: OPTIONS_SET=AVX AVX2 ICONV MANPAGES MMX OPENGL OPTIMIZATION OPTIMIZED_CFLAGS OPTIMIZED_FLAGS SIMD SSE SSE2 THREADS UNICODE X11
9: OPTIONS_UNSET=DEBUG DOCS EXAMPLES HTMLDOCS IPV6 PHP TEST
You may also download “freebsd-10p1-make.conf” directly or view “freebsd-10p1-make.conf” on GitHub Gist.

Finally, I installed BASH by typing make -C /usr/ports/shells/bash config-recursive first to configure it (deselect “implicitcd” when the first configuration screen appears) and then make -C /usr/ports/shells/bash install to actually install it. BASH will not work straight away as it needs two specific file-systems to exist. You can make them with appear at boot with the following two commands.

1: echo "fdesc    /dev/fd    fdescfs    rw    0    0" >> /etc/fstab
2: echo "proc    /proc    procfs    rw    0    0" >> /etc/fstab
You may also download “freebsd-10p1-cmds6.sh” directly or view “freebsd-10p1-cmds6.sh” on GitHub Gist.

BASH should now be there ready for you to use - after you do a reboot.

§2.2 Adding Users

I created all of the infrastructure for users to use the ZFS file-system, have a data store and a Time Capsule. This resulted in quite a few ZFS commands, but hopefully they will all make sense (and most are duplicates).

01: # Create their home
02: zfs create pool1/home/user1
03: zfs create pool1/home/user2
04: zfs create pool1/home/user3
05: zfs create pool1/home/user4
06: # Limit their home
07: zfs set refquota=100G pool1/home/user1
08: zfs set refquota=100G pool1/home/user2
09: zfs set refquota=100G pool1/home/user3
10: zfs set refquota=100G pool1/home/user4
11: # Create their data store
12: zfs create pool1/data/user1
13: zfs create pool1/data/user2
14: zfs create pool1/data/user3
15: zfs create pool1/data/user4
16: # Create their time machine store
17: zfs create pool1/timecapsule/user1
18: zfs create pool1/timecapsule/user2
19: zfs create pool1/timecapsule/user3
20: zfs create pool1/timecapsule/user4
21: 
You may also download “freebsd-10p1-cmds7.sh” directly or view “freebsd-10p1-cmds7.sh” on GitHub Gist.

Note that there is no need to put quotas on the Time Capsules as they are handled in the configuration file later on and work quite well. Users are added using the aptly named adduser command. Once that is done I modified their groups and the permissions of their directories.

01: # add all users to "staff"
02: pw groupmod staff -m user1,user2,user3,user4
03: # add "user1" to "wheel"
04: pw groupmod wheel -m user1
05: # add SFTP-only users to custom group
06: pw groupadd sftp_only -m user3,user4
07: # forbid "user4" from logging in
08: pw usermod user4 -s /usr/sbin/nologin
09: # Set secure umasks
10: echo umask=0077 > /home/user1/.profile
11: echo umask=0077 > /home/user2/.profile
12: echo umask=0077 > /home/user3/.profile
13: echo umask=0077 > /home/user4/.profile
14: # Setup "vim"
15: echo "syntax on" > /home/user1/.vimrc
16: echo "syntax on" > /home/user2/.vimrc
17: echo "syntax on" > /home/user3/.vimrc
18: echo "syntax on" > /home/user4/.vimrc
19: # Make users own their folders
20: chown user1:staff /home/user1 /mnt/data/user1 /mnt/timecapsule/user1
21: chown user2:staff /home/user2 /mnt/data/user2 /mnt/timecapsule/user2
22: chown user3:staff /home/user3 /mnt/data/user3 /mnt/timecapsule/user3
23: chown user4:staff /home/user4 /mnt/data/user4 /mnt/timecapsule/user4
24: # Make private folders private
25: chmod 700 /home/user1 /home/user2 /home/user3 /home/user4
26: chmod 700 /mnt/timecapsule/user1 /mnt/timecapsule/user2 /mnt/timecapsule/user3 /mnt/timecapsule/user4
27: # Make shared (parent) folders group read-only
28: chmod 755 /mnt/data/user1 /mnt/data/user2 /mnt/data/user3 /mnt/data/user4
29: # Make sure that users can get to their folders
30: chgrp staff /home /mnt /mnt/data /tmp
You may also download “freebsd-10p1-cmds8.sh” directly or view “freebsd-10p1-cmds8.sh” on GitHub Gist.

§2.3 Installing Software

The first thing that I did at this stage is to download the source code for FreeBSD. If you are just going to use this machine as a server or some such then you don’t need to do this. However, if you intend on using it as a build platform for other FreeBSD machines (such as ARM boards) then you’ll need the source code. You can grab it with the command svnlite co https://svn0.us-west.freebsd.org/base/head /usr/src/.

Next, I installed Portmaster as it is a handy tool for installing software from the ports tree. To install it you’ll need to run the command make -C /usr/ports/ports-mgmt/portmaster install.

Now I start installing other software that I need, using Portmaster instead of Make. Below you’ll find a list of all of the software that I have installed. I do each one in turn using the command portmaster -y foo/bar (i.e., portmaster -y editors/vim). Next to some of them is a description of what configuration settings I changed from default, some of them are configuration options for [grand-]daughters.

That will take a long time to run/wait for. Once it is all done I like to create a link to Python using this command cd /usr/local/bin && ln -s python2.7 python. I then append some lines to the end of “/etc/rc.conf” so that some of the recently installed software starts automatically. You can do this with the following commands.

01: # Enables SMART monitoring of hard drives
02: echo 'smartd_enable="YES"' >> /etc/rc.conf
03: # Enables the Time Capsules
04: echo 'netatalk_enable="YES"' >> /etc/rc.conf
05: echo 'cnid_metad_enable="YES"' >> /etc/rc.conf
06: echo 'afpd_enable="YES"' >> /etc/rc.conf
07: # Enable UPnP/DNLA
08: echo 'mediatomb_enable="YES"' >> /etc/rc.conf
09: echo 'mediatomb_flags="-i 192.168.1.110 -p 49152"' >> /etc/rc.conf
10: # Enable SMB
11: echo 'samba_enable="YES"' >> /etc/rc.conf
12: # Configure NTP
13: echo 'ntpd_sync_on_start="YES"' >> /etc/rc.conf
14: echo 'ntpd_hosts="uk.pool.ntp.org"' >> /etc/rc.conf
15: # Enable mail
16: echo 'sendmail_enable="YES"' >> /etc/rc.conf
17: # Enable DAAP
18: echo 'firefly_enable="YES"' >> /etc/rc.conf
19: # Enable web server
20: echo 'nginx_enable="YES"' >> /etc/rc.conf
21: # Enable protection of SSH
22: echo 'sshguard_enable="YES"' >> /etc/rc.conf
23: # HACK: Bug 174018 from https://www.freshports.org/security/sshguard
24: echo 'sshguard_blacklist=""' >> /etc/rc.conf
You may also download “freebsd-10p1-cmds9.sh” directly or view “freebsd-10p1-cmds9.sh” on GitHub Gist.

At this point it is probably worth doing a reboot just to check that everything is working OK for you. A lot of the daemons that have just been installed will probably not run because we have not configured them yet, but the fact that they print an error message means that they are at least trying to start up - which is good.

§2.4 Configuring Cron

Here is a copy of my Cron file, I think that it is fairly self-explanatory. All it does is correct permissions of user’s private/shared folders, check for updates in both FreeBSD and the ports system; and finally it performs some maintenance of the ZFS pool.

01: # Define environment variables
02: LANG=en_GB.UTF-8
03: PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/games:/usr/local/bin:/usr/local/sbin
04: SHELL=/usr/local/bin/bash
05: 
06: # Ensure user/group/permissions are correct for personal/shared folders
07: 0 0 * * * /usr/bin/find /mnt/data/user2/ ! -user user2 -exec chown user2 {} \;
08: 0 0 * * * /usr/bin/find /mnt/data/user2/ ! -group staff -exec chgrp staff {} \;
09: 0 0 * * * /usr/bin/find /mnt/data/user2/ -type f ! -perm 600 -exec chmod 600 {} \;
10: 0 0 * * * /usr/bin/find /mnt/data/user2/ -depth +0 -type d ! -perm 700 -exec chmod 700 {} \;
11: 0 0 * * * /usr/bin/find /mnt/data/user3/ ! -user user3 -exec chown user3 {} \;
12: 0 0 * * * /usr/bin/find /mnt/data/user3/ ! -group staff -exec chgrp staff {} \;
13: 0 0 * * * /usr/bin/find /mnt/data/user3/ -type f ! -perm 600 -exec chmod 600 {} \;
14: 0 0 * * * /usr/bin/find /mnt/data/user3/ -depth +0 -type d ! -perm 700 -exec chmod 700 {} \;
15: 0 0 * * * /usr/bin/find /mnt/data/user4/ ! -user user4 -exec chown user4 {} \;
16: 0 0 * * * /usr/bin/find /mnt/data/user4/ ! -group staff -exec chgrp staff {} \;
17: 0 0 * * * /usr/bin/find /mnt/data/user4/ -type f ! -perm 644 -exec chmod 644 {} \;
18: 0 0 * * * /usr/bin/find /mnt/data/user4/ -depth +0 -type d ! -perm 755 -exec chmod 755 {} \;
19: 
20: # Fetch updates for FreeBSD and Ports then clean the ports tree
21: 0 0 * * * /usr/sbin/freebsd-update cron > /dev/null
22: 0 0 * * * /usr/sbin/portsnap cron update > /dev/null
23: 0 2 * * * /usr/local/sbin/portmaster -y --clean-distfiles > /dev/null
24: 2 2 * * * /usr/local/sbin/portmaster -y --clean-packages > /dev/null
25: 4 2 * * * /usr/local/sbin/portmaster --check-depends
26: 
27: # Provide ZFS usage statistics
28: 0 0 * * * /sbin/zfs list -o name,used,avail,compressratio -r pool1/home > /home/USAGE.txt
29: 0 0 * * * /sbin/zfs list -o name,used,avail,compressratio -r pool1/mnt/data > /mnt/data/USAGE.txt
30: 0 0 * * * /sbin/zfs list -o name,used,avail,compressratio -r pool1/mnt/timecapsule > /mnt/timecapsule/USAGE.txt
31: 
32: # Scrub ZFS
33: 0 0 * * 1 /sbin/zpool scrub pool1
34: 
35: # Create ZFS snapshots
36: 0 0 * * 1-6 /usr/local/sbin/zfs-auto-snapshot daily  6
37: 0 0 * *   0 /usr/local/sbin/zfs-auto-snapshot weekly 5
You may also download “freebsd-10p1-crontab” directly or view “freebsd-10p1-crontab” on GitHub Gist.

§2.5 Configuring Software

I have highlighted a few keys pieces of software (such as media servers and the like) to describe how I configured them for my needs.

§2.5.1 Time Machine

The settings for the Time Capsules are kept in “/usr/local/etc/AppleVolumes.default”. Use your favourite editor (I use Vim) to change the file so that the un-commented lines look like the ones below. Now Mac computers should be able to see, and use, your FreeBSD machine as a Time Capsule.

1: :DEFAULT: options:upriv,usedots
2: 
3: /mnt/timecapsule/user1 Time Capsule-user1 options:tm volsizelimit:500000 allow:user1
4: /mnt/timecapsule/user2 Time Capsule-user2 options:tm volsizelimit:500000 allow:user2
5: /mnt/timecapsule/user3 Time Capsule-user3 options:tm volsizelimit:500000 allow:user3
6: /mnt/timecapsule/user4 Time Capsule-user4 options:tm volsizelimit:500000 allow:user4
You may also download “freebsd-10p1-AppleVolumes.default” directly or view “freebsd-10p1-AppleVolumes.default” on GitHub Gist.
§2.5.2 Web Server

Above I installed Nginx so that I could have a very light weight webserver on my local network. The settings for Nginx are in “/usr/local/etc/nginx/nginx.conf”. Use your favourite editor (I use Vim) to change the file so that it looks like the one below.

01: # Basic settings
02: worker_processes            auto;
03: 
04: events {
05:     worker_connections          1024;
06: }
07: 
08: http {
09:     # Basic settings
10:     charset                     utf8;
11:     gzip_static                 on;
12:     server_tokens               off;
13:     sendfile                    off;
14:     tcp_nopush                  on;
15:     limit_conn_zone             $binary_remote_addr zone=addr:10m;
16: 
17:     # Buffers
18:     client_body_buffer_size     1k;
19:     client_header_buffer_size   1k;
20:     client_max_body_size        1k;
21:     large_client_header_buffers 2 1k;
22: 
23:     # Timeouts
24:     client_body_timeout         10;
25:     client_header_timeout       10;
26:     keepalive_timeout           5 5;
27:     send_timeout                10;
28: 
29:     # Define the default MIME types for files
30:     include                     mime.types;
31:     default_type                application/octet-stream;
32: 
33:     server {
34:         # Basic settings
35:         listen                      80;
36:         server_name                 www.thomasguymer.co.uk;
37:         if ($request_method !~ ^(GET|HEAD)$ ) {
38:             return                      444;
39:         }
40: 
41:         location / {
42:             root                        /usr/local/www/thomasguymer.co.uk/www;
43:             index                       index.html;
44:             limit_conn                  addr 5;
45:         }
46: 
47:         # List all available MIME types
48:         types {
49:             application/javascript      js;
50:             application/pdf             pdf;
51:             application/rss+xml         rss;
52:             application/xml             xml;
53:             application/zip             zip;
54:             image/x-icon                ico;
55:             image/jpeg                  jpg;
56:             image/png                   png;
57:             image/svg+xml               svg;
58:             text/css                    css;
59:             text/html                   html;
60:             text/plain                  log txt;
61:             text/x-suse-ymp             ymp;
62:         }
63:     }
64: }
You may also download “freebsd-10p1-nginx.conf” directly or view “freebsd-10p1-nginx.conf” on GitHub Gist.
§2.5.3 Media Servers

I have installed two separate media servers to serve my music and videos over different protocols: Firefly (a.k.a. mt-daapd) serves content over DAAP for Apple devices; and Mediatomb serves content over UPnP/DNLA for Android, Linux and Windows devices. The two configuration files for these pieces of software are “/usr/local/etc/firefly/mt-daapd.conf” and “/usr/local/etc/mediatomb/config.xml” respectively. Below I have included both configuration files - Firefly first and Mediatomb second.

01: [general]
02: web_root = /usr/local/share/mt-daapd/admin-root
03: port = 3689
04: admin_pw = PASSWORD
05: db_type = sqlite3
06: db_parms = /var/db/firefly
07: mp3_dir = /mnt/data/user1/Music
08: servername = HOST
09: runas = daapd
10: extensions = .mp3
11: logfile = /var/log/mt-daapd.log
12: debuglevel = 1
13: rescan_interval = 1800
14: scan_type = 2
15: 
16: [plugins]
17: plugin_dir = /usr/local/lib/mt-daapd/plugins
18: 
19: [scanning]
20: process_playlists = 0
21: process_itunes = 0
22: process_m3u = 0
You may also download “freebsd-10p1-mt-daapd.conf” directly or view “freebsd-10p1-mt-daapd.conf” on GitHub Gist.
01: <?xml version="1.0" encoding="UTF-8"?>
02: <config xmlns="http://mediatomb.cc/config/2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="2" xsi:schemaLocation="http://mediatomb.cc/config/2 http://mediatomb.cc/config/2.xsd">
03:     <server>
04:         <ui enabled="yes" show-tooltips="yes">
05:             <accounts enabled="yes" session-timeout="30">
06:                 <account user="mediatomb" password="PASSWORD"/>
07:             </accounts>
08:         </ui>
09:         <name>HOST</name>
10:         <udn>uuid:9dafbdf1-904a-4016-9af9-b0bc756c7537</udn>
11:         <home>/var/mediatomb</home>
12:         <webroot>/usr/local/share/mediatomb/web</webroot>
13:         <storage caching="yes">
14:             <sqlite3 enabled="yes">
15:                 <database-file>mediatomb.db</database-file>
16:             </sqlite3>
17:             <mysql enabled="no">
18:                 <host>localhost</host>
19:                 <username>mediatomb</username>
20:                 <database>mediatomb</database>
21:             </mysql>
22:         </storage>
23:         <protocolInfo extend="no"/>
24:         <custom-http-headers>
25:             <add header="transferMode.dlna.org: Streaming"/>
26:             <add header="contentFeatures.dlna.org: DLNA.ORG_OP=01;DLNA.ORG_CI=0;DLNA.ORG_FLAGS=017000 00000000000000000000000000"/>
27:         </custom-http-headers>
28:         <extended-runtime-options>
29:             <ffmpegthumbnailer enabled="yes">
30:                 <thumbnail-size>128</thumbnail-size>
31:                 <seek-percentage>5</seek-percentage>
32:                 <filmstrip-overlay>no</filmstrip-overlay>
33:                 <workaround-bugs>no</workaround-bugs>
34:                 <image-quality>8</image-quality>
35:             </ffmpegthumbnailer>
36:             <mark-played-items enabled="no" suppress-cds-updates="yes"/>
37:         </extended-runtime-options>
38:     </server>
39:     <import hidden-files="no">
40:         <scripting script-charset="UTF-8">
41:             <common-script>/usr/local/share/mediatomb/js/common.js</common-script>
42:             <playlist-script>/usr/local/share/mediatomb/js/playlists.js</playlist-script>
43:             <virtual-layout type="builtin">
44:                 <import-script>/usr/local/share/mediatomb/js/import.js</import-script>
45:             </virtual-layout>
46:         </scripting>
47:         <mappings>
48:             <extension-mimetype ignore-unknown="no">
49:                 <map from="flac" to="audio/x-flac"/>
50:                 <map from="flv" to="video/x-flv"/>
51:                 <map from="m3u" to="audio/x-mpegurl"/>
52:                 <map from="mp3" to="audio/mpeg"/>
53:                 <map from="mp4" to="video/mp4"/>
54:                 <map from="mpg" to="video/mpeg"/>
55:                 <map from="pls" to="audio/x-scpls"/>
56:                 <map from="wma" to="audio/x-ms-wma"/>
57:                 <map from="wmv" to="video/x-ms-wmv"/>
58:             </extension-mimetype>
59:             <mimetype-upnpclass>
60:                 <map from="audio/*" to="object.item.audioItem.musicTrack"/>
61:                 <map from="image/*" to="object.item.imageItem"/>
62:                 <map from="video/*" to="object.item.videoItem"/>
63:             </mimetype-upnpclass>
64:             <mimetype-contenttype>
65:                 <treat mimetype="audio/mpeg" as="mp3"/>
66:                 <treat mimetype="audio/x-flac" as="flac"/>
67:                 <treat mimetype="audio/x-mpegurl" as="playlist"/>
68:                 <treat mimetype="audio/x-scpls" as="playlist"/>
69:                 <treat mimetype="audio/x-wav" as="pcm"/>
70:                 <treat mimetype="image/jpeg" as="jpg"/>
71:                 <treat mimetype="video/x-msvideo" as="avi"/>
72:             </mimetype-contenttype>
73:         </mappings>
74:     </import>
75:     <transcoding enabled="no"/>
76: </config>
You may also download “freebsd-10p1-config.xml” directly or view “freebsd-10p1-config.xml” on GitHub Gist.

For advice on mt-daapd then check out this excellent article. I feel like the Mediatomb one can still be trimmed out some more. It is derived from the one that is included in the package but I do feel that a lot of the MIME-type related lines are superfluous.

§2.5.4 Samba

I found Chapter 9 of the Samba documentation particularly useful for this part. The settings for Samba are kept in “/usr/local/etc/smb4.conf”. Use your favourite editor (I use Vim) to change the file so that the un-commented lines look like the ones below.

01: [global]
02:     netbios name = HOST
03:     encrypt passwords = yes
04:     server string = HOST Samba Folders
05:     workgroup = WORKGROUP
06:     hosts allow = 192.168.1.
07:     load printers = no
08:     log file = /var/log/samba4.log
09:     map to guest = bad user
10: 
11: [Music]
12:     comment = Music Share
13:     path = /mnt/data/user1/Music
14:     read only = yes
15:     guest ok = yes
16:     only guest = yes
17: 
18: [Videos]
19:     comment = Videos Share
20:     path = /mnt/data/user1/Videos
21:     read only = yes
22:     guest ok = yes
23:     only guest = yes
24: 
25: [Origin]
26:     comment = Origin Backup
27:     path = /mnt/data/user4
28:     read only = no
29:     create mask = 0644
30:     directory mask = 0755
31:     valid users = user4
You may also download “freebsd-10p1-smb4.conf” directly or view “freebsd-10p1-smb4.conf” on GitHub Gist.

Now you need to go ahead and actually create the Samba users (and their passwords), which are in addition to the FreeBSD users. To create the guest account run adduser like below.

01: Username: guest
02: Full name: guest
03: Uid (Leave empty for default):
04: Login group [guest]: staff
05: Login group is staff. Invite guest into other groups? []: sftp_only
06: Login class [default]:
07: Shell (sh csh tcsh bash rbash git-shell nologin) [sh]: nologin
08: Home directory [/home/guest]:
09: Home directory permissions (Leave empty for default):
10: Use password-based authentication? [yes]:
11: Use an empty password? (yes/no) [no]:
12: Use a random password? (yes/no) [no]:
13: Enter password:
14: Enter password again:
15: Lock out the account after creation? [no]:
16: Username   : guest
17: Password   : *****
18: Full Name  : guest
19: Uid        : 1011
20: Class      :
21: Groups     : staff sftp_only
22: Home       : /home/guest
23: Home Mode  :
24: Shell      : /usr/sbin/nologin
25: Locked     : no
26: OK? (yes/no): yes
27: adduser: INFO: Successfully added (guest) to the user database.
28: Add another user? (yes/no): no
29: Goodbye!
You may also download “freebsd-10p1-adduser” directly or view “freebsd-10p1-adduser” on GitHub Gist.

Now it is time to add the Samba users: this is done by using smbpasswd like below.

1: smbpasswd -a user1
2: smbpasswd -a user2
3: smbpasswd -a user3
4: smbpasswd -a user4
5: smbpasswd -e user1
6: smbpasswd -e user2
7: smbpasswd -e user3
8: smbpasswd -e user4
You may also download “freebsd-10p1-cmds10.sh” directly or view “freebsd-10p1-cmds10.sh” on GitHub Gist.
§2.5.5 SSH

The settings for the SSH server are kept in “/etc/ssh/sshd_config”. Use your favourite editor (I use Vim) to change the file so that the un-commented lines look like the ones below.

01: # Accept environment variables LANG, LC_*, TIMEFORMAT and TZ
02: AcceptEnv LANG LC_* TIMEFORMAT TZ
03: 
04: # Show banner before logging in
05: Banner /etc/banner
06: 
07: # Make sure that users are still connected
08: ClientAliveInterval 5m
09: 
10: # Kill in-active login attempts
11: LoginGraceTime 5m
12: 
13: # Log all activity
14: LogLevel VERBOSE
15: 
16: # Number of password attempts
17: MaxAuthTries 4
18: 
19: # Number of hanging connections
20: MaxStartups 4
21: 
22: # Disable root login
23: PermitRootLogin no
24: 
25: # Enable port forwarding
26: PermitTunnel yes
27: 
28: # Maintain privacy
29: PrintLastLog no
30: 
31: # Use internal SFTP process with increased logging and file mode restrictions
32: Subsystem sftp internal-sftp -l VERBOSE -u 077
33: 
34: # Disable reverse lookup
35: UseDNS no
36: 
37: # Enable X11 forwarding
38: X11Forwarding yes
39: 
40: # Specific options for "sftp_only" group members
41: Match group sftp_only
42:     # Disallow agent forwarding
43:     AllowAgentForwarding no
44: 
45:     # Disallow TCP forwarding
46:     AllowTCPForwarding no
47: 
48:         # Chroot to the ZFS data partition
49:         ChrootDirectory /mnt/data
50: 
51:     # Force all connections to be SFTP
52:     ForceCommand internal-sftp -l VERBOSE -u 077
53: 
54:     # Disable port forwarding
55:     PermitTunnel no
56: 
57:     # Disable X11 forwarding
58:     X11Forwarding no
59: 
60: # Specific options for "user1" user
61: Match user user1
62:     # Hide banner
63:     Banner none
You may also download “freebsd-10p1-sshd_config” directly or view “freebsd-10p1-sshd_config” on GitHub Gist.

All the options are pretty self-explanatory I think but I would like to draw your attention to the line ForceCommand internal-sftp -l VERBOSE -u 077 under the section Match group sftp_only. These two lines mean that users who are members of the group “sftp_only” (“user3” and “user4” in this tutorial) cannot run commands over SSH on my server but do have the ability to use it’s storage over an SFTP connection. If one of those users tries to connect using ssh user3@HOST then it will always fail but they can still transfer files using an encrypted SFTP connection. In essence, my server is simply a data store for them and it will refuse all attempts to log in, run commands, forward ports or open GUIs.

§2.5.6 SMART Daemon

The settings for the SMART daemon are kept in “/usr/local/etc/smartd.conf”. Use your favourite editor (I use Vim) to change the file so that the un-commented lines look like the ones below. This will start a short self-test on all disks between 2AM and 3AM on Tuesday morning and any changes in their health will be emailed to you.

1: DEFAULT -m user@example.com
2: /dev/ada0 -s S/../../2/02
3: /dev/ada1 -s S/../../2/02
4: /dev/ada2 -s S/../../2/02
5: /dev/ada3 -s S/../../2/02
6: /dev/ada4 -s S/../../2/02
7: /dev/ada5 -s S/../../2/02
You may also download “freebsd-10p1-smartd.conf” directly or view “freebsd-10p1-smartd.conf” on GitHub Gist.

§3 Workarounds

For some reason the default installation of Python from the ports system does not load up the machine’s SSL certificates correctly. This means that Python scripts which connect to servers securely, such as youtube-dl, fail with “CERTIFICATE_VERIFY_FAILED” errors. After a fair amount of searching I found other people had similar issues. I was able to implement a workaround by typing the following command as root ln -s /usr/local/etc/ssl/cert.pem /etc/ssl/cert.pem.

§4 Usage Notes

To find a port in the port tree run whereis bar.

To perform an audit of all known security holes in the installed software run pkg audit. Before you update the software using portmaster foo/bar you should check UPDATING to see if there are any issues for updating any specific package.

To update the base of FreeBSD run freebsd-update fetch install.

To restart servers run one of the following commands:

To update the time after a long down period run (during which the clock may have drifted) ntpdate -v -b uk.pool.ntp.org.

To upgrade all installed ports in the correct order run portmaster -afy.

To list all ports that have upgrades available run portmaster -L | grep "New version available" | sort.

To upgrade from 10.0-RELEASE to 10.1-RELEASE (according to the Installation Notes) run the following commands (I have added the -m DISABLE_VULNERABILITIES=yes because sometimes updating FreeBSD to a secure version is more important than ensuring all installed packages are themselves secure):

1: freebsd-update fetch install
2: reboot
3: freebsd-update upgrade -r 10.1-RELEASE
4: freebsd-update install
5: reboot
6: freebsd-update install
7: portmaster -B -f -m DISABLE_VULNERABILITIES=yes -a
8: # Run "portmaster -B -R -f -m DISABLE_VULNERABILITIES=yes -a" if you were interrupted and need to restart from where you left off
9: freebsd-update install
10: reboot
You may also download “freebsd-10p1-cmds11.sh” directly or view “freebsd-10p1-cmds11.sh” on GitHub Gist.

If you have changed the umask of the root account then some permissions will be more restrictive than intended. To correct port’s documentation installation (which assumes default umask) run the following commands:

1: find /usr/local/share/doc ! -perm 755 -type d -exec chmod 755 {} \;
2: find /usr/local/share/doc ! -perm 644 -type f -exec chmod 644 {} \;
You may also download “freebsd-10p1-cmds12.sh” directly or view “freebsd-10p1-cmds12.sh” on GitHub Gist.

If you have changed the umask of the root account then some permissions will be more restrictive than intended. To correct a PIP installation (which assumes default umask) run the following commands:

1: find /usr/local/lib/python2.7/site-packages ! -perm 755 -type d -exec chmod 755 {} \;
2: find /usr/local/lib/python2.7/site-packages ! -perm 644 -type f -exec chmod 644 {} \;
You may also download “freebsd-10p1-cmds13.sh” directly or view “freebsd-10p1-cmds13.sh” on GitHub Gist.

To update all PHP 5.6 packages, for example, run portmaster $(portmaster -l | egrep -o php56.+ | sed "s/-5.6.*//g;s/-1.0//g").

§5 References

For advice on which folders to copy on Windows to backup Origin downloads then check out this informative article (remembering to connect to the SMB share as “WORKGROUP\user4” from Windows).

§6 Word Of Warning

A xkcd comic about installing *BSD
© xkcd