Posted in ubuntu Read more
Recently I decided to replace NFS on a small network with something that was more secure and more resistant to network failures (as in, Nautilus wouldn’t hang because of a symlink to a directory in an autofs mounted NFS share). Most importantly, I wanted something that was secure, simple and robust. I naturally thought of SFTP, but there were at least two problems with a naive SFTP implementation, both of which I decided I must solve to meet the ‘secure’ criteria:
Of course, there are other alternatives that I could have pursued: sshfs, shfs, SFTP with scponly, patching OpenSSH to support chrooting (see ‘Update’ below), NFSv4, IPsec, etc. Adding all the infrastructure to properly support NFSv4 and IPsec did not meet the simplicity requirement, and neither did running a patched OpenSSH server. sshfs, shfs, and SFTP with scponly did not really fit the bill either.
What did I come up with? A combination of SFTP, a hardened sshd configuration and AppArmor on Ubuntu.
This was easy because sftp is enabled by default on Ubuntu and Debian systems. This should be enough:
$ sudo apt-get install openssh-server
Just make sure you have the following set in /etc/ssh/sshd_config (see man sshd_config for details).
Subsystem sftp /usr/lib/openssh/sftp-server
With that in place, all I needed to do was add users with strong passwords:
$ sudo adduser sftupser1
$ sudo adduser sftpuser2
Then you can test if it worked with:
$ sftp sftpuser1@server
sftp> ls /
Now that SFTP is working, we need to limit access. One way to do this is via a Match rule that uses a ForceCommand. Combined with AllowUsers, adding something like this to /etc/ssh/sshd_config is pretty powerful:
AllowUsers adminuser email@example.com sftpuser2
Match User sftpuser1,sftpuser2
ForceCommand /usr/lib/openssh/sftp-server -l INFO
Remember to restart ssh with ‘/etc/init.d/ssh restart’. The above allows normal shell access for the adminuser, SFTP-only access to sftpuser1 from 192.168.0.10 and to sftupser2 from anywhere. One can imagine combining this with ‘PasswordAuthentication no’ or GSSAPI to enforce more stringent authentication so access is even more tightly controlled.
The above does a lot to increase security over the standard NFS shares that I had before. Good encryption, strong authentication and reliable UID mappings for DAC (POSIX discretionary access controls) are all in place. However, it doesn’t have the ability to confine access to a certain directory like NFS does. A simple AppArmor profile can achieve this and give even more granularity than just DAC. Imagine the following directory structure:
Now add to /etc/apparmor.d/usr.lib.openssh.sftp-server (and enable with ‘sudo apparmor_parser -r /etc/apparmor.d/usr.lib.openssh.sftp-server’):
# Served files
# Need read access for every parent directory
owner /var/exports/** rwkl,
# don't require ownership to read shared files
This is a very simple profile for sftp-server itself, with access to files in /var/exports. Notice that by default the owner must match for any files in /var/exports, but in /var/exports/shared it does not. AppArmor works in conjunction with DAC, so that if DAC denies access, AppArmor is not consulted. If DAC permits access, then AppArmor is consulted and may deny access.
This is but one man’s implementation for a simple, secure and robust file service. There are limitations with the method as described, notably managing the sshd_config file and not supporting some traditional setups such as $HOME on NFS. That said, with a little creativity, a lot of possibilities exist for file serving with this technique. For my needs, the combination of standard OpenSSH and AppArmor on Ubuntu was very compelling. Enjoy!
OpenSSH 4.8 and higher (available in Ubuntu 8.10 and later) contains the ChrootDirectory option, which may be enough for certain environments. It is simpler to setup (ie AppArmor is not required), but doesn’t have the same granularity and sftp-server protection that the AppArmor method provides. See comment 32 and comment 34 for details. Combining ChrootDirectory and AppArmor would provide even more defense in depth. It’s great to have so many options for secure file sharing! :)
After a lot of hard work by John Johansen and the Ubuntu kernel team, bug #375422 is well on its way to be fixed. More than just forward ported for Ubuntu, AppArmor has been reworked to use the updated kernel infrastructure for LSMs. As seen in #apparmor on Freenode a couple of days ago:
11:24 < jjohansen> I am working to a point where I can try upstreaming again, base off of the security_path_XXX patches instead of the vfs patches
11:24 < jjohansen> so the module is mostly self contained again
These patches are in the latest 9.10 kernel, and help testing AppArmor in Karmic is needed. To get started, verify you have at least 2.6.31-3.19-generic:
$ cat /proc/version_signature
AppArmor will be enabled by default for Karmic just like in previous Ubuntu releases, but it is off for now until a few kinks are worked out. To test it right away, you’ll need to reboot, adding ‘security=apparmor’ to the kernel command line. Then fire up ‘aa-status’ to see if it is enabled. A fresh install of 9.10 as of today should look something like:
$ sudo aa-status
apparmor module is loaded.
8 profiles are loaded.
8 profiles are in enforce mode.
0 profiles are in complain mode.
2 processes have profiles defined.
2 processes are in enforce mode :
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.
Please throw all your crazy profiles at it as well as testing the packages with existing profiles, then file bugs:
Thank you Ubuntu Kernel team and especially John for all the hard work on getting the “MAC system for human beings” (as I like to call it) not only working again, but upstreamable — this is really great stuff! :)
Short answer: Of course not!
Ubuntu has had AppArmor1 on by default for a while now, and with each new release more and more profiles are added. The Ubuntu community has worked hard to make the installed profiles work well, and by far and large, most people happily use their Ubuntu systems without noticing AppArmor is even there.
Of course, like with any software, there are bugs. I know these AppArmor profile bugs can be frustrating, but because AppArmor is a path-based system, diagnosing, fixing and even working around profile bugs is actually quite easy. AppArmor has the ability to disable specific profiles rather than simply turning it on or off, yet I’ve seen people in IRC and forums advise others to disable AppArmor completely. This is totally misguided and YOU SHOULD NEVER DISABLE APPARMOR ENTIRELY to work around a profiling problem. That is like trying to open your front door with dynamite– it will work, but it’ll leave a big hole and you’ll likely hurt yourself. Think about it, on my regular ol’ Jaunty laptop system, I have 4 profiles in place installed via Ubuntu packages (to see the profiles on your system, look in /etc/apparmor.d). Why would I want to disable all of AppArmor (and therefore all of those profiles) instead of dealing with just the one that is causing me problems? Obviously, the more software you install with AppArmor protection, the more you have to lose by disabling AppArmor completely.
So, when dealing with a profile bug, there are only a few things you need to know:
With this in mind, let’s say tcpdump is misbehaving. You can check /var/log/kern.log for entries like:
Jul 7 12:21:15 laptop kernel: [272480.175323] type=1503 audit(1246987275.634:324): operation="inode_create" requested_mask="a::" denied_mask="a::" fsuid=0 name="/opt/foo.out" pid=24113 profile="/usr/sbin/tcpdump"
That looks complicated, but it isn’t really, and it tells you everything you need to know to file a bug and fix the problem yourself. Specifically, “/usr/sbin/tcpdump” was denied “a” access to “/opt/foo.out”.
So now what?
If you are using the program with a default configuration or non-default but common configuration, then by all means, file a bug. If unsure, ask on IRC, on a mailing list or just file it anyway.
If you are a non-technical user or just need to put debugging this issue on hold, then you can disable this specific profile (there are others ways of doing this, but this method works best):
$ sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.tcpdump
$ sudo ln -s /etc/apparmor.d/usr.sbin.tcpdump /etc/apparmor.d/disable/usr.sbin.tcpdump
What that does is remove the profile for tcpdump from the kernel, then disables the profile such that AppArmor won’t load it when it is started (eg, on reboot). Now you can use the application without AppArmor protection, but leaving all those other applications with profiles protected.
If you are technically minded, dive into /etc/apparmor.d/usr.sbin.tcpdump and adjust the profile, then reload it:
$ sudo <your favorite editor> /etc/apparmor.d/usr.sbin.tcpdump
$ sudo apparmor_parser -r /etc/apparmor.d/usr.sbin.tcpdump
This will likely be an iterative process, but you can base your new or updated rules on what is already the profile– it is pretty straightforward. After a couple of times, it will be second nature and you might want to start contributing to developing new profiles. Once the profile is working for you, please add your proposed fix to the bug report you filed earlier.
The DebuggingApparmor page has information on how to triage, fix and work-around AppArmor profile bugs. To learn more about AppArmor and the most frequently used access rules, install the apparmor-docs package, and read /usr/share/doc/apparmor-docs/techdoc.pdf.gz.
1. For those of you who don’t know, AppArmor is a path-based (as opposed to SELinux, which is inode-based) mandatory access control (MAC) system that limits access a process has to a predefined set of files and operations. These access controls are known as ‘profiles’ in AppArmor parlance.
It all started back in the good ol’ days of the Jaunty development cycle when I heard this new fangled filesystem thingy called ext4 was going to be an option in Jaunty. It claimed to be faster with much shorter fsck times. So, like any good Ubuntu developer, I tried it. It was indeed noticeably faster and fsck times were much improved.
The honeymoon was over when I ended up hitting bug #317781. That was no fun as it ate several virtual machines and quite a few other things (I had backups for all but the VMs). This machine is on a UPS, uses raid1, and on modern hardware (dual core Intel system with 4GB of RAM). In other words, this is not some flaky system but one that normally is only rebooted when there is a kernel upgrade (well, that is a white lie, but you get the point– it is a stable machine). According to Ted Ts’o, I shouldn’t be seeing this. Frustrated, I spent the better part of a weekend shuffling disks around to try to move my data off of ext4 and reformat the drive back to ext3. I was, how shall I put it, disappointed.
Some welcome patches were applied to Jaunty’s kernel soon after that to make ext4 behave more like ext3. By all accounts this stops ext4 from eating files under adverse conditions. So, now not only does the filesystem perform well, it doesn’t eat files. Life was good…
… until I noticed that under certain conditions I would get a total system freeze. Naturally, there was nothing in the logs (something I always appreciate ;). I thought it might have been several things, but I couldn’t prove any of them. Yesterday, however, I was able to reliably freeze my system. Basically, I was compiling a Jaunty kernel (2.6.28) in a schroot and using this command:
$ CONCURRENCY_LEVEL=2 AUTOBUILD=1 NOEXTRAS=1 skipabi=true fakeroot debian/rules binary-generic
Things were going along fine until I tried to delete a ton of files in a deep directory during the compile, then it would freeze. I was able to reproduce this 3 times in a row. Finally, I shuffled some things around and put my /home partition back on ext3 and I could not reproduce the freeze. There are several bugs in Launchpad talking about ext4 and system freezes, and after a bit more research I will add my comments, but for now, I am simply hopeful I will not see the freezes any more.
To be fair, ext4 is not the default filesystem on 9.04, and while it is supposed to be in 9.10, people I know running ext4 on 9.10 aren’t seeing these problems (yes, I’ve asked around). I do continue to use ext4 for ‘/’ on Jaunty systems with a separate /home partition as ext3, because it really does perform better, and this seems to be a good compromise. Having been burned by ext4 a couple of times now, I think it’ll probably be awhile before I trust ext4 for my important data though. Time will tell. :)
While not exactly news as it happened sometime last month, ufw is now in Debian and is even available in Squeeze. What is new is that the fine folks in Debian have started to translate the debconf strings in ufw, and during the process the strings are much better. Thanks Debian!
In other news, ufw trunk now has support for filtering by interface. To use it, do something like:
$ sudo ufw allow in on eth0 to any port 80
See the man page for more information. This feature will be in ufw 0.28 which is targeted for Ubuntu Karmic and I also hope to add egress filtering this cycle. I haven’t started on egress filtering yet, but I have a good idea on how to proceed. Stay tuned!
© 2010 Canonical Ltd. Ubuntu and Canonical are registered trademarks of Canonical Ltd.