nigel.mcnie.name

Why Web Application Control Panels Should Die

Sat, 06 Mar 2010 01:33:00 GMT+12

The control panel. It's a habit that I'm sure has been driven by lazy developers and open-source applications that have to run on shared hosting. But when you're making an application for the big bad interwebs, a control panel is a ticking timebomb just waiting for a clever hacker to exploit for their own nefarious purposes. It's time they died, and were replaced with the control application.

The Timebomb

What is a control panel anyway? It's the part of a webapp that administrators can use to perform administration tasks. Deleting user accounts, resetting passwords, undeleting users, resetting more passwords... you get the idea. Depending on the site, the control panel can have all sorts of functionality, such as showing site metrics and the ability to flick the site into maintenance mode.

Typically, security is provided by the control panel allowing site administrators only. I think this is a naive approach, relying on a combination of your imagined security prowess and magic fairy dust. There are several avenues of attack for a someone wanting to abuse your control panel, here are some of them:

If you make one mistake in the authentication code, just one, it can be enough for an attacker to pry wide open. Things have certainly gotten better than the heady days of magic_quotes_gpc, but how sure are you that your authentication logic is bullet proof? Would you bet your life on it? Your business? How well have you tested it? Have you fuzz tested it?
Even if your code is perfect - which you know deep down it might not be - many control panels include the ability to promote and demote administrators. One slip, and you or an employee might accidentally grant someone rights they should never have.
And even if your people are sensible, you just need to make one mistake again, and a hacker will abuse a CSRF or XSS hole in your site to perform an admin action. You might have a form library that does session keys for you, a templating system that outputs everything HTML escaped by default, and yet still one mistake could provide an open door for an attacker to pry their way into your system. Again, I don't think you'd bet your life on it.
Finally, an oldie but a goodie - an attacker could simply guess the username and password of an administrator account. This is possible in any number of ways, and as the number of people with administrator rights grows, this danger grows proportionately.

Root-cause Analysis

If you're now hearing the ticking sound of doom when thinking about your control panel, this is a good sign [1]. Now you're aware of the problem, let's have a look at how it came about and how we can fix it.

The problem we are trying to solve is administering the site and its users without resorting to hacking around in the database. We want the ability to perform some quite complicated management tasks easily, as eventually Support will be given the job of looking after the site, and they need to be able to perform as many tasks as possible without bothering Engineering in any way.

As I see it, the problem with the control panel approach is that it puts far too much code into harms way. All the due dilligence in the world can't prevent one honest mistake causing an access breach, and the control panel is full with goodies to play with. Delete this user? check. Deface the homepage? Check. Steal confidential reports and site metrics? Check. And so on...

Here's a question: Do actions like resetting passwords need to require using your application?

Answer: Your application's libraries and models would certainly make it easier! But the other parts of your app aren't necessary.

I think this answer holds the key to a good solution. Rather than build a control panel on the side of the application, why not put some of your code to double use as a scaffolding for two apps - the user one, and a control application?

The Control Application

This app is built using underlying libraries for your site, but just contains administration functions. In MVC terms, you simply have a different set of controllers for each application. They talk to the same database, but perform completely different functions.

Here's the benefit - you can run the control application inside your network only. Your users don't see this application, they can't see it, it simply doesn't exist to them. Even better, should one of them decide to hack your site, there isn't any admin code to exploit. There's no admin panel to defeat the authentication of and no way an CSRF attack will result in a sticky end for your site.

In fact, the control application could be an instance of your site with some extra controllers included. Then your public site doesn't even need site administrators anymore, as your support staff can do the user-interaction tasks like assuming identities from within the control app.

With this change, you've massively reduced the attack surface of your application without compromising any functionality. There's no site administration accounts available for hackers to compromise and no passwords they can guess.

Discussion

This strategy works as long as all site admins can access the network where the control app runs. This is a fair requirement for most apps, and not for some. In particular, if some users need some "admin" type functionality, you'll still need to build that into the public facing application. But you've still benefited by cutting down the attack surface available, as it's unlikely that those users will need the nuclear weapon features like mass account deletion.

There's a few different ways you can deal with the admin accounts. As the user database is shared, it may make sense for the control application to look in a different database for authentication credentials. This really drives home the separation between your users and site administrators. You could also include admin accounts within your user database, flat out rejecting admin logins on the public site while rejecting user logins in the control application.

This doesn't entirely prevent the possibility of CSRF. If your admin application is running at http://foobar:8888/, a hacker could still try to make someone visit a page containing an CSRF attack on http://foobar:8888/accounts/delete_all or similar - but they'd have to know the control application's location in the first place. At least, unlike with a control panel, they don't have an easily guessable URL like http://example.com/admin/ to aim at.

Sorry, open source peeps, while you insist on supporting shared hosting you're a bit stuck. But with a little imagination, I think you could do better than you are. For example, it's impossible to disable the Drupal admin panel, though a brutal rm -rf drupal/admin might do it. Why not go one step further, and allow a drupal to be configured to run in 'control application' mode, using a different database for authentication?

As with the disassociation method of deleting user accounts, we're implementing this on Get Your Game On. I'll report back on how it works out as we go.

[1]	Not the ticking obviously, you should get yourself checked for Tinnitus if symptoms persist.

Mahara Book

Fri, 05 Mar 2010 12:17:32 GMT+12

In case you missed the announcement, the Mahara book has been published. I helped review the content, and I think it's a great way learn about the Mahara system. Most of Mahara 1.2's features are covered well by Derrin and his team, so if you're a Mahara user I recommend you check it out.

Note: the book covers the user features, not the administration side - I understand that's coming in another book that will be a great companion to this one.

Who Likes Puppies?

Fri, 26 Feb 2010 00:16:47 GMT+12

You do, that's who! What kind of terrorist doesn't like them?

If you do, check out Mum's new Golden Retriever site - Salhouse Goldens. Lots of photos of goldies and other animals. I persuaded Mum to CC license them (BY-NC-ND), so if you need some excellent animal photos, check it out. Mum really does get some great pictures of the dogs on the long walks they do.

How to Delete User Accounts Properly: The Disassociation Method

Sat, 20 Feb 2010 01:30:50 GMT+12

We've all used websites where a user account is required. What happens when you decide that you don't want an account on a certain site any more?

The number of sites who haven't thought about this problem and implemented a sensible account deletion policy is staggering. Yet I believe that sites can deal with this problem in a way that is easy to implement, will reduce support costs and increase goodwill among their (ex-)user base.

Read on, and see what you think. We're going to put this method to the test with Get Your Game On, and I'll post back here with the results once it's been in play for a while.

The User Perspective

The site that raised my ire today was Posterous. They offer a cool service, and I thought it might be a good way for my mother to keep a website about her Golden Retrievers, but it turned out Posterous wasn't a good fit for her (she was more interested in having lots of static pages and image galleries, so we went with Google Sites in the end).

This left me today hunting around in the Posterous UI for the "Delete my account" button - a hunt that ended in failure. It's not possible to delete a posterous account.

As it happens, you can do some clicking around and delete almost everything until you're left with one empty blog. But why can't an account be deleted? While we might not have any blog posts in the system any more, Posterous still holds my mother's e-mail address and other details, and in theory could decide to contact her in future - something that she clearly won't desire [1].

To my mind, this is the key point: Deleting an account is actually sending a message to the site saying I do not give you permission to use my data any more. Please do not display my data on your site, tell people that I am a user of your site, or contact me in any way about your site.

The Site Perspective

As a software engineer, I know it's possible to implement user account deletion. But I also know it's a lot harder than simply going through the site with a chainsaw, deleting anything remotely connected with the user. Here are a few considerations:

It's completely impractical to suggest that user data be deleted from site backups.
Any data the user added to the system which is displayed in the context of other users' data is nearly impossible to delete sensibly. For example, forum posts in a discussion forum - if we were to simply delete all posts by the user, we'd completely trash the meaning of the threads they were taking part in.
Some users delete their accounts accidentally, then frantically contact support asking to be undeleted!

Here are some things a site can do when a user account is deleted without too much bother:

Stop displaying any of the user's data that exists on its own - for example, a profile page.
Prevent anyone from logging in to the account
Don't count the user in any user lists or reports about active/existing users

Most sites realise soon enough that it doesn't matter how cool they are - people are going to want to delete their accounts. This is an operation that has no inherent financial value for them, therefore they want to minimise the time spent dealing with it. This includes support time - a constant stream of people requesting account deletion is a support headache and a source of friction a growing site doesn't need.

Website needs, User needs

Having seen both sides, we can now make a list of the things users and sites want regarding user account deletion:

Users do not want to be associated with the site any more.
However, users do sometimes want to be able to undelete their account.
Users would like as much of their data deleted as possible.
Sites don't want to waste time trying to implement impossible and undesired things like removing user forum posts.
Sites don't want to deal with a constant stream of requests for accounts to be deleted

I believe all of these objectives can be achieved with a little bit of thought on the part of the site.

A Working Implementation

Let's call this the "Disassociation Method" of account deletion:

The site should provied an account deletion button. It doesn't have to be prominent, or referenced in any documentation if that is not desired (and let's be honest, what site really wants to highlight this?) - but there should be such a button available to all users.

If the user clicks this button and confirms their action, then from their point of view their account should appear to be deleted. In the Posterous case, visiting my Mother's blog should result in a 404. If they were sending her any e-mails, those e-mails should cease immediately.

An important note here is that the site probably shouldn't tell the user that their account can be undeleted at any time, even though this is possible. The site is looking to reassure the user that deleting means the user won't be associated with the site.

After some time, if the user decides that they want to un-delete their account, they can contact support and ask for this to be done. It's at the site's discretion whether to undelete the account or not, but at least the site is able to make that choice.

Ethics

"Whoa!", you might be saying, "You just redefined delete!".

Ethically, I agree that you have a point. From a practical perspective, you could try using words like "disassociate", and try explain to users that you're keeping their data to save them from themselves, but I'm not sure if that's going to help do anything other than soothe your consience. Perhaps you might be better off with explaining the details of how account deletion works in your terms and conditions.

Practicalities

"We've tried implementing account undeletion before, and it's too hard!".

Rubbish. Go read this article about how you messed up your schema, then go do it properly.

Summing Up

Account deletion is currently done poorly by most sites. With a little thought however, sites can largely neutralise any problems it causes without spending much of their precious time.

[1]

Yes yes I know that we could fudge her details, turn off notifications, maybe change the e-mail address if that's allowed - although it seems posterous requires you always have a valid e-mail account on file with them - but this is all beside the point. Posterous still thinks my mother is a user. Deleting an account shouldn't require playing games with the system.

Skype 2.1 on 64 bit Debian

Mon, 08 Feb 2010 09:21:02 GMT+12

As of 2010/02/08, it is possible to get Skype 2.1 working on 64 bit Debian. You just need a bit of hax.

All instructions should be carried out as root.

1. Install Required Packages

Install the debian packages required for Skype. You need some qt4 libs and some 32bit compatibility libraries:

apt-get install ia32-libs ia32-libs-gtk libqt4-core libqt4-gui

2. Get Skype

Download and install the Debian version of the Skype deb. You need to force architecture when installing:

dpkg -i --force-architecture /path/to/downloaded/skype.deb

3. Install extra i386 libs for Skype 2.1

It turns out you need some more i386 libraries installed before Skype 2.1 will work.

First, download them:

To install them:

# Uncompress them in a new directory
mkdir -p /emul/ia32-linux
dpkg -X /path/to/downloaded/libwrap0.deb /emul/ia32-linux
dpkg -X /path/to/downloaded/libgdbm3.deb /emul/ia32-linux

# Add some paths to your lib search path:
echo '/emul/ia32-linux/lib' > /etc/ld.so.conf.d/ia32.conf
echo '/emul/ia32-linux/usr/lib' >> /etc/ld.so.conf.d/ia32.conf

# Re-run ldconfig
ldconfig

Then skype should be good to go; just run 'skype' from the command line, or however you normally run it.

The acid test is whether you can log in - Skype 2.1 will run without those extra libraries being present, but it will crash while logging you in if they're not there.

Sources:

http://wiki.debian.org/skype - Doesn't have instructions for Skype 2.1 (yet?). Does have instructions for video. Suggests you should run skype as linux32 skype, it seems to run fine just as skype for me though.
https://developer.skype.com/jira/browse/SCL-510 - near the bottom is a list of similar instructions, my instructions don't involve copying files into directories that should only be populated by installing packages.
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=543448 - debian bug about including the extra libs in ia32-libs.

One Year

Sun, 07 Feb 2010 00:44:36 GMT+12

If you could swap a year of your life to work at 2x speed for a year, would you? What does it say about your choice of work if you wouldn't?

For most people, the answer is "no". There's either no benefit to them working harder, or it wouldn't be fun.

If your answer is "yes", that's probably because you see real benefits to working harder - which might mean you have goals that are important to you, and a path to reach them that is in your control. If so, congratulations!

I reckon a good step towards living the life you want would be to find something to work on where you would answer "yes" to this question.

Handling All Filenames In Bash

Tue, 29 Dec 2009 19:21:30 GMT+12

If you want to process a list of filenames in a loop in bash (let's say you're using file to generate the list)

... AND you want it to work even when the files have spaces in their names

... AND you want it to work even when the files have backslashes in their names

... then don't do what anyone else on the internet suggests. Instead, do this:

find . -type f | while read -r F ; do
    echo "$F"
done

And everywhere you use the filename ($F above), enclose it in double quotes.

for F in $(find . -type f) inevitably ends up failing when the filenames have spaces in them.

Smarter people tend to suggest the | while read F solution, but they forget the -r which turn off backslashes counting as escape characters.

Smarter people still just use perl [1] [2].

[1]	How did I find all this out, you may ask. Ahem, well one day I thought I'd write a waffer thin script to... er, look, an eagle!

[2]

Also, in case it isn't obvious, this is another example of bash's total failure to be useful for anything at all. You can't even trust it to handle filenames correctly without a bunch of arcane syntax and piping - and I just bet some GNU/Beard is dying to e-mail me right now to say how this solution doesn't work when the file has the unicode bunny-rabbit-doing-a-poopy symbol in it...

Our New Startup: Get Your Game On!

Wed, 23 Dec 2009 15:48:32 GMT+12

We have launched a splash page for our new venture, Get Your Game On. If you have ever had to manage a competition, or know someone who does, you (or they) will be interested in what we're doing. Please spread the word!

There's nothing to try out yet, but we're working on that right now. You can follow @getyourgameonnz to stay updated.

ps: Yeah it's a .co.nz, but we're not restricting ourselves to just New Zealanders of course!

A Five Minute Guide to Linux Containers for Debian

Fri, 11 Dec 2009 23:12:21 GMT+12

Linux Containers - otherwise known as LXC - are a virtualisation system that's in the Linux kernel right now and ready to use. You can use them to set up vserver/openvz style guests on a host - e.g. separate processes, file systems, but same kernel.

Word on the street is that vservers are being deprecated in Debian beyond squeeze, so it seems like a good time to learn about containers. Here is my five minute guide to getting them up and running on your debian unstable (squeeze) box.

Setup

apt-get install lxc

You want kernel 2.6.29 or later, 2.6.31 is currently in unstable.

Once lxc is installed, lxc-checkconfig will check if your kernel has everything enabled that is needed. It reports most things except for the memory controller for me. As I haven't gotten around to playing with resource limiting in the containers, this doesn't bother me.

You need to mount a cgroup filesystem somewhere to control things:

sudo mkdir /var/local/cgroup
sudo vim /etc/fstab
# add this line:
# cgroup  /var/local/cgroup  cgroup  defaults  0  0
sudo mount cgroup

Networking Prep

You have to set up a bridge so the containers will be able to talk to the outside world (and the outside world can talk back) [1].

This isn't hard, it just involves editing /etc/network/interfaces. Just make sure you stop networking before making the changes:

/etc/init.d/networking stop
vim /etc/network/interfaces

# find the section talking about your physical interface, it's normally
# eth0 or eth1
auto eth0
iface eth0 inet manual # change from 'dhcp' to 'manual'

# add this section
auto br0
iface br0 inet dhcp
    bridge_ports eth0
    bridge_stp off
    bridge_fd 0
    bridge_maxwait 0

Then /etc/init.d/networking start. Try pinging something, it should work just fine. If not, revert your changes and ask a network ninja to help you.

Creating Containers

The lxc-debian script in /usr/share/doc/lxc/examples is useful. I have made some modifications to it, to make it a bit more friendly. You can get it from my git repository:

git clone git://git.nigel.mcnie.name/lxc-debian.git

If you don't have git, just download it from here.

Put the lxc-debian script in /usr/local/bin, and put the init script at /etc/init.d/lxc. Then you're ready to create your first container:

lxc-debian create

You'll have to pick a name and hostname, then some networking related things. You can use route -n to work out your gateway if you don't know it already, and just pick a free IP on your subnet if you don't know anything else to choose.

Once you're done, it will download a bunch of packages with debootstrap, and set up your container. The next time you create a container, it will use the packages from the first time, so it'll only be slow once.

Starting and Accessing your Container

You start it with lxc-start -n [name]. This will dump you at the login prompt - e.g., this is like starting a real server, it's in the foreground. Login is 'root', no password.

You probably actually wanted to daemonize it, so halt it and then type lxc-stop -n [name] in another terminal to shut it down [2].

To start it in the background, do lxc-start -n [name] -d. This will exit immediately and the container will start. You can use lxc-info -n [name] to find out whether it's started up yet, though they tend to start pretty quickly.

Once you're done, you can connect to a tty in the container with lxc-console -n [name] -t 1. You can replace 1 with any tty number (1-6 are started for you). Control-A, q exits.

Or, the lxc-debian script installs and configures an ssh server for you, so you can just ssh root@[ipaddress] to get into it as root. I tend to use this method all the time.

I want to remove my container

lxc-debian destroy and answer the questions. This basically does:

lxc-destroy -n [name]
rm -rf /path/to/rootfs

What Next?

Now that you can create, start, connect to and destroy containers, there's not much else you need to know to use containers for development environments or other such uses. Later, I might write an article about some more advanced topics, such as resource limiting.

I'm interested in improving lxc-debian - if you are as well then get in touch, I'd love to hear from you.

Sources

http://lxc.teegra.net/ - has notes, targetted at arch linux. lxc-debian and the modern lxc-create seem to handle most of the container setup that article suggests.

man lxc - A worthwhile read, in slightly idomatic english but understandable enough.

http://openvz.org/pipermail/devel/2008-September/014314.html - lxc with libvirt

http://jim.studt.net/depository/index.php/using-linux-containers-with-debian-lenny - similar to the experiments I've had

http://jim.studt.net/depository/index.php/letting-your-lxc-containers-use-consoles

http://www.stgraber.org/2009/11/06/lxc-containers-or-extremely-fast-virtualization

[1]	Actually, this is debatable. Martyn seems to be getting along fine with configuring his containers to use the physical interface directly; I had less luck. The bridge Works For Me, and conveniently is what you need for heavier virtualisation like kvm, so I'm sticking with it.

[2]	I've found that starting containers in the foreground and then lxc-stopping them can cause the terminal they were in to start behaving strangely. No idea why this is, I tend to always start my containers in the background so it's not an issue for me.

THIS IS YOUR INTREPID REPORTER SIGNING OFF

Wed, 25 Nov 2009 10:48:31 GMT+12

THE NOTORIOUS LONG TIME SPAMMER, TROLL AND CURRY OBSESSED PSYCHOPATH, KNOWN ONLY BY THE 1337 HACKER HANDLE OF "YOUR INTREPID REPORTER", HAS BEEN CAUGHT AND SENTENCED TO THREE HUNDRED YEARS IN THE CURRYMINES OF RAJ "CAN THE NEXT CURRYMAIL WRITER PLEASE JUST TALK ABOUT MY BLOODY CURRIES!" PATEL, EVIL CURRY DICTATOR, FOR CRIMES INCLUDING SPAMMING, TROLLING AND BEING A CURRY OBSESSED PSYCHOPATH.