Politics and Technology.

Saturday, September 27, 2008

Active Directory Authentication and RHEL, pt1

Ok, so it, again, has been a couple of months since my last technical post. The whole RHEL cluster thing has been tossed to the side for now so there was nothing to report.

In searching for the command to extend the Satellite Server's database, I came across my own blog entry for just that. I realized that I've neglected my own blog so I figured I should add something.

One project I've been working on over the past month or so has to do with overcoming the expected abysmal connectivity from Red China during the 2008 Olympics. The application folks were turned on to Aspera from our colleagues at Fox News. This software is, essentially, a fork of OpenSSH. It is designed to send file transfers through a UDP based data channel (they call it "fasp"). This has the effect of beating TCP based transfers over high-latency connections. Basically, Aspera transfers beats FTP hands down. It beats it so well, transfers can move at wire speed and overwhelm all other traffic. Fortunately, Aspera has built in throttling mechanisms into fasp that help guard against killing networks.

So, what does this have to do with Active Directory authentication on RHEL? Plenty for us. Since we'd be placing this thing on the bad-bad internet expecting reporters from Beijing to upload hours worth of Star Spangled Banner from any ol' internet cafe, authentication is a concern. The speed of this project forced us to put it in place before getting any SA or Help Desk resources on board with support. I don't like the idea of an SA or engineer getting called at 3am because some luddite in Beijing can't figure out their password. Linking in logins to the Aspera server to AD was a critical component.

Since Aspera runs on Solaris, RHEL and Windows, we expected to stick it on Windows and call it a day. Alas, there was one problem: lockout DoS attacks. Windows natively doesn't protect against this. RHEL does with pam_tally. RHEL also can handle LDAP authentication with an AD server. In trying to stick with a non-complicated roll-out in order to make ongoing SA support easy, RHEL became the OS of choice here.

So, there we were with the following solution: Aspera and RHEL with pam_tally and Active Directory authentication. Unfortunately, a lot became trial and error since there seems to be a dearth of real world examples on how to get RHEL to talk to an AD server. It turns out to be very straightforward and simple.

The weird thing about Aspera's software, however, is that they spent a lot of effort making FASP and then slapped on a generic web interface. They expect you to hire consolutants to make the Web interface professional looking. We tried in vain to get Apache to authenticate via LDAP without opening ourselves up to the lockout DoS. We even tried to compile in "mod_auth_pam" to take advantage of pam_tally, but it seems to refuse to use LDAP over SSL through pam, even though LDAP in the clear worked fine. I believe it has something to do with getting non-privileged UID access to the OpenSSL libraries or maybe the originating LDAPS port.

In the intrest of getting up and running in time for capturing team USA kick ass on video, we gave up trying to figure out the issue and decided to stick with a regular "htpasswd" authentication scheme. In the end, I left the mod_auth_pam in just to use pam_tally.

What follows is the fruit of our labors in this endeavor.

Install RHEL 5

Install Apache

Download the Aspera Enterprise Server for RHEL from the Aspera web site.


First, install the RPM as you would any other RPM. The bulk of the software installs in /opt, though binaries drop into the usual places under /usr. Next, configure the license with the provided license key by placing the keys into “/opt/aspera/etc/fasplicense.txt”.

Configure the Aspera Enterprise Server Software

Make a unix account with the same name (say, “ASPERA”). It is ok to lock the account from logins and even create a shell of “/bin/false”. Set the home directory to be the directory where all file transfers will work out of (“/home/ASPERA” is sufficient). We will use a UID of 7000 in our setup.

Now, add user ASPERA's UID and home directory to “/opt/aspera/etc/docroot” on one line, separated by a “:”.

# cat > docroot
7000:/home/ASPERA ^D
# cat docroot

Set the world writable and executable permissions for user videotransfer’s home directory so that everyone can access its contents.

# chmod 777 /home/ASPERA

Finally, you will need to hack one of Aspera’s perl scripts to allow everyone to share the same directory for file transfers. Near line 826 of “/opt/aspera/var/webtools/scripts/aspera-dirlist.pl”, change the command

$sought = $id.':'; To read $sought = '7000:';

Whereas “7000” is the UID of the unix account “ASPERA”.

Edit “/opt/aspera/etc/aspera.conf” to reflect these settings for bandwidth caps and target rate flows.

Under "Central":


Under "Web":

Be sure to restart the aspera service.

# /usr/sbin/service asperacentral restart

Configure Apache

Within “/opt/aspera/etc”, create group login and password in the” htpasswd” format within a file called “webpasswd”. We choose to use “ASPERA” as the username.

# htpasswd -b webpasswd ASPERA SOMEPASSWORD

Add thes “ASPERA” stanza to the bottom of “/etc/httpd/conf/httpd.conf”. We want to be able to have everyone reach the Aspera directory without using "http://asperaserver.corp.com/aspera/user" all the time. Keeping root pointed at "aspera/user" is a good idea for this.

(httpd.conf entry to follow on another post)

Set DocumentRoot to be the Aspera directory.

DocumentRoot "/opt/aspera/var/webtools"

Copy “.htaccess” from “/opt/aspera/var/webtools/user” to “/opt/aspera/var/webtools”.

Configure LDAP

Create a new “/etc/ldap.conf” with the following entries. LDAP will be used to talk to the Active Directory servers, over SSL, to provide password authentication. You need to have a read only AD account created for the initial binding. This user account is used to lookup accounts to provide the password hash.

host WINDOWSDC1.domain.corp.com WINDOWSDC2.domain.corp.com

base dc=domain,dc=corp,dc=com
binddn CN=READONLYUSER,OU=Users,OU=USA,DC=domain,DC=corpm,DC=com


timelimit 120

bind_timelimit 120

idle_timelimit 3600

pam_login_attribute samaccountname

ssl on

tls_cacertdir /etc/pki/tls/certs

debug 5

Download your Certificate Authority certificates. Be sure that they are in PEM format. Ours come as PC friendly pkcs7.

# openssl pkcs7 -inform DER -in CA.p7b -print_certs -text -out CA.pem

Our CA certs have mulitple entries. The certificates from our pem file had to be separated out into different files. Fortunately someone addressed this issue. This PERL script can do the work for you.

# Splits a certficate file with multiple entries up into
# one certificate perl file
# Artistic License
# v0.0.1 Nick Burch
my $filename = shift;
unless($filename) die("Usage:\n cert-split.pl <certificate-file>\n");
open INP, "<$filename" or die("Unable to load \"$filename\"\n"); my $ifile = ""; my $thisfile = ""; while(<inp>) {

$ifile .= $_;
$thisfile .= $_;
if($_ =~ /^\-+END(\s\w+)?\sCERTIFICATE\-+$/) {

print "Found a complete certificate:\n";
print `echo "$thisfile" | openssl x509 -noout -issuer -subject`;
print "\n";
print "What file should this be saved to?\n";

my $fname = <>;

open CERT, ">$fname";
print CERT $thisfile;

close CERT;

$thisfile = "";

print "Certificate saved\n\n";

close INP;
print "Completed\n";

Next, move these new PEM files to “/etc/pki/tls/certs” and softlink their x509 hash values to their names. Hashes are calculated with “/usr/bin/openssl x509 –noout –hash –in FILENAME”. OpenSSL uses the x509 hash of a signature to match against a filename of the same name in order to know which certificate to use to verify authenticity. Observe the soft linking example below.

# cp *.pem /etc/pki/tls/certs; cd /etc/pki/tls/certs
# ls -alF

total 376

drwxr-xr-x 2 root root 4096 Jul 2 11:59 ./

drwxr-xr-x 7 root root 4096 Jul 2 10:22 ../

-rw-r--r-- 1 root root 249373 Jan 12 2007 ca-bundle.crt

-rw-r--r-- 1 root root 5054 Jul 2 10:31 ca1.pem

-rw-r--r-- 1 root root 4066 Jul 2 10:31 ca2.pem

-rw-r--r-- 1 root root 5060 Jul 2 10:31 ca3.pem

-rw-r--r-- 1 root root 5055 Jul 2 10:31 ca4.pem

-rw-r--r-- 1 root root 5054 Jul 2 10:31 ca5.pem

-rw-r--r-- 1 root root 610 Jan 12 2007 make-dummy-cert

-rw-r--r-- 1 root root 1832 Jan 12 2007 Makefile

# for i in `ls *pem`; do ln -s $i `openssl x509 -noout -hash -in $i`.0; done

# ls -alF

total 396

drwxr-xr-x 2 root root 4096 Jul 2 12:06 ./

drwxr-xr-x 7 root root 4096 Jul 2 10:22 ../

lrwxrwxrwx 1 root root 9 Jul 2 12:06 184d21f0.0 -> ca1.pem
lrwxrwxrwx 1 root root 9 Jul 2 12:06 3cea7904.0 -> ca4.pem lrwxrwxrwx 1 root root 9 Jul 2 12:06 64cc7d0a.0 -> ca2.pem lrwxrwxrwx 1 root root 9 Jul 2 12:06 a4a62ce2.0 -> ca5.pem -rw-r--r-- 1 root root 249373 Jan 12 2007 ca-bundle.crt
lrwxrwxrwx 1 root root 9 Jul 2 12:06 d32b38ca.0 -> ca3.pem
-rw-r--r-- 1 root root 5054 Jul 2 10:31 ca1.pem
-rw-r--r-- 1 root root 4066 Jul 2 10:31 ca2.pem

-rw-r--r-- 1 root root 5060 Jul 2 10:31 ca3.pem

-rw-r--r-- 1 root root 5055 Jul 2 10:31 ca4.pem

-rw-r--r-- 1 root root 5054 Jul 2 10:31 ca5.pem

-rw-r--r-- 1 root root 610 Jan 12 2007 make-dummy-cert

-rw-r--r-- 1 root root 1832 Jan 12 2007 Makefile


Configure PAM

In “/etc/pam.d/sshd”, configure the pam stack to include pam_tally and pam_ldap. The pam_tally module will prevent failed password attempts on this server from triggering an account lockout from the Active Directory. If a user enters in a wrong password five times (“deny=5” in the option list), they will be locked out from this server only for 61 minutes (one minute longer than Active Directory’s automatic reset). After 61 minutes, pam_tally will automatically reset and allow them to try again. The pam_ldap module will allow users to authenticate their passwords from Active Directory rather than from “/etc/shadow”. Password management for those accounts should then come from the Dow Jones Help Desk, as with all other Active Directory passwords. The order in which these modules are referenced within “/etc/pam.d/sshd” is important. The example below allows a local user account to authenticate against “/etc/shadow” after trying against the Active Directory server. This allows for normal Unix logins.

# cat /etc/pam.d/sshd
auth requisite pam_tally.so onerr=fail deny=5\ unlock_time=3660

auth sufficient pam_ldap.so

auth include system-auth

account required pam_nologin.so

account requisite pam_tally.so

account sufficient pam_ldap.so

account include system-auth

password include system-auth

session optional pam_keyinit.so force revoke

session include system-auth
session required pam_loginuid.so

Create User Accounts
Create user accounts for every person expected to use this software. The user account names must match their Windows Acitve Directory name and their shell must be “/bin/aspshell”. Their unix passwords in “/etc/shadow” can remain locked for added security.

No comments: