30 Nov 2013
In the massive amount of Linux systems I've seen over the years not a single one of them had a great sudoers file. Some were doing command line restrictions for one or two accounts, others were doing nothing more than allowing people to sudo straight to root and some had LDAP integration that gave certain groups permissions. But with each one of these the groups/users that had sudo was a very generic group of administrators that may or may not know Linux very well. That was the choice of the managers and directors, which I understand is a very low level item to put time into. However, their time would be well worth the effort of securing authorization from a security, compliance and keep your job type of situation.
I've known how to use sudo and sudoers file for a long time, but I decided to do a ton of research on sudo and how the sudoers file really works specifically the commands assigned to users and how to restrict users to those commands. Then I also did research on how to become root without the privileges, from vim breakouts to creating substitute files that give you root. With this I was able to create a sudoers file that I believe is very secure, but flexible enough to give the necessary access to users of the system.
These types of systems were usually multiple users accessing root privileges, not a VPS or single user access. And it would be multiple team roles like; developers, administrators, release engineers and database administrators.
Below is a simple example of a sudoers file, without any extra privileges, but with restricted shells and a couple other things.
Cmnd_Alias SHELLS_RESTRICTED=/bin/sh,/sbin/sh,/sbin/jsh,/usr/bin/sh,/bin/csh,/usr/bin/csh,/bin/ksh, \
/usr/local/share/bin/tcsh,/usr/local/share/bin/bash,/usr/local/bin/tcsh,/usr/local/bin/bash,/usr/bin/rsh, \
/usr/local/bin/zsh,/usr/bin/ksh
Cmnd_Alias EDITORS_RESTRICTED=/bin/more,/bin/less,/usr/bin/less,/usr/bin/vim,/usr/bin/vi,/bin/vi
Cmnd_Alias PROGRAMMING_RESTRICTED=/usr/bin/perl,/usr/bin/python,/usr/bin/ruby,/usr/bin/irb
Defaults requiretty
Defaults always_set_home
Defaults env_reset
Defaults env_keep = "COLORS DISPLAY HOSTNAME HISTSIZE INPUTRC KDEDIR LS_COLORS"
Defaults env_keep += "MAIL PS1 PS2 QTDIR USERNAME LANG LC_ADDRESS LC_CTYPE"
Defaults env_keep += "LC_COLLATE LC_IDENTIFICATION LC_MEASUREMENT LC_MESSAGES"
Defaults env_keep += "LC_MONETARY LC_NAME LC_NUMERIC LC_PAPER LC_TELEPHONE"
Defaults env_keep += "LC_TIME LC_ALL LANGUAGE LINGUAS _XKB_CHARSET XAUTHORITY"
Defaults secure_path = /sbin:/bin:/usr/sbin:/usr/bin
Defaults!SHELLS_RESTRICTED noexec
Defaults!EDITORS_RESTRICTED noexec
Defaults!PROGRAMMING_RESTRICTED noexec
Defaults loglinelen = 0, logfile =/var/log/sudo.log, log_year, log_host, syslog=auth
root ALL = (ALL) ALL
To explain the above sudoers file...
- You'll see a couple Cmnd_Alias directives, these list out files that could be compromised to allow a person to become root if they were run with the sudo command.
- Then we add environmental variables that we allow in when someone does a sudo.
- I kept requiretty because I don't want any other session, but a shell login session to be allowed to sudo. So if someone creates a cronjob that runs sudo or cgi-bin script that runs sudo they will be blocked. I've also added a sudo log, to see what is running sudo commands.
- I've added a secure_path variable to make sure this is the path used during sudo commands.
It's not a matter of trust, it's a matter of doing things correctly and the same. Too many times I've seen it where people expect to have full sudo rights because they don't know how to do something without sudo. Ask a systems administrator to do things with sudo in front of it without being root, it's a very different shell world when they start to do that. (or at least try)
And when they start to put sudo in front of their commands they begin to respect the abilities of sudo, root and authorization to the Linux system. And they can also design systems or commands a bit differently, which is a good thing in my opinion.
**NOTE it does matter where you put things in a sudoers file so beware of that.
Okay now that we'e gotten a basic sudoers file out of the way, let's create a sudoers file that makes it so a systems administrator can run sudo as themselves and we can capture those commands for audits and such.
Cmnd_Alias HALT=/usr/sbin/halt,/usr/sbin/fasthalt,/sbin/shutdown -h,/sbin/shutdown -H, \
/sbin/shutdown -P,/sbin/reboot -p
Cmnd_Alias REBOOT=/sbin/shutdown -r,/sbin/reboot -f
Cmnd_Alias SERVICE=/sbin/chkconfig,/sbin/service
Cmnd_Alias FILE=/bin/chmod,/bin/chown,/bin/ln,/bin/mkdir,/usr/bin/chattr,/usr/bin/lsattr, \
/bin/touch,/bin/rm,/bin/rmdir,/bin/cat,/bin/ls,/usr/bin/diff,/usr/bin/locate,/sbin/mkfs,/sbin/fsck, \
/sbin/lvm,/sbin/lvmconf,/sbin/lvmdump,/bin/mount,/bin/umount,/usr/bin/hexdump,/usr/bin/head, \
/usr/bin/tail,/bin/grep,/bin/echo,/usr/bin/md5sum,/usr/bin/test
Cmnd_Alias PROCESS=/bin/kill,/usr/bin/killall,/bin/ps
Cmnd_Alias STATISTICS=/usr/sbin/tcpdump,/usr/sbin/lnstat,/usr/bin/iostat,/usr/bin/free, \
/usr/bin/sar,/bin/netstat,/usr/bin/vmstat,/usr/bin/du
Cmnd_Alias NETWORK=/usr/bin/nslookup,/bin/traceroute
Cmnd_Alias SYSTEM=/usr/bin/strace,/bin/env,/usr/bin/nohup,/usr/sbin/lsof
Cmnd_Alias PACKAGE_MANAGEMENT=/usr/bin/yum,/bin/rpm,/usr/bin/gem
Cmnd_Alias SUDO_EDIT=sudoedit /etc/fstab #add more on here as necessary
Cmnd_Alias SYSTEM_ADMINISTRATORS=SERVICE,FILE,PROCESS,STATISTICS,NETWORK, \
SYSTEM,PACKAGE_MANAGEMENT,SUDO_EDIT,HALT,REBOOT
%system_administrators ALL = (ALL) NOPASSWD: SYSTEM_ADMINISTRATORS
Now the group system_administrators have access to what they need access to. They cannot become root, but they can do their specific tasks they need on the system. Granted the sudoers file tells the systems administrators what they can do, but isn't that what setting up and knowing what a system does is all about anyways? I never liked the argument of we want no restrictions because we are the systems administrators. What I say to that is if you do not fully understand your application then you haven't integrated with the developers or database teams well enough to do a throughout job of restricting only what is needed. You haven't done your systems administrator's job.
And what is the benefit of doing these restrictions? Security, compliance, full working knowledge of what the application does and needs and proper authorization without blocking anyone from doing what they need to do. Now this makes it so your application's base OS configuration is customized to allow people to do what they need for the application. So I do acknowledge that it requires extra work, but we do the same thing when we do nice, ulimit or other types of restrictions.
Oh! and a couple other thing to say is...
These sudoers examples are not for every type of system, just ones you have fully baked in a particular environment. If you fully understand the application and want to secure it in production, then use sudoers to lock it down and restrict it to what is needed.
Make sure your delete your root user's password after you have figured out your sudo situation, you don't need people trying to brute force the root user. And delete any other user's passwords, stop using passwords and start using sudo and ssh public keys.
Please don't ssh in as root or login as the root user.
And remember many systems these days are throw away because they are virtual machines that can be duplicated.
---
I plan to write more about sudo. Just need to get some thoughts put together and I'll have something out soon.
30 Nov 2013
Amazon Linux which runs only on EC2 instances implements an auto upgrade function when it first boots. When you have a much older AMI like 2011.09 or 2012.03 it tries to do a yum update with a newer repository like the recently released 2013.09. As you can tell the older the AMI the longer the update will take. When I run Chef and other cloud-init type functions I run into a yum lock issue. What you want to do to "fix" this problem is to add user-data when the EC2 instance starts up. Below is the necessary cloud-init.
#cloud-config
repo_releasever: 2013.03
repo_upgrade: none
- #cloud-config tells us to use this data for yum config.
- repo_releasever tells the AMI to keep yum at 2013.03 instead of the latest which is default
- repo_upgrade tells the AMI to not do security updates on boot
29 Nov 2013
There are not many examples for the ruby gem fog, what I use is the rdoc pages for the information about the fog API. Usually that works for me, but sometimes there is trial and error, which takes time. I've been making these short posts about different things you can do with fog so that it helps someone else down the line.
For this post you'll learn how to make a EC2 security group and then authorize Amazon ELBs and an IP address range.
require 'fog'
connection = Fog::Compute.new(:provider => 'AWS', :aws_access_key_id => access_key, :aws_secret_access_key => secret_key, :region => "us-east-1")
sg = connection.security_groups.new(:name => "myservers_security_group", :description => "For my web servers running Linux")
sg.save
#Let's do a get, just to make sure it's there
sg = connection.security_groups.get("myservers_security_group")
#Now let's authorize a CIDR
result = sg.authorize_port_range(22..22, {:cidr_ip => "192.168.1.100/32"})
if result.status != 200
puts "Failed to authorize cidr"
end
#Now let's authorize another EC2 security group
result = sg.authorize_port_range(80..80, {:group => { 'amazon-elb' => 'amazon-elb-sg'}, :ip_protocol => 'tcp'})
if result .status != 200
puts "Failed to authorized group"
end
If you look at the authorize EC2 group you'll see "amazon-elb", which is the accountID and "amazon-elb-sg" is the security group name. So for your account, you would put the actual accountID (which should be a bunch of numbers)
29 Nov 2013
Building a cloudwatch alarm using the ruby gem fog took me a while to figure out. It wasn't as clear cut as other fog methods. Below you'll see how to create the metric and then build the alarm for that metric.
require 'fog'
connection = Fog::AWS::CloudWatch.new(:aws_access_key_id => access_key, :aws_secret_access_key => secret_key, :region => "us-east-1")
result = connection.put_metric_data("AWS/EC2", ["MetricName"=>"my_custom_metric", "Value"=>1.0, "Unit"=>"Count"])
if result.status != 200
puts "Failed creating metric"
end
#You'll need to wait a very long time if this is a new metric
waiting = nil
while waiting.nil?
sleep 10
connection.metrics #this reload's the metrics available from the API
waiting = connection.metrics.get("AWS/EC2","my_custom_metric")
end
#Now let's create the alarm
alarm_config = {}
alarm_config.store("AlarmName","custom_metric_alarm")
alarm_config.store("AlarmActions",[])
alarm_config.store("MetricName","my_custom_metric")
alarm_config.store("Namespace","AWS/EC2")
alarm_config.store("AlarmDescription","Notifies me when custom metric is too high")
alarm_config.store("Statistic","Average")
alarm_config.store("Unit",nil)
alarm_config.store("ComparisonOperator","GreaterThanOrEqualToThreshold") #check Amazon docs for others
alarm_config.store("Period",300) #seconds
alarm_config.store("EvaluationPeriods",1)
alarm_config.store("Threshold",10.0)
alarm_config.store("Dimensions",[])
result = connection.put_metric_alarm(alarm_config)
if result.status != 200
puts "Alarm creation failed"
end
#You can get the alarm like this
my_alarm = cs.alarms.get("custom_metric_alarm")
You'll notice "AWS/EC2" in many of the above methods, this is the namespace Amazon needs to have to place it in the proper location. There are many namespaces and you can also create custom namespaces. Like "ApplicationName:server_stats"
Also, I didn't put an example above, but when creating an Alarm you can specify an Action for it to take. "AlarmActions" is an array of strings that point to an ARN for an autoscaling group policy or email.
27 Nov 2013
I researched pam and ldap authentication setups for a long time and have found how to do it properly. When I say properly I mean your Linux user account get's the same uid and gid for every server connected to the same Active Directory infrastructure.
The documents/discussions don't just come out and say it. I tested this for a while and below is how you can setup PAM with LDAP and keep the same uid and gid on each Linux server.
First let's install the necessary packages:
apt-get install winbind libpam-krb5 smbclient krb5-user ntpdate ntp nscd
Let's setup samba smb.conf for (/etc/samba/smb.conf):
[global]
server string = %h server (Samba, Ubuntu)
dns proxy = no
log file = /var/log/samba/log.%m
max log size = 1000
log level = 1
syslog only = yes
syslog = 0
panic action = /usr/share/samba/panic-action %d
security = ads
realm = myADdomain.com
workgroup = myADdomain
restrict anonymous = 2
encrypt passwords = true
obey pam restrictions = yes
unix password sync = yes
passwd program = /usr/bin/passwd %u
passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* .
pam password change = yes
map to guest = bad user
#This is where the uid/gid matching with ldap id comes in
idmap uid = 10000-100000000
idmap gid = 10000-100000000
idmap config * : backend = rid
idmap config * : range = 10000-100000000
template shell = /bin/bash
template homedir = /home/%U
client use spnego = yes
client ntlmv2 auth = yes
winbind enum groups = yes
winbind enum users = yes
winbind use default domain = yes
winbind cache time = 10
winbind nested groups = yes
usershare allow guests = yes
load printers = no
Next setup the kerberos configuration (/etc/krb5.conf):
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
[libdefaults]
default_realm = MYADDOMAIN.COM
krb4_config = /etc/krb.conf
krb4_realms = /etc/krb.realms
kdc_timesync = 1
ccache_type = 4
forwardable = true
proxiable = true
dns_lookup_realm = true
dns_lookup_kdc = true
v4_instance_resolve = false
v4_name_convert = {
host = {
rcmd = host
ftp = ftp
}
plain = {
something = something-else
}
}
fcc-mit-ticketflags = true
[realms]
MYADDOMAIN.COM = {
kdc = myADdomain.com
admin_server = myADdomain.com
default_domain = myADdomain.com
}
[domain_realm]
myADdomain.com = MYADDOMAIN.COM
[login]
krb4_convert = true
krb4_get_tickets = false
[appdefaults]
pam = {
debug = false
ticket_lifetime = 36000
renew_lifetime = 36000
forwardable = true
krb4_convert = false
}
Now that we've configured samba and kerberos we can now add the server to the domain (you'll need priviledges into the Computers OU:
net rpc join -U mydomainadmin_username
ldconfig
From there we want the Linux server to recognize kerberos and samba authentications by updating a couple files, see below:
/etc/nsswitch.conf
#Change the ones you see in nsswitch to these below (keep in this order)
passwd: compat winbind
group: compat winbind
shadow: compat winbind
hosts: files dns wins
/etc/pam.d/common-session
#Add these to the top of the file
session required pam_mkhomedir.so umask=0022 skel=/etc/skel
session sufficient pam_winbind.so
session required pam_unix.so
/etc/pam.d/common-account
#Add these to the top of the file
account sufficient pam_winbind.so
account required pam_unix.so
/etc/pam.d/common-auth
#Add these to the top of the file
auth sufficient pam_winbind.so krb5_auth krb5_ccache_type=FILE
auth sufficient pam_unix.so nullok_secure use_first_pass
auth required pam_deny.so
#And comment out this line:
auth [success=1 default=ignore] pam_winbind.so krb5_auth krb5_ccache_type=FILE cached_login try_first_pass
#And add this line right below it:
auth [success=1 default=ignore] pam_winbind.so require_membership_of=myUnixUsersGroup krb5_auth krb5_ccache_type=FILE cached_login try_first_pass
#Restart winbind
That's it, you've now setup your Linux box to accept requests from Active Directory users.
Also did you see in the above PAM config common-auth the "myUnixUsersGroup" is the group we attached that are allowed to login to Linux servers. Below is a listing of other files you may want to configure and/or check.
/etc/ssh/sshd_config
#Make sure this is set properly in the file
UsePAM yes
/etc/sudoers
#Let's allow only the Linnux Admins group to sudo
%grpUnixAdmin ALL=(ALL) ALL