Installation notes for ifp-32
From SpeechWiki
To add users
useradd -u <uid> -g<gid> <userName>
so that uid and gid match the existing ones
rocks sync users cluster-fork '/sbin/service autofs restart'
to replicate the login info immediately, otherwise they get sent out once an hour This copies their password/login info and autofs entries into /etc/auto.home for the compute nodes. DO NOT EDIT /etc/auto.home
Allow access to wordpress and ganglia from everywhere via https
cd /etc/sysconfig/ chmod u+w iptables emacs iptables
add line
-A INPUT -m state --state NEW -p tcp --dport https -j ACCEPT
Hash known hosts, so a hacked account on one system won't propagate to others so easily
cd /etc/ssh/ chmod u+w ssh_config emacs ssh_config
add line under Host*
HashKnownHosts yes
This actually shares the dirs through the NFS
chmod u+w exports emacs exports /ws/ifp-32-1 10.0.1.0/255.255.255.0(rw) /ws/ifp-32-2 10.0.1.0/255.255.255.0(rw) /etc/rc.d/init.d/nfs restart
You have to do the analogous thing on the compute nodes, if you want to share their something from them.
This sets up the automounts on the /cworkspace
cd /etc/ chmod u+w auto.* emacs auto.master
add line
/cworkspace /etc/auto.share --timeout=1200
emacs auto.share
add lines
apps ifp-32.local:/export/& install ifp-32.local:/export/home/& c1-1 compute-1-1.local:/ws/c1-1 c1-2 compute-1-2.local:/ws/c1-2 c1-3 compute-1-3.local:/ws/c1-3 c1-4 compute-1-4.local:/ws/c1-4 c1-5 compute-1-5.local:/ws/c1-5 c1-6 compute-1-6.local:/ws/c1-6 c1-7 compute-1-7.local:/ws/c1-7 c1-8 compute-1-8.local:/ws/c1-8 c1-9 compute-1-9.local:/ws/c1-9 c1-10 compute-1-10.local:/ws/c1-10 c1-11 compute-1-11.local:/ws/c1-11 c1-12 compute-1-12.local:/ws/c1-12 c1-13 compute-1-13.local:/ws/c1-13 c1-14 compute-1-14.local:/ws/c1-14 c1-15 compute-1-15.local:/ws/c1-15 c1-16 compute-1-16.local:/ws/c1-16 ifp-32-1 ifp-32.ifp.uiuc.edu:/ws/ifp-32-1 ifp-32-2 ifp-32.ifp.uiuc.edu:/ws/ifp-32-2 usr_local_linux_cluster ifp-32.local:/export/usr_local_linux_cluster
cluster-fork '/sbin/service autofs restart'
To change scheduling to equal cpu allocation among waiting users
qconf -mconf
add lines
enforce_user auto auto_user_fshare 100
qconf -msconf
add lines
weight_tickets_functional 10000
To make qstat show jobs of all users
cd /opt/gridengine/default/common emacs sge_qstat
add line
-u *
get a latish subversion from rpmforge (the one from centos is too old):
wget http://packages.sw.be/subversion/subversion-1.5.6-0.2.el5.rf.x86_64.rpm rpm -ivh subversion-1.5.6-0.2.el5.rf.x86_64.rpm
get a latish version of python
this will install /usr/bin/python2.6 but won't touch the /usr/bin/python that was there originally
wget http://www.python.org/ftp/python/2.6.2/Python-2.6.2.tar.bz2 tar -xjvf Python-2.6.2.tar.bz2 cd Python-2.6.2 ./configure yum install readline-devel make -j 4 make altinstall
reduce the rate of brute-force attack
emacs /etc/sysconfig/iptables
Change the line
-A INPUT -m state --state NEW -p tcp --dport ssh -j ACCEPT
to
#don't allow too many ssh connections from the same IP address #this reduces brute-force attacks and keeps the secure logs clean. Arthur #-A INPUT -m state --state NEW -p tcp --dport ssh -j ACCEPT -A INPUT -m state --state NEW -p tcp --dport ssh -j SSH_CHECK
and add above that
#Arthur's change to limit the number of brute-force ssh attacks #allows only 3 logins from the same ip address every 60 sec. -N SSH_CHECK -A SSH_CHECK -m recent --set --name SSH -A SSH_CHECK -m recent --update --seconds 60 --hitcount 3 --name SSH -j DROP -A SSH_CHECK -j ACCEPT
Energy usage
Dell powervault MD-1000 with 12 7200RPM disks, with one of the two redundant power supplies turned off: 1.6 Amps idle, 1.9 Amps while doing "hdparm -t" benchmark.
Dell powervault MD-1000 with 8 7200RPM disks, with one of the two redundant power supplies turned off: 1.5 Amps under a typical workload 1.7 Amps under typical workload simultanously with "hdparm -t" benchmark.
Dell Poweredge 2850 1.8 Amps with a light workload 2.7 with cpuburn running on each of the 4 cpus (full cpu load)
Dell powerconnect RPS-600 .4 amps idle (nothing plugged into it) .5 amps with one powerconnect 5324 plugged into it, and the powerconnect 5324's own power unplugged .2 amps powerconnect 5324 with own power plugged in, and no DC power supply (powerconnect RPS-600) plugged in
