(Almost) 100% Success for Glasgow!

We noticed today that (apart from that pesky red mark for CMS over the past 6 months, and some yellow on the WMS tests) we're looking incredibly green and functional on the Glasgow Dashboard at the moment. So, we took a picture before that changed...

(The Glasgow Dashboard is Mike's mashup of all the useful metrics on the web concerning UKI-SCOTGRID-GLASGOW, now over two, alternating, pages. It's actually quite useful, and remarkably festive at this time of year.)

openmpi magic

I have just rebuilt openmpi-1.3.4 for use with CASTEP. This is built to a useful /opt location with Torque, F90 support for gfortran44.

gLite support for OPENMPI is fairly generic and means any openmpi rpm install does not have any useful batch or interconnect support. So anything out of the ordinary requires a custom build.

I will stick the RPM available for download here in the next few days.

The magic for building from a src rpm:

cd /usr/src/redhat/SPECS/
rpmbuild -ba --define '_prefix /opt/openmpi-1.3.4' --define '_mandir %{_prefix}/share/man'
--define 'configure_options --prefix=/opt/openmpi-1.3.4 --with-tm=/usr/
FC=gfortran44 F77=gfortran44 CC=gcc44 CXX=g++44 FFLAGS=-O2 FCFLAGS=-O2 CFLAGS=-O2 CXXFLAGS=-O2'
openmpi-1.3.4.spec

Full Instructions are here.

issues with gfortran43/44 and mpich

I am finally getting to the bottom of what has been going wrong with re-compiling MPICH for F90/F95 (required for CASTEP - a demanding Fortran code). I have now narrowed it down to one issue between recompiling MPICH with gfortran43/44 for SL5 usage with CASTEP instead of plain old gfortran.

FC="gfortran44" ; export FC;
F90="gfortran44" ; export F90;
...
--enable-f90modules

The SAM MPICH test runs after recompiling with F90 support using gfortran and in fact it works fine on SL4 and SL5. So that was not the issue.

message size            transfertime            bandwidth
32 bytes                0.000000 sec            inf MB/s
2048 bytes              0.000117 sec            17.476267 MB/s
131072 bytes            0.001445 sec            90.687654 MB/s
8388608 bytes           0.078437 sec            106.946397 

It turns out that MPICH just doesn't work when compiled with gfortran43/44. Leaving me in a bit of an pickle as CASTEP will not compile on SL5 with gfortran, you have to use gfortran43/44!

Time for the backup plan ..... openmpi.

lightening testing of glexec with SCAS

Well since it is looking increasingly lightly that we will be moving to some form of identity switching at our sites to give us more information about who is running their jobs via their pilot frameworks. I thought I would give it whirl.

So in some lightening tests, a phrase I am stealing from lightening talks sometimes given at technical conferences, I am trialling glexec for identity switching coupled with SCAS for centralised allow/deny decisions.

Here is what was tested:

an install of SCAS
and install and test GLEXEC with SCAS on LCG-CE
and install and test GLEXEC with SCAS on CREAM [1]
and install and test GLEXEC on WN (SL4)
and install and test GLEXEC on WN (SL5)

Detailed Instructions and Results can be found here

The short and long of it is that it is very easy to set-up SCAS and use it on whatever service you want. So easy infact that once you SCAS server is up and running you cn direct calls to it from your CE's in a matter of minutes. glexec on the WN is just as easy, all that remains would be for someone to use it.

We currently have not rolled any of this into production but I am confident that it could be done quickly and safely. Since we are into real data taking, safely is the keyword. We want no unnecessary downtimes, which I think is achievable.

Thanks to Oscar at Nikhef for answering questions.

1: there appeared to be a certificate permission issue when calling SCAS from CREAM that prevented job submission. It looks like you need to copy the hostcert/key by hand to another cert owned by the tomcat user.


-rw-r--r-- 1 tomcat tomcat 2187 Dec 4 10:44 tomcathostcert.pem
-r-------- 1 tomcat tomcat 1863 Dec 4 10:44 tomcathostkey.pem