torque submit filters

After debating whether to add node properties for SL4 and SL5 into the job managers for both cream and the lcg-ce I read Derek's post from RAL about using submit filters. So I thought I would have a go and see if I could tweak the node specification, keep the number of nodes requested intact for MPI and add additional property for the particular CE. Turns out its easy to implement but as usual there is some wierdness. You should be able to write your filter in whatever language you like and just specify the torque.cfg i.e.

Here is a simple example in bash:
/usr/local/sbin# cat torque_submit_filter.sh

#!/bin/sh
while read i
  do
    if [[ $i =~ "^#PBS -l nodes=[0-9]" ]]
    then
      export i="${i}:SL4"
    fi
    echo $i
  done

/var/spool/pbs# cat torque.cfg

SUBMITFILTER /usr/local/sbin/torque_submit_filter.sh

This works with cream but not with the lcg-ce.

So lets try again but this time in perl:

/usr/local/sbin# cat torque_submit_filter.pl

#!/usr/bin/perl -w

use strict;

# Echo all other input
while ()
  {
    # By default just copy the line.
    my $line = $_;

    if ($line =~ m/^#PBS -l nodes=[0-9]/)
      {
        chomp($line);
        $line = $line . ":SL5\n";
      }

    print ($line);
  }

Now this works with both cream and lcg-ce! Obviously you can do whatever takes your fancy to the qsub input and make it more intelligent.

A word of warning. We used the same queues for both CE's which meant that SL4 and SL5 resources were indistinguishable to users unless they used OS specific CE requirements. We ended flooded on the SL4 queues, with lots of free slots on the SL5 queues. So in the end we have created a new set of queues for the SL4 CE. Hopefully this will be explicit enough for users to target the correct CE.