Users: The Good, The Bad and The Ugly...

Last Thursday I went to see a few potential ScotGrid users. There are a couple of users in Civil Engineering who'd like to use the cluster. One of them has monte carlo code and this should be no problem (the good!) Another has a real need for MPI code (the bad!), which is challenging, but we obviously hope to build on the good work done here in GridIreland (see their wiki).

Finally, I spoke to a postgrad student in IBLS (Institute of Biomedical and Life Sciences) who has a lot of code to run on protein data (don't really understand the problem, actually). Now, she's got to run code she hasn't written herself, and neither does she have the source code and the damn thing asks questions in an interactive mode before it runs. (The ugly!)

On the computing service cluster she has to run the code in PBS interactive mode, in order to answer the questions. Then she has to hope that the network stays up between her machine and the cluster, because if it goes down (and often it does) it kills her job.

The first thing I did was show her how to use screen, so that at least she can run in a detached terminal. (I found a good tutorial.)

Last night, I started to write a python wrapper for the program which will take a defined set of default options, which can then be overridden on the command line, and pass them to the program. This turned out to be quite troublesome - using the python Popen3 class (pipe open) I just could not get the read() or readline() methods to behave properly with the select() call (even in nonblocking mode). select() would return a filehandle with data only once; and neither did the write() call seem to push the output properly into the program. However, when I switched to using the os.read() and os.write() calls, instead of the class methods, things started to work exactly as I expected. Hopefully this will run the program properly in a non-interactive mode and open up the possibility of running on our cluster.

Of course, there's a different track to this as well - we urgently need to get VOs setup for these users so that they can use edg-job-* commands or ganga to run their jobs. Watch this space...