Just a short note from the EGEE 09 conference. It's been very gratifying to have had so much interest in gqsub at the conference - I even had emails about it scant hours after the poster was put up (and before the offical poster session!).
In response to the comments recieved, I've put a roadmap of planned features up on the gqsub page, which gives an idea of where it's headed.
In addition, v 1.2.0 is out, which implements auto staging back of output. This means that in cases where there is not a shared filesystem between the UI and the worker node, but there is GridFTP server on the UI, then gqsub will pull out the JDL tricks we used earilier with the Lumerical deployment. This results in the illusion of a shared filesystem - the job is submitted, and the output appears in the right places as if it was done in a shared filesystem.
Showing posts with label gqsub. Show all posts
Showing posts with label gqsub. Show all posts
Thursday, September 24, 2009
Wednesday, September 09, 2009
Canna I no just use qsub?
Ah, the endless refrain.
Anytime a user with cluster experience is introduced to the gLite submission mechanism, some question of that order (although not always with a Scottish accent) is inevitable.
Pulling out my Human-Computer Interaction hat, I first came to the conclusion that, despite the occasional hints to the contrary, users are indeed Human. Hot on the heels of this realisation, a little bit of analysis of the gLite job submission and control tools indicated that, whilst very powerful, they work in a very different fashion to qsub.
It's not clear that qsub is in any sense a better iterface than the native command line tools, but it is clear that it is different.
The general idea was to resolve this difference by providing a different interface to grid job submission that was more familiar to users with existing experience of cluster computing. Wether it's going to be a better approch for a user without that experience is not clear; but it will make it simpler for users to use the Grid as an offload for a local cluster (i.e. use a cluster, when it's full, send the jobs to the Grid).
It turns out that the POSIX defintion of qsub isn't too far away, conceptually, from a Grid system, so all that was needed to act as an interface transalation layer was a relativly straightforward python script.
Rather than relay all the gory details here, let me direct you to the gqsub download page, with the manual.
For users on svr020, it's installed in the default path, so you can just use it. Note that to properly mirror the expected behaviour you probably want to make sure you run from within $CLUSTER_SHARED.
But to answer the original question: "Aye!"
Anytime a user with cluster experience is introduced to the gLite submission mechanism, some question of that order (although not always with a Scottish accent) is inevitable.
Pulling out my Human-Computer Interaction hat, I first came to the conclusion that, despite the occasional hints to the contrary, users are indeed Human. Hot on the heels of this realisation, a little bit of analysis of the gLite job submission and control tools indicated that, whilst very powerful, they work in a very different fashion to qsub.
It's not clear that qsub is in any sense a better iterface than the native command line tools, but it is clear that it is different.
The general idea was to resolve this difference by providing a different interface to grid job submission that was more familiar to users with existing experience of cluster computing. Wether it's going to be a better approch for a user without that experience is not clear; but it will make it simpler for users to use the Grid as an offload for a local cluster (i.e. use a cluster, when it's full, send the jobs to the Grid).
It turns out that the POSIX defintion of qsub isn't too far away, conceptually, from a Grid system, so all that was needed to act as an interface transalation layer was a relativly straightforward python script.
Rather than relay all the gory details here, let me direct you to the gqsub download page, with the manual.
For users on svr020, it's installed in the default path, so you can just use it. Note that to properly mirror the expected behaviour you probably want to make sure you run from within $CLUSTER_SHARED.
But to answer the original question: "Aye!"
Subscribe to:
Comments (Atom)