The gridengine scheduler determines when and on which machines your jobs will run. It's quite complicated, with many tunable parameters. This page provides a high-level explanation of how you can expect gridengine to behave.
Goals
- Our primary goal is fairness. All users should have an equal share of grid resources.
- Users that donate equipment to the grid are afforded priority access.
Implementation
Slots == Cores
No machine in the grid may run more jobs than it has cores. This guarantees that every process has a dedicated core.
Run Time Limits: Short, Long, Very Long
Run time limits are imposed by separate queues. The short queue will not allow jobs to run for more than 1 hour. The long queue limits jobs to 24 hours. The very long queue has no time limit.
Jobs run by default in the short queue.
Every machine has slots in each queue equal to the number of cores it has. So a two-core machine has six slots: two for each queue. No more than two jobs can run at any time on it, but they can be from any queue.
Long Job Quotas
To keep long and very long jobs from crowding out shorter jobs, there are grid-wide quotas imposed on them. No more than 440 long jobs can be running at any time, and no more than 220 very long jobs.
Memory Resources
Memory usage is considered only when scheduling a job. Actual memory use on each machine is compared with the amount of memory requested by a job, and jobs only run on machines with sufficient free memory to accommodate the request.
By default, jobs request 2G of memory.
Priorities
Job priorities start out the same, but rise gradually with the length of time spent waiting to run.
Job priorities do not currently behave as expected. Heavy grid users should have lower priority (with a half-life of 7 days), and jobs submitted by new grid users should jump the queue. A thorough review of the scheduling algorithm and configuration is pending, and this section will be rewritten when we understand it better, and improve it.
Grid administrators can apply override tickets to jobs to change their priority and allow them to run ahead of others.