pbspy

PyPI - Version docs Python Version PyPI - License build

A python package for working with the Portable Batch System (PBS) job scheduler.

pbspy

Utilities for PBS job submission and output retrieval.

pbspy.gadi

Queue resource limits for the Gadi supercomputer (Australia).

Usage

Submitting a Job

To submit a job to the PBS scheduler, create a JobDescription and submit() it:

from pbspy import Job, JobDescription

job_description = (
    JobDescription(
        name="job_tiny",
        queue="normal",
        ncpus=1,
        mem="100MB",
        jobfs="0MB",
        walltime="00:05:00",
    )
    .add_command(["echo", "A"])
    .add_command(["echo", "B"])
)
job: Job = job_description.submit()
print(job.job_id)

The Job holds a job_id and other attributes.

The from_nodes() constructor of JobDescription creates a job description that uses all available resources for some number of nodes. Resource limits for a queue must be provided in a QueueLimits.

from pbspy import Job, JobDescription

queue_limits = QueueLimits(
    cpus_per_node=48, max_mem_per_node=190, max_jobfs_per_node=400
)
job_description = JobDescription.from_nodes(
    nnodes=2,
    name="job_1node",
    queue="normal",
    queue_limits=queue_limits,
    walltime="00:05:00",
)

pbspy does not currently support querying the QueueLimits of a PBS queue. Note that some PBS systems do not make such information queryable anyway.

The pbspy.gadi submodule has pre-defined QueueLimits for the Gadi supercomputer.

Note

Contributions are welcomed for QueueLimits for other clusters.

Retrieving Job Output

The output of a job can be retrieved by calling result() on a Job:

result: JobResult = job.result()
print(result.output)

result() waits for the job to complete before returning a JobResult that holds the stdout, stderr, exit_code, and other information output by PBS.

Wait for multiple jobs to complete with the result_all() static method of Job.

License

pbspy is licensed under the MIT License <http://opensource.org/licenses/MIT>.

API Documentation