Reference Guide

You can find the full API here but some endpoints might not be available depending on your deployment.

Status

GET /status/services

List of services

Returns a list containing all available micro services with a name, description, and status.

Status Codes:
GET /status/services/{servicename}

Get service information

Returns a single service descriptor (name, description and status) from its name.

Parameters:
  • servicename (string) – The service name
Status Codes:
GET /status/systems

List of systems

Returns a list containing all available systems and response status.

Status Codes:
GET /status/systems/{machinename}

Get system information

Returns a single system from its name.

Parameters:
  • machinename (string) – The system name
Status Codes:
GET /status/parameters

List of API parameters

Returns list of parameters that can be configured in environment files.

Status Codes:
  • 200 OK – List of parameters and values

Utilities

GET /utilities/ls

List directory contents

Returns a list of contents at the specified path on the {X-Machine-Name} filesystem.

Query Parameters:
 
  • targetPath (string) – Absolute path to destination (Required)
  • numericUid (boolean) – Show UID/GID instead of names
  • showhidden (boolean) – Show entries starting with ‘.’
Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Permission-Denied – User does not have permissions to access machine or path
  • X-Invalid-Path – <path> is an invalid path
  • X-Timeout – Command has finished with timeout signal
POST /utilities/mkdir

Creates a directory

Create a directory at the specified path on the {X-Machine-Name} filesystem.

Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Permission-Denied – User does not have permissions to access machine or path
  • X-Exists – <path> already exists
  • X-Invalid-Path – <path> is an invalid path
  • X-Timeout – Command has finished with timeout signal
PUT /utilities/rename

Rename/move a file, directory, or symlink

Rename/move a file, directory, or symlink at the sourcePath to the targetPath on the {X-Machine-Name} filesystem.

Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Permission-Denied – User does not have permissions to access machine or paths
  • X-Not-Found – <oldpath> not found
  • X-Exists – <newpath> already exists
  • X-Invalid-Path – <oldpath> and/or <newpath> are invalid paths
  • X-Timeout – Command has finished with timeout signal
PUT /utilities/chmod

Change file mode bits

Change the file mod bits of a given file according to the specified mode.

Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Permission-Denied – User does not have permissions to access machine or paths
  • X-Invalid-Path – <path> is an invalid path
  • X-Timeout – Command has finished with timeout signal
  • X-Invalid-Mode – <mode> is an invalid mode
PUT /utilities/chown

Change file owner and group

Changes the user and/or group ownership of a given file. If only owner or group information is passed, only that information will be updated.

Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Permission-Denied – User does not have permissions to access machine or paths
  • X-Invalid-Path – <path> is an invalid path
  • X-Timeout – Command has finished with timeout signal
  • X-Invalid-Owner – <owner> is an invalid user
  • X-Invalid-Group – <group> is an invalid group
POST /utilities/copy

Copy file from a filesystem path to another

Copies file from {sourcePath} to {targetPath}.

Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Permission-Denied – User does not have permissions to access machine or paths
  • X-Invalid-Path – <path> is an invalid path
  • X-Timeout – Command has finished with timeout signal
  • X-Exists – <targetPath> already exists
  • X-Not-Found – <sourcePath> not found
GET /utilities/file

determine file type

Uses the file linux application to determine the type of a file on the {X-Machine-Name} filesystem.

Query Parameters:
 
  • targetPath (string) – Absolute filesystem path (Required)
Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Permission-Denied – User does not have permissions to access machine or paths
  • X-Invalid-Path – <path> is an invalid path
  • X-Timeout – Command has finished with timeout signal
GET /utilities/head

Prints first part of a file

Returns the first part (10 lines by default) from a file. Options ‘bytes’ and ‘lines’ can not by specified simultaneously.

Query Parameters:
 
  • targetPath (string) – Absolute path to destination (Required)
  • bytes (string) – Returns the first N bytes, accepts K, KB, MB, etc
  • lines (string) – Returns the first N lines, accepts K, KB, MB, etc
  • skip_ending (string) – The output will be the whole file, without the last NUM bytes/lines of each file. NUM should be specified in the respective argument through bytes or lines.
Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Permission-Denied – User does not have permissions to access machine or path
  • X-Invalid-Path – <path> is invalid.
  • X-Timeout – Command has finished with timeout signal
GET /utilities/stat

determines the status of a file

Uses the stat application to determine the status of a file on the {X-Machine-Name} filesystem. The options are “%f %i %d %h %u %g %s %X %Y %Z” and the outputs are converted to integers.

Query Parameters:
 
  • targetPath (string) – Absolute filesystem path (Required)
  • dereference (boolean) – Follow symbolic links
Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Invalid-Path – <path> is an invalid path
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Not-A-Directory – targetPath is not a directory
  • X-Permission-Denied – User does not have permissions to access machine or paths
  • X-Timeout – Command has finished with timeout signal
POST /utilities/symlink

Create a symlink

Create a symbolic link (symlink) on the {X-Machine-Name} filesystem.

Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Permission-Denied – User does not have permissions to access machine or paths
  • X-Exists – <source> already exists
  • X-Invalid-Path – <target> and/or <source> are invalid paths
  • X-Timeout – Command has finished with timeout signal
GET /utilities/tail

Prints last part of a file

Returns the last part (10 lines by default) from a file. Options ‘bytes’ and ‘lines’ can not by specified simultaneously.

Query Parameters:
 
  • targetPath (string) – Absolute path to destination (Required)
  • bytes (string) – Returns the last N bytes, accepts K, KB, MB, etc
  • lines (string) – Returns the last N lines, accepts K, KB, MB, etc
  • skip_beginning (boolean) – The output will start with byte/line NUM of each file. NUM should be specified in the respective argument through bytes or lines.
Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Permission-Denied – User does not have permissions to access machine or path
  • X-Invalid-Path – <path> is invalid.
  • X-Timeout – Command has finished with timeout signal
GET /utilities/download

Download a small file (binary or text)

Blocking call that returns the file from the specified path on the {X-Machine-Name} filesystem.

Query Parameters:
 
  • sourcePath (string) – Path to the file to download (Required)
Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Invalid-Path – <path> is invalid
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Permission-Denied – User does not have permissions to access machine or path
  • X-Size-Limit – File exceeds size limit
POST /utilities/upload

Uploads a small file

Uploads a file to the specified path on the {X-Machine-Name} filesystem.

Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Permission-Denied – User does not have permissions to access machine or path
  • X-Invalid-Path – <path> is invalid.
DELETE /utilities/rm

Delete a small file

Delete a file to the specified path on the {X-Machine-Name} filesystem.

Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Permission-Denied – User does not have permissions to access machine or path
  • X-Invalid-Path – <path> is invalid.
  • X-Timeout – Command has finished with timeout signal
GET /utilities/checksum

Calculate the checksum of a given file

Calculate the SHA256 (256-bit) checksum of a specified file in {targetPath} on the {X-Machine-Name} filesystem.

Query Parameters:
 
  • targetPath (string) – Path to the file to calculate checksum (Required)
Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Permission-Denied – User does not have permissions to access machine or path
  • X-Invalid-Path – <path> is invalid.
  • X-Timeout – Command has finished with timeout signal
  • X-A-Directory – <targetPath> is a directory, can’t checksum directories
GET /utilities/view

View the content of a plain text file (UTF8)

View the content of a plain text file (UTF8 chars only) in {targetPath} on the {X-Machine-Name} filesystem.

Query Parameters:
 
  • targetPath (string) – Path to the file to view (Required)
Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-A-Directory – <targetPath> is a directory, can’t checksum directories
  • X-Invalid-Path – <path> is invalid
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Permission-Denied – User does not have permissions to access machine or path
  • X-Size-Limit – File exceeds size limit
  • X-Timeout – Command has finished with timeout signal
GET /utilities/whoami

Returns current system username

Returns the current username on the {X-Machine-Name}.

Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Permission-Denied – User does not have permissions to access machine or path
  • X-Timeout – Command has finished with timeout signal

Compute

POST /compute/jobs/upload

Submit Job by uploading a local sbatch file

Non-blocking call. Submits a batch script to SLURM on the target system. The batch script is uploaded as a file to the microservice which then stores it in a temporal directory in preparation to be submitted to the workload manager. The operation returns the task id associated to the Task microservice that will contain information of the SLURM job once it is created.

Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Permission-Denied – User does not have permissions to access machine
  • X-sbatch-error – sbatch returned error
POST /compute/jobs/path

Submit Job by a given remote sbatch file

Non-blocking call. Submits a batch script to SLURM on the target system. The batch script is uploaded as a file to the microservice which then stores it in a temporal directory in preparation to be submitted to the workload manager. The operation returns the task id associated to the Task microservice that will contain information of the SLURM job once it is created.

Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Permission-Denied – User does not have permissions to access machine
  • X-sbatch-error – sbatch returned error
GET /compute/jobs

Retrieves information from all jobs

Information about jobs on the SLURM scheduling queue. This call uses the squeue command.

Query Parameters:
 
  • jobs (array) – Comma-separated list of job IDs to retrieve.
  • pageSize (integer) – Number of entries returned (default is set only when pageNumber is set)
  • pageNumber (integer) – Page number
Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Permission-Denied – User does not have permissions to access machine
GET /compute/jobs/{jobid}

Retrieves information from a job

Information about a job on the SLURM scheduling queue. This call uses the squeue command.

Parameters:
  • jobid (string) – ID of the job
Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Permission-Denied – User does not have permissions to access machine
DELETE /compute/jobs/{jobid}

Delete Job

Cancel job from SLURM, using the scancel command.

Parameters:
  • jobid (string) – ID of the job
Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Permission-Denied – User does not have permissions to access machine
GET /compute/acct

Job account information

Reports accounting data of job in the SLURM job accounting log, this includes information from completed jobs. This call uses the sacct command.

Query Parameters:
 
  • jobs (array) – Comma-separated list of job IDs to retrieve
  • pageSize (integer) – Number of entries returned (default is set only when pageNumber is set)
  • pageNumber (integer) – Page number
  • starttime (string) – Start date/time of job’s query. Allowed formats are HH:MM[:SS] [AM|PM] MMDD[YY] or MM/DD[/YY] or MM.DD[.YY] MM/DD[/YY]-HH:MM[:SS] YYYY-MM-DD[THH:MM[:SS]]
  • endtime (string) – End time (and/or date) of sacct query. Allowed formats are HH:MM[:SS] [AM|PM] MMDD[YY] or MM/DD[/YY] or MM.DD[.YY] MM/DD[/YY]-HH:MM[:SS] YYYY-MM-DD[THH:MM[:SS]]
Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Permission-Denied – User does not have permissions to access machine

Storage

POST /storage/xfer-internal/rsync

rsync

Data transfer between internal CSCS file systems. To transfer files and folders from /users, /project or /store to the /scratch file systems for stage-in or stage-out jobs. Reference: https://user.cscs.ch/storage/transfer/internal/

Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
POST /storage/xfer-internal/mv

move (rename) files

‘Move files between internal CSCS file systems. Rename sourcePath to targetPath, or move sourcePath to targetPath /users, /project or /store to the /scratch file systems. Possible to stage-out jobs providing the SLURM Id of a production job. Reference: https://user.cscs.ch/storage/data_transfer/internal_transfer/

Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
POST /storage/xfer-internal/cp

copy files and directories

‘Copy files and directories between internal CSCS file systems. Copy sourcePath to targetPath /users, /project or /store to the /scratch file systems. Possible to stage-out jobs providing the SLURM Id of a production job. Reference: https://user.cscs.ch/storage/data_transfer/internal_transfer/

Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
POST /storage/xfer-internal/rm

remove files or directories

‘Remove files or directories in the internal CSCS file systems, with options rm -rf. With targetPath pointing to file system /users, /project, /store, or /scratch. Possible to stage-out jobs providing the SLURM Id of a production job. Reference: https://user.cscs.ch/storage/data_transfer/internal_transfer/

Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
POST /storage/xfer-external/upload

Upload a file

Starts an asynchronous upload to a specific path, the upload workflow is coordinated by a task in the Tasks microservice. This returns a task task that will provide a persisten URL at which the target file can be uploaded to, the persistent URL is encoded with a random hash and is available for an extended period of time (it does not depend on the lifetime of the OIDC token).

Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Permission-Denied – User does not have permissions to access path
  • X-Not-Found – targetPath not found
  • X-Invalid-Path – targetPath is an invalid path.
  • X-Not-A-Directory – targetPath is not a directory
  • X-Error – Error
POST /storage/xfer-external/download

Download a file

Start an asynchronous download by creating a new task in the Tasks microservice. The new task will first copy the file to the file transfer server, once this copy has been completed a persisten URL will be provided by the task. This persistent URL is encoded with a random hash and is available for an extended period of time (it does not depend on the lifetime of the OIDC token).

Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Permission-Denied – User does not have permissions to access path
  • X-Not-Found – targetPath not found
  • X-Invalid-Path – targetPath is an invalid path.
  • X-A-Directory – targetPath is a directory, can’t download directories
  • X-Error – Error
POST /storage/xfer-external/invalidate

Invalidate temporary URL

Remove a temporary URL attached to a given Task Id

Status Codes:
Request Headers:
 
  • X-Task-Id – Task Id associated to the upload/download task (Required)

Tasks

GET /tasks

Returns all tasks (or a subset)

List all recorded tasks and their status.

Query Parameters:
 
  • tasks (string) – Comma-separated list of {taskid}
Status Codes:
POST /tasks

Creates a new task

Creates a new task.

Status Codes:
Request Headers:
 
  • X-Firecrest-Service – Name of the service for which the task will be created (“compute” or “storage”) (Required)
  • X-Machine-Name – Name of the system for which the task will be created (must match configuration) (Required)
GET /tasks/{taskid}

Task status information

A long running task progress and result is tracked through a {taskid}.

Parameters:
  • taskid (string) – id of task
Status Codes:
  • 200 OK – Current status of a task with taskid. Depending on the type of task (Compute or Storage) and of the current status, the schema could change. Check the field status to match the specific response.
PUT /tasks/{taskid}

Updates a task

Updates a task entry that keeps track of progress

Parameters:
  • taskid (string) – id of task
Status Codes:
Request Headers:
 
  • X-Firecrest-Service – Name of the service for which the task will be created (“compute”/”storage”) (Required)
DELETE /tasks/{taskid}

Delete task

Delete a already existing task

Parameters:
  • taskid (string) – id of task
Status Codes:
POST /tasks/expire/{task_id}

Marks tasks with an expiration time

Marks tasks with an expiration time. After that the tasks will be removed.

Status Codes:

Reservation

GET /reservations

Returns all reservations

List all active reservations and their status

Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Timeout – Command has finished with timeout signal
  • X-Permission-Denied – User does not have permissions to access machine
POST /reservations

Creates a new reservation

Creates a new reservation with {reservation} name for a given SLURM groupname

Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Error – Error
  • X-Timeout – Command has finished with timeout signal
  • X-Permission-Denied – User does not have permissions to access machine
PUT /reservations/{reservation}

Updates reservation {reservation}

Updates an already created reservation named {reservation}

Parameters:
  • reservation (string) – reservation name
Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Error – Error
  • X-Timeout – Command has finished with timeout signal
  • X-Permission-Denied – User does not have permissions to access machine
DELETE /reservations/{reservation}

Deletes reservation {reservation}

Deletes an already created reservation named {reservation}

Parameters:
  • reservation (string) – reservation name
Status Codes:
Request Headers:
 
  • X-Machine-Name – The system name (Required)
Response Headers:
 
  • X-Machine-Does-Not-Exist – Machine does not exist
  • X-Machine-Not-Available – Machine is not available
  • X-Error – Error
  • X-Timeout – Command has finished with timeout signal
  • X-Permission-Denied – User does not have permissions to access machine

Certificator

POST /certificator

Creates SSH certificates

creates an SSH certificate to be used for system command execution

Status Codes: