Orion CLI

Options

The Orion CLI has several top level options that modify the behavior of the commands that are run.

Timeout

The --timeout flag can be used to set the request timeout in seconds, i.e. if the server takes longer to respond to a request than the specified number of seconds, the client disconnects. The default value is 10, but that may be too short for some cases:

  • Slower or inconsistent internet connections

  • Sharing a package with many cubes to many users

  • Uploading large files

JSON

The --json flag can be used to have the Orion CLI return JSON rather than plaintext to make it easier parse the results

ocli --json <command>

Profile

The --profile flag specifies which profile configuration to use while making requests to the Orion API. The default is the default profile

ocli --profile=testing <command>

ocli config directory and file permissions are now forced to be read/write for the owner

Max Retries

The --max-retries flag specifies the number of times that the client will retry a connection if it fails or gets a 404 error. This is set to 2 by default. There is back off implemented with the retries, so the time will increase exponentially (to a certain point) with the number of retries.

Shard uploads and downloads are unaffected by --max-retries.

ocli --max-retries=5 <command>

Configuration Commands

To configure an Orion Profile

ocli config profile --url <hostname> --username <username> [--password <text>] [--with-token] [--project <project_id>]

To configure an Orion Profile for use with SAML requires first creating a token in the web interface’s user profile page

ocli config profile --url <hostname> --username <username> --with-token [--project <project_id>]

To change the default project of an Orion Profile

ocli config project

To delete an Orion Profile

ocli config delete <profile_name>

To retrieve information about the current profile

ocli config info

To list the configured profiles

ocli config list

To refresh your Orion Profile token

ocli config token

Storage Commands

Folders

Commands to interact with Orion’s virtual filesystem

Note that folder commands are TOP LEVEL - e.g. ocli ls <path> instead of ocli folders ls <path>.

Folders contain Files, Datasets, ShardCollections, and other folders. They can take the following forms:

/project/<project-id>/user/<username>/<path>

Project folders which are private to user with the given username.

/project/<project-id>/team/<path>

Project shared folders. All project members have access to these folders and their contents.

/project/<project-id>/workspace/<workspace-id>/<path>

Project shared workspace folders. Project members are given permission by sharing the workspace with them.

/organization/<path>

Organization shared folders.

List contents of a folder, or details of an item, use the following command. Setting --recurse=True will list all subdirectories and contents of the provided path(s). Items can be specified by path or file://<file_id>|dataset://<dataset_id>.

ocli ls <path(s)|item(s)> [--recurse=bool]

To create a folder, use the following command

ocli mkdir <path(s)>

To copy to/from Orion. This works similarly to scp, but dest must be a folder or local directory. Currently, only file and folder upload is supported. Setting --recurse=True will copy children recursively. Setting --dry-run=True will list the items that would be copied and then quit.

ocli cp <source> <dest> [--recurse=<bool>] [--dry-run=<bool>]

To peek inside of a file or dataset in Orion, use the following command. Items can be specified by path or by file://<file_id>|dataset://<dataset_id>. --number= or -n will optionally specify the number of lines for a file or the number of records for a dataset.

ocli head <path(s)|item(s)> [-n, --number=<integer>]

To move a folder or an item. This will move all the children if the path refers to a Folder. Setting --dry-run=True will list the move(s) and then quit. Note: items will inherit the destination folder’s permissions. Therefore, mv can be used to share across teams, workspaces, or organization.

ocli mv <source> <dest> [--dry-run=<bool>]

To remove a folder or item. Setting --dry-run=True will list the item(s) to be removed and then quit. Note: currently will not remove a folder with children. In the future this will use the trash-can api.

ocli rm <path(s)> [--dry-run=<bool>]

Workspaces

The CLI provides a number of commands to interact with workspaces, shared areas for Orion files, datasets, and collections organized into folders.

To create a workspace, use the create command and give it a slug, a name unique to this project. The backend will automatically give the workspace a unique id, which can be used in subsequent calls to add or remove users by sharing the workspace.

ocli workspaces create <workspace_slug>

To share the workspace and everything it contains with other users, use the following

Datasets

The CLI provides several ways to interact with datasets

Using the id of a dataset it is possible to access all of the information associated with the dataset

ocli datasets info <resource_id>

To delete a dataset, use the following command

ocli datasets delete <resource_id>

To list the datasets the current orion profile has access to. The arguments passed after list are used as query parameters. For example tags=2D,3D name=foo or name=aspirin.

ocli datasets list [--project=<integer>] [--name=<string>] \
    [--limit=<integer>]      \
    [--offset=<integer>]     \
    [owner=<integer>]        \
    [deleted=<boolean>]      \
    [expired=<boolean>]      \
    [tags=TAG1,...]          \
    [tag_ids=<integer>,...]  \
    [systags=<tag>,...]      \
    [exclude_tags=<tag>,...] \
    [tag_match=[default: exact](exact | iexact | contains | icontains | startswith | istartswith | endswith | iendswith)] \
    [order_by=[-](created | name | count | owner)] \
    [<key>=<value>,...]

ex:
    ocli datasets list
    ocli datasets list --name=aspirin_ --limit=30
    ocli datasets list \
        --project=12 \
        --limit=20 \
        tag_match=exact \
        order_by=-size \
        tags=3D,foo_jobs \
        exclude_tags=2D,bar_jobs \

To upload a dataset, pass the local path to the file and the client will create a new dataset. Handles any OEChem readable format, except JSON.

ocli datasets upload <file> [--project=<identifier>]

The following command will download a dataset, with any OEChem writable format except JSON, to your local file system.

ocli datasets download <resource_id> <file>

If the creation of a dataset is interrupted (e.g. a job fails or is canceled as it is writing records), then it will be left in a dirty state. The following command will finalize a dataset, which sets dirty to false and provides an accurate count of records contained in the dataset.

ocli datasets finalize <resource_id>

Files

The Orion CLI provides several ways to interact with Files.

To retrieve information about an Orion File

ocli files info <resource_id>

To delete an Orion File

ocli files delete <resource_id>

To list Orion Files. Possible to filter using query parameters. For example tags=foo will filter Orion Files that have the tag foo.

ocli files list [--project=<integer>] [--name=<string>] \
    [--limit=<integer>]  \
    [--offset=<integer>] \
    [owner=<integer>]    \
    [deleted=<boolean>]  \
    [expired=<boolean>]  \
    [tags=TAG1,...]      \
    [tag_ids=<integer>,...] \
    [tag_match=(exact | iexact | contains | icontains | startswith | istartswith | endswith | iendswith)] \
    [order_by=[-](created | name | size | owner | id)] \
    [<key>=<value>,...]

ex:
    ocli files list
    ocli files list --name=foo_ --limit=10
    ocli files list \
        --project=12 \
        --limit=20 \
        tag_match=exact \
        order_by=-size \
        tags=mol_jobs,foo_jobs \
        exclude_tags=bar_jobs \

Uploading a file from the local file system to Orion

ocli files upload <file> [--project=<identifier>]

Downloading an Orion File to the local file system

ocli files download <resource_id> <file>

Shard Collections

Commands for managing collections.

To list the collections that you have access to

ocli collections list [--limit=<integer>] [--offset=<integer>] \
    [owner=<integer>] [project=<integer>] \
    [deleted=<boolean>] [expired=<boolean>] \
    [tags=<tag>,...] [tag_ids=<integer>,...] \
    [tag_match=(exact | iexact | contains | icontains | startswith | istartswith | endswith | iendswith)] \
    [order_by=[-](created | name | size | owner | id)] \
    [<key>=<value>,...]

ex:
    ocli collections list
    ocli collections list --name=foo_
    ocli collections list \
        --project=12 \
        --limit=20 \
        tag_match=exact \
        order_by=-size \
        tags=mol_jobs,foo_jobs \
        exclude_tags=bar_jobs \

To get information on a specific collection

ocli collections info <resource_id>

To list shards in a collection

ocli collections list-shards <resource_id> \
    [--limit=<integer>] [--offset=<integer>] \
    [name=<name>] [after=<integer>]

ex:
    ocli collections list-shards 344923
    ocli collections list-shards 344923 --limit=5
    ocli collections list-shards 344923 name=foo_12

To get information on a specific shard in a collection

ocli collections info-shard <resource_id> <shard_id>

To upload a list of files to a new collection

ocli collections upload <name> <file> [<file> ...]

To upload a list of files to an existing collection

ocli collections upload <resource_id> <file> [<file> ...]

To open an existing, closed collection in order to add to it

ocli collections open <resource_id>

To close an existing, open collection in order to prevent adding to it

ocli collections close <resource_id>

To download all shards in an existing collection to a directory

ocli collections download <resource_id> <destination>

To download a list of shards in an existing collection to a directory

ocli collections download-shards <resource_id> [<shard_id> ...] <destination>

To delete an existing collection

ocli collections delete <resource_id>

To delete individual shards in an existing collection

ocli collections delete-shards <resource_id> <shard_id> ...

Secrets

Secrets are a way of storing tokens in Orion. All values are encrypted upon upload and are decrypted when request through the API

To get information on a specific secret

ocli secrets info <resource_id>

To delete an existing secret

ocli secrets delete <resource_id>

To list the secrets that you have access to

ocli secrets list [--name=<string>] \
    [--limit=<integer>] [--offset=<integer>] \
    [description=<description>] [owner=<integer>] \
    [<key>=<value>,...]

ex:
    ocli secrets list
    ocli secrets list --name=secret_foo

To create a new secret.

If --value is not provided, a prompt will provide the ability to set the value using a password input to avoid storing the value in the terminal history.

ocli secrets create <secret_name> [--value=<secret_value>] [--description=<string>]

Services

Orion Services is a registry of internal or external services integrated with the larger Orion system

To get information on a specific service

ocli services info <resource_id>

To delete an existing service

ocli services delete <resource_id>

To list the services that you have access to

ocli services list [--name=<string>] \
    [--limit=<integer>] [--offset=<integer>] \
    [owner=<integer>] [<key>=<value>,...]

ex:
    ocli services list
    ocli services list --limit=5 owner=10

To create a new service

ocli services create <service_name> [--description=<string>]

Projects

Commands for managing projects

To get information on a specific project

ocli projects info <resource_id>

To delete an existing project

ocli projects delete <resource_id>

To list the projects that you have access to

ocli projects list [--limit=<integer>] [--offset=<integer>] \
    [name=<name>] [as_admin=<boolean>] \
    [deleted=<boolean>] [expired=<boolean>] \
    [<key>=<value>,...]

ex:
    ocli projects list
    ocli projects list name=foo_
    ocli projects list deleted=true --limit=5

To create a new project

ocli projects create <project_name>

To share a project

ocli projects share <project_id> user <user_id>

To unshare a project

ocli projects share <project_id> user <user_id> [--new-owner=<user_id>]

To leave a project

ocli projects leave <project_id> [--new-owner=<user_id>]

  • Any member of a project can use this command to leave a project.

  • If --new-owner isn’t specified, the project assets belonging to the leaving user will be transferred to the project owner.

  • If the user specified as --new-owner isn’t already a member of the project, they will automatically join the project.

  • If the leaving user is the project owner --new-owner must be specified.

Floe Commands

The following commands are a set of resources in Orion that relate to Floe which includes Packages, WorkFloes, and Jobs.

Packages

Commands related to Floe Packages

Initializing a package

ocli packages init

The ocli packages init command:

  • Sets up a skeleton of an Orion package containing cubes and floes.

  • Provides simple example cube and floe.

  • Supports testing setup using PyTest including working tests for the example cube and floe.

  • Provides a setup.py file with commands to run tests or build the package.

  • Provides version configuration via storing the version only in the module’s __init__.py file.

Retrieving the information about a Floe Package

ocli packages info <resource_id>

Retrieving the Environment build log for a Floe Package

ocli packages environment <resource_id>

Retrieving the Inspection information for a Floe Package

ocli packages inspection <resource_id>

Deleting a Floe Package

ocli packages delete <resource_id>

Listing Floe Packages

ocli packages list [--limit=<integer>] [--offset=<integer>] \
    [states=<state>,... (ready, deprecated, queued, error)] \
    [name=<name>] [UUID=<uuid>] [owner=<integer>] \
    [<key>=<value>,...]

ex:
    ocli packages list
    ocli packages list states=error
    ocli packages list --limit=20 states=ready,queued

Upload a Floe Package

ocli packages upload <file_or_url>

WorkFloes

Commands for accessing Workfloes in Orion. Workfloes are a collection of cubes that defines the flow of data.

Retrieving the information about a WorkFloe

ocli workfloes info <resource_id>

Deleting a Workfloe

ocli workfloes delete <resource_id>

Listing Workfloes

ocli workfloes list [--package=<integer>] [--name=<string>] \
    [--limit=<integer>] [--offset=<integer>] \
    [states=<state>,... (ready, deprecated, hidden, invalid)] \
    [systags=<tag>,...] [tags=<tag>,...] [owner=<integer>] \
    [classification=<string> [classification_depth=<integer>]
    [order_by=(recently_used | name)]
    [<key>=<value>,...]

ex:
    ocli workfloes list
    ocli workfloes list tags=tag1,tag2
    ocli workfloes list --name=collection_to_ --limit=10 order_by=recently_used
    ocli workfloes list --package=10 states=ready,deprecated

Listing Classifications

ocli workfloes list-classifications [--name=<string>] [--path=<string>] \
    [--depth=<integer>] [--limit=<integer>] [--offset=<integer>] \
    [<key>=<value>,...]

ex:
    ocli workfloes list-classifications
    ocli workfloes list-classifications --path=Example/Test
    ocli workfloes list-classifications --name=exam --depth=1 --limit=10

Jobs

Commands for interacting with Jobs in Orion. Jobs are an instantiation of a Workfloe that performs the computation defined by the Workfloe.

Retrieving the information about a Job

ocli jobs info <resource_id>

Retrieve the logs of a Job

ocli jobs logs <resource_id>

Canceling a running Job

ocli jobs cancel <resource_id>

Deleting a Job

ocli jobs delete <resource_id>

Listing existing Jobs

ocli jobs list [--limit=<integer>] [--offset=<integer>] \
    [name=<name>] [project=<integer>] [workfloe=<integer>] \
    [job_type=(system | user)] [owner=<integer>] \
    [states=<state>,... (queued, running, complete)] \
    [tags=<tag>,...] [<key>=<value>,...]

ex:
    ocli jobs list
    ocli jobs list workfloe=20
    ocli jobs list tags=tag1,tag2 states=complete
    ocli jobs list job_type=user name=aspirin_ states=queued,running

Starting a Job

ocli jobs start <workfloe_id> <job_name> [--wait] \
    [FLOE_PARAMETERS]

ex:

ocli jobs start 6836 job_name0 --in 601 --out run_output.oedb

Note

Adding -- --help to the ocli jobs start command will print the parameters that are expected by the Workfloe.

Watching a Job

ocli jobs watch <workfloe_id> <job_name> [CUBES] [OPTIONS]

Collect debug information

ocli jobs debug-export <job_id>

Cost Exporting

Generating Storage Use Report

ocli cost export --report-type=storage_use [USER IDS]

ex:

ocli cost export --report-type=storage_use 10 11 20

Generating Spend Summary Report

ocli cost export --report-type=spend_summary [USER IDS]

ex:

ocli cost export --report-type=spend_summary 10 11 20

MolSearch

Databases

Commands to manage MolSearch databases

To create a molsearch database from the specified MAAS collection

ocli molsearch create-database --collection=<collection_id> [--cpu=<integer>] \
    [memory-mb=<integer>] [disk-space=<integer>]\
    [gpu=<integer>] [instance-type=<string>] \
    [pools=<pools>,... (system, default,...)]

To list molsearch databases you have access to

ocli molsearch list-databases [--limit=<integer>] \
    [--offset=<integer>]

To display details of a specific molsearch database

ocli molsearch get-database-detail <database_id>

To delete a molsearch database

ocli molsearch delete-database <database_id>

To share a molsearch database with a user

ocli molsearch share-database <database_id> user <user_id(s)> [--write] [--delete] [--share]

To unshare a molsearch database with a user

ocli molsearch unshare-database <database_id> user <user_id(s)>

To share a molsearch database with all users in the current organization

ocli molsearch share-database <database_id> organization

To unshare a molsearch database with all users in the current organization

ocli molsearch unshare-database <database_id> organization