Cube Basics

A cube is a python object for doing an independent unit of computation in the Orion cloud environment. Cubes are building blocks for Floes, which are specific complex computations. A cube can be used in any number of Floes and is should be written as a small reusable computation.

All cubes derive from the class floe.api.cubes.ComputeCube, but we provide mixins such as orionplatform.mixins.RecordPortsMixin that will provide default ports.

Note

While, the Orion Scheduler supports sending many kinds of units of work to a cube, as a practical matter the unit of work for almost all cubes is an datarecord.OEMolRecord. Therefore the items of work sent to cubes are referred to as records in the rest of the cube writing docs.

Member Functions

To implement any cube the following cube method must be implemented:

  • def process(self, record, port): Processes a record. Called for each record.

This function defines the basic logic of how a cube processes a record on which this cube should operate. The port argument stores the name of the port via which record arrived. This can be important for cubes which have multiple inputs and whose logic depends on the origin of the record.

Other cube methods which may be implemented are:

  • def begin(self): Called once to initialize the cube before records are passed to process.

  • def end(self): Cleanup routine. Called after all units of work have been sent through process.

Ports

Ports are the I/O mechanism for cubes. Records come in through input ports and are sent to the cubes process method along with the name of the port (see Port Basics for exceptions to this rule). To output a record it must be explicitly emitted to an output port in either the begin, process or end method. orionplatform.mixins.RecordPortsMixin automatically includes the following standard ports:

  • intake : Input Port

  • success : Output Port for successfully processed records.

  • failure : Output Port for records than could not be processed.

This cube takes records from the intake input port and passes them to the success output port unchanged. In the this example, and in all cubes, the output port is an object attached to the cube with an emit function records can be passed to.

from floe.api import ComputeCube
from orionplatform.mixins import RecordPortsMixin


class PassThroughCube(RecordPortsMixin, ComputeCube):
    def process(self, record, port):
        self.success.emit(record)

Now lets add a counter to the previous cube so we know how many records are passed along. This cube makes use of the begin and end functions to initialize the counter and uses the cubes logging capability to write the count to the log file after all records have been sent to the cube.

from floe.api import ComputeCube
from orionplatform.mixins import RecordPortsMixin


class PassThroughAndCountCube(RecordPortsMixin, ComputeCube):
    def begin(self):
        self.count = 0

    def process(self, record, port):
        self.count += 1
        self.success.emit(record)

    def end(self):
        self.log.info(f"Counted {self.count} records")

Parameters

The final element to add to our cube is a parameter. Parameter are variables that can be set at runtime to control the behavior of the cube. In the example below we add a boolean parameter that controls whether the records are sent to the success or failure port.

from orionplatform.mixins import RecordPortsMixin
from floe.api import ComputeCube, BooleanParameter


class SwitchAndCountCube(RecordPortsMixin, ComputeCube):
    send_to_failure_port = BooleanParameter(
        "send_to_failure_port",
        default=False,
        title="Fail",
        description="If true sends all records to the failure port.",
    )

    def begin(self):
        self.count = 0

    def process(self, record, port):
        self.count += 1

        if self.args.send_to_failure_port:
            self.failure.emit(record)
        else:
            self.success.emit(record)

    def end(self):
        self.log.info(f"Counted {self.count} records")

We used a boolean parameter in the example above. The Floe library includes the following basic parameters

These parameters are all declared in the same way at the prior example, with the name of the variable the parameter class is assigned to always passed as a string to the first argument of the constructor. The values of the parameters can be accessed in either the begin, process or end methods of the cube via “self.args.<parameter name>”.

Parameters that interact with fields on an datarecord.OEMolRecord are another common type of parameter. There are many Orion Platform Parameters for the different datarecord.OEField types.

Exceptions

Floe contains a module for exceptions which trigger special behaviors in Orion. For now, there is only one, namely, :py:class:`floe.api.exceptions.HardwareError. When a cube running in Orion raises this error, it will cause the instance it is running on to stop(after finishing all existing work) and terminate.