Release Notes

v0.12.0: November 2020

  • Support for a parameter order attribute, to influence user interface display, has been added.
  • Support for converting from newer versions of the Floe spec to older versions with FloeSpecConverter has been added.
  • Floe spec version 1.1 has been introduced.
  • Command line option --help now displays WorkFloe parameters sorted according to order attribute.
  • Command line option --help-all to display all WorkFloe parameters has been added.

Bugfixes

  • An issue that caused port counts for CubeGroup initializers to not increment has been resolved.
  • An issue that caused port counts to be duplicated on connections leaving groups has been resolved.
  • An issue where logging Exception would cause cube failure has been resolved.
  • An issue that prevented modify_parameter from overriding values set by parameter_overrides has been resolved.
  • An issue that caused floes to hang if an initializer port was not consumed has been resolved.
  • An issue that prevented parameter_overrides from setting the value of parameters with many=True has been resolved.

Breaking Changes

  • Parameters that are promoted to the same name, but with different default values, now raise a validation error.

v0.11.0: August 2020

  • Support for Python versions that are less than or equal to 3.6 is being deprecated in the next Orion release.
  • Cube parameters and parameter groups are copied as Cube instance attributes.
  • cube.modify_parameter(cube.param, **fields) checks if param belongs to cube.
  • When promoting a parameter with modify_parameter, setting both promoted_name and promoted is no longer necessary.
  • The documentation of the disk_space and memory_mb parameters has been fixed to reflect the backend implementation. The units of both parameters are mebibytes (MiB == 1048576 B) rather than megabytes (MB). Also, notes regarding taking into account overheads by requesting a couple hundred MiB more than is required have been added.
  • The instance_tags parameter is deprecated and will be removed in the future.

Bugfixes

  • An issue that caused Floes running outside of Orion to not honor item_count has been resolved.
  • An issue causing local parallel Cubes in a cycle to hang has been fixed.
  • An issue preventing large Cube groups from passing validation has been fixed.
  • floe.api.ordering.cube_precedents now correctly returns upstream Cubes.
  • An issue where the max value of Parallel Cube parameter item_count would allow overflows has been resolved and reduced from 100000 to 65535.
  • An issue where sink Cubes would hang when using an initializer has been resolved.
  • An issue where parameters with many and required set would allow no input to be set has been resolved.
  • An issue related to the default validation of parameters with many=False has been resolved.
  • An issue where the required attribute of promoted parameters would be ignored has been fixed.

Breaking Changes

  • cube.parameters() and cube.parameter_groups() methods now list parameters that are attributes of cube instance, not attributes of cube class.
  • A ValidationError is now raised when the name of a Parameter or Port is the same as the name of one of the Cube attributes, which previously caused this Cube attribute to be shadowed unintentionally.

v0.10.3: May 2020

Bugfixes

  • An issue related to memory overflows has been fixed.
  • floe detect now enables the Python fault handler.

v0.10.2: March 2020

Bugfixes

  • Issues with Cube inheritance and constructor parameters are now more thoroughly validated.

v0.10.1: March 2020

Bugfixes

  • An issue that resulted in errors running a Floe due to very slow Python imports has been resolved.

v0.10.0: March 2020

  • CubeTestRunner.set_parameters has been modified to trigger parameter validation and conversion on the underlying Cube in order to support testing parameters which override get_runtime_value.

v0.9.3: March 2020

  • floe detect now has a flag --no-json to not inspect JSON files as WorkFloes.

Bugfixes

  • Validation of ports has been improved.

v0.9.2: February 2020

  • floe lint now incorporates errors and returns a non-zero exit code if it encounters an error.
  • floe.api.parameter has been deprecated in favor of floe.api.parameters. floe.api.parameter will be removed in next major release of Floe package.

Bugfixes

  • An issue that resulted in the loss of parameter groups at package ingestion has been resolved.
  • An issue that resulted in Cubes and WorkFloes not being detected if the same UUID was used.

v0.9.1: January 2020

  • Internal Floe code, which was deprecated in previous Orion releases, and is no longer used, was removed.

v0.9.0: December 2019

  • New parameters, which optionally enable metric collection for Cubes have been added.

v0.8.11: November 2019

  • A new method WorkFloe.check_arguments(), which performs basic validation of runtime arguments without actually running a Floe has been added.

Bugfixes

  • ImportError for Floe parameters when calling WorkFloe.from_json(no_imports=True) is no longer raised.
  • Parameter defaults are now validated when calling WorkFloe.validate(), unless the parameter requires a value to be given.
  • Validation of command line arguments for Floes created with no_imports=True has been improved.
  • A ValidationError is now raised for unknown command line arguments.
  • The WorkFloe UUID in a specification is now included in the WorkFloe object upon instantiation.

v0.8.10: November 2019

  • Removes unused code from Cube parsing logic.

Bugfixes

  • A bug preventing HyperCubes from being defined without a name has been fixed.

v0.8.9: November 2019

  • An error is raised when attempting to emit more than the maximum allowed bytes onto a port.

Bugfixes

  • A bug where Cubes defined with a UUID would be detected once for every instantiation has been fixed.

v0.8.8: October 2019

  • A new command floe html <path/to/file> has been added to generate a simple HTML representation of a Floe.

Bugfixes

  • Avoid importing code from Cubes outside a Cube group
  • Fix exporting of cycle information to json when ProxyCubes are involved
  • An argument has been added to WorkFloe.from_json() in order to optionally remove rank and cycle details when reading a specification so that it can be re-sorted.

v0.8.7: August 2019

  • A function modify_parameter has been added to WorkFloe to give promoted parameters a default name.

v0.8.6: August 2019

  • OutputPort.connect() has been made idempotent.

v0.8.5: July 2019

Bugfixes

  • A bug affecting promoted parameters in a ProxyCube instance is now fixed.

v0.8.4: July 2019

  • Parameter validation has been cleaned up in order to produce clearer messages.
  • Parsing parameters without imports has been enabled.

v0.8.3: July 2019

  • Command line parameter messages have been improved to make it easier to determine which parameters are unsatisfied.

v0.8.2: July 2019

Bugfixes

  • TCP keepalive on Cube to Cube connections has been enabled to avoid idle connections being dropped by firewalls.

v0.8.1: June 2019

Bugfixes

  • A bug has been fixed related to verifying that no Cube in a group is part of a cycle.

v0.8.0: June 2019

  • A simplified syntax has been introduced for Port, Parameter and Cube, which allows instances of these to omit a name upon instantiation.

v0.7.1: June 2019

Bugfixes

  • A bug in the messaging layer related to initializers which would cause upstream Cubes to hang indefinitely has been fixed.

v0.7.0: May 2019

  • The function get_runtime_value() has been added to Parameter as a hook for transforming primitive parameter values into complex objects.
  • A context manager has been added to CubeTestRunner for more concise testing.
  • Parameter.validate_default() has been removed.

v0.6.27: May 2019

Bugfixes

  • A bug with Cube group messaging has been fixed.

v0.6.26: April 2019

  • Floe detection has been improved to catch more issues and display precisely where issues occurred.
  • Duplicate Cube names in a WorkFloe are now rejected.
  • Validation has been added to WorkFloe to ensure that all ports with required=True are used.
  • min_parallel default value of 1 has been reduced to 0.

Bugfixes

  • An issue where a required parameter with choices would require that the default is in the choices has been fixed. None is now be a valid choice.

v0.6.25: March 2019

Bugfixes

  • An issue where validate_default() for StringParameter incorrectly handled a default value of None has been fixed.
  • A description has been added to the min_parallel and max_parallel parameters. This bug resulted in needless warnings when running floe lint <path>.

v0.6.24: March 2019

Bugfixes

  • A data buffering issue which resulted in corrupted/lost messages during high network traffic has been fixed.

v0.6.23: February 2019

  • set_initializer_input and set_parameters functions have been added to CubeTestRunner.

Bugfixes

  • A bug causing cyclic WorkFloes to hang has been fixed.
  • A defined variable bug in cube_status and logs unexpected Cube type has been fixed.

v0.6.22: February 2019

  • Default memory requirement of Cubes has been reduced to 1800Mb.

v0.6.21: February 2019

  • Support for many parameters, which are implemented as lists of the underlying parameter type, has been added.

v0.6.20: January 2019

Bugfixes

  • An issue where a routine parallel worker exit in a locally-running Floe leading to Floe failure has been fixed.

v0.6.19: December 2018

Bugfixes

  • A bug causing ProxyCube instances to have their specifications cleared has been fixed.

v0.6.18: November 2018

Bugfixes

  • An issue where Cube parameter_overrides weren’t being honored locally has been fixed.

v0.6.17: November 2018

  • Adds default value of 1000 for max_parallel parameter on Parallel Cubes

Bugfixes

  • A bug which caused failures when running Floes locally in Windows has been fixed.

v0.6.16: October 2018

  • WorkFloe specification validation has been improved.

v0.6.15: October 2018

Bugfixes

  • A bug where parallel workers would incorrectly propagate failures when run in Orion has been fixed.

v0.6.14: October 2018

Bugfixes

  • A bug with promoted parameter titles in HyperCubes has been fixed.

v0.6.13: October 2018

  • WorkFloe specification validation has been improved.

v0.6.12: September 2018

  • Validation errors in parameters have been improved to include parameter names.

Bugfixes

  • A bug causing data overflows in downstream Cubes has been fixed.

v0.6.11: September 2018

Bugfixes

  • A bug causing parallel Floes to hang in Orion has been fixed.

v0.6.10: September 2018

Bugfixes

  • A bug affecting parallel groups with multiple initializer ports has been fixed.

v0.6.9: August 2018

Bugfixes

  • A bug causing data overflows in downstream Cubes has been fixed.

v0.6.8: August 2018

  • Paths containing WorkFloes with duplicate names are now rejected by floe detect <path>.

Bugfixes

  • A bug where non-group inputs to parallel Cube groups could have incorrect counts has been fixed.
  • A bug related to promoting a ports in nested HyperCubes has been fixed.

v0.6.6: August 2018

Bugfixes

  • A bug where None was not accepted as a default value for a DecimalParameter instance with required=True has been fixed.

v0.6.5: August 2018

  • Support for promoting separate ports within a HyperCube instance to the same name has been added.

Bugfixes

  • A bug which caused initializer ports connected to multiple ports to hang indefinitely has been fixed.

v0.6.4: August 2018

  • Validation for staged WorkFloes has been added

Bugfixes

  • JSONParameter has been made compatible with Python 3

v0.6.3: August 2018

Bugfixes

  • A bug where no a Floe with no Cubes would hang indefinitely has been fixed.

v0.6.2: July 2018

Bugfixes

  • An issue with termination detection in cyclic Floes has been fixed.

v0.6.1: July 2018

  • Error handling in local Floe-running code has been improved so that Floes exit properly in order to avoid spawning zombie processes.

v0.6.0: July 2018

  • Default parameter values are now validated. This is a breaking change that affects a custom parameter’s prepare() method. See Parameter documentation for details.
  • WorkFloe.json() has been made to validate that valid JSON will be produced.

Bugfixes

  • A bug relating to intake port counts has been fixed.

v0.5.19: July 2018

  • Several issues related to mixed packages have been fixed.

v0.5.18: July 2018

  • Several issues related to mixed packages have been fixed.

v0.5.17: June 2018

  • Several issues related to mixed packages have been fixed.

v0.5.16: June 2018

  • The connection of multiple ports to an initializer port has been enabled.

v0.5.15: June 2018

Bugfixes

  • A bug relating to promoting a port in a HyperCube nested within another HyperCube has been fixed.

v0.5.14: June 2018

  • The title attribute has been added to WorkFloe JSON specification.

v0.5.13: June 2018

  • WorkFloe sorting logic has been improved.
  • Error detection/reporting on Floe/Cube parameters has been improved.
  • Support for initializer ports has been added.

Bugfixes

  • Several HyperCube bugs have been fixed.
  • A bug relating to mixed (i.e. a package containing Cubes from other packages) Floe package parsing has been fixed.
  • Several bugs in local-Floe running resulting in zombie processes have been fixed.
  • A bug in termination detection for cyclic Floes has been fixed.

v0.2.182: Oct 2017

  • The ability to override additional parameter fields in a Cube subclass has been added.

Bugfixes

  • A bug where termination was not properly detected in certain cyclic WorkFloes has been fixed.

v0.2.181: Oct 2017

  • The ability to make Floe ignore Cubes & WorkFloes has been added. Prefix the name of a subclass of Cube or WorkFloe with an underscore to indicate that Floe should skip it.
  • An attribute, title, has been added to Port.
  • A warning is now provided by the linter for promoted parameter name collisions.
  • A method for handling parallel Cube failures, ParallelMixin.process_failure() has been added.
  • A warning is now provided by the linter if no files were linted.
  • The class ParallelMixin has been created to simplify the definition of parallel/serial Cubes.

Bugfixes

  • A bug where parameters in a subclass of Cube could shadow parameters of the base class has been fixed.
  • A bug in dynamic imports related to custom packages has been fixed.