Using the Recovery Dataset for MD Floes

The recovery dataset (default name md_recovery) is a helpful output that can be used by the Extend MD Simulations Floe to continue unfinished jobs. This dataset is created after a cycle or iteration has been completed in the MD Floes. Cycles are dictated by the Checkpoint Interval (hours) parameter in the FE-NES for Binding Affinity Prediction Floe (Cube Max Run Time parameter in other MD floes in the MD Affinity package). This parameter is set to one hour by default. After each cycle, the recovery dataset is updated with the latest MD data. If this parameter is lowered, the recovery dataset will be updated more frequently; however, fewer MD steps will be performed per cycle. This parameter cannot be increased past 11 hours without significant risk of job failure because of the AWS 12-hour time limit.

When is the recovery dataset written?

By default, the recovery dataset is not written by these floes. This leaves storage free for other outputs intended for long-term use.

When a job nears completion (the calculated number of cycles is completed for the number of MD steps), the data in the recovery dataset is overwritten to an output dataset (e.g., md_output dataset). If the Parallel Iteration Checker and MD Stage Concatenation Cubes have finished running, then the recovery dataset is not useful any more and is deleted. However, if a job is canceled, terminated, or fails before these cubes have finished, then the recovery dataset is saved. It can be used as input to Extend MD Simulations Floe.

Note

The run time and number of MD steps per cycle are highly dependent upon the size of your flask (system). If the default value (1 hour) is used for the Cube Max Run Time parameter and the number of atoms of your flask (system) is smaller than 200,000, it sets the per-cycle runtime shorter than 1 hour to have shorter per-cycle runtimes for small flasks.

The recovery dataset can be used as input for the Extend MD Simulations Floe to continue MD simulations. All parameters that were set in the previous floe that created the recovery dataset will be maintained for the Extend MD Simulations Floe.

Note

The recovery dataset output from the Short Trajectory MD with Analysis Floe currently only holds information for the bound MD simulation. It does not contain information for the unbound simulation.

Example Case:

A job is submitted to run for 50 ns; the number of cycles/iterations needed to complete the MD simulation is calculated to be 50 cycles based on the size of the system and other parameters. After one cycle, the recovery dataset has been built and continues to grow after each iteration. The job reaches a cost limit after 30 cycles and is terminated. The recovery dataset can be used as input for the Extend MD Simulations Floe, and all of the original job settings will be applied to override the default settings of the Extend MD Simulations Floe. The job will continue until the 50 iterations are complete (starting at the 30th iteration), producing a new md_output dataset. If an additional 50 ns is desired, the md_output dataset can be used as the input for the Extend MD Simulations Floe. However, new settings must be provided for the new run because MD parameters and settings are not carried over into the new MD simulations.