Depends
Workflow management for research and visual effects.
A Flexible, Easy-To-Use Workflow Manager
Depends is an open source scientific workflow management system tuned specifically to address the needs of researchers and visual effects artists. As with many scientific workflow managers, Depends focuses on ease and clarity of use, the preservation of data provenance, and compatibility with existing software and pipelines. It does this by providing an interface to organize a collection of preexisting command-line applications into a directed acyclic dependency graph (DAG), concurrently maintaining a list of available data as a scenegraph. Depends is written using Python 2.6.6 and the pyside bindings for the Qt windowing toolkit, and runs under multiple operating systems.
Depends is flexible in that it can incorporate existing applications into its dependency graph without the need to change their behavior. Using a lightweight Python interface, a command-line program can be executed by simply gathering a list of parameters from the Depends user interface and passing them to the program, or as complex as the Python programming language will allow. This flexibility is key to making Depends useful for individuals or groups of individuals in various production and research facilities.
Depends simplifies many of the tedius aspects of traditional workflow management without taking too much control away from the user. Scroll down for a list of ways Depends may help you solve your workflow management challenges.Strict Typing
Dependency nodes are defined with explicit input and output data types. Depends alerts the user when these types don't match, potentially saving time.
Execution Validation
A large range of checks are executed before sending a series of time-consuming operations to the execution environment, helping the user avoid costly mistakes.
Built-in Versioning
Versioning workflow progress is essential to gleaning the provenance of data long after it was generated. Depends has simple mechanisms to make versioning natural and easy.
Stale Data Tracking
If a change occurs upstream in the dependency graph, each node that depends on the data must also be re-executed. Depends makes these dependencies clear.
File Linting
Depends checks the presence and validity of files on disk. This insures files that are presumed to be present are, in fact, present.
Sequence Types
Not only are individual file types checked, but sequences of related files are also rolled up into types, further simplifying workflow management.
Use Any Existing Software
Depends' flexible Python API allows the user to add nearly any program that can be run from the commandline to a workflow in a flexible and quick manner.
Execution Order Guarantees
Depends uses its directed acyclic graph to insure processes are executed in the proper order - no matter how complex the dependencies become.
External Software Interfaces
Depends has a socket API that allows it to quickly visualize and manipulate data in external programs such as Autodesk's Maya or 3d Studio Max.
Flexible Execution
Workflows can be executed in the environment of your choice - whether it be on your local machine or using a process management software on a farm.
Developer Tools
Depends recognizes that creating one's own nodes can be an iterative process. Functionality is provided to make this as painless as possible.
Internal Variables
Depends maintains a list of internal "environment variables" that can be set from the command-line or in its internal variable editor.
Native Process Wedging
Depends' environment variable system along with its commandline interface allows for simple wedging of various parameters in a workflow.
Plaintext Workflows
Each wokflow is saved to disk using JSON text files, facilitating inspection and manipulation outside the Depends user interface.
Plugin system
Depends allow the user to change the execution environment, file browser, node definitions, and data types using a simple Python plugin interface.