Optimization of media processing workflows with adaptive operator behaviors

In this paper, we present the ARIA media processing workflow architecture that processes, filters, and fuses sensory inputs and actuates responses in real-time. The components of the architecture are programmable and adaptable; i.e. the delay, size, and quality/precision characteristics of the individual operators can be controlled via a number of parameters. Each data object processed by qStream components is subject to transformations based on the parameter values. For instance, the quality of an output data object and the corresponding processing delay and resource usage depend on the values assigned to parameters of the operators in the object flow path. In Candan, Peng, Ryu, Chatha, Mayer (Efficient stream routing in quality- and resource-adaptive flow architectures. In: Workshop on multimedia information systems, 2004), we introduced a class of flow optimization problems that promote creation and delivery of small delay or small resource-usage objects to the actuators in single-sensor, single-actuator workflows. In this paper, we extend our attention to multi-sensor media processing workflow scenarios. The algorithms we present take into account the implicit dependencies between various system parameters, such as resource consumption and object sizes. We experimentally show the effectiveness and efficiency of the algorithms.