A multifactorial evaluation framework for gene regulatory network reconstruction

In the past years, many computational methods have been developed to infer the structure of gene regulatory networks from time series data. However, the applicability and accuracy presumptions of such algorithms remain unclear due to experimental heterogeneity. This paper assesses the performance of recent and successful network inference strategies under a novel, multifactorial evaluation framework in order to highlight pragmatic tradeoffs in experimental design. The effects of data quantity and systems perturbations are addressed, thereby formulating guidelines for efficient resource management. Realistic data were generated from six widely used benchmark models of rhythmic and non-rhythmic gene regulatory systems with random perturbations mimicking the effect of gene knock-out or chemical treatments. Then, time series data of increasing lengths were provided to five state-of-the-art network inference algorithms representing distinctive mathematical paradigms. The performances of such network reconstruction methodologies are uncovered under various experimental conditions. We report that the algorithms do not benefit equally from data increments. Furthermore, at least for the studied rhythmic system, it is more profitable for network inference strategies to be run on long time series rather than short time series with multiple perturbations. By contrast, for the non-rhythmic systems, increasing the number of perturbation experiments yielded better results than increasing the sampling frequency. We expect that future benchmark and algorithm design would integrate such multifactorial considerations to promote their widespread and conscientious usage.