Literature Studies

Page summary

Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). Please extend this list by creating a new page and adding a link below. Use the guidelines described here.

1 Results from Literature

1.1 Classification

2003

• Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data

2005

• A review and comparison of classification algorithms for medical decision making

2016

• Predicting Breast Cancer Survivability Using Data Mining Techniques

1.2 Selection of Differential Features and Regions

1.2.1 Identifying differential features

2006

• Rat toxicogenomic study reveals analytical consistency across microarray platforms

2010

• A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium

2017

• Identification of differentially expressed peptides in high-throughput proteomics data
• In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values
• Strategies for analyzing bisulfite sequencing data

2018

• Identification of Differentially Methylated Sites with Weak Methylation Effects

1.2.2 Identifying differential regions (e.g. DMRs)

2015

• De novo identification of differentially methylated regions in the human genome
• MethylAction: detecting differentially methylated regions that distinguish biological subtypes
• metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data

2016

• seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data
• Statistical methods for detecting differentially methylated regions based on MethylCap-seq data

2017

• DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data

2018

• Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus
• DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts
• MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)

1.2.3 Identifying sets of features (e.g. gene set analyses)

2009

• A general modular framework for gene set enrichment analysis
• Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16

2018

• Gene set analysis methods: a systematic comparison

1.2.4 Dimension reduction

2008

• On the Relationship Between Feature Selection and Classification Accuracy

2015

• Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data

1.3 Imputation methods for missing values

2001

• Missing value estimation methods for DNA microarrays

2002

• Imputation of missing longitudinal data: a comparison of methods

2008

• Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes

2011

• Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline

2014

• Recursive partitioning for missing data imputation in the presence of interaction effects.

2015

• Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics.

2016

• Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies
• Multiple imputation and analysis for high-dimensional incomplete proteomics data

2018

• Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data

1.4 ODE-based Modelling

2001

• Ways to Fit a PK Model with Some Data Below the Quantification Limit

2008

• Hybrid optimization method with general switching strategy for parameter estimation

2011

• Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis

2013

• Lessons Learned from Quantitative Dynamical Modeling in Systems Biology
• ODE parameter inference using adaptive gradient matching with Gaussian processes

2017

• Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems
• Data-driven reverse engineering of signaling pathways using ensembles of dynamic models
• Fast derivatives of likelihood functionals for ODE based models using adjoint-state method
• Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy
• Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks

2018

• Continuous analogue to iterative optimization for PDE-constrained inverse problems
• Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology
• Hierarchical optimization for the efficient parametrization of ODE models
• Inference for differential equation models using relaxation via dynamical systems
• Input-dependent structural identifiability of nonlinear systems
• Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis
• Performance of objective functions and optimization procedures for parameter estimation in system biology models

2019

• A comparison of methods for quantifying prediction uncertainty in systems biology
• Benchmark problems for dynamic modeling of intracellular processes
• Benchmarking optimization methods for parameter estimation in large kinetic models
• Efficient computation of steady states in large-scale ODE models of biochemical reaction networks
• Full observability and estimation of unknown inputs, states and parameters of nonlinear biological models
• Mini-batch optimization enables training of ODE models on large-scale datasets
• Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach
• Parameter estimation in models of biological oscillators: an automated regularised estimation approach
• Robust calibration of hierarchical population models for heterogeneous cell populations
• Scalable nonlinear programming framework for parameter estimation in dynamic biological system models
• Testing structural identifiability by a simple scaling method
• Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations

2020

• An application of Conditional RobustCalibration (CRC) to ordinary differential equations (ODEs) models in computational systems biology: a comparison of two sampling strategies
• Efficient parameterization of large-scale dynamic models based on relative measurements

1.5 Omics Workflows

2015

• ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines

2017

• A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation
• A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies

2019

• A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines
• Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays

1.6 Preprocessing high-throughput data

Year	Country
2000	First Author	Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions
2001	Japan
1999	Germanay

2003

• A comparison of normalization methods for high density oligonucleotide array data based on variance and bias

2005

• Comparison of Affymetrix GeneChip Expression Measures
• Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays

2006

• Using RNA sample titrations to assess microarray platform performance and normalization techniques
• Normalization regarding non-random missing values in high-throughput mass spectrometry data

2007

• Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data

2008

• Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix

2009

• Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations

2010

• Consistency of predictive signature genes and classifiers generated using different microarray platforms
• Detecting and correcting systematic variation in large-scale RNA sequencing data
• Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments
• Normalization of RNA-seq data using factor analysis of control genes or samples

2011

• Affymetrix GeneChip microarray preprocessing for multivariate analyses

2012

• A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis

2014

• Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets
• Prevention, diagnosis and treatment of high-throughput sequencing data pathologies

2015

• Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions
• Optimization of miRNA-seq data preprocessing

2016

• Chemometric methods in data processing of mass spectrometry-based metabolomics: A review

2018

• Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening