Literature Studies

Page summary

Here outcomes of benchmarking studies from the literature are collected. The primary aim is a comprehensive overview about neutral benchmark studies, i.e. assessments which were performed independenty on publication of a new approach. Studies which are not neutral are put in brackets. The focus is on computational methods for analyzing experimental data (instead of comparing experimental techniques or platforms). Please extend this list by creating a new page and adding a link below. Use the guidelines described here.

1 Results from Literature

1.1 Classification

2003

• Comparison of statistical methods for classification of ovarian cancer using mass spectrometry data

2005

• A review and comparison of classification algorithms for medical decision making

2016

• Predicting Breast Cancer Survivability Using Data Mining Techniques

1.2 Selection of Differential Features and Regions

1.2.1 Identifying differential features

2006

• Rat toxicogenomic study reveals analytical consistency across microarray platforms

2010

• A comprehensive assessment of RNA-seq accuracy, reproducibility and information content by the sequencing Quality control consortium

2017

• Identification of differentially expressed peptides in high-throughput proteomics data
• In-depth method assessments of differentially expressed protein detection for shotgun proteomics data with missing values
• Strategies for analyzing bisulfite sequencing data

2018

• Identification of Differentially Methylated Sites with Weak Methylation Effects

1.2.2 Identifying differential regions (e.g. DMRs)

2015

• De novo identification of differentially methylated regions in the human genome
• MethylAction: detecting differentially methylated regions that distinguish biological subtypes
• metilene: Fast and sensitive calling of differentially methylated regions from bisulfite sequencing data

2016

• seqlm: an MDL based method for identifying differentially methylated regions in high density methylation array data
• Statistical methods for detecting differentially methylated regions based on MethylCap-seq data

2017

• DMRfinder: efficiently identifying differentially methylated regions from MethylC-seq data

2018

• Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus
• DMRcaller: a versatile R/Bioconductor package for detection and visualization of differentially methylated regions in CpG and non-CpG contexts
• MethCP: Differentially Methylated Region Detection with Change Point Models (bioRxiv)

1.2.3 Identifying sets of features (e.g. gene set analyses)

2009

• A general modular framework for gene set enrichment analysis
• Comparing gene set analysis methods on single-nucleotide polymorphism data from Genetic Analysis Workshop 16

2018

• Gene set analysis methods: a systematic comparison

1.2.4 Dimension reduction

2008

• On the Relationship Between Feature Selection and Classification Accuracy

2015

• Comparing feature selection methods for highdimensional imbalanced data: identifying rheumatoid arthritis cohorts from routine data

1.3 Imputation methods for missing values

1996

• Partially parametric techniques for multiple imputation (Schenker)

1999

• Imputing Missing Data for Gene Expression Arrays

2001

• Missing value estimation methods for DNA microarrays (Troyanskaya)

2002

• Imputation of missing longitudinal data: a comparison of methods

2003

• A Bayesian missing value estimation method for gene expression profile data (Oba)

2005

• Nonlinear PCA: a missing data approach (Scholz)

2007

• pcaMethods—a bioconductor package providing PCA methods for incomplete data (Stacklies)
• Sequential imputation for missing values (Verboven)

2008

• Which missing value imputation method to use in expression profiles: a comparative study and two selection schemes

2011

• Iterative stepwise regression imputation using standard and robust methods (Templ)
• Missing values in mass spectrometry based metabolomics: an undervalued step in the data processing pipeline

2012

• MissForest—non-parametric missing value imputation for mixed-type data (Stekhoven)

2013

• Accounting for undetected compounds in statistical analyses of mass spectrometry ‘omic studies (Taylor)
• Comparison of imputation methods for missing laboratory data in medicine (Waljee)

2014

• Comparison of Random Forest and Parametric Imputation Models for Imputing Missing Data Using MICE: A CALIBER Study (Shah)
• Comparison of methods for imputing limited-range variables: a simulation study (Rodwell)
• Tuning multiple imputation by predictive mean matching and local residual draws (Morris)
• Recursive partitioning for missing data imputation in the presence of interaction effects.

2015

• Review, evaluation, and discussion of the challenges of missing value imputation for mass spectrometry-based label-free global proteomics.

2016

• Accounting for the Multiple Natures of Missing Values in Label-Free Quantitative Proteomics Data Sets to Compare Imputation Strategies
• Multiple imputation and analysis for high-dimensional incomplete proteomics data

2018

• Missing Value Imputation Approach for Mass Spectrometry-based Metabolomics Data (Wei)
• Gap-filling a spatially explicit plant trait database: comparing imputation methods and different levels of environmental information (Poyatos)
• The effects of nonignorable missing data on label-free mass spectrometry proteomics experiments (O'Brien)

1.4 ODE-based Modelling

2001

• Ways to Fit a PK Model with Some Data Below the Quantification Limit

2008

• Hybrid optimization method with general switching strategy for parameter estimation

2011

• Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis

2013

• Lessons Learned from Quantitative Dynamical Modeling in Systems Biology
• ODE parameter inference using adaptive gradient matching with Gaussian processes

2017

• Comprehensive benchmarking of Markov chain Monte Carlo methods for dynamical systems
• Data-driven reverse engineering of signaling pathways using ensembles of dynamic models
• Fast derivatives of likelihood functionals for ODE based models using adjoint-state method
• Parameter estimation in large-scale systems biology models: a parallel and self-adaptive cooperative strategy
• Performance of objective functions and optimization procedures for parameter estimation in system biology models
• Scalable Parameter Estimation for Genome-Scale Biochemical Reaction Networks

2018

• Evaluation of Derivative-Free Optimizers for Parameter Estimation in Systems Biology
• Hierarchical optimization for the efficient parametrization of ODE models
• Inference for differential equation models using relaxation via dynamical systems
• Input-dependent structural identifiability of nonlinear systems
• Optimization and profile calculation of ODE models using second order adjoint sensitivity analysis

2019

• A comparison of methods for quantifying prediction uncertainty in systems biology
• Benchmark problems for dynamic modeling of intracellular processes
• Benchmarking optimization methods for parameter estimation in large kinetic models
• Efficient computation of steady states in large-scale ODE models of biochemical reaction networks
• Full observability and estimation of unknown inputs, states and parameters of nonlinear biological models
• Mini-batch optimization enables training of ODE models on large-scale datasets
• Parameter Estimation and Variable Selection for Big Systems of Linear Ordinary Differential Equations: A Matrix-Based Approach
• Parameter estimation in models of biological oscillators: an automated regularised estimation approach
• Robust calibration of hierarchical population models for heterogeneous cell populations
• Scalable nonlinear programming framework for parameter estimation in dynamic biological system models
• Testing structural identifiability by a simple scaling method
• Tracking for parameter and state estimation in possibly misspecified partially observed linear Ordinary Differential Equations

2020

• An application of Conditional RobustCalibration (CRC) to ordinary differential equations (ODEs) models in computational systems biology: a comparison of two sampling strategies
• Efficient parameterization of large-scale dynamic models based on relative measurements

1.5 Omics Workflows

2015

• ComparingVariant Call Files for Performance Benchmarkingof Next-Generation Sequencing Variant Calling Pipelines

2017

• A comprehensive evaluation of popular proteomics software workflows for label-free proteome quantification and imputation
• A benchmarking of workflows for detecting differential splicing and differential expression at isoform level in human RNA-seq studies

2019

• A Systematic Evaluation of Single CellRNA-Seq Analysis Pipelines
• Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays

Year	First Author	Title
2003		A comparison of normalization methods for high density oligonucleotide array data based on variance and bias

1.6 Preprocessing high-throughput data

Year	First Author	Title
2003		A comparison of normalization methods for high density oligonucleotide array data based on variance and bias
2005		Comparison of Affymetrix GeneChip Expression Measures
2005	Meleth S	The case for well-conducted experiments to validate statistical protocols for 2D gels: different pre-processing = different lists of significant proteins
2005		Comparison of background correction and normalization procedures for high-density oligonucleotide microarrays
2006		Using RNA sample titrations to assess microarray platform performance and normalization techniques
2006	Wang P	Normalization regarding non-random missing values in high-throughput mass spectrometry data
2007	Carvalho B	Exploration, normalization, and genotype calls of high-density oligonucleotide SNP array data
2007	Cannataro M	MS‐Analyzer: preprocessing and data mining services for proteomics applications on the Grid
2008		Comparison of preprocessing methods for the hgU133+2 chip from Affymetrix
2009		Comparison of Affymetrix data normalization methods using 6,926 experiments across five array generations
2010		Consistency of predictive signature genes and classifiers generated using different microarray platforms
2010		Detecting and correcting systematic variation in large-scale RNA sequencing data
2010		Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments
2010		Normalization of RNA-seq data using factor analysis of control genes or samples
2011		Affymetrix GeneChip microarray preprocessing for multivariate analyses
2012		A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis
2014		Normalyzer: A Tool for Rapid Evaluation of Normalization Methods for Omics Data Sets
2014	Zhou X	Prevention, diagnosis and treatment of high-throughput sequencing data pathologies
2015	Caraus I	Detecting and overcoming systematic bias in high-throughput screening technologies: a comprehensive review of practical issues and methodological solutions
2015	Tam S	Optimization of miRNA-seq data preprocessing
2016	Yi L	Chemometric methods in data processing of mass spectrometry-based metabolomics: A review
2018	Mazoure B	Identification and Correction of Additive and Multiplicative Spatial Biases in Experimental High-Throughput Screening