Difference between revisions of "Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus"
(→General aspects) |
(Added text) |
||
| Line 4: | Line 4: | ||
[https://doi.org/10.1186/s12859-018-2037-1 https://doi.org/10.1186/s12859-018-2037-1] | [https://doi.org/10.1186/s12859-018-2037-1 https://doi.org/10.1186/s12859-018-2037-1] | ||
| − | |||
=== Summary === | === Summary === | ||
| Line 10: | Line 9: | ||
=== Study outcomes === | === Study outcomes === | ||
| − | |||
==== Outcome O1 ==== | ==== Outcome O1 ==== | ||
| − | The performance of . | + | The performance of Defiant, MethylKit, MethylSig, Metilene, RADmeth and RnBeads was compared in terms of precision, recall and false negatives. The following result is obtained: |
| + | * Metilene and Defiant clearly outperform MethylKit, MethylSig, RADmeth and RnBeads | ||
| + | * Sometimes precision and recall equal to zero is obtained which might indicate switched class labels. | ||
| − | Outcome O1 is presented as Figure | + | Outcome O1 is presented as Figure 3 in the original publication. |
| + | Since "Defiant" is occuring twice in the figure legend and RnBeads is missing, we guess that the magenta markers belong to RnBead (i.e. like the colors chosen in Fig. 4). | ||
==== Outcome O2 ==== | ==== Outcome O2 ==== | ||
| − | . | + | The performance of Defiant, MethylKit, MethylSig, Metilene, RADmeth and RnBeads was compared in terms of the F1-score which is the harmonic mean of precision and recall. The following outcome is observed: |
| + | * Metilene is superior | ||
| + | * Metilene and Defiant clearly outperform MethylKit, MethylSig, RADmeth and RnBeads | ||
| − | Outcome O2 is presented as Figure | + | Outcome O2 is presented as Figure 4 in the original publication. |
==== Outcome On ==== | ==== Outcome On ==== | ||
| Line 34: | Line 37: | ||
* The paper presents a new approach (defiant) and at the same times provides several analyses for comparing the performance of the new approach with existing algorithms. Such a study setting is very frequently found in the literature but has a high risk for biased outcomes. One reason for such a bias might be that typically application examples are selected to nicely demonstrate performance benefits. Moreover, new approaches are often established if existing methods have minor performance in a new application setup. For such a setup, a new approach then has good chances to outperform and it remains rather unclear how performance comparisons translates to new application settings. | * The paper presents a new approach (defiant) and at the same times provides several analyses for comparing the performance of the new approach with existing algorithms. Such a study setting is very frequently found in the literature but has a high risk for biased outcomes. One reason for such a bias might be that typically application examples are selected to nicely demonstrate performance benefits. Moreover, new approaches are often established if existing methods have minor performance in a new application setup. For such a setup, a new approach then has good chances to outperform and it remains rather unclear how performance comparisons translates to new application settings. | ||
| − | ==== Design for Outcome O1 ==== | + | ==== Design for Outcome O1 and O2 ==== |
* 16 "benchmark" data sets were analyzed taken from [27] | * 16 "benchmark" data sets were analyzed taken from [27] | ||
* Configuration parameters were chosen ... | * Configuration parameters were chosen ... | ||
* ... | * ... | ||
| + | |||
==== Design for Outcome O2 ==== | ==== Design for Outcome O2 ==== | ||
| − | * The outcome | + | * The outcome of O1 might indicate switched class labels. The outcome O2 would also be in agreement with switched class labels because also F1-score equals to zero occurs. |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
* Configuration parameters were chosen ... | * Configuration parameters were chosen ... | ||
* ... | * ... | ||
Revision as of 14:20, 25 January 2019
Contents
1 Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus
David E. Condon, Phu V. Tran, Yu-Chin Lien, Jonathan Schug, Michael K. Georgieff, Rebecca A. Simmons and Kyoung-Jae Won, Defiant: (DMRs: easy, fast, identification and ANnoTation) identifies differentially Methylated regions from iron-deficient rat hippocampus, 2018, BMC Bioinformatics, 19:31.
https://doi.org/10.1186/s12859-018-2037-1
1.1 Summary
The paper considers identification of differentially methylated regions (DMRs) from bisulfite sequencing data (BSSEQ). A new package (defiant) is introduced. The paper claims that shows superior performance to other approaches as shown in analyses of a series of benchmarking tests on artificial and real data.
1.2 Study outcomes
1.2.1 Outcome O1
The performance of Defiant, MethylKit, MethylSig, Metilene, RADmeth and RnBeads was compared in terms of precision, recall and false negatives. The following result is obtained:
- Metilene and Defiant clearly outperform MethylKit, MethylSig, RADmeth and RnBeads
- Sometimes precision and recall equal to zero is obtained which might indicate switched class labels.
Outcome O1 is presented as Figure 3 in the original publication. Since "Defiant" is occuring twice in the figure legend and RnBeads is missing, we guess that the magenta markers belong to RnBead (i.e. like the colors chosen in Fig. 4).
1.2.2 Outcome O2
The performance of Defiant, MethylKit, MethylSig, Metilene, RADmeth and RnBeads was compared in terms of the F1-score which is the harmonic mean of precision and recall. The following outcome is observed:
- Metilene is superior
- Metilene and Defiant clearly outperform MethylKit, MethylSig, RADmeth and RnBeads
Outcome O2 is presented as Figure 4 in the original publication.
1.2.3 Outcome On
...
Outcome On is presented as Figure X in the original publication.
1.2.4 Further outcomes
If intended, you can add further outcomes here.
1.3 Study design and evidence level
1.3.1 General aspects
- The paper presents a new approach (defiant) and at the same times provides several analyses for comparing the performance of the new approach with existing algorithms. Such a study setting is very frequently found in the literature but has a high risk for biased outcomes. One reason for such a bias might be that typically application examples are selected to nicely demonstrate performance benefits. Moreover, new approaches are often established if existing methods have minor performance in a new application setup. For such a setup, a new approach then has good chances to outperform and it remains rather unclear how performance comparisons translates to new application settings.
1.3.2 Design for Outcome O1 and O2
- 16 "benchmark" data sets were analyzed taken from [27]
- Configuration parameters were chosen ...
- ...
1.3.3 Design for Outcome O2
- The outcome of O1 might indicate switched class labels. The outcome O2 would also be in agreement with switched class labels because also F1-score equals to zero occurs.
- Configuration parameters were chosen ...
- ...
1.4 Further comments and aspects
1.5 References
[27] Jühling F, Kretzmer H, Bernhart SH, Otto C, Stadler PF, Hoffmann S. Metilene: fast and sensitive calling of differentially methylated regions from bisulfite sequencing data. Genome Res. 2015; https://doi.org/10.1101/gr.196394.11
