PrecisionFDA Truth Challenge V2: Calling variants from short and long reads in difficult-to-map regions

Nathan D. Olson; Justin Wagner; Jennifer McDaniel; Sarah H. Stephens; Samuel T. Westreich; Anish G. Prasanna; Elaine Johanson; Emily Boja; Ezekiel J. Maier; Omar Serang; David Jáspez; José M. Lorenzo-Salazar; Adrián Muñoz-Barrera; Luis A. Rubio-Rodríguez; Carlos Flores; Konstantinos Kyriakidis; Andigoni Malousi; Kishwar Shafin; Trevor Pesout; Miten Jain; Benedict Paten; Pi-Chuan Chang; Alexey Kolesnikov; Maria Nattestad; Gunjan Baid; Sidharth Goel; Howard Yang; Andrew Carroll; Robert Eveleigh; Mathieu Bourgey; Guillaume Bourque; Gen Li; ChouXian MA; LinQi Tang; YuanPing DU; ShaoWei Zhang; Jordi Morata; Raúl Tonda; Genís Parra; Jean-Rémi Trotta; Christian Brueffer; Sinem Demirkaya-Budak; Duygu Kabakci-Zorlu; Deniz Turgut; Özem Kalay; Gungor Budak; Kübra Narcı; Elif Arslan; Richard Brown; Ivan J. Johnson; Alexey Dolgoborodov; Vladimir Semenyuk; Amit Jain; H. Serhat Tetikol; Varun Jain; Mike Ruehle; Bryan Lajoie; Cooper Roddey; Severine Catreux; Rami Mehio; Mian Umair Ahsan; Qian Liu; Kai Wang; Sayed Mohammad Ebrahim Sahraeian; Li Tai Fang; Marghoob Mohiyuddin; Calvin Hung; Chirag Jain; Hanying Feng; Zhipan Li; Luoqi Chen; Fritz J. Sedlazeck; Justin M Zook

doi:10.1016/j.xgen.2022.100129

PrecisionFDA Truth Challenge V2: Calling variants from short and long reads in difficult-to-map regions

Nathan D. Olson, Justin Wagner, Jennifer McDaniel, Sarah H. Stephens, Samuel T. Westreich, Anish G. Prasanna, Elaine Johanson, Emily Boja, Ezekiel J. Maier, Omar Serang, David Jáspez, José M. Lorenzo-Salazar, Adrián Muñoz-Barrera, Luis A. Rubio-Rodríguez, Carlos Flores, Konstantinos Kyriakidis, Andigoni Malousi, Kishwar Shafin, Trevor Pesout, Miten JainBenedict Paten, Pi-Chuan Chang, Alexey Kolesnikov, Maria Nattestad, Gunjan Baid, Sidharth Goel, Howard Yang, Andrew Carroll, Robert Eveleigh, Mathieu Bourgey, Guillaume Bourque, Gen Li, ChouXian MA, LinQi Tang, YuanPing DU, ShaoWei Zhang, Jordi Morata, Raúl Tonda, Genís Parra, Jean-Rémi Trotta, Christian Brueffer, Sinem Demirkaya-Budak, Duygu Kabakci-Zorlu, Deniz Turgut, Özem Kalay, Gungor Budak, Kübra Narcı, Elif Arslan, Richard Brown, Ivan J. Johnson, Alexey Dolgoborodov, Vladimir Semenyuk, Amit Jain, H. Serhat Tetikol, Varun Jain, Mike Ruehle, Bryan Lajoie, Cooper Roddey, Severine Catreux, Rami Mehio, Mian Umair Ahsan, Qian Liu, Kai Wang, Sayed Mohammad Ebrahim Sahraeian, Li Tai Fang, Marghoob Mohiyuddin, Calvin Hung, Chirag Jain, Hanying Feng, Zhipan Li, Luoqi Chen, Fritz J. Sedlazeck, Justin M Zook

Research output: Contribution to journal › Article › peer-review

Abstract

The precisionFDA Truth Challenge V2 aimed to assess the state of the art of variant calling in challenging genomic regions. Starting with FASTQs, 20 challenge participants applied their variant-calling pipelines and submitted 64 variant call sets for one or more sequencing technologies (Illumina, PacBio HiFi, and Oxford Nanopore Technologies). Submissions were evaluated following best practices for benchmarking small variants with updated Genome in a Bottle benchmark sets and genome stratifications. Challenge submissions included numerous innovative methods, with graph-based and machine learning methods scoring best for short-read and long-read datasets, respectively. With machine learning approaches, combining multiple sequencing technologies performed particularly well. Recent developments in sequencing and variant calling have enabled benchmarking variants in challenging genomic regions, paving the way for the identification of previously unknown clinically relevant variants.

Original language	English
Article number	100129
Pages (from-to)	1-12
Journal	Cell Genomics
Volume	2
Issue number	5
DOIs	https://doi.org/10.1016/j.xgen.2022.100129
Publication status	Published - 2022 May 11

Subject classification (UKÄ)

Bioinformatics and Systems Biology
Bioinformatics (Computational Biology)
Medical Genetics

Keywords

DNA
variant
shot-read sequencing
long-read sequencing
benchmark

Access to Document

10.1016/j.xgen.2022.100129Licence: CC BY

1 Preprint (in preprint archive)

precisionFDA Truth Challenge V2: Calling variants from short- and long-reads in difficult-to-map regions
Olson, N. D., Wagner, J., McDaniel, J., Stephens, S. H., Westreich, S. T., Prasanna, A. G., Johanson, E., Maier, E. J., Boja, E., Serang, O., Jáspez, D., Lorenzo-Salazar, J. M., Muñoz-Barrera, A., Rubio-Rodríguez, L. A., Flores, C., Kyriakidis, K., Malousi, A., Shafin, K., Pesout, T., Jain, M., & 53 othersPaten, B., Chang, P-C., Kolesnikov, A., Nattestad, M., Baid, G., Goel, S., Yang, H., Carroll, A., Eveleigh, R., Bourgey, M., Bourque, G., Li, G., MA, C., Tang, L., DU, Y., Zhang, S., Morata, J., Tonda, R., Parra, G., Trotta, J-R., Brueffer, C., Demirkaya-Budak, S., Kabakci-Zorlu, D., Turgut, D., Kalay, Ö., Budak, G., Narcı, K., Arslan, E., Brown, R., Johnson, I. J., Dolgoborodov, A., Semenyuk, V., Jain, A., Tetikol, H. S., Jain, V., Ruehle, M., Lajoie, B., Roddey, C., Catreux, S., Mehio, R., Ahsan, M. U., Liu, Q., Wang, K., Sahraeian, S. M. E., Fang, L. T., Mohiyuddin, M., Hung, C., Jain, C., Feng, H., Li, Z., Chen, L., Sedlazeck, F. J. & Zook, J. M., 2020 Nov 15, bioRxiv.
Research output: Working paper/Preprint › Preprint (in preprint archive)

Cite this

Olson, N. D., Wagner, J., McDaniel, J., Stephens, S. H., Westreich, S. T., Prasanna, A. G., Johanson, E., Boja, E., Maier, E. J., Serang, O., Jáspez, D., Lorenzo-Salazar, J. M., Muñoz-Barrera, A., Rubio-Rodríguez, L. A., Flores, C., Kyriakidis, K., Malousi, A., Shafin, K., Pesout, T., ... Zook, J. M. (2022). PrecisionFDA Truth Challenge V2: Calling variants from short and long reads in difficult-to-map regions. Cell Genomics, 2(5), 1-12. [100129]. https://doi.org/10.1016/j.xgen.2022.100129

@article{321a29e20c7d465990cce5a073131432,

title = "PrecisionFDA Truth Challenge V2: Calling variants from short and long reads in difficult-to-map regions",

abstract = "The precisionFDA Truth Challenge V2 aimed to assess the state of the art of variant calling in challenging genomic regions. Starting with FASTQs, 20 challenge participants applied their variant-calling pipelines and submitted 64 variant call sets for one or more sequencing technologies (Illumina, PacBio HiFi, and Oxford Nanopore Technologies). Submissions were evaluated following best practices for benchmarking small variants with updated Genome in a Bottle benchmark sets and genome stratifications. Challenge submissions included numerous innovative methods, with graph-based and machine learning methods scoring best for short-read and long-read datasets, respectively. With machine learning approaches, combining multiple sequencing technologies performed particularly well. Recent developments in sequencing and variant calling have enabled benchmarking variants in challenging genomic regions, paving the way for the identification of previously unknown clinically relevant variants.",

keywords = "DNA, variant, shot-read sequencing, long-read sequencing, benchmark",

author = "Olson, {Nathan D.} and Justin Wagner and Jennifer McDaniel and Stephens, {Sarah H.} and Westreich, {Samuel T.} and Prasanna, {Anish G.} and Elaine Johanson and Emily Boja and Maier, {Ezekiel J.} and Omar Serang and David J{\'a}spez and Lorenzo-Salazar, {Jos{\'e} M.} and Adri{\'a}n Mu{\~n}oz-Barrera and Rubio-Rodr{\'i}guez, {Luis A.} and Carlos Flores and Konstantinos Kyriakidis and Andigoni Malousi and Kishwar Shafin and Trevor Pesout and Miten Jain and Benedict Paten and Pi-Chuan Chang and Alexey Kolesnikov and Maria Nattestad and Gunjan Baid and Sidharth Goel and Howard Yang and Andrew Carroll and Robert Eveleigh and Mathieu Bourgey and Guillaume Bourque and Gen Li and ChouXian MA and LinQi Tang and YuanPing DU and ShaoWei Zhang and Jordi Morata and Ra{\'u}l Tonda and Gen{\'i}s Parra and Jean-R{\'e}mi Trotta and Christian Brueffer and Sinem Demirkaya-Budak and Duygu Kabakci-Zorlu and Deniz Turgut and {\"O}zem Kalay and Gungor Budak and K{\"u}bra Narcı and Elif Arslan and Richard Brown and Johnson, {Ivan J.} and Alexey Dolgoborodov and Vladimir Semenyuk and Amit Jain and Tetikol, {H. Serhat} and Varun Jain and Mike Ruehle and Bryan Lajoie and Cooper Roddey and Severine Catreux and Rami Mehio and Ahsan, {Mian Umair} and Qian Liu and Kai Wang and Sahraeian, {Sayed Mohammad Ebrahim} and Fang, {Li Tai} and Marghoob Mohiyuddin and Calvin Hung and Chirag Jain and Hanying Feng and Zhipan Li and Luoqi Chen and Sedlazeck, {Fritz J.} and Zook, {Justin M}",

year = "2022",

month = may,

day = "11",

doi = "10.1016/j.xgen.2022.100129",

language = "English",

volume = "2",

pages = "1--12",

journal = "Cell Genomics",

issn = "2666-979X",

publisher = "Cell Press",

number = "5",

}

Olson, ND, Wagner, J, McDaniel, J, Stephens, SH, Westreich, ST, Prasanna, AG, Johanson, E, Boja, E, Maier, EJ, Serang, O, Jáspez, D, Lorenzo-Salazar, JM, Muñoz-Barrera, A, Rubio-Rodríguez, LA, Flores, C, Kyriakidis, K, Malousi, A, Shafin, K, Pesout, T, Jain, M, Paten, B, Chang, P-C, Kolesnikov, A, Nattestad, M, Baid, G, Goel, S, Yang, H, Carroll, A, Eveleigh, R, Bourgey, M, Bourque, G, Li, G, MA, C, Tang, L, DU, Y, Zhang, S, Morata, J, Tonda, R, Parra, G, Trotta, J-R, Brueffer, C, Demirkaya-Budak, S, Kabakci-Zorlu, D, Turgut, D, Kalay, Ö, Budak, G, Narcı, K, Arslan, E, Brown, R, Johnson, IJ, Dolgoborodov, A, Semenyuk, V, Jain, A, Tetikol, HS, Jain, V, Ruehle, M, Lajoie, B, Roddey, C, Catreux, S, Mehio, R, Ahsan, MU, Liu, Q, Wang, K, Sahraeian, SME, Fang, LT, Mohiyuddin, M, Hung, C, Jain, C, Feng, H, Li, Z, Chen, L, Sedlazeck, FJ & Zook, JM 2022, 'PrecisionFDA Truth Challenge V2: Calling variants from short and long reads in difficult-to-map regions', Cell Genomics, vol. 2, no. 5, 100129, pp. 1-12. https://doi.org/10.1016/j.xgen.2022.100129

TY - JOUR

T1 - PrecisionFDA Truth Challenge V2: Calling variants from short and long reads in difficult-to-map regions

AU - Olson, Nathan D.

AU - Wagner, Justin

AU - McDaniel, Jennifer

AU - Stephens, Sarah H.

AU - Westreich, Samuel T.

AU - Prasanna, Anish G.

AU - Johanson, Elaine

AU - Boja, Emily

AU - Maier, Ezekiel J.

AU - Serang, Omar

AU - Jáspez, David

AU - Lorenzo-Salazar, José M.

AU - Muñoz-Barrera, Adrián

AU - Rubio-Rodríguez, Luis A.

AU - Flores, Carlos

AU - Kyriakidis, Konstantinos

AU - Malousi, Andigoni

AU - Shafin, Kishwar

AU - Pesout, Trevor

AU - Jain, Miten

AU - Paten, Benedict

AU - Chang, Pi-Chuan

AU - Kolesnikov, Alexey

AU - Nattestad, Maria

AU - Baid, Gunjan

AU - Goel, Sidharth

AU - Yang, Howard

AU - Carroll, Andrew

AU - Eveleigh, Robert

AU - Bourgey, Mathieu

AU - Bourque, Guillaume

AU - Li, Gen

AU - MA, ChouXian

AU - Tang, LinQi

AU - DU, YuanPing

AU - Zhang, ShaoWei

AU - Morata, Jordi

AU - Tonda, Raúl

AU - Parra, Genís

AU - Trotta, Jean-Rémi

AU - Brueffer, Christian

AU - Demirkaya-Budak, Sinem

AU - Kabakci-Zorlu, Duygu

AU - Turgut, Deniz

AU - Kalay, Özem

AU - Budak, Gungor

AU - Narcı, Kübra

AU - Arslan, Elif

AU - Brown, Richard

AU - Johnson, Ivan J.

AU - Dolgoborodov, Alexey

AU - Semenyuk, Vladimir

AU - Jain, Amit

AU - Tetikol, H. Serhat

AU - Jain, Varun

AU - Ruehle, Mike

AU - Lajoie, Bryan

AU - Roddey, Cooper

AU - Catreux, Severine

AU - Mehio, Rami

AU - Ahsan, Mian Umair

AU - Liu, Qian

AU - Wang, Kai

AU - Sahraeian, Sayed Mohammad Ebrahim

AU - Fang, Li Tai

AU - Mohiyuddin, Marghoob

AU - Hung, Calvin

AU - Jain, Chirag

AU - Feng, Hanying

AU - Li, Zhipan

AU - Chen, Luoqi

AU - Sedlazeck, Fritz J.

AU - Zook, Justin M

PY - 2022/5/11

Y1 - 2022/5/11

N2 - The precisionFDA Truth Challenge V2 aimed to assess the state of the art of variant calling in challenging genomic regions. Starting with FASTQs, 20 challenge participants applied their variant-calling pipelines and submitted 64 variant call sets for one or more sequencing technologies (Illumina, PacBio HiFi, and Oxford Nanopore Technologies). Submissions were evaluated following best practices for benchmarking small variants with updated Genome in a Bottle benchmark sets and genome stratifications. Challenge submissions included numerous innovative methods, with graph-based and machine learning methods scoring best for short-read and long-read datasets, respectively. With machine learning approaches, combining multiple sequencing technologies performed particularly well. Recent developments in sequencing and variant calling have enabled benchmarking variants in challenging genomic regions, paving the way for the identification of previously unknown clinically relevant variants.

AB - The precisionFDA Truth Challenge V2 aimed to assess the state of the art of variant calling in challenging genomic regions. Starting with FASTQs, 20 challenge participants applied their variant-calling pipelines and submitted 64 variant call sets for one or more sequencing technologies (Illumina, PacBio HiFi, and Oxford Nanopore Technologies). Submissions were evaluated following best practices for benchmarking small variants with updated Genome in a Bottle benchmark sets and genome stratifications. Challenge submissions included numerous innovative methods, with graph-based and machine learning methods scoring best for short-read and long-read datasets, respectively. With machine learning approaches, combining multiple sequencing technologies performed particularly well. Recent developments in sequencing and variant calling have enabled benchmarking variants in challenging genomic regions, paving the way for the identification of previously unknown clinically relevant variants.

KW - DNA

KW - variant

KW - shot-read sequencing

KW - long-read sequencing

KW - benchmark

U2 - 10.1016/j.xgen.2022.100129

DO - 10.1016/j.xgen.2022.100129

M3 - Article

C2 - 35720974

SN - 2666-979X

VL - 2

SP - 1

EP - 12

JO - Cell Genomics

JF - Cell Genomics

IS - 5

M1 - 100129

ER -

PrecisionFDA Truth Challenge V2: Calling variants from short and long reads in difficult-to-map regions

Abstract

Subject classification (UKÄ)

Keywords

Access to Document

Fingerprint

Research output

precisionFDA Truth Challenge V2: Calling variants from short- and long-reads in difficult-to-map regions

Cite this