External Quality Assessment of SARS-CoV-2 Sequencing: an ESGMD-SSM Pilot Trial across 15 European Laboratories

Fanny Wegner; Tim Roloff; Michael Huber; Samuel Cordey; Alban Ramette; Yannick Gerth; Claire Bertelli; Madlen Stange; Helena M.B. Seth-Smith; Alfredo Mari; Karoline Leuzinger; Lorenzo Cerutti; Keith Harshman; Ioannis Xenarios; Philippe Le Mercier; Pascal Bittel; Stefan Neuenschwander; Onya Opota; Jonas Fuchs; Marcus Panning; Charlotte Michel; Marie Hallin; Thomas Demuyser; Ricardo De Mendonca; Paul Savelkoul; Jozef Dingemans; Brian van der Veer; Stefan A. Boers; Eric C.J. Claas; Jordy P.M. Coolen; Willem J.G. Melchers; Marianne Gunell; Teemu Kallonen; Tytti Vuorinen; Antti J. Hakanen; Eva Bernhoff; Marit Andrea Klokkhammer Hetland; Hadar Golan Berman; Sheera Adar; Jacob Moran-Gilad; Dana G. Wolf; Stephen L. Leib; Oliver Nolte; Laurent Kaiser; Stefan Schmutz; Verena Kufner; Maryam Zaheri; Alexandra Trkola; Hege Vangstein Aamot; Hans H. Hirsch; Gilbert Greub; Adrian Egli

doi:10.1128/JCM.01698-21

External Quality Assessment of SARS-CoV-2 Sequencing: an ESGMD-SSM Pilot Trial across 15 European Laboratories

Fanny Wegner^*, Tim Roloff, Michael Huber, Samuel Cordey, Alban Ramette, Yannick Gerth, Claire Bertelli, Madlen Stange, Helena M.B. Seth-Smith, Alfredo Mari, Karoline Leuzinger, Lorenzo Cerutti, Keith Harshman, Ioannis Xenarios, Philippe Le Mercier, Pascal Bittel, Stefan Neuenschwander, Onya Opota, Jonas Fuchs, Marcus PanningCharlotte Michel, Marie Hallin, Thomas Demuyser, Ricardo De Mendonca, Paul Savelkoul, Jozef Dingemans, Brian van der Veer, Stefan A. Boers, Eric C.J. Claas, Jordy P.M. Coolen, Willem J.G. Melchers, Marianne Gunell, Teemu Kallonen, Tytti Vuorinen, Antti J. Hakanen, Eva Bernhoff, Marit Andrea Klokkhammer Hetland, Hadar Golan Berman, Sheera Adar, Jacob Moran-Gilad, Dana G. Wolf, Stephen L. Leib, Oliver Nolte, Laurent Kaiser, Stefan Schmutz, Verena Kufner, Maryam Zaheri, Alexandra Trkola, Hege Vangstein Aamot, Hans H. Hirsch, Gilbert Greub, Adrian Egli

^*Corresponding author for this work

Department of Microbiology and Molecular Genetics

Research output: Contribution to journal › Article › peer-review

9 Scopus citations

Abstract

This first pilot trial on external quality assessment (EQA) of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) whole-genome sequencing, initiated by the European Society of Clinical Microbiology and Infectious Diseases (ESCMID) Study Group for Genomic and Molecular Diagnostics (ESGMD) and the Swiss Society for Microbiology (SSM), aims to build a framework between laboratories in order to improve pathogen surveillance sequencing. Ten samples with various viral loads were sent out to 15 clinical laboratories that had free choice of sequencing methods and bioinformatic analyses. The key aspects on which the individual centers were compared were the identification of (i) single nucleotide polymorphisms (SNPs) and indels, (ii) Pango lineages, and (iii) clusters between samples. The participating laboratories used a wide array of methods and analysis pipelines. Most were able to generate whole genomes for all samples. Genomes were sequenced to various depths (up to a 100-fold difference across centers). There was a very good consensus regarding the majority of reporting criteria, but there were a few discrepancies in lineage and cluster assignments. Additionally, there were inconsistencies in variant calling. The main reasons for discrepancies were missing data, bioinformatic choices, and interpretation of data. The pilot EQA was overall a success. It was able to show the high quality of participating laboratories and provide valuable feedback in cases where problems occurred, thereby improving the sequencing setup of laboratories. A larger follow-up EQA should, however, improve on defining the variables and format of the report. Additionally, contamination and/or minority variants should be a further aspect of assessment.

Original language	American English
Article number	e0169821
Journal	Journal of Clinical Microbiology
Volume	60
Issue number	1
DOIs	https://doi.org/10.1128/JCM.01698-21
State	Published - Jan 2022

Bibliographical note

Funding Information:
Costs of preparing the EQA and shipping costs were borne by the Clinical Bacteriology and Mycology group of the University Hospital Basel and the Swiss Society of Microbiology (SSM) (Adrian Egli), whereas sequencing costs were supported by each participating laboratory. This work was in part supported by B-FAST (Fkz 01KX2021) in the framework of the National Research Network of University Medical Centers on COVID-19 (Nationales Forschungsnetzwerk der Universitätsmedizin zu Covid-19), Federal Ministry of Education and Research of Germany (Bundesministerium für Bildung und Forschung [BMBF]).

Publisher Copyright:
Copyright © 2022 Wegner et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International license.

Keywords

COVID-19
Humans
Laboratories
Laboratories, Clinical
Pilot Projects
SARS-CoV-2

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1128/JCM.01698-21

Cite this

Wegner, F., Roloff, T., Huber, M., Cordey, S., Ramette, A., Gerth, Y., Bertelli, C., Stange, M., Seth-Smith, H. M. B., Mari, A., Leuzinger, K., Cerutti, L., Harshman, K., Xenarios, I., Le Mercier, P., Bittel, P., Neuenschwander, S., Opota, O., Fuchs, J., ... Egli, A. (2022). External Quality Assessment of SARS-CoV-2 Sequencing: an ESGMD-SSM Pilot Trial across 15 European Laboratories. Journal of Clinical Microbiology, 60(1), Article e0169821. https://doi.org/10.1128/JCM.01698-21

@article{faf4a5208a65429ea228523d563f8624,

title = "External Quality Assessment of SARS-CoV-2 Sequencing: an ESGMD-SSM Pilot Trial across 15 European Laboratories",

abstract = "This first pilot trial on external quality assessment (EQA) of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) whole-genome sequencing, initiated by the European Society of Clinical Microbiology and Infectious Diseases (ESCMID) Study Group for Genomic and Molecular Diagnostics (ESGMD) and the Swiss Society for Microbiology (SSM), aims to build a framework between laboratories in order to improve pathogen surveillance sequencing. Ten samples with various viral loads were sent out to 15 clinical laboratories that had free choice of sequencing methods and bioinformatic analyses. The key aspects on which the individual centers were compared were the identification of (i) single nucleotide polymorphisms (SNPs) and indels, (ii) Pango lineages, and (iii) clusters between samples. The participating laboratories used a wide array of methods and analysis pipelines. Most were able to generate whole genomes for all samples. Genomes were sequenced to various depths (up to a 100-fold difference across centers). There was a very good consensus regarding the majority of reporting criteria, but there were a few discrepancies in lineage and cluster assignments. Additionally, there were inconsistencies in variant calling. The main reasons for discrepancies were missing data, bioinformatic choices, and interpretation of data. The pilot EQA was overall a success. It was able to show the high quality of participating laboratories and provide valuable feedback in cases where problems occurred, thereby improving the sequencing setup of laboratories. A larger follow-up EQA should, however, improve on defining the variables and format of the report. Additionally, contamination and/or minority variants should be a further aspect of assessment.",

keywords = "COVID-19, Humans, Laboratories, Laboratories, Clinical, Pilot Projects, SARS-CoV-2",

author = "Fanny Wegner and Tim Roloff and Michael Huber and Samuel Cordey and Alban Ramette and Yannick Gerth and Claire Bertelli and Madlen Stange and Seth-Smith, {Helena M.B.} and Alfredo Mari and Karoline Leuzinger and Lorenzo Cerutti and Keith Harshman and Ioannis Xenarios and {Le Mercier}, Philippe and Pascal Bittel and Stefan Neuenschwander and Onya Opota and Jonas Fuchs and Marcus Panning and Charlotte Michel and Marie Hallin and Thomas Demuyser and {De Mendonca}, Ricardo and Paul Savelkoul and Jozef Dingemans and {van der Veer}, Brian and Boers, {Stefan A.} and Claas, {Eric C.J.} and Coolen, {Jordy P.M.} and Melchers, {Willem J.G.} and Marianne Gunell and Teemu Kallonen and Tytti Vuorinen and Hakanen, {Antti J.} and Eva Bernhoff and Hetland, {Marit Andrea Klokkhammer} and Berman, {Hadar Golan} and Sheera Adar and Jacob Moran-Gilad and Wolf, {Dana G.} and Leib, {Stephen L.} and Oliver Nolte and Laurent Kaiser and Stefan Schmutz and Verena Kufner and Maryam Zaheri and Alexandra Trkola and Aamot, {Hege Vangstein} and Hirsch, {Hans H.} and Gilbert Greub and Adrian Egli",

note = "Funding Information: Costs of preparing the EQA and shipping costs were borne by the Clinical Bacteriology and Mycology group of the University Hospital Basel and the Swiss Society of Microbiology (SSM) (Adrian Egli), whereas sequencing costs were supported by each participating laboratory. This work was in part supported by B-FAST (Fkz 01KX2021) in the framework of the National Research Network of University Medical Centers on COVID-19 (Nationales Forschungsnetzwerk der Universit{\"a}tsmedizin zu Covid-19), Federal Ministry of Education and Research of Germany (Bundesministerium f{\"u}r Bildung und Forschung [BMBF]). Publisher Copyright: Copyright {\textcopyright} 2022 Wegner et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International license.",

year = "2022",

month = jan,

doi = "10.1128/JCM.01698-21",

language = "American English",

volume = "60",

journal = "Journal of Clinical Microbiology",

issn = "0095-1137",

publisher = "American Society for Microbiology",

number = "1",

}

Wegner, F, Roloff, T, Huber, M, Cordey, S, Ramette, A, Gerth, Y, Bertelli, C, Stange, M, Seth-Smith, HMB, Mari, A, Leuzinger, K, Cerutti, L, Harshman, K, Xenarios, I, Le Mercier, P, Bittel, P, Neuenschwander, S, Opota, O, Fuchs, J, Panning, M, Michel, C, Hallin, M, Demuyser, T, De Mendonca, R, Savelkoul, P, Dingemans, J, van der Veer, B, Boers, SA, Claas, ECJ, Coolen, JPM, Melchers, WJG, Gunell, M, Kallonen, T, Vuorinen, T, Hakanen, AJ, Bernhoff, E, Hetland, MAK, Berman, HG, Adar, S, Moran-Gilad, J, Wolf, DG, Leib, SL, Nolte, O, Kaiser, L, Schmutz, S, Kufner, V, Zaheri, M, Trkola, A, Aamot, HV, Hirsch, HH, Greub, G & Egli, A 2022, 'External Quality Assessment of SARS-CoV-2 Sequencing: an ESGMD-SSM Pilot Trial across 15 European Laboratories', Journal of Clinical Microbiology, vol. 60, no. 1, e0169821. https://doi.org/10.1128/JCM.01698-21

TY - JOUR

T1 - External Quality Assessment of SARS-CoV-2 Sequencing

T2 - an ESGMD-SSM Pilot Trial across 15 European Laboratories

AU - Wegner, Fanny

AU - Roloff, Tim

AU - Huber, Michael

AU - Cordey, Samuel

AU - Ramette, Alban

AU - Gerth, Yannick

AU - Bertelli, Claire

AU - Stange, Madlen

AU - Seth-Smith, Helena M.B.

AU - Mari, Alfredo

AU - Leuzinger, Karoline

AU - Cerutti, Lorenzo

AU - Harshman, Keith

AU - Xenarios, Ioannis

AU - Le Mercier, Philippe

AU - Bittel, Pascal

AU - Neuenschwander, Stefan

AU - Opota, Onya

AU - Fuchs, Jonas

AU - Panning, Marcus

AU - Michel, Charlotte

AU - Hallin, Marie

AU - Demuyser, Thomas

AU - De Mendonca, Ricardo

AU - Savelkoul, Paul

AU - Dingemans, Jozef

AU - van der Veer, Brian

AU - Boers, Stefan A.

AU - Claas, Eric C.J.

AU - Coolen, Jordy P.M.

AU - Melchers, Willem J.G.

AU - Gunell, Marianne

AU - Kallonen, Teemu

AU - Vuorinen, Tytti

AU - Hakanen, Antti J.

AU - Bernhoff, Eva

AU - Hetland, Marit Andrea Klokkhammer

AU - Berman, Hadar Golan

AU - Adar, Sheera

AU - Moran-Gilad, Jacob

AU - Wolf, Dana G.

AU - Leib, Stephen L.

AU - Nolte, Oliver

AU - Kaiser, Laurent

AU - Schmutz, Stefan

AU - Kufner, Verena

AU - Zaheri, Maryam

AU - Trkola, Alexandra

AU - Aamot, Hege Vangstein

AU - Hirsch, Hans H.

AU - Greub, Gilbert

AU - Egli, Adrian

N1 - Funding Information: Costs of preparing the EQA and shipping costs were borne by the Clinical Bacteriology and Mycology group of the University Hospital Basel and the Swiss Society of Microbiology (SSM) (Adrian Egli), whereas sequencing costs were supported by each participating laboratory. This work was in part supported by B-FAST (Fkz 01KX2021) in the framework of the National Research Network of University Medical Centers on COVID-19 (Nationales Forschungsnetzwerk der Universitätsmedizin zu Covid-19), Federal Ministry of Education and Research of Germany (Bundesministerium für Bildung und Forschung [BMBF]). Publisher Copyright: Copyright © 2022 Wegner et al. This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International license.

PY - 2022/1

Y1 - 2022/1

N2 - This first pilot trial on external quality assessment (EQA) of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) whole-genome sequencing, initiated by the European Society of Clinical Microbiology and Infectious Diseases (ESCMID) Study Group for Genomic and Molecular Diagnostics (ESGMD) and the Swiss Society for Microbiology (SSM), aims to build a framework between laboratories in order to improve pathogen surveillance sequencing. Ten samples with various viral loads were sent out to 15 clinical laboratories that had free choice of sequencing methods and bioinformatic analyses. The key aspects on which the individual centers were compared were the identification of (i) single nucleotide polymorphisms (SNPs) and indels, (ii) Pango lineages, and (iii) clusters between samples. The participating laboratories used a wide array of methods and analysis pipelines. Most were able to generate whole genomes for all samples. Genomes were sequenced to various depths (up to a 100-fold difference across centers). There was a very good consensus regarding the majority of reporting criteria, but there were a few discrepancies in lineage and cluster assignments. Additionally, there were inconsistencies in variant calling. The main reasons for discrepancies were missing data, bioinformatic choices, and interpretation of data. The pilot EQA was overall a success. It was able to show the high quality of participating laboratories and provide valuable feedback in cases where problems occurred, thereby improving the sequencing setup of laboratories. A larger follow-up EQA should, however, improve on defining the variables and format of the report. Additionally, contamination and/or minority variants should be a further aspect of assessment.

AB - This first pilot trial on external quality assessment (EQA) of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) whole-genome sequencing, initiated by the European Society of Clinical Microbiology and Infectious Diseases (ESCMID) Study Group for Genomic and Molecular Diagnostics (ESGMD) and the Swiss Society for Microbiology (SSM), aims to build a framework between laboratories in order to improve pathogen surveillance sequencing. Ten samples with various viral loads were sent out to 15 clinical laboratories that had free choice of sequencing methods and bioinformatic analyses. The key aspects on which the individual centers were compared were the identification of (i) single nucleotide polymorphisms (SNPs) and indels, (ii) Pango lineages, and (iii) clusters between samples. The participating laboratories used a wide array of methods and analysis pipelines. Most were able to generate whole genomes for all samples. Genomes were sequenced to various depths (up to a 100-fold difference across centers). There was a very good consensus regarding the majority of reporting criteria, but there were a few discrepancies in lineage and cluster assignments. Additionally, there were inconsistencies in variant calling. The main reasons for discrepancies were missing data, bioinformatic choices, and interpretation of data. The pilot EQA was overall a success. It was able to show the high quality of participating laboratories and provide valuable feedback in cases where problems occurred, thereby improving the sequencing setup of laboratories. A larger follow-up EQA should, however, improve on defining the variables and format of the report. Additionally, contamination and/or minority variants should be a further aspect of assessment.

KW - COVID-19

KW - Humans

KW - Laboratories

KW - Laboratories, Clinical

KW - Pilot Projects

KW - SARS-CoV-2

UR - http://www.scopus.com/inward/record.url?scp=85122954298&partnerID=8YFLogxK

U2 - 10.1128/JCM.01698-21

DO - 10.1128/JCM.01698-21

M3 - Article

C2 - 34757834

AN - SCOPUS:85122954298

SN - 0095-1137

VL - 60

JO - Journal of Clinical Microbiology

JF - Journal of Clinical Microbiology

IS - 1

M1 - e0169821

ER -

External Quality Assessment of SARS-CoV-2 Sequencing: an ESGMD-SSM Pilot Trial across 15 European Laboratories

Abstract

Bibliographical note

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this