11 Pipeline overview
11.1 Overview of variant calling pipeline
The chart below provides a high-level overview of the different steps involved in the processing of genomic data. In the following chapter we will focus on the variant calling pipeline, which deals with:
- Assessing the quality of sequence reads and filtering if necessary.
- Aligning the reads to a reference genome - this is also referred to as read mapping.
- Identify variants, i.e. where do my samples differ from the reference genome - this is known as _variant calling.
In the diagram below the various tools (e.g., bwa, samtools, gatk) and data formats (fastq, bam, vcf, etc.) are depicted as well. Some of these we have already encountered in the previous chapters. You might also recall the pipeline scripts (stored in ./training/scripts) that were mentioned in the chapter on Unix scripting. These already showcased a number of the different steps involved in the pipeline.
A more simplified overview of the pipeline can be seen here:
We will mostly focus on the AmpliSeq pipeline available here as it was introduced by Kattenberg et al. (2022) and Kattenberg et al. (2023).
For the whole-genome sequencing parts, while largely similar, we try to adhere to standard conventions established by examples like the GATK best practices and MalariaGEN’s genomic databases (2023).
11.2 Additional resources
We have collected a number of related resources below, which have served as an inspiration while preparing our own course:
- https://genomics.sschmeier.com/
- https://training.galaxyproject.org/training-material/topics/sequence-analysis/
- https://www.melbournebioinformatics.org.au/tutorials/tutorials/variant_calling_gatk1/variant_calling_gatk1/
- https://speciationgenomics.github.io/
- https://ngs-course.readthedocs.io/en/praha-november-2023/
- https://people.duke.edu/~ccc14/duke-hts-2018/index.html
- https://biomedicalhub.github.io/genomics/
- https://hbctraining.github.io/variant_analysis/schedule/self-learning.html
- https://datacarpentry.org/wrangling-genomics/
- https://mmbdtp.github.io/modules/sequencing/week-2-programme/
- https://eriqande.github.io/eca-bioinf-handbook/overview-of-bioinformatic-analyses.html
- https://rnnh.github.io/bioinfo-notebook/#contents
- https://learn.gencore.bio.nyu.edu/
- https://conmeehan.github.io/PathogenDataCourse/PathogenCourse.html
- https://bio-courses1.rsb.anu.edu.au/biol3161-data/ReadsToVariants.html
The video below by Tobias Rausch @ EMBL-EBI (Rausch 2022) also provides an excellent overview of various topics that come up in genomic variant calling, but it is broader in scope than just AmpliSeq or molecular surveillance of parasites: