Sergey Koren PacBio User Group Meeting

Transcription

1 Sergey Koren PacBio User Group Meeting Marco Island, FL February 15, 2012

2 Disclaimer This presentation was developed and funded under Agreement No. HSHQDC-07-C awarded by the U.S. Department of Homeland Security for the management and operation of the National Biodefense Analysis and Countermeasures Center (NBACC), a Federally Funded Research and Development Center. The views and conclusions contained in this presentation are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of the U.S. Department of Homeland Security. The Department of Homeland Security does not endorse any products or commercial services mentioned in this presentation. Hybrid Error Correction/

3 Hybrid strategy Second-gen Accurate and plentiful Short reads Biased error / coverage Third-gen Long and unbiased Low accuracy Low throughput Goal Long and accurate High-quality assemblies Hybrid Error Correction/

4 Read length matters k = 50 k = 1,000 k = 5,000 Assembly complexity of prokaryotic genomes using short reads. Kingsford C, Schatz MC, Pop M (2010) BMC Bioinformatics. Hybrid Error Correction/

5 Overlap versus de Bruijn Overlap Repeat complexity depends on read length Read placements kept Robust to error Tangled by high-coverage de Bruijn Repeat complexity depends on word length Read placements lost Sensitive to error Robust to high coverage '" #"" #'" $"" $'" #""!"#$%&%'()(*% BCD(,HI&E BCD(,HI#$J Spring 11 Summer 11!"" #""" #$"" #%"" #&"" ()*+,+-./,0-1234,56789:.3-/; Fall 11 Hybrid Error Correction/

6 Sequencing Errors and Assembly Sequencing errors add complexity to assembly 15% read error, 30% overlap error, is challenging Indel error is even worse Obscure overlaps, require shorter words Complicates graph by introducing spurs, bubbles, tips, etc. Increases the effective repeat rate Potentially high error rate in consensus Hybrid Error Correction/

7 Reduced positional and GC bias Empirical Error Rate Read Position Feng Chen, JGI Hybrid Error Correction/

8 Find easier overlaps first seed size 30% IDY 15% IDY Hybrid Error Correction/

9 Hybrid correction and assembly Pre-process reads?! Map short to long reads Separate repeats R! R'! Compute layout Trim at coverage gaps Compute consensus Assemble corrected reads with OLC strategy R! R'! Hybrid Error Correction/

10 Repeat separation Estimate long-read coverage from mappings Select top C mappings Forces uniform coverage of short-read mapping Mappings per short-read (20x long read depth) Mapping depth (50x short read depth) Frequency Frequency Hybrid Error Correction/

11 Correction accuracy Independent of short-read depth Hybrid Error Correction/

12 E. coli contig improvements Hybrid Error Correction/

13 Full results Hybrid Error Correction/

14 Affect of sequencing coverage Illumina coverage Up to 100x, improves both throughput and assembly PacBio coverage Rapid gains up to 10x Back to Sanger coverage!"#$%&' % Bases !!!!!!!! Throughput Continuity Contig N50 (Kbp) 3&!+, :; "!! #!! $!! %!! %&%+<+'()*+7=>#""; Illumina Coverage! "! #! $! %! &! '()*+,-./012/ Hybrid Error Correction/

15 Nearing perfection Simulated PacBio reads for Francisella tularensis 7 contigs at 40x Assembly broken by 2 x 30 Kbp repeat and 4 x 5 Kbp repeat Simulated # Contigs L/W Expected # Islands truenum num Automatic bacterial closure is here, stay tuned for Friday Hybrid Error Correction/

16 The Budgie Genome Budgerigar project Melopsittacus undulatus ~1.2 Gbp haploid length 454 data (15.4x) Shotgun, 15 runs (median ~419) 3kb paired-end, 4 runs 8kb paired-end, 8 runs 20kb paired-end, 4 runs FLX+, 15 runs (median ~667) Illumina data (54x) 200bp Illumina paired-end PacBio data (5.5x) Original (median 1,308; max 16, 947) Corrected via 54x of Illumina (median 997; max 13,079) See Erich Jarvis on Friday Hybrid Error Correction/

17 Budgie Results Assembler Parrot-Celera Parrot-Celera Parrot-Celera Finch-PCAP Chicken-PCAP Penguin- SOAP Method PacBio Illumina Sanger Sanger Illumina Coverage 19X 15.4X 18X 6X 7.1X 60X Genome size 1.2Gb 1.2Gb 1.2GB 1.2Gb 1.05Gb 1.63 Gb [Scaffolds] TotalBasesInScaffolds 1,136,220,675 1,101,870,942 1,083,510,132 1,224,525,252 1,047,124,295 1,229,920,113 # of Scaffolds 3,465 1,916 1,138 37,698 23,776 29,970 AvgScaffoldSize 22,055 28,994 44,319 32,482 44,041 41,958 N50ScaffoldSize 1,300,229 9,103,021 9,133,401 10,409,499 11,125,310 5,071,598 LargestScaffoldSize 7,777,437 46,598,737 39,887,647 56,620,707 51,053,708 28,260,285 [Contigs] # of Contigs 15,081 16,574 21, ,053 85, ,943 AvgContigSize 15,344 15,148 15,334 9,714 12,291 10,686 N50ContigSize 99,573 75,178 46,805 38,549 45,280 30,459 LargestContigSize 1,238, , , , , ,109 Assemblathon 2 TBD Hybrid Error Correction/

18 Assembly Correctness Coverage of PacBio Assembly by Illumina 10Kbp mates Count Clone Coverage Coverage of PacBio Joins By Illumina 10Kbp mates Count 0e+00 2e+05 4e+05 6e Clone Coverage Hybrid Error Correction/

19 Gene Content (vs zebra finch) % Number of CDS PBcR 454 Illumina % CDS Coverage Hybrid Error Correction/

20 RNA-Seq Correction of corn transcriptome 130K PacBio sequences, 230M Illumina pairs Missing exons recovered, Indel rate reduced to 0.06%, 0.02% 11% mapped by BLAT pre-correction, 99% after correction Hybrid Error Correction/

21 Demonstrations Small genome assembly and closure Bacteria and Yeast 5x N50 size versus second-generation sequencing Vertebrate genome assembly Parrot genome hybrid assembly Better continuity than gold-standard Sanger Full-length transcript sequencing Corn transcriptome End-to-end transcripts and alternate isoforms Working on single cell Hybrid Error Correction/

22 Conclusions High error does not prohibit assembly Difficult to use alone, but powerful in combination Co-assembly guards against mis-assembly Potential applications The return of genome closure : one chromosome, one contig Complex assembly : eukaryotic regulatory sequence Mixed sample separation : haplotype phasing Transcriptome annotation Future challenges Sequencing : throughput, accuracy, length, cost Protocols : sample prep, library construction Informatics : efficiency, robust methods, scaffolding Hybrid Error Correction/

23 Brief PBcR Tutorial Hybrid Error Correction/

24 Get Correction/Assembly Pipeline (part of Celera Assembler) free downloads open source license = GPL wiki documentation public bug reporting pacbiotoca Available starting with CA v7.0. Currently available through CVS. Sample data and an online tutorial are available online. Hybrid Error Correction/

25 Installing CA Download Pre-Build Package Hybrid Error Correction/

26 Installing CA Compile From CVS Hybrid Error Correction/

27 Correction Pipeline Introduction The options (-length, -partitions, and -threads, -noclean, -sge, and -sgecorrection) are optional. The -libraryname option is mandatory. It provides a UID name for the library that the reads are placed into. This name is completely up to the end user, but must contain no white spaces or commas. The -s pacbio.spec option is mandatory. It should point to a a CA spec file to specify how to overlap the illumina and PacBio data for correction (which comes included with the sample data.) Two templates are provided for SGE and high-memory multi-core environment. The -t X option specifies that the correction program itself is run using X threads. It should be set to the number of processors available wherever this pipeline runs. The -fastq specifies the pacbio fragments to correct. The correction will also work with unfiltered reads if you don't have a filtered fastq file. Hybrid Error Correction/

28 Correction Pipeline Output Pacbio.fasta >25001_1 AAGCAGTCGATTGCCTCATTGCGTCGCTTTTTGCTGTCCCACAGTATTTTTTTCCTGCCA TCCACCCATTTTTCGACCTGCTCTTCAGCAGTCAGCTGCTGCGCTTCGGTCAGATCAAAA ATATCCGGGTTATTCGGGAAGTGAACGGCACCGGGAAGCGGTTCATCCCCTTCCGGCGTC AGTGTGAAGCGGTTATAAATCTGCTCTTTCGCGGTATCCGTACCGATTTCGGTAAGGTAA ACCCCGTTTTTGTTTCGCTTACGTGGCATGCTGGCCACCGGCTTTCCGTAGACGGATGCC CCTTTAATGGGGATCACCCGGAACAGCCCATGTTTTTTCGAGCGTTCATACACAATGGTC GGGTCAATCCCGCCAGTATCCCAGCAGATACGGGATATCGACATTTCTGCACCATTCCGG CGGGTATAGGTTTTATTGATGGCCTCATCCACACGCAGCAGCGTCTGTTCATCGTCGTGG CGGCCCATAATAATCTGCCGGTCAATCAGCCAGCTTTCCTCACCCGGCCCCCATCCCCAT Pacbio.frg {VER ver:2 } {LIB act:a acc:pacbio ori:u mea:0.0 std:0.0 src:. nft:13 fea: forcebogunitigger=1 donottrusthomopolymerruns=0 Pacbio.qual Hybrid Error Correction/

29 Getting data into CA SFF (454) sfftoca clear 454 trim chop linker titanium FastQ (Illumina) fastqtoca insertsize type illumina fastq A,B FastA (PacBio) convert-fasta-to-v2.pl -pacbio -s reads.fna -q reads.fna.qual -l pacbioccs > css.frg Hybrid Error Correction/

30 Correction Example On Lambda Phage Paired-end Illumina sequences, 200bp mean (illumina.1/2.fastq) FastQ of PacBio sequences (pacbio.filtered_subreads.fastq) 1. Convert Illumina fastq to CA format fastqtoca -insertsize libraryname illumina - type sanger innie -mates illumina.1.fastq,illumina. 2.fastq > illumina.frg 2. Run correction pipeline pacbiotoca -length 500 -partitions 200 -l pacbio -t 16 -s pacbio.spec -fastq pacbio.filtered_subreads.fastq illumina.frg 3. Check for expected output -rw-r--r-- -rw-r--r-- -rw-r--r-- 2.6M Oct 21 10:06 pacbio.fasta 7.7M Oct 21 10:06 pacbio.qual 5.3M Oct 21 10:06 pacbio.frg Hybrid Error Correction/

31 Working With Corrected Data FastA file available for use by any tools FRG file for input to CA for assembly 1. Run CA assembly runca -p asm -d asm -s asm.spec pacbio.frg 2. Check for assembly results ls lha asm/9-terminator/ -rw-r--r-- -rw-r--r-- -rw-r--r-- -rw-r--r-- 671K Oct 21 10:14 asm.asm 48K Oct 21 10:14 asm.ctg.fasta 104K Oct 21 10:14 asm.posmap.frgctg 3.2K Oct 21 10:14 asm.qc 3. Check assembly quality cat asm/9-terminator/asm.qc [Contigs] TotalContigsInScaffolds=1 MeanContigLength=48452 MinContigLength=48452 MaxContigLength=48452 N50ContigBases=48452 Hybrid Error Correction/

32 Celera Assembler Output Assembly.ctg.fasta >scf138 CTTAATTTTGGAATGAGCAAATCAAACGGCTCTCTTTCTTCTCTCTTCCGACGATCATTTCTCTCCTTCA TCCTTTTTCTCTATTGAATTTATGGTTGTGAGACACTTGCCTATGCCGTTCAAAATTTTACATTAGAGTG GATCTAAACCAAATTGGACACTGTGTTATCTGGTTTTAGCTTGAAGTAATTTCTAACAAGAATGGTTGCT TCAAAAAGACAAATATAAAGTGAAATTGATTGCCTCATTCTCTTCACCCTCATTCAAACTAAAATTGAAG Assembly.asm CTTAAAGAAAGAAGTAAAGGAATCAAAAATGGCAGCCTGCACGTATGAGATGACGAATTCTAAAATTTCA {MDI AGCTCTGCGTTATATTTATTTTGAGTTATACGTTATCTTAATTAATTGGGTTGCTATAATTATAACTTAT ref:(20kb,1) TTCGCTCCACCTAGATTTTTAACGGCCAAAGTGAGTTTAGTTGGTGGTATTAAAATGACTTTTTTATTTA Assembly.posmap.frgctg mea: AAAATCGAAGGTTTAGTTTTTACTATGCAATTAGTGTACCAAAAGAAATTTTTATACCTATTTAAATCAA Assembly.qc FPQMBNN02SL55F f std: [Scaffolds] FPMLNGL01BTUZ f min:1539 TotalScaffolds=4127 FO77AEH01COVYY f max:31474 TotalContigsInScaffolds=8653 FPKRC2D02RHQ3C r buc:28 MeanContigsPerScaffold=2.10 FPDPMZT01DTV0D f his: MinContigsPerScaffold=1 FPBG22E02Q51Y r 8 MaxContigsPerScaffold=73 FPMX3IK01CS2XJ f 0 TotalBasesInScaffolds= FPMLNGL02QFBKX f 1 MeanBasesInScaffolds=48621 FO6AT3P02QPVK r 52 MinBasesInScaffolds=446 FO6AT3P02TCI7F r 69 MaxBasesInScaffolds= FO77AEH01A5XKF r 59 N25ScaffoldBases= FO77AEH01CKY2Z r 104 N50ScaffoldBases= FO77AEH01BD0T r 146 N75ScaffoldBases= FO76IZV02R25OI f 85 ScaffoldAt = FO76IZV02P6FZH f 66 ScaffoldAt =1027 FPDPEIT01BA86D r Hybrid Error Correction/

33 Acknowledgements NBACC Adam Phillippy Cold Spring Harbor Michael Schatz, Richard McCombie J. Craig Venter Institute Brian Walenz DOE Joint Genome Institute Jeffrey Martin, Zhong Wang Duke University / Howard Hughes Medical Institute Erich Jarvis, Jason Howard, Ganeshkumar Ganapathy University of Maryland Institute for Genome Sciences David Rasko USDA Timothy Smith, Gregory Harhay, Dayna Harhay, Scott McVey Pacific Biosciences Edwin Hauw, Aaron Klammer The Assemblathon Hybrid Error Correction/

34 This Document was prepared for the Department of Homeland Security (DHS) by the Battelle National Biodefense Institute, LLC (BNBI) as part of contract HSHQDC-07-C to manage and operate the National Biodefense Analysis and Countermeasures Center (NBACC), a Federally Funded Research and Development Center. In no event shall the DHS, BNBI or NBACC have any responsibility or liability for any use, misuse, inability to use, or reliance upon the information contained herein. In addition, no warranty of fitness for a particular purpose, merchantability, accuracy or adequacy is provided regarding the contents of this document.