Tape/Disk and Format Guidelines Tape From 1.4 Mbytes to 4 Terabytes Disk A Little History Lesson And Where the Future Takes Us 1
What s In Your Library Tape Disk Various Formats Format Compatibility across Disciplines? 5 July 2012 2
Not Rocket Science Do It Once - Do It Right 3
Tape over the years Analogue Digital 1.4 Mbytes to 4/5 Terabytes History? How to you organise this quantity of data Area of Interest Data Type Encapsulation Tar RODE Lacy Internal 4
ANALOGUE FM AM Carter United Belt Techno 24 Trace Techno 48 Trace Smov Magnadisk Maximum 4 shots (only Magnadisk) Maximum 48 Traces (only Techno double width) 5
1 st 7 Track Technology IBM 726 NOT ADOPTED 2 Mbytes of data 1952 Read data both backwards and forwards!! 6
21 Track (Texas Instruments) 1958 197? 356 bpi 1200 ft 20 Mbytes 356 bpi 2400ft 40 Mbytes 712 bpi 1200ft 40 Mbytes 712 bpi 2400ft 80 Mbytes 7
7 Track (1964 1979) 200 bpi 556 bpi 800 bpi 6 Data Tracks, 1 Parity Track Capacity of: 8
9 Track (1964 1994) 800 bpi 1600 bpi 3200bpi 6250 bpi (9042 bpi) Error Correction 2 Tracks Capacity 20 Mbytes 170 Mbytes (113 Mbytes if small blocks) 9
3480 and 3490 1984-2004 200 800 Mbytes 3 Mbytes per Second 10
3590 3590B 10 Gig 3590E 20 Gig 3590H 30 Gig 1995 2006 DLT Ceased Production 2009 11
Media Failures 36 Track HDDR D2 D1 Exabyte TK50 etc DLT
TS1140 4rd Generation of 3592 enterprise tape drive Introducing new Barium Ferrite media types with up to 4 TB native capacity Re-Writable and Write Once Read Many (WORM) cartridge at 4TB Economy cartridge available at 500GB 250 MBps native drive data rate Dual 8Gb fibre channel interfaces with 650 MB/s max compressed data rate Supports data partitioning, data encryption and key management Differentiated Media New media types usable at higher capacity on future drive generations Media re-use of existing JB/JX media types with automatic Upformat support 100 % factory tested media, improved physical durability and usage life MES upgrade for TS1130 available (Model Conversion) Attaches to Selected HP and Sun Microsystems servers Selected versions of Microsoft Windows Selected Linux editions Supported in IBM TS3500 tape libraries IBM Rack 13
StorageTek T10000C 2011 -? 5 Terabytes Native 240 Mbytes / second Sectored? Barium Ferrite 14
Appendix F The table below indicates the maximum allowable block size for accepted types of media. It is expected that this table will need to be updated approximately once per year. Device Type Maximum Block Size 3480 128 Kilobytes 3490, 3490E 256 Kilobytes 3590 512 Kilobytes DST 1,199,840 Bytes Redwood 256 Kilobytes 1/2 Round Tape 64 Kilobytes Kilobyte is defined as 1024 bytes 15
Oil Company Workflow Specification Contract Acquire (SEGD) Storage (QC?) Processes (SEGY) Storage (QC?) Delivery of Data to Oil Company Project Interpretation 16
Exchange Formats Data and the Exchange Formats The basis of all, data, information and knowledge Without Digital Samples You Cannot Find Oil and Gas Tape And/Or Disk 17
TAPE Sample Data Exchange Formats Began As Tape Formats 18
TAPE MAP 19
ENCAPSULATION or NATIVE SEGD and SEGY Native on Disk Uses information in header record to create byte stream SEGY Binary Header, number of samples per trace SEGD Calculates number of samples per trace 20
SEGY EBCDIC Header 3205-3208 * Line number (only one line per reel). 3209-3212 * Reel number. 3213-3214 * Number of data traces per record 3215-3216 * Number of auxiliary traces per record 3217-3218 * Sample interval in µsec (for this reel of data) 3219-3220 * Sample interval in µsec (for original field recording) 3221-3222 * Number of samples per data trace (for this reel ) 3223-3224 * Number of samples per data trace (for original ) 3225-3226 * Data sample format code 3227-3228 * CDP fold (expected number of traces per CDP/ensemble). 21
SEGY Binary Header 3201-3204 Job number: 375860 3205-3208 * Line number: 1234 3209-3212 * Reel number: 1 3213-3214 * Data traces per ensemble: 240 3215-3216 * Aux traces per ensemble: 2 3221-3222 * Sample interval (this reel): 2000 3221-3222 * Samples per trace (this reel): 3072 3225-3226 * Sample format code: 1 (IBM 32-bit FP) 3229-3230 Trace sort code: 1 (as recorded (field)) 22
SEGY Trace Header 1-4 * Trace sequence number within line-numbers continue to increase if additional reels are required on same line. 9 12 * Original field record number. 13-16 * Trace number within the original field record. 29-30 * Trace identification code: 1 = seismic data 4 = time break 7 = timing 2 = dead 5 = uphole 8 = water break 3 = dummy 6 = sweep 9---- N = optional use 115-116 117-118 * Number of samples in this trace. * Sample interval in µsec for this trace 23
Stanza Example SEGY Rev 1.0 Stanza ((JJ ESeis: Microseismic Geometry Definition ver 1.0)) Definer name = J and J Example Seismic Ltd. Line Name Convention = CDA Line Name = Sample MicroSeismic 1 First Trace In Data Set = 101 Last Trace In Data Set = 1021 First SP In Data Set = 2001 Last SP In Data Set = 6032 ((SEG: Coverage Perimeter ver 1.0)) Coverage type =full-fold Perimeter coordinate type =I,J Perimeter node number =10 Perimeter node coordinates =334.0000,908.0000 Perimeter node coordinates =654.0000,908.0000 Perimeter node coordinates =654.0000,833.0000 Perimeter node coordinates =900.0000,833.0000 Perimeter node coordinates =900.0000,721.0000 Coverage Perimeter comment =48 fold data ((SEG: Measurement Units ver 1.0)) Data Sample Measurement Unit =Millivolts Volt conversion =0.001 additional stanzas or blank records to end of 3200-byte Extended Textual Header ((SEG: EndText)) blank records to end of 3200-byte Extended Textual Header First Trace Header 24
SEGD Header ---- Format recognized as SEGD_DMX ---- File number: 128 SEGD format code: 8024 (16-bit quaternary exponent demultiplexed) SEGD format revision: 0000 Base scan interval: 2 millisec Record length: 5632 millisec Date and time: 94/049 15:55:14 Record type: 8 (normal record) Manufacturer code: 22 (Geco/Prakla) Scan Chan Chan No Sample Start End Type Set Type Chans Int Time Time 1 1 6 27 2 0 5632 1 2 1 960 2 0 5632 Normal {Normal completion} 25
QUALITY CONTROL - Format Format Compliance e.g. SEGD Checker SEG-D Checker Test Summary CheckerVersion v1.0 BuildDate 21:59:14 Aug 29 2011 TestTime 17:21:50 30 Jul 2011 TapeLabelPresent 1 SEGD Rev 3.0 NumShots 1 ShotsPassed 1 ShotsFailedDecode 0 ShotsFailedValidate 0 NumErrors 0 NumWarnings 0 26
SEGD ON DISK T-Max Minus T Zero End Time Minus Start Time DT First Sample at Time Zero = Extra Sample 27
SEGD ON DISK - LAYOUT 28
The Future SEGD 3.0 Improved Handshake to QC and Processing Measurements Table (Open Spirit/Energistics) Table of Contents - Survey Information Co-ordinate Reference System- EPSG (OGP) Compliant Units of Measurement Table (Energistics) Time Stamp GPS EpochMu Trailer Edits etc Increased Flexibility Data on Disk Multi-discipline 29
OGP Example 5 July 2012 30
Standards within Standards The Future SEGY SPS P1/P2 P6 Header Similarity Extension for Disciplines CSV? XML? 31
ENCAPSULATION TIF - Tape Image File Baker Hughes method for writing BIT wire line format data to disk Native limit, pointer signed 2 Gig Native limit, pointer unsigned 4Gig Virtual Pointers Extended TIF many versions, not a published or ratified exchange standard 32
ENCAPSULATION TAR Tape ARchive also known as tarball Standardized by POSIX.1-1988 and later POSIX.1-2001. Implementations Unix TAR, Linux TAR, Windows WinTar, 32 bit TAR, 64 bit TAR 33
ENCAPSULATION RODE Record Oriented Data Encapsulation SEG data exchange standard Meta-data entry fields Based on RP66 as is DLIS Complex but published and reliable if using correct implementation 34
MBytes/s CERN Report Slide 1 Writing Data - Transfer Rates MBytes/s with blocksize, I00004 40 35 30 25 20 15 10 5 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Blocks (K) 35
Seconds/file Cartridge Capacity Labelled files/s with size in 256Kblocks, 163AAC 1 0.8 0.6 0.4 0.2 0 1 3 5 7 9 11 13 15 17 19 Blocks (256 K) 36
SEG Standards within Standards Technical Standards Committee OGP Surveying and Positioning Committee Energistics Units of Measure Standards Within Standards Positioning Measurements Tables Manufacturers Codes 37
Updates OGP positioning exchange formats Headers in line with SEGD Energistics Expanding the Units of Measurements Tables SEG SPS format Headers in line with OGP and SEG formats SEG Update of SEGD Enable reading of formats as cave and xml files 38
Revisit Old Data Quote from IBM web-site Effective September 29, 2006, IBM will withdraw from marketing the 3590 Enterprise Tape Drive. Tape currently most common 600 Gig 3592 High Enterprise Drive (the only one) Now released the 4 Terabyte TS 1140 tape Robot TS 3500 2.5 Exabytes (2,500 Petabytes) 39
What do They Say? SEG Compliant SEGD Nothing Cut and Paste Contracts Needle in Haystack How do you organize 600 Gbytes of data 4 Hrs to 20 Hrs 5 July 2012 40
350 square kilometre 3D seismic survey on Block L in Brunei Darusallam. The survey, costing a total of approximately US$15.3 million, is expected to commence in Q1 of 2008. 41
Apache 3D + 4D 30,000 BOE To 75,000 BOE + 42
In Conclusion Use Standards and Good Practices Properly specify what you want Free advise from professional body From TODAY Improve things SEG, OGP and Energistics working in harmony NPD Mandate SEGD 3.0 for Norwegian Waters Historic Data 10 year cycle of proactive data management 43
SEG TSC Pro-Active Geophysical Data Management Do It Once Do It Right 44
Thank You Jill Lewis, Managing Director Troika International Technical Standards Committee, SEG Liaison SEG/OGP Co-Chair Global Affairs Committee Europe and FSU SEG, PESGB, AAPG, SPE, EI, HGS, GSH, GOSH, EAGE, APSG, SPE... 45