Preservation and Handling of Media & Digital Works Howard Besser, Director Moving Image Archiving & Preservation Program NYU s Tisch School of the Arts http://www.tisch.nyu.edu/preservation Preservation and Handling of Media & Digital Works The problems posed by Digital Works, and the need for Digital Curation and Stewardship Obsolescence & Reformatting, and how Audiovisual collections have been doing this for a long time NYU s MIAP Program & our work with Libraries and other collections The Short Life of Digital Info: Digital Longevity Problems- Digital Preservation Digital Preservation The Viewing Problem The Scrambling Problem Digital Curation The Inter-relation Problem The Custodial Problem The Translation Problem The Viewing Problem Digital Info requires a whole infrastructure to view it Each piece of that infrastructure is changing at an incredibly rapid rate How can we ever hope to deal with all the permutations and combinations Viewing Problem Requires new file formats and new physical strata at regular intervals Needs a serious Managed Environment Main InterPARES finding--the need for complete lifecycle management archivist needs to be incolved when record is created and throughout active life 1
The Scrambling Problem Dangers from: Digital Curation Compression to ease storage & delivery Encryption to enhance digital commerce The Inter-relation Problem -Info is increasingly inter-related to other info -How do we make our own Info persist when it points to and integrates with Info owned by others? -What is the boundary of a set of information (or even of a digital object)? The Custodial Problem In the past, much of survival was due to redundancy How do we decide what to save? Who should save it? Mellon-funded E-Journal Archives How should they save it?- The Custodial Problem: How to save information? Methods for later access Refreshing Migration Emulation Issues of authenticity and evidence The Translation Problem Content translated into new delivery devices changes meaning -A photo vs. a painting -If Info is produced originally in digital form in one encoded format, will it be the same when translated into another format? Behaviors
The Translation Problem Thinking of the Future (1/) Screens will be different resolutions and different aspect ratios CRTs won t exist A decade or from now, today s user interfaces will look like arrow-key navigation looks like today The Translation Problem Thinking of the Future (/) Today s streaming media are small windows, slow speeds As bandwidth increases, viewers will expect higher quality streams Creators may need to consider how they ll be able to deliver higher-bandwidth streams Delivery Derivatives vs. Masters encoded w/standards May also want to re-edit the piece to take advantage of changes in technology, viewer expectations, society- Screen Formats Responding to serious Longevity Problems Previous formats required little ongoing intervention (remote storage facilities, Iron Mtn); digital formats require intense ongoing management Need for: Preservation Repositories Preservation Metadata Digital Stewardship definitions acquire, manage, organize, preserve, and provide access to massive amounts of data for use and re-use by a variety of interdisciplinary and heterogeneous communities over time Preservation + Use Besser-Media Art Conservation, /7/09 Digital Stewardship: Harvard Library OIS definition the management of digital objects over the long term through careful digital asset management practices. Collection managers and DRS staff must work together to manage stored digital objects throughout all phases of the objects' life cycle. This section describes the major areas of digital stewardship in the life cycle of a digital object: Assessment and selection phase: Collection manager performs a curatorial assessment of materials intended for DRS storage. Acquisition and creation phase: Collection manager (in consultation with HUL analysts) selects digital formats and defines technical specifications and workflow processes for creation of objects and related metadata. Deposit phase: DRS ensures a successful deposit by validating each package of digital objects and related metadata that is submitted to DRS. Archive and preservation phase: DRS staff perform periodic checks to ensure the usability of digital objects over time. This includes periodic reports to collection managers about their objects. Besser-Media Art Conservation, /7/09 3
Digital Stewardship Digital Curation Includes much of what cultural institutions have always done (select, acquire, preserve, provide access, maintain context) But also adds some of the activities that audiovisual collections have always done (maintain knowledge of the original equipment and formats, manage process of reformatting when older hardware is difficult to maintain Also includes important notions from the digital world about ongoing management of re-formatting information, technical metadata, contextual info, repository issues, Besser-Media Art Conservation, /7/09 Besser-Media Art Conservation, /7/09 DCC Digital Curation Life-cycle http://www.dcc.ac.uk/ Digital Curation Planning & risk management Open flexible file formats Format registries OAIS repositories Extensive Metadata Individuals & organizations- Besser-Media Art Conservation, /7/09 Besser-Media Art Conservation, /7/09 Organisation to Engage & Collaborate communities of practice: users Associates Network Industry testbeds & tools Besser-Media Art Conservation, /7/09 OCLC October 006 service definition & delivery a centre of expertise in data curation and preservation community support & outreach management & admin support development co-ordination research curation organisations eg DPC standards bodies research collaborators Digital Preservation Players Collection staff (need to reach agreement on SIP/DIP and acceptable AIP transformations) preservation/conservation staff metadata staff access staff Repository staff Agreement negotiators Besser-Media Art Conservation, /7/09 4
Preservation Repositories: too difficult for small institutions Too complex for small institutions to manage Will be done through partnering (small museum or dance company with University) or through consortia (museum association, state-wide organization, ) or through service bureaus (OCLC) Archive or museum will direct what is needed, but digital repository will carry out the actual work (as defined in SIP/DIP/AIP agreement) Storage Media Removable media (like CDs) is not a long -term answer The long-term answer requires ongoing management, and involves regular migration or emulation. This solution is only viable with storage on spinning disks- Besser-Media Art Conservation, /7/09 Besser-Media Art Conservation, /7/09 Storing on CDs becomes a big problem over time Consumers replace their CDs with a hard disk (& so should you) Besser-Media Art Conservation, /7/09 Besser-Media Art Conservation, /7/09 Plain DVDs are no longer the latest format Obsolescence Besser-Media Art Conservation, /7/09 5
Difficult Materials become obsolete relatively quickly Obsolete or deteriorated Physical Carriers The physical carriers decay or become obsolete The technology required to view the carriers changes frequently The encoding formats needed to decode the content shift Obsolete Carriers & Info Techn Obsolete Carriers Obsolete Carrier viewing Technology? Kodak stops making some films 6
List of old Audio Formats Format Description Years in Use Wax Cylinder Records - or 4-minute formats, wax 1888 199 or wax compound Recordable Disc Records 7, 1, or 16, recorded at 199 1960s (Direct or Acetate Discs) 33 or 78 revolutions per minute (rpm). Generally vinyl on a paper, glass or metal base Recording Wire Spooled wire, usually in 15- c. 1945 1955 to 30- minute lengths, one direction only Open reel recording tape 1/4, 3 10 1/" reels, c. 1945 Present 1 7/8 30 inches per second (IPS) speeds Compact Cassette 1/8 tape in hard case, 1 7/8 1965 Present IPS format Microcassette/Minicassette Very small -4 cm cassette 1977 Present Digital disk, MP3, and other digital recorders tapes Audio recorded directly in digital files to optical disks or internal hard drives 000 Present Old Video Formats (www.vidipax.com) Keep Old Video Equipment--NARA Keep Old Video Equipment--NARA Keep Old Audio Equipment-NARA Reformatting and Ethics 7
Re-formatting is not a new idea What is Reformatting? A form of copying Usually copied onto a medium having different physical characteristics than the original physical strata Examples Document on acidic paper onto non-acidic paper Newspaper microfilming History of Conservation & Preservation Reformatting In ancient times, in the library of Pamphilus at Caesaria, badly damaged papyrus manuscript pages were replaced with parchment (which was stronger) -Saint Jerome The Bible was hand-copied for millenia 1964 - US Newberry Library (Paul Banks) began 1st US institutional preservation program 1987 - US NEH begins funding massive microfilming of brittle paper (mainly newspapers) Why do we Reformat?- Brittle Newspapers (Australia Battye Library) Film Decay (LC Dayton) 8
Why do we Reformat? Edison Because we cannot sustain the original object (its physical characteristics are deteriorating too fast) Because continued access and handling of the original object will rapidly decay its physical characteristics (so we create a surrogate for users and store the original in very good conditions, away from users) Because viewing the work requires some kind of technology, and we can t keep that technology working very far into the future- Metal sound recording Disks Casa Rui Barbosa Paper print (LC Dayton) Record Turntables Slide Projector 9
Limitations of Reformatting Authenticity issues (more later) User behaviors (newspaper, book, video game, ) Users mistaking the reformatted work for the original Critiques of Reformatting Mainly User Behaviors Can t view outside the library Only sequential access Viewing and studying is awkward But unless we Reformat, we totally lose some kinds of works (particularly audiovisual works like film) 50% of all titles produced before 1950 have vanished (approximate number as of late 1970s) This reflects full-length features; survival rates are much lower for other types (studio newsreels, shorts, docs, independent, ), and these orphans are particularly in peril Fewer than 0% of features from 190s survive in complete form; survival rates of 1910s is <10% (& none of these are negatives) -Film Preservation 1993: A Study of the Current State of American Film Preservation, Vol 1: Report, June 1993, Report of the Librarian of Congress (http://www.loc.gov/film/study.html) And sometimes we have to reformat because of technology changes We don t have video players to play tapes made 5 years ago We don t have 8-inch floppy disk drives, syquest drives, zip drives We don t have Windows 3 operating systems But this is something that conservators have always dealt with Authenticity Issues with reformatting- Is the work what it purports to be? Commercial reformatting examples Media archivists and interpretation Beyond Film & Video Commercial Reformatting Issues: PR at 1967 re-issue of GWTW Spherical Blow-Up In the Splendor of 70mm. Wide screen and full stereophonic sound! For the thousands who remember its unparalleled drama, action and romance! For the new thousands to whom the wonders will be revealed for the first time! Breathtaking spectacle, inspired acting by the greatest cast ever assembled! The screen's most exciting love story! The most-talked about picture ever made! http://www.imdb.com/title/tt0031381/taglines 10
But most people didn t know that 70mm widescreen is different shape than 35mm normal Change in Aspect Ratio forces cutting Pan and Scan example This meant eliminating part of the frame 7 Brides for 7 Brothers, Stanley Donen, MGM, 1954 from http:/ /en.wikipedia.org/wiki/pan_and_scan Eliminating even in famous scenes But the re-release wasn t governed by artistic concerns Intellectual Property is owned by MGM, not by Fleming or Selznick MGM is motivated maximizing profit, not in maintaining artistic integrity Bigger is always better Not radically different than another blow to artistic integrity/originality in Atlanta 0 years later 11
Atlanta was also home of 1980s Colorization movement Sometimes even the Director wants to go back and change their original film Star Wars 1977 vs 004 Someone needs to maintain the integrity of artistic works This means preserving original versions even when commercial interests want to replace the older version with something new and fancy even when the artist wants to use more recent technological developments to improve their work Media Archivists Viewing Context (images like in Dayton-Hudson) Preserve original works Provide access to older versions Maintain the integrity of the original in any restoration process Champion works that do not have commercial entities pushing for their preservation and distribution Try to make sure that works are viewed within their original context- 1
Be concerned about For preservation you may need to re-format, but with recent changes in copyright laws, you may not have the right to re-format Intellectual property rights are very difficult, particularly considering that most films and videos have extensive underlying rights that you could never get prior permission for (stock footage, historical footage, music composition, music performance, ) [ Eyes on the Prize ] And even if you have the right to re-format for preservation, you might not have the right to show what you have preserved Possible endless need for reformatting implies Possible loss with each generation Requires managed environment Can lead to violations Managed Environment Media Repositories- More than temperature & humidity control Periodic monitoring of the works Periodic monitoring of the technical environment for viewing the works (software, systems, hardware) Trusted repositories Univ of GA Walter J Brown Media Archives Univ of GA Walter J Brown Media Archives 13
Univ of GA Walter J Brown Media Archives Univ of GA Walter J Brown Media Archives Univ of GA Walter J Brown Media Archives Hogan Jazz Archive Tulane Univ Library Cinemateca Brasileira (video storage) Hampton Collection (interviews) 14
Various Formats Intermixed (Hampton) MIAP Projects (Paper Tiger) MIAP Projects (Paper Tiger) Orphan Works Home Movies--Emanuel Goldberg Amateur Film-- UGA Walter J Brown Media Archive Athens GA, 1947 Shows contrast btwn White and African -American neighborhoods http://www.libs.uga.edu/media/collections/homemovies/housing.html 15
Wolfson Florida Moving Image Archive http://youtube.com/watch?v=vhwa307owq0 Palmour Street (1957) Gainesville, GA http://www.youtube.com/watch?v=vpgwzcehx30 Educational film from Southern Educational Film Production Service Prelinger Archives Physical Deterioration Hampton Collection (atmosphere cntrl) Academy-Atmosphere % RH IPI Preservation Index temperature/humidity, Years until noticeable deterioration Tempera ture Cº º 7º 13º 18º 4º 9º 35º 0 150 600 50 15 60 30 16 30 900 400 00 90 45 5 1 40 700 300 150 150 70 35 18 10 50 500 50 100 50 5 14 7 60 350 175 80 40 0 11 6 70 50 15 60 30 16 9 5 80 00 100 50 5 13 7 4 16
Temperature & Humidity for Tape Storage Variance of less than ºC and 5% RH per 4 hours Ideally 8ºC and 5% RH Other options 0 C (68 F) and 0-30% RH 15 C (59 F) and 0-40% RH 10 C (50 F) and 0-50% RH Never store below 8ºC amianet.org 003, & ISO 1893 Oral/Visual Histories- New distribution: itunes-u UCB Oral Histories itunes-u UCB Campus Events itunes-u Duster Video Do we trust itunes to preserve these? 17
The Re-Mix generation Dueling Videos: Scholar Creates Remix of Another Academic's YouTube Hit http://chronicle.com/wiredcampus/article/707/dueling-videos-scholar-creates-spoof-of-another-academics-youtube-hit Chronicle report on Center for Social Media Re-Mix project http://chronicle.com/wiredcampus/article /638/video-report-a-legal-defense-of -videos-riffing-on-popular-culture Some Re-Mix Videos Baby Got Book http://www.godtube.com/view_video.php?viewkey=97759aa7a0c99bff671 Bush vs Zombies http://www.youtube.com/watch?v=ioxgrtdysly George Bush Don t Like Black People http://revver.com/video/71633/george-bush-dont-like-black-people/ Victory in Iraq http://revver.com/video/79501/victory-in-iraq/ Fox News Edits a Democrat to Make Him Look Worse http://www.youtube.com/watch?v=ogqpxn7njqm xx Born-digital images Where is the original? Born-digital works are both easier and harder to preserve than analog works +With a born-digital work, we don t have to worry so much about the original artifact (there really isn t one) -We know that digital works face a whole range of obsolescence problems, so they must be reformatted at least once per decade 18
What can you do now? For both Film & Video Label elements as well as you can Try to keep things at a low humidity and temperature Limit the number of formats as much as possible Save important production elements Risk Management We can t say definitively that we can make every digital work persist What we CAN say is that the more a digital work conforms to standards and best practices, the greater the likelihood that we can assure persistance Our preservation repositories can even accept deposits of non -conforming works, but the less they conform, the less likely that they ll be salvageable Persistance is most likely for works that share standards, metadata, and best practices Reformatting Best Practices (still images) Think about users (and potential users), uses, and type of material/collection Scan at the highest quality that does not exceed the likely potential users/uses /material Do not let today s delivery limitations influence your scanning file sizes; understand the difference between digital masters and derivative files used for delivery Many documents which appear to be bitonal actually are better represented with greyscale scans Include color bar and ruler in the scan Use objective measurements to determine scanner settings (do NOT attempt to make the image good on your particular monitor or use image processing to color correct) Don t use lossy compression Store in a common (standardized) file format Capture as much metadata as is reasonably possible (including metadata about the scanning process itself) So, with electronic works, the focus should be less on stable temperature (Helsinki underground vaults) And less on the construction of Vaults (Helsinki underground vaults) Physical preservation Paradigms Shifts needed Old atmospheric cntrl New ongoing mgmt What to save? artifact idea + ancillary material & documentation Cataloging Later access Individual work in hand Artifact & documentation FRBR Restaging, ancillary material & documentation 19
Preservation and Handling of Media & Digital Works Howard Besser NYU Moving Image Archiving & Preservation Program http://www.tisch.nyu.edu/preservation http://besser.tsoa.nyu.edu/howard/talks/ http://www.ptvdigitalarchive.org/ http://www.dcc.ac.uk/ http://www.digitalpreservation.gov/ http://www.amianet.org/ http://www.clir.org/pubs/reports/pub54/ http://sunsite.berkeley.edu/longevity/ http://www.imagepermanenceinstitute.org/ http://www.screensound.gov.au/screensound/screenso.nsf/ 0