UNITED STATES PATENT AND TRADEMARK OFFICE BEFORE THE PATENT TRIAL AND APPEAL BOARD. Samsung Electronics America, Inc., Petitioner

Transcription

1 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 UNITED STATES PATENT AND TRADEMARK OFFICE BEFORE THE PATENT TRIAL AND APPEAL BOARD Samsung Electronics America, Inc., Petitioner v. Prisua Engineering Corp., Patent Owner. Patent No. 8,650,591 Filing Date: March 8, 2011 TITLE: VIDEO ENABLED DIGITAL DEVICES FOR EMBEDDING USER DATA IN INTERACTIVE APPLICATIONS DECLARATION OF EDWARD DELP III, PH.D. Inter Partes Review No Petitioner Samsung 1003

2 TABLE OF CONTENTS PAGE I. INTRODUCTION AND QUALIFICATIONS... 1 A. Education... 1 B. Career... 1 C. Materials Considered... 7 II. LEGAL PRINCIPLES USED IN THE ANALYSIS... 8 A. Person Having Ordinary Skill in the Art... 8 B. Prior Art C. Identification of Grounds of Unpatentability D. Broadest Reasonable Interpretations III. THE 591 PATENT IV. GROUND 1: CLAIMS 1, 2, 8, AND 11 ARE ANTICIPATED OR RENDERED OBVIOUS BY SENFTNER A. Overview of Senftner B. Challenged Claims V. GROUND 2: CLAIMS 3 AND 4 ARE OBVIOUS OVER SENFTNER IN VIEW OF LEVOY VI. VII. A. Overview of Levoy B. Motivation to Combine Senftner and Levoy C. Challenged Claims GROUND 3: CLAIMS 1, 2, 8, AND 11 ARE OBVIOUS OVER SITRICK A. Overview of Sitrick B. Challenged Claims GROUND 4: CLAIMS 3 AND 4 ARE OBVIOUS OVER SITRICK IN VIEW OF LEVOY A. Motivation to Combine Sitrick and Levoy B. Challenged Claims VIII. CONCLUDING STATEMENT APPENDICES A-E... A-1 2

3 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 I, Edward J. Delp III, declare as follows: I. INTRODUCTION AND QUALIFICATIONS 1. I have been engaged by Petitioner Samsung Electronics America, Inc., ( Samsung ) to opine on certain matters regarding U.S. Patent No. 8,650,591 ( 591 patent ). Specifically, this declaration addresses the validity of claims 1-4, 8, and 11 (the Challenged Claims ) of the 591 patent in light of the prior art. I receive $650 per hour for my services. No part of my compensation is dependent on my opinions or on the outcome of this proceeding. I have no financial interest, beneficial or otherwise, in any of the parties to this review. A. Education 2. I received my Bachelor of Science degree in Electrical Engineering from the University of Cincinnati in 1973; my Master of Science degree from the University of Cincinnati in 1975; and my Ph.D. in electrical engineering from Purdue University in In May 2002, I received an Honorary Doctor of Technology from the Tampere University of Technology in Tampere, Finland; this award cited my work in signal processing. My C.V. is appended to the end of this Declaration as Appendix E. B. Career 3. I am a Distinguished Professor in the School of Electrical and 1 3

4 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 Computer Engineering at Purdue University in West Lafayette, Indiana. I also have a joint appointment in the Weldon School of Biomedical Engineering. My professorship is sponsored by the Charles William Harrison endowment, which donated the $1.5 million dollars to Purdue to establish my position. My official title is the Charles William Harrison Distinguished Professor of Electrical and Computer Engineering. There are 81 Distinguished Professors at Purdue University out of a total of 10,900 faculty members. 4. Purdue University is one of the largest and oldest engineering schools in the United States. It is consistently ranked in the top 10 engineering schools in the Unites States. One out of every 14 engineers in the United States is a Purdue graduate. Purdue graduates have had a tremendous impact on engineering developments and practices in the U.S. For example, the first (Neil Armstrong) and last (Eugene Cernan) persons to walk on the moon were Purdue graduates. Two of the engineers who won Emmy awards for their work in imaging technology, Bill Beyers and Lauren Christopher, are Purdue graduates. 5. My expertise includes digital signal processing, the processing and coding of image, video, and audio signals, and embedded and mobile applications. The image compression algorithm I developed as part of my Ph.D. thesis, block truncation coding, has been used extensively in many applications 2 4

5 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 and was one of the final candidates of the JPEG compression standard. It was used by NASA to send images back to Earth from the Mars Pathfinder MicroRover, which landed safely on Mars on July 4, I am a Professional Engineer. I am registered in the State of Ohio (registration number E-45364). 7. As a professor at Purdue University and the University of Michigan, I have performed extensive research relating to signal, image, and video processing techniques and embedded systems. Over the last ten years, I have supervised the research and preparation of more than 45 Ph.D. theses relating to topics in signal, image, and video processing. As part of my continuing research in the field of signal, image, and video processing, I have studied new developments in the field of signal, image, and video processing and embedded systems, analyzed the publications of other leaders in this field, and participated in industry organizations chartered to study the processing of images, video, and audio signals. My experience in the field of signal processing includes over 30 years of directed research, as well as the materials taught in my classes at Purdue University and the University of Michigan. 8. I have been studying subject matter relating to image and video processing since approximately 1975, in connection with grants provided by the 3 5

6 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 National Science Foundation, the National Institutes of Health, the Department of Defense, the Department of Homeland Security, Department of Energy and various corporations, including Texas Instruments, Samsung, Motorola, Nokia, Google and Thomson. 9. I also have been elected a Fellow of the Institute of Electrical and Electronics Engineers (IEEE), the Society for Imaging Science and Technology (IS&T), the International Society for Optical Engineering (SPIE), and the American Institute of Medical and Biological Engineering. In 2004, I received the Technical Achievement Award from the IEEE Signal Processing Society for my work in image and video compression and multimedia security. In 2008, I received the Society Award from IEEE Signal Processing Society (SPS). This is the highest award given by the SPS. 10. In 2009, I received the Purdue College of Engineering Faculty Excellence Award for Research. 11. In 2015, I was named Electronic Imaging Scientist of the Year by the IS&T and SPIE. The Scientist of the Year award is given annually to a member of the electronic imaging community who has demonstrated excellence and commanded the respect of his/her peers by making significant and substantial contributions to the field of electronic imaging via research, publications and 4 6

7 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 service. I was cited for my contributions to multimedia security and image and video compression. 12. I received the 2017 SPIE Technology Achievement Award. The SPIE Technology Achievement award is awarded annually to recognize outstanding technical accomplishment in optics, electro-optics, photonic engineering, or imaging. The SPIE Awards Committee has made this recommendation in recognition of my pioneering work in multimedia security including watermarking and device forensics and for his contributions to image and video compression. 13. In 2016, I received the Purdue College of Engineering Mentoring Award for my work in mentoring junior faculty and women graduate students. 14. In 1990, I received the Honeywell Award and in 1992 the D. D. Ewing Award, both for excellence in teaching. In 2001, I received the Raymond C. Bowman Award for fostering education in imaging science from the Society for Imaging Science and Technology (IS&T). In 2004, I received the Wilfred Hesselberth Award for Teaching Excellence. In 2000, I was selected a Distinguished Lecturer of the IEEE Signal Processing Society, and I gave lectures in France, Spain, and Australia on signal, image, and video processing. I 5 7

8 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 am a Distinguished Professor in the School of Electrical and Computer Engineering at Purdue University in West Lafayette, Indiana. I also have a joint appointment in the Weldon School of Biomedical Engineering. My professorship is sponsored by the Charles William Harrison endowment, which donated the $1.5 million dollars to Purdue to establish my position. My official title is the Charles William Harrison Distinguished Professor of Electrical and Computer Engineering. There are 81 Distinguished Professors at Purdue University out of a total of 10,900 faculty members. 15. I am a Professional Engineer. I am registered in the State of Ohio (registration number E-45364). 16. As a professor at Purdue University and the University of Michigan, I have performed extensive research relating to signal, image, and video processing techniques and embedded systems. Over the last ten years, I have supervised the research and preparation of more than 45 Ph.D. theses relating to topics in signal, image, and video processing. As part of my continuing research in the field of signal, image, and video processing, I have studied new developments in the field of signal, image, and video processing and embedded systems, analyzed the publications of other leaders in this field, and participated in industry organizations chartered to study the processing of images, 6 8

9 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 video, and audio signals. My experience in the field of signal processing includes over 30 years of directed research, as well as the materials taught in my classes at Purdue University and the University of Michigan. C. Materials Considered 17. The analysis that I provide in this Declaration is based on my experience, as well as the documents I have considered, including U.S. Patent No. 8,650,591 (the 591 patent) (Ex. 1001). I have also reviewed the prosecution history of the 591 patent and the materials listed below: LIST OF EXHIBITS Exhibit Number Document 1001 U.S. Patent No. 8,650,591 ( 591 patent ) 1002 File history of U.S. Patent No. 8,650,591 ( 591 FH ) 1004 Deposition of Dr. Yolanda Prieto, Prisua Engineering Corp. v. Samsung Electronics Co., Ltd., Case No. 16-cv KMM (Jan. 17, 2017) 1005 Joint Claim Construction Statement, Prisua Engineering Corp. v. Samsung Electronics Co., Ltd., Case No. 16-cv KMM, Docket No U.S. Patent No. 7,460,731 to Senftner et al. ( Senftner ) 1007 U.S. Patent Application Publication No. 2005/ to Sitrick ( Sitrick ) 1008 U.S. Patent Application Publication No. 2009/ to Levoy et al. ( Levoy ) 1009 Negahdaripour Decl. ISO Patent Owner s Opening Claim Construction Brief ( Negahdaripour Decl. ) 1010 U.S. Patent Application Publication No. 2006/

10 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 Exhibit Number Document 1011 U.S. Patent No. 7,307,623 to Enomoto 1012 U.S. Patent No. 4,686,332 to Greanias et al Edward Delp Decl. ISO Petitioner s Responsive Claim Construction Brief 1014 U.S. Patent Application Publication No. 2008/ to Zeev Russak et al. II. LEGAL PRINCIPLES USED IN THE ANALYSIS 18. Attorneys for the Petitioner have explained certain legal principles to me that I have relied upon in forming my opinions set forth in this report. I have also relied on my personal knowledge gained through experiences and exposure to the field of patent law. A. Person Having Ordinary Skill in the Art 19. I understand that my assessment of claims of the 591 patent must be undertaken from the perspective of what would have been known or understood by a person having ordinary skill in the art, reading the 591 patent on its relevant filing date and in light of the specification and file history of the 591 patent. I will refer to such a person as a POSITA. 20. The 591 patent claims priority to U.S. Provisional Patent Application No. 61/311,892, filed March 9, For purposes of this 8 10

11 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 declaration only, I have assumed that all the Challenged Claims are entitled to a priority date of March 9, Counsel has advised me that, to determine the appropriate level of one of ordinary skill in the art, the following four factors may be considered: (a) the types of problems encountered by those working in the field and prior art solutions thereto; (b) the sophistication of the technology in question, and the rapidity with which innovations occur in the field; (c) the educational level of active workers in the field; and (d) the educational level of the inventor. 22. I am well acquainted with the level of ordinary skill required to implement the subject matter of the 591 patent. I have direct experience with and am capable of rendering an informed opinion on what the level of ordinary skill in the art was for the relevant field as of March 9, The 591 patent describes the field of invention as follows: The invention disclosed broadly relates to the field of data base administration and more particularly relates to the field of altering index objects in tables. (Ex at 1:23-26.) 24. As an example, claim 1 is a method claim. It recites: 9 11

12 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650, An interactive media apparatus for generating a displayable edited video data stream from an original video data stream, wherein at least one pixel in a frame of said original video data stream is digitally extracted to form a first image, said first image then replaced by a second image resulting from a digital extraction of at least one pixel in a frame of a user input video data stream, said apparatus comprising: an image capture device capturing the user input video data stream; an image display device displaying the original video stream; a data entry device, operably coupled with the image capture device and the image display device, operated by a user to select the at least one pixel in the frame of the user input video data stream to use as the second image, and further operated by the user to select the at least one pixel to use as the first image; wherein said data entry device is selected from a group of devices consisting of: a keyboard, a display, a wireless communication capability device, and an external memory device; a digital processing unit operably coupled with the data entry device, said digital processing unit performing: identifying the selected at least one pixel in the frame of the user input video data stream; extracting the identified at least one pixel as the second image; 10 12

13 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 storing the second image in a memory device operably coupled with the interactive media apparatus; receiving a selection of the first image from the original video data stream; extracting the first image; spatially matching an area of the second image to an area of the first image in the original video data stream, wherein spatially matching the areas results in equal spatial lengths and widths between said two spatially matched areas; and performing a substitution of the spatially matched first image with the spatially matched second image to generate the displayable edited video data stream from the original video data stream. (Ex. 1001, claim 1.) 25. The ʼ591 patent is directed to a system that creates a new composite video by substituting a portion of an original video data stream with an image in a user input video data stream. To understand the specification and to make and use the claimed inventions without undue experimentation, one of ordinary skill would have at least an engineer with a Bachelor of Science degree and at least three years of imaging and signal processing experience or would have earned a Master s Degree in Electrical Engineering and at least two years of 11 13

14 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 professional experience in signal, image, and video processing. B. Prior Art 26. I understand that the law provides categories of information that constitute prior art that may be used to anticipate or render obvious patent claims. Prior to the American Invents Act ( pre-aia ), I understand that, to be prior art to a particular patent claim under 35 U.S.C. Section 102(a), a reference must have been known or used in this country, or patented or described in a printed publication before the priority date of the Challenged Claims. To be prior art under pre-aia 35 U.S.C. Section 102(b), I further understand that a reference must have been in public use or on sale in this country, or patented or described in a printed publication more than one year prior to the date of application for the Challenged Claims. To be prior art under pre-aia 35 U.S.C. Section 102(e), I further understand that a patent application must have been published or a patent application subsequently granted must have been filed before the priority date. I also understand that the POSITA is presumed to have knowledge of all relevant prior art. Below is a table identifying the main prior art references that will be discussed in detail in this declaration

15 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 Table 1 Reference Title Date Senftner U.S. Patent No. 7,460,731 to Senftner et al. Issued ( Senftner ) Dec. 2, 2008 Sitrick Levoy U.S. Patent Application Publication No. 2005/ to Sitrick ( Sitrick ) U.S. Patent Application Publication No. 2009/ to Levoy et al. ( Levoy ) C. Identification of Grounds of Unpatentability Published Jul. 14, 2005 Published Dec. 17, I understand that the Petitioner is requesting inter partes review of claims 1-4, 8 and 11 of the 591 patent under the grounds set forth in Table 2, below

16 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 Table 2 Ground of Unpatentability 591 Patent Claim(s) Basis for Rejection Ground 1 1, 2, 8, and 11 Ground 2 3 and 4 Ground 3 1, 2, 8, and 11 Ground 4 3 and 4 Anticipated or rendered obvious by U.S. Patent No. 7,460,731 ( Senftner ) Rendered obvious by Senftner in view of U.S. Patent Application Publication No. 2009/ ( Levoy ) Rendered obvious by U.S. Patent Application Publication No. 2005/ ( Sitrick ) Rendered obvious by Sitrick in view of Levoy D. Broadest Reasonable Interpretations 28. I understand that, in an inter partes review, the claim terms are to be given their broadest reasonable interpretation (BRI) in light of the specification. See 37 C.F.R (b). 29. In performing my analysis and rendering my opinions, I have interpreted claim terms for which the Petitioner has not proposed a BRI by giving them the ordinary meaning they would have to a POSITA, reading the ʼ591 patent with its priority filing date (March 9, 2010) in mind, and in light of its specification and prosecution history

17 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 III. THE 591 PATENT 30. As indicated on its face, the 591 patent issued from U.S. Application No. 13/042,955, which was filed March 8, The 491 patent claims priority to U.S. provisional application No. 61/311,892, filed on March 9, The ʼ591 patent is directed to a system that creates a new composite video by substituting a portion of an original video data stream with an image in a user input video data stream. FIG. 3 (reproduced below) shows the preferred image substitution described by the patent. According to the specification, a user input 150 of a photo image of the user used to replace the face of the image shown on the device 108. The user transmits the photo image 150 by wired or wireless means to the device 108. The image substitution is performed and the device 108 shows the substituted image 190. (Ex at 2:66-3:4 (emphasis added))

18 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 (Ex at Fig. 3.) IV. GROUND 1: CLAIMS 1, 2, 8, AND 11 ARE ANTICIPATED OR RENDERED OBVIOUS BY SENFTNER A. Overview of Senftner 32. Senftner relates to the creation of personalized video through partial image replacement in an original video. (Ex. 1006, Abstract). Senftner is entitled Personalizing a video. 33. Senftner discloses systems and methods for replacing images 16 18

19 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 and videos of an original actor or target with a new actor or target replacement. (Id. at 2:33-54; 5:42-47; 9:25-31; FIG. 1). 34. Figure 1 in Senftner lays out several steps: process 100 relates to obtaining images of a new actor; process 200 relates to preparing an original video, having the original actor, for substitution; and process 300 relates to making a personalized video based on processes 100 and 200. (Id. at 9:32-52). Motion correction can also be applied. (Id. at 6:10-14; 17:10-23). B. Challenged Claims [1-PREAMBLE-i] An interactive media apparatus for generating a displayable edited video data stream from an original video data stream, 35. In my opinion, Senftner discloses the first part of the preamble of claim 1. (See, e.g., Ex. 1006, Abstract; 5:20-25; 2:41-54). [1-PREMABLE-ii]... wherein at least one pixel in a frame of said original video data stream is digitally extracted to form a first image, said first image then replaced by a second image resulting from a digital extraction of at least one pixel in a frame of a user input video data stream, said apparatus comprising: 36. In my opinion, Senftner discloses the second part of the preamble of claim 1. (See, e.g., id. at 2:41-54; 5:15-28; 5:33-40; 6:8-14; 8:67-9:1; 9:6-9; 10:3-28; 11:7-12; 11:42-59; 12:27-45; 13:15-25; 17:46-49; 8:52-54; FIGs. 1-3)

20 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 [1a] an image capture device capturing the user input video data stream; 37. In my opinion, Senftner discloses limitation 1a. (See, e.g., id. at 17:45-48; FIGs. 1, 8-11). [1b] an image display device displaying the original video stream; 38. In my opinion, Senftner discloses limitation 1b. (See, e.g., id. at 21:6-8; FIG. 10). [1c-i] a data entry device, operably coupled with the image capture device and the image display device 39. In my opinion, Senftner discloses limitation 1c. (See, e.g., id. at 20:24, 20:35-36; 20:62-64; 21:5-7; FIGs ). [1c-ii]... operated by a user to select the at least one pixel in the frame of the user input video data stream to use as the second image, and further operated by the user to select the at least one pixel to use as the first image; 40. In my opinion, Senftner discloses limitation 1c-ii. (See, e.g., id. at 2:8-10; 5:5-6; 2:33-45; 8:52-67; 10:3-16; 11:57; 12:37-45; 17:23-24; 18:1-18; 18:45-46; 20:24-39; 20:56-57; FIGs. 8-11). 41. In particular, a POSITA would understand that each frame of a digital video is comprised of pixels and each frame of a video contains a 2D image. (Ex at 2:8-10; 11:57). 42. A POSITA would also recognize that the pixel information of the first image and the second image must be used by the system because the 18 20

21 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 disclosed replacement process in Senftner manipulates a digital video (original video data stream) on a pixel-by-pixel and frame-by-frame basis. (Ex at 8:52-67). 43. A POSITA would understand that when a 2D image (a second image), primarily capturing the new actor s face, is selected by the requester for the actor modeling process, the requester also necessarily selects the at least one pixel comprising the selected 2D image. (Ex at 2:41-45; 10:3-16; Fig. 10; 20:24-39.) 44. Likewise, a POSITA would understand that when the requester selects an original object (target/first image) to be replaced by a target replacement (second image), the requester necessarily selects at least one pixel comprising the image of the original object. (Ex at 2:41-45; 8:60-64; Fig. 10; 20:24-39.) [1d] wherein said data entry device is selected from a group of devices consisting of: a keyboard, a display, a wireless communication capability device, and an external memory device; 45. In my opinion, Senftner discloses limitation 1d. (See, e.g., id. at 20:65-21:11). [1e] a digital processing unit operably coupled with the data entry device, said digital processing unit performing: 46. In my opinion, Senftner discloses limitation 1e. (See, e.g., id. at 19 21

22 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 20:56-21:11; FIGs ). [1e-i] identifying the selected at least one pixel in the frame of the user input video data stream; 47. In my opinion, Senftner discloses limitation 1e-i. (See, e.g., id. at 10:3-12; 12:27-45, 18:1-22; 8:52-9:5; 8:60-62). [1e-ii] extracting the identified at least one pixel as the second image; 48. In my opinion, Senftner discloses limitation 1e-ii. (See, e.g., id. at 10:3-12; 4:15-24). 49. I also understand that Patent Owner may argue for a narrow construction of digital extraction, in which case one or more pixels must actually be removed from the data stream. Although I do not agree that this narrow interpretation is consistent with the language of the claim, and I do not see any support for such an interpretation in the specification of the 591 patent, it is nonetheless my opinion that Senftner discloses this limitation. 50. In particular, a POSITA would understand that there are two ways to obtain new actor images. One way, which Senftner discloses (see id. at 10:3-12 and 4:15-24), is copying the applicable pixels of a new actor s image during the modeling process. The second way would be to extract the necessary data from the new actor s data stream, including the pixels associated with the new actor s image. Senftner discloses removal of the selected target in the 20 22

23 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 original video data stream. (See, e.g., id. at 2:51-54; 2:58-62; 3:22-28.) 51. A POSITA would recognize that it is simple and routine to change from a copying operation to a cutting operation, i.e., for copying the original pixels are not deleted, while for cutting the original pixels are deleted. (See, e.g., Ex at 2-16.) Thus, in my opinion, Senftner still discloses, teaches, and suggests to a POSITA the extracting step of limitation 1e-ii, because a POSITA would consider both copying and cutting the associated pixel data to be simple, routine, and known alternatives. [1e-iii] storing the second image in a memory device operably coupled with the interactive media apparatus; 52. In my opinion, Senftner discloses limitation 1e-iii. (See, e.g., id. at 17:65-67; 18:11-12; 21:23-29; FIG. 8). [1e-iv] receiving a selection of the first image from the original video data stream; 53. In my opinion, Senftner discloses limitation 1e-iv. (See, e.g., id. at 2:41-44; 9:6-9; 10:29-12:17; FIGs. 1-2). [1e-v] extracting the first image; 54. In my opinion, Senftner discloses limitation 1e-v. (See, e.g., id. at 2:33-54; 5:42-59; 6:8-14; 8:58-9:5; 11:7-12). [1e-vi] spatially matching an area of the second image to an area of the first image in the original video data stream, wherein spatially matching the areas 21 23

24 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 results in equal spatial lengths and widths between said two spatially matched areas; and 55. In the district court case between the Patent Owner and the Petitioner ( Litigation ), I submitted my declaration in support of Petitioner s responsive claim construction brief, which is attached as Exhibit In the Litigation, I found the spatially matching term to be indefinite. My opinion still remains that this term is indefinite. However, I have been instructed by counsel that, in an IPR, all claimed terms must be construed to compare them to the prior art. In this context only, an assumed interpretation of the indefinite term is used i.e., a spatially matching involves aligning pixels in the spatial domain or resizing of one image to the size of another image such that the two images are matched in the X-Y dimensions (length-width). Under this assumed interpretation, I find the spatially matching term of claims 1 and 11, disclosed, taught, or suggested, as provided below. 56. In my opinion, Senftner discloses limitation 1e-vi. (See, e.g., id. at 10:29-46; 12:27-45). [1e-vii] performing a substitution of the spatially matched first image with the spatially matched second image to generate the displayable edited video data stream from the original video data stream 57. In my opinion, Senftner discloses limitation 1e-vii. (See, e.g., id. at 2:33-54; 5:42-59; 8:58-9:5; 12:27-45)

25 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 [Claim 2] The interactive media apparatus of claim 1 wherein the digital processing unit is further capable of performing: computing motion vectors associated with the first image; and applying the motion vectors to the second image extracted from the user input video data stream, wherein the generated displayable edited video data stream resulting from the substitution maintains an overall motion of the original video data stream. 58. In my opinion, Senftner discloses claim 2. (See, e.g., id. at 2:41-54; 6:8-14; 17:10-23). [Claim 8] The interactive media apparatus of claim 1, wherein the substitution performed by the digital processing device replaces at least a face of a first person from the original video data stream by at least a face of a second person from the user input video data stream. 9). [Claim 11] 59. In my opinion, Senftner discloses claim 8. (See, e.g., id. at 9:6-60. In my opinion, Senftner discloses claim 11, which consists of method steps that are analogous to the limitations of claims 1 and 2, as indicated by the mapping below. [11P-i] A method for generating a displayable edited video data stream from an original video data stream, 61. In my opinion, Senftner discloses the limitations of [11P-ii] for the reasons identified above for [1-PREAMBLE-i]. [11P-ii] wherein at least one pixel in a frame of the original video data stream is digitally extracted to form a first image, said first image then replaced by a 23 25

26 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 second image resulting from a digital extraction of at least one pixel in a frame of a user input video data stream, said method comprising: 62. In my opinion, Senftner discloses the limitation of [11P-ii] for the reasons identified above for [1P-ii]. [11a] capturing a user input video data stream by using a digital video capture device; 63. In my opinion, Senftner discloses limitation [11a] for the reasons identified above for limitation [1a]. [11b] using a data entry device operably coupled with the digital video capture device and a digital display device, selecting the at least one pixel in the frame of the input video data stream; 64. In my opinion, Senftner discloses limitation [11b] for the reasons identified above for limitation [1c]. [11c] wherein the data entry device is selected from a group of devices consisting of: a keyboard, a display, a wireless communication capability device, and an external memory device; and 65. In my opinion, Senftner discloses limitation [11c] for the reasons identified above for limitation [1d]. [11d] using a digital processing unit operably coupled with the data entry device, performing: 66. In my opinion, Senftner discloses limitation [11d] for the reasons identified above for limitation [1e]

27 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 [11d-i] identifying the selected at least one pixel in the frame of the input video stream; 67. In my opinion, Senftner discloses limitation [11d-i] for the reasons identified above for limitation [1e-i]. [11d-ii] extracting the identified at least one pixel as the second image; 68. In my opinion, Senftner discloses limitation [11d-ii] for the reasons identified above for limitation [1e-ii]. [11d-iii] storing the second image in a memory device operably coupled with the digital processing unit; 69. In my opinion, Senftner discloses limitation [11d-iii] for the reasons identified above for limitation [1e-iii]. [11d-iv] receiving a selection of the first image from the user operating the data entry device; 70. In my opinion, Senftner discloses limitation [11d-iv] for the reasons identified above for limitation [1e-iv]. [11d-v] extracting the first image from the original video data stream; 71. In my opinion, Senftner discloses limitation [11d-v] for the reasons identified above for limitation [1e-v]. [11d-vi] spatially matching an area of the second image to an area of the first image in the original video data stream, wherein spatially matching the areas results in equal spatial lengths and widths between said two spatially matched areas; 25 27

28 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650, In my opinion, Senftner discloses limitation [11d-vi] for the reasons identified above for limitation [1e-vi]. [11d-vii] performing a substitution of the spatially matched first image with the spatially matched second image to generate a the displayable edited video data stream from the original video data stream; 73. In my opinion, Senftner discloses limitation [11d-vii] for the reasons identified above for limitation [1e-vii]. [11d-viii] computing motion vectors associated with the first image; and 74. In my opinion, Senftner discloses limitation [11d-viii] for the reasons identified above for claim 2 (specifically, the interactive media apparatus of claim 1 wherein the digital processing unit is further capable of performing: computing motion vectors associated with the first image ). [11d-ix] applying the motion vectors to the second image, wherein the generated displayable edited video data stream resulting from the substitution maintains an overall motion of the original video data stream. 75. In my opinion, Senftner discloses limitation [11d-ix] for the reasons identified above for claim 2 (specifically, applying the motion vectors to the second image extracted from the user input video data stream, wherein the generated displayable edited video data stream resulting from the substitution maintains an overall motion of the original video data stream )

29 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 V. GROUND 2: CLAIMS 3 AND 4 ARE OBVIOUS OVER SENFTNER IN VIEW OF LEVOY A. Overview of Levoy 76. Levoy relates to systems and methods for selecting portions of an image via a touch screen device, and for incorporating those selections into another image. This allows users to create a new composite image, e.g., on a mobile phone or other touch-screen type device. Figure 3 provides one example of this functionality, and illustrates how image fragments can be incorporated into an underlying image using a touch screen. (Ex. 1008, FIG. 3). Figure 3 is reproduced below: 27 29

30 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650, As Figure 3 shows, a sample burst image 310 is shown on the display of a mobile device 300. (Id. at 46). Device 300 may be a mobile device. (See, e.g., id. at 21; 23; 47). A user can use a fingertip to touch boxed area 320 in order to generate image fragments 330, 340, and 350. (Id. at 46-47). These fragments can then be selected and incorporated into the underlying image. (Id. at 46; 50). 78. Paragraphs 47 and 50 of Levoy further explain this image selection and incorporation process: [T]he apparatus 100 may include various means for receiving a selection of a particular burst image, which may include the processor 105, the presenter 134, the user interface 115, a display (e.g., a touch screen display or a conventional display), algorithms executed by the foregoing or other elements for receiving a selection of a particular burst image described herein and/or the like. In this regard, a user may interact with the user interface 115 to select one of the burst images via the presentations of the burst image fragments. For example, a user may tap on a touch screen in the location of a particular burst image fragment to select the underlying burst image. The selection may be obtained by the user interface 115 and transmitted to the processor 105 to be received by the processor 105. (Id. at 47.) The processor 105 may also be configured to generate a composite image based on one or more selected burst images and the 28 30

31 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 corresponding one or more selected locations associated with the selected burst images. In some exemplary embodiments, the processor may also be configured to provide for the presentation of the composite image after generation. In this regard, a composite image may be generated in any known manner. However, the inputs to the generation of the composite image may be derived for the selected burst images and the selected locations associated with the selected burst images. (Id. at 50.) B. Motivation to Combine Senftner and Levoy 79. As I described above, Senftner describes video personalization using partial image replacement. (Ex. 1006, Abstract; 2:41-54). Levoy also describes creating composite images based on selecting a portion of a burst image and a corresponding portion of another burst image. (Ex. 1008, Abstract; 58-61). Thus, both references involve creating new image data by substituting a portion of image data from one image with image data from another image. 80. Senftner describes a variety of computers that can be used to select original images and new images for replacement, including computer tablets,... personal digital assistants (PDAs), portable computers, and laptop computers. (Ex. 1006, 21:25-29). However, Senftner does not expressly disclose that any of these devices necessarily have a touch screen. 81. Levoy, however, shows that touch screen devices were well 29 31

32 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 known to those of ordinary skill in the art prior to the 591 patent. (See, e.g., Ex. 1008, 10, 21, 46-50). In fact, long before the 591 patent was filed, touch screen devices were known to be easier to use and more versatile than keyboards. (See, e.g., Ex. 1010, 4, see also Ex. 1011, 1:25-36). Touch screens were also known to provide a natural and user-friendly experience for operators. (See, e.g., Ex. 1012, 1:24-40). 82. Thus, a POSITA would have been motivated to use conventional touch screen technology (as opposed to conventional, non-touch technologies) with any of the computer devices identified in Senftner, including tablets, PDAs, portable computers, and laptops. C. Challenged Claims [Claim 3] The interactive media apparatus of claim 1 wherein the digital processing unit is further capable of extracting the at least one pixel from the user entering data in the data entry display device. 83. I understand that Petitioner contends that claim 3 is invalid on certain grounds related to 35 U.S.C I have been asked to assume for purposes of my analysis that the data entry display device of claim 3 is a display, per limitation 1d. 84. In my opinion, Senftner in view of Levoy teaches and suggests claim 3. Senftner discloses various computer devices that can be used to select 30 32

33 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 original images and new images for replacement, such as tablets, PDAs, portable computers, and laptops. (See, e.g., Ex. 1006, 21:25-29). However, Senftner does disclose that any of these devices have a touch screen display. 85. Nonetheless, as shown by Levoy, touch screen devices were well known prior to the 591 patent, and as I described above with respect to the Motivation to Combine section, touch screens were known to provide numerous benefits over conventional non-touch screen devices. (See V.B). Thus, a POSITA would have been motivated to use conventional touch screen display devices, such as touch-screen enabled tablets, PDAs, portable computers, or laptops, as the computerized device of Senftner, instead of non-touch screen displays. [Claim 4] The interactive media apparatus of claim 3 wherein the digital processing unit is further capable of extracting the at least one pixel from the user pointing to a spatial location in a displayed video frame, 86. As with claim 3 above, I understand that Petitioner contends that claim 4 is invalid on certain grounds related to 35 U.S.C I have been asked to assume for purposes of my analysis that the at least one pixel limitation of claim 4 refers to the user input data stream of claim In my opinion, Senftner in view of Levoy teaches and suggests claim 4. Although Senftner does not expressly disclose a touch screen computer 31 33

34 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 device, a POSITA would be motivated to add the touch screen functionality disclosed in Levoy to Senftner for the reasons I described above with respect to the Motivation to Combine section. (See V.B). As a POSITA would understand, users attempting to edit pictures with touch screen technology would naturally point to a spatial location in a displayed video frame, as shown in Levoy. (See, e.g., Ex. 1008, 47). VI. GROUND 3: CLAIMS 1, 2, 8, AND 11 ARE OBVIOUS OVER SITRICK A. Overview of Sitrick 88. Sitrick relates to a system and method for processing a video input signal providing for tracking a selected portion in a predefined audio-visual system and integrating selected user images into the selected portion of the predefined audiovisual presentation. (Ex at Abstract.) 89. In Sitrick a user selected image [a second image] is selectively integrated into a predefined presentation in place of a tracked portion [a first image] of the predefined audiovisual presentation [an original video data stream]. (Ex at 11). The image substitution process in Sitrick enables the substitution of a facial image from an external source into an original video, in order to create an edited video: 32 34

35 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 (See, e.g., Ex at FIG. 1; see also id. at 31). B. Challenged Claims [1-PREAMBLE-i] An interactive media apparatus for generating a displayable edited video data stream from an original video data stream, 90. In my opinion, Sitrick discloses the first part of the preamble of claim 1. (See, e.g., Ex. 1007, Abstract; 31). [1-PREMABLE-ii]... wherein at least one pixel in a frame of said original video data stream is digitally extracted to form a first image, said first image then replaced by a second image resulting from a digital extraction of at least one pixel in a frame of a user input video data stream, said apparatus comprising: 91. In my opinion, Sitrick discloses the second part of the preamble of claim 1. (See, e.g., id. at 19; 48-49; 54; 57; 71-72; 82; 87; FIGs. 5, 7). 92. In particular, Sitrick includes Figure 7: 33 35

36 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650, In paragraph 48, Sitrick explains that a mask 750 is formed by analyzing the image 710. (Id. at 48). As a POSITA would understand, this analysis involves extracting pixel information from the image 710 in order to determine the relative position of objects in the picture, such as the location of the face 711. The mask 750 in Sitrick is then formed based on this extracted pixel information. The mask can then be replaced with a user s face, by overlaying the user s face on the mask. (See, e.g., id. at 87; 54; 19; FIG. 5). 94. Sitrick also discloses techniques for extracting information about a reference object from an image, e.g., the reference object s position within the visual image, a rotational orientation, color information, size information, geometric information such as a wire-frame mesh, mask information, and other 34 36

37 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 information. (See, e.g., id. at 82; 49). In view of this disclosure, a POSITA would understand that Sitrick necessarily extracts pixel information relating to a given reference object, thereby enabling the system to detect the reference object in each frame of a data stream. Indeed, a POSITA would understand that Sitrick discloses forming an image when the mask is produced, and when the image of the reference object is created to be used by the tracking subsystem. [1a] an image capture device capturing the user input video data stream; 12; 139). 95. In my opinion, Sitrick discloses limitation 1a. (See, e.g., id. at [1b] an image display device displaying the original video stream; 96. In my opinion, Sitrick discloses limitation 1b. (See, e.g., id. at FIGs. 1-6). [1c-i] a data entry device, operably coupled with the image capture device and the image display device 97. In my opinion, Sitrick discloses limitation 1c. In particular, Sitrick discloses a system that is implemented on a general purpose computer. (See, e.g., id. at 29; 41-43, 46, 69-70, 79-80, 95, , 115, 118, ). 98. As a POSITA would understand, general purpose computers include data entry devices, such as a keyboard. 99. Sitrick also discloses, teaches, and suggests an integration 35 37

38 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 subsystem that is implemented in a general purpose computer. In particular, Sitrick discloses a general purpose computer, as shown in Figure 1. Sitrick also discloses an integration subsystem that is used to create the output video 190 based on the program data 120 and the user image data 135. (Id. at 31). As a POSITA would know, this subsystem would typically be implemented within the general purpose computer. (See id. at 29; FIG. 13) The integration subsystem in Sitrick is also coupled to an external source of image data. (Id. at 31). A POSITA would recognize that this image data is provided by a capture device, such as a video camera. (Id. at 12). A POSITA would further recognize that the data entry device(s) connected to the general purpose computer would be operably connected to the image capture device as well, which would enable user interaction with the image capture device. Indeed, Sitrick refers to user selected image[s] that are selectively integrated into a predefined audiovisual presentation. (Id. at 11). A POSITA would understand that the user selects images using a data entry device, such as a keyboard Sitrick also discloses that its general purpose computer is coupled to an image display device, such as the display for showing videos depicted in figure 1. (See also id. at FIG. 13; 121). Here again, a POSITA 36 38

39 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 would recognize that the general purpose computer receives video input signals (id. at 121) and displays such signals on an image display device. Moreover, a POSITA would understand that the data entry device for the general purpose computer would be used to interact with the computer to select content for viewing on the display device, as well as to interact with that content. (Id. at 13). [1c-ii]... operated by a user to select the at least one pixel in the frame of the user input video data stream to use as the second image, and further operated by the user to select the at least one pixel to use as the first image; 5; 11) In my opinion, Sitrick discloses limitation 1c-ii. (See, e.g., id. at 103. A POSITA would also understand from Sitrick that a user would necessarily have to select one or more pixels in order to select an image or portion of an image, as disclosed in Sitrick. (Id. at 13). [1d] wherein said data entry device is selected from a group of devices consisting of: a keyboard, a display, a wireless communication capability device, and an external memory device; 104. In my opinion, Sitrick discloses limitation 1d, and in particular discloses a general purpose computer that would include a data entry device, as described above with respect to limitation 1c-i As discussed in [1c-ii], above, since the user operates the Sitrick 37 39

40 Declaration of Edward Delp Petition for Inter Partes Review of U.S. Patent No. 8,650,591 system implemented on a general purpose computer to select one or more pixels in a frame of a video, a POSITA would understand that a data entry device would necessarily be utilized as an input device mean. (Id. at 11.) A POSITA would understand that a keyboard was an obvious choice as one of the plurality of input device means disclosed in Sitrick. (Id.) [1e] a digital processing unit operably coupled with the data entry device, said digital processing unit performing: 115) In my opinion, Sitrick discloses limitation 1e. (See, e.g., id. at [1e-i] identifying the selected at least one pixel in the frame of the user input video data stream; 107. In my opinion, Sitrick discloses limitation 1e-i. (See, e.g., id. at 11; 31; 40; 87; 104; Figs. 1, 5) As I explained above with respect to 1-PREAMBLE-ii and limitation 1c, Sitrick describes a system in which a user s face is selected from the user s image data and then overlaid on top of a mask from the program video. A POSITA would understand that this involves identifying and selecting the pixels comprising the user s face (in a reference object) in order to overlay those pixels on the mask. (See id. at 104). [1e-ii] extracting the identified at least one pixel as the second image; 38 40