(12) Ulllted States Patent (10) Patent N0.: US 8,396,311 B2 Chono (45) Date of Patent: Mar. 12, 2013

Transcription

1 US B2 (12) Uted States Patent (10) Patent N0.: US 8,396,311 B2 Chono (45) Date of Patent: Mar. 12, 2013 (54) MAGE ENCODNG APPARATUS, MAGE 6,987,468 B1 * 1/2006 Mavar /59 ENCODNG METOD, AND MAGE * ; yam??? ~~~~~~~~~~~~~~~~~ ~~ 341/51 agas 1 et a. ENCODNG PROGRAM 2006/ A1 5/2006 Smirnov _ / A1* 8/2006 Mohsenian / (75) nventor. Keiichi Chono, Tokyo (JP) 2006/ A1 10/2006 Cha (73) Assignee: NEC Corporation, Tokyo (JP) (Continued) ( * ) Notice: Subject to any discaimer, the term of this FOREGN PATENT DOCUMENTS patent is extended or adjusted under 35 JP A 4/2004 U.S.C. 154(b) by 592 days. JP A 4/2007 (21) App. No.: 12/601,484 Continued ( ) (22) PCT Fied: Apr. 28, 2008 OTER PUBLCATONS (86) PCT NO : PCT/JP2008/ Zhang et a1, igh-performance CABAC Engine for.264/avc igh De?nition Rea-Time Decoding, CCE 2007, Digest of Tech (6X1), nica Papers. nternationa Conference on Consumer Eectronics, (2), (4) Date: NOV. 23, 2009 Jan 1044, 2007 Las Vegas, NV} (87) PCT Pub. No.: WO2008/ (Continued) PCT Pub. Date: Nov. 27, 2008 (65) Prior Pubication Data Primary Examiner * Wenpeng Chen Assistant Examiner * Feng Niu US 2010/ A1 Ju. 8, 2010 (30) Foreign Appication Priority Data (57) ABSTRACT May 21, 2007 (JP) An image encoding apparatus, an image encoding method, and an image encoding program capabe of minimizing (5 1) nt C image degradation, controing a code amount in units smaer than a picture, and ensuring that encoding of the picture is ( ' ) _ / _ / competed Within a certain time period is provided. A symbo (52) US. C /239, , number estimating device estimates the totabin number Ofa (58) sred ofrciss??catiqon searcht """" None puraity of macro bocks constituting a picture. An entropy ee app 102 on e or Comp e e Seam 15 Dry encoding seector outputs the entropy encoding mode seect (56) References Cited ing signa to an entropy encoder using the inputted bin num U.S. PATENT DOCUMENTS her, in order to seect one of a CABAC device or a VLC device. A CPU performs image encoding by executing a contro program stored in a program storing portion. 5,574,449 A * 11/1996 Goin /65 5,654,702 A 8/1997 Ran 5,774,081 A * 6/1998 Cheng et a /107 6,856,701 B2 * 2/2005 KarcZeWicZ et a / Caims, 12 Drawing Sheets 209 Emmy M mcomm swam mm SELECTOR _,205 uuussn Esnmme DEVCE zoo ) q 201 C 207 L "M PM 31%? K \ $888M RWY 4+ BUFFER, BUFFER _ii \ Device DER 2nd 991; 203; 22 m NVERSE-GONVERSDN 212* ; ' 216" NVERSE-QUANTZATON E PORNO" M 12,411 4' i W cm DEGODED _ PCTURE ~ ? Bum

2 US 8,396,311 B2 Page 2 US. PATENT DOCUMENTS 2007/ A1* 2/2007 Senda / / A1* 5/2007 Smirnov 382/ / A1* 5/2008 Maadi et a /24022 FOREGN PATENT DOCUMENTS JP A 6/2008 WO A 4/2004 OTER PUBLCATONS SO/EC nformation technoogy4coding of audio-vi sua ObjGCSiPAT 10: Advanced Video Coding, Fifth edition, May 15, E. Viscito, Macrobock eve imit on the number of binary symbos in CABAC, Joint Video Team (JV T) of SO/EC MPEG & TU-T VCEG (SO/EC JTC1/SC29/WG11 and TU-T SG16 Q6), XP , Dec C. Gomia et a., New features and appications of the.264 video coding standard, nformation Technoogy: Research and Education, EEE, 2003, pp D. Marpe et a., Context-Based Adaptive Binary Arithmetic Coding in the.264/avc Video Compression Standard, EEE Transactions on Circuits and Systems for Video Technoogy, vo. 13, No. 7, Ju. 2003, pp nternationa Search Report for PCT/JP2008/ maied Aug. 5, K. Chono et a., A PCM Coding Method using Decoded mages for obeying the upper imit on the number of bits of MB in.264 encoding, PCSJ (Picture Coding Symposium of Japan), Session P5-17, Nov Okamoto et a., Switched Entropy Coding Method for Rea-Time igh De?nation Appications of.264/avc, The nstitute of Eec tronics, nformation and Communication Engineers Genera Confer ence, D-11-2, Mar Taiwanese Of?ce Action for No issued on Ju. 24, * cited by examiner

3 US. Patent Mar. 12, 2013 Sheet US 8,396,311 B2 a? M EN Na g max gag, " $53 mwmamx ".wzmawé mccm \ h K $582M AztiaiaaaiiwL M 3N?. ~ "?jgzg W M mean -... g + 3mm "3N 5 u?mg? wag $55 Em W.15 m m. W mg Egg zigzag m?at ii. zmz?t?é,, g u w. 3% M n 1 zoimx?ggwwga 3? n. W mega 25% 1% m n \sa "m A \fi N W wig v gag m wig. W m h m gwi M w?mgwwgvmmwm?g... m _ W,w

4 US. Patent Mar. 12, 2013 Sheet US 8,396,311 B2 (PCTURE EQCGDENG) 836,,\ SYiABGL mam "' PRUGESS 3382M ENTRDPY ENCOQWG SELEGTiNG PRGCESS mama RAPE. Sm smms _ PROCESS man max 3304 ENGQDiNG Paucass LAST MACE!) BLQCK? ' PMURE CGMPLEXWY 83% F i g. 2

5 .. US. Patent Mar. 12, 2013 Sheet US 8,396,311 B2 A gig m 1.,, N $ aégwmm?w M if n Ema? was 3?jm?éw, W g m 55 E?gsm Eggg ma,., 1,. Egg ", wwwwmwmw mg} gamma 5?g 1 M 53% 3,, W.,1 mwmgmm sag hmmwm?mmm Q, W immmm.. \, mmm.,,, N3 1» T1 $12M gamma > NE? wwwmwm " mg, gag mwmg? Va; GEE g m v,3?hmmm WE, \3i\.1. a _ mm, W a, mwgsm 2% m _,w w +111 magma as wmag? Egm w?? Va; w 1.,,,.W., iii wwmmmmm WE Ea $23? $3,

6 US. Patent Mar. 12, 2013 Sheet US 8,396,311 B2 {PREL MNARY PEGTURE ENCUDNG} S5Bi " * PARAMETER SETYNE 35Q211 \ PRELiMFNARY MAGRO BLGCK ENCUBiNG 8503 a,, * LAST '. _MAGRO BLBGK? 1153 S535 f REL A 1 NARY ' """ ENCGB E N6 P i CTURE CGMPLEX TY UPDATE F 5g, ~4

7 US. Patent Mar. 12, 2013 Sheet 5 0f 12 US 8,396,311 B2 {MAW PiGTBRE ENGQMNG) s52: SYMBGL NU?éBER ES? WATEGN ENTRGPY ENCGDENG SELECT 352m», PRCYURE RATE SETUKG + MAW W163i) BLQGK ENGQDWG was"? mam max? PiCTURE CGMPLEXTY Fi g, 5

8 US. Patent Mar. 12, 2013 Sheet US 8,396,311 B2 rrrttrrttrrrrrrrrrrrtttttttttr+eicereti\iiiiiéééii)i Q é 2 1 a 1 i i R i a e t i t i S 4 1?t wit. ~ m; 1 115mm %=.ANQ vs 5 QUE SMWX M Ru a U U Riiwi. 11R?u UT? 3 6.< M E n: QR?! R20 5 CC? 3, ERR C nunuon RM Eiuu 5, nuurc, Rvfnu GE 3P3,, GTF. 1, P3P Wm 1, w an % mew 1,, > i > E mm 9P3 Wm M, i ii f 311,, Wm * K n L v M, F.r m 3 R w M 7..a: N * tit-1:1:vtttin iiaitiufi'rrtrr Mm 6 * '. T! _ a.. ME 2. "m a, & m ".8 /. M 5m "E nu, NV \ a - u 12 T. w _ m,. 4 i K93 "m N R w r0 on an R Ru 25% mm m an E nu. v: E YE,Mn:.1 1 R MR, 1.. :9, Dn 1. E E. an EE VD Dn?u VinU. An?u ND 6.? UL Rnu.?nu Rnu. 1111?? R Nnu {nu Nnu _. :m 3... mm mm mm. w. mm 1m,3 mm. u v \40111 "! nuw NE $5 1,5. 5 G1... ME LEM vii YRK UK. WK, Gm \TK \nu 19K " v :w mam am am ma ea, 1M gm mww Mam w i! a. { mnu RL \PsL,RiL "1&6, MG 1. 15L 2 3 L MT, PB,\ B %,D..B _ p.371, he 2 PB 1! $3 1.. TB.... EN m D,. _ DUDG 2. LU TAU 1 Nnu,1 DG. RQE EU 1} SR.2 mums 3 RD, TCBL _ N am: as: 5%,. mm mags, M a; 1.. 1,, PA an FM 6 SM 6. T u. L h 1. UM. _ Q n * 6 w a _ ~ a ~ ~ ~ a a u a u ~ ~ ~ u u n u n v u - r v n r m - w R R R t t Q t n, r, x i u a S a r u i Q. u a t u G i t t 1 x KL

9 US. Patent Mar. 12, 2013 Sheet US 8,396,311 B2 {PREL i M NARY Pi CTURE ENGGD NE} s1m~-~~ PARAMEYER smms W, 11mm Bum 5m mama i - 11s - i, * mm mm - - F i g, 7

10 US. Patent Mar. 12, 2013 Sheet US 8,396,311 B2 {MAW PiCYURE - SYMSGL NzmBER Sm \ EST EMAGN Sm EMEGBWG SELEC Y PCWRE RATE S723 SEWNG W, BLQGK Sm GP s?meaa i V MACRG BLGEK 81,25 ENCDQiNG ' LAST MCRG BLOCK? SW PECTURE cammsx: 11/ F i g. 8

11 US. Patent Mar. 12, 2013 Sheet US 8,396,311 B2 m 5 A, 558mm mm N21,. m xawmwémiwm?é m z2 ww$> Main N + M., 1, + W m? E} N32,,.. wm?ua wzgaw?.. imam v, 55D as,.\ wow Er m,1, Q2 zsw?gu mg: mag \Y zmz?éée iii 1,1.. Wang. W 2 1., _. 11w: Q5?ut W \1 Q2 Eta 7156,85... N9

12 US. Patent Mar. 12, 2013 Sheet US 8,396,311 B2 x / En NF m m 3, E m Egg wc E

13 US. Patent Mar. 12, 2013 Sheet US 8,396,311 Ciia-((( magma 3mg E QM. magma a} N no 0.: ~21,... ~ ~ ttttittiittt1i $i$ii Tuv i! t $5.5m mmmcuzm a.

14 US. Patent Mar. 12, 2013 Sheet US 8,396,311 B2 $145 2' " ' " " " " " * ' ' ' " * " * * " " ' * "*z : 15s ; J51 152;" * * 1 * - "455 <\ Em? 1' j,1 i 1 =,3 1,, BiNARiZATBN : : 1 M"? " DEME w ; ; ENCGQER MW"; 3 W. 15? E 2 $ ,159 i m E'W" ggfgggm E \w _. ; GABAC DEWCE A : Fig. 12 g i 5v 1,,_ VARMBLE LENGT 3 if, ; ; 5110mm 3

15 1 MAGE ENCODNG APPARATUS, MAGE ENCODNG METOD, AND MAGE ENCODNG PROGRAM This appication is the Nationa Phase of PCT/JP2008/ ,?ed Apr. 28, 2008, Which is based upon and caims priority from Japanese Patent Appication No ?ed May 21, BACKGROUND OF TE NVENTON 1. Technica Fied The present invention reates to an image encoding appa ratus, an image encoding method, and an image encoding program using image encoding technoogy. n particuar, the present invention reates to an image encoding apparatus, an image encoding method, and an image encoding program capabe of seecting an entropy encoding scheme. 2. Background Art An image encoding apparatus encodes externa input image data in accordance With a predetermined image encod ing scheme to generate a bit stream..264/avc is known as one of image encoding schemes used for such encoding pro cess (see Non-patent document 1). This scheme conforms to MPEG (Moving Picture Experts Group) -4 Written Standards Part 10, and Joint mode scheme is known as its encoding reference mode. An image encoding apparatus based on the Joint mode scheme, Which is reated to the present invention, is caed reated art image encoding apparatus hereinafter. FG. 9 shows a structure of a reated art image encoding apparatus. The reated art image encoding apparatus 100 incudes an image frame buffer 102 to successivey store image frames Which constitute a target image data 101 to be compressed. An image data 103 is divided into and outputted as macro bocks having a predetermined image area size from the image frame buffer 102, and inputted to a macro bock encoder 104 in Which the image data is encoded in units of macro bocks. A code amount controer 105 and a decoded picture buffer 106 are connected to the macro bock encoder 104. The macro bock encoder 104 outputs an encoded bit stream 107. The de?nitions of the macro bock and picture is expained ater. The macro bock encoder 104 of such image encoding apparatus 100 incudes a macro bock buffer 111 to receive the image data 103, a predicting device 112 connected to the macro bock buffer 111, a cacuating device 113 to subtract the output from the predicting device 112 from the output of the macro bock buffer 111, a conversion and quantization device 114 to convert and quantize the cacuation resut from the cacuating device 113 under the contro of the code amount controer 105, an entropy encoder 115 and inverse conversion and inverse-quantization device 116 arranged on the output side of the conversion and quantization device 114, and an adder 117 arranged on the output side of the inverse conversion and inverse-quantization device 116. Assume that the image data 101 inputted to the image encoding apparatus 100 has a QCF (Quarter Common nter mediate Format). The QCF is one of image signa formats de?ned in the TU (nternationa Teecommunication Union). FG. 10 shows an image frame in the QCF image format. The image?ame in the QCF is composed of macro bocks of 176 bocks in Wide by 144 bocks in high. One image frame is composed of one frame picture in the progressive scanning. Furthermore, one image frame is composed of two?ed pic tures in the interace scanning. They are caed simpy pic ture(s) in the foowing expanation. US 8,396,311 B Each macro bock, Which is the unit constituting the pic ture, is composed of brightness pixes in 16x16 pixes, and coor-difference pixes of Cr (coor-difference signas) and Cb (coor-difference signas), each in 8x8 pixes. FG. 10 shows a brightness position (x) and coor-difference position (0) of an 8><8 pixe bock on a pixe-by-pixe basis When the macro bock is divided into 16 parts. The macro bock encoder 104 shown in FG. 9 encodes the image data 103 in units of macro bocks. n this case, the encoding is successivey performed With raster scanning diagonay from the upper eft of the picture to the ower right in simiar manner to the raster scan for a teevision system. Firsty, the macro bock buffer 111 of the macro bock encoder 104 reads a target image data to be encoded in macro bocks, temporay stores it, and suppies it to the conversion and quantization device 114 arranged at the subsequent stage. At this point, the cacuating device 113 subtracts a prediction image 122 outputted from the predicting device 112 from the image 121 read from the macro bock buffer 111 in macro bocks, and the suppies the prediction error image 123 (i.e., the cacuation resut) to the conversion and quantization device 114. ncidentay, there are two types of the prediction error image 123, i.e., a prediction image generated based on inter frame prediction and a prediction image generated based on intra-frame prediction. The inter-frame prediction is gener ated using the correation between separate image frames, i.e., using the image of an image frame Which Was encoded and reconstructed before and Which has a different dispay time from the current target image frame to be encoded. On the other hand, the intra-frame prediction is generated using the correation Within the image frame, i.e., using the image of an image frame Which Was encoded and reconstructed before the current target image frame to be encoded and has the same dispay time as the current target image frame. A set (sice) of macro bocks Which can be encoded using the intra-frame prediction aone is caed -sice hereinafter. Furthermore, a sice of macro bocks Which can be encoded using both the intra-frame prediction and inter-frame predic tion is caed P-sice. Furthermore, a sice of macro bocks Which can be encoded using the inter-frame prediction Where not ony the image of one image frame but aso the images of two image frames can be used simutaneousy is caed B-sice. Furthermore, a picture Which can be encoded using the -sice is caed -picture, and a picture Which can be encoded using both the -sice and P-sice is caed P-pic ture. Furthermore, a picture Which can be encoded using not ony the -sice and P-sice but aso the B-sice is caed B-picture. The conversion and quantization device 114 frequency converts the prediction error image 123 in smaer units than macro bocks. t converts the prediction error image 123 from a spatia domain to a frequency domain. n the AVC (Ad vanced Video Coding) Standards, a frequency conversion in units of 8x8 bocks or 4x4 bocks is appicabe to brightness pixes. A prediction error image converted to a frequency domain is caed conversion coef?cient hereinafter. This conversion coe?icient is quantized based on a parameter 125 suppied from the code amount controer 105, and suppied to the entropy encoder 115 as code data 126. This quantized conversion coe?icient is caed quantized vaue in the speci?cation. The code data 126 is aso suppied to the inverse-conver sion and inverse-quantization device 116. The inverse-con version and inverse-quantization device 116 inverse-quan tizes the quantized vaue suppied from the conversion and

16 3 quantization device 114, and further inverse-frequency-con verts it to the origina spatia domain. Then, the adder 117 adds the prediction image 122 suppied from the predicting device 112 to the prediction error image Which Was restored to the spatia domain to obtain a decoded image 128. This decoded image 128 is stored in the decodedpicture buffer 106 for subsequent encoding. The entropy encoder 115 entropy-encodes the inputted code data 126 and outputs a bit stream 107. The term entropy-encoding means compression of data in Which a code having different ength is assigned depending on the occurring probabiity of the data. Since the present invention cosey reates to the entropy encoder 115, the detai of it Wi be expained ater. The predicting device 112 suppies a generating parameter of a prediction image to the entropy encoder 115 as code data 129. The generating parameter may incudes, for exampe, a prediction mode indicating the type of prediction such as inter-frame prediction and intra-frame prediction, a index of a decoded frame used in inter-frame prediction, a motion vector used in inter-frame prediction, and intra-frame predic tion direction used in intra-frame prediction. As expained above, the decoded picture buffer 106 stores the decoded image 128 suppied from the inverse-conversion and inverse-quantization device 116. Then, it US 8,396,311 B2 manages decoded image pictures reconstructed from the decoded image 128 (Which is simpy caed decoded pictures here inafter). The code amount controer 105 monitors a bit stream 131 outputted from the entropy encoder 115 to encode a picture With a desired bit number. Then, if the bit number of the bit stream 131 is arger than the desired bit number, it outputs a parameter indicating the increase of a quantizing step size as a quantizing parameter 125. On the other hand, if the bit number of the bit stream 131 is smaer than the desired bit number, it outputs a parameter indicating the decrease of a quantizing step size as the quantizing parameter 125. The entropy encoder 115 aso monitors a symbo number (bin number) Which is inputted to an arithmetic encoder (Which is expained ater) in the case Where CABAC (Con text-based Adaptive Binary Arithmetic Coding) is used as entropy encoding With a entropy encoding seecting signa 132 (the detai of the CABAC is expained ater). Then, the quantizing parameter Wi be adjusted such that the ratio between a bit number and a bin number satis?es the ratio speci?ed in the above-mentioned AVC Standards. FG. 11 shows the speci?c structure of this entropy encoder. The entropy encoder 115 incudes a?rst seector 141 to receive the image code data 126 outputted from the con version and quantization device 114 shown in FG. 9, a CABAC device 142 having an input side connected to one of the output sides of the?rst seector 141, a VLC (Variabe Length Coding) device 143 having an input side connected to the other output side of the?rst seector 141, and a second seector 144 to seectivey receive one of the outputs from the CABAC device 142 and VLC device 143 Which are used as the devices for these two types of coding. An entropy encoding mode seecting signa 132 is pro vided to the entropy encoder 115 for the switching of the?rst seector 141 and second seector 144. The entropy encoding mode seecting signa 132 is a signa to seect one of the CABAC device 142 and VLC device 143. n this manner, in the AVC (Advanced Video Coding) Standards, entropy encoding is performed on the code data 126 of macro bocks on a picture-by-picture basis by seecting one of the coding by CABAC device 142 or the coding by VLC device n the entropy encoder 115 shown in FG. 11, the CABAC device 142 receives a code data 1261 outputted from the?rst seector 141, and outputs a bit stream 1071 to the second seector 144. The CABAC device 142 aso outputs a bin number data 145 representing a bin number. The speci?c structure of the CABAC device 142 is expained ater. The VLC device 143 receives a code data 1262 outputted from the?rst seector 141, and outputs a bit stream 1072 to the second seector 144. The speci?c structure of the VLC device 143 is expained ater. The second seector 144 aso outputs a bit number data 146 representing a bit number as We as the bit stream 107. ncidentay, the CABAC device 142 achieves higher encoding ef?ciency than the VLC device 143. owever, the CABAC device 142 requires arger processing effort than the VLC device 143. Therefore, in genera, the CABAC device 142 is used for a higher pro?e (e.g., igh pro?e or Main pro?e) Which supports compicated process. MeanWhie, the VLC device 143 is used for a ower pro?e (e.g., Base-ine pro?e) Which does not support compicated process. ow ever, since code data in higher eve ayers than a macro bock ayer has a reativey smaer code data amount Which occu pies a bit stream, the VLC device 143 is used for code data in such ayers to prioritize comparabiity among each pro?e. FG. 12 shows the speci?c structure of a CABAC device. The CABAC device 142 incudes a binarization device 151 to receive a code data 1261 through the?rst seector 141 shown in FG. 11 and converting it to binary, and a switch 153 to switch the output of the binary outputted from the binariza tion device 151.A bin 155 outputted from the switch 153 as a binary symbo is suppied to an arithmetic encoder 156 and a context cacuator 157. The binarization device 151 is adapted to convert inputted code data 1261 to a binary string in accordance With proce dure speci?ed in the AVC Standards and output it as a binary string data 152. The arithmetic encoder 156 encodes the binary string of the bin 155 Which is successivey suppied from the switch 153 to binary arithmetic code by using a dominant symbo 158 and a state number 159 suppied from the context cacuator 157. Furthermore, it successivey sup pies an updated dominant symbo 158 and an updated state number 159 to the context cacuator 157. The term state number 159 means a tabe number of a tabe storing a vaue corresponding to the occurring probabiity of dominant sym bo speci?ed in the AVC Standards. The context cacuator 157 suppies stored dominant sym bo 158 and state number 159 corresponding to the bin 155 Which is successivey suppied from the switch 153 as a symbo. Furthermore, it aso stores the dominant symbo 158 and state number 159 Which are updated by the binary arith metic encoding at the arithmetic encoder 156. The switch 153 outputs a bin number data 145 to the outside of the CABAC device 142. Furthermore, the arith metic encoder 156 outputs a bit stream 1071 to the outside of the CABAC device 142. As shown in FG. 11, the bit stream 1071 is suppied to the second seector 144 in parae to a bit stream 1072 outputted from the VLC device 143. FG. 13 shows the speci?c structure of avlc device. The VLC device 143 incudes a variabe ength encoder 161 to receive a code data 1262 through the?rst seector 141 shown in FG. 11, and a tabe seector 162. The variabe ength encoder 161 encodes the code data 1262 to variabe ength code in accordance With a tabe speci?ed by the tabe seector 162, and outputs a bit stream The tabe seector 162 contains variabe ength encoding tabes (not shown) corresponding to the types of code data 1262 such as a prediction mode, a quantized vaue, and the

17 5 ike. Then, it suppies a tabe 163 seected from variabe ength encoding tabes to the variabe ength encoder 161. n accordance With the image encoding apparatus 100 described above Which is reated to the present invention, When the VLC device 143 (FG. 11) is used for entropy encoding, the competion time of the picture encoding by this image encoding apparatus 100 is determined by the amount of inputted code data. On the other hand, When the CABAC device 142 shown in FG. 12 is used as the bock for entropy encoding, the compe tion time of the picture encoding by this image encoding apparatus 100 is determined by the number of the bin 155 inputted to the arithmetic encoder 156. ncidentay, in the case Where a bock other than the CABAC device 142 is used, the process Wi be aways competed Within?nite ength of time since the amount of input data is?nite. n the.264/avc Standards, the foowing two items are used as restrictions to imit the processing amount of entropy encoding per picture on the decoding side. (a) Encoding must be controed such that the bit number of a picture becomes equa to or ess than a vaue speci?ed in the.264/avc Standards. (b) Encoding must be controed such that the ratio between the bit number and bin number of a picture becomes equa to or ess than a vaue speci?ed in the.264/avc Standards. Since the VLC device 143 shown in FG. 13 aows process in units of the code data 1262, its process is simper than that of the CABAC device 142 shown in FG. 12. Therefore, assuming that the process is aways competed Within a cer tain time period, encoding of the image encoding apparatus 100 needs to be controed such that the bin number of a picture does not exceeds the maximum bin number for Which the arithmetic encoder 156 can process during the processing time. A foowing method is one of exempary methods for such encoding controing. (i) For a picture having a arge bin number, there is a method of encoding using PCM (Puse Code Moduation) mode in Which a input image in macro bocks is directy outputted as a bit stream (Patent document 1 and Non-patent document 2). (ii) There is another method in Which CABAC device 142 and avlc device 143 are operated in parae for the code data of a picture. Then, if the process of the CABAC device 142 is competed Within a predeter mined time, the encoding output from this CABAC device 142 is seected for the encoding. f not, the encoding output from this VLC device 143 is seected for the encoding (N on patent document 3). [Patent document 1] Japanese unexamined appication pub ication No (paragraphs , and FG. 1). [Non-patent document 1] SO/EC Advanced Video Coding. [Non-patent document 2] PCM ENCODNG METOD CONFORMNG TO.264 MB UPPER LMT BT USNG LOCAL DECODE MAGE by Chono et a, PCSJ (Picture Coding Symposium of Japan), pp. 5-17, Novem ber [Non-patent document 3] ENTROPY ENCODNG TEC NQUE N.264/AVC FOR G-RESOLUTON by Okamoto et a, The nstitute of Eectronics, nformation and Communication Engineers Genera Conference, D-11-2, March SUMMARY The former method using a PCM mode is effective for appications in Which the bin number of one macro bock is US 8,396,311 B restricted. owever, there is a probem associated With degraded images in such appications in Which the bin num ber of a picture is restricted (i.e., the number of bins of a puraity of macro bocks is restricted). Because When the PCM mode is heaviy used in a picture to impement the method stated in the above item (i), a macro bock Which is encoded after the heavy use of the PCM mode is coarsey quantized since the PCM mode has a arge output bit number. MeanWhie, in the atter method in Which a CABAC device 142 and avlc device 143 are operated in parae, and one of the encoding outputs is seected, it becomes after-the-event seection. Therefore, it poses a critica probem in the image encoding apparatus 100 that the encoding cannot be con troed in units smaer than the picture. Because the bit num ber for the code data 126 inputted to the entropy encoder 115 is unknown unti the seected encoding output becomes known. ncidentay, Patent document 1 proposes a method in Which When the bin number of code data encoded from an image is arger than a predetermined vaue, an aternative bit stream of code data separatey encoded from the above-men tioned image With a different encoding parameter (e.g., dif ferent quantizing parameter) (such that its bin number becomes smaer than the predetermined vaue) is outputted. owever, since this method is aso the after-the-event seec tion, it has a critica probem simiar to the above-mentioned probem. The present invention has been made in view of these circumstances, and provides an image encoding apparatus, an image encoding method, and an image encoding program capabe of minimizing image degradation, controing a code amount in units smaer than pictures, and ensuring that encoding of the picture is competed Within a certain time period. n accordance With one aspect of the present invention, an image encoding apparatus incudes: (a) a code data generator to transform and quantize an image bock to generate code data; (b) a variabe ength code encoder to encode the code data into bit-string data on the basis of appying variabe ength encoding; (c) an arithmetic encoder to encode the code data into bit-string data on the basis of converting the code data into binary symbos and appying binary arithmetic encoding; (d) a binary symbo number estimator to estimate the tota number of the binary symbos that are converted from the code data generated from a puraity of image bocks; and (e) a encoding seector to seect, for the encoding of the code data, either the variabe ength code encoder or the arithmetic encoder for the encoding of the code data on the basis of the tota number estimated by the binary symbo number estimator. That is, the present invention in one aspect achieves the above-mentioned object by estimating the bin number of the image of a target picture to be encoded; and controing entropy encoding and the target bit number (assigned picture rate) of the picture using the estimated bin number and com pression ratios of the variabe ength encoder and arithmetic encoder. n accordance With another aspect of the present invention, an image encoding method incudes: (a) transforming and quantizing an image bock to generate code data; (b) estimat ing the tota number of the binary symbos that are converted from the code data generated from a puraity of image bocks; and (c) seecting the encoding method of the code data from variabe ength encoding and arithmetic encoding on the basis of the tota number estimated, the variabe ength encod ing being adapted to encode the code data into bit-string data on the basis of appying variabe ength encoding, and the

18 7 arithmetic encoding being adapted to encode the code data into bit-string data on the basis of converting the code data into binary symbos and appying binary arithmetic encoding. That is, the present invention in one aspect expresses simi ar technica idea to the above-mentioned image encoding apparatus as the transitions of processes. n accordance with another aspect, the present invention provides a computer-readabe recording medium recording an image encoding program to instruct a computer to execute: (a) a code data generating process to transform and quantize an image bock to generate code data; (b) a binary symbo number estimating process to estimate the tota number of the binary symbos that are converted from the code data gener ated from a puraity of image bocks; and (c) an encoding seecting process to seect the encoding method of the code data from variabe ength encoding and arithmetic encoding on the basis of the tota number estimated by the binary symbo number estimating process, the variabe ength encoding being adapted to encode the code data into bit-string data on the basis of appying variabe ength encoding, and the arithmetic encoding being adapted to encode the code data into bit-string data on the basis of converting the code data into binary symbos and appying binary arithmetic encoding. That is, the present invention in one aspect expresses simi ar technica idea to the above-mentioned image encoding apparatus as a program executed by a computer. The above and other objects, features and advantages of the present invention wi become more fuy understood from the detaied description given hereinbeow and the accompa nying drawings which are given by way of iustration ony, and thus are not to be considered as imiting the present invention. BREF DESCRPTON OF TE DRAWNGS FG. 1 is a bock diagram showing the structure of an image encoding apparatus in accordance with a?rst embodiment of the present invention; FG. 2 is a?owchart showing the operation of the image encoding apparatus in accordance with the?rst embodiment from the start of picture encoding to the end thereof; FG. 3 is a bock diagram showing the structure of an image encoding apparatus in accordance with a second embodiment of the present invention; FG. 4 is a?owchart showing the operation of the image encoding apparatus in accordance with the second embodi ment for preiminary picture encoding; FG. 5 is a?owchart showing the operation of the image encoding apparatus in accordance with the second embodi ment for main picture encoding; FG. 6 is a bock diagram showing the structure of an image encoding apparatus in accordance with a third embodiment of the present invention; FG. 7 is a?owchart showing the operation of the image encoding apparatus in accordance with the third embodiment for preiminary picture encoding; FG. 8 is a?owchart showing the operation of the image encoding apparatus in accordance with the third embodiment for main picture encoding; FG. 9 is a bock diagram showing the structure of an image encoding apparatus of reated art; FG. 10 is an expanatory diagram showing an image frame in the QCF image format; FG. 11 is a bock diagram showing the speci?c structure of an entropy encoder; FG. 12 is a bock diagram showing the speci?c structure of a CABAC device shown in FG. 11; and US 8,396,3 B FG. 13 is a bock diagram showing the speci?c structure of a VLC device shown in FG. 11. DESCRPTON OF TE REFERENCE NUMERALS 142(242) CABCA DEVCE 143(243) vlc DEVCE 200, 400, 600 MAGE ENCODNG APPARATUS 202 MACRO BLOCK BUFFER 204 MACRO BLOCK ENCODER 205, 422 CODE AMOUNT CONTROLLER 208, 424, 608 symbol NUMBER EsTMATNG DEvCE 209 BN NUMBER 210, 425, 609 ENTROPY ENCODNG selector 212 PREDCTNG DEvCE 214 CONvERsON AND QUANTZATON DEvCE 215 ENTROPY ENCODER 232 ENTROPY ENCODNG MODE selectng sgnal 271,431,671 CPU 272, 432, 672 PROGRAM storng PORTON 404 PRELMNARY ENCODER 405 MAN ENCODER 604 PRELMNARY PARALLEL ENCODER 605 MAN PARALLEL ENCODER 611 PRELMNARY MACRO BLOCK ENCODER 621 MAN MACRO BLOCK ENCODER EXEMPLARY EMBODMENT Exempary embodiments in accordance with the present invention are expained hereinafter. First Embodiment FG. 1 shows the structure of an image encoding apparatus in accordance with a?rst embodiment of the present inven tion. An image encoding apparatus 200 in accordance with a?rst embodiment incudes an image frame buffer 202 to suc cessivey store target image data 201 to be compressed. The image frame buffer 202 outputs image data 203 in units of macro bocks in simiar manner to the above-expained image encoding apparatus 100 (FG. 9) in the reated art to the present invention. This image data 203 is inputted to a macro bock encoder 204, and encoded in units of macro bocks. A code amount controer 205 and a decoded picture buffer 206 are connected to the macro bock encoder 204. A symbo number estimating device 208 which estimates the bin number of macro bocks of a target picture to be encoded, i.e., the tota bin number of a puraity of macro bocks constituting the picture, is connected to the image frame buffer 202. The symbo number estimating device 208 reads the image of a target picture to be encoded which is stored in the image frame buffer 202, and an assigned picture rate for the target picture to be encoded at the 205, and performs the above-mentioned estimation. The bin number of the macro bocks of the target picture to be encoded which was estimated by the symbo number esti mating device 208 is inputted to an entropy encoding seector 210. The entropy encoding seector 210 outputs an entropy encoding mode seecting signa 232, which is used to seect a entropy encoding mode, based on the inputted bin number 209. The macro bock encoder 204 incudes, in comparison to the macro bock encoder 104 shown in FG. 9, a macro bock buffer 211 to receive the image data 203, a predicting device 212 connected to the macro bock buffer 211, a cacuating device 213 to subtract the output from the predicting device 212 from the output of the macro bock buffer 211, a conver

19 9 sion and quantization device 214 to convert and quantize the cacuation resut from the cacuating device 213 under the contro of the code amount controer 205, an entropy encoder 215 and inverse-quantization and inverse-conversion device 216 arranged on the output side of the conversion and quan tization device 214, and an adder 217 arranged on the output side of the inverse-conversion and inverse-quantization device 216. The macro bock encoder 204 outputs an encoded bit stream 207. The image encoding apparatus 200 in accordance With this embodiment incudes a CPU (Centra Processing Unit) 271, a program storing portion 272 such as a disk drive and an optica disk to store a contro program executed by this CPU 271, and a Working memory 273 such as semiconductor memory to temporay store various data during the time When the CPU 271 is executing the contro program stored in the program storing portion 272. At east some part of the components shown in FG. 1 may be achieved by software operations by the CPU 271 executing the contro program stored in the program storing portion 272. n the image encoding apparatus 200 having structure expained above, the code amount controer 205 speci?es an assigned picture rate depending on the indicated content of the entropy encoding mode seecting signa 232 estabished by the entropy encoding seector 210. Therefore, its opera tions are different from the code amount controer 105 shown in FG. 9. That is, the code amount controer 105 shown in FG. 9 is set up in accordance With an instruction manua, and in genera, one of the CABAC device 242 or VLC device 243 Which corresponds to the CABAC device 142 and VLC device 143 respectivey in FG. 11 and is seected in a?xed manner in accordance With a pro?e. The macro bock encoder 204 in accordance With the?rst embodiment of the present invention has substantiay the same components such as the macro bock buffer 211 and entropy encoder 215 as those of the macro bock encoder 104. Therefore, the structures shown in FGS are appied to this embodiment Without any modi?cation. owever, in this embodiment, the number in hundreds pace of the sign rep resenting each component is changed from 1 to 2 in the foowing expanation. ncidentay, the image encoding apparatus 200 of this embodiment determines the content of the entropy encoding mode seecting signa 232 by operating the symbo number estimating device 208 and entropy encoding seector 210 at the start of the picture encoding. The image encoding appa ratus 200 of this embodiment updates the assigned picture rate by operating the code amount controer 205 after the decision of the content of the entropy encoding mode seect ing signa 232 and before the start of the encoding of macro bocks Within a target picture to be encoded. After the picture rate update, the image encoding apparatus 200 successivey encodes macro bocks Within the target picture to be encoded by operating the macro bock encoder 204. Then, after the Whoe macro bocks Within the picture are encoded, the image encoding apparatus 200 performs predetermined processes and starts the encoding of the next picture. FG. 2 shows the operation of the image encoding appara tus in accordance With this embodiment from the start to the end of picture encoding. The foowing expanation is made With reference to FG. 1 and FGS Firsty, the image encoding apparatus 200 of this embodi ment starts the picture encoding With symbo number esti mating process using the symbo number estimating device 208 (Step S301). Assume that the brightness pixe data of the target picture to be encoded Which is stored at the image frame buffer 202 is src[y][x] (Wherein Oéxéwidth-, US 8,396,311 B Oéyéheight-) at the start of the picture encoding When an image data 201 is sent from a image data source (not shown). n the formuas, x is a horizonta position of a pixe Within the picture, and y is a vertica position of the pixe. Furthermore, Widt is the horizonta pixe count of the picture, and height is the vertica pixe count. n the symbo number estimating process of the step S301, the symbo number estimating device 208 estimates the bin number from the brightness pixe data src [y] [x]. To this end, an activity mb_act[i] is cacuated for each macro bock With the foowing equation (1) using this brightness pixe data n the equation (1), ave[i] is expressed by the foowing equation (2). As seen from the equation (1), the activities represent variations of the pixe vaues of macro bocks. Therefore, it is possibe to use the standard deviation or vari ance of the pixe vaues of the macro bocks. ave[i] n the equations (1) and (2), i is a macro bock address in order of raster scanning Within a picture. Furthermore, mbx (i) is a horizonta position of a macro bock Within the picture corresponding to the macro bock address i, and mby(i) is a vertica position of the macro bock Within the picture corre sponding to the macro bock address i. Then, in the symbo number estimating process of the step S301, estimated bin number compexity of the target picture to be encoded epic_bin_x is cacuated With the foowing equation (3) using the activity for each macro bock mb_act [i]. EPCibiXOLXPCiCt+5 (3) n the equation, pic_act is expressed by the foowing equation (4). picisizeimbsi piciact : mbiact?] [:0 n the equations (3) and (4), 0t and [3 represent the gra dient and intercept respectivey of the regression ine of picture bin number compexity for the activity. Furthermore, pic_size_mbs is the number of macro bocks contained in the picture (c:(width*height)/256). The term picture bin number compexity pic_bin_x means the product of the picture bin number pic_bin and the average quantizing step size of the macro bock pic_qs (i.e., pic_bin_x:pic_bin*pic_qs) When the target picture to be encoded is encoded as picture. At the end of the symbo number estimating process of the step S301, estimated bin number of the target picture to be encoded epic_bin is cacuated With the foowing equation (5) using the estimated bin number compexity of the target picture to be encoded epic_bin_x and the assigned picture rate for the target picture to be encoded pic_rate. (1) (2) (4)

20 11 epicibin:max(epicibinix/qsirgyxpicirate) (5) n the equation, qs_rc is expressed by the foowing equation (6). qsircpicix/picirate (6) n the equations (5) and (6), the picture compexity pic_x is the product of the occurring picture rate of the picture which has the same picture type as the target picture to be encoded (which is expained ater) and is encoded asty i.e., ast_pi c_rate and the average quantizing step size of the macro bock ast_pic_qs. Furthermore, y is the ratio between the symbo number bin and the bit number bit (Ybin/bit), n the equation, y is arger number than 1. The y may be a statistica ratio for each picture type, or a ratio cacuated from the data on a asty encoded picture for each picture type. After the symbo number estimating process of the step S301 is competed in this manner, an entropy encoding seect ing process is performed (Step S302). At the step S302, the entropy encoding seector 210 seects the entropy encoding mode seecting signa 232 using the estimated bin number of the target picture to be encoded epic_bin. Then, it cacuates the average quantizing step size qs_cabac using the foowing equation (7) for the case where the target picture is encoded by the CABAC device 242 (see FG. 12) qsicabacjyxpicix/mimepicibinppicibin) (7) n the equation, ppic_bin is expressed by the foowing equation (8). ppicibinpmbibinxpicisizeimbs (8) n the equations (7) and (8), pmb_bin is the bin number for which the arithmetic encoder of the CABAC device 242 (see FG. 12) of the entropy encoder 215 can process within the encoding time of macro bock. Therefore, the average quantizing step size qs_cabac is the image quaity by the CABAC device 242 under the condition where the upper imit bit number is imposed by the processibe bin number within a predetermined time period. The smaer the average quan tizing step size qs_cabac, the better the image quaity becomes. Then, in the entropy encoding seecting process of the step S302, it cacuates the average quantizing step size qs_vc using the foowing equation (9) for the case where the target picture is encoded by the VLC device 243. qsivcjyxpicix/picirate (9) n the equation (9), y is the compression ratio between the bit number when the code data of a picture is processed by the VLC device 243 vc_bit and the bit number when it is arithmeticay encoded cabac_bit (yq/c_bit/cabac_bit). Fur thermore, y is arger number than 1, and a statistica ratio for each picture type. As seen from the de?nition of y, the average quantizing step size qs_vc is the image quaity by the VLC device 243 under the condition where the reduction in compression ef?ciency by the VLC device 243 is imposed. At the end of the entropy encoding seecting process of the step S302, the entropy encoding mode seecting signa 232 is determined by the foowing equation (10) using the average quantizing step size qs_cabac and the average quantizing step size qs_vc. mode : CABAC if (qsicabac g qsivc) (10) VLC... Otherwise. US 8,396,311 B After the entropy encoding seecting process of the step S302 is competed in this manner, a picture rate setting pro cess is performed (Step S303). n this picture rate setting process, the code amount controer 205 updates the assigned picture rate of the target picture to be encoded pic_rate based on the entropy encoding mode seecting signa 232 (mode) using the foowing equation (11). picix/qsivc if (mode: VLC). { (1 1) picirate : picix/ qsicabac... Otherwise ncidentay, the quantizing step size qs_rc to achieve the assigned picture rate pic_rate can be cacuated with the fo owing equation (12). {qsivc if (mode : VLC) (12) qsirc : qsicabac... Otherwise. After the picture rate setting process of the step S303 is competed in this manner, a macro bock encoding process is performed (Step S304). n this macro bock encoding pro cess, the macro bock encoder 204 encodes one macro bock of the target picture to be encoded using the decided entropy encoding mode seecting signa 232. At this point, simiar to the code amount controer 105 of the image encoding apparatus 100 shown in FG. 9, the code amount controer 205 monitors a bit stream 207 outputted from the entropy encoder 215. Then, if the bit number of the bit stream 207 outputted from the entropy encoder 215 is arger than the assigned picture rate, it outputs a quantizing parameter indicating the increase of a quantizing step size. On the other hand, if the bit number of the bit stream 207 is smaer than the assigned picture rate, the code amount con troer 205 outputs a quantizing parameter indicating the decrease of a quantizing step size. After the macro bock encoding process of the step S304 is performed in this manner, it proceeds to the next step (Step S305). n the step S305, it determines whether or not the whoe macro bocks in the target picture to be encoded are encoded. f it is not competed (N), it returns to the step S304 to continue the process. f the encoding of the whoe macro bocks in the target picture to be encoded were competed (Step S305: Y), the picture compexity pic_x is updated with the foowing equation (13) depending on the decided entropy encoding mode seecting signa 232 (mode) at the end of the picture encoding (Step S306), and a series of the picture encoding process is competed (END)., { A X piciactirate >< qsipic if (mode : VLC) (3) piciactiratex qsipic Otherwise picix : n the equation, pic_act_rate is the occurring bit number of a picture which is encoded at that time. Furthermore, qs_pic is the average quantizing step size of the macro bock of the picture which is encoded at that time. As aready expained above, the picture compexity pic_x stores its vaue for each picture type. As expained above, the image encoding apparatus 200 in accordance with the?rst embodiment of the present invention can generate a bit stream 207 by successivey appying the processes to encode a picture as shown in FG. 2 to externay

21

22

23

24

25

26

27

28