Available online at ScienceDirect. Procedia Computer Science 73 (2015 ) 48 55

Transcription

1 Avaiabe oie at ScieceDirect Procedia Comuter Sciece 73 (2015 ) The Iteratioa Coferece o Advaced Wireess, Iformatio, ad Commuicatio Techoogies (AWICT 2015) A Tutoria o Seech Sythesis Modes Tabet Youcef a *, Boughazi Mohamed b, Affifi Sade b a Uiversity of Boumerdes,35000 Ageria b Uiversity of Aaba,23000 Ageria Abstract For Seech Sythesis, the uderstadig of the hysica ad mathematica modes of seech is essetia. Hece, Seech Modeig is a arge fied, ad is we documeted i iterature. The aim i this aer is to rovide a bacgroud review of severa seech modes used i seech sythesis, secificay the Source Fiter Mode, Liear Predictio Mode, Siusoida Mode, ad Harmoic/Noise Mode. The most imortat modes of seech sigas wi be described startig from the earier oes u uti the ast oes, i order to highight major imrovemets over these modes. It woud be desirabe a arametric mode of seech, that is reativey sime, fexibe, high quaity, ad robust i re-sythesis. Emhasis wi be give i Harmoic / Noise Mode, sice it seems to be more romisig ad robust mode of seech The Authors. Pubished by Esevier B.V. This is a oe access artice uder the CC BY-NC-ND icese (htt://creativecommos.org/iceses/by-c-d/4.0/) The Authors. Pubished Esevier B.V. Peer-review uder resosibiity of orgaizig committee of the Iteratioa Coferece o Advaced Wireess, Iformatio, ad Commuicatio Techoogies (AWICT 2015) 2015). Keywords: Liear Predictio Mode; Siusoida Mode; Harmoic/Noise Mode * tabet2402@yahoo.fr The Authors. Pubished by Esevier B.V. This is a oe access artice uder the CC BY-NC-ND icese (htt://creativecommos.org/iceses/by-c-d/4.0/). Peer-review uder resosibiity of orgaizig committee of the Iteratioa Coferece o Advaced Wireess, Iformatio, ad Commuicatio Techoogies (AWICT 2015) doi: /j.rocs

2 Tabet Youcef et a. / Procedia Comuter Sciece 73 ( 2015 ) Itroductio Seech is the most atura form of huma commuicatio. The temora-sectra variatios of seech sigas covey such iformatio as words, itetio, exressio, itoatio, accet, seaer idetity, geder, stye of seaig, state of heath of the seaer ad emotio 1. The time siga evoutio ca either be rereseted by a mode or ot. The advatages of usig a mode are its caacity to reduce the acoustic siga s redudacy ad to defie arameters that are better fitted to acoustic rocessig 2. The seech roductio mode ad the siusoida mode are the two mai modes used i seech sythesis. The first oe is a mode with severa i series-system that rereset the differet stages of the huma seech roductio, i.e. excitatio system, voca tract, ad is system. The secod mode decomoses the observed siga ito a sum of siewave comoets, i.e. a sum of frequecy ad/or amitude moduated cosies. Seectig a mode deeds o may factors such as quaity of sythesized seech, ease of arameter extractio, modificatio of arameters, umber of arameters ad comutatio oad. The remaider of this aer is orgaized as foows. First, reated wors is give i sectio 2. Next, i sectio 3 we reset the source fiter mode. After that the iear redictio mode is deicted i sectio 4. I sectio 5, the siusoida mode is described. This is foowed by the harmoic/oise mode i sectio 6. Fiay, cocusios are give i sectio Reated wors Severa aroaches have bee roosed for seech modeig, some of them wi be briefy discussed beow ad the most imortat oe wi be described i detai i the ext sectios. Serra 3 suggested a hybrid system for the aaysis, trasformatio, ad sythesis of soud based o a determiistic/stochastic decomositio. This system is desiged to obtai musicay usefu itermediate reresetatios for soud trasformatios. The determiistic comoet is rereseted by a series of o-ecessary harmoic siusoids cacuated by Short Term Fourier Trasform (STFT)-based ea-icig method. The stochastic comoet is rereseted by a series of magitude-sectrum eveoes that wor as a time varyig fiter excited by a white oise. This aroach is abe to create ew souds out of the reresetatio of a articuar soud. The determiistic siga is obtaied by sythesizig a siusoid from each trajectory. The, the residua betwee the determiistic comoet ad the origia soud is modeed by a series of eveoes. Fiay, the stochastic siga is geerated by a iverse STFT. This system is very fexibe ad aows for trasformatios of the soud by maiuatig each comoet searatey. George ad Smith 4, i their wor, used a system based o the combiatio of a overa-add (OLA) siusoida mode with a aaysis-by-sythesis (ABS) techique to determie the siusoida mode arameters. They itroduce a equivaet frequecy domai agorithm. I additio, a refied overa-add siusoida mode is derived. There is a we correatio betwee the refied overa-add sythesis ad aaysis-by-sythesis. The roosed aaysis-bysythesis/overa-add (ABS) system achieves very high sythetic seech quaity. To mode the trasiet art of seech siga, severa modes have bee roosed. For exame, a fexibe aaysis/sythesis mode for trasiet sigas is roosed i 5. The mode reseted is a arametric mode for trasiets that aows for a wide rage of siga trasformatios. I 6 a mode that gathers the roerties of a siusoida reresetatio ad a OLA rocessig ste is reseted. This mode is acowedged as beig efficiet for rederig of time-ocaized evets. Aso, the authors i 7, 8, 9 roosed the Exoetiay Damed Siusoida Mode (EDSM), aog with more owerfu arameter estimatio schemes based o either matchig ursuit or subsace methods. Subsace methods have good sectra roerties ad do ot suffer from the time frequecy tradeoff embedded i other methods. However, they are comutatioay itesive. Recety, i order to refie the roosed modes usig more owerfu arameter estimatio schemes, Fa-chir trasform that emoys a adative aaysis basis comosed of quadratic chirs is reseted i 10. A siusoida aaysis of seech simiar to the mode roosed i 11 but with the Fa-chir trasform istead of the Fast Fourier Trasform has bee coducted 12,13 with very satisfactory resuts.

3 50 Tabet Youcef et a. / Procedia Comuter Sciece 73 ( 2015 ) Source Fiter Mode I the Source-Fiter decomositio, the seech siga resuts from the combiatio of some acoustic eergy (iteractio of the ugs ad the aryx) coued with a trasfer fuctio that is determied by the shae of the sura-gotic cavities. I the cotext of siga rocessig, the Source-Fiter mode describes the seech siga as the covoutio of a excitatio siga by a time varyig fiter. The excitatio characterizes the variatio of acoustic ressure i the aryx ad the fiter reresets the time-frequecy behavior of the voca tract trasfer fuctio 14. For the most art, it is sufficiet to mode the roductio of a samed seech siga by discrete-time system mode. I this mode the uvoiced excitatio is assumed to be a radom oise sequece, ad the voiced excitatio is assumed to be a eriodic imusio trai with imuses saced by the itch eriod rouded to the earest same 15. The discrete-time system mode is aso caed source fiter mode. This system ca be described by the covoutio exressio s( ) m0 h( m) u( m) (1) Where s() is the etire seech siga, h() is the imuse resose, ad u() is the voiced/uvoiced excitatio. To simify aaysis, it is ofte assumed the system is a a-o fiter with system fuctio of the form H ( z) G (2) 1-1 where G ad are resectivey the gai ad the order of the fiter. The iear system is assumed to mode the comosite sectrum effects of radiatio, voca tract tube, ad gotta excitatio use shae (for voiced seech oy). Over a short time iterva the iear system i the mode is commoy referred to as simy the voca tract system ad the corresodig resose is caed the voca tract imuse resose. For a-oe iear systems, as rereseted by (2), the iut ad the outut are reated by a differece equatio of the form s( ) a s( ) Gu( ) (3) 1 This Source fiter mode is itimatey reated to the Liear Predictio mode discussed i the foowig sectio. 4. Liear Predictio Mode I 1960, Fa itroduced a iear mode of the time-domai waveform of the seech siga. To the Source-Fiter hyothesis, he added the hyothesis of ideedece betwee the gotta waveform ad the voca tract. The voca tract is modeed as a a oe fiter, aso caed Auto Regressive. The gotta waveform is modeed roughy as a use trai with a fudameta eriod equa to the itch (for voiced soud) ad as a white oise with zero mea ad uit variace (for uvoiced souds) 2, 14. The term Liear redictio refers to the mechaism of usig a iear combiatio of the ast time-domai sames, s(-1),s(-2),, s(-), to aroximate or to redict the curret time-domai same s() 16,17. So, a iear redictor of order P, with redictio coefficiets { }, is defied as a system whose outut is

4 Tabet Youcef et a. / Procedia Comuter Sciece 73 ( 2015 ) s ( ) s( ) 1 (4) Where s () is the redicted siga The redictio error is give by e( ) s( ) s ( ) s( ) s( ) (5) 1 Equatio 5 ca be reseted i the z-domai as E( z) s( z) A( z) (6) Where E(z) is the z-trasform of e(), S(z) is the z-trasform of s(), ad A(z) is the z-trasform of the redictio error fiter give by A z ) 1 1 ( z (7) Uo further isectio of equatio 3 ad 5, it ca be see that if the mode is exacty accurate for the seech siga, ad if {a } = { }, the e() = Gu(). Thus, A(z) becomes the iverse fiter of the system H(z) of equatio 2 G H ( z) A( z) (8) The mai robem of iear redictive aaysis thus becomes the estimatio of the redictor coefficiets { } so that the redictio error e[] is miimized uder some criterio. The mea squared error is by far the most utiized otimizatio criterio. The coefficiets { } that miimize the mea squared error are assumed to be the arameters of the system fuctio H(z) of equatio 2. The squared redictio error E i a short-time frame s (m) startig at same is defied as E e 2 ( m) m (9) 2 E ( s ( m) s ( m)) (10) E m m 2 ( s ( m) s ( m )) (11) 1 Where s (m) = s (+m) Two major aroaches to the comutatio of the LPC coefficiets have bee deveoed: the autocorreatio method ad the covariace method. Hece, the miimizatio of the equatio 11 eads to orma equatios which ca be soved usig severa agorithms.

5 52 Tabet Youcef et a. / Procedia Comuter Sciece 73 ( 2015 ) The Liear redictive aaysis is aied o a frame-by-frame basis to the seech siga. Hece, for each frame a iear redictive fiter is geerated. This fiter modes the gotta excitatio use shae, voca tract ad i radiatios effects. Durig voiced seech, a sime use trai excites the iear redictive fiter. However, for uvoiced seech the fiter is excited by a white oise. The method of iear redictive aaysis was oe of the most owerfu seech aaysis techiques because it is sime, fast, ad has a imited umber of arameters. The mai drawbac of this method is that is iherety buzzy due to its arametric ature, ad this degrades the seech quaity. Aso, hoemes such asas caot be modeed by the iear redictio mode because they cotai ati-formats, ad this mode is a a-oe mode. However, the method of iear redictio has bee the redomiat techique for estimatig the basic seech arameters, e.g., itch, formats, sectre, voca tract area fuctios, ad for reresetig seech for ow bit rate trasmissio or storage uti the ed of the 1980s, after which it gave way to more comex techiques which offered a better siga quaity, e.g., siusoida mode ad its derivatives. 5. Siusoida Mode The Siusoida mode reseted i 11, reresets a seech as a sum of siusoida fuctios, evovig over time. This mode has the caacity of frequecy resoutio (sius resoutio) ad time resoutio (evoutio of each siusoid over time). The seech siga is assumed to be the outut of a sowy time varyig digita fiter with a excitatio that cature the ature of the voiced/uvoiced distictio i seech roductio (Excitatio exressed as a sum of siusoids). The seech siga resutig from the fu mode is writte s( ) a ( ) cos( w ) (12) L 1 Where L is the umber of siusoida segmets, a, w, rereset resectivey the amitude, frequecy ad hase of the siewave. The aaysis scheme is based o the otio of siusoida trac, referrig to the comoets of the sum i the sythesis formua 12. The umber L of tracs varies with time: each trac is active durig a give ase of time ad this has to be determied by a tracig agorithm. It is thus ecessary to estimate the umber of comoets, their amitudes, ad frequecies. This aaysis ste is based o the Short Term Fourier Trasform. For each frame, the sectra eas are obtaied by searchig for a oca maxima o the amitude sectrum ad the eimiatig those whose amitude is beow a give threshod. The ositio of the eas rovides frequecies ad amitudes of the siusoida comoets. Phases of these comoets are cacuated as the hase of the Short Term Fourier Trasform for a give frequecy. For each frame, a set of L sectra eas is thus obtaied. The sythesis siga is cacuated by a overa-add of the short-term siga from equatio 12. I this case, the siusoida tracs are ot exicit. It is much more advatageous to foow the siusoida tracs exicity ad the to iteroate the sythesis arameters aog these tracs. So, this aterative sythesis method is erformed i severa stes. The first oe is the arameter matchig, the secod oe is the arameter iteroatio, ad fiay the sythetic waveform is cacuated from equatio 12. The matchig rocedure is doe betwee arameter vaues comuted at two cosecutive frame boudaries i order to sove the robem of the variabe umber of the eas across frames. The agorithm roosed i 11 has to be ut ito oeratio for detectig the birth, cotiuatio, ad death of the siusoida comoets across frames. After ayig arameter matchig agorithm, the ext ste is about arameter iteroatio i order to avoid abrut chages from oe frame to the ext. The amitudes are ieary iteroated betwee two successive frames ad the hases ad frequecies are iteroated usig a cubic fuctio. Fiay, equatio 12 is used to cacuate the sythetic seech siga. The mai advatage of the siusoida mode is that it erforms seech modificatio by fidig the siusoida comoets for a waveform ad erformig modificatio by aterig the arameters of the equatio 12, amey the amitudes, hases, ad frequecies 18. So, the seech deivered by this mode is ercetuay idetica to the origia

6 Tabet Youcef et a. / Procedia Comuter Sciece 73 ( 2015 ) oe. Because, the siusoida mode is suited for modeig harmoic souds, the uvoiced souds are oory rereseted by this mode. Aso, this mode has imitatio ie arge database ad comutatioa comexity. The eed of more comex modes that coud hade the o-harmoic comoet of seech soud ed to the Determiistic/Stochastic mode 3 or to the Harmoic/Noise mode Harmoic/Noise Mode Harmoic/Noise Mode (HNM), divides the seech siga ito two arts: the harmoic art ad the oise art. The harmoic oe reresets the quasi-eriodic comoets of the seech siga such as vowes ad some voiced cosoats ad the oise art reresets the o-eriodic art such as fricative asiratio oise, bursts, uvoiced seech, etc. The harmoic art is modeed through a set of harmoicay reated siusoids with sowy varyig amitudes ad frequecies. However, the oise art is usuay modeed as white Gaussia oise assig through a shaig fiter. The seech sectrum is divided ito two sub bads deimited by a time varyig maximum voiced frequecy. A umber of modes based o the ricie of decomosig the seech siga ito harmoic art ad oise art have bee roosed. Give its ouarity we exore here the Harmoic/Noise mode roosed i 19, 20, 21. The seech siga i such mode ca be exressed by a combiatio of harmoic ad oise ie modes as s( ) s ( ) s ( ) h (13) The siga i the harmoic art ca be modeed as where s ( ) A ( ) cos( ( ) ( )) (14) h L 1 ( ) 0 ( u) du (15) Ad where A (), () deote the amitude ad hase at time of the th harmoic resectivey, 0 is the fudameta frequecy ad L is the umber of harmoics icuded i the harmoic art. However, the oise art ca be modeed as s ( ) e( )[ h( ; )* b( )] (16) where b is a white Gaussia oise; h is a time varyig ormaized a-oe fiter; e is a eergy eveoe fuctio aied to give the fitered oise the correct temora atter. Aaysis: The estimatio of the itch is the first ste i Harmoic/Noise Mode Aaysis. It is obtaied by usig a time domai aroach, i.e. searchig the miimum vaue of a error fuctio (a autocorreatio aroach). From this iitia itch estimatio, a harmoic mode is fitted to each frame ad the voiced/uvoiced decisio is made by usig criterio which taes accout ito how cose this harmoic mode is to the origia mode. For voiced frames, we the estimate the maximum voiced frequecy Fm. This estimatio is based o a ea icig agorithm. Oce the maximum voiced frequecy has bee foud, accurate itch estimatio is ecessary. This is doe by miimizig a give error betwee the iitia itch estimatio ad the rage of frequecies cassified as voiced from the revious ste. The amitudes ad hases of the harmoics are foud i time domai usig a weighted east square error betwee the rea ad the sythetic wave form. For the estimatio of the arameters of the oise comoet (uvoiced art), i each aaysis frame, a sectra desity of the origia siga is modeed by a autoregressive fiter. This fiter wi be excited by a white oise ad

7 54 Tabet Youcef et a. / Procedia Comuter Sciece 73 ( 2015 ) the dyamic characteristics are cosidered by usig a variace eveoe which moduates the excitatio. Aso, a triaguar- ie time domai eergy eveoe moduates the oise comrisig the secod art of a voiced sectrum. A high ass fiter of F m (maximum voiced frequecy) cut frequecy is used to searate the harmoic art from the oise oe. The fia ste i Harmoic/Noise mode aaysis is hase modificatio for seech sychroizatio. This is doe by usig a time domai techique to adjust the reative ositios of the waveforms withi the frames to esure that the a aig. Sythesis: The sythesis is erformed i a itch sychroous way. The amitudes ad the hases of the harmoic comoet, are estimated via a east square criterio, ad are ieary iteroated betwee successive frames. Oy, the hases are uwraed before ayig iteroatio. A overa-add (OLA) method is used to sythesis the oise art: A Gaussia oise b is assed through the fiter h severa times er frame i order to esure that the temora characteristics are successfuy geerated; the oise is high ass fitered with a cut-off frequecy equa to the maximum voiced frequecy Fm. Next, to esure that the oise is sychroized with the harmoic art, it is moduated i the time domai. The fia sythetic seech siga is obtaied by addig the two arts, i.e. harmoic art us oise art. Ivestigatios show that Harmoic/Noise mode outerforms amost a modes of seech siga i term of aturaess, ideibiity ad easatess which are of re-requisite i may seech sythesizers 20,21. Harmoic/Noise mode is a itch sychroous system, ad uie other cocateative aroaches hece it eimiates the robem of sychroizatio of seech frames, ad shows the caabiities of rovidig high quaity rosodic modificatios without busiess whe comared to other methods 22. For these reasos Harmoic/Noise mode was chose by AT&T to serve as the baced i their Next Geeratio Text to Seech (TTS) System which is caimed to roduce extremey high quaity sythetic seech. Aso Harmoic/Noise mode is widey used i other framewors. For exame, it has bee quite aturay ivoved i the deveomet of voice modificatio, trasformatio, ad coversio systems 23, 24, 25. Harmoic/Noise mode has bee used for the deveomet of a high quaity vocoder aicabe i statistica framewor articuary i moder seech sythesizer 26. Oe mai drawbac of the harmoic oise mode is its comexity Cocusios I this aer, we have reseted a overview of severa seech modes used i seech sythesis. Severa aroaches have bee roosed for seech sythesis modeig, some of them have bee briefy discussed ad the most imortat oes, amey, Liear Predictio mode, siusoida mode, ad Harmoic/oise mode were discussed i some detais. The iear redictio mode was oe of the most imortat seech sythesis modes uti the ed of the 1980s, after which it gave way to more comex techiques which offered a better siga quaity, e.g., siusoida mode ad its derivatives. Siusoida mode is eseciay suited for modeig voiced souds, due to the icaabiity of the mode to cature oisy soud we, it is ecessary to use other tyes of modes which decomose the seech siga ito severa arts as the determiistic/stochastic mode, or the Harmoic/Noise mode. However, these modes caot be used to mode the trasiet art of seech siga. The soutio is to use aother tye of modes caed Harmoic/Trasiet/Noise mode or Determiistic/Trasiet/Stochastic mode. Comared to the existig seech sythesis modes, Harmoic/Noise mode shows more ractica otetia for seech sythesis. It is widey used i severa seech sythesis framewors as it offers high quaity seech with a reativey smaer umber of arameters, ad with ease itch ad time scae modificatio. To imrove the quaity of the sythesized seech, ew methods for extractig certai arameters of the Harmoic/Noise Mode are roosed i the iterature.

8 Tabet Youcef et a. / Procedia Comuter Sciece 73 ( 2015 ) Refereces 1. Saeed V. Vaseghi. Mutimedia Siga Processig.Theory ad Aicatios i Seech, Music ad Commuicatios.Wiey MariaiJ.Laguage ad seech rocessig. ITSE, WELLEY Serra X. A System for Soud Aaysis, Trasformatio, Sythsis based o a Determistic us Stochastic Decomositio.PhD thesis, Staford Uiversity George E.B.adSmith M. J.T. Seech aaysis/sythesis ad modificatio usig a aaysis-by-sythesis/overa-add siusoida mode.ieee Trasactios o Seech ad Audio Processig (5): Verma T., Levie S., ad Meg T. Trasiet modeig sythesis: a fexibe aaysis/sythesis too for trasiet sigas. I Proc. of the ICMC 1997, Thessaoii, Greece Peeters G. ad RODET X. Sioa. A ew aaysis/sythesis method usig sectrum ea shae distortios, hase ad reassiged sectrum.i Proceedigs of the Iteratioa Comuter Music Coferece (ICMC 99) Nieuwehuijse J., Heusdes R., ad Derettere E. Robust exoetia modeig of audio sigas. IICASSP 1998, Seatte, WA, USA.May Jese J, Jese S. H, ad Hase E. Exoetia siusoida modeig of trasitioa seech segmets.i Proceedigs of IEEE Iteratioa Coferece o Acoustic, Seech, ad Siga Processig (ICASSP) Vo 1, Jese J., Heusdes R., adjese S. A ercetua subsace aroach for modeig of seech ad audio sigas with damed siusoids.ieee Trasactio o Seech ad Audio Processig (2): Keesi M. ad Weruaga L. Adative chir-based time-frequecy aaysis of seech. Seech Commuicatio, : Mcauay R.J., Quatieri T.F. Seech aaysis/sythesis based o a siusoida reresetatio. IEEE Tras. o ASSP.1986.vo. 34, o Du R. B. ad Quatieri T. F. Siewave Aaysis/Sythesis Based o the Fa-Chir Trasform. Worsho o Aicatios of Siga Processig to Audio ad Acoustics (WASPAA). October, Du R. B., Quatieri T. F., ad MaysaN. Siewave arameter estimatio usig the fast fa-chir trasform.iworsho oaicatios of Siga Processig to Audio ad Acoustics (WASPAA) Fat G. Acoustic Theory of Seech Productio, Mouto, La Hague-Paris Rabier L. R. ad Schafer R. W. Itroductio to digita seech rocessig. Foudatios ad Treds i Siga Processig vo. 1, o. 1, Rabier L. R. ad Schafer R. W., Digita Processig of Seech Sigas. Pretice-Ha Ic Ata B. S. ad Haauer S. L. Seech aaysis ad sythesis by iear redictio of the seech wave. Joura of the Acoustica Society of America Tayor P. Text-to-Seech Sythesis.Cambridge Uiversity Press Styiaou Y. Harmoic us Noise Modes for Seech combied with Statistica Methods, for Seech ad Seaer Modificatio. PhD thesis, E.N.S.T Paris Laroche J., Styiaou Y., ad Mouies E. HNS: Seech modificatio based o a harmoic + oise mode. Proceedigs of the Iteratioa Coferece o Acoustics, Seech, ad Siqa Processig : Styiaou Y., Laroche J., ad Mouies E. High quaity seech modificatio based o a harmoic + oise mode. Proceedigs of EURO SPEECH Syrda A., Styiaou Y., Garriso L., Coie A., ad Schroeter J. TDPSOLA versus harmoic us oise mode i dihoe based seech sythesis. Proceedigs of the Iteratioa Coferece o Acoustics, Seech ad Siga Processig Styiaou Y., Caé O. ad Mouies E. Statistica methods for voice quaity trasformatio.i Proceedigs of the Euroea Coferece o Seech Commuicatio ad Techoogy (EUROSPEECH 95) Styiaou Y. Caé O. ad Mouies E. Cotiuous robabiistic trasform for voice coversio.ieee trasactios o Acoustics, Seech ad Siga Processig (ASSP) (2): Oudot M. Aicatio du modèe siusoïdes et bruit au décodage, au débruitage et à a modificatio des sos de aroe. PhD thesis, Ecoe Natioae Suérieure des Téécommuicatios, Paris Saiz I., Navas E., Heraez I. Harmoic us oise mode based vocoder for statistica arametric seech sythesis. Seected Toics i Siga Processig, IEEE (2).