H. L et al.: A Scalable HDD Vdeo Recordng Soluton Usng A Real-tme Fle System A Scalable HDD Vdeo Recordng Soluton Usng A Real-tme Fle System Hong L, Stephen R. Cumpson Member, IEEE, Robert Jochemsen, Jan Korst and Nek Lambert 663 Abstract We present a scalable soluton that enables a sngle manstream hard dsk drve to smultaneously record and play back multple real-tme vdeo streams, n addton to supportng non real-tme PC-lke applcatons. Our soluton can handle dfferent types of dsks, dfferent numbers of streams wth dfferent bt rates, and arbtrary non-rt applcatons at the same tme. Index Terms HDD, vdeo recordng, real-tme, fle system. I. INTRODUCTION anstream IDE Hard Dsk Drves (HDD) have enabled Mrandomly accessble mass storage for home audo/vdeo (A/V) applcatons. The contnuously mproved HDD performance and capacty have enabled revolutonary functons on a CE devce, such as Personal Vdeo Recordng (PVR), tme-shftng, network surfng, games, ntellgent collectng and navgatng of prvate and publc A/V content and nformaton usng broadcast and network servces, and other nterestng Personal Computer (PC) functons. For an advanced Home Meda Server (HMS) that can support the above exctng functons, the HDD needs to support multple smultaneous real-tme (RT) A/V streams n addton to non-rt applcaton requests. Such a HDD recordng soluton should be scalable and should not mpose restrctons on the applcaton software mplementaton. Although manstream HDDs have reached hgh throughput, such as ~2 MB/s for contnuous readng or wrtng, the actual throughput of scattered data blocks could be unacceptably low due to the swtch overhead of the HDD. For an advanced HMS that supports multple users wth multple A/V streams, the system requres HDD throughput schedulng and proper fragmentaton lmts for A/V fles on the dsk to guarantee real-tme (RT) streamng throughput for the streams. Typcal embedded system mplementatons based on PC fle systems cannot provde streamng guarantees, and often the amount of PC-lke (non-rt) dsk requests has to be strctly lmted. Several mplementatons use pre-allocated fles on a PC fle system to avod fragmentaton for A/V fles. But ths needs separate admnstraton code for A/V content. Another problem for PC fle systems, e.g. a FAT fle system, s that t could become corrupt due to a power falure. Typcal embedded mplementatons have no specal HDD The authors are all wth Phlps Research Laboratores (Endhoven), Prof. Holstlaan 4, 5656AA Endhoven, The Netherlands. (e-mal of Hong L: hong.r.l@phlps.com) Manuscrpt receved June 17, 23 98 363/ $1. 23 IEEE tme assgnment functon. The dsk throughput assgnment s done at applcaton level, or s applcaton dependent. Therefore t works only for dedcated applcatons. Some mplementatons do not care about the streamng performance at all; they smply assume hgh enough dsk throughput and smple enough applcatons runnng on the system. The above mplementatons often requre sgnfcant reengneerng effort, f a developer wants to swtch to dfferent types of dsks, or to support a dfferent number of streams wth dfferent bt rates, or to upgrade wth varous connectvty features and other PC functons. The soluton we present has the requred scalablty and t s robust aganst power falures. II. GUARANTEEING REAL-TIME PERFORMANCE FOR STREAMING When enjoyng A/V content, consumers do not accept vsble or audble A/V artfacts, such as lost frames, (MPEG2) block artfacts, mage hccups, strange sounds, etc. These artfacts could be caused by any component n the stream process chan, e.g. a HDD, an encoder/decoder, buffers, buses, CPU, etc. Focusng on usng the HDD n ths paper, we want to guarantee real-tme streamng performance by guaranteeng suffcent throughput for each stream. In other words, the streamng buffer between the HDD and a stream processng chan never overflows (for recordng to the HDD) or underflows (for playng back). We also want the buffer sze to be small for latency and cost reasons. We want guaranteed streamng performance ndependent of random non-rt dsk requests. The above requrements can only be reached wth predctable HDD performance. However, for current standard (ATA-6 and pror) IDE dsks, the tme penalty caused by the Error Recovery Procedure (ERP) of the drve s not predctable. The streamng throughput guarantee n ths paper s a nomnal guarantee, whch does not take the rare ERP events nto account. We assume that the CPU latency s low enough for ssung HDD commands n tme. Ths s usually true because onemllsecond latency s acceptable for HDD control. We also assume that the bus throughput and the rest of the stream process chan have suffcent performance. A. Stream Bt Rate Upper Bound Although a compressed A/V stream, e.g. an MPEG2 stream, may have a varable bt rate, the maxmum bt rate s usually a good upper bound to calculate the guaranteed dsk throughput for a stream.
664 The maxmum bt rate condton leads to over-reservaton of HDD throughput n realty. For typcal DVB broadcast encoders, the average bt rate can be half of the maxmum bt rate. The system may waste dsk bandwdth, f other dsk requests cannot take advantage of the unconsumed tme that s reserved for a varable bt rate stream. Such slack tme can be used to mprove the throughput for non-rt dsk requests. B. Predctable HDD Performance Manstream HDDs organze ther meda platters n tracks and sectors. A track conssts of all sectors on a crcle and a sector conssts of 512 contnuous bytes. Ignorng the rare random ERP events, the nomnal command completon tme of a dsk request s predctable and conssts of seek tme, rotatonal delay and transfer tme. The seek tme s for movng the head to the target track. It can be modeled wth a lnear functon of the seek dstance. The rotatonal delay s spent on watng for the frst sector of the data block on the track. On average t s a half rotaton. The maxmum rotatonal delay s a full rotaton. The transfer tme s for readng or wrtng the contguous sectors. It vares over the platter surface, because the number of sectors per track vares due to zonng. The performance vares for dfferent HDDs, e.g. wth dfferent meda densty, n dfferent operaton modes and n dfferent form factors. Reference [1] presents methods to characterze the HDD performance and to predct the nomnal command completon tme for gven dsk requests. Accessng scattered sectors costs sgnfcant dsk swtch overhead, whch ncludes seek tme and rotatonal delay. For a manstream HDD, a full rotaton s more than the tme requred for transferrng 1 Kbytes. The maxmum seek tme (between the nner- and the outer-most tracks) s often three to four tmes that of a full rotaton. Ths tme s equvalent to that of transferrng 5 Kbytes. A careless desgn could easly waste 9% of HDD tme on the swtch overhead: e.g. f the head moves after access of every few sectors. C. Data Fragmentaton on the Dsk Wthout a proper data fragmentaton lmt, a HDD vdeo recordng system cannot effectvely guarantee dsk throughput for A/V streams. The FAT fle systems, whch are standard n the PC world, cannot provde sutable fragmentaton lmts for vdeo applcatons, because they use small fle allocaton unts, such as 4-64 KB. Unless usng specal technques lke preallocated or pre-clustered fles, a vdeo fle can be scattered n small blocks on the dsk, especally when the system has been runnng wth an almost full dsk, whch s normal for a PVR or an HMS applcaton. Usng a large fle allocaton unt e.g. hundreds or thousands of sectors, can provde proper fragmentaton lmts for A/V fles. But such large allocaton unts waste valuable dsk capacty for small (e.g. a few klobytes) fles, snce a unt can only be used for one fle. IEEE Transactons on Consumer Electroncs, Vol. 49, No. 3, AUGUST 23 Usng dfferent allocaton unt szes can save capacty, but t ncreases the complexty of the fle system. A practcal soluton s to use multple dsk parttons wth dfferent fle systems, although end-users mght not lke the fxed partton sze. D. HDD Throughput Schedulng Suffcent dsk throughput can be guaranteed for each RT stream by usng a dsk scheduler. The scheduler repeatedly assgns dsk tme to each stream accordng to ts bt rate, so streams wll not run out of data durng playback or out of space durng recordng. Cycle-based schedulng methods, such as sngle sweep schedulng and dual sweep schedulng [2], optmze dsk swtch overhead by re-orderng the sequence of servng the streams. In a cycle perod these methods serve every stream once. The sequence of servng the streams s ordered so that the dsk head moves monotoncally n one or two sweeps every cycle. The swtch overhead s therefore mnmzed, no matter where the stream fles are allocated on the dsk. Because ths knd of scheduler s desgned wth worst-case boundary condtons, such as the maxmum bt rates, the maxmum swtch overhead and sustaned transfer speed, the dsk on average takes less tme than estmated for servng all the streams. The saved tme can be used for non-rt requests. If the desgner wants a certan non-rt throughput he can also reserve some tme for non-rt requests. E. HDD Error Recovery Most dsk commands take a predctable amount of tme. However, a very lmted number of dsk commands take much more tme due to dsk access errors, such as data error and seek error, etc. Current IDE HDDs use an unnterruptble Error Recovery Procedure (ERP) to do retres for error recovery, whch may take up to several seconds. The ERP tme s not predctable and cannot drectly be taken nto account n guaranteeng the streamng performance. If the HDD hts an error, all streams may be nterrupted. To make the ERP tme predctable, a Streamng Data Transfer Feature Set proposal has been defned n the ATA standardzaton commttee [3]. Ths feature set allows grantng a tme budget to every dsk streamng command. If the HDD cannot complete the command n tme, t wll store the status and return to the host n tme, but wth an error. It s the responsblty of the host (e.g. the schedulng algorthm) to determne whether t has tme to ssue a command to resolve the error. In such a way, one dsk access error only nfluences the stream that ssued the command. The other streams wll be unaffected. Because data ntegrty s mportant for non-rt requests, the scheduler should resolve errors of a non-rt request. The feature set allows the scheduler to nterleave the recovery wth streamng commands to guarantee the real-tme throughput. It s also useful to have an extra mechansm to dagnose real dsk
H. L et al.: A Scalable HDD Vdeo Recordng Soluton Usng A Real-tme Fle System 665 defects from the reported errors. The Streamng Data Transfer Feature Set could be approved n the comng ATA standard. III. SOLUTION To acheve a scalable HDD vdeo recordng soluton for advanced HDD vdeo recorders and home meda servers, we have developed a real-tme fle system (RTFS) to guarantee streamng performance n addton to arbtrary non-rt dsk requests. Fg. 1. Scalable HDD recordng soluton usng the RTFS. A. Archtecture Fg. 1 shows the structure of our soluton. Applcatons can access fles on the HDD n streamng mode, or n non-rt mode that s the same as normal HDD access on a PC. The RTFS s central to the soluton. Real-tme streamng s acheved usng the RTFS between the HDD and streamng devces, such as an encoder, a decoder, a dgtal tuner, a home network channel or other dgtal nput/output devces. A streamng buffer s used for each stream, wth optmzed sze determned by the RTFS. Non-RT dsk requests, whch are ssued va the RTFS or a legacy FS (embedded n the operatng system), are completed through the RTFS scheduler. Supportng the legacy fle system enables use and/or reuse of exstng non-rt applcatons. In ths archtecture, the RTFS fle system layer and the legacy FS manage fle allocatons (n ther own parttons) on the dsk. Only those fles n the RTFS partton can be streamed wth guaranteed throughput, because the RTFS fle system layer can provde proper fle fragmentaton lmts for guaranteeng throughput. The RTFS scheduler s a component on top of the HDD drver, whch guarantees throughput of regstered streams. Usng a dsk performance model [1] to estmate nomnal command completon tme, t schedules the pendng requests receved from the RTFS fle system layer and the legacy FS, and executes them n the scheduled sequence usng the HDD drver, so that the throughput of each stream s guaranteed. In our soluton, streams work as follows: An applcaton can dynamcally start a new stream wth a requred bt rate usng an admsson control negotaton mechansm of the RTFS. Durng the negotaton, the RTFS scheduler returns the streamng buffer sze to the applcaton. When a recordng stream has started, the streamng devce contnuously produces data n the streamng buffer. Such data producton events are sgnaled to the streamng thread around the RTFS, whch montors the avalable data n the streamng buffer. A streamng thread n the fle system layer ssues streamng requests to the scheduler. Usng a cycle-based schedulng method, the scheduler makes an executon sequence of all pendng requests (from all streams and non-rt applcatons) at the begnnng of a cycle perod. The sequence guarantees streamng throughput and mnmzes seek overhead. Non-RT requests are nserted nto the sequence f the total nomnal command completon tme s less than the cycle perod. Then the scheduler sends the sequence of requests one by one to the HDD. When a request s completed by the HDD, the scheduler returns an acknowledgement to the fle system layer or the legacy FS. If t s a RT request, the streamng thread wll receve the acknowledgement and wll then release the space n the streamng buffer. Immedately after the completon of the sequence, a new cycle starts f the scheduler has receved new requests. The streamng contnues so untl the applcaton stops the stream. The fnalzaton of the stream wll stop the streamng devce, delete the stream regstraton and ts allocated bandwdth n the scheduler, release the buffer and kll the stream thread tself. The same holds for a play back stream, where data s streamed from the dsk to the stream devce. The RTFS s applcaton ndependent. Ths enables us to use the RTFS for a varety of applcatons. The dsk scheduler and the HDD n combnaton behave as a streamng dsk. The addton of the fle system makes the RTFS a streamng fle system. B. Real-tme Fle System Some detals of the fle system layer and the dsk scheduler explan how streamng throughputs are guaranteed. 1) The fle system layer The FS layer uses a farly large dsk allocaton unt optmzed for vdeo to mnmze the negatve mpact of dsk head jumpng and fragmentaton whlst streamng. Small fles are expected n other parttons on the dsk, snce the legacy FS as embedded n the operatng system s used for bootng, applcaton loadng, etc. To mnmze swtch overhead, t s stll preferable to avod request splts at the boundary of allocaton unts n a fle. The RTFS uses a fracton of the allocaton unt as the data block sze for streamng access of the HDD. The data block sze can be derved from the requred stream bt rate and the allocaton unt sze.
666 It s mportant for a CE devce to prevent fle system corrupton due to a power falure. The FS layer uses a number of tables for fle system management nformaton on the dsk. The tables are atomcally updated n an alternatng way, ensurng that there s always a vald set of tables on the dsk and total system ntegrty s mantaned. However, a power falure may stll lead to loss of some of the latest content that has just been wrtten nto the fle, because the fle system management nformaton of the fle has not been saved to the dsk. In order to mnmze potental data loss after a power falure, system desgners can also choose to frequently update the fle nformaton on dsk. The FS layer has normal fle management tools, such as lstng the fles and drectores, openng and closng fles, etc. A brdge can be used to map these tools to the standard fle system nterface of the operatng system, so that any thrdparty applcaton can access the RTFS partton n normal non- RT mode va the legacy FS nterface. 2) The dsk scheduler The RTFS dsk scheduler uses cycle-based methods [2], whch schedules dsk requests n repeatng cycle perods. In each cycle perod, t serves one request per stream wth a request sze large enough for the stream to survve a whole perod. If the total command completon tme for all regstered streams s less than the cycle perod, the streams are schedulable, as shown n (1). B n P =1 B / r + SW ( n ) P r d where, P s the cycle perod, r s the requred bt rate and B s the data block sze transferred for stream n a cycle, r d s the sustaned transfer rate of the dsk, and SW(n) s an upper bound on the total swtch overhead for the n streams n the cycle. The sustaned transfer rate of a dsk s the rate of transferrng a contnuous data block on the nner most tracks, whch could be half of the transfer rate on the outermost tracks. Ths s because the outermost tracks have more sectors. The total swtch overhead ncludes all seek tme and the rotatonal delay for all the streamng requests n the cycle. Wthout seek optmzaton, e.g. Round Robn schedulng that serves streams n a fxed sequence every cycle, the SW(n) should count a full seek and a full rotaton for each stream. Ths becomes sgnfcant n the cycle perod, when the number of streams ncreases. For sngle sweep schedulng that orders all requests n a cycle n a monotonc sequence for seekng, the swtch overhead upper bound can be derved usng a lnear (along the seek dstance) seek tme model [4]. SW s s 2 s s n 1 ( n) = ( s + ) + ( n 1) ( s + ) + n R (1) (2) IEEE Transactons on Consumer Electroncs, Vol. 49, No. 3, AUGUST 23 In (2), s s the full-dstance seek tme, s s the offset (wth zero seek dstance) of the lnear seek tme model, n s the number of streams. The last part of (2) counts a full dsk rotaton R for each stream n the upper bound. The frst part s a half full-dstance seek, whch bounds the seek tme to the nearest nner/outer-most request n the sequence. The mddle part bounds the seek tme for other requests n the sequence whch, n worst case, equals the total seek tme when the requests are evenly dstrbuted over the dsk. The buffer sze s determned by the data block sze B and the schedulng method. For example, for the varable bt rate case t s twce the block sze for Round Robn schedulng: one block for the dsk request and the other for bufferng data used by the streamng devce n the cycle perod. The buffer requred for the sngle sweep schedulng s three tmes the block sze, the thrd block s requred due to request reorderng. The cycle perod s chosen durng the ntalzaton of the dsk scheduler. Usng a larger cycle perod reduces the relatve swtch overhead n a cycle, but requres more memory. It also nfluences the response tme of a user acton. Worst-case response tmes go up, but the effect on average response tme s much less clear and depends on several factors. 3) Dsk performance model The dsk performance model of the RTFS conssts of small tables of dsk parameters, such as capacty, rotaton speed, and zone nformaton. It uses a square-root model for short seeks, and a lnear model for long seeks. It also has nterface functons used by the scheduler. Durng streamng the dsk performance model s used to estmate the command completon tme of dsk requests. Durng the admsson control for startng new streams, t s also used to calculate the schedulablty and dsk streamng parameters, such as the block sze and the buffer sze. C. Non-RT Requests In our soluton, non-rt requests come from both the RTFS and the legacy FS. These requests are used to update the fle system management nformaton, to support non-rt applcatons lke content browsng, to manage a metadata database, and to support arbtrary thrd-party applcatons. The system desgner has no knowledge and no control of when and how many non-rt requests are ssued. 1) Usng slack tme Non-RT dsk requests are handled whenever the RTFS scheduler sees slack tme n a cycle. The slack tme s the free tme excludng the tme used for the streams. So non-rt requests have no nfluence on the streamng performance. The slack tme s there because the streamng requests usually spend less tme than the scheduled tme, whch s calculated wth the maxmum streamng bt rate and the overhead upper bound SW(n) durng the admsson control. In practce t s not necessary for the RTFS to reserve a certan
H. L et al.: A Scalable HDD Vdeo Recordng Soluton Usng A Real-tme Fle System 667 dsk throughput for non-rt requests. All dsk requests from the legacy FS are handled as non-rt requests by the scheduler. As a result, random legacy applcaton requests to the dsk are handled wthout breakng the real-tme guarantees for streamng. 2) Splttng a non-rt request The RTFS scheduler can splt and complete a non-rt request n several cycles, because the sze of the request can be too bg to ft n the free tme of one cycle. The request sze, e.g. of a thrd-party applcaton, can actually be too bg for the whole cycle, because the desgners never know the requrements of each other. 3) Reorderng non-rt requests For seek optmzaton, t s advantageous to reorder non-rt requests. Ths s not always allowed though, because reorderng can, for example, create nconsstences n case of a power falure. The RTFS does not do reorderng on the requests of the legacy fle system for ths reason. Requests related to dfferent fles on the RTFS can be reordered, because they are ndependent. Note that request reorderng s not a problem for tmeshftng of a stream, or playback durng the recordng. Before a recordng request s completed, the data block s not a part of the fle. So the playback cannot ssue a command to read the block. IV. SCALABILITY For whatever reasons, be t hgher performance, lower cost, new features and/or more functons, scalablty s becomng more mportant for HDD recorders and Home Meda Servers wth respect to dsk types, streamng requrements, non-rt throughput, etc. A. HDD Types and Modes The performance of dfferent types of dsk drves vares due to dfferent rotatonal speed, seek tme, meda geometry, and form factor, e.g. 1/1.8/2.5/3.5 nch. Usng a dsk performance model generated by measurements, the system can adapt to changes n dsk performance, both when a dfferent dsk s used and when dsk performance deterorates over tme. In our soluton, performance models for dfferent dsks can be extracted n the desgn phase or at run tme. The system software does not need to be changed when a dfferent dsk s connected. A CE devce s expected to keep ts operatonal nose level low enough to keep the devce unobtrusve n a bedroom or lvng room. Manstream HDDs usually have two operatonal modes: a normal mode that has the same performance as n a PC, and a slent mode that has a reduced seek performance but s much queter. Our performance model uses dfferent parameters for both modes. It can swtch to the correct set of parameters at run tme accordng to the preferred dsk confguraton. The dsk performance may deterorate over tme due to wear and be affected by envronmental changes (e.g. hgh temperature) [5]. Our soluton allows usng a performance montor to update the performance model. In such a way, the system can follow the envronmental change and the performance deteroraton, and stll provde best guaranteed streamng. B. Varous Streams Typcal HDD recordng solutons are mplemented for fxed stream requrements, such as a fxed number of streams wth fxed bt rate. It may requre complete re-engneerng to swtch to dfferent streamng requrements. 1) Varous streams Dfferent A/V streams are nterestng for a HMS, such as varous MPEG2 SDTV streams (<1Mbps), MPEG2 HDTV streams (~2 Mbps), and varous MPEG4 vdeo streams from very low bt rates (<1 Mbps) to hgh bt rate. Audo streams have even lower (~2 Kbps) bt rates and are usually handled n non-rt mode. 2) Admsson control The RTFS acheves applcaton flexblty by usng an admsson control mechansm. New streams wth varous bt rates can be admtted at run tme by an applcaton. The admsson control negotaton takes a few steps: 1. Check schedulablty: Frst, the applcaton asks the RTFS whether a new stream s schedulable wth a guaranteed bt rate. If schedulable, the RTFS scheduler wll calculate the streamng block sze and the buffer sze, and return them to the FS layer and the applcaton. 2. Prepare stream: When the applcaton receves those parameters, t creates a streamng buffer and lnks t to the streamng devce. 3. Start the stream: Then the applcaton asks the RTFS to start the stream. The RTFS checks agan the schedulablty, adds the stream to the scheduler. A new stream starts streamng. 3) Schedulablty of the soluton Under worst-case condtons, such as full fle system fragmentaton, our soluton can smultaneously handle, for example, four HDTV (2 Mbps) streams plus four SDTV ( 1 Mbps) streams on a 3.5-nch manstream IDE dsk. C. Non-RT Throughput Besdes guaranteeng streamng throughput, the soluton has no restrcton on HDD access for applcatons n the non-rt mode. Furthermore, t also optmzes the performance of the non-rt access. 1) Usng maxmum tme adaptvely In many streamng solutons, the assgned throughput for non-rt requests by the scheduler s fxed n the desgn. In our
668 soluton the scheduler can use all of the slack tme for the non- RT requests. The throughput of the non-rt requests s hgher than n the fxed case, under the same streamng load. The throughput can be even hgher when the stream load decreases. 2) Prortzaton of non-rt requests The RTFS scheduler has several prortes for non-rt requests. Requests wth hgher prorty are served earler. For example, updatng fle system management nformaton uses the hghest prorty to mnmze the probablty of data loss, whle background defect detecton may use the lowest to ncrease bandwdth for user operaton. Prortzaton helps to use lmted resources, such as the dsk bandwdth and the power n a moble case, for the most mportant thngs. 3) Legacy FS Dfferent approaches are possble n the way the RTFS handles dsk requests from legacy fle systems, e.g. n Lnux, n WnCE or n other operatng systems. A good soluton should allow the system to do bootng and vrtual memory on the HDD. One method s to redrect the HDD drver calls of the legacy FS to the RTFS scheduler. It s a smple approach, f clear and usable drver nterfaces can be found. A dsadvantage s that the throughput of such legacy FS requests could be reduced. Ths s because the request s handled synchronously at the HDD drver level and the scheduler can schedule only one request n a cycle. For sophstcated fle systems lke the one n Lnux, t s dffcult to redrect the drver nterface to RTFS, because the sophstcated memory management and devce management n Lnux kernel s complcated. We found a way to add a RTFS hook nto the kernel. The hook can send a snapshot of the pendng request lst from the kernel to the RTFS scheduler. The scheduler then determnes whch Lnux requests can be executed wthout mpact on streamng guarantees. Although ths s not a very clean soluton, t keeps the RTFS code hghly ndependent from the operatng system, and provdes good performance. Wth ths soluton we can run Lnux applcatons, such as Netscape Internet browser and complng the whole kernel, whle the same dsk s recordng and playng back 12 DVB streams. 4) Improvng non-rt performance Usually one applcaton synchronously ssues non-rt dsk requests: when one s fnshed, the next s ssued. It s also usual that the applcaton ssues non-rt requests only at certan moments. But when t needs dsk access, t wants the full throughput. The same holds for the users. It s a good dea to try to create a large tme slot for the non- RT requests, because of ths bursty behavor of non-rt requests. For nstance, after completng a scheduled request sequence, the scheduler uses the remanng tme of the cycle only for the non-rt requests. In such a way non-rt requests get the full HDD bandwdth, although n part-tme. IEEE Transactons on Consumer Electroncs, Vol. 49, No. 3, AUGUST 23 Creatng a large tme slot for non-rt requests does not conflct wth the dea of decreasng user response tme, whch s the reason for the scheduler to start a new cycle mmedately after the end of the prevous one. User actons are usually related to non-rt dsk requests, e.g. changng a menu or browsng content nformaton. These actons can typcally be handled faster n ths way. V. IMPLEMENTATION & RESULTS Our soluton s operatng system ndependent, except the legacy FS lnk. The mplementaton uses a small operatng system abstracton layer, ncludng memory allocaton, semaphores, message queues, etc. The frst RTFS verson was mplemented on the embedded operatng system psos. The later versons are bascally stll operatng system ndependent, but were developed n user space on Lnux. One demonstraton of the soluton s on a Lnux PC wth a sngle IDE hard dsk. The PC has a 35MHz Pentum II processor. The manstream dsk contans an RTFS partton for A/V streams and Lnux parttons for normal PC functons. Dedcated PCI cards are used for DVB nput, as well as for decodng vdeo. A 2.4x wrtng speed DVD+RW drve s connected to the second IDE controller as a DVD archver. The streamng applcatons we run on the demonstrator are multple tme-shfters, whch smultaneously records DVB broadcastng channels on the dsk and plays the recordngs back. Meanwhle, we run varous Lnux functons: from Internet browsng to software development. We also run a DVD archvng functon usng Open Source software, whch creates a DVD vdeo mage from a user selected recordng on the HDD, and then burns a DVD dsc wth the DVD+RW drve. Usng a manstream 3.5-nch IDE dsk and a sngle sweep scheduler wth 6 ms cycle perod, the soluton can smultaneously handle 6 ndependent tme-shfters, or 12 DVB streams. The DVB streams are typcal TV channels broadcast n Europe wth varable bt rates n the range of 4-1 Mbps. Each stream uses a 1.4 MB buffer. Wth 12 DVB streams accessng the dsk, all Lnux functons work well. The DVD archvng functon works roughly n real-tme. No matter what Lnux applcatons are runnng, the tme-shftng vdeo streams look perfect. Users can also do trck-play, such as pause, fast/back-forward and jump-to-lve, etc, although the fast/back-forward has no guaranteed performance. Usng a 1-nch IDE dsk, the soluton can smultaneously handle 3 DVB streams. We measured the throughput of the streams and Lnux applcatons wth dfferent stream settngs on a 3.5-nch manstream IDE dsk. The dsk has 4 GB capacty, 72 rpm rotatonal speed, and 18.8 MB/s sustaned bt rate. It works n slent mode wth 32 ms full-dstance seek tme, and the offset of the lnear seek tme model s 14 ms. The streams are for the tme-shfters. The fles are evenly dstrbuted over the RTFS partton, whch s 8% of the whole dsk. The throughput of
H. L et al.: A Scalable HDD Vdeo Recordng Soluton Usng A Real-tme Fle System 669 Lnux applcatons s measured usng a huge contnuous dsk copy applcaton. Note that Lnux tself also ssues some dsk requests, whch are not counted n the throughput. The streams have varable bt rates. Ths leads to large varatons n the measurements. Stll, the trade-off between real-tme and besteffort throughput s clearly vsble. As shown n Table I, the Lnux dsk copy applcaton stll has at least 1 MB/s throughput, n the case of 12 DVB streams. TABLE I MEASURED THROUGHPUT OF STREAMS AND LINUX APPLICATIONS # DVB streams 12 1 8 6 4 2 1 Total stream (MB/s) 7.3 5.6 4.1 3.4 2.4 1.6.7 Lnux (MB/s) 1. 3.3 4. 5.6 6.8 8.4 9.9 The streams have varable bt rate The throughput ncreases when the number of streams decreases or when the total stream bt rate drops. Non-RT applcatons automatcally get ncreased throughput when less streamng throughput s needed. The total throughput of the streams and the non-rt requests should ncrease when the number of streams drops. Ths s because for the case of a large sequental Lnux dsk copy acton the swtch overhead drops. But n measurements t bascally remans constant. The total throughput s only about the half of the sustaned bt rate. Ths matches our calculaton only for 12-stream case, because the swtch overhead s more than 5% of the cycle perod. When the number of streams drops, the non-rt requests throughput dd not take full advantage of ths. The reason s that the scheduler receves at one tme only a small part (128 KB) of the large requests ssued by the Lnux dsk copy applcaton, and t takes some tme before the new request arrves at the scheduler. Ths results n a loss of non- RT throughput. The whole 12-stream applcatons cost approxmately 5% processor load for the Pentum II processor. Admttng fewer streams results n progressvely more bandwdth for PC functons, whch demonstrates scalablty. Smlar results are observed wth other manstream dsks from all the major HDD manufacturers. VI. CONCLUSION The RTFS uses a tuned combnaton of technques lke dsk schedulng and fle allocaton management. It s dsk ndependent, and supports multple real tme A/V streams n combnaton wth tradtonal PC functons. It enables scalable soluton for convergng CE functonalty and PC functonalty on advanced products lke Home Meda Servers. ACKNOWLEDGMENT The authors thank our colleagues: Ruud Wjnands, Özcan Mesut, Mscha Geldermans, Ronald van Neuwberg and Ad Denssen, for ther great contrbuton to ths project. REFERENCES [1] Ö. Mesut, N. Lambert, HDD Characterzaton for A/V Streamng Applcatons, IEEE Trans. Consumer Electron., vol. 48, no. 3, pp. 82-87, Aug. 22. [2] J. Korst, V. Pronk, P. Coumans, G. van Doren, and E. Aarts, Comparng dsk schedulng algorthms for VBR data streams, Computer Communcatons, 21(1998) pp. 1328 1343. [3] Darren Bulk, Proposal for Streamng Data Transfer Feature Set, T13 ATA pro-posal d99128, http://www.t13.org/techncal/d991281.pdf [4] Y.-J. Oyang, A tght upper bound of the lumped dsk seek tme for the scan dsk schedulng polcy", Informaton Processng Letters, vol. 54, No. 6, pp. 355-358, 1995 [5] R. Severt and S.E. Frankln, Degradaton of AV Streamng Performance of Hard Dsk Drves at Elevated temperature, ASME nformaton storage and processng sys-tems conference, Santa Clara, CA, 22 Hong L receved hs B.Sc. (1988) on Electroncs at TsngHua Unversty n Bejng, Chna. He worked on dynamcal ECG acquston and analyss for hs MSc (1992) and PhD (1995) degrees at the same unversty. Snce 1996, he worked on mage presentaton and recognton as a research fellow n Pattern Recognton Group at Delft Unversty of Technology, the Netherlands. He joned Phlps Research Laboratores n Endhoven (Netherlands) n December 1998, worked on HDD A/V storage systems, real-tme fle system and PVR. He s currently workng on home meda servers ncludng content management and dgtal rghts management. Stephen Cumpson (M' 92) receved BSc/MEng n Electrcal & Electronc Engneerng, Unversty of Manchester 1992; PhD, Unversty of Manchester 1996. He joned Phlps Research Laboratores n Endhoven (Netherlands) n 1996. He worked on dgtal magnetc recordng for DV, ADR, thermally asssted magnetc recordng, real-tme fle system, personal vdeo recordng and HDD usage n CE applcatons. He s currently workng on dstrbuted storage n the connected home. Robert Jochemsen receved hs MSc n Computng Scence at the Unversty of Gronngen, The Netherlands n 1996. He joned Phlps Research Laboratores n Endhoven n 1999. He has snce worked on robust real-tme fle system technology for sold state devces and hard dsk based vdeo recorders. Jan Korst receved hs MSc degree n mathematcs from Delft Unversty of Technology, and hs PhD degree from Endhoven Unversty of Technology, both n the Netherlands. Snce 1985, he has been wth the Phlps Research Laboratores n Endhoven. Over the years he has been workng manly on schedulng and resource management problems n the areas of IC desgn and multmeda systems. Nek Lambert receved Ir. degree n Physcs, Delft Unversty of Technology 1981; PhD, Delft Unversty of Technology 1986; Phlps Research Laboratores n Endhoven (Netherlands) and Brarclff Manor (USA) snce 1986; has worked on superconductors, cryocoolers, varous thn flat dsplays, reflectve lqud crystal projecton dsplays, real-tme fle system and personal vdeo recordng, and s now nvolved n sold state memory research.