CHALLENGES IN NEW MEDIA NETWOKING APPLICATIONS John Moulder, Göran Appelquist Digital Vision AB, Stockholm, Sweden ABSTRACT The rapid acceptance of MPEG2 compression and stream transport technology has opened up new avenues for media distribution and management. Paradoxically though, these new digital formats and standards have introduced pitfalls and limitations that in many ways mirror some of the weaknesses of the analogue PAL, SECAM and NTSC systems that took years to master. Simple economics dictate that decompressing signals for processing then recompressing them for subsequent transmission is often neither feasible nor practicable. A new genre of equipment that can allow for compressed media to be both comprehensibly and affordably processed in the MPEG system domain with a high level of quality will open the door to digital television for all. INTRODUCTION The challenge on offer to broadcast manufacturers today is that of producing a new generation of equipment that can overcome the gamut of MPEG system idiosyncrasies whilst offering new, affordable and efficient applications that facilitate optimal commercial exploitation of this powerful delivery system. Some examples of new MPEG technology applications include advertisement inserters, stream splicers, rich data inserters, bitrate reducers, re-coders, format converters, logo generators and metadata modifiers. This paper summarises some of today s current challenges and describes emergent product applications which could complete the digital jig-saw and provide for a continuous MPEG system path leading right from station input all the way to the consumer. LEGACY: OLD HEADACHES, NEW PROBLEMS Over the past fifty years new developments in television have often later been found to cause problems for emergent applications. Legacy issues are often encountered; for example interlace, television s first analogue domain compression system, is still used for the majority of MPEG digital video formats. There are two established high level control and modulation standards and a proposed third, DVB and ATSC with COFDM and 8VSB with their complexity and argued relative application merits often mirror the two main analogue colour modulation standards of PAL and NTSC. The recently proposed Japanese digital modulation scheme has some unique features reminiscent of SECAM s idiosyncrasies. MPEG digital formats include field frequencies of 60 and 59.94 Hz reflecting the 1/1000 th drop in frequency introduced with NTSC to prevent video subcarrier interference with the sound subcarrier by interleaving their spectral lines! Even the 24-frame digital film format has a 23.98 Hz variant to allow for 3/2 pull-down film conversion to 59.94 interlaced applications.
SIMPLE IDEAS, COMPLEX APPLICATIONS MPEG2 compression, by exploiting psycho-visual elements of temporal and spatial redundancy can reduce a video signal to less than 3% of its original bandwidth, making it easily and affordably transmissible. However the compression methods employed can introduce significant obstacles to even some of the simplest concepts for coded signal manipulation. Direct processing of MPEG coded data can be viable if it offers at least some of the following advantages compared to the established methods for decoding, processing then re-encoding: Lower cost Less signal degradation Lower number of components and smaller size Greater reliability Advertisement Inserters For an affiliate or local television station, an every day requirement is to insert adverts into gaps within incoming programmes, sometimes also substituting different material in place of certain pieces of incoming content. To achieve this it is necessary to switch at selected intervals between two MPEG sources (incoming and local) this technique is known as seamless splicing. Figure1 below shows what could happen to a VBV buffer if you just switched between two video elementary streams without performing a seamless splice, the result is an overflow. Splice Point VBV Level Stream 1 VBV Level Stream 2 Effective Unconstrained VBV Buffer Overflow ES 1 ES 2 Buffer Overflow!! Figure 1 - Example of VBV buffer overflow with uncontrained splicing
The problem shown in figure 1 derives from MPEG s use of VBV buffer prediction, a nonframe rate related technique designed to steal bandwidth by averaging data rates for transmission between different frame types. I frames, for example, normally take much longer to transmit than P or B frames all of whose overall data content and size varies with respect to the relative complexity of material versus time. elementary data may also be VBR encoded, which adds further complications. Additionally, MPEG transport streams just cannot be easily switched or edited, due in the main to the GOP sequence which comprises sequences of MPEG coded frames starting with a single intra compression coded anchor (I) frame followed by alternating groups of forward (P) and backward/forward (B) predicted compressed frames. It should be noted that GOP sequences are normally not closed, that is a B frame takes a prediction from a future I or P frame. If a stream is cut such that it contains B frames looking to I or P frames beyond the edit point, then severe errors will be generated if new material is inserted after the edit point. Transport streams also contain audio, Teletext/subtitles and many type of metadata, these normally do not time align with video frame boundaries and will all require separate treatment. It should be noted that SMPTE 312M splice points only work if all up stream encoders can generate suitable splice points and splice PID data streams. Thus complex signal processing will often be required to seamlessly splice in locally originated programme material. The attraction of working in MPEG throughout is that almost always, locally inserted material could be affordably MPEG encoded offline. Since local stations often have relatively small operating budgets MPEG Advert Insertion is feasible coupled with offline encoding, providing a modestly priced Splicer unit which can execute in real time the powerful algorithms necessary to overcome the above issues which dwarf PAL & NTSC colour framing and 8 field issues in comparison. Trans-coders and Format Conversion In MPEG terms transcoding describes a process of converting between certain profiles (say 4:2:2 to 4:2:0), bit-rates (say 15 to 5MB/s) and GOP sequences (e.g. IBBP to IBIB or I Frame only), or combinations thereof. These processes can currently all be achieved by a full decode to baseband, followed by an encode to the new format. This is costly procedure, which can involve a significant degradation of signal quality when several of these processes are concatenated, as you might find feeding a signal through a typical distribution chain. MPEG compression involves a number of algorithmic processes, which since the standard is defined in terms of a standardised decoder model can involve many equally valid variations of the encoding process. These include critical encoding decisions such as Motion Vector calculation, and signal quantisation based on relative energy content, with Discrete Cosine Transform (DCT) and Entropy () calculations which are much more mechanistic. Cascaded sets of various different manufacturers encoders with incumbent variations in their processing methodology and configuration, can further add to the potential for concatenation errors. Research done for the Atlantic project has found that where original coding information such as the GOP sequence, including the location of I, P, B frame types, along with original motion vectors, is preserved over the base-band interface between successive decode and re-encode stages, then considerably reduced signal degradation is found, compared to normal uncorrelated concatenations. The most significant measurable degradation in uncorrelated cascades is where, for example, a reconstructed B Frame is then chosen as a subsequent anchor or I frame on a following encode. For those familiar with MPEG encoding this drift is well known fact,
whose relatively obscure origin lies in distortions to the IDCT/DCT process caused by the mixing of MPEG specified Intra coded rounding quantisers (as used for I Frames) with the mandated Inter-coded truncated quantisers (as used for P/B Frames). To date, one of the major outcomes of this work has been the proposal for the use of so called dumb encoders and smart decoders, which enable the original motion vector calculations, GOP sequence and phasing to be passed on through multiple stage cascades. Standards have now been proposed for carrying these encoding decisions through baseband. But for many distribution applications, where as in our previous heading, adverts are inserted, then the consequent discontinuities would break the encoding chain, thus forcing the use of smart encoders to make the required encoding calculations over these breaks. However, for some types of transcoding, a natural economy can first be achieved by only going part way to base band and then back again. This can eliminate several process stages by partial/hybrid algorithms, reducing cost and complexity without compromising accuracy. The major advantage that can then be achieved, is when this partial transcoding process is also co-sited with a function such as seamless splicing. What at first glance seems like a complication actually considerably simplifies the signal processing whilst allowing full MPEG signal integrity and control to be retained. Figure 2 below illustrates such a hybrid transcoder. GOP (In) GOP (Out) Motion Vectors (Out) Motion Vectors (In) Adaptive Motion Compensated Prediction Analyser DCT Weighting & Re-quantize Scan DCT Stream (In) DeMux Previous Frame Macroblocks (In) Current Frame Next Frame Macro blocks (Out) Mux Stream (Out) Decode Scan Quantization matrices (In) Weighting & De-quantize - + Weighting & Re-quantize Quantization Optimisation Scan Code Quantization matrices (Out) Bitrate reducers Figure 2 - Hybrid MPEG2 Transcoder Many operations, particularly cable head ends take their material from multiple origins. Cable, has a large but finite bandwidth and it is often economically necessary to allocate bandwidths according to the relative sales value of the programming content (i.e. premium movies versus niche programming). Today, incoming material may well be MPEG digital off-
air or from a down-link, without doubt some of these programmes will be received with bitrates which are well in excess of the bandwidth slot allocated for their retransmission. Again, this application needs to offer a cost and/or performance advantage over discreet decoding then re-encoding. The previous section on transcoding and format conversion showed that non-correlated MPEG video decoding then re-coding could introduce significant quality degradations. High quality major bitrate reduction requires a full transcoder as illustrated in figure 2, however separating the original motion vectors and DCT data offers the option of a simplified procssing where simple, but more exacting, quantisation decisions are made than were originally. For modest bitrate changes, a reduction in quality is acheivable similar to that which would be expected, were those quantisation decisions made at source, rather than downstream. Cascaded transcoding artefacts are potentially much more significant where you need say a 4.5 to 3.5 MB/s reduction at 4:2:0. Here the incoming material is at a significantly lower bitrate than is usually employed in a more controlled contribution environment, where the incoming material typically has at least two to three times the output bitrate, for example 15 MB/s 4:2:2 going to 5 MB/s 4:2:0, thus making errors potentially much less noticeable. All the transcoding techniques described in this paper need to be employed to help reduce transcoding artefacts when performing bitrate reductions. In particular Intra (I) and Inter (P/B) coded frames need to be identified for separate quantisation treatments. Figure 3 below shows the functionality required for a modest bitrate reduction. Motion Vectors Stream (In) Demux Macro block (In) Decode I P B Quantization matrices(in) Bitrate Reduction Requantiser I P B Code Macro block (Out) Quantization matrices(out) Mux Stream (Out) Inter/Intra Field/Frame DCT metadata Rich Data Content and Metadata handling Figure 3 - Simplified bitrate reducer Today, the broadcast industry is paying increasing attention to the idea of Rich Data Content, with potential applications being evaluated in areas such as value added broadcasting and interactive services. MPEG2 is now being opened for carriage of all kinds of data from programme notes to interactive video clips to selective IP content and so called T Commerce; Rich Data provides a graphic description of the many forms that value added data content can take and its revenue value will soon be measured.
Metadata is data that identifies content, including Rich Data. For example a consumer might want a hard copy of a recipe from a cookery programme, a link to a website which is selling the special implement the chef is using and perhaps may want to access a streamed clip showing how to perform a particular procedure referred to in the programme. Metadata holds the key to all this rich content, some metadata may be streamed at source like teletext for example, while some may need be varied locally for each region or country. Rich data insertion is the natural territory for a transport stream processor, which must also be able to add Metadata and potentally modify any that is already present in incoming transport streams. Generic Transport Stream Processor The broad range of non base band processing functions described in this paper such as multiplexing, splicing, bit rate reduction, format conversion and data handling all have very similar core transport stream handling mechanisms. So perhaps it is not suprising that a basic transport stream processor hardware platform can be constructed which then allows individual applications to be developed and mapped as a firmware and software package into the hardware. Commonality of components and the massive economies of scale that can be achieved when there is a single processor platform, offer the prospect of compact, high performance and keenly priced products. Also, even more superior performance can be achieved by combining several functions. For example, a superior advert splicer/inserter may employ transcoding algorithms for bit rate reduction, as well as metadata modification for the local area, local station schedule trimming and finally logo insertion; with all these functions embedded in a single MPEG in to MPEG out unit. The block drawing in figure 4 below shows the key core hardware elements required for a generic transport stream processor engine. Ext Ref Local Clock TS 1 (IN) PCR 1 Extract PCR 1 Correct 'A' I/P Buffer 1 Process Algorithm Engine TS n (IN) PCR n Extract PCR n Correct 'A' I/P Buffer n Process Algorithm Engine n MUX PCR Correct 'B' O/P Buffer TS(OUT) Metadata Process & modify Figure 4 - Generic Transport Stream Processor
CONCLUSIONS This paper has described a generic transport stream processor that goes some way towards overcoming some of the problems introduced by MPEG compression systems. However such a device has to offer at least a lower cost and less signal degradation than using conventional decode signal process encode chain. A multi-purpose transport stream processor, used as a common platform for executing a variety of co-sited MPEG processing functions could be a design that through its economies of scale will offer a viable alternative to decode-then-encode processing. Such an algorithmic engine may well help navigate a common route through the plethora of obstacles we can see are generated as a side effect of MPEG s power. REFERENCES 1. O Grady, W., Balakrishnan, M., Radha, H., Real-time switching of MPEG-2 bitstreams, Proceedings of 1997 International Broadcasting Convention, pp. 166 to 170. 2. McGrath, E., Digital insertion of advertising into a digital stream (DID), Proceedings of 1997 International Broadcasting Convention, pp. 258 to 261. 3. Birch, C.H., MPEG splicing and bandwidth management, Proceedings of 1997 International Broadcasting Convention, pp. 541 to 545. 4. SMPTE 312M Splice Points for MPEG-2 Transport Streams. 5. Wilkinson J.H, Mihara K, MPEG Trancoding: Techniques and Applications. Proceeding of 1998 International Broadcasting Convention. 6. Tudor P.N, Werner O.H, Real Time Transcoding of MPEG2 Bit Streams. Proceedings of 1997 International Broadcasting Convention, pp. 286-301. 7. Generic Coding of Moving Pictures and associated audio: Systems. ISO/IEC 13818-1. 8. Generic Coding of Moving Pictures and associated audio:. ISO/IEC 13818-2. 9. Rowe R, Streaming Metadata, Applications and Challenges. Proceeding of 2001 35 th SMPTE Advanced Motion Imaging Conference.