Tvheadend - Bug #2171 MP2 radio stations hickups

Tvheadend - Bug #2171 MP2 radio stations hickups 2014-07-07 17:14 - J H Status: Fixed Start date: 2014-07-07 Priority: Normal Due date: Assignee: % Done: 100% Category: Estimated time: 0.00 hour Target version: Found in version: 3.9.959~g6fab551 Affected Versions: Description I've noticed hickups in the audio of certain radio stations when using the git version of Tvheadend. The stable version plays fine. It's only the Music Choice channels, audio stream is MP2 and my provider is Ziggo in The Netherlands. I'm using Sundtek tuners and I've already did some testing with the support engineer of Sundtek to rule out the tuners. Tvheadend log stays empty during playback, same for the playing software like XBMC, VLC etc. OS: Ubuntu 14.04 LTS Tvheadend v. 3.9.959~g6fab551 I've added a recording of a Music Choice channel. You can hear 4 hickups. Associated revisions Revision 67938bc9-2014-12-08 20:20 - Jaroslav Kysela descrambler: implement proper TS queue flush before key change, fixes #2171 History #1-2014-07-28 13:21 - J H - File Stutter.ts added Added another maybe better example. #2-2014-09-19 14:51 - J H Now on build 3.9.1539~g286509b and still existing. Also noticed stuttering on an unencrypted channel. Could someone please tell me if there's any way to log this in tvheadend? Normal debugging log shows no errors during playback. #3-2014-09-19 15:32 - Jaroslav Kysela Could you provide the whole mux dump? Use the 'play' link in the mux tab to gather the whole infomation. But only if the contents is not crypted.. #4-2014-09-19 16:01 - J H I'm sorry but I don't have a FTA channel that's affected. I was wrong. 2018-05-13 1/9

It seems only some encrypted radio stations are wrong. I can provide you with a mux dump of one of those muxes if that is of any help? #5-2014-09-21 15:38 - J H Is there any other information I can give to solve this issue? #6-2014-09-21 19:33 - Jaroslav Kysela Could you grab the output stream from tvheadend using wireshark or tcpdump? Only the TCP audio stream.. It looks like a timing issue.. #7-2014-09-22 12:59 - J H - File stream-wireshark2.pcapng added Added wireshark file. Not sure if this is the format you're looking for. If you need something else please say so. #8-2014-10-15 11:39 - J H Did you spot a cause for this issue in the wireshark file? #9-2014-12-04 12:04 - J H Ziggo added a unencrypted Christmas Music Choice channel. Guess what... no stutter in playback. My best guess is there's something wrong with radio stations and encryption. Could someone please take a look at it again?version running now is 3.9.2165~g3579d04 #10-2014-12-04 12:05 - J H Didn't mean unencrypted but unencrypted #11-2014-12-04 13:40 - Jaroslav Kysela Just some notes: Trying to analyze Stutter.ts - it looks like a raw (passthrough) TS file from TVH. There are missing frames in the audio stream: A python script to analyze frame pts diffs: import sys fp = sys.stdin prev = None for l in fp.readlines(): if l.startswith('pkt_pts_time='): next = float(l[13:-1]) if prev: print "%.6f (diff %.6f)" % (next, next - prev) 2018-05-13 2/9

prev = next Cmd line: ffprobe -show_frames ~/Download/Stutter.ts python a.py Result: 90132.396111 (diff 0.026000) 90132.422111 (diff 0.026000) 90132.605111 (diff 0.183000) # gap - 1 + 6 missing frames 90132.631111 (diff 0.026000) 90132.683111 (diff 0.052000) # gap - 1 + 1 missing frame 90132.710111 (diff 0.027000) 90132.736111 (diff 0.026000) 90157.369111 (diff 0.026000) 90157.395111 (diff 0.026000) 90157.421111 (diff 0.026000) 90157.604111 (diff 0.183000) # gap - 1 + 6 missing frames 90157.630111 (diff 0.026000) 90157.683111 (diff 0.053000) # gap - 1 + 1 missing frame 90157.709111 (diff 0.026000) 90157.735111 (diff 0.026000) 90182.368111 (diff 0.026000) 90182.394111 (diff 0.026000) 90182.421111 (diff 0.027000) 90182.603111 (diff 0.182000) # gap - 1 + 6 missing frames 90182.630111 (diff 0.027000) 90182.682111 (diff 0.052000) # gap - 1 + 1 missing frame 90182.708111 (diff 0.026000) 90182.734111 (diff 0.026000) Note that the time difference between gaps is 25 seconds each (descrambling key change?). #12-2014-12-04 13:41 - Jaroslav Kysela Could you provide --trace descrambler,capmt,cwc? #13-2014-12-04 18:24 - J H - File tvheadend-debug.log added Here you go. What I noticed is that the stutter in audio lines up perfectly with the ECM message in the log. Hope this puts you on the right track! 2018-05-13 3/9

#14-2014-12-04 20:48 - Jaroslav Kysela Could you do same test with this one-line addition? diff --git a/src/descrambler/descrambler.c b/src/descrambler/descrambler.c index d42683e..94f7b3b 100755 --- a/src/descrambler/descrambler.c +++ b/src/descrambler/descrambler.c @@ -370,6 +370,7 @@ descrambler_descramble ( service_t *t, sbuf_free(&dr->dr_buf); } ki = tsb[3]; + tvhtrace("descrambler", "ki = %02x", ki); if ((ki & 0x80)!= 0x00) { if (key_valid(dr, ki) == 0) { if (tvhlog_limit(&dr->dr_loglimit_key, 10)) #15-2014-12-04 21:31 - J H - File tvheadend-debug.log added Same problem, log added #16-2014-12-05 11:10 - Jaroslav Kysela Which CSA do you use? $ grep CSA.config.mk CONFIG_TVHCSA = yes CONFIG_DVBCSA = no #17-2014-12-05 12:39 - J H xbmc@mediacenter:~/tvheadend$ grep CSA.config.mk CONFIG_TVHCSA = yes CONFIG_DVBCSA = no #18-2014-12-05 16:05 - Jaroslav Kysela Another debug code: diff --git a/src/descrambler/descrambler.c b/src/descrambler/descrambler.c index d42683e..d838b1d 100755 --- a/src/descrambler/descrambler.c +++ b/src/descrambler/descrambler.c @@ -326,6 +326,9 @@ descrambler_descramble ( service_t *t, 2018-05-13 4/9

lock_assert(&t->s_stream_mutex); + static int didx = 0; + tvhtrace("descrambler", "descramble ts packet %i (PID %i)", didx++, st->es_pid); + if (dr == NULL) return -1; count = failed = 0; diff --git a/src/input/mpegts/tsdemux.c b/src/input/mpegts/tsdemux.c index 2b4f7e0..d3016e3 100644 --- a/src/input/mpegts/tsdemux.c +++ b/src/input/mpegts/tsdemux.c @@ -256,6 +256,10 @@ ts_recv_packet2(mpegts_service_t *t, const uint8_t *tsb) elementary_stream_t *st; int pid = (tsb[1] & 0x1f) << 8 tsb[2]; + static int didx = 0; + tvhtrace("descrambler", "output ts packet %i PID %i", didx++, pid); + tvhlog_hexdump("descrambler", tsb, 188); + if((st = service_stream_find((service_t*)t, pid))!= NULL) ts_recv_packet0(t, st, tsb); } #19-2014-12-05 16:55 - J H - File tvheadend-debug.log added Log added, for example audio stutter at: - 16:47:34-16:47:54-16:48:14 And so on with 20 seconds interval #20-2014-12-05 18:21 - Jaroslav Kysela OK. No descrambled TS packets are lost. So, it appears that the packets are missing before they enter to the descrambler. Could you try simultaneous streaming the whole mux while the radio service is used, if it changes something? (Something like 'wget -O /dev/null <mux_play_url_from_the_mux_grid>'.) #21-2014-12-05 19:09 - J H Just tried, no changes. 2018-05-13 5/9

#22-2014-12-05 20:25 - Jaroslav Kysela I need to implement a way to insert the descrambling keys to the whole mux stream and also implement the re-play functionality, so I can do testing directly on my machine. It may take some time. #23-2014-12-05 22:41 - J H Ok, thanks very much! Please say when you need more info from me. #24-2014-12-07 21:25 - Jaroslav Kysela OK. Grab latest sources. Use --enable-tsdebug as the configure option. Then run tvheadend with "--tsdebug <directory>" arguments. Then run the test and you should see in directory two files. I need them for analyze what's going wrong. One file is raw input from the linuxdvb subsystem and the second file is almost same (tvh does some processing - no contents change) but with added descrambler keys. #25-2014-12-08 09:36 - J H - File 658MHz in Ziggo-1417994463-0x7f6d1c009bc0-input.ts added - File 658MHz in Ziggo-1417994463-0x7f6d1c009bc0-mux.ts added Resulted files are added. #26-2014-12-08 16:18 - Jaroslav Kysela Tried to re-play and guess what - no packets are lost, but I had to fix a little issue with the PCR extraction - v3.9-2214-g8668734 for tsfile. I don't think that it's related for the live playback, but could you test the recent code? #27-2014-12-08 16:33 - J H - File 658MHz in Ziggo-1418052479-0x16e5200-input.ts added - File 658MHz in Ziggo-1418052479-0x16e5200-mux.ts added Issue not resolved as you already suspected, added the.ts files #28-2014-12-08 16:54 - Jaroslav Kysela Could you try this? diff --git a/src/descrambler/descrambler.c b/src/descrambler/descrambler.c index a15aa64..0706c9b 100755 --- a/src/descrambler/descrambler.c +++ b/src/descrambler/descrambler.c @@ -179,6 +179,8 @@ descrambler_keys ( th_descrambler_t *td, int type, if (tvhcsa_set_type(&dr->dr_csa, type) < 0) return; 2018-05-13 6/9

+ usleep(500000); + pthread_mutex_lock(&t->s_stream_mutex); LIST_FOREACH(td2, &t->s_descramblers, td_service_link) BTW: Also the second re-play was successful (no gaps in the output stream). #29-2014-12-08 17:04 - J H TVH crashes with that code added. Do you need a crashlog and where can I find it? #30-2014-12-08 17:26 - Jaroslav Kysela diff --git a/src/descrambler/cwc.c b/src/descrambler/cwc.c index bac4aa4..a36c8c2 100755 --- a/src/descrambler/cwc.c +++ b/src/descrambler/cwc.c @@ -809,7 +809,9 @@ forbid: es->es_keystate = ES_RESOLVED; es->es_resolved = 1; + pthread_mutex_unlock(&ct->cs_cwc->cwc_mutex); descrambler_keys((th_descrambler_t *)ct, DESCRAMBLER_DES, msg + 3, msg + 3 + 8); + pthread_mutex_lock(&ct->cs_cwc->cwc_mutex); } else { tvhlog(log_debug, "cwc", "Received ECM reply%s for service \"%s\" " diff --git a/src/descrambler/descrambler.c b/src/descrambler/descrambler.c index a15aa64..0706c9b 100755 --- a/src/descrambler/descrambler.c +++ b/src/descrambler/descrambler.c @@ -179,6 +179,8 @@ descrambler_keys ( th_descrambler_t *td, int type, if (tvhcsa_set_type(&dr->dr_csa, type) < 0) return; + usleep(500000); + pthread_mutex_lock(&t->s_stream_mutex); LIST_FOREACH(td2, &t->s_descramblers, td_service_link) #31-2014-12-08 18:21 - J H I'm very happy to tell you this seemed to fix the issue. Tested multiple radio channels en playback seems fine again. Thanks very much for your time and effort! 2018-05-13 7/9

#32-2014-12-08 18:58 - Jaroslav Kysela It's not a fix, but just a debug code. It looks like a timing issue (wrong key is used for decoding some frames for a short period). The patch just delays the new key processing by 500 ms. #33-2014-12-08 19:24 - J H Ok I understand. If there's something to test please say so. #34-2014-12-08 19:30 - Jaroslav Kysela I found the real issue. Basically, the key was changed too quickly and tvh buffers some TS packets for the DES decoder. These buffered TS packets must be flushed (processed) before key changes, otherwise they are decoded with the new key which is invalid for them (thus some pieces of the audio stream weren't parsed properly). I'm working on a fix. I only wonder, why we haven't noted before, because this bug is really serious. #35-2014-12-08 19:35 - J H I made a forum post here asking if anybody else was having this issue, no replies. I also asked on tweakers.net in the tvheadend topic if anyone else noticed... no one. Guess I'm the only one left here listening to radio. #36-2014-12-08 20:20 - Jaroslav Kysela - Status changed from New to Fixed - % Done changed from 0 to 100 Applied in changeset commit:tvheadend 67938bc9d9774dff7b99e4e134d5c38bde55cce9. #37-2014-12-08 20:21 - Jaroslav Kysela The above commit contains the proper fix for your issue. Could you test? #38-2014-12-08 20:37 - J H Tested and confirmed to fix the issue. Thanks again! #39-2014-12-08 20:59 - Jaroslav Kysela J H wrote: I made a forum post here asking if anybody else was having this issue, no replies. I also asked on tweakers.net in the tvheadend topic if anyone else noticed... no one. Guess I'm the only one left here listening to radio. I think that the problem was that most of radio stations on SAT are not encrypted (at least it's true for my country and I think also for most of European countries). 2018-05-13 8/9

#40-2014-12-08 21:43 - Jaroslav Kysela - File deleted (tvheadend-debug.log) #41-2014-12-08 21:43 - Jaroslav Kysela - File deleted (658MHz in Ziggo-1417994463-0x7f6d1c009bc0-mux.ts) #42-2014-12-08 21:43 - Jaroslav Kysela - File deleted (stream-wireshark2.pcapng) #43-2014-12-08 21:43 - Jaroslav Kysela - File deleted (Stutter.ts) #44-2014-12-08 21:43 - Jaroslav Kysela - File deleted (recording.ts) #45-2014-12-08 21:43 - Jaroslav Kysela - File deleted (658MHz in Ziggo-1417994463-0x7f6d1c009bc0-input.ts) #46-2014-12-08 21:43 - Jaroslav Kysela - File deleted (658MHz in Ziggo-1418052479-0x16e5200-mux.ts) #47-2014-12-08 21:43 - Jaroslav Kysela - File deleted (658MHz in Ziggo-1418052479-0x16e5200-input.ts) Files tvheadend-debug.log 22 KB 2014-12-04 J H tvheadend-debug.log 1.04 MB 2014-12-04 J H 2018-05-13 9/9