With the same scanefile-data kaffeine 1.3.1 finds in my setup 500 channels with a scan but with versions 2.0.1 to 2.0.3 only 120 channels. Running the scan on a different card doesn't help. The scanfile-data consists of only one transponder. Reproducible: Always Steps to Reproduce: 1. Start kaffeine 2. At the menu bar go to Television -> Channels or press C 3. Press "Start Scan"-Button Actual Results: Finds only 120 Channels Expected Results: To find 500 Channels
Could you provide some additional data? What's the transponder you're trying to tune? Please also post the debug logs with both 1.3.1 and 2.0.x. So far, I suspect that this could be a timeout issue, but it is hard to know without having more data and doing some tests. There is a per-device parameter that controls the tuner timeout. You could try to increase it, and see if the number of discovered channel changes. That's said, this parameter is not propagated to libdvbv5. It means that, if the issue is related to the needed time to lock into a frequency, just changing it won't fix, but if the issue is related to the time Kaffeine takes to get the needed MPEG-TS data, it may solve your issue. I'll take a better look on how to better pass the tuning timeout to libdvbv5.
(In reply to Mauro Carvalho Chehab from comment #1) > Could you provide some additional data? What's the transponder you're trying > to tune? Please also post the debug logs with both 1.3.1 and 2.0.x. > > So far, I suspect that this could be a timeout issue, but it is hard to know > without having more data and doing some tests. > > There is a per-device parameter that controls the tuner timeout. You could > try to increase it, and see if the number of discovered channel changes. > > That's said, this parameter is not propagated to libdvbv5. It means that, if > the issue is related to the needed time to lock into a frequency, just > changing it won't fix, but if the issue is related to the time Kaffeine > takes to get the needed MPEG-TS data, it may solve your issue. Actually, after double-checking the code, the only timeout passed through the libdvbv5 is the scan transponder timeout, with Kaffeine currently doesn't use. So, changing the timeout should work fine. Please try increasing it for us to see what's going wrong. Are you having problems with DVB-C, DVB-T/T2 or DVB-S/S2?
It's DVB-C. Scanfile entry: C 410000000 6900000 NONE QAM64 The following tests were done with kaffeine 2.0.3 and kaffeine was restarted every time after resetting the tuner timeout values. Changing the tuner timeout from 1500ms to the maximum of 5000ms didn't yield any different results, even lowering the tuner timeout to 500ms didn't make any difference: 500ms: 126 Channels 1500ms: 126 Channels 3000ms: 126 Channels 5000ms: 126 Channels I attached the logs during the scan and a dvb dump.
Created attachment 99509 [details] Kaffeine 1.3.1 Channels scan 1500ms log
Created attachment 99510 [details] Kaffeine 2.0.3 Channels scan 1500ms log
Created attachment 99511 [details] Kaffeine 2.0.3 Channels scan 5000ms log
Created attachment 99512 [details] Kaffeine 2.0.3 Channels scan 1500ms dumpdvb log
Created attachment 99513 [details] Kaffeine 2.0.3 Channels scan DvbDump
Ok, it is not a timeout issue. I need to check if Kaffeine is getting all transponders. I found a bug there, that might be causing some troubles, but I suspect that the issue is not solved yet. Could you please compile Kaffeine from the git tree, at either one of the sites below: https://git.linuxtv.org/mchehab/kaffeine.git/log/ https://quickgit.kde.org/?p=kaffeine.git and run Kaffeine with the new --debug option? I'd like the logs when running with it. It should show the transponders it found, plus the scan info. It also helps if you could record 30 seconds of the content of the initial transponder (the one at the scanflie.dvb) using -P option. Something like: dvbv5-zap -r -P -t 30 -c /devel/v4l/dtv-scan-tables/dvb-c/de-Berlin 394000000 With that, I can use the file to simulate using a DVB-C signal generator. Tthe above assumes that you have a copy of the dtv-scan-tables stored on your home directory, and you're using the de-Berlin file. You can download the more updated scantable from: https://linuxtv.org/downloads/dtv-scan-tables/dtv-scan-tables-LATEST.tar.bz2 PS.: It is likely that the file will be bigger than the maximum size that bugs.kde.org accepts. So, the best is if you could place it somewhere and send me a download link directly via e-mail.
I added the requested log file from a version built from commit d027d52. As requested I will send you an E-Mail regarding the file containing the complete transport stream. This isn't related to the current problem, but with the current git version I lost a lot of icons in kaffeine. I saw that you changed the icon-generation but couldn't see if I have to do something more than cmake, make, make install. Are additional steps in the building process necessary?
Created attachment 99523 [details] kaffeine 2.0.3.d027d52 debug dumpdvb channels scan log
(In reply to lev.cohan from comment #10) > I added the requested log file from a version built from commit d027d52. > > As requested I will send you an E-Mail regarding the file containing the > complete transport stream. Received, thanks! I'm doing some tests with the stream. > This isn't related to the current problem, but with the current git version > I lost a lot of icons in kaffeine. I saw that you changed the > icon-generation but couldn't see if I have to do something more than cmake, > make, make install. Are additional steps in the building process necessary? It should now honor the icons that are set on KDE, falling back to the breeze icons if the current theme doesn't have it.
(In reply to lev.cohan from comment #11) > Created attachment 99523 [details] > kaffeine 2.0.3.d027d52 debug dumpdvb channels scan log Ok, from the logs, it got 132 channels via the SDT table. Could you please run dvbv5-scan and post its results? I'm wandering if it will get the channels that are missing. I'm starting to suspect that maybe your old Kaffeine 1.3.1 has some extra patch applying to it, parsing some other MPEG-TS descriptors or tables.
(In reply to Mauro Carvalho Chehab from comment #13) > (In reply to lev.cohan from comment #11) > > Created attachment 99523 [details] > > kaffeine 2.0.3.d027d52 debug dumpdvb channels scan log > > Ok, from the logs, it got 132 channels via the SDT table. Could you please > run dvbv5-scan and post its results? I'm wandering if it will get the > channels that are missing. > > I'm starting to suspect that maybe your old Kaffeine 1.3.1 has some extra > patch applying to it, parsing some other MPEG-TS descriptors or tables. Btw, I found some issues on the SDT table parser that could be the culprit. Just pushed a patch fixing it (plus I also fixed the lack of icons). Please test.
(In reply to Mauro Carvalho Chehab from comment #12) > (In reply to lev.cohan from comment #10) > > I added the requested log file from a version built from commit d027d52. > > > > As requested I will send you an E-Mail regarding the file containing the > > complete transport stream. > > Received, thanks! I'm doing some tests with the stream. > > > This isn't related to the current problem, but with the current git version > > I lost a lot of icons in kaffeine. I saw that you changed the > > icon-generation but couldn't see if I have to do something more than cmake, > > make, make install. Are additional steps in the building process necessary? > > It should now honor the icons that are set on KDE, falling back to the > breeze icons if the current theme doesn't have it. Thanks to 2eac0a1 the icons are back again. (In reply to Mauro Carvalho Chehab from comment #14) > (In reply to Mauro Carvalho Chehab from comment #13) > > (In reply to lev.cohan from comment #11) > > > Created attachment 99523 [details] > > > kaffeine 2.0.3.d027d52 debug dumpdvb channels scan log > > > > Ok, from the logs, it got 132 channels via the SDT table. Could you please > > run dvbv5-scan and post its results? I'm wandering if it will get the > > channels that are missing. > > Not all channels are plain TV or Radio Channels. I added the dvb_channel.conf . > > I'm starting to suspect that maybe your old Kaffeine 1.3.1 has some extra > > patch applying to it, parsing some other MPEG-TS descriptors or tables. > > Btw, I found some issues on the SDT table parser that could be the culprit. > Just pushed a patch fixing it (plus I also fixed the lack of icons). Please > test. Still the same with kaffeine built from commit c40ad97, the corresponding log has been added.
Created attachment 99574 [details] kaffeine 2.0.3.c40ad97 debug dumpdvb channels scan log
Created attachment 99575 [details] dvbv5-scan dvb_channels.conf
(In reply to lev.cohan from comment #15) > (In reply to Mauro Carvalho Chehab from comment #12) > > (In reply to lev.cohan from comment #10) > > It should now honor the icons that are set on KDE, falling back to the > > breeze icons if the current theme doesn't have it. > > Thanks to 2eac0a1 the icons are back again. Good! > > > Ok, from the logs, it got 132 channels via the SDT table. Could you please > > > run dvbv5-scan and post its results? I'm wandering if it will get the > > > channels that are missing. > > Not all channels are plain TV or Radio Channels. I added the > dvb_channel.conf . dvbv5-scan found 128 channels, Kaffeine found 127. The only difference was the channel "#5010", with doesn't have an entry at the SDT table. such "channel" doesn't provide neither video or audio PIDs. It is probably some service channel with software upgrades and service activation data. I'm running out of ideas about why you found other channels with Kaffeine 1.3. Can you provide me a MPEG-TS sample with a missing channel with video or audio?
The MPEG-TS has two NIT tables and two SDT tables. By using dvbv5-scan with the "-p" option, it was able to get 52 transponders instead of 12. However, the second SDT tables contain programs that aren't actually there. Not sure if the channels at the other NIT tables are valid or not. Could you please test scanning it with: $ dvbv5-scan -a0 -f1 ~/dtv-scan-tables/dvb-c/ch-Zuerich-upc-cablecom -p PS.: there's a rationale for "-p" to not be the default: it takes a lot more time to scan. Also, there's a bug with dvb-utils causing it to not work. I just fixed it upstream: https://git.linuxtv.org/v4l-utils.git/log/
Created attachment 99590 [details] Hack to force Kaffeine to get all programs From what I've seen, channels with video and audio were properly added to Kaffeine 2.0, on the transponders it identified from the main NIT table. The enclosed hack would make available also the channels with pure data (with Kaffeine won't do anything, except silently play them). Are those the missing channels?
Created attachment 99591 [details] Hack to force Kaffeine to also use frequencies from other NIT tables This is a second hack, made against the very latest git version. It makes Kaffeine to use the other NIT tables. That makes it to get 83 transponders, but I'm not sure if those transponders are really there. Please check if it makes any difference. Please notice that parsing other NIT tables is something that we don't usually want, as those tables are meant to carry on frequencies for other Satellites and other networks. However, on DVB-C, nothing really prevents some carrier to re-use for its own purpose.
(In reply to Mauro Carvalho Chehab from comment #19) > The MPEG-TS has two NIT tables and two SDT tables. By using dvbv5-scan with > the "-p" option, it was able to get 52 transponders instead of 12. However, > the second SDT tables contain programs that aren't actually there. Not sure > if the channels at the other NIT tables are valid or not. > > Could you please test scanning it with: > $ dvbv5-scan -a0 -f1 ~/dtv-scan-tables/dvb-c/ch-Zuerich-upc-cablecom -p > > PS.: there's a rationale for "-p" to not be the default: it takes a lot more > time to scan. Also, there's a bug with dvb-utils causing it to not work. I > just fixed it upstream: > https://git.linuxtv.org/v4l-utils.git/log/ I did build v4l-utils from the commits 11c6e2c earlier and 9b8e58f now and with neither version I manage to make a complete scan with "dvbv5-scan -p", it always segfaults or stalls. Using the verbose command line parameter it crashes even faster. There are also a lot of error messages of "channels not found on PMT" shown and the channels are not stored with their "names" but in the pattern "FREQUENCY#SERVICE_ID", which is somewhat cumbersome. In comparison to "dvbv5-scan -p" the old "scan -n" from https://linuxtv.org/hg/dvb-apps manages to do this flawlessly, even the different "Network Names" (probably the different Network-IDs) are identified. I attached the resulting file "dvbv3_channels.conf" with the 533 services found. By the way, with my TV set I have to enter the Network-ID (here 43016) for the channels scan. Could this enhance or speed up the scanning process if the user could define that and it would be incorporated in the scanning process?
Created attachment 99615 [details] dvb-apps scan dvbv3_channels.conf
(In reply to lev.cohan from comment #22) > I did build v4l-utils from the commits 11c6e2c earlier and 9b8e58f now and > with neither version I manage to make a complete scan with "dvbv5-scan -p", > it always segfaults or stalls. Using the verbose command line parameter it > crashes even faster. Ok. It would be good later send me what's happening there, for me to fix the bug at the tool as well. I'm planning switch the Kaffeine scan algorithm to use the one at v4l-utils in some future, in order to be able to maintain just one code. So, it will benefit Kaffeine if we can make it more reliable. > There are also a lot of error messages of "channels not > found on PMT" shown and the channels are not stored with their "names" but > in the pattern "FREQUENCY#SERVICE_ID", which is somewhat cumbersome. That's because it will use the "other SDT" tables too. For your cable TV provider, it seems that it needs to use NIT + other NIT + SDT tables and ignore other SDT tables. I need to change the logic at libdvbv5 to append new SDT entries instead of replace the tables. > In comparison to "dvbv5-scan -p" the old "scan -n" from > https://linuxtv.org/hg/dvb-apps > manages to do this flawlessly, even the different "Network Names" > (probably the different Network-IDs) are identified. I attached the > resulting file "dvbv3_channels.conf" with the 533 services found. Ok. Yeah, those other Network IDs come from the other NIT tables. The patch at https://bugs.kde.org/attachment.cgi?id=99591 should force Kaffeine to handle such tables. Please test and report if it is able to find all channels. > By the way, with my TV set I have to enter the Network-ID (here 43016) for > the channels scan. Could this enhance or speed up the scanning process if > the user could define that and it would be incorporated in the scanning > process? Yes. The normal NIT table has ID = 0x40 and it is just one table, and, on your case, it lists 12 transponders. The "other NIT" is actually a series of different extensions, all identified with Table ID 0x41, bu with different extension IDs. Each extension provides a set of transponder frequencies. The total amount of transponders there is 83 transponders. If it gets just one of those tables, the number of transponders will be less than 83, speeding up the tuning. So, for a final patch, it is likely interesting to add some options to avoid the user the need of scanning transponders that will never be available for him, e. g. allowing them to completely disable other NIT or to filter just one (or a few?) other NITs. Doing an easy to use GUI interface for it could be tricky, though.
(In reply to Mauro Carvalho Chehab from comment #24) > (In reply to lev.cohan from comment #22) > > > I did build v4l-utils from the commits 11c6e2c earlier and 9b8e58f now and > > with neither version I manage to make a complete scan with "dvbv5-scan -p", > > it always segfaults or stalls. Using the verbose command line parameter it > > crashes even faster. > > Ok. It would be good later send me what's happening there, for me to fix the > bug at the tool as well. I'm planning switch the Kaffeine scan algorithm to > use the one at v4l-utils in some future, in order to be able to maintain > just one code. So, it will benefit Kaffeine if we can make it more reliable. How should I report it? Via E-Mail? > > There are also a lot of error messages of "channels not > > found on PMT" shown and the channels are not stored with their "names" but > > in the pattern "FREQUENCY#SERVICE_ID", which is somewhat cumbersome. > > That's because it will use the "other SDT" tables too. For your cable TV > provider, it seems that it needs to use NIT + other NIT + SDT tables and > ignore other SDT tables. I need to change the logic at libdvbv5 to append > new SDT entries instead of replace the tables. > > > In comparison to "dvbv5-scan -p" the old "scan -n" from > > https://linuxtv.org/hg/dvb-apps > > manages to do this flawlessly, even the different "Network Names" > > (probably the different Network-IDs) are identified. I attached the > > resulting file "dvbv3_channels.conf" with the 533 services found. > > Ok. Yeah, those other Network IDs come from the other NIT tables. The patch > at https://bugs.kde.org/attachment.cgi?id=99591 should force Kaffeine to > handle such tables. Please test and report if it is able to find all > channels. Kaffeine built from commit f03047d and patched finds 514 channels. That's pretty good. Do you have any script to export the channels form the sqlite.db? I attached the gzipped log file for the scan in which you can see that for every transponder it adds all 83 transponders multiple times. > > By the way, with my TV set I have to enter the Network-ID (here 43016) for > > the channels scan. Could this enhance or speed up the scanning process if > > the user could define that and it would be incorporated in the scanning > > process? > > Yes. The normal NIT table has ID = 0x40 and it is just one table, and, on > your case, it lists 12 transponders. > The "other NIT" is actually a series of different extensions, all identified > with Table ID 0x41, bu with different extension IDs. Each extension provides > a set of transponder frequencies. The total amount of transponders there is > 83 transponders. If it gets just one of those tables, the number of > transponders will be less than 83, speeding up the tuning. > So, for a final patch, it is likely interesting to add some options to avoid > the user the need of scanning transponders that will never be available for > him, e. g. allowing them to completely disable other NIT or to filter just > one (or a few?) other NITs. Doing an easy to use GUI interface for it could > be tricky, though. How about adding a network-id entry to the DVBv5 file format for the dtv-scan-tables? Adding a text-field to the device tab would also be a possibility.
Created attachment 99633 [details] kaffeine 2.0.3.f03047d patched debug dumdvb channels scan gzipped log
(In reply to lev.cohan from comment #25) > (In reply to Mauro Carvalho Chehab from comment #24) > > Ok. It would be good later send me what's happening there, for me to fix the bug at the tool as well. > How should I report it? Via E-Mail? Yes. Send an e-mail to linux-media@vger.kernel.org. > Kaffeine built from commit f03047d and patched finds 514 channels. > That's pretty good. Good! From the logs, the other NIT tables pointed to 83 transponders. 26 ones failed to tune. I suspect that such transponders are available on other cities/states/countries where this cable operator provides services. > Do you have any script to export the channels form the sqlite.db? No, but it shouldn't be hard to get the channels from there using the command line. It should be almost identical to what dvbscan found. I suspect that the other channels got by dvbscan are the data only channels. > I attached the gzipped log file for the scan in which you can see that for > every transponder it adds all 83 transponders multiple times. Every time it gets a new transponder, it will parse the NIT tables provided there, as NIT table(s) may be different. Btw, where I live, different channels provide different NIT tables. > > So, for a final patch, it is likely interesting to add some options to avoid > > the user the need of scanning transponders that will never be available for > > him, e. g. allowing them to completely disable other NIT or to filter just > > one (or a few?) other NITs. Doing an easy to use GUI interface for it could > > be tricky, though. > > How about adding a network-id entry to the DVBv5 file format for the dtv-scan-tables? It could, but I don't see any advantage. It will still need to parse the other NIT tables, as other places using the same scan file could be using different transponders. > Adding a text-field to the device tab would also be a possibility. The problem is that the screen is already too crowd on devices that support multiple delivery systems, like this one: https://mchehab.fedorapeople.org/kaffeine_screenshots/tmp/kaffeine-terrestrial-cable.png The problem is that the usage of other NIT tables should happen per delivery system type. So, a frontend that support 3 different delivery systems (like the one shown on the above image) would need 3 extra lines for a button to enable "all other network tables" or 6 lines if we also add text input field for the network ID(s) that will be used. To make it even worse,, the Kernel currently maps as two frontends single devices that support both Satellite and Cable (or Terrestrial) delivery systems. We'll eventually fix this bug. So, a single device could support up to 8 different delivery systems (right now, there are devices with 6 different delivery systems on the same frontend). As some people use Kaffeine with 21:9 monitors, adding those extra lines could be a problem. So, I'm actually thinking that I to add an extra button (and eventually a combo box) at the channel scan window that would make it parse the other NIT tables. That would avoid polluting the channel configuration tabs.
Git commit 408a151d61a546a03923b0f7a67339d762cbbf18 by Mauro Carvalho Chehab. Committed on 21/06/2016 at 00:35. Pushed by mauroc into branch 'master'. dvbscan: add an option to handle other NIT tables Specially for DVB-C, Kaffeine should allow the user to also seek for new transponders at the other NIT tables, as it may have other transponders that are not shown at the main NIT table. Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com> M +18 -13 src/dvb/dvbscan.cpp M +4 -3 src/dvb/dvbscan.h M +7 -3 src/dvb/dvbscandialog.cpp M +1 -0 src/dvb/dvbscandialog.h http://commits.kde.org/kaffeine/408a151d61a546a03923b0f7a67339d762cbbf18
(In reply to Mauro Carvalho Chehab from comment #27) > (In reply to lev.cohan from comment #25) > > How about adding a network-id entry to the DVBv5 file format for the dtv-scan-tables? > > It could, but I don't see any advantage. It will still need to parse the > other NIT tables, as other places using the same scan file could be using > different transponders. After sleeping on this, it could indeed be helpful to have a network id entry at dtv-scan-tables. The most complex part would be to add support for this new field at the scanning tools, to allow them to auto-detect the need of a network ID and to identify what's the right ID to be stored. The tuner algorithm would need to be different, and the file output logic could only happen after parsing the full file.