A background parser should, of course, be parsing in the background. But there would appear to be some stupid ugly locks that is making the Open dialogue box a $^@#^ nightmare. The delays of the dialogue updating can be in excess of 20 seconds. This is at a time when I/O and CPU load are not that bad (CPU load is all kdvelop) and all of the dirents (and probably inodes) of the entire project tree should be in memory (cached in the kernel) since I'm actively doing git operations on the tree. This is working with gcc as the project, which contains a number of absurdly sized files that take quite a long time to parse -- this is what's causing me to suspect some locking issue. I could be wrong, but the performance is completely unacceptable. If click "stop all" then performance is again fine. In one instance, I was able to at least queue up several UI events that it did later catch -- ctrl-clicking several files that I wanted to open and dragging the thumbnail up and down... there was NO change on the gui for a very long time, but when it did update it was correct.
Are you, by any chance, out of memory? Lowering the amount of synchroneous parsing threads in the settings dialog might also help. For a project as huge as gcc, it might also make sense to exclude some parts from the project you are not interested in at the moment.
(In reply to Sven Brauch from comment #1) > Are you, by any chance, out of memory? Sorry for my slow response. As I said, there was very little I/O at the time and iirc, I had a few hundred MiB in swap that had been there for days (probably from garbage sitting in /tmp). I was able to look over at my computer and see that the HDD light was not coming on. This is what indicated to me that this was some type of locking issue. I do not wish to lower the number of parsing threads below the number of CPU cores that I have and I refuse to believe that the kernel could *possibly* do such a horrible job as scheduling that a UI thread would be completely starved for an entire 20 seconds. I have already excluded areas of the GCC project that I'm not working on (in addition to testsuites as they will crash clang). That being said, parsing a file with 25k lines like gcc/config/i386/i386.c takes quite a long time. Of course, that shouldn't be a reason for the UI to not respond for that duration. I'll try to reproduce this later this week and see if I can discover any other helpful details.
I should probably also make the distinction between the whole UI being a bit sluggish when the project is parsing and the open file dialogue being 100% unresponsive for periods of up to 20 seconds.
Seemingly weird question, is the welcome page (with the KDevelop logo and the seesion list) showing when you experience this problem? I don't have that issue here and there is quite certainly not a lock in place causing this ... the dialog is provided by a completely separate component, and has no shared code with the rest of the IDE ...
(In reply to Sven Brauch from comment #4) > Seemingly weird question, is the welcome page (with the KDevelop logo and > the seesion list) showing when you experience this problem? I guess that's not weird, I know it's been a bit sluggish sometimes for me in the past (still using a Phenom-based machine!), but it isn't open. > I don't have that issue here and there is quite certainly not a lock in > place causing this ... the dialog is provided by a completely separate > component, and has no shared code with the rest of the IDE ... I guess this is one of those wonderful mysteries. So I ran iotop while doing this and trying a few different things out. The I/O hovers between 35 and 50% during parsing, so I wouldn't expect I/O waits for the open dialog to be very long (I have a 3-device software RAID 5 array). Moving the scroll bar or typing in the code editor lags by around 100 to 500-ish milliseconds during parsing with 4 threads. In the open dialog, it is sometimes perfectly responsive, with HID events being processed in less time that I can perceive (this may be towards the beginning of parsing?). Then it will suddenly change to where response times vary between 2 to 20-ish seconds, with an average somewhere around 8-10. Unfortunately I'm unfamiliar with both KDE and QT, so I wouldn't be sure where to start trying to diagnose the underlying cause. If I "stop-all" then the open dialog behaves normally. I'll see if I can learn anything in gdb.
Created attachment 108108 [details] gdb backtrace OK, looking at the debug output with QT_LOGGING_RULES="kdev*=true" gdb --args kdevelop -s <session> I can see that it is completely responsive *prior* to the start of parsing. So I waited until it was unresponsive for a few seconds and then I did an alt-tab and SIGINT. Attached is the full backtrace from gdb.
I'm running Gentoo, so here are the specifics of my system and how kdevelop and kdevplatform were built: $ emerge --info $(qlist -IC /kdev) !!! Repository 'rion' has sync-uri attribute, but is missing sync-type attribute Portage 2.3.8 (python 3.4.6-final-0, default/linux/amd64/13.0/desktop/plasma, gcc-5.4.0, glibc-2.25-r4, 4.9.39-gentoo x86_64) ================================================================= System Settings ================================================================= System uname: Linux-4.9.39-gentoo-x86_64-AMD_Phenom-tm-_9850_Quad-Core_Processor-with-gentoo-2.4.1 KiB Mem: 8175848 total, 141316 free KiB Swap: 16795944 total, 15193188 free Timestamp of repository gentoo: Fri, 22 Sep 2017 03:30:01 +0000 Head commit of repository gentoo: 4df0a1dcb6838efbf99faba8007e8950e28e2844 sh bash 4.4_p12 ld GNU ld (Gentoo 2.28 p1.2) 2.28 distcc 3.2rc1 x86_64-pc-linux-gnu [enabled] ccache version 3.3.4 [enabled] app-shells/bash: 4.4_p12::gentoo dev-java/java-config: 2.2.0-r3::gentoo dev-lang/perl: 5.24.2::gentoo dev-lang/python: 2.7.13::gentoo, 3.4.6::gentoo dev-util/ccache: 3.3.4-r1::gentoo dev-util/cmake: 3.9.1::gentoo dev-util/pkgconfig: 0.29.2::gentoo sys-apps/baselayout: 2.4.1-r2::gentoo sys-apps/openrc: 0.29::gentoo sys-apps/sandbox: 2.10-r4::gentoo sys-devel/autoconf: 2.13::gentoo, 2.64::gentoo, 2.69-r4::gentoo sys-devel/automake: 1.11.6-r2::gentoo, 1.13.4-r1::gentoo, 1.15.1-r1::gentoo sys-devel/binutils: 2.28-r2::gentoo, 2.28.1::gentoo sys-devel/gcc: 4.9.4::gentoo, 5.4.0-r3::gentoo, 6.3.0::gentoo, 6.4.0::gentoo sys-devel/gcc-config: 1.8-r1::gentoo sys-devel/libtool: 2.4.6-r4::gentoo sys-devel/make: 4.2.1-r1::gentoo sys-kernel/linux-headers: 4.11::gentoo (virtual/os-headers) sys-libs/glibc: 2.25-r4::gentoo Repositories: gentoo location: /usr/portage sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 loudmouth-local location: /usr/local/portage masters: gentoo priority: 0 gamerlay location: /var/lib/layman/gamerlay masters: gentoo priority: 50 pentoo location: /var/lib/layman/pentoo masters: gentoo priority: 50 rion location: /var/lib/layman/rion sync-uri: https://github.com/rion-overlay/rion-overlay.git masters: gentoo priority: 50 sunrise location: /var/lib/layman/sunrise masters: gentoo priority: 50 zugaina location: /var/lib/layman/zugaina masters: gentoo priority: 50 ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -ggdb -march=amdfam10 -msahf -mpopcnt -mlzcnt -mtune=amdfam10 --param l1-cache-size=64 --param l1-cache-line-size=64 --param l2-cache-size=512" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/lib64/libreoffice/program/sofficerc /usr/share/config /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-O2 -ggdb -march=amdfam10 -msahf -mpopcnt -mlzcnt -mtune=amdfam10 --param l1-cache-size=64 --param l1-cache-line-size=64 --param l2-cache-size=512" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="-j3 --load-average=8 --with-bdeps=y" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs buildpkg candy ccache cgroup compress-build-logs config-protect-if-modified distcc distlocks ebuild-locks fixlafiles installsources merge-sync multilib-strict news parallel-fetch preserve-libs protect-owned sandbox sfperms split-elog split-log splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://gentoo.mirrors.easynews.com/linux/gentoo/ http://gentoo.cites.uiuc.edu/pub/gentoo/ http://www.gtlib.gatech.edu/pub/gentoo" LANG="en_US.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j18 --load-average=6" PKGDIR="/usr/portage/packages" PORTAGE_COMPRESS="pxz" PORTAGE_COMPRESS_FLAGS="-9e -T4" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/tmp" USE="7zip X a52 aac aacplus aacs acl acpi activities aio alsa amd64 apache2 appindicator autotools bazaar berkdb bindist bluetooth bluray branding bzip2 cairo cdda cddb cdio cdr celt cg clang cli cmake consolekit cracklib crypt cups cvs cxx dbus declarative designer development device-mapper dirac directfb dot dri dts dvd dvdr emboss encode exif extraengine faac fam ffmpeg fftw firefox flac fluidsynth fortran frei0r fuse gcj gdbm geoip gif git glamor glew glut gme gnome-keyring gnutls gphoto2 gpm graphviz gsm gstreamer gtk gtk3 hardened highlight iconv id3tag introspection ipv6 java jpeg jpeg2k kate kde kdrive kipi kwallet ladspa lame lcms ldap lensfun libass libcaca libnotify libsamplerate libv4l lto lua lz4 lzma lzo mad matroska md5sum mercurial midi mmx mmxext mng modules mp3 mp4 mpeg mtp mudflap multicall multilib multitarget musepack natspec ncurses networkmanager nls nptl nsplugin objc objc++ objc-gc ogg okteta omr openal opencl openexr opengl openmp opus pam pango pch pcre pdf perl phonon plasma playlist png policykit ppds projectm python qemu qml qt3support qt4 qt5 qthelp quvi rar raw readline reviewboard rfc3779 sasl sbsms scanner schroedinger sdl seccomp semantic-desktop session sid smp soundtouch speex spell spice sqlite sqlite3 sse sse2 sse3 ssh ssl startup-notification static-libs subversion svg syslog systemtap taglib tcpd theora threads thumbnail tiff tk tools tremor truetype twolame udev udisks unicode upnp upnp-av upower usb usbredir utils v4l vaapi valgrind vcdx vde vdpau vhost-net vim-syntax vlc vorbis vpx vte wavpack webkit webp widgets wifi wxwidgets x264 x265 xattr xcb xcomposite xine xinerama xml xpm xrandr xscreensaver xv xvid xvmc zeroconf zlib zvbi" ABI_X86="64 32" ALSA_CARDS="hdsp emu10k1 hda-intel" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="pc" INPUT_DEVICES="keyboard mouse joystick evdev wacom vmmouse" KERNEL="linux" L10N="en-US" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en_US" LLVM_TARGETS="AMDGPU BPF NVPTX X86 ARM PowerPC" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6" POSTGRES_TARGETS="postgres9_5" PYTHON_SINGLE_TARGET="python3_4" PYTHON_TARGETS="python2_7 python3_4" QEMU_SOFTMMU_TARGETS="aarch64 arm armeb i386 m68k mips mips64 mips64el mipsel mipsn32 mipsn32el ppc ppc64 ppc64abi32 ppc64le x86_64" QEMU_USER_TARGETS="aarch64 arm armeb i386 m68k mips mips64 mips64el mipsel mipsn32 mipsn32el ppc ppc64 ppc64abi32 ppc64le x86_64" RUBY_TARGETS="ruby19 ruby20 ruby22 ruby23 ruby24" USERLAND="GNU" VIDEO_CARDS="nvidia radeon qxl vmware" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_RSYNC_EXTRA_OPTS ================================================================= Package Settings ================================================================= dev-util/kdevelop-5.1.2::gentoo was built with the following: USE="debug handbook plasma qmake -gdbui -okteta -test" ABI_X86="(64)" dev-util/kdevelop-pg-qt-2.1.0::gentoo was built with the following: USE="-debug -test" ABI_X86="(64)" dev-util/kdevplatform-5.1.2::gentoo was built with the following: USE="cvs debug reviewboard subversion welcomepage -test" ABI_X86="(64)"
Being that I don't know QT or KDE, if you can give me a hint as to which thread(s) service the open dialogue and which are the parsing threads I will play with it some more and see what I can figure out. My wild guesses are that "QQmlThread" is servicing the open dialogue (defined in qml?), "pool" is the main thread pool thread and that each of the "Queue" threads are members of that pool (that do parsing)? Futexes were fairly recently introduced (3.18 I think?), so maybe this flaw is something new-ish in the framework? I don't know if it's normal, but one of those threads has 22 levels of recursion.
Note, to get more standard traces with multiple threads, use "thread apply all bt". The trace you posted points to looking up the mime type of a document. Being just one sample, that doesn't really tell what the time is spent on ... I don't know, I don't see the problem here. Which file dialog are you using? How large is the directory it opens on?
(In reply to Sven Brauch from comment #9) > Note, to get more standard traces with multiple threads, use "thread apply > all bt". Ahh, thanks for that! > > The trace you posted points to looking up the mime type of a document. Being > just one sample, that doesn't really tell what the time is spent on ... > > I don't know, I don't see the problem here. Which file dialog are you using? > How large is the directory it opens on? Well crap, I forgot that I booted into Mate a few weeks ago because I was having problems with KDE ignoring alt-tab after an update. :( Sorry about that, so this is a KDevelop w/ Mate problem. Again, the problem isn't anything normal, it's not the size of the directory because the whole thing loads and populates the list, I can scroll through the entire list at normal speed until the project loading gets to the point where it's parsing files.
Oh, so it's the GTK3 file dialog? Ok, interesting to know ... I should test that some day ...
Thank you for reporting this issue in KDE software. As it has been a while since this issue was reported, can we please ask you to see if you can reproduce the issue with a recent software version? If you can reproduce the issue, please change the status to "REPORTED" when replying. Thank you!
Dear Bug Submitter, This bug has been in NEEDSINFO status with no change for at least 15 days. Please provide the requested information as soon as possible and set the bug status as REPORTED. Due to regular bug tracker maintenance, if the bug is still in NEEDSINFO status with no change in 30 days the bug will be closed as RESOLVED > WORKSFORME due to lack of needed information. For more information about our bug triaging procedures please read the wiki located here: https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging If you have already provided the requested information, please mark the bug as REPORTED so that the KDE team knows that the bug is ready to be confirmed. Thank you for helping us make KDE software even better for everyone!
This bug has been in NEEDSINFO status with no change for at least 30 days. The bug is now closed as RESOLVED > WORKSFORME due to lack of needed information. For more information about our bug triaging procedures please read the wiki located here: https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging Thank you for helping us make KDE software even better for everyone!