Bug 311408

Summary: Partitionmanager - libsolid doesn't correctly detect hard disks
Product: [Frameworks and Libraries] solid Reporter: Mattia <mattia.verga>
Component: generalAssignee: Lukáš Tinkl <lukas>
Status: RESOLVED INTENTIONAL    
Severity: grave CC: adaptee, andrius, asturm, chris.dart, entisten, kde.bugs.sp242, kde, kevin.kofler, lukas, njc, rdieter, sqn14, treeve, yesyoucanspamme
Priority: NOR    
Version: unspecified   
Target Milestone: 4.11   
Platform: unspecified   
OS: Linux   
Latest Commit: Version Fixed In:
Attachments: partitionmanager window
same disk from gparted
systemsettings.desktop
backtrace of partition manager crash
backtrace of partition manager crash
output of solid-hardware list detail with udisks1
output of solid-hardware list detail with udisks2
njc's print screen

Description Mattia 2012-12-09 13:20:48 UTC
I'm using the last SVN version self compiled.
Partitionmanager only shows me the swap partition: it detects as device /dev/sda5 instead of /dev/sda
sda5 (swap) and sda6 (root) are both under an extended partition (/dev/sda3).
Here is the output of fdisk -l
# fdisk -l

Disk /dev/sda: 500.1 GB, 500107862016 bytes, 976773168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Identificativo disco: 0x000d2ae4

Dispositivo Boot      Start         End      Blocks   Id  System
/dev/sda1            2048   209717247   104857600    7  HPFS/NTFS/exFAT
/dev/sda2   *   209717248   210741247      512000   83  Linux
/dev/sda3       210741248   517325129   153291941    5  Esteso
/dev/sda4       517325130   976768064   229721467+  83  Linux
/dev/sda5       210743296   227127295     8192000   82  Linux swap / Solaris
/dev/sda6       227129344   517324799   145097728   83  Linux


Reproducible: Always
Comment 1 Mattia 2012-12-09 13:21:44 UTC
Created attachment 75748 [details]
partitionmanager window
Comment 2 Mattia 2012-12-09 13:22:07 UTC
Created attachment 75749 [details]
same disk from gparted
Comment 3 Andrius Štikonas 2012-12-09 16:02:35 UTC
Is this still the case with latest SVN? I accidentally introduced integer overflow today (sorry) which is now fixed.
Comment 4 Mattia 2012-12-09 17:03:43 UTC
Same thing, problem is still there.
Can be a problem introduced by parted 3.1? Last time I've compiled for Fedora 17 (parted 3.0) it worked, now in Fedora 18 (parted 3.1) it doesn't... I will try to rebuild locally with parted 3.0 instead of using Fedora buildsystem to see if it changes something.
Comment 5 Mattia 2012-12-09 18:14:11 UTC
Tried to compile with parted 3.0 under Fedora 18, but the problem persists.
Luckily I had an outdated Fedora 17 virtual machine, I tried to compile last SVN of partitionmanager in that and it works... I'm going to do a full update of that machine and see if it give me the problem.
Comment 6 Mattia 2012-12-09 21:19:43 UTC
Ok, so: last SVN version on Fedora 17 with parted 3.0 - works
last SVN version on Fedora 18 with parted 3.1 - doesn't work
last SVN version or previous compiled version for Fedora 17 on Fedora 18 with parted 3.0 - doesn't work

It's not a parted version related problem.
Some other information: the log in partitionmanager shows "Scanning devices - Device found: unknown" and I can see a lot (20 or more) of "partitionmanage" (yes, without the final 'r') zombie processes in top while the application is running.
Comment 7 Andrius Štikonas 2012-12-09 21:36:58 UTC
Could it be that Fedora 18 uses udisks2 and Fedora 17 uses udisks1?
(In reply to comment #6)
> Ok, so: last SVN version on Fedora 17 with parted 3.0 - works
> last SVN version on Fedora 18 with parted 3.1 - doesn't work
> last SVN version or previous compiled version for Fedora 17 on Fedora 18
> with parted 3.0 - doesn't work
> 
> It's not a parted version related problem.
> Some other information: the log in partitionmanager shows "Scanning devices
> - Device found: unknown" and I can see a lot (20 or more) of
> "partitionmanage" (yes, without the final 'r') zombie processes in top while
> the application is running.
Comment 8 Mattia 2012-12-10 16:05:11 UTC
Created attachment 75769 [details]
systemsettings.desktop

(In reply to comment #7)
> Could it be that Fedora 18 uses udisks2 and Fedora 17 uses udisks1?
> (In reply to comment #6)

Just checked: yes, Fedora 17 ships udisks1 as default, while Fedora 18 uses udisks2.
Is partitionmanager incompatible with udisks2? Should I figure out how to compile it to use udisks1?
Comment 9 Andrius Štikonas 2012-12-10 16:28:44 UTC
Partition manager does not use udisks directly. It uses Solid and whatever 
backend Solid provides. The question is whether kdelibs in Fedora support 
udisks2. I think that there were some effors to backport udisks2 support from 
KDE Frameworks 5 to kdelibs 4.x but I'm not sure if Fedora 18 includes them...
But that would be really strange because how can they detect removable media 
in other KDE applications.

I haven't yet tried udisks2 myself on Gentoo but kdelibs package has optional 
udisks2 USE flag...
Comment 10 Mattia 2012-12-10 16:46:33 UTC
Well, in Fedora 18 I can see that all KDE packages depends on udisks2 (if I try to uninstall it, yum tries to remove also kdelibs package). So KDE is using udisks2.
I will ask on Fedora development mailing list how I can bypass this problem, but I don't think they will rebuild all KDE stuff without udisks2 flag...
Comment 11 Andrius Štikonas 2012-12-10 21:02:21 UTC
I've compiled udisks2 and recompiled kdelibs to use it but for now I can't get partitionmanager to work with it at all. It simply crashes but according to backtrace the crash is in Solid. I suspect that this is not a bug in partitionmanager...
Hopefully, future versions of kdelibs and Solid would work better with udisks2.
Comment 12 Rex Dieter 2012-12-11 03:08:22 UTC
Can you post backtraces for the crashes you encounter?   I'll poke some solid/udisk guru (ltinkl for starters) to look into it.
Comment 13 Andrius Štikonas 2012-12-11 13:01:12 UTC
Created attachment 75782 [details]
backtrace of partition manager crash

I'm attaching the requested backtrace.
Comment 14 Lukáš Tinkl 2012-12-11 13:47:06 UTC
Can you please install all the needed debugging symbols to get a more meaningful BT?
Comment 15 Rex Dieter 2012-12-11 14:04:44 UTC
Interestingly, installing and running kde-partitionmanager-1.0.3-8.201202025svn.fc18 works fine on my fedora 18 box (no crash, detects my disks/partitions correctly).

though, for various reasons selinux is off/disabled, could you try running in selinux permissive mode and see if that changes anything?
Comment 16 Andrius Štikonas 2012-12-11 15:16:13 UTC
I've recompiled Qt, glib and glibc on Gentoo and it still shows the same backtrace. I'm not sure why backtrace is not complete...
Comment 17 Andrius Štikonas 2012-12-11 15:21:55 UTC
sorry, I think I know why backtraces are incomplete. I'll upload a new one once I recompile Qt/kdelibs...
Comment 18 Andrius Štikonas 2012-12-11 16:37:20 UTC
Created attachment 75785 [details]
backtrace of partition manager crash
Comment 19 Kevin Kofler 2012-12-11 17:12:29 UTC
Debugging information for libsolid is still missing.
Comment 20 Andrius Štikonas 2012-12-11 17:54:59 UTC
This is really strange. I am sure that kdelibs, libsolid and solid-runtime were all compiled with debug symbols (and these symbols were not stripped). Something is really weird.

(In reply to comment #19)
> Debugging information for libsolid is still missing.
Comment 21 Mattia 2012-12-11 18:06:24 UTC
(In reply to comment #15)
> Interestingly, installing and running
> kde-partitionmanager-1.0.3-8.201202025svn.fc18 works fine on my fedora 18
> box (no crash, detects my disks/partitions correctly).
> 
> though, for various reasons selinux is off/disabled, could you try running
> in selinux permissive mode and see if that changes anything?

I tried on my main PC and in a virtual machine with both selinux on and off, but the result is still the same. I also installed today kdelibs updates and checked, no change.
Can you please try the latest svn version? It's available in koji, but I didn't pushed it into updates-testing... http://koji.fedoraproject.org/koji/buildinfo?buildID=371979

I also had some random crash at startup, but now I'm not able to reproduce that... if I get the crash I will install debugging symbols to create a backtrace.
Comment 22 Rex Dieter 2012-12-11 18:18:37 UTC
That build is ok for me too.  Full disclosure, forgot to mention this box of mine is running kde-4.9.90

Mattia, mind testing against kde-4.9.4 (if you hadn't already), it included some good solid/udisks2 fixes,
https://admin.fedoraproject.org/updates/FEDORA-2012-20061
Comment 23 Mattia 2012-12-11 20:05:23 UTC
(In reply to comment #22)
> That build is ok for me too.  Full disclosure, forgot to mention this box of
> mine is running kde-4.9.90
> 
> Mattia, mind testing against kde-4.9.4 (if you hadn't already), it included
> some good solid/udisks2 fixes,
> https://admin.fedoraproject.org/updates/FEDORA-2012-20061

Yes, I've installed kde-4.9.4 with today updates, but I get the same problem.

I've cloned my Fedore 18 virtual machine, enabled rawhide repository and installed kdelibs 4.9.90 (and related dependencies): this way I confirm partitionmanager is working.
Comment 24 Andrius Štikonas 2012-12-12 10:07:11 UTC
OK, I've updated to KDE 4.9.4 and it seems that Solid related crashes are fixed.
I think that we are seeing 2 different problems in this bug. It seems that one of them is fixed by 4.94.
Comment 25 Volker Lanz 2012-12-14 09:31:21 UTC
So what to make of this? This looks like it has the potential to take hours to even try and _reproduce_ it, not having KDE 4.9.90 or udisks2 running yet.

Is there anything I need to do or even _can_ do? Or do we close this? Or reassign to solid?
Comment 26 Mattia 2012-12-14 20:22:32 UTC
(In reply to comment #25)
> So what to make of this? This looks like it has the potential to take hours
> to even try and _reproduce_ it, not having KDE 4.9.90 or udisks2 running yet.
> 
> Is there anything I need to do or even _can_ do? Or do we close this? Or
> reassign to solid?

Since partitionmanager works fine with kde 4.9.90 and udisks2 I think there's something wrong on how solid/kdelibs 4.9.4 interfaces with udisks2.
So I think it should be reassigned to kdelibs?
Comment 27 Rex Dieter 2012-12-14 20:42:56 UTC
Let me amend that, fedora builds prior to kdelibs-4.9.90-3 were inadvertently using the udisks(1) backend, not udisks2.  I've confirmed that udisks2 backend with kdelibs-4.9.90-3 behaves badly too.

Ideally, if we could find a minimal test-case, based on the solid calls made in partitionmanager to highlight the problem(s), that would help identify what's going wrong.
Comment 28 Mattia 2012-12-15 16:52:28 UTC
I'm not a programmer, but I did some tests...
I attach two files made in my Virtual Machine, showing the output of
'solid-hardware list details'
(only related to block_devices and drives).
As you can see with udisks1 the udi = '/org/freedesktop/UDisks/devices/sda'
is listed with a property of
Block.device = '/dev/sda'  (string)

Instead in udisks2 the udi = '/org/freedesktop/UDisks2/drives/VBOX_HARDDISK_VB3ace5ba5_0dc3560f'
is listed with a property of
Block.device = '/dev/sda3'  (string)

I think that's the error, in fact partitionmanager shows '/dev/sda3' as to be the hard disk instead of '/dev/sda'. I checked also on my main PC, as you can look from the previous screenshot partitionmanager shows the disk as '/dev/sda5' and the output of solid-hardware shows "Block.device = '/dev/sda5'  (string)"
Comment 29 Mattia 2012-12-15 16:53:06 UTC
Created attachment 75849 [details]
output of solid-hardware list detail with udisks1
Comment 30 Mattia 2012-12-15 16:53:24 UTC
Created attachment 75850 [details]
output of solid-hardware list detail with udisks2
Comment 31 Rex Dieter 2012-12-15 19:05:11 UTC
thanks, confirmed and reassigning to solid (for now)
Comment 32 Mattia 2012-12-26 18:19:38 UTC
Can the problem be in the conditional of the IF at line 56 of solid / solid / backends / udisks2 / udisksblock.cpp ?

I think it should say something like "IF the device is type: partition table than get the path". The actual conditional is always true and it gets the first partition it parses (wich is the last on the disk). (but, again, my knowledge of C is very poor and mine is only a guess)
Comment 33 Mattia 2013-02-03 13:01:42 UTC
ping?
Comment 34 Mattia 2013-02-09 11:22:00 UTC
Tested with kdelibs 4.9.98, same problem
Comment 35 Drift 2013-02-17 17:52:46 UTC
This bug happens to me as well.
Fedora 18 w/ kde-unstable + kde-testing + kde (KDE 4.10)
kde-partitionmanager-1.0.3-8.20120205svn.fc18.x86_64

It detects only one partition per device.
I have no unencrypted swap partitions: it detects one random partition.
On my setup, it detects
/dev/sda4 rather than /dev/sda
/dev/sdb11 rather than /dev/sdb.

sda has a GPT partition table.
/dev/sda4 happens to be a 14GiB ext4 partition.
/dev/sda4 is seen as if it were a block device (as if it were /dev/sda): /dev/sda41 (!) is the "perceived" 14GiB partition. partitionmanager cannot recognize that it is in ext4 anyway.

sdb has a msdos partition table.
/dev/sdb11 is a plain dm-crypt partition.
"No valid partition table found".
Comment 36 Mattia 2013-03-07 17:44:24 UTC
Lukáš, can you please say us something on this?
Comment 37 Lukáš Tinkl 2013-03-07 18:55:53 UTC
Can't reproduce, doesn't crash on KDE 4.10
Comment 38 Rex Dieter 2013-03-07 19:03:26 UTC
Crashing isn't the problem (anymore, as far as I'm aware).

The problem now is that apparently, partitionmanager doesn't grok the output from udisks2 backend.  see the attachments here "output of solid-hardware list detail with udisks1" vs "output of solid-hardware list detail with udisks2" as an example of the difference(s).

I guessing  partitionmanager may make some invalid assumptions on the type or format of the data returned, but hard to say without looking at the code.
Comment 39 Mattia 2013-03-07 20:17:30 UTC
(In reply to comment #37)
> Can't reproduce, doesn't crash on KDE 4.10

My original problem wasn't a crash, it was that Partitionmanager detects the entire hard disk with the name of a partition on that disk (in my case instead of /dev/sda it detected /dev/sda5).
Looking at the output of solid with 'solid-hardware list details' I found that seems a problem of libsolid-udisks2 that gets an udi related to a partition associated to what should be the entire hard disk. See comment #28
Comment 40 njc 2013-03-23 13:22:16 UTC
Created attachment 78315 [details]
njc's print screen

it's my print screen
Comment 41 njc 2013-03-23 13:24:08 UTC
Comment on attachment 78315 [details]
njc's print screen

It's my disks configuration:
[root@njc-n73sv ~]# fdisk -l

Disk /dev/sda: 750.2 GB, 750156374016 bytes, 1465149168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0xf3c6f6a7

Устр-во Загр     Начало       Конец       Блоки   Id  Система
/dev/sda1            2048    52430847    26214400   1c  Скрытый W95 FAT32 (LBA)
/dev/sda2   *    52430848   693467135   320518144    7  HPFS/NTFS/exFAT
/dev/sda3       693467136  1465147391   385840128    f  W95 расшир. (LBA)
/dev/sda5       693469184  1465147391   385839104    7  HPFS/NTFS/exFAT

Disk /dev/sdb: 500.1 GB, 500107862016 bytes, 976773168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x366ec4cf

Устр-во Загр     Начало       Конец       Блоки   Id  Система
/dev/sdb1   *        2048     1026047      512000   83  Linux
/dev/sdb2         1026048   103426047    51200000   83  Linux
/dev/sdb3       103426048   205826047    51200000   83  Linux
/dev/sdb4       205826048   976773119   385473536    5  Расширенный
/dev/sdb5       205830144   218118143     6144000   82  Linux своп / Solaris
/dev/sdb6       218130633   527719184   154794276   83  Linux
Partition 6 does not start on physical sector boundary.
/dev/sdb7       527722496   528746495      512000   83  Linux
/dev/sdb8       528748544   633606143    52428800   83  Linux
/dev/sdb9       633608192   645666815     6029312   82  Linux своп / Solaris
[root@njc-n73sv ~]#
Comment 42 njc 2013-03-23 13:29:46 UTC
Comment on attachment 78315 [details]
njc's print screen

[njc@njc-n73sv ~]$ uname -a
Linux njc-n73sv.localdomain 3.8.3-203.fc18.x86_64 #1 SMP Mon Mar 18 12:59:28 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
[njc@njc-n73sv ~]$ 


and 
KDE 4.10.1
Comment 43 Mattia 2013-04-19 18:41:36 UTC
KDE 4.10.2 libsolid still doesn't detect partitions in a correct way using udisks2.
Comment 44 Jekyll Wu 2013-04-19 22:22:31 UTC
*** Bug 318608 has been marked as a duplicate of this bug. ***
Comment 45 Esteban Taroni 2013-04-23 21:07:54 UTC
Running gentoo with solid 4.10.1.
Downgrading udisks to 1.0.4-r5 don't solve the problem.

The problem seems to be with libsolid.
Can't test 4.9 with udisks2 because I remove backup of 4.9 install
Comment 46 Esteban Taroni 2013-04-23 21:40:18 UTC
Sorry my test was wrong because I forget to unmerve udisks2.
Make more test
Comment 47 Jekyll Wu 2013-05-30 01:19:07 UTC
*** Bug 316005 has been marked as a duplicate of this bug. ***
Comment 48 Andrius Štikonas 2013-06-23 09:44:40 UTC
SVN commit 1358151 by stikonas:

Add an initial support for UDisks2 solid backend.
Currently, UDisks2 support can only be enabled at compile time.

 M  +5 -0      CMakeLists.txt  
 M  +5 -1      src/plugins/libparted/libpartedbackend.cpp  
 M  +5 -0      src/util/helpers.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1358151
Comment 49 Andreas Sturmlechner 2013-06-23 12:25:02 UTC
Excellent, builds fine, now partitionmanager lists both SSD and HDD correctly on my system.
Comment 50 Mattia 2013-06-23 13:48:30 UTC
I confirm now partitionmanager works with udisks2 backend, thank you Andrius.
Comment 51 Lukáš Tinkl 2013-06-23 14:39:24 UTC
Good work!
Comment 52 Andrius Štikonas 2013-06-24 00:10:07 UTC
I've noticed that partitionmanager fails to see the device if it contains no partition table. E.g. you first zero the device or fill it with the random data.

I'll see if this can be fixed but it still works much better then before if you use udisks2, so it should be good as a temporary solution.
Comment 53 Kevin Kofler 2013-06-29 21:44:54 UTC
I don't think that's the right approach to fix this. Having #ifdefs based on the Solid backend defeats the whole purpose of having the Solid abstraction in the first place! IMHO, this really needs to be fixed on the Solid end.
Comment 54 Andrius Štikonas 2013-06-29 22:22:06 UTC
(In reply to comment #53)
> I don't think that's the right approach to fix this. Having #ifdefs based on
> the Solid backend defeats the whole purpose of having the Solid abstraction
> in the first place! IMHO, this really needs to be fixed on the Solid end.

I agree but this bug was too serious and we need at least temporary solution. Of course this is somewhat ugly and it does not work 100% perfectly (partitionmanager doesn't see device if it doesn't contain partition table) but at least people are able to use it now.
Comment 55 Mattia 2013-07-07 08:42:47 UTC
So, shouldn't this bug remain open and assigned to solid?
Or someone that have identified the problem in solid code should open a new bug?
Comment 56 Chris Dart 2013-07-08 10:00:01 UTC
Partition manager does not understand raid disks. Shows both parts of the raid group separatly but does not show the raid group as a unit. 
I suspect installer is using 'PartitionManager' and as a result unable to load kde as dual boot with win 7. Win 7 with Mint 15 and Mate 1.6 is no problem but KDE will not work. 
Installer is instructed to load grub to 'disk5' but always crashes when trying to load it to 'sda' - it was not told to load to sda !!!! Devices show as sda; sdb; sdc etc in PartitionManager but as disk1; disk2; disk3 etc in installer. I am totaly confused.
Comment 57 Andrius Štikonas 2014-07-08 10:08:29 UTC
*** Bug 332151 has been marked as a duplicate of this bug. ***
Comment 58 Andrius Štikonas 2015-02-18 21:11:56 UTC
I guess we can close this bug for now. Partitionmanager is not affected at all now and there is no obvious way to fix this.