Bug 252353 - Please introduce unified handwriting recognition on the desktop for tablet computers
Summary: Please introduce unified handwriting recognition on the desktop for tablet co...
Status: CONFIRMED
Alias: None
Product: plasmashell
Classification: Plasma
Component: general (show other bugs)
Version: 5.19.5
Platform: Fedora RPMs Linux
: NOR wishlist
Target Milestone: 1.0
Assignee: Christoph Feck
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-09-25 16:29 UTC by Otso Helenius
Modified: 2021-03-07 18:15 UTC (History)
7 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Otso Helenius 2010-09-25 16:29:41 UTC
Version:           4.5 (using KDE 4.5.0) 
OS:                Linux

KDE needs an unified method for handwriting recognition on tablet computers. So far I've been trying to experiment with Cellwrite, but it's far from a useful option.

The current method on Vista is actually quite good:
http://www.youtube.com/watch?v=vF0n2IvdCpI

Even if the docking problems would be fixed with Cellwrite, it's still cumbersome. There is no good way to write sentences, and to rewrite and remove words in between.

Also tools for controlling text after entering it into another application (as Cellwrite does) are lacking.

My suggestion is:

- Fork the Cellwrite code and enable docking it permanently or popping up at cursor position with a configurable hotkey
- Allow removing whole words from sentences with single tap
- Enable adjustable delay after which written text is sent to the active window. Also introduce direct buttons for e.g. delete and backspace for quick correcting of the already entered text
- Integrate the app with the spellchecking capabilities in KDE (e.g KOffice)

I think KDE Tablet interface could benefit from a snappy unintrusive handwriting interface especially now that the amount of devices capable running of KDE but lacking in keyboard is increasing.

Reproducible: Didn't try
Comment 1 Christoph Feck 2010-09-25 16:46:31 UTC
Moved to Plasma, because it is very likely this needs to be integrated with the current plasma keyboard.

Added myself as the assignee not because I have working code, but because I am probably the only KDE core developer that has a tablet PC with a Wacom pen (feel free to prove me wrong ;)

If someone wants to start the project, I can help mentoring it. Eventually I might start it myself, too, but that's far future seeing my current to do list.

Cellwriter unfortunately is unusable on KDE 4 because of bug 234926.
Comment 2 Otso Helenius 2011-09-28 10:54:11 UTC
Any chance of someone picking this up for 4.8?
Comment 3 Otso Helenius 2012-06-18 13:44:29 UTC
With cellwriter 1.3.5 the blocker bug is resolved finally. Would there be any possibility to get a pop-up writing area for example added as a panel now that KDE has taken a clear step towards supporting tablet devices?
Comment 4 Christoph Feck 2012-07-31 23:59:14 UTC
I extracted the CellWriter code and can now use its recognition engine from inside a Qt application. Still have to check how it can be integrated into the plasma-keyboard.
Comment 5 Otso Helenius 2012-08-15 05:32:08 UTC
This sounds fantastic! I'm ready to do beta testing when you have the plasma version up at github/have rpms/sources.
Comment 6 Christoph Feck 2012-11-06 11:22:54 UTC
Could not find the plasma-keyboard, it looks like Plasma now uses Maliit as the keyboard.
Comment 7 Michael Hasselmann 2013-02-22 06:10:00 UTC
Plasma Active is already using Maliit. Maliit's architecture allows for plugins that can handle handwriting recognition, yet I am unaware of any good open-source implementation that could do that? Bonus points if it uses Qt already.
Comment 8 Christoph Feck 2013-02-22 12:06:45 UTC
Michael, I extracted the code from CellWriter, and made a Qt API for it. Works fine so far, if you could give me some pointers how to integrate it with Maliit, I would appreciate that. Alternatively, if you are working with Maliit source already, you could try that yourself, given that I cannot start this before May, if at all.

Source is here: http://skulpture.maxiom.de/playground/kritzel.tar.xz
Comment 9 Peter Potrowl 2013-04-01 15:55:59 UTC
How difficult would be the integration with Maliit? What about proposing this as a Google Summer of Code project?

http://community.kde.org/GSoC/2013/Ideas
Comment 10 Michael Hasselmann 2013-04-01 16:11:22 UTC
Yeah, might be a good idea for GSoC as this would be a relatively isolated task. Am not sure I'd make a good mentor, however.
Comment 11 Michael Hasselmann 2013-04-01 16:26:47 UTC
I played a bit with cellwriter now, very cool UI. Not sure I'd copy it 1:1 for Maliit.
Comment 12 Nate Graham 2018-06-08 20:06:31 UTC
Hello!

This feature request was filed for KDE Plasma 4, which reached end-of-support status in August 2015. KDE Plasma 5's desktop shell has been almost completely rewritten for better performance and usability, so it is likely that this feature request is already implemented in Plasma 5, or is no longer applicable.

Accordingly, we hope you understand why we must close this feature request. If the requested feature is still desired but not implemented in KDE Plasma 5.12 or later, please feel free to open a new ticket in the "plasmashell" product after reading https://community.kde.org/Get_Involved/Bug_Reporting

If you would like to get involved in KDE's bug triaging effort so that future mass bug closes like this are less likely, please read https://community.kde.org/Get_Involved#Bug_Triaging

Thanks for your understanding!

Nate Graham
Comment 13 Christoph Feck 2018-10-10 13:17:25 UTC
There is a GPL3 recognizer at https://github.com/phatware/WritePad-Handwriting-Recognition-Engine

This would need to be integrated into QtVirtualKeyboard now that Maliit is obsolete.
Comment 14 Nate Graham 2020-09-28 23:50:30 UTC
Actually last I heard, KWin and Plasma Mobile's developers are looking to go back to Maliit. Perhaps that could be added as a plugin?
Comment 15 Aleix Pol 2020-10-28 23:42:07 UTC
It could be added as a plugin to maliit, it could be added as a separate application (which is probably easier to implement).

Something that would be useful would be implementing a standalone Qt Virtual Keyboard (which supports handwriting, https://doc.qt.io/qt-5/handwriting.html) that implements the input-method-v1 protocol (which is the one that maliit implements. It would allow us to leverage the tool without having to depend explicitly on it.

If anyone wants to work on it and initial guidance is needed, I'll be happy to assist.