Bug 410807

Does the script you are trying to run contain a call to sys.argv? Because that's the part that's crashing and not part of our code. Embedded Python does not have access to command line args because it's never run from a command line.

The exception is thrown even when debugging a simple Hello World script or even empty files :(

I can confirm this on Windows 10 with the Krita 4.2.5 download from the Krita website.

I also belatedly noticed that the debugger doesn't work on Linux either (4.2.5. appimage)—it just dies with a different error in a less noticeable way. I get a traceback in the Linux terminal: 

qt.svg: Cannot open file ':/icons/debug_arrow.svg', because: No such file or directory
Error in sys.excepthook:
Traceback (most recent call last):
  File "/tmp/.mount_krita-kLnui4/usr/lib/krita-python-libs/krita/excepthook.py", line 23, in on_error
    dlg = ExceptHookDialog(exc_type, exc_obj, exc_tb)
  File "/tmp/.mount_krita-kLnui4/usr/lib/krita-python-libs/krita/excepthook.py", line 60, in __init__
    html = cgitb.text((exc_type, exc_obj, exc_tb))
  File "/tmp/.mount_krita-kLnui4/usr/lib/python3.5/cgitb.py", line 212, in text
    formatvalue=lambda value: '=' + pydoc.text.repr(value))
  File "/tmp/.mount_krita-kLnui4/usr/lib/python3.5/inspect.py", line 1231, in formatargvalues
    specs.append(convert(args[i]))
  File "/tmp/.mount_krita-kLnui4/usr/lib/python3.5/inspect.py", line 1228, in convert
    return formatarg(name) + formatvalue(locals[name])
KeyError: 'exc'

Original exception was:
Traceback (most recent call last):
  File "/tmp/.mount_krita-kLnui4/usr/share/krita/pykrita/scripter/ui_scripter/actions/debugaction/debugaction.py", line 45, in debug
    self.scripter.debugcontroller.start(self.scripter.documentcontroller.activeDocument)
  File "/tmp/.mount_krita-kLnui4/usr/share/krita/pykrita/scripter/debugcontroller.py", line 39, in start
    loop.run_until_complete(self._debugger.start())
  File "/tmp/.mount_krita-kLnui4/usr/lib/python3.5/asyncio/base_events.py", line 387, in run_until_complete
    return future.result()
  File "/tmp/.mount_krita-kLnui4/usr/lib/python3.5/asyncio/futures.py", line 274, in result
    raise self._exception
  File "/tmp/.mount_krita-kLnui4/usr/lib/python3.5/asyncio/tasks.py", line 241, in _step
    result = coro.throw(exc)
  File "/tmp/.mount_krita-kLnui4/usr/share/krita/pykrita/scripter/debugger_scripter/debugger.py", line 115, in start
    yield self.display()
RuntimeError: yield was used instead of yield from for generator in task <Task pending coro=<Debugger.start() running at /tmp/.mount_krita-kLnui4/usr/share/krita/pykrita/scripter/debugger_scripter/debugger.py:115> cb=[_run_until_complete_cb() at /tmp/.mount_krita-kLnui4/usr/lib/python3.5/asyncio/base_events.py:164]> with <generator object Debugger.display at 0x7ff4fc0c4728>

Git commit 1e447588703b64bfbee5dfb258a58e3e590d4b2d by Rebecca Breu.
Committed on 02/09/2019 at 19:03.
Pushed by rbreu into branch 'master'.

BUG 410807: Python Scripter Debugger won't run (Linux)
This fixes the issues on Linux.

The 'yield from' syntax has been introduced in Python 3.3, so this won't run on older Python versions, including 2.x. According to https://vfxplatform.com/ we can drop Python 2 support in 2020, and Boud said to drop Python 2 support now in master, seeing how Krita 4.3 will come out the end of 2019 anyway.

M  +3    -3    plugins/python/scripter/debugger_scripter/debugger.py
M  +1    -1    plugins/python/scripter/ui_scripter/actions/debugaction/debugaction.py

https://invent.kde.org/kde/krita/commit/1e447588703b64bfbee5dfb258a58e3e590d4b2d

That commit only fixed the Linux error. Looking into what's up with Windows now.

Git commit 4fd16ad8cadaac7f16f2a8b483f67dfef9e3599a by Boudewijn Rempt, on behalf of Rebecca Breu.
Committed on 10/09/2019 at 08:25.
Pushed by rempt into branch 'krita/4.2'.

BUG 410807: Python Scripter Debugger won't run (Linux)
This fixes the issues on Linux.

The 'yield from' syntax has been introduced in Python 3.3, so this won't run on older Python versions, including 2.x. According to https://vfxplatform.com/ we can drop Python 2 support in 2020, and Boud said to drop Python 2 support now in master, seeing how Krita 4.3 will come out the end of 2019 anyway.

M  +3    -3    plugins/python/scripter/debugger_scripter/debugger.py
M  +1    -1    plugins/python/scripter/ui_scripter/actions/debugaction/debugaction.py

https://invent.kde.org/kde/krita/commit/4fd16ad8cadaac7f16f2a8b483f67dfef9e3599a

Unfortunately, this is a very tricky issue.

The debugger wants to spawn a subprocess for debugging the script. On Linux and (I assume) MacOS, this works by forking the current process.

On Windows, there's no way to fork/clone the current process, so as a workaround, Python's multiprocessing library starts a new process from scratch and gives it the parent process's argv to provide such things as the Python executable, Python script etc. However, since we are running an embedded Python, the parent process has no argv and trying to access that attribute fails with the error posted by the bug reporter. It's possible to set argv manually before spawning the subprocess, but that way, we could only provide a system Python executable (if it even exists), not Krita's embedded Python since that's not an executable.

See for example here:
https://stackoverflow.com/questions/15636266/embedded-python-multiprocessing-not-working

Here's the same issue with a user script in Blender:
https://blender.stackexchange.com/questions/8530/how-to-get-python-multiprocessing-module-working-on-windows

The only fix that I can think of is rewriting the debugger to use threads instead of subprocesses, but that could potentially cause other issues, like thread-safety and issues with Python's GIL.

Not sure if Blender has a debugger like this—might be worth looking into?

If we can't solve it, we could at least deactivate the debugger on Windows, or catch the error with a meaningful popup for the user.

Note to self: Try if spawning subprocesses actually still works on Linux, or whether we run into the same sqlite db access issues as https://bugs.kde.org/show_bug.cgi?id=420078

"Do not open an SQLite database connection, then fork(), then try to use that database connection in the child process. All kinds of locking problems will result and you can easily end up with a corrupt database."

https://www.sqlite.org/howtocorrupt.html#_carrying_an_open_database_connection_across_a_fork_

When does Krita try to access the DB? Can it potentially happen from within a Python script? What about threading?

*** Bug 478341 has been marked as a duplicate of this bug. ***

I just checked that on Linux, the Debugger still works. 

BUG 478341 has a slightly different traceback than the original one here, possibly since our current Python version is trying to deal with the forking issue a bit differently, but I think the underlying cause is still the same. Since I failed to find a solution and am not currently working on it, I'm unassigning myself.

Confirming that Krita 5.2.2 on macOS gives a similar "TypeError: cannot pickle 'ScripterExtension' object" as reported on Windows in bug 478341.
It seems the debugger only works on Linux.