Bug 383177 - Provide a copy of all logs when executing an Observation Plan with the Scheduler
Summary: Provide a copy of all logs when executing an Observation Plan with the Scheduler
Status: RESOLVED FIXED
Alias: None
Product: kstars
Classification: Applications
Component: general (show other bugs)
Version: unspecified
Platform: unspecified Unspecified
: NOR wishlist
Target Milestone: ---
Assignee: Jasem Mutlaq
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-08-05 20:17 UTC by TallFurryMan
Modified: 2017-08-29 07:53 UTC (History)
0 users

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description TallFurryMan 2017-08-05 20:17:01 UTC
It would be interesting to get a copy of the different logs (KStars, Ekos, INDI, ...) generated while an Observation Plan executes under the Scheduler.

Tracking, focusing, aligning and guiding could then be logged as jobs are executed. This would allow us to more easily understand and debug problems when things go wrong.

A checkbox in the Scheduler could configure KStars and Ekos to provide maximum log levels for each scheduled item. INDI log levels should not be altered because some may disrupt reactivity of drivers.
Comment 1 Jasem Mutlaq 2017-08-05 20:31:59 UTC
Isn't this already implemented? Go to KStars --> Configure --> Advanced and configure logs for Ekos. There are no control over INDI logs since these are generated by INDI.
Comment 2 TallFurryMan 2017-08-06 09:13:36 UTC
Yes, logs can be generated by KStars, Ekos and INDI (when the .indi/logs folder exists!), but the only relation with the current observation plan is the name of the target and index of the scheduler job. The log doesn't say which job XML is processed for instance. 
Having the excerpts of logs stored along with either the scheduler job XML, or the capture sequence job XML, or the storage for captures, would be helpful to gather relevant information for an observation. This would help posting debug information in issues too. 
However I realize this is harder to specify than it seems, as the notions of input (the scheduler job, the sequence job, the input fits...) and output (the captured frames,... ) have no well-defined hierarchy or dependency. Jobs could be in a read-only location for the scheduler for instance, which would prevent the tool to use that place for output.
Perhaps introducing variables that may be used as location placeholders could be a solution. I'm thinking about the way Eclipse manages paths here and the default "Target" placeholder naming capture files. "$SchedulerOutput", which could be used to define "$SequenceOutput" or even "$IndiOutput" for client-side data. This way all relevant output information can be stored in the same location.
Use case for captures would be:
- create a capture sequence job, assign an output using the undefined "$SchedulerOutput" as prefix for the sequence job, say "$SchedulerOutput/some/path", this defines "$SequenceOutput" for the job. 
- create a scheduler job, assign a capture sequence, assign an output, this defines "$SchedulerOutput" for the scheduler job, say "/home/me/yyyymmdd/a_target".
- when the scheduler job executes, the capture sequence job uses "$SequenceOutput", which is ultimately "/home/me/yyyymmdd/a_target/some/path". 
- any module that write logs or store data can then have those copied to the path resulting from those variables.
Comment 3 Jasem Mutlaq 2017-08-06 10:28:16 UTC
I understand. Actually, KStars should be using Qt Logging Framework. The logging stuff was added before the Qt Logging Framework become mature. I will take a look at that again.
Comment 4 Jasem Mutlaq 2017-08-06 19:29:49 UTC
Git commit 5028e958caf97eaadd5c30cd442254e37fd84088 by Jasem Mutlaq.
Committed on 06/08/2017 at 19:26.
Pushed by mutlaqja into branch 'master'.

Migrating KStars to use QLoggingCategory framework which supports granular control over logging in runtime.

The migration is still not complete yet but the base structure is there and operational and we should migrate any qDebug..et al when appropiate to use the QLoggingCategory framework. When you write code, you must abide by this, I will not accept any code that does not use the framework when logging.
CCMAIL:kstars-devel@kde.org

M  +1    -0    CMakeLists.txt
M  +2    -0    datahandlers/CMakeLists.txt
M  +17   -16   datahandlers/catalogdb.cpp
M  +12   -0    kstars/CMakeLists.txt
M  +0    -1    kstars/auxiliary/kspaths.cpp
M  +15   -13   kstars/auxiliary/ksuserdb.cpp
M  +47   -14   kstars/auxiliary/ksutils.cpp
M  +6    -1    kstars/auxiliary/ksutils.h
M  +3    -2    kstars/ekos/align/align.cpp
M  +12   -19   kstars/ekos/capture/capture.cpp
M  +8    -8    kstars/ekos/ekosmanager.cpp
M  +38   -72   kstars/ekos/focus/focus.cpp
M  +3    -2    kstars/ekos/guide/guide.cpp
M  +3    -2    kstars/ekos/scheduler/scheduler.cpp
M  +33   -61   kstars/fitsviewer/fitsdata.cpp
M  +4    -2    kstars/fitsviewer/fitsviewer.cpp
M  +7    -12   kstars/indi/drivermanager.cpp
M  +3    -2    kstars/indi/indidevice.cpp
M  +5    -9    kstars/indi/indilistener.cpp
M  +8    -12   kstars/indi/inditelescope.cpp
M  +4    -3    kstars/indi/servermanager.cpp
M  +6    -2    kstars/kstars.cpp
M  +3    -1    kstars/kstarsdata.cpp
M  +3    -1    kstars/kstarsdbus.cpp
M  +2    -2    kstars/main.cpp
M  +12   -0    kstars/options/opsadvanced.cpp
M  +1    -0    kstars/options/opsadvanced.h
M  +3    -1    kstars/skycomponents/constellationlines.cpp
M  +5    -3    kstars/skycomponents/deepstarcomponent.cpp
M  +6    -6    kstars/time/simclock.cpp

https://commits.kde.org/kstars/5028e958caf97eaadd5c30cd442254e37fd84088
Comment 5 TallFurryMan 2017-08-10 06:59:49 UTC
This is excellent. I didn't understand where the logs would end up from the patch (my guess is that they are still at the same place), so I need to read documentation about the Qt logging framework. I assume adding a file backend dynamically is possible. Great job!
Comment 6 Jasem Mutlaq 2017-08-10 09:11:54 UTC
Yes it's the same file (if you choose file logging) but now all categories and their logging levels (info, warn, debug..etc) is controllable at run time and via environment variables, in addition to KStars control via the GUI.
Comment 7 TallFurryMan 2017-08-29 06:18:18 UTC
I think there's a possible regression on log record.
On one hand folders supposed to contain logs are NOT created automatically.
When creating them manually, logs start to record when launching the application.
However only KStars/Ekos logs are recorded. INDI logs are nowhere to be found.
Comment 8 Jasem Mutlaq 2017-08-29 07:37:12 UTC
INDI logs are not related to KStars, they are created by INDI server and drivers.
Comment 9 TallFurryMan 2017-08-29 07:47:35 UTC
Yes, true, they are merged into the main log. 
I probably missed a configuration change as I was used to look for them on the server. I remember the checkbox on the INDI panel, and it's probably not checked.
My remark about the need to create the ".local" folder holds however. Unless it is KDE related? I recently switched to a Lubuntu non-kde by default for lightness.
Comment 10 Jasem Mutlaq 2017-08-29 07:51:14 UTC
I thought .local is standard, even in non-KDE environments.
Comment 11 TallFurryMan 2017-08-29 07:53:17 UTC
Sorry I meant the full path to the logs folder.
From memory that is ".local/kstars/logs" or something? "logs" doesn't exist by default.