Bug 364655 - git master 2016-06-22 - re-creation of proxy files...
Summary: git master 2016-06-22 - re-creation of proxy files...
Status: RESOLVED FIXED
Alias: None
Product: kdenlive
Classification: Applications
Component: User Interface & Miscellaneous (show other bugs)
Version: unspecified
Platform: Arch Linux Linux
: NOR wishlist
Target Milestone: ---
Assignee: Jean-Baptiste Mardelle
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-06-23 06:33 UTC by Evert Vorster
Modified: 2016-06-25 05:36 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Evert Vorster 2016-06-23 06:33:40 UTC
Hi there. 

It seems that the default for creating proxies has changed. Previously, once a proxy was generated, it stayed generated, and re-used whenever that clip was loaded into another project. This was a great time saver, especially on GoPro footage that tends to be hours long.

The current solution is sub-optimal for my use case, as I tend to re-use GoPro footage often. 

In my mind, a proxy should be generated for a clip, and stored together with the clip, much like the extracted .mlt files for stabilization, or clips that are generated when a zone is extracted, etc. 
Then, when a clip is loaded, it would be easy to check for the existence of a proxy, and then check to see if the proxy matches the desired resolution. It might even be an idea to incorporate the proxy profile name into the filename of the proxy. 

For instance, if you have a clip called GoPro0001.mp4, and your proxy profile is called 720p, then the proxy name could be:
GoPro0001.Proxy_720p.mp4
This would even allow keeping various versions of the proxy, like
GoPro0001.Proxy_360p.mp4
GoPro0001.Proxy_1080p.mp4
and so on. 

I think that keeping proxies are valuable, as they don't change, unlike the project's temporary renders, thumbnails, etc. 

It may even simplify hunting down errors where proxies are used in final renders instead of the original. 

Reproducible: Always
Comment 1 Jean-Baptiste Mardelle 2016-06-23 21:14:09 UTC
In my recent move to more standard XDG data location, I moved the proxy clips inside the project's cache folder. This effectively means that a proxy created in one project will not be available in another project. I agree that depending on your workflow that might not be a good idea.

However I am not really comfortable with putting the proxy clips in the source clip folder. Proxy clips are in my opinion temporary data (more valuable than thumbnails I agree). For example, if the proxy profile is changed in the project, existing proxies should be deleted (just tested this and changing project's proxy profile is in fact not working :-( ). I don't want to start deleting stuff that is mixed with original source clips.

So I would rather put them like we did before all in one folder, maybe $HOME/.cache/kdenlive/proxies/.

This would make proxy clips available cross project. Also not sure keeping different proxy copies is really needed. Proxy clips are not supposed to match the project resolution, they are supposed to be low resolution copies allowing a faster workflow so I don't think it makes sense to have 720P proxies and 1080P proxies.

Would the change to a cross project folder be ok for you?
Comment 2 Evert Vorster 2016-06-24 02:27:13 UTC
Hi there, JB. 

My /home is rather small, and I would symlink out any cache directory to a larger faster disk if not given the option to configure it in the UI. 

I did not mean that the proxy clip should match the project resolution, but that the proxy name should be what the proxy generator is called in Kdenlive. 
I have my proxy generators labelled to what size video they will produce, and I can easily see how the miscommunication happened. My apologies. 

Say I have a 1080p project, with proxies enabled. My proxy generator is called 720p, but it could also have been called MPEGTS... a proxy generated with it should just have the source clip name, and the name of the proxy generator, and maybe the string "proxy" in the name. 

It's good to be careful about deleting stuff. If you filter carefully, and catch empty variables, your chances of actually deleting source footage is quite small. 
When deleting en masse, give a list of files up for deletion, and give the option to cancel as a final check. 

In the end, Kdenlive is mostly your project, and whatever you do with it is fine. If I don't like it I will either learn to live with it, or get a workaround. :)

I think I have an ancient open bug about proxies not being re-created when the proxy generator profile changes. With my scheme, you would not have to delete proxies at all, unless you modified the parameters of a proxy generator. 

Deleting really needs it's own UI, where you can filter and select files, rather than trying to do it automatically. 

Thanks for some awesome software!
Comment 3 Jean-Baptiste Mardelle 2016-06-24 06:09:00 UTC
Git commit b79722496d202132cffe1258255267d02818fdce by Jean-Baptiste Mardelle.
Committed on 24/06/2016 at 06:08.
Pushed by mardelle into branch 'master'.

* Store all proxy clips in the same folder: $HOME/.cache/kdenlive/proxy to allow reusing proxy in different projects
* Proxy are now saved with a reference to size and extension, so you can keep several proxy versions depending on your proxy profile
* Fix proxy clips not rebuild when switching project's proxy profile
TODO: delete proxies when the proxy profile is edited and allow deletion of proxies

M  +26   -2    src/bin/bin.cpp
M  +1    -0    src/bin/bin.h
M  +30   -11   src/doc/kdenlivedoc.cpp
M  +1    -1    src/doc/kdenlivedoc.h
M  +3    -6    src/mainwindow.cpp
M  +1    -0    src/project/jobs/proxyclipjob.cpp

http://commits.kde.org/kdenlive/b79722496d202132cffe1258255267d02818fdce
Comment 4 Evert Vorster 2016-06-24 06:24:16 UTC
Wow, that's a lot of work you put into this one. 

Thank you very, very much!
I'm building it now to test the changes made. I'll put the ticket to remind for the deletion part of it.
Comment 5 Jean-Baptiste Mardelle 2016-06-24 06:30:26 UTC
Still work in progress. Currently I created a subfolder for each profile, for example:

proxy/mpg-640/93184480418410.mpg
proxy/mpg-320/93184480418410.mpg

But I think it is in fact better to add profile info to the proxy filename itself rather than put them in separate dirs, so be prepared for some more changes.

You will notice that I still use a hash for the name of the proxy file. I know it feels more opaque to the user but allows us to keep the proxy even if source clip is renamed and prevents conflicts it 2 source clips have the same name.
Comment 6 Jean-Baptiste Mardelle 2016-06-24 07:01:04 UTC
Git commit 32ccaa68f72b9759a90509fdfbb0072ad9b13fc5 by Jean-Baptiste Mardelle.
Committed on 24/06/2016 at 07:00.
Pushed by mardelle into branch 'master'.

Store proxy profile info in proxy clip name, no more separate folders, fix info & deletion of proxies in cached data widget

M  +11   -0    src/bin/bin.cpp
M  +2    -0    src/bin/bin.h
M  +10   -10   src/doc/kdenlivedoc.cpp
M  +2    -0    src/doc/kdenlivedoc.h
M  +22   -16   src/project/dialogs/temporarydata.cpp
M  +2    -1    src/project/dialogs/temporarydata.h

http://commits.kde.org/kdenlive/32ccaa68f72b9759a90509fdfbb0072ad9b13fc5
Comment 7 Wegwerf 2016-06-24 08:49:41 UTC
Using hashed names for proxy files is actually necessary for my typical projects: some audio and media files have generic recurring names, such as intro.mp4, et cetera. Using the original name in my scenario would cause clashes, followed by a lot of confusion as I regularily work on two or more projects simultaneously. So the current scheme looks quite useful to me. Thank you, Jean-Baptiste, for spending your time on Kdenlive!
Comment 8 Evert Vorster 2016-06-25 05:36:51 UTC
Works perfectly. 

Thanks!