Bug 499230 - Auto-Tags is not able to tag textured image, only photography. Clarifai or Nyckel AI models could fix this issue.
Summary: Auto-Tags is not able to tag textured image, only photography. Clarifai or Ny...
Status: REPORTED
Alias: None
Product: digikam
Classification: Applications
Component: Tags-AutoAssignement (other bugs)
Version First Reported In: 8.6.0
Platform: Microsoft Windows Microsoft Windows
: NOR wishlist
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2025-01-28 06:00 UTC by 44.yoel.44
Modified: 2025-04-12 05:38 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed/Implemented In:
Sentry Crash Report:


Attachments
A texture that should have easily tagged, but is not even close. (801.46 KB, image/png)
2025-01-28 06:00 UTC, 44.yoel.44
Details

Note You need to log in before you can comment on or make changes to this bug.
Description 44.yoel.44 2025-01-28 06:00:23 UTC
Created attachment 177746 [details]
A texture that should have easily tagged, but is not even close.

I'm trying to use this software to organize my texture asset library, but the AI models can only identify objects, not textures, so I'm getting gibberish instead.


STEPS TO REPRODUCE
1. Get a few textures (3D Total has a bunch of free packs for example).
2. Put them in your library and make DigiKam scan them to add automatic tags (I used the slowest YOLOv5 XLarge).
3. Check the tags that it has automatically added.


OBSERVED RESULT
It identifies a wood texture as "person, car, pastry, ...", just as an example.
Basically, if there is no visible silhouette of an object, it invents the tags, and even when there is a full object, it has issues if it's lacking context.


EXPECTED RESULT
The model should try to interpret the patterns and color of the texture to understand what material could it be.


SOFTWARE/OS VERSIONS
Windows: 
macOS: 
(available in the Info Center app, or by running `kinfo` in a terminal window)
Linux/KDE Plasma: 
KDE Plasma Version: 
KDE Frameworks Version: 
Qt Version: 

ADDITIONAL INFORMATION
Searching a bit online I've found Clarifai and Nyckel may be able to do that, I don't know if they could be integrated into DigiKam, and if so, how to do it.
Nyckel at least offers API access, but I don't know how it works.
If you need some testers I could help, my library is over 60GB with textures of all kinds of materials from all over the world, higher and lower quality, non-tileable and seamless, photoreal and digital, materials and effects, all you can ask for, I probably have.
Comment 1 44.yoel.44 2025-01-28 07:26:20 UTC
Further investigation has led me to this collection of datasets that include a few based on already tagged textures, if only there was an already trained model that could be run locally...

https://github.com/thetoby9944/awesome-texture-datasets
Comment 2 Maik Qualmann 2025-01-28 07:43:28 UTC
Please try the pre-release version of digiKam-8.6.0 with improved auto tags engine and YOLOv11.

https://files.kde.org/digikam/

Maik
Comment 3 Michael Miller 2025-02-01 13:41:50 UTC
Hi Yoel,
I don't think the new autotagging models in 8.6.0 will help, sorry.  The models are general object detection and image classification models, and aren't made for detecting textures.  I'm researching the idea of a specialized model library for 8.7.0 or 8.8.0 where the end user could select additional optional models to download.  I'm looking at specialized models for wildlife, and one specifically for birds.  Adding an additional model for textures is something that can be done relatively easily if we decide to add the specialized model library.  Do you have any links for models in mind as a starting point for me?

Cheers,
Mike
Comment 4 44.yoel.44 2025-02-03 21:57:54 UTC
(In reply to Michael Miller from comment #3)
> Do you have any links for models in mind as a starting point for me?

There is no downloadable pretrained models specific for textures, there is training data, linked in my first reply, and there is a couple of models that can be only accessed on-line (as far as I know).
All I know is that for textures, image classification would be more efficient than object recognition, I was thinking on training a model myself, but that would be my first time doing so and I don't even know were to start from.