- Java 87.9%
- Python 11.7%
- Shell 0.4%
Darktable's Deepface Companion
Choose from available detectors and models. Compare model-detector-pairs, see tables.
Info
Make sure you have a backup of your pictures before using this program.
Licence: MIT
You are invited to contribute, e.g. code, or install package for Flatpak (Linux), or install packages for Mac and Windows.
The silver bullet would be to show the face area and name (XMP RegionInfo) directly in Darktable.
For now Darkable reads the XMP HierarchicalSubject (face name) only, see details further below.
Acknowledgements
- Deepface for face detection and recognition
- darktable-cli to convert raw images. Supported cameras and file formats with version 4.6.
- Exiftool for reading meta data from images and writing into images
- Icons
- JSON handling
- Slider with double thumbs
- Wikimedia for pictures in screenhots
Requirements
You need to install:
- Java (mandatory)
- Python (mandatory)
- Deepface
- Exiftools
- Darktable
To have the full functionality you need
- to install all of the above
- recommended 16 GB RAM or more
Automated Installation and first Program Start using a Script
Automated Installation
There are install scripts available for Linux.
You will need the version control system git.
Install git
su -c 'apt install git'
Change to your home directory (to just give you an example).
cd ~
Then use git to download the project files.
git clone https://codeberg.org/ojrandom/ddc.git
Change to the repository.
cd ddc
For Debian use install_debian.sh:
bash install/install_debian.sh
This installs everything for you, Java, Python, Deepface with optional detectors, Exiftool, and Darktable.
It is recommended to watch the CPU and RAM.
First Program Start
Start the program under Linux as usual, see below for Gnome...
It is recommended to watch the CPU and RAM during the first program start.
Recommendation: Avoid using Darktable and Darktable's Deepface Companion (DDC) at the same time if you have camera raw pictures in your collection. Why? DDC uses darktable-cli to convert camera raw pictures. Both Darktable and darktable-cli access the same SQLite database. SQLite db does not allow concurrent access.
The program calls a backend script to check what detectors and models are installed...
The detectors and models are checked one by one to avoid to exceed the system RAM.
BE PATIENT, the models are serveral GB big and have to be downloaded once.
Under the hood: The results will be written to a file.
~/.ddc/messages/available_models.csv
Example: The file content below shows that all additional addons where installed too.
models,Facenet512,VGG-Face,ArcFace,SFace,Dlib,Facenet,OpenFace,DeepID,GhostFaceNet
demography,Emotion,Age,Gender,Race
detectors,yolov8,retinaface,mtcnn,fastmtcnn,ssd,opencv,mediapipe,dlib,yunet
Updates
Once Darktable's Deepface Companion is up and running, you can upgrade it in the 'settings' dialog.
Screenshot above: The program is looking for the current versions of Deepface and the optional detectors when you open the settings dialog.
Unter the hood the project files are updated by
git pull
and the backend (Deepface) is updated by upgrade.sh. Basically it does...
pip install --upgrade deepface
pip install --upgrade ultralytics
...
Manual Installation
Under MS Windows and Mac the installation was not tested but should work because Java, Python, Exiftool and Darktable can be installed on these OSs. You are invited to contribute.
Under Linux you might update and upgrade first, for Debian...
su -c 'apt update && apt upgrade'
Deepface - Manual Installation
You need Deepface to detect and recognize faces in pictures.
Without Deepface installed, you are still able to display faces read from pictures directly (or their sidecar files) and to write names; see comments on Exiftool further below.
Java - Manual Installation
The frontend (GUI... graphical user interface) of the program is written in Java. To install Java manually, open a terminal and type...
su -c 'apt install default-jre'
Python and Deepface - Manual Installation
The backend of the program is written in Python. To install Python and Deepface manually, open a terminal and type...
su -c 'apt install python3-pip python3.11-venv'
Optionally if you want to use dlib...
su -c 'apt install cmake'
Install the Deepface packages in a virtual environment...
Step 1: Create and activate a virtual environment...
python3 -m venv ~/.ddc/deepface && source ~/.ddc/deepface/bin/activate
Make sure to use the same name 'deepface' for the virtual environment.
If you do not use a virtual environment, do not forget to deactivate the checkbox 'use venv' in the settings dialog of the frontend, see screenshot below.
Step 2: Install Deepface and optional modules in this virtual environment.
pip install deepface mediapipe dlib facenet-pytorch ultralytics
Warning: You probalby need more than 16 GB of RAM to install dlib.
The optional detectors are:
- dlib: 'pip install dlib', needs more than 16 GB of system RAM during installation
- FastMtcnn: 'pip install facenet-pytorch'
- MediaPipe: 'pip install mediapipe'
- Yolo: 'pip install ultralytics'
For later upgrades...
pip install --upgrade deepface mediapipe dlib facenet-pytorch ultralytics
Leave the virtual environment
deactivate
Troubelshooting
At the time of writing (04-2024) an error shows...
ModuleNotFoundError: No module named 'tf_keras'
Fix the error by downgrading keras and tensorflow...
pip install --upgrade tf-keras==2.15 tensorflow==2.15
At the time of writing (03-2025) the face recognition stopped to work after an update of Debian testing. An install/update of deepface failed with errors like...
ERROR: Could not find a version that satisfies the requirement mediapipe (from versions: none)
The solution was to downgrade the Python version, for example from Python 3.13 to Python 3.9.13.
For more details see the wiki.
Exiftool - Manual Installation
You need Exiftool to read and write metadata from and to pictures.
Read
- region info and names of faces
- names as hierarchical subjects
- date and time a picture was taken
Write
- region info and names of faces
- names as hierarchical subjects
How to check if Exiftool is installed? Open a terminal and type...
exiftool -ver
This prints the version of Exiftool. How to install Exiftool?
For Debian Linux...
su -c 'apt install libimage-exiftool-perl'
For Windows, Mac see.
Darktable - Manual Installation
You need Darktable if you want to use camera RAW pictures.
How to install Darktable?
For Debian Linux...
su -c 'apt install darktable'
For Windows, Mac see.
Download and Start Darktable's Deepface Companion - Manual Installation
To download the program itself
- Download the project files as zip and unzip, or
- Click through the repository, or
- Download via the version control system GIT
To use git...
First install git
su -c 'apt install git'
Change to your home directory (to just give you an example)
cd ~
Then use git to download...
git clone https://codeberg.org/ojrandom/ddc.git
Change to the repository
cd ddc
Git makes it easy to update the Darktabe's Deepface Companion...
cd ~/ddc
git pull
To start the program...
java -jar ddc.jar
Log Messages
Write debug messages for both Java frontend and Python backend...
java -jar ddc.jar --verbose
The logfiles are
~/.ddc/log/java.log
~/.ddc/log/python.log
If you want to set the log level for the Python backend only. Look for the file
~/.ddc/preferences.props
and set the log level in this line
org.ojrandom.ddc.MainFrameExif.log_level_python_NORMAL_0,_TRACE_1,_DEBUG_2,_DATA_3,_ALL_4=1
Make sure to omit the "--verbose" parameter in the command
java -jar ddc.jar
because this would overwrite the log level for Python to "2" = DEBUG.
Use the Program - Basics
Add Pictures
To add pictures
- drag&drop pictures or directories into the program window or
- open a file dialog
The program will start to
- Convert camera RAW pictures to JPGs.
- Read metadata of pictures or their sidecar files
- region and name of faces (XMP RegionInfo)
- date the photo was taken
- Read the files faces.json and demography.json where the program has written to all results of previous runs. (You should backup these files along with your pictures.)
The taskbar shows what's going on, below the progress of reading metadata from pictures or XMPs...
...The program will stop to read metadata as soon as you start the face detection. That's because the database SQLite does not allow concurrent access. In our case
- the Java frontend writes metadata (dates, face regions and names) to the database and
- the Python backend reads and writes to the database too.
Progress indicator in taskbar while converting RAW images to JPG...
Start Detection of Faces
To start the face detection and recognition...
You can stop the backend before it has finished...
Set Names
Set names...
Recognize Faces
Start again to find (recognize) a persons in other pictures.
The programm writes also recognized faces into the pictures / sidecar files if you select the toogle button as shown below...
Warning: This action can not be made undone. But you can
- correct names and
- overwrite the faces.
So make sure the recognition produces acceptable results before using this feature.
Search
Dates and Names
Brigitte AND Jaques together
The program will do an AND search, e.g. Bardot AND Charrier will find pictures that show both Brigitte Bardot and Jacques Charrier.
Dates - Last Modified versus Picture Taken
Per default the date is the last modified time (file system). Toggle the button to use the date the picture was taken. Often this date was removed from pictures for privacy reasons or by conversions between formats.
Facial Attributes (Demography)
Activate
(Do not expect too much from the accuracy of the results.)
Search for facial attributes as in this example for a happy white woman between 25 and 45 years...
Darktable Settings
Darktable presents the dialog below if updated sidecar files are found.
If you have set (or confirmed) the name of a person it will show up like this:
For details see explanation on Exiftool.
Program Settings
Proposed Settings to start with
Fast and overall accurate. The (more fine granular) "confidence" produced by yolov8 might help to sort out faces, and thus tends to result in less false positives for large picture collections.
- yolov8 - Facenet512 - euclidian_L2=1.04
If you prefer accuracy over speed and switch off most filters for the training data...
- retinaface - Facenet512 - euclidian_L2=1.04, or
- retinaface - Facenet512 - cosine=0.55, or
- retinaface - VGG-Face - euclidian=0.95 or euclidian_L2=0.95
Feel free to experiment and come up with settings that work best for you.
Settings for larger Picture Collections
The following settings get interesting if you work with larger picture collections.
Take care of the minimum sizes of faces to be detected....
Why?
In this step all the tasks that consume much time, cpu and ram are done:
- detect face
- generate face representation (multidimensional vector)
- calculate confidence, eye distance, sharpness
The good thing: This is done ONCE.
The search for a face is just comparing face representations (vectors) of faces that meet certain conditions:
- date
- confidence (highly depended on detector)
- sharpness
- width of face
- eye distance
You can change these conditions at any time and start another search.
To get information on how to tune the settings, click onto a face to view it's values
- confidence (highly depended on detector)
- sharpness
- width of face
- eye distance
Discussion
At the moment of writing the sharpness is calculated...
face = img[y:y+h, x:x+w]
gray = cv2.cvtColor(face, cv2.COLOR_BGR2GRAY)
sharpness = np.max(cv2.convertScaleAbs(cv2.Laplacian(gray, 3)))
It seems to produce more useful results overall than these two other methods...
sharpness = np.mean(cv2.Canny(gray, 50,250))
sharpness = cv2.Laplacian(gray, cv2.CV_64F).var()
You are invited to come up with a better solution. The code is in calculate_sharpness extract.py. Find discussions on the methods here and here.
Some Background on Face Detection and Recognition
A modern face recognition pipeline consists of 5 common stages: detect, align, normalize, represent and verify.
The settings page reflects the 5 stages...
- detect: setting: a) choose from a list of available detectors, and b) detector settings for minimum size of faces to be detected in pixel and in percent
- align: no setting available
- normalize: no setting available
- represent: setting: choose from a list of available models
- verify: setting: choose from a list of available distance metrics and change the default thresholds
Detector - Model - Distance Metric - Threshold
The main program window will show results for one single detector-model-pair only.
If you change the detector-model-pair, do no forget to start the backend again.
Detector - What it does
A detector (yolov8 in the example below) will detect and cut off faces...
...in a picture...
Depending on the detector the additional steps are
- alignment
- normalization
For more details see.
You can define minimum sizes of faces to be detected. Smaller faces will be ignored.
Changing these two values does not effect pictures (and faces) a detector has processed already. A simple way to run a detection again is to delete faces from the database...
Hidden feature: How to write faces that are cut out by the detector as show in the screenshots further above.
Double click onto the taskbar as seen in the screenshot below. The backend will write all detected faces of the next run - and only the next run - into the directory
~/.dcc/faces
Model - What it does
A model (Facenet512, VGG-Face, ArcFace,...) takes the face from a detector and generates a multidimensional vector, called representation or embedding. The length of the vector depends on the model. Below is a represention of Brigittte Bardot's face by ArcFace. It contains 512 floats.
-0.1507522165775299 0.10644867271184921 0.16636240482330322 -0.20085033774375916 0.13641738891601562 0.21131213009357452 0.07841136306524277 0.09227259457111359 -0.07799510657787323 0.054123178124427795 0.12919586896896362 -0.24851877987384796 -0.21473953127861023 -0.14870214462280273 -0.2882217466831207 0.11269964277744293 -0.028646040707826614 -0.024518344551324844 0.03603978455066681 -0.332613080739975 -0.069401815533638 0.04467102140188217 0.24837370216846466 -0.14879679679870605 0.03479159623384476 0.02125582844018936 -0.20345911383628845 0.024075262248516083 0.018966030329465866 -0.19389286637306213 -0.19772294163703918 0.012669045478105545 -0.03523429483175278 0.008643034845590591 -0.05167267471551895 -0.09624893963336945 0.08120505511760712 0.01816486194729805 0.12689095735549927 0.1426890790462494 0.12057355046272278 0.03072037175297737 0.06143832206726074 0.07322122156620026 -0.06702066212892532 -0.28818026185035706 -0.07872183620929718 0.13170026242733002 -0.10484801977872849 -0.04916970059275627 0.1880270093679428 0.08184927701950073 -0.09990928322076797 -0.33760684728622437 -0.17320258915424347 0.1795213222503662 0.17804330587387085 -0.12485195696353912 -0.1037316769361496 -0.13089972734451294 0.14982926845550537 0.16705651581287384 0.06311194598674774 -0.23448975384235382 0.24045898020267487 0.027878761291503906 0.1541891247034073 -0.38070011138916016 -0.012173713184893131 0.0008213635301217437 0.10632570087909698 0.05701422691345215 0.005696340464055538 -0.0668611153960228 -0.3117189109325409 0.05028689280152321 0.1289585679769516 0.12108305096626282 -0.011235770769417286 -0.1351291835308075 -0.017610635608434677 -0.014757946133613586 0.03441938757896423 -0.06725314259529114 0.11628571152687073 0.16386926174163818 -0.1726088970899582 -0.07605179399251938 -0.03823022544384003 0.12528827786445618 0.1454046070575714 -0.011175083927810192 -0.06526698172092438 0.15341514348983765 -0.23985923826694489 0.020440582185983658 0.10978319495916367 -0.16470760107040405 0.21423158049583435 0.035984478890895844 0.06097329407930374 0.06495784223079681 0.08980787545442581 -0.01758900098502636 0.20221464335918427 0.25195056200027466 0.124274343252182 0.24287402629852295 -0.005929925944656134 -0.06382106244564056 0.1099344864487648 0.02769046649336815 0.09732621163129807 0.14924302697181702 0.015032706782221794 0.16838043928146362 -0.17711889743804932 -0.3333938717842102 0.3144294023513794 -0.06142628937959671 0.0684928223490715 -0.08290739357471466 -0.11211231350898743 -0.02298089489340782 -0.006862804293632507 -0.2679087519645691 0.004045800305902958 0.20269621908664703 -0.10197412967681885 0.38325533270835876 -0.003350699320435524 -0.03649201989173889 0.06105433404445648 -0.033733952790498734 -0.043835125863552094 -0.027866220101714134 0.03162357956171036 0.0903366282582283 0.43959811329841614 0.11632244288921356 0.07430676370859146 -0.08505959808826447 0.09498604387044907 0.08640249073505402 -0.027155719697475433 -0.06422708183526993 0.17042002081871033 -0.10106413811445236 0.1089707762002945 0.05321715772151947 -0.045227937400341034 0.11472900211811066 -0.05700601637363434 -0.11753926426172256 -0.13733506202697754 0.20347344875335693 0.10027375817298889 -0.03979342430830002 -0.012665625661611557 0.08749467134475708 0.03302755206823349 -0.01312285102903843 -0.20873886346817017 0.06376330554485321 0.08016959577798843 -0.13752608001232147 -0.14772504568099976 -0.1819135546684265 -0.315667062997818 -0.0812205970287323 0.3002954125404358 -0.15718142688274384 -0.15746921300888062 0.2614154815673828 -0.014574557542800903 -0.02932816743850708 0.014374012127518654 0.08739123493432999 -0.10393965244293213 0.11112555861473083 -0.03115697391331196 -0.23205535113811493 -0.2460319846868515 -0.1041032150387764 0.026223445311188698 0.01753084547817707 0.12151439487934113 0.10079660266637802 -0.06428595632314682 -0.11986757069826126 0.20049342513084412 0.18343020975589752 -0.09353023022413254 -0.026341427117586136 0.23840925097465515 0.32338187098503113 0.028589198365807533 -0.0794629454612732 -0.07305850833654404 0.13466335833072662 0.053892310708761215 0.31542283296585083 0.2827756404876709 -0.07492711395025253 -0.008916620165109634 -0.017431335523724556 -0.23638547956943512 0.24319005012512207 -0.26907819509506226 0.0377027690410614 -0.040804672986269 -0.06069974973797798 -0.03135532885789871 -0.05568736419081688 0.12602867186069489 -0.13932955265045166 -0.09535728394985199 0.03887559846043587 -0.008352230302989483 -0.010711132548749447 -0.01790476404130459 0.19983230531215668 -0.01893114112317562 0.06687531620264053 0.1367560625076294 0.09406091272830963 -0.25066298246383667 -0.15604817867279053 0.12281902134418488 -0.019094668328762054 -0.00360923632979393 0.027470901608467102 0.04514884203672409 -0.12321417033672333 0.11392609775066376 0.09290620684623718 0.10461821407079697 0.0018251659348607063 0.040152378380298615 0.020828736945986748 0.2045879364013672 -0.22299957275390625 -0.06893196702003479 0.12036115676164627 0.013537619262933731 0.032317399978637695 -0.12646958231925964 -0.13597014546394348 0.3505982458591461 -0.035777732729911804 -0.2961054742336273 0.11768462508916855 0.05624621361494064 0.07822993397712708 0.06516047567129135 0.1601024866104126 0.020654216408729553 0.17184188961982727 -0.029512308537960052 -0.21932949125766754 0.08358342945575714 0.08099365234375 0.32322248816490173 0.19374272227287292 0.004623560234904289 -0.30636805295944214 -0.04382973536849022 -0.09415486454963684 -0.033258672803640366 0.06268031150102615 0.08798806369304657 -0.2621958553791046 -0.07060398161411285 0.15111058950424194 -0.1884179264307022 0.07675564289093018 0.246672123670578 0.1762048304080963 -0.07453955709934235 0.0871090218424797 -0.06642098724842072 0.013501439243555069 -0.05305338650941849 0.04595085605978966 -0.07593151926994324 0.1649615317583084 0.10419350862503052 -0.04312421754002571 0.017659328877925873 -0.24797287583351135 -0.19023871421813965 -0.12498864531517029 0.12386095523834229 -0.062148839235305786 -0.07461570203304291 -0.08460785448551178 0.12619853019714355 0.2805732190608978 -0.09784522652626038 0.010104748420417309 0.05613849312067032 -0.029250510036945343 -0.1783324033021927 0.03891514986753464 0.009782659821212292 -0.06167210265994072 0.26013997197151184 -0.04611312597990036 -0.1580944061279297 0.011692697182297707 0.014968447387218475 -0.11795603483915329 0.12023823708295822 0.1294676959514618 -0.256094753742218 0.059003688395023346 0.05787768214941025 -0.25522634387016296 0.012784406542778015 0.14864183962345123 -0.021433550864458084 0.3876439929008484 0.009681733325123787 -0.02716796286404133 0.04353249818086624 -0.1026572436094284 0.09020930528640747 0.32391154766082764 -0.11091437190771103 0.1256718784570694 -0.16253070533275604 0.053998760879039764 -0.0004847962409257889 -0.10641917586326599 0.1897924840450287 -0.27838870882987976 0.18340696394443512 -0.19172441959381104 -0.42361584305763245 0.048830606043338776 -0.3553275465965271 0.17278511822223663 0.3162100613117218 0.18911638855934143 -0.2881825864315033 -0.08557673543691635 -0.0077012162655591965 -0.27251580357551575 -0.01794370450079441 0.14268377423286438 0.12700876593589783 -0.10156956315040588 -0.0825706422328949 -0.11109267920255661 -0.35889506340026855 -0.016574449837207794 -0.2959628403186798 -0.19211116433143616 -0.14855548739433289 -0.13573908805847168 0.04814746975898743 0.12199203670024872 0.1367264986038208 0.28752604126930237 0.007124496623873711 0.23474448919296265 -0.22610917687416077 0.3621109426021576 0.08496185392141342 -0.043305639177560806 -0.046303026378154755 -0.031399913132190704 0.031083011999726295 -0.052136652171611786 0.10667471587657928 -0.006952858529984951 -0.27124449610710144 0.1438112109899521 -0.06948913633823395 -0.021916892379522324 -0.14295653998851776 -0.12810596823692322 -0.08717750757932663 0.22628876566886902 0.30750009417533875 -0.08133987337350845 -0.18289445340633392 -0.05688611790537834 -0.13174396753311157 0.03699856996536255 0.24333713948726654 -0.10501306504011154 0.09491296857595444 0.20181529223918915 0.17027446627616882 0.0616217777132988 0.13454791903495789 -0.058057527989149094 0.0006804205477237701 0.18260584771633148 -0.2648466229438782 -0.02506742812693119 -0.04504009708762169 0.07334963977336884 -0.15601257979869843 0.04095267131924629 0.07692281901836395 -0.1786699891090393 0.17696787416934967 -0.12048609554767609 -0.08992348611354828 -0.029878992587327957 -0.030783530324697495 -0.2985697388648987 0.04669687896966934 -0.38920220732688904 -0.06587663292884827 -0.105284683406353 0.11612103134393692 -0.1436300426721573 -0.06751015037298203 -0.07263731211423874 0.059816744178533554 -0.12212423235177994 -0.020010408014059067 0.07969047129154205 0.11178172379732132 0.11266111582517624 -0.21907860040664673 -0.3193401098251343 -0.15480278432369232 -0.02522195316851139 0.015751369297504425 -0.1943005919456482 0.060452498495578766 0.15539981424808502 -0.07071623206138611 0.21059346199035645 -0.09310675412416458 0.26978570222854614 -0.1487477719783783 -0.11950097233057022 -0.09268397837877274 -0.008734793402254581 -0.042502954602241516 -0.13967444002628326 0.260219007730484 -0.04365608096122742 0.007656067609786987 -0.2742755115032196 0.07259612530469894 -0.143938809633255 -0.1497926563024521 -0.12346705794334412 0.14574697613716125 -0.10294898599386215 -0.12203621864318848 0.07150642573833466 -0.21570545434951782 -0.010902742855250835 0.2912830412387848 -0.11013668775558472 0.18695847690105438 0.18695203959941864 -0.08964511752128601 0.012960018590092659 -0.09975399821996689 0.2735804319381714 0.14325568079948425 0.11992636322975159 0.030361389741301537 0.07109463214874268 -0.23505300283432007 0.033754028379917145 -0.25786468386650085 0.051138658076524734 -0.012879357673227787 -0.2482716292142868 0.11318746209144592 0.10455406457185745 -0.008434411138296127 -0.06359345465898514 -0.06784755736589432 -0.1812153458595276 -0.12134749442338943 -0.01580974832177162 -0.1502992957830429 -0.17404013872146606 0.32265546917915344 -0.19792091846466064 -0.0040230536833405495 0.03398857265710831 0.12231706827878952 -0.22685186564922333 0.027347661554813385 0.17875322699546814 -0.002978881821036339 -0.15458913147449493 0.05608511343598366 0.1733793020248413 -0.2634865641593933 0.010002431459724903 0.18746455013751984 -0.01773480698466301 -0.21092776954174042 0.27708566188812256 -0.18009983003139496 0.01957426220178604 -0.031101014465093613 0.021790748462080956 0.22839631140232086 -0.26474347710609436
Distance Metrics and Thresholds - What they do
To recognize a person the representations are compared. The representations of the same person should be similar. Similarity is calculated by different metrics
- Cosine Similarity
- Euclidean Distance
- Euclidean L2 Distance
The similarity (distance) that is used to decide if a representation is likely to belong to the same person is called a threshold.
For more on thresholds see
Tables Probe and Compare
Activate "Run Compare and Probe" to produce some statistics.
Why?
- to give some insights,
- to provide a starting point for changes.
Advice
Do not cling too much to the figures in the tables. Always check with:
- Settings for training data, especially confidence (highly dependend on detector), but also blur, eye distance, face width.
- Real outcome in the main window. Example: Despite the table "Compare" shows no false negative for a detector-model-pair you might still see false negatives in some pictures.
Table Probe
Table Probe - What Data is collected and processed?
Faces
- that carry a name the user has set or confirmed the name
- that belong to the detector-model-pairs as defined in the settings page
- filtered by dates from-to as defined by active search values
- filtered by confidence, sharpness, eye distance, size of face as defined in the settings page
To illustrate it for the more technical-interested reader, see function get_names in file dabase.py...
"SELECT faces.id, faces.name, faces.model, faces.detector, faces.embedding"
+ " FROM faces"
+ " LEFT JOIN pictures ON pictures.id=faces.file_id"
+ " WHERE"
+ " model = '" + model + "'"
+ " AND detector = '" + detector + "'"
+ " AND name != ''"
+ " AND embedding != ''"
+ " AND unknown != 1"
+ " AND ignore != 1"
+ " AND sharpness >= " + str(min_sharpness)
+ " AND eye_distance >= " + str(min_eye_distance)
+ " AND confidence >= " + str(min_confidence)
+ " AND w >= " + str(min_face_width)
+ " AND pictures." + columnDate + " >= '" + date_from + "'"
+ " AND pictures." + columnDate + " <= '" + date_to + "'"
+ ";"
The corresponding filter in the GUI...
Table Probe - Values
As mentioned, only faces are processed that carry a name. All of the steps below are applied for each distance metric.
Steps
- Compare a representation of a face with the representation of all other faces. Then...
- Column "count false positives": Does the most similar representation carry the same name? If not it is counted.
- Column "same person": Find the distance to all representations carrying the same name. Get the average distance.
- Column "nearest same person": Find the distance to the most similar representation carrying the same name.
- Column "nearest other person": Find the distance to the most similar representation carrying not the same name.
- Column "all faces": Count of faces compared to each other.
- Repeat with every representation
- Calculate average values for columns same person, nearest same person, nearest other person. These averages are displayed in the table.
- Column "% difference": Calculate the difference in percent between the average of "same person" and "nearest other person"
The code is in probe.py.
Table Compare
Table Compare - What Data is collected and processed?
The data collected is the same as in "Probe", see above.
Difference to "Probe"
Apply the current thresholds as defined in the settings...
Table Compare - Values
As mentioned, only faces are processed that carry a name.
Steps
- Compare a representation of a face with the representation of all other faces. Apply the threshold. Then...
- Column "positives": Does the most similar representation carry the same name? If YES it is counted.
- Column "false_positives": Does the most similar representation carry the same name? If NO it is counted.
- Column "nothing": Was any representation found (within the threshold)? If NO it is counted.
- Column "all faces": Count of faces compared to each other.
- Repeat with every representation
The code is in compare.py.
Technical Details
Exiftool - Reading and Writing Metadata
The xmp:HierarchicalSubject does the magic to hand over face tags to Darktable.
The programm reads and writes metadata from/to
- a picture, or
- a sidecar file (*.xmp) of a picture if it does exist. This is the preferred method.
Why?
- If the information about a face is stored inside the picture itself it will never get lost.
- If you name faces they are stored in hierarchical tags in the pictures or sidcar files (for details see further below). The photo editor Darktable is able to show hierarchical tags and to use them to filter and search pictures. In Darktable make sure to activate the preferences storage - Look for updated XMP files on startup, see screenshot below.
The program does not write into camera RAW pictures. This is possible but is seen as too risky. Instead, a sidecar file is created. (Darktable creates a side care file for each picture by default anyway.)
What and how?
- The "frames" around faces and the names are stored inside the "RegionInfo", see.
- Hierarchical tags like "face|Brigitte Bardot" will be stored as "HierarchicalSubject" as part of the picture's metadata. Darktable shows hierarchical tags and is able to use them to filter and search pictures.
The underlying command to read information about faces from the image:
From the sidecar file of the image...
exiftool -struct -j -xmp:all path path-to-your/image.jpg.xmp
...or from the image itself if a sidecar file is not found...
exiftool -struct -j -xmp:all path path-to-your/image.jpg
The above command will give you something like...
[
{
"RegionInfo": {
"AppliedToDimensions": {
"H": 2848,
"Unit": "pixel",
"W": 4288
},
"RegionList": [
{
"Area": {
"H": 0.17,
"Unit": "normalized",
"W": 0.15,
"X": 0.3,
"Y": 0.4
},
"Name": "Brigitte Bardot",
"Description": "A beauty",
"Type": "Face"
},
{
"Area": {
"H": 0.09,
"Unit": "normalized",
"W": 0.06,
"X": 0.5,
"Y": 0.6
},
"Name": "Anna von Trio",
"Description": "some girl",
"Type": "Face"
}
]
},
"HierarchicalSubject": [
"face|Brigitte Bardot",
"face|Anna von Trio"
]
}
]
To write write faces into the sidecar file or the picture itself...
exiftool -RegionInfo="{AppliedToDimensions={W=4288,H=2848,Unit=pixel},RegionList=[{Area={W=0.15,H=0.17,X=0.3,Y=0.4,Unit=normalized},Description=A beauty,Name=Brigitte Bardot,Type=Face},{Area={W=0.06,H=0.09,X=0.5,Y=0.6,Unit=normalized},Name=Anna,Description=some girl}]}" path-to-your/image.jpg.xmp
To add a hierarchical tag...
exiftool -struct -xmp:HierarchicalSubject+="face|Brigitte Bardot"" path-to-your/image.jpg.xmp
To remove a hierarchical tag...
exiftool -struct -xmp:HierarchicalSubject-="face|Anna von Trio"" path-to-your/image.jpg.xmp
For more information on
- Metadata of pictures see and there the linked official MWG specification
- Serialisation of this data see
- Hints on hierarchical tags see
Webpage of Exiftool for more information.
To view all metadata:
exiftool path-to-your/image.jpg
View Tags and Region Info in other Programs
Some examples on how to have a look what's going on under the hood.
Eye of GNOME
You can check the tag and region data in programs like Eye of GNOME (eog)
DB Browser for SQLite
Have a look into the database that both the Java frontend and Python backend use, screenshot of DB Browser for SQLite:
Darktable - Converting RAW Images
RAW pictures are converted to JPEG and stored in the directory ~/.ddc/exported.
darktable-cli picture.CR3 . --out-ext ".jpg"
CR3 is Canon's raw format. Many other cameras are supported.
List of formats Darktable can handle with version 4.6...
Preview Images
To speed up the previews all pictures are scaled down and stored as low resolution images in the directory ~/.ddc/lowres.
Long-Term Storage
The program will write two files in every directory where in finds faces:
- faces.json - holds all face representations, names, regions of faces,...
- demography.json - holds all data about age, gender, race and emotion.
You should backup both files along with your pictures.
The programm reads these two files when you import your pictures on a different computer for example.
Other programms should be able to handle the data as well thanks to the standard format JSON used.
In faces.json a face lookd like this
{
"w": 0.2743055555555556,
"x": 0.4236111111111111,
"y": 0.5960648148148149,
"h": 0.4652777777777778,
"name": "",
"name_recognized": "Brigitte Bardot",
"detector": "yolov8",
"model": "Facenet512",
"file_original": "Tradita_-_Brigitte_Bardot.png",
"confidence": 0.83,
"sharpness": 113,
"eye_distance": 0.41,
"created": 1713178208495,
"time_shot": 0,
"date_shot": "",
"time_file": 1709987502731,
"date_file": "2024年03月09日",
"named": -1,
"distance": 0.8524836719177552,
"recognized": 1713178912281,
"imported_xmp": 1,
"unknown": -1,
"imported": 1,
"ignore": -1,
"id": 1213,
"export": 1,
"export_demography": 0,
"is_xmp": 0,
"message": "",
"file_id": 1,
"distance_metric": "euclidean_l2",
"imported_demography": 1,
"embedding": "-0.4103320240974426 2.0419790744781494 0.7966465353965759 -0.782355785369873 -0.02778051048517227 0.7400485873222351 -1.3580586910247803 0.373167484998703 1.5517343282699585 0.49372994899749756 -0.3957640528678894 1.7124834060668945 0.8587837219238281 -0.20644888281822205 -2.777188301086426 1.281241774559021 2.181084156036377 1.6427803039550781 0.1514243334531784 0.2480032593011856 -0.39725351333618164 -1.6468416452407837 0.10768511891365051 -0.17916908860206604 ... -1.223793387413025 1.0967347621917725"
}
The embedding contains 512 float values in the original data.
In demography.json a face looks like this
{
"w": 0.2604166666666667,
"h": 0.4583333333333333,
"x": 0.421875,
"y": 0.5879629629629629,
"age": 33,
"dominant_gender": "Woman",
"dominant_race": "white",
"dominant_emotion": "neutral",
"gender": "{Woman: 99.99557733535767, Man: 0.004422274651005864}",
"emotion": "{angry: 41.597030568476846, disgust: 1.9412437341609813e-06, fear: 2.4004639058752173, happy: 0.0005611055953111349, sad: 9.071191063977354, surprise: 0.00028217954242260295, neutral: 46.930468448344655}",
"race": "{asian: 0.07937956252135336, indian: 0.06576888263225555, black: 0.002156127811758779, white: 87.31628060340881, middle eastern: 6.118746101856232, latino hispanic: 6.417664885520935}",
"file_original": "Tradita_-_Brigitte_Bardot.png",
"created": 1713167967729,
"time_file": 1709987502731,
"date_file": "2024年03月09日",
"date_shot": "",
"time_shot": 0,
"imported_xmp": 1,
"imported": 1,
"id": 1,
"export": 0,
"export_demography": 1,
"message": "",
"file_id": 1,
"detector": "retinaface",
"imported_demography": 1
}
Location of Database and Temporary Files
You can change the location and name of the database.
The directory where the database is located is used to write
- converted raw files
- low resolution pictures for previews
Troubleshooting
Taskbar: Counter for Metadata Is Stuck
Observation:
Taskbar > counter for metadata > is stuck
Effects:
- The date when a picture was shot can not be read and stored (used to filter pictures).
- Faces stored in the picture or its sidecar file (xmp) can not be read.
Possible cause:
Exiftool hangs.
Solution:
Check the processes of your OS for exiftool.
Scrolling is Dead Slow
Observation:
Scrolling is really slow.
Cause:
The low resolution images are not created yet. The progress is shown in the taskbar...
Solution:
Just wait.
Backend Stops
Possible reason: Maximum system RAM exceeded a certain value, default 80%.
The more faces in a picture the more RAM is consumed.
Example: 40 faces need about 10 GB of RAM using
- detector = yoloV8
- model = Facenet512
- facial attributes switched on.
The more detectors and models are used the more RAM is consumed.
Solution:
- Change the value to a higher one under Settings > Database > Maximum RAM
- Use only one detector and one model
- Switch facial attributes off (age, gender, emotion, race)
- Use detectors and models that consume less RAM
Author
OJ Random (Tom Wiedenhöft), built 2025年07月06日.