I save and backup all the photos on a Synology NAS instead of using one of the online providers. However Synology Photos doesn’t have good search capabilities. So I built a project to search through the images using natural language captions, and found that it works really well.
I have published the project publicly with a GPL-3.0 license - synology-photos-nlp-search. Anyone is welcome to use and contribute to the project.
It was really cool to see that I can try two search terms like food and eating, and the embeddings model would understand the difference and provide relevant images for both.
The project runs the model and stores any model-related files locally, so besides downloading the model and necessary python packages, there are no API calls being made to any outside services. I have containerized the application to make it easier to deploy and use. That said, some programming experience might be needed as it’s not an open-and-use application.
This is my first major project that I am publishing, and would welcome any feedback for improvements from the community.
This is really cool!
I saw that Synology Photos is getting some sort of object recognition in an upcoming update. But I love the idea of running your app on a container on my Mac (with lots of cores to run the ML code) while keeping the photos homed on the NAS. Definitely going to play with it :)
There’s been some work getting CLIP to run in pure C++ with quantization in GGML, and there’s a curious FasterViT model I’ve seen months ago, so hopefully this can be made faster to inference and easier to host as one binary soon enough.
