Learn how combining OpenCLIP image embeddings with all-MiniLM-L12-v2 text embeddings creates a powerful photo recommendation system. Discover the maths behind late fusion, how a single tunable weight blends visual and contextual signals, and how to build the full pipeline with a lightweight Node.js and Python setup — no database or vector store required.

Discover how OpenCLIP, a free open-source vision model, can automatically categorise, search, and visualise a personal photo library using zero-shot classification and vector embeddings. Learn how to build a semantic photo search engine with FastAPI, pgvector, and vanilla TypeScript, and explore your library as an interactive 3D cluster cloud powered by UMAP and Three.js. All running locally — no API keys, no monthly bills.

The idea for this experimental project, which I am going to demonstrate in this blog article, came to me from the Office Works website. Here's how it all happened. As a hobbyist landscape photographer, I have accumulated hundreds of images over the years. Recently, I decided to print these photos in a 10x12" photo book. I immediately checked the Office Works website to see if they offer such a service, and I was in luck. Office Works has an incredible platform that allows customers to easily create photo books using their online editor. While I was able to accomplish what I wanted, there was one feature that I was missing but would be nice to have the ability to sort all of the images based on their colours. In my case, I uploaded 160 images, and manually sorting all of them was a bit tedious.