← Library · Why it's better · How it works
Anonymised notes and results behind the deck. Participants are P1 to P6; no names or contact details are kept. Quotes are paraphrased from moderator notes unless marked otherwise.
Method: 15-minute calls; the participant searched their own phone live while the moderator watched; then the Retrace idea was described. Counts only (n = 6). Quotes are as reported in the moderator's notes (paraphrased unless marked). P4 is an older adult and a low-confidence searcher: the edge of the target segment, kept in and flagged.
| P1 | P2 | P3 | P4 | P5 | P6 | Count | |
|---|---|---|---|---|---|---|---|
| Photos / years | ~10k / 5+ | ~20k / 6-7 | ~1k / 2 (Android) | n.n. | 15-20k / ~10 | 3-4k / 4-5 | |
| Usual way to find photos | scroll | scroll | search | scroll | Google Memories & albums | scroll | scroll first 4 of 6 |
| Placed the photo by an event, trip, activity or period | Shimla trip | COVID period; Ladakh trip | Kerala trip | a 2017 wedding | paragliding 2016; Dubai | n/a (objects, people) | 5 of 5 who described a moment |
| Unsure of month or year | yes | yes | n.n. | n.n. | n.n. | n.n. | 2 noted |
| Raised wrong dates from their own library | WhatsApp photos | phone change | phone changed many times | 3 of 6 | |||
| A search needed rewording or failed first | "suit/blazer/coat" failed | "check" failed, "bank check" worked | 2 of 6 | ||||
| Place/people + thing did not narrow | "cafe in Shimla" | "museum in Ladakh" | "wedding" spread across years | skywalk mostly ok | "kids with grandmother" | 4 of 6 | |
| Object/topic search worked | café (noisy) | prescription | medicinal plants | paragliding | bank cheque (2nd try) | 5 of 6 | |
| Interested in Retrace | yes | yes | yes, excited | yes | yes | yes | 6 of 6 |
| Privacy | worried ("on the internet") | "almost all apps do that" | none voiced | not worried | fine with Google; worried about non-Google AI | yes, "needs some guardrails" | conditions or worry 3 of 6 |
research/mvp_tests/README.md.research/mvp_tests/engine_eval.md.Date of call: 4 Oct 2026. Format: not noted. Phone: Motorola, Android. Main photo app: not noted (gallery search used). Years of photos: at least 5. About 10,000 photos; storage almost full. Age band / city: not noted.
Method note: the moderator asked P1 to use search rather than scroll (P1 normally scrolls). P1 asked what to type; the moderator answered "Whatever you feel like you can write."
Date of call: 4 Oct 2026. Format: not noted. Phone: Android, current phone about 2 years. Apps: Google Photos and the phone gallery. Photos: about 20,000; phone almost full. Years of photos: 6 to 7. Age band / city: not noted.
Date of call: 4 Oct 2026. Format: not noted. A quick call. Phones: iPhone (Apple Photos) and Android (Google Photos). The Android phone has about 2 years of photos, about 1,000. Age band / city: not noted. Self-described as tech-savvy.
Date of call: 5 Oct 2026. Format: not noted. Older adult ("elderly"). Phone, app, years and number of photos: not noted. Phone changed many times. Segment note: older and a low-confidence searcher, so an edge of the target segment (long-time users with years of photos). Report separately where it matters, never hide.
Date of call: 5 Oct 2026. Format: not noted. Phone: Android, Google Photos with cloud backup to their Google account (which they don't like). Photos: 15,000 to 20,000; about 10 years. Age band / city: not noted.
Date of call: 5 Oct 2026. Format: not noted. Phone: Android; Google Photos and the phone gallery. Photos: 3,000 to 4,000; 4 to 5 years. Age band / city: not noted.
One file per participant: P<n>.tsv is the "Copy results" output of /test, pasted as is.
| When | Change | Sessions run before it |
|---|---|---|
| 5 Oct, after the first dry run | Preview photo blurred and self-hiding (text on the prescription was being copied into search); T3/T4 stories no longer contain the label word; log records Retrace's own result | dry run (excluded) |
| 5 Oct, after P1 | Normal search made fairer: ignores words it doesn't know ("three"), ignores filler words ("when", "was"), shows best partial matches when no photo has every word ("dengue prescription"). Years and months stay hard date filters. Retrace shows a rescued photo first when it is the one searched for. | P1 run 1 |
P1 run 1 (5 Oct): ran under the stricter normal search. Their rephrasing under normal search ("party with three friends" x3, "dengue prescription") was partly caused by that strictness, so do not count their normal-search searches/time as a fair baseline. Retrace results and the consent/trust answers stand.
P1 run 2 (5 Oct, P1_run2.tsv, entered as participant 2 so modes swapped): a repeat; P1 had seen all four
photos. All 4 found in 1 search each, ease 5 throughout, normal 8 s and 14 s vs Retrace 11 s and 16 s. Expected: once
you know the photo and its words, both modes work. Useful only as a check that the fairer baseline behaves: normal
search found the mandap ("wedding mandap") and the café ("beach side cafe"). One real signal: for "party with friends"
plain search returned 6 group selfies but not the dancing photo; Retrace placed the 12 misfiled party photos
(0 matches, 12 moved). Consent unchanged ("Yes, if I can choose which apps it uses"); reasons made sense 5/5.
P2 (5 Oct, P2.tsv, entered as participant 3): first run, fair baseline. All 4 found with 1 search each, ease 5
throughout. Normal: "club party" (25 results, 16 s), "wedding mandap" (10 s). Retrace: "doctor's prescription" (19 s,
1 photo rescued from the wrong date), "beach cafe" (12 s). No difference between modes for P2: they left the year
out of the party search and knew the word "mandap", so normal search did not hit the wrong-date trap. Consent: "Yes, if
I can choose which apps it uses" + "only if my data is stored locally and not online"; reasons made sense 4/5.
P3 (6 Oct, P3.tsv, entered as participant 4): first run, fair baseline. All 4 found, 1 search each, ease 5.
Both Retrace tasks were real wrong-date cases: "party in 2018 with friends" (plain search: 1 result, a chat
screenshot; target missed) and "wedding jaipur" (plain: 16 results without the mandap, which has no location and is
filed under Feb 2024). Retrace found both (19 s, 13 s). Consent: "Yes" ("As long as search becomes easier"); reasons 4/5.
P5 (6 Oct, P5.tsv, entered as participant 5): first run, fair baseline. All 4 found. Normal T1 took 4 searches
and 36 s: "party with 3 friends" and "party with friends" (twice) returned only group selfies, "It earlier gave me only
selfie photos not the dance photos", until plain "party" (25 results). Normal T2 "Doctor's prescription" 12 s.
Retrace: "riya wedding" 12 s (plain search with the same words misses the mandap), "fries and cold coffee cafe" 11 s.
Consent: "Yes, if I can choose which apps it uses"; reasons 5/5.
P6 (6 Oct, P6.tsv, entered as participant 6): first run, fair baseline. All 4 found, 1 search each, ease 5.
Retrace: "fresher's party" 12 s (plain search with the same words: 1 result, a chat; target missed), "doctor's
prescription" 19 s. Normal: "riya's wedding" 10 s (26 results, mandap included), "sunset beach cafe goa" 12 s.
Consent: "Yes"; reasons 5/5.
For every query typed in a Retrace task, the same words were re-run through the plain search (script check, 6 Oct).
| Session | Typed in a Retrace task | Plain search with the same words | Retrace |
|---|---|---|---|
| P1 run 1 T3 | wedding Mandap | target included (6 results) | found |
| P1 run 1 T4 | cafe at the beach | target included (7) | found |
| P2 T2 | doctor's prescription | target included (4) | found |
| P2 T4 | beach cafe | target included (7) | found |
| P3 T1 | party in 2018 with friends | missed (1 result, a chat) | found |
| P3 T3 | wedding jaipur | missed (16 results, no mandap) | found |
| P5 T3 | riya wedding | missed (14 results, no mandap) | found |
| P5 T4 | fries and cold coffee cafe | target included (#13 of 19) | found (in the top group) |
| P6 T1 | fresher's party | missed (1 result, a chat) | found |
| P6 T2 | doctor's prescription | target included (#3 of 4) | found |
| P1 run 2 T1 (repeat) | party with friends | missed (6 group selfies) | found |
| P1 run 2 T2 (repeat) | medicine prescription | target included (4) | found |
Rescue (first runs only): 4 of 4 searches where plain search with the same words would have missed the photo, Retrace found it. In the other 6 first-run Retrace searches plain search would also have worked; Retrace did no worse.
| Normal search (10 tasks) | Retrace (10 tasks) | |
|---|---|---|
| Found | 10 of 10 | 10 of 10 |
| Tasks needing more than 1 search | 3 of 10 (P1 T1: 4, P1 T2: 2, P5 T1: 4) | 0 of 10 |
| Searches typed, total | 17 | 10 |
| Median time | 13 s (range 10 to 36) | 13 s (range 11 to 51) |
| Ease, median (1 to 5) | 5 (range 2 to 5) | 5 (range 3 to 5) |
| Rescued (plain search with the same words misses it) | n/a | 4 of 4 |
Fair-baseline only (P2, P3, P5, P6; P1 ran under the stricter search): normal needed more than one search in 1 of 8 tasks (P5 T1: 4 searches, 36 s); Retrace in 0 of 8.
Reading. Everyone found every photo, and median time is the same (13 s), so the difference is effort and rescue, not speed: under normal search people reworded when their words did not match the photo's labels ("party with friends" brought selfies, not the dance photo); with Retrace nobody did, and every time the words would have missed the photo in plain search (4 of 4: "party in 2018 with friends", "wedding jaipur", "riya wedding", "fresher's party"), Retrace found it. Different tasks per mode and n = 5: report counts, not percentages or significance.
| Calendar / Gmail | Reasons made sense | Words | |
|---|---|---|---|
| P1 | Yes, if I can choose which apps it uses | 4/5 | call: worried photos would be "on the internet all the time" |
| P2 | Yes, if I can choose which apps it uses | 4/5 | "only if my data is stored locally and not online" |
| P3 | Yes | 4/5 | "As long as search becomes easier" |
| P5 | Yes, if I can choose which apps it uses | 5/5 | call: fine with Google; worried about non-Google AI and cloud backup |
| P6 | Yes | 5/5 | call: "it needs some guardrails to protect our data" |
Totals: Calendar/Gmail yes from 5 of 5, 3 of them conditional on choosing which apps; reasons made sense 4, 4, 4, 5, 5.
Question: Retrace must work on how people actually phrase a half-remembered photo, not on sentences it was tuned on.
Method. Two query sets, each written by a separate AI agent that never saw the code, labels or library data; it got
only a plain description of each target photo and was told to write queries the way different real people type them:
very short, natural sentences, a wrong or fuzzy year, Hinglish or typos, and who/what/relative time. Each query is run
through (a) the keyword-search stand-in and (b) Retrace as the person sees it (the grouped results page). Rank is among
distinct photos (near-duplicates collapsed). Script: mvp/app/scripts/heldout.ts.
| Keyword search | Retrace | |
|---|---|---|
| Set B: target found at all | 34 / 60 (57%) | 54 / 60 (90%) |
| Set B: target in top 5 | 32 / 60 (53%) | 49 / 60 (82%) |
| Set B: target is the top result | 18 / 60 (30%) | 28 / 60 (47%) |
| Set A before fixes: found / top 5 / top 1 | 50 / 47 / 33 of 80 | 70 / 66 / 51 of 80 |
| Set A after fixes (no longer clean) | 50 / 47 / 33 of 80 | 78 / 73 / 61 of 80 |
Each file is the "Copy results" export, pasted unchanged. Participant numbers in the files are /test numbers (P1 = 1, P2 = 3, P3 = 4, P5 = 5, P6 = 6; 2 = P1's repeat run). P4 did not take the test.
participant task mode found seconds searches moment_taps refines fixes opened wrong_picks ease queries comment
1 T1 normal 1 22 4 0 0 0 1 0 3 party with three friends (0) | party with three friends (0) | party with three friends (0) | party (25) I searched party with three friends it didn't show me anything, then I searched party then it gave me bunch of photos
1 T2 normal 1 18 2 0 0 0 1 0 2 dengue prescription (0) | prescription (4) When I searched for dengue prescription, it gave no results but when I searched for prescription it then gave me a result
1 T3 retrace 1 51 1 0 0 0 1 0 3 wedding Mandap (6 plain) | [Retrace: 1 matches, 9 moved, top moment Riya's wedding (Jaipur)] found it
1 T4 retrace 1 13 1 0 0 0 1 0 5 cafe at the beach (7 plain) | [Retrace: 7 matches, 0 moved, top moment Goa trip]
participant 1 consent: Yes, if I can choose which apps it uses trust_reasons: 4 why:
# P2 (call notes in research/sessions/P2.md). First run. Entered as participant 3 in /test (1 and 2 were P1's runs).
participant task mode found seconds searches moment_taps refines fixes opened wrong_picks ease queries comment
3 T1 normal 1 16 1 0 0 0 1 0 5 club party (25) easy to find
3 T2 retrace 1 19 1 0 0 0 1 0 5 doctor's prescription (4 plain) | [Retrace: 3 matches, 1 moved, top moment When you were unwell (Aug 2022)]
3 T3 normal 1 10 1 0 0 0 1 0 5 wedding mandap (6)
3 T4 retrace 1 12 1 0 0 0 1 0 5 beach cafe (7 plain) | [Retrace: 7 matches, 0 moved, top moment Goa trip]
participant 3 consent: Yes, if I can choose which apps it uses trust_reasons: 4 why: only if my data is stored locally and not online
# P3 (call notes in research/sessions/P3.md). First run. Entered as participant 4 in /test.
participant task mode found seconds searches moment_taps refines fixes opened wrong_picks ease queries comment
4 T1 retrace 1 19 1 0 0 0 1 0 5 party in 2018 with friends (1 plain) | [Retrace: 8 matches, 12 moved, top moment Freshers' Night]
4 T2 normal 1 14 1 0 0 0 1 0 5 prescription (4)
4 T3 retrace 1 13 1 0 0 0 1 0 5 wedding jaipur (16 plain) | [Retrace: 16 matches, 9 moved, top moment Riya's wedding (Jaipur)]
4 T4 normal 1 11 1 0 0 0 1 0 5 goa sunset cafe (7)
participant 4 consent: Yes trust_reasons: 4 why: As long as search becomes easier
# P5 (call notes in research/sessions/P5.md). First run. Entered as participant 5 in /test.
participant task mode found seconds searches moment_taps refines fixes opened wrong_picks ease queries comment
5 T1 normal 1 36 4 0 0 0 1 0 4 party with 3 friends (6) | party with friends (6) | party with friends (6) | party (25) It earlier gave me only selfie photos not the dance photos
5 T2 normal 1 12 1 0 0 0 1 0 5 Doctor's prescription (4)
5 T3 retrace 1 12 1 0 0 0 1 0 5 riya wedding (14 plain) | [Retrace: 18 matches, 9 moved, top moment Riya's wedding (Jaipur)]
5 T4 retrace 1 11 1 0 0 0 1 0 5 fries and cold coffee cafe (19 plain) | [Retrace: 19 matches, 0 moved, top moment Goa trip]
participant 5 consent: Yes, if I can choose which apps it uses trust_reasons: 5 why:
# P6 (call notes in research/sessions/P6.md). First run. Entered as participant 6 in /test.
participant task mode found seconds searches moment_taps refines fixes opened wrong_picks ease queries comment
6 T1 retrace 1 12 1 0 0 0 1 0 5 fresher's party (1 plain) | [Retrace: 1 matches, 12 moved, top moment Freshers' Night]
6 T2 retrace 1 19 1 0 0 0 1 0 5 doctor's prescription (4 plain) | [Retrace: 3 matches, 1 moved, top moment When you were unwell (Aug 2022)]
6 T3 normal 1 10 1 0 0 0 1 0 5 riya's wedding (26)
6 T4 normal 1 12 1 0 0 0 1 0 5 sunset beach cafe goa (7)
participant 6 consent: Yes trust_reasons: 5 why:
# REPEAT RUN: P1 had already seen all four target photos in run 1. Not fresh data; report separately, never pooled.
# Entered as participant 2 so modes were swapped. Ran after the fairer normal search (5 Oct).
participant task mode found seconds searches moment_taps refines fixes opened wrong_picks ease queries comment
2 T1 retrace 1 11 1 0 0 0 1 0 5 party with friends (6 plain) | [Retrace: 0 matches, 12 moved, top moment Freshers' Night]
2 T2 retrace 1 16 1 0 0 0 1 0 5 medicine prescription (4 plain) | [Retrace: 3 matches, 1 moved, top moment When you were unwell (Aug 2022)]
2 T3 normal 1 8 1 0 0 0 1 0 5 wedding mandap (6)
2 T4 normal 1 14 1 0 0 0 2 0 5 beach side cafe (7)
participant 2 consent: Yes, if I can choose which apps it uses trust_reasons: 5 why:
Everyday photos without people in the prototype library are free stock images under a CC0 (public domain) licence, which needs no attribution; sources are listed for transparency. 133 images are used (near-duplicate copies made from some of them are not listed separately).
The fictional people were created from character reference portraits licensed from Adobe Stock; the scenes of their fictional life were generated with ChatGPT image generation. Documents, tickets, receipts and chat screenshots were drawn in code.
| Image | Licence | Source |
|---|---|---|
| food_001 | CC0 | www.rawpixel.com |
| food_002 | CC0 | www.rawpixel.com |
| food_003 | CC0 | commons.wikimedia.org |
| food_008 | CC0 | commons.wikimedia.org |
| food_010 | CC0 | www.flickr.com |
| food_014 | CC0 | wordpress.org |
| food_015 | CC0 | www.flickr.com |
| food_016 | CC0 | www.flickr.com |
| food_017 | CC0 | www.flickr.com |
| food_018 | CC0 | wordpress.org |
| food_019 | CC0 | wordpress.org |
| food_020 | CC0 | wordpress.org |
| food_021 | CC0 | www.rawpixel.com |
| food_022 | CC0 | www.flickr.com |
| food_023 | CC0 | www.flickr.com |
| food_024 | CC0 | www.flickr.com |
| food_025 | CC0 | www.flickr.com |
| food_026 | CC0 | www.flickr.com |
| food_027 | CC0 | www.flickr.com |
| food_028 | CC0 | www.flickr.com |
| food_029 | CC0 | www.flickr.com |
| food_030 | CC0 | www.flickr.com |
| food_031 | CC0 | www.flickr.com |
| food_032 | CC0 | www.flickr.com |
| food_034 | CC0 | www.rawpixel.com |
| food_035 | CC0 | wordpress.org |
| food_036 | CC0 | www.rawpixel.com |
| food_037 | CC0 | wordpress.org |
| food_038 | CC0 | wordpress.org |
| food_039 | CC0 | wordpress.org |
| food_040 | CC0 | wordpress.org |
| sky_001 | CC0 | www.flickr.com |
| sky_002 | CC0 | stocksnap.io |
| sky_003 | CC0 | www.flickr.com |
| sky_004 | CC0 | stocksnap.io |
| sky_006 | CC0 | stocksnap.io |
| sky_007 | CC0 | stocksnap.io |
| sky_008 | CC0 | stocksnap.io |
| sky_009 | CC0 | stocksnap.io |
| sky_010 | CC0 | commons.wikimedia.org |
| sky_011 | CC0 | www.flickr.com |
| sky_012 | CC0 | stocksnap.io |
| sky_013 | CC0 | stocksnap.io |
| sky_014 | CC0 | stocksnap.io |
| sky_015 | CC0 | www.flickr.com |
| sky_016 | CC0 | stocksnap.io |
| sky_017 | CC0 | stocksnap.io |
| sky_018 | CC0 | stocksnap.io |
| sky_019 | CC0 | www.flickr.com |
| sky_020 | CC0 | www.flickr.com |
| sky_021 | CC0 | www.flickr.com |
| sky_022 | CC0 | www.flickr.com |
| sky_023 | CC0 | www.flickr.com |
| sky_024 | CC0 | www.flickr.com |
| sky_025 | CC0 | www.flickr.com |
| streets_003 | CC0 | www.flickr.com |
| streets_006 | CC0 | www.flickr.com |
| streets_012 | CC0 | www.rawpixel.com |
| streets_013 | CC0 | www.rawpixel.com |
| streets_015 | CC0 | www.flickr.com |
| streets_016 | CC0 | www.flickr.com |
| streets_017 | CC0 | www.flickr.com |
| streets_019 | CC0 | www.flickr.com |
| streets_020 | CC0 | www.flickr.com |
| streets_021 | CC0 | www.flickr.com |
| streets_022 | CC0 | www.flickr.com |
| streets_023 | CC0 | www.flickr.com |
| streets_024 | CC0 | www.flickr.com |
| streets_025 | CC0 | www.flickr.com |
| nature_001 | CC0 | www.flickr.com |
| nature_002 | CC0 | www.flickr.com |
| nature_003 | CC0 | www.flickr.com |
| nature_004 | CC0 | www.flickr.com |
| nature_005 | CC0 | www.flickr.com |
| nature_006 | CC0 | www.flickr.com |
| nature_007 | CC0 | www.flickr.com |
| nature_008 | CC0 | www.flickr.com |
| nature_009 | CC0 | www.flickr.com |
| nature_010 | CC0 | www.flickr.com |
| nature_011 | CC0 | www.flickr.com |
| nature_012 | CC0 | www.flickr.com |
| nature_013 | CC0 | www.flickr.com |
| nature_014 | CC0 | www.flickr.com |
| nature_015 | CC0 | www.flickr.com |
| nature_016 | CC0 | www.flickr.com |
| nature_017 | CC0 | www.flickr.com |
| nature_018 | CC0 | www.flickr.com |
| nature_019 | CC0 | www.flickr.com |
| nature_020 | CC0 | www.flickr.com |
| things_002 | CC0 | www.flickr.com |
| things_003 | CC0 | www.flickr.com |
| things_004 | CC0 | stocksnap.io |
| things_005 | CC0 | www.flickr.com |
| things_006 | CC0 | stocksnap.io |
| things_007 | CC0 | stocksnap.io |
| things_008 | CC0 | stocksnap.io |
| things_009 | CC0 | www.flickr.com |
| things_010 | CC0 | www.flickr.com |
| things_011 | CC0 | www.rawpixel.com |
| things_013 | CC0 | www.rawpixel.com |
| things_014 | CC0 | www.rawpixel.com |
| things_015 | CC0 | www.rawpixel.com |
| things_021 | CC0 | www.flickr.com |
| things_023 | CC0 | www.flickr.com |
| things_024 | CC0 | www.flickr.com |
| things_025 | CC0 | stocksnap.io |
| things_026 | CC0 | stocksnap.io |
| things_027 | CC0 | stocksnap.io |
| things_028 | CC0 | stocksnap.io |
| things_029 | CC0 | stocksnap.io |
| things_030 | CC0 | stocksnap.io |
| notes_001 | CC0 | www.flickr.com |
| notes_002 | CC0 | stocksnap.io |
| notes_003 | CC0 | stocksnap.io |
| notes_004 | CC0 | www.flickr.com |
| notes_006 | CC0 | www.flickr.com |
| notes_007 | CC0 | www.flickr.com |
| notes_014 | CC0 | wordpress.org |
| random_001 | CC0 | stocksnap.io |
| random_002 | CC0 | www.flickr.com |
| random_003 | CC0 | stocksnap.io |
| random_004 | CC0 | stocksnap.io |
| random_005 | CC0 | stocksnap.io |
| random_006 | CC0 | www.flickr.com |
| random_007 | CC0 | www.flickr.com |
| random_008 | CC0 | www.flickr.com |
| random_009 | CC0 | www.flickr.com |
| random_010 | CC0 | stocksnap.io |
| random_011 | CC0 | www.rawpixel.com |
| random_012 | CC0 | www.rawpixel.com |
| random_013 | CC0 | www.rawpixel.com |
| random_014 | CC0 | wordpress.org |
| random_015 | CC0 | www.flickr.com |