# THIS YOU — Mandatory Product Visual References & Screenshot Fallback

## Goal

Make product visual fidelity truly multimodal.

The current product fidelity pipeline already stores:

- `visual_references`
- `visual_spec`
- `visual_lock_fingerprint`
- `fidelity_check_fields`

But storing an image URL inside JSON is **not enough**.

The system must guarantee that the HERO generator and the PRODUCT FIDELITY CHECK receive **real image inputs**, not only product URLs or text descriptions.

This is a HARD requirement.

---

# Core rule

> Every visible selected product must have at least one successfully loaded real visual reference image before HERO generation or fidelity checking can proceed.

A URL string alone does not count as a loaded visual reference.

A textual `visual_spec` alone does not count as a visual reference.

The product image itself must be available to the multimodal model.

---

# Required priority order

For every selected product, obtain a visual reference using this order:

```text
1. Exact official product image asset
        ↓ if unavailable
2. Exact official product-page screenshot
        ↓ if unavailable
3. Human-uploaded exact product reference
        ↓ if unavailable
4. STOP workflow
```

Never continue with text-only visual fidelity.

---

# 1. Preferred source — official product image

First try to fetch the exact official image associated with the selected:

- product;
- colour;
- variant;
- exact product URL.

Preferred image set:

```text
front
detail
on_model
back
```

Minimum:

```text
1 usable image
```

Recommended for visible garments:

```text
front + detail
```

Especially for:

- jackets;
- shirts;
- coats;
- trousers with unusual construction;
- shoes with distinctive panels/colour blocking.

---

# 2. Screenshot fallback

If direct image fetch fails, do not abandon visual fidelity.

Open the exact product page and take a screenshot.

The screenshot must come from:

```text
exact product URL
+
exact selected colour / variant
```

Then crop the screenshot to the selected product or relevant detail.

Example:

```text
Product image CDN blocks server fetch
↓
Open retailer product page
↓
Take screenshot
↓
Crop exact front garment image
↓
Store as approved visual reference
```

The screenshot becomes a valid visual reference.

---

# Screenshot provenance

Store:

```json
{
  "type": "front",
  "source": "official_product_page_screenshot",
  "product_url": "https://exact-product-page",
  "asset_id": "product-assets/231/front-screenshot.png",
  "captured_at": "2026-08-16T..."
}
```

If useful:

```json
"source_image_url": null
```

because the visual evidence came from the rendered product page rather than a directly fetched image asset.

---

# 3. Human-upload fallback

If both:

```text
official image fetch
AND
official product-page screenshot
```

fail, allow the admin/human reviewer to upload or select the exact product image manually.

Example:

```json
{
  "type": "front",
  "source": "human_uploaded_exact_product_reference",
  "product_url": "https://exact-product-page",
  "asset_id": "product-assets/231/manual-front.png"
}
```

Human must confirm:

```text
This image is the exact selected product and exact selected colour/variant.
```

---

# 4. STOP condition

If a visible selected product has no usable image reference:

```text
visual_reference_status = missing
```

The system must not:

- generate final HERO;
- mark Product Visual Lock complete;
- run a normal fidelity score;
- return PASS based only on text.

Instead:

```text
status = insufficient_visual_reference
human_action_required = true
```

---

# 5. Product Visual Lock must be image-based

PRODUCT VISUAL LOCK must analyze the actual product images.

Do not generate `visual_spec` from:

- product title alone;
- product description alone;
- URL text;
- search result snippets;
- previous cached text.

Valid inputs:

```text
real product image
+
optional product-page text
```

The image is the primary source for visible construction.

---

# 6. Fidelity checker must be multimodal

The fidelity checker must receive:

```text
CURRENT HERO IMAGE
+
REAL PRODUCT REFERENCE IMAGE(S)
+
VISUAL SPEC
```

For every selected product.

Example conceptual request:

```text
Image 1 = HERO
Image 2 = Weekday jacket front
Image 3 = Weekday jacket detail
Image 4 = shirt front
Image 5 = trousers front
Image 6 = shoes front

Text =
visual_spec + fidelity_check_fields
```

Do not send only:

```json
"url": "https://media.weekday.com/..."
```

and assume the vision model has seen the image.

The backend must actually load/download/attach the image to the multimodal request.

---

# 7. URL is provenance, not visual evidence

This:

```json
{
  "url": "https://media.weekday.com/image.jpg"
}
```

means:

```text
we know where the image came from
```

It does NOT automatically mean:

```text
the model has received the image pixels
```

Therefore store both:

```json
{
  "url": "https://official-image-url",
  "asset_id": "product-assets/231/front.jpg",
  "load_status": "ready"
}
```

Recommended fields:

```json
{
  "type": "front",
  "source": "official_product_page",
  "url": "https://...",
  "asset_id": "product-assets/231/front.jpg",
  "load_status": "ready",
  "captured_at": "..."
}
```

---

# 8. Required image state

Recommended state machine:

```text
discovered
↓
fetching
↓
ready
```

Fallback:

```text
fetching
↓
direct_fetch_failed
↓
screenshot_capture
↓
ready
```

Fallback again:

```text
screenshot_capture
↓
screenshot_failed
↓
human_upload_required
↓
ready
```

Terminal failure:

```text
human_upload_required
↓
no_reference
↓
blocked
```

---

# 9. HERO generation gate

Before HERO generation:

```pseudo
for each visible selected_product:
    if product.visual_reference_count_loaded < 1:
        BLOCK HERO
```

Recommended:

```pseudo
if garment is construction-sensitive:
    prefer >= 2 references
```

Construction-sensitive examples:

```text
jackets
shirts
coats
structured trousers
complex sneakers
```

---

# 10. Fidelity-check gate

Before running PRODUCT FIDELITY CHECK:

```pseudo
for each selected_product:
    if no loaded visual reference:
        product.status = insufficient_visual_reference
```

The checker must NOT return:

```text
pass
```

for that product.

Do not calculate a normal product score from text only.

---

# 11. New result state

Recommended fidelity result extension:

```json
{
  "reference_status": "ready"
}
```

Allowed:

```text
ready
insufficient_visual_reference
```

Example:

```json
{
  "product_id": 231,
  "reference_status": "ready",
  "checks": {
    "front_closure": "pass"
  }
}
```

Failure:

```json
{
  "product_id": 231,
  "reference_status": "insufficient_visual_reference",
  "checks": {},
  "critical_failures": [],
  "not_visible": [],
  "score": null
}
```

---

# 12. Do not silently fall back to text

Forbidden behavior:

```text
Image fetch failed
↓
Use visual_spec text instead
↓
Return fidelity PASS
```

Correct behavior:

```text
Image fetch failed
↓
Try screenshot
↓
Try human-upload reference
↓
STOP if no image exists
```

---

# 13. Screenshot cropping

A full retailer-page screenshot can contain too much irrelevant content.

Preferred flow:

```text
capture page screenshot
↓
identify selected product image region
↓
crop product image / relevant construction detail
↓
store crop as reference asset
```

Keep the original full screenshot optionally for provenance/debugging.

Example:

```json
{
  "asset_id": "product-assets/231/front-crop.png",
  "source_screenshot_asset_id": "product-assets/231/product-page-full.png"
}
```

---

# 14. Exact variant safety

Screenshot fallback must still verify:

- product name;
- selected colour;
- variant/style number where available.

Do not capture:

- another colour;
- recommendation carousel;
- "you may also like" product;
- similar garment;
- another variant from the same product family.

---

# 15. Cache policy

Once a product visual reference has been human-approved:

```text
cache/store the exact image asset
```

This is good.

Reason:

The visual lock must remain stable between:

```text
product approval
and
HERO generation
```

Do not unexpectedly replace the approved visual reference with a later retailer image.

Store:

```text
asset
source URL
product URL
captured_at
visual_lock_fingerprint
```

---

# 16. Price cache remains separate

Do not confuse visual caching with price verification.

Price:

```text
must be freshly verified / revalidated
```

Approved visual reference:

```text
may remain cached and locked
```

because visual consistency is desirable.

---

# 17. HERO correction pass

The targeted correction model must also receive the actual product image(s) for the failed product.

Example:

```text
CURRENT HERO
+
Weekday jacket front image
+
Weekday jacket detail image
+
failed fields:
overall_length
zipper_visibility
pocket_shape
```

Do not perform targeted correction from text only.

---

# 18. Reference minimum by product type

Recommended defaults:

## Simple T-shirt / sweater

```text
1 front reference
```

## Shirt

```text
front + detail preferred
```

## Jacket / coat

```text
front + detail required when available
```

## Trousers

```text
front + on-model preferred
```

## Shoes

```text
side/front + detail preferred
```

These are recommended quality targets, but the absolute hard minimum remains:

```text
at least 1 real image
```

---

# 19. Admin UI

For each product show:

```text
VISUAL REFERENCE

✓ Loaded
Source: Official product image
Front
[thumbnail]
```

Fallback example:

```text
✓ Loaded
Source: Official product-page screenshot
[thumbnail]
```

Failure:

```text
⚠ VISUAL REFERENCE MISSING

Direct image fetch failed.
Screenshot capture failed.

UPLOAD EXACT PRODUCT IMAGE
```

Do not show:

```text
Visual Lock Approved
```

while no real reference exists.

---

# 20. Debugging visibility

Admin/debug UI should expose:

```text
source
asset_id
load_status
image dimensions
captured_at
product_url
visual_lock_fingerprint
```

This makes it possible to confirm whether the model actually received an image.

---

# 21. Multimodal request logging

For debugging, log something equivalent to:

```json
{
  "hero_asset_attached": true,
  "product_reference_assets_attached": [
    "product-assets/231/front.jpg",
    "product-assets/231/detail.jpg",
    "product-assets/232/front.jpg",
    "product-assets/233/front.jpg",
    "product-assets/234/front.jpg"
  ]
}
```

This is important.

A successful request should make it obvious that image assets were attached.

---

# 22. Acceptance criteria

## Product Visual Lock

- [ ] Every visible product has at least one real image reference.
- [ ] Direct official image fetch is attempted first.
- [ ] Failed direct fetch automatically triggers screenshot fallback.
- [ ] Screenshot comes from the exact product page.
- [ ] Screenshot/crop represents the exact selected colour/variant.
- [ ] Failed screenshot triggers human-upload fallback.
- [ ] No image reference = workflow blocked.
- [ ] `visual_spec` is generated from actual visual evidence.
- [ ] Text-only Visual Lock is forbidden.

## HERO generation

- [ ] Real product reference assets are attached to the multimodal request.
- [ ] Product URLs are not treated as equivalent to image attachments.
- [ ] Reader images define identity.
- [ ] Product images define garment construction.

## Fidelity check

- [ ] HERO image is attached as an actual image.
- [ ] At least one actual product image is attached for every product being checked.
- [ ] Checker cannot return normal PASS when product reference image failed to load.
- [ ] Text-only fidelity scoring is forbidden.
- [ ] Screenshot fallback references are valid multimodal inputs.
- [ ] `not_visible` applies only to HERO visibility, not missing product references.

## Correction pass

- [ ] Failed product's actual reference images are attached again.
- [ ] Correction is based on image + visual_spec.
- [ ] Text-only correction is forbidden.

---

# 23. Expected failure handling

Correct:

```text
Official image CDN blocked
↓
Screenshot exact product page
↓
Crop selected jacket
↓
Visual Lock proceeds
```

Also correct:

```text
Official image blocked
↓
Screenshot blocked
↓
Admin uploads exact reference
↓
Visual Lock proceeds
```

Correct final failure:

```text
No usable image can be obtained
↓
BLOCK
```

Forbidden:

```text
No image
↓
Use product description
↓
Guess construction
↓
PASS
```

---

# Final rule

> **Images are mandatory. URLs and text are supplementary metadata only.**

For THIS YOU product fidelity:

```text
REAL PRODUCT IMAGE
        +
VISUAL SPEC
        +
HERO IMAGE
        =
VALID FIDELITY CHECK
```

Without the real product image:

```text
NO VALID FIDELITY CHECK
```