rzen
049c6c5377
Disambiguates the one-letter `find` (item) vs `finds` (roundup) split that
kept causing confusion. Audience-facing now: `/bundles/<slug>/`, "Bundle"
labels in home/footer/search.
Internals: `getCollection('finds')` → `('bundles')`, `Entry.type 'finds'`
→ `'bundle'`, `OgFallback 'finds'` → `'bundles'`, `findListMosaic` →
`bundleMosaic`, `FindsPost` → `Bundle`, `findsReferencing` →
`bundlesReferencing`. Build script + generated mosaic dir follow
(`scripts/build-bundle-mosaics.mjs`, `src/generated/bundle-mosaics/`).
Skills `build-finds` → `build-bundle`, `cluster-finds` →
`cluster-bundles`. The `find` collection (individual items) is unchanged.
No URL redirects: low external-link surface area.
123 lines
8.7 KiB
Markdown
123 lines
8.7 KiB
Markdown
---
|
|
name: daily-finds
|
|
description: Capture-only daily sweep for unique.rzen.dev. Scans every source in sources.json, writes one find file per qualifying item to src/content/find/, and nominates any new sources cited in the items it reads (plus optionally via WebSearch). Does NOT pick a winner, write a bundle, or address the user/audience. Picking happens via /build-bundle and /build-review. Use when the user asks for "today's finds", "morning sweep", "capture", or runs /daily-finds.
|
|
disable-model-invocation: true
|
|
---
|
|
|
|
# daily-finds
|
|
|
|
Pure capture: every item that passes the bar becomes a `find` file. No daily cap, no theme detection, no audience-facing post. Whether any of these finds appears in a `bundle` or graduates to a review is a separate editorial decision.
|
|
|
|
Invoke from inside the `unique.rzen.dev` repo.
|
|
|
|
## Inputs
|
|
|
|
- `sources.json` at the project root — curated source list with per-source `fetch` strategy (`webfetch` / `curl` / `skip`) and optional `fetchUrl`.
|
|
- `daily-finds.log.md` — running log of every find slug ever written (used to skip duplicates).
|
|
- `src/content/find/*.mdx` — already-captured finds (every existing slug is a "do not re-suggest" item).
|
|
- `src/content/reviews/*.mdx` — already-reviewed items (do not re-suggest).
|
|
- `candidate-sources.md` — newly nominated sources awaiting user vetting.
|
|
|
|
## The bar
|
|
|
|
A find qualifies if and only if it is **(a)** delightful, unusual, or distinctive in a way the audience would care about *and* **(b)** usable daily (relax to "memorable for a visitor" for travel). Drop generic mass-market items, ad-copy listicles, speculative / not-yet-shipping products, self-promo, and anything not traceable to a real currently-buyable / currently-visitable thing.
|
|
|
|
Crucially: do **not** filter by "is it the best of the day." Capture broadly. Editorial selection happens later.
|
|
|
|
## Procedure
|
|
|
|
### 1. Scan every source
|
|
|
|
For each source in `sources.json`:
|
|
|
|
- **Default (`fetch: "webfetch"` or no field)**: WebFetch against `fetchUrl` if present, else `url`. Extract recently-featured items (name, link, one-line description).
|
|
- **`fetch: "curl"`**: shell out to `curl -sL -A "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.0 Safari/605.1.15" --max-time 25 "<fetchUrl|url>"` and parse the HTML body. For Reddit JSON endpoints, pipe through `python3 -c '…'` to extract titles, links, and selftext. Treat HTTP 403 with a non-empty HTML body as a content response (Atlas Obscura quirk) — only treat empty bodies or Cloudflare challenge markers as blocked.
|
|
- **`fetch: "skip"`**: log the source as skipped (with `skipReason`) and move on. Do not retry.
|
|
|
|
Parallelize WebFetch in batches of ~8 concurrent requests; finish all sources before proceeding.
|
|
|
|
For each source record: items extracted, or `empty` / `blocked` / `skipped`.
|
|
|
|
### 2. Filter
|
|
|
|
For each extracted item, drop:
|
|
|
|
- Anything whose slug already exists in `src/content/find/`.
|
|
- Anything whose name appears anywhere in `daily-finds.log.md`.
|
|
- Anything reviewed in `src/content/reviews/` (compare by `name` field).
|
|
- Items that fail the bar above.
|
|
|
|
### 3. Write find files
|
|
|
|
For every survivor, write one file at `src/content/find/<slug>.mdx`:
|
|
|
|
- Slug: kebab-case form of the name (lowercase, ASCII, no leading numbers).
|
|
- Frontmatter: `name`, `subtitle` (optional, one-liner), `date` (today), `link` (external URL — required), `linkText` (optional), `source` (the source name from sources.json — required, exact match), `topics` (copied from the source's `topics`), `tags` (free editorial tags inferred from the item — keep to 2-4), `description` (optional meta override).
|
|
- Body: 2-3 sentence note. Concrete, name a *specific* virtue. Bad: "great design". Good: "the silicone bumper means it survives drops onto concrete; the lanyard hole is positioned where your thumb naturally rests".
|
|
- No daily cap — write every survivor.
|
|
|
|
If `link` for a Reddit-discovered item points at a v.redd.it / i.redd.it URL, fetch the post's selftext or comments to find the actual product URL before writing the file.
|
|
|
|
#### Supporting media
|
|
|
|
For every find, try to retrieve a single representative image from the linked page so the find has visual support if it later graduates to a review (or gets surfaced on a tile).
|
|
|
|
- **Where to look (in priority order):** `og:image` meta tag → JSON-LD `image` field → main product photo / hero image markup → first article image.
|
|
- **Where to save:** `src/assets/finds/<slug>/hero.<ext>`. Preserve the original extension (`.jpg`, `.png`, `.webp`, `.gif`). One file per find — `hero` is the canonical name.
|
|
- **What to skip:** images with marketing/sales overlay text (Amazon-style "BEST PRESS / NO GRIT" tags), tiny thumbnails (<400px on the long edge), generic site logos, paywalled CDN URLs that 403 to curl. If the only available shots are marketing collages, skip rather than save a noisy image — a missing hero is fine, a bad one isn't.
|
|
- **For Mac/iOS apps:** prefer the App Store hero or developer-site screenshot. For physical products: prefer a clean studio shot or lifestyle photo without overlay copy. For travel destinations: prefer a Wikimedia Commons / Wikipedia image (free-to-use only). For text-only finds (quotes, dad jokes, essays): no media expected — skip.
|
|
- **Fetch strategy:** `curl -sL --max-time 25 -A "Mozilla/5.0 (...Safari...)" "<image-url>" -o <path>`. WebFetch is for HTML, not binaries.
|
|
- **Don't register heroes in `src/lib/hero.ts` at this stage.** That's a review-time decision (see `build-review`). The asset just needs to exist on disk so it's there if/when the find graduates.
|
|
|
|
Log image-fetch failures (404, 403, marketing-only) as a one-line note in the capture-run section of `daily-finds.log.md` so we can revisit broken media-source patterns later — same way we log fetch issues for source pages.
|
|
|
|
### 4. Nominate new sources
|
|
|
|
Watch for new sources organically and via search. No cap on nominees per run — if a day's reading surfaces five plausible new sources, capture all five.
|
|
|
|
- **From inline references in source posts.** When a post on a known source credits or links out to another publication as where they originally discovered the thing, capture that publication as a candidate source. Trigger phrases to watch for: *"via X"*, *"first seen on Y"*, *"spotted by Z"*, *"h/t X"*, *"originally posted on Y"*, *"thanks to X for the find"*, or any byline / sidebar link pointing to a different domain that itself looks like a curated feed. Cool Tools, Tools and Toys, Recomendo, Kottke, and Daring Fireball especially tend to credit their finds — read posts carefully enough to catch these credits.
|
|
- **From general observation.** When a publication is mentioned in a way that suggests it'd fit the brief (curated, taste-driven, focused on delightful or unusual items), nominate it.
|
|
- **From targeted search.** If the run produced few or no organic candidates, run one `WebSearch` for `"best <topic> newsletter"` or `"<topic> curator blog"`.
|
|
|
|
Append every nominee to `candidate-sources.md` under today's date heading. For each: name, URL, a one-line case for adding, and how it was surfaced — name the originating source post when applicable (e.g. *"via a Cool Tools post about the Spyderco Dragonfly"*), or *"WebSearch"* for search-derived candidates.
|
|
|
|
Don't auto-add to `sources.json` — that's a user decision.
|
|
|
|
### 5. Update the log
|
|
|
|
Append to `daily-finds.log.md`:
|
|
|
|
```
|
|
## YYYY-MM-DD — capture run
|
|
|
|
Sources scanned: <N> total — <hits> hits, <empty> empty, <blocked> blocked, <skipped> skipped.
|
|
Productive sources: <comma-separated list of sources that produced at least one written find>.
|
|
|
|
Captured finds:
|
|
- <slug> — <name> — source: <source>
|
|
- <slug> — <name> — source: <source>
|
|
- ...
|
|
|
|
New source nominees: <count> (see candidate-sources.md).
|
|
```
|
|
|
|
### 6. Report to the user
|
|
|
|
Short summary in chat:
|
|
|
|
```
|
|
Captured <N> finds across <M> productive sources today.
|
|
Blocked: <K> sources. New source nominees: <count>.
|
|
Run /build-bundle to draft a bundle from the recent pool, or /build-review <slug> to graduate one to a review.
|
|
```
|
|
|
|
That's it. No theme, no candidates list, no "pick a winner."
|
|
|
|
## Notes
|
|
|
|
- "Why it fits" / find descriptions should name a *specific* virtue. Bad: "great design". Good: "magnetic connector means it never strains the cable when a foot catches it".
|
|
- Travel destinations relax the daily-use criterion — a place qualifies if a visitor would remember it years later.
|
|
- If a source's homepage is too noisy, prefer adding a `fetchUrl` to `sources.json` rather than working around it inside the skill.
|
|
- Never invent slugs that collide with existing finds or reviews. If a name would produce a colliding slug, append a year or distinguishing word.
|
|
- Source attribution must be exact — the `source` field in each find file must match a `name` in `sources.json` so the `/sources` page cross-reference works.
|