diff --git a/CHANGELOG.md b/CHANGELOG.md index 6a35fdb..1b300d9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,8 @@ ## July 2026 +- Feeds for everything: alongside the existing `/rss.xml`, add a JSON Feed 1.1 at `/feed.json`, per-category RSS at `/categories//rss.xml`, and per-tag RSS at `/tags//rss.xml` — all derived from a shared `src/lib/feed.ts` item source (reviews + posts + bundles, same visibility rules as the site). Category/tag pages advertise their scoped feed via `` autodiscovery; a new `/feeds/` directory page (linked from the footer) lists every feed with entry counts; every RSS feed carries an `atom:link` self reference and renders as a readable page in browsers via `public/feed.xsl` +- Programmatic-reader pass: build-generated `/llms.txt` (llmstxt.org format — site description, feed/endpoint map, and every review, bundle, and post with one-line descriptions), `SearchAction` on the home-page `WebSite` JSON-LD wired to `/search/?q=`, and a "For robots" section on `/feeds/` documenting `search.json`, `llms.txt`, the sitemap, and JSON-LD coverage - Autonomous pipeline overhaul: new `/daily-pipeline` skill chains capture → bundle → review graduation → pick-of-the-day → build → commit → push and ends with a digest; `daily-finds` gains a score-and-shortlist stage (up to 12 `shortlist: true` finds per run, top 1-3 flagged as review candidates), feed-first fetching (`fetch: "feed"` + `feedUrl` in sources.json), cadence-aware scanning, and authority to apply mechanical source maintenance itself; `build-bundle`/`cluster-bundles`/`build-review` gain unattended `auto` modes (publish + log instead of pausing for confirmation), and `build-bundle weekly` writes a date-based "week's catch" so shortlisted finds never sit unbundled for weeks - Review heroes are now auto-discovered: `src/lib/hero.ts` globs `reviews/*/hero.` (images and `hero.mp4`), so a new review needs no registration — only legacy non-`hero`-named assets remain as explicit overrides; both hero globs also accept `.avif` - Pick of the day moves from a hardcoded constant in `src/pages/index.astro` to `src/data/pick.json` (falls back to the latest review); the dev-mode "Make today's pick" button writes the JSON file diff --git a/README.md b/README.md index 01a2d60..913f90b 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,9 @@ Built with [Astro](https://astro.build) + MDX. - Reviews render as two-line index entries: bold name + muted subtitle (`Cotypist` / *Autocomplete that lives everywhere on your Mac*) - Auto-generated `/categories//` and `/tags//` listing pages — finds excluded by design - **Amazon affiliate handling** in `src/lib/affiliate.ts`: outbound links to Amazon (`amazon.com`, `amzn.to`, `a.co`, `smile.amazon.com`) get the configured tracking ID appended automatically; the link gets `rel="sponsored"`, an "Affiliate" badge appears next to it, and a one-line disclosure is shown beneath. Finds and reviews also take an optional `amazonLink` frontmatter field (canonical `/dp/` form) rendered as an "Also on Amazon" link — the capture pipeline populates it when a shortlisted physical product has an exact Amazon match. Add additional marketplace IDs (e.g. `amazon.co.uk`) to the `AMAZON_TAGS` map when needed -- **SEO surface**: single `` component owning canonical, Open Graph, Twitter Card, and JSON-LD; per-page structured data (`Review`, `BlogPosting`, `Product`, `Article`, `CollectionPage`, `ItemList`, `BreadcrumbList`, `WebSite`, `Organization`); auto-generated `sitemap-index.xml` (excludes `/grid/` and `/index3/`); RSS feed at `/rss.xml`; `robots.txt`; static OG image fallbacks under `/public/og/`; promoted finds canonical to their review +- **SEO surface**: single `` component owning canonical, Open Graph, Twitter Card, feed autodiscovery, and JSON-LD; per-page structured data (`Review`, `BlogPosting`, `Product`, `Article`, `CollectionPage`, `ItemList`, `BreadcrumbList`, `WebSite` with `SearchAction`, `Organization`); auto-generated `sitemap-index.xml` (excludes `/grid/` and `/index3/`); `robots.txt`; static OG image fallbacks under `/public/og/`; promoted finds canonical to their review +- **Feeds everywhere**: site-wide RSS 2.0 at `/rss.xml` and JSON Feed 1.1 at `/feed.json`, plus per-category (`/categories//rss.xml`) and per-tag (`/tags//rss.xml`) RSS feeds, all derived from a single item source in `src/lib/feed.ts`; category/tag pages advertise their scoped feed via ``; a human-readable directory at `/feeds/` (footer-linked) lists every feed with counts; RSS renders as a readable page in browsers via `public/feed.xsl` +- **Programmatic surface**: build-generated `/llms.txt` (llmstxt.org format — site map with every review/bundle/post and one-line descriptions, plus feed and endpoint pointers) and the full-site JSON index at `/search.json`, both documented in the "For robots" section of `/feeds/` - Custom 404 page with recent-reviews and navigation fallback - Light/dark mode via `prefers-color-scheme` - **Cloudflare Web Analytics** beacon in `BaseLayout.astro` for cookieless, privacy-preserving page-view + referrer + Core Web Vitals tracking scoped to `unique.rzen.dev` diff --git a/public/feed.xsl b/public/feed.xsl new file mode 100644 index 0000000..b4ec17d --- /dev/null +++ b/public/feed.xsl @@ -0,0 +1,79 @@ + + + + + + + + <xsl:value-of select="title"/> — feed + + + + + +

+ This is an RSS feed. Copy this page's address into + your feed reader to subscribe. All feeds are listed at + /feeds/. +

+

+

+ +
+

+ + + + +

+

+

+
+ + + + + diff --git a/src/components/SEO.astro b/src/components/SEO.astro index e4f0751..3a967cd 100644 --- a/src/components/SEO.astro +++ b/src/components/SEO.astro @@ -19,6 +19,8 @@ interface Props { modifiedTime?: Date; articleTags?: string[]; jsonLd?: Record | Record[]; + /** Page-scoped feeds (category/tag) advertised alongside the site-wide ones. */ + feeds?: Array<{ title: string; href: string }>; } const { @@ -33,6 +35,7 @@ const { modifiedTime, articleTags = [], jsonLd, + feeds = [], } = Astro.props; const SITE_NAME = 'Unique'; @@ -59,6 +62,10 @@ const jsonLdBlocks = jsonLd + +{feeds.map((f) => ( + +))} diff --git a/src/layouts/BaseLayout.astro b/src/layouts/BaseLayout.astro index 2f2b7e4..0429ae8 100644 --- a/src/layouts/BaseLayout.astro +++ b/src/layouts/BaseLayout.astro @@ -20,6 +20,8 @@ interface Props { modifiedTime?: Date; articleTags?: string[]; jsonLd?: Record | Record[]; + /** Page-scoped feeds (category/tag) advertised alongside the site-wide ones. */ + feeds?: Array<{ title: string; href: string }>; /** Opt into the wider content tier on roomy viewports (grid/listing pages). */ wide?: boolean; } @@ -40,6 +42,7 @@ const { modifiedTime, articleTags, jsonLd, + feeds, wide, } = Astro.props; const siteName = 'Unique'; @@ -77,6 +80,7 @@ const year = new Date().getFullYear(); modifiedTime={modifiedTime} articleTags={articleTags} jsonLd={jsonLd} + feeds={feeds} /> @@ -123,6 +127,7 @@ const year = new Date().getFullYear();
  • Home
  • Grid
  • Sources
    • +
  • Feeds
    • diff --git a/src/lib/feed.ts b/src/lib/feed.ts new file mode 100644 index 0000000..46a168b --- /dev/null +++ b/src/lib/feed.ts @@ -0,0 +1,130 @@ +import rss from '@astrojs/rss'; +import { getCollection } from 'astro:content'; + +/** + * One source of truth for everything syndicated: the main /rss.xml and + * /feed.json feeds plus every per-category and per-tag feed derive their + * items from here. Mirrors getAllEntries() (reviews + posts + bundles — + * finds stay bundle-only) but keeps the description field feeds need. + */ +export type FeedItem = { + title: string; + description: string; + pubDate: Date; + /** Site-relative path, e.g. /reviews/foo/ */ + link: string; + category: string; + tags: string[]; +}; + +export const FEED_TITLE = 'Unique'; +export const FEED_DESCRIPTION = + 'A small log of delightful, unique things — products, apps, phenomena, oddities.'; + +let itemsCache: Promise | null = null; + +export function getFeedItems(): Promise { + if (!itemsCache) itemsCache = buildFeedItems(); + return itemsCache; +} + +async function buildFeedItems(): Promise { + const [reviews, posts, bundles] = await Promise.all([ + getCollection('reviews'), + getCollection('posts'), + getCollection('bundles'), + ]); + + const items: FeedItem[] = [ + ...reviews.map((r) => ({ + title: r.data.name, + description: r.data.description ?? r.data.subtitle ?? '', + pubDate: r.data.date, + link: `/reviews/${r.id}/`, + category: r.data.category, + tags: r.data.tags, + })), + ...posts.map((p) => ({ + title: p.data.title, + description: p.data.description ?? '', + pubDate: p.data.date, + link: `/posts/${p.id}/`, + category: p.data.category, + tags: p.data.tags, + })), + ...bundles.map((b) => ({ + title: b.data.title, + description: b.data.description ?? b.data.blurb ?? '', + pubDate: b.data.date, + link: `/bundles/${b.id}/`, + category: b.data.category, + tags: b.data.tags, + })), + ]; + + return items.sort((a, b) => b.pubDate.valueOf() - a.pubDate.valueOf()); +} + +/** Shape @astrojs/rss expects. */ +export function toRssItems(items: FeedItem[]) { + return items.map((i) => ({ + title: i.title, + description: i.description, + pubDate: i.pubDate, + link: i.link, + categories: [i.category, ...i.tags], + })); +} + +/** + * Render an RSS 2.0 response with the trimmings every feed on the site + * shares: language, an atom:link self reference, and the browser-facing + * XSL stylesheet. + */ +export function renderRss(opts: { + site: URL; + /** Site-relative path of the feed itself, e.g. /rss.xml */ + path: string; + title: string; + description: string; + items: FeedItem[]; +}) { + const selfUrl = new URL(opts.path, opts.site).href; + return rss({ + title: opts.title, + description: opts.description, + site: opts.site, + items: toRssItems(opts.items), + stylesheet: '/feed.xsl', + xmlns: { atom: 'http://www.w3.org/2005/Atom' }, + customData: [ + 'en-us', + ``, + ].join(''), + }); +} + +export type FeedScope = { name: string; count: number }; + +export async function getFeedCategories(): Promise { + return scopeCounts((i) => [i.category]); +} + +export async function getFeedTags(): Promise { + return scopeCounts((i) => i.tags); +} + +async function scopeCounts( + pick: (i: FeedItem) => string[] +): Promise { + const items = await getFeedItems(); + const counts = new Map(); + for (const item of items) { + for (const name of pick(item)) { + counts.set(name, (counts.get(name) ?? 0) + 1); + } + } + return [...counts.entries()] + .map(([name, count]) => ({ name, count })) + .sort((a, b) => a.name.localeCompare(b.name)); +} diff --git a/src/lib/seo.ts b/src/lib/seo.ts index 7dd5758..9a5a10f 100644 --- a/src/lib/seo.ts +++ b/src/lib/seo.ts @@ -106,6 +106,14 @@ export const WEBSITE = { 'A small log of delightful, unique things — products, apps, phenomena, oddities.', publisher: { '@id': `${SITE_URL}/#org` }, inLanguage: 'en-US', + potentialAction: { + '@type': 'SearchAction', + target: { + '@type': 'EntryPoint', + urlTemplate: `${SITE_URL}/search/?q={search_term_string}`, + }, + 'query-input': 'required name=search_term_string', + }, }; const SOFTWARE_CATEGORIES = new Set([ diff --git a/src/pages/categories/[category].astro b/src/pages/categories/[category].astro index 4c306a5..3927d97 100644 --- a/src/pages/categories/[category].astro +++ b/src/pages/categories/[category].astro @@ -60,6 +60,7 @@ const breadcrumbs = buildBreadcrumbs([ lastUpdated={lastUpdated} ogImage={ogImage} jsonLd={[collectionPage, itemList, breadcrumbs]} + feeds={[{ title: `Unique — ${category} (RSS)`, href: `/categories/${category}/rss.xml` }]} wide > diff --git a/src/pages/categories/[category]/rss.xml.js b/src/pages/categories/[category]/rss.xml.js new file mode 100644 index 0000000..886be45 --- /dev/null +++ b/src/pages/categories/[category]/rss.xml.js @@ -0,0 +1,18 @@ +import { getFeedItems, getFeedCategories, renderRss, FEED_TITLE } from '../../../lib/feed'; + +export async function getStaticPaths() { + const categories = await getFeedCategories(); + return categories.map(({ name }) => ({ params: { category: name } })); +} + +export async function GET(context) { + const { category } = context.params; + const items = (await getFeedItems()).filter((i) => i.category === category); + return renderRss({ + site: context.site, + path: `/categories/${category}/rss.xml`, + title: `${FEED_TITLE} — ${category}`, + description: `Entries in ${category} on ${FEED_TITLE}.`, + items, + }); +} diff --git a/src/pages/feed.json.js b/src/pages/feed.json.js new file mode 100644 index 0000000..e76de82 --- /dev/null +++ b/src/pages/feed.json.js @@ -0,0 +1,28 @@ +import { getFeedItems, FEED_TITLE, FEED_DESCRIPTION } from '../lib/feed'; + +// JSON Feed 1.1 — https://jsonfeed.org/version/1.1 +export async function GET(context) { + const items = await getFeedItems(); + + const feed = { + version: 'https://jsonfeed.org/version/1.1', + title: FEED_TITLE, + home_page_url: context.site.href, + feed_url: new URL('/feed.json', context.site).href, + description: FEED_DESCRIPTION, + language: 'en-US', + authors: [{ name: 'rzen', url: new URL('/about/', context.site).href }], + items: items.map((i) => ({ + id: new URL(i.link, context.site).href, + url: new URL(i.link, context.site).href, + title: i.title, + content_text: i.description, + date_published: i.pubDate.toISOString(), + tags: [i.category, ...i.tags], + })), + }; + + return new Response(JSON.stringify(feed, null, 2), { + headers: { 'Content-Type': 'application/feed+json' }, + }); +} diff --git a/src/pages/feeds.astro b/src/pages/feeds.astro new file mode 100644 index 0000000..d4287d1 --- /dev/null +++ b/src/pages/feeds.astro @@ -0,0 +1,149 @@ +--- +import BaseLayout from '../layouts/BaseLayout.astro'; +import { getFeedCategories, getFeedTags } from '../lib/feed'; +import { buildOgImage, buildBreadcrumbs, absoluteUrl } from '../lib/seo'; + +const categories = await getFeedCategories(); +const tags = await getFeedTags(); + +const description = + 'Subscribe to Unique: RSS and JSON feeds for everything, plus per-category and per-tag feeds.'; + +const ogImage = buildOgImage(undefined, 'site', 'Feeds on Unique'); + +const collectionPage = { + '@context': 'https://schema.org', + '@type': 'CollectionPage', + '@id': absoluteUrl('/feeds/#collection'), + name: 'Feeds — Unique', + description, + url: absoluteUrl('/feeds/'), + inLanguage: 'en-US', +}; + +const breadcrumbs = buildBreadcrumbs([ + { name: 'Home', url: '/' }, + { name: 'Feeds', url: '/feeds/' }, +]); +--- + + +

    Feeds

    +

    + Everything published here — reviews, bundles, posts — is available as a + feed. Paste any of these into your feed reader. +

    + +

    Everything

    + + +

    By category

    +

    Each category page also advertises its own feed for autodiscovery.

    +
      + {categories.map(({ name, count }) => ( +
    • + {name} + ({count}) + RSS +
      • + ))} +
    + +

    By tag

    +
      + {tags.map(({ name, count }) => ( +
    • + #{name} + ({count}) + RSS +
      • + ))} +
    + +

    For robots

    +

    Reading the site programmatically? You may also want:

    +
      +
    • + /search.json — the full index of everything on + the site as JSON, including individual finds (which don't appear in the + feeds — they surface through bundles) +
      • +
    • + /llms.txt — a plain-text map of the site for + language models and other tools +
      • +
    • + /sitemap-index.xml — every page, for crawlers +
      • +
    • + Every page embeds schema.org JSON-LD (Review, + Product, Article, CollectionPage…) +
      • +
    + + +     diff --git a/src/pages/llms.txt.ts b/src/pages/llms.txt.ts new file mode 100644 index 0000000..4db360e --- /dev/null +++ b/src/pages/llms.txt.ts @@ -0,0 +1,81 @@ +import type { APIContext } from 'astro'; +import { getCollection } from 'astro:content'; +import { FEED_DESCRIPTION } from '../lib/feed'; + +// llms.txt — https://llmstxt.org/ — a plain-Markdown map of the site for +// language models and other programmatic readers. Regenerated on every +// build from the content collections, so it never goes stale. +export async function GET(context: APIContext) { + const site = context.site!; + const abs = (path: string) => new URL(path, site).href; + + const [reviews, bundles, posts, finds] = await Promise.all([ + getCollection('reviews'), + getCollection('bundles'), + getCollection('posts'), + getCollection('find'), + ]); + + const byDateDesc = (a: T, b: T) => + b.data.date.valueOf() - a.data.date.valueOf(); + + const reviewLines = reviews + .sort(byDateDesc) + .map( + (r) => + `- [${r.data.name}](${abs(`/reviews/${r.id}/`)}): ${r.data.description ?? r.data.subtitle}` + ); + + const bundleLines = bundles + .sort(byDateDesc) + .map( + (b) => + `- [${b.data.title}](${abs(`/bundles/${b.id}/`)}): ${b.data.description ?? b.data.blurb ?? `${b.data.items.length} finds`}` + ); + + const postLines = posts + .sort(byDateDesc) + .map( + (p) => `- [${p.data.title}](${abs(`/posts/${p.id}/`)}): ${p.data.description ?? ''}` + ); + + const body = `# Unique + +> ${FEED_DESCRIPTION} One person's editorial log, published daily at ${site.href} — every entry is something genuinely delightful, hand-picked and written up. + +Content lives in four collections: **reviews** (full write-ups), **bundles** (themed roundups), **posts** (essays), and **finds** (short captures — ${finds.length} so far — that surface only through the bundles referencing them, at /find//). + +## Feeds + +- [Everything (RSS 2.0)](${abs('/rss.xml')}) +- [Everything (JSON Feed 1.1)](${abs('/feed.json')}) +- Per-category RSS at /categories//rss.xml and per-tag RSS at /tags//rss.xml — all listed at [/feeds/](${abs('/feeds/')}) + +## Machine-readable + +- [Full site index as JSON](${abs('/search.json')}): every review, bundle, post, and find with title, URL, date, tags, and blurb +- [Sitemap](${abs('/sitemap-index.xml')}) +- Every page embeds schema.org JSON-LD (Review, Product, SoftwareApplication, Article, CollectionPage, BreadcrumbList) + +## Reviews + +${reviewLines.join('\n')} + +## Bundles + +${bundleLines.join('\n')} + +## Posts + +${postLines.join('\n')} + +## About + +- [About the site](${abs('/about/')}) +- [Legal & disclosures](${abs('/legal/')}): affiliate-link policy, privacy +`; + + return new Response(body, { + headers: { 'Content-Type': 'text/plain; charset=utf-8' }, + }); +} diff --git a/src/pages/rss.xml.js b/src/pages/rss.xml.js index 6d9fb2e..eff5e7f 100644 --- a/src/pages/rss.xml.js +++ b/src/pages/rss.xml.js @@ -1,43 +1,11 @@ -import rss from '@astrojs/rss'; -import { getCollection } from 'astro:content'; +import { getFeedItems, renderRss, FEED_TITLE, FEED_DESCRIPTION } from '../lib/feed'; export async function GET(context) { - const [reviews, posts, bundles] = await Promise.all([ - getCollection('reviews'), - getCollection('posts'), - getCollection('bundles'), - ]); - - const items = [ - ...reviews.map((r) => ({ - title: r.data.name, - description: r.data.description ?? r.data.subtitle ?? '', - pubDate: r.data.date, - link: `/reviews/${r.id}/`, - categories: [r.data.category, ...r.data.tags], - })), - ...posts.map((p) => ({ - title: p.data.title, - description: p.data.description ?? '', - pubDate: p.data.date, - link: `/posts/${p.id}/`, - categories: [p.data.category, ...p.data.tags], - })), - ...bundles.map((b) => ({ - title: b.data.title, - description: b.data.description ?? b.data.blurb ?? '', - pubDate: b.data.date, - link: `/bundles/${b.id}/`, - categories: [b.data.category, ...b.data.tags], - })), - ].sort((a, b) => b.pubDate.valueOf() - a.pubDate.valueOf()); - - return rss({ - title: 'Unique', - description: - 'A small log of delightful, unique things — products, apps, phenomena, oddities.', + return renderRss({ site: context.site, - items, - customData: 'en-us', + path: '/rss.xml', + title: FEED_TITLE, + description: FEED_DESCRIPTION, + items: await getFeedItems(), }); } diff --git a/src/pages/tags/[tag].astro b/src/pages/tags/[tag].astro index ccd9fc9..e7390b2 100644 --- a/src/pages/tags/[tag].astro +++ b/src/pages/tags/[tag].astro @@ -60,6 +60,7 @@ const breadcrumbs = buildBreadcrumbs([ lastUpdated={lastUpdated} ogImage={ogImage} jsonLd={[collectionPage, itemList, breadcrumbs]} + feeds={[{ title: `Unique — #${tag} (RSS)`, href: `/tags/${tag}/rss.xml` }]} wide > diff --git a/src/pages/tags/[tag]/rss.xml.js b/src/pages/tags/[tag]/rss.xml.js new file mode 100644 index 0000000..a90ad44 --- /dev/null +++ b/src/pages/tags/[tag]/rss.xml.js @@ -0,0 +1,18 @@ +import { getFeedItems, getFeedTags, renderRss, FEED_TITLE } from '../../../lib/feed'; + +export async function getStaticPaths() { + const tags = await getFeedTags(); + return tags.map(({ name }) => ({ params: { tag: name } })); +} + +export async function GET(context) { + const { tag } = context.params; + const items = (await getFeedItems()).filter((i) => i.tags.includes(tag)); + return renderRss({ + site: context.site, + path: `/tags/${tag}/rss.xml`, + title: `${FEED_TITLE} — #${tag}`, + description: `Entries tagged #${tag} on ${FEED_TITLE}.`, + items, + }); +}