Agent Skills package for researching any topic across Reddit, X, YouTube, and web. Installable across Claude Code (most common host), Codex, Cursor, GitHub Copilot, Gemini CLI, Grok (xAI), and 50+ other Agent Skills hosts. Python scripts with multi-source search aggregation.
skills/last30days/SKILL.md— canonical skill definition / runtime spec the model reads when the slash command fires; its reference-routing table gates the stage and mode instructions inskills/last30days/references/skills/last30days/scripts/last30days.py— main research engineskills/last30days/scripts/lib/— search, enrichment, rendering modulesskills/last30days/scripts/lib/vendor/bird-search/— vendored X search clientdocs/solutions/— documented solutions to past problems (bugs, best practices, workflow patterns), organized by category with YAML frontmatter (module,tags,problem_type)CONCEPTS.md— shared domain vocabulary (Skill, Engine, Harness, Beta channel) — relevant when orienting to the codebase or discussing project terminologyCONFIGURATION.md— user-facing knobs (env vars, flags, per-host install patterns); keep in sync per the rules belowCHANGELOG.md— structured release history built by towncrier at release time (launch copy lives in GitHub Releases)changelog.d/— per-PR news fragments; feature PRs write here, never editCHANGELOG.mddirectlyCONTRIBUTING.md— setup, fragments, and release notes for humans and agents (towncrier is release-only).github/scripts/prepare_release.py— lockstep version bump + towncrier build (release PRs only)HERMES_SETUP.md— install instructions for the Hermes harness specificallymcp/— Go MCP server (.mcpbbundle for Claude Desktop) exposingresearchandpreflight(what a research run would read, write, and contact) tools. It embeds a copy ofskills/last30days/scripts/(gitignoredmcp/internal/engine/vendored/, regenerated bymcp/scripts/sync-engine.sh) and shells out topython3. Seemcp/README.md.
- This is an Agent Skills package, not a CLI tool. The product is the slash-command-invoked skill (
/last30days <topic>in most harnesses);scripts/last30days.pyis implementation. Claude Code is the most common host but not the only one — features must work across every harness the skill installs into. - Feature design starts from the slash-command UX. A new engine flag with no SKILL.md integration (the root or a routed reference) is incomplete — the model invoking the skill won't know the flag exists.
- README and PR examples show
/last30days <topic>first. Direct CLI invocation (python3 scripts/last30days.py ...) is a fallback for scripting, cron, and dev-time engine testing; label it as such, never as the primary path. - Slash commands don't pass shell mechanics through.
/last30days OpenClaw --emit=html | pbcopyis invalid in any harness — either use the slash form (no flags or pipes; let the model translate user intent into engine flags) or use the direct CLI form (fullpython3 ...with explicit flags and a real shell).
# Dev/fallback: direct engine invocation (scripting, cron, or engine testing only).
# Saves to $LAST30DAYS_MEMORY_DIR when set in shell or ~/.config/last30days/.env;
# add --save-dir <path> for a one-off override. Mirrors LAST30DAYS_STORE convention.
python3 skills/last30days/scripts/last30days.py "test query" --emit=compact
npx skills add . -g -y # copies skill into ~/.agents/skills/<name>/ (frozen at install time); re-run to sync working-tree edits — see Rules below
# Tests (pytest; tests/ discovery is configured in pyproject.toml)
uv run pytest # full suite
uv run pytest tests/test_dedupe_v3.py # single file
uv run pytest tests/test_dedupe_v3.py -k some_case # single case
uv run pytest --cov # with coverage (skips lib/vendor/)
PYTHONPATH=skills/last30days/scripts uv run python -m unittest tests.test_subproc -v # subprocess tests (Windows CI job; unittest skips conftest.py, so set the path)
uv run pytest tests/eval -x -s # research-quality eval (separate CI job)
# MCP server (Go, under mcp/)
bash mcp/scripts/sync-engine.sh # re-mirror the Python engine into the Go embed dir
(cd mcp && go test -race ./...)
# Release prep (maintainers / release automation — not feature PRs):
# Prefer GitHub Actions → "Prepare release". Local equivalent:
uv run python .github/scripts/prepare_release.py --bump patch # or --version X.Y.ZPython 3.12+ required. Use uv for the env; the venv lives at .venv/. CI (.github/workflows/validate.yml) runs the full suite in parallel with coverage (uv run pytest -n auto --dist worksteal --cov, failing below the fail_under floor), plus the subprocess tests, the eval suite, and the Go MCP tests after sync-engine.sh.
Agents open most PRs. Follow this so CHANGELOG.md stops conflicting and versions stay lockstep:
- Feature/fix PRs: add
changelog.d/<pr-or-issue>.<type>.md(added/changed/fixed/removed/deprecated/security) when the change belongs in the next release notes. Seechangelog.d/README.mdandCONTRIBUTING.md. Fill the PR template’s Summary, Agent disclosure, and Relationship sections. - Never edit
CHANGELOG.mdin a feature PR. Never bump version strings inpyproject.toml,SKILL.md, plugin/marketplace JSON, oruv.lockoutside a release PR. CI (changelog-guard.yml) enforces this. - Nothing for release notes: omit the fragment, check Skip changelog in the template, and add the
skip-changeloglabel. Dependabot PRs (authordependabot[bot]) skip that requirement automatically. - Cutting a release: run Actions → Prepare release (patch/minor/major). That opens a
chore(release): bump version to X.Y.ZPR which runs towncrier and bumps every lockstep surface. Merging tomaintriggers Tag release, which pushesvX.Y.Zand existingrelease.ymlpublishes.skill/.mcpbartifacts. Do not hand-edit ten version files. Contributors do not need a global towncrier install —uv sync --group dev(or the Action) provides it for release prep only. - Lockstep gate remains
tests/test_plugin_contract.py::test_versions_match_across_manifests. Workflow contract:tests/test_changelog_workflow.py.
lib/__init__.pymust be bare package marker (comment only, NO eager imports)- One-time setup:
npx skills add . -g -ycopies the skill into~/.agents/skills/<name>/(real directory) and, for harnesses that support symlinked skill dirs, drops a per-host symlink pointing at that copy. Working-tree edits do NOT propagate automatically — the~/.agents/skills/<name>/copy is frozen at install time. To sync after edits, re-runnpx skills add . -g -y. For live-edit on a dev machine, move that real directory to a backup outside~/.agents/skills/first, then create the working-tree symlink:ln -sfn "$PWD/skills/last30days" ~/.agents/skills/last30days(run from the repo root).ln -sfnreplaces an existing symlink but does not replace a real directory; with the install copy still present, it creates a link inside that directory and leaves the stale skill active. - Git remote: origin = public (
mvanhorn/last30days-skill) - Do not reduce
fail_underinpyproject.toml([tool.coverage.report]) without documenting why in the PR. The coverage gate is a floor meant to rise over time, not to be relaxed when new code is under-tested. - Every
lib/*.pycall tolog.source_log(...)must passtty_only=False. The default isTrue, which silently drops every line when stderr isn't a TTY (Claude Code, Codex, CI, captured output) — turning source observability into invisible failure. Enforced bytests/test_source_log_visibility.py. - CLI-gated optional sources (Digg via
digg-pp-cli, YouTube viayt-dlp) activate only whenshutil.whichresolves the binary on the agent subprocess PATH — not merely when the file exists on disk. First-run setup installs Digg through@mvanhorn/printing-press-library(default$HOME/.local/bin); Hermes/OpenClaw gateways often need that directory on PATH. Setup must distinguish PATH-visible installs from off-PATH binaries and must not claim "now active" unless the engine gate would pass. Seedocs/solutions/integration-issues/digg-cli-agent-path-setup-wizard.md. - First-run onboarding is consent-driven, model-led, and host-split. Spec lives in the root
SKILL.mdgate and Step 0 ofreferences/setup-wizard.md; contract locked bytests/test_onboarding_contract.py(andtests/test_grok_bot_host_contract.py).- The setup subprocess does only mechanical work (cookie reads, tool installs, GitHub device-auth, emitting the engine-owned welcome via
--welcome). It cannot prompt, so consent lives in the skill instructions. - Step 0 has THREE branches:
- Claude Code Modal Flow for hosts with modals — the restored v3.0.0
AskUserQuestion-driven NUX (welcome, Auto/Manual/Skip, cookie consent, ScrapeCreators offer,INCLUDE_SOURCESopt-in, first-topic picker). The welcome pitch is embedded in the modal's question, because the modal is the only always-fully-visible surface — a separate welcome message or--welcomeBash run gets buried behind "ctrl+o to expand". - Non-Modal Prose Flow for hosts without modals (OpenClaw, Codex, Cursor, Gemini CLI, Grok CLI) — relays
last30days.py --welcomeverbatim. - Grok Bot Prose Flow for
LAST30DAYS_HOST=grok-bot— leads with the bot's own X lanes (its built-in X tools first, then the X for Grok Bot connector;LAST30DAYS_X_HOST_LANE=1plus a--x-postsenvelope the model fetches itself, exported only when the fetch returned posts), persists any backup key (X_BEARER_TOKEN/XAI_API_KEY) only through the engine'ssetup --store-keypath, and never reads a browser session.
- Claude Code Modal Flow for hosts with modals — the restored v3.0.0
- The logged-in Reddit session has its own consent question (
references/reddit-session.md, routed from the root): asked after the X cookie answer (yes or skip) in the Modal and Non-Modal flows (a Reddit yes is its own consent and outranks a savedBROWSER_CONSENT=false; onlyFROM_BROWSER=offblocks it), once for returning users through the root gate'sREDDIT_OFFERtoken (REDDIT_SESSION_OFFEREDrecords it), and never on a Grok Bot host. X consent never implies it. - Do NOT collapse the modal flow back into a bare silent
setupcall or flatten it to prose-only — the guided modals are the feature (they eroded once and were restored). - The first two flows ask before reading cookies and surface the macOS Full Disk Access fix on permission-denied; all three offer the ScrapeCreators GitHub signup (10,000 free calls) on every first run.
- GitHub device auth is a two-command split:
setup --github-startreturns the code fast (foreground, copies to clipboard),setup --github-pollwaits for authorization (setup --githubchains both for back-compat). Success persistsSCRAPECREATORS_API_KEYviasetup_wizard.write_api_key(0o600) and masks the key in stdout. - Step 5 source opt-in has two tiers, both comment-enabled: Recommended (TikTok + Instagram posts and top comments, plus YouTube comments —
INCLUDE_SOURCES=tiktok,instagram,youtube_comments,tiktok_comments,instagram_comments) and Everything (also Threads + Pinterest). Comments are on by default (posts on → comments on for all three platforms). Threads and Pinterest are the only opt-in extras — they appear only in the Step 5 Everything option, never in the welcome or the Step 4 offer.
- The setup subprocess does only mechanical work (cookie reads, tool installs, GitHub device-auth, emitting the engine-owned welcome via
- Comment ranking. Instagram comments come from ScrapeCreators (
/v2/instagram/post/comments, ranked bycomment_like_count) with vote-weighting parity to YouTube/TikTok (a dedicated_instagram_engagementcarve-out plus_VOTE_LOG_REFERENCE/label/threshold entries). The "Top Community Comments" list (render._render_top_comments) is the report's quote pool, chosen and ordered by position, never by like count across items: each item (thread, video, reel, story) gives its top comment and its #2 by its own stored order (metadata["top_comments"]: score for Reddit, like count for TikTok and Instagram, YouTube's "top comments" order, Hacker News thread order), nothing below. All top comments come first, in the order of the stories (shown clusters as numbered, then other kept items as ranked), each story's standalone X/Threads/Bluesky/LinkedIn posts (10+ likes, labelledpost, a LinkedIn post only with a real author) and what its YouTube, TikTok or Instagram video said (one line per video with 10+ likes, labelledsaid: YouTube'stranscript_highlights, or the best highlightyoutube_yt.extract_transcript_highlightsdraws from a reel's or TikTok's spoken caption) after its top comments, then all #2 comments in the same order; within a story and position, entries on a candidate that cleared_BEST_TAKE_FUNNY_FLOORlead and endfunny(fun_scorebelongs to the candidate). The per-platform display floors (YouTube 50, TikTok 500...) do not apply; on a platform that counts likes, upvotes or reactions (Instagram, TikTok, YouTube, X, Reddit, Threads, Bluesky, LinkedIn, GitHub) an item joins only when its top community comment has at least 5 (a Reddit thread's floor scales with its own score,min(5, max(2, score // 20)), so a small on-topic thread is not shut out), and its #2 needs at least 1, while Hacker News comments are exempt. A bot's comment ([bot]and bot-named authors, "automated review" text, HTML-comment markup), a comment by the item's own author (channel, creator, OP, submitter; read from the stored comment'sis_uploaderflag where the source provides one, names compared only when it is absent) and one with nothing quotable left are never community comments: the next comment takes the position. Quote text also drops lines quoting the parent (>), and markdown emphasis markers. Every eligibility test runs before duplicates are removed. Up to 40 one-line entries, shown clusters covered first and taking turns for slots (one top comment each, then round by round, so a busy cluster cannot starve a later one), each in the quote line's own shape (dot, author,on r/subor platform,· count), then metadata: URL,cluster Norre: "title", andtop comment/#2 comment/post. Quote text is one line with raw URLs dropped, inner double quotes made single, a sentence-end cut when over 240 characters, and leading reply handles stripped from X posts. Compact output also prints## Posts by @handleand## Polymarket Markets(each on-topic market as a ready-to-copy odds line,📊 Polymarket · {subject}: {outcome} {price} · ..., at most three outcomes, movement in points after the outcome it belongs to, built from theoddssummarypolymarket._odds_summarystores on the item; then URL and metadata), and the command line (not the renderer) appends the## Before you write the reportchecklist to--emit=compactstdout only (lib/report_checklist.py).
- Never commit real API keys, browser cookies, auth tokens, app passwords, access tokens, or
.envcontents. - Use the env-based auth patterns in
skills/last30days/scripts/lib/env.py; tests and fixtures must use obvious dummy values only. - Keep examples safe by redacting secrets and avoiding copy/pasteable live credentials in docs, fixtures, and test data.
- Do not weaken or disable the advisory security workflow (
.github/workflows/security.yml) without explaining why in the PR description or review thread.
README.md is the canonical English README. When changing it, reflect the same substantive updates in README.fr.md, README.de.md, README.es.md, README.pt-BR.md, README.ja.md, README.zh-CN.md, README.ar.md, README.fa.md, and README.tr.md, preserving commands, links, tables, and reciprocal language navigation.
CONFIGURATION.md is the user-facing configuration reference — save paths, per-source API keys, web-search backend priority, trend-monitoring stack, per-client install patterns. Distinct from SKILL.md (the canonical runtime spec).
Update CONFIGURATION.md when:
- adding a new env var (e.g.
LAST30DAYS_*,BSKY_*,*_API_KEY) - adding a new CLI flag that affects configuration (e.g.
--store,--web-backend) - adding a new per-client install pattern (Claude Code, Gemini, Codex, Cursor, Grok, Hermes…)
- adding a new optional source that requires its own credential
- changing the priority order of config layers (per-run flag > env >
.envfile > defaults)
Keep the existing structure organized by how often each layer is touched: per-run flags → env vars / .env → optional trend-monitoring stack → per-client patterns. Add new content into the right section rather than appending at the end.
When a new config concept lands in SKILL.md or AGENTS.md, mirror the user-facing knob in CONFIGURATION.md so non-agent readers can configure the skill without reverse-engineering it from the runtime spec.
The repo doubles as a native Grok Build plugin via .grok-plugin/plugin.json + .grok-plugin/marketplace.json. Grok also reads .claude-plugin/* for compatibility; the native pair is the first-class lane and what an official xAI marketplace listing points at. The self-hosted catalog uses a bare Git URL source ({"source":"url","url":"https://github.com/mvanhorn/last30days-skill.git"}) so grok plugin marketplace add mvanhorn/last30days-skill tracks HEAD — not a self-referential local path: "." (Grok does not enumerate those). Version lockstep with Claude/Codex/Gemini manifests is enforced by tests/test_plugin_contract.py. Validate with grok plugin validate ..
Listing in xAI's official catalog (xai-org/plugin-marketplace) is a post-merge outbound PR to their repo that pins a full 40-char commit SHA — unlike our own .grok-plugin/marketplace.json, which uses a bare URL source and tracks HEAD. Step-by-step procedure: docs/reference/xai-marketplace-submission.md.
Experimental changes get tested on mvanhorn/last30days-skill-private, which installs as a parallel /last30days-beta slash command. Beta-only changes never ship to public without a review PR here. Workflow guide (and the plan that established this setup) lives at BETA.md in the private repo.