Repository navigation
Run kubo on go-libp2p v0.52.0 on every content and bootstrap node - #41
Conversation
go-libp2p v0.52.0 is the first release with the inbound-slot leak fix (libp2p/go-libp2p#3568). No kubo release uses it yet, so sing-1's content node and chic-1's bootstrap-b run ipfs/kubo v0.43.1 rebuilt on it, in place of v0.43.1 plus only the fix, before every box moves to it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude reviewThis PR changes only the kubo image digest for the two canaries (sing-1 content, chic-1 bootstrap-b), plus comments and one README sentence. The digest is the same full 64-hex value in both files. The old fix-only digest remains only in a comment, which is correct for the rollback note. The controls (lith-1, chic-1 No issues found. claude-sonnet-5-5 at high effort · requested by TakGN · re-run with |
No kubo release ships go-libp2p v0.52.0 yet, and one will take a while. Move the content and bootstrap role defaults to our v0.43.1 rebuild on v0.52.0, which fixes the inbound-slot leak, and drop the canary overrides on sing-1 and chic-1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Part of libp2p/go-libp2p#3564 and ipfs/kubo#11480. Follows #37.
Why
go-libp2p v0.52.0 (2026-11-14) is the first release with the inbound-slot leak fix, libp2p/go-libp2p#3568. Upstream asked us to move to the release and keep reporting the resource-manager gauges. No kubo release uses it yet, and one will take a while, so this moves every content and bootstrap kubo to our own build of it now. kubo v0.43.1 replaces go-libp2p with
lidel/go-libp2p@9c4ee60, so a release-built kubo can't pick it up withgo get; ipfs/kubo#11480 asks for a kubo release with it.The fix-only canaries, 2026-11-11 to 2026-11-16
These are the two canaries #37 put on v0.43.1 plus only the five lines of #3568. Leaked inbound slots = rcmgr system inbound − transient − live inbound (swarm opened − closed). Hub Prometheus, 2026-11-16 14:43 UTC:
Over those five days the canaries read 0–1, the scrape-timing noise upstream told us to expect. Their traffic matches their controls': bootstrap-b holds 17,008 live inbound connections (9,708 QUIC, 6,330 TCP) against 17,515 on bootstrap next to it. Both open about 20 inbound connections a second. Neither canary restarted or fired an alert. The results are in libp2p/go-libp2p#3564 (18 h comparison); they are why the fix went into v0.52.0.
What else changes
Unlike the fix-only build, this one changes more than the fix. Moving from lidel's v0.49-based fork to v0.52.0,
go mod tidyalso raises shared dependencies:Upstream has only checked that kubo starts on it. The fix itself has five days of production behind it; the dependency bumps don't. So the rollout below goes one box at a time, starting with the two former canary nodes, and the rollback is one digest.
The image
ghcr.io/ipni/kubo:v0.43.1-libp2p-v0.52.0(sha256:71d7b748…), amd64, pushed by hand, pullable without logging in.acc53c0) with kubo's own Dockerfile, unchanged. The only source change is ingo.mod/go.sum:go mod edit -replace github.com/libp2p/go-libp2p=github.com/libp2p/go-libp2p@v0.52.0, thengo mod tidy. Thego 1.26.5line is unchanged.p2p/net/upgrader/listener.gohas the fix.go version -magainst upstreamipfs/kubo:v0.43.1shows the same Go 1.26.5,CGO_ENABLED=1,-trimpathand amd64 v1, with=> github.com/libp2p/go-libp2p v0.52.0replacing the lidel line, plus the dependency bumps above. The version string is unchanged (0.43.1-acc53c0)./debug/metrics/prometheusstill exportslibp2p_rcmgr_connectionsandlibp2p_swarm_connections_*, so the leak gauge and the*InboundSlotsLeakingalerts in ipni/infra keep working.Change
roles/content/defaults/main.yml:content_kubo_image→ the v0.52.0 build, for every content box. The comment records the build, the fix-only results and the upstream digest to roll back to.roles/bootstrap/defaults/main.yml:bootstrap_default_kubo_image→ the same digest, for every bootstrapper instance.host_vars/sing-1/content.yml,host_vars/chic-1/bootstrap.yml: drop the canary overrides from Run a kubo with the go-libp2p inbound-slot leak fix on sing-1's content node and chic-1's bootstrap-b #37; both nodes now take the role default.README.md: the Images note says what kubo runs and why.Checked
ansible-playbook --syntax-checkoncontent.ymlandbootstrap.yml.scripts/render-manifests.ymlrendersghcr.io/ipni/kubo@sha256:71d7b748…for bothk8s/contentandk8s/bootstrap; nokubo_imageoverride is left inhost_varsorgroup_vars.scripts/recover-upstream-pinset.shruns the content default locally, so on an arm64 machine it now runs under emulation.yamllint,ansible-lint(CI runs them).After merging
Merging deploys nothing. The baseline for every content and bootstrap kubo was taken on 2026-11-16 at 14:43 UTC from the hub Prometheus: leak, live connections by direction and transport, rcmgr blocks and streams, RSS, CPU, goroutines and fds.
Each run restarts that box's kubo(s), which clears the leaked slots. One box at a time,
--check --difffirst (expect only the kubo image lines to change), and a look at the box's kubo (pod up, peers, no alerts) before the next:Then, for a few days, compare against the baseline:
The
*InboundSlotsLeakingalerts in ipni/infra stay as they are, as a guard.Rollback: set both role defaults back to
ipfs/kubosha256:b293923d…(upstream v0.43.1; the full digest is in the content defaults comment), or toghcr.io/ipni/kubosha256:44f66498…(v0.43.1 plus only the fix), and re-run the same commands.When kubo ships a release with go-libp2p ≥ v0.52.0 (ipfs/kubo#11480), move both defaults back to
ipfs/kubo.🤖 Generated with Claude Code