Repository navigation
Conversation
javascript-sdk dedups holdoutIds at install (insertUniqueSorted); java-sdk kept the duplicate in the applicable-holdout set. One holdout exposure was still queued (the cached holdout assignment's CAS), but the duplicate counted in holdoutSetMatches, so a refresh that only dropped it re-resolved the covered experiment and published a second exposure. Pinned by cross-sdk-tests Holdout Conformance scenario 500. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nder Unreleased
A mutant that only drops a repeat of the last resolved holdout passed the
suite, since the only test repeated ids back to back. {11, 12, 11} now pins
that any repeat counts once. 1.9.0 is already released, so the fix is listed
under a new Unreleased section.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…larify the changelog Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A dedup that only compared against the first kept holdout passed the suite. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. WalkthroughThe holdout resolver now adds each resolved holdout only once, while continuing to ignore missing or malformed references and sort results by ID. New tests check exposure counts for adjacent and non-adjacent duplicate IDs. They also check that removing duplicates during refresh preserves assignments and pending exposure counts. The changelog records the change for Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested reviewers: Merge Risk:
|
|
Independent sdk-review on eb9d467: APPROVE, 3 P2 fixed across 4 reviewer passes (a4da49c, 1a2fee6, 1eab5ae, eb9d467). Remaining P3s (an unnumbered release heading, and a reverse-refresh test) are tracked below and will be addressed in the review loop. Final reviewer report (eb9d467), with evidence logAPPROVE at Compared published Must fixNone. No P1 or P2. Tests needed (current implementation correct)10 full-suite mutations (core-api + core-api-java6, JDK 11): 9 killed, 1 survived.
Compatibility / parity / forward-compat evidence
PerformanceStatic finding; no timing claimed. The dedup runs once per experiment at data install ( P3 / notes
Version and release notesHead Pre-existing on main (not this PR)
Verification limits / provenance: Spotless (formatter download), Animal Sniffer ( sdk-review evidence: java-sdk (branch fix/dedup-duplicate-holdout-ids, no PR yet)
1. Trees
2. Backwards compatibility
3. Parity
4. Tests and mutation
5. Performance
6. Forward compatibility
7. Build / CI
8. Docs / CHANGELOG
Findings
Verification limits
|
|
Review submitted The automated review has been submitted. See the review for the verdict and any findings. Head: |
|
Question Q1 — Confirm duplicate-holdout contract before merge Has the responsible holdout/cross-SDK owner confirmed that repeated holdoutIds must count as one holdout, so this Java behavior change is intended to merge? The PR description explicitly leaves this decision open. Please confirm the intended contract (or link the owner's decision). Context: PR description explicitly says “owner decision needed before merge” on dedup as the cross-SDK contract; joalves's evidence comment repeats that it is pending. Context.java:2302 now deduplicates applicable holdouts, and tests pin unchanged exposures when refresh removes duplicate IDs. Checked all four commits, PR conversation and existing reviews; no confirmation is recorded. Why I'm asking: The implementation and tests can be verified independently, but approval would choose an explicitly unresolved product contract. The owner's answer determines merge readiness, not merely test adequacy. Suggested answers (free text is always welcome):
@joalves, please include |
|
Q1 (#25 (comment)): Not confirmed yet; keep the PR draft pending the owner's decision. No owner decision is recorded. This is open question 7 in the holdouts plan. The PR stays a draft until the owner confirms. Please review the implementation and tests on their merits: whether this fix makes Java match javascript-sdk, which already dedups at install, and does nothing else. The merge itself waits for that decision. |
jervasion-absmartly
left a comment
There was a problem hiding this comment.
❓ Needs a human decision
No blocking implementation defect found. The change deduplicates applicable holdouts at installation, consistent with javascript-sdk, and prevents duplicate-only refreshes from generating an extra experiment exposure. Merge readiness still depends on the explicitly open cross-SDK contract decision.
Next step: Keep this PR draft until the holdout/cross-SDK owner decides whether duplicate holdout IDs must count once.
Suggestions (optional)
- [nit] Three short comments repeat the adjacent code or test setup; optional cleanup is listed in the review details.
Human review required
Question: Should repeated holdoutIds count as one holdout across SDKs?
Why it matters: This head implements that behavior; an approval would endorse a contract that the PR explicitly leaves undecided. This is not a code-defect finding.
Evidence checked: Installation and refresh/cache consumers, both added regression tests, javascript-sdk's insertUniqueSorted installation path at 311cfca, the PR description, and joalves's Q1 answer, which confirms that no owner decision is recorded.
Needed from: The holdout/cross-SDK owner, now asked directly by the author.
What I checked
- ✅ Both artifacts pass their full suites on JDK 11 and JDK 25: 3,018 test executions, zero failures or skips.
- ✅ Adjacent and non-adjacent duplicates preserve pending exposure counts across refresh and publish the expected experiment/holdout exposures. Both new tests fail in both artifacts when only the deduplication condition is reverted.
- ✅ Independent error-handling review found no introduced failure-path defect; missing/malformed references and refresh failure propagation remain intact.
- ✅ Exact-head CI passed its full build, including formatting, compatibility and coverage gates.
Review details
Scope and intent
- Head:
eb9d467a45d5617401d452fdbdad3042faaed293. Base and merge-base:07807b516df5ad0d7d92e5b682041982802cd871. Initial review, round 1; resumed after Q1 was answered. Head/base remain unchanged. - Reviewed all four PR commits and the complete three-file diff: runtime resolver,
ContextHoldoutTest, and changelog. No API signatures, decoder, hashing or assignment algorithms change. Java 6 sources/tests are generated from the same source path. - The author reports JDK 11 tests and 313 cross-SDK scenarios passing. This review independently ran the repository suites and a production-hunk revert; it did not rerun the external cross-SDK orchestrator. JavaScript parity was additionally checked against its install-time ID comparator and unique-insertion implementation.
- Verified: the ID-keyed map supplies the same object for a repeated ID.
Experiment.equalsreturns immediately for identity and rejects distinct IDs before comparing nested fields. Thuscontainsdeduplicates IDs without collapsing distinct holdouts. Unique-reference inputs produce the same resolved array as before. Sorting and cache comparison remain unchanged. - Work shape: per experiment with k holdout references and u resolved unique holdouts, installation adds up to k list scans of at most u entries (worst-case quadratic); cached reads are unchanged. The PR describes these lists as a handful of entries. No measured latency claim is made.
Validation
- Verified Java/Javac 11.0.33; Gradle ran with
JAVA_HOME="${JAVA_HOME_11:?Select a verified JDK 11 first}". ./gradlew build --no-daemon: local execution stopped during Spotless configuration becausedownload.eclipse.orgcould not resolve. This was an environment failure, not a formatting result. Full-build coverage comes from exact-head CI below../gradlew :core-api:test :core-api-java6:test :core-api:testJdk25 :core-api-java6:testJdk25 --no-daemon: passed. XML totals: core-api 754 on each runtime; core-api-java6 755 on each runtime; zero failures, errors or skips.- Scratch revert changed only
if ((holdout != null) && !applicable.contains(holdout))back toif (holdout != null). Command:./gradlew :core-api:test :core-api-java6:test --tests '*ContextHoldoutTest.duplicateHoldoutIdIsOneHoldout' --tests '*ContextHoldoutTest.nonAdjacentDuplicateHoldoutIdIsOneHoldout' --continue --no-daemon. The core-api task collected 754 tests and failed only the two new tests; java6 collected two and failed both. Failures were attributable to post-refresh pending counts: expected 2/3, actual 3/4. Tests were unmodified. git diff --check: passed.- CI: Java-SDK run 44144165280, exact head, completed successfully; build job
134254623494, including successful Build Java SDK step. Workflow runs./gradlew -Porg.gradle.java.installations.fromEnv=JAVA_HOME_25_X64 clean build. Commit statuses contain no individual statuses; their empty aggregatependingis not a running check.
Passes
Passes (non-blocking): run: core runtime/API/data (resolver and refresh/exposure consumers); error handling/messages (mandatory independent pr-review-errors pass completed for installation, malformed references and refresh failure propagation; no material coverage gap); tests/CI/packaging (two new tests, observable exposure behavior and generated Java6 artifact); cross-cutting simplicity (local audit of the one-condition runtime change); comment hygiene; diff composition.
Not run: deployment/operations/observability (no manifests, release procedures or telemetry emissions changed); shared-component/frontend (no frontend source changed); behaviour-bearing data/policy (no runtime configuration, schemas, fixtures or policy changed).
Comment hygiene
Comment hygiene (non-blocking): 10 comment / 52 code lines, ratio ~1:5.2, 3 blocks flagged.
threshold T=5; restatements counted: yes (10 × C ≥ E).
core-api/src/test/java/com/absmartly/sdk/ContextHoldoutTest.java:638:own exposure + one holdout exposure— restates the assertion/expected payload.- Same file, line 660:
A repeat need not be adjacent or of the first id: {11, 12, 11, 12} is holdouts 11 and 12.— restates the named test and fixture. - Same file, line 670:
own exposure + one per holdout— restates the assertion/expected payload.
Three qualifying added lines are below T. Preserved the comments explaining cross-SDK semantics and the non-obvious relationship between duplicate references, cache matching and re-exposure.
Diff composition
Diff composition (non-blocking): every file is required for the change; none flagged.
Ran the pinned introduced_blobs.py helper successfully for the full baseline-to-head range: nine content-write occurrences across four commits and three paths. Reviewed their commit deltas as well as the head diff; no scratch artifacts or credential-bearing content found.
Simplifications
Simplifications (non-blocking): none. The existing list plus a membership guard is a local solution without new state or abstraction.
Author questions
- Q1: answered by joalves; confirmation is still absent, and the PR remains draft pending the owner's decision. The answer is consistent with the PR description and does not resolve the contract itself.
- Unanswered author questions: none. The outstanding owner decision is recorded above under Human review required.
|
Closing: the owner confirmed the backend never sends duplicate holdoutIds, so there's no behaviour to change. |
What
holdoutIdslisting the same holdout more than once (adjacent or not) now resolves to one holdout, as in javascript-sdk, which dedups at install withinsertUniqueSorted.Why
holdoutSetMatches.Change
resolveApplicableHoldoutsskips a holdout that is already in the list. It is O(n²) over one experiment'sholdoutIds, which is a handful of entries, and it is computed once persetData.core-api-java6is generated.ContextHoldoutTest:{11, 11}: one holdout exposure; a refresh to{11}re-exposes nothing; the publish carries the experiment and one holdout exposure.{11, 12, 11, 12}: a refresh to{11, 12}re-exposes nothing, and the publish carries the experiment and one exposure per holdout. This kills mutants that only drop adjacent repeats or only repeats of the first id.Unreleased→Fixedsection, since 1.9.0 is already released.Verification
./gradlew test -PskipJdk25on JDK 11: core-api 754 tests and core-api-java6 755 tests, 0 failures. Both new tests fail on main.holdoutIdsDedup: trueand passes all 313 scenarios, including 500.Linked: absmartly/cross-sdk-tests#23 (H1-H8; java lists
holdoutIdsDedupas a reviewed exception until this merges).🤖 Generated with Claude Code
Summary by CodeRabbit