Skip to content

fix(sqs): harden delayed long-poll, attribute MD5, and host-derived queue URLs - #1

Merged
ehlerst merged 1 commit into
ehlerst:mainfrom
tehlers320:sqs/cid-integration-hardening
Nov 21, 2026
Merged

ehlerst merged 1 commit into
ehlerst:mainfrom
tehlers320:sqs/cid-integration-hardening

Conversation

@tehlers320

@tehlers320 tehlers320 commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

SQS Hardening

Fixes

  1. Long-poll ReceiveMessage missed delayed messages. With queue-level or per-message DelaySeconds and a WaitTimeSeconds long poll, the poll loop only re-scanned on new-send notifications, so delayed messages becoming visible mid-poll were not picked up until the next call. The loop now also wakes at the earliest future visible_at.
  2. Missing canonical MD5OfMessageAttributes. SendMessage/SendMessageBatch (JSON and query) now return SQS's canonical message-attribute digest (attributes sorted by name; per attribute: 4-byte big-endian length + name, length + DataType, transport byte 0x01 String / 0x02 Binary, length + value). The JSON SendMessageBatch handler also previously dropped entry MessageAttributes — now parsed, stored, echoed on receive, and digested per batch entry.
  3. Queue URLs were hardcoded to http://localhost:4566/<acct>/<name>. CreateQueue/GetQueueUrl/ListQueues now build URLs from the request's Host header (LocalStack parity), so SDKs pointed at the emulator on a container host or non-default port keep the returned URLs usable; falls back to the previous default when no Host header is present. Queue lookup remains name-based, so previously issued URLs and AWS-shaped URLs keep working.

Tests

  • test_sqs_delayed_long_poll_lifecycle_on_json_protocol — end-to-end JSON lifecycle: create-with-DelaySeconds, GetQueueUrl, SendMessage through an AWS-shaped URL, delayed-message long-poll receive, purge + empty long-poll.
  • test_sqs_send_message_attributes_canonical_md5 — attribute MD5 digest asserted against reference vectors, plus attribute round-trip on receive.
  • test_sqs_send_message_batch_keeps_attributes — batch entry attributes are kept and reported with their canonical digest.
  • test_sqs_queue_urls_follow_request_host — queue URLs reflect the request Host header; cross-host queue addressing still resolves.
  • Engine unit tests for calculate_message_attributes_md5 and resolve_queue_name.

Audit of the content-image-demand SQS sender path (Kotlin SqsSenderService
awsJson1.0 SendMessage + its integration wiring, and the cid-emissions
Rust SqsTransport wire payloads) found three gaps; this fixes them:

- ReceiveMessage long-poll now wakes when delayed messages become
  visible: queue-level DelaySeconds + WaitTimeSeconds long-polls
  previously slept to the deadline and missed the message.
- SendMessage/SendMessageBatch now return the canonical SQS
  MD5OfMessageAttributes (sorted names, 4-byte big-endian length
  prefixes, 0x01 String / 0x02 Binary transport bytes), and the JSON
  batch handler no longer drops entry MessageAttributes.
- CreateQueue/GetQueueUrl/ListQueues queue URLs follow the request
  Host header instead of a hardcoded localhost:4566 (LocalStack
  parity), so SDKs inside containers keep the returned URLs usable.

Adds CID-shaped compat tests (create-with-delay + GetQueueUrl + send
through a real-AWS-shaped queue URL + delayed long-poll receive +
purge/empty-poll, attribute MD5 vectors, batch attributes, Host-header
URLs) and engine unit tests for the attribute digest and queue-URL
resolution.

@ehlerst ehlerst left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed, compiled, tested, and benchmarked locally:

  • cargo fmt --all -- --check: pass
  • RUSTFLAGS="-D warnings" cargo clippy --workspace --all-targets --all-features: pass
  • cargo test --workspace --all-targets --all-features: all tests pass (including the 4 new SQS compat tests and 2 new unit tests)
  • Independently verified the AWS canonical MD5OfMessageAttributes test vectors
  • Benchmarks (ruststack-benchmarks --service sqs + Criterion sqs_bench vs main): zero regressions (receive_and_delete_10 -4.8%, all other ops within noise, A/A+ ratings maintained)

Merging.

@ehlerst
ehlerst merged commit f90ddd9 into ehlerst:main Nov 21, 2026
@ehlerst ehlerst changed the title fix(sqs): harden SendMessage path for CID integration testing fix(sqs): harden delayed long-poll, attribute MD5, and host-derived queue URLs Nov 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants