Skip to content

About

Rust utility toolbox (std-only core, feature-gated): case conversion, slice/map/date helpers, snowflake/order/UUID IDs, Django-style filters, MySQL DDL parsing, priority event loop, data aggregation, weighted random, captcha, JWT, AES/SM2/SM3/SM4 crypto, password hashing, TLS setup, geoip, translation, distributed locks, BIN lookup, field masking.

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Repository files navigation

rust-utils

English | 简体中文 | 日本語


A Rust toolbox: case-style conversion, slice/map helpers, Snowflake IDs, order IDs, query-condition parsing, MySQL DDL parsing, a priority event loop, relation backfill, password hashing, AES/HMAC, JWT, random nickname generation, bank-card BIN lookup, image CAPTCHA, IP geolocation, machine translation, distributed locks, Chinese national crypto (SM), and more.

Design principles:

  • Core modules are zero-dependency (std only); capability areas with heavier dependencies (time, randomness, crypto, JWT, …) are enabled per-feature;
  • The whole crate carries #![forbid(unsafe_code)];
  • Boundary reads (geoip, ddl_parser, …) return errors or empty results on out-of-range or malformed input — never panic.

Usage

[dependencies]
rust-utils = "0.2"

# Enable features as needed, e.g.:
# rust-utils = { version = "0.2", features = ["chrono", "rand", "password"] }
# or everything: features = ["full"]

Modules

Module Feature Description
byteutil default Integer/byte conversions, ASCII case flips
stringcase default camel/snake/kebab conversion, acronym- and digit-segment aware
stringutil default Lenient numeric/boolean parsing, JSON field-value rewriting
sliceutil default Find family, intersection/difference/union, dedup, chunking
sqlutil 默认 Lexical SQL masking (MaskSQL) and table extraction (ExtractTables) for PostgreSQL-flavored SQL
maputil default keys/values/merge/drop/filter
mathutil default Statistics, hand-written Gaussian, Erfc/Ierfc
sensitive default Sensitive-word filtering: trie longest-match, interspersed-char skipping, masking
netutil 默认 SSRF protection (URL validation + blocked CIDRs + DNS resolve check), spoof-proof client IP via trusted proxy chain
pagination default Pagination offsets
cryptocurrency default Crypto wallet address format validation
query_parser json (for the JSON form) Django-style field__op filter/sort parsing
ddl_parser default MySQL CREATE TABLE parsing (hand-written lexer, zero deps)
eventloop default Single-threaded priority event loop, with a frame-driven mode
eventbus 默认 In-process event bus: subscribe/publish + middleware chain + multi-bus manager
aggregator default Relation backfill (lists/trees) + parallel executor + cached batch loader
converter 默认 REST path / operationID → resource:action permission code
authorizer 默认 (casbin/opa engines via authorizer-casbin/authorizer-opa) Authorization manager shell: Engine abstraction + noop/deny-all + built-in RBAC engine + casbin/OPA engines (fail-closed)
id default (uuid optional) Snowflake IDs, order IDs, machine codes (built-in SHA-256), UUID v4/v7
fsutil default (glob optional) File/path helpers
dateutil chrono Day-granularity truncation, range checks
timeutil chrono Today/yesterday/this-month/last-month ranges, time differences, format conversion
random rand Random toolkit: weighted choice/alias table/dice/jitter/gaussian/phone numbers, etc.
name_generator name-generator Random nicknames and Chinese/English/Japanese names (embedded 2.5 MB dictionaries)
slug slug URL slugs (unicode transliteration)
structutil structutil Struct field traversal: serde-flavored ForEach/ToMap (rename/skip semantics carried by serde attributes)
password password PBKDF2/bcrypt/argon2/HMAC/SHA hashing strategies + complexity validation
ossutil ossutil Object-storage helpers: MIME/magic-byte → bucket & extension mapping, filename strategies, upload guards, Aliyun OSS presigned URLs
crypto crypto AES-CBC/AES-GCM/HMAC/SHA-2, Encryptor (enc: prefix) envelope, unified Cipher/Signer faces
auditutil auditutil Audit log: client IP + tamper-evidence chain (deterministic hashing + ECDSA-P256 signing)
doctext doctext (PDF via doctext-pdf, xlsx via doctext-xlsx) Plain-text extraction from txt/md/csv/html/docx/pdf/xlsx
fieldperm fieldperm Field-level read/write trimming (path semantics aligned with the protoreflect behavior)
mailer mailer SMTP mail: STARTTLS / implicit SSL, sender display name
notify notify Multi-channel notification: webhook/IM/email/SMS/vendor push, SSRF-guarded outbound
jwt jwt JWT generate/parse/verify/refresh: HS256 shared-secret + RS*/PS*/ES* PEM asymmetric (alg pinned against confusion)
bank_card bank-card Luhn validation + BIN lookup (2,013 embedded records)
captcha captcha Image CAPTCHA: digit/alphanumeric/arithmetic/Chinese text drivers + slide/click/rotate, self-drawn rendering, JSON output compatible with the base64Captcha front-end component, pluggable storage (captcha-redis adds a Redis store)
fieldmask fieldmask Field masks: nested mask-tree building + filter/prune/overwrite/validate/path normalization, over serde_json::Value
tls tls TLS certificate loading: rustls server/client configs assembled from PEM files/bytes (one-way/mutual)
geoip geoip IP geolocation with three backends: qqwry (GB18030, province/city splitting), ip2region xdb v2 (vector index + segment bisection, searcher pool), MaxMind mmdb reader; data files are loaded by the caller
translator translator Translators for four backends: Baidu (MD5 signature) / Alibaba (RPC signature) / Google (bare v1 endpoint, v2/v3 REST) / Volc (HMAC-SHA256 derivation chain); one request per call, no automatic retry; request construction and signatures are offline-verifiable
distlock distlock Distributed locks: Locker/Lock abstractions with acquire options (Redis store via distlock-redis, wire-compatible with the bsm/redislock protocol; etcd store via distlock-etcd, lease+txn acquisition with block-wait support)
sm sm Chinese national crypto SM2 (C1C3C2/ASN.1 encryption & signatures) / SM3 / SM4-CBC
otp otp RFC 4226 HOTP / RFC 6238 TOTP (SHA-1/256/512, 6–8 digits, configurable window), otpauth:// URI building & parsing
qrcode qrcode (PNG via qrcode-png) QR codes: module matrix + SVG + terminal block chars; decode roundtrip covered in tests
xlsx xlsx Excel read/write: calamine reader + rust_xlsxwriter writer, write→read roundtrip tested in-crate
code_generator code-generator Template-driven codegen: mustache-subset renderer (missing fields fail fast) + guarded file output
mapper mapper entity↔DTO field mapping over serde_json: rename/nested move/enum conversion/defaults/whitelist
printrender printrender HTML/URL → PDF (margins/scale/header-footer/page ranges) and PNG/JPEG screenshots via local Chrome/Edge CDP

Examples

Case conversion (acronym- and digit-segment aware)

use rust_utils::stringcase;

assert_eq!(stringcase::snake_case("HTTPStatusCode"), "http_status_code");
assert_eq!(stringcase::snake_case("Numbers123Test"), "numbers123_test");
assert_eq!(stringcase::upper_camel_case("parse_url.do_parse"), "ParseUrlDoParse");
assert_eq!(stringcase::kebab_case("Hello World!"), "hello-world");

Snowflake IDs and order IDs

use rust_utils::id;

let node = id::SnowflakeNode::new(1).unwrap();
let a = node.generate();
let b = node.generate();
assert!(b > a); // monotonically increasing

let order = id::generate_order_id_with_random("ORD", None);         // ORD + 14-digit timestamp + random digits
let order = id::generate_order_id_with_increase_index("ORD", None); // ORD + 14-digit timestamp + incrementing index

Query condition parsing

use rust_utils::query_parser;

let mut got = Vec::new();
query_parser::parse_filter_query_string(
    "name__icontains:%E5%BC%A0,age__gte:18",
    |field, op, value| got.push((field.to_string(), op.to_string(), value.to_string())),
).unwrap();
// got: [("name", "icontains", "张"), ("age", "gte", "18")]

MySQL DDL parsing

use rust_utils::ddl_parser;

let table = ddl_parser::parse_create_table(
    "CREATE TABLE `users` (\
        `id` BIGINT AUTO_INCREMENT PRIMARY KEY, \
        `userName` VARCHAR(100) NOT NULL COMMENT '用户名'\
    ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='用户表'",
).unwrap();

assert_eq!(table.name, "users");
assert!(table.columns[0].auto_increment);
assert_eq!(table.columns[1].comment, "用户名");
assert_eq!(table.engine, "innodb");

Priority event loop

use rust_utils::eventloop::{new_request_event, EventLoop, EventProcessor, EventResult};
use std::sync::{Arc, Mutex};
use std::time::Duration;

struct Collector(Arc<Mutex<Vec<String>>>);

impl EventProcessor<String> for Collector {
    fn process(&mut self, event: Event<String>) -> EventResult<String> {
        self.0.lock().unwrap().push(event.event_type.clone());
        EventResult::ok(event.data)
    }
}

let seen = Arc::new(Mutex::new(Vec::new()));
let loop_ = EventLoop::new(16, Collector(seen.clone()), false);
loop_.start();

// request events carry a receipt, handy for waiting on completion
let (ev, rx) = new_request_event("ping", Some("hello".to_string()));
loop_.submit(ev).unwrap();
let reply = rx.recv_timeout(Duration::from_secs(5)).unwrap();
loop_.stop();

assert_eq!(reply.data.as_deref(), Some("hello"));

Relation backfill (avoids N+1)

use rust_utils::aggregator::populate;
use std::collections::HashMap;

struct Order { id: u32, user_name: Option<String> }

let mut orders = vec![Order { id: 1, user_name: None }];
let users: HashMap<u32, String> = [(1, "张三".to_string())].into();

populate(&mut orders, &users, |o| o.id, |o, name| o.user_name = Some(name));
assert_eq!(orders[0].user_name.as_deref(), Some("张三"));

Password hashing

use rust_utils::password::{Argon2Crypto, Crypto};

let algo = Argon2Crypto;
let hash = algo.encrypt("s3cret!").unwrap();       // standard PHC format
assert!(algo.verify("s3cret!", &hash).unwrap());

CAPTCHA service

Seven drivers: four text drivers (digit/alphanumeric/arithmetic/Chinese) plus three interactive ones (slide/click/rotate); rendering is fully programmatic, with no external image assets (see assets/captcha/README.md for glyph asset notes). Text drivers return PNG data URIs; interactive drivers return JSON compatible with the base64Captcha front-end component.

use rust_utils::captcha::{Captcha, Config, DriverKind};

let cap = Captcha::with_config(Config::with_driver(DriverKind::Digit));
let (id, b64, answer) = cap.generate().unwrap();
// text drivers return a PNG data URI; slide/click/rotate drivers return
// JSON (with embedded JPEG master / PNG thumbnail images)
assert!(b64.starts_with("data:image/png;base64,"));
assert!(cap.verify(&id, &answer).unwrap()); // one-shot verification

With the captcha-redis feature enabled, swap Captcha::with_store(MemoryStore::new(), cfg) for RedisStore::new("redis://127.0.0.1/")? — the service-level API stays exactly the same.

Cached batch loading

use rust_utils::aggregator::Loader;
use std::sync::atomic::{AtomicUsize, Ordering};

let calls = std::sync::Arc::new(AtomicUsize::new(0));
let calls2 = calls.clone();
let loader = Loader::new(move |keys: &[u32]| {
    calls2.fetch_add(1, Ordering::Relaxed);
    keys.iter().map(|k| (*k, k * 10)).collect()
});

let got = loader.load_many(&[1, 2, 3]);
assert_eq!(got[&2].as_ref(), &20);
loader.load_many(&[2, 3]); // all cache hits, no refetch
assert_eq!(calls.load(Ordering::Relaxed), 1);

Development

cargo test                 # core-module tests (zero-dependency)
cargo test --all-features  # all-module tests (1085 unit + 62 doc tests, 1 marked ignore)
cargo clippy --all-features --all-targets
cargo fmt

Release

CI publishes to crates.io on v* tags (.github/workflows/release.yml):

  1. Configure the CARGO_REGISTRY_TOKEN secret once (crates.io → Settings → Tokens, publish-only);
  2. Bump version in Cargo.toml + update CHANGELOG.md;
  3. git tag v0.2.0 && git push origin v0.2.0 — tag/Cargo.toml consistency is checked, full tests run against live redis/etcd, then the crate is published.

License

MIT

About

Rust utility toolbox (std-only core, feature-gated): case conversion, slice/map/date helpers, snowflake/order/UUID IDs, Django-style filters, MySQL DDL parsing, priority event loop, data aggregation, weighted random, captcha, JWT, AES/SM2/SM3/SM4 crypto, password hashing, TLS setup, geoip, translation, distributed locks, BIN lookup, field masking.

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages