- Fix SGen collection/dictionary null-handling: always emit PropertySkip for nulls, preventing NREs regardless of nullable annotation. - Add micro-opt CV threshold (1.5%) to benchmark output for finer-grained result flagging; update reporting and context. - Benchmark loop: add inter-sample settle delay, trimmed median, and branchless progress for more reliable measurements. - Add regression tests for SGen null-handling (complex, collection, dictionary; null/non-null; SGen/reflection; FastMode/Default). - Update docs: clarify SGen null-check contract, add AQN binder security plan, and cross-reference related issues. - Misc: code cleanups, improved comments, and minor doc clarifications. |
||
|---|---|---|
| .. | ||
| AyCode.Core.Serializers.Console.csproj | ||
| BenchmarkLoop.cs | ||
| Configuration.cs | ||
| Menu.cs | ||
| Program.cs | ||
| README.md | ||
README.md
AyCode.Core.Serializers.Console
Interactive console runner for the serializer benchmark suite. Targets .NET 9.
Companion: shares its workload + reporting infrastructure with the BDN runner in
AyCode.Benchmark/via<ProjectReference>. See that project's README for the full dual-runner architecture.
Role
This is the fast-iteration half of the benchmark stack — a custom adaptive measure engine optimized for short turnaround (~1-3 min full run) during micro-optimization loops. The BDN half lives in AyCode.Benchmark and produces statistically tighter numbers (~5-15 min full run) for before-commit validation. Both runners emit the same .log / .LLM / .output triplet to Test_Benchmark_Results/Benchmark/ — Console prefixes with Console., BDN with Bdn..
Compared serializers
- AcBinary — multiple options presets:
FastMode(Compact wire, no ref handling, no interning),Default(with ref handling + interning), plus SGen / Runtime dispatch variants and Compact / Fast wire modes. - MemoryPack — SOTA baseline, wire-mode-aligned with AcBinary for apples-to-apples encoding comparison (UTF-8 ↔ Compact, UTF-16 ↔ Fast).
- MessagePack — JIT-only (AOT incompatible due to dynamic resolver).
- System.Text.Json — reference comparison (commented out in
CreateSerializersby default).
Key files
Program.cs— entry point. Parses CLI args (Core/Comprehensive/Edge/ per-cell / op-mode / serializer-set) or falls into interactiveMenu.Menu.cs— interactive layer/serializer-set selection + nested settings (iteration counts, wire mode, charset).BenchmarkLoop.cs— custom adaptive measure engine. CPU 0 affinity pin + High priority for stabilization, JIT pre-warmup, phase-isolated Ser/Des warmup→measure withGC.Collectat every boundary, 10-sample median + pilot discard, adaptive iter calibration to ~250ms/cell wall-clock, dedicated allocation-only sample.Configuration.cs— Console-side state (SelectedWireMode,WarmupIterations,BenchmarkSamples,TargetSampleMs, charset selection,BuildConfigurationconst from#if DEBUG/RELEASE/AYCODE_NATIVEAOT).
Workload + reporting types — ISerializerBenchmark, BenchmarkResult, BenchmarkOptions, BenchmarkEnums, BenchmarkReportWriter, ReportingContext, the 12 concrete *Benchmark<T> classes (AcBinaryBenchmark, MemoryPackBenchmark, AcBinaryBufferWriterBenchmark, ...), RoundTripValidator — live in AyCode.Benchmark/Workloads/Scenarios/ and AyCode.Benchmark/Reporting/.
Test data
5 cells, provided by AyCode.Core.Tests.TestModels.BenchmarkTestDataProvider*:
- Small (2×2×2×2)
- Medium (3×3×3×4)
- Large (5×5×5×10)
- Repeated Strings (10 items, string-deduplication stress)
- Deep Nested (2×4×4×8, depth stress)
20% IId reference rate by default. Two graph variants (TestOrder_All_False / _All_True) are built per cell — AcBinary's option preset picks which variant gets fed to it (UsesAllFalseVariant rule in BenchmarkLoop).
Charset profiles (Menu → Settings → Charset)
Controls the BenchmarkTestDataProvider.LongStringSuffix — the string-tail appended to property values. Influences string-marker selection on the wire (FixStrAscii vs StringSmall / Medium / Big / StringAscii), interning hit rates, and UTF-8 encode cost.
Consistent length across all charsets (UTF-16 char count): every *Short = 40 char, every *Long = 280 char (= Short × 7). Isolates the workload variable to UTF-8 byte content per charset (1-byte ASCII vs 2-byte Latin1 / Cyrillic vs 3-byte CJK vs mixed) — wire-size and encode/decode cost differences are pure charset effects, not length effects.
| Profile | UTF-16 char | UTF-8 byte (approx) | Tier |
|---|---|---|---|
Latin1FixAscii |
0 | 0 | FixStrAscii / FixStr-equivalent (baseline-only) |
AsciiShort |
40 | 40 | StringAscii (167) |
AsciiLong |
280 | 280 | StringAscii (167) |
Latin1Short |
40 | ~72 | StringSmall (91) |
Latin1Long (default) |
280 | ~504 | StringMedium (94) |
CjkBmpShort |
40 | ~104 | StringSmall |
CjkBmpLong |
280 | ~728 | StringMedium |
CyrillicShort |
40 | ~72 | StringSmall |
CyrillicLong |
280 | ~504 | StringMedium |
MixedShort |
40 | ~88 | StringSmall |
MixedLong |
280 | ~616 | StringMedium |
CLI
dotnet run -c Release --project AyCode.Core.Serializers.Console -- [arg]
| Arg | Result |
|---|---|
| (no args) | Interactive menu — pick layer (Core / Comprehensive / Edge / Small / Medium / Large / Repeated / Deep / All) × serializer-set (Standard / FastestByte ["F"] / AsyncPipe ["P"]). |
Core / Comprehensive / Edge / Small / Medium / Large / Repeated / Deep / All |
Run that layer at Standard serializer-set, All op-mode. |
FastestByte / AsyncPipe / Standard |
Run that serializer-set, All layer, All op-mode. |
Serialize / Deserialize / All |
Run that op-mode, All layer, Standard serializer-set. |
quick |
Single-sample fast mode (Debug-equivalent — very loose numbers, smoke-test only). |
Output: Test_Benchmark_Results/Benchmark/Console.FullBenchmark_<Build>_<timestamp>.{log,LLM,output}.
Dependencies
| Dependency | Purpose |
|---|---|
AyCode.Core (ProjectReference) |
AcBinary serializer |
AyCode.Core.Tests (ProjectReference) |
Test data factory + test models |
AyCode.Benchmark (ProjectReference) |
Shared workload + reporting (ISerializerBenchmark, BenchmarkResult, BenchmarkReportWriter, ReportingContext, the 12 concrete benchmark classes) |
MemoryPack |
Comparison target (also via Workloads) |
MessagePack |
Comparison target |
Newtonsoft.Json |
Comparison target (currently disabled) |
Build & publish notes
<StartupObject>AyCode.Core.Serializers.Console.Program</StartupObject>in the csproj explicitly disambiguates the entry point — necessary because this Exe references another Exe (AyCode.Benchmark), and the build would otherwise complain about multipleMainmethods.- AOT publish (
dotnet publish -c Release) is configured via'$(_IsPublishing)' == 'true'PropertyGroup. The Benchmark project's BDN-stack (BenchmarkDotNet, Iced disassembler, MongoDB.Bson) is pulled in transitively — accepted tradeoff for the unified workload sharing.