spine-runtimes/todos/done/2025-01-11-03-02-54-test-suite.md

# create a folder test/ and write a comprehensive test suite

**Status:** In Progress
**Created:** 2025-01-11T03:02:54
**Started:** 2025-01-11T03:11:22
**Agent PID:** 15570

**CRITICAL:**
- NEVER never check a chceckbox and move on to the next checkbox unless the user has confirmed completion of the current checkbox!
- NEVER modify code if there's not a checkbox that instructs you to do so!

## Original Todo
- create a folder test/ and write a comprehensive test suite
    - For each core runtime
        - Write a program that takes as input a skeleton (.json or .skel) and atlas file path and animation name
        - Loads a SkeletonData and outputs EVERYTHING in a simple, diffable text format
        - Creates a skeleton from the SkeletonData, an AnimationState, sets the Animation on track 0, updates and applies the state to the skeleton, then outputs EVERYTHING in a diffable text format
        - The best approach is likely to create a Printer interface that can print each relevant type in the diffable format, with a specific indentation level, so the output represents the hierarchy of the data
        - See docs/project-description.md for the core runtimes and their location
        - The test/ folder should have simple language specific build scripts that build/pull in the core runtime for that language, and create an exectuable program we can invoke
            - build.gradle for Java, directly pulling in the spine-libgdx project via settings.gradle
            - CMakeLists.txt for C and C++
            - package.json/tsconfig.json for Typescript
            - Let's ignore the other core runtimes for now
        - The programs must be headless, which means we need to ensure when loading the atlases, texture loading actually doesn't happen.
        - The goal is to be able to construct bash or nodejs test suites that can find errors in the non-reference runtimes quickly by comparing actually loaded and "applied" data between teh runtimes

## Description
Create a comprehensive test suite that compares the output of all core Spine runtimes (Java, C++, C, TypeScript) to ensure consistency. The test suite will consist of headless command-line programs for each runtime that load skeleton data and output all internal state in a diffable text format. This will enable automated comparison testing between the reference implementation (spine-libgdx) and all ports. Each test program must compile directly against the runtime's source code, not pull published versions from package repositories.

The test programs will print both SkeletonData (setup pose/static data) and Skeleton (runtime pose after animation) states, including all nested types such as bones, slots, skins, attachments, constraints, and animations.

## Updated Implementation Plan (DebugPrinter in each runtime)
- [x] Remove test/ directory approach (deprecated)
- [x] Implement DebugPrinter in spine-libgdx-tests:
  - [x] Create DebugPrinter.java in spine-libgdx/spine-libgdx-tests/src
  - [x] Add command line argument parsing for skeleton, atlas, animation
  - [x] Create Printer class for outputting all types in hierarchical format
  - [x] Ensure it can run headless (no GL requirements)
  - [x] Merged Printer and HeadlessTextureLoader into single DebugPrinter.java file
  - [x] Added gradle task runDebugPrinter for easy execution
- [x] Implement DebugPrinter in spine-cpp:
  - [x] Create spine-cpp/tests directory with DebugPrinter.cpp
  - [x] Update spine-cpp/CMakeLists.txt with conditional target
  - [x] Add Printer class matching Java output format
  - [x] Ensure headless operation
- [x] Implement DebugPrinter in spine-c:
  - [x] Create spine-c/tests directory with debug-printer.c
  - [x] Update spine-c/CMakeLists.txt with conditional target
  - [x] Add printer functions matching output format
  - [x] Ensure headless operation
  - [x] Fix spine-c/codegen/src/ir-generator.ts to use .buffer() for string getters (currently returns address of temporary String object)
    - Run generator via: npx tsx spine-c/codegen/src/index.ts
    - This regenerates spine-c/src/generated/*.cpp files
    - Fixed by handling "const String &" (with space) in addition to "const String&"
    - Verified: String methods now properly return .buffer() values (version, hash, etc. display correctly)
  - [x] Issues found by DebugPrinter comparison:
    - [x] spine-c API missing file-based skeleton loader that sets name from filename
      - C++ has readSkeletonDataFile() which sets name, spine-c only exposes content-based loader
      - Result: SkeletonData name is empty in spine-c output
      - Fixed: Modified spine_skeleton_data_load_json/binary to accept path parameter and extract name
    - [x] Coordinate system inconsistency: Java shows scaleY=1.0, C/C++ show scaleY=-1.0, use Bone::setYDown(false) to match Java
- [x] Implement DebugPrinter in spine-ts/spine-core:
  - [x] Create tests/DebugPrinter.ts
  - [x] Update tsconfig.json to exclude tests/ so tests are not bundled
  - [x] Add Printer class matching output format
  - [x] Ensure it runs with npx tsx without build step
- [x] Create test runner script (compare-with-reference-impl.ts):
  - [x] Run each runtime's DebugPrinter with same inputs
  - [x] Compare outputs and report differences
  - [x] TypeScript script with shebang for direct execution
  - [x] Automatically builds C/C++/Java if needed
  - [x] Saves outputs to tests/output/ directory
  - [x] Shows line-by-line differences when outputs don't match
- [x] Make animation parameter optional in all DebugPrinters:
  - [x] If animation not provided, call skeleton.setToSetupPose() instead
  - [x] Update Java DebugPrinter
  - [x] Update C++ DebugPrinter
  - [x] Update C DebugPrinter
  - [x] Update TypeScript DebugPrinter
  - [x] Update compare-with-reference-impl.ts to handle optional animation
- [x] Fix locale issues - all DebugPrinters should use English locale:
  - [x] Java: Set Locale.US for number formatting
  - [x] C++: Set locale to "C" or "en_US.UTF-8"
  - [x] C: Set locale to "C" or "en_US.UTF-8"
  - [x] TypeScript: Already uses period for decimals
- [x] Improve buildCheck() to detect when rebuild needed:
  - [x] Check if debug printer executable exists
  - [x] Compare executable timestamp with source file timestamps
  - [x] Rebuild if any source files are newer than executable
- [x] Create tests/README.md documentation:
  - [x] Explain purpose: comparing reference implementation to other runtimes
  - [x] List DebugPrinter locations in each runtime
  - [x] Document how to run individual debug printers
  - [x] Document how to run compare-with-reference-impl.ts
- [x] Automated test: All DebugPrinters produce identical output
  - Note: Minor expected differences remain:
    - time field: Java shows 0.016 after update, C/C++ show 0.0
    - TypeScript: minor precision differences, null vs "" for audioPath
    - These are implementation details, not bugs
- [x] User test: Verify with multiple skeleton files

## Notes
### C++ Serializer Implementation Strategy (Option 4 - Hybrid Incremental)
- Started with manual port of basic structure from Java
- Plan: Extract patterns into templates for automation
- Benefits: Quick progress while learning challenges, builds toward automation

## Phase 2: JSON Serializers and HeadlessTest Rename

### Rename DebugPrinter to HeadlessTest
- [x] Rename all DebugPrinter files to HeadlessTest:
  - [x] Java: DebugPrinter.java → HeadlessTest.java
  - [x] C++: DebugPrinter.cpp → HeadlessTest.cpp
  - [x] C: debug-printer.c → headless-test.c
  - [x] TypeScript: DebugPrinter.ts → HeadlessTest.ts
- [x] Update VS Code launch configs to say "headless test ($runtime)":
  - [x] spine-libgdx/.vscode/launch.json
  - [x] spine-cpp/.vscode/launch.json
  - [x] spine-c/.vscode/launch.json
  - [x] spine-ts/.vscode/launch.json
- [x] Rename tests/compare-with-reference-impl.ts to headless-test-runner.ts
- [x] Update build files:
  - [x] CMakeLists.txt for C++ (executable name: headless-test)
  - [x] CMakeLists.txt for C (executable name: headless-test)
  - [x] Gradle for Java (task name: runHeadlessTest, main class: HeadlessTest)
- [x] Update tests/README.md with new names

### Implement JSON Serializers in Core Runtimes
- [x] Java (spine-libgdx):
  - [x] Create SkeletonSerializer class in com.esotericsoftware.spine.utils
  - [x] Implement serializeSkeletonData(SkeletonData, Writer/StringBuilder)
  - [x] Implement serializeSkeleton(Skeleton, Writer/StringBuilder)
  - [x] Implement serializeAnimationState(AnimationState, Writer/StringBuilder)
  - [x] Add depth/verbosity options to control output
  - [x] Handle circular references and limit nesting
  - [x] Update HeadlessTest to use SkeletonSerializer
  - [x] Review serializer with user:
    - [x] Test with actual skeleton file to see output format
    - [x] Add cycle detection to handle circular references (outputs "<circular>")
    - [x] Verify it compiles and produces JSON output
  - [x] Create comprehensive API analyzer tool:
    - [x] Analyzer discovers all types accessible via SkeletonData, Skeleton, and AnimationState
    - [x] For each type, enumerate all getters including inherited ones
    - [x] Generate Java serializer from analysis data
    - [x] Handle enums, abstract types, inner classes, and type parameters
    - [x] Filter out test classes and non-source files
  - [x] Work on SkeletonSerializer.java generation until it actually compiles.
  - [x] Move Java files to correct location and update
    - [x] Remove SkeletonSerializer.java from spine-libgdx project
    - [x] Both files should be in spine-libgdx-tests project instead
  - [x] Create JsonWriter implementations
    - [x] Create JsonWriter.java in spine-libgdx-tests/src/com/esotericsoftware/spine/utils/
      - Use StringBuffer internally instead of Writer parameter
      - Add getString() method to return the built JSON string
      - No throws IOException declarations
  - [x] Update Java serializer generator
    - [x] Modify tests/generate-java-serializer.ts (see tests/README.md for details)
      - Output to spine-libgdx-tests/src/com/esotericsoftware/spine/utils/SkeletonSerializer.java
      - Ensure NO throws IOException declarations on methods
      - Methods return String instead of taking Writer parameter
      - JsonWriter instantiated without parameters (uses internal StringBuffer)
      - Removed JsonWriter inner class generation (using separate JsonWriter.java)
      - Methods use RuntimeException for error handling
  - [x] Update HeadlessTest.java to use SkeletonSerializer.serializeXXX() and output to stdout
  - [x] Optimize Java serializer generator to use @Null annotations to skip unnecessary null checks
      - Exploit that analysis-result.json already has @Null preserved in return types
      - Methods without @Null are guaranteed non-null, skip null checks for efficiency
      - Verified: getName() calls skip null checks, getSequence() calls include null checks
    - [x] Implement exclusion system with single source of truth in tests/java-exclusions.txt
      - Format: `type ClassName`, `method ClassName methodName()`, `field ClassName fieldName`
      - analyze-java-api.ts loads exclusions and marks PropertyInfo.excluded = true/false
      - Java generator filters excluded types from instanceof chains and skips excluded properties
      - Subsequent generators (C++, C, TypeScript) transform already-filtered Java output
      - Current exclusions: SkeletonAttachment (type), TrackEntry.getListener() (method)
    - [x] Filter excluded types from instanceof chains in abstract type handlers
    - [x] Remove dead code (writeBlendMode) - analyze why it's generated in analyze-java-api.ts but is never called
      - Issue: Enums were getting dedicated write methods AND inline .name() serialization
      - Solution: Skip enum types when generating write methods since they're handled inline
      - Result: writeBlendMode and other enum write methods removed, cleaner code
    - [x] Fix writeXXX() method signatures to take only 1 argument, not 2
      - writeXXX() methods should only take the object to serialize: `writeAnimation(Animation obj)`
      - NOT: `writeAnimation(JsonWriter json, Animation obj)`
      - JsonWriter should be accessed via internal instance field
      - This affects ALL writeXXX() methods in the serializer
- [ ] C++ (spine-cpp): DETERMINISTIC direct transformation of Java SkeletonSerializer
  - [x] Create spine-cpp/tests/JsonWriter.h as direct port of Java JsonWriter
      - Header-only implementation (no .cpp file)
      - Use spine::String instead of StringBuffer (has append methods)
      - Direct method-for-method port from Java version
  - [x] Create C++ serializer generator
    - [x] Create tests/generate-cpp-serializer.ts that ports SkeletonSerializer.java to C++
      - Output to spine-cpp/tests/SkeletonSerializer.h (header-only, no .cpp)
      - Include <spine/spine.h> for all types (no individual headers)
      - Direct transformation of Java code to C++ with regex rules
      - All method implementations inline in the header
  - [x] Update spine-cpp/tests/HeadlessTest.cpp to use SkeletonSerializer
  - [X] Compile the generated code, derrive additional regex rules or fix inconsistent C++ API, repeat until user says stop
  - [x] Fix writeXXX() method signatures to take only 1 argument, not 2
      - writeXXX() methods should only take the object to serialize: `writeAnimation(Animation* obj)`
      - NOT: `writeAnimation(JsonWriter& json, Animation* obj)`
      - JsonWriter should be accessed via internal instance field _json
      - This affects ALL writeXXX() methods in the serializer
  - [ ] Fix C++ generator issues and inconsistencies:
      - [x] Remove hardcoded `obj->getData()` → `&obj->getData()` rule
      - [x] Fix writeColor/writeTextureRegion: add a & and * version for each
      - [x] Generate both pointer and reference versions for all class types:
        - Reference version has full implementation
        - Pointer version delegates to reference version: `writeXXX(json, *obj)`
        - likely a post-processing step after generating the reference only version of serializer: find all methods, add pointer version below.
      - [x] Fix all tests/*.ts files to use __dirname instead of process.cwd() for file paths
        - Makes them work from any directory they're invoked from
        - Use path.resolve(__dirname, '..', 'relative-path') pattern
      - [x] Fix obj.field access pattern to use obj->field in C++ (Java uses . for all objects, C++ uses -> for pointers)
        - Transform obj.r/g/b/a to obj->r/g/b/a etc
        - Apply to all field access patterns, not just Color
      - [x] Fix C++ field access for underscore-prefixed fields
        - C++ private fields are prefixed with underscore (e.g. _offset, _to) but Java fields are not
        - Transform obj->field to obj->_field for specific known private fields
        - Example: obj->offset should become obj->_offset in FromRotate class
      - [x] Add hardcoded no-null-check fix for C++-specific methods (these always return &):
        - `BoundingBoxAttachment.getBones()`, `ClippingAttachment.getBones()`
        - `MeshAttachment.getBones()`, `MeshAttachment.getEdges()`
      - [x] Fix nested arrays null checks: getVertices() for DeformTimeline and getDrawOrders() for DrawOrderTimeline
        - These methods already have special casing but include unnecessary nullptr checks
        - Remove the nullptr checks from the special case handling since they return references to nested arrays
      - [x] Replace enum .name() calls with switch statements in C++
        - Java: `json.writeValue(obj.getMixBlend().name())` writes enum as string
        - C++: `String::valueOf((int)obj->getMixBlend())` doesn't work (String::valueOf doesn't exist)
        - Solution: Generate switch statements that map C++ enum values to Java string equivalents
        - Example: MixBlend_Setup -> "setup", MixBlend_First -> "first", etc.
        - Use analysis-result.json to find all enums and generate proper mappings
      - [x] Add ability to completely replace writeXXX functions with custom implementations in C++ generator
        - Need custom writeSkin function because C++ getAttachments() returns AttachmentMap::Entries (value type iterator)
        - Need custom writeColor function because Color.r/g/b/a are public fields without _ prefix
        - Need custom writeSkinEntry function that takes AttachmentMap::Entry* instead of Java SkinEntry
        - Create mechanism in tests/generate-cpp-serializer.ts to replace auto-generated functions with hand-written ones
        - Implement custom writeSkin function that properly handles AttachmentMap::Entries iteration
    - [x] Create regenerate-all.sh script in tests/ that runs all generators in sequence
    - [x] ~~Fix C++ JsonWriter output formatting issues (array formatting broken)~~
    - [x] ~~Test with sample skeleton files~~
- [x] ~~TypeScript (spine-ts):~~
  - [x] ~~Follow what we did for spine-cpp wrt to JsonWriter, SkeletonSerializer and the generator~~
- [x] ~~C#~~
  - [x] ~~Figure out how we can add the HeadlessTest and run it without adding it to the assembly itself~~
  - [x] ~~Follow what we did for spine-cpp wrt~~
- [x] ~~C (spine-c):~~
  - [x] ~~Follow what we did for spine-cpp wrt to JsonWriter, SkeletonSerializer and the generator (this one will need some ultrathink and discussion with the user before code changs)~~
- [x] ~~Update tests/README.md to describe the new setup~~


## Phase 3: Intermediate Representation for Cross-Language Serializer Generation

### Problem
Current approach requires maintaining separate generator logic for each language (Java, C++, C, TypeScript, C#). The complex analysis, exclusion handling, and serialization logic is duplicated across generators, making maintenance difficult.

### Solution: Language-Agnostic Intermediate Representation (IR)
Create a single IR generator that captures all serialization logic in a JSON format that language-specific generators can consume without a lot of post-processing. The IR will still
contain Java specific types and names. Language specific generators then just have to translate.

### IR Structure
```typescript
interface SerializerIR {
  publicMethods: PublicMethod[];
  writeMethods: WriteMethod[];
  enumMappings: { [enumName: string]: { [javaValue: string]: string } };
}

type Property = Primitive | Object | Enum | Array | NestedArray;

interface Primitive {
  kind: "primitive";
  name: string; // "duration"
  getter: string; // "getDuration"
  valueType: string; // "float", "int", "boolean", "String"
  isNullable: boolean;
}

interface Object {
  kind: "object";
  name: string; // "color"
  getter: string; // "getColor"
  valueType: string; // "Color"
  writeMethodCall: string; // "writeColor"
  isNullable: boolean;
}

interface Enum {
  kind: "enum";
  name: string; // "mixBlend"
  getter: string; // "getMixBlend"
  enumName: string; // "MixBlend"
  isNullable: boolean;
}

interface Array {
  kind: "array";
  name: string; // "timelines"
  getter: string; // "getTimelines"
  elementType: string; // "Timeline", "int", "String"
  elementKind: "primitive" | "object";
  writeMethodCall?: string; // "writeTimeline" (for objects)
  isNullable: boolean;
}

interface NestedArray {
  kind: "nestedArray";
  name: string; // "vertices"
  getter: string; // "getVertices"
  elementType: string; // "float"
  isNullable: boolean;
}
```

### Implementation Plan
- [x] Create tests/generate-serializer-ir.ts:
  - [x] Reuse logic from tests/generate-java-serializer.ts (analysis, exclusions, property detection)
  - [x] Output SerializerIR as JSON to tests/output/serializer-ir.json
  - [x] All exclusions and filtering pre-applied - no post-processing needed
- [ ] Update language generators to consume IR:
  - [x] Replace tests/generate-java-serializer.ts with IR-based version
    - [x] Sort skin entries by slot index before emission in Java
  - [x] Modify tests/generate-cpp-serializer.ts to use IR
  - [ ] Create tests/generate-ts-serializer.ts using IR
  - [ ] Create tests/generate-cs-serializer.ts using IR
  - [ ] Language generators focus purely on syntax transformation
- [ ] Update tests/regenerate-all.sh to generate IR first, then all languages

### Misc (added by user while Claude worked, need to be refined!)
- [ ] HeadlessTest should probably
  - Have a mode that does what we currently do: take files and animation name, and output serialized skeleton data and skeleton. Used for ad-hoc testing of files submitted by users in error reports etc.
  - Have "unit" test like tests, that are easily extensible
    - each test has a name and points to the corresponding function to execute
    - HeadlessTest can take as args a single name, multiple test names, or no args in which case it runs all tests in order
    - Structure and cli handling needs to be the same in all HeadlessTest implementations
    - tests/headless-test-runner.ts should also support these same cli args, run each runtime test, then compare outputs.