h8-536-decoder/README.md

# H8/536 ROM Decompiler

The ROM used is from a SONY RCP-TX7 Camera Panel. Some of the code in this repo may be bias to the functions of that particular use case with the H8/536.

This repo now includes a standalone Python helper for the H8/536 ROM image:

```powershell
python h8536_decompiler.py ROM\M27C512@DIP28_1.BIN --out build\rom_decompiled.asm --json build\rom_decompiled.json
```

If you are using the repo-local venv:

```powershell
.\.venv\Scripts\python.exe h8536_decompiler.py --out build\rom_decompiled.asm --json build\rom_decompiled.json --cycles --callgraph-dot build\callgraph.dot
```

To turn the structured decompile output into conservative C-like pseudocode:

```powershell
.\.venv\Scripts\python.exe h8536_pseudocode.py build\rom_decompiled.json --out build\rom_pseudocode.c --cycles
```

To generate a focused RX/TX serial-path pseudocode view from the reconstruction metadata:

```powershell
.\.venv\Scripts\python.exe h8536_serial_pseudocode.py build\rom_decompiled.json --out build\rom_serial_pseudocode.c
```

To run the newer sidecar protocol and gate/queue analysis tools:

```powershell
.\.venv\Scripts\python.exe h8536_serial_gate.py build\rom_decompiled.json --out build\rom_serial_gate.txt
.\.venv\Scripts\python.exe h8536_report_source_trace.py build\rom_decompiled.json --out build\rom_report_sources.txt
.\.venv\Scripts\python.exe h8536_table_xrefs.py --out build\rom_table_xrefs.txt
.\.venv\Scripts\python.exe h8536_consistency.py build\rom_decompiled.json --out build\rom_consistency.txt
.\.venv\Scripts\python.exe h8536_protocol_capture.py ROM\rcp-txd-idle-only.txt
```

To start the current emulator harness:

```powershell
.\.venv\Scripts\python.exe h8536_emulator.py --max-steps 1000000 --stop-on-heartbeat --interval-steps 512
.\.venv\Scripts\python.exe h8536_emulator_probe.py --max-steps 4000000 --stop-on-tx
.\.venv\Scripts\python.exe h8536_emulator_probe.py --max-steps 1000000 --stop-on-tx --p9-fast-path
.\.venv\Scripts\python.exe h8536_emulator_rx_probe.py --preset connect-lcd
.\.venv\Scripts\python.exe scripts\bench_connect_lcd_sequence.py --port COM5 --relay-port COM6 --prompt-screen
.\.venv\Scripts\python.exe h8536_emulator_bench_replay.py captures\bench-connect-lcd-sequence-20260525-214411.txt --assert-bench-parity
```

The real-device bench helper uses `pyserial`; install repo dependencies with `.\.venv\Scripts\python.exe -m pip install -r requirements.txt` if needed.

## What It Does

- Decodes the H8/500 instruction set used by the H8/536.
- Reads the H8/536 minimum-mode vector table from the ROM.
- Recursively traces reachable code from reset, interrupt, and trap vectors.
- Emits labels for branch and call targets.
- Tracks `LDC.B #xx, BR` along traced control flow so later short absolute `@aa:8` operands can resolve automatically.
- Annotates H8/536 register accesses such as `P1DDR`, `SYSCR1`, `WCR`, watchdog, timer/SCI/A-D, and RAM-control registers.
- Decodes register bitfields and selected hardware semantics for setup writes.
- Annotates interrupt priority registers and DTC enable routing registers.
- Emits memory-region metadata for vector, DTC, RAM, register-field, and mode-dependent program/external space.
- Parses the DTC vector table described by the manual and decodes DTC register-information blocks.
- Tracks SCI setup writes and can infer baud rates from SMR/BRR when `--clock-hz` is supplied.
- Annotates SCI protocol actions such as TDRE waits, TDR writes, RDR reads, RX/TX interrupt enables, and receive-error clears.
- Reconstructs evidence-supported SCI1 serial frame candidates, including the apparent six-byte TX/RX units and XOR checksum seeded by `0x5A`.
- Infers candidate serial protocol semantics from validated frames, including `RX[0] & 0x07` command dispatch, likely index/value byte roles, and response staging through `F850-F854`.
- Generates a focused RX/TX serial-path pseudocode view from those serial reconstruction and protocol-semantic candidates.
- Marks H8 word-destination writes fed by byte immediates as explicit zero-extension in pseudocode, including the heartbeat queue write at `loc_4067`.
- Emits a decompiler/pseudocode consistency report for width semantics that are easy to misread.
- Decodes observed serial byte captures into six-byte frames, validates checksums, labels capture-observed heartbeat/call/camera-power candidates, and summarizes heartbeat cadence.
- Accepts both analyzer-style lines such as `RX 006 bytes ...` and the idle reference `frame 006 ...` format in `ROM/rcp-txd-idle-only.txt`.
- Reconstructs the autonomous serial gate/queue state-machine around `loc_3FD3`, `loc_BAF2`, `F9B0/F9B5`, `FAA2/FAA3/FAA5`, the `F9C4`/FRT2 idle heartbeat gate at `loc_4046`, and the resend path through `BE9E/BED5`.
- Traces direct callers to `loc_3E54` to identify report queue sources and conservatively flags whether observed report indexes such as `0x0007` are ROM-proven constants or runtime/capture observations.
- Generates table/index cross-reference reports for candidate value/current/secondary/flag tables and LCD text correlations.
- Adds a Sony RCP-TX7 board profile that ties H8/536 pin 66 `P95/TXD` and pin 67 `P96/RXD` to the MAX202 RS232 transceiver.
- Flags/manual-annotates TEMP-register access ordering for FRT and A/D 16-bit peripheral registers.
- Scans unreached ROM ranges for ASCII strings and pointer-table candidates.
- Scans likely LCD/menu text records, groups display-text regions, and reports literal/near matches for terms such as `CONNECT`.
- Emits function summaries and a direct-call graph in JSON, with optional Graphviz DOT output.
- Tracks conservative per-basic-block register/control-register dataflow in JSON and comments known value changes.
- Discovers RAM/external/global symbols from memory references and pointer tables, including read/write counts and xrefs.
- Adds indirect `JSR/JMP @Rn` flow hints when a nearby indexed word load looks like a pointer table dispatch.
- Adds Appendix A cycle estimates to JSON and can append them to ASM comments.
- Summarizes straight-line block timing and backward-branch loop timing when requested.
- Handles the E-clock transfer instructions `MOVFPE` and `MOVTPE`.
- Recognizes likely LCD E-clock access routines at `H'F200`/`H'F201`, including busy-flag polling and data/control writes.
- Generates a separate C-like pseudocode view from the JSON, preserving labels, calls, branches, register names, inferred symbols, metadata comments, optional cycle notes, and simple structured `if`/`do while` patterns.
- Provides an early H8/536 emulator harness with ROM/RAM/register memory mapping, reset-vector boot, SCI1 transmit capture, MOV condition-code updates, `SCB/F`, stack/call/return support, indirect `JMP/JSR @Rn` dispatch, scaffolded SCI1 RXI/ERI/TXI and interval timer scheduling, manual-derived FRT1/FRT2 OCIA cycle scheduling, a P9 bit-banged bus model, a 16x4 LCD bus/DDRAM model for `H'F200`/`H'F201`, and an opt-in P9 transfer fast path.
- Includes an emulator probe that reports hot PCs, recent P9/SCI accesses, serial report queue/gate traces, RAM lifecycle watches, final SCI1/TXI state, and captured P9 byte candidates while running the real ROM.
- Includes an RX command probe that boots until SCI1 RXI is serviceable, injects host six-byte frames through RDR/RDRF, listens for device TX frames, and reports serial latch/table/LCD-buffer and emulated-LCD effects.
- Includes a bench helper for replaying the emulator-derived CONNECT LCD frame sequence against the real device through COM5, with optional COM6 relay power cycling and timestamped capture logs.
- Includes a bench-log replay harness that feeds recorded host TX frames back into the ROM emulator and asserts parity against the real device's observed response/LCD state.

Current serial observations:

- Idle capture reference: `ROM/rcp-txd-idle-only.txt`.
- Idle frame: `00 00 00 00 80 DA`.
- Capture-side label: `heartbeat_alive_candidate`.
- Idle cadence from the reference file: 54 frames, average about 699.9 ms, min 601 ms, max 803 ms.
- Static/runtime finding: `F9C4` is a candidate idle heartbeat/report countdown. Init loads `H'14`, `loc_BA26` reloads `H'07` after a send, FRT2 OCIA decrements it, and `loc_4046` can enqueue report `H'0000` when it reaches zero and the queue is empty.
- Emulator timing finding: the ROM initializes FRT2 with `TCR=H'02` and `OCRA=H'7A12`; using the manual's `phi/32` prescaler gives a 1,000,000-cycle OCIA period, so the default `--clock-hz 10000000` models that tick as 100 ms and the post-send `F9C4=H'07` heartbeat delay as about 700 ms.
- Runtime-confirmed heartbeat path: `loc_4067` writes `H'0000` into the queue via a zero-extended word move, `loc_BAF2/loc_BB08` dequeue it, `loc_BB1C/loc_BB20/loc_BB2B` stage the TX bytes, and `loc_BA26` emits `00 00 00 00 80 DA`.
- Emulator LCD finding: the ROM writes the boot/no-active-session message to the LCD bus as ` CONNECT:NOT ACT` on line 0 by the time SCI1 RX is serviceable. Valid and invalid six-byte host frames leave that display active while normal serial replies/heartbeats continue.
- RX probe finding: with calibrated FRT timing, the `--preset connect-lcd` sequence reaches the command-`0x04` handler but leaves the emulated LCD at ` CONNECT:NOT ACT` and falls back to heartbeat output; the earlier `CONNECT:  OK`/`02 00 02 00 00 5A` result is now treated as a legacy step-timer artifact.
- Bench follow-up: replaying the emulator CONNECT sequence on the real device did not switch the LCD to OK. The real device answered the `04 00 00 80 00 DE` step with `07 80 C0 60 20 5D` in the captured run and remained at `CONNECT NOT ACT`, so the next mismatch to chase is the missing visible `07 80 C0 60 20 5D` response/session context rather than the LCD OK branch.
- Observed capture labels such as `cam_power_button_candidate` and `call_button_candidate` are deliberately treated as capture overlays, not protocol facts hard-coded in ROM.

The generated listing is written to:

```text
build/rom_decompiled.asm
```

The optional JSON output is useful for scripts or later analysis:

```text
build/rom_decompiled.json
```

Common derived outputs:

```text
build/rom_pseudocode.c
build/rom_serial_pseudocode.c
build/rom_serial_gate.txt
build/rom_report_sources.txt
build/rom_table_xrefs.txt
build/rom_consistency.txt
build/callgraph.dot
```

## Useful Options

```powershell
python h8536_decompiler.py --help
```

- `--mode min|max`: vector format. This ROM appears to be minimum mode; `min` is the default.
- `--entry H'1234`: add an extra entry point to recursive tracing.
- `--linear`: linear-sweep the selected range instead of tracing from vectors.
- `--start H'1000 --end H'D100`: constrain the decode range.
- `--br H'FE`: resolve short absolute `@aa:8` operands through a known base-register value.
- `--clock-hz 16000000`: infer SCI baud rates from manual BRR formulas.
- `--board-profile sony_rcp_tx7|none`: include or suppress known board-trace annotations.
- `--cycles`: append Appendix A cycle estimates to assembly comments.
- `--timing`: include straight-line block and backward-branch loop timing summaries.
- `--callgraph-dot build\callgraph.dot`: write a Graphviz DOT call graph.

For pseudocode:

```powershell
python h8536_pseudocode.py --help
```

- `--no-asm`: omit original assembly text from pseudocode line comments.
- `--no-addresses`: omit instruction addresses from pseudocode line comments.
- `--cycles`: include cycle estimates from the JSON.
- `--no-structure`: preserve label/goto output instead of simple structured `if`/loop output.
- `--max-functions N`: emit only the first `N` functions for focused review.

For focused serial pseudocode:

```powershell
python h8536_serial_pseudocode.py --help
```

- `--tx-only`: emit only the candidate transmit path.
- `--rx-only`: emit only the candidate receive path.
- `--no-evidence`: omit evidence-address comments.
- `--no-manual`: omit manual-reference comments.
- `--no-board`: omit board/MAX202 comments.
- `--no-semantics`: omit candidate command/field semantics.

For protocol trace and capture logs:

```powershell
python h8536_protocol_trace.py --help
python h8536_protocol_capture.py --help
```

- `h8536_protocol_trace.py --direction tx 00 00 15 80 00 CF`: decode raw bytes as protocol frames.
- `h8536_protocol_capture.py ROM\rcp-txd-idle-only.txt`: parse timestamped captures, recombine split chunks, validate checksums, and summarize cadence/gate hints.
- `--json` on the capture tool emits machine-readable frame and cadence data.

For gate/queue and table reports:

```powershell
python h8536_serial_gate.py --help
python h8536_report_source_trace.py --help
python h8536_table_xrefs.py --help
python h8536_consistency.py --help
```

- `h8536_serial_gate.py`: reports the autonomous TX gate and report queue evidence.
- `h8536_report_source_trace.py`: traces direct `loc_3E54` report enqueue sources. Current finding: no direct static `R3 = 0x0007` enqueue in the JSON, so CAM power `0x0007` remains runtime/capture-observed unless a later indirect/table path proves it.
- `h8536_table_xrefs.py`: emits candidate table/index xrefs and LCD text correlation hints.
- `h8536_consistency.py`: flags JSON-to-pseudocode semantic hazards such as byte immediates written to word destinations.

For the emulator harness:

```powershell
python h8536_emulator.py --help
python h8536_emulator_probe.py --help
python h8536_emulator_rx_probe.py --help
```

- `--rom PATH`: use an explicit ROM path instead of auto-discovering `ROM\M27C512@DIP28_1.BIN`.
- `--max-steps N`: bound execution.
- `--trace`: print executed instructions.
- `--stop-on-heartbeat`: stop only if `00 00 00 00 80 DA` is emitted through SCI1 TDR.
- `--interval-steps N`: tune the scaffolded interval timer cadence.
- `--clock-hz N`: set the CPU/phi clock used for calibrated FRT1/FRT2 compare timing; the default is 10 MHz.
- `--frt1-ocia-steps N` / `--frt2-ocia-steps N`: optional legacy overrides for forcing rough FRT compare cadence in targeted tests.
- `--p9-fast-path`: shortcut known P9 transfer routines for exploration.
- `--trace-report-gates`, `--trace-report-queue`, and `--trace-ram-lifecycle`: inspect the serial report queue, `loc_4046`/`F9C4` gate, and watched RAM byte history.
- `--target-frame "00 00 00 00 80 DA"`: compare staged/emitted TX bytes against an expected six-byte frame.
- `h8536_emulator_rx_probe.py "04 00 00 80 00"`: append the checksum, inject the host frame through SCI1 RX, and summarize responses.
- `h8536_emulator_rx_probe.py --preset connect-lcd`: replay the current CONNECT LCD activation candidates.
- `scripts\bench_connect_lcd_sequence.py --port COM5 --relay-port COM6 --prompt-screen`: power-cycle the bench device, wait for heartbeat readiness, send `04 00 00 40 00 1E`, `04 00 00 80 00 DE`, `04 00 00 C0 00 9E`, log RX/TX, and prompt for observed LCD text.
- `h8536_emulator_bench_replay.py captures\bench-connect-lcd-sequence-20260525-214411.txt --assert-bench-parity`: replay a real bench log into the emulator and intentionally fail while any response/LCD state still diverges from the bench-observed `CONNECT NOT ACT` plus `07 80 C0 60 20 5D` path.
- Current status: boots from `H'1000`, initializes SCI1, models the first P9 bit-banged handshakes, captures P9 byte candidates, can optionally fast-path known P9 routines, schedules FRT1/FRT2 OCIA from timer registers and `--clock-hz`, captures the ROM-driven LCD line ` CONNECT:NOT ACT`, and emits the observed heartbeat frame `00 00 00 00 80 DA`.

## Code Layout

- `h8536_decompiler.py`: compatibility wrapper for the CLI.
- `h8536/cli.py`: argument parsing and end-to-end orchestration.
- `h8536/decoder.py`: instruction and effective-address decoding.
- `h8536/tables.py`: manual-derived opcode/vector/register tables.
- `h8536/vectors.py`: exception and DTC vector parsing.
- `h8536/dtc.py`: DTC register-information block decoding.
- `h8536/analysis.py`: recursive tracing, linear sweep, labels, function grouping, and call graph analysis.
- `h8536/data_analysis.py`: unreached string and pointer-table candidate scans.
- `h8536/memory.py`: manual-derived memory-region tagging.
- `h8536/cycles.py`: Appendix A cycle estimate tables.
- `h8536/dataflow.py`: conservative register/control-register value tracking.
- `h8536/symbols.py`: RAM/external/global symbol discovery from references and data tables.
- `h8536/indirect.py`: indirect call/jump and pointer-table dispatch hints.
- `h8536/lcd_text.py`: LCD/menu text record scanning, fuzzy search, and text xrefs.
- `h8536/lcd_driver.py`: LCD E-clock access and busy-poll recognizer.
- `h8536/timing.py`: block and loop cycle summaries.
- `h8536/sci.py`: SCI setup tracking and baud inference.
- `h8536/sci_protocol.py`: SCI transmit/receive/status semantic annotations.
- `h8536/serial_reconstruction.py`: cautious higher-level SCI frame reconstruction from decompiled evidence.
- `h8536/serial_semantics.py`: candidate command/field semantics inferred from serial frame use.
- `h8536/serial_pseudocode.py`: focused RX/TX protocol pseudocode generation from reconstruction metadata.
- `h8536/protocol_trace.py`: raw six-byte protocol frame decoder/checksum validator.
- `h8536/protocol_capture.py`: timestamped serial capture parser, frame recombiner, and cadence/gate-session analyzer.
- `h8536/serial_gate.py`: autonomous TX gate/queue state-machine reconstruction.
- `h8536/report_source_trace.py`: direct `loc_3E54` report enqueue source tracer.
- `h8536/table_xrefs.py`: table/index xrefs and LCD correlation report generation.
- `h8536/consistency.py`: decompiler/pseudocode semantic consistency checks.
- `h8536/emulator/`: early H8/536 emulator package split into CPU state, memory map, SCI1 TX capture, P9 bus model, LCD model, manual-derived FRT timer scheduling, runner, probe, CLI, and peripheral scaffolding.
- `h8536/emulator/rx_probe.py`: host-frame injection and response/listener probe for SCI1 RX experiments.
- `h8536/board_profile.py`: Sony RCP-TX7 board-trace annotations, including the MAX202 RS232 path.
- `h8536/peripheral_access.py`: FRT/A-D TEMP-register access analysis.
- `h8536/pseudocode.py`: JSON-to-C-like pseudocode generation.
- `h8536/render.py`: assembly and JSON output.
- `h8536/model.py`, `h8536/rom.py`, `h8536/formatting.py`: shared data structures and helpers.
- `h8536_pseudocode.py`: pseudocode CLI wrapper.
- `h8536_serial_pseudocode.py`: focused serial pseudocode CLI wrapper.
- `h8536_protocol_trace.py`, `h8536_protocol_capture.py`: protocol analysis CLI wrappers.
- `h8536_serial_gate.py`, `h8536_report_source_trace.py`, `h8536_table_xrefs.py`, `h8536_consistency.py`: sidecar analysis CLI wrappers.
- `h8536_emulator.py`, `h8536_emulator_probe.py`, `h8536_emulator_rx_probe.py`, `h8536_emulator_bench_replay.py`: emulator CLI wrappers.
- `scripts/bench_connect_lcd_sequence.py`: real-device COM5/COM6 bench runner for the CONNECT LCD sequence.