emulator improvements

README.md

@@ -26,6 +26,21 @@ To generate a focused RX/TX serial-path pseudocode view from the reconstruction
 .\.venv\Scripts\python.exe h8536_serial_pseudocode.py build\rom_decompiled.json --out build\rom_serial_pseudocode.c
 ```
 
+To run the newer sidecar protocol and gate/queue analysis tools:
+
+```powershell
+.\.venv\Scripts\python.exe h8536_serial_gate.py build\rom_decompiled.json --out build\rom_serial_gate.txt
+.\.venv\Scripts\python.exe h8536_report_source_trace.py build\rom_decompiled.json --out build\rom_report_sources.txt
+.\.venv\Scripts\python.exe h8536_table_xrefs.py --out build\rom_table_xrefs.txt
+.\.venv\Scripts\python.exe h8536_protocol_capture.py ROM\rcp-txd-idle-only.txt
+```
+
+To start the current emulator harness:
+
+```powershell
+.\.venv\Scripts\python.exe h8536_emulator.py --max-steps 1000000 --stop-on-heartbeat --interval-steps 512
+```
+
 ## What It Does
 
 - Decodes the H8/500 instruction set used by the H8/536.
@@ -43,6 +58,11 @@ To generate a focused RX/TX serial-path pseudocode view from the reconstruction
 - Reconstructs evidence-supported SCI1 serial frame candidates, including the apparent six-byte TX/RX units and XOR checksum seeded by `0x5A`.
 - Infers candidate serial protocol semantics from validated frames, including `RX[0] & 0x07` command dispatch, likely index/value byte roles, and response staging through `F850-F854`.
 - Generates a focused RX/TX serial-path pseudocode view from those serial reconstruction and protocol-semantic candidates.
+- Decodes observed serial byte captures into six-byte frames, validates checksums, labels capture-observed heartbeat/call/camera-power candidates, and summarizes heartbeat cadence.
+- Accepts both analyzer-style lines such as `RX 006 bytes ...` and the idle reference `frame 006 ...` format in `ROM/rcp-txd-idle-only.txt`.
+- Reconstructs the autonomous serial gate/queue state-machine around `loc_3FD3`, `loc_BAF2`, `F9B0/F9B5`, `FAA2/FAA3/FAA5`, and the resend path through `BE9E/BED5`.
+- Traces direct callers to `loc_3E54` to identify report queue sources and conservatively flags whether observed report indexes such as `0x0007` are ROM-proven constants or runtime/capture observations.
+- Generates table/index cross-reference reports for candidate value/current/secondary/flag tables and LCD text correlations.
 - Adds a Sony RCP-TX7 board profile that ties H8/536 pin 66 `P95/TXD` and pin 67 `P96/RXD` to the MAX202 RS232 transceiver.
 - Flags/manual-annotates TEMP-register access ordering for FRT and A/D 16-bit peripheral registers.
 - Scans unreached ROM ranges for ASCII strings and pointer-table candidates.
@@ -56,6 +76,15 @@ To generate a focused RX/TX serial-path pseudocode view from the reconstruction
 - Handles the E-clock transfer instructions `MOVFPE` and `MOVTPE`.
 - Recognizes likely LCD E-clock access routines at `H'F200`/`H'F201`, including busy-flag polling and data/control writes.
 - Generates a separate C-like pseudocode view from the JSON, preserving labels, calls, branches, register names, inferred symbols, metadata comments, optional cycle notes, and simple structured `if`/`do while` patterns.
+- Provides an early H8/536 emulator harness with ROM/RAM/register memory mapping, reset-vector boot, SCI1 transmit capture, `SCB/F`, stack/call/return support, and scaffolded SCI1 TXI/interval interrupt scheduling.
+
+Current serial observations:
+
+- Idle capture reference: `ROM/rcp-txd-idle-only.txt`.
+- Idle frame: `00 00 00 00 80 DA`.
+- Capture-side label: `heartbeat_alive_candidate`.
+- Idle cadence from the reference file: 54 frames, average about 699.9 ms, min 601 ms, max 803 ms.
+- Observed capture labels such as `cam_power_button_candidate` and `call_button_candidate` are deliberately treated as capture overlays, not protocol facts hard-coded in ROM.
 
 The generated listing is written to:
 
@@ -69,6 +98,17 @@ The optional JSON output is useful for scripts or later analysis:
 build/rom_decompiled.json
 ```
 
+Common derived outputs:
+
+```text
+build/rom_pseudocode.c
+build/rom_serial_pseudocode.c
+build/rom_serial_gate.txt
+build/rom_report_sources.txt
+build/rom_table_xrefs.txt
+build/callgraph.dot
+```
+
 ## Useful Options
 
 ```powershell
@@ -111,6 +151,42 @@ python h8536_serial_pseudocode.py --help
 - `--no-board`: omit board/MAX202 comments.
 - `--no-semantics`: omit candidate command/field semantics.
 
+For protocol trace and capture logs:
+
+```powershell
+python h8536_protocol_trace.py --help
+python h8536_protocol_capture.py --help
+```
+
+- `h8536_protocol_trace.py --direction tx 00 00 15 80 00 CF`: decode raw bytes as protocol frames.
+- `h8536_protocol_capture.py ROM\rcp-txd-idle-only.txt`: parse timestamped captures, recombine split chunks, validate checksums, and summarize cadence/gate hints.
+- `--json` on the capture tool emits machine-readable frame and cadence data.
+
+For gate/queue and table reports:
+
+```powershell
+python h8536_serial_gate.py --help
+python h8536_report_source_trace.py --help
+python h8536_table_xrefs.py --help
+```
+
+- `h8536_serial_gate.py`: reports the autonomous TX gate and report queue evidence.
+- `h8536_report_source_trace.py`: traces direct `loc_3E54` report enqueue sources. Current finding: no direct static `R3 = 0x0007` enqueue in the JSON, so CAM power `0x0007` remains runtime/capture-observed unless a later indirect/table path proves it.
+- `h8536_table_xrefs.py`: emits candidate table/index xrefs and LCD text correlation hints.
+
+For the emulator harness:
+
+```powershell
+python h8536_emulator.py --help
+```
+
+- `--rom PATH`: use an explicit ROM path instead of auto-discovering `ROM\M27C512@DIP28_1.BIN`.
+- `--max-steps N`: bound execution.
+- `--trace`: print executed instructions.
+- `--stop-on-heartbeat`: stop only if `00 00 00 00 80 DA` is emitted through SCI1 TDR.
+- `--interval-steps N`: tune the scaffolded interval timer cadence.
+- Current status: boots from `H'1000`, initializes SCI1, supports the first stack/call/interrupt pieces, but does not yet reach the heartbeat. The next emulator work is a better external P9 bit-banged device/handshake model around the `BFE0/BFFE/C08B/C0DB/C121` routines.
+
 ## Code Layout
 
 - `h8536_decompiler.py`: compatibility wrapper for the CLI.
@@ -134,6 +210,12 @@ python h8536_serial_pseudocode.py --help
 - `h8536/serial_reconstruction.py`: cautious higher-level SCI frame reconstruction from decompiled evidence.
 - `h8536/serial_semantics.py`: candidate command/field semantics inferred from serial frame use.
 - `h8536/serial_pseudocode.py`: focused RX/TX protocol pseudocode generation from reconstruction metadata.
+- `h8536/protocol_trace.py`: raw six-byte protocol frame decoder/checksum validator.
+- `h8536/protocol_capture.py`: timestamped serial capture parser, frame recombiner, and cadence/gate-session analyzer.
+- `h8536/serial_gate.py`: autonomous TX gate/queue state-machine reconstruction.
+- `h8536/report_source_trace.py`: direct `loc_3E54` report enqueue source tracer.
+- `h8536/table_xrefs.py`: table/index xrefs and LCD correlation report generation.
+- `h8536/emulator/`: early H8/536 emulator package split into CPU state, memory map, SCI1 TX capture, runner, CLI, and peripheral scaffolding.
 - `h8536/board_profile.py`: Sony RCP-TX7 board-trace annotations, including the MAX202 RS232 path.
 - `h8536/peripheral_access.py`: FRT/A-D TEMP-register access analysis.
 - `h8536/pseudocode.py`: JSON-to-C-like pseudocode generation.
@@ -141,3 +223,6 @@ python h8536_serial_pseudocode.py --help
 - `h8536/model.py`, `h8536/rom.py`, `h8536/formatting.py`: shared data structures and helpers.
 - `h8536_pseudocode.py`: pseudocode CLI wrapper.
 - `h8536_serial_pseudocode.py`: focused serial pseudocode CLI wrapper.
+- `h8536_protocol_trace.py`, `h8536_protocol_capture.py`: protocol analysis CLI wrappers.
+- `h8536_serial_gate.py`, `h8536_report_source_trace.py`, `h8536_table_xrefs.py`: sidecar analysis CLI wrappers.
+- `h8536_emulator.py`: emulator CLI wrapper.