System Architecture
Deep dive into the 4-stage data pipeline: streaming ingestion, deterministic rules engine, local AI synthesis, and post-mortem generation.
Last Signal was architected to solve the transparency gap in autonomous agent debugging. Instead of relying on monolithic cloud agents or opaque vector stores, Last Signal employs a deterministic, transparent pipeline.
Pipeline Overview
Execution-to-Autopsy Forensic Pipeline
Streaming Ingestion
Last Signal launches your agent as a supervised subprocess via subprocess.Popen(shell=False). Background worker threads read stdout and stderr concurrently without buffering, extracting JSONL event markers and catching uncaught Python exceptions.
Stage 1: Streaming Ingestion
When you invoke lastsignal run -- <command>, the supervisor initializes an isolated subprocess using Python's subprocess.Popen. To avoid deadlocks and buffer saturation on high-throughput agent logs, Last Signal creates two dedicated daemon reader threads:
- stdout Reader: Intercepts raw application stdout line by line, scans for embedded JSONL event markers (__LS_EVENT__ or raw JSON), and writes clean traces to .lastsignal/traces/.
- stderr Reader: Captures uncaught Python tracebacks, native OS crash signals, and runtime warnings.
Stage 2: Deterministic Rules Engine
The trace reader parses JSONL into typed EventModel objects. These events are processed sequentially by four state-machine rules:
- RULE-REPEATED-TOOL-FAILURE: Tracks failure frequency per tool name. Flags violations when failures >= 2.
- RULE-UNPRODUCTIVE-RETRY-LOOP: Maintains an active retry window. Flags violations when retries accumulate without a success event.
- RULE-TERMINAL-TOOL-FAILURE: Calculates temporal delta between the last tool failure and process exit timestamp.
- RULE-UNRECOVERED-EXECUTION-FAILURE: Validates process exit code and verifies whether any recorded recovery path mitigated the failure.
Stage 3: Local AI Grounding (Ollama)
Once the deterministic rules engine establishes verified findings, Last Signal constructs a strictly grounded prompt. The prompt supplies only the rule findings, chronological tool sequence, and error messages to a local Ollama instance (http://localhost:11434).
Stage 4: Post-Mortem Generation
The final step compiles all evidence into a structured Markdown document written to reports/postmortem_<trace_id>.md. The report includes rule tables, chronological event timelines, exact line numbers, and copyable remediations.
Subprocess Execution & Security
Last Signal strictly sets shell=False when invoking subprocesses. This prevents shell injection vulnerabilities, ensures direct signal propagation (such as SIGINT and SIGTERM), and guarantees that child PIDs map 1:1 to the monitored agent process.