# systemg Full LLM Guide

systemg is an agent-friendly general process composer. It manages a graph of local
programs from a YAML manifest, starts dependencies in order, supervises long
running services, runs cron-style units, captures logs, reports status, and
supports targeted service control through a single CLI named `sysg`.

Current release: `0.56.1`

Primary links:

- Documentation: https://sysg.dev
- Quickstart: https://sysg.dev/quickstart
- Installation: https://sysg.dev/installation
- Upgrading sysg: https://sysg.dev/installation/upgrading
- Commands: https://sysg.dev/how-it-works/commands
- Logs command: https://sysg.dev/how-it-works/commands/logs
- Status command: https://sysg.dev/how-it-works/commands/status
- Inspect command: https://sysg.dev/how-it-works/commands/inspect
- Validate command: https://sysg.dev/how-it-works/commands/validate
- Claude Code integration: https://sysg.dev/integrations/claude-code
- Configuration: https://sysg.dev/how-it-works/configuration
- Runtime state: https://sysg.dev/how-it-works/state
- Logging model: https://sysg.dev/how-it-works/logs
- Debugging method: https://github.com/ra0x3/systemg/blob/main/DEBUGGING.md
- Repository: https://github.com/ra0x3/systemg

## What Agents Need To Know In 0.56.1

- Global `--plain` mode for non-interactive callers.
- `sysg logs` no longer blocks by default when called from pipes, files, SSH, or
  agent sessions.
- Manifest schema `2`, multi-project state isolation, and `-p` project targeting.
- Typed `SGXXXX` diagnostics for start, selectors, restart, purge, and upgrades.
- Foreground starts stream `service | line`; Ctrl-C stops only that project.
- Clean exits remain done regardless of restart policy.
- Health checks separate per-probe `attempt_timeout` from `total_timeout`.
- `supervisor.xml` exposes log caps and lifecycle timeout defaults.
- Persisted and command-output XML is nested and two-space indented.
- Compatible upgrades use same-PID live re-execution without restarting workloads.

## Agent Mode

Prefer `--plain` whenever an LLM, script, CI job, remote shell, or parser is
driving `sysg`.

```sh
sysg --plain status
sysg --plain logs -s api
sysg --plain inspect -s api --format json
```

`--plain` is global, so it can be placed before or after the subcommand. It is
equivalent to setting `SYSTEMG_AGENT=1` for the current process. Agent mode is
also enabled if either `SYSTEMG_AGENT` or `NO_COLOR` is set in the environment.

Agent mode is intended to:

- Disable color and decorative terminal output.
- Avoid banners and paging where applicable.
- Preserve full unit names instead of truncating for a terminal table.
- Make `sysg logs` one-shot by default instead of following forever.
- Strip ANSI escape sequences from log output by default.

Recommended environment for tools that run many `sysg` commands:

```sh
export SYSTEMG_AGENT=1
```

## Install

Install latest:

```sh
curl --proto '=https' --tlsv1.2 -fsSL https://sh.sysg.dev/ | sh
```

Install or activate a specific version:

```sh
curl --proto '=https' --tlsv1.2 -fsSL https://sh.sysg.dev/ | sh -s -- --version 0.56.1
```

System deployments use `scripts/install-systemg.sh` to set up `/usr/bin/sysg`,
`/etc/systemg`, and `/var/lib/systemg`. In system mode (`--sys`), state lives in
`/var/lib/systemg` and logs live in `/var/log/systemg`.

Starting with 0.56.0, the normal installer can live-upgrade a compatible newer
release. The resident supervisor keeps its PID and workloads while replacing
its binary. From 0.57.1 forward, matching protocol and handoff schema—not
major/minor boundaries—determine compatibility. Residents through 0.57.0,
downgrades, and contract changes may require a stopped supervisor. SG0501-SG0505
explain validation, compatibility, handoff safety, and rollback failures.

## Core Mental Model

systemg runs one resident supervisor. Schema-v2 manifests may load one or many
projects from a `systemg.yaml` configuration file. Units can be services or cron
jobs. Services are long-running processes; cron units are scheduled jobs, not
persistent services.

A foreground `start` is a terminal attachment to one project, not a separate
supervisor. Ctrl-C stops that project while the supervisor and sibling projects
remain. `--daemonize` starts without retaining the attachment.

Common operations:

```sh
sysg start
sysg start --config systemg.yaml
sysg start --daemonize
sysg stop
sysg restart
sysg status
sysg inspect -s api
sysg logs -s api
```

Most service-specific commands accept:

```sh
--service <name>
-s <name>
--project <project-id>
-p <project-id>
--config <path>
-c <path>
```

For `logs`, bare `--service <name>` targets only the loose bundle; include
`--project` for every project-owned unit. Other service selectors require a
project when multiple loaded projects contain the same name.

## Status

Use `status` to inspect supervisor health.

Best commands for LLMs:

```sh
sysg --plain status
sysg status --format json
sysg status --project <project-id> --format json
sysg status --service api --format json
```

`sysg status` can run without a config file when a supervisor is already
running. If no supervisor is running and no config is provided, it reports that
there is no running supervisor.

Useful status options:

- `--format json`: structured machine-readable output. If `--format` is passed
  without a value, JSON is the default.
- `--format xml`: XML status output.
- `--plain`: disable color and print full, un-truncated unit names.
- `--project <id>`: filter by stable project id.
- `--service <name>`: filter to one unit.
- `--all`: include orphaned state outside the selected project/config set.
- `--live`: force immediate runtime collection instead of using the configured
  snapshot mode.
- `--stream <duration>`: continuously refresh status. Avoid this for one-shot
  LLM calls unless streaming is explicitly requested.

Exit-code convention for rendered status and inspect views:

- `0`: healthy
- `1`: warning
- `2`: failing

## Inspect

Use `inspect` for one service or cron unit.

Best commands for LLMs:

```sh
sysg --plain inspect -s api
sysg inspect -s api --format json
sysg inspect -s api --project <project-id> --format json
```

Useful inspect options:

- `--format json`: structured detail for parsers.
- `--format xml`: XML detail.
- `--plain`: inherited global flag for agent-friendly output.
- `--project <id>`: disambiguate duplicate service names.
- `--live`: force immediate runtime collection.
- `--stream <duration>`: continuously refresh. Avoid for one-shot agent calls.

## Validate

Use `validate` to check a manifest before running it. It parses the file,
resolves the dependency graph, and reports precise, fixable diagnostics.

Best commands for LLMs:

```sh
sysg validate -c sysg.yaml
sysg validate -c sysg.yaml --format json
sysg --plain validate -c sysg.yaml
```

Behavior:

- Exit code `0` when the manifest is valid, `1` when it has problems.
- Human output shows the offending line with a caret, plus a plain-language
  `why`, a suggested `fix`, and a docs link. It is colored for terminals and
  falls back to plain text under `--plain` / `--no-color` / agent mode.
- `--format json` emits `{config, valid, diagnostics[]}` where each diagnostic
  has `line`, `column`, `kind`, `message`, `why`, `suggestion`, and `doc`.

Checks performed: schema version `2`, project or loose-bundle services with a
`command`, valid YAML syntax, `deployment.health_check` with a `url` or
`command`, `depends_on` references that exist and contain no cycle, valid named
project ids, and `${VAR}` interpolations that resolve. Run `sysg validate` after writing or
editing any config, and gate deploys on it:

```sh
sysg validate -c production.yaml --plain || exit 1
sysg start -c production.yaml --daemonize
```

## Logs

Use `logs` to read service output captured by systemg.

Best commands for LLMs:

```sh
sysg --plain logs -s api
sysg logs -s api --format json
sysg logs -s api --raw
sysg logs -s api --grep ERROR --since 2h
sysg logs -s api --all --grep "panic|ERROR|failed"
sysg logs --supervisor --format json
sysg logs --path
sysg logs -s api --path --all
```

Important log-following behavior:

- `--follow` or `-f` always follows until interrupted.
- `--no-follow` always prints one snapshot and exits.
- With neither flag, systemg follows only when stdout is an interactive terminal
  and agent mode is unset.
- In pipes, files, SSH commands, and `SYSTEMG_AGENT=1` sessions, logs are
  one-shot by default.

This means these commands are safe for LLMs and automation:

```sh
sysg logs -s api
sysg --plain logs -s api
sysg logs -s api | grep ERROR
ssh host 'SYSTEMG_AGENT=1 sysg logs -s api'
```

Use `--lines 0 --follow` when a long-running, new-lines-only tail is
intentional:

```sh
sysg logs -s api --lines 0 --follow
```

### Structured Logs

Use JSON-lines output for reliable parsing:

```sh
sysg logs -s api --format json
```

Each emitted line is a JSON object with:

```json
{"ts":"2026-07-07T12:00:00Z","stream":"stdout","service":"api","line":"request completed"}
```

Properties:

- `ts`: systemg capture timestamp.
- `stream`: `stdout`, `stderr`, or supervisor stream label.
- `service`: service name when available.
- `line`: original application line after requested stripping/prefix handling.

`--format json` drops banners and section headers, and strips ANSI escapes by
default.

### Raw Logs

Use raw output when the application line itself is the only useful payload:

```sh
sysg logs -s api --raw
```

`--raw` removes systemg's capture timestamp and stream label. ANSI escapes are
stripped by default.

### Time And Pattern Filtering

Use bounded log reads instead of asking for unbounded output:

```sh
sysg logs -s api --since 30m
sysg logs -s api --since 2h --grep ERROR
sysg logs -s api --since 2026-07-07 --until 2026-07-07T12:00:00Z
```

Accepted time forms:

- RFC3339 timestamp, such as `2026-07-07T14:00:00Z`.
- Bare UTC date, such as `2026-07-07`.
- Relative age in the past, such as `30m`, `2h`, or `7d`.

`--grep` is a regular expression. It composes with `--kind`, `--since`,
`--until`, and `--all`.

Any `--since`, `--until`, or `--all` read is a one-shot snapshot because an
upper-bounded or historical read cannot be followed continuously.

### Active And Rotated History

By default, `sysg logs` reads the full active log; use `--lines N` to bound it.
Use `--all` to read active plus rotated history:

```sh
sysg logs -s api --all --grep ERROR
```

The output is ordered oldest to newest, with rotated backups before the active
file. `--all` ignores `--lines`.

### Log Paths

Use `--path` to locate logs without relying on internal path conventions:

```sh
sysg logs --path
sysg logs -s api --path
sysg logs -s api --path --all
```

With no service, `--path` prints the log directory. With a service, it prints
that service's active log path. With `--all`, it prints active plus rotated
backup paths.

### ANSI Handling

systemg records service output verbatim, including ANSI escapes. For downstream
tools:

- ANSI stripping is on by default for `--format json`.
- ANSI stripping is on by default for `--raw`.
- ANSI stripping is on by default for non-interactive output.
- ANSI stripping is on by default in agent mode.
- Use `--strip-ansi` to force stripping.
- Use `--no-strip-ansi` to preserve escapes.

### Log Maintenance

Truncate retained logs or prune rotated backups when disk usage matters:

```sh
sysg logs --purge
sysg logs -s api --purge
sysg logs --prune --max-size 100MB
sysg logs --prune --max-age 7d
```

`--purge` truncates log files (all services plus `supervisor.log`, or one
service with `-s`). `--prune` deletes rotated backups by total size or age and
requires at least one of `--max-size` or `--max-age`.

## Configuration Keys

Manifest reference (see `docs/how-it-works/configuration.mdx` for details):

- Top level: `version: "2"` (required), canonical `projects` map or deprecated
  singular `project`, loose `services`, `env`, `logs` (`sink: file|none`, `max_bytes`, `max_files`),
  `status` (`snapshot_mode: off|summary|detailed`, `snapshot_interval_secs`),
  `metrics` (`retention_minutes`, `sample_interval_secs`, `max_memory_bytes`,
  `spillover_path`), and `services` (required).
- Per service: `command` (required), `depends_on`, `env` (`vars`, `file`,
  `inherit_env`, `clear_session_vars`, `strip`), `restart_policy`
  (`always|on-failure|never`; clean exits never restart), `backoff`,
  `max_restarts`, `hooks` (`onstart` after successful starts and `onerr` after
  unsuccessful exits), `cron` (`expression`, `timezone`),
  `deployment` (`strategy: rolling|immediate`, `pre_start`, `health_check`,
  `grace_period`, `blue_green`), `logs`, `skip`, `spawn` (`mode`, `limits`).
- Privileged mode only: `user`, `group`, `supplementary_groups`,
  `capabilities`, `limits`, `isolation`.

Health checks live under `deployment.health_check`, not at the service level.
`attempt_timeout` bounds one probe; `total_timeout` keeps fast failures from
exhausting a slow service's readiness window before both the retry floor and
time budget are exhausted.

Supervisor-wide defaults live outside project manifests in
`~/.local/share/systemg/supervisor.xml`, or `/var/lib/systemg/supervisor.xml` in
system mode. Its `logs` block sets `max_bytes` and `max_files`; its `timeouts`
block sets `pre_start_secs`, `startup_stability_ms`, and `stop_verify_secs`.
Compact legacy XML remains readable, while new XML output uses two-space
indentation.

## Command Recipes For LLMs

Get a parseable view of the system:

```sh
sysg status --format json
```

Get a readable view without terminal decoration:

```sh
sysg --plain status
```

Inspect one service:

```sh
sysg inspect -s api --format json
```

Check recent errors:

```sh
sysg logs -s api --format json --grep "ERROR|WARN|panic|failed" --since 2h
```

Get only application log text:

```sh
sysg logs -s api --raw --since 30m
```

Search all retained history:

```sh
sysg logs -s api --all --grep "database|timeout|connection"
```

Disambiguate a service name across projects:

```sh
sysg status --project <project-id> --service api --format json
sysg inspect --project <project-id> -s api --format json
sysg logs --project <project-id> -s api --format json
```

Restart one service:

```sh
sysg restart -s api
```

Stop one service:

```sh
sysg stop -s api
```

Restart a service in a specific project:

```sh
sysg restart --project <project-id> -s api
```

## Avoid These In One-Shot Agent Calls

Avoid long-running or interactive commands unless the user explicitly asked for
streaming:

```sh
sysg logs -s api --lines 0 --follow
sysg status --stream 5
sysg inspect -s api --stream 5
sysg logs -s api --stream 2
```

Prefer one-shot alternatives:

```sh
sysg --plain logs -s api --lines 200
sysg status --format json
sysg inspect -s api --format json
```

## Output Selection

For LLM and automation workflows:

- Prefer JSON for parsing and decision making.
- Prefer `--plain` for human-readable summaries.
- Prefer `--raw` when only application log text matters.
- Prefer `--grep`, `--since`, and `--lines` to bound log volume.
- Prefer `--project` when service names may be duplicated across projects.
- Prefer `--path` when another tool should read the logs directly.

## Claude Code Integration

systemg ships an official Claude Code plugin and skill from a self-hosted
marketplace in the repository. The skill teaches the `sysg` CLI, the config
schema, deployments, cron units, and log inspection, and instructs the agent to
use `--plain` in non-interactive contexts, prefer `--format json`, and run
`sysg validate` before starting a stack.

Install:

```text
/plugin marketplace add ra0x3/systemg
/plugin install systemg
```

The plugin version tracks the crate version (CI stamps
`.claude-plugin/plugin.json` from `Cargo.toml` on release). Refresh with
`/plugin marketplace update systemg`; Claude Code prompts for updates
automatically. Full details: https://sysg.dev/integrations/claude-code

## Relevant Local Docs

These docs contain the canonical user-facing descriptions:

- `docs/how-it-works/commands/index.mdx`: global flags, including `--plain`.
- `docs/how-it-works/commands/status.mdx`: status behavior and options.
- `docs/how-it-works/commands/validate.mdx`: config validation and diagnostics.
- `docs/how-it-works/commands/logs.mdx`: logs behavior, filters, JSON/raw
  output, ANSI handling, path lookup, and rotation.
- `docs/how-it-works/logs.mdx`: logging architecture and retention model.
- `docs/how-it-works/configuration.mdx`: service and project configuration.
- `docs/integrations/claude-code.mdx`: the Claude Code plugin and skill.
- `DEBUGGING.md`: evidence-first live-stack debugging and verification gates.