Server Configuration File

ChronoLog server executables share a single JSON configuration file. The repository ships a template at default_conf.json.in; a fully expanded example is installed as conf/default-chrono-conf.json and is used by tools/deploy/deploy_local.sh.

The file is organized into two shared blocks (clock, authentication) and four per-component sections (chrono_visor, chrono_keeper, chrono_grapher, chrono_player). See the Overview page for the loader architecture behind these sections.

{
  "clock":          { ... },
  "authentication": { ... },
  "chrono_visor":   { ... },
  "chrono_keeper":  { ... },
  "chrono_grapher": { ... },
  "chrono_player":  { ... }
}

---

Shared Blocks

These blocks are parsed once by ConfigurationManager and are visible to every process.

clock

Controls the clock source used for event timestamping. See Performance Tuning → Clock Source for the effect of each option.

Field	Type	Default	Description
clocksource_type	string	"CPP_STYLE"	One of "C_STYLE", "CPP_STYLE", "TSC".
drift_cal_sleep_sec	integer	10	Seconds between clock drift calibrations.
drift_cal_sleep_nsec	integer	0	Additional nanoseconds added to the calibration interval.

Backed by ClockConf in src/chrono-common/include/ConfigurationBlocks.h.

authentication

Field	Type	Default	Description
auth_type	string	"RBAC"	Authentication mechanism. Currently only "RBAC" is used.
module_location	string	"/path/to/auth_module"	Path to the authentication module shared object. The template's value is a placeholder; a file without the key gets "".

Backed by AuthConf in src/chrono-common/include/ConfigurationBlocks.h.

---

Common Per-Component Sub-Blocks

Several JSON sub-blocks appear in more than one component. They always have the same shape, parsed by the same C++ building block.

rpc — Thallium endpoint

Every service endpoint is described by an rpc block, parsed by RPCProviderConf.

Field	Type	Example	Description
protocol_conf	string	"ofi+sockets"	Thallium/Mercury transport string. See Network & RPC.
service_ip	string	"127.0.0.1"	IP the service binds to (servers) or connects to (clients).
service_base_port	integer	5555	TCP port.
service_provider_id	integer	55	Thallium provider ID; multiplexes services on the same address.

Monitoring — logging

Per-component log sink. Wrapped under a monitor key and parsed by LogConf.

"Monitoring": {
  "monitor": {
    "type": "file",
    "file": "chrono-visor-1.log",
    "level": "debug",
    "name": "ChronoVisor",
    "filesize": 104857600,
    "filenum": 3,
    "flushlevel": "warning"
  }
}

Field	Type	Description
type	string	Log sink type. Currently only "file" is supported.
file	string	Log file name.
level	string	Minimum log level: trace, debug, info, warning, error, critical, off.
name	string	Logger name that appears in log lines.
filesize	integer	Maximum file size in bytes before rotation.
filenum	integer	Number of rotated files to keep.
flushlevel	string	Minimum level that triggers an immediate flush to disk.

DataStoreInternals — story-chunk, tail and retention tuning

Appears in chrono_keeper, chrono_grapher, and chrono_player. Parsed by DataStoreConf. See Performance Tuning → Story Chunk Settings for the semantics of each field.

The defaults below are the template's. The first four keys have a different template value in each component. A key left out of the file takes the value compiled into the component instead, which for those four keys differs from the template: story_chunk_duration_secs 30 on the keeper and 60 on the grapher and player, acceptance_window_secs 10 on the keeper and 180 on the grapher and player, inactive_story_delay_secs 180 on the keeper and 300 on the grapher and player, and max_story_chunk_size 4096. The other keys compile to their template values.

Field	Type	Default	Description
max_story_chunk_size	integer	4096	Not used for now. Each component reads it, but nothing limits a chunk by its event count: a chunk covers a time range of story_chunk_duration_secs.
story_chunk_duration_secs	integer	keeper 10, grapher 30, player 60	How long a chunk remains open.
acceptance_window_secs	integer	keeper 15, grapher 60, player 300	Maximum allowed age of an incoming event relative to wall-clock time.
inactive_story_delay_secs	integer	keeper 120, grapher 300, player 300	Idle time before an in-memory story is evicted.
tail_capacity	integer	65536	(Keeper only) Maximum most-recent sealed events indexed per story for tail reads. A chunk with events in the index stays in memory; the index is released when the story retires. Must be $>0$.
live_tail_read	boolean	false	(Keeper only) When true, tail reads also serve unsealed events from the active timeline in addition to sealed chunks, dropping visibility latency to sub-second.
retention_cap_mb	integer	4096	(Keeper only) Retained-chunk memory, in MB, above which the keeper logs a warning while it waits for ChronoGrapher to persist chunks. Nothing is dropped. Set it above the memory a healthy keeper holds, or the warning fires in normal operation: about 3 × 125 s × peak per-keeper MB/s with the template's windows. 0 turns the warning off. See Durable Chunk Retention. The keeper's WAL block has its own cap, durable_cap_mb, on the part of that memory the grapher has not confirmed.
watermark_resend_timeout_secs	integer	300	(Keeper only) Seconds a keeper waits for ChronoGrapher to confirm a chunk written before sending it again. Keep it at least twice ChronoGrapher's story_chunk_duration_secs plus acceptance_window_secs; see Timing rules across components. Must be positive.
archive_visibility_delay_secs	integer	10	(Keeper only) Seconds a keeper keeps a chunk after ChronoGrapher confirms it written, serving its events to replays meanwhile. ChronoGrapher records the file in the archive manifest before it confirms, and a player reads the manifest before each archive read, so this only covers a replay already under way when the confirmation arrives; see Timing rules across components. 0 frees on confirmation. See Durable Chunk Retention.
shutdown_confirm_timeout_secs	integer	150	(Keeper only) Seconds a keeper stopped with SIGTERM waits for ChronoGrapher to confirm its chunks written before it exits. Cover ChronoGrapher's story_chunk_duration_secs plus acceptance_window_secs; see Timing rules across components. 0 exits without waiting.
watermark_report_interval_secs	integer	1	(Grapher only) How often ChronoGrapher sends changed persisted watermarks and unwritten receipts to the keepers.

Timing rules across components

Several timing settings only work together with a setting of another component: a keeper waits for ChronoGrapher to write its chunks, and a player has to find a file before the keeper lets its copy go. Each component reads only its own settings, so nothing checks these relations when a component starts. If you change any of the settings below, keep the rules. The values are the template's.

Rule	Template values	Why	If it is broken
grapher acceptance_window_secs > keeper story_chunk_duration_secs + keeper acceptance_window_secs	60 > 10 + 15	A keeper ships a chunk that long after its first event; it has to reach the grapher before the grapher's window for that time closes.	Every keeper chunk arrives after its window was written and lands in a second file for that window.
grapher story_chunk_duration_secs + acceptance_window_secs (the write window) stays short	30 + 60 = 90 s	An event reaches the archive within the write window, and a keeper holds each chunk at least that long.	Keepers hold more memory and replays read more from the keepers.
keeper watermark_resend_timeout_secs ≥ 2 × the grapher's write window	300 ≥ 180	A keeper sends a chunk again only when the grapher has had time to write it.	A delivery about to be confirmed is replaced by a new one that has to be confirmed from the start; chunks are sent and written again, and a keeper can wait indefinitely.
keeper shutdown_confirm_timeout_secs ≥ the grapher's write window	150 ≥ 90	A keeper stopped with SIGTERM waits for the grapher to write the chunks it just sent.	The keeper exits before its last chunks are confirmed, and frees them unconfirmed with a warning.
keeper retention_cap_mb ≥ 3 × peak per-keeper MB/s × (keeper story_chunk_duration_secs + acceptance_window_secs + the grapher's write window + grapher watermark_report_interval_secs + keeper archive_visibility_delay_secs)	4096 ≥ 3 × 10 × 126	That is how long a healthy keeper holds each chunk; the factor 3 leaves room for a short grapher stall.	The retention warning fires in normal operation and stops meaning anything. Nothing is dropped.
grapher watermark_report_interval_secs well below keeper archive_visibility_delay_secs	1 vs 10	Keepers learn that a chunk is written from these reports.	Chunks stay on the keepers longer than they need to.
grapher inactive_story_delay_secs > keeper story_chunk_duration_secs + keeper acceptance_window_secs	300 > 25	After a story is released, a keeper seals its last chunk that long later; the grapher's pipeline for the story should still be there.	The last chunks arrive after the pipeline retired and are written as reopened windows.

No rule ties a player's settings to a keeper's: a player finds files through the archive manifest, which ChronoGrapher appends to before it confirms a window written, and reads it before each archive read.

It also helps to make the grapher's story_chunk_duration_secs a multiple of the keeper's (30 and 10), so that a keeper's chunk does not straddle two grapher windows.

A client waits up to 180 s for a replay (fixed in the client library). A player gives each keeper 5 s to answer before it returns a partial result, so this leaves the archive read most of that time.

WAL — keeper write-ahead log

Appears in chrono_keeper only. Parsed by WalConf (src/chrono-keeper/include/ChronoKeeperConfiguration.h). The WAL holds the events of durable writes on the keeper's disk until ChronoGrapher confirms the chunks that contain them. With enabled false, which is the default, a durable log_event is refused with CL_ERR_DURABILITY_UNAVAILABLE and the keeper behaves as it did before the WAL existed. A missing WAL block leaves every field at its default.

"WAL": {
  "enabled": false,
  "dir": "",
  "segment_bytes": 67108864,
  "max_bytes": 1073741824,
  "group_commit_max_bytes": 4194304,
  "group_commit_window_us": 0,
  "reserve_min_bytes": 268435456,
  "reserve_percent": 10,
  "durable_cap_mb": 4096
}

Field	Type	Default	Description
enabled	boolean	false	Turns the WAL on. deploy_local.sh sets it with --wal; deploy_cluster.sh and single_user_deploy.sh with --wal or --wal-dir. All three also read WAL_ENABLED=1 from the environment.
dir	string	""	Absolute path of the WAL directory. Required when enabled is true. The deploy scripts fill it in for each keeper.
segment_bytes	integer	67108864	Size of one WAL segment file, 64 MiB. At least 1048576 (1 MiB). Checked even when the WAL is off.
max_bytes	integer	1073741824	Total size of the WAL segments above which the keeper logs a warning, at most once a minute, while it waits for ChronoGrapher to confirm events. Nothing is refused or dropped.
group_commit_max_bytes	integer	4194304	Largest group of queued events written and synced together, 4 MiB. A group always holds at least one event.
group_commit_window_us	integer	0	Microseconds the committer waits after the first queued event for more events before it syncs. 0 syncs as soon as the committer is free, so the group size follows the load. The wait also ends early when the queued bytes reach group_commit_max_bytes. At most 10000.
reserve_min_bytes	integer	268435456	Free space, in bytes, the WAL directory's file system keeps (256 MiB). The reserve is the larger of this and reserve_percent of the file system.
reserve_percent	integer	10	Percentage of the WAL directory's file system kept free. 0 to 100. Below the reserve, durable writes get CL_ERR_DURABILITY_FULL until space is back.
durable_cap_mb	integer	4096	Memory, in MB, of retained chunks the grapher has not confirmed, above which a durable write gets CL_ERR_DURABILITY_FULL. Durable writes are admitted again once it is down to three quarters of the cap. A confirmed chunk the tail index still holds does not count; retention_cap_mb watches all retained memory and only warns.

Validation runs when the keeper starts, and a refused value stops the keeper:

• enabled must be a boolean, dir a string, and the numeric fields integers.
• With enabled true, dir must be non-empty and begin with /.
• Every size (segment_bytes, max_bytes, group_commit_max_bytes, reserve_min_bytes, durable_cap_mb) must be at least 1.
• segment_bytes must be at least 1 MiB, group_commit_window_us at most 10000, and reserve_percent between 0 and 100.
• durable_cap_mb must not exceed the largest value whose size in bytes fits in a signed 64-bit integer.
• A key the keeper does not know is printed to standard error and ignored.

The keeper also exits at startup when it cannot open dir or when the recovery of the WAL fails.

The longest durable event is 256 MiB minus 36 bytes. That limit is fixed and has no key. A longer durable event gets CL_ERR_INVALID_ARG and is not stored.

ExtractionModule

Present under chrono_keeper and chrono_grapher. Parsed by ExtractionModuleConfiguration (src/chrono-common/include/ExtractionModuleConfiguration.h). Defines the configurable pipeline of extractors that drains retired StoryChunks. See Architecture → ChronoKeeper for the concept and Performance Tuning → Extraction Pipeline for tuning guidance.

Field	Type	Description
extraction_stream_count	integer	Number of parallel extraction-stream threads draining the extraction queue. Default: 2.
extraction_protocol	string	Thallium protocol of the keeper's engine for its RDMA extractors. Default: "ofi+sockets". Read by the keeper only.
extractors	object	Map of extractor entries. Each key is a label of your choice (the template uses extractor_to_grapher and hdf5_archive_extractor), and each entry must set "type" to one of the extractor types listed below; additional fields depend on the type. A component keeps one extractor per type: of two entries with the same type, only the last one in the file is used. An entry whose type the component does not know is ignored.

Supported extractor types

type	Available in	Required fields	Behavior
csv_extractor	keeper, grapher	csv_archive_dir (string)	Writes each StoryChunk as a CSV file under csv_archive_dir.
single_endpoint_rdma_extractor	keeper	receiving_endpoint (object: protocol_conf, service_ip, service_base_port, service_provider_id)	Drains each StoryChunk to one RDMA endpoint, the ChronoGrapher's KeeperGrapherDrainService.
dual_endpoint_rdma_extractor	keeper	grapher_receiving_endpoint and player_receiving_endpoint (objects with the same fields as receiving_endpoint)	Sends each StoryChunk to the ChronoGrapher's KeeperGrapherDrainService and to the ChronoPlayer's PlayerRecordingService.
hdf5_extractor	grapher (required)	hdf5_archive_dir (string)	Serializes each StoryChunk into the HDF5 archive under hdf5_archive_dir.
logging_extractor	keeper, grapher	none	Logs one INFO line per StoryChunk (story, time range, event count) and stores nothing.

A ChronoGrapher's chain must include hdf5_extractor: it is the only extractor that confirms chunks written, and keepers free a chunk only once the grapher confirms it. A grapher whose extractors list is empty or absent gets an hdf5_extractor with hdf5_archive_dir /tmp; one whose list leaves it out, such as a csv_extractor alone, fails to start. Other extractors may run alongside it.

Keepers ship with single_endpoint_rdma_extractor to ChronoGrapher. Replay reads recent events from the keepers directly, so the ChronoPlayer copy that dual_endpoint_rdma_extractor sends is no longer needed.

See the keeper and grapher blocks in conf/default_conf.json.in for fully expanded examples.

ArchiveReaders — story files directory

Present under chrono_player. Parsed by ExtractorReaderConf.

Field	Type	Description
story_files_dir	string	The archive directory the player reads from: ChronoGrapher's hdf5_archive_dir. The player finds files through the archive manifest there; see HDF5 Archive and Archive on a Shared File System.

IngestionThreadCount — ingestion-thread parallelism

A scalar field that sits inside the ingestion service object of each component:

Component	Path	Default	Description
ChronoKeeper	chrono_keeper.KeeperRecordingService.IngestionThreadCount	4	Number of worker threads serving the Keeper recording RPC.
ChronoGrapher	chrono_grapher.KeeperGrapherDrainService.IngestionThreadCount	1	Number of worker threads serving the Grapher drain RPC.
ChronoPlayer	chrono_player.PlaybackQueryService.IngestionThreadCount	1	Number of worker threads serving the Player playback-query RPC.
ChronoPlayer	chrono_player.PlayerRecordingService.IngestionThreadCount	1	Number of worker threads receiving chunks from a keeper's dual_endpoint_rdma_extractor.

---

chrono_visor

Parsed by VisorConfiguration (src/chrono-visor/include/ChronoVisorConfiguration.h).

Field	Type	Description
VisorClientPortalService	object	{ "rpc": { ... } } — endpoint used by clients to connect to ChronoVisor.
VisorKeeperRegistryService	object	{ "rpc": { ... } } — endpoint where ChronoKeeper/Grapher/Player register and heartbeat.
Monitoring	object	See Monitoring.
delayed_data_admin_exit_in_secs	integer	Seconds ChronoVisor keeps its admin RPC client for a keeper, grapher or player after that process unregisters or registers again, and while ChronoVisor shuts down, before destroying it, so calls already sent on it can finish. It holds no events. Default 3; a value outside 1–59 becomes 5.

chrono_keeper

Parsed by KeeperConfiguration (src/chrono-keeper/include/ChronoKeeperConfiguration.h).

Field	Type	Description
RecordingGroup	integer	Logical group this keeper belongs to. Must match its paired ChronoGrapher.
KeeperRecordingService	object	{ "rpc": { ... } } — endpoint that receives events from ChronoVisor for recording.
KeeperDataStoreAdminService	object	{ "rpc": { ... } } — endpoint for admin actions on the keeper's data store.
VisorKeeperRegistryService	object	{ "rpc": { ... } } — endpoint on ChronoVisor where this keeper registers.
KeeperGrapherDrainService	object	{ "rpc": { ... } } — endpoint on ChronoGrapher where drained chunks are delivered.
Monitoring	object	See Monitoring.
DataStoreInternals	object	See DataStoreInternals.
ExtractionModule	object	See ExtractionModule.
WAL	object	Write-ahead log for durable writes. Optional; off by default. See WAL.

KeeperRecordingService carries an IngestionThreadCount field — see IngestionThreadCount.

chrono_grapher

Parsed by GrapherConfiguration (src/chrono-grapher/include/ChronoGrapherConfiguration.h).

Field	Type	Description
RecordingGroup	integer	Logical group this grapher serves.
KeeperGrapherDrainService	object	{ "rpc": { ... } } — endpoint where this grapher receives drained chunks from a keeper.
DataStoreAdminService	object	{ "rpc": { ... } } — endpoint for admin actions on the grapher's data store.
VisorRegistryService	object	{ "rpc": { ... } } — endpoint on ChronoVisor where this grapher registers.
Monitoring	object	See Monitoring.
DataStoreInternals	object	See DataStoreInternals.
ExtractionModule	object	See ExtractionModule.

KeeperGrapherDrainService carries an IngestionThreadCount field — see IngestionThreadCount.

chrono_player

Parsed by PlayerConfiguration (src/chrono-player/include/ChronoPlayerConfiguration.h).

Field	Type	Description
RecordingGroup	integer	Logical group this player serves.
PlayerStoreAdminService	object	{ "rpc": { ... } } — endpoint for admin actions on the player's data store.
PlaybackQueryService	object	{ "rpc": { ... } } — endpoint that receives playback queries from clients.
VisorRegistryService	object	{ "rpc": { ... } } — endpoint on ChronoVisor where this player registers.
Monitoring	object	See Monitoring.
DataStoreInternals	object	See DataStoreInternals.
PlayerRecordingService	object	{ "rpc": { ... } } — endpoint that receives chunks from a keeper's dual_endpoint_rdma_extractor. Keepers ship with single_endpoint_rdma_extractor, so nothing sends to it by default.
ArchiveReaders	object	See ArchiveReaders.

PlaybackQueryService and PlayerRecordingService carry an IngestionThreadCount field — see IngestionThreadCount.

---

Full Example

The repository ships a fully expanded example at conf/default_conf.json.in. A key left out of the file takes the value set in the constructor of the component's configuration class, which is not always the template's value; see DataStoreInternals.