Construction agent

Skill-guided self-evolving construction

Prepare and optimize graph schemas for graph-driven analysis: preserve relational semantics, freeze evaluation queries, explore structures, and retain validation results and domain experience.

#How skills guide construction

Self-evolving construction provides graph schema preparation and optimization within iGraph’s graph-driven analysis. When relational sources need a queryable graph or an existing graph schema needs optimization for fixed queries, the construction agent prepares and validates candidates. Agent Runtime skills organize dataset-state checks, tool steps, and edit approvals; the Graph Engine construction subagent handles graph schemas, candidate evaluation, and domain experience. The SQL reference executor independently computes source answers so graph schema changes are tested against the same queries, while existing suitable graphs remain directly available to the main analysis agent.

LayerExecutionRetention and reuse
Procedural skillsLocalSkills loads bundled SKILL.md files and approved skills for the current user. graph_construction defines status inspection, loading, optimization previews, and commits.Reviewed candidate skills enter the user skill directory and become available in subsequent tasks.
Construction strategiesTabularConstructionSubAgent applies constrained transformations under a frozen workload and independently compares source-table and graph answers.DomainExperienceStore records correctness, cost gains, and outcome types. Similar tasks retrieve evidence to rank candidates.

The two layers have separate storage boundaries: candidate SKILL.md files are not loaded by LocalSkills before approval, and construction strategy experience is stored in SQLite under the default workspace owner. Updating construction experience neither rewrites built-in SKILL.md files nor promotes it to an approved user skill.

Procedural skills define how a task is executed; construction strategies determine which graph schema is used for the current data. For example, graph_construction resolves the dataset belonging to the current attachment and checks its state. It loads an existing graph when ready and previews edits before submission. Inside the construction subagent, choosing path materialization or attribute entities depends on the current input, workload, and evidence. A valid outer workflow does not prove a candidate valid, and successful optimization does not approve later user edits.

Agent Runtime uses Agno LocalSkills to load built-in skills and active skills in the default workspace. Candidate directories are excluded; approval moves a candidate into the active directory. The construction subagent uses only the retained sources, construction-specific tools, and domain experience for the current workspace, without main-session messages, general memory, or filesystem tools.

#Construction and evolution loop

Self-evolving construction: iteration and experienceOpen diagram ↗Open diagram ↗
Self-evolving construction: iteration and experienceSelf-evolving construction: iteration and experience

The circle shows candidate iteration within one construction run: choose a strategy from the domain, query hotspots, and experience; transform a copy; validate independently; accept or reject; then use feedback to choose again. Acceptance updates the retained graph, while rejection keeps it for further exploration. The outer dashed return shows cross-run reuse: eligible strategy evidence is saved only after exploration ends and final validation passes, then informs candidate priorities in later runs.

The frozen evaluation queries, independent SQL reference executor, candidate optimization, and domain experience apply to SQL, CSV, and TXT imported as tables or line records. Only txt_mode=entities takes a separate path: chunked LLM extraction, checking that both relationship endpoints occur in the same source chunk, deduplication, and construction_graph.v1 validation. Completing this mode does not establish SQL-verified relationship semantics.

  • Relational inputs build property graph G₀ from primary and foreign keys while retaining row identity, types, NULLs, and direction; semantic TXT enters a separate chunked extraction path.
  • Read the domain description and schema, generate business queries or accept supplied queries, then freeze the workload and record its SHA-256.
  • Run the source SQL oracle and graph executor separately to measure correctness, index preparation cost, and query latency.
  • Select strategies using domain priors, graph structure, and historical evidence; instantiate candidates with real tables, columns, and node IDs from this task.
  • Transform a copy, check full source integrity, and execute every frozen query. Accept improvements that meet the criteria, otherwise retain the current graph.
  • Record candidate outcomes. After an independent final validation, save the graph and evolution-report.json and retain structured evidence for subsequent builds.

Default llm mode uses a model to generate evaluation queries and plan candidates with separate construction model settings. rules mode uses deterministic queries and code-generated candidates for offline validation; it currently tries materialize_join, generate_entity_table, and transform_batch rather than every transformation. An LLM failure does not silently fall back to rules and claim success.

PhaseAvailable toolsProgress condition
Workload generationinspect_schema, set_workloadProvide exactly query_count valid queries and retain categories covered by the seeds; one successful freeze completes the phase.
Candidate evolutioninspect_schema, evaluate_queries, eight transformation entry points, finishThe workload is frozen; transformation feedback decides whether the current graph changes. Workload editing tools are no longer available.
Final validationExecuted by the workerRecheck source integrity, the workload fingerprint, and every query result; the model cannot bypass this step.

Query generation allows up to three model responses by default. The model must call set_workload; prose alone does not freeze queries. Missing fields, duplicate plans, or reduced coverage return structured errors that can be corrected within budget. Candidate optimization allows up to twelve model responses, each with at most ten tool calls. Calls run sequentially and share the default total of 24 with query generation.

finish requests completion; it does not publish a graph. LLM mode requires at least one candidate to reach query evaluation and the current graph to pass every frozen query before accepting finish. final_validation_pending=true means final validation is outstanding. The iteration limit also leads to final validation: a run can succeed if a candidate was evaluated and the retained graph passes. A run consisting only of invalid calls fails with no_evolution_candidate_evaluated.

json
{
  "tool": "finish",
  "arguments": {
    "reason": "All frozen queries pass; no further candidate is justified."
  }
}

#Source data preview and field explanations

After uploading SQL, CSV, or TXT, the construction panel can preview the retained source before a build starts. The preview remains available after construction. SQL and CSV provide a table selector, fields, declared types, keys, descriptions, and sample rows; TXT displays an excerpt from the beginning of the original text. Reading the preview neither starts construction nor changes a saved or active session graph.

Description sourceInterpretation
Comment · commentAn identifiable SQL column comment takes precedence over other descriptions. Not every database dialect’s comment syntax is supported.
Schema · schemaExplains a declared primary key, foreign key, or referenced field: structural facts, not a complete business definition.
Name hint · inferredOffers a tentative hint for recognized names such as name, status, quantity, and DisplayName. It does not invent currency, units, population definitions, or status-code mappings.
Unavailable · unavailableShown explicitly when no comment, key declaration, or recognized name hint is available; consult the data dictionary or domain documentation.
BoundaryCurrent limit
SamplesAt most 5 rows per table, 300 displayed characters per cell, and a 4,000-character TXT excerpt.
Source and responseSQL/CSV preview caps the source at 32 MiB. TXT reads only a bounded prefix, so a larger file does not by itself prevent an excerpt. Responses are capped at 4 MiB UTF-8, with at most 128 tables and 256 columns per table; CSV bundles also check total uncompressed size.
Row countsCounts appear when available. A partial TXT excerpt or read-only SQLite sample may have no total count. These are initial records, not a random sample or full statistics.
Separate budgetsSQL preview has separate parsing-time, token-count, and statement-size budgets; SQLite reads limit time and individual values. The full_scale construction profile does not raise preview limits.

SQL-text preview parses CREATE TABLE and INSERT ... VALUES as data without executing uploaded SQL; unsupported statements or values produce warnings or placeholders. SQLite images are read-only: virtual tables are skipped, while unsupported structures such as generated columns are rejected. CSV can be previewed again with the panel’s selected delimiter; TXT always shows the original excerpt. Large integers may be returned as strings to preserve browser precision, and binary values are not displayed verbatim.

http
GET /api/personal-datasets/{dataset_id}/construction/source-preview
GET /api/projects/{project_id}/datasets/{dataset_id}/construction/source-preview

# Optional CSV delimiter, URL-encoded when necessary
GET /api/personal-datasets/{dataset_id}/construction/source-preview?delimiter=%09

#Inputs and relational semantics

InputSupport and constraints
CSVUTF-8 / BOM, headers, strict column counts; cell values remain strings.
SQLSQLite-compatible ordinary DDL, literal INSERTs, supported dump transaction markers, or a SQLite database image with a .sql extension.
TXTUTF-8 / BOM. auto recognizes a strict tab-delimited table or creates line records, lines forces line records, and entities uses an LLM to identify entities and relationships.
  • Each row becomes a node. Reliable primary keys determine stable identities, including full composite keys.
  • Foreign keys connect child rows to actual referenced parent rows. Matching column names alone do not create relationships.
  • Without a reliable primary key, row positions are used and a warning is recorded. Dangling references do not create fake nodes.
  • SQL inputs do not connect to production databases or execute arbitrary scripts. Unsupported views and triggers must be converted into ordinary tables first.

Plain text may contain prose or one or more facts per line. entities needs no header and does not require one edge per line. It requires mode=llm and cannot be combined with delimiter, primary_key, foreign_keys, or tables. The system sends the full text to the construction model in source order, using chunks with bounded overlap. Both endpoints of a retained relationship must occur in the same source chunk; otherwise the candidate is discarded. If a chunk still returns neither valid output nor an explicit no-relationship result after retries, or its call fails, the entire build fails without saving a partial graph. These checks do not prove that the model interpreted the relationship correctly; review the original text before use.

text
张三就职于星河公司。
李四向张三汇报。
星河公司位于合肥。

The SQL loader identifies database images by the SQLite header and parses other input as UTF-8 SQL text. The default bounded profile imports images or SQL text into in-memory SQLite; an explicit full_scale profile accesses SQLite images as read-only files. sqlite_master, table_xinfo, and foreign_key_list supply tables, columns, ordered primary keys, and grouped foreign keys. Text accepts only ordinary table/index definitions, literal INSERTs, and supported dump transaction markers; SELECT, WITH, UPDATE, RETURNING, and other unsupported inputs are rejected. Images are also checked for views, triggers, virtual tables, and generated columns and cannot bypass construction input constraints.

Relational comparison is not just visible string equality. SQL imports retain declared types and BINARY, NOCASE, or RTRIM collations. Foreign-key matching places the parent column on the left so SQLite applies its affinity and collation. A parent key declared TEXT COLLATE NOCASE can match a reference with different casing; the graph must not substitute arbitrary string comparison. SQL identifiers use SQLite’s ASCII case rules, while cell contents are not lowercased.

CSV headers and row widths are validated strictly, and values remain strings: 0012 is not converted to a number, and empty strings are not universally converted to NULL. An empty CSV foreign-key value means no reference; an SQL empty string remains a text value that may match a parent key. For a directly uploaded single CSV, API options.table_name, primary_key, foreign_keys, and inline tables can supply structure. Inline tables accept data and key definitions, not external file paths. column_types is descriptive and does not convert CSV strings to numbers. CSV bundles configured in the workspace use the schema retained with their files; see multi-table inputs.

Cells must be finite JSON scalars: strings, numbers, booleans, or NULL. BLOBs, nested arrays, objects, and nonfinite numbers need preprocessing. A CSV amount="120.0" remains a string and cannot directly serve as a numeric SUM/AVG column. Prepare typed SQL data or an inline table containing numeric scalars for numeric aggregation.

The following illustrative CSV options assume an uploaded header of order_id,customer_id,amount and provide customers inline to demonstrate key declarations and type boundaries.

json
{
  "table_name": "orders",
  "primary_key": [
    "order_id"
  ],
  "foreign_keys": [
    {
      "columns": [
        "customer_id"
      ],
      "ref_table": "customers",
      "ref_columns": [
        "id"
      ]
    }
  ],
  "tables": [
    {
      "name": "customers",
      "columns": [
        "id",
        "name"
      ],
      "rows": [
        {
          "id": "C001",
          "name": "Alice"
        }
      ],
      "primary_key": [
        "id"
      ]
    }
  ]
}

Row identities are derived from the table name, key mode, and complete PK value list using SHA-256. A composite key cannot be reduced to its first column; reordering rows does not change keyed identities. Without a reliable key, identity uses the table name and row position, so reordering can change it and produces a warning. CSV key inference checks only explicit names such as id, table_name_id, and singular_table_name_id and verifies nonempty uniqueness; accidental uniqueness is insufficient.

text
Keyed row:
row_id = "row_" + SHA256(JSON([table, "pk", [key_1, key_2, ...]]))[:40]

Keyless row:
row_id = "row_" + SHA256(JSON([table, "position", row_index]))[:40]

JSON is canonicalized; complete composite keys participate in identity.

Initial construction creates nodes for every row before adding actual declared-FK matches. Non-NULL referenced keys must be unique; duplicate targets fail import. A composite FK containing NULL creates no relationship. A non-NULL unmatched reference retains its row and produces a warning. Types and collations are stored in relational_schema on the first row of each table; later graph schemas must preserve those semantics as well as business values.

TXT auto only detects a tab-delimited table with consistent row widths and at least one data row; otherwise each line becomes a record with a text field, including blank lines. lines forces line records. An explicit delimiter imports a strict table but cannot be combined with lines or entities. Line import does not infer semantic relationships; select entity-and-relationship extraction explicitly in the workspace when needed.

#Multi-table CSV and key definitions

Uploading one or more CSV files in the workspace opens a schema dialog. Each file represents one table; review its table name, headers, primary key, and foreign keys before submission. Name-based key suggestions are not verified relationships: confirm their business meaning, and construction will check uniqueness and actual references. The dialog currently reads comma-separated headers; convert other delimiters or use API input that supports delimiter.

The upload API accepts files and csv_schema. csv_schema uses version="csv_bundle.v1" and one tables entry per file, declaring file_name, name, columns, primary_key, and foreign_keys. The server retains the CSV contents and schema as one dataset. This is not an arbitrary ZIP-upload interface, and a multi-file request cannot mix CSV with SQL or TXT. A bundle contains 1–128 tables with unique file and table names.

Construction uses the saved bundle definition and rejects overrides through options.tables, primary_key, or foreign_keys. For SQL, multiple tables and their foreign keys come from the same SQL file. Inline tables for a directly uploaded single CSV are a separate API mechanism and are not interchangeable with a retained bundle.

#Frozen evaluation queries and DSL

set_workload accepts queries once during workload generation, with exactly query_count plans. The source-table oracle computes expected answers; the model supplies no answers. Questions and plans cannot be changed after freezing to improve the score. Each plan is compiled to SQL and independently executed by GraphExecutor on the current graph.

DSL fieldSemantics
root / joinsThe root alias is t0. Follow declared foreign keys forward or in reverse using inner/left joins, up to three hops.
filtersUse qualified column names such as t0.amount. Comparisons support eq, ne, lt, le, gt, ge, in, contains, is_null, and not_null.
projection / group_byNonaggregate projections must satisfy grouping constraints; results retain duplicate rows.
aggregationscount, sum, min, max, avg; sum/avg require numeric columns. Empty groups yield 0 for count and null for the other aggregates.
order_by / limitlimit requires deterministic ordering with output aliases covered by order_by; ordering and tie-group semantics are validated.

GraphExecutor reads only the current graph, reconstructing relational rows from row nodes, bridge edges, property entities, and relocated attributes. It cannot access source tables or oracle answers. source_integrity_errors also traverses every source record and declared FK, checking identity, fields, relationships, and column semantics, including data outside the workload.

Coverage includes relationships, optional joins, multiple hops, value filters, NULLs, aggregation, grouping, and sorting. A result of 15/15 means every query in that frozen workload passes; it does not establish correctness for arbitrary natural-language questions.

The generator builds query pools per table rather than letting one wide table consume the workload. Tables contribute counts, details, attribute lookups, ordering, grouping, NULL checks, and other plans. Business categories such as status/category and measures such as amount/price rank ahead of identifiers. Keys support exact lookup, but are not treated as amounts to sum or preferred business grouping categories. Filter constants come from actual samples rather than invented values that always return empty results.

Relationship plans expand through source FK-match indexes for at most three steps. Generation avoids immediate backtracking over one FK and repeated use of a directed relation. Multi-hop counts and details require reachable source rows; LEFT JOIN plans retain unmatched records. Selection balances root-table coverage before rotating categories and considers at most 2,000 generated plans. Insufficient distinct executable plans cause an explicit failure rather than filling the set with reworded duplicates.

There are three workload sources: fixed plans in options.queries, generator output in rules mode, and model-generated plans in LLM mode. The model receives valid seeds, domain context, input structure, samples, and required categories; set_workload validates its response. Coverage requirements are relative to the seeds, so a source without FKs is not forced to invent relationship queries. Supplied queries still undergo structural and semantic validation, but do not have to fill the seed-category set.

Invalid planRejection
Only changing the ID or wording of an identical planThe semantic fingerprint is unchanged; this adds no distinct coverage.
is_null with a value argumentNULL predicates take no value; use {column, op}.
limit=10 with only some output columns orderedEvery output alias must be ordered to make limiting deterministic.
Using arbitrary column equality instead of a declared FKjoins must match a complete FK declared in the input.
Comparing a SQL TEXT column directly with a floatImplicit float-to-TEXT comparison is unsupported; provide an explicit text literal.

Before freezing, validation fills omitted lists and limit:null, deduplicates semantic plans, and creates a normalized JSON copy. workload_sha256 is checked before each candidate and before returning. The illustrative COUNT plan below can be placed in queries with query_count=1. It counts missing amount values without treating NULL as zero.

json
{
  "id": "missing_amount",
  "question": "How many orders have no amount?",
  "root": {
    "table": "orders",
    "alias": "t0"
  },
  "joins": [],
  "filters": [
    {
      "column": "t0.amount",
      "op": "is_null"
    }
  ],
  "projection": [],
  "group_by": [],
  "aggregations": [
    {
      "function": "count",
      "column": "*",
      "as": "missing_count"
    }
  ],
  "order_by": [],
  "limit": null
}

For in-memory SQL input, the independent reference executor copies the original connection to preserve definitions and comparison rules. For a full_scale read-only SQLite file, it verifies file identity and opens a separate read-only connection. CSV and inline data use temporary tables without additional SQLite affinity conversion, preserving scalar types. The same DSL is compiled with qualified identifiers and bound parameters; filter values are not concatenated into executable SQL. The graph executes the logic independently, and the two paths meet only at result comparison.

The column references in filters, projection, group_by, and aggregation inputs address source columns as alias.column, such as t0.status or t1.amount. Aggregate output aliases such as cnt are available to order_by, not to filters as source fields; the DSL currently has no HAVING. COUNT(*) can use *, while other aggregates require a valid column. A question that needs post-aggregation filtering cannot be preserved by substituting an arbitrary source column for cnt.

JOIN direction is relative to the foreign-key declaration. For orders.customer_id → customers.id, forward starts from an orders alias and enters customers; reverse starts from a customers alias and enters orders. A reverse join still declares foreign_key.table=orders and ref_table=customers: do not swap child and parent. from is an existing alias, table names the table added by this step, and alias assigns its new alias. Both directions must use a complete declared foreign key.

#Construction skills and transformations

Across its phases, the private construction subagent exposes twelve distinct tool names: four read/control tools, seven single-transformation entry points, and transform_batch for composition. optimize_graph contains six operations, giving twelve single-transformation strategies in total. Composition reuses these operations rather than adding a thirteenth standalone strategy. These tools are separate from the main agent’s public MCP catalog.

Control toolArgumentsBehavior
inspect_schema{}Read source/current graph schemas, samples, and real node/edge IDs.
set_workloadqueries[]Freeze the workload once during generation; expected answers are not accepted.
evaluate_queries{}Return per-query validation, pass count, errors, and timing for the current graph.
finishreasonRequest final validation. LLM mode requires at least one valid candidate evaluated against the oracle, and all current queries passing.
TransformationRequired argumentsGraph schema changes and constraints
connect_columnstable, columns[], ref_table, ref_columns[]Join column values. Declared FKs retain source semantics; other relations carry column_join metadata. Optional edge_type.
materialize_joinpath[]Materialize a 2–3-step FK path while preserving intermediate node_ids and multiplicity. Each step has table, columns, ref_table, ref_columns, and direction. Optional edge_type.
generate_entity_tabletable, column, node_typeTurn distinct attribute values into shared entities while retaining source values. Optional edge_type; node types cannot overwrite source tables.
unfold_multi_categorytable, column, node_typeSplit multivalue categories and retain position on relationships without rewriting source cells. Optional edge_type and delimiter.
add_primary_keytable, columns[]Require a non-null unique candidate key; record identity_key while retaining source DDL and stable row IDs.
remove_primary_keytableRemove a proposed identity_key constraint, retaining source PKs and row identity.
optimize_graphoperations[]Apply the six structural operations below; all candidates undergo the same integrity and query validation.
transform_batchsteps[]Compose 1–6 single transformations and evaluate the final graph as one candidate. No nested batches; structural or attribute changes must precede materialize_join steps.
opRequired fields besides opValidation semantics
collapse_nodenode_id, edge_typeCollapse only a bridge row with exactly two current FKs and no incoming FK. Preserve row data and multiplicity on its edge.
node_property_to_edgenode_id, property, edge_idsMove a business field onto currently incident edges. Optional target_property and remove_from_node.
extract_property_nodenode_ids, property, node_type, edge_typeStore value on a shared entity and mark edges with extracted_attribute. Optional remove_from_node.
deletetarget, idsOptional property and cascade; no source record or declared relationship may be lost.
reverse_edgesedge_idsReverse specific existing edges; source FK direction remains subject to relational integrity.
rename_typetarget, old_type, new_typeRename business node/edge types while retaining identity and semantics.

property refers to business fields inside properties.data, such as status. Internal fields such as data, table, and row_index cannot be moved as ordinary attributes. Non-path transformations invalidate and remove old shortcuts. Repeated candidates with the same graph version and arguments are rejected.

Graph schemas can change, but logical source rows must remain recoverable from the current graph. resolve_rows validates node/edge IDs, actual endpoints, and table, row_index, and data, then reads current row-node data, bridge rows on edges, and values reached through extracted-attribute links or relocation records. It returns detached rows for execution without reading historical graphs or repairing missing fields from source tables.

Current graph schemaResolutionFailure example
Row nodeproperties.table / row_index / dataDuplicated logical identity or data that is not a column map.
Bridge row edgeIts own data and complete FK roles in row_endpointsA row_endpoints entry is not an actual edge endpoint.
Extracted attributeRow → attribute entity; extracted_attribute identifies the original columnLinking the wrong table’s row, missing entity value, or a conflicting row value.
Attribute moved to an edgerelocated_attributes supplies row_id, column, and valuerow_id is not incident to the edge, or storage locations disagree.

When order status becomes a shared entity, queries read its current value. If it says paid while the row stores pending, resolve_rows rejects the conflict rather than choosing whichever passes. If both the row field and its attribute relationship are removed, integrity validation reports missing source data. The candidate remains invalid even when no workload query filters status.

collapse_node has stricter source-row conditions: no declared FK may reference its table, the row must have exactly two incident FK edges belonging to that table, and there can be no other incident edges or self-collapse. The resulting edge retains the original row and both FK roles; repeated purchases remain separate row edges. This targets bridge records rather than arbitrary highly connected nodes.

materialize_join stores more than endpoints: each shortcut retains the path and intermediate nodes. GraphExecutor checks every step against current FK adjacency, then counts paths to verify that the set for a start node is complete. An incomplete shortcut set is not enabled; execution falls back to original relationships so join multiplicity is not silently reduced.

Candidate examples are cached against the graph version and regenerated after accepting a new graph. Rejected candidates leave the incumbent unchanged. Duplicate detection fingerprints the graph, tool, and arguments, preventing repeated evaluation of the same proposal. Non-path transformations remove old shortcuts because changed nodes, fields, or directions can invalidate them; further materialization must use the new graph.

Each transform_batch step uses its tool’s original {tool, arguments} schema. Only the complete resulting graph is evaluated for correctness and cost; intermediate changes do not each need to show a speed gain. Failure of any step discards the candidate. Consecutive materializations can retain distinct shortcut paths, but an ordinary transformation after them is rejected because it would invalidate those paths.

Attribute entities can also provide a physical access path. GraphExecutor first verifies from the current graph that every non-NULL row of a table/column is covered and each entity value exactly matches its live row value, then builds an attribute index for t0 root predicates. Partial, stale, or tokenized category structures cannot narrow a scan, and source tables are not consulted to complete the index. physical_plan, source_rows_examined, and attribute_index_hits record whether it was used.

#Correctness and cost acceptance

Let A be the passed query count, L the sum of per-query median runtimes, P index preparation time, and R expected query runs. Cost amortizes index preparation over anticipated use instead of considering query latency alone.

text
Cost(G) = L(G) + P(G) / expected_query_runs
gain(round) = (Cost(current, round) - Cost(candidate, round)) / Cost(current, round)

performance_pass = all_rounds_correct
  AND median(gain) > min_speedup
  AND wins_over_threshold >= ceil(2 * comparison_rounds / 3)
  AND median_gain_when_current_first > min_speedup
  AND median_gain_when_candidate_first > min_speedup

accept = source_integrity_ok AND graph_growth_within_limit AND (
  A(candidate) > A(current)
  OR (A(candidate) = A(current) = query_count AND performance_pass)
)

Defaults are expected_query_runs=100, min_speedup=0.05, and max_graph_growth=3.0. When both graphs answer every query correctly, comparison_rounds paired measurements alternate which graph runs first rather than comparing the candidate with a historical baseline timing. Every pair must remain correct; the overall median gain and the median for each order must strictly exceed the threshold, and at least two-thirds of pairs must exceed it. The retained graph is independently validated again before completion.

The report records source-integrity errors, query differences, resource limits, and outcomes such as no_measured_improvement. Only after the final graph passes can eligible candidates with strategy signatures produce accepted, no_gain, or unsafe evidence. no_change, graph_growth_budget_exceeded, comparison_query_failure, and unsigned candidates do not produce strategy observations. Rejecting a candidate retains the current graph.

Acceptance applies several gates before queries run. Invalid arguments or graph structure are rejected; an unchanged graph returns no_change; total nodes plus edges above initial size multiplied by max_graph_growth returns graph_growth_budget_exceeded; source-row or FK mismatch returns source_fidelity_failed. Only candidates passing the gates reach full query evaluation. source_fidelity_failed is the fixed protocol code for a source-integrity failure.

Correctness counts passing queries rather than scoring language-answer quality. A slower candidate can become the incumbent if it passes more queries; this intermediate acceptance does not satisfy publication requirements. Cost is compared against min_speedup only when both graphs pass every query. Final validation reruns the graph actually retained rather than treating an earlier candidate or a model finish request as the result.

Each query receives one warmup followed by benchmark_repeats timing samples. A sample adaptively repeats execution toward min_query_seconds, subject to iteration and overall budgets. Its elapsed time is divided by its execution count; the median per-query value contributes to L. Each round separately measures index preparation as P, while SQL-reference time is excluded from graph cost. Previews show at most 20 rows, but comparison uses complete results, preserving duplicates and validating ordering and ties where specified. These are in-process GraphExecutor measurements, not production NebulaGraph end-to-end latency.

Per-query model feedback includes correctness, row counts, errors, and timings, but excludes concrete answers in expected_preview and actual_preview. Those previews remain in the private run report. The model can identify error types and slow queries while executors remain responsible for answers and final acceptance.

history[].comparison retains each pair’s order, costs, gains, threshold wins, and acceptance criteria. Per-query timing_iterations, measured_query_runs, and warmup_runs describe actual executions; benchmark_repeats is not the total query-call count. materialized_path_hits counts shortcut use during timed executions, not added edges.

#How domain experience changes selection

Domain profiles support finance, academic, commerce, healthcare, social, and general. Automatic detection uses the domain description plus table/column names. Description matches have twice the schema weight; the leading score must reach 3; when the runner-up also reaches 3, a lead below 1.75× falls back to general. Explicit categories have confidence 1.

DomainExperienceStore retains users/<user>/construction-experience/strategies.sqlite3, with <user> identifying the default workspace owner. Experience stores strategy types, structural counts, query mixes, correctness, and cost gains; transferable experience excludes raw rows, SQL, prompts, table and column names, and complete tool arguments. A logical-source digest derived from structure and logical rows deduplicates experience; source_sha256 separately records the original file-byte digest.

StepComputation or rule
Match contextCompare log-transformed structural counts, query-family Jaccard, and normalized query mix. Filter incompatible contexts, require similarity ≥0.35, and return up to eight by default.
Outcome rewardaccepted: 0.25 + 0.75 × gain; no_gain: −0.25 + 0.75 × min(gain, 0); unsafe: −1. Clip gain to [−1, 1].
Policy scoreSimilarity-weight rewards, smooth the denominator by +2, multiply by mean similarity, and clip to [−1, 1].
Evidence confidencesupported requires ≥3 independent sources, acceptance ≥2/3, no unsafe outcome, and positive mean gain. Otherwise unsafe outcomes, no_gain ≥ accepted, or nonpositive scores yield cautioned; remaining patterns are candidate.
Rank candidatesCombine domain/structure priors with similarity-weighted evidence. The evidence weight is 1 for supported and 0.35 otherwise; unsafe evidence adds a 0.75 penalty.
text
policy_score = clip(
  (sum(reward_i * similarity_i) / (sum(similarity_i) + 2))
  * mean(similarity_i), -1, 1
)

priority = domain_and_structure_prior
         + evidence_weight * similarity * policy_score
         - safety_penalty

Keep only the latest evidence per source, domain, strategy, and structural pattern; rerunning the same source does not increase independent-source counts. Negative evidence lowers priority without permanently disabling a strategy. Similar structures still require validation on new data. Retain at most 2,048 source-pattern records and 256 patterns. Write receipts include the latest 4,096 plus older receipts still referenced by evidence.

Experience is retrieved after workload freezing and baseline evaluation, so context includes the actual query mix rather than only a commerce label. Candidate feedback returns to the model during the current run. The persistent store is updated only after the final graph passes and serves later tasks. Repeating an input replaces that source-pattern’s latest observation instead of counting retries as additional sources.

Signatures use interpretable structural features. A batch mixing different optimize_graph operations is not credited to one strategy. A candidate without an extractable signature can still be evaluated, but adds no transferable evidence. Unchanged candidates and graph-growth rejections also create no observation. accepted and no_gain require counts showing every query passed; otherwise the observation becomes unsafe and its gain is reset to zero.

Experience is optional guidance. experience_enabled=false disables both reads and writes; an unconfigured store reports no_store, retrieval failure reports unavailable, and write failure reports write_failed. A temporarily unavailable store does not invalidate a graph that passes full validation. These states differ from an available but empty cold-start store.

Final proposal ordering also uses measured hotspots: search_priority = measured_query_cost_share + 0.1 × priority. The first term is the share of current query time covered by the proposal; the second contributes domain and experience priority. transform_batch currently has no standalone strategy signature: it can be evaluated and recorded in history, but its combined gain is not assigned to one strategy in the experience store.

#Start construction and inspect results

In the application, upload a personal dataset and start construction from its dataset panel. The main agent’s graph_construction skill checks status and calls load_constructed_graph when ready. Initial builds use the Backend dataset API; there is no equivalent public MCP build tool.

bash
curl -X POST http://localhost:21108/api/personal-datasets \

# Replace DATASET_ID with dataset.dataset_id from the upload response.
curl -X POST http://localhost:21108/api/personal-datasets/DATASET_ID/construction/build \
  -H "Content-Type: application/json" \
  -d '{"options":{"mode":"rules","query_count":15}}'

When diagnosing your input, verify row-node and FK-edge counts, inspect workload_coverage for business scope, check candidate reasons in history, then compare experience retrieval and write states. If every query passes but accepted_candidates=0, inspect no_measured_improvement and cost rather than declaring the build failed. The configuration and query snippets on this page illustrate the mechanisms.

#Construction lifecycle and recovery

State or responseMeaning and next step
source_ready / not_builtThe source is retained but has no constructed graph to load. Review samples and options, then build.
buildingRead phase, message, and progress from status. Closing the page or ending the HTTP request does not automatically cancel the server task.
readyThe saved graph is available. A repeated build reuses it rather than rebuilding with new options. Use a new dataset to compare another build configuration.
failedInspect failure_code, messages, and the report, correct the input or options, then retry. A failure is not a usable graph.
construction_running_unconfirmedThe HTTP wait timed out and the server may still be running. Check status before starting another task.
construction_build_conflict / 429The dataset is building with different options, or construction capacity is full. Wait for the active task before submitting again.

Progress reports actual phases and available metrics; it does not publish an intermediate graph. Edits are rejected while construction or recovery checks are in progress. After success, read the current revision before editing. Saving a construction result and loading it into a session are separate actions; see the dataset and workspace guides.

#Preview, commit, and revision conflicts

Constructed graphs use construction_graph.v1. Read the latest graph and revision before editing. Preview validates candidate operations; commit persists them. Include base_revision to prevent overwriting a newer revision with a stale graph.

MethodPersonal dataset pathPurpose
GET/api/personal-datasets/{id}/constructionCapabilities and status
GET/api/personal-datasets/{id}/construction/source-previewOriginal samples and field descriptions, available before construction
GET/api/personal-datasets/{id}/construction/schemaSaved-graph type/property statistics and source-table structure; no original sample values
POST/api/personal-datasets/{id}/construction/buildStart initial construction; reuse the saved graph when already ready
GET/api/personal-datasets/{id}/construction/graphRead graph and revision
POST/api/personal-datasets/{id}/construction/previewValidate edits with { base_revision, operations }
POST/api/personal-datasets/{id}/construction/commitSame request shape; persist a new revision

Automatic construction and saved-graph editing are separate phases. The subagent explores isolated candidates and selects results using source integrity and the frozen workload. Later preview/commit operations implement user-requested revisioned edits. Read actual IDs and base_revision, then commit exactly the operations successfully previewed. After revision_conflict, reread and preview instead of incrementing the revision and replaying the request.

A successful commit increments the dataset graph revision, but the analysis session must still call load_constructed_graph to use it. Check size and state with get_graph_info before analysis. The construction report describes that build; it is not evidence that later manual edits still pass the old workload.

These views serve different purposes: source-preview inspects uploaded content, schema describes saved graph types and property statistics, and POST preview validates versioned edits. Both personal and project datasets expose the construction routes. Project routes use /api/projects/{project_id}/datasets/{id} and enforce dataset ownership.

#Budgets and run reports

SettingDefaultMeaning
max_source_bytes32 MiBCan be raised to 64 MiB in the bounded profile
max_source_rows200,000Maximum rows; fails instead of silently truncating
source_timeout_seconds10SQL parsing seconds; maximum 60
text_max_characters1,000,000Full-text character cap for entities; range 1–4,000,000
text_chunk_tokens / text_overlap_tokens1,000 / 150Approximate semantic chunk size and overlap; overlap must be smaller
text_max_chunks / text_max_calls64 / 96Complete chunk count and total model-call cap including retries
GRAPH_CONSTRUCTION_QUERY_COUNT15Frozen query count
GRAPH_CONSTRUCTION_MAX_ITERATIONS12Candidate improvement iterations
GRAPH_CONSTRUCTION_MAX_SECONDS900Construction time budget in seconds
GRAPH_CONSTRUCTION_CONCURRENCY1Per Graph Engine process; at most one simultaneous build per user

Relational construction also defaults to max_tool_calls=24, benchmark_repeats=5, comparison_rounds=3, min_query_seconds=0.003, and llm_timeout_seconds=120. In the bounded profile, a graph query permits at most 10 seconds and 200,000 intermediate rows. Semantic TXT allows up to three attempts per chunk and shares max_seconds with the overall task; character, chunk, call, or graph-size overflow fails the build. Neither path treats a graph produced from truncated input as a complete success.

The source defaults above belong to resource_profile="bounded". API/CLI callers may explicitly select full_scale, raising source limits to 8 GiB, 10,000,000 rows, and 7,200 seconds for SQL import, with larger graph and internal-query budgets. This does not promise automatic streaming: CSV and text still load into memory; only SQLite images use read-only file access. HTTP construction still limits max_seconds to 1–3,600 seconds, and the overall deadline takes precedence over longer internal budgets; the CLI handles full_scale separately. Deployment resources and entry-point limits may impose lower practical bounds.

Report fieldContents
config / llm_used / modelExecution configuration, whether a model was used, and model identity.
source_schema / domain_profileSource structure, domain category, and confidence.
queries / workload_sha256 / workload_coverageFrozen questions, DSL plans, workload checksum, and coverage.
baseline / finalPer-query correctness, errors, answer previews, and timings.
historyCandidate arguments, strategy, acceptance, failure reasons, and incumbent/candidate costs.
experienceMatched context, retrieved evidence, candidate priorities, write results, and updated policy scores.
initial_size / final_size / performance / phase_secondsGraph size, cost gains, and phase timings.

evolution-report.json uses tabular_evolution.v1 to describe a build. The frozen queries, baseline, candidate history, and strategy experience listed above apply to the relational path. Semantic TXT reports construction_kind=text_entities and semantic_extraction for chunked extraction rather than running the same SQL-reference evaluation. Graph Engine saves revisions and reports, with experience recorded according to the path. Users separately choose when to load the saved result into a session.

Construction optionDefault / rangePurpose
query_count15 / 1–100Frozen-plan count; supplied queries must match.
max_iterations12 / 1–30LLM evolution responses; limits attempted candidates in rules mode.
max_tool_calls24 / 1–100Model tool-call budget shared across generation and evolution.
workload_attempts3 / 1–5Model-response attempts to freeze a valid workload.
benchmark_repeats5 / 1–15Timing samples per graph query; each sample may execute the query repeatedly.
sample_rows3 / 1–10Samples per table sent to the construction model; does not change the source preview’s fixed five-row cap.
expected_query_runs100 / 1–100,000Expected runs for amortizing preparation cost.
llm_timeout_seconds120 / 1–600Per-request timeout, also bounded by remaining total time.
max_seconds900 / 1–3,600Overall task deadline for the HTTP build endpoint; deployment settings can change the default.
min_speedup0.05 / 0–0.9Required cost gain when all queries are correct.
max_graph_growth3.0 / 1–10Maximum multiple of initial nodes plus edges.
experience_top_k8 / 1–32Maximum retrieved strategy records.
experience_enabledtrue / booleanEnable cross-build experience reads and writes.
comparison_rounds3 / 2–9Paired comparison rounds with alternating current/candidate order.
min_query_seconds0.003 / 0–0.1Target seconds per timing sample, still bounded by iteration and deadline limits.

Explicit options override corresponding GRAPH_CONSTRUCTION_<UPPERCASE_FIELD> variables, followed by defaults. Counts require actual integers; booleans do not stand in for 1. Timeouts and ratios must be finite and within range. mode is llm or rules; domain_category accepts auto and six domains. Source byte, row, and SQL-import limits are separate parsing options; not every arbitrary setting has an environment-variable equivalent.

The overall deadline propagates through context-local state. Nested parsing, transformations, and queries use the earlier applicable deadline; long scans check on entry, every 1,024 items, and on exit. A model request uses the smaller of llm_timeout_seconds and remaining overall time. SQL import and queries obey both their own limits and remaining total time. Service mode adds process supervision for native code or network calls that cannot return promptly at Python checkpoints.

OutputContents and use
graph.jsonValidated directed, nodes, and edges; graph JSON excludes the report, and the CLI adds no version field.
evolution-report.jsontabular_evolution.v1 with phases, frozen plans, baseline, history, final validation, options, and experience status. Atomically updated by phase for diagnosis.
strategies.sqlite3Structured strategy experience under the default workspace owner. Its location is derived from a scope-checked source directory, not an arbitrary model-supplied path.
Worker receiptContains build_id, the graph.json filename and file SHA-256, or failure_code. Supervision uses it to identify this run’s output rather than an old file.

Report hashes identify different objects: source_sha256 hashes input bytes, workload_sha256 hashes normalized plans, and final_graph_sha256 hashes normalized graph content. Deduplication uses a separate logical-source digest, while the worker receipt’s sha256 hashes actual graph.json bytes. They are not interchangeable; changed file bytes do not necessarily indicate a new logical source.

Failure code or reasonDiagnosis
workload_generation_failedNo valid workload within the attempt limit; inspect workload_validation_errors.
tool_call_budget_exhaustedCombined tool-call budget exceeded; inspect tool_calls and its trace.
no_evolution_candidate_evaluatedNo LLM-mode candidate passed prechecks and reached evaluation.
query_accuracy_target_not_metThe final graph failed frozen plans; inspect final.results.
construction_time_budget_exhaustedThe overall deadline expired, possibly detected inside a nested step; reduce data/workload size or adjust supported budgets.

The workspace shows construction status, progress, and saved results but currently has no dedicated download button for the complete evolution-report.json. CLI reports reside in the --output directory; a deployed Graph Engine stores them under its data root at users/<user_id>/datasets/<dataset_id>/construction/evolution-report.json. An authorized operator can read the relevant file for diagnosis; the UI progress summary is not the complete report.

iGraphDocumentation