Migrations

A migration expresses portable schema intent and owns a stable history identifier:

public sealed class InitialSocialGraph : NodalMigration
{
    public override string Id => "20260816_001_initial_social_graph";

    protected override void Up(MigrationBuilder migration) => migration
        .CreateNode<Person>()
        .CreateRelation<Knows, Person, Person>()
        .CreateIndex<Person, string>(person => person.Name);

    protected override void Down(MigrationBuilder migration) => migration
        .DropRelation<Knows>()
        .DropNode<Person>();
}

Planning is side-effect free. Plans contain deterministic checksums and provider-specific commands:

MigrationPlan preview = await context.Database.PlanMigrationsAsync(migrations);
MigrationPlan applied = await context.Database.MigrateAsync(migrations);

Neo4j commits one homogeneous schema-command batch transactionally, then records its migration state in a separate graph-write transaction. This separation is required by Neo4j: schema modifications and graph writes cannot share one transaction. Nodal writes Applying first and uses idempotent DDL plus Applied/Failed history to make an interrupted migration reviewable and retryable. TigerGraph compiles schema operations into a deterministic job and requires a verified ITigerGraphAdministrativeControlPlane.

TigerGraph durable job lifecycle

TigerGraph schema jobs are not a REST++ data transaction. Nodal therefore keeps two independent metadata records: __NodalMigration is authoritative applied history, while __NodalSchemaJob journals creation, execution, cleanup, and history reconciliation. Cleanup uses its own bounded cancellation token.

If a process restarts after schema success, the journal resumes cleanup and history persistence without running the schema job again. If cancellation makes the RUN outcome unknowable, automatic replay stops with TigerGraphMigrationRecoveryRequiredException:

TigerGraphSchemaJobJournalEntry? state =
    await provider.MigrationRecovery.InspectAsync(migrationId);

// Call exactly one after independently inspecting the live schema.
await provider.MigrationRecovery.ConfirmSchemaAppliedAsync(migrationId);
// await provider.MigrationRecovery.ConfirmSchemaNotAppliedAsync(migrationId);

The next ordinary migration run finishes the selected recovery path. Changing the checksum, direction, or deterministic job envelope during an unfinished recovery is rejected as journal drift.

Schema snapshots and reviewable diffs

NodalSchemaSnapshotFactory captures the registered model without connecting to a database. Snapshots have their own wire-format version, deterministic ordering and a stable SHA-256 hash. Provider introspectors can capture the live Neo4j or TigerGraph schema into the same contract while preserving provider name, version and storage types.

NodalSchemaSnapshot desired = NodalSchemaSnapshotFactory.FromModel(context.Model);
NodalSchemaSnapshot current = await provider.SchemaIntrospector.CaptureAsync();

NodalSchemaMigrationPlan plan = NodalSchemaMigrationMapper.Map(current, desired);
string review = NodalSchemaMigrationPlanSerializer.ToMarkdown(plan);
string automation = NodalSchemaMigrationPlanSerializer.Serialize(plan);

Property renames are never inferred. Supply an explicit rename hint such as node:people:name -> display_name; otherwise the diff remains an add/drop pair. Relation shape changes, changed schema-object definitions and compound indexes that cannot be represented safely by the portable M1 operations are placed in ManualReview. Unknown snapshot format versions fail with NodalSchemaSnapshotVersionException; a future format must provide an explicit upgrade path instead of silently reinterpreting persisted metadata.

Migration CLI

Nodal.Tool exposes the provider-neutral review stage as the nodal .NET tool. The commands read versioned snapshots and never connect to a graph database or apply schema changes:

nodal migrations snapshot \
  --input model.snapshot.json \
  --output nodal.snapshot.json

nodal migrations diff \
  --from deployed.snapshot.json \
  --to nodal.snapshot.json \
  --format json \
  --output migration-diff.json

nodal migrations plan \
  --from deployed.snapshot.json \
  --to nodal.snapshot.json \
  --output migration-plan.md

nodal migrations validate --snapshot nodal.snapshot.json

snapshot validates, normalizes, and hashes a snapshot generated by the model API. diff and plan produce deterministic text or JSON suitable for pull request artifacts. Unknown options are rejected without echoing their values, and file or JSON failures use stable non-zero exit codes. Provider composition is supplied only at the execution boundary, keeping read-only planning side-effect free and provider-neutral.

Use --format github with diff, plan, or list to emit escaped GitHub workflow annotations. This makes manual-review findings visible in a pull request without exposing option values or credentials.

Immutable migration bundles

A bundle freezes the migration ID, verified provider identity, Nodal version, required capabilities, ordered provider commands, transaction semantics, and destructive-operation flags behind a canonical SHA-256 checksum. Bundle input is an explicit manifest produced after provider compilation:

migration.manifest.json

{
  "migrationId": "20260825_001_people",
  "providerName": "Neo4j",
  "providerVersion": "5.26",
  "frameworkVersion": "0.1.0-alpha.1",
  "requirements": ["SchemaWrite"],
  "commands": [
    {
      "name": "create-index",
      "text": "CREATE INDEX nodal_people_name IF NOT EXISTS FOR (n:people) ON (n.name)",
      "transactional": true,
      "destructive": false,
      "kind": "schema",
      "direction": "up"
    }
  ]
}

nodal migrations bundle \
  --manifest migration.manifest.json \
  --output bundles/20260825_001_people.nodalbundle.json

nodal migrations list --directory bundles --format github

Requirements are deduplicated and sorted, while command order is preserved. Reading a bundle recomputes its checksum; modified content is rejected before execution. Command text resembling an assigned password, access token, client secret, authorization value, or bearer credential is rejected during bundle creation. Bundles intentionally contain no connection strings or environment secrets.

Bundles can also contain ordered down commands. The provider-neutral NodalMigrationBundleExecutor verifies the checksum, provider name and version, required capabilities, persisted migration history, and destructive approval before mutation. Repeated apply and rollback calls are idempotent. When a provider exposes IGraphMigrationLockProvider, history inspection and execution run under one exclusive lease. Dry-run mode performs the same safety checks without changing provider state; rollback always requires explicit destructive approval and fails clearly when no down commands exist.

Apply and rollback from a deployment host

apply and rollback require a trusted host assembly whose public, parameterless type implements INodalMigrationBundleExecutionHost. The host is the composition root: it loads credentials from its deployment secret store, constructs the selected provider and verified NodalMigrationBundleTarget, and delegates to NodalMigrationBundleExecutor. The CLI loads it through two environment variables so secrets never become process arguments:

export NODAL_MIGRATION_HOST_ASSEMBLY=/app/Contoso.Graph.Migrations.dll
export NODAL_MIGRATION_HOST_TYPE=Contoso.Graph.DeploymentMigrationHost

nodal migrations apply --bundle bundles/20260825_001_people.nodalbundle.json --dry-run true
nodal migrations apply --bundle bundles/20260825_001_people.nodalbundle.json --approve-destructive true
nodal migrations rollback --bundle bundles/20260825_001_people.nodalbundle.json --approve-destructive true

Only load an assembly produced and reviewed by the deployment pipeline. The CLI does not accept connection strings, passwords, or access tokens. A dynamically loaded host that implements IAsyncDisposable is disposed after execution.

The repository includes compile-checked Neo4jMigrationHost and TigerGraphMigrationHost examples under samples/Nodal.Samples.MigrationHost. Both read one JSON document from NODAL_MIGRATION_HOST_CONFIGURATION; use a GitHub Environment secret or the deployment platform's equivalent. For Neo4j, the shape is:

{
  "endpoint": "neo4j://graph.internal:7687",
  "providerVersion": "5.26",
  "capabilities": ["SchemaWrite"],
  "username": "deployment-user",
  "password": "loaded-from-secret-store",
  "database": "neo4j"
}

TigerGraph additionally uses graphName, gsqlFile, and gsqlPrefixArguments. These snippets document the contract only; never commit a populated configuration document.

The manual Apply Immutable Migration Bundle GitHub workflow downloads a reviewed artifact from an exact workflow run, installs an exact Nodal.Tool version, confines bundle and host paths to that artifact, and executes inside the selected staging or production GitHub Environment. Configure required reviewers on both environments; production approval is the final human gate before destructive execution. The optional NODAL_MIGRATION_HOST_CONFIGURATION environment secret is passed only to the trusted host process and is never printed by the workflow or CLI.

Every alpha publication also emits nodal-release-evidence.json and the canonical capability JSON-LD beside the verified packages. The evidence records the commit, exact package version, package set, migration bundle format, certified provider baselines, and SHA-256 hash of the capability graph.

Safe backfills and recovery

Backfills must be bounded. Use IMigrationBackfillCheckpointStore when a backfill can outlive one process or request. The executor persists a checkpoint only after the callback reports a successful batch, resumes from that token, and removes the checkpoint after completion.

Each MigrationBackfillContext exposes a deterministic IdempotencyKey. A provider callback should pass this key to its write transaction or durable deduplication record. If a process fails after the provider write but before the checkpoint save, the retry receives the same key and must treat the batch as already applied.

Recovery procedure

1. Stop the migration worker and inspect migration history and the backfill checkpoint.
2. Verify the provider and server version recorded by the deployment.
3. Resume with the same migration name, batch size, and callback contract.
4. For a type rewrite, validate source and target counts before cleanup.
5. Remove old properties, indexes, or constraints only after validation succeeds.
6. If recovery is not safe, revert the reversible schema migration and restore from the provider backup before retrying.

Neo4j checkpoints are stored as __NodalBackfillCheckpoint metadata nodes. TigerGraph uses ITigerGraphBackfillCheckpointTransport, because the supported administrative channel differs between self-managed and managed deployments. Non-transactional provider commands are surfaced during preflight with a warning and require the recorded history state plus the provider recovery procedure.

Neo4j schema boundaries

Neo4j is schema-optional. Node labels, relationship types and ordinary properties can exist without DDL. Nodal therefore treats CreateNode, CreateRelation and flexible property add/remove operations as model metadata and emits no schema command for them. Indexes and uniqueness constraints are physical schema objects with deterministic Nodal names.

The certified baseline is Neo4j 5.26 Community. Property-existence and property-type constraints are Enterprise Edition features, so the Community provider does not advertise or silently emulate them. Application-level validation is not presented as a database constraint. Enterprise deployments opt in explicitly:

var provider = new Neo4jProvider(new Neo4jOptions
{
    Endpoint = new Uri("neo4j://localhost:7687"),
    Username = "neo4j",
    Password = configuration["Neo4j:Password"]!,
    EnterpriseSchemaConstraintsEnabled = true,
});

The portable migration API can then declare node or relationship constraints:

migration
    .CreateNodePropertyExistenceConstraint<Person, string>(person => person.Email)
    .CreateNodePropertyTypeConstraint<Person, string>(person => person.Email)
    .CreateRelationPropertyExistenceConstraint<Knows, DateTime>(relation => relation.Since);

Without the explicit capability, preflight reports the operation as unsupported before any command reaches Neo4j. This preserves portable intent while preventing Community and Enterprise deployments from silently diverging.