Internals

Repository layout

ailine/
  cli/             # Click entry point + terminal formatters
  config/          # .ailine.yml loaders + defaults + path constants
  fingerprint/     # environment fingerprint
  integrations/    # MLflow UI subprocess, MLflow run-context plugin, git URL helpers
  linkage/         # DVC discovery + linkage classification
  persistence/     # SQLite schema, migrations, repository facade
  run/             # CLI run-command capture, session orchestrator, cleanup
  snapshot/        # repo scan, manifest, content-addressed objects
  web/             # Flask app factory + route modules + templates

Tests sit under tests/ and follow test_<module>.py naming.

Lineage data model

One SQLite table (tree) carries every recorded run. The fields you care about:

• id — either a Git commit SHA (when the worktree was clean) or a snapshot content hash (when it was dirty).
• type — git or snapshot.
• parent — the parent commit when type = snapshot.
• status — in_progress, done, or failed. Set to in_progress before the child subprocess starts and finalized to done / failed based on its exit code.
• mlflow_run — populated mid-flight via the correlation tag (default), pre-link, or post-hoc lookup.
• snapshot_path / manifest_path / metadata_path / diff_path — on-disk artifacts produced by the snapshot pipeline.
• dvc_linkage_json / env_fingerprint_json / run_command_json — structured JSON blobs for DVC linkage, environment fingerprint, and the exact argv / cwd / timestamp.

Old rows where status is NULL are interpreted as done for backward-compatible display.

.ailine/ state directory

AIline's auto-generated artifacts live under .ailine/ at the repo root, so the project root stays clean.

User-controlled paths (mlruns/, .ailine.yml, .ailineignore) are never relocated. On the first invocation in an older checkout AIline transparently moves any legacy root-level artifacts (ailine_tree.db, ailine.log, ailine_config.txt) into .ailine/. Each path can be overridden with the matching AILINE_* environment variable (AILINE_DB_PATH, AILINE_LOG_PATH, AILINE_CONFIG_PATH, AILINE_STORAGE_DIR, AILINE_STATE_DIR).

MLflow plugin and correlation flow

AIline registers a RunContextProvider via Poetry's plugin metadata. MLflow auto-discovers any provider declared under the mlflow.run_context_provider entry-point group and applies its tags() to every run started in the same Python process.

[tool.poetry.plugins."mlflow.run_context_provider"]
ailine_correlation_id = "ailine.integrations.mlflow_plugin:AilineRunContextProvider"

The provider is a no-op when AILINE_CORRELATION_ID is unset. When it is set, it adds a single tag:

ailine.correlation_id = <uuid>

Sequence per ailine track invocation:

1. Generate uuid4().hex.
2. Inject AILINE_CORRELATION_ID into the child env before subprocess.run.
3. Spawn a daemon thread that calls MlflowClient.search_runs with filter_string='tags."ailine.correlation_id" = "<uuid>"' every track.mlflow.link_poll_seconds.
4. On first match: repository.set_mlflow_run(record_id, run_id) with a WHERE mlflow_run IS NULL OR mlflow_run = '' guard, then exit the thread.
5. On subprocess exit: signal the thread to stop, do one final retry, log a clear warning if no match was ever found.

The NULL/empty guard prevents stomping a value the user's script may have already supplied via MLFLOW_RUN_ID.

Reproducibility scope and non-goals

AIline is aimed at Python-centric ML projects. Per recorded run we persist enough to argue the next execution can be brought as close as is reasonable to the recorded one — not a hardware certification or a line-by-line audit of every third-party library.

What we maximize

• Pre-run worktree state (snapshot manifest + diff + per-file objects).
• Exact argv and cwd for the child process.
• Coarse environment: Python version, platform, lockfile fingerprint, configured package versions.
• DVC linkage where present (paths and hashes; you still own remotes and pulls).
• MLflow run association for cross-checking metrics and artifacts.

What we explicitly do not guarantee

• Bit-identical reruns of stochastic training (same seed never implies same weights across GPUs / drivers / cuDNN / worker counts).
• Files created only after the snapshot step (mid-run downloads, checkpoints written during training).
• Reads outside the worktree, or paths excluded by policy.
• Hardware equivalence or per-file verification of vendored / compiled dependencies beyond version metadata.

For stronger output parity, treat outputs as first-class artifacts (DVC outs, MLflow logged files) — not as something implied by the pre-run snapshot alone.

Releasing

Versions come from Git tags via poetry-dynamic-versioning. There is no manual version = ... bump in pyproject.toml; the tag is the version. The configured tag pattern accepts vX.Y.Z and PEP 440 pre-releases (vX.Y.Za1, vX.Y.Zb2, vX.Y.Zrc1).

One-time per developer machine:

poetry self add "poetry-dynamic-versioning[plugin]"

Cut a release:

git tag v0.5.1
git push origin v0.5.1

Pushing a v*.*.* tag triggers the GitHub Actions release workflow, which runs tests, calls poetry build, attaches dist/*.whl and dist/*.tar.gz to a GitHub Release, and publishes to TestPyPI.

Demo commands

AIline ships three commands behind explicit *-demo names for the original tutorial flow. They clone a sample repo into ./repo and pretend-train. They are not recommended for real projects — use ailine track instead.

ailine init-demo <git_repo_url>     # clone into ./repo
ailine run --script train.py        # demo: wraps in MLflow, records a snapshot
ailine reset-demo                   # remove ./repo, the DB, and mlruns/