Skip to main content

Filesystem layout

A data product in source control is usually one repository, or a clear directory inside a monorepo, that holds what you need to compile Spine and publish a version through the control plane. The tree below is a convention; names can differ as long as you point the CLI at the right paths with --source-dir, --out, and --lib.

Conventional tree​

project-root/
├── dataspine.json              # Data Product Manifest (build + product metadata)
├── spine-src/                  # Spine sources (optional other directory name)
│   ├── main.spine
│   └── …                       # more .spine files if you split by domain
├── lib/                        # optional: type-library JSON for --lib
├── test/                       # optional: golden tests (see Test framework)
│   └── some-scenario/
│       ├── main.spine
│       ├── ingests/
│       └── expected/
└── build/                      # optional: local compile output (often gitignored)
    └── artifact.json

What goes where​

  • dataspine.json — The Data Product Manifest: root namespace, library references, product metadata, and other fields the toolchain and control plane read from Git.

  • Spine sources — One or more *.spine files. Many teams use a single directory (for example spine-src/) and pass --source-dir to dataspine check and dataspine compile so every .spine file under that tree is picked up recursively. You may split types and flows across files; namespaces and imports must stay consistent.

  • Type libraries — JSON (or directories of JSON) for external types, passed with --lib or via manifest-driven build settings your pipeline supports. Placement is a team choice: beside the product or in a shared library repo.

  • Golden tests — Optional folders with ingests/ and expected/ beside a main.spine (or your team’s equivalent). Rules are in the Test framework and Prepare test data.

  • Build artifacts — dataspine compile --out writes a build artifact JSON for upload and deployment. A local build/ folder is often gitignored; CI may write the same file in a temp directory.

  • Generated clients — Per-data-product packages (npm, Maven, Rust crates) are usually built and published from CI into your registry, not kept as primary source next to .spine, unless you explicitly vendor them.

Monorepos and multiple products​

When several data products share one Git repository, each product typically has its own dataspine.json and Spine tree under a dedicated subdirectory, so CI can run check / compile with different --source-dir and --root-namespace per job.

See also​