Bundle, deposit & query:
results crates into a lakehouse

Crate-out

Finished results become a self-describing bundle

Use case 04 pulled crates in to assemble inputs. This is the symmetric step: the integrative model, its refined components, the maps, and the validation reports get packaged back out as one RO-Crate. The crate carries a LAMBDA-BER Dataset describing everything inside — so the bundle is not a folder of mystery files, it is a typed, checksummed, schema-validatable record.

chd1-ncp-integrative.crate/
├── ro-crate-metadata.json      # RO-Crate descriptor (the manifest)
├── dataset.yaml                # LAMBDA-BER Dataset — validates against the schema
├── models/
│   └── integrative_model.cif    # IMP output, the deliverable
├── maps/
│   └── complex_sharpened.map    # cryo-EM, 3.2 Å
├── components/
│   └── ncp_refined.cif          # phenix.refine output (X-ray)
├── saxs/
│   └── chd1.dat                 # I(q), P(r)
└── reports/
    ├── molprobity.json          # validation
    └── mtriage_fsc.json         # map resolution

Two destinations

The same crate goes two places

A finished crate serves two very different audiences. Deposition is about publishing a single result for the world; the lakehouse is about accumulating many results for your own future questions. One bundle, both jobs.

Architecture

Strata: metadata in the warehouse, bytes in the lake

The lakehouse split is the whole point. Heavy data — maps, movies, MTZs — never moves into a warehouse; it stays in object storage at the facility, referenced by checksummed pointers. Only the LAMBDA-BER metadata is normalized into query tables. You get warehouse-grade SQL over lake-scale data.

The query payoff

Every past study, one SQL away

Because each crate lands the same LAMBDA-BER classes, the lakehouse holds a uniform table across facilities and years. Questions that used to mean emailing three labs become a single query over a flattened analytics view.

SELECT s.sample_code, em.resolution_ang, sx.rg_ang
FROM   experiment_run em
JOIN   experiment_sample_assoc a  USING (experiment_id)
JOIN   sample s                    ON s.id = a.sample_id
JOIN   experiment_run sx          ON sx.sample_id = s.id
WHERE  em.technique = 'cryo_em'
  AND  em.resolution_ang <= 3.0
  AND  sx.technique = 'saxs';

-- walk outputs back to the beamtime that produced them
SELECT w.workflow_code, w.software_name,
       d.file_name, d.checksum
FROM   workflow_run w
JOIN   workflow_output_assoc o USING (workflow_id)
JOIN   data_file d             ON d.id = o.data_file_id
WHERE  w.software_name = 'IMP';

The bytes referenced by data_file.checksum are still out in object storage — the query returns the pointer, and the agent streams the file only if the next step needs it. Same federation principle as use case 04, now running in reverse.

What lands in the lakehouse

The crate's metadata, normalized

Depositing is publishing one crate; the lakehouse ingest shreds that crate's dataset.yaml into the relational tables it already maps to.

The agent at work

Package, deposit, ingest

One pass: the agent validates the crate against the schema, files the public depositions, and ingests the metadata into the lakehouse — leaving the heavy data where it sits.

Payoff

What bundling + lakehousing buys