Artifacts

Introduction

An artifact is a versioned, named bundle of files attached to a project — a trained model, a dataset, a set of evaluation outputs. Logging files as artifacts (rather than leaving them on disk) gives you:

• Versioning — each time you log under the same name, Trackio records a new version (v0, v1, …). Identical content is de-duplicated, so re-logging unchanged files reuses the existing version.
• Aliases — moving pointers such as latest (assigned automatically) or your own (prod, best) that always resolve to a single version.
• Lineage — artifacts are linked to the runs that produced and consumed them, so you can trace which run created a model and which runs used it.

Artifacts work fully offline against local storage, and sync to a Hugging Face Space (or self-hosted server) when one is configured, exactly like metrics.

Logging an artifact

The quickest way is to log a file or directory directly. This creates a new artifact, adds the path, and records it as an output of the current run:

import trackio

trackio.init(project="my_project")

# ... training ...

trackio.log_artifact("checkpoints/model.safetensors", type="model")

When you pass a path, the artifact name defaults to run-<run_id>-<basename> and type defaults to "unspecified". Pass name and type to set them explicitly:

trackio.log_artifact("checkpoints/", name="my-model", type="model")

Artifact types

Artifact type is a free-form category string. Trackio does not enforce a fixed list of types, but common conventions are:

Use stable type names within a project so you can filter artifacts consistently and use trackio.use_artifact(..., type="...") to assert that a run is consuming the expected kind of artifact.

Building an artifact explicitly

For finer control — multiple files, custom logical paths, a description, or metadata — construct an Artifact, add files to it, then log it:

artifact = trackio.Artifact(
    name="my-model",
    type="model",
    description="Fine-tuned on the v2 dataset",
    metadata={"base_model": "bert-base", "epochs": 3},
)
artifact.add_file("checkpoints/model.safetensors")
artifact.add_file("checkpoints/config.json")
artifact.add_dir("tokenizer/")

logged = trackio.log_artifact(artifact)

After logging, the artifact is frozen and its version, aliases, size, and manifest are populated:

print(logged.version)   # "v0"
print(logged.aliases)   # ("latest",)

Versions and aliases

Each log under an existing name creates the next integer version and moves the latest alias onto it. Assign your own aliases with aliases=:

trackio.log_artifact(artifact, aliases=["prod", "best"])

Aliases are moving pointers: logging a new version with the same alias re-points it. Version specifiers (v0, v1, …) are reserved and cannot be used as aliases.

If you log content identical to an existing version, Trackio reuses that version instead of creating a new one. Any explicit aliases you pass still rotate onto it, but latest is left where it is — re-logging identical (or older) content never moves it backward.

Using an artifact

use_artifact() fetches an artifact and records it as an input to the current run, which is what builds lineage between a consuming run and the artifact it used:

trackio.init(project="my_project")

artifact = trackio.use_artifact("my-model")          # resolves to :latest
artifact = trackio.use_artifact("my-model:v2")       # pin a version
artifact = trackio.use_artifact("my-model:prod")     # resolve an alias

Pass type to assert the artifact’s type, raising if it doesn’t match:

artifact = trackio.use_artifact("my-model", type="model")

Downloading files

use_artifact returns an Artifact whose files you materialize with download():

artifact = trackio.use_artifact("my-model:latest")
path = artifact.download()
# files are now under ./.trackio/artifact-downloads/my_project/my-model_v2/

By default, files are written to ./.trackio/artifact-downloads/<project>/<name>_v<version>/, keyed by project so same-named artifacts from different projects never collide; pass root to choose another directory. download() is idempotent — files already present are skipped — and when the run is backed by a Space, any file missing locally is fetched from the remote.

Referencing external data

Some data is too large to copy, or already lives in durable storage you don’t want duplicated. add_reference records such data by URI in the manifest without staging any bytes into Trackio’s storage, so you still get versioning, de-duplication, aliases, and lineage over data that never moves:

artifact = trackio.Artifact(name="training-set", type="dataset")
artifact.add_reference("file:///mnt/nvme/datasets/corpus.parquet")
artifact.add_reference("file:///mnt/nvme/datasets/shards/")            # a directory — expands to one entry per file
artifact.add_reference("https://example.com/data/eval.json")
artifact.add_reference("hf://datasets/username/my-dataset/train.parquet")
trackio.log_artifact(artifact)

Built-in schemes are file://, http(s)://, and hf:// — all served by Trackio’s existing dependencies. Any other scheme (s3://, gs://, Azure blob URLs, …) is supported by registering a custom handler; see Referencing other stores below for complete S3, GCS, and Azure examples. With checksum=True (the default), Trackio probes the source to fill in size and a checksum:

• file:// is stream-hashed to a sha256; its size is read from disk.
• hf:// uses the LFS sha256 (or the git blob id); http(s):// uses the server ETag.

Every checksum is stored in a single digest field — the sha256 for local/LFS references, or the provider’s ETag for HTTP or object-store references.

A URI that denotes a directory (file://) or a repo path (hf://) is expanded into one reference entry per object, each named by its path relative to the prefix (nested under name= when given). max_objects (default 10,000) caps the expansion and raises if exceeded.

When no checksum can be obtained — for example a server with no ETag — the reference is still recorded, using the URI itself as the digest, and a warning is emitted. A local file:// path that doesn’t exist is the exception: it raises rather than being recorded, since a missing local file is almost always a mistake — use add_file to stage a local file. Pass checksum=False to skip the per-object checksum — a single object then records just its URI — and name= to set the logical path.

A reference’s version identity comes from its URI, size, and digest, so re-logging identical references de-duplicates to one version, while a changed URI, size, or checksum creates a new one — exactly like file content. When the URI is used as the digest, identity therefore tracks the URI and not the content.

URIs are recorded verbatim, and manifests can be synced to a Hugging Face Dataset. A URI that embeds a credential — a presigned https:// URL or an Azure SAS token — would expose that credential to anyone who can read the Dataset, so add_reference warns when a URI’s query string looks signed; prefer the object’s canonical unsigned URI (e.g. s3://bucket/key).

Reference bytes are never uploaded to a Space or bucket, but they can be fetched on demand. download() downloads each referenced object into the directory (via httpx, huggingface_hub, or your registered handler), which can transfer a large amount of data. Before fetching, each object’s source is re-probed and its current checksum compared against the recorded one — a mismatch warns that the source may have changed but the download proceeds. Each object is written atomically, and one already present on disk is skipped without re-checking the source, so repeated calls are cheap and idempotent:

artifact = trackio.use_artifact("training-set:latest")
path = artifact.download()   # downloads staged files AND referenced objects

To work with the pointers without downloading, read them from the artifact instead:

print(artifact.references)                  # [{'path': ..., 'ref': 's3://...', 'size': ..., 'digest': ...}, ...]
print(artifact.get_entry_uri("eval.json"))  # "s3://my-bucket/eval.json"

An unfetchable reference (a missing local file, an unreachable URL, or a scheme with no registered handler) raises when download() reaches it.

Referencing other stores (S3, GCS, Azure)

Trackio deliberately ships no cloud-vendor SDKs. To reference objects in a store it doesn’t handle natively, subclass trackio.ReferenceHandler and register an instance with trackio.register_reference_handler. A handler implements four methods:

• matches(scheme, uri) — whether this handler owns the URI. Registered handlers are consulted before the built-ins, so a handler may also claim specific https:// hosts (as the Azure example below does) ahead of the generic HTTP handler.
• resolve(uri, checksum, max_objects) — probe the source and return a list of trackio.ResolvedReference entries: one entry with relkey=None for a single object, or one entry per object (with relkey set to the object’s path relative to the prefix) when the URI denotes a prefix. Set size and digest when the source can be probed (the provider’s ETag works fine as a digest — Trackio treats reference digests as opaque tokens); return them as None to record the reference un-checksummed, in which case Trackio falls back to the URI as the digest and warns.
• fetch(uri, dest) — download the single object at uri to the local path dest (called by Artifact.download()).
• hint() — a one-line actionable message shown in the warning when resolve produced no checksum.

Keys discovered during prefix expansion should be percent-encoded when synthesizing each entry’s uri (and decoded in fetch), since manifest URIs may not contain spaces.

S3 ( s3:// , via boto3 )

from urllib.parse import quote, unquote, urlsplit

import boto3

import trackio


class S3Handler(trackio.ReferenceHandler):
    def matches(self, scheme, uri):
        return scheme == "s3"

    def resolve(self, uri, checksum, max_objects):
        parts = urlsplit(uri)
        bucket, key = parts.netloc, unquote(parts.path.lstrip("/"))
        client = boto3.client("s3")
        if key and not key.endswith("/"):
            head = client.head_object(Bucket=bucket, Key=key)
            return [
                trackio.ResolvedReference(
                    relkey=None,
                    uri=uri,
                    size=int(head["ContentLength"]),
                    digest=head["ETag"].strip('"') if checksum else None,
                )
            ]
        base = key[: key.rfind("/") + 1]
        entries = []
        pages = client.get_paginator("list_objects_v2").paginate(
            Bucket=bucket, Prefix=key
        )
        for page in pages:
            for obj in page.get("Contents", []):
                if obj["Key"].endswith("/"):
                    continue
                if len(entries) >= max_objects:
                    raise ValueError(
                        f"{uri} expands to more than {max_objects} objects; "
                        "pass max_objects= to raise the limit."
                    )
                entries.append(
                    trackio.ResolvedReference(
                        relkey=obj["Key"][len(base):],
                        uri=f"s3://{bucket}/{quote(obj['Key'], safe='/')}",
                        size=int(obj["Size"]),
                        digest=obj["ETag"].strip('"') if checksum else None,
                    )
                )
        return entries

    def fetch(self, uri, dest):
        parts = urlsplit(uri)
        with open(dest, "wb") as f:
            boto3.client("s3").download_fileobj(
                parts.netloc, unquote(parts.path.lstrip("/")), f
            )

    def hint(self):
        return "Configure AWS credentials so the object can be probed for an ETag."


trackio.register_reference_handler(S3Handler())

artifact = trackio.Artifact(name="training-set", type="dataset")
artifact.add_reference("s3://my-bucket/shards/")   # expands to one entry per object
trackio.log_artifact(artifact)

Google Cloud Storage ( gs:// , via google-cloud-storage )

from urllib.parse import quote, unquote, urlsplit

from google.cloud import storage

import trackio


class GcsHandler(trackio.ReferenceHandler):
    def matches(self, scheme, uri):
        return scheme == "gs"

    def resolve(self, uri, checksum, max_objects):
        parts = urlsplit(uri)
        bucket, key = parts.netloc, unquote(parts.path.lstrip("/"))
        client = storage.Client()
        if key and not key.endswith("/"):
            blob = client.bucket(bucket).get_blob(key)
            if blob is None:
                raise ValueError(f"No such GCS object: {uri}")
            return [
                trackio.ResolvedReference(
                    relkey=None,
                    uri=uri,
                    size=blob.size,
                    digest=blob.etag.strip('"') if checksum else None,
                )
            ]
        base = key[: key.rfind("/") + 1]
        entries = []
        for blob in client.list_blobs(bucket, prefix=key):
            if blob.name.endswith("/"):
                continue
            if len(entries) >= max_objects:
                raise ValueError(
                    f"{uri} expands to more than {max_objects} objects; "
                    "pass max_objects= to raise the limit."
                )
            entries.append(
                trackio.ResolvedReference(
                    relkey=blob.name[len(base):],
                    uri=f"gs://{bucket}/{quote(blob.name, safe='/')}",
                    size=blob.size,
                    digest=blob.etag.strip('"') if checksum else None,
                )
            )
        return entries

    def fetch(self, uri, dest):
        parts = urlsplit(uri)
        key = unquote(parts.path.lstrip("/"))
        storage.Client().bucket(parts.netloc).blob(key).download_to_filename(
            str(dest)
        )

    def hint(self):
        return "Configure GCP credentials so the object can be probed for an ETag."


trackio.register_reference_handler(GcsHandler())

Azure Blob Storage (via azure-storage-blob + azure-identity )

Azure blobs are addressed by plain https:// URLs (https://<account>.blob.core.windows.net/<container>/<blob>), so this handler claims them by host suffix — registering it makes it win over Trackio’s generic HTTP handler:

from urllib.parse import quote, unquote, urlsplit

from azure.identity import DefaultAzureCredential
from azure.storage.blob import BlobServiceClient

import trackio


class AzureHandler(trackio.ReferenceHandler):
    def matches(self, scheme, uri):
        return scheme in ("http", "https") and urlsplit(
            uri
        ).netloc.lower().endswith(".blob.core.windows.net")

    def _split(self, uri):
        parts = urlsplit(uri)
        container, _, blob_name = parts.path.lstrip("/").partition("/")
        return f"{parts.scheme}://{parts.netloc}", container, unquote(blob_name)

    def _service(self, account_url):
        return BlobServiceClient(account_url, credential=DefaultAzureCredential())

    def resolve(self, uri, checksum, max_objects):
        account_url, container, blob_name = self._split(uri)
        service = self._service(account_url)
        if blob_name and not blob_name.endswith("/"):
            props = service.get_blob_client(container, blob_name).get_blob_properties()
            return [
                trackio.ResolvedReference(
                    relkey=None,
                    uri=uri,
                    size=props.size,
                    digest=props.etag.strip('"') if checksum else None,
                )
            ]
        base = blob_name[: blob_name.rfind("/") + 1]
        entries = []
        container_client = service.get_container_client(container)
        for blob in container_client.list_blobs(name_starts_with=blob_name):
            if blob.name.endswith("/"):
                continue
            if len(entries) >= max_objects:
                raise ValueError(
                    f"{uri} expands to more than {max_objects} objects; "
                    "pass max_objects= to raise the limit."
                )
            entries.append(
                trackio.ResolvedReference(
                    relkey=blob.name[len(base):],
                    uri=f"{account_url}/{container}/{quote(blob.name, safe='/')}",
                    size=blob.size,
                    digest=blob.etag.strip('"') if checksum else None,
                )
            )
        return entries

    def fetch(self, uri, dest):
        account_url, container, blob_name = self._split(uri)
        downloader = (
            self._service(account_url)
            .get_blob_client(container, blob_name)
            .download_blob()
        )
        with open(dest, "wb") as f:
            downloader.readinto(f)

    def hint(self):
        return "Configure Azure credentials so the blob can be probed for an ETag."


trackio.register_reference_handler(AzureHandler())

These handlers raise when the source can’t be probed; if you’d rather record an un-checksummed reference in that case (for example, when cataloging data behind auth you can’t reach at log time), catch the provider error inside resolve and return [trackio.ResolvedReference(relkey=None, uri=uri)] — Trackio will fall back to the URI as the digest and warn, using your hint().

Register handlers in every process that adds or downloads such references: Artifact.download() needs the handler to fetch the bytes, and a URI whose scheme has no registered handler raises.

Browsing artifacts in the dashboard

The dashboard’s Artifacts tab (shown once a project has artifacts) lists artifacts grouped by type, with each version’s aliases, size, and file count. Selecting a version shows its file manifest — click a file to download it — plus its metadata and lineage: the run that produced it and the runs that used it, each linking to the run’s detail page.

Run detail pages show the same lineage from the other direction, with Output artifacts and Input artifacts sections linking back to the Artifacts tab.

Remote storage

When your run targets a Hugging Face Space or self-hosted server (see Track), artifact files are content-addressed and uploaded once: Trackio skips blobs the server already has, so re-logging shared files is cheap. Artifact metadata and blobs are persisted alongside your other run data to the configured HF Dataset or bucket.