asdf-pydantic

PyPI - Version PyPI - Python Version Documentation Status

Tip

For v1, see https://asdf-pydantic.readthedocs.io/en/v1/

Type-validated scientific data serialization with ASDF and Pydantic models

import asdf
from asdf_pydantic import AsdfPydanticModel

class Rectangle(AsdfPydanticModel):
    _tag = "asdf://asdf-pydantic/examples/tags/rectangle-1.0.0"

    width: float
    height: float

# After creating extension and install ...

af = asdf.AsdfFile()
af["rect"] = Rectangle(width=1, height=1)
ASDF File 
print(af.dumps())

#ASDF 1.0.0
#ASDF_STANDARD 1.5.0
%YAML 1.1
%TAG ! tag:stsci.edu:asdf/
--- !core/asdf-1.1.0
asdf_library: !core/software-1.0.0 {
    author: The ASDF Developers,
    homepage: 'http://github.com/asdf-format/asdf',
    name: asdf,
    version: 2.14.3}
history:
  extensions:
  - !core/extension_metadata-1.0.0
    extension_class: asdf.extension.BuiltinExtension
    software: !core/software-1.0.0 {
        name: asdf,
        version: 2.14.3}
  - !core/extension_metadata-1.0.0 {
    extension_class: mypackage.shapes.ShapesExtension,
    extension_uri: 'asdf://asdf-pydantic/shapes/extensions/shapes-1.0.0'}
rect: !<asdf://asdf-pydantic/shapes/tags/rectangle-1.0.0> {
    height: 1.0,
    width: 1.0}
...
ASDF Schema 
print(af["rect"].model_asdf_schema())

%YAML 1.1
---
$schema: http://stsci.edu/schemas/asdf/asdf-schema-1.0.0
id: asdf://asdf-pydantic/examples/tags/rectangle-1.0.0/schema
title: Rectangle
type: object
properties:
  width:
    title: Width
    type: number
  height:
    title: Height
    type: number
required:
- width
- height
JSON Schema 
print(af["rect"].model_json_schema())

{
    "properties": {
        "width": {
            "title": "Width",
            "type": "number"
        },
        "height": {
            "title": "Height",
            "type": "number"
        }
    },
    "required": [
        "width",
        "height"
    ],
    "title": "Rectangle",
    "type": "object"
}

Features

  • Create ASDF tag from your pydantic models with batteries (converters) included

  • Automatically generate ASDF schemas

  • Validate data models as you create them, and not only when reading and writing ASDF files

  • Preserve Python types when deserializing ASDF files

  • All the benefits of pydantic (e.g., JSON encoder, JSON schema, pydantic types).

Installation

pip install "asdf-pydantic>=2a"

Usage

Define your data model with AsdfPydanticModel. For pydantic fans, this has all the features of pydantic’s BaseModel.

# mypackage/shapes.py
from asdf_pydantic import AsdfPydanticModel

class Rectangle(AsdfPydanticModel):
    _tag = "asdf://asdf-pydantic/examples/tags/rectangle-1.0.0"

    width: Annotated[
        u.Quantity[u.m], AsdfTag("tag:stsci.edu:asdf/core/unit/quantity-1.*")
    ]
    height: Annotated[
        u.Quantity[u.m], AsdfTag("tag:stsci.edu:asdf/core/unit/quantity-1.*")
    ]

Then create an ASDF extension with the help of the provided converter class AsdfPydanticConverter.

# mypackage/extensions.py
from asdf.extension import Extension
from asdf_pydantic.converter import AsdfPydanticConverter
from mypackage.shapes import Rectangle

converter = AsdfPydanticConverter()
converter.add_models(Rectangle)

class ShapesExtension(Extension):
    extension_uri = "asdf://asdf-pydantic/examples/extensions/shapes-1.0.0"
    converters = [converter]
    tags = [*converter.tags]

After your extension is installed with either the entrypoint method or temporarily with asdf.get_config():

import asdf
from mypackage.extensions import ShapeExtension

asdf.get_config().add_extension(ShapesExtension())

af = asdf.AsdfFile()
af["rect"] = Rectangle(width=1, height=1)

# Write
af.write_to("shapes.asdf")

# Read back and validate
with asdf.open("shapes.asdf", "rb") as af:
    print(af["rect"])

Read Further