06. Pipelines

Gyokhan Kochmarla edited this page Nov 16, 2025 · 1 revision

HoloGen uses a modular, protocol-based pipeline architecture to generate synthetic hologram datasets. The pipeline transforms object-domain patterns through holographic propagation, applies realistic noise, and persists the results to disk.

Pipeline Flow

The dataset generation pipeline consists of four main stages:

Object Generation: Create object-domain patterns using shape generators
Hologram Creation: Transform objects into holograms using holography strategies
Reconstruction: Recover object-domain fields from holograms
Persistence: Write samples to disk in NumPy and PNG formats

graph LR
 A[ObjectDomainProducer] --> B[ObjectToHologramConverter]
 B --> C[HologramDatasetGenerator]
 C --> D[DatasetWriter]
 
 A -.-> A1[Shape patterns]
 B -.-> B1[Hologram fields]
 C -.-> C1[Complete samples]
 D -.-> D1[.npz + .png files]
 
 style A fill:#e1f5ff
 style B fill:#fff4e1
 style C fill:#ffe1f5
 style D fill:#e1ffe1

Key Design Principles

Protocol-based: Components implement protocols (duck typing) for flexibility
Immutable configuration: Configuration objects are passed through the pipeline
Strategy pattern: Holography methods (inline/off-axis) are interchangeable strategies
Composable: Mix and match components to create custom pipelines
Type-safe: Explicit type hints for NumPy arrays and data structures

Component Interaction Diagram

The following diagram shows how components interact during sample generation:

sequenceDiagram
 participant User
 participant Generator as HologramDatasetGenerator
 participant Producer as ObjectDomainProducer
 participant ShapeGen as ShapeGenerator
 participant Converter as ObjectToHologramConverter
 participant Strategy as HolographyStrategy
 participant Noise as NoiseModel
 participant Writer as DatasetWriter
 
 User->>Generator: generate(count, config, rng)
 loop For each sample
 Generator->>Producer: generate(grid, rng)
 Producer->>ShapeGen: generate(grid, rng)
 ShapeGen-->>Producer: object pixels
 Producer-->>Generator: ObjectSample
 
 Generator->>Converter: create_hologram(sample, config, rng)
 Converter->>Strategy: create_hologram(field, config)
 Strategy-->>Converter: hologram field
 
 opt If noise enabled
 Converter->>Noise: apply(hologram, config, rng)
 Noise-->>Converter: noisy hologram
 end
 
 Converter-->>Generator: hologram
 
 Generator->>Converter: reconstruct(hologram, config)
 Converter->>Strategy: reconstruct(hologram, config)
 Strategy-->>Converter: reconstruction
 Converter-->>Generator: reconstruction
 
 Generator-->>User: yield HologramSample
 end
 
 User->>Writer: save(samples, output_dir)
 Writer-->>User: Files written to disk

Data Transformation Flow

This diagram illustrates how data is transformed through the pipeline:

graph TD
 A[Random Shape Selection] --> B[Binary Amplitude Image]
 B --> C{Complex Field?}
 
 C -->|Legacy| D[Real Amplitude Array]
 C -->|Complex| E[Complex Field Array]
 
 D --> F[Convert to Complex]
 E --> F
 F --> G[Angular Spectrum Propagation]
 
 G --> H{Holography Method}
 H -->|Inline| I[Propagated Field]
 H -->|Off-Axis| J[Propagated + Reference]
 
 I --> K{Noise?}
 J --> K
 
 K -->|Yes| L[Extract Intensity]
 K -->|No| M[Hologram Field]
 
 L --> N[Apply Noise Models]
 N --> O[Reconstruct Complex Field]
 O --> M
 
 M --> P[Backward Propagation]
 P --> Q{Output Format}
 
 Q -->|Legacy| R[Intensity Arrays]
 Q -->|Complex| S[Complex Field Arrays]
 
 R --> T[NPZ + PNG Files]
 S --> U[NPZ + Amplitude/Phase PNGs]
 
 style A fill:#e1f5ff
 style B fill:#e1f5ff
 style G fill:#fff4e1
 style N fill:#ffe1e1
 style P fill:#fff4e1
 style T fill:#e1ffe1
 style U fill:#e1ffe1

Components

1. ObjectDomainProducer

Purpose: Generate object-domain samples by randomly selecting from registered shape generators.

Protocol: None (concrete class)

Key Methods:

generate(grid, rng): Produce a legacy intensity-based object sample
generate_complex(grid, rng, phase_shift, mode): Produce a complex field object sample

Configuration: Initialized with a tuple of shape generators

Example:

from hologen import ObjectDomainProducer, CircleGenerator, RectangleGenerator
from hologen.types import GridSpec
import numpy as np
# Create producer with specific generators
producer = ObjectDomainProducer(
 shape_generators=(CircleGenerator(), RectangleGenerator())
)
# Generate object sample
grid = GridSpec(height=512, width=512, pixel_pitch=5e-6)
rng = np.random.default_rng(seed=42)
sample = producer.generate(grid, rng)
print(f"Generated {sample.name} with shape {sample.pixels.shape}")
# Output: Generated circle with shape (512, 512)

Complex Field Generation:

# Generate phase-only object
complex_sample = producer.generate_complex(
 grid=grid,
 rng=rng,
 phase_shift=np.pi/2, # 90-degree phase shift
 mode="phase"
)
print(f"Field representation: {complex_sample.representation}")
# Output: Field representation: phase

2. ObjectToHologramConverter

Purpose: Transform object-domain fields into holograms and perform reconstruction using holography strategies.

Protocol: None (concrete class)

Key Methods:

create_hologram(sample, config, rng): Generate hologram from object sample
reconstruct(hologram, config): Recover object field from hologram

Configuration:

strategy_mapping: Dictionary mapping holography methods to strategy implementations
noise_model: Optional noise model to apply during hologram creation
output_config: Configuration for field representations

Example:

from hologen import ObjectToHologramConverter
from hologen.holography.inline import InlineHolographyStrategy
from hologen.holography.off_axis import OffAxisHolographyStrategy
from hologen.types import HolographyMethod, HolographyConfig, OpticalConfig
# Create converter with both strategies
converter = ObjectToHologramConverter(
 strategy_mapping={
 HolographyMethod.INLINE: InlineHolographyStrategy(),
 HolographyMethod.OFF_AXIS: OffAxisHolographyStrategy(),
 }
)
# Configure holography parameters
config = HolographyConfig(
 grid=grid,
 optics=OpticalConfig(wavelength=532e-9, propagation_distance=0.1),
 method=HolographyMethod.INLINE
)
# Create hologram
hologram = converter.create_hologram(sample, config, rng)
reconstruction = converter.reconstruct(hologram, config)

With Noise:

from hologen.noise import SensorNoiseModel
# Add sensor noise
noise_model = SensorNoiseModel(
 name="sensor",
 read_noise=10.0,
 shot_noise=True,
 dark_current=5.0,
 bit_depth=12
)
converter = ObjectToHologramConverter(
 strategy_mapping={HolographyMethod.INLINE: InlineHolographyStrategy()},
 noise_model=noise_model
)
# Noise is applied during hologram creation
noisy_hologram = converter.create_hologram(sample, config, rng)

3. HologramDatasetGenerator

Purpose: Orchestrate the complete pipeline to generate multiple hologram samples.

Protocol: Implements DatasetGenerator protocol

Key Methods:

generate(count, config, rng, phase_shift, mode, use_complex): Yield hologram samples

Configuration:

object_producer: ObjectDomainProducer instance
converter: ObjectToHologramConverter instance

Example:

from hologen import HologramDatasetGenerator
# Assemble pipeline
generator = HologramDatasetGenerator(
 object_producer=producer,
 converter=converter
)
# Generate 10 samples
samples = list(generator.generate(
 count=10,
 config=config,
 rng=rng
))
print(f"Generated {len(samples)} samples")
for i, sample in enumerate(samples[:3]):
 print(f" Sample {i}: {sample.object_sample.name}")

Complex Field Pipeline:

# Generate complex field samples
complex_samples = list(generator.generate(
 count=10,
 config=config,
 rng=rng,
 phase_shift=np.pi/4,
 mode="phase",
 use_complex=True
))
# Access complex fields
sample = complex_samples[0]
print(f"Object representation: {sample.object_sample.representation}")
print(f"Hologram representation: {sample.hologram_representation}")
print(f"Reconstruction representation: {sample.reconstruction_representation}")

4. DatasetWriter

Purpose: Persist generated samples to disk in NumPy archives and PNG previews.

Protocol: Implements DatasetWriter protocol

Implementations:

NumpyDatasetWriter: Legacy intensity-based format
ComplexFieldWriter: Complex field format with multiple representations

Key Methods:

save(samples, output_dir): Write samples to specified directory

Example (Legacy Format):

from hologen.utils.io import NumpyDatasetWriter
from pathlib import Path
# Create writer with PNG previews
writer = NumpyDatasetWriter(save_preview=True)
# Save samples
writer.save(samples, output_dir=Path("dataset"))
# Directory structure:
# dataset/
# sample_00000_circle.npz
# sample_00000_circle_object.png
# sample_00000_circle_hologram.png
# sample_00000_circle_reconstruction.png
# ...

Example (Complex Field Format):

from hologen.utils.io import ComplexFieldWriter
# Create writer with phase colormap
writer = ComplexFieldWriter(
 save_preview=True,
 phase_colormap="twilight"
)
# Save complex samples
writer.save(complex_samples, output_dir=Path("complex_dataset"))
# Directory structure:
# complex_dataset/
# sample_00000_circle_object.npz
# sample_00000_circle_object_amplitude.png
# sample_00000_circle_object_phase.png
# sample_00000_circle_hologram.npz
# sample_00000_circle_hologram_amplitude.png
# sample_00000_circle_hologram_phase.png
# ...

Configuration Objects

GridSpec

Defines the spatial sampling grid for all fields in the pipeline.

Fields:

height (int): Number of pixels along vertical axis
width (int): Number of pixels along horizontal axis
pixel_pitch (float): Sampling interval in meters

Example:

×ばつ512 grid with 5 μm pixels grid = GridSpec( width="512," pixel_pitch=5e-6) # Physical dimensions physical_height = grid.height * grid.pixel_pitch # 2.56 mm physical_width = grid.width * grid.pixel_pitch # 2.56 mm">

from hologen.types import GridSpec
# ×ばつ512 grid with 5 μm pixels
grid = GridSpec(height=512, width=512, pixel_pitch=5e-6)
# Physical dimensions
physical_height = grid.height * grid.pixel_pitch # 2.56 mm
physical_width = grid.width * grid.pixel_pitch # 2.56 mm

OpticalConfig

Specifies the physical parameters for wave propagation.

Fields:

wavelength (float): Illumination wavelength in meters
propagation_distance (float): Object-to-sensor distance in meters

Example:

from hologen.types import OpticalConfig
# Green laser at 10 cm distance
optics = OpticalConfig(
 wavelength=532e-9, # 532 nm
 propagation_distance=0.1 # 10 cm
)

HolographyConfig

Bundles grid, optical, and method parameters for the pipeline.

Fields:

grid (GridSpec): Spatial sampling specification
optics (OpticalConfig): Physical propagation parameters
method (HolographyMethod): Holography strategy (INLINE or OFF_AXIS)
carrier (OffAxisCarrier | None): Carrier configuration for off-axis

Example (Inline):

from hologen.types import HolographyConfig, HolographyMethod
config = HolographyConfig(
 grid=GridSpec(height=512, width=512, pixel_pitch=5e-6),
 optics=OpticalConfig(wavelength=532e-9, propagation_distance=0.1),
 method=HolographyMethod.INLINE
)

Example (Off-Axis):

from hologen.types import OffAxisCarrier
config = HolographyConfig(
 grid=GridSpec(height=512, width=512, pixel_pitch=5e-6),
 optics=OpticalConfig(wavelength=532e-9, propagation_distance=0.1),
 method=HolographyMethod.OFF_AXIS,
 carrier=OffAxisCarrier(
 frequency_x=2e5, # 200 cycles/mm
 frequency_y=2e5, # 200 cycles/mm
 gaussian_width=5e4 # 50 cycles/mm filter width
 )
)

OutputConfig

Controls field representations in the output samples.

Fields:

object_representation (FieldRepresentation): Object domain representation
hologram_representation (FieldRepresentation): Hologram representation
reconstruction_representation (FieldRepresentation): Reconstruction representation

Representations:

INTENSITY: |E|2 (intensity)
AMPLITUDE: |E| (amplitude)
PHASE: arg(E) (phase)
COMPLEX: E (full complex field)

Example:

from hologen.types import OutputConfig, FieldRepresentation
# Save amplitude and phase separately
output_config = OutputConfig(
 object_representation=FieldRepresentation.AMPLITUDE,
 hologram_representation=FieldRepresentation.COMPLEX,
 reconstruction_representation=FieldRepresentation.AMPLITUDE
)
converter = ObjectToHologramConverter(
 strategy_mapping={HolographyMethod.INLINE: InlineHolographyStrategy()},
 output_config=output_config
)

NoiseConfig

Configures all noise and aberration parameters.

Fields:

Sensor Noise:
- sensor_read_noise (float): Gaussian read noise std dev
- sensor_shot_noise (bool): Enable Poisson shot noise
- sensor_dark_current (float): Mean dark current
- sensor_bit_depth (int | None): ADC quantization bits
Speckle Noise:
- speckle_contrast (float): Contrast ratio [0, 1]
- speckle_correlation_length (float): Correlation length in pixels
Aberrations:
- aberration_defocus (float): Defocus coefficient
- aberration_astigmatism_x (float): Astigmatism X coefficient
- aberration_astigmatism_y (float): Astigmatism Y coefficient
- aberration_coma_x (float): Coma X coefficient
- aberration_coma_y (float): Coma Y coefficient

Example:

from hologen.types import NoiseConfig
from hologen.converters import create_noise_model
# Configure realistic sensor noise
noise_config = NoiseConfig(
 sensor_read_noise=10.0,
 sensor_shot_noise=True,
 sensor_dark_current=5.0,
 sensor_bit_depth=12,
 speckle_contrast=0.3,
 speckle_correlation_length=2.0
)
# Create composite noise model
noise_model = create_noise_model(noise_config)
# Use in converter
converter = ObjectToHologramConverter(
 strategy_mapping={HolographyMethod.INLINE: InlineHolographyStrategy()},
 noise_model=noise_model
)

End-to-End Examples

Example 1: Basic Dataset Generation

Generate a simple dataset with default settings:

from hologen import (
 ObjectDomainProducer,
 ObjectToHologramConverter,
 HologramDatasetGenerator,
)
from hologen.shapes import available_generators
from hologen.holography.inline import InlineHolographyStrategy
from hologen.types import (
 GridSpec,
 OpticalConfig,
 HolographyConfig,
 HolographyMethod,
)
from hologen.utils.io import NumpyDatasetWriter
import numpy as np
from pathlib import Path
# Setup
grid = GridSpec(height=512, width=512, pixel_pitch=5e-6)
optics = OpticalConfig(wavelength=532e-9, propagation_distance=0.1)
config = HolographyConfig(grid=grid, optics=optics, method=HolographyMethod.INLINE)
rng = np.random.default_rng(seed=42)
# Build pipeline
producer = ObjectDomainProducer(shape_generators=tuple(available_generators()))
converter = ObjectToHologramConverter(
 strategy_mapping={HolographyMethod.INLINE: InlineHolographyStrategy()}
)
generator = HologramDatasetGenerator(object_producer=producer, converter=converter)
# Generate samples
samples = list(generator.generate(count=100, config=config, rng=rng))
# Save to disk
writer = NumpyDatasetWriter(save_preview=True)
writer.save(samples, output_dir=Path("basic_dataset"))
print(f"Generated {len(samples)} samples in basic_dataset/")

Example 2: Off-Axis with Noise

Generate off-axis holograms with realistic noise:

from hologen.holography.off_axis import OffAxisHolographyStrategy
from hologen.types import OffAxisCarrier, NoiseConfig
from hologen.converters import create_noise_model
# Configure off-axis holography
config = HolographyConfig(
 grid=GridSpec(height=512, width=512, pixel_pitch=5e-6),
 optics=OpticalConfig(wavelength=532e-9, propagation_distance=0.1),
 method=HolographyMethod.OFF_AXIS,
 carrier=OffAxisCarrier(
 frequency_x=2e5,
 frequency_y=2e5,
 gaussian_width=5e4
 )
)
# Configure noise
noise_config = NoiseConfig(
 sensor_read_noise=15.0,
 sensor_shot_noise=True,
 sensor_dark_current=10.0,
 sensor_bit_depth=12,
 speckle_contrast=0.4,
 speckle_correlation_length=2.5
)
noise_model = create_noise_model(noise_config)
# Build pipeline with noise
producer = ObjectDomainProducer(shape_generators=tuple(available_generators()))
converter = ObjectToHologramConverter(
 strategy_mapping={HolographyMethod.OFF_AXIS: OffAxisHolographyStrategy()},
 noise_model=noise_model
)
generator = HologramDatasetGenerator(object_producer=producer, converter=converter)
# Generate and save
samples = list(generator.generate(count=100, config=config, rng=rng))
writer = NumpyDatasetWriter(save_preview=True)
writer.save(samples, output_dir=Path("offaxis_noisy_dataset"))

Example 3: Complex Field Pipeline

Generate phase-only objects with complex field output:

from hologen.types import OutputConfig, FieldRepresentation
from hologen.utils.io import ComplexFieldWriter
# Configure complex field output
output_config = OutputConfig(
 object_representation=FieldRepresentation.PHASE,
 hologram_representation=FieldRepresentation.COMPLEX,
 reconstruction_representation=FieldRepresentation.COMPLEX
)
# Build pipeline
producer = ObjectDomainProducer(shape_generators=tuple(available_generators()))
converter = ObjectToHologramConverter(
 strategy_mapping={HolographyMethod.INLINE: InlineHolographyStrategy()},
 output_config=output_config
)
generator = HologramDatasetGenerator(object_producer=producer, converter=converter)
# Generate complex samples
config = HolographyConfig(
 grid=GridSpec(height=512, width=512, pixel_pitch=5e-6),
 optics=OpticalConfig(wavelength=532e-9, propagation_distance=0.1),
 method=HolographyMethod.INLINE
)
complex_samples = list(generator.generate(
 count=100,
 config=config,
 rng=rng,
 phase_shift=np.pi/2, # 90-degree phase shift
 mode="phase",
 use_complex=True
))
# Save with complex field writer
writer = ComplexFieldWriter(save_preview=True, phase_colormap="twilight")
writer.save(complex_samples, output_dir=Path("phase_only_dataset"))

Example 4: Custom Shape Generators

Use only specific shape generators:

from hologen import CircleGenerator, RingGenerator
# Create producer with specific generators
producer = ObjectDomainProducer(
 shape_generators=(CircleGenerator(), RingGenerator())
)
# Rest of pipeline as usual
converter = ObjectToHologramConverter(
 strategy_mapping={HolographyMethod.INLINE: InlineHolographyStrategy()}
)
generator = HologramDatasetGenerator(object_producer=producer, converter=converter)
samples = list(generator.generate(count=50, config=config, rng=rng))
print(f"Generated samples with shapes: {set(s.object_sample.name for s in samples)}")
# Output: Generated samples with shapes: {'circle', 'ring'}

Protocol-Based Design

HoloGen uses Python protocols (structural subtyping) to define component interfaces. This enables duck typing and makes the pipeline highly extensible.

Architecture Overview

The following diagram shows the protocol-based architecture:

classDiagram
 class ObjectShapeGenerator {
 <<Protocol>>
 +name: str
 +generate(grid, rng) ArrayFloat
 +generate_complex(grid, rng, phase_shift, mode) ArrayComplex
 }
 
 class HolographyStrategy {
 <<Protocol>>
 +create_hologram(field, config) ArrayComplex
 +reconstruct(hologram, config) ArrayComplex
 }
 
 class NoiseModel {
 <<Protocol>>
 +apply(hologram, config, rng) ArrayFloat
 }
 
 class DatasetWriter {
 <<Protocol>>
 +save(samples, output_dir) None
 }
 
 class DatasetGenerator {
 <<Protocol>>
 +generate(count, config, rng) Iterable
 }
 
 class ObjectDomainProducer {
 -shape_generators: tuple
 +generate(grid, rng) ObjectSample
 +generate_complex(grid, rng, phase_shift, mode) ComplexObjectSample
 }
 
 class ObjectToHologramConverter {
 -strategy_mapping: dict
 -noise_model: NoiseModel
 -output_config: OutputConfig
 +create_hologram(sample, config, rng) Array
 +reconstruct(hologram, config) Array
 }
 
 class HologramDatasetGenerator {
 -object_producer: ObjectDomainProducer
 -converter: ObjectToHologramConverter
 +generate(count, config, rng, ...) Iterable
 }
 
 class InlineHolographyStrategy {
 +create_hologram(field, config) ArrayComplex
 +reconstruct(hologram, config) ArrayComplex
 }
 
 class OffAxisHolographyStrategy {
 +create_hologram(field, config) ArrayComplex
 +reconstruct(hologram, config) ArrayComplex
 }
 
 class CircleGenerator {
 +name: str
 +generate(grid, rng) ArrayFloat
 }
 
 class SensorNoiseModel {
 +apply(hologram, config, rng) ArrayFloat
 }
 
 class NumpyDatasetWriter {
 +save(samples, output_dir) None
 }
 
 ObjectDomainProducer --> ObjectShapeGenerator : uses
 ObjectToHologramConverter --> HolographyStrategy : uses
 ObjectToHologramConverter --> NoiseModel : uses
 HologramDatasetGenerator --> ObjectDomainProducer : contains
 HologramDatasetGenerator --> ObjectToHologramConverter : contains
 HologramDatasetGenerator ..|> DatasetGenerator : implements
 
 InlineHolographyStrategy ..|> HolographyStrategy : implements
 OffAxisHolographyStrategy ..|> HolographyStrategy : implements
 CircleGenerator ..|> ObjectShapeGenerator : implements
 SensorNoiseModel ..|> NoiseModel : implements
 NumpyDatasetWriter ..|> DatasetWriter : implements

Core Protocols

ObjectShapeGenerator

Defines the interface for shape generators:

from typing import Protocol
from hologen.types import GridSpec, ArrayFloat
from numpy.random import Generator
class ObjectShapeGenerator(Protocol):
 @property
 def name(self) -> str:
 """Return the canonical name of the generator."""
 
 def generate(self, grid: GridSpec, rng: Generator) -> ArrayFloat:
 """Create a binary object-domain image."""

Any class implementing these methods can be used as a shape generator.

HolographyStrategy

Defines the interface for holography methods:

from typing import Protocol
from hologen.types import ArrayComplex, HolographyConfig
class HolographyStrategy(Protocol):
 def create_hologram(
 self, object_field: ArrayComplex, config: HolographyConfig
 ) -> ArrayComplex:
 """Create a hologram from an object-domain complex field."""
 
 def reconstruct(
 self, hologram: ArrayComplex, config: HolographyConfig
 ) -> ArrayComplex:
 """Recover an object-domain complex field from a hologram."""

Implement this protocol to create custom holography methods.

DatasetWriter

Defines the interface for dataset persistence:

from typing import Protocol
from collections.abc import Iterable
from pathlib import Path
from hologen.types import HologramSample
class DatasetWriter(Protocol):
 def save(self, samples: Iterable[HologramSample], output_dir: Path) -> None:
 """Persist a sequence of hologram samples."""

Implement this protocol to create custom output formats.

NoiseModel

Defines the interface for noise simulation:

from typing import Protocol
from hologen.types import ArrayFloat, HolographyConfig
from numpy.random import Generator
class NoiseModel(Protocol):
 def apply(
 self, hologram: ArrayFloat, config: HolographyConfig, rng: Generator
 ) -> ArrayFloat:
 """Apply noise to a hologram."""

Implement this protocol to create custom noise models.

Creating Custom Components

Custom Shape Generator

from hologen.types import GridSpec, ArrayFloat
from numpy.random import Generator
import numpy as np
class StarGenerator:
 """Generate star-shaped objects."""
 
 @property
 def name(self) -> str:
 return "star"
 
 def generate(self, grid: GridSpec, rng: Generator) -> ArrayFloat:
 # Implementation details...
 pixels = np.zeros((grid.height, grid.width))
 # Draw star shape
 return pixels
# Use in pipeline
producer = ObjectDomainProducer(
 shape_generators=(StarGenerator(), CircleGenerator())
)

Custom Holography Strategy

from hologen.types import ArrayComplex, HolographyConfig
class CustomHolographyStrategy:
 """Custom holography implementation."""
 
 def create_hologram(
 self, object_field: ArrayComplex, config: HolographyConfig
 ) -> ArrayComplex:
 # Custom hologram generation logic
 pass
 
 def reconstruct(
 self, hologram: ArrayComplex, config: HolographyConfig
 ) -> ArrayComplex:
 # Custom reconstruction logic
 pass
# Use in pipeline
converter = ObjectToHologramConverter(
 strategy_mapping={
 HolographyMethod.INLINE: CustomHolographyStrategy()
 }
)

Custom Writer

from collections.abc import Iterable
from pathlib import Path
from hologen.types import HologramSample
import h5py
class HDF5Writer:
 """Write samples to HDF5 format."""
 
 def save(self, samples: Iterable[HologramSample], output_dir: Path) -> None:
 output_dir.mkdir(parents=True, exist_ok=True)
 
 with h5py.File(output_dir / "dataset.h5", "w") as f:
 for i, sample in enumerate(samples):
 grp = f.create_group(f"sample_{i:05d}")
 grp.create_dataset("object", data=sample.object_sample.pixels)
 grp.create_dataset("hologram", data=sample.hologram)
 grp.create_dataset("reconstruction", data=sample.reconstruction)
# Use in pipeline
writer = HDF5Writer()
writer.save(samples, output_dir=Path("hdf5_dataset"))

API Reference

ObjectDomainProducer

@dataclass(slots=True)
class ObjectDomainProducer:
 shape_generators: tuple[ObjectShapeGenerator, ...]
 
 def generate(self, grid: GridSpec, rng: Generator) -> ObjectSample:
 """Produce a legacy intensity-based object sample."""
 
 def generate_complex(
 self,
 grid: GridSpec,
 rng: Generator,
 phase_shift: float = 0.0,
 mode: str = "amplitude",
 ) -> ComplexObjectSample:
 """Produce a complex field object sample."""

ObjectToHologramConverter

@dataclass(slots=True)
class ObjectToHologramConverter:
 strategy_mapping: dict[HolographyMethod, HolographyStrategy]
 noise_model: NoiseModel | None = None
 output_config: OutputConfig = field(default_factory=OutputConfig)
 
 def create_hologram(
 self,
 sample: ObjectSample | ComplexObjectSample,
 config: HolographyConfig,
 rng: Generator | None = None,
 ) -> ArrayFloat | ArrayComplex:
 """Generate a hologram for the provided object sample."""
 
 def reconstruct(
 self, hologram: ArrayFloat | ArrayComplex, config: HolographyConfig
 ) -> ArrayFloat | ArrayComplex:
 """Reconstruct an object-domain field from a hologram."""

HologramDatasetGenerator

@dataclass(slots=True)
class HologramDatasetGenerator(DatasetGenerator):
 object_producer: ObjectDomainProducer
 converter: ObjectToHologramConverter
 
 def generate(
 self,
 count: int,
 config: HolographyConfig,
 rng: Generator,
 phase_shift: float = 0.0,
 mode: str = "amplitude",
 use_complex: bool = False,
 ) -> Iterable[HologramSample | ComplexHologramSample]:
 """Yield hologram samples as an iterable sequence."""

Factory Functions

def default_object_producer() -> ObjectDomainProducer:
 """Create the default object domain producer with built-in shapes."""
def default_converter(
 noise_model: NoiseModel | None = None,
) -> ObjectToHologramConverter:
 """Create the default converter with inline and off-axis strategies."""
def generate_dataset(
 count: int,
 config: HolographyConfig,
 rng: Generator,
 writer: DatasetWriter,
 generator: HologramDatasetGenerator | None = None,
 output_dir: Path | None = None,
) -> None:
 """Generate and persist a holography dataset using the pipeline."""
def create_noise_model(config: NoiseConfig) -> NoiseModel | None:
 """Create a composite noise model from configuration."""

Best Practices

1. Use Factory Functions for Defaults

The factory functions provide sensible defaults:

from hologen.converters import default_object_producer, default_converter
producer = default_object_producer() # All built-in shapes
converter = default_converter() # Both inline and off-axis

2. Reuse Configuration Objects

Create configuration objects once and reuse them:

# Good: Reuse config
config = HolographyConfig(...)
samples = list(generator.generate(count=1000, config=config, rng=rng))
# Avoid: Creating config in loop
for i in range(1000):
 config = HolographyConfig(...) # Wasteful

3. Use Generators for Memory Efficiency

The pipeline yields samples lazily. Process them in batches:

# Memory-efficient: Process in batches
for batch_start in range(0, 10000, 100):
 samples = list(generator.generate(count=100, config=config, rng=rng))
 writer.save(samples, output_dir=Path(f"batch_{batch_start}"))

4. Validate Configuration Early

Check configuration validity before generating large datasets:

# Generate one sample to validate
test_samples = list(generator.generate(count=1, config=config, rng=rng))
assert len(test_samples) == 1, "Pipeline validation failed"
# Now generate full dataset
samples = list(generator.generate(count=10000, config=config, rng=rng))

5. Use Type Hints

Leverage type hints for better IDE support:

from hologen.types import HologramSample
samples: list[HologramSample] = list(
 generator.generate(count=10, config=config, rng=rng)
)

Troubleshooting

Issue: "Unknown holography method"

Cause: The converter's strategy mapping doesn't include the requested method.

Solution: Ensure the strategy mapping includes the method specified in config:

# Wrong: Missing OFF_AXIS strategy
converter = ObjectToHologramConverter(
 strategy_mapping={HolographyMethod.INLINE: InlineHolographyStrategy()}
)
config = HolographyConfig(..., method=HolographyMethod.OFF_AXIS) # Error!
# Correct: Include both strategies
converter = ObjectToHologramConverter(
 strategy_mapping={
 HolographyMethod.INLINE: InlineHolographyStrategy(),
 HolographyMethod.OFF_AXIS: OffAxisHolographyStrategy(),
 }
)

Issue: "Off-axis holography requires carrier configuration"

Cause: Using OFF_AXIS method without providing carrier parameters.

Solution: Include OffAxisCarrier in the configuration:

from hologen.types import OffAxisCarrier
config = HolographyConfig(
 grid=grid,
 optics=optics,
 method=HolographyMethod.OFF_AXIS,
 carrier=OffAxisCarrier(
 frequency_x=2e5,
 frequency_y=2e5,
 gaussian_width=5e4
 )
)

Issue: Noise not being applied

Cause: Forgot to pass rng to create_hologram().

Solution: Always pass the random number generator when using noise:

# Wrong: No rng, noise won't be applied
hologram = converter.create_hologram(sample, config)
# Correct: Pass rng for noise application
hologram = converter.create_hologram(sample, config, rng)

Issue: Memory usage too high

Cause: Generating all samples at once.

Solution: Process samples in batches or use streaming:

# Memory-efficient approach
batch_size = 100
for batch_idx in range(0, total_samples, batch_size):
 samples = list(generator.generate(
 count=min(batch_size, total_samples - batch_idx),
 config=config,
 rng=rng
 ))
 writer.save(samples, output_dir=Path(f"batch_{batch_idx // batch_size}"))

06. Pipelines

Pipeline Flow

Key Design Principles

Component Interaction Diagram

Data Transformation Flow

Components

1. ObjectDomainProducer

2. ObjectToHologramConverter

3. HologramDatasetGenerator

4. DatasetWriter

Configuration Objects

GridSpec

OpticalConfig

HolographyConfig

OutputConfig

NoiseConfig

End-to-End Examples

Example 1: Basic Dataset Generation

Example 2: Off-Axis with Noise

Example 3: Complex Field Pipeline

Example 4: Custom Shape Generators

Protocol-Based Design

Architecture Overview

Core Protocols

ObjectShapeGenerator

HolographyStrategy

DatasetWriter

NoiseModel

Creating Custom Components

Custom Shape Generator

Custom Holography Strategy

Custom Writer

API Reference

ObjectDomainProducer

ObjectToHologramConverter

HologramDatasetGenerator

Factory Functions

Best Practices

1. Use Factory Functions for Defaults

2. Reuse Configuration Objects

3. Use Generators for Memory Efficiency

4. Validate Configuration Early

5. Use Type Hints

Troubleshooting

Issue: "Unknown holography method"

Issue: "Off-axis holography requires carrier configuration"

Issue: Noise not being applied

Issue: Memory usage too high

See Also

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally