Skip to content

Settings

Settings

Bases: OpenAlephSettings

ingest-file settings management using pydantic-settings

Note

All settings can be set via environment variables, prepending INGESTORS_ (except for those with another alias) via runtime or in a .env file.

Source code in ingestors/settings.py
class Settings(OpenAlephSettings):
    """
    `ingest-file` settings management using
    [pydantic-settings](https://docs.pydantic.dev/latest/concepts/pydantic_settings/)

    Note:
        All settings can be set via environment variables, prepending
        `INGESTORS_` (except for those with another alias) via runtime or in a
        `.env` file.
    """

    model_config = SettingsConfigDict(
        env_prefix="ingestors_",
        env_file=".env",
        extra="ignore",  # other envs in .env file
    )

    convert_timeout: int = 300
    """Headless libreoffice document convert timeout in seconds"""

    unoserver_uri: str | None = None
    """URI of a running unoserver XML-RPC listener
    to convert office documents to PDF with."""

    soffice_bin: str = "/usr/bin/libreoffice"
    """Path to the LibreOffice/soffice binary"""

    lakehouse_flush_size: int = 10_000
    """Size cap until when the journal should be flushed to parquet"""

    crawl_threads: int = 1
    """Crawl parallelization, worth raising if archive storage is over the network"""

    @property
    def tags_database_uri(self) -> str:
        """Resolve the tags database URI from fragments_uri, forcing psycopg3
        driver for PostgreSQL."""
        uri = self.fragments_uri
        if uri and uri.startswith("postgresql://"):
            uri = uri.replace("postgresql://", "postgresql+psycopg://", 1)
        return uri

convert_timeout = 300 class-attribute instance-attribute

Headless libreoffice document convert timeout in seconds

crawl_threads = 1 class-attribute instance-attribute

Crawl parallelization, worth raising if archive storage is over the network

lakehouse_flush_size = 10000 class-attribute instance-attribute

Size cap until when the journal should be flushed to parquet

soffice_bin = '/usr/bin/libreoffice' class-attribute instance-attribute

Path to the LibreOffice/soffice binary

tags_database_uri property

Resolve the tags database URI from fragments_uri, forcing psycopg3 driver for PostgreSQL.

unoserver_uri = None class-attribute instance-attribute

URI of a running unoserver XML-RPC listener to convert office documents to PDF with.