openaleph_procrastinate.helpers
Helper functions to access the archive and FollowTheMoney entities of a dataset within Jobs.
Everything here goes through
the repository, which speaks to either
the legacy stores or the lakehouse depending on the
lakehouse
setting.
entity_writer(dataset, origin=OPAL_ORIGIN)
Get the EntityStore for
the given dataset to write to. It is flushed and closed when leaving the
context – a store that buffers writes holds a database connection until
then, so callers that build one themselves have to close it.
Source code in openaleph_procrastinate/helpers.py
get_localpath(dataset, content_hash)
Load a file from the archive and store it in a local temporary path for further processing. The file is cleaned up after leaving the context. Reference
Source code in openaleph_procrastinate/helpers.py
load_entities(dataset, entity_ids)
open_file(dataset, content_hash)
Load a file from the archive and store it in a local temporary path for further processing. Returns an open file handler. The file is closed and cleaned up after leaving the context. Reference