Module lunar.ml.edd_service
Classes
class EDDService (config: Config)
-
Methods
def copy_model_to_s3(self, model_path: str, model_name: str, model_version: str) ‑> Dict[str, Any]
-
Copy model binaries and meta files from EDD to BAP model registry.
Args
- model_path: (str) Relative path of model binaries and meta (Caution: Must not start with slash and must end with slash)
- model_name: (str) Name of a model
- model_version: (str) Version of a model
Returns
dict
Example
response = copy_model_to_s3(model_path="myfiles/models/test_model/v1/", model_name="test_model", version="v1")
def copy_table_to_s3(self, database_name: str, table_name: str, db_type: str, operation: str, target_name: str = None, s3_dt: str = None, edd_table_partition: str = None) ‑> Dict[str, Any]
-
Copy a table from EDD S3 to BAP S3 and Database(dynamodb or docdb).
Args
- database_name: (str) Database name of EDD S3 ('Datalake' is not available)
- table_name: (str) Name of a table on EDD S3
- db_type: (str) Type of database to be loaded on BAP ('dynamodb' | 'docdb' | 'es' | 's3')
- operation: (str) Type of operation ('put' (db_type 's3' is only available on 'put') | 'update' | 'upsert' (only for db_type 'dynamodb') | 'delete')
- target_name: (optional) (str) Target name on database to be loaded (Default: Value of
table_name
) - s3_dt: (optional) (str) Set a dt partition on BAP S3. If db_type is
docdb
, it is necessary - edd_table_partition: (optional) (str) Partition value of table on EDD S3
Returns
dict
Example
response = copy_table_to_s3( database_name="apollo", table_name="test_table", db_type="docdb", operation="put", target_name="test_data", s3_dt="20211101", edd_table_partition="dt=202109/type=put", )