-
Notifications
You must be signed in to change notification settings - Fork 29
Open
Description
local data service has logic that can handle both a full path or the filename resulting in several problematic calls to get_dataset(). the filename is expected in all other data services, not the full path. the problematic calls are:
parent_library_model_column_order.py
parent_dataframe = self.data_service.get_dataset(
dataset_name=parent_datasets[0].full_path
)
variable_count.py
data: pd.DataFrame = self.data_service.get_dataset(
dataset_name=dataset.full_path
)
variable_exists.py
dataframe = self.data_service.get_dataset(dataset_name=self.params.dataset_path)
variable_is_null.py
dataframe = self.data_service.get_dataset(
dataset_name=self.params.dataset_path
)
variable_value_count.py
if dataset_metadata.is_split:
corresponding_datasets = get_corresponding_datasets(
self.params.datasets, dataset_metadata
)
data: DatasetInterface = self.data_service.concat_split_datasets(
self.data_service.get_dataset, corresponding_datasets
)
else:
data: DatasetInterface = self.data_service.get_dataset(
dataset_name=os.path.join(
self.params.directory_path, dataset_metadata.filename
)
)
data = tag_source(data, dataset_metadata)
dataset_preprocessor.py
def _download_dataset(self, filename: str) -> DatasetInterface:
return self._data_service.get_dataset(
dataset_name=os.path.join(
os.path.dirname(self._dataset_metadata.full_path), filename
)
)
ruleprocessor.py
operation_params.dataframe = self.data_service.get_dataset(
dataset_name=file_path
)
sdtm_utilities.py
class_name = data_service._handle_custom_domains(
data_service.get_dataset(dataset_name=dataset_metadata.full_path),
dataset_metadata,
dataset_path,
datasets,
)
these should be resolved, with the ability to process a complete path removed from local data service, as well as tests updated
Reactions are currently unavailable