Problem Description
SDV currently expects the inputs/outputs of the synthetic data process to be different based on the data modality (aka single vs. sequential vs. multi-table). As part of SDV 2.0, we'd like to make all the inputs/outputs consistent across the modalities to make all the workflows similar.
Expected behavior
Modeling: All synthesizers should accept a dictionary that maps a table name to a DataFrame, regardless of how many tables there are. Single-table and sequential synthesizers would expect there to only be 1 table name (key) in the dictionary.
- This change affects all features related to modeling and setting up modeling configs which includes:
fit, preprocess, fit_processed_data, auto_assign_transformers, detect_constraints
- Within a single-table (or sequential) context, the functions should error out if there are multiple items in the dictionary – aka multiple tables
Sampling:
- The main
sample call should just accept a table_name and num_rows as required parameters. (a) Multi-table synthesizers should scale the other tables based on the requested table. (b) Sequential synthesizers won't yet implement this method; throw an error saying to use the sample_sequences function instead (see below).
- The batching parameters should also be included in the
sample call; for multi-table, they should apply to each table independently.
- The
output_file_path parameter should be changed to output_folder_path. All the synthesized CSVs should then end up here.
- Create a new function called
sample_sequences that can be used with sequential synthesizers only. This takes in num_sequences and sequence_length similar to today's sample call.
- Nothing needs to be changed for conditional sampling
Evaluation:
- The quality and diagnostic evaluations have already been consolidated to the same i/o for all modalities -- nothing to be done here
- The visualization functions should be updated to only import from
sdv.evaluation. They should take in a table_name parameter (which is required for multi-table data)
Utility functions: All relevant utility functions should also update to dictionaries that maps a table name to a DataFrame, regardless of how many tables there are.
- This includes:
get_random_sequence_subset (t should validate that there is only 1 key in the dictionary -- aka 1 table)
- The
download_demo functions should return a dictionary not matter what the modality
Additional context
No backwards compatibility is needed since this is for SDV 2.0.
Merge into a feature branch for SDV 2.0, as there will be other issues that are also needed for this version.
Problem Description
SDV currently expects the inputs/outputs of the synthetic data process to be different based on the data modality (aka single vs. sequential vs. multi-table). As part of SDV 2.0, we'd like to make all the inputs/outputs consistent across the modalities to make all the workflows similar.
Expected behavior
Modeling: All synthesizers should accept a dictionary that maps a table name to a DataFrame, regardless of how many tables there are. Single-table and sequential synthesizers would expect there to only be 1 table name (key) in the dictionary.
fit,preprocess,fit_processed_data,auto_assign_transformers,detect_constraintsSampling:
samplecall should just accept atable_nameandnum_rowsas required parameters. (a) Multi-table synthesizers should scale the other tables based on the requested table. (b) Sequential synthesizers won't yet implement this method; throw an error saying to use thesample_sequencesfunction instead (see below).samplecall; for multi-table, they should apply to each table independently.output_file_pathparameter should be changed tooutput_folder_path. All the synthesized CSVs should then end up here.sample_sequencesthat can be used with sequential synthesizers only. This takes innum_sequencesandsequence_lengthsimilar to today's sample call.Evaluation:
sdv.evaluation. They should take in atable_nameparameter (which is required for multi-table data)Utility functions: All relevant utility functions should also update to dictionaries that maps a table name to a DataFrame, regardless of how many tables there are.
get_random_sequence_subset(t should validate that there is only 1 key in the dictionary -- aka 1 table)download_demofunctions should return a dictionary not matter what the modalityAdditional context
No backwards compatibility is needed since this is for SDV 2.0.
Merge into a feature branch for SDV 2.0, as there will be other issues that are also needed for this version.