Skip to content

SDV 2.0: Make the I/O consistent for all modalities #2950

Description

@npatki

Problem Description

SDV currently expects the inputs/outputs of the synthetic data process to be different based on the data modality (aka single vs. sequential vs. multi-table). As part of SDV 2.0, we'd like to make all the inputs/outputs consistent across the modalities to make all the workflows similar.

Expected behavior

Modeling: All synthesizers should accept a dictionary that maps a table name to a DataFrame, regardless of how many tables there are. Single-table and sequential synthesizers would expect there to only be 1 table name (key) in the dictionary.

  • This change affects all features related to modeling and setting up modeling configs which includes: fit, preprocess, fit_processed_data, auto_assign_transformers, detect_constraints
  • Within a single-table (or sequential) context, the functions should error out if there are multiple items in the dictionary – aka multiple tables

Sampling:

  • The main sample call should just accept a table_name and num_rows as required parameters. (a) Multi-table synthesizers should scale the other tables based on the requested table. (b) Sequential synthesizers won't yet implement this method; throw an error saying to use the sample_sequences function instead (see below).
    • The batching parameters should also be included in the sample call; for multi-table, they should apply to each table independently.
    • The output_file_path parameter should be changed to output_folder_path. All the synthesized CSVs should then end up here.
  • Create a new function called sample_sequences that can be used with sequential synthesizers only. This takes in num_sequences and sequence_length similar to today's sample call.
  • Nothing needs to be changed for conditional sampling

Evaluation:

  • The quality and diagnostic evaluations have already been consolidated to the same i/o for all modalities -- nothing to be done here
  • The visualization functions should be updated to only import from sdv.evaluation. They should take in a table_name parameter (which is required for multi-table data)

Utility functions: All relevant utility functions should also update to dictionaries that maps a table name to a DataFrame, regardless of how many tables there are.

  • This includes: get_random_sequence_subset (t should validate that there is only 1 key in the dictionary -- aka 1 table)
  • The download_demo functions should return a dictionary not matter what the modality

Additional context

No backwards compatibility is needed since this is for SDV 2.0.

Merge into a feature branch for SDV 2.0, as there will be other issues that are also needed for this version.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions