Repository navigation
Replies: 1 comment
|
Hi, my team at the University of Toronto are able to pick this work up as a part of our open source group project. Looking over the proposed scope and steps, I think adding a bulk registration workflow makes sense. Based on the current flow, after selecting a single table in the dataset we have the option to "create and explore". When adding the ability to import datasets in bulk we would let users select multiple tables from the dropdown without retrying it every time. As an initial option, we would make this a synchronous workflow, and detect when multiple tables are selected via a checklist in the UI. A new query would be made, mimicking the single processing query currently available. Instead of creating multiple charts (as the workflow currently does), the V1 of this feature should just create the datasets, without routing users to the "create chart" page. Our proposed idea includes:
As we continue, we may be able to incorporate more of the functionality above as well. This includes progress tracking, auto skip-detection, generating default metrics and options to overwrite/sync existing DBs. |
Uh oh!
There was an error while loading. Please reload this page.
As a Superset 5.0.0 user managing multiple databases, I often face two major pain points:
There’s no easy way to bulk-import datasets.
Superset currently requires manually creating each dataset (table or view) through the UI.
When a database contains dozens or even hundreds of tables, this process becomes extremely time-consuming and error-prone.
The existing “import/export datasource” YAML or ZIP mechanism is helpful for migration but doesn’t provide an automated way to scan and register all tables/views in a database as datasets.
Chart creation lacks database-level filtering.
When creating a new chart, the dataset selector lists all datasets across all databases in one long dropdown.
If multiple databases share similar table names or schemas, it becomes difficult to find the correct dataset.
A database (and optionally schema) filter would make the dataset selection process much faster and less confusing.
Proposed Solution
Bulk Import Functionality
Add a feature in the UI or API to automatically scan a database (and optionally a schema) and register all tables/views as datasets in one operation.
Include configurable options such as:
SUM(),AVG())Support both synchronous and asynchronous execution with progress tracking.
Database Filter in Chart Creation Dialog
Benefits
All reactions