pypi installation vs. dependencies

[maederm]

Hi

I'm using Superset using the pypi installation method. It makes it easier to add DB drivers and other dependencies.

However, it seems that some workarounds are needed for it to work, which I wouldn’t need with the Docker method. As of 6.1.0, I needed to add these extra dependencies which I assume should live in the project's pyproject.toml.

Additional entries required for 6.1.0:

# Workaround for https://github.com/apache/superset/issues/40962
cachetools = ">=5.3.0,<7"
# Workaround for missing apache-superset dependency (didn't find an issue)
rich = "^15.0.0"
# Workaround for https://github.com/apache/superset/discussions/40172#discussioncomment-16980371
flask-sqlalchemy = "^2.5.1"

It seems that I can't expect the dependencies in pyproject.toml to be up to date. Is that assumption correct?
Should I use something like uv pip compile with the base.txt instead?
Or is this something that should work, and could I contribute a fix

[dosubot[bot]]

Hi @maederm! I'm Dosu and I'm helping the Apache Superset team.

Your observation is correct — there is a real gap between Docker and PyPI installations. The root cause is that Docker builds install from requirements/base.txt [1], which is a compiled lockfile with all transitive dependencies pinned to tested versions, while pip install apache-superset uses only pyproject.toml with version ranges and relies on dynamic resolution of transitive dependencies [2].

Here's what's happening with your three workarounds:

• cachetools — This is a confirmed bug (cachetools is a Superset requirement but is not explicitly required #40962). Superset directly imports cachetools (in aws_iam.py), but it was only pulled in transitively via google-auth. When google-auth v2.53.0+ dropped cachetools from its dependencies, PyPI installs broke. Docker still works because base.txt pins google-auth==2.43.0 which still includes it [3]. This should be added as an explicit dependency in pyproject.toml.

• flask-sqlalchemy — The pyproject.toml doesn't pin this directly; it comes in via flask-appbuilder and flask-migrate [4]. PyPI resolution can pull in v3.x, which has breaking API changes. Docker pins it to 2.5.1.

• rich — This is a transitive dependency via flask-limiter [5] and isn't declared directly in pyproject.toml.

To answer your questions:

1. Is pyproject.toml expected to be incomplete? Not intentionally, but in practice it only declares top-level dependencies with loose ranges, and the project is primarily tested against the pinned base.txt versions. So yes, PyPI installs can drift into untested territory.

2. Should you use uv pip compile with base.txt? Using requirements/base.txt directly (or compiling from it) would give you the same pinned environment Docker uses — that's the most reliable approach for PyPI-based installs today.

3. Can you contribute a fix? Absolutely — adding cachetools as an explicit dependency in pyproject.toml (as suggested in cachetools is a Superset requirement but is not explicitly required #40962) and tightening the flask-sqlalchemy version constraint would be welcome contributions. These are legitimate gaps that affect PyPI users.

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.

[FrancescoCastaldi]

Your observation is correct: pyproject.toml and setup.py in Apache Superset specify loose dependency ranges to allow flexibility, but running an unconstrained pip install apache-superset will resolve recent transient releases of sub-dependencies (like cachetools, rich, or flask-sqlalchemy) that break runtime contracts.

To achieve a reliable, deterministic PyPI installation identical to official release containers, you must use Superset's official constraints file.

The Supported PyPI Installation Method

When installing from PyPI, supply the version-pinned constraints file maintained in the repository for that exact release tag:

# Set your target version
export SUPERSET_VERSION=6.1.0

# Install using the release constraint file
pip install apache-superset==${SUPERSET_VERSION} \
  -c https://raw.githubusercontent.com/apache/superset/${SUPERSET_VERSION}/requirements/base.txt

Why this is necessary

• Superset relies on strict pins for core packages (such as Flask-AppBuilder, Werkzeug, SQLAlchemy, and Flask-Caching) because upstream patch releases frequently introduce breaking changes in internal extension points.
• The requirements/base.txt file in the GitHub tag contains the fully resolved, tested dependency lock tree used to build the official Docker images.

If you use uv pip compile

If you manage your environment with uv, do not compile from pyproject.toml directly. Instead, compile against the official requirements/base.txt:

uv pip install -r https://raw.githubusercontent.com/apache/superset/6.1.0/requirements/base.txt
uv pip install apache-superset==6.1.0 --no-deps

This avoids any manual dependency patching while making it easy to install custom database drivers alongside Superset.