Skip to content

Load: Harden LOAD TSFILE source path validation#17624

Merged
jt2594838 merged 9 commits into
masterfrom
load-pri
May 12, 2026
Merged

Load: Harden LOAD TSFILE source path validation#17624
jt2594838 merged 9 commits into
masterfrom
load-pri

Conversation

@Caideyipi
Copy link
Copy Markdown
Collaborator

@Caideyipi Caideyipi commented May 9, 2026

Description

This PR hardens LOAD TSFILE execution by adding explicit authorization checks and limiting user-issued load sources to
configured allowed directories.

Main changes:

  • Restrict source paths for user-issued LOAD TSFILE.
    • Add load_tsfile_allowed_dirs config.
    • If unset, the allowed source directories default to IoTDB internal load TsFile directories.
    • Canonicalize source paths before validation to reject paths outside the configured allowlist, including
      traversal-style paths.
  • Preserve internal load flows.
    • Add unchecked constructors/factory methods for internal paths such as pipe receiver loading, active load,
      scheduler retry, and type-conversion retry paths.
  • Add tests covering:
    • Source files outside load_tsfile_allowed_dirs are rejected.
    • Sub-statements preserve database information under the new path validation behavior.

This PR has:

  • been self-reviewed.
    • concurrent read
    • concurrent write
    • concurrent read and write
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods.
  • added or updated version, license, or notice information
  • added comments explaining the "why" and the intent of the code wherever would not be obvious
    for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold
    for code coverage.
  • added integration tests.
  • been tested in a test IoTDB cluster.

Key changed/added classes (or packages if there are too many classes) in this PR

@Caideyipi Caideyipi marked this pull request as draft May 9, 2026 07:16
@codecov
Copy link
Copy Markdown

codecov Bot commented May 9, 2026

Codecov Report

❌ Patch coverage is 68.14159% with 36 lines in your changes missing coverage. Please review.
✅ Project coverage is 40.35%. Comparing base (d4be5c8) to head (fb20447).
⚠️ Report is 10 commits behind head on master.

Files with missing lines Patch % Lines
...ain/java/org/apache/iotdb/db/conf/IoTDBConfig.java 75.55% 11 Missing ⚠️
...ngine/plan/statement/crud/LoadTsFileStatement.java 65.62% 11 Missing ⚠️
...ryengine/plan/analyze/load/LoadTsFileAnalyzer.java 0.00% 6 Missing ⚠️
...ueryengine/plan/relational/sql/ast/LoadTsFile.java 50.00% 2 Missing ⚠️
...ngine/plan/scheduler/load/LoadTsFileScheduler.java 0.00% 2 Missing ⚠️
...rageengine/load/active/ActiveLoadTsFileLoader.java 0.00% 2 Missing ⚠️
.../receiver/protocol/legacy/loader/TsFileLoader.java 0.00% 1 Missing ⚠️
...eceiver/protocol/thrift/IoTDBDataNodeReceiver.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #17624      +/-   ##
============================================
+ Coverage     40.23%   40.35%   +0.12%     
  Complexity     2554     2554              
============================================
  Files          5177     5177              
  Lines        348880   349073     +193     
  Branches      44624    44663      +39     
============================================
+ Hits         140363   140861     +498     
+ Misses       208517   208212     -305     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Caideyipi Caideyipi changed the title Load: Harden LOAD TSFILE authorization and source path validation Load: Harden LOAD TSFILE source path validation May 11, 2026
@Caideyipi Caideyipi marked this pull request as ready for review May 11, 2026 03:11
@sonarqubecloud
Copy link
Copy Markdown

@jt2594838 jt2594838 merged commit 5029a0a into master May 12, 2026
29 of 30 checks passed
@jt2594838 jt2594838 deleted the load-pri branch May 12, 2026 09:11
jt2594838 pushed a commit that referenced this pull request May 13, 2026
…#17654)

* Load: Harden LOAD TSFILE source path validation (#17624)

* Load pri

* sp

* MAINTAIN

* rollback

* Add

* change

* canonical

* line

* Pre

* Update LoadTsFileStatementTest.java

* Update LoadTsFileStatementTest.java

* Update IoTDBDescriptor.java
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants